Gradient Accelerated Stochastic Dual Dynamic Programming For Economic

Multistage stochastic economic dispatch (ED) solutions play a crucial role in obtaining reliable and cost-effective operations. The stochastic dual dynamic programming (SDDP) has been proposed to achieve optimal solutions for multistage stochastic programming (SP) problems under the probabilistic convergence rate. The fast SDDP (FSDDP) method can guarantee finite convergence by solving bilevel problems for generating deterministic updates while suffering from the high computational complexity of large-scale decision-making systems. Thus, we leverage the recent advances in the primal–dual reformulation method of solving bilevel problems, which allows for first-order gradient descent/ascent (GDA) updates. Based on the primal–dual bilevel optimizer (PDBO), we propose the gradient-accelerated SDDP (GA-SDDP) for solving stochastic ED problems. Taking advantage of the finite termination of PDBO for solving bilevel problems, the finite convergence of GA-SDDP is verified by the analysis.

Numerical tests on the IEEE 33-bus, 69-bus, and 123-bus systems have shown that GA-SDDP can achieve finite termination compared to SDDP and improved computational performance over FSDDP. A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.© Copyright 2025 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions. Scientific Reports volume 15, Article number: 40389 (2025) Cite this article In modern machine learning, optimization algorithms are crucial; they steer the training process by skillfully navigating through complex, high-dimensional loss landscapes. Among these, stochastic gradient descent with momentum (SGDM) is widely adopted for its ability to accelerate convergence in shallow regions.

However, SGDM struggles in challenging optimization landscapes, where narrow, curved valleys can lead to oscillations and slow progress. This paper introduces dual enhanced SGD (DESGD), which addresses these limitations by dynamically adapting both momentum and step size on the same update rules of SGDM. In two optimization test functions, the Rosenbrock and Sum Square functions, the suggested optimizer typically performs better than SGDM and Adam. For example, it accomplishes comparable errors while achieving up to 81–95% fewer iterations and 66–91% less CPU time than SGDM and 67–78% fewer iterations with 62–70% quicker runtimes than Adam. On the MNIST dataset, the proposed optimizer achieved the highest accuracies and lowest test losses across the majority of batch sizes. Compared to SGDM, they consistently improved accuracy by about 1–2%, while performing on par with or slightly better than Adam in accuracy and error.

Although SGDM remained the fastest per-step optimizer, our method’s computational cost is aligned with that of other adaptive optimizers like Adam. This marginal increase in per-iteration overhead is decisively justified by the substantial gains in model accuracy and reduction in training loss, demonstrating a favorable cost-to-performance ratio. The results demonstrate that DESGD is a promising practical optimizer to handle scenarios demanding stability in challenging landscapes. Machine learning (ML) optimization is critical to the development of models that exhibit efficiency, scalability and superior performance. With the continuous advancement of modern ML approaches, the necessity of optimization in the training of complex models, such as deep neural networks, is becoming more essential. Recent developments, such as gradient-based approaches, adaptive learning rate strategies and stochastic optimization techniques, have greatly enhanced the performance of models in various applications like disease diagnosis1,2,3, photovoltaic power forecasting4,5, large language models training6...

In addition, improving model performance plays a crucial role in enhancing computational efficiency, resulting in reduced training time, lower resource utilization and an increase in the accessibility and deployment of ML solutions in real... Thus, the area of ML optimization remains a promising research area, offering significant ideas that enhance the progress of artificial intelligence across diverse sectors. One of the powerful methods used in ML optimization is the gradient descent (GD), which minimizes the loss function \(J(\theta )\) where \(\theta\) are the model’s parameters by updating them in the negative direction... The step size to reach a local minimum is found by the learning rate \(\alpha\). The GD method has different variants according to the number of data samples that will be fed into the optimization process. Stochastic gradient descent (SGD) performs the parameters update using one data sample at a time instead of using all the data samples and completes one epoch i.e.

one iteration, after finishing all the data samples. About Ministry of Education Key Lab for Intelligent Networks and Network Security, Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China About Ministry of Education Key Lab for Intelligent Networks and Network Security, Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China About Ministry of Education Key Lab for Intelligent Networks and Network Security, Xi’an Jiaotong University, No.

28, Xianning West Road, Xi’an, 710049, Shaanxi, China About Ministry of Education Key Lab for Intelligent Networks and Network Security, Xi’an Jiaotong University, No. 28, Xianning West Road, Xi’an, 710049, Shaanxi, China Please login to MyJ-GLOBAL to see full information. You also need to select "Display abstract, etc. of medical articles" in your MyJ-GLOBAL account page in order to see abstracts, etc.

of medical articles. arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.© Copyright 2025 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

People Also Search

• Gradient accelerated stochastic dual dynamic programming for economic ...
• Fast Stochastic Dual Dynamic Programming for Economic Dispatch in ...
• Multi‐stage stochastic dual dynamic programming to low‐carbon economic ...
• A dual enhanced stochastic gradient descent method with dynamic ...
• Stochastic Dual Dynamic Programming and Its Variants: A Review
• Dual dynamic programming for stochastic programs over an infinite horizon
• Dual Stochastic Dual Dynamic Programming for Multi-Stage Economic ...