Learning to Optimize by Differentiable Programming

Liping Tao; Xindi Tong; Chee Wei Tan

Home/Archive/arXiv

arXiv · 2601.16510

Learning to Optimize by Differentiable Programming

Liping Tao·Xindi Tong·Chee Wei Tan

View original source ↗Read source document ↗Export metadata ↓

Abstract

Solving massive-scale optimization problems requires scalable first-order methods with low per-iteration cost. This tutorial highlights a shift in optimization: using differentiable programming not only to execute algorithms but to learn how to design them. Modern frameworks such as PyTorch, TensorFlow, and JAX enable this paradigm through efficient automatic differentiation. Embedding first-order methods within these systems allows end-to-end training that improves convergence and solution quality. Guided by Fenchel-Rockafellar duality, the tutorial demonstrates how duality-informed iterative schemes such as the alternating direction method of multipliers, and the primal-dual hybrid gradient can be learned and adapted through representative case studies.

Explore related subjects

cs.MS cs.LG math.OC

Keep this discovery

BibTeX RIS

Liping Tao, Xindi Tong, Chee Wei Tan. 2026-08-30. Learning to Optimize by Differentiable Programming. https://arxiv.org/abs/2601.16510

Cite the original work for its findings. Save a collection to share your selection of sources.

Discover connections

cs.LG subject cs.MS subject math.OC subject

Connections use source metadata and explicit phrase matches, not verified experimental comparisons.

KEEP EXPLORING

Related discoveries

arXiv2026

Fast Relax-and-Round Unit Commitment with Economic Horizons

The US energy system is increasingly under pressure to serve expanding data loads and to accommodate a larger number of generating units with varying technologies and own- ership structures. Therefore, developing new unit commitment methods remains a priority for reliable and affordable grid operations. We expand our novel computational method for unit commitment (UC) to include ramping constraints and long- horizon planning and provide a theoretical bound on its error. We introduce a fast novel algorithm to commit hydro-generators. We solve problems with thousands of generators at 5-minute market intervals. We show that our method can solve UC problems with over 20,000 generators in approximately 10 seconds on commodity hardware and that an increased planning horizon leads to sizable operational cost savings. We attain this runtime improvement by introducing a heuristic tailored for UC problems. Our method can be implemented using existing continuous optimization solvers and adapted for different applications. We prove a bound on the error of these solvers and show that it vanishes (in relative terms) as the problem becomes larger. We also introduce a fast and accurate hydro UC algorithm. Combined, these algorithms would allow an operator to make horizon-aware economic decisions for large systems with hydro units.

math.OC↗

arXiv2026

Rates of Convergence in the Central Limit Theorem for Markov Chains, with an Application to TD Learning

We prove a non-asymptotic central limit theorem for vector-valued martingale differences using Stein's method, and use Poisson's equation to extend the result to functions of Markov Chains. We then show that these results can be applied to establish a non-asymptotic central limit theorem for Temporal Difference (TD) learning with averaging.

math.PR↗

arXiv2026

Sharp Restricted Isometry Thresholds for Global Minima of Rank-Restricted Matrix LASSO

We determine the sharp restricted isometry threshold for recovery at global minima of the rank-restricted matrix LASSO. For target rank $r_{\star}$, if the rank-$k$ RIP constant satisfies $δ<δ_{\mathrm{sharp}}(k/r_{\star})$, where $δ_{\mathrm{sharp}}(t)=t/(4-t)$ for $0<t<4/3$ and $δ_{\mathrm{sharp}}(t)=\sqrt{(t-1)/t}$ for $t\ge4/3$, then every global minimizer has Frobenius error $\lesssim\sqrt{r_{\star}}λ$ for all $λ\gtrsim\|\mathcal{A}^{*}(ξ)\|_{\mathrm{op}}$ and at every search rank $r\ge r_{\star}$. The constants depend only on the RIP constant and $t=k/r_{\star}$, and in particular are independent of the search rank. When the rank restriction is inactive, the result specializes to the ordinary convex matrix LASSO. We also obtain the analogous results for sparsity-restricted vector LASSO. Conversely, we show that the threshold $δ<δ_{\mathrm{sharp}}(k/r_{\star})$ cannot be improved, due to the existence of counterexamples whose global minimizers fail to recover the ground truth.

stat.ML↗