arXiv subjects

Kansei Ushiyama

Publications and source records attributed to Kansei Ushiyama.

6 recordsLinked to original sources

arXiv2026

A restart-free accelerated algorithm for nonconvex minimization: continuous- and discrete-time analysis

We propose two deterministic momentum-based first-order methods for minimizing lower-bounded nonconvex functions with Lipschitz-continuous gradients and Hessians. Both methods attain an $\varepsilon$-approximate first-order stationary point in $O(\varepsilon^{-7/4})$ function and gradient evaluations, and their update parameters do not depend on the target accuracy or a prescribed terminal iteration. The convergence guarantees concern the best point among weighted averages of the iterates. Instead of relying on momentum-resetting restarts, both methods employ a velocity-control mechanism that adaptively reduces the velocity. The first method uses a differentiable velocity-control rule, making its last iterate differentiable with respect to the initial point. To the best of our knowledge, this is the first deterministic first-order method to combine this differentiability property with the stated complexity. The second method adaptively estimates the gradient and Hessian Lipschitz constants and requires no prior knowledge of their values. Numerical experiments show that it outperforms existing parameter-free methods on some test problems. Both methods are derived by discretizing a new continuous-time model described by an ordinary differential equation. We develop parallel continuous- and discrete-time convergence analyses within the performance estimation problem framework.

math.OC↗

arXiv2026

A $\sqrt{2}$-accelerated FISTA for composite strongly convex problems

In this paper, we propose a novel accelerated forward-backward splitting algorithm for minimizing composite convex functions expressed as the sum of a smooth function and a possibly nonsmooth function. When the composite objective is strongly convex, the proposed method achieves a linear convergence rate in objective value that improves the leading constant in the exponent by a factor of $\sqrt{2}$ relative to FISTA and also improves upon the best previously known convergence rate for this problem class. Our convergence analysis remains valid even when one of the two component functions is weakly convex, provided that their sum remains convex. The proposed algorithm is derived by discretizing a continuous-time model of the Information-Theoretic Exact Method (ITEM), an optimal first-order method for unconstrained smooth strongly convex minimization.

math.OC↗

arXiv2025

Essential Convergence Rates of Continuous-Time Models for Optimization Methods

Designing and analyzing optimization methods via continuous-time models expressed as ordinary differential equations (ODEs) is a promising approach for its intuitiveness and simplicity. A key concern, however, is that the convergence rates of such models can be arbitrarily modified by time rescaling, rendering the task of seeking ODEs with ``fast'' convergence meaningless. To eliminate this ambiguity of the rates, we introduce the notion of the essential convergence rate. We justify this notion by proving that, under appropriate assumptions on discretization, no method obtained by discretizing an ODE can achieve a faster rate than its essential convergence rate.

math.OC↗

arXiv2024

Differentiable Pareto-Smoothed Weighting for High-Dimensional Heterogeneous Treatment Effect Estimation

There is a growing interest in estimating heterogeneous treatment effects across individuals using their high-dimensional feature attributes. Achieving high performance in such high-dimensional heterogeneous treatment effect estimation is challenging because in this setup, it is usual that some features induce sample selection bias while others do not but are predictive of potential outcomes. To avoid losing such predictive feature information, existing methods learn separate feature representations using inverse probability weighting (IPW). However, due to their numerically unstable IPW weights, these methods suffer from estimation bias under a finite sample setup. To develop a numerically robust estimator by weighted representation learning, we propose a differentiable Pareto-smoothed weighting framework that replaces extreme weight values in an end-to-end fashion. Our experimental results show that by effectively correcting the weight values, our proposed method outperforms the existing ones, including traditional weighting schemes. Our code is available at https://github.com/ychika/DPSW.

stat.ML↗

arXiv2023

A new unified framework for designing convex optimization methods with prescribed theoretical convergence estimates: A numerical analysis approach

We propose a new unified framework for describing and designing gradient-based convex optimization methods from a numerical analysis perspective. There the key is the new concept of weak discrete gradients (weak DGs), which is a generalization of DGs standard in numerical analysis. Via weak DG, we consider abstract optimization methods, and prove unified convergence rate estimates that hold independent of the choice of weak DGs except for some constants in the final estimate. With some choices of weak DGs, we can reproduce many popular existing methods, such as the steepest descent and Nesterov's accelerated gradient method, and also some recent variants from numerical analysis community. By considering new weak DGs, we can easily explore new theoretically-guaranteed optimization methods; we show some examples. We believe this work is the first attempt to fully integrate research branches in optimization and numerical analysis areas, so far independently developed.

math.OC↗

arXiv2022

Essential convergence rate of ordinary differential equations appearing in optimization

Some continuous optimization methods can be connected to ordinary differential equations (ODEs) by taking continuous limits, and their convergence rates can be explained by the ODEs. However, since such ODEs can achieve any convergence rate by time scaling, the correspondence is not as straightforward as usually expected, and deriving new methods through ODEs is not quite direct. In this letter, we pay attention to stability restriction in discretizing ODEs and show that acceleration by time scaling basically implies deceleration in discretization; they balance out so that we can define an attainable unique convergence rate which we call an "essential convergence rate".

math.NA↗