arXiv subjects

Wei Bian

Publications and source records attributed to Wei Bian.

At least 19 recordsLinked to original sources

arXiv2026

A nonmonotone globalization framework for Anderson acceleration for contractive and nonexpansive fixed point problems

Anderson acceleration (AA) is an effective technique for accelerating fixed point iterations, but it generally lacks global convergence guarantees. We propose a nonmonotone globalized Anderson acceleration framework that retains the local acceleration of AA while ensuring global convergence. For contractive mappings, without requiring prior knowledge of the contraction factor, we prove that the proposed method reduces exactly to pure AA after finitely many iterations and enjoys global $r$-linear convergence for memory size $m\geq1$ and global residual $q$-linear convergence for $m=1$, with convergence factors no greater than the contraction factor of the underlying fixed point mapping. For nonexpansive mappings, we prove that the fixed point residuals converge globally to zero. A class-agnostic parameter selection rule is further developed to ensure these convergence properties in both settings. Numerical experiments demonstrate the robustness and efficiency of the proposed method.

math.NA↗

arXiv2026

An extra gradient Anderson-accelerated algorithm for pseudomonotone variational inequalities

This paper proposes an extra gradient Anderson-accelerated algorithm for solving pseudomonotone variational inequalities, which uses the extra gradient scheme with line search to guarantee the global convergence and Anderson acceleration to have fast convergent rate. We prove that the sequence generated by the proposed algorithm from any initial point converges to a solution of the pseudomonotone variational inequality problem without assuming the Lipschitz continuity and contractive condition, which are used for convergence analysis of the extra gradient method and Anderson-accelerated method, respectively in existing literatures. Numerical experiments, particular emphasis on Harker-Pang problem, fractional programming, nonlinear complementarity problem and PDE problem with free boundary, are conducted to validate the effectiveness and good performance of the proposed algorithm comparing with the extra gradient method and Anderson-accelerated method.

math.OC↗

arXiv2026

An Anderson-accelerated stochastic extragradient method for stochastic variational inequalities

In this paper, we propose an Anderson-accelerated stochastic extragradient algorithm for solving a class of stochastic variational inequalities, by incorporating Anderson acceleration into the stochastic extragradient method under a stochastic approximation framework. A key challenge in our setting is that the pseudomonotonicity assumption is only imposed on the expectation of the stochastic operator, rather than on the individual stochastic operator itself and the sample averages utilized in the algorithm. We prove that, despite the lack of pseudomonotonicity in the sampled operators, the sequence generated by the proposed algorithm converges almost surely to a solution of the stochastic variational inequality problem. Additionally, we establish the sublinear convergence rate of the proposed algorithm in terms of the mean residual function, along with its optimal oracle complexity. Finally, we validate the effectiveness of the proposed algorithm through numerical experiments.

math.OC↗

arXiv2026

TV Subgradient-Guided Multi-Source Fusion for Spectral Imaging in Dual-Camera CASSI Systems

Balancing spectral, spatial, and temporal resolutions is a key challenge in spectral imaging. The Dual-Camera Coded Aperture Snapshot Spectral Imaging (DC-CASSI) system alleviates this trade-off but suffers from severely ill-posed reconstruction problems due to its high compression ratio. Existing methods are constrained by scene-specific tuning or excessive reliance on paired training data. To address these issues, we propose a Total Variation (TV) subgradient-guided multi-source fusion framework for DC-CASSI reconstruction, comprising three core components: (1) An end-to-end Single-Disperser CASSI (SD-CASSI) observation model based on the tensor-form Kronecker $δ$, which establishes a rigorous mathematical foundation for physical constraints while enabling efficient adjoint operator implementation; (2) An adaptive spatial reference generator that integrates SD-CASSI's physical model and RGB subspace constraint, generating the reference image as reliable spatial prior; (3) A TV subgradient-guided regularization term that encodes local structural directions from the reference image into spectral reconstruction, achieving high-quality fused results. The framework is validated on simulated datasets and real-world datasets. Experimental results demonstrate that it achieves state-of-the-art reconstruction performance and robust noise resilience. This work not only establishes an interpretable theoretical foundation for subgradient-guided fusion but also provides a practical fusion-based paradigm for high-fidelity spectral image reconstruction in DC-CASSI systems. Source code: https://github.com/bestwishes43/ADMM-TVDS.

cs.CV↗

arXiv2026

Robust principal component analysis with rank and cardinality regularization under matrix factorization

Robust principal component analysis is an important representative method in data analysis. It is usually viewed as an optimization problem involving the rank and $\ell_0$-norm of matrices. In this paper, we study the rank and $\ell_0$ regularized optimization problem and its matrix factorization problem. We establish their equivalences on global minimizers and stationary points, respectively. Furthermore, we construct a broadly applicable equivalent nonconvex relaxation framework for the constrained factorization model in the sense of global minimizers and stationary points with strong optimality conditions (called strong stationary points). For the general factorization problem with lower semicontinuous regularizers and a loss function whose gradient is locally Lipschitz, we propose a novel proximal gradient-based algorithm based on joint and alternating calculation with convergence to its limiting-critical points. The algorithm can attain the stationary points of the original problem and its adaptive counterpart can attain the strong stationary points of the factorization problem.

math.OC↗

arXiv2025

Normal Transformer: Extracting Surface Geometry from LiDAR Points Enhanced by Visual Semantics

High-quality surface normal can help improve geometry estimation in problems faced by autonomous vehicles, such as collision avoidance and occlusion inference. While a considerable volume of literature focuses on densely scanned indoor scenarios, normal estimation during autonomous driving remains an intricate problem due to the sparse, non-uniform, and noisy nature of real-world LiDAR scans. In this paper, we introduce a multi-modal technique that leverages 3D point clouds and 2D colour images obtained from LiDAR and camera sensors for surface normal estimation. We present the Hybrid Geometric Transformer (HGT), a novel transformer-based neural network architecture that proficiently fuses visual semantic and 3D geometric information. Furthermore, we developed an effective learning strategy for the multi-modal data. Experimental results demonstrate the superior effectiveness of our information fusion approach compared to existing methods. It has also been verified that the proposed model can learn from a simulated 3D environment that mimics a traffic scene. The learned geometric knowledge is transferable and can be applied to real-world 3D scenes in the KITTI dataset. Further tasks built upon the estimated normal vectors in the KITTI dataset show that the proposed estimator has an advantage over existing methods.

cs.CV↗

arXiv2025

Extrapolated Hard Thresholding Algorithms with Finite Length for Composite $\ell_0$ Penalized Problems

For a class of sparse optimization problems with the penalty function of $\|(\cdot)_+\|_0$, we first characterize its local minimizers and then propose an extrapolated hard thresholding algorithm to solve such problems. We show that the iterates generated by the proposed algorithm with $ε>0$ (where $ε$ is the dry friction coefficient) have finite length, without relying on the Kurdyka-Łojasiewicz inequality. Furthermore, we demonstrate that the algorithm converges to an $ε$-local minimizer of this problem. For the special case that $ε=0$, we establish that any accumulation point of the iterates is a local minimizer of the problem. Additionally, we analyze the convergence when an error term is present in the algorithm, showing that the algorithm still converges in the same manner as before, provided that the errors asymptotically approach zero. Finally, we conduct numerical experiments to verify the theoretical results of the proposed algorithm.

math.OC↗

arXiv2025

A consensus-based optimization method for nonsmooth nonconvex programs with approximated gradient descent scheme

In this paper, we are interested in finding the global minimizer of a nonsmooth nonconvex unconstrained optimization problem. By combining the discrete consensus-based optimization (CBO) algorithm and the gradient descent method, we develop a novel CBO algorithm with an extra gradient descent scheme evaluated by the forward-difference technique on the function values, where only the objective function values are used in the proposed algorithm. First, we prove that the proposed algorithm can exhibit global consensus in an exponential rate in two senses and possess a unique global consensus point. Second, we evaluate the error estimate between the objective function value on the global consensus point and its global minimum. In particular, as the parameter $β$ tends to $\infty$, the error converges to zero and the convergence rate is $\mathcal{O}\left(\frac{\logβ}β\right)$. Third, under some suitable assumptions on the objective function, we provide the number of iterations required for the mean square error in expectation to reach the desired accuracy. It is worth underlining that the theoretical analysis in this paper does not use the mean-field limit. Finally, we illustrate the improved efficiency and promising performance of our novel CBO method through some experiments on several nonconvex benchmark problems and the application to train deep neural networks.

math.OC↗

arXiv2025

Smoothing Iterative Consensus-based Optimization Algorithm for Nonsmooth Nonconvex Optimization Problems with Global Optimality

In this paper, we focus on finding the global minimizer of a general unconstrained nonsmooth nonconvex optimization problem. Taking advantage of the smoothing method and the consensus-based optimization (CBO) method, we propose a novel smoothing iterative consensus-based optimization (SICBO) algorithm. First, we prove that the solution process of the proposed algorithm here exponentially converges to a common stochastic consensus point almost surely. Second, we establish a detailed theoretical analysis to ensure the small enough error between the objective function value at the consensus point and the optimal function value, to the best of our knowledge, which provides the first theoretical guarantee to the global optimality of the proposed algorithm for nonconvex optimization problems. Moreover, unlike the previously introduced CBO methods, the theoretical results are valid for the cases that the objective function is nonsmooth, nonconvex and perhaps non-Lipschitz continuous. Finally, several numerical examples are performed to illustrate the effectiveness of our proposed algorithm for solving the global minimizer of the nonsmooth and nonconvex optimization problems.

math.OC↗

arXiv2025

A Smoothing Consensus-Based Optimization Algorithm for Nonsmooth Nonconvex Optimization

Lately, a novel swarm intelligence model, namely the consensus-based optimization (CBO) algorithm, was introduced to deal with the global optimization problems. Limited by the conditions of Ito's formula, the convergence analysis of the previous CBO finite particle system mainly focuses on the problem with smooth objective function. With the help of smoothing method, this paper achieves a breakthrough by proposing an effective CBO algorithm for solving the global solution of a nonconvex, nonsmooth, and possible non-Lipschitz continuous minimization problem with theoretical analysis, which dose not rely on the mean-field limit. We indicate that the proposed algorithm exhibits a global consensus and converges to a common state with any initial data. Then, we give a more detailed error estimation on the objective function values along the state of the proposed algorithm towards the global minimum. Finally, some numerical examples are presented to illustrate the appreciable performance of the proposed method on solving the nonsmooth, nonconvex minimization problems.

math.OC↗

arXiv2024

A smoothing Anderson acceleration algorithm for nonsmooth fixed point problem with linear convergence

In this paper, we consider the Anderson acceleration method for solving the contractive fixed point problem, which is nonsmooth in general. We define a class of smoothing functions for the original nonsmooth fixed point mapping, which can be easily formulated for many cases (see section3). Then, taking advantage of the Anderson acceleration method, we proposed a Smoothing Anderson(m) algorithm, in which we utilized a smoothing function of the original nonsmooth fixed point mapping and update the smoothing parameter adaptively. In theory, we first demonstrate the r-linear convergence of the proposed Smoothing Anderson(m) algorithm for solving the considered nonsmooth contractive fixed point problem with r-factor no larger than c, where c is the contractive factor of the fixed point mapping. Second, we establish that both of the Smoothing Anderson(1) and the Smoothing EDIIS(1) algorithms are q-linear convergent with q-factor no larger than c. Finally, we present three numerical examples with practical applications from elastic net regression, free boundary problems for infinite journal bearings and non-negative least squares problem to illustrate the better performance of the proposed Smoothing Anderson(m) algorithm comparing with some popular methods.

math.OC↗

arXiv2024

Global-guided Focal Neural Radiance Field for Large-scale Scene Rendering

Neural radiance fields~(NeRF) have recently been applied to render large-scale scenes. However, their limited model capacity typically results in blurred rendering results. Existing large-scale NeRFs primarily address this limitation by partitioning the scene into blocks, which are subsequently handled by separate sub-NeRFs. These sub-NeRFs, trained from scratch and processed independently, lead to inconsistencies in geometry and appearance across the scene. Consequently, the rendering quality fails to exhibit significant improvement despite the expansion of model capacity. In this work, we present global-guided focal neural radiance field (GF-NeRF) that achieves high-fidelity rendering of large-scale scenes. Our proposed GF-NeRF utilizes a two-stage (Global and Focal) architecture and a global-guided training strategy. The global stage obtains a continuous representation of the entire scene while the focal stage decomposes the scene into multiple blocks and further processes them with distinct sub-encoders. Leveraging this two-stage architecture, sub-encoders only need fine-tuning based on the global encoder, thus reducing training complexity in the focal stage while maintaining scene-wide consistency. Spatial information and error information from the global stage also benefit the sub-encoders to focus on crucial areas and effectively capture more details of large-scale scenes. Notably, our approach does not rely on any prior knowledge about the target scene, attributing GF-NeRF adaptable to various large-scale scene types, including street-view and aerial-view scenes. We demonstrate that our method achieves high-fidelity, natural rendering results on various types of large-scale datasets. Our project page: https://shaomq2187.github.io/GF-NeRF/

cs.CV↗

arXiv2024

On solving a rank regularized minimization problem via equivalent factorized column-sparse regularized models

Rank regularized minimization problem is an ideal model for the low-rank matrix completion/recovery problem. The matrix factorization approach can transform the high-dimensional rank regularized problem to a low-dimensional factorized column-sparse regularized problem. The latter can greatly facilitate fast computations in applicable algorithms, but needs to overcome the simultaneous non-convexity of the loss and regularization functions. In this paper, we consider the factorized column-sparse regularized model. Firstly, we optimize this model with bound constraints, and establish a certain equivalence between the optimized factorization problem and rank regularized problem. Further, we strengthen the optimality condition for stationary points of the factorization problem and define the notion of strong stationary point. Moreover, we establish the equivalence between the factorization problem and its a nonconvex relaxation in the sense of global minimizers and strong stationary points. To solve the factorization problem, we design two types of algorithms and give an adaptive method to reduce their computation. The first algorithm is from the relaxation point of view and its iterates own some properties from global minimizers of the factorization problem after finite iterations. We give some analysis on the convergence of its iterates to the strong stationary point. The second algorithm is designed for directly solving the factorization problem. We improve the PALM algorithm introduced by Bolte et al. (Math Program Ser A 146:459-494, 2014) for the factorization problem and give its improved convergence results. Finally, we conduct numerical experiments to show the promising performance of the proposed model and algorithms for low-rank matrix completion.

math.OC↗

arXiv2024

Nonsmooth convex-concave saddle point problems with cardinality penalties

In this paper, we focus on a class of convexly constrained nonsmooth convex-concave saddle point problems with cardinality penalties. Although such nonsmooth nonconvex-nonconcave and discontinuous min-max problems may not have a saddle point, we show that they have a local saddle point and a global minimax point, and some local saddle points have the lower bound properties. We define a class of strong local saddle points based on the lower bound properties for stability of variable selection. Moreover, we give a framework to construct continuous relaxations of the discontinuous min-max problems based on the convolution, such that they have the same saddle points with the original problem. We also establish the relations between the continuous relaxation problems and the original problems regarding local saddle points, global minimax points, local minimax points and stationary points. Finally, we illustrate our results with distributionally robust sparse convex regression, sparse robust bond portfolio construction and sparse convex-concave logistic regression saddle point problems.

math.OC↗

arXiv2022

Anderson Acceleration for Nonsmooth Fixed Point Problems

We give new convergence results of Anderson acceleration for the composite $\max$ fixed point problem. We prove that Anderson(1) and EDIIS(1) are q-linear convergent with a smaller q-factor than existing q-factors. Moreover, we propose a smoothing approximation of the composite max function in the contractive fixed point problem. We show that the smoothing approximation is a contraction mapping with the same fixed point as the composite $\max$ fixed point problem. Our results rigorously confirm that the nonsmoothness does not affect the convergence rate of Anderson acceleration method when we use the proposed smoothing approximation for the composite $\max$ fixed point problem. Numerical results for constrained minimax problems, complementarity problems and nonsmooth differential equations are presented to show the efficiency and good performance of the proposed Anderson acceleration method with smoothing approximation.

math.OC↗

arXiv2022

DC algorithms for a class of sparse group $\ell_0$ regularized optimization problems

In this paper, we consider a class of sparse group $\ell_0$ regularized optimization problems. Firstly, we give a continuous relaxation model of the considered problem and establish the equivalence of these two problems in the sense of global minimizers. Then, we define a class of stationary points of the relaxation problem, and prove that any defined stationary point is a local minimizer of the considered sparse group $\ell_0$ regularized problem and satisfies a desirable property of its global minimizers. Further, based on the difference-of-convex (DC) structure of the relaxation problem, we design two DC algorithms to solve the relaxation problem. We prove that any accumulation point of the iterates generated by them is a stationary point of the relaxation problem. In particular, all accumulation points have a common support set and a unified lower bound for the nonzero entries, and their zero entries can be attained within finite iterations. Moreover, we prove the convergence of the entire iterates generated by the proposed algorithms. Finally, we give some numerical experiments to show the efficiency of the proposed algorithms.

math.OC↗

arXiv2022

Unsupervised Domain Adaptation with Implicit Pseudo Supervision for Semantic Segmentation

Pseudo-labelling is a popular technique in unsuper-vised domain adaptation for semantic segmentation. However, pseudo labels are noisy and inevitably have confirmation bias due to the discrepancy between source and target domains and training process. In this paper, we train the model by the pseudo labels which are implicitly produced by itself to learn new complementary knowledge about target domain. Specifically, we propose a tri-learning architecture, where every two branches produce the pseudo labels to train the third one. And we align the pseudo labels based on the similarity of the probability distributions for each two branches. To further implicitly utilize the pseudo labels, we maximize the distances of features for different classes and minimize the distances for the same classes by triplet loss. Extensive experiments on GTA5 to Cityscapes and SYNTHIA to Cityscapes tasks show that the proposed method has considerable improvements.

cs.CV↗

arXiv2021

Fast inertial dynamic algorithm with smoothing method for nonsmooth convex optimization

In order to solve the minimization of a nonsmooth convex function, we design an inertial second-order dynamic algorithm, which is obtained by approximating the nonsmooth function by a class of smooth functions. By studying the asymptotic behavior of the dynamic algorithm, we prove that each trajectory of it weakly converges to an optimal solution under some appropriate conditions on the smoothing parameters, and the convergence rate of the objective function values is o(t^-2). We also show that the algorithm is stable, that is, this dynamic algorithm with a perturbation term owns the same convergence properties when the perturbation term satisfies certain conditions. Finally, we verify the theoretical results by some numerical experiments.

math.OC↗