arXiv subjects

Yuning Yang

Publications and source records attributed to Yuning Yang.

At least 19 recordsLinked to original sources

arXiv2026

Optimal Parameter-Free Gradient Minimization in $\ell_p$ Geometry

We study the first-order oracle complexity of finding a queried point with small gradient in $\ell_p$ geometry, with particular attention to the information needed to adapt the unknown smoothness and distance scales. In the strict counted local value--gradient model, no finite complexity bound can depend only on $LR/\eps$ without a nondegenerate local scale observation: a one-dimensional construction keeps $LR/\eps=4$ while defeating every prescribed finite query budget. We resolve Diakonikolas's general-$\ell_p$ parameter-free extension question for every fixed $1 2$, we first establish the dimension-free deterministic known-parameter upper exponent $p/(p+2)$ in $K=LR/\eps$, matching the published lower polynomial exponent under its horizon and dimension qualifications. The finite local routine fits the same observable scale--radius procedure, so this exponent is preserved without knowing $L$ or $R$. Writing $\Kbar=\max\{1,LR/\eps\}$, the post-initialization pair-oracle complexity is $O_p(\Kbar^{1/2})$ for $1 2$, together with the additive calibration cost $O_p(\log(e+L/M_0))$ in every regime.

math.OC↗

arXiv2026

Improving Sketching Algorithms for Low-Rank Matrix Approximation via Sketch-Power Iterations

Power iteration can improve the accuracy of randomized SVD, but requires multiple data passes, making it impractical in streaming or memory-constrained settings. We introduce a lightweight yet effective sketch-power iteration, allowing power-like iterations with only a single pass of the data, which can be incorporated into one-pass algorithms for low-rank approximation. As an example, we integrate the sketch-power iteration into a one-pass algorithm proposed by Tropp et al., and introduce strategies to reduce its storage cost. We establish meaningful error bounds: given a fixed storage budget, the sketch sizes derived from the bounds closely match the optimal ones observed in reality. This allows one to preselect reasonable parameters. Numerical experiments on both synthetic and real-world datasets indicate that, under the same storage constraints, applying one or two sketch-power iterations can substantially improve the approximation accuracy of the considered one-pass algorithms. In particular, experiments on real data with flat spectrum show that the method can approximate the dominant singular vectors well.

math.NA↗

arXiv2025

Towards Group Fairness with Multiple Sensitive Attributes in Federated Foundation Models

The deep integration of foundation models (FM) with federated learning (FL) enhances personalization and scalability for diverse downstream tasks, making it crucial in sensitive domains like healthcare. Achieving group fairness has become an increasingly prominent issue in the era of federated foundation models (FFMs), since biases in sensitive attributes might lead to inequitable treatment for under-represented demographic groups. Existing studies mostly focus on achieving fairness with respect to a single sensitive attribute. This renders them unable to provide clear interpretability of dependencies among multiple sensitive attributes which is required to achieve group fairness. Our paper takes the first attempt towards a causal analysis of the relationship between group fairness across various sensitive attributes in the FFM. We extend the FFM structure to trade off multiple sensitive attributes simultaneously and quantify the causal effect behind the group fairness through causal discovery and inference. Extensive experiments validate its effectiveness, offering insights into interpretability towards building trustworthy and fair FFM systems.

cs.LG↗

arXiv2024

Value-Spectrum: Quantifying Preferences of Vision-Language Models via Value Decomposition in Social Media Contexts

The recent progress in Vision-Language Models (VLMs) has broadened the scope of multimodal applications. However, evaluations often remain limited to functional tasks, neglecting abstract dimensions such as personality traits and human values. To address this gap, we introduce Value-Spectrum, a novel Visual Question Answering (VQA) benchmark aimed at assessing VLMs based on Schwartz's value dimensions that capture core human values guiding people's preferences and actions. We design a VLM agent pipeline to simulate video browsing and construct a vector database comprising over 50,000 short videos from TikTok, YouTube Shorts, and Instagram Reels. These videos span multiple months and cover diverse topics, including family, health, hobbies, society, technology, etc. Benchmarking on Value-Spectrum highlights notable variations in how VLMs handle value-oriented content. Beyond identifying VLMs' intrinsic preferences, we also explore the ability of VLM agents to adopt specific personas when explicitly prompted, revealing insights into the adaptability of the model in role-playing scenarios. These findings highlight the potential of Value-Spectrum as a comprehensive evaluation set for tracking VLM preferences in value-based tasks and abilities to simulate diverse personas. The complete code and data are available at: https://github.com/Jeremyyny/Value-Spectrum.

cs.CL↗

arXiv2024

TimeAutoDiff: A Unified Framework for Generation, Imputation, Forecasting, and Time-Varying Metadata Conditioning of Heterogeneous Time Series Tabular Data

We present TimeAutoDiff, a unified latent-diffusion framework for four fundamental time-series tasks: unconditional generation, missing-data imputation, forecasting, and time-varying-metadata conditional generation. The model natively supports heterogeneous features including continuous, binary, and categorical variables. We unify all tasks using a masked-modeling strategy in which a binary mask specifies which time-series cells are observed and which must be generated. TimeAutoDiff combines a lightweight variational autoencoder, which maps mixed-type features into a continuous latent sequence, with a diffusion model that learns temporal dynamics in this latent space. Two architectural choices provide strong speed and scalability benefits. The diffusion model samples an entire latent trajectory at once rather than denoising one timestep at a time, greatly reducing reverse-diffusion calls. In addition, the VAE compresses along the feature axis, enabling efficient modeling of wide tables in a low-dimensional latent space. Empirical evaluation shows that TimeAutoDiff matches or surpasses strong baselines in synthetic sequence fidelity and consistently improves imputation and forecasting performance. Metadata conditioning enables realistic scenario exploration, allowing users to edit metadata sequences and produce coherent counterfactual trajectories that preserve cross-feature dependencies. Ablation studies highlight the importance of the VAE's feature encoding and key components of the denoiser. A distance-to-closest-record audit further indicates that the model generalizes without excessive memorization. Code is available at https://github.com/namjoonsuh/TimeAutoDiff

cs.LG↗

arXiv2024

SPD-CFL: Stepwise Parameter Dropout for Efficient Continual Federated Learning

Federated Learning (FL) is a collaborative machine learning paradigm for training models on local sensitive data with privacy protection. Pre-trained transformer-based models have emerged as useful foundation models (FMs) to be fine-tuned for a wide range of downstream tasks. However, large-scale pre-trained models make it challenging for traditional FL due to high communication overhead in the resource-constrained IoT. This has inspired the field of parameter-efficient fine-tuning (PEFT) research. Existing PEFT methods attempt to optimize model performance at the given dropout level. Such an approach places the burden on human users to find a dropout rate that provides a satisfactory level of performance through trial-and-error, which is time consuming and resource intensive. To address this limitation, we propose the Step-wise Parameter Dropout for Continual Federated Learning (SPD-CFL) approach. Instead of pre-defining a desired dropout rate, it allows users to specify the target level of performance and then attempts to find the most suitable dropout rate for the given FL model. Specifically, on the server side, SPD-CFL drops trainable parameters in a stepwise manner to improve communication efficiency by reducing the rank of low-rank adaptation (LoRA). The sensitivity-based gradient consistency (SGC) measure is designed to facilitate the adaptive adjustment of parameter dropout. In addition, SPD-CFL introduces continual learning (CL) on the client side to mitigate performance degradation due to the inconsistent optima with distinct parameter dropout rates under heterogeneous FL. Extensive experiments on the public benchmark dataset CIFAR-10 and a real-world medical Face dataset demonstrate significant superiority of SPD-CFL over state-of-the-art methods. Compared to the best-performing baseline, it achieves a 2.07% higher test AUC while reducing communication overhead by 29.53%.

cs.LG↗

arXiv2024

Randomized Large-Scale Quaternion Matrix Approximation: Practical Rangefinders and One-Pass Algorithm

Recently, randomized algorithms for low-rank approximation of quaternion matrices have received increasing attention. However, for large-scale problems, existing quaternion orthonormalizations are inefficient, leading to slow rangefinders. To address this, by appropriately leveraging efficient scientific computing libraries in the complex arithmetic, this work devises two practical quaternion rangefinders, one of which is non-orthonormal yet well-conditioned. They are then integrated into the quaternion version of a one-pass algorithm, which originally takes orthonormal rangefinders only. We establish the error bounds and demonstrate that the error is proportional to the condition number of the rangefinder. The probabilistic bounds are exhibited for both quaternion Gaussian and sub-Gaussian embeddings. Numerical experiments demonstrate that the one-pass algorithm with the proposed rangefinders significantly outperforms previous techniques in efficiency. Additionally, we tested the algorithm in a 3D Navier-Stokes equation ($5.22$GB) and a 4D Lorenz-type chaotic system ($5.74$GB) data compression, as well as a $31365\times 27125$ image compression to demonstrate its capability for handling large-scale applications.

math.NA↗

arXiv2023

A Parallelizable Quaternion Higher-Order Singular Value Decomposition with Applications

Higher-order singular value decomposition (HOSVD) is a celebrated tool for tensor data analysis. The sequential HOSVD was recently generalized to the quaternion domain, while a naive quaternion extension of the classical HOSVD% by De Lathauwer et al., which can be excecuted in parallel, incurs issues. To leverage the power of parallel computing, this work introduces a two-sided quaternion HOSVD (TS-QHOSVD) that can be parallelized on two processors. It is proved that TS-QHOSVD (i) preserves the HOSVD ordering property, (ii) inherits the orthogonality property at the first and the last modes, and (iii) satisfies the weak orthogonality at all modes. The truncated TS-QHOSVD is then developed, with its error bound being established. We apply the proposed model on color video denoising as well as scientific data compression arising from 3D Navier-Stokes equation and Lorentz system to demonstrate its efficacy.

math.NA↗

arXiv2023

On Finite-Step Convergence of the Non-Greedy Algorithm and Proximal Alternating Minimization Method with Extrapolation for $L_1$-Norm PCA

The classical non-greedy algorithm (NGA) and the recently proposed proximal alternating minimization method with extrapolation (PAMe) for $L_1$-norm PCA are revisited and their finite-step convergence are studied. It is first shown that NGA can be interpreted as a conditional subgradient or an alternating maximization method. By recognizing it as a conditional subgradient, we prove that the iterative points generated by the algorithm will be constant in finitely many steps under a certain full-rank assumption; such an assumption can be removed when the projection dimension is one. By treating the algorithm as an alternating maximization, we then prove that the objective value will be fixed after at most $\left\lceil\frac{F^{\max}}{\tau_0} \right\rceil$ steps, where the stopping point satisfies certain optimality conditions. Then, a slight modification of NGA with improved convergence properties is analyzed. It is shown that the iterative points generated by the modified algorithm will not change after at most $\left\lceil\frac{2F^{\max}}{\tau} \right\rceil$ steps; furthermore, the stopping point satisfies certain optimality conditions if the proximal parameter $\tau$ is small enough. For PAMe, it is proved that the sign variable will remain constant after finitely many steps and the algorithm can output a point satisfying certain optimality condition, if the parameters are small enough and a full rank assumption is satisfied. Moreover, if there is no proximal term on the projection matrix related subproblem, then the iterative points generated by this modified algorithm will not change after at most $\left\lceil \frac{4F^{\max}}{\tau(1-\gamma)} \right\rceil$ steps and the stopping point also satisfies certain optimality conditions, provided similar assumptions as those for PAMe. The full rank assumption can be removed when the projection dimension is one.

math.OC↗

arXiv2022

Practical Approximation Algorithms for $\ell_1$-Regularized Sparse Rank-$1$ Approximation to Higher-Order Tensors

Two approximation algorithms are proposed for $\ell_1$-regularized sparse rank-1 approximation to higher-order tensors. The algorithms are based on multilinear relaxation and sparsification, which are easily implemented and well scalable. In particular, the second one scales linearly with the size of the input tensor. Based on a careful estimation of the $\ell_1$-regularized sparsification, theoretical approximation lower bounds are derived. Our theoretical results also suggest an explicit way of choosing the regularization parameters. Numerical examples are provided to verify the proposed algorithms.

math.OC↗

arXiv2022

Hot-SVD: Higher-Order t-Singular Value Decomposition for Tensors based on Tensor-Tensor Product

This paper considers a way of generalizing the t-SVD of third-order tensors (regarded as tubal matrices) to tensors of arbitrary order N (which can be similarly regarded as tubal tensors of order (N-1)). \color{black}Such a generalization is different from the t-SVD for tensors of order greater than three [Martin, Shafer, Larue, SIAM J. Sci. Comput., 35 (2013), A474--A490]. The decomposition is called Hot-SVD since it can be recognized as a tensor-tensor product version of HOSVD. The existence of Hot-SVD is proved. To this end, a new transpose for third-order tensors is introduced. This transpose is crucial in the verification of Hot-SVD, since it serves as a bridge between tubal tensors and their unfoldings. We establish some properties of Hot-SVD, analogous to those of HOSVD, and in doing so we emphasize the perspective of tubal tensors. The truncated and sequentially truncated Hot-SVD are then introduced, whose error bounds are $\sqrt{N}$ for an $(N+1)$-th order tensor. We provide numerical examples to validate Hot-SVD, truncated Hot-SVD, and sequentially truncated Hot-SVD.

math.NA↗

arXiv2022

Computation over Tensor Stiefel Manifold: A Preliminary Study

Let $*$ denote the t-product between two third-order tensors. The purpose of this work is to study fundamental computation over the set $St(n,p,l):= \{\mathcal Q\in \mathbb R^{n\times p\times l} \mid \mathcal Q^{\top}* \mathcal Q = \mathcal I \}$, where $\mathcal Q$ is a third-order tensor of size $n\times p \times l$ and $\mathcal I$ ($n\geq p$) is the identity tensor. It is first verified that $St(n,p,l)$ endowed with the usual Frobenius norm forms a Riemannian manifold, which is termed as the (third-order) \emph{tensor Stiefel manifold} in this work. We then derive the tangent space, Riemannian gradient, and Riemannian Hessian on $St(n,p,l)$. In addition, formulas of various retractions based on t-QR, t-polar decomposition, Cayley transform, and t-exponential, as well as vector transports, are presented. It is expected that analogous to their matrix counterparts, the formulas derived in this study may serve as building blocks for analyzing optimization problems over the tensor Stiefel manifold and designing Riemannian algorithms for them.

math.OC↗

arXiv2021

A Study on Nonnegative Tubal Matrices

Tubal scalars are usual vectors, and tubal matrices are matrices with every element being a tubal scalar. Such a matrix is often recognized as a third-order tensor. The product between tubal scalars, tubal vectors, and tubal matrices can be done by the powerful t-product. In this paper, we define nonnegative/positive/strongly positive tubal scalars/vectors/matrices, and establish several properties that are analogous to their matrix counterparts. In particular, we introduce the irreducible tubal matrix, and provide two equivalent characterizations. Then, the celebrated Perron-Frobenius theorem is established on the nonnegative irreducible tubal matrices. We show that some conclusions of the PF theorem for nonnegative irreducible matrices can be generalized to the tubal matrix setting, while some are not. One reason is the defined positivity here has a different meaning to its usual sense. For those conclusions that can not be extended, weaker conclusions are proved. We also show that, if the nonnegative irreducible tubal matrix contains a strongly positive tubal scalar, then most conclusions of the matrix PF theorem hold.

math.SP↗

arXiv2021

On the Largest Singular Value/Eigenvalue of a Random Tensor

This short note presents upper bounds of the expectations of the largest singular values/eigenvalues of various types of random tensors in the non-asymptotic sense. For a standard Gaussian tensor of size $n_1\times\cdots\times n_d$, it is shown that the expectation of its largest singular value is upper bounded by $\sqrt {n_1}+\cdots+\sqrt {n_d}$. For the expectation of the largest $\ell^d$-singular value, it is upper bounded by $2^{\frac{d-1}{2}}\prod_{j=1}^{d}n_j^{\frac{d-2}{2d}}\sum^d_{j=1}n_j^{\frac{1}{2}}$. We also derive the upper bounds of the expectations of the largest Z-/H-($\ell^d$)/M-/C-eigenvalues of symmetric, partially symmetric, and piezoelectric-type Gaussian tensors, which are respectively upper bounded by $d\sqrt n$, $d\cdot 2^{\frac{d-1}{2}}n^{\frac{d-1}{2}}$, $2\sqrt m+2\sqrt n$, and $3\sqrt n$.

math.SP↗

arXiv2020

On Approximation Algorithm for Orthogonal Low-Rank Tensor Approximation

The goal of this work is to fill a gap in [Yang, SIAM J. Matrix Anal. Appl, 41 (2020), 1797--1825]. In that work, an approximation procedure was proposed for orthogonal low-rank tensor approximation; however, the approximation lower bound was only established when the number of orthonormal factors is one. To this end, by further exploring the multilinearity and orthogonality of the problem, we introduce a modified approximation algorithm. Approximation lower bound is established, either in deterministic or expected sense, no matter how many orthonormal factors there are. In addition, a major feature of the new algorithm is its flexibility to allow either deterministic or randomized procedures to solve a key step of each latent orthonormal factor involved in the algorithm. This feature can reduce the computation of large SVDs, making the algorithm more efficient. Some numerical studies are provided to validate the usefulness of the proposed algorithm.

math.OC↗

arXiv2020

Several Approximation Algorithms for Sparse Best Rank-1 Approximation to Higher-Order Tensors

Sparse tensor best rank-1 approximation (BR1Approx), which is a sparsity generalization of the dense tensor BR1Approx, and is a higher-order extension of the sparse matrix BR1Approx, is one of the most important problems in sparse tensor decomposition and related problems arising from statistics and machine learning. By exploiting the multilinearity as well as the sparsity structure of the problem, four approximation algorithms are proposed, which are easily implemented, of low computational complexity, and can serve as initial procedures for iterative algorithms. In addition, theoretically guaranteed worst-case approximation lower bounds are proved for all the algorithms. We provide numerical experiments on synthetic and real data to illustrate the effectiveness of the proposed algorithms.

math.NA↗

arXiv2020

Finding the Global Optimum of a Class of Quartic Minimization Problem

We consider a special nonconvex quartic minimization problem over a single spherical constraint, which includes the discretized energy functional minimization problem of non-rotating Bose-Einstein condensates (BECs) as one of the important applications. Such a problem is studied by exploiting its characterization as a nonlinear eigenvalue problem with eigenvector nonlinearity (NEPv), which admits a unique nonnegative eigenvector, and this eigenvector is exactly the global minimizer to the quartic minimization. With these properties, any algorithm converging to the nonnegative stationary point of this optimization problem finds its global minimum, such as the regularized Newton (RN) method. In particular, we obtain the global convergence to global optimum of the inexact alternating direction method of multipliers (ADMM) for this problem. Numerical experiments for applications in non-rotating BEC validate our theories.

math.NA↗

arXiv2020

Half-Quadratic Alternating Direction Method of Multipliers for Robust Orthogonal Tensor Approximation

Higher-order tensor canonical polyadic decomposition (CPD) with one or more of the latent factor matrices being columnwisely orthonormal has been well studied in recent years. However, most existing models penalize the noises, if occurring, by employing the least squares loss, which may be sensitive to non-Gaussian noise or outliers, leading to bias estimates of the latent factors. In this paper, based on the maximum a posterior estimation, we derive a robust orthogonal tensor CPD model with Cauchy loss, which is resistant to heavy-tailed noise or outliers. By exploring the half-quadratic property of the model, a new method, which is termed as half-quadratic alternating direction method of multipliers (HQ-ADMM), is proposed to solve the model. Each subproblem involved in HQ-ADMM admits a closed-form solution. Thanks to some nice properties of the Cauchy loss, we show that the whole sequence generated by the algorithm globally converges to a stationary point of the problem under consideration. Numerical experiments on synthetic and real data demonstrate the efficiency and robustness of the proposed model and algorithm.

math.OC↗