arXiv subjects

Tianqi Zhong

Publications and source records attributed to Tianqi Zhong.

4 recordsLinked to original sources

arXiv2026

Environment-Conditioned Tail Reweighting for Invariant Learning under Mixed Shifts

Out-of-distribution generalization becomes challenging when spurious correlations vary across environments while difficult or underrepresented samples remain insufficiently emphasized within them. Invariant learning and reweighting address complementary aspects of this mixed-shift problem, yet applying reweighting only to prediction leaves the invariance constraint evaluated on a different training risk. We introduce Environment-Conditioned Tail Reweighting (ECTR), which couples cross-environment TV invariance with within-environment adversarial tail weighting through a shared weighted risk. Sample weights are normalized within each environment, and the resulting risk is used consistently for both predictive learning and the total-variation (TV) stationarity penalty, while an environment-wise KL term controls adversarial concentration. Our analysis characterizes the stationarity-dependent weighting signal and its KL-regularized distributional interpretation, alongside conditional-risk and optimization properties under the stated assumptions. Controlled mixed-shift ablations consistently support the shared-risk coupling across three difficulty settings. Across synthetic and real-world benchmarks, ECTR achieves favorable results under both observed- and inferred-environment settings, including improvements over TV-based parent methods and strong performance against additional invariant-learning baselines. Training-corruption experiments further show that ECTR assigns less excess weight to corrupted samples than the tested fixed-tail rules. Together, these results provide a unified theoretical and empirical account of environment-conditioned tail reweighting for invariant learning under mixed shifts.

cs.LG↗

arXiv2025

Out-of-distribution Generalization for Total Variation based Invariant Risk Minimization

Invariant risk minimization is an important general machine learning framework that has recently been interpreted as a total variation model (IRM-TV). However, how to improve out-of-distribution (OOD) generalization in the IRM-TV setting remains unsolved. In this paper, we extend IRM-TV to a Lagrangian multiplier model named OOD-TV-IRM. We find that the autonomous TV penalty hyperparameter is exactly the Lagrangian multiplier. Thus OOD-TV-IRM is essentially a primal-dual optimization model, where the primal optimization minimizes the entire invariant risk and the dual optimization strengthens the TV penalty. The objective is to reach a semi-Nash equilibrium where the balance between the training loss and OOD generalization is maintained. We also develop a convergent primal-dual algorithm that facilitates an adversarial learning scheme. Experimental results show that OOD-TV-IRM outperforms IRM-TV in most situations.

cs.LG↗

arXiv2024

Benchmarking and Improving Compositional Generalization of Multi-aspect Controllable Text Generation

Compositional generalization, representing the model's ability to generate text with new attribute combinations obtained by recombining single attributes from the training data, is a crucial property for multi-aspect controllable text generation (MCTG) methods. Nonetheless, a comprehensive compositional generalization evaluation benchmark of MCTG is still lacking. We propose CompMCTG, a benchmark encompassing diverse multi-aspect labeled datasets and a crafted three-dimensional evaluation protocol, to holistically evaluate the compositional generalization of MCTG approaches. We observe that existing MCTG works generally confront a noticeable performance drop in compositional testing. To mitigate this issue, we introduce Meta-MCTG, a training framework incorporating meta-learning, where we enable models to learn how to generalize by simulating compositional generalization scenarios in the training phase. We demonstrate the effectiveness of Meta-MCTG through achieving obvious improvement (by at most 3.64%) for compositional testing performance in 94.4% cases.

cs.CL↗

arXiv2023

Air-Decoding: Attribute Distribution Reconstruction for Decoding-Time Controllable Text Generation

Controllable text generation (CTG) aims to generate text with desired attributes, and decoding-time-based methods have shown promising performance on this task. However, in this paper, we identify the phenomenon of Attribute Collapse for the first time. It causes the fluency of generated text to rapidly decrease when the control strength exceeds a critical value, rendering the text completely unusable. This limitation hinders the effectiveness of decoding methods in achieving high levels of controllability. To address this problem, we propose a novel lightweight decoding framework named Air-Decoding. Its main idea is reconstructing the attribute distributions to balance the weights between attribute words and non-attribute words to generate more fluent text. Specifically, we train prefixes by prefix-tuning to obtain attribute distributions. Then we design a novel attribute distribution reconstruction method to balance the obtained distributions and use the reconstructed distributions to guide language models for generation, effectively avoiding the issue of Attribute Collapse. Experiments on multiple CTG tasks prove that our method achieves a new state-of-the-art control performance.

cs.CL↗