arXiv subjects

Xi Chen

Publications and source records attributed to Xi Chen.

At least 667 records · Page 37Linked to original sources

arXiv2023

PreSTU: Pre-Training for Scene-Text Understanding

The ability to recognize and reason about text embedded in visual inputs is often lacking in vision-and-language (V&L) models, perhaps because V&L pre-training methods have often failed to include such an ability in their training objective. In this paper, we propose PreSTU, a novel pre-training recipe dedicated to scene-text understanding (STU). PreSTU introduces OCR-aware pre-training objectives that encourage the model to recognize text from an image and connect it to the rest of the image content. We implement PreSTU using a simple transformer-based encoder-decoder architecture, combined with large-scale image-text datasets with scene text obtained from an off-the-shelf OCR system. We empirically demonstrate the effectiveness of this pre-training approach on eight visual question answering and four image captioning benchmarks.

cs.CV↗

arXiv2023

Scattering of relativistic electrons and analogies with optical phenomena: A study of longitudinal and transverse shifts at step potentials

We investigate the behavior of relativistic electrons encountering a potential step through analogies with optical phenomena. By accounting for the conservation of Dirac current, we elucidate that the Goos-Hänchen shift can be understood as a combination of two components: one arising from the current entering the transmission region and the other originating from the interference between the incident and reflected beams. This result has been proven to be consistent with findings obtained utilizing the stationary phase method. Moreover, we explore the transverse Imbert-Fedorov shift, by applying both current conservation and total angular momentum conservation, revealing intriguing parallel to the spin Hall effect. Beyond enriching our comprehension of fundamental quantum phenomena, our findings have potential applications for designing and characterizing devices using Dirac and topological materials.

quant-ph↗

arXiv2023

Teacher-Student Architecture for Knowledge Distillation: A Survey

Although Deep neural networks (DNNs) have shown a strong capacity to solve large-scale problems in many areas, such DNNs are hard to be deployed in real-world systems due to their voluminous parameters. To tackle this issue, Teacher-Student architectures were proposed, where simple student networks with a few parameters can achieve comparable performance to deep teacher networks with many parameters. Recently, Teacher-Student architectures have been effectively and widely embraced on various knowledge distillation (KD) objectives, including knowledge compression, knowledge expansion, knowledge adaptation, and knowledge enhancement. With the help of Teacher-Student architectures, current studies are able to achieve multiple distillation objectives through lightweight and generalized student networks. Different from existing KD surveys that primarily focus on knowledge compression, this survey first explores Teacher-Student architectures across multiple distillation objectives. This survey presents an introduction to various knowledge representations and their corresponding optimization objectives. Additionally, we provide a systematic overview of Teacher-Student architectures with representative learning algorithms and effective distillation schemes. This survey also summarizes recent applications of Teacher-Student architectures across multiple purposes, including classification, recognition, generation, ranking, and regression. Lastly, potential research directions in KD are investigated, focusing on architecture design, knowledge quality, and theoretical studies of regression-based learning, respectively. Through this comprehensive survey, industry practitioners and the academic community can gain valuable insights and guidelines for effectively designing, learning, and applying Teacher-Student architectures on various distillation objectives.

cs.LG↗

arXiv2023

In Search of Infall Motion in Molecular Clumps. IV. Mapping of the Global Infall Sources

We have used the IRAM 30-m telescope to map some targets with HCO$^+$ (1-0) and H$^{13}$CO$^+$ (1-0) lines in order to search for gas infall evidence in the clumps. In this paper, we report the mapping results for 13 targets. All of these targets show HCO$^+$ emissions, while H$^{13}$CO$^+$ emissions are observed in ten of them. The HCO$^+$ integrated intensity maps of ten targets show clear clumpy structures, and nine targets show clumpy structures in the H$^{13}$CO$^+$ maps. Using the RADEX radiative transfer code, we estimate the column density of H$^{13}$CO$^+$, and determine the abundance ratio [H$^{13}$CO$^+$]/[H$_2$] to be approximately 10$^{-12}$ to 10$^{-10}$. Based on the asymmetry of the HCO$^+$ line profiles, we identify 11 targets show blue profiles, while six clumps have global infall evidence. We use the RATRAN and two-layer models to fit the HCO$^+$ line profiles of these infall sources, and analyze their spatial distribution of the infall velocity. The average infall velocities estimated by these two models are 0.24 -- 1.85 km s$^{-1}$ and 0.28 -- 1.45 km s$^{-1}$, respectively. The mass infall rate ranges from approximately 10$^{-5}$ to 10$^{-2}$ M$_{\odot}$ yr$^{-1}$, which suggests that intermediate- or high-mass stars may be forming in the target regions.

astro-ph.GA↗

arXiv2023

Flexible Coherent Optical Access: Architectures, Algorithms, and Demonstrations

To cope with the explosive bandwidth demand, significant progress has been made in the ITU-T standardization sector to define a higher-speed passive optical network (PON) with a 50Gb/s line rate. Recently, 50G PON becomes mature gradually, which means it is time to discuss beyond 50G PON. For ensuring an acceptable optical power budget, beyond 50G PON will potentially use coherent technologies, which can simultaneously promote the applications of flexible multiple access such as time/frequency-domain multiple access (TFDMA). In this paper, we will introduce the architectures, algorithms, and demonstrations for TFDMA-based coherent PON. The system architectures based on an ultra-simple coherent transceiver and specific signal spectra are designed to greatly reduce the cost of ONUs. Meanwhile, fast and low-complexity digital signal processing (DSP) algorithms are proposed for dealing with upstream and downstream signals. Based on the architectures and algorithms, we experimentally demonstrate the first real-time TFDMA-based coherent PON, which can support at most 256 end users, and peak line rates of 100Gb/s and 200Gb/s in the upstream and downstream scenarios, respectively. In conclusion, the proposed technologies for the coherent PON make it more possible to be applied in the future beyond 50G PON.

cs.NI↗

arXiv2023

The Infrared Extinction Law in the Ophiuchus Molecular Cloud based on UKIDSS and Spitzer

Investigating the extinction properties in dense molecular clouds is of significant importance for understanding the behavior of interstellar dust and its impact on observations. In this study, we comprehensively examined the extinction law in the Ophiuchus cloud across a wavelength range from 0.8$\,μ\rm m$ to 8$\,μ\rm m$. To achieve this, we analyzed NIR and MIR data obtained from the UKIDSS GCS and the Spitzer c2d survey, respectively. By fitting a series of color-color diagrams, we determined color-excess ratios $E_{J-λ}/E_{J-K}$ for seven passbands. These ratios were then directly converted to derive the relative extinction law $A_λ/A_K$. Our findings demonstrate that the Ophiuchus cloud exhibits a characteristic of flat MIR extinction, consistent with previous studies. Additionally, our results reveal variations in the extinction law with extinction depth, indicating a flatter trend from the NIR to MIR bands as extinction increases. Notably, our analysis reveals no significant difference in the MIR extinction law among the four dark clouds: L1712, L1689, L1709, and L1688. However, distinct variations were observed in the extinction law for regions outside the dark clouds, specifically L1688N and L1688W. These regions displayed lower color-excess ratios $E_{J-λ}/E_{J-K}$ in the Spitzer/IRAC bands. This observation lends support to the dust growth occurring in the dense regions of the Ophiuchus cloud.

astro-ph.GA↗

arXiv2023

Convolutional Occupancy Models for Dense Packing of Complex, Novel Objects

Dense packing in pick-and-place systems is an important feature in many warehouse and logistics applications. Prior work in this space has largely focused on planning algorithms in simulation, but real-world packing performance is often bottlenecked by the difficulty of perceiving 3D object geometry in highly occluded, partially observed scenes. In this work, we present a fully-convolutional shape completion model, F-CON, which can be easily combined with off-the-shelf planning methods for dense packing in the real world. We also release a simulated dataset, COB-3D-v2, that can be used to train shape completion models for real-word robotics applications, and use it to demonstrate that F-CON outperforms other state-of-the-art shape completion methods. Finally, we equip a real-world pick-and-place system with F-CON, and demonstrate dense packing of complex, unseen objects in cluttered scenes. Across multiple planning methods, F-CON enables substantially better dense packing than other shape completion methods.

cs.RO↗

arXiv2023

Fairness-aware Online Price Discrimination with Nonparametric Demand Models

Price discrimination, which refers to the strategy of setting different prices for different customer groups, has been widely used in online retailing. Although it helps boost the collected revenue for online retailers, it might create serious concerns about fairness, which even violates the regulation and laws. This paper studies the problem of dynamic discriminatory pricing under fairness constraints. In particular, we consider a finite selling horizon of length $T$ for a single product with two groups of customers. Each group of customers has its unknown demand function that needs to be learned. For each selling period, the seller determines the price for each group and observes their purchase behavior. While existing literature mainly focuses on maximizing revenue, ensuring fairness among different customers has not been fully explored in the dynamic pricing literature. This work adopts the fairness notion from Cohen et al. (2022). For price fairness, we propose an optimal dynamic pricing policy regarding regret, which enforces the strict price fairness constraint. In contrast to the standard $\sqrt{T}$-type regret in online learning, we show that the optimal regret in our case is $\tilde{O}(T^{4/5})$. We further extend our algorithm to a more general notion of fairness, which includes demand fairness as a special case. To handle this general class, we propose a soft fairness constraint and develop a dynamic pricing policy that achieves $\tilde{O}(T^{4/5})$ regret. We also demonstrate that our algorithmic techniques can be adapted to more general scenarios such as fairness among multiple groups of customers.

cs.LG↗

arXiv2023

RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control

We study how vision-language models trained on Internet-scale data can be incorporated directly into end-to-end robotic control to boost generalization and enable emergent semantic reasoning. Our goal is to enable a single end-to-end trained model to both learn to map robot observations to actions and enjoy the benefits of large-scale pretraining on language and vision-language data from the web. To this end, we propose to co-fine-tune state-of-the-art vision-language models on both robotic trajectory data and Internet-scale vision-language tasks, such as visual question answering. In contrast to other approaches, we propose a simple, general recipe to achieve this goal: in order to fit both natural language responses and robotic actions into the same format, we express the actions as text tokens and incorporate them directly into the training set of the model in the same way as natural language tokens. We refer to such category of models as vision-language-action models (VLA) and instantiate an example of such a model, which we call RT-2. Our extensive evaluation (6k evaluation trials) shows that our approach leads to performant robotic policies and enables RT-2 to obtain a range of emergent capabilities from Internet-scale training. This includes significantly improved generalization to novel objects, the ability to interpret commands not present in the robot training data (such as placing an object onto a particular number or icon), and the ability to perform rudimentary reasoning in response to user commands (such as picking up the smallest or largest object, or the one closest to another object). We further show that incorporating chain of thought reasoning allows RT-2 to perform multi-stage semantic reasoning, for example figuring out which object to pick up for use as an improvised hammer (a rock), or which type of drink is best suited for someone who is tired (an energy drink).

cs.RO↗

arXiv2023

2D-Shapley: A Framework for Fragmented Data Valuation

Data valuation -- quantifying the contribution of individual data sources to certain predictive behaviors of a model -- is of great importance to enhancing the transparency of machine learning and designing incentive systems for data sharing. Existing work has focused on evaluating data sources with the shared feature or sample space. How to valuate fragmented data sources of which each only contains partial features and samples remains an open question. We start by presenting a method to calculate the counterfactual of removing a fragment from the aggregated data matrix. Based on the counterfactual calculation, we further propose 2D-Shapley, a theoretical framework for fragmented data valuation that uniquely satisfies some appealing axioms in the fragmented data context. 2D-Shapley empowers a range of new use cases, such as selecting useful data fragments, providing interpretation for sample-wise data values, and fine-grained data issue diagnosis.

cs.LG↗

arXiv2023

Photonic counterdiabatic quantum optimization algorithm

We propose a hybrid quantum-classical approximate optimization algorithm for photonic quantum computing, specifically tailored for addressing continuous-variable optimization problems. Inspired by counterdiabatic protocols, our algorithm significantly reduces the required quantum operations for optimization as compared to adiabatic protocols. This reduction enables us to tackle non-convex continuous optimization and countably infinite integer programming within the near-term era of quantum computing. Through comprehensive benchmarking, we demonstrate that our approach outperforms existing state-of-the-art hybrid adiabatic quantum algorithms in terms of convergence and implementability. Remarkably, our algorithm offers a practical and accessible experimental realization, bypassing the need for high-order operations and overcoming experimental constraints. We conduct proof-of-principle experiments on an eight-mode nanophotonic quantum chip, successfully showcasing the feasibility and potential impact of the algorithm.

quant-ph↗

arXiv2023

Uniformity Testing over Hypergrids with Subcube Conditioning

We give an algorithm for testing uniformity of distributions supported on hypergrids $[m_1] \times \cdots \times [m_n]$, which makes $\smash{\widetilde{O}(\text{poly}(m)\sqrt{n}/ε^2)}$ many queries to a subcube conditional sampling oracle with $m=\max_i m_i$. When $m$ is a constant, our algorithm is nearly optimal and strengthens the algorithm of [CCK+21] which has the same query complexity but works for hypercubes $\{\pm 1\}^n$ only. A key technical contribution behind the analysis of our algorithm is a proof of a robust version of Pisier's inequality for functions over hypergrids using Fourier analysis.

cs.DS↗

arXiv2023

Grain size in low loss superconducting Ta thin films on c-axis sapphire

In recent years, the implementation of thin-film Ta has led to improved coherence times in superconducting circuits. Efforts to further optimize this materials set have become a focus of the subfield of materials for superconducting quantum computing. It has been previously hypothesized that grain size could be correlated with device performance. In this work, we perform a comparative grain size experiment with $α$-Ta on $c$-axis sapphire. Our evaluation methods include both room-temperature chemical and structural characterization and cryogenic microwave measurements, and we report no statistical difference in device performance between small- and larger-grain-size devices with grain sizes of 924 nm$^2$ and 1700 nm$^2$, respectively. These findings suggest that grain size is not correlated with loss in the parameter regime of interest for Ta grown on c-axis sapphire, narrowing the parameter space for optimization of this materials set.

quant-ph↗

arXiv2023

Global well-posedness of quadratic and subquadratic half wave Schr{ö}dinger equations

We consider the following $p$ order nonlinear half wave Schr{ö}dinger equations$$\left(i \partial\_{t}+\partial\_{x }^2-\left|D\_{y}\right|\right) u=\pm|u|^{p-1} u$$on the plane $\mathbb{R}^2$ with $1<p\leq 2$. This equation is considered as a toy model motivated by the study of solutions to weakly dispersive equations. In particular, the global well-posedness of this equation is a difficult problem due to the anisotropic property of the equation, with one direction corresponding to the half-wave operator, which is not dispersive. In this paper, we prove the global well-posedness of this equation in $L\_x^2 H\_y^s(\mathbb{R}^2) \cap H\_x^1 L\_y^2(\mathbb{R}^2)$($\frac{1}{2}\leq s \leq 1$), which is the first global well-posedness result of nonlinear half wave Schr{ö}dinger equations. With the global well-posedness in the energy space for the focusing equation and the study on the solitary wave in [1], we complete the proof of the stability of the set of ground states. Moreover, we consider the half wave Schr{ö}dinger equations on $\mathbb{R}\_{x}\times\mathbb{T}\_{y}$, which can also be called the wave guide Schr{ö}dinger equations on $\mathbb{R}\_{x}\times\mathbb{T}\_{y}$. Using a similar approach in the analysis of the Cauchy problem of half wave Schr{ö}dinger equations on $\mathbb{R}^2$, we can also deduce the global well-posedness of $p$ ($1<p\leq2$) order wave guide Schr{ö}dinger equations in $L\_x^2 H\_y^s(\mathbb{R}\times\mathbb{T}) \cap H\_x^1 L\_y^2(\mathbb{R}\times\mathbb{T})$ with $\frac{1}{2}\leq s \leq 1$. With the global well-posedness in the energy space for the focusing wave guide Schr{ö}dinger equations and the study on the ground states in [2], we complete the proof of the orbital stability of the ground states with small frequencies.

math.AP↗

arXiv2023

Open-vocabulary Panoptic Segmentation with Embedding Modulation

Open-vocabulary image segmentation is attracting increasing attention due to its critical applications in the real world. Traditional closed-vocabulary segmentation methods are not able to characterize novel objects, whereas several recent open-vocabulary attempts obtain unsatisfactory results, i.e., notable performance reduction on the closed vocabulary and massive demand for extra data. To this end, we propose OPSNet, an omnipotent and data-efficient framework for Open-vocabulary Panoptic Segmentation. Specifically, the exquisitely designed Embedding Modulation module, together with several meticulous components, enables adequate embedding enhancement and information exchange between the segmentation model and the visual-linguistic well-aligned CLIP encoder, resulting in superior segmentation performance under both open- and closed-vocabulary settings with much fewer need of additional data. Extensive experimental evaluations are conducted across multiple datasets (e.g., COCO, ADE20K, Cityscapes, and PascalContext) under various circumstances, where the proposed OPSNet achieves state-of-the-art results, which demonstrates the effectiveness and generality of the proposed approach. The code and trained models will be made publicly available.

cs.CV↗

arXiv2023

Pruning Ternary Quantization

Inference time, model size, and accuracy are three key factors in deep model compression. Most of the existing work addresses these three key factors separately as it is difficult to optimize them all at the same time. For example, low-bit quantization aims at obtaining a faster model; weight sharing quantization aims at improving compression ratio and accuracy; and mixed-precision quantization aims at balancing accuracy and inference time. To simultaneously optimize bit-width, model size, and accuracy, we propose pruning ternary quantization (PTQ): a simple, effective, symmetric ternary quantization method. We integrate L2 normalization, pruning, and the weight decay term to reduce the weight discrepancy in the gradient estimator during quantization, thus producing highly compressed ternary weights. Our method brings the highest test accuracy and the highest compression ratio. For example, it produces a 939kb (49$\times$) 2bit ternary ResNet-18 model with only 4\% accuracy drop on the ImageNet dataset. It compresses 170MB Mask R-CNN to 5MB (34$\times$) with only 2.8\% average precision drop. Our method is verified on image classification, object detection/segmentation tasks with different network structures such as ResNet-18, ResNet-50, and MobileNetV2.

cs.CV↗

arXiv2023

Avalanche Instability as Nonequilibrium Quantum Criticality

A fundamental instability in the nonequilibrium conduction band under a electric field bias is proposed via the spontaneous emission of coherent phonons. Analytic theory, supported by numerical calculations, establishes that the quantum avalanche, an abrupt nonequilibrium occupation of excited bands, results from the competition between the collapse of the band minimum via the phonon emission and the dephasing of the electron with the environment. The continuous avalanche transition is a quantum phase transition with the nonequilibrium phase diagram determined by the avalanche parameter $β$, with peculiar reentrant avalanche domes close to the phase boundary. We further confirm the nature of the quantum avalanche with the temperature dependence.

cond-mat.str-el↗

arXiv2023

Gradient-based Bi-level Optimization for Deep Learning: A Survey

Bi-level optimization, especially the gradient-based category, has been widely used in the deep learning community including hyperparameter optimization and meta-knowledge extraction. Bi-level optimization embeds one problem within another and the gradient-based category solves the outer-level task by computing the hypergradient, which is much more efficient than classical methods such as the evolutionary algorithm. In this survey, we first give a formal definition of the gradient-based bi-level optimization. Next, we delineate criteria to determine if a research problem is apt for bi-level optimization and provide a practical guide on structuring such problems into a bi-level optimization framework, a feature particularly beneficial for those new to this domain. More specifically, there are two formulations: the single-task formulation to optimize hyperparameters such as regularization parameters and the distilled data, and the multi-task formulation to extract meta-knowledge such as the model initialization. With a bi-level formulation, we then discuss four bi-level optimization solvers to update the outer variable including explicit gradient update, proxy update, implicit function update, and closed-form update. Finally, we wrap up the survey by highlighting two prospective future directions: (1) Effective Data Optimization for Science examined through the lens of task formulation. (2) Accurate Explicit Proxy Update analyzed from an optimization standpoint.

cs.LG↗