arXiv subjects

Zhan Shi

Publications and source records attributed to Zhan Shi.

At least 73 records · Page 4Linked to original sources

arXiv2023

Aperture Diffraction for Compact Snapshot Spectral Imaging

We demonstrate a compact, cost-effective snapshot spectral imaging system named Aperture Diffraction Imaging Spectrometer (ADIS), which consists only of an imaging lens with an ultra-thin orthogonal aperture mask and a mosaic filter sensor, requiring no additional physical footprint compared to common RGB cameras. Then we introduce a new optical design that each point in the object space is multiplexed to discrete encoding locations on the mosaic filter sensor by diffraction-based spatial-spectral projection engineering generated from the orthogonal mask. The orthogonal projection is uniformly accepted to obtain a weakly calibration-dependent data form to enhance modulation robustness. Meanwhile, the Cascade Shift-Shuffle Spectral Transformer (CSST) with strong perception of the diffraction degeneration is designed to solve a sparsity-constrained inverse problem, realizing the volume reconstruction from 2D measurements with Large amount of aliasing. Our system is evaluated by elaborating the imaging optical theory and reconstruction algorithm with demonstrating the experimental imaging under a single exposure. Ultimately, we achieve the sub-super-pixel spatial resolution and high spectral resolution imaging. The code will be available at: https://github.com/Krito-ex/CSST.

cs.CV↗

arXiv2023

Regression-Oriented Knowledge Distillation for Lightweight Ship Orientation Angle Prediction with Optical Remote Sensing Images

Ship orientation angle prediction (SOAP) with optical remote sensing images is an important image processing task, which often relies on deep convolutional neural networks (CNNs) to make accurate predictions. This paper proposes a novel framework to reduce the model sizes and computational costs of SOAP models without harming prediction accuracy. First, a new SOAP model called Mobile-SOAP is designed based on MobileNetV2, achieving state-of-the-art prediction accuracy. Four tiny SOAP models are also created by replacing the convolutional blocks in Mobile-SOAP with four small-scale networks, respectively. Then, to transfer knowledge from Mobile-SOAP to four lightweight models, we propose a novel knowledge distillation (KD) framework termed SOAP-KD consisting of a novel feature-based guidance loss and an optimized synthetic samples-based knowledge transfer mechanism. Lastly, extensive experiments on the FGSC-23 dataset confirm the superiority of Mobile-SOAP over existing models and also demonstrate the effectiveness of SOAP-KD in improving the prediction performance of four specially designed tiny models. Notably, by using SOAP-KD, the test mean absolute error of the ShuffleNetV2x1.0-based model is only 8% higher than that of Mobile-SOAP, but its number of parameters and multiply-accumulate operations (MACs) are respectively 61.6% and 60.8% less.

cs.CV↗

arXiv2023

SonoTransformers: Ultrafast Transformable Wireless Microscale Machines

Shape transformation, a key mechanism for organismal survival and adaptation, has gained importance across fields as diverse as electronics and medicine. However, designing and controlling microscale shape-shifting materials remains a fundamental challenge in various actuation modalities. Here, we introduce SonoTransformer, an acoustically activated micromachine that delivers ultrafast transformability using preprogrammed soft hinges. These hinges concentrate energy through intensified oscillation and provide the necessary torque for transformation. We have created new machine designs to predetermine the folding state, enabling tailoring and milliseconds transformation. Additionally, we have shown selective transformation by adjusting acoustic power, realizing high degrees of control and functional versatility. Our findings open new research avenues in acoustics, physics, and soft matter, offering new design paradigms and development opportunities in robotics, metamaterials, adaptive optics, and microtechnology.

physics.app-ph↗

arXiv2023

The stochastic Jacobi flow

The problem of conditioning on the occupation field was investigated for the Brownian motion in 1998 independently by Aldous [4] and Warren and Yor [34] and recently for the loop soup at intensity $1/2$ by Werner [35], Sabot and Tarrès [30], and Lupu, Sabot and Tarrès [22]. We consider this problem in the case of the Brownian loop soup on the real line, and show that it is connected with a flow version of Jacobi processes, called Jacobi flow. We give a pathwise construction of this flow simultaneously for all parameters by means of a common Brownian motion, via the perturbed reflecting Brownian motion. The Jacobi flow is related to Fleming--Viot processes, as established by Bertoin and Le Gall [9] and Dawson and Li [11]. This relation allows us to interpret Perkins' disintegration theorem between Feller continuous state branching-processes and Fleming--Viot processes as a decomposition of Gaussian measures. Our approach gives a unified framework for the problems of disintegrating on the real line. The connection with Bass--Burdzy flows which was drawn in Warren [33] and Lupu, Sabot and Tarrès [23] is shown to be valid in the general case.

math.PR↗

arXiv2023

The dual Derrida-Retaux conjecture

We consider a recursive system $(X_n)$ which was introduced by Collet et al. [10] as a spin glass model, and later by Derrida, Hakim, and Vannimenus [13] and by Derrida and Retaux [14] as a simplified hierarchical renormalization model. The system $(X_n)$ is expected to possess highly nontrivial universalities at or near criticality. In the nearly supercritical regime, Derrida and Retaux [14] conjectured that the free energy of the system decays exponentially with exponent $(p-p_c)^{-\frac12}$ as $p \downarrow p_c$. We study the nearly subcritical regime ($p \uparrow p_c$) and aim at a dual version of the Derrida-Retaux conjecture; our main result states that as $n \to \infty$, both $\E(X_n)$ and $¶(X_n\neq 0)$ decay exponentially with exponent $(p_c-p)^{\frac12 +o(1)}$, where $o(1) \to 0$ as $p \uparrow p_c$.

math-ph↗

arXiv2022

Speaker recognition with two-step multi-modal deep cleansing

Neural network-based speaker recognition has achieved significant improvement in recent years. A robust speaker representation learns meaningful knowledge from both hard and easy samples in the training set to achieve good performance. However, noisy samples (i.e., with wrong labels) in the training set induce confusion and cause the network to learn the incorrect representation. In this paper, we propose a two-step audio-visual deep cleansing framework to eliminate the effect of noisy labels in speaker representation learning. This framework contains a coarse-grained cleansing step to search for the peculiar samples, followed by a fine-grained cleansing step to filter out the noisy labels. Our study starts from an efficient audio-visual speaker recognition system, which achieves a close to perfect equal-error-rate (EER) of 0.01\%, 0.07\% and 0.13\% on the Vox-O, E and H test sets. With the proposed multi-modal cleansing mechanism, four different speaker recognition networks achieve an average improvement of 5.9\%. Code has been made available at: \textcolor{magenta}{\url{https://github.com/TaoRuijie/AVCleanse}}.

eess.AS↗

arXiv2022

Visual Semantic Parsing: From Images to Abstract Meaning Representation

The success of scene graphs for visual scene understanding has brought attention to the benefits of abstracting a visual input (e.g., image) into a structured representation, where entities (people and objects) are nodes connected by edges specifying their relations. Building these representations, however, requires expensive manual annotation in the form of images paired with their scene graphs or frames. These formalisms remain limited in the nature of entities and relations they can capture. In this paper, we propose to leverage a widely-used meaning representation in the field of natural language processing, the Abstract Meaning Representation (AMR), to address these shortcomings. Compared to scene graphs, which largely emphasize spatial relationships, our visual AMR graphs are more linguistically informed, with a focus on higher-level semantic concepts extrapolated from visual input. Moreover, they allow us to generate meta-AMR graphs to unify information contained in multiple image descriptions under one representation. Through extensive experimentation and analysis, we demonstrate that we can re-purpose an existing text-to-AMR parser to parse images into AMRs. Our findings point to important future research directions for improved scene understanding.

cs.CV↗

arXiv2022

Graph-based Active Learning for Semi-supervised Classification of SAR Data

We present a novel method for classification of Synthetic Aperture Radar (SAR) data by combining ideas from graph-based learning and neural network methods within an active learning framework. Graph-based methods in machine learning are based on a similarity graph constructed from the data. When the data consists of raw images composed of scenes, extraneous information can make the classification task more difficult. In recent years, neural network methods have been shown to provide a promising framework for extracting patterns from SAR images. These methods, however, require ample training data to avoid overfitting. At the same time, such training data are often unavailable for applications of interest, such as automatic target recognition (ATR) and SAR data. We use a Convolutional Neural Network Variational Autoencoder (CNNVAE) to embed SAR data into a feature space, and then construct a similarity graph from the embedded data and apply graph-based semi-supervised learning techniques. The CNNVAE feature embedding and graph construction requires no labeled data, which reduces overfitting and improves the generalization performance of graph learning at low label rates. Furthermore, the method easily incorporates a human-in-the-loop for active learning in the data-labeling process. We present promising results and compare them to other standard machine learning methods on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset for ATR with small amounts of labeled data.

cs.LG↗

arXiv2022

Driver Side and Traffic Based Evaluation Model for On-Street Parking Solutions

Parking has been a painful problem for urban drivers. The parking pain exacerbates as more people tend to live in cities in the context of global urbanization. Thus, it is demanding to find a solution to mitigate d rivers' parking headaches. Many solutions tried to resolve the parking issue by predicting parking occupancy. Their focuses were on the accuracy of the theoretical side but lacked a standardized model to evaluate these proposals in practice. This paper develops a Driver Side and Traffic Based Evaluation Model (DSTBM), which provides a general evaluation scheme for different parking solutions. Two common parking detection methods, fixed sensing and mobile sensing are analyzed using DSTBM. The results indicate first, DSTBM examines different solutions from the driver's perspective and has no conflicts with other evaluation schemes; second, DSTBM confirms that fixed sensing performs better than mobile sensing in terms of prediction accuracy.

cs.CY↗

arXiv2021

Unsupervised Pre-training with Structured Knowledge for Improving Natural Language Inference

While recent research on natural language inference has considerably benefited from large annotated datasets, the amount of inference-related knowledge (including commonsense) provided in the annotated data is still rather limited. There have been two lines of approaches that can be used to further address the limitation: (1) unsupervised pretraining can leverage knowledge in much larger unstructured text data; (2) structured (often human-curated) knowledge has started to be considered in neural-network-based models for NLI. An immediate question is whether these two approaches complement each other, or how to develop models that can bring together their advantages. In this paper, we propose models that leverage structured knowledge in different components of pre-trained models. Our results show that the proposed models perform better than previous BERT-based state-of-the-art models. Although our models are proposed for NLI, they can be easily extended to other sentence or sentence-pair classification problems.

cs.CL↗

arXiv2021

Unsupervised Conversation Disentanglement through Co-Training

Conversation disentanglement aims to separate intermingled messages into detached sessions, which is a fundamental task in understanding multi-party conversations. Existing work on conversation disentanglement relies heavily upon human-annotated datasets, which are expensive to obtain in practice. In this work, we explore to train a conversation disentanglement model without referencing any human annotations. Our method is built upon a deep co-training algorithm, which consists of two neural networks: a message-pair classifier and a session classifier. The former is responsible for retrieving local relations between two messages while the latter categorizes a message to a session by capturing context-aware information. Both networks are initialized respectively with pseudo data built from an unannotated corpus. During the deep co-training process, we use the session classifier as a reinforcement learning component to learn a session assigning policy by maximizing the local rewards given by the message-pair classifier. For the message-pair classifier, we enrich its training data by retrieving message pairs with high confidence from the disentangled sessions predicted by the session classifier. Experimental results on the large Movie Dialogue Dataset demonstrate that our proposed approach achieves competitive performance compared to the previous supervised methods. Further experiments show that the predicted disentangled conversations can promote the performance on the downstream task of multi-party response selection.

cs.CL↗

arXiv2021

The sustainability probability for the critical Derrida-Retaux model

We are interested in the recursive model $(Y_n, \, n\ge 0)$ studied by Collet, Eckmann, Glaser and Martin [9] and by Derrida and Retaux [12]. We prove that at criticality, the probability ${\bf P}(Y_n>0)$ behaves like $n^{-2 + o(1)}$ as $n$ goes to infinity; this gives a weaker confirmation of predictions made in [9], [12] and [6]. Our method relies on studying the number of pivotal vertices and open paths, combined with a delicate coupling argument.

math.PR↗

arXiv2021

Path decompositions of perturbed reflecting Brownian motions

We are interested in path decompositions of a perturbed reflecting Brownian motion (PRBM) at the hitting times and at the minimum. Our study relies on the loop soups developed by Lawler and Werner [10] and Le Jan [13]-[14], in particular on a result discovered by Lupu [15] identifying the law of the excursions of the PRBM above its past minimum with the loop measure of Brownian bridges.

math.PR↗

arXiv2021

Points of infinite multiplicity of planar Brownian motion: measures and local times

It is well-known (see Dvoretzky, Erd{\H o}s and Kakutani [8] and Le Gall [12]) that a planar Brownian motion $(B_t)_{t\ge 0}$ has points of infinite multiplicity, and these points form a dense set on the range. Our main result is the construction of a family of random measures, denoted by $\{{\mathcal M}_{\infty}^α\}_{0< α<2}$, that are supported by the set of the points of infinite multiplicity. We prove that for any $α\in (0, 2)$, almost surely the Hausdorff dimension of ${\mathcal M}_{\infty}^α$ equals $2-α$, and ${\mathcal M}_{\infty}^α$ is supported by the set of thick points defined in Bass, Burdzy and Khoshnevisan [1] as well as by that defined in Dembo, Peres, Rosen and Zeitouni [5]. Our construction also reveals that with probability one, ${\mathcal M}_\infty^α({\rm d} x)$-almost everywhere, there exists a continuous nondecreasing additive functional $({\mathfrak L}_t^x)_{t\ge 0}$, called local times at $x$, such that the support of $ {\rm d} {\mathfrak L}_t^x$ coincides with the level set $\{t: B_t=x\}$.

math.PR↗

arXiv2020

An infinite-dimensional representation of the Ray-Knight theorems

The classical Ray-Knight theorems for Brownian motion determine the law of its local time process either at the first hitting time of a given value a by the local time at the origin, or at the first hitting time of a given position b by Brownian motion. We extend these results by describing the local time process jointly for all a and all b, by means of stochastic integral with respect to an appropriate white noise. Our result applies to $μ$-processes, and has an immediate application: a $μ$-process is the height process of a Feller continuous-state branching process (CSBP) with immigration (Lambert [10]), whereas a Feller CSBP with immigration satisfies a stochastic differential equation driven by a white noise (Dawson and Li [7]); our result gives an explicit relation between these two descriptions and shows that the stochastic differential equation in question is a reformulation of Tanaka's formula.

math.PR↗

arXiv2020

Learned Hardware/Software Co-Design of Neural Accelerators

The use of deep learning has grown at an exponential rate, giving rise to numerous specialized hardware and software systems for deep learning. Because the design space of deep learning software stacks and hardware accelerators is diverse and vast, prior work considers software optimizations separately from hardware architectures, effectively reducing the search space. Unfortunately, this bifurcated approach means that many profitable design points are never explored. This paper instead casts the problem as hardware/software co-design, with the goal of automatically identifying desirable points in the joint design space. The key to our solution is a new constrained Bayesian optimization framework that avoids invalid solutions by exploiting the highly constrained features of this design space, which are semi-continuous/semi-discrete. We evaluate our optimization framework by applying it to a variety of neural models, improving the energy-delay product by 18% (ResNet) and 40% (DQN) over hand-tuned state-of-the-art systems, as well as demonstrating strong results on other neural network architectures, such as MLPs and Transformers.

cs.LG↗

arXiv2020

Improving Image Captioning with Better Use of Captions

Image captioning is a multimodal problem that has drawn extensive attention in both the natural language processing and computer vision community. In this paper, we present a novel image captioning architecture to better explore semantics available in captions and leverage that to enhance both image representation and caption generation. Our models first construct caption-guided visual relationship graphs that introduce beneficial inductive bias using weakly supervised multi-instance learning. The representation is then enhanced with neighbouring and contextual nodes with their textual and visual features. During generation, the model further incorporates visual relationships using multi-task learning for jointly predicting word and object/predicate tag sequences. We perform extensive experiments on the MSCOCO dataset, showing that the proposed framework significantly outperforms the baselines, resulting in the state-of-the-art performance under a wide range of evaluation metrics.

cs.CV↗

arXiv2020

The stable Derrida--Retaux system at criticality

The Derrida--Retaux recursive system was investigated by Derrida and Retaux (2014) as a hierarchical renormalization model in statistical physics. A prediction of Derrida and Retaux (2014) on the free energy has recently been rigorously proved (Chen, Dagard, Derrida, Hu, Lifshits and Shi (2019+)), confirming the Berezinskii--Kosterlitz--Thouless-type phase transition in the system. Interestingly, it has been established in Chen, Dagard, Derrida, Hu, Lifshits and Shi (2019+) that the prediction is valid only under a certain integrability assumption on the initial distribution, and a new type of universality result has been shown when this integrability assumption is not satisfied. We present a unified approach for systems satisfying a certain domination condition, and give an upper bound for derivatives of all orders of the moment generating function. When the integrability assumption is not satisfied, our result allows to identify the large-time order of magnitude of the product of the moment generating functions at criticality, confirming and completing a previous result in Collet, Eckmann, Glaser and Martin (1984).

math.PR↗