arXiv subjects
Publications and source records attributed to Zhan Shi.
We review recent results and conjectures for a simplified version of the depinning problem in presence of disorder which was introduced by Derrida and Retaux in 2014. For this toy model, the depinning transition has been predicted to be of the Berezinskii--Kosterlitz--Thouless type. Here we discuss under which integrability conditions this prediction can be proved and how it is modified otherwise.
We are interested in the nearly supercritical regime in a family of max-type recursive models studied by Collet, Eckman, Glaser and Martin and by Derrida and Retaux, and prove that under a suitable integrability assumption on the initial distribution, the free energy vanishes at the transition with an essential singularity with exponent $\tfrac12$. This gives a weaker answer to a conjecture of Derrida and Retaux. Other behaviours are obtained when the integrability condition is not satisfied.
As the performance of computer systems stagnates due to the end of Moore's Law, there is a need for new models that can understand and optimize the execution of general purpose code. While there is a growing body of work on using Graph Neural Networks (GNNs) to learn representations of source code, these representations do not understand how code dynamically executes. In this work, we propose a new approach to use GNNs to learn fused representations of general source code and its execution. Our approach defines a multi-task GNN over low-level representations of source code and program state (i.e., assembly code and dynamic memory states), converting complex source code constructs and complex data structures into a simpler, more uniform format. We show that this leads to improved performance over similar methods that do not use execution and it opens the door to applying GNN models to new tasks that would not be feasible from static code alone. As an illustration of this, we apply the new model to challenging dynamic tasks (branch prediction and prefetching) from the SPEC CPU benchmark suite, outperforming the state-of-the-art by 26% and 45% respectively. Moreover, we use the learned fused graph embeddings to demonstrate transfer learning with high performance on an indirectly related task (algorithm classification).
We introduce a class of stochastic algorithms for minimizing weakly convex functions over proximally smooth sets. As their main building blocks, the algorithms use simplified models of the objective function and the constraint set, along with a retraction operation to restore feasibility. All the proposed methods come equipped with a finite time efficiency guarantee in terms of a natural stationarity measure. We discuss consequences for nonsmooth optimization over smooth manifolds and over sets cut out by weakly-convex inequalities.
The problem of adversarial examples has highlighted the need for a theory of regularisation that is general enough to apply to exotic function classes, such as universal approximators. In response, we give a very general equality result regarding the relationship between distributional robustness and regularisation, as defined with a transportation cost uncertainty set. The theory allows us to (tightly) certify the robustness properties of a Lipschitz-regularised model with very mild assumptions. As a theoretical application we show a new result explicating the connection between adversarial learning and distributional robustness. We then give new results for how to achieve Lipschitz regularisation of kernel classifiers, which are demonstrated experimentally.
We show that a coalescence equation exhibits a variety of critical behaviors, depending on the initial condition. This equation was introduced a few years ago to understand a toy model {studied by Derrida and Retaux to mimic} the depinning transition in presence of disorder. It was shown recently that this toy model exhibits the same critical behaviors as the equation studied in the present work. Here we find several families of exact solutions of this coalescence equation, in particular a family of scaling functions which are closely related to the different possible critical behaviors. These scaling functions lead to new conjectures, in particular on the shapes of the critical trees, that we have checked numerically.
We study deviation probabilities for the number of high positioned particles in branching Brownian motion, and confirm a conjecture of Derrida and Shi (2016). We also solve the corresponding problem for the two-dimensional discrete Gaussian free field. Our method relies on an elementary inequality for inhomogeneous Galton-Watson processes.
We consider a simple max-type recursive model which was introduced in the study of depinning transition in presence of strong disorder, by Derrida and Retaux. Our interest is focused on the critical regime, for which we study the extinction probability, the first moment and the moment generating function. Several stronger assertions are stated as conjectures.
The last few years have seen a staggering number of empirical studies of the robustness of neural networks in a model of adversarial perturbations of their inputs. Most rely on an adversary which carries out local modifications within prescribed balls. None however has so far questioned the broader picture: how to frame a resource-bounded adversary so that it can be severely detrimental to learning, a non-trivial problem which entails at a minimum the choice of loss and classifiers. We suggest a formal answer for losses that satisfy the minimal statistical requirement of being proper. We pin down a simple sufficient property for any given class of adversaries to be detrimental to learning, involving a central measure of "harmfulness" which generalizes the well-known class of integral probability metrics. A key feature of our result is that it holds for all proper losses, and for a popular subset of these, the optimisation of this central measure appears to be independent of the loss. When classifiers are Lipschitz -- a now popular approach in adversarial training --, this optimisation resorts to optimal transport to make a low-budget compression of class marginals. Toy experiments reveal a finding recently separately observed: training against a sufficiently budgeted adversary of this kind improves generalization.
Winograd Schema Challenge (WSC) was proposed as an AI-hard problem in testing computers' intelligence on common sense representation and reasoning. This paper presents the new state-of-theart on WSC, achieving an accuracy of 71.1%. We demonstrate that the leading performance benefits from jointly modelling sentence structures, utilizing knowledge learned from cutting-edge pretraining models, and performing fine-tuning. We conduct detailed analyses, showing that fine-tuning is critical for achieving the performance, but it helps more on the simpler associative problems. Modelling sentence dependency structures, however, consistently helps on the harder non-associative subset of WSC. Analysis also shows that larger fine-tuning datasets yield better performances, suggesting the potential benefit of future work on annotating more Winograd schema sentences.
Robust risk minimisation has several advantages: it has been studied with regards to improving the generalisation properties of models and robustness to adversarial perturbation. We bound the distributionally robust risk for a model class rich enough to include deep neural networks by a regularised empirical risk involving the Lipschitz constant of the model. This allows us to interpretand quantify the robustness properties of a deep neural network. As an application we show the distributionally robust risk upperbounds the adversarial training risk.
Text generation is a crucial task in NLP. Recently, several adversarial generative models have been proposed to improve the exposure bias problem in text generation. Though these models gain great success, they still suffer from the problems of reward sparsity and mode collapse. In order to address these two problems, in this paper, we employ inverse reinforcement learning (IRL) for text generation. Specifically, the IRL framework learns a reward function on training data, and then an optimal policy to maximum the expected total reward. Similar to the adversarial models, the reward and policy function in IRL are optimized alternately. Our method has two advantages: (1) the reward function can produce more dense reward signals. (2) the generation policy, trained by "entropy regularized" policy gradient, encourages to generate more diversified texts. Experiment results demonstrate that our proposed method can generate higher quality texts than the previous methods.
We are interested in a simple max-type recursive model studied by Derrida and Retaux (2014) in the context of a physics problem, and find a wide range for the exponent in the free energy in the nearly supercritical regime.
We consider a class of biased random walks on infinite graphs and present several general results on the spectral radius of biased random walk.
The uniform spanning forest measure ($\mathsf{USF}$) on a locally finite, infinite connected graph $G$ with conductance $c$ is defined as a weak limit of uniform spanning tree measure on finite subgraphs. Depending on the underlying graph and conductances, the corresponding $\mathsf{USF}$ is not necessarily concentrated on the set of spanning trees. Pemantle~\cite{PR1991} showed that on $\mathbb{Z}^d$, equipped with the unit conductance $ c=1$, $\mathsf{USF}$ is concentrated on spanning trees if and only if $d \leq 4$. In this work we study the $\mathsf{USF}$ associated with conductances induced by $λ$--biased random walk on $\mathbb{Z}^d$, $d \geq 2$, $0 < λ< 1$, i.e. conductances are set to be $c(e) = λ^{-|e|}$, where $|e|$ is the graph distance of the edge $e$ from the origin. Our main result states that in this case $\mathsf{USF}$ consists of finitely many trees if and only if $d = 2$ or $3$. More precisely, we prove that the uniform spanning forest has $2^d$ trees if $d = 2$ or $3$, and infinitely many trees if $d \geq 4$. Our method relies on the analysis of the spectral radius and the speed of the $λ$--biased random walk on $\mathbb{Z}^d$.
Neural word segmentation has attracted more and more research interests for its ability to alleviate the effort of feature engineering and utilize the external resource by the pre-trained character or word embeddings. In this paper, we propose a new neural model to incorporate the word-level information for Chinese word segmentation. Unlike the previous word-based models, our model still adopts the framework of character-based sequence labeling, which has advantages on both effectiveness and efficiency at the inference stage. To utilize the word-level information, we also propose a new long short-term memory (LSTM) architecture over directed acyclic graph (DAG). Experimental results demonstrate that our model leads to better performances than the baseline models.
We study the dynamics of a pulse-gated semiconductor double quantum dot qubit. In our experiments, the qubit coherence times are relatively long, but the visibility of the quantum oscillations is low. We show that these observations are consistent with a theory that incorporates decoherence arising from charge noise that gives rise to detuning fluctuations of the double dot. Because effects from charge noise are largest near the singlet-triplet avoided level crossing, the visibility of the oscillations are low when the singlet-triplet avoided level crossing occurs in the vicinity of the charge degeneracy point crossed during the manipulation, but there is only modest dephasing at the large detuning value at which the quantum phase accumulates. This theory agrees well with experimental data and predicts that the visibility can be increased greatly by appropriate tuning of the interdot tunneling rate.
We have shown recently how to calculate the large deviation function of the position $X_{\max}(t) $ of the right most particle of a branching Brownian motion at time $t$. This large deviation function exhibits a phase transition at a certain negative velocity. Here we extend this result to more general branching random walks and show that the probability distribution of $X_{\max}(t)$ has, asymptotically in time, a prefactor characterized by non trivial power law.