arXiv subjects

Qian Zhang

Publications and source records attributed to Qian Zhang.

At least 271 records · Page 15Linked to original sources

arXiv2023

Improving Few-shot Image Generation by Structural Discrimination and Textural Modulation

Few-shot image generation, which aims to produce plausible and diverse images for one category given a few images from this category, has drawn extensive attention. Existing approaches either globally interpolate different images or fuse local representations with pre-defined coefficients. However, such an intuitive combination of images/features only exploits the most relevant information for generation, leading to poor diversity and coarse-grained semantic fusion. To remedy this, this paper proposes a novel textural modulation (TexMod) mechanism to inject external semantic signals into internal local representations. Parameterized by the feedback from the discriminator, our TexMod enables more fined-grained semantic injection while maintaining the synthesis fidelity. Moreover, a global structural discriminator (StructD) is developed to explicitly guide the model to generate images with reasonable layout and outline. Furthermore, the frequency awareness of the model is reinforced by encouraging the model to distinguish frequency signals. Together with these techniques, we build a novel and effective model for few-shot image generation. The effectiveness of our model is identified by extensive experiments on three popular datasets and various settings. Besides achieving state-of-the-art synthesis performance on these datasets, our proposed techniques could be seamlessly integrated into existing models for a further performance boost.

cs.CV↗

arXiv2023

VMA: Divide-and-Conquer Vectorized Map Annotation System for Large-Scale Driving Scene

High-definition (HD) map serves as the essential infrastructure of autonomous driving. In this work, we build up a systematic vectorized map annotation framework (termed VMA) for efficiently generating HD map of large-scale driving scene. We design a divide-and-conquer annotation scheme to solve the spatial extensibility problem of HD map generation, and abstract map elements with a variety of geometric patterns as unified point sequence representation, which can be extended to most map elements in the driving scene. VMA is highly efficient and extensible, requiring negligible human effort, and flexible in terms of spatial scale and element type. We quantitatively and qualitatively validate the annotation performance on real-world urban and highway scenes, as well as NYC Planimetric Database. VMA can significantly improve map generation efficiency and require little human effort. On average VMA takes 160min for annotating a scene with a range of hundreds of meters, and reduces 52.3% of the human cost, showing great application value. Code: https://github.com/hustvl/VMA.

cs.CV↗

arXiv2023

VAD: Vectorized Scene Representation for Efficient Autonomous Driving

Autonomous driving requires a comprehensive understanding of the surrounding environment for reliable trajectory planning. Previous works rely on dense rasterized scene representation (e.g., agent occupancy and semantic map) to perform planning, which is computationally intensive and misses the instance-level structure information. In this paper, we propose VAD, an end-to-end vectorized paradigm for autonomous driving, which models the driving scene as a fully vectorized representation. The proposed vectorized paradigm has two significant advantages. On one hand, VAD exploits the vectorized agent motion and map elements as explicit instance-level planning constraints which effectively improves planning safety. On the other hand, VAD runs much faster than previous end-to-end planning methods by getting rid of computation-intensive rasterized representation and hand-designed post-processing steps. VAD achieves state-of-the-art end-to-end planning performance on the nuScenes dataset, outperforming the previous best method by a large margin. Our base model, VAD-Base, greatly reduces the average collision rate by 29.0% and runs 2.5x faster. Besides, a lightweight variant, VAD-Tiny, greatly improves the inference speed (up to 9.3x) while achieving comparable planning performance. We believe the excellent performance and the high efficiency of VAD are critical for the real-world deployment of an autonomous driving system. Code and models are available at https://github.com/hustvl/VAD for facilitating future research.

cs.RO↗

arXiv2023

Conditional Perceptual Quality Preserving Image Compression

We propose conditional perceptual quality, an extension of the perceptual quality defined in \citet{blau2018perception}, by conditioning it on user defined information. Specifically, we extend the original perceptual quality $d(p_{X},p_{\hat{X}})$ to the conditional perceptual quality $d(p_{X|Y},p_{\hat{X}|Y})$, where $X$ is the original image, $\hat{X}$ is the reconstructed, $Y$ is side information defined by user and $d(.,.)$ is divergence. We show that conditional perceptual quality has similar theoretical properties as rate-distortion-perception trade-off \citep{blau2019rethinking}. Based on these theoretical results, we propose an optimal framework for conditional perceptual quality preserving compression. Experimental results show that our codec successfully maintains high perceptual quality and semantic quality at all bitrate. Besides, by providing a lowerbound of common randomness required, we settle the previous arguments on whether randomness should be incorporated into generator for (conditional) perceptual quality compression. The source code is provided in supplementary material.

eess.IV↗

arXiv2023

Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learning

Learning-based approaches have achieved remarkable performance in the domain of autonomous driving. Leveraging the impressive ability of neural networks and large amounts of human driving data, complex patterns and rules of driving behavior can be encoded as a model to benefit the autonomous driving system. Besides, an increasing number of data-driven works have been studied in the decision-making and motion planning module. However, the reliability and the stability of the neural network is still full of uncertainty. In this paper, we introduce a hierarchical planning architecture including a high-level grid-based behavior planner and a low-level trajectory planner, which is highly interpretable and controllable. As the high-level planner is responsible for finding a consistent route, the low-level planner generates a feasible trajectory. We evaluate our method both in closed-loop simulation and real world driving, and demonstrate the neural network planner has outstanding performance in complex urban autonomous driving scenarios.

cs.RO↗

arXiv2023

On the stability of critical points of the Hardy-Littlewood-Sobolev inequality

This paper is concerned with the quantitative stability of critical points of the Hardy-Littlewood-Sobolev inequality. Namely, we give quantitative estimates for the Choquard equation: $$-Δu=(I_μ\ast|u|^{2_μ^*}) u^{2_μ^*-1}\ \ \text{in}\ \ \R^N,$$ where $u>0,\ N\geq 3,\ μ\in(0,N)$, $I_μ$ is the Riesz potential and $2_μ^* \coloneqq \frac{2N-μ}{N-2}$ is the upper Hardy-Littlewood-Sobolev critical exponent. The Struwe's decomposition (see M. Struwe: Math Z.,1984) showed that the equation $Δu + u^{\frac{N+2}{N-2 }}=0$ has phenomenon of ``stable up to bubbling'', that is, if $u\geq0$ and $\|Δu+u^{\frac{N+2}{N-2}}\|_{(\mathcal{D}^{1,2})^{-1}}$ approaches zero, then $d(u)$ goes to zero, where $d(u)$ denotes the $\mathcal{D}^{1,2}(\R^N)$-distance between $u$ and the set of all sums of Talenti bubbles. Ciraolo, F{}igalli and Maggi (Int. Math. Res. Not.,2017) obtained the f{}irst quantitative version of Struwe's decomposition with single bubble in all dimensions $N\geq 3$, i.e, $\displaystyle d(u)\leq C\|Δu+u^{\frac{N+2}{N-2}}\|_{L^{\frac{2N}{N+2}}}.$ For multiple bubbles, F{}igalli and Glaudo (Arch. Rational Mech. Anal., 2020) obtained quantitative estimates depending on the dimension, namely $$ d(u)\leq C\|Δu+u^{\frac{N+2}{N-2}}\|_{(\mathcal{D}^{1,2})^{-1}}, \hbox{ where } 3\leq N\leq 5,$$ which is invalid as $N\geq 6.$ \vskip0.1in \quad In this paper, we prove the quantitative estimate of the Hardy-Littlewood-Sobolev inequality, we get $$d(u)\leq C\|Δu +(I_μ\ast|u|^{2_μ^*})|u|^{2_μ^*-2}u\|_{(\mathcal{D}^{1,2})^{-1}}, \hbox{ when } N=3 \hbox{ and } 5/2< μ<3.$$

math.AP↗

arXiv2023

Reward Selection with Noisy Observations

We study a fundamental problem in optimization under uncertainty. There are $n$ boxes; each box $i$ contains a hidden reward $x_i$. Rewards are drawn i.i.d. from an unknown distribution $\mathcal{D}$. For each box $i$, we see $y_i$, an unbiased estimate of its reward, which is drawn from a Normal distribution with known standard deviation $σ_i$ (and an unknown mean $x_i$). Our task is to select a single box, with the goal of maximizing our reward. This problem captures a wide range of applications, e.g. ad auctions, where the hidden reward is the click-through rate of an ad. Previous work in this model [BKMR12] proves that the naive policy, which selects the box with the largest estimate $y_i$, is suboptimal, and suggests a linear policy, which selects the box $i$ with the largest $y_i - c \cdot σ_i$, for some $c > 0$. However, no formal guarantees are given about the performance of either policy (e.g., whether their expected reward is within some factor of the optimal policy's reward). In this work, we prove that both the naive policy and the linear policy are arbitrarily bad compared to the optimal policy, even when $\mathcal{D}$ is well-behaved, e.g. has monotone hazard rate (MHR), and even under a "small tail" condition, which requires that not too many boxes have arbitrarily large noise. On the flip side, we propose a simple threshold policy that gives a constant approximation to the reward of a prophet (who knows the realized values $x_1, \dots, x_n$) under the same "small tail" condition. We prove that when this condition is not satisfied, even an optimal clairvoyant policy (that knows $\mathcal{D}$) cannot get a constant approximation to the prophet, even for MHR distributions, implying that our threshold policy is optimal against the prophet benchmark, up to constants.

cs.GT↗

arXiv2023

BrainCog: A Spiking Neural Network based Brain-inspired Cognitive Intelligence Engine for Brain-inspired AI and Brain Simulation

Spiking neural networks (SNNs) have attracted extensive attentions in Brain-inspired Artificial Intelligence and computational neuroscience. They can be used to simulate biological information processing in the brain at multiple scales. More importantly, SNNs serve as an appropriate level of abstraction to bring inspirations from brain and cognition to Artificial Intelligence. In this paper, we present the Brain-inspired Cognitive Intelligence Engine (BrainCog) for creating brain-inspired AI and brain simulation models. BrainCog incorporates different types of spiking neuron models, learning rules, brain areas, etc., as essential modules provided by the platform. Based on these easy-to-use modules, BrainCog supports various brain-inspired cognitive functions, including Perception and Learning, Decision Making, Knowledge Representation and Reasoning, Motor Control, and Social Cognition. These brain-inspired AI models have been effectively validated on various supervised, unsupervised, and reinforcement learning tasks, and they can be used to enable AI models to be with multiple brain-inspired cognitive functions. For brain simulation, BrainCog realizes the function simulation of decision-making, working memory, the structure simulation of the Neural Circuit, and whole brain structure simulation of Mouse brain, Macaque brain, and Human brain. An AI engine named BORN is developed based on BrainCog, and it demonstrates how the components of BrainCog can be integrated and used to build AI models and applications. To enable the scientific quest to decode the nature of biological intelligence and create AI, BrainCog aims to provide essential and easy-to-use building blocks, and infrastructural support to develop brain-inspired spiking neural network based AI, and to simulate the cognitive brains at multiple scales. The online repository of BrainCog can be found at https://github.com/braincog-x.

cs.NE↗

arXiv2023

Monocular Road Planar Parallax Estimation

Estimating the 3D structure of the drivable surface and surrounding environment is a crucial task for assisted and autonomous driving. It is commonly solved either by using 3D sensors such as LiDAR or directly predicting the depth of points via deep learning. However, the former is expensive, and the latter lacks the use of geometry information for the scene. In this paper, instead of following existing methodologies, we propose Road Planar Parallax Attention Network (RPANet), a new deep neural network for 3D sensing from monocular image sequences based on planar parallax, which takes full advantage of the omnipresent road plane geometry in driving scenes. RPANet takes a pair of images aligned by the homography of the road plane as input and outputs a $γ$ map (the ratio of height to depth) for 3D reconstruction. The $γ$ map has the potential to construct a two-dimensional transformation between two consecutive frames. It implies planar parallax and can be combined with the road plane serving as a reference to estimate the 3D structure by warping the consecutive frames. Furthermore, we introduce a novel cross-attention module to make the network better perceive the displacements caused by planar parallax. To verify the effectiveness of our method, we sample data from the Waymo Open Dataset and construct annotations related to planar parallax. Comprehensive experiments are conducted on the sampled dataset to demonstrate the 3D reconstruction accuracy of our approach in challenging scenarios.

cs.CV↗

arXiv2023

Universal Rates for Multiclass Learning

We study universal rates for multiclass classification, establishing the optimal rates (up to log factors) for all hypothesis classes. This generalizes previous results on binary classification (Bousquet, Hanneke, Moran, van Handel, and Yehudayoff, 2021), and resolves an open question studied by Kalavasis, Velegkas, and Karbasi (2022) who handled the multiclass setting with a bounded number of class labels. In contrast, our result applies for any countable label space. Even for finite label space, our proofs provide a more precise bounds on the learning curves, as they do not depend on the number of labels. Specifically, we show that any class admits exponential rates if and only if it has no infinite Littlestone tree, and admits (near-)linear rates if and only if it has no infinite Daniely-Shalev-Shwartz-Littleston (DSL) tree, and otherwise requires arbitrarily slow rates. DSL trees are a new structure we define in this work, in which each node of the tree is given by a pseudo-cube of possible classifications of a given set of points. Pseudo-cubes are a structure, rooted in the work of Daniely and Shalev-Shwartz (2014), and recently shown by Brukhim, Carmon, Dinur, Moran, and Yehudayoff (2022) to characterize PAC learnability (i.e., uniform rates) for multiclass classification. We also resolve an open question of Kalavasis, Velegkas, and Karbasi (2022) regarding the equivalence of classes having infinite Graph-Littlestone (GL) trees versus infinite Natarajan-Littlestone (NL) trees, showing that they are indeed equivalent.

cs.LG↗

arXiv2023

Enhanced Population on Ionic Excited States by Synchronized Ionization and Multiphoton Resonance

We study population distributions and lasing actions of N_2^+ driven by femtosecond lasers with various wavelengths, and uncover an efficient ionic excitation mechanism induced by synchronized ionization and multiphoton resonance. Our results show that the strongest N_2^+ lasing appears around 1000 nm pump wavelength. At the optimal wavelength, the pump-energy threshold for air lasing generation is reduced by five folds compared with that required by the previous 800 nm pump laser. Simulations based on the ionization-coupling model indicate that although the Stark-assisted three-photon resonance can be satisfied within a broad pump wavelength range, the optimal pump wavelength arises when the dynamic three-photon resonance temporally synchronizes with the ionization injection. In this case, the ionic dipoles created at each half optical cycle have the same phase. The dipole phase locking promotes the continuous population transfer from ionic ground state to the excited state, giving rise to a dramatic increase of excited-state population. This work provides new insight on the photoexcitation mechanism of ions in strong laser fields, and opens up a route for optimizing ionic radiations.

physics.atom-ph↗

arXiv2023

ProRes: Exploring Degradation-aware Visual Prompt for Universal Image Restoration

Image restoration aims to reconstruct degraded images, e.g., denoising or deblurring. Existing works focus on designing task-specific methods and there are inadequate attempts at universal methods. However, simply unifying multiple tasks into one universal architecture suffers from uncontrollable and undesired predictions. To address those issues, we explore prompt learning in universal architectures for image restoration tasks. In this paper, we present Degradation-aware Visual Prompts, which encode various types of image degradation, e.g., noise and blur, into unified visual prompts. These degradation-aware prompts provide control over image processing and allow weighted combinations for customized image restoration. We then leverage degradation-aware visual prompts to establish a controllable and universal model for image restoration, called ProRes, which is applicable to an extensive range of image restoration tasks. ProRes leverages the vanilla Vision Transformer (ViT) without any task-specific designs. Furthermore, the pre-trained ProRes can easily adapt to new tasks through efficient prompt tuning with only a few images. Without bells and whistles, ProRes achieves competitive performance compared to task-specific methods and experiments can demonstrate its ability for controllable restoration and adaptation for new tasks. The code and models will be released in \url{https://github.com/leonmakise/ProRes}.

cs.CV↗

arXiv2023

Augmenting Greybox Fuzzing with Generative AI

Real-world programs expecting structured inputs often has a format-parsing stage gating the deeper program space. Neither a mutation-based approach nor a generative approach can provide a solution that is effective and scalable. Large language models (LLM) pre-trained with an enormous amount of natural language corpus have proved to be effective for understanding the implicit format syntax and generating format-conforming inputs. In this paper, propose ChatFuzz, a greybox fuzzer augmented by generative AI. More specifically, we pick a seed in the fuzzer's seed pool and prompt ChatGPT generative models to variations, which are more likely to be format-conforming and thus of high quality. We conduct extensive experiments to explore the best practice for harvesting the power of generative LLM models. The experiment results show that our approach improves the edge coverage by 12.77\% over the SOTA greybox fuzzer (AFL++) on 12 target programs from three well-tested benchmarks. As for vulnerability detection, \sys is able to perform similar to or better than AFL++ for programs with explicit syntax rules but not for programs with non-trivial syntax.

cs.CR↗

arXiv2023

FireFly: A High-Throughput Hardware Accelerator for Spiking Neural Networks with Efficient DSP and Memory Optimization

Spiking neural networks (SNNs) have been widely used due to their strong biological interpretability and high energy efficiency. With the introduction of the backpropagation algorithm and surrogate gradient, the structure of spiking neural networks has become more complex, and the performance gap with artificial neural networks has gradually decreased. However, most SNN hardware implementations for field-programmable gate arrays (FPGAs) cannot meet arithmetic or memory efficiency requirements, which significantly restricts the development of SNNs. They do not delve into the arithmetic operations between the binary spikes and synaptic weights or assume unlimited on-chip RAM resources by using overly expensive devices on small tasks. To improve arithmetic efficiency, we analyze the neural dynamics of spiking neurons, generalize the SNN arithmetic operation to the multiplex-accumulate operation, and propose a high-performance implementation of such operation by utilizing the DSP48E2 hard block in Xilinx Ultrascale FPGAs. To improve memory efficiency, we design a memory system to enable efficient synaptic weights and membrane voltage memory access with reasonable on-chip RAM consumption. Combining the above two improvements, we propose an FPGA accelerator that can process spikes generated by the firing neuron on-the-fly (FireFly). FireFly is the first SNN accelerator that incorporates DSP optimization techniques into SNN synaptic operations. FireFly is implemented on several FPGA edge devices with limited resources but still guarantees a peak performance of 5.53TOP/s at 300MHz. As a lightweight accelerator, FireFly achieves the highest computational density efficiency compared with existing research using large FPGA devices.

cs.NE↗

arXiv2023

Multi-Granularity Archaeological Dating of Chinese Bronze Dings Based on a Knowledge-Guided Relation Graph

The archaeological dating of bronze dings has played a critical role in the study of ancient Chinese history. Current archaeology depends on trained experts to carry out bronze dating, which is time-consuming and labor-intensive. For such dating, in this study, we propose a learning-based approach to integrate advanced deep learning techniques and archaeological knowledge. To achieve this, we first collect a large-scale image dataset of bronze dings, which contains richer attribute information than other existing fine-grained datasets. Second, we introduce a multihead classifier and a knowledge-guided relation graph to mine the relationship between attributes and the ding era. Third, we conduct comparison experiments with various existing methods, the results of which show that our dating method achieves a state-of-the-art performance. We hope that our data and applied networks will enrich fine-grained classification research relevant to other interdisciplinary areas of expertise. The dataset and source code used are included in our supplementary materials, and will be open after submission owing to the anonymity policy. Source codes and data are available at: https://github.com/zhourixin/bronze-Ding.

cs.CV↗

arXiv2023

Existence of positive ground state solutions for the coupled Choquard system with potential

In this paper, we study the following coupled Choquard system in $\mathbb R^N$: $$\left\{\begin{align}&-Δu+A(x)u=\frac{2p}{p+q} \bigl(I_α\ast |v|^q\bigr)|u|^{p-2}u,\\ &-Δv+B(x)v=\frac{2q}{p+q}\bigl(I_α\ast|u|^p\bigr)|v|^{q-2}v,\\ &\ u(x)\to0\ \ \hbox{and}\ \ v(x)\to0\ \ \hbox{as}\ |x|\to\infty,\end{align}\right.$$ where $α\in(0,N)$ and $\frac{N+α}{N}<p,\ q<2_*^α$, in which $2_*^α$ denotes $\frac{N+α}{N-2}$ if $N\geq 3$ and $2_*^α:= \infty$ if $N=1,\ 2$. The function $I_α$ is a Riesz potential. By using Nehari manifold method, we obtain the existence of positive ground state solution in the case of bounded potential and periodic potential respectively. In particular, the nonlinear term includes the well-studied case $p=q$ and $u(x)=v(x)$, and the less-studied case $p\neq q$ and $u(x)\neq v(x)$. Moreover it seems to be the first existence result for the case of $p\neq q$.

math.AP↗

arXiv2023

Existence and asymptotics of normalized solutions for logarithmic Schrödinger system

This paper is concerned with the following logarithmic Schrödinger system: $$\left\{\begin{align} \ &\ -Δu_1+ω_1u_1=μ_1 u_1\log u_1^2+\frac{2p}{p+q}|u_2|^{q}|u_1|^{p-2}u_1,\\ \ &\ -Δu_2+ω_2u_2=μ_2 u_2\log u_2^2+\frac{2q}{p+q}|u_1|^{p}|u_2|^{q-2}u_2,\\ \ &\ \int_Ω|u_i|^2\,dx=ρ_i,\ \ i=1,2,\\ \ &\ (u_1,u_2)\in H_0^1(Ω;\mathbb R^2),\end{align}\right.$$ where $Ω=\mathbb{R}^N$ or $Ω\subset\mathbb R^N(N\geq3)$ is a bounded smooth domain, $ω_i\in\mathbb R$, $μ_i,\ ρ_i>0,\ i=1,2.$ Moreover, $p,\ q\geq1,\ 2\leq p+q\leqslant 2^*$, where $2^*:=\frac{2N}{N-2}$. By using a Gagliardo-Nirenberg inequality and careful estimation of $u\log u^2$, firstly, we will provide a unified proof of the existence of the normalized ground states solution for all $2\leq p+q\leqslant 2^*$. Secondly, we consider the stability of normalized ground states solutions. Finally, we analyze the behavior of solutions for Sobolev-subcritical case and pass the limit as the exponent $p+q$ approaches to $2^*$. Notably, the uncertainty of sign of $u\log u^2$ in $(0,+\infty)$ is one of the difficulties of this paper, and also one of the motivations we are interested in. In particular, we can establish the existence of positive normalized ground states solutions for the Brézis-Nirenberg type problem with logarithmic perturbations (i.e., $p+q=2^*$). In addition, our study includes proving the existence of solutions to the logarithmic type Brézis-Nirenberg problem with and without the $L^2$-mass $\int_Ω|u_i|^2\,dx=ρ_i(i=1,2)$ constraint by two different methods, respectively. Our results seems to be the first result of the normalized solution of the coupled nonlinear Schrödinger system with logarithmic perturbation.

math.AP↗

arXiv2023

Existence and concentration of ground state solution to a nonlocal Schrödinger equation

We study a class of Schrödinger-Kirchhoff system involving critical exponent. We aim to find suitable conditions to assure the existence of a positive ground state solution of Nehari-Pohouzaev type $u_{\varepsilon}$ with exponential decay at infinity for $\varepsilon$ and $ u_{\varepsilon}$ concentrates around a global minimum point of $ V$ as $ \varepsilon\rightarrow0^{+}.$ The nonlinear term includes the nonlinearity $f(u)\sim|u|^{p-1}u$ for the well-studied case $ p\in[3,5)$, and the less-studied case $p\in(2,3)$.

math.AP↗