arXiv subjects

Zheng Zhu

Publications and source records attributed to Zheng Zhu.

At least 127 records · Page 7Linked to original sources

arXiv2025

HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation

Human-motion video generation has been a challenging task, primarily due to the difficulty inherent in learning human body movements. While some approaches have attempted to drive human-centric video generation explicitly through pose control, these methods typically rely on poses derived from existing videos, thereby lacking flexibility. To address this, we propose HumanDreamer, a decoupled human video generation framework that first generates diverse poses from text prompts and then leverages these poses to generate human-motion videos. Specifically, we propose MotionVid, the largest dataset for human-motion pose generation. Based on the dataset, we present MotionDiT, which is trained to generate structured human-motion poses from text prompts. Besides, a novel LAMA loss is introduced, which together contribute to a significant improvement in FID by 62.4%, along with respective enhancements in R-precision for top1, top2, and top3 by 41.8%, 26.3%, and 18.3%, thereby advancing both the Text-to-Pose control accuracy and FID metrics. Our experiments across various Pose-to-Video baselines demonstrate that the poses generated by our method can produce diverse and high-quality human-motion videos. Furthermore, our model can facilitate other downstream tasks, such as pose sequence prediction and 2D-3D motion lifting.

cs.CV↗

arXiv2025

Exploring Large Language Models in Healthcare: Insights into Corpora Sources, Customization Strategies, and Evaluation Metrics

This study reviewed the use of Large Language Models (LLMs) in healthcare, focusing on their training corpora, customization techniques, and evaluation metrics. A systematic search of studies from 2021 to 2024 identified 61 articles. Four types of corpora were used: clinical resources, literature, open-source datasets, and web-crawled data. Common construction techniques included pre-training, prompt engineering, and retrieval-augmented generation, with 44 studies combining multiple methods. Evaluation metrics were categorized into process, usability, and outcome metrics, with outcome metrics divided into model-based and expert-assessed outcomes. The study identified critical gaps in corpus fairness, which contributed to biases from geographic, cultural, and socio-economic factors. The reliance on unverified or unstructured data highlighted the need for better integration of evidence-based clinical guidelines. Future research should focus on developing a tiered corpus architecture with vetted sources and dynamic weighting, while ensuring model transparency. Additionally, the lack of standardized evaluation frameworks for domain-specific models called for comprehensive validation of LLMs in real-world healthcare settings.

cs.CL↗

arXiv2024

Engineering Fractional Chern Insulators through Periodic Strain in Monolayer Graphene and Transition Metal Dichalcogenides

We propose the realization of interaction-driven insulators in periodically strained monolayer graphene and transition metal dichalcogenides (TMDs). Through extensive many-body exact diagonalization, we provide compelling evidence for various fractional Chern insulators (FCIs) in both strained monolayer graphene and TMDs, including the Laughlin states, Halperin states, and FCIs with tunable topological orders in Chern number |C| = 2 bands. We also discuss the relationship among band geometry, band filling and spin polarization. Notably, by examining both the entanglement spectrum and many-body Chern number, we reveal a state with Laughlin-like topological order emerging in the |C| = 2 band, which challenges the existing theoretical understanding of high Chern number (high-C) FCIs. These findings suggest that periodically strained monolayer graphene and TMDs provide promising platforms for engineering fractional Chern insulators and underscore the need for further investigation into high-C FCIs.

cond-mat.str-el↗

arXiv2024

OpenStereo: A Comprehensive Benchmark for Stereo Matching and Strong Baseline

Stereo matching aims to estimate the disparity between matching pixels in a stereo image pair, which is important to robotics, autonomous driving, and other computer vision tasks. Despite the development of numerous impressive methods in recent years, determining the most suitable architecture for practical application remains challenging. Addressing this gap, our paper introduces a comprehensive benchmark focusing on practical applicability rather than solely on individual models for optimized performance. Specifically, we develop a flexible and efficient stereo matching codebase, called OpenStereo. OpenStereo includes training and inference codes of more than 10 network models, making it, to our knowledge, the most complete stereo matching toolbox available. Based on OpenStereo, we conducted experiments and have achieved or surpassed the performance metrics reported in the original paper. Additionally, we conduct an exhaustive analysis and deconstruction of recent developments in stereo matching through comprehensive ablative experiments. These investigations inspired the creation of StereoBase, a strong baseline model. Our StereoBase ranks 1st on SceneFlow, KITTI 2015, 2012 (Reflective) among published methods and achieves the best performance across all metrics. In addition, StereoBase has strong cross-dataset generalization. Code is available at \url{https://github.com/XiandaGuo/OpenStereo}.

cs.CV↗

arXiv2024

ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restoration

Closed-loop simulation is crucial for end-to-end autonomous driving. Existing sensor simulation methods (e.g., NeRF and 3DGS) reconstruct driving scenes based on conditions that closely mirror training data distributions. However, these methods struggle with rendering novel trajectories, such as lane changes. Recent works have demonstrated that integrating world model knowledge alleviates these issues. Despite their efficiency, these approaches still encounter difficulties in the accurate representation of more complex maneuvers, with multi-lane shifts being a notable example. Therefore, we introduce ReconDreamer, which enhances driving scene reconstruction through incremental integration of world model knowledge. Specifically, DriveRestorer is proposed to mitigate artifacts via online restoration. This is complemented by a progressive data update strategy designed to ensure high-quality rendering for more complex maneuvers. To the best of our knowledge, ReconDreamer is the first method to effectively render in large maneuvers. Experimental results demonstrate that ReconDreamer outperforms Street Gaussians in the NTA-IoU, NTL-IoU, and FID, with relative improvements by 24.87%, 6.72%, and 29.97%. Furthermore, ReconDreamer surpasses DriveDreamer4D with PVG during large maneuver rendering, as verified by a relative improvement of 195.87% in the NTA-IoU metric and a comprehensive user study.

cs.CV↗

arXiv2024

DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation

Closed-loop simulation is essential for advancing end-to-end autonomous driving systems. Contemporary sensor simulation methods, such as NeRF and 3DGS, rely predominantly on conditions closely aligned with training data distributions, which are largely confined to forward-driving scenarios. Consequently, these methods face limitations when rendering complex maneuvers (e.g., lane change, acceleration, deceleration). Recent advancements in autonomous-driving world models have demonstrated the potential to generate diverse driving videos. However, these approaches remain constrained to 2D video generation, inherently lacking the spatiotemporal coherence required to capture intricacies of dynamic driving environments. In this paper, we introduce DriveDreamer4D, which enhances 4D driving scene representation leveraging world model priors. Specifically, we utilize the world model as a data machine to synthesize novel trajectory videos, where structured conditions are explicitly leveraged to control the spatial-temporal consistency of traffic elements. Besides, the cousin data training strategy is proposed to facilitate merging real and synthetic data for optimizing 4DGS. To our knowledge, DriveDreamer4D is the first to utilize video generation models for improving 4D reconstruction in driving scenarios. Experimental results reveal that DriveDreamer4D significantly enhances generation quality under novel trajectory views, achieving a relative improvement in FID by 32.1%, 46.4%, and 16.3% compared to PVG, S3Gaussian, and Deformable-GS. Moreover, DriveDreamer4D markedly enhances the spatiotemporal coherence of driving agents, which is verified by a comprehensive user study and the relative increases of 22.6%, 43.5%, and 15.6% in the NTA-IoU metric.

cs.CV↗

arXiv2024

Sobolev Versus Homogeneous Sobolev Extension

In this paper, we study the relationship between Sobolev extension domains and homogeneous Sobolev extension domains. Precisely, we obtain the following results. 1- Let $1\leq q\leq p\leq \infty$. Then a bounded $(L^{1, p}, L^{1, q})$-extension domain is also a $(W^{1, p}, W^{1, q})$-extension domain. 2- Let $1\leq q\leq p<q^\star\leq \infty$ or $n< q \leq p\leq \infty$. Then a bounded domain is a $(W^{1, p}, W^{1, q})$-extension domain if and only if it is an $(L^{1, p}, L^{1, q})$-extension domain. 3- For $1\leq q<n$ and $q^\star<p\leq \infty$, there exists a bounded domain $Ω\subset\mathbb{R}^n$ which is a $(W^{1, p}, W^{1, q})$-extension domain but not an $(L^{1, p}, L^{1, q})$-extension domain for $1 \leq q <p\leq n$.

math.FA↗

arXiv2024

EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video Generation

Video generation has emerged as a promising tool for world simulation, leveraging visual data to replicate real-world environments. Within this context, egocentric video generation, which centers on the human perspective, holds significant potential for enhancing applications in virtual reality, augmented reality, and gaming. However, the generation of egocentric videos presents substantial challenges due to the dynamic nature of egocentric viewpoints, the intricate diversity of actions, and the complex variety of scenes encountered. Existing datasets are inadequate for addressing these challenges effectively. To bridge this gap, we present EgoVid-5M, the first high-quality dataset specifically curated for egocentric video generation. EgoVid-5M encompasses 5 million egocentric video clips and is enriched with detailed action annotations, including fine-grained kinematic control and high-level textual descriptions. To ensure the integrity and usability of the dataset, we implement a sophisticated data cleaning pipeline designed to maintain frame consistency, action coherence, and motion smoothness under egocentric conditions. Furthermore, we introduce EgoDreamer, which is capable of generating egocentric videos driven simultaneously by action descriptions and kinematic control signals. The EgoVid-5M dataset, associated action annotations, and all data cleansing metadata will be released for the advancement of research in egocentric video generation.

cs.CV↗

arXiv2024

A Modulo Sampling Hardware Prototype and Reconstruction Algorithm Evaluation

Analog-to-digital converters (ADCs) play a vital important role in any devices via manipulating analog signals in a digital manner. Given that the amplitude of the signal exceeds the dynamic range of the ADCs, clipping occurs and the quality of the digitized signal degrades significantly. In this paper, we design a joint modulo sampling hardware and processing prototype which improves the ADCs' dynamic range by folding the signal before sampling. Both the detailed design of the hardware and the recovery results of various state-of-the-art processing algorithms including our proposed unlimited sampling line spectral estimation (USLSE) algorithm are presented. Additionally, key issues that arise during implementation are also addressed. It is demonstrated that the USLSE algorithm successfully recovers the original signal with a frequency of 2.5 kHz and an amplitude 10 times the ADC's dynamic range, and the linear prediction (LP) algorithm successfully recovers the original signal with a frequency of 3.5 kHz and an amplitude 10 times the ADC's dynamic range.

eess.SP↗

arXiv2024

Ubiquitous Field Transportation Robots with Robust Wheel-Leg Transformable Modules

This paper introduces two field transportation robots. Both robots are equipped with transformable wheel-leg modules, which can smoothly switch between operation modes and can work in various challenging terrains. SWhegPro, with six S-shaped legs, enables transporting loads in challenging uneven outdoor terrains. SWhegPro3, featuring four three-impeller wheels, has surprising stair-climbing performance in indoor scenarios. Different from ordinary gear-driven transformable mechanisms, the modular wheels we designed driven by self-locking electric push rods can switch modes accurately and stably with high loads, significantly improving the load capacity of the robot in leg mode. This study analyzes the robot's wheel-leg module operation when the terrain parameters change. Through the derivation of mathematical models and calculations based on simplified kinematic models, a method for optimizing the robot parameters and wheel-leg structure parameters is finally proposed.The design and control strategy are then verified through simulations and field experiments in various complex terrains, and the working performance of the two field transportation robots is calculated and analyzed by recording sensor data and proposing evaluation methods.

cs.RO↗

arXiv2024

Weak limits of Sobolev homeomorphisms are one to one

We prove that the key property in models of Nonlinear Elasticity which corresponds to the non-interpenetration of matter, i.e. injectivity a.e., can be achieved in the class of weak limits of homeomorphisms under very minimal assumptions. Let $Ω\subseteq \mathbb{R}^n$ be a domain and let $p>\left\lfloor\frac{n}{2}\right\rfloor$ for $n\geq 4$ or $p\geq 1$ for $n=2,3$. Assume that $f_k\in W^{1,p}$ is a sequence of homeomorphisms such that $f_k\rightharpoonup f$ weakly in $W^{1,p}$ and assume that $J_f>0$ a.e. Then we show that $f$ is injective a.e.

math.FA↗

arXiv2024

On the Identifiability from Modulo Measurements under DFT Sensing Matrix

Modulo sampling (MS) has been recently introduced to enhance the dynamic range of conventional ADCs by applying a modulo operator before sampling. This paper examines the identifiability of a measurement model where measurements are taken using a discrete Fourier transform (DFT) sensing matrix, followed by a modulo operator (modulo-DFT). Firstly, we derive a necessary and sufficient condition for the unique identification of the modulo-DFT sensing model based on the number of measurements and the indices of zero elements in the original signal. Then, we conduct a deeper analysis of three specific cases: when the number of measurements is a power of $2$, a prime number, and twice a prime number. Additionally, we investigate the identifiability of periodic bandlimited (PBL) signals under MS, which can be considered as the modulo-DFT sensing model with additional symmetric and conjugate constraints on the original signal. We also provide a necessary and sufficient condition based solely on the number of samples in one period for the unique identification of the PBL signal under MS, though with an ambiguity in the direct current (DC) component. Furthermore, we show that when the oversampling factor exceeds $3(1+1/P)$, the PBL signal can be uniquely identified with an ambiguity in the DC component, where $P$ is the number of harmonics, including the fundamental component, in the positive frequency part. Finally, we also present a recovery algorithm that estimates the original signal by solving integer linear equations, and we conduct simulations to validate our conclusions.

cs.IT↗

arXiv2024

Subharmonic oscillations in the Floquet circuit with the frequency-synthesis dimension

The period-doubling oscillation emerges with the coexistence between zero and π modes in Floquet topological insulator. Here, utilized the flexibility of the circuit, we construct the Floquet circuit with frequency-synthetic dimension and find the topological-protected deeply-subharmonic oscillations with the period extensively exceeding the doubling-driven period. In the construction framework, the periodically-driven mechanism is attained by implementing the circuit-oscillator hierarchy with the stepping-variation resonances in frequency domain. The zero and π modes that arise at the Floquet band in the circuit indicate the anomalous boundary-bulk correspondence. The coexistence of zero and π modes, results in a subharmonic oscillation with the extremely-low frequency on the edge of the Floquet circuit. Furthermore, we explore the Floquet band with the enhanced periodically-driven strength tailored by the component flexibility of the circuit. Our method provides a flexible scheme to study Floquet topological phases, and open a new path for realizing the deeply subwavelength system.

cond-mat.mes-hall↗

arXiv2024

Mean Field Study of Superconductivity in the Square Lattice $t$-$J$ Model with Three-Site Hopping

It remains an open question whether the two-dimensional single-band pure Hubbard model and its related pure $t$-$J$ model truly capture the superconducting order in cuprates. Recent numerical studies on this issue have raised a notable disparity in superconducting order between the pure Hubbard model and the pure $t$-$J$ model. Inspired by these, we investigate the role of the three-site hopping term in $d$-wave superconductivity, such a term is usually neglected in the effective Hamiltonian of the Hubbard model, though its amplitude is of the same order as the superexchange coupling $J$ in the $t$-$J$ model. Our slave-boson mean-field solution demonstrates the suppression of $d$-wave superconducting order by incorporating the three-site hopping term, consistent with numerical observations by the density matrix renormalization group. This suppression could be understood as a result of competition between superexchange interaction and three-site hopping, the former favors $d$-wave pairing while the latter favors $s$-wave pairing. We also discussed its role in quasiparticle dispersion and boson-condensation temperature. Our findings may offer an alternative understanding of the recent numerical contrasting findings in the strong coupling regime: the absent or weak superconductivity in the pure Hubbard model, while the robust superconductivity in the $t$-$J$ model without including the three-site hopping term.

cond-mat.str-el↗

arXiv2024

OpenPSG: Open-set Panoptic Scene Graph Generation via Large Multimodal Models

Panoptic Scene Graph Generation (PSG) aims to segment objects and recognize their relations, enabling the structured understanding of an image. Previous methods focus on predicting predefined object and relation categories, hence limiting their applications in the open world scenarios. With the rapid development of large multimodal models (LMMs), significant progress has been made in open-set object detection and segmentation, yet open-set relation prediction in PSG remains unexplored. In this paper, we focus on the task of open-set relation prediction integrated with a pretrained open-set panoptic segmentation model to achieve true open-set panoptic scene graph generation (OpenPSG). Our OpenPSG leverages LMMs to achieve open-set relation prediction in an autoregressive manner. We introduce a relation query transformer to efficiently extract visual features of object pairs and estimate the existence of relations between them. The latter can enhance the prediction efficiency by filtering irrelevant pairs. Finally, we design the generation and judgement instructions to perform open-set relation prediction in PSG autoregressively. To our knowledge, we are the first to propose the open-set PSG task. Extensive experiments demonstrate that our method achieves state-of-the-art performance in open-set relation prediction and panoptic scene graph generation. Code is available at \url{https://github.com/franciszzj/OpenPSG}.

cs.CV↗

arXiv2024

Local Sobolev-Poincare imbedding domains

In this article, we study local Sobolev-Poincaré imbedding domains. The main result reads as below. \begin{enumerate} \item for $1\leq p\leq n$, a bounded uniform domain is also a local Sobolev-Poincaré imbedding domain of order $p$; conversely a local Sobolev-Poincaré imbedding domain of order $p$ is locally linearly connected $(LLC)$. A uniform domain is $(LLC)$. Conversely, with some very weak connecting assumption, a $(LLC)$ domain is uniform. \item for $n<p<\fz$, a bounded domain is a local Sobolev-Poincaré imbedding domain of order $p$ if and only if it is an $α$-cigar domain for $α=(p-n)/(p-1)$. Hence, a domain is a local Sobolev-Poincaré imbedding domain of oder $p$ if and only if it is a (global) Sobolev-Poincaré imbedding domain. \end{enumerate}

math.FA↗

arXiv2024

The SkatingVerse Workshop & Challenge: Methods and Results

The SkatingVerse Workshop & Challenge aims to encourage research in developing novel and accurate methods for human action understanding. The SkatingVerse dataset used for the SkatingVerse Challenge has been publicly released. There are two subsets in the dataset, i.e., the training subset and testing subset. The training subsets consists of 19,993 RGB video sequences, and the testing subsets consists of 8,586 RGB video sequences. Around 10 participating teams from the globe competed in the SkatingVerse Challenge. In this paper, we provide a brief summary of the SkatingVerse Workshop & Challenge including brief introductions to the top three methods. The submission leaderboard will be reopened for researchers that are interested in the human action understanding challenge. The benchmark dataset and other information can be found at: https://skatingverse.github.io/.

cs.CV↗

arXiv2024

Phase Diagram of the Square-Lattice $t$-$J$-$V$ Model for Electron-Doped Cuprates

Motivated by significant discrepancies between experimental observations of electron-doped cuprates and numerical results of the Hubbard and $t$-$J$ models, we investigate the role of inter-site interactions $V$ by studying the $t$-$J$-$V$ model on square lattices. Based on large-scale density matrix renormalization group simulations, we identify the ground-state phase diagram across varying inter-site interactions $V$ and doping concentration $δ$. We find that the phase diagram with finite inter-site interactions $2\lesssim V/J\lesssim3$ offers a more accurate description of electron-doped cuprates than the conventional Hubbard and $t$-$J$ models. Moreover, we reveal the role of inter-site interactions $V$ at varying doping levels: at light doping, inter-site interactions favor Néel antiferromagnetic order, and suppress both superconductivity and charge density wave; around optimal doping, these interactions support a pseudogap-like phase while suppressing superconductivity, and we further perform the slave boson mean-field analysis to understand the numerical results microscopically; at higher doping, the effects of inter-site interactions become insignificant, with our numerical predictions suggesting the emergence of incommensurate spin density wave phase. Our specific focus around optimal doping with various inter-site interactions identifies successive phases including phase separation, uniform $d$-wave SC and a pseudogap-like phase, and reveals a relative insensitivity of charge density wave to superconductivity. Our study suggests the $t$-$J$-$V$ model as the minimal model to capture the essential physics of the electron-doped cuprates.

cond-mat.str-el↗