arXiv subjects

Ziyang Liu

Publications and source records attributed to Ziyang Liu.

At least 55 records · Page 3Linked to original sources

arXiv2025

Rapid MRI-Based Synthetic CT Simulations for Precise tFUS Targeting

Accurate targeting is critical for the effectiveness of transcranial focused ultrasound (tFUS) neuromodulation. While CT provides accurate skull acoustic properties, its ionizing radiation and poor soft tissue contrast limit clinical applicability. In contrast, MRI offers superior neuroanatomical visualization without radiation exposure but lacks skull property mapping. This study proposes a novel, fully CT free simulation framework that integrates MRI-derived synthetic CT (sCT) with efficient modeling techniques for rapid and precise tFUS targeting. We trained a deep-learning model to generate sCT from T1-weighted MRI and integrated it with both full-wave (k-Wave) and accelerated simulation methods, hybrid angular spectrum (kWASM) and Rayleigh-Sommerfeld ASM (RSASM). Across five skull models, both full-wave and hybrid pipelines using sCT demonstrated sub-millimeter targeting deviation, focal shape consistency (FWHM ~3.3-3.8 mm), and <0.2 normalized pressure error compared to CT-based gold standard. Notably, the kW-ASM and RS-ASM pipelines reduced simulation time from ~3320 s to 187 s and 34 s respectively, achieving ~94% and ~90% time savings. These results confirm that MRI-derived sCT combined with innovative rapid simulation techniques enables fast, accurate, and radiation-free tFUS planning, supporting its feasibility for scalable clinical applications.

physics.med-ph↗

arXiv2025

Anti-aliasing Algorithm Based on Three-dimensional Display Image

3D-display technology has been a promising emerging area with potential to be the core of next-generation display technology. When directly observing unprocessed images and text through a naked-eye 3D display device, severe distortion and jaggedness will be displayed, which will make the display effect much worse. In this work, we try to settle down such degradation with spatial and frequency processing, furthermore, we make efforts to extract degenerate function of columnar lens array thus fundamentally eliminating degradation.

eess.IV↗

arXiv2025

Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal Learning

Modern robot navigation systems encounter difficulties in diverse and complex indoor environments. Traditional approaches rely on multiple modules with small models or rule-based systems and thus lack adaptability to new environments. To address this, we developed Astra, a comprehensive dual-model architecture, Astra-Global and Astra-Local, for mobile robot navigation. Astra-Global, a multimodal LLM, processes vision and language inputs to perform self and goal localization using a hybrid topological-semantic graph as the global map, and outperforms traditional visual place recognition methods. Astra-Local, a multitask network, handles local path planning and odometry estimation. Its 4D spatial-temporal encoder, trained through self-supervised learning, generates robust 4D features for downstream tasks. The planning head utilizes flow matching and a novel masked ESDF loss to minimize collision risks for generating local trajectories, and the odometry head integrates multi-sensor inputs via a transformer encoder to predict the relative pose of the robot. Deployed on real in-house mobile robots, Astra achieves high end-to-end mission success rate across diverse indoor environments.

cs.RO↗

arXiv2025

MCCD: Multi-Agent Collaboration-based Compositional Diffusion for Complex Text-to-Image Generation

Diffusion models have shown excellent performance in text-to-image generation. Nevertheless, existing methods often suffer from performance bottlenecks when handling complex prompts that involve multiple objects, characteristics, and relations. Therefore, we propose a Multi-agent Collaboration-based Compositional Diffusion (MCCD) for text-to-image generation for complex scenes. Specifically, we design a multi-agent collaboration-based scene parsing module that generates an agent system comprising multiple agents with distinct tasks, utilizing MLLMs to extract various scene elements effectively. In addition, Hierarchical Compositional diffusion utilizes a Gaussian mask and filtering to refine bounding box regions and enhance objects through region enhancement, resulting in the accurate and high-fidelity generation of complex scenes. Comprehensive experiments demonstrate that our MCCD significantly improves the performance of the baseline models in a training-free manner, providing a substantial advantage in complex scene generation.

cs.CV↗

arXiv2025

Establishment and Solution of a Multi-Stage Decision Model Based on Hypothesis Testing and Dynamic Programming Algorithm

This paper introduces a novel multi-stage decision-making model that integrates hypothesis testing and dynamic programming algorithms to address complex decision-making scenarios.Initially,we develop a sampling inspection scheme that controls for both Type I and Type II errors using a simple random sampling method without replacement,ensuring the randomness and representativeness of the sample while minimizing selection bias.Through the application of hypothesis testing theory,a hypothesis testing model concerning the defect rate is established,and formulas for the approximate distribution of the sample defect rate and the minimum sample size required under two different scenarios are derived. Subsequently,a multi-stage dynamic programming decision model is constructed.This involves defining the state transition functions and stage-specific objective functions,followed by obtaining six optimal decision strategies under various conditions through backward recursion.The results demonstrate the model's potent capability for multi-stage decision-making and its high interpretability,offering significant advantages in practical applications.

eess.SY↗

arXiv2025

Naked Eye Three-dimensional Display System Based on Time-multiplexed Technology

Our group is developing a multi-user eye-tracked 3D display, an evolution of the single-user eye-tracked 3D display that we have already successfully developed. This display utilizes a slanted lenticular setup, where multiple perspective views are shown across the viewing field. Due to the constraints of the lenticular lens parameters, identical views are repeated across the field, limiting eye tracking to a single user. However, this limitation can be addressed using spatio-temporal multiplexing, where view zone groups are presented sequentially with a high frame rate liquid crystal display (LCD) and driver, in combination with a synchronized directional light emitting diode (LED) array. In this paper, we describe the operation and results of the backlight drive electronics, where a prototype using a white LED illumination matrix, a simplified LCD panel, and a linear Fresnel lens array serves as a test bed.

eess.SY↗

arXiv2025

A generative approach for lensless imaging in low-light conditions

Lensless imaging offers a lightweight, compact alternative to traditional lens-based systems, ideal for exploration in space-constrained environments. However, the absence of a focusing lens and limited lighting in such environments often result in low-light conditions, where the measurements suffer from complex noise interference due to insufficient capture of photons. This study presents a robust reconstruction method for high-quality imaging in low-light scenarios, employing two complementary perspectives: model-driven and data-driven. First, we apply a physic-model-driven perspective to reconstruct in the range space of the pseudo-inverse of the measurement model as a first guidance to extract information in the noisy measurements. Then, we integrate a generative-model based perspective to suppress residual noises as the second guidance to suppress noises in the initial noisy results. Specifically, a learnable Wiener filter-based module generates an initial noisy reconstruction. Then, for fast and, more importantly, stable generation of the clear image from the noisy version, we implement a modified conditional generative diffusion module. This module converts the raw image into the latent wavelet domain for efficiency and uses modified bidirectional training processes for stabilization. Simulations and real-world experiments demonstrate substantial improvements in overall visual quality, advancing lensless imaging in challenging low-light environments.

eess.IV↗

arXiv2024

MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understanding

Large Vision-Language Models (LVLMs) have achieved remarkable performance in many vision-language tasks, yet their capabilities in fine-grained visual understanding remain insufficiently evaluated. Existing benchmarks either contain limited fine-grained evaluation samples that are mixed with other data, or are confined to object-level assessments in natural images. To holistically assess LVLMs' fine-grained visual understanding capabilities, we propose using document images with multi-granularity and multi-modal information to supplement natural images. In this light, we construct MMDocBench, a benchmark with various OCR-free document understanding tasks for the evaluation of fine-grained visual perception and reasoning abilities. MMDocBench defines 15 main tasks with 4,338 QA pairs and 11,353 supporting regions, covering various document images such as research papers, receipts, financial reports, Wikipedia tables, charts, and infographics. Based on MMDocBench, we conduct extensive experiments using 13 open-source and 3 proprietary advanced LVLMs, assessing their strengths and weaknesses across different tasks and document image types. The benchmark, task instructions, and evaluation code will be made publicly available.

cs.CV↗

arXiv2024

TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Data

In this work, we address question answering (QA) over a hybrid of tabular and textual data that are very common content on the Web (e.g. SEC filings), where discrete reasoning capabilities are often required. Recently, large language models (LLMs) like GPT-4 have demonstrated strong multi-step reasoning capabilities. We then consider harnessing the amazing power of LLMs to solve our task. We abstract a Step-wise Pipeline for tabular and textual QA, which consists of three key steps, including Extractor, Reasoner and Executor, and initially design an instruction to instantiate the pipeline and validate that GPT-4 outperforms all existing methods. However, utilizing an online LLM like GPT-4 holds various challenges in terms of cost, latency, and data security risk, which motivates us to specialize smaller LLMs in this task. We develop a TAT-LLM language model by fine-tuning LLaMA 2 with the training data generated automatically from existing expert-annotated datasets following the Step-wise Pipeline. The experimental results have verified that our TAT-LLM model can outperform all baseline models, including the previous best fine-tuned models and very large-scale LLMs like GPT-4 on FinQA, TAT-QA and TAT-DQA benchmarks.

cs.CL↗

arXiv2024

Neumann Series-based Neural Operator for Solving Inverse Medium Problem

The inverse medium problem, inherently ill-posed and nonlinear, presents significant computational challenges. This study introduces a novel approach by integrating a Neumann series structure within a neural network framework to effectively handle multiparameter inputs. Experiments demonstrate that our methodology not only accelerates computations but also significantly enhances generalization performance, even with varying scattering properties and noisy data. The robustness and adaptability of our framework provide crucial insights and methodologies, extending its applicability to a broad spectrum of scattering problems. These advancements mark a significant step forward in the field, offering a scalable solution to traditionally complex inverse problems.

math-ph↗

arXiv2024

NSNO: Neumann Series Neural Operator for Solving Helmholtz Equations in Inhomogeneous Medium

In this paper, we propose Neumann Series Neural Operator (NSNO) to learn the solution operator of Helmholtz equation from inhomogeneity coefficients and source terms to solutions. Helmholtz equation is a crucial partial differential equation (PDE) with applications in various scientific and engineering fields. However, efficient solver of Helmholtz equation is still a big challenge especially in the case of high wavenumber. Recently, deep learning has shown great potential in solving PDEs especially in learning solution operators. Inspired by Neumann series in Helmholtz equation, we design a novel network architecture in which U-Net is embedded inside to capture the multiscale feature. Extensive experiments show that the proposed NSNO significantly outperforms the state-of-the-art FNO with at least 60\% lower relative $L^2$-error, especially in the large wavenumber case, and has 50\% lower computational cost and less data requirement. Moreover, NSNO can be used as the surrogate model in inverse scattering problems. Numerical tests show that NSNO is able to give comparable results with traditional finite difference forward solver while the computational cost is reduced tremendously.

math.NA↗

arXiv2023

Graph Contrastive Learning with Generative Adversarial Network

Graph Neural Networks (GNNs) have demonstrated promising results on exploiting node representations for many downstream tasks through supervised end-to-end training. To deal with the widespread label scarcity issue in real-world applications, Graph Contrastive Learning (GCL) is leveraged to train GNNs with limited or even no labels by maximizing the mutual information between nodes in its augmented views generated from the original graph. However, the distribution of graphs remains unconsidered in view generation, resulting in the ignorance of unseen edges in most existing literature, which is empirically shown to be able to improve GCL's performance in our experiments. To this end, we propose to incorporate graph generative adversarial networks (GANs) to learn the distribution of views for GCL, in order to i) automatically capture the characteristic of graphs for augmentations, and ii) jointly train the graph GAN model and the GCL model. Specifically, we present GACN, a novel Generative Adversarial Contrastive learning Network for graph representation learning. GACN develops a view generator and a view discriminator to generate augmented views automatically in an adversarial style. Then, GACN leverages these views to train a GNN encoder with two carefully designed self-supervised learning losses, including the graph contrastive loss and the Bayesian personalized ranking Loss. Furthermore, we design an optimization framework to train all GACN modules jointly. Extensive experiments on seven real-world datasets show that GACN is able to generate high-quality augmented views for GCL and is superior to twelve state-of-the-art baseline methods. Noticeably, our proposed GACN surprisingly discovers that the generated views in data augmentation finally conform to the well-known preferential attachment rule in online networks.

cs.LG↗

arXiv2023

Deep learning-based reduced order model for three-dimensional unsteady flow using mesh transformation and stitching

Artificial intelligence-based three-dimensional(3D) fluid modeling has gained significant attention in recent years. However, the accuracy of such models is often limited by the processing of irregular flow data. In order to bolster the credibility of near-wall flow prediction, this paper presents a deep learning-based reduced order model for three-dimensional unsteady flow using the transformation and stitching of multi-block structured meshes. To begin with, full-order flow data is provided by numerical simulations that rely on multi-block structured meshes. A mesh transformation technique is applied to convert each structured grid with data into a corresponding uniform and orthogonal grid, which is subsequently stitched and filled. The resulting snapshots in the transformed domain contain accurate flow information for multiple meshes and can be directly fed into a structured neural network without requiring any interpolation operation. Subsequently, a network model based on a fully convolutional neural network is constructed to predict flow dynamics accurately. To validate the strategy's feasibility, the flow around a sphere with Re=300 was investigated, and the results obtained using traditional Cartesian interpolation were used as the baseline for comparison. All the results demonstrate the preservation and accurate prediction of flow details near the wall, with the pressure correlation coefficient on the wall achieving an impressive value of 0.9985. Moreover, the periodic behavior of flow fields can be faithfully predicted during long-term inference.

physics.flu-dyn↗

arXiv2023

PANE-GNN: Unifying Positive and Negative Edges in Graph Neural Networks for Recommendation

Recommender systems play a crucial role in addressing the issue of information overload by delivering personalized recommendations to users. In recent years, there has been a growing interest in leveraging graph neural networks (GNNs) for recommender systems, capitalizing on advancements in graph representation learning. These GNN-based models primarily focus on analyzing users' positive feedback while overlooking the valuable insights provided by their negative feedback. In this paper, we propose PANE-GNN, an innovative recommendation model that unifies Positive And Negative Edges in Graph Neural Networks for recommendation. By incorporating user preferences and dispreferences, our approach enhances the capability of recommender systems to offer personalized suggestions. PANE-GNN first partitions the raw rating graph into two distinct bipartite graphs based on positive and negative feedback. Subsequently, we employ two separate embeddings, the interest embedding and the disinterest embedding, to capture users' likes and dislikes, respectively. To facilitate effective information propagation, we design distinct message-passing mechanisms for positive and negative feedback. Furthermore, we introduce a distortion to the negative graph, which exclusively consists of negative feedback edges, for contrastive training. This distortion plays a crucial role in effectively denoising the negative feedback. The experimental results provide compelling evidence that PANE-GNN surpasses the existing state-of-the-art benchmark methods across four real-world datasets. These datasets include three commonly used recommender system datasets and one open-source short video recommendation dataset.

cs.IR↗

arXiv2022

Knowledge Distillation based Contextual Relevance Matching for E-commerce Product Search

Online relevance matching is an essential task of e-commerce product search to boost the utility of search engines and ensure a smooth user experience. Previous work adopts either classical relevance matching models or Transformer-style models to address it. However, they ignore the inherent bipartite graph structures that are ubiquitous in e-commerce product search logs and are too inefficient to deploy online. In this paper, we design an efficient knowledge distillation framework for e-commerce relevance matching to integrate the respective advantages of Transformer-style models and classical relevance matching models. Especially for the core student model of the framework, we propose a novel method using $k$-order relevance modeling. The experimental results on large-scale real-world data (the size is 6$\sim$174 million) show that the proposed method significantly improves the prediction accuracy in terms of human relevance judgment. We deploy our method to the anonymous online search platform. The A/B testing results show that our method significantly improves 5.7% of UV-value under price sort mode.

cs.IR↗

arXiv2022

TeKo: Text-Rich Graph Neural Networks with External Knowledge

Graph Neural Networks (GNNs) have gained great popularity in tackling various analytical tasks on graph-structured data (i.e., networks). Typical GNNs and their variants follow a message-passing manner that obtains network representations by the feature propagation process along network topology, which however ignore the rich textual semantics (e.g., local word-sequence) that exist in many real-world networks. Existing methods for text-rich networks integrate textual semantics by mainly utilizing internal information such as topics or phrases/words, which often suffer from an inability to comprehensively mine the text semantics, limiting the reciprocal guidance between network structure and text semantics. To address these problems, we propose a novel text-rich graph neural network with external knowledge (TeKo), in order to take full advantage of both structural and textual information within text-rich networks. Specifically, we first present a flexible heterogeneous semantic network that incorporates high-quality entities and interactions among documents and entities. We then introduce two types of external knowledge, that is, structured triplets and unstructured entity description, to gain a deeper insight into textual semantics. We further design a reciprocal convolutional mechanism for the constructed heterogeneous semantic network, enabling network structure and textual semantics to collaboratively enhance each other and learn high-level network representations. Extensive experimental results on four public text-rich networks as well as a large-scale e-commerce searching dataset illustrate the superior performance of TeKo over state-of-the-art baselines.

cs.SI↗

arXiv2022

RDU: A Region-based Approach to Form-style Document Understanding

Key Information Extraction (KIE) is aimed at extracting structured information (e.g. key-value pairs) from form-style documents (e.g. invoices), which makes an important step towards intelligent document understanding. Previous approaches generally tackle KIE by sequence tagging, which faces difficulty to process non-flatten sequences, especially for table-text mixed documents. These approaches also suffer from the trouble of pre-defining a fixed set of labels for each type of documents, as well as the label imbalance issue. In this work, we assume Optical Character Recognition (OCR) has been applied to input documents, and reformulate the KIE task as a region prediction problem in the two-dimensional (2D) space given a target field. Following this new setup, we develop a new KIE model named Region-based Document Understanding (RDU) that takes as input the text content and corresponding coordinates of a document, and tries to predict the result by localizing a bounding-box-like region. Our RDU first applies a layout-aware BERT equipped with a soft layout attention masking and bias mechanism to incorporate layout information into the representations. Then, a list of candidate regions is generated from the representations via a Region Proposal Module inspired by computer vision models widely applied for object detection. Finally, a Region Categorization Module and a Region Selection Module are adopted to judge whether a proposed region is valid and select the one with the largest probability from all proposed regions respectively. Experiments on four types of form-style documents show that our proposed method can achieve impressive results. In addition, our RDU model can be trained with different document types seamlessly, which is especially helpful over low-resource documents.

cs.AI↗

arXiv2021

DSRGAN: Detail Prior-Assisted Perceptual Single Image Super-Resolution via Generative Adversarial Networks

The generative adversarial network (GAN) is successfully applied to study the perceptual single image superresolution (SISR). However, the GAN often tends to generate images with high frequency details being inconsistent with the real ones. Inspired by conventional detail enhancement algorithms, we propose a novel prior knowledge, the detail prior, to assist the GAN in alleviating this problem and restoring more realistic details. The proposed method, named DSRGAN, includes a well designed detail extraction algorithm to capture the most important high frequency information from images. Then, two discriminators are utilized for supervision on image-domain and detail-domain restorations, respectively. The DSRGAN merges the restored detail into the final output via a detail enhancement manner. The special design of DSRGAN takes advantages from both the model-based conventional algorithm and the data-driven deep learning network. Experimental results demonstrate that the DSRGAN outperforms the state-of-the-art SISR methods on perceptual metrics and achieves comparable results in terms of fidelity metrics simultaneously. Following the DSRGAN, it is feasible to incorporate other conventional image processing algorithms into a deep learning network to form a model-based deep SISR.

eess.IV↗