arXiv subjects

Jie Zhang

Publications and source records attributed to Jie Zhang.

At least 613 records · Page 34Linked to original sources

arXiv2024

LearnedFTL: A Learning-Based Page-Level FTL for Reducing Double Reads in Flash-Based SSDs

We present LearnedFTL, a new on-demand page-level flash translation layer (FTL) design, which employs learned indexes to improve the address translation efficiency of flash-based SSDs. The first of its kind, it reduces the number of double reads induced by address translation in random read accesses. LearnedFTL proposes three key techniques: an in-place-update linear model to build learned indexes efficiently, a virtual PPN representation to obtain contiguous PPNs for sorted LPNs, and a group-based allocation and model training via GC/rewrite strategy to reduce the training overhead. By tightly integrating the aforementioned key techniques, LearnedFTL considerably speeds up address translation while reducing the number of flash read accesses caused by the address translation. Our extensive experiments on a FEMU-based prototype show that LearnedFTL can reduce up to 55.5\% address translation-induced double reads. As a result, LearnedFTL reduces the P99 tail latency by 2.9$\times$ $\sim$ 12.2$\times$ with an average of 5.5$\times$ and 8.2$\times$ compared to the state-of-the-art TPFTL and LeaFTL schemes, respectively.

cs.AR↗

arXiv2024

ICDM 2020 Knowledge Graph Contest: Consumer Event-Cause Extraction

Consumer Event-Cause Extraction, the task aimed at extracting the potential causes behind certain events in the text, has gained much attention in recent years due to its wide applications. The ICDM 2020 conference sets up an evaluation competition that aims to extract events and the causes of the extracted events with a specified subject (a brand or product). In this task, we mainly focus on how to construct an end-to-end model, and extract multiple event types and event-causes simultaneously. To this end, we introduce a fresh perspective to revisit the relational event-cause extraction task and propose a novel sequence tagging framework, instead of extracting event types and events-causes separately. Experiments show our framework outperforms baseline methods even when its encoder module uses an initialized pre-trained BERT encoder, showing the power of the new tagging framework. In this competition, our team achieved 1st place in the first stage leaderboard, and 3rd place in the final stage leaderboard.

cs.CL↗

arXiv2024

PM2D: A parallel GPU-based code for the kinetic simulation of laser plasma instabilities in large scale plasmas

Laser plasma instabilities (LPIs) have significant influences on the laser energy deposition efficiency, hot electron generation, and uniformity of irradiation in inertial confined fusion (ICF). In contrast to theoretical analysis of linear development of LPIs, numerical simulations play a more and more important role in revealing the complex physics of LPIs. Since LPIs are typically a three-wave coupling process, the precise kinetic simulation of LPIs requires to resolve the laser period (around one femtosecond) and laser wavelength (less than one micron). In this paper, a full wave fluid model of LPIs is constructed and numerically solved by the particle-mesh method, where the plasma is described by macro particles that can move across the mesh grids freely. Based upon this model, a two-dimensional (2D) GPU code named PM2D is developed. It can simulate the kinetic effects of LPIs self-consistently as normal particle-in-cell (PIC) codes. Moreover, as the physical model adopted in the PM2D code is specifically constructed for LPIs, the required macro particles per grid in the simulations can be largely reduced and thus overall simulation cost is considerably reduced comparing with typical PIC codes. Moreover, the numerical noise in our PM2D code is much lower, which makes it more robust than PIC codes in the simulation of LPIs for the long-time scale above 10 picoseconds. After the distributed computing is realized, our PM2D code is able to run on GPU clusters with a total mesh grids up to several billions, which meets the typical requirements for the simulations of LPIs at ICF experimental scale with reasonable cost.

physics.plasm-ph↗

arXiv2024

A Platform for All-optical Thomson/ Compton Scattering with Versatile Parameters

A dual-beam platform for all-optical electron-photon scattering, or Thomson/Compton scattering, with adjustable collision-angle and parameter tuning ability has been developed, which, in principle, can be used for the verification of strong-field quantum electrodynamics effects. Combining this platform with a 200 TW Ti:Sapphire laser system, we demonstrated the generation of inverse Compton scattering X/gamma-rays with tunable energies from tens of keV to MeV. The polarization of X/gamma radiation was manipulated by controlling the polarization of scattering laser. In the near future, by combining this experimental platform with multi-PW laser facilities, it is proposed to experimentally generate X/gamma radiation with orbital angular momentum for the nuclear isomer excitation, and more importantly, to explore the regime transition from nonlinear Thomson scattering to nonlinear Compton scattering, eventually to demonstrate the verification of theories on extremely strong field quantum electrodynamics effects.

hep-ex↗

arXiv2024

Toward stochastic neural computing

The highly irregular spiking activity of cortical neurons and behavioral variability suggest that the brain could operate in a fundamentally probabilistic way. Mimicking how the brain implements and learns probabilistic computation could be a key to developing machine intelligence that can think more like humans. In this work, we propose a theory of stochastic neural computing (SNC) in which streams of noisy inputs are transformed and processed through populations of nonlinearly coupled spiking neurons. To account for the propagation of correlated neural variability, we derive from first principles a moment embedding for spiking neural network (SNN). This leads to a new class of deep learning model called the moment neural network (MNN) which naturally generalizes rate-based neural networks to second order. As the MNN faithfully captures the stationary statistics of spiking neural activity, it can serve as a powerful proxy for training SNN with zero free parameters. Through joint manipulation of mean firing rate and noise correlations in a task-driven way, the model is able to learn inference tasks while simultaneously minimizing prediction uncertainty, resulting in enhanced inference speed. We further demonstrate the application of our method to Intel's Loihi neuromorphic hardware. The proposed theory of SNC may open up new opportunities for developing machine intelligence capable of computing uncertainty and for designing unconventional computing architectures.

cs.NE↗

arXiv2024

Facility Location Problems with Capacity Constraints: Two Facilities and Beyond

In this paper, we investigate the Mechanism Design aspects of the $m$-Capacitated Facility Location Problem ($m$-CFLP) on a line. We focus on two frameworks. In the first framework, the number of facilities is arbitrary, all facilities have the same capacity, and the number of agents is equal to the total capacity of all facilities. In the second framework, we aim to place two facilities, each with a capacity of at least half of the total agents. For both of these frameworks, we propose truthful mechanisms with bounded approximation ratios with respect to the Social Cost (SC) and the Maximum Cost (MC). When $m>2$, the result sharply contrasts with the impossibility results known for the classic $m$-Facility Location Problem \cite{fotakis2014power}, where capacity constraints are not considered. Furthermore, all our mechanisms are (i) optimal with respect to the MC (ii) optimal or nearly optimal with respect to the SC among anonymous mechanisms. For both frameworks, we provide a lower bound on the approximation ratio that any truthful and deterministic mechanism can achieve with respect to the SC and MC.

cs.GT↗

arXiv2024

Distribution Network Restoration: Resource Scheduling Considering Coupled Transportation-Power Networks

Optimal decision-making is key to efficient allocation and scheduling of repair resources (e.g., crews) to service affected nodes of large power grid networks. Traditional manual restoration methods are inadequate for modern smart grids sprawling across vast territories, compounded by the unpredictable nature of damage and disruptions in power and transportation networks. This paper develops a method that focuses on the restoration and repair efforts within power systems. We expand upon the methodology proposed in the literature and incorporate a real-world transportation network to enhance the realism and practicality of repair schedules. Our approach carefully devises a reduced network that combines vulnerable components from the distribution network with the real transportation network. Key contributions include dynamically addressing a coupled resource allocation and capacitated vehicle routing problem over a new reduced network model, constructed by integrating the power grid with the transportation network. This is performed using network heuristics and graph theory to prioritize securing critical grid segments. A case study is presented for the 8500 bus system.

math.OC↗

arXiv2024

Re2LLM: Reflective Reinforcement Large Language Model for Session-based Recommendation

Large Language Models (LLMs) are emerging as promising approaches to enhance session-based recommendation (SBR), where both prompt-based and fine-tuning-based methods have been widely investigated to align LLMs with SBR. However, the former methods struggle with optimal prompts to elicit the correct reasoning of LLMs due to the lack of task-specific feedback, leading to unsatisfactory recommendations. Although the latter methods attempt to fine-tune LLMs with domain-specific knowledge, they face limitations such as high computational costs and reliance on open-source backbones. To address such issues, we propose a Reflective Reinforcement Large Language Model (Re2LLM) for SBR, guiding LLMs to focus on specialized knowledge essential for more accurate recommendations effectively and efficiently. In particular, we first design the Reflective Exploration Module to effectively extract knowledge that is readily understandable and digestible by LLMs. To be specific, we direct LLMs to examine recommendation errors through self-reflection and construct a knowledge base (KB) comprising hints capable of rectifying these errors. To efficiently elicit the correct reasoning of LLMs, we further devise the Reinforcement Utilization Module to train a lightweight retrieval agent. It learns to select hints from the constructed KB based on the task-specific feedback, where the hints can serve as guidance to help correct LLMs reasoning for better recommendations. Extensive experiments on multiple real-world datasets demonstrate that our method consistently outperforms state-of-the-art methods.

cs.AI↗

arXiv2024

Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities

Past research has examined how well these models grasp code syntax, yet their understanding of code semantics still needs to be explored. We extensively analyze seven code models to investigate how code models represent code syntax and semantics. This includes four prominent code pre-trained models (CodeBERT, GraphCodeBERT, CodeT5, and UnixCoder) and three large language models (StarCoder, CodeLlama, and CodeT5+). We have developed four probing tasks to evaluate the models' abilities to learn code syntax and semantics. These tasks focus on reconstructing code syntax and semantic structures-such as AST, CFG, CDG, and DDG - within the models' representation spaces. These structures are fundamental to understanding code. Additionally, we explore the role of syntax tokens in each token representation and the extended dependencies among code tokens. Furthermore, we examine the distribution of attention weights concerning code semantic structures. Through detailed analysis, our results emphasize the strengths and weaknesses of various code models in mastering code syntax and semantics. The findings reveal that these models are proficient in grasping code syntax, effectively capturing the relationships and roles of syntax tokens. However, their ability to encode code semantics shows more variability. This study enriches our understanding of the capabilities of code models in analyzing syntax and semantics. Our findings offer valuable insights for future code model enhancements, helping optimize their application across a range of code-related tasks.

cs.SE↗

arXiv2024

Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness

Large-scale pre-trained vision-language models like CLIP have demonstrated impressive performance across various tasks, and exhibit remarkable zero-shot generalization capability, while they are also vulnerable to imperceptible adversarial examples. Existing works typically employ adversarial training (fine-tuning) as a defense method against adversarial examples. However, direct application to the CLIP model may result in overfitting, compromising the model's capacity for generalization. In this paper, we propose Pre-trained Model Guided Adversarial Fine-Tuning (PMG-AFT) method, which leverages supervision from the original pre-trained model by carefully designing an auxiliary branch, to enhance the model's zero-shot adversarial robustness. Specifically, PMG-AFT minimizes the distance between the features of adversarial examples in the target model and those in the pre-trained model, aiming to preserve the generalization features already captured by the pre-trained model. Extensive Experiments on 15 zero-shot datasets demonstrate that PMG-AFT significantly outperforms the state-of-the-art method, improving the top-1 robust accuracy by an average of 4.99%. Furthermore, our approach consistently improves clean accuracy by an average of 8.72%. Our code is available at https://github.com/serendipity1122/Pre-trained-Model-Guided-Fine-Tuning-for-Zero-Shot-Adversarial-Robustness.

cs.CV↗

arXiv2024

High Noise Scheduling is a Must

Consistency models possess high capabilities for image generation, advancing sampling steps to a single step through their advanced techniques. Current advancements move one step forward consistency training techniques and eliminates the limitation of distillation training. Even though the proposed curriculum and noise scheduling in improved training techniques yield better results than basic consistency models, it lacks well balanced noise distribution and its consistency between curriculum. In this study, it is investigated the balance between high and low noise levels in noise distribution and offered polynomial noise distribution to maintain the stability. This proposed polynomial noise distribution is also supported with a predefined Karras noises to prevent unique noise levels arises with Karras noise generation algorithm. Furthermore, by elimination of learned noisy steps with a curriculum based on sinusoidal function increase the performance of the model in denoising. To make a fair comparison with the latest released consistency model training techniques, experiments are conducted with same hyper-parameters except curriculum and noise distribution. The models utilized during experiments are determined with low depth to prove the robustness of our proposed technique. The results show that the polynomial noise distribution outperforms the model trained with log-normal noise distribution, yielding a 33.54 FID score after 100,000 training steps with constant discretization steps. Additionally, the implementation of a sinusoidal-based curriculum enhances denoising performance, resulting in a FID score of 30.48.

cs.LG↗

arXiv2024

Nanophotonic inspection of deep-subwavelength integrated optoelectronic chips

Artificial nanostructures with ultrafine and deep-subwavelength feature sizes have emerged as a paradigm-shifting platform to advanced light field management, becoming a key building block for high-performance integrated optoelectronics and flat optics. However, direct optical inspection of such integrated chips with densely packed complex and small features remains a missing metrology gap that hinders quick feedback between design and fabrications. Here, we demonstrate that photothermal nonlinear scattering microscopy can be utilized for direct imaging and resolving of integrated optoelectronic chips beyond the diffraction limit. We reveal that the inherent coupling among deep-subwavelength nanostructures supporting leaky resonances allows for the pronounced heating effect to access reversible nonlinear modulations of the confocal reflection intensity, leading to optical resolving power down to 80 nm (~lambda/7). The versatility of this approach has been exemplified by direct imaging of silicon grating couplers and metalens with a minimum critical dimension of 100 nm, as well as central processing unit (CPU) chip with 45 nm technology, unfolding the long-sought possibility of in-situ, non-destructive, high-throughput optical inspection of integrated optoelectronic chips and nanophotonic chips.

physics.optics↗

arXiv2024

TeleChat Technical Report

In this technical report, we present TeleChat, a collection of large language models (LLMs) with parameters of 3 billion, 7 billion and 12 billion. It includes pretrained language models as well as fine-tuned chat models that is aligned with human preferences. TeleChat is initially pretrained on an extensive corpus containing a diverse collection of texts from both English and Chinese languages, including trillions of tokens. Subsequently, the model undergoes fine-tuning to align with human preferences, following a detailed methodology that we describe. We evaluate the performance of TeleChat on various tasks, including language understanding, mathematics, reasoning, code generation, and knowledge-based question answering. Our findings indicate that TeleChat achieves comparable performance to other open-source models of similar size across a wide range of public benchmarks. To support future research and applications utilizing LLMs, we release the fine-tuned model checkpoints of TeleChat's 7B and 12B variant, along with code and a portion of our pretraining data, to the public community.

cs.CL↗

arXiv2024

RU22Fact: Optimizing Evidence for Multilingual Explainable Fact-Checking on Russia-Ukraine Conflict

Fact-checking is the task of verifying the factuality of a given claim by examining the available evidence. High-quality evidence plays a vital role in enhancing fact-checking systems and facilitating the generation of explanations that are understandable to humans. However, the provision of both sufficient and relevant evidence for explainable fact-checking systems poses a challenge. To tackle this challenge, we propose a method based on a Large Language Model to automatically retrieve and summarize evidence from the Web. Furthermore, we construct RU22Fact, a novel multilingual explainable fact-checking dataset on the Russia-Ukraine conflict in 2022 of 16K samples, each containing real-world claims, optimized evidence, and referenced explanation. To establish a baseline for our dataset, we also develop an end-to-end explainable fact-checking system to verify claims and generate explanations. Experimental results demonstrate the prospect of optimized evidence in increasing fact-checking performance and also indicate the possibility of further progress in the end-to-end claim verification and explanation generation tasks.

cs.CL↗

arXiv2024

How memories are stored in the brain: the declarative memory model

The ability to form memories is a basic feature of learning and accumulating knowledge. But where is memory information stored in the brain? Within the scientific research community, it is generally believed that memory information is stored in the synapse. However, this widely accepted dogma has been challenged by more and more evidence in recent years. In this paper, I will present a memory model, the neural circuit construction theory, created from a perspective that may completely subvert traditional thinking. Instead of looking for whether memories are stored inside neurons or in the synapses, the theory proposes that the human brain does not have any container like memory storage, rather human memory is hardwired through live wiring. The model consists of two parts, the semantic memory model and the episodic memory model.

q-bio.NC↗

arXiv2024

Our Model Achieves Excellent Performance on MovieLens: What Does it Mean?

A typical benchmark dataset for recommender system (RecSys) evaluation consists of user-item interactions generated on a platform within a time period. The interaction generation mechanism partially explains why a user interacts with (e.g., like, purchase, rate) an item, and the context of when a particular interaction happened. In this study, we conduct a meticulous analysis of the MovieLens dataset and explain the potential impact of using the dataset for evaluating recommendation algorithms. We make a few main findings from our analysis. First, there are significant differences in user interactions at the different stages when a user interacts with the MovieLens platform. The early interactions largely define the user portrait which affects the subsequent interactions. Second, user interactions are highly affected by the candidate movies that are recommended by the platform's internal recommendation algorithm(s). Third, changing the order of user interactions makes it more difficult for sequential algorithms to capture the progressive interaction process. We further discuss the discrepancy between the interaction generation mechanism that is employed by the MovieLens system and that of typical real-world recommendation scenarios. In summary, the MovieLens platform demonstrates an efficient and effective way of collecting user preferences to address cold-starts. However, models that achieve excellent recommendation accuracy on the MovieLens dataset may not demonstrate superior performance in practice, for at least two kinds of differences: (i) the differences in the contexts of user-item interaction generation, and (ii) the differences in user knowledge about the item collections. While results on MovieLens can be useful as a reference, they should not be solely relied upon as the primary justification for the effectiveness of a recommendation system model.

cs.IR↗

arXiv2024

Decision-making with Speculative Opponent Models

Opponent modelling has proven effective in enhancing the decision-making of the controlled agent by constructing models of opponent agents. However, existing methods often rely on access to the observations and actions of opponents, a requirement that is infeasible when such information is either unobservable or challenging to obtain. To address this issue, we introduce Distributional Opponent-aided Multi-agent Actor-Critic (DOMAC), the first speculative opponent modelling algorithm that relies solely on local information (i.e., the controlled agent's observations, actions, and rewards). Specifically, the actor maintains a speculated belief about the opponents using the tailored speculative opponent models that predict the opponents' actions using only local information. Moreover, DOMAC features distributional critic models that estimate the return distribution of the actor's policy, yielding a more fine-grained assessment of the actor's quality. This thus more effectively guides the training of the speculative opponent models that the actor depends upon. Furthermore, we formally derive a policy gradient theorem with the proposed opponent models. Extensive experiments under eight different challenging multi-agent benchmark tasks within the MPE, Pommerman and StarCraft Multiagent Challenge (SMAC) demonstrate that our DOMAC successfully models opponents' behaviours and delivers superior performance against state-of-the-art methods with a faster convergence speed.

cs.AI↗

arXiv2024

Real-Fake: Effective Training Data Synthesis Through Distribution Matching

Synthetic training data has gained prominence in numerous learning tasks and scenarios, offering advantages such as dataset augmentation, generalization evaluation, and privacy preservation. Despite these benefits, the efficiency of synthetic data generated by current methodologies remains inferior when training advanced deep models exclusively, limiting its practical utility. To address this challenge, we analyze the principles underlying training data synthesis for supervised learning and elucidate a principled theoretical framework from the distribution-matching perspective that explicates the mechanisms governing synthesis efficacy. Through extensive experiments, we demonstrate the effectiveness of our synthetic data across diverse image classification tasks, both as a replacement for and augmentation to real datasets, while also benefits such as out-of-distribution generalization, privacy preservation, and scalability. Specifically, we achieve 70.9% top1 classification accuracy on ImageNet1K when training solely with synthetic data equivalent to 1 X the original real data size, which increases to 76.0% when scaling up to 10 X synthetic data.

cs.LG↗