arXiv subjects

Yixuan Chen

Publications and source records attributed to Yixuan Chen.

44 records · Page 3Linked to original sources

arXiv2024

STALL+: Boosting LLM-based Repository-level Code Completion with Static Analysis

Repository-level code completion is challenging as it involves complicated contexts from multiple files in the repository. To date, researchers have proposed two technical categories to enhance LLM-based repository-level code completion, i.e., retrieval-augmented generation (RAG) and static analysis integration. This work performs the first study on the static analysis integration in LLM-based repository-level code completion by investigating both the effectiveness and efficiency of static analysis integration strategies across different phases of code completion. We first implement a framework STALL+, which supports an extendable and customizable integration of multiple static analysis strategies into the complete pipeline of LLM-based repository-level code completion; and based on STALL+, we perform extensive experiments by including different code LLMs on the latest repository-level code completion benchmark CrossCodeEval. Our findings show that integrating file-level dependencies in prompting phase performs the best while the integration in post-processing phase performs the worse. Additionally, we observe different improvements from static analysis between dynamic languages and static languages, i.e., the best combination is prompting-phase with decoding-phase integration for Java while the best combination is prompting-phase with post-processing-phase integration for Python given the limitations of statically analyzing dynamic languages. Additionally, we find the complementarity between RAG and static analysis integration as well as their cost-effectiveness after combination.

cs.SE↗

arXiv2024

No More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generation

Unit testing is essential in detecting bugs in functionally-discrete program units. Manually writing high-quality unit tests is time-consuming and laborious. Although traditional techniques can generate tests with reasonable coverage, they exhibit low readability and cannot be directly adopted by developers. Recent work has shown the large potential of large language models (LLMs) in unit test generation, which can generate more human-like and meaningful test code. ChatGPT, the latest LLM incorporating instruction tuning and reinforcement learning, has performed well in various domains. However, It remains unclear how effective ChatGPT is in unit test generation. In this work, we perform the first empirical study to evaluate ChatGPT's capability of unit test generation. Specifically, we conduct a quantitative analysis and a user study to systematically investigate the quality of its generated tests regarding the correctness, sufficiency, readability, and usability. The tests generated by ChatGPT still suffer from correctness issues, including diverse compilation errors and execution failures. Still, the passing tests generated by ChatGPT resemble manually-written tests by achieving comparable coverage, readability, and even sometimes developers' preference. Our findings indicate that generating unit tests with ChatGPT could be very promising if the correctness of its generated tests could be further improved. Inspired by our findings above, we propose ChatTESTER, a novel ChatGPT-based unit test generation approach, which leverages ChatGPT itself to improve the quality of its generated tests. ChatTESTER incorporates an initial test generator and an iterative test refiner. Our evaluation demonstrates the effectiveness of ChatTESTER by generating 34.3% more compilable tests and 18.7% more tests with correct assertions than the default ChatGPT.

cs.SE↗

arXiv2024

Train Faster, Perform Better: Modular Adaptive Training in Over-Parameterized Models

Despite their prevalence in deep-learning communities, over-parameterized models convey high demands of computational costs for proper training. This work studies the fine-grained, modular-level learning dynamics of over-parameterized models to attain a more efficient and fruitful training strategy. Empirical evidence reveals that when scaling down into network modules, such as heads in self-attention models, we can observe varying learning patterns implicitly associated with each module's trainability. To describe such modular-level learning capabilities, we introduce a novel concept dubbed modular neural tangent kernel (mNTK), and we demonstrate that the quality of a module's learning is tightly associated with its mNTK's principal eigenvalue $λ_{\max}$. A large $λ_{\max}$ indicates that the module learns features with better convergence, while those miniature ones may impact generalization negatively. Inspired by the discovery, we propose a novel training strategy termed Modular Adaptive Training (MAT) to update those modules with their $λ_{\max}$ exceeding a dynamic threshold selectively, concentrating the model on learning common features and ignoring those inconsistent ones. Unlike most existing training schemes with a complete BP cycle across all network modules, MAT can significantly save computations by its partially-updating strategy and can further improve performance. Experiments show that MAT nearly halves the computational cost of model training and outperforms the accuracy of baselines.

cs.LG↗

arXiv2023

Adaptive Distributionally Robust Planning for Renewable-Powered Fast Charging Stations Under Decision-Dependent EV Diffusion Uncertainty

When deploying fast charging stations (FCSs) to support long-distance trips of electric vehicles (EVs), there exist indirect network effects: while the gradual diffusion of EVs directly influences the timing and capacities of FCS allocation, the decisions for FCS allocations, in turn, impact the drivers' willingness to adopt EVs. This interplay, if neglected, can result in uncovered EVs and security issues on the grid side and even hinder the effective diffusion of EVs. In this paper, we explicitly incorporate this interdependence by quantifying EV adoption rates as decision-dependent uncertainties (DDUs) using decision-dependent ambiguity sets (DDASs). Then, a two-stage decision-dependent distributionally robust FCS planning (D$^3$R-FCSP) model is developed for adaptively deploying FCSs with on-site sources and expanding the coupled distribution network. A multi-period capacitated arc cover-path cover (MCACPC) model is incorporated to capture the EVs' recharging patterns to ensure the feasibility of FCS locations and capacities. To resolve the nonlinearity and nonconvexity, the D$^3$R-FCSP model is equivalently reformulated into a single-level mixed-integer linear programming by exploiting its strong duality and applying the McCormick envelope. Finally, case studies highlight the superior out-of-sample performances of our model in terms of security and cost-efficiency. Furthermore, the byproduct of accelerated EV adoption through an implicit positive feedback loop is highlighted.

eess.SY↗

arXiv2023

ClassEval: A Manually-Crafted Benchmark for Evaluating LLMs on Class-level Code Generation

In this work, we make the first attempt to evaluate LLMs in a more challenging code generation scenario, i.e. class-level code generation. We first manually construct the first class-level code generation benchmark ClassEval of 100 class-level Python code generation tasks with approximately 500 person-hours. Based on it, we then perform the first study of 11 state-of-the-art LLMs on class-level code generation. Based on our results, we have the following main findings. First, we find that all existing LLMs show much worse performance on class-level code generation compared to on standalone method-level code generation benchmarks like HumanEval; and the method-level coding ability cannot equivalently reflect the class-level coding ability among LLMs. Second, we find that GPT-4 and GPT-3.5 still exhibit dominate superior than other LLMs on class-level code generation, and the second-tier models includes Instruct-Starcoder, Instruct-Codegen, and Wizardcoder with very similar performance. Third, we find that generating the entire class all at once (i.e. holistic generation strategy) is the best generation strategy only for GPT-4 and GPT-3.5, while method-by-method generation (i.e. incremental and compositional) is better strategies for the other models with limited ability of understanding long instructions and utilizing the middle information. Lastly, we find the limited model ability of generating method-dependent code and discuss the frequent error types in generated classes. Our benchmark is available at https://github.com/FudanSELab/ClassEval.

cs.CL↗

arXiv2021

A new parsimonious method for classifying Cancer Tissue-of-Origin Based on DNA Methylation 450K data

DNA methylation is a well-studied genetic modification that regulates gene transcription of Eukaryotes. Its alternations have been recognized as a significant component of cancer development. In this study, we use the DNA methylation 450k data from The Cancer Genome Atlas to evaluate the efficacy of DNA methylation data on cancer classification for 30 cancer types. We propose a new method for gene selection in high dimensional data(over 450 thousand). Variance filtering is first introduced for dimension reduction and Recursive feature elimination (RFE) is then used for feature selection. We address the problem of selecting a small subsets of genes from large number of methylated sites, and our parsimonious model is demonstrated to be efficient, achieving an accuracy over 91%, outperforming other studies which use DNA micro-arrays and RNA-seq Data . The performance of 20 models, which are based on 4 estimators (Random Forest, Decision Tree, Extra Tree and Support Vector Machine) and 5 classifiers (k-Nearest Neighbours, Support Vector Machine, XGboost, Light GBM and Multi-Layer Perceptron), is compared and robustness of the RFE algorithm is examined. Results suggest that the combined model of extra tree plus catboost classifier offers the best performance in cancer identification, with an overall validation accuracy of 91% , 92.3%, 93.3% and 93.5% for 20, 30, 40 and 50 features respectively. The biological functions in cancer development of 50 selected genes is also explored through enrichment analysis and the results show that 12 out of 16 of our top features have already been identified to be specific with cancer and we also propose some more genes to be tested for future studies. Therefore, our method may be utilzed as an auxiliary diagnostic method to determine the actual clinicopathological status of a specific cancer.

q-bio.TO↗

arXiv2020

Electric field induced metallic behavior in thin crystals of ferroelectric α-In2Se3

Ferroelectric semiconductor field effect transistors (FeSmFETs), which employ ferroelectric semiconducting thin crystals of α-In2Se3 as the channel material as opposed to the gate dielectric in conventional ferroelectric FETs (FeFETs) were prepared and measured from room to the liquid-helium temperatures. These FeSmFETs were found to yield evidence for the reorientation of the electrical polarization and an electric field induced metallic state in α-In2Se3. Our findings suggest that FeSmFETs can serve as a platform for the fundamental study of ferroelectric metals as well as the exploration of the integration of data storage and logic operations in the same device.

cond-mat.mtrl-sci↗

arXiv2015

Temperature-dependent transformation thermotics: From switchable thermal cloaks to macroscopic thermal diodes

The macroscopic control of ubiquitous heat flow remains poorly explored due to the lack of a fundamental theoretical method. Here, by establishing temperature-dependent transformation thermotics for treating materials whose conductivity depends on temperature, we show analytical and simulation evidence for switchable thermal cloaking and a macroscopic thermal diode based on the cloaking. The latter allows heat flow in one direction but prohibits the flow in the opposite direction, which is also confirmed by our experiments. Our results suggest that the temperature-dependent transformation thermotics could be a fundamental theoretical method for achieving macroscopic heat rectification, and provide guidance both for macroscopic control of heat flow and for the design of the counterparts of switchable thermal cloaks or macroscopic thermal diodes in other fields like seismology, acoustics, electromagnetics, or matter waves.

cond-mat.mtrl-sci↗