arXiv subjects

Junjie Wang

Publications and source records attributed to Junjie Wang.

At least 235 records · Page 13Linked to original sources

arXiv2023

Chatting with GPT-3 for Zero-Shot Human-Like Mobile Automated GUI Testing

Mobile apps are indispensable for people's daily life, and automated GUI (Graphical User Interface) testing is widely used for app quality assurance. There is a growing interest in using learning-based techniques for automated GUI testing which aims at generating human-like actions and interactions. However, the limitations such as low testing coverage, weak generalization, and heavy reliance on training data, make an urgent need for a more effective approach to generate human-like actions to thoroughly test mobile apps. Inspired by the success of the Large Language Model (LLM), e.g., GPT-3 and ChatGPT, in natural language understanding and question answering, we formulate the mobile GUI testing problem as a Q&A task. We propose GPTDroid, asking LLM to chat with the mobile apps by passing the GUI page information to LLM to elicit testing scripts, and executing them to keep passing the app feedback to LLM, iterating the whole process. Within it, we extract the static context of the GUI page and the dynamic context of the iterative testing process, design prompts for inputting this information to LLM, and develop a neural matching network to decode the LLM's output into actionable steps to execute the app. We evaluate GPTDroid on 86 apps from Google Play, and its activity coverage is 71%, with 32% higher than the best baseline, and can detect 36% more bugs with faster speed than the best baseline. GPTDroid also detects 48 new bugs on the Google Play with 25 of them being confirmed/fixed. We further summarize the capabilities of GPTDroid behind the superior performance, including semantic text input, compound action, long meaningful test trace, and test case prioritization.

cs.SE↗

arXiv2023

The Good, the Bad, and the Missing: Neural Code Generation for Machine Learning Tasks

Machine learning (ML) has been increasingly used in a variety of domains, while solving ML programming tasks poses unique challenges because of the fundamentally different nature and construction from general programming tasks, especially for developers who do not have ML backgrounds. Automatic code generation that produces a code snippet from a natural language description can be a promising technique to accelerate ML programming tasks. In recent years, although many deep learning-based neural code generation models have been proposed with high accuracy, the fact that most of them are mainly evaluated on general programming tasks calls into question their effectiveness and usefulness in ML programming tasks. In this paper, we set out to investigate the effectiveness of existing neural code generation models on ML programming tasks. For our analysis, we select six state-of-the-art neural code generation models, and evaluate their performance on four widely used ML libraries, with newly-created 83K pairs of natural-language described ML programming tasks. Our empirical study reveals some good, bad, and missing aspects of neural code generation models on ML tasks, with a few major ones listed below. (Good) Neural code generation models perform significantly better on ML tasks than on non-ML tasks. (Bad) Most of the generated code is semantically incorrect. (Bad) Code generation models cannot significantly improve developers' completion time. (Good) The generated code can help developers write more correct code by providing developers with clues for using correct APIs. (Missing) The observation from our user study reveals the missing aspects of code generation for ML tasks, e.g., decomposing code generation for divide-and-conquer into two tasks: API sequence identification and API usage generation.

cs.SE↗

arXiv2023

Spectral condition for spanning $k$-ended trees in $t$-connected graphs

For any integer $k\geq 2$, a spanning $k$-ended tree is a spanning tree with at most $k$ leaves. In this paper, we provide a tight spectral radius condition for the existence of a spanning $k$-ended tree in $t$-connected graphs, which generalizes a result of Ao, Liu and Yuan (2023).

math.CO↗

arXiv2023

NER-to-MRC: Named-Entity Recognition Completely Solving as Machine Reading Comprehension

Named-entity recognition (NER) detects texts with predefined semantic labels and is an essential building block for natural language processing (NLP). Notably, recent NER research focuses on utilizing massive extra data, including pre-training corpora and incorporating search engines. However, these methods suffer from high costs associated with data collection and pre-training, and additional training process of the retrieved data from search engines. To address the above challenges, we completely frame NER as a machine reading comprehension (MRC) problem, called NER-to-MRC, by leveraging MRC with its ability to exploit existing data efficiently. Several prior works have been dedicated to employing MRC-based solutions for tackling the NER problem, several challenges persist: i) the reliance on manually designed prompts; ii) the limited MRC approaches to data reconstruction, which fails to achieve performance on par with methods utilizing extensive additional data. Thus, our NER-to-MRC conversion consists of two components: i) transform the NER task into a form suitable for the model to solve with MRC in a efficient manner; ii) apply the MRC reasoning strategy to the model. We experiment on 6 benchmark datasets from three domains and achieve state-of-the-art performance without external data, up to 11.24% improvement on the WNUT-16 dataset.

cs.CL↗

arXiv2023

Fengshenbang 1.0: Being the Foundation of Chinese Cognitive Intelligence

Nowadays, foundation models become one of fundamental infrastructures in artificial intelligence, paving ways to the general intelligence. However, the reality presents two urgent challenges: existing foundation models are dominated by the English-language community; users are often given limited resources and thus cannot always use foundation models. To support the development of the Chinese-language community, we introduce an open-source project, called Fengshenbang, which leads by the research center for Cognitive Computing and Natural Language (CCNL). Our project has comprehensive capabilities, including large pre-trained models, user-friendly APIs, benchmarks, datasets, and others. We wrap all these in three sub-projects: the Fengshenbang Model, the Fengshen Framework, and the Fengshen Benchmark. An open-source roadmap, Fengshenbang, aims to re-evaluate the open-source community of Chinese pre-trained large-scale models, prompting the development of the entire Chinese large-scale model community. We also want to build a user-centered open-source ecosystem to allow individuals to access the desired models to match their computing resources. Furthermore, we invite companies, colleges, and research institutions to collaborate with us to build the large-scale open-source model-based ecosystem. We hope that this project will be the foundation of Chinese cognitive intelligence.

cs.CL↗

arXiv2023

Giant atom induced zero modes and localization in the nonreciprocal Su-Schrieffer-Heeger chain

A notable feature of non-Hermitian systems with skin effects is the sensitivity of their spectra and eigenstates to the boundary conditions. In the literature, three types of boundary conditions-periodic boundary condition,open boundary condition and a defect in the system as a boundary, are explored. In this work we introduce the other type of boundary condition provided by a giant atom. The giant atom couples to a nonreciprocal SuSchrieffer-Heeger chain at two points and plays the role of defects. We study the spectrum and localization of eigenstates of the system and find that the giant atom can induce asymmetric zero modes. A remarkable feature is that bulk states might localize at the left or the right chain-atom coupling sites in weak localization regimes. This bipolar localization leads to Bloch-like states, even though translational invariance is broken. Moreover, we find that the localization is obviously weaker than the case with two small atoms or open boundary conditions even in strong coupling regimes. These intriguing results indicate that nonlocal coupling of giant atom to a nonreciprocal SSH chain weakens localization of the eigenstates. We also show that the Lyapunov exponent in the long-time dynamics in real space can act as a witness of the localized bulk states.

quant-ph↗

arXiv2023

Hierarchical Motion Planning under Probabilistic Temporal Tasks and Safe-Return Constraints

Safety is crucial for robotic missions within an uncertain environment. Common safety requirements such as collision avoidance are only state-dependent, which can be restrictive for complex missions. In this work, we address a more general formulation as safe-return constraints, which require the existence of a return-policy to drive the system back to a set of safe states with high probability. The robot motion is modeled as a Markov Decision Process (MDP) with probabilistic labels, which can be highly non-ergodic. The robotic task is specified as Linear Temporal Logic (LTL) formulas over these labels, such as surveillance and transportation. We first provide theoretical guarantees on the re-formulation of such safe-return constraints, and a baseline solution based on computing two complete product automata. Furthermore, to tackle the computational complexity, we propose a hierarchical planning algorithm that combines the feature-based symbolic and temporal abstraction with constrained optimization. It synthesizes simultaneously two dependent motion policies: the outbound policy minimizes the overall cost of satisfying the task with a high probability, while the return policy ensures the safe-return constraints. The problem formulation is versatile regarding the robot model, task specifications and safety constraints. The proposed hierarchical algorithm is more efficient and can solve much larger problems than the baseline solution, with only a slight loss of optimality. Numerical validations include simulations and hardware experiments of a search-and-rescue mission and a planetary exploration mission over various system sizes.

cs.RO↗

arXiv2023

Spectral conditions for graphs having all (fractional) $[a,b]$-factors

Let $a\leq b$ be two positive integers. We say that a graph $G$ has all $[a,b]$-factors if it has an $h$-factor for every function $h: V(G)\rightarrow \mathbb{Z}^+$ such that $a\le h(v) \le b$ for all $v\in V(G)$ and $\sum_{v\in V(G)}h(v)\equiv 0\pmod 2$, and has all fractional $[a,b]$-factors if it has a fractional $p$-factor for every $p: V(G) \rightarrow \mathbb{Z}^+$ such that $a\le p(v)\le b$ for all $v\in V(G)$. In this paper, we provide tight spectral radius conditions for graphs having all $[a,b]$-factors ($3\leq a<b$) and all fractional $[a,b]$-factors ($1\leq a<b$), respectively.

math.CO↗

arXiv2023

Superconductivity in an Orbital-reoriented SnAs Square Lattice: a Case Study of Li0.6Sn2As2 and NaSnAs

Searching for functional square lattices in layered superconductor systems offers an explicit clue to modify the electron behavior and find exotic properties. The trigonal SnAs3 structural units in SnAs-based systems are relatively conformable to distortion, which provides the possibility to achieve structurally topological transformation and higher superconducting transition temperatures. In the present work, the functional As square lattice was realized and activated in Li0.6Sn2As2 and NaSnAs through a topotactic structural transformation of trigonal SnAs3 to square SnAs4 under pressure, resulting in a record-high Tc among all synthesized SnAs-based compounds. Meanwhile, the conductive channel transfers from the out-of-plane pz orbital to the in-plane px+py orbitals, facilitating electron hopping within the square 2D lattice and boosting the superconductivity. The reorientation of p-orbital following a directed local structure transformation provides an effective strategy to modify layered superconductors.

cond-mat.supr-con↗

arXiv2022

Prediction of surface reconstructions using MAGUS

In this paper, we present a new module to predict the potential surface reconstruction configurations of given surface structures in the framework of our machine learning and graph theory assisted universal structure searcher (MAGUS). In addition to random structures generated with specific lattice symmetry, we made full use of bulk materials to obtain a better distribution of population energy, namely, randomly appending atoms to a surface cleaved from bulk structures or moving/removing some of the atoms on the surface, which is inspired by natural surface reconstruction processes. In addition, we borrowed ideas from cluster predictions to spread structures better between different compositions, considering that surface models of different atom numbers usually have some building blocks in common. To validate this newly developed module, we tested it with studies on the surface reconstructions of Si (100), Si (111) and 4H-SiC(1-102)-c(2x2), respectively. We successfully gave the known ground states as well as a new SiC surface model in an extremely Si-rich environment.

cond-mat.mtrl-sci↗

arXiv2022

Fill in the Blank: Context-aware Automated Text Input Generation for Mobile GUI Testing

Automated GUI testing is widely used to help ensure the quality of mobile apps. However, many GUIs require appropriate text inputs to proceed to the next page which remains a prominent obstacle for testing coverage. Considering the diversity and semantic requirement of valid inputs (e.g., flight departure, movie name), it is challenging to automate the text input generation. Inspired by the fact that the pre-trained Large Language Model (LLM) has made outstanding progress in text generation, we propose an approach named QTypist based on LLM for intelligently generating semantic input text according to the GUI context. To boost the performance of LLM in the mobile testing scenario, we develop a prompt-based data construction and tuning method which automatically extracts the prompts and answers for model tuning. We evaluate QTypist on 106 apps from Google Play and the result shows that the passing rate of QTypist is 87%, which is 93% higher than the best baseline. We also integrate QTypist with the automated GUI testing tools and it can cover 42% more app activities, 52% more pages, and subsequently help reveal 122% more bugs compared with the raw tool.

cs.SE↗

arXiv2022

The Metamorphosis: Automatic Detection of Scaling Issues for Mobile Apps

As the bridge between users and software, Graphical User Interface (GUI) is critical to the app accessibility. Scaling up the font or display size of GUI can help improve the visual impact, readability, and usability of an app, and is frequently used by the elderly and people with vision impairment. Yet this can easily lead to scaling issues such as text truncation, component overlap, which negatively influence the acquirement of the right information and the fluent usage of the app. Previous techniques for UI display issue detection and cross-platform inconsistency detection cannot work well for these scaling issues. In this paper, we propose an automated method, dVermin, for scaling issue detection, through detecting the inconsistency of a view under the default and a larger display scale. The evaluation result shows that dVermin achieves 97% precision and 97% recall in issue page detection, and 84% precision and 91% recall for issue view detection, outperforming two state-of-the-art baselines by a large margin. We also evaluate dVermin with popular Android apps on F-droid, and successfully uncover 21 previously-undetected scaling issues with 20 of them being confirmed/fixed.

cs.SE↗

arXiv2022

Prototypical context-aware dynamics generalization for high-dimensional model-based reinforcement learning

The latent world model provides a promising way to learn policies in a compact latent space for tasks with high-dimensional observations, however, its generalization across diverse environments with unseen dynamics remains challenging. Although the recurrent structure utilized in current advances helps to capture local dynamics, modeling only state transitions without an explicit understanding of environmental context limits the generalization ability of the dynamics model. To address this issue, we propose a Prototypical Context-Aware Dynamics (ProtoCAD) model, which captures the local dynamics by time consistent latent context and enables dynamics generalization in high-dimensional control tasks. ProtoCAD extracts useful contextual information with the help of the prototypes clustered over batch and benefits model-based RL in two folds: 1) It utilizes a temporally consistent prototypical regularizer that encourages the prototype assignments produced for different time parts of the same latent trajectory to be temporally consistent instead of comparing the features; 2) A context representation is designed which combines both the projection embedding of latent states and aggregated prototypes and can significantly improve the dynamics generalization ability. Extensive experiments show that ProtoCAD surpasses existing methods in terms of dynamics generalization. Compared with the recurrent-based model RSSM, ProtoCAD delivers 13.2% and 26.7% better mean and median performance across all dynamics generalization tasks.

cs.LG↗

arXiv2022

Bloch band structures and linear response theory of nonlinear systems

We investigate the Bloch bands and develop a linear response theory for nonlinear systems, where the interplay between topological parameters and nonlinearity leads to new band structures. The nonlinear system under consideration is described by the Qi-Wu-Zhang model with Kerr-type nonlinearity, which can be treated as a nonlinear version of Chern insulator. We explore the eigenenergies of the Hamiltonian and discuss its Bloch band structures as well as the condition of gap closing. A cone structure in the ground Bloch band and tubed structure in the excited Bloch band is found. We also numerically calculate the linear response of the nonlinear Chern insulator to external fields, finding that these new band structures break the condition of adiabatic evolution and make the linear response not quantized. This feature of response can be understood by examining the dynamics of the nonlinear system.

quant-ph↗

arXiv2022

Corrected Evaluation Results of the NTCIR WWW-2, WWW-3, and WWW-4 English Subtasks

Unfortunately, the official English (sub)task results reported in the NTCIR-14 WWW-2, NTCIR-15 WWW-3, and NTCIR-16 WWW-4 overview papers are incorrect due to noise in the official qrels files; this paper reports results based on the corrected qrels files. The noise is due to a fatal bug in the backend of our relevance assessment interface. More specifically, at WWW-2, WWW-3, and WWW-4, two versions of pool files were created for each English topic: a PRI ("prioritised") file, which uses the NTCIRPOOL script to prioritise likely relevant documents, and a RND ("randomised") file, which randomises the pooled documents. This was done for the purpose of studying the effect of document ordering for relevance assessors. However, the programmer who wrote the interface backend assumed that a combination of a topic ID and a document rank in the pool file uniquely determines a document ID; this is obviously incorrect as we have two versions of pool files. The outcome is that all the PRI-based relevance labels for the WWW-2 test collection are incorrect (while all the RND-based relevance labels are correct), and all the RND-based relevance labels for the WWW-3 and WWW-4 test collections are incorrect (while all the PRI-based relevance labels are correct). This bug was finally discovered at the NTCIR-16 WWW-4 task when the first seven authors of this paper served as Gold assessors (i.e., topic creators who define what is relevant) and closely examined the disagreements with Bronze assessors (i.e., non-topic-creators; non-experts). We would like to apologise to the WWW participants and the NTCIR chairs for the inconvenience and confusion caused due to this bug.

cs.IR↗

arXiv2022

Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective

We propose a new paradigm for zero-shot learners that is format agnostic, i.e., it is compatible with any format and applicable to a list of language tasks, such as text classification, commonsense reasoning, coreference resolution, and sentiment analysis. Zero-shot learning aims to train a model on a given task such that it can address new learning tasks without any additional training. Our approach converts zero-shot learning into multiple-choice tasks, avoiding problems in commonly used large-scale generative models such as FLAN. It not only adds generalization ability to models but also significantly reduces the number of parameters. Our method shares the merits of efficient training and deployment. Our approach shows state-of-the-art performance on several benchmarks and produces satisfactory results on tasks such as natural language inference and text classification. Our model achieves this success with only 235M parameters, which is substantially smaller than state-of-the-art models with billions of parameters. The code and pre-trained models are available at https://github.com/IDEA-CCNL/Fengshenbang-LM .

cs.CL↗

arXiv2022

LeVoice ASR Systems for the ISCSLP 2022 Intelligent Cockpit Speech Recognition Challenge

This paper describes LeVoice automatic speech recognition systems to track2 of intelligent cockpit speech recognition challenge 2022. Track2 is a speech recognition task without limits on the scope of model size. Our main points include deep learning based speech enhancement, text-to-speech based speech generation, training data augmentation via various techniques and speech recognition model fusion. We compared and fused the hybrid architecture and two kinds of end-to-end architecture. For end-to-end modeling, we used models based on connectionist temporal classification/attention-based encoder-decoder architecture and recurrent neural network transducer/attention-based encoder-decoder architecture. The performance of these models is evaluated with an additional language model to improve word error rates. As a result, our system achieved 10.2\% character error rate on the challenge test set data and ranked third place among the submitted systems in the challenge.

eess.AS↗

arXiv2022

Are We Building on the Rock? On the Importance of Data Preprocessing for Code Summarization

Code summarization, the task of generating useful comments given the code, has long been of interest. Most of the existing code summarization models are trained and validated on widely-used code comment benchmark datasets. However, little is known about the quality of the benchmark datasets built from real-world projects. Are the benchmark datasets as good as expected? To bridge the gap, we conduct a systematic research to assess and improve the quality of four benchmark datasets widely used for code summarization tasks. First, we propose an automated code-comment cleaning tool that can accurately detect noisy data caused by inappropriate data preprocessing operations from existing benchmark datasets. Then, we apply the tool to further assess the data quality of the four benchmark datasets, based on the detected noises. Finally, we conduct comparative experiments to investigate the impact of noisy data on the performance of code summarization models. The results show that these data preprocessing noises widely exist in all four benchmark datasets, and removing these noisy data leads to a significant improvement on the performance of code summarization. We believe that the findings and insights will enable a better understanding of data quality in code summarization tasks, and pave the way for relevant research and practice.

cs.SE↗