arXiv subjects

Shuo Jiang

Publications and source records attributed to Shuo Jiang.

At least 55 records · Page 3Linked to original sources

arXiv2023

A Survey on Robotic Manipulation of Deformable Objects: Recent Advances, Open Challenges and New Frontiers

Deformable object manipulation (DOM) for robots has a wide range of applications in various fields such as industrial, service and health care sectors. However, compared to manipulation of rigid objects, DOM poses significant challenges for robotic perception, modeling and manipulation, due to the infinite dimensionality of the state space of deformable objects (DOs) and the complexity of their dynamics. The development of computer graphics and machine learning has enabled novel techniques for DOM. These techniques, based on data-driven paradigms, can address some of the challenges that analytical approaches of DOM face. However, some existing reviews do not include all aspects of DOM, and some previous reviews do not summarize data-driven approaches adequately. In this article, we survey more than 150 relevant studies (data-driven approaches mainly) and summarize recent advances, open challenges, and new frontiers for aspects of perception, modeling and manipulation for DOs. Particularly, we summarize initial progress made by Large Language Models (LLMs) in robotic manipulation, and indicates some valuable directions for further research. We believe that integrating data-driven approaches and analytical approaches can provide viable solutions to open challenges of DOM.

cs.RO↗

arXiv2023

ProtoCode: Leveraging Large Language Models for Automated Generation of Machine-Readable Protocols from Scientific Publications

Protocol standardization and sharing are crucial for reproducibility in life sciences. In spite of numerous efforts for standardized protocol description, adherence to these standards in literature remains largely inconsistent. Curation of protocols are especially challenging due to the labor intensive process, requiring expert domain knowledge of each experimental procedure. Recent advancements in Large Language Models (LLMs) offer a promising solution to interpret and curate knowledge from complex scientific literature. In this work, we develop ProtoCode, a tool leveraging fine-tune LLMs to curate protocols which can be interpretable by both human and machine interfaces. Our proof-of-concept, focused on polymerase chain reaction (PCR) protocols, retrieves information from PCR protocols at an accuracy ranging 69-100% depending on the information content. In all the tested protocols, we demonstrate that ProtoCode successfully converts literature-based protocols into correct operational files for multiple thermal cycler systems. In conclusion, ProtoCode can alleviate labor intensive curation and standardization of life science protocols to enhance research reproducibility by providing a reliable, automated means to process and standardize protocols. ProtoCode is freely available as a web server at https://curation.taxila.io/ProtoCode/.

q-bio.QM↗

arXiv2023

Hierarchical RL-Guided Large-scale Navigation of a Snake Robot

Classical snake robot control leverages mimicking snake-like gaits tuned for specific environments. However, to operate adaptively in unstructured environments, gait generation must be dynamically scheduled. In this work, we present a four-layer hierarchical control scheme to enable the snake robot to navigate freely in large-scale environments. The proposed model decomposes navigation into global planning, local planning, gait generation, and gait tracking. Using reinforcement learning (RL) and a central pattern generator (CPG), our method learns to navigate in complex mazes within hours and can be directly deployed to arbitrary new environments in a zero-shot fashion. We use the high-fidelity model of Northeastern's slithering robot COBRA to test the effectiveness of the proposed hierarchical control approach.

cs.RO↗

arXiv2023

Snake Robot with Tactile Perception Navigates on Large-scale Challenging Terrain

Along with the advancement of robot skin technology, there has been notable progress in the development of snake robots featuring body-surface tactile perception. In this study, we proposed a locomotion control framework for snake robots that integrates tactile perception to augment their adaptability to various terrains. Our approach embraces a hierarchical reinforcement learning (HRL) architecture, wherein the high-level orchestrates global navigation strategies while the low-level uses curriculum learning for local navigation maneuvers. Due to the significant computational demands of collision detection in whole-body tactile sensing, the efficiency of the simulator is severely compromised. Thus a distributed training pattern to mitigate the efficiency reduction was adopted. We evaluated the navigation performance of the snake robot in complex large-scale cave exploration with challenging terrains to exhibit improvements in motion efficiency, evidencing the efficacy of tactile perception in terrain-adaptive locomotion of snake robots.

cs.RO↗

arXiv2023

Robot Learning in the Era of Foundation Models: A Survey

The proliferation of Large Language Models (LLMs) has s fueled a shift in robot learning from automation towards general embodied Artificial Intelligence (AI). Adopting foundation models together with traditional learning methods to robot learning has increasingly gained recent interest research community and showed potential for real-life application. However, there are few literatures comprehensively reviewing the relatively new technologies combined with robotics. The purpose of this review is to systematically assess the state-of-the-art foundation model techniques in the robot learning and to identify future potential areas. Specifically, we first summarized the technical evolution of robot learning and identified the necessary preliminary preparations for foundation models including the simulators, datasets, foundation model framework. In addition, we focused on the following four mainstream areas of robot learning including manipulation, navigation, planning, and reasoning and demonstrated how the foundation model techniques can be adopted in the above scenarios. Furthermore, critical issues which are neglected in the current literatures including robot hardware and software decoupling, dynamic data, generalization performance with the presence of human, etc. were discussed. This review highlights the state-of-the-art progress of foundation models in robot learning and future research should focus on multimodal interaction especially dynamics data, exclusive foundation models for robots, and AI alignment, etc.

cs.RO↗

arXiv2023

Synthetic IMU Datasets and Protocols Can Simplify Fall Detection Experiments and Optimize Sensor Configuration

Falls represent a significant cause of injury among the elderly population. Extensive research has been devoted to the utilization of wearable IMU sensors in conjunction with machine learning techniques for fall detection. To address the challenge of acquiring costly training data, this paper presents a novel method that generates a substantial volume of synthetic IMU data with minimal real fall experiments. First, unmarked 3D motion capture technology is employed to reconstruct human movements. Subsequently, utilizing the biomechanical simulation platform Opensim and forward kinematic methods, an ample amount of training data from various body segments can be custom generated. An LSTM model is trained, achieving testing accuracies of 91.99% and 86.62% on two distinct datasets of actual fall-related IMU data, demonstrated the comparable performance of models trained using genuine IMU data. Building upon the simulation framework, this paper further optimized the single IMU attachment position and multiple IMU combinations on fall detection. The proposed method simplifies fall detection data acquisition experiments, provides novel venue for generating low cost synthetic data in scenario where acquiring data for machine learning is challenging and paves the way for customizing machine learning configurations.

q-bio.QM↗

arXiv2023

Exploring assessment method of technological advancement based on literature cross-citation

Assessing advancements of technology is essential for creating science and technology policies and making informed investments in the technology market. However, current methods primarily focus on the characteristics of the technologies themselves, making it difficult to accurately assess technologies across various fields and generations. To address this challenge, we propose a novel approach that uses bibliometrics, specifically literature citation networks, to measure changes in knowledge flow throughout the evolution of technology. This method can identify diverse trends in technology development and is an effective tool for evaluating technological advancements. We demonstrate its accuracy and applicability by applying it to mobile communication technology and comparing its quantitative results with other assessment methods. Our work provides critical support for assessing different technical routes and formulating technology policy.

stat.AP↗

arXiv2022

Indoor Future Person Localization from an Egocentric Wearable Camera

Accurate prediction of future person location and movement trajectory from an egocentric wearable camera can benefit a wide range of applications, such as assisting visually impaired people in navigation, and the development of mobility assistance for people with disability. In this work, a new egocentric dataset was constructed using a wearable camera, with 8,250 short clips of a targeted person either walking 1) toward, 2) away, or 3) across the camera wearer in indoor environments, or 4) staying still in the scene, and 13,817 person bounding boxes were manually labelled. Apart from the bounding boxes, the dataset also contains the estimated pose of the targeted person as well as the IMU signal of the wearable camera at each time point. An LSTM-based encoder-decoder framework was designed to predict the future location and movement trajectory of the targeted person in this egocentric setting. Extensive experiments have been conducted on the new dataset, and have shown that the proposed method is able to reliably and better predict future person location and trajectory in egocentric videos captured by the wearable camera compared to three baselines.

cs.CV↗

arXiv2022

Technology Fitness Landscape for Design Innovation: A Deep Neural Embedding Approach Based on Patent Data

Technology is essential to innovation and economic prosperity. Understanding technological changes can guide innovators to find new directions of design innovation and thus make breakthroughs. In this work, we construct a technology fitness landscape via deep neural embeddings of patent data. The landscape consists of 1,757 technology domains and their respective improvement rates. In the landscape, we found a high hill related to information and communication technologies (ICT) and a vast low plain of the remaining domains. The landscape presents a bird's eye view of the structure of the total technology space, providing a new way for innovators to interpret technology evolution with a biological analogy, and a biologically-inspired inference to the next innovation.

cs.LG↗

arXiv2022

Patent Data for Engineering Design: A Critical Review and Future Directions

Patent data have long been used for engineering design research because of its large and expanding size, and widely varying massive amount of design information contained in patents. Recent advances in artificial intelligence and data science present unprecedented opportunities to develop data-driven design methods and tools, as well as advance design science, using the patent database. Herein, we survey and categorize the patent-for-design literature based on its contributions to design theories, methods, tools, and strategies, as well as the types of patent data and data-driven methods used in respective studies. Our review highlights promising future research directions in patent data-driven design research and practice.

cs.DL↗

arXiv2022

Exceptional Point modulated by Kerr effect in Anti-Parity-Time Symmetry System

With respect to parity-time (PT) symmetry, anti-parity-time (APT) symmetric system exhibits much easier readout mechanism due to its real frequency splitting. Generally, such systems need to be operated at exceptional points (EPs) to obtain the best performance. However, strict conditons to locate APT symmetric systems at their EPs precisely put restraints on their practical applications. To overcome this problem, we propose a scheme to manipulate the EPs in APT symmetric configuration by Kerr effect. It is demonstrated that operating EPs by self-phase modulation alone will impede the frequency splitting caused by external perturbations, while cross-phase modulation can enhance the response to measurable perturbations. We also investigate the thermal effect induced by high light intensity, which could reduce the power to manipulate EPs. This proposed scheme can pave a new way in fabricating devices based on APT symmetry.

physics.optics↗

arXiv2022

Deep Learning for Technical Document Classification

In large technology companies, the requirements for managing and organizing technical documents created by engineers and managers have increased dramatically in recent years, which has led to a higher demand for more scalable, accurate, and automated document classification. Prior studies have only focused on processing text for classification, whereas technical documents often contain multimodal information. To leverage multimodal information for document classification to improve the model performance, this paper presents a novel multimodal deep learning architecture, TechDoc, which utilizes three types of information, including natural language texts and descriptive images within documents and the associations among the documents. The architecture synthesizes the convolutional neural network, recurrent neural network, and graph neural network through an integrated training process. We applied the architecture to a large multimodal technical document database and trained the model for classifying documents based on the hierarchical International Patent Classification system. Our results show that TechDoc presents a greater classification accuracy than the unimodal methods and other state-of-the-art benchmarks. The trained model can potentially be scaled to millions of real-world multimodal technical documents, which is useful for data and knowledge management in large technology companies and organizations.

cs.LG↗

arXiv2021

Data-Driven Design-by-Analogy: State of the Art and Future Directions

Design-by-Analogy (DbA) is a design methodology wherein new solutions, opportunities or designs are generated in a target domain based on inspiration drawn from a source domain; it can benefit designers in mitigating design fixation and improving design ideation outcomes. Recently, the increasingly available design databases and rapidly advancing data science and artificial intelligence technologies have presented new opportunities for developing data-driven methods and tools for DbA support. In this study, we survey existing data-driven DbA studies and categorize individual studies according to the data, methods, and applications in four categories, namely, analogy encoding, retrieval, mapping, and evaluation. Based on both nuanced organic review and structured analysis, this paper elucidates the state of the art of data-driven DbA research to date and benchmarks it with the frontier of data science and AI research to identify promising research opportunities and directions for the field. Finally, we propose a future conceptual data-driven DbA system that integrates all propositions.

cs.AI↗

arXiv2021

An Intelligent Passive Food Intake Assessment System with Egocentric Cameras

Malnutrition is a major public health concern in low-and-middle-income countries (LMICs). Understanding food and nutrient intake across communities, households and individuals is critical to the development of health policies and interventions. To ease the procedure in conducting large-scale dietary assessments, we propose to implement an intelligent passive food intake assessment system via egocentric cameras particular for households in Ghana and Uganda. Algorithms are first designed to remove redundant images for minimising the storage memory. At run time, deep learning-based semantic segmentation is applied to recognise multi-food types and newly-designed handcrafted features are extracted for further consumed food weight monitoring. Comprehensive experiments are conducted to validate our methods on an in-the-wild dataset captured under the settings which simulate the unique LMIC conditions with participants of Ghanaian and Kenyan origin eating common Ghanaian/Kenyan dishes. To demonstrate the efficacy, experienced dietitians are involved in this research to perform the visual portion size estimation, and their predictions are compared to our proposed method. The promising results have shown that our method is able to reliably monitor food intake and give feedback on users' eating behaviour which provides guidance for dietitians in regular dietary assessment.

cs.CV↗

arXiv2020

Exceptional surface-enhanced Rotation Sensing with Robustness in an optical microcavity

Exceptional points (EPs) are special singularities of non-Hermitian Hamiltonians. At an EP, two or more eigenvalues and the corresponding eigenstates coalesce. Recently, EP-based optical gyroscope near an EP was extensively investigated to improve the response to rotation. However, the highly resultant dimensionality in the eigenstate space causing from the EP-based system is sensitivity to more external perturbations, that is, the great response at EPs requires strict implementation conditions. To solve this problem, a new non-Hermitian structure was proposed that realizes an exceptional surface (EP surface) constructed of numerous EPs embedded in a high-dimensional parameter space. With respect to the isolated EPs, non-Hermitian EP surface provides additional degrees of freedom to permit the undesired perturbations (such as fabrication errors) shifts along the EP surface which corresponds to the robustness. On the other hand, the rotation will force the system away from these EPs obtaining a sensitivity enhancement of three orders of magnitude, compared to the traditional Sagnac effect rotation detecting. The EP surface system has a potential to combine robustness with enhanced sensitivity of the rotation.

physics.optics↗

arXiv2020

A Convolutional Neural Network-based Patent Image Retrieval Method for Design Ideation

The patent database is often used in searches of inspirational stimuli for innovative design opportunities because of its large size, extensive variety and rich design information in patent documents. However, most patent mining research only focuses on textual information and ignores visual information. Herein, we propose a convolutional neural network (CNN)-based patent image retrieval method. The core of this approach is a novel neural network architecture named Dual-VGG that is aimed to accomplish two tasks: visual material type prediction and international patent classification (IPC) class label prediction. In turn, the trained neural network provides the deep features in the image embedding vectors that can be utilized for patent image retrieval and visual mapping. The accuracy of both training tasks and patent image embedding space are evaluated to show the performance of our model. This approach is also illustrated in a case study of robot arm design retrieval. Compared to traditional keyword-based searching and Google image searching, the proposed method discovers more useful visual information for engineering design.

cs.CV↗

arXiv2020

High-sensitivity bio-sensor based on the real-splitting indirectly coupled Anti-Parity time symmetric WGMs

Detecting the size of single nanoparticle with high precision is crucial to understanding the characteristic of the nanoparticle. In this paper, we research the single particle detection based on the Anti-parity time symmetric (APT) indirectly coupled WGMs. The results show that the Anti-parity time symmetric WGM nanoparticle sensor exhibits giant enhancement in frequency splitting compared with single WGM sensor, when the system operating at exceptional point (EP). With respect to the parity-time symmetric nanoparticle sensor, our research exhibits a real eigenfrequency splitting, which can be directly detected.

physics.optics↗

arXiv2016

On the Statistical Analysis of Practical SPARQL Queries

In this paper, we analyze some basic features of SPARQL queries coming from our practical world in a statistical way. These features include three statistic features such as the occurrence frequency of triple patterns, fragments, well-designed patterns and four semantic features such as monotonicity, non-monotonicity, weak monotonicity (old solutions are still served as parts of new solutions when some new triples are added) and satisfiability. All these features contribute to characterize SPARQL queries in different dimensions. We hope that this statistical analysis would provide some useful observation for researchers and engineers who are interested in what practical SPARQL queries look like, so that they could develop some practical heuristics for processing SPARQL queries and build SPARQL query processing engines and benchmarks. Besides, they can narrow the scope of their problems by avoiding those cases that do possibly not happen in our practical world.

cs.DB↗