arXiv subjects

Wentao Yu

Publications and source records attributed to Wentao Yu.

45 records · Page 3Linked to original sources

arXiv2021

Know Your Surroundings: Panoramic Multi-Object Tracking by Multimodality Collaboration

In this paper, we focus on the multi-object tracking (MOT) problem of automatic driving and robot navigation. Most existing MOT methods track multiple objects using a singular RGB camera, which are prone to camera field-of-view and suffer tracking failures in complex scenarios due to background clutters and poor light conditions. To meet these challenges, we propose a MultiModality PAnoramic multi-object Tracking framework (MMPAT), which takes both 2D panorama images and 3D point clouds as input and then infers target trajectories using the multimodality data. The proposed method contains four major modules, a panorama image detection module, a multimodality data fusion module, a data association module and a trajectory inference model. We evaluate the proposed method on the JRDB dataset, where the MMPAT achieves the top performance in both the detection and tracking tasks and significantly outperforms state-of-the-art methods by a large margin (15.7 and 8.5 improvement in terms of AP and MOTA, respectively).

cs.CV↗

arXiv2021

Fusing information streams in end-to-end audio-visual speech recognition

End-to-end acoustic speech recognition has quickly gained widespread popularity and shows promising results in many studies. Specifically the joint transformer/CTC model provides very good performance in many tasks. However, under noisy and distorted conditions, the performance still degrades notably. While audio-visual speech recognition can significantly improve the recognition rate of end-to-end models in such poor conditions, it is not obvious how to best utilize any available information on acoustic and visual signal quality and reliability in these models. We thus consider the question of how to optimally inform the transformer/CTC model of any time-variant reliability of the acoustic and visual information streams. We propose a new fusion strategy, incorporating reliability information in a decision fusion net that considers the temporal effects of the attention mechanism. This approach yields significant improvements compared to a state-of-the-art baseline model on the Lip Reading Sentences 2 and 3 (LRS2 and LRS3) corpus. On average, the new system achieves a relative word error rate reduction of 43% compared to the audio-only setup and 31% compared to the audiovisual end-to-end baseline.

eess.AS↗

arXiv2020

Multimodal Integration for Large-Vocabulary Audio-Visual Speech Recognition

For many small- and medium-vocabulary tasks, audio-visual speech recognition can significantly improve the recognition rates compared to audio-only systems. However, there is still an ongoing debate regarding the best combination strategy for multi-modal information, which should allow for the translation of these gains to large-vocabulary recognition. While an integration at the level of state-posterior probabilities, using dynamic stream weighting, is almost universally helpful for small-vocabulary systems, in large-vocabulary speech recognition, the recognition accuracy remains difficult to improve. In the following, we specifically consider the large-vocabulary task of the LRS2 database, and we investigate a broad range of integration strategies, comparing early integration and end-to-end learning with many versions of hybrid recognition and dynamic stream weighting. One aspect, which is shown to provide much benefit here, is the use of dynamic stream reliability indicators, which allow for hybrid architectures to strongly profit from the inclusion of visual information whenever the audio channel is distorted even slightly.

eess.AS↗

arXiv2020

Intermittent "Turbulence" in a Many-body System

In natural settings, intermittent dynamics are ubiquitous and often arise from a coupling between external driving and spatial heterogeneities. A well-known example is the generation of transient, turbulent puffs of fluid through a pipe with rough walls. Here we show how similar dynamics can emerge in a discrete, crystalline system of particles driven by noise. Polydispersity in particle masses leads to localized vibrational modes that effectuate a transition to a gas-like phase. A minimal model for the evolution of the system's mechanical energies exhibits quasi-cyclic oscillations, and a single, dimensionless number captures the essential features of the intermittent dynamics, analogous to the Reynolds number for pipe flow.

nlin.AO↗

arXiv2016

High-resolution Collinear Chiral Sum Frequency Generation Microscopy by Using Vectorial Beam

In chiral sum frequency generation (C-SFG), the chiral nature of $χ^{(2)}$ requires the three involved electric fields to be pairwise non-parallel, leading to the traditional non-collinear configuration which is a hindrance for achieving diffraction limited resolution while utilizing it as a label-free imaging contrast mechanism . Here we propose a collinear C-SFG (CC-SFG) microscopy modality by using longitudinal z-polarized vectorial field. Label-free chiral imaging with enhanced spatial resolution (~1.4 times improvement in one lateral and the longitudinal directions over the traditional non-collinear scheme) is demonstrated, providing a new path for SFG microscopy with diffraction-limited resolution for mapping chirality.

physics.optics↗

arXiv2015

Super-resolution deep imaging with hollow Bessel beam STED microscopy

Stimulated emission depletion (STED) microscopy has become a powerful imaging and localized excitation method beating the diffraction barrier for improved lateral spatial resolution in cellular imaging, lithography, etc. Due to specimen-induced aberrations and scattering distortion, it has been a great challenge for STED to maintain consistent lateral resolution deeply inside the specimens. Here we report on a deep imaging STED microscopy by using Gaussian beam for excitation and hollow Bessel beam for depletion (GB-STED). The proposed scheme shows the improved imaging depth up to ~155μm in solid agarose sample, ~115μm in PDMS and ~100μm in phantom of gray matter in brain tissue with consistent super resolution, while the standard STED microscopy shown a significantly reduced lateral resolution at the same imaging depth. The results indicate the excellent imaging penetration capability of GB-STED, making it a promising tool for deep 3D imaging optical nanoscopy and laser fabrication.

physics.optics↗

arXiv2010

Novel Multifunctional Materials Based on Oxide Thin Films and Artificial Heteroepitaxial Multilayers

Transition metal oxides show fascinating physical properties such as high temperature superconductivity, ferro- and antiferromagnetism, ferroelectricity or even multiferroicity. The enormous progress in oxide thin film technology allows us to integrate these materials with semiconducting, normal conducting, dielectric or non-linear optical oxides in complex oxide heterostructures, providing the basis for novel multi-functional materials and various device applications. Here, we report on the combination of ferromagnetic, semiconducting, metallic, and dielectric materials properties in thin films and artificial heterostructures using laser molecular beam epitaxy. We discuss the fabrication and characterization of oxide-based ferromagnetic tunnel junctions, transition metal-doped semiconductors, intrinsic multiferroics, and artificial ferroelectric/ferromagetic heterostructures - the latter allow for the detailed study of strain effects, forming the basis of spin-mechanics. For characterization we use X-ray diffraction, SQUID magnetometry, magnetotransport measurements, and advanced methods of transmission electron microscopy with the goal to correlate macroscopic physical properties with the microstructure of the thin films and heterostructures.

cond-mat.mtrl-sci↗

arXiv2009

The relation between 13CO(2-1) line width in molecular clouds and bolometric luminosity of associated IRAS sources

We search for evidence of a relation between properties of young stellar objects (YSOs) and their parent molecular clouds to understand the initial conditions of high-mass star formation. A sample of 135 sources was selected from the Infrared Astronomical Satellite (IRAS) Point Source Catalog, on the basis of their red color to enhance the possibility of discovering young sources. Using the Kolner Observatorium fur SubMillimeter Astronomie (KOSMA) 3-m telescope, a single-point survey in 13CO(2-1) was carried out for the entire sample, and 14 sources were mapped further. Archival mid-infrared (MIR) data were compared with the 13CO emissions to identify evolutionary stages of the sources. A 13CO observed sample was assembled to investigate the correlation between 13CO line width of the clouds and the luminosity of the associated YSOs. We identified 98 sources suitable for star formation analyses for which relevant parameters were calculated. We detected 18 cores from 14 mapped sources, which were identified with eight pre-UC HII regions and one UC HII region, two high-mass cores earlier than pre-UC HII phase, four possible star forming clusters, and three sourceless cores. By compiling a large (360 sources) 13CO observed sample, a good correlation was found between the 13CO line width of the clouds and the bolometric luminosity of the associated YSOs, which can be fitted as a power law: lg(dV13/km/s)=-0.023+0.135lg(Lbol/Lsolar). Results show that luminous (>10^3Lsolar) YSOs tend to be associated with both more massive and more turbulent (dV13>2km/s) molecular cloud structures.

astro-ph.SR↗

arXiv2004

A study of high velocity molecular outflows with an up-to-date sample

A statistical study of the properties of molecular outflows is performed based on an up-to-date sample. 391 outflows were identified in published articles or preprints before February 28, 2003. The parameters of position, morphology, mass, energy, outflow dynamics and central source luminosity are presented for each outflow source. Outflow lobe polarity is known for all the sources, and 84% are found to be bipolar. The sources are divided into low mass and high mass groups according to either the available bolometric luminosity of the central source or the outflow mass.Energetic mass ejection may be a common aspect of the formation of high mass as well as low mass stars. Outflow masses are correlated strongly with bolometric luminosity of the center sources, which was obtained for the first time. There are also correlations between the central source luminosity and the parameters of mechanical luminosity and the thrust or force necessary to drive the outflow. The results show that flow mass, momentum and energy depend on the nature of the central source. Despite their similarity, there are differences between the high mass and low mass outflows on collimation, flow masses, mass-dynamical time relation and associated objects such as HH objects and water masers.Sources with characteristics of collapse or infall comprise 12% of the entire outflow sample. The spatial distribution of the outflow sources in the Galaxy is presented and the local occurrence rate is compared with the stellar birth rate.

astro-ph↗