Search arXiv⌕ Search

arXiv subjects

Junjie Li

Publications and source records attributed to Junjie Li.

At least 127 records · Page 7Linked to original sources
arXiv2023

SoccerNet 2023 Challenges Results

The SoccerNet 2023 challenges were the third annual video understanding challenges organized by the SoccerNet team. For this third edition, the challenges were composed of seven vision-based tasks split into three main themes. The first theme, broadcast video understanding, is composed of three high-level tasks related to describing events occurring in the video broadcasts: (1) action spotting, focusing on retrieving all timestamps related to global actions in soccer, (2) ball action spotting, focusing on retrieving all timestamps related to the soccer ball change of state, and (3) dense video captioning, focusing on describing the broadcast with natural language and anchored timestamps. The second theme, field understanding, relates to the single task of (4) camera calibration, focusing on retrieving the intrinsic and extrinsic camera parameters from images. The third and last theme, player understanding, is composed of three low-level tasks related to extracting information about the players: (5) re-identification, focusing on retrieving the same players across multiple views, (6) multiple object tracking, focusing on tracking players and the ball through unedited video streams, and (7) jersey number recognition, focusing on recognizing the jersey number of players from tracklets. Compared to the previous editions of the SoccerNet challenges, tasks (2-3-7) are novel, including new annotations and data, task (4) was enhanced with more data and annotations, and task (6) now focuses on end-to-end approaches. More information on the tasks, challenges, and leaderboards are available on https://www.soccer-net.org. Baselines and development kits can be found on https://github.com/SoccerNet.

cs.CV↗
arXiv2023

Hydrogen jet diffusion modeling by using physics-informed graph neural network and sparsely-distributed sensor data

Efficient modeling of jet diffusion during accidental release is critical for operation and maintenance management of hydrogen facilities. Deep learning has proven effective for concentration prediction in gas jet diffusion scenarios. Nonetheless, its reliance on extensive simulations as training data and its potential disregard for physical laws limit its applicability to unseen accidental scenarios. Recently, physics-informed neural networks (PINNs) have emerged to reconstruct spatial information by using data from sparsely-distributed sensors which are easily collected in real-world applications. However, prevailing approaches use the fully-connected neural network as the backbone without considering the spatial dependency of sensor data, which reduces the accuracy of concentration prediction. This study introduces the physics-informed graph deep learning approach (Physic_GNN) for efficient and accurate hydrogen jet diffusion prediction by using sparsely-distributed sensor data. Graph neural network (GNN) is used to model the spatial dependency of such sensor data by using graph nodes at which governing equations describing the physical law of hydrogen jet diffusion are immediately solved. The computed residuals are then applied to constrain the training process. Public experimental data of hydrogen jet is used to compare the accuracy and efficiency between our proposed approach Physic_GNN and state-of-the-art PINN. The results demonstrate our Physic_GNN exhibits higher accuracy and physical consistency of centerline concentration prediction given sparse concentration compared to PINN and more efficient compared to OpenFOAM. The proposed approach enables accurate and robust real-time spatial consequence reconstruction and underlying physical mechanisms analysis by using sparse sensor data.

cs.CE↗
arXiv2023

Exploring the Potential of Integrated Optical Sensing and Communication (IOSAC) Systems with Si Waveguides for Future Networks

Advanced silicon photonic technologies enable integrated optical sensing and communication (IOSAC) in real time for the emerging application requirements of simultaneous sensing and communication for next-generation networks. Here, we propose and demonstrate the IOSAC system on the silicon nitride (SiN) photonics platform. The IOSAC devices based on microring resonators are capable of monitoring the variation of analytes, transmitting the information to the terminal along with the modulated optical signal in real-time, and replacing bulk optics in high-precision and high-speed applications. By directly integrating SiN ring resonators with optical communication networks, simultaneous sensing and optical communication are demonstrated by an optical signal transmission experimental system using especially filtering amplified spontaneous emission spectra. The refractive index (RI) sensing ring with a sensitivity of 172 nm/RIU, a figure of merit (FOM) of 1220, and a detection limit (DL) of 8.2*10-6 RIU is demonstrated. Simultaneously, the 1.25 Gbps optical on-off-keying (OOK) signal is transmitted at the concentration of different NaCl solutions, which indicates the bit-error-ratio (BER) decreases with the increase in concentration. The novel IOSAC technology shows the potential to realize high-performance simultaneous biosensing and communication in real time and further accelerate the development of IoT and 6G networks.

eess.SP↗
arXiv2023

Rethinking the visual cues in audio-visual speaker extraction

The Audio-Visual Speaker Extraction (AVSE) algorithm employs parallel video recording to leverage two visual cues, namely speaker identity and synchronization, to enhance performance compared to audio-only algorithms. However, the visual front-end in AVSE is often derived from a pre-trained model or end-to-end trained, making it unclear which visual cue contributes more to the speaker extraction performance. This raises the question of how to better utilize visual cues. To address this issue, we propose two training strategies that decouple the learning of the two visual cues. Our experimental results demonstrate that both visual cues are useful, with the synchronization cue having a higher impact. We introduce a more explainable model, the Decoupled Audio-Visual Speaker Extraction (DAVSE) model, which leverages both visual cues.

cs.SD↗
arXiv2023

Dual-band polarized upconversion photoluminescence enhanced by resonant dielectric metasurfaces

Lanthanide-doped upconversion nanoparticles emerged recently as an attractive material platform underpinning a broad range of innovative applications such as optical cryptography, luminescent probes, and lasing. However, the intricate 4f-associated electronic transition in upconversion nanoparticles leads only to a weak photoluminescence intensity and unpolarized emission, hindering many applications that demand ultrabright and polarized light sources. Here, we uncover a new strategy for achieving ultrabright and dual-band polarized upconversion photoluminescence. We employ resonant dielectric metasurfaces supporting high-quality resonant modes at dual upconversion bands enabling two-order-of-magnitude amplification of upconversion emissions. We demonstrate that dual-band resonances can be selectively switched on polarization, endowing cross-polarization controlled upconversion luminescence with ultra-high degrees of polarization, reaching approximately 0.86 and 0.91 at dual emission wavelengths of 540 nm and 660 nm, respectively. Our strategy offers an effective approach for enhancing photon upconversion processes paving the way toward efficient low-threshold polarization upconversion lasers.

physics.optics↗
arXiv2023

Phase Field Model for Non-equilibrium Interface Conditions

This article presents a new phase-field formulation for non-equilibrium interface conditions in rapid phase transformations. With a particular way of defining concentration fields, the classical sharp and diffuse (thick) interface theories are unified into the present phase-field model. Each point inside the diffuse interface is comparable to the classical sharp interface model, governed by phase boundary migration and short-range atomic exchanges and connected by long-range diffusion. Their thermodynamic forces and fluxes are following the Onsager reciprocal relation and consistent with the classical irreversible thermodynamics. Furthermore, we establish the balance of driving forces and energy dissipations for both the diffuse interface and the single point inside it. The model is then validated by rapid solidification of Al-Cu alloys. With an effective mobility of considering non-equilibrium long-range diffusion, the model represents the main characteristics of non-equilibrium interface kinetics, i.e., solute trapping and solute drag. Especially, we reproduce the complete trapping at the high-velocity regime and provide a thermodynamic consistent description of the partial drag phenomena at the low-velocity region. Moreover, the dissipation analysis indicates that even the far-from-equilibrium diffuse interface is composed of numerous representative volume elements near the equilibrium, providing a new understanding of non-equilibrium interfaces.

cond-mat.mtrl-sci↗
arXiv2023

Detecting Out-of-distribution Examples via Class-conditional Impressions Reappearing

Out-of-distribution (OOD) detection aims at enhancing standard deep neural networks to distinguish anomalous inputs from original training data. Previous progress has introduced various approaches where the in-distribution training data and even several OOD examples are prerequisites. However, due to privacy and security, auxiliary data tends to be impractical in a real-world scenario. In this paper, we propose a data-free method without training on natural data, called Class-Conditional Impressions Reappearing (C2IR), which utilizes image impressions from the fixed model to recover class-conditional feature statistics. Based on that, we introduce Integral Probability Metrics to estimate layer-wise class-conditional deviations and obtain layer weights by Measuring Gradient-based Importance (MGI). The experiments verify the effectiveness of our method and indicate that C2IR outperforms other post-hoc methods and reaches comparable performance to the full access (ID and OOD) detection method, especially in the far-OOD dataset (SVHN).

cs.LG↗
arXiv2023

Thermodynamics of Non-equilibrium Diffuse-Interfaces in Mesoscale Phase Transformations

We present a new phase-field formulation for the non-equilibrium interface kinetics. The diffuse interface is considered an integral of numerous representative volume elements (RVEs), in which there is a two-phase mixture with two conserved and two non-conserved concentration fields. This particular way of separating concentration fields leads to two distinct dissipative processes besides the phase boundary migration, i.e., trans-sharp-interface diffusion between different phases within a single RVE and trans-diffuse-interface diffusion between different RVEs within a single phase. A two-part mechanism is proposed to balance the driving forces and energy dissipation for diffusionless and diffusional processes in RVEs. Moreover, the classical interface friction and solute drag can be recovered by integrating the diffuse interface. Compared with the sharp interface theory, the present diffuse interface can reproduce the atomic simulated partial-drag self-consistently in thermodynamics.

cond-mat.mtrl-sci↗
arXiv2023

Exploiting the Textual Potential from Vision-Language Pre-training for Text-based Person Search

Text-based Person Search (TPS), is targeted on retrieving pedestrians to match text descriptions instead of query images. Recent Vision-Language Pre-training (VLP) models can bring transferable knowledge to downstream TPS tasks, resulting in more efficient performance gains. However, existing TPS methods improved by VLP only utilize pre-trained visual encoders, neglecting the corresponding textual representation and breaking the significant modality alignment learned from large-scale pre-training. In this paper, we explore the full utilization of textual potential from VLP in TPS tasks. We build on the proposed VLP-TPS baseline model, which is the first TPS model with both pre-trained modalities. We propose the Multi-Integrity Description Constraints (MIDC) to enhance the robustness of the textual modality by incorporating different components of fine-grained corpus during training. Inspired by the prompt approach for zero-shot classification with VLP models, we propose the Dynamic Attribute Prompt (DAP) to provide a unified corpus of fine-grained attributes as language hints for the image modality. Extensive experiments show that our proposed TPS framework achieves state-of-the-art performance, exceeding the previous best method by a margin.

cs.CV↗
arXiv2023

Video Semantic Segmentation with Inter-Frame Feature Fusion and Inner-Frame Feature Refinement

Video semantic segmentation aims to generate accurate semantic maps for each video frame. To this end, many works dedicate to integrate diverse information from consecutive frames to enhance the features for prediction, where a feature alignment procedure via estimated optical flow is usually required. However, the optical flow would inevitably suffer from inaccuracy, and then introduce noises in feature fusion and further result in unsatisfactory segmentation results. In this paper, to tackle the misalignment issue, we propose a spatial-temporal fusion (STF) module to model dense pairwise relationships among multi-frame features. Different from previous methods, STF uniformly and adaptively fuses features at different spatial and temporal positions, and avoids error-prone optical flow estimation. Besides, we further exploit feature refinement within a single frame and propose a novel memory-augmented refinement (MAR) module to tackle difficult predictions among semantic boundaries. Specifically, MAR can store the boundary features and prototypes extracted from the training samples, which together form the task-specific memory, and then use them to refine the features during inference. Essentially, MAR can move the hard features closer to the most likely category and thus make them more discriminative. We conduct extensive experiments on Cityscapes and CamVid, and the results show that our proposed methods significantly outperform previous methods and achieves the state-of-the-art performance. Code and pretrained models are available at https://github.com/jfzhuang/ST_Memory.

cs.CV↗
arXiv2022

Prediction of nonlinear interface dynamics in the unidirectional freezing of particle suspensions with rigid compacted layer

Water freezing in particle suspensions widely exists in nature. As a typical physical system of free boundary problem, the spatiotemporal evolution of the solid/liquid interface not only origins from phase transformation but also from permeation flow in front of ice. Physical models have been proposed in previous efforts to describe the interface dynamic behaviors in unidirectional freezing of particle suspensions. However, there are several physical parameters difficult to be determined in previous investigations dedicated to describing the spatiotemporal evolution in unidirectional freezing of particle suspensions. Here, based on the fundamental momentum theorem, we propose a consistent theoretical framework to address the unidirectional freezing process in the particle suspensions coupled with the effect of water permeation. An interface undercooling-dependent pushing force exerted on the compacted layer with a specific formula is derived based on the surface tension. Then a dynamic compacted layer is considered and analyzed. Numerical solutions of the nonlinear models reveal the dependence of system dynamics on some typical physical parameters, particle radius, initial particle concentration in the suspensions, freezing velocity and so on. The system dynamics are characterized by interface velocity, interface undercooling and interface recoil as functions of time. The models allow us to reconsider the formation mechanism of ice spears in freezing of particle suspensions in a simpler but novel way, with potential implications for both understanding and controlling not only ice formation in porous media but also crystallization processes in other complex systems.

cond-mat.mtrl-sci↗
arXiv2022

Dual-stage structural response to quenching charge order in magnetite

The Verwey transition in magnetite (Fe3O4 ) is the prototypical metal-insulator transition and has eluded a comprehensive explanation for decades. A major element of the challenge is the complex interplay between charge order and lattice distortions. Here we use ultrafast electron diffraction (UED) to disentangle the roles of charge order and lattice distortions by tracking the transient structural evolution after charge order is melted via ultrafast photoexcitation. A dual stage response is observed in which X3, X1, and Delta5 type structural distortions occur on markedly different timescales of 0.7 to 3.2 ps and longer than 3.2 ps. We propose that these distinct timescales arise because X3 type distortions strongly couple to the trimeron charge order, whereas the Delta5-distortions are more strongly associated with monoclinic to cubic distortions of the overall lattice. Our work aids in clarifying the charge lattice interplay using UED method and illustrates the disentanglement of the complex phases in magnetite.

cond-mat.str-el↗
arXiv2022

Robust Security Energy Efficiency Optimization for RIS-Aided Cell-Free Networks with Multiple Eavesdroppers

In this paper, we investigate the energy efficiency (EE) problem under reconfigurable intelligent surface (RIS)-aided secure cell-free networks, where multiple legitimate users and eavesdroppers (Eves) exist. We formulate a max-min secure EE optimization problem by jointly designing the distributed active beamforming and artificial noise at base stations as well as the passive beamforming at RISs under practical constraints. To deal with it, we first divide the original optimization problem into two sub-ones, and then propose an iterative optimization algorithm to solve each sub-problem based on the fractional programming, constrained convex-convex procedure (CCCP) and semi-definite programming (SDP) techniques. After that, these two sub-problems are alternatively solved until convergence, and the final solutions are obtained. Next, we extend to the imperfect channel state information of the Eves' links, and investigate the robust SEE beamforming optimization problem by bringing the outage probability constraints. Based on this, we first transform the uncertain outage probability constraints into the certain ones by the bernstein-type inequality and sphere boundary techniques, and then propose an alternatively iterative algorithm to obtain the solutions of the original problem based on the S-procedure, successive convex approximation, CCCP and SDP techniques. Finally, the simulation results are conducted to show the effectiveness of the proposed schemes.

cs.IT↗
arXiv2022

Efficient Cavity Searching for Gene Network of Influenza A Virus

High order structures (cavities and cliques) of the gene network of influenza A virus reveal tight associations among viruses during evolution and are key signals that indicate viral cross-species infection and cause pandemics. As indicators for sensing the dynamic changes of viral genes, these higher order structures have been the focus of attention in the field of virology. However, the size of the viral gene network is usually huge, and searching these structures in the networks introduces unacceptable delay. To mitigate this issue, in this paper, we propose a simple-yet-effective model named HyperSearch based on deep learning to search cavities in a computable complex network for influenza virus genetics. Extensive experiments conducted on a public influenza virus dataset demonstrate the effectiveness of HyperSearch over other advanced deep-learning methods without any elaborated model crafting. Moreover, HyperSearch can finish the search works in minutes while 0-1 programming takes days. Since the proposed method is simple and easy to be transferred to other complex networks, HyperSearch has the potential to facilitate the monitoring of dynamic changes in viral genes and help humans keep up with the pace of virus mutations.

q-bio.GN↗
arXiv2022

IE-GAN: An Improved Evolutionary Generative Adversarial Network Using a New Fitness Function and a Generic Crossover Operator

The training of generative adversarial networks (GANs) is usually vulnerable to mode collapse and vanishing gradients. The evolutionary generative adversarial network (E-GAN) attempts to alleviate these issues by optimizing the learning strategy with multiple loss functions. It uses a learning-based evolutionary framework, which develops new mutation operators specifically for general deep neural networks. However, the evaluation mechanism in the fitness function of E-GAN cannot truly reflect the adaptability of individuals to their environment, leading to an inaccurate assessment of the diversity of individuals. Moreover, the evolution step of E-GAN only contains mutation operators without considering the crossover operator jointly, isolating the superior characteristics among individuals. To address these issues, we propose an improved E-GAN framework called IE-GAN, which introduces a new fitness function and a generic crossover operator. In particular, the proposed fitness function, from an objective perspective, can model the evolutionary process of individuals more accurately. The crossover operator, which has been commonly adopted in evolutionary algorithms, can enable offspring to imitate the superior gene expression of their parents through knowledge distillation. Experiments on various datasets demonstrate the effectiveness of our proposed IE-GAN in terms of the quality of the generated samples and time efficiency.

cs.NE↗
arXiv2022

VCSE: Time-Domain Visual-Contextual Speaker Extraction Network

Speaker extraction seeks to extract the target speech in a multi-talker scenario given an auxiliary reference. Such reference can be auditory, i.e., a pre-recorded speech, visual, i.e., lip movements, or contextual, i.e., phonetic sequence. References in different modalities provide distinct and complementary information that could be fused to form top-down attention on the target speaker. Previous studies have introduced visual and contextual modalities in a single model. In this paper, we propose a two-stage time-domain visual-contextual speaker extraction network named VCSE, which incorporates visual and self-enrolled contextual cues stage by stage to take full advantage of every modality. In the first stage, we pre-extract a target speech with visual cues and estimate the underlying phonetic sequence. In the second stage, we refine the pre-extracted target speech with the self-enrolled contextual cues. Experimental results on the real-world Lip Reading Sentences 3 (LRS3) database demonstrate that our proposed VCSE network consistently outperforms other state-of-the-art baselines.

cs.CV↗
arXiv2022

SoccerNet 2022 Challenges Results

The SoccerNet 2022 challenges were the second annual video understanding challenges organized by the SoccerNet team. In 2022, the challenges were composed of 6 vision-based tasks: (1) action spotting, focusing on retrieving action timestamps in long untrimmed videos, (2) replay grounding, focusing on retrieving the live moment of an action shown in a replay, (3) pitch localization, focusing on detecting line and goal part elements, (4) camera calibration, dedicated to retrieving the intrinsic and extrinsic camera parameters, (5) player re-identification, focusing on retrieving the same players across multiple views, and (6) multiple object tracking, focusing on tracking players and the ball through unedited video streams. Compared to last year's challenges, tasks (1-2) had their evaluation metrics redefined to consider tighter temporal accuracies, and tasks (3-6) were novel, including their underlying data and annotations. More information on the tasks, challenges and leaderboards are available on https://www.soccer-net.org. Baselines and development kits are available on https://github.com/SoccerNet.

cs.CV↗
arXiv2022

Exploring High-quality Target Domain Information for Unsupervised Domain Adaptive Semantic Segmentation

In unsupervised domain adaptive (UDA) semantic segmentation, the distillation based methods are currently dominant in performance. However, the distillation technique requires complicate multi-stage process and many training tricks. In this paper, we propose a simple yet effective method that can achieve competitive performance to the advanced distillation methods. Our core idea is to fully explore the target-domain information from the views of boundaries and features. First, we propose a novel mix-up strategy to generate high-quality target-domain boundaries with ground-truth labels. Different from the source-domain boundaries in previous works, we select the high-confidence target-domain areas and then paste them to the source-domain images. Such a strategy can generate the object boundaries in target domain (edge of target-domain object areas) with the correct labels. Consequently, the boundary information of target domain can be effectively captured by learning on the mixed-up samples. Second, we design a multi-level contrastive loss to improve the representation of target-domain data, including pixel-level and prototype-level contrastive learning. By combining two proposed methods, more discriminative features can be extracted and hard object boundaries can be better addressed for the target domain. The experimental results on two commonly adopted benchmarks (\textit{i.e.}, GTA5 $\rightarrow$ Cityscapes and SYNTHIA $\rightarrow$ Cityscapes) show that our method achieves competitive performance to complicated distillation methods. Notably, for the SYNTHIA$\rightarrow$ Cityscapes scenario, our method achieves the state-of-the-art performance with $57.8\%$ mIoU and $64.6\%$ mIoU on 16 classes and 13 classes. Code is available at https://github.com/ljjcoder/EHTDI.

cs.CV↗