arXiv subjects

Jin Li

Publications and source records attributed to Jin Li.

At least 163 records · Page 9Linked to original sources

arXiv2024

Open-Vocabulary Scene Text Recognition via Pseudo-Image Labeling and Margin Loss

Scene text recognition is an important and challenging task in computer vision. However, most prior works focus on recognizing pre-defined words, while there are various out-of-vocabulary (OOV) words in real-world applications. In this paper, we propose a novel open-vocabulary text recognition framework, Pseudo-OCR, to recognize OOV words. The key challenge in this task is the lack of OOV training data. To solve this problem, we first propose a pseudo label generation module that leverages character detection and image inpainting to produce substantial pseudo OOV training data from real-world images. Unlike previous synthetic data, our pseudo OOV data contains real characters and backgrounds to simulate real-world applications. Secondly, to reduce noises in pseudo data, we present a semantic checking mechanism to filter semantically meaningful data. Thirdly, we introduce a quality-aware margin loss to boost the training with pseudo data. Our loss includes a margin-based part to enhance the classification ability, and a quality-aware part to penalize low-quality samples in both real and pseudo data. Extensive experiments demonstrate that our approach outperforms the state-of-the-art on eight datasets and achieves the first rank in the ICDAR2022 challenge.

cs.CV↗

arXiv2024

Unsupervised Incremental Learning with Dual Concept Drift Detection for Identifying Anomalous Sequences

In the contemporary digital landscape, the continuous generation of extensive streaming data across diverse domains has become pervasive. Yet, a significant portion of this data remains unlabeled, posing a challenge in identifying infrequent events such as anomalies. This challenge is further amplified in non-stationary environments, where the performance of models can degrade over time due to concept drift. To address these challenges, this paper introduces a new method referred to as VAE4AS (Variational Autoencoder for Anomalous Sequences). VAE4AS integrates incremental learning with dual drift detection mechanisms, employing both a statistical test and a distance-based test. The anomaly detection is facilitated by a Variational Autoencoder. To gauge the effectiveness of VAE4AS, a comprehensive experimental study is conducted using real-world and synthetic datasets characterized by anomalous rates below 10\% and recurrent drift. The results show that the proposed method surpasses both robust baselines and state-of-the-art techniques, providing compelling evidence for their efficacy in effectively addressing some of the challenges associated with anomalous sequence detection in non-stationary streaming data.

cs.CE↗

arXiv2024

Integrated Optical Vortex Microcomb

The explorations of physical degrees of freedom with infinite dimensionalities, such as orbital angular momentum and frequency of light, have profoundly reshaped the landscape of modern optics with representative photonic functional devices including optical vortex emitters and frequency combs. In nanophotonics, whispering gallery mode microresonators naturally support orbital angular momentum of light and have been demonstrated as on-chip emitters of monochromatic optical vortices. On the other hand, whispering gallery mode microresonators serve as a highly efficient nonlinear optical platform for producing light at different frequencies - i.e., microcombs. Here, we interlace the optical vortices and microcombs by demonstrating an optical vortex comb on an III-V integrated nonlinear microresonator. The angular-grating-dressed nonlinear microring simultaneously emits spatiotemporal light springs consisting of 50 orbital angular momentum modes that are each spectrally addressed to the frequency components (longitudinal whispering gallery modes) of the generated microcomb. We further experimentally generate optical pulses with time-varying orbital angular momenta by carefully introducing a specific intermodal phase relation to spatiotemporal light springs. This work may immediately boost the development of integrated nonlinear/quantum photonics for exploring fundamental optical physics and advancing photonic quantum technology.

physics.optics↗

arXiv2024

ContriMix: Scalable stain color augmentation for domain generalization without domain labels in digital pathology

Differences in staining and imaging procedures can cause significant color variations in histopathology images, leading to poor generalization when deploying deep-learning models trained from a different data source. Various color augmentation methods have been proposed to generate synthetic images during training to make models more robust, eliminating the need for stain normalization during test time. Many color augmentation methods leverage domain labels to generate synthetic images. This approach causes three significant challenges to scaling such a model. Firstly, incorporating data from a new domain into deep-learning models trained on existing domain labels is not straightforward. Secondly, dependency on domain labels prevents the use of pathology images without domain labels to improve model performance. Finally, implementation of these methods becomes complicated when multiple domain labels (e.g., patient identification, medical center, etc) are associated with a single image. We introduce ContriMix, a novel domain label free stain color augmentation method based on DRIT++, a style-transfer method. Contrimix leverages sample stain color variation within a training minibatch and random mixing to extract content and attribute information from pathology images. This information can be used by a trained ContriMix model to create synthetic images to improve the performance of existing classifiers. ContriMix outperforms competing methods on the Camelyon17-WILDS dataset. Its performance is consistent across different slides in the test set while being robust to the color variation from rare substances in pathology images. We make our code and trained ContriMix models available for research use. The code for ContriMix can be found at https://gitlab.com/huutan86/contrimix

eess.IV↗

arXiv2024

From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Models

Multi-modal Large Language Models (MLLMs) have made significant strides in expanding the capabilities of Large Language Models (LLMs) through the incorporation of visual perception interfaces. Despite the emergence of exciting applications and the availability of diverse instruction tuning data, existing approaches often rely on CLIP or its variants as the visual branch, and merely extract features from the deep layers. However, these methods lack a comprehensive analysis of the visual encoders in MLLMs. In this paper, we conduct an extensive investigation into the effectiveness of different vision encoders within MLLMs. Our findings reveal that the shallow layer features of CLIP offer particular advantages for fine-grained tasks such as grounding and region understanding. Surprisingly, the vision-only model DINO, which is not pretrained with text-image alignment, demonstrates promising performance as a visual branch within MLLMs. By simply equipping it with an MLP layer for alignment, DINO surpasses CLIP in fine-grained related perception tasks. Building upon these observations, we propose a simple yet effective feature merging strategy, named COMM, that integrates CLIP and DINO with Multi-level features Merging, to enhance the visual capabilities of MLLMs. We evaluate COMM through comprehensive experiments on a wide range of benchmarks, including image captioning, visual question answering, visual grounding, and object hallucination. Experimental results demonstrate the superior performance of COMM compared to existing methods, showcasing its enhanced visual capabilities within MLLMs.

cs.CV↗

arXiv2024

Understanding Missingness in Time-series Electronic Health Records for Individualized Representation

With the widespread of machine learning models for healthcare applications, there is increased interest in building applications for personalized medicine. Despite the plethora of proposed research for personalized medicine, very few focus on representing missingness and learning from the missingness patterns in time-series Electronic Health Records (EHR) data. The lack of focus on missingness representation in an individualized way limits the full utilization of machine learning applications towards true personalization. In this brief communication, we highlight new insights into patterns of missingness with real-world examples and implications of missingness in EHRs. The insights in this work aim to bridge the gap between theoretical assumptions and practical observations in real-world EHRs. We hope this work will open new doors for exploring directions for better representation in predictive modelling for true personalization.

cs.LG↗

arXiv2024

Entropies of the various components of the universe

In this work, we study the entropies of photons, dust (baryonic matter), dark matter, and dark energy in the context of cosmology. When these components expand freely with the universe, we calculate the entropy and specific entropy of each component from the perspective of statistics. Under specific assumptions and conditions, the entropies of these components can satisfy the second law of thermodynamics independently. Our calculations show that the specific entropy of matter cannot be a constant during the expansion of the universe except for photons. When these components interact with the space-time background, there could exist the phenomenon of particle production (annihilation). We study the influence of the interaction on the entropies of these components, and obtain the conditions guaranteeing that the entropy of each component satisfies the second law of thermodynamics.

gr-qc↗

arXiv2024

Observational appearance and additional photon rings of the asymmetric thin-shell wormhole in Horndeski theory

In this paper, we study the observational appearance of the asymmetric thin-shell wormhole (ATW) in Horndeski theory by employing the ray-tracing method. We first calculate the effective potential and null geodesic of the ATW, and then we obtain the deflection angle of the photon in the ATW spacetime. Based on the impact parameter of the photon, the trajectory of the photon can be classified into three cases. Two typical emission models of the thin accretion disk are considered to analyze the observational appearance of the ATW. By comparing the observational appearances of the ATW and a black hole with the same mass parameter, we find additional features in the observational appearance of the ATW, such as the ``lensing band" and ``photon ring group".

gr-qc↗

arXiv2024

Understanding the predication mechanism of deep learning through error propagation among parameters in strong lensing case

The error propagation among estimated parameters reflects the correlation among the parameters. We study the capability of machine learning of "learning" the correlation of estimated parameters. We show that machine learning can recover the relation between the uncertainties of different parameters, especially, as predicted by the error propagation formula. Gravitational lensing can be used to probe both astrophysics and cosmology. As a practical application, we show that the machine learning is able to intelligently find the error propagation among the gravitational lens parameters (effective lens mass $M_{L}$ and Einstein radius $θ_{E}$) in accordance with the theoretical formula for the singular isothermal ellipse (SIE) lens model. The relation of errors of lens mass and Einstein radius, (e.g. the ratio of standard deviations $\mathcal{F}=σ_{\hat{ M_{L}}}/ σ_{\hat{θ_{E}}}$) predicted by the deep convolution neural network are consistent with the error propagation formula of SIE lens model. As a proof-of-principle test, a toy model of linear relation with Gaussian noise is presented. We found that the predictions obtained by machine learning indeed indicate the information about the law of error propagation and the distribution of noise. Error propagation plays a crucial role in identifying the physical relation among parameters, rather than a coincidence relation, therefore we anticipate our case study on the error propagation of machine learning predictions could extend to other physical systems on searching the correlation among parameters.

astro-ph.GA↗

arXiv2023

GanFinger: GAN-Based Fingerprint Generation for Deep Neural Network Ownership Verification

Deep neural networks (DNNs) are extensively employed in a wide range of application scenarios. Generally, training a commercially viable neural network requires significant amounts of data and computing resources, and it is easy for unauthorized users to use the networks illegally. Therefore, network ownership verification has become one of the most crucial steps in safeguarding digital assets. To verify the ownership of networks, the existing network fingerprinting approaches perform poorly in the aspects of efficiency, stealthiness, and discriminability. To address these issues, we propose a network fingerprinting approach, named as GanFinger, to construct the network fingerprints based on the network behavior, which is characterized by network outputs of pairs of original examples and conferrable adversarial examples. Specifically, GanFinger leverages Generative Adversarial Networks (GANs) to effectively generate conferrable adversarial examples with imperceptible perturbations. These examples can exhibit identical outputs on copyrighted and pirated networks while producing different results on irrelevant networks. Moreover, to enhance the accuracy of fingerprint ownership verification, the network similarity is computed based on the accuracy-robustness distance of fingerprint examples'outputs. To evaluate the performance of GanFinger, we construct a comprehensive benchmark consisting of 186 networks with five network structures and four popular network post-processing techniques. The benchmark experiments demonstrate that GanFinger significantly outperforms the state-of-the-arts in efficiency, stealthiness, and discriminability. It achieves a remarkable 6.57 times faster in fingerprint generation and boosts the ARUC value by 0.175, resulting in a relative improvement of about 26%.

cs.CR↗

arXiv2023

Subtraction of the confusion foreground and parameter uncertainty of resolvable galactic binaries on the networks of space-based gravitational-wave detectors

There are tens of millions of compact binary systems in the Milky Way, called galactic binaries (GBs), most of which are unresolved, and the Gravitational waves (GWs) emitted overlap to form foreground confusion. By simulating such foreground confusion, we have studied how LISA, Taiji and TianQin, including their alternative orbital configurations, subtract resolvable GBs when they combine as some networks. The results of our research indicate that the order of detected number for a single detector from high to low is: Taiji-m, Taiji-p (c), LISA, TianQin I, TianQin II. For detector combinations on the network, the foreground confusion is effectively reduced as the number of detectors grows, and the optimal combinations with different numbers are: Taiji-m, LISA+Taiji-m, LISA+Taiji-m+TianQin I, and LISA+Taiji-m+TianQin I+II. The sensitivity curve is optimized as the number of detectors increases, which renders it possible to detect other gravitational wave sources more precisely and decrease the resolvable GBs parameter uncertainty. Based on this, we discuss the parameter uncertainty of resolvable GBs detected by the combinations above and find that GW detection can promote electromagnetic (EM) detection. On the contrary, we discovered that by utilizing EM detection, determining the inclination angle can reduce the uncertainty of GW strain amplitude by $\sim$93%, and determining the sky position can reduce the uncertainty of the phase by $\sim$30%, further strengthening the connection between GW detection and EM detection, and contributing to the research of Multi-messenger astronomy.

gr-qc↗

arXiv2023

Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNs

Despite Graph neural networks' significant performance gain over many classic techniques in various graph-related downstream tasks, their successes are restricted in shallow models due to over-smoothness and the difficulties of optimizations among many other issues. In this paper, to alleviate the over-smoothing issue, we propose a soft graph normalization method to preserve the diversities of node embeddings and prevent indiscrimination due to possible over-closeness. Combined with residual connections, we analyze the reason why the method can effectively capture the knowledge in both input graph structures and node features even with deep networks. Additionally, inspired by Curriculum Learning that learns easy examples before the hard ones, we propose a novel label-smoothing-based learning framework to enhance the optimization of deep GNNs, which iteratively smooths labels in an auxiliary graph and constructs many gradual non-smooth tasks for extracting increasingly complex knowledge and gradually discriminating nodes from coarse to fine. The method arguably reduces the risk of overfitting and generalizes better results. Finally, extensive experiments are carried out to demonstrate the effectiveness and potential of the proposed model and learning framework through comparison with twelve existing baselines including the state-of-the-art methods on twelve real-world node classification benchmarks.

cs.LG↗

arXiv2023

Deep learning forecasts of cosmic acceleration parameters from DECi-hertz Interferometer Gravitational-wave Observatory

Validating the accelerating expansion of the universe is an important issue for understanding the evolution of the universe. By constraining the cosmic acceleration parameter $X_H$, we can discriminate between the $Λ\mathrm{CDM}$ (cosmological constant plus cold dark matter) model and LTB (the Lemaître-Tolman-Bondi) model. In this paper, we explore the possibility of constraining the cosmic acceleration parameter with the inspiral gravitational waveform of neutron star binaries (NSBs) in the frequency range of 0.1Hz-10Hz, which can be detected by the second-generation space-based gravitational wave detector DECIGO. We use a convolutional neural network (CNN), a long short-term memory (LSTM) network combined with a gated recurrent unit (GRU), and Fisher information matrix to derive constraints on the cosmic acceleration parameter $X_H$. Based on the simulated gravitational wave data with a time duration of 1 month, we conclude that CNN can limit the relative error to 14.09%, while LSTM network combined with GRU can limit the relative error to 13.53%. Additionally, using Fisher information matrix for gravitational wave data with a 5-year observation can limit the relative error to 32.94%. Compared with the Fisher information matrix method, deep learning techniques will significantly improve the constraints on the cosmic acceleration parameters at different redshifts. Therefore, DECIGO is expected to provide direct measurements of the acceleration of the universe, by observing the chirp signals of coalescing binary neutron stars.

astro-ph.GA↗

arXiv2023

Parameter Estimation for Intermediate-Mass Binary Black Holes through Gravitational Waves Observed by DECIGO

With the anticipated launch of space-based gravitational wave detectors, including LISA, TaiJi, TianQin, and DECIGO, expected around 2030, the detection of gravitational waves generated by intermediate-mass black hole binaries (IMBBHs) becomes a tangible prospect. However, due to the detector's reception of a substantial amount of non-Gaussian, non-stationary data, employing traditional Bayesian inference methods for parameter estimation would result in significant resource demands and limitations in the waveform template library. Therefore, in this paper, we simulated foreground noise induced by stellar-origin binary black holes (SOBBHs), which is non-Gaussian and non-stationary, and we explore the use of Gaussian process regression (GPR) and deep learning for parameter estimation of Intermediate Mass Binary Black Holes (IMBBHs) in the presence of such non-Gaussian, non-stationary background noise. By comparing these results from deep learning and GPR, we demonstrate that deep learning can offer improved precision in parameter estimation compared to traditional GPR. Furthermore, compared to GPR, deep learning can provide posterior distributions of the sample parameters faster.

gr-qc↗

arXiv2023

The ability of Lisa, Taiji, and their networks to detect the stochastic gravitational wave background generated by Cosmic Strings

The cosmic string contributes to our understanding and revelation of the fundamental structure and evolutionary patterns of the universe, unifying our knowledge of the cosmos and unveiling new physical laws and phenomena. Therefore, we anticipate the detection of Stochastic Gravitational Wave Background (SGWB) signals generated by cosmic strings in space-based detectors. We have analyzed the detection capabilities of individual space-based detectors, Lisa and Taiji, as well as the joint space-based detector network, Lisa-Taiji, for SGWB signals produced by cosmic strings, taking into account other astronomical noise sources. The results indicate that the Lisa-Taiji network exhibits superior capabilities in detecting SGWB signals generated by cosmic strings and can provide strong evidence. The Lisa-Taiji network can achieve an uncertainty estimation of $ΔGμ/Gμ<0.5$ for cosmic string tension $Gμ\sim4\times10^{-17}$, and can provide evidence for the presence of SGWB signals generated by cosmic strings at $Gμ\sim10^{-17}$, and strong evidence at $Gμ\sim10^{-16}$. Even in the presence of only SGWB signals, it can achieve a relative uncertainty of $ΔGμ/Gμ<0.5$ for cosmic string tension $Gμ<10^{-18}$, and provide strong evidence at $Gμ\sim10^{-17}$.

astro-ph.CO↗

arXiv2023

AiluRus: A Scalable ViT Framework for Dense Prediction

Vision transformers (ViTs) have emerged as a prevalent architecture for vision tasks owing to their impressive performance. However, when it comes to handling long token sequences, especially in dense prediction tasks that require high-resolution input, the complexity of ViTs increases significantly. Notably, dense prediction tasks, such as semantic segmentation or object detection, emphasize more on the contours or shapes of objects, while the texture inside objects is less informative. Motivated by this observation, we propose to apply adaptive resolution for different regions in the image according to their importance. Specifically, at the intermediate layer of the ViT, we utilize a spatial-aware density-based clustering algorithm to select representative tokens from the token sequence. Once the representative tokens are determined, we proceed to merge other tokens into their closest representative token. Consequently, semantic similar tokens are merged together to form low-resolution regions, while semantic irrelevant tokens are preserved independently as high-resolution regions. This strategy effectively reduces the number of tokens, allowing subsequent layers to handle a reduced token sequence and achieve acceleration. We evaluate our proposed method on three different datasets and observe promising performance. For example, the "Segmenter ViT-L" model can be accelerated by 48% FPS without fine-tuning, while maintaining the performance. Additionally, our method can be applied to accelerate fine-tuning as well. Experimental results demonstrate that we can save 52% training time while accelerating 2.46 times FPS with only a 0.09% performance drop. The code is available at https://github.com/caddyless/ailurus/tree/main.

cs.CV↗

arXiv2023

Electromagnetic counterparts of high-frequency gravitational waves in a rotating laboratory frame system and their detection

We consider the perturbative photon flows (PPFs, i.e., electromagnetic (EM) counterparts) generated by the EM resonance response to high-frequency gravitational waves (HFGWs) with additional polarization states in a rotating laboratory frame system. It is found that when the propagating direction of HFGWs and the symmetrical axis of the laboratory frame system are the same, the PPFs have the maximum value. In this case, using the rotation (the rotation of the azimuth $ϕ$) of the EM detection system, all six possible polarization states of HFGWs can be separated and displayed. For the current experimental conditions, it is quite prospective to detect the PPFs generated by the HFGWs predicted in the braneworld models, the primordial black hole theories, the interaction mechanism between astrophysical plasma and intense EM radiation, etc., due to the large amplitudes (or high spectral densities) and spectral characteristics of these HFGWs. Detecting the primordial HFGWs from inflation faces great challenges at present, but it is not impossible.

gr-qc↗

arXiv2023

Testing the spatial geometry of the universe with TianQin: the prospect of using supermassive black hole binaries

The determination of the spatial geometry of the universe plays an important role in modern cosmology. Any deviation from the cosmic curvature $Ω_K=0$ would have a profound impact on the primordial inflation paradigm and fundamental physics. In this paper, we carry out a systematic study of the prospect of measuring cosmic curvature with the inspiral signal of supermassive black hole binaries (SMBHBs) that could be detected with TianQin. The study is based on a cosmological-model-independent method that extended the application of gravitational wave (GW) standard sirens in cosmology. By comparing the distances from future simulated GW events and simulated $H(z)$ data, we evaluate if TianQin would produce robust constraints on the cosmic curvature parameter $Ω_{k}$. More specifically, we consider 3-yr to 10-yr observations of supermassive black hole binaries with total masses ranging from $10^{3}M_\odot$ to $10^{7}M_\odot$. Our results show that in the future, with the synergy of 10-yr high-quality observations, we can tightly constrain the curvature parameter at the level of $1σ$ $Ω_k=-0.002\pm0.061$. Moreover, our findings indicate that the total mass of SMBHB does influence the estimation of cosmic curvature, implied by the analysis performed on different subsamples of gravitational wave data. Therefore, TianQin is expected to provide a powerful and competitive probe of the spatial geometry of the universe, compared to future spaced-based detectors such as DECIGO.

astro-ph.CO↗