arXiv subjects

Xin Wang

Publications and source records attributed to Xin Wang.

At least 1,189 records · Page 66Linked to original sources

arXiv2022

Shared Loss between Generators of GANs

Generative adversarial networks are generative models that are capable of replicating the implicit probability distribution of the input data with high accuracy. Traditionally, GANs consist of a Generator and a Discriminator which interact with each other to produce highly realistic artificial data. Traditional GANs fall prey to the mode collapse problem, which means that they are unable to generate the different variations of data present in the input dataset. Recently, multiple generators have been used to produce more realistic output by mitigating the mode collapse problem. We use this multiple generator framework. The novelty in this paper lies in making the generators compete against each other while interacting with the discriminator simultaneously. We show that this causes a dramatic reduction in the training time for GANs without affecting its performance.

cs.LG↗

arXiv2022

LiDAL: Inter-frame Uncertainty Based Active Learning for 3D LiDAR Semantic Segmentation

We propose LiDAL, a novel active learning method for 3D LiDAR semantic segmentation by exploiting inter-frame uncertainty among LiDAR frames. Our core idea is that a well-trained model should generate robust results irrespective of viewpoints for scene scanning and thus the inconsistencies in model predictions across frames provide a very reliable measure of uncertainty for active sample selection. To implement this uncertainty measure, we introduce new inter-frame divergence and entropy formulations, which serve as the metrics for active selection. Moreover, we demonstrate additional performance gains by predicting and incorporating pseudo-labels, which are also selected using the proposed inter-frame uncertainty measure. Experimental results validate the effectiveness of LiDAL: we achieve 95% of the performance of fully supervised learning with less than 5% of annotations on the SemanticKITTI and nuScenes datasets, outperforming state-of-the-art active learning methods. Code release: https://github.com/hzykent/LiDAL.

cs.CV↗

arXiv2022

Large Language Models with Controllable Working Memory

Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP), owing to their excellent understanding and generation abilities. Remarkably, what further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. While many downstream applications provide the model with an informational context to aid its performance on the underlying task, how the model's world knowledge interacts with the factual information presented in the context remains under explored. As a desirable behavior, an LLM should give precedence to the context whenever it contains task-relevant information that conflicts with the model's memorized knowledge. This enables model predictions to be grounded in the context, which can then be used to update or correct specific model predictions without frequent retraining. By contrast, when the context is irrelevant to the task, the model should ignore it and fall back on its internal knowledge. In this paper, we undertake a first joint study of the aforementioned two properties, namely controllability and robustness, in the context of LLMs. We demonstrate that state-of-the-art T5 and PaLM (both pretrained and finetuned) could exhibit poor controllability and robustness, which do not scale with increasing model size. As a solution, we propose a novel method - Knowledge Aware FineTuning (KAFT) - to strengthen both controllability and robustness by incorporating counterfactual and irrelevant contexts to standard supervised datasets. Our comprehensive evaluation showcases the utility of KAFT across model architectures and sizes.

cs.CL↗

arXiv2022

Unified Loss of Pair Similarity Optimization for Vision-Language Retrieval

There are two popular loss functions used for vision-language retrieval, i.e., triplet loss and contrastive learning loss, both of them essentially minimize the difference between the similarities of negative pairs and positive pairs. More specifically, Triplet loss with Hard Negative mining (Triplet-HN), which is widely used in existing retrieval models to improve the discriminative ability, is easy to fall into local minima in training. On the other hand, Vision-Language Contrastive learning loss (VLC), which is widely used in the vision-language pre-training, has been shown to achieve significant performance gains on vision-language retrieval, but the performance of fine-tuning with VLC on small datasets is not satisfactory. This paper proposes a unified loss of pair similarity optimization for vision-language retrieval, providing a powerful tool for understanding existing loss functions. Our unified loss includes the hard sample mining strategy of VLC and introduces the margin used by the triplet loss for better similarity separation. It is shown that both Triplet-HN and VLC are special forms of our unified loss. Compared with the Triplet-HN, our unified loss has a fast convergence speed. Compared with the VLC, our unified loss is more discriminative and can provide better generalization in downstream fine-tuning tasks. Experiments on image-text and video-text retrieval benchmarks show that our unified loss can significantly improve the performance of the state-of-the-art retrieval models.

cs.CV↗

arXiv2022

Block Format Error Bounds and Optimal Block Size Selection

The amounts of data that need to be transmitted, processed, and stored by the modern deep neural networks have reached truly enormous volumes in the last few years calling for the invention of new paradigms both in hardware and software development. One of the most promising and rapidly advancing frontiers here is the creation of new numerical formats. In this work we focus on the family of block floating point numerical formats due to their combination of wide dynamic range, numerical accuracy, and efficient hardware implementation of inner products using simple integer arithmetic. These formats are characterized by a block of mantissas with a shared scale factor. The basic Block Floating Point (BFP) format quantizes the block scales into the nearest powers of two on the right. Its simple modification - Scaled BFP (SBFP) - stores the same scales in full precision and thus allows higher accuracy. In this paper, we study the statistical behavior of both these formats rigorously. We develop asymptotic bounds on the inner product error in SBFP- and BFP-quantized normally distributed vectors. Next, we refine those asymptotic results to finite dimensional settings and derive high-dimensional tight bounds for the same errors. Based on the obtained results we introduce a performance measure assessing accuracy of any block format. This measure allows us to determine the optimal parameters, such as the block size, yielding highest accuracy. In particular, we show that if the precision of the BFP format is fixed at 4 bits, the optimal block size becomes 64. All theoretical derivations are supported by numerical experiments and studies on the weights of publicly available pretrained neural networks.

cs.LG↗

arXiv2022

CEERS Epoch 1 NIRCam Imaging: Reduction Methods and Simulations Enabling Early JWST Science Results

We present the data release and data reduction process for the Epoch 1 NIRCam observations for the Cosmic Evolution Early Release Science Survey (CEERS). These data consist of NIRCam imaging in six broadband filters (F115W, F150W, F200W, F277W, F356W and F444W) and one medium band filter (F410M) over four pointings, obtained in parallel with primary CEERS MIRI observations (Yang et al. in prep). We reduced the NIRCam imaging with the JWST Calibration Pipeline, with custom modifications and reduction steps designed to address additional features and challenges with the data. Here we provide a detailed description of each step in our reduction and a discussion of future expected improvements. Our reduction process includes corrections for known pre-launch issues such as 1/f noise, as well as in-flight issues including snowballs, wisps, and astrometric alignment. Many of our custom reduction processes were first developed with pre-launch simulated NIRCam imaging over the full 10 CEERS NIRCam pointings. We present a description of the creation and reduction of this simulated dataset in the Appendix. We provide mosaics of the real images in a public release, as well as our reduction scripts with detailed explanations to allow users to reproduce our final data products. These represent one of the first official public datasets released from the Directors Discretionary Early Release Science (DD-ERS) program.

astro-ph.IM↗

arXiv2022

Recognizing License Plates in Real-Time

License plate detection and recognition (LPDR) is of growing importance for enabling intelligent transportation and ensuring the security and safety of the cities. However, LPDR faces a big challenge in a practical environment. The license plates can have extremely diverse sizes, fonts and colors, and the plate images are usually of poor quality caused by skewed capturing angles, uneven lighting, occlusion, and blurring. In applications such as surveillance, it often requires fast processing. To enable real-time and accurate license plate recognition, in this work, we propose a set of techniques: 1) a contour reconstruction method along with edge-detection to quickly detect the candidate plates; 2) a simple zero-one-alternation scheme to effectively remove the fake top and bottom borders around plates to facilitate more accurate segmentation of characters on plates; 3) a set of techniques to augment the training data, incorporate SIFT features into the CNN network, and exploit transfer learning to obtain the initial parameters for more effective training; and 4) a two-phase verification procedure to determine the correct plate at low cost, a statistical filtering in the plate detection stage to quickly remove unwanted candidates, and the accurate CR results after the CR process to perform further plate verification without additional processing. We implement a complete LPDR system based on our algorithms. The experimental results demonstrate that our system can accurately recognize license plate in real-time. Additionally, it works robustly under various levels of illumination and noise, and in the presence of car movement. Compared to peer schemes, our system is not only among the most accurate ones but is also the fastest, and can be easily applied to other scenarios.

cs.CV↗

arXiv2022

Attention Augmented ConvNeXt UNet For Rectal Tumour Segmentation

It is a challenge to segment the location and size of rectal cancer tumours through deep learning. In this paper, in order to improve the ability of extracting suffi-cient feature information in rectal tumour segmentation, attention enlarged ConvNeXt UNet (AACN-UNet), is proposed. The network mainly includes two improvements: 1) the encoder stage of UNet is changed to ConvNeXt structure for encoding operation, which can not only integrate multi-scale semantic information on a large scale, but al-so reduce information loss and extract more feature information from CT images; 2) CBAM attention mechanism is added to improve the connection of each feature in channel and space, which is conducive to extracting the effective feature of the target and improving the segmentation accuracy.The experiment with UNet and its variant network shows that AACN-UNet is 0.9% ,1.1% and 1.4% higher than the current best results in P, F1 and Miou.Compared with the training time, the number of parameters in UNet network is less. This shows that our proposed AACN-UNet has achieved ex-cellent results in CT image segmentation of rectal cancer.

eess.IV↗

arXiv2022

Higher angular momentum pairing states in Sr$_2$RuO$_4$ in the presence of longer-range interactions

The superconducting symmetry of Sr$_2$RuO$_4$ remains a puzzle. Time-reversal symmetry breaking $d_{x^2-y^2} + ig_{xy(x^2-y^2)}$ pairing has been proposed for reconciling multiple key experiments. However, its stability remains unclear. In this work, we theoretically study the superconducting instabilities in Sr$_2$RuO$_4$, including the effects of spin-orbit coupling (SOC), in the presence of both local and longer-range interactions within a random phase approximation. We show that the inclusion of second nearest neighbor repulsions, together with non-local SOC in the $B_{2g}$ channel or orbital-anisotropy of the non-local interactions, can have a significant impact on the stability of both $d_{x^2-y^2}$- and $g$-wave pairing channels. We analyze the properties, such as Knight shift and spontaneous edge current, of the realized $d_{x^2-y^2} + ig$, $s^{\prime}+id_{xy}$ and mixed helical pairings in different parameter spaces and find that the $d_{x^2-y^2} + ig$ solution is in better agreement with the experimental data.

cond-mat.supr-con↗

arXiv2022

Tomographic Alcock-Paczynski Method with Redshift Errors

The tomographic Alcock-Paczynski (AP) method is a promising method that uses the redshift evolution of the anisotropic clustering in redshift space to calibrate cosmology. It extends the applicable range of AP method to substantially nonlinear scales, yielding very tight cosmological constraints. For future stage-IV slitless spectroscopic surveys, the non-negligible redshift errors might reduce the advantage of the tomographic AP method by suppressing the resolution of the nonlinear structure along the line of sight. The present work studies how redshift errors propagate to cosmological parameters in the tomographic AP analysis. We use a formula $σ_z = σ(1+z)^α $ to model the redshift errors, with $σ$ varying from 0.001 to 0.006 and $α$ varying from 0.5 to 1.5. The redshift errors produce a signal of anisotropic clustering that is similar to a strong finger-of-god effect, which smears out both the AP signal and the contamination caused by the redshift space distortions (RSD). For the target precision of the Chinese Space Station Telescope optical survey ($σ\lesssim 0.002$), the decrement of constraining power on the dark energy equation of state is mild ($\lesssim 50\%$), and the suppression of RSD contamination leads to a smaller bias-to-signal ratio. Our results indicate that the tomographic AP method will remain a useful and complementary tool for analyses of future slitless spectroscopic surveys.

astro-ph.CO↗

arXiv2022

Multimessenger Picture of J1048+7143

We draw the multimessenger picture of J1048+7143, a flat-spectrum radio quasar known to show quasi-periodic oscillations in the $γ$-ray regime. We generate the adaptively-binned Fermi Large Area Telescope light curve of this source above 168 MeV to find three major $γ$-ray flares of the source, such that all three flares consist of two-two sharp sub-flares. Based on radio interferometric imaging data taken with the Very Large Array, we find that the kpc-scale jet is directed towards west, while our analysis of $8.6$-GHz very long baseline interferometry data, mostly taken with the Very Long Baseline Array, revealed signatures of two pc-scale jets, one pointing towards east, one pointing towards south. We suggest that the misalignment of the kpc- and pc-scale jets is a revealing signature of jet precession. We also analyze the $5$-GHz total flux density curve of J1048+7143 taken with the Nanshan(Ur) and RATAN-600 single dish radio telescopes and find two complete radio flares, slightly lagging behind the $γ$-ray flares. We model the timing of $γ$-ray flares as signature of the spin-orbit precession in a supermassive black hole binary, and find that the binary could merge in the next $\sim 60-80$ years. We show that both the Pulsar Timing Arrays and the planned Laser Interferometer Space Antenna lack sensitivity and frequency coverage to detect the hypothetical supermassive black hole binary in J1048$+$7143. We argue that the identification of sources similar to J1048+7143 plays a key role to reveal periodic high-energy sources in the distant Universe.

astro-ph.HE↗

arXiv2022

Detection of Real-time DeepFakes in Video Conferencing with Active Probing and Corneal Reflection

The COVID pandemic has led to the wide adoption of online video calls in recent years. However, the increasing reliance on video calls provides opportunities for new impersonation attacks by fraudsters using the advanced real-time DeepFakes. Real-time DeepFakes pose new challenges to detection methods, which have to run in real-time as a video call is ongoing. In this paper, we describe a new active forensic method to detect real-time DeepFakes. Specifically, we authenticate video calls by displaying a distinct pattern on the screen and using the corneal reflection extracted from the images of the call participant's face. This pattern can be induced by a call participant displaying on a shared screen or directly integrated into the video-call client. In either case, no specialized imaging or lighting hardware is required. Through large-scale simulations, we evaluate the reliability of this approach under a range in a variety of real-world imaging scenarios.

cs.CV↗

arXiv2022

InFIP: An Explainable DNN Intellectual Property Protection Method based on Intrinsic Features

Intellectual property (IP) protection for Deep Neural Networks (DNNs) has raised serious concerns in recent years. Most existing works embed watermarks in the DNN model for IP protection, which need to modify the model and lack of interpretability. In this paper, for the first time, we propose an interpretable intellectual property protection method for DNN based on explainable artificial intelligence. Compared with existing works, the proposed method does not modify the DNN model, and the decision of the ownership verification is interpretable. We extract the intrinsic features of the DNN model by using Deep Taylor Decomposition. Since the intrinsic feature is composed of unique interpretation of the model's decision, the intrinsic feature can be regarded as fingerprint of the model. If the fingerprint of a suspected model is the same as the original model, the suspected model is considered as a pirated model. Experimental results demonstrate that the fingerprints can be successfully used to verify the ownership of the model and the test accuracy of the model is not affected. Furthermore, the proposed method is robust to fine-tuning attack, pruning attack, watermark overwriting attack, and adaptive attack.

cs.CV↗

arXiv2022

GGViT:Multistream Vision Transformer Network in Face2Face Facial Reenactment Detection

Detecting manipulated facial images and videos on social networks has been an urgent problem to be solved. The compression of videos on social media has destroyed some pixel details that could be used to detect forgeries. Hence, it is crucial to detect manipulated faces in videos of different quality. We propose a new multi-stream network architecture named GGViT, which utilizes global information to improve the generalization of the model. The embedding of the whole face extracted by ViT will guide each stream network. Through a large number of experiments, we have proved that our proposed model achieves state-of-the-art classification accuracy on FF++ dataset, and has been greatly improved on scenarios of different compression rates. The accuracy of Raw/C23, Raw/C40 and C23/C40 was increased by 24.34%, 15.08% and 10.14% respectively.

cs.CV↗

arXiv2022

3D Matting: A Benchmark Study on Soft Segmentation Method for Pulmonary Nodules Applied in Computed Tomography

Usually, lesions are not isolated but are associated with the surrounding tissues. For example, the growth of a tumour can depend on or infiltrate into the surrounding tissues. Due to the pathological nature of the lesions, it is challenging to distinguish their boundaries in medical imaging. However, these uncertain regions may contain diagnostic information. Therefore, the simple binarization of lesions by traditional binary segmentation can result in the loss of diagnostic information. In this work, we introduce the image matting into the 3D scenes and use the alpha matte, i.e., a soft mask, to describe lesions in a 3D medical image. The traditional soft mask acted as a training trick to compensate for the easily mislabelled or under-labelled ambiguous regions. In contrast, 3D matting uses soft segmentation to characterize the uncertain regions more finely, which means that it retains more structural information for subsequent diagnosis and treatment. The current study of image matting methods in 3D is limited. To address this issue, we conduct a comprehensive study of 3D matting, including both traditional and deep-learning-based methods. We adapt four state-of-the-art 2D image matting algorithms to 3D scenes and further customize the methods for CT images to calibrate the alpha matte with the radiodensity. Moreover, we propose the first end-to-end deep 3D matting network and implement a solid 3D medical image matting benchmark. Its efficient counterparts are also proposed to achieve a good performance-computation balance. Furthermore, there is no high-quality annotated dataset related to 3D matting, slowing down the development of data-driven deep-learning-based methods. To address this issue, we construct the first 3D medical matting dataset. The validity of the dataset was verified through clinicians' assessments and downstream experiments.

eess.IV↗

arXiv2022

Early results from GLASS-JWST VIII: An Extremely Magnified Blue Supergiant Star at Redshift 2.65 in the Abell 2744 Cluster Field

We report the discovery of an extremely magnified star at redshift $z=2.65$ in James Webb Space Telescope (JWST) NIRISS pre-imaging of the Abell 2744 galaxy-cluster field. The star's background host galaxy lies on a fold caustic of the foreground lens, and the cluster creates a pair of images of the region close to the lensed star. We identified the bright transient in one of the merging images at a distance of $\sim 0.15"$ from the critical curve, by subtracting the JWST F115W and F150W imaging from coadditions of archival Hubble Space Telescope (HST) F105W and F125W images and F140W and F160W images, respectively. Since the time delay between the two images should be only hours, the transient must be the microlensing event of an individual star, as opposed to a luminous stellar explosion which would persist for days to months. Analysis of individual exposures suggests that the star's magnification is not changing rapidly during the observations. From photometry of the point source through the F115W, F150W, and F200W filters, we identify a strong Balmer break, and modeling allows us to constrain the star's temperature to be approximately 7,000--12,000 K.

astro-ph.GA↗

arXiv2022

Investigating Active-learning-based Training Data Selection for Speech Spoofing Countermeasure

Training a spoofing countermeasure (CM) that generalizes to various unseen data is desired but challenging. While methods such as data augmentation and self-supervised learning are applicable, the imperfect CM performance on diverse test sets still calls for additional strategies. This study took the initiative and investigated CM training using active learning (AL), a framework that iteratively selects useful data from a large pool set and fine-tunes the CM. This study compared a few methods to measure the data usefulness and the impact of using different pool sets collected from various sources. The results showed that the AL-based CMs achieved better generalization than our strong baseline on multiple test tests. Furthermore, compared with a top-line CM that simply used the whole data pool set for training, the AL-based CMs achieved similar performance using less training data. Although no single best configuration was found for AL, the rule of thumb is to include diverse spoof and bona fide data in the pool set and to avoid any AL data selection method that selects the data that the CM feels confident in.

eess.AS↗

arXiv2022

Liquid Crystals as Multifunctional Interfaces for Trapping and Characterizing Microplastics

Identifying and removing microplastics (MPs) from the environment is a global challenge. This study explores how the colloidal fraction of common MPs behave at aqueous interfaces of liquid crystal (LC) films. We observed polyethylene (PE) and polystyrene (PS) microparticles to be captured at the LC interface and exhibit distinct two-dimensional aggregation patterns. The addition of low concentrations of surfactant (sodium dodecylsulfate (SDS)) was found to further amplify the differences in PS/PE aggregation patterns, with PS changing from a linear chain-like morphology to a singly dispersed state with increasing SDS concentration and PE forming dense clusters at all SDS concentrations. Statistical characterization of assembly patterns using fractal geometric theory-based machine learning and a deep learning image recognition model yielded highly accurate classification of PE vs PS (>99%). Additionally, by performing feature importance analysis on our deep learning model, dense, multi-branched assemblies were confirmed to be unique features of PE relative to PS. To obtain additional insight into the origin of these key features, we performed microscopic characterization of LC ordering at the microparticle surfaces. These observations led us to predict that both microparticle types should generate LC-mediated interactions (due to elastic strain) with a dipolar symmetry, a prediction consistent with the observed interfacial organization of PS but not PE. We conclude that the non-equilibrium organization of the PE microparticles arises from their polycrystalline nature, which leads to rough particle surfaces and weakened LC elastic interactions and enhanced capillary forces. Overall, our results highlight the potential utility of LC interfaces for surface-sensitive characterization of colloidal MPs.

cond-mat.soft↗