arXiv subjects

Ming Yang

Publications and source records attributed to Ming Yang.

At least 217 records · Page 12Linked to original sources

arXiv2024

Tactics2D: A Highly Modular and Extensible Simulator for Driving Decision-making

Simulation is a prospective method for generating diverse and realistic traffic scenarios to aid in the development of driving decision-making systems. However, existing simulators often fall short in diverse scenarios or interactive behavior models for traffic participants. This deficiency underscores the need for a flexible, reliable, user-friendly open-source simulator. Addressing this challenge, Tactics2D adopts a modular approach to traffic scenario construction, encompassing road elements, traffic regulations, behavior models, physics simulations for vehicles, and event detection mechanisms. By integrating numerous commonly utilized algorithms and configurations, Tactics2D empowers users to construct their driving scenarios effortlessly, just like assembling building blocks. Users can effectively evaluate the performance of driving decision-making models across various scenarios by leveraging both public datasets and user-collected real-world data. For access to the source code and community support, please visit the official GitHub page for Tactics2D at https://github.com/WoodOxen/Tactics2D.

cs.LG↗

arXiv2024

Unpaired Multi-view Clustering via Reliable View Guidance

This paper focuses on unpaired multi-view clustering (UMC), a challenging problem where paired observed samples are unavailable across multiple views. The goal is to perform effective joint clustering using the unpaired observed samples in all views. In incomplete multi-view clustering, existing methods typically rely on sample pairing between views to capture their complementary. However, that is not applicable in the case of UMC. Hence, we aim to extract the consistent cluster structure across views. In UMC, two challenging issues arise: uncertain cluster structure due to lack of label and uncertain pairing relationship due to absence of paired samples. We assume that the view with a good cluster structure is the reliable view, which acts as a supervisor to guide the clustering of the other views. With the guidance of reliable views, a more certain cluster structure of these views is obtained while achieving alignment between reliable views and other views. Then we propose Reliable view Guidance with one reliable view (RG-UMC) and multiple reliable views (RGs-UMC) for UMC. Specifically, we design alignment modules with one reliable view and multiple reliable views, respectively, to adaptively guide the optimization process. Also, we utilize the compactness module to enhance the relationship of samples within the same cluster. Meanwhile, an orthogonal constraint is applied to latent representation to obtain discriminate features. Extensive experiments show that both RG-UMC and RGs-UMC outperform the best state-of-the-art method by an average of 24.14\% and 29.42\% in NMI, respectively.

cs.CV↗

arXiv2024

Tianyu: search for the second solar system and explore the dynamic universe

Giant planets like Jupiter and Saturn, play important roles in the formation and habitability of Earth-like planets. The detection of solar system analogs that have multiple cold giant planets is essential for our understanding of planet habitability and planet formation. Although transit surveys such as Kepler and TESS have discovered thousands of exoplanets, these missions are not sensitive to long period planets due to their limited observation baseline. The Tianyu project, comprising two 1-meter telescopes (Tianyu-I and II), is designed to detect transiting cold giant planets in order to find solar system analogs. Featuring a large field of view and equipped with a high-speed CMOS camera, Tianyu-I will perform a high-precision photometric survey of about 100 million stars, measuring light curves at hour-long cadence. The candidates found by Tianyu-I will be confirmed by Tianyu-II and other surveys and follow-up facilities through multi-band photometry, spectroscopy, and high resolution imaging. Tianyu telescopes will be situated at an elevation about 4000 meters in Lenghu, China. With a photometric precision of 1% for stars with V < 18 mag, Tianyu is expected to find more than 300 transiting exoplanets, including about 12 cold giant planets, over five years. A five-year survey of Tianyu would discover 1-2 solar system analogs. Moreover, Tianyu is also designed for non-exoplanetary exploration, incorporating multiple survey modes covering timescales from sub-seconds to months, with a particular emphasis on events occurring within the sub-second to hour range. It excels in observing areas such as infant supernovae, rare variable stars and binaries, tidal disruption events, Be stars, cometary activities, and interstellar objects. These discoveries not only enhance our comprehension of the universe but also offer compelling opportunities for public engagement in scientific exploration.

astro-ph.IM↗

arXiv2024

SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery

Prior studies on Remote Sensing Foundation Model (RSFM) reveal immense potential towards a generic model for Earth Observation. Nevertheless, these works primarily focus on a single modality without temporal and geo-context modeling, hampering their capabilities for diverse tasks. In this study, we present SkySense, a generic billion-scale model, pre-trained on a curated multi-modal Remote Sensing Imagery (RSI) dataset with 21.5 million temporal sequences. SkySense incorporates a factorized multi-modal spatiotemporal encoder taking temporal sequences of optical and Synthetic Aperture Radar (SAR) data as input. This encoder is pre-trained by our proposed Multi-Granularity Contrastive Learning to learn representations across different modal and spatial granularities. To further enhance the RSI representations by the geo-context clue, we introduce Geo-Context Prototype Learning to learn region-aware prototypes upon RSI's multi-modal spatiotemporal features. To our best knowledge, SkySense is the largest Multi-Modal RSFM to date, whose modules can be flexibly combined or used individually to accommodate various tasks. It demonstrates remarkable generalization capabilities on a thorough evaluation encompassing 16 datasets over 7 tasks, from single- to multi-modal, static to temporal, and classification to localization. SkySense surpasses 18 recent RSFMs in all test scenarios. Specifically, it outperforms the latest models such as GFM, SatLas and Scale-MAE by a large margin, i.e., 2.76%, 3.67% and 3.61% on average respectively. We will release the pre-trained weights to facilitate future research and Earth Observation applications.

cs.CV↗

arXiv2024

Zippo: Zipping Color and Transparency Distributions into a Single Diffusion Model

Beyond the superiority of the text-to-image diffusion model in generating high-quality images, recent studies have attempted to uncover its potential for adapting the learned semantic knowledge to visual perception tasks. In this work, instead of translating a generative diffusion model into a visual perception model, we explore to retain the generative ability with the perceptive adaptation. To accomplish this, we present Zippo, a unified framework for zipping the color and transparency distributions into a single diffusion model by expanding the diffusion latent into a joint representation of RGB images and alpha mattes. By alternatively selecting one modality as the condition and then applying the diffusion process to the counterpart modality, Zippo is capable of generating RGB images from alpha mattes and predicting transparency from input images. In addition to single-modality prediction, we propose a modality-aware noise reassignment strategy to further empower Zippo with jointly generating RGB images and its corresponding alpha mattes under the text guidance. Our experiments showcase Zippo's ability of efficient text-conditioned transparent image generation and present plausible results of Matte-to-RGB and RGB-to-Matte translation.

cs.CV↗

arXiv2024

Pink: Unveiling the Power of Referential Comprehension for Multi-modal LLMs

Multi-modal Large Language Models (MLLMs) have shown remarkable capabilities in various multi-modal tasks. Nevertheless, their performance in fine-grained image understanding tasks is still limited. To address this issue, this paper proposes a new framework to enhance the fine-grained image understanding abilities of MLLMs. Specifically, we present a new method for constructing the instruction tuning dataset at a low cost by leveraging annotations in existing datasets. A self-consistent bootstrapping method is also introduced to extend existing dense object annotations into high-quality referring-expression-bounding-box pairs. These methods enable the generation of high-quality instruction data which includes a wide range of fundamental abilities essential for fine-grained image perception. Moreover, we argue that the visual encoder should be tuned during instruction tuning to mitigate the gap between full image perception and fine-grained image perception. Experimental results demonstrate the superior performance of our method. For instance, our model exhibits a 5.2% accuracy improvement over Qwen-VL on GQA and surpasses the accuracy of Kosmos-2 by 24.7% on RefCOCO_val. We have also attained the top rank on the leaderboard of MMBench. This promising performance is achieved by training on only publicly available data, making it easily reproducible. The models, datasets, and codes are publicly available at https://github.com/SY-Xuan/Pink.

cs.CV↗

arXiv2024

Discovery of a Magnetic Topological Semimetal Eu$_3$In$_2$As$_4$ with a Single Pair of Weyl Points

Magnetic Weyl semimetal (MWS) is a unique topological state with open surface Fermi arc states and other exotic transport phenomena. However, most reported MWSs show multiple pairs of Weyl points and complicated Fermi surfaces, which increases the difficulty of the investigation into the intrinsic chiral transport property. In this wor, we successfully synthesized a soft magnetic Weyl semimetal Eu$_3$In$_2$As$_4$ with a single pair of Weyl points under magnetic fields. The Shubnikov de Haas (SdH) oscillation with a single frequency, as well as a linear hall resistance with the same carrier density, is observed up to 50 Tesla, indicating a single pair of Weyl points around the Fermi level with a massless fermion ($m^* = 0.121 m_0$, $π$ Berry phase). Such a single pair of Weyl points is further confirmed by the density functional theory calculations. The magnetic ordering and band topology can be easily tuned by the external magnetic field. The field-induced MWS Eu$_3$In$_2$As$_4$ with a single pair of Weyl points is a good platform to detect chiral transport properties, including possible quantum anomalous Hall effect.

cond-mat.mes-hall↗

arXiv2024

One-Step Multi-View Clustering Based on Transition Probability

The large-scale multi-view clustering algorithms, based on the anchor graph, have shown promising performance and efficiency and have been extensively explored in recent years. Despite their successes, current methods lack interpretability in the clustering process and do not sufficiently consider the complementary information across different views. To address these shortcomings, we introduce the One-Step Multi-View Clustering Based on Transition Probability (OSMVC-TP). This method adopts a probabilistic approach, which leverages the anchor graph, representing the transition probabilities from samples to anchor points. Our method directly learns the transition probabilities from anchor points to categories, and calculates the transition probabilities from samples to categories, thus obtaining soft label matrices for samples and anchor points, enhancing the interpretability of clustering. Furthermore, to maintain consistency in labels across different views, we apply a Schatten p-norm constraint on the tensor composed of the soft labels. This approach effectively harnesses the complementary information among the views. Extensive experiments have confirmed the effectiveness and robustness of OSMVC-TP.

cs.LG↗

arXiv2024

Learning Dynamic Tetrahedra for High-Quality Talking Head Synthesis

Recent works in implicit representations, such as Neural Radiance Fields (NeRF), have advanced the generation of realistic and animatable head avatars from video sequences. These implicit methods are still confronted by visual artifacts and jitters, since the lack of explicit geometric constraints poses a fundamental challenge in accurately modeling complex facial deformations. In this paper, we introduce Dynamic Tetrahedra (DynTet), a novel hybrid representation that encodes explicit dynamic meshes by neural networks to ensure geometric consistency across various motions and viewpoints. DynTet is parameterized by the coordinate-based networks which learn signed distance, deformation, and material texture, anchoring the training data into a predefined tetrahedra grid. Leveraging Marching Tetrahedra, DynTet efficiently decodes textured meshes with a consistent topology, enabling fast rendering through a differentiable rasterizer and supervision via a pixel loss. To enhance training efficiency, we incorporate classical 3D Morphable Models to facilitate geometry learning and define a canonical space for simplifying texture learning. These advantages are readily achievable owing to the effective geometric representation employed in DynTet. Compared with prior works, DynTet demonstrates significant improvements in fidelity, lip synchronization, and real-time performance according to various metrics. Beyond producing stable and visually appealing synthesis videos, our method also outputs the dynamic meshes which is promising to enable many emerging applications.

cs.CV↗

arXiv2024

The Star Formation History in Local Group Galaxies. I. Ten Dwarf Galaxies

The star formation histories (SFHs) of galaxies provide valuable insights into galaxy evolution and stellar physics. Understanding the SFHs enables the study of chemical enrichment of galaxies, star formation triggered by interactions, and the behavior of various stellar populations. This work investigates the SFHs of ten dwarf galaxies in the Local Group (LG), which spans a wide range of types, masses, luminosities, and metallicities. The analysis is based on our new sample of the member stars in the LG after removing the foreground dwarf stars by the near-infrared color-color diagram and the Gaia astrometric information. The samples include the most complete and pure red supergiants and asymptotic giant branch stars to gain valuable insights into the recent SFHs of the galaxies. The CMD fitting method is introduced to measure the SFH. The Padova isochrones are used to generate initial model CMDs, accounting for photometric errors and completeness through star field simulations to match the completeness and error distributions of the observed CMDs. Subsequently, the SFHs, distance modulus, and metallicity of the ten dwarf galaxies are determined by fitting the CMDs. The results indicate that the star formation rates (SFRs) of dwarf irregulars show a gradual increase, while those of dwarf ellipticals exhibit a gradual decrease from the past to the present. Furthermore, this work shows that the star formation activity in dwarf ellipticals persisted up to 30 Myr ago. A significant increasing feature in the SFH of NGC 6822 reveals star formation activity triggered by an interaction event.

astro-ph.GA↗

arXiv2024

M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining

Vision-language foundation models like CLIP have revolutionized the field of artificial intelligence. Nevertheless, VLM models supporting multi-language, e.g., in both Chinese and English, have lagged due to the relative scarcity of large-scale pretraining datasets. Toward this end, we introduce a comprehensive bilingual (Chinese-English) dataset BM-6B with over 6 billion image-text pairs, aimed at enhancing multimodal foundation models to well understand images in both languages. To handle such a scale of dataset, we propose a novel grouped aggregation approach for image-text contrastive loss computation, which reduces the communication overhead and GPU memory demands significantly, facilitating a 60% increase in training speed. We pretrain a series of bilingual image-text foundation models with an enhanced fine-grained understanding ability on BM-6B, the resulting models, dubbed as $M^2$-Encoders (pronounced "M-Square"), set new benchmarks in both languages for multimodal retrieval and classification tasks. Notably, Our largest $M^2$-Encoder-10B model has achieved top-1 accuracies of 88.5% on ImageNet and 80.7% on ImageNet-CN under a zero-shot classification setting, surpassing previously reported SoTA methods by 2.2% and 21.1%, respectively. The $M^2$-Encoder series represents one of the most comprehensive bilingual image-text foundation models to date, so we are making it available to the research community for further exploration and development.

cs.CV↗

arXiv2024

M2-RAAP: A Multi-Modal Recipe for Advancing Adaptation-based Pre-training towards Effective and Efficient Zero-shot Video-text Retrieval

We present a Multi-Modal Recipe for Advancing Adaptation-based Pre-training towards effective and efficient zero-shot video-text retrieval, dubbed M2-RAAP. Upon popular image-text models like CLIP, most current adaptation-based video-text pre-training methods are confronted by three major issues, i.e., noisy data corpus, time-consuming pre-training, and limited performance gain. Towards this end, we conduct a comprehensive study including four critical steps in video-text pre-training. Specifically, we investigate 1) data filtering and refinement, 2) video input type selection, 3) temporal modeling, and 4) video feature enhancement. We then summarize this empirical study into the M2-RAAP recipe, where our technical contributions lie in 1) the data filtering and text re-writing pipeline resulting in 1M high-quality bilingual video-text pairs, 2) the replacement of video inputs with key-frames to accelerate pre-training, and 3) the Auxiliary-Caption-Guided (ACG) strategy to enhance video features. We conduct extensive experiments by adapting three image-text foundation models on two refined video-text datasets from different languages, validating the robustness and reproducibility of M2-RAAP for adaptation-based pre-training. Results demonstrate that M2-RAAP yields superior performance with significantly reduced data (-90%) and time consumption (-95%), establishing a new SOTA on four English zero-shot retrieval datasets and two Chinese ones. We are preparing our refined bilingual data annotations and codebase, which will be available at https://github.com/alipay/Ant-Multi-Modal-Framework/tree/main/prj/M2_RAAP.

cs.CV↗

arXiv2024

Evolved Massive Stars at Low-metallicity VII. the Lower Mass Limit of Red Supergiant Population in the Large Magellanic Cloud

The precise definition of the lower mass limit of red supergiant stars (RSGs) is an open question in astrophysics and does not attract too much attention. Here we assemble a spectroscopic evolved cool star sample with 6,602 targets, including RSGs, asymptotic giant branch stars, and red giant branch stars, in the Large Magellanic Cloud based on \textit{Gaia} DR3 and SDSS-IV/APOGEE-2. The reference spectrum of each stellar population is built according to the quantile range of relative intensity ($1\%\sim99\%$). Five different methods, e.g., chi-square ($χ^2$), cosine similarity (CS), machine learning (ML), equivalent width (EW), and line ratio (LR), are used in order to separate different stellar populations. The ML and $χ^2$ provide the best and relatively consistent prediction of certain population. The derived lower limit of the RSG population is able to reach to the $\rm K_S$-band tip of red giant branch ($\rm K_S~$$\approx12.0$ mag), indicating a luminosity as low as about $10^{3.5}~L_{\sun}$, which corresponds to a stellar radius only about $100~R_{\sun}$. Given the mass-luminosity relation of $L/L_\sun =f(M/M_\sun)^3$ with $f\approx15.5\pm3$ and taking into account of the mass loss of faint RSGs up to now, the minimal initial mass of the RSG population would be about $6.1\pm0.4~M_\sun$, which is much lower than the traditional threshold of $8~M_\sun$ for the massive stars. This is the first spectroscopic evidence, indicating that the lower mass limit of RSG population is around $6~M_\sun$. However, the destinies of such faint RSGs are still elusive and may have large impact on the stellar evolutionary and supernova models.

astro-ph.SR↗

arXiv2024

Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learning

In recent years, the explosion of web videos makes text-video retrieval increasingly essential and popular for video filtering, recommendation, and search. Text-video retrieval aims to rank relevant text/video higher than irrelevant ones. The core of this task is to precisely measure the cross-modal similarity between texts and videos. Recently, contrastive learning methods have shown promising results for text-video retrieval, most of which focus on the construction of positive and negative pairs to learn text and video representations. Nevertheless, they do not pay enough attention to hard negative pairs and lack the ability to model different levels of semantic similarity. To address these two issues, this paper improves contrastive learning using two novel techniques. First, to exploit hard examples for robust discriminative power, we propose a novel Dual-Modal Attention-Enhanced Module (DMAE) to mine hard negative pairs from textual and visual clues. By further introducing a Negative-aware InfoNCE (NegNCE) loss, we are able to adaptively identify all these hard negatives and explicitly highlight their impacts in the training loss. Second, our work argues that triplet samples can better model fine-grained semantic similarity compared to pairwise samples. We thereby present a new Triplet Partial Margin Contrastive Learning (TPM-CL) module to construct partial order triplet samples by automatically generating fine-grained hard negatives for matched text-video pairs. The proposed TPM-CL designs an adaptive token masking strategy with cross-modal interaction to model subtle semantic differences. Extensive experiments demonstrate that the proposed approach outperforms existing methods on four widely-used text-video retrieval datasets, including MSR-VTT, MSVD, DiDeMo and ActivityNet.

cs.CV↗

arXiv2024

Evolved Massive Stars at Low-metallicity VI. Mass-Loss Rate of Red Supergiant Stars in the Large Magellanic Cloud

Mass loss is a crucial process that affects the observational properties, evolution path and fate of highly evolved stars. However, the mechanism of mass loss is still unclear, and the mass-loss rate (MLR) of red supergiant stars (RSGs) requires further research and precise evaluation. To address this, we utilized an updated and complete sample of RSGs in the Large Magellanic Cloud (LMC) and employed the 2-DUST radiation transfer model and spectral energy distribution (SED) fitting approach to determine the dust-production rates (DPRs) and dust properties of the RSGs. We have fitted 4,714 selected RSGs with over 100,000 theoretical templates of evolved stars. Our results show that the DPR range of RSGs in the LMC is $10^{-11}\, \rm{M_{\odot}\, yr^{-1}}$ to $10^{-7}\, \rm{M_{\odot}\, yr^{-1}}$, and the total DPR of all RSGs is 1.14 $\times 10^{-6} \, \rm{M_{\odot} \, yr^{-1}}$. We find that $63.3\%$ RSGs are oxygen-rich, and they account for $97.2\%$ of the total DPR. The optically thin RSG, which comprise $30.6\%$ of our sample, contribute only $0.1\%$ of the total DPR, while carbon-rich RSGs ($6.1\%$) produce $2.7\%$ of the total DPR. Overall, 208 RSGs contributed $76.6\%$ of the total DPR. We have established a new relationship between the MLR and luminosity of RSGs in the LMC, which exhibits a positive trend and a clear turning point at $\log{L/L_{\odot}} \approx 4.4$.

astro-ph.GA↗

arXiv2024

SyCoCa: Symmetrizing Contrastive Captioners with Attentive Masking for Multimodal Alignment

Multimodal alignment between language and vision is the fundamental topic in current vision-language model research. Contrastive Captioners (CoCa), as a representative method, integrates Contrastive Language-Image Pretraining (CLIP) and Image Caption (IC) into a unified framework, resulting in impressive results. CLIP imposes a bidirectional constraints on global representation of entire images and sentences. Although IC conducts an unidirectional image-to-text generation on local representation, it lacks any constraint on local text-to-image reconstruction, which limits the ability to understand images at a fine-grained level when aligned with texts. To achieve multimodal alignment from both global and local perspectives, this paper proposes Symmetrizing Contrastive Captioners (SyCoCa), which introduces bidirectional interactions on images and texts across the global and local representation levels. Specifically, we expand a Text-Guided Masked Image Modeling (TG-MIM) head based on ITC and IC heads. The improved SyCoCa can further leverage textual cues to reconstruct contextual images and visual cues to predict textual contents. When implementing bidirectional local interactions, the local contents of images tend to be cluttered or unrelated to their textual descriptions. Thus, we employ an attentive masking strategy to select effective image patches for interaction. Extensive experiments on five vision-language tasks, including image-text retrieval, image-captioning, visual question answering, and zero-shot/finetuned image classification, validate the effectiveness of our proposed method.

cs.CV↗

arXiv2024

Evolutionary Alternating Direction Method of Multipliers for Constrained Multi-Objective Optimization with Unknown Constraints

Constrained multi-objective optimization problems (CMOPs) pervade real-world applications in science, engineering, and design. Constraint violation has been a building block in designing evolutionary multi-objective optimization algorithms for solving constrained multi-objective optimization problems. However, in certain scenarios, constraint functions might be unknown or inadequately defined, making constraint violation unattainable and potentially misleading for conventional constrained evolutionary multi-objective optimization algorithms. To address this issue, we present the first of its kind evolutionary optimization framework, inspired by the principles of the alternating direction method of multipliers that decouples objective and constraint functions. This framework tackles CMOPs with unknown constraints by reformulating the original problem into an additive form of two subproblems, each of which is allotted a dedicated evolutionary population. Notably, these two populations operate towards complementary evolutionary directions during their optimization processes. In order to minimize discrepancy, their evolutionary directions alternate, aiding the discovery of feasible solutions. Comparative experiments conducted against five state-of-the-art constrained evolutionary multi-objective optimization algorithms, on 120 benchmark test problem instances with varying properties, as well as two real-world engineering optimization problems, demonstrate the effectiveness and superiority of our proposed framework. Its salient features include faster convergence and enhanced resilience to various Pareto front shapes.

cs.NE↗

arXiv2023

Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving

Simulators play a crucial role in autonomous driving, offering significant time, cost, and labor savings. Over the past few years, the number of simulators for autonomous driving has grown substantially. However, there is a growing concern about the validity of algorithms developed and evaluated in simulators, indicating a need for a thorough analysis of the development status of the simulators. To bridge the gap in research, this paper analyzes the evolution of simulators and explains how the functionalities and utilities have developed. Then, the existing simulators are categorized based on their task applicability, providing researchers with a taxonomy to swiftly assess a simulator's suitability for specific tasks. Recommendations for select simulators are presented, considering factors such as accessibility, maintenance status, and quality. Recognizing potential hazards in simulators that could impact the confidence of simulation experiments, the paper dedicates substantial effort to identifying and justifying critical issues in actively maintained open-source simulators. Moreover, the paper reviews potential solutions to address these issues, serving as a guide for enhancing the credibility of simulators.

cs.RO↗