arXiv subjects

Kai Wang

Publications and source records attributed to Kai Wang.

At least 397 records · Page 22Linked to original sources

arXiv2024

PowerRadio: Manipulate Sensor Measurementvia Power GND Radiation

Sensors are key components enabling various applications, e.g., home intrusion detection and environmental monitoring. While various software defenses and physical protections are used to prevent sensor manipulation, this paper introduces a new threat vector, PowerRadio, that bypasses existing protections and changes sensor readings from a distance. PowerRadio leverages interconnected ground (GND) wires, a standard practice for electrical safety at home, to inject malicious signals. The injected signal is coupled by the sensor's analog measurement wire and eventually survives the noise filters, inducing incorrect measurement. We present three methods to manipulate sensors by inducing static bias, periodical signals, or pulses. For instance, we show adding stripes into the captured images of a surveillance camera or injecting inaudible voice commands into conference microphones. We study the underlying principles of PowerRadio and identify its root causes: (1) the lack of shielding between ground and data signal wires and (2) the asymmetry of circuit impedance that enables interference to bypass filtering. We validate PowerRadio against a surveillance system, broadcast systems, and various sensors. We believe that PowerRadio represents an emerging threat, exhibiting the advantages of both radiated and conducted EMI, e.g., expanding the effective attack distance of radiated EMI yet eliminating the requirement of line-of-sight or approaching physically. Our insights shall provide guidance for enhancing the sensors' security and power wiring during the design phases.

eess.SP↗

arXiv2024

Deformations and homotopy theory of Nijenhuis associative algebras

This paper is the first in a series of works devoted to an operadic study of Nijenhuis structures, focusing on Nijenhuis associative algebras. We introduce the concept of homotopy Nijenhuis associative algebras and demonstrate that the differential graded (=dg) operad $\NjAoperad_{\infty}$ governing these structures serves as the minimal model of the operad $\NjAoperad$ for Nijenhuis associative algebras. Additionally, we determine the Koszul dual homotopy cooperad of $\NjAoperad$. We construct an $L_\infty$-algebra that controls the simultaneous deformations of associative products and Nijenhuis operators. The Maurer-Cartan elements of this $L_\infty$-algebra correspond bijectively to Nijenhuis associative algebra structures. From this, we derive a cochain complex (deformation complex) and an associated cohomology theory of Nijenhuis associative algebras. Finally, we explore the connection between homotopy relative Rota-Baxter associative algebras of weight $0$ and homotopy Nijenhuis associative algebras. A sequel to this work will extend the study to Nijenhuis Lie algebras, with applications to Nijenhuis geometry.

math.KT↗

arXiv2024

CALLIC: Content Adaptive Learning for Lossless Image Compression

Learned lossless image compression has achieved significant advancements in recent years. However, existing methods often rely on training amortized generative models on massive datasets, resulting in sub-optimal probability distribution estimation for specific testing images during encoding process. To address this challenge, we explore the connection between the Minimum Description Length (MDL) principle and Parameter-Efficient Transfer Learning (PETL), leading to the development of a novel content-adaptive approach for learned lossless image compression, dubbed CALLIC. Specifically, we first propose a content-aware autoregressive self-attention mechanism by leveraging convolutional gating operations, termed Masked Gated ConvFormer (MGCF), and pretrain MGCF on training dataset. Cache then Crop Inference (CCI) is proposed to accelerate the coding process. During encoding, we decompose pre-trained layers, including depth-wise convolutions, using low-rank matrices and then adapt the incremental weights on testing image by Rate-guided Progressive Fine-Tuning (RPFT). RPFT fine-tunes with gradually increasing patches that are sorted in descending order by estimated entropy, optimizing learning process and reducing adaptation time. Extensive experiments across diverse datasets demonstrate that CALLIC sets a new state-of-the-art (SOTA) for learned lossless image compression.

cs.CV↗

arXiv2024

LiPar: A Lightweight Parallel Learning Model for Practical In-Vehicle Network Intrusion Detection

With the development of intelligent transportation systems, vehicles are exposed to a complex network environment. As the main network of in-vehicle networks, the controller area network (CAN) has many potential security hazards, resulting in higher generalization capability and lighter security requirements for intrusion detection systems to ensure safety. Among intrusion detection technologies, methods based on deep learning work best without prior expert knowledge. However, they all have a large model size and usually rely on large computing power such as cloud computing, and are therefore not suitable to be installed on the in-vehicle network. Therefore, we explore computational resource allocation schemes in in-vehicle network and propose a lightweight parallel neural network structure, LiPar, which achieve enhanced generalization capability for identifying normal and abnormal patterns of in-vehicle communication flows to achieve effective intrusion detection while improving the utilization of limited computing resources. In particular, LiPar adaptationally allocates task loads to in-vehicle computing devices, such as multiple electronic control units, domain controllers, computing gateways through evaluates whether a computing device is suitable to undertake the branch computing tasks according to its real-time resource occupancy. Through experiments, we prove that LiPar has great detection performance, running efficiency, and lightweight model size, which can be well adapted to the in-vehicle environment practically and protect the in-vehicle CAN bus security. Furthermore, with only the common multi-dimensional branch convolution networks for detection, LiPar can have a high potential for generalization in spatial and temporal feature fusion learning.

cs.CR↗

arXiv2024

The Art of Deception: Color Visual Illusions and Diffusion Models

Visual illusions in humans arise when interpreting out-of-distribution stimuli: if the observer is adapted to certain statistics, perception of outliers deviates from reality. Recent studies have shown that artificial neural networks (ANNs) can also be deceived by visual illusions. This revelation raises profound questions about the nature of visual information. Why are two independent systems, both human brains and ANNs, susceptible to the same illusions? Should any ANN be capable of perceiving visual illusions? Are these perceptions a feature or a flaw? In this work, we study how visual illusions are encoded in diffusion models. Remarkably, we show that they present human-like brightness/color shifts in their latent space. We use this fact to demonstrate that diffusion models can predict visual illusions. Furthermore, we also show how to generate new unseen visual illusions in realistic images using text-to-image diffusion models. We validate this ability through psychophysical experiments that show how our model-generated illusions also fool humans.

cs.CV↗

arXiv2024

Single-View Graph Contrastive Learning with Soft Neighborhood Awareness

Most graph contrastive learning (GCL) methods heavily rely on cross-view contrast, thus facing several concomitant challenges, such as the complexity of designing effective augmentations, the potential for information loss between views, and increased computational costs. To mitigate reliance on cross-view contrasts, we propose \ttt{SIGNA}, a novel single-view graph contrastive learning framework. Regarding the inconsistency between structural connection and semantic similarity of neighborhoods, we resort to soft neighborhood awareness for GCL. Specifically, we leverage dropout to obtain structurally-related yet randomly-noised embedding pairs for neighbors, which serve as potential positive samples. At each epoch, the role of partial neighbors is switched from positive to negative, leading to probabilistic neighborhood contrastive learning effect. Furthermore, we propose a normalized Jensen-Shannon divergence estimator for a better effect of contrastive learning. Surprisingly, experiments on diverse node-level tasks demonstrate that our simple single-view GCL framework consistently outperforms existing methods by margins of up to 21.74% (PPI). In particular, with soft neighborhood awareness, SIGNA can adopt MLPs instead of complicated GCNs as the encoder to generate representations in transductive learning tasks, thus speeding up its inference process by 109 times to 331 times. The source code is available at https://github.com/sunisfighting/SIGNA.

cs.LG↗

arXiv2024

A multimodal ensemble approach for clear cell renal cell carcinoma treatment outcome prediction

Purpose: A reliable cancer prognosis model for clear cell renal cell carcinoma (ccRCC) can enhance personalized treatment. We developed a multi-modal ensemble model (MMEM) that integrates pretreatment clinical data, multi-omics data, and histopathology whole slide image (WSI) data to predict overall survival (OS) and disease-free survival (DFS) for ccRCC patients. Methods: We analyzed 226 patients from The Cancer Genome Atlas Kidney Renal Clear Cell Carcinoma (TCGA-KIRC) dataset, which includes OS, DFS follow-up data, and five data modalities: clinical data, WSIs, and three multi-omics datasets (mRNA, miRNA, and DNA methylation). Separate survival models were built for OS and DFS. Cox-proportional hazards (CPH) model with forward feature selection is used for clinical and multi-omics data. Features from WSIs were extracted using ResNet and three general-purpose foundation models. A deep learning-based CPH model predicted survival using encoded WSI features. Risk scores from all models were combined based on training performance. Results: Performance was assessed using concordance index (C-index) and AUROC. The clinical feature-based CPH model received the highest weight for both OS and DFS tasks. Among WSI-based models, the general-purpose foundation model (UNI) achieved the best performance. The final MMEM model surpassed single-modality models, achieving C-indices of 0.820 (OS) and 0.833 (DFS), and AUROC values of 0.831 (3-year patient death) and 0.862 (cancer recurrence). Using predicted risk medians to stratify high- and low-risk groups, log-rank tests showed improved performance in both OS and DFS compared to single-modality models. Conclusion: MMEM is the first multi-modal model for ccRCC patients, integrating five data modalities. It outperformed single-modality models in prognostic ability and has the potential to assist in ccRCC patient management if independently validated.

cs.CV↗

arXiv2024

Multi-Modal Environmental Sensing Based Path Loss Prediction for V2I Communications

The stability and reliability of wireless data transmission in vehicular networks face significant challenges due to the high dynamics of path loss caused by the complexity of rapidly changing environments. This paper proposes a multi-modal environmental sensing-based path loss prediction architecture (MES-PLA) for V2I communications. First, we establish a multi-modal environment data and channel joint acquisition platform to generate a spatio-temporally synchronized and aligned dataset of environmental and channel data. Then we designed a multi-modal feature extraction and fusion network (MFEF-Net) for multi-modal environmental sensing data. MFEF-Net extracts features from RGB images, point cloud data, and GPS information, and integrates them with an attention mechanism to effectively leverage the strengths of each modality. The simulation results demonstrate that the Root Mean Square Error (RMSE) of MES-PLA is 2.20 dB, indicating a notable improvement in prediction accuracy compared to single-modal sensing data input. Moreover, MES-PLA exhibits enhanced stability under varying illumination conditions compared to single-modal methods.

eess.SP↗

arXiv2024

Influence Minimization via Blocking Strategies

We study the influence minimization problem: given a graph $G$ and a seed set $S$, blocking at most $b$ nodes or $b$ edges such that the influence spread of the seed set is minimized. This is a pivotal yet underexplored aspect of network analytics, which can limit the spread of undesirable phenomena in networks, such as misinformation and epidemics. Given the inherent NP-hardness of the problem under the IC and LT models, previous studies have employed greedy algorithms and Monte Carlo Simulations for its resolution. However, existing techniques become cost-prohibitive when applied to large networks due to the necessity of enumerating all the candidate blockers and computing the decrease in expected spread from blocking each of them. This significantly restricts the practicality and effectiveness of existing methods, especially when prompt decision-making is crucial. In this paper, we propose the AdvancedGreedy algorithm, which utilizes a novel graph sampling technique that incorporates the dominator tree structure. We find that AdvancedGreedy can achieve a $(1-1/e-ε)$-approximation in the problem under the LT model. Experimental evaluations on real-life networks reveal that our proposed algorithms exhibit a significant enhancement in efficiency, surpassing the state-of-the-art algorithm by three orders of magnitude, while achieving high effectiveness.

cs.DB↗

arXiv2024

A Stitch in Time Saves Nine: Small VLM is a Precise Guidance for Accelerating Large VLMs

Vision-language models (VLMs) have shown remarkable success across various multi-modal tasks, yet large VLMs encounter significant efficiency challenges due to processing numerous visual tokens. A promising approach to accelerating large VLM inference is using partial information, such as attention maps from specific layers, to assess token importance and prune less essential tokens. However, our study reveals three key insights: (i) Partial attention information is insufficient for accurately identifying critical visual tokens, resulting in suboptimal performance, especially at low token retention ratios; (ii) Global attention information, such as the attention map aggregated across all layers, more effectively preserves essential tokens and maintains comparable performance under aggressive pruning. However, the attention maps from all layers requires a full inference pass, which increases computational load and is therefore impractical in existing methods; and (iii) The global attention map aggregated from a small VLM closely resembles that of a large VLM, suggesting an efficient alternative. Based on these findings, we introduce a \textbf{training-free} method, \underline{\textbf{S}}mall VLM \underline{\textbf{G}}uidance for accelerating \underline{\textbf{L}}arge VLMs (\textbf{SGL}). Specifically, we employ the attention map aggregated from a small VLM to guide visual token pruning in a large VLM. Additionally, an early exiting mechanism is developed to fully use the small VLM's predictions, dynamically invoking the larger VLM only when necessary, yielding a superior trade-off between accuracy and computation. Extensive evaluations across 11 benchmarks demonstrate the effectiveness and generalizability of SGL, achieving up to 91\% pruning ratio for visual tokens while retaining competitive performance.

cs.CV↗

arXiv2024

Entangling independent particles by path identity

Quantum entanglement -- correlations of particles that are stronger than any classical analogue -- is the basis for research on the foundations of quantum mechanics and for practical applications such as quantum networks. Traditionally, entanglement is achieved through local interactions or via entanglement swapping, where entanglement at a distance is generated through previously established entanglement and Bell-state measurements. However, the precise requirements enabling the generation of quantum entanglement without traditional local interactions remain less explored. Here we demonstrate that independent particles can be entangled without the need for direct interaction, prior established entanglement, or Bell-state measurements, by exploiting the indistinguishability of the origins of photon pairs. Our demonstrations challenge the long-standing belief that the prior generation and measurement of entanglement are necessary prerequisites for generating entanglement between independent particles that do not share a common past. In addition to its foundational interest, we show that this technique might lower the resource requirements in quantum networks, by reducing the complexity of photon sources and the overhead photon numbers.

quant-ph↗

arXiv2024

Revisiting the hyperfine interval for the $2s2p$ $^3\!P_{J}$ state in $^9$Be

Using relativistic multiconfiguration Dirac-Hartree-Fock method, we calculate the hyperfine-structure properties of the $2s2p$ $^3\!P_{J}$ state in $^9$Be. The hyperfine-structure properties encompass first-order hyperfine-structure parameters, as well as second-order and third-order corrections arising from the hyperfine mixing of different $2s2p$ $^3\!P_{J}$ levels. Based on our theoretical results, we reanalyze the previously reported measurement of the hyperfine interval for the $2s2p$ $^3\!P$ state in $^9$Be [A. G. Blachman and A. Lurio, Phys. Rev. 153, 164(1967)], yielding updated hyperfine-structure constants. Our results show that the hyperfine-structure constant $B$ of $2s2p$ $^3\!P_{1}$ is notably sensitive to second-order correction. Conversely, accurately determining the hyperfine-structure constant $B$ of $2s2p$ $^3\!P_{2}$ necessitates consideration of the hyperfine-structure constant $C$ in the first-order hyperfine interaction equation. The updated hyperfine-structure constant $B$ of the $2s2p$ $^3\!P_{2}$ state is found to be $1.4542(67)$~MHz, which is approximately $1.7\%$ larger than the previous value of $1.427(9)$~MHz. By combining our theoretical results with the updated hyperfine-structure constant for the $2s2p$ $^3\!P_{2}$ state, we extract the electric quadrupole moment $Q$ of $^9$Be nucleus to be $0.05320(50)$~b. This value is consistent with the most recent determination using the few-body precision calculation method. Additional, we also discuss the reasons for the discrepancy between the $Q$ values obtained through few-body and previous many-body calculations.

physics.atom-ph↗

arXiv2024

Computational metaoptics for imaging

Metasurfaces -- ultrathin structures composed of subwavelength optical elements -- have revolutionized light manipulation by enabling precise control over electromagnetic waves' amplitude, phase, polarization, and spectral properties. Concurrently, computational imaging leverages algorithms to reconstruct images from optically processed signals, overcoming limitations of traditional imaging systems. This review explores the synergistic integration of metaoptics and computational imaging, "computational metaoptics," which combines the physical wavefront shaping ability of metasurfaces with advanced computational algorithms to enhance imaging performance beyond conventional limits. We discuss how computational metaoptics addresses the inherent limitations of single-layer metasurfaces in achieving multifunctionality without compromising efficiency. By treating metasurfaces as physical preconditioners and co-designing them with reconstruction algorithms through end-to-end (inverse) design, it is possible to jointly optimize the optical hardware and computational software. This holistic approach allows for the automatic discovery of optimal metasurface designs and reconstruction methods that significantly improve imaging capabilities. Advanced applications enabled by computational metaoptics are highlighted, including phase imaging and quantum state measurement, which benefit from the metasurfaces' ability to manipulate complex light fields and the computational algorithms' capacity to reconstruct high-dimensional information. We also examine performance evaluation challenges, emphasizing the need for new metrics that account for the combined optical and computational nature of these systems. Finally, we identify new frontiers in computational metaoptics which point toward a future where computational metaoptics may play a central role in advancing imaging science and technology.

physics.optics↗

arXiv2024

Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesis

Although text-to-image (T2I) models exhibit remarkable generation capabilities, they frequently fail to accurately bind semantically related objects or attributes in the input prompts; a challenge termed semantic binding. Previous approaches either involve intensive fine-tuning of the entire T2I model or require users or large language models to specify generation layouts, adding complexity. In this paper, we define semantic binding as the task of associating a given object with its attribute, termed attribute binding, or linking it to other related sub-objects, referred to as object binding. We introduce a novel method called Token Merging (ToMe), which enhances semantic binding by aggregating relevant tokens into a single composite token. This ensures that the object, its attributes and sub-objects all share the same cross-attention map. Additionally, to address potential confusion among main objects with complex textual prompts, we propose end token substitution as a complementary strategy. To further refine our approach in the initial stages of T2I generation, where layouts are determined, we incorporate two auxiliary losses, an entropy loss and a semantic binding loss, to iteratively update the composite token to improve the generation integrity. We conducted extensive experiments to validate the effectiveness of ToMe, comparing it against various existing methods on the T2I-CompBench and our proposed GPT-4o object binding benchmark. Our method is particularly effective in complex scenarios that involve multiple objects and attributes, which previous methods often fail to address. The code will be publicly available at \url{https://github.com/hutaihang/ToMe}.

cs.CV↗

arXiv2024

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark

In the age of large-scale language models, benchmarks like the Massive Multitask Language Understanding (MMLU) have been pivotal in pushing the boundaries of what AI can achieve in language comprehension and reasoning across diverse domains. However, as models continue to improve, their performance on these benchmarks has begun to plateau, making it increasingly difficult to discern differences in model capabilities. This paper introduces MMLU-Pro, an enhanced dataset designed to extend the mostly knowledge-driven MMLU benchmark by integrating more challenging, reasoning-focused questions and expanding the choice set from four to ten options. Additionally, MMLU-Pro eliminates the trivial and noisy questions in MMLU. Our experimental results show that MMLU-Pro not only raises the challenge, causing a significant drop in accuracy by 16% to 33% compared to MMLU but also demonstrates greater stability under varying prompts. With 24 different prompt styles tested, the sensitivity of model scores to prompt variations decreased from 4-5% in MMLU to just 2% in MMLU-Pro. Additionally, we found that models utilizing Chain of Thought (CoT) reasoning achieved better performance on MMLU-Pro compared to direct answering, which is in stark contrast to the findings on the original MMLU, indicating that MMLU-Pro includes more complex reasoning questions. Our assessments confirm that MMLU-Pro is a more discriminative benchmark to better track progress in the field.

cs.CL↗

arXiv2024

Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning

Accurate emotion perception is crucial for various applications, including human-computer interaction, education, and counseling. However, traditional single-modality approaches often fail to capture the complexity of real-world emotional expressions, which are inherently multimodal. Moreover, existing Multimodal Large Language Models (MLLMs) face challenges in integrating audio and recognizing subtle facial micro-expressions. To address this, we introduce the MERR dataset, containing 28,618 coarse-grained and 4,487 fine-grained annotated samples across diverse emotional categories. This dataset enables models to learn from varied scenarios and generalize to real-world applications. Furthermore, we propose Emotion-LLaMA, a model that seamlessly integrates audio, visual, and textual inputs through emotion-specific encoders. By aligning features into a shared space and employing a modified LLaMA model with instruction tuning, Emotion-LLaMA significantly enhances both emotional recognition and reasoning capabilities. Extensive evaluations show Emotion-LLaMA outperforms other MLLMs, achieving top scores in Clue Overlap (7.83) and Label Overlap (6.25) on EMER, an F1 score of 0.9036 on MER2023-SEMI challenge, and the highest UAR (45.59) and WAR (59.37) in zero-shot evaluations on DFEW dataset.

cs.AI↗

arXiv2024

Aligning Large Language Models with Representation Editing: A Control Perspective

Aligning large language models (LLMs) with human objectives is crucial for real-world applications. However, fine-tuning LLMs for alignment often suffers from unstable training and requires substantial computing resources. Test-time alignment techniques, such as prompting and guided decoding, do not modify the underlying model, and their performance remains dependent on the original model's capabilities. To address these challenges, we propose aligning LLMs through representation editing. The core of our method is to view a pre-trained autoregressive LLM as a discrete-time stochastic dynamical system. To achieve alignment for specific objectives, we introduce external control signals into the state space of this language dynamical system. We train a value function directly on the hidden states according to the Bellman equation, enabling gradient-based optimization to obtain the optimal control signals at test time. Our experiments demonstrate that our method outperforms existing test-time alignment techniques while requiring significantly fewer resources compared to fine-tuning methods. Our code is available at https://github.com/Lingkai-Kong/RE-Control.

cs.AI↗

arXiv2024

Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability,Reproducibility, and Practicality

Recent text-to-video (T2V) technology advancements, as demonstrated by models such as Gen2, Pika, and Sora, have significantly broadened its applicability and popularity. Despite these strides, evaluating these models poses substantial challenges. Primarily, due to the limitations inherent in automatic metrics, manual evaluation is often considered a superior method for assessing T2V generation. However, existing manual evaluation protocols face reproducibility, reliability, and practicality issues. To address these challenges, this paper introduces the Text-to-Video Human Evaluation (T2VHE) protocol, a comprehensive and standardized protocol for T2V models. The T2VHE protocol includes well-defined metrics, thorough annotator training, and an effective dynamic evaluation module. Experimental results demonstrate that this protocol not only ensures high-quality annotations but can also reduce evaluation costs by nearly 50\%. We will open-source the entire setup of the T2VHE protocol, including the complete protocol workflow, the dynamic evaluation component details, and the annotation interface code. This will help communities establish more sophisticated human assessment protocols.

cs.CV↗