Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 2506.23094

TOMI: Transforming and Organizing Music Ideas for Multi-Track Compositions with Full-Song Structure

Qi He·Gus Xia·Ziyu Wang
View original source ↗Read source document ↗Export metadata ↓

Abstract

Hierarchical planning is a powerful approach to model long sequences structurally. Aside from considering hierarchies in the temporal structure of music, this paper explores an even more important aspect: concept hierarchy, which involves generating music ideas, transforming them, and ultimately organizing them--across musical time and space--into a complete composition. To this end, we introduce TOMI (Transforming and Organizing Music Ideas) as a novel approach in deep music generation and develop a TOMI-based model via instruction-tuned foundation LLM. Formally, we represent a multi-track composition process via a sparse, four-dimensional space characterized by clips (short audio or MIDI segments), sections (temporal positions), tracks (instrument layers), and transformations (elaboration methods). Our model is capable of generating multi-track electronic music with full-song structure, and we further integrate the TOMI-based model with the REAPER digital audio workstation, enabling interactive human-AI co-creation. Experimental results demonstrate that our approach produces higher-quality electronic music with stronger structural coherence compared to baselines.

Explore related subjects

cs.SDcs.AIeess.AS

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Qi He, Gus Xia, Ziyu Wang. 2025-06-29. TOMI: Transforming and Organizing Music Ideas for Multi-Track Compositions with Full-Song Structure. https://arxiv.org/abs/2506.23094

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Retain-Free Machine Unlearning for Speech Emotion Recognition

Speech Emotion Recognition (SER) infers a speaker's emotional state from speech and is increasingly deployed in human-computer interaction, education, and healthcare. Because speech also carries sensitive personal information, speakers may ask that some of their recordings be deleted, which requires removing the influence of those samples from an already trained SER model. Most machine unlearning methods can meet this request only with access to the remaining training data alongside the samples to be forgotten; this is impractical when the remaining data cannot be redistributed or has itself been deleted, and it adds storage and computation as the data grows. To this end, we propose a retain-free unlearning method that updates a pre-trained SER model using only the forget set. Our key idea is to synthesise adversarial samples from the forget set as a surrogate for the unavailable remaining data, and to constrain each parameter update by its estimated importance so that forgetting does not erase general knowledge. The experiments over several emotional-speech corpora and self-supervised backbones show that our method drives forget-set performance down to near chance while retaining much of the model's utility on the remaining and unseen-speaker data, narrowing the gap to methods that rely on the remaining set.

cs.SD↗
arXiv2026

SHINE: Sequential Hierarchical Integration Network for EEG and MEG

How natural speech is represented in the brain constitutes a major challenge for cognitive neuroscience. Reconstructing the speech envelope and Mel spectrogram from EEG and MEG provides a time-resolved way to study its temporal and spectral structure. Speech-related neural activity spans sensors and temporal scales; extracting these representations while adapting the use of context to each acoustic target is a central problem in speech reconstruction. We propose SHINE, a Sequential Hierarchical Integration Network for EEG and MEG. A residual sensor adapter unifies input dimensions, intermediate dilated-block states retain temporal depth, and a target- and time-dependent gate fuses local hierarchical and attention-enhanced context predictions. Across two EEG and two MEG datasets, SHINE has the highest mean envelope and mean-Mel Pearson correlations among nine local baseline implementations on all eight dataset-metric combinations. SHINE also placed second in the speech-detection Extended Track of the NeurIPS 2025 PNPL Competition. Code will be released at https://github.com/xuxiran/SHINE.

cs.SD↗
arXiv2026

Self-Distilled Pronunciation and Accent Control for Neural Text-to-Speech

Text-to-speech that reads raw text has no lexicon: a rare word is read as guessed, and a native Japanese listener accepts a word only if its reading and pitch accent are both right. A known remedy installs a reading-and-accent channel into a released model, but it needs many recordings. This paper removes the recordings: the frozen backbone reads a sentence containing a common word it already says correctly, and that output serves as the teacher for the same sentence with the word replaced by an annotated reading with its pitch accent. Screened raters judged the tag right on 0.80 to 0.93 of unseen difficult words on four backbones spanning autoregressive, diffusion, and encoder-decoder synthesis; plain kana, which cannot express an accent, got 0.38 to 0.60. On words needing no edit, naturalness is non-inferior on one backbone; on the other three, listeners prefer the unedited rendition by 0.19 to 0.26.

cs.SD↗