Search arXivSearch

arXiv subjects

Miki Okamura

Publications and source records attributed to Miki Okamura.

3 recordsLinked to original sources

Onomatopoeia Cursor: Verbal Mirroring of Mouse Movement with Comic-Style Lettering

The mouse cursor has remained visually mute for half a century: it shows where we point, but says nothing about how we move. We present the Onomatopoeia Cursor, a shipped macOS overlay that classifies cursor kinematics in real time and renders Japanese mimetic words (onomatopoeia) as animated comic-style lettering above the cursor -- "kyorokyoro" (glancing around) for rapid horizontal reversals, "sorosoro" (cautiously) for slow careful motion, "byuun" (whoosh) for fast straight strokes. The system reads seven input channels and displays roughly sixty word forms across five languages. Crucially, the form of each word fluctuates with the manner of action through a morphological generator grounded in Japanese sound symbolism (voicing = weight, gemination = abruptness, elongation = extent, reduplication = iteration). Beyond the rule generator, an on-device onomatopoeia-only transformer (0.4M parameters, 9.7 ms per word) trained on 2,782 mimetic words synthesizes novel forms from the manner of an action; for sound-definite events, a family-anchored generator constrains synthesis to the correct phonetic family. Characters are rendered with hand-drawn outline perturbation, brush-style deformation, and per-character animation grounded in manga lettering conventions. We articulate the design space of verbal motion mirroring, formalize the pipeline as a learnable differentiable mapping, and report a technical evaluation of what is actually implemented and measured. Our central conjecture concerns the sense of agency: formative first-person use suggests that naming a movement while it happens perturbs the felt authorship of the action -- modulation, amplification, and interference -- and we outline a within-subjects study with salience-matched controls as future work. No user-study results are claimed; the contribution is the concept, the working system, and its design space.

cs.HC

OnomaCompass: A Texture Exploration Interface that Shuttles between Words and Images

Humans can finely perceive material textures, yet articulating such somatic impressions in words is a cognitive bottleneck in design ideation. We present OnomaCompass, a web-based exploration system that links sound-symbolic onomatopoeia and visual texture representations to support early-stage material discovery. Instead of requiring users to craft precise prompts for generative AI, OnomaCompass provides two coordinated latent-space maps--one for texture images and one for onomatopoeic term--built from an authored dataset of invented onomatopoeia and corresponding textures generated via Stable Diffusion. Users can navigate both spaces, trigger cross-modal highlighting, curate findings in a gallery, and preview textures applied to objects via an image-editing model. The system also supports video interpolation between selected textures and re-embedding of extracted frames to form an emergent exploration loop. We conducted a within-subjects study with 11 participants comparing OnomaCompass to a prompt-based image-generation workflow using Gemini 2.5 Flash Image ("Nano Banana"). OnomaCompass significantly reduced workload (NASA-TLX overall, mental demand, effort, and frustration; p < .05) and increased hedonic user experience (UEQ), while usability (SUS) favored the baseline. Qualitative findings indicate that OnomaCompass helps users externalize vague sensory expectations and promotes serendipitous discovery, but also reveals interaction challenges in spatial navigation. Overall, leveraging sound symbolism as a lightweight cue offers a complementary approach to Kansei-driven material ideation beyond prompt-centric generation.

cs.HC

Dance Generation by Sound Symbolic Words

This study introduces a novel approach to generate dance motions using onomatopoeia as input, with the aim of enhancing creativity and diversity in dance generation. Unlike text and music, onomatopoeia conveys rhythm and meaning through abstract word expressions without constraints on expression and without need for specialized knowledge. We adapt the AI Choreographer framework and employ the Sakamoto system, a feature extraction method for onomatopoeia focusing on phonemes and syllables. Additionally, we present a new dataset of 40 onomatopoeia-dance motion pairs collected through a user survey. Our results demonstrate that the proposed method enables more intuitive dance generation and can create dance motions using sound-symbolic words from a variety of languages, including those without onomatopoeia. This highlights the potential for diverse dance creation across different languages and cultures, accessible to a wider audience. Qualitative samples from our model can be found at: https://sites.google.com/view/onomatopoeia-dance/home/.

cs.LG