arXiv subjects

Haining Zhang

Publications and source records attributed to Haining Zhang.

7 recordsLinked to original sources

arXiv2026

Measuring Smartphone User Experience through a Hierarchical Metric Framework via Social Media Reviews

Smartphone user experience (UX) is widely expressed in user-generated online discourse across platforms, creating opportunities for in-the-wild measurement at scale. However, existing UX instruments and review-mining approaches do not provide a smartphone-oriented, theory-grounded hierarchical measurement specification that supports consistent aggregation and comparison across heterogeneous platforms. In this research, we propose a hierarchical smartphone UX measurement framework and an interpretable computational pipeline that translates cross-platform reviews into structured UX metrics. The pipeline extracts localized experience evidence units, maps them to the hierarchy via coarse-to-fine classification, and quantifies evaluations with a unified five-level satisfaction sentiment model. We apply the approach to a stratified subset of approximately 20,000 Chinese social media reviews covering four major smartphone brands across three platforms. The resulting metrics separate what users discuss, captured by normalized mention frequency, from how they evaluate it, captured by mean five-level sentiment scores. This work provides a scalable and interpretable evidence base for cross-brand comparison and metric-level interpretation beyond raw review volume or single-platform observations.

cs.HC↗

arXiv2026

EvalMem: An Operation-Level Diagnostic Framework for Long-Term Memory Systems

Long-horizon interactions with LLM-based assistants require memory systems that preserve and update user states, preferences, and interaction histories. Existing evaluations report end-to-end QA accuracy and cannot determine whether errors arise from encoding, retrieval, or generation. We introduce EvalMem, an operation-level diagnostic framework with three parallel Examiners. For each query, the Encoding Examiner checks whether the target fact is stored, the Retrieval Examiner assesses whether the native retriever returns usable evidence, and the Generation Examiner tests whether the model can answer from oracle evidence. Their outputs form fine-grained multi-label defect codes. To improve store-level diagnosis, we adapt agentic RAG with a recall-first strategy that searches using both the query and source evidence, increasing recall of present evidence on LoCoMo from 70.2% to 95.6%. Evaluations of seven memory systems on LoCoMo, LongMemEval-S, and dynamic DynaMem-Bench identify retrieval as the most frequently attributed failure layer; in default LoCoMo, retrieval defects reach 22.1%, compared with 7.7% for encoding and 6.5% for generation. Guided by this diagnosis, MemWiki, a search-friendly auxiliary structure built from each system's memory export, improves mean accuracy by 2.5 and 2.3 percentage points on LoCoMo and LongMemEval-S, respectively.

cs.CL↗

arXiv2026

Division of Labor and Collaboration Between Parents in Family Education

Homework tutoring work is a demanding and often conflict-prone practice in family life, and parents often lack targeted support for managing its cognitive and emotional burdens. Through interviews with 18 parents of children in grades 1-3, we examine how homework-related labor is divided and coordinated between parents, and where AI might meaningfully intervene. We found three key insights: (1) Homework labor encompasses distinct dimensions: physical, cognitive, and emotional, with the latter two often remaining invisible. (2) We identified father-mother-child triadic dynamics in labor division, with children's feedback as the primary factor shaping parental labor adjustments. (3) Building on prior HCI research, we propose an AI design that prioritizes relationship maintenance over task automation or broad labor mitigation. By employing labor as a lens that integrates care work, we explore the complexities of labor within family contexts, contributing to feminist and care-oriented HCI and to the development of context-sensitive coparenting practices.

cs.HC↗

arXiv2026

A3: Android Agent Arena for Mobile GUI Agents with Essential-State Procedural Evaluation

The advancement of Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) has catalyzed the development of mobile graphic user interface (GUI) AI agents, which is designed to autonomously perform tasks on mobile devices. However, a significant gap persists in mobile GUI agent evaluation, where existing benchmarks predominantly rely on either static frame assessments such as AndroidControl or offline static apps such as AndroidWorld and thus fail to capture agent performance in dynamic, real-world online mobile apps. To address this gap, we present Android Agent Arena (A3), a novel "essential-state" based procedural evaluation system for mobile GUI agents. A3 introduces a benchmark of 100 tasks derived from 20 widely-used, dynamic online apps across 20 categories from the Google Play Store, ensuring evaluation comprehension. A3 also presents a novel "essential-state" based procedural evaluation method that leverages MLLMs as reward models to progressively verify task completion and process achievement. This evaluation approach address the limitations of traditional function based evaluation methods on online dynamic apps. Furthermore, A3 includes a toolkit to streamline Android device interaction, reset online environment and apps and facilitate data collection from both human and agent demonstrations. The complete A3 system, including the benchmark and tools, will be publicly released to provide a robust foundation for future research and development in mobile GUI agents.

cs.AI↗

arXiv2025

Context-Aware Intelligent Chatbot Framework Leveraging Mobile Sensing

With the rapid advancement of large language models (LLMs), intelligent conversational assistants have demonstrated remarkable capabilities across various domains. However, they still mainly rely on explicit textual input and do not know the real world behaviors of users. This paper proposes a context-sensitive conversational assistant framework grounded in mobile sensing data. By collecting user behavior and environmental data through smartphones, we abstract these signals into 16 contextual scenarios and translate them into natural language prompts, thus improving the model's understanding of the user's state. We design a structured prompting system to guide the LLM in generating a more personalized and contextually relevant dialogue. This approach integrates mobile sensing with large language models, demonstrating the potential of passive behavioral data in intelligent conversation and offering a viable path toward digital health and personalized interaction.

cs.HC↗

arXiv2025

An Automated Multi-modal Evaluation Framework for Mobile Intelligent Assistants Based on Large Language Models and Multi-Agent Collaboration

With the rapid development of mobile intelligent assistant technologies, multi-modal AI assistants have become essential interfaces for daily user interactions. However, current evaluation methods face challenges including high manual costs, inconsistent standards, and subjective bias. This paper proposes an automated multi-modal evaluation framework based on large language models and multi-agent collaboration. The framework employs a three-tier agent architecture consisting of interaction evaluation agents, semantic verification agents, and experience decision agents. Through supervised fine-tuning on the Qwen3-8B model, we achieve a significant evaluation matching accuracy with human experts. Experimental results on eight major intelligent agents demonstrate the framework's effectiveness in predicting users' satisfaction and identifying generation defects.

cs.AI↗

arXiv2025

Design and Challenges of Mental Health Assessment Tools Based on Natural Language Interaction

Mental health assessments are of central importance to individuals' well-being. Conventional assessment methodologies predominantly depend on clinical interviews and standardised self-report questionnaires. Nevertheless, the efficacy of these methodologies is frequently impeded by factors such as subjectivity, recall bias, and accessibility issues. Furthermore, concerns regarding bias and privacy may result in misreporting in data collected through self-reporting in mental health research. The present study examined the design opportunities and challenges inherent in the development of a mental health assessment tool based on natural language interaction with large language models (LLMs). An interactive prototype system was developed using conversational AI for non-invasive mental health assessment, and was evaluated through semi-structured interviews with 11 mental health professionals (six counsellors and five psychiatrists). The analysis identified key design considerations for future development, highlighting how AI-driven adaptive questioning could potentially enhance the reliability of self-reported data while identifying critical challenges, including privacy protection, algorithmic bias, and cross-cultural applicability. This study provides an empirical foundation for mental health technology innovation by demonstrating the potential and limitations of natural language interaction in mental health assessment.

cs.HC↗