Computation and Language · Cross-listed from cs.IR
On-Device Commercial Intent Retrieval Under Size, Latency, and Privacy Constraints: A 3 MiB Retrieval System with Typed Egress Boundaries
Hyojung Han
We study commercial intent inference that runs entirely on the user's device, under three constraints frozen before the work began: the downloaded payload under 3 MiB, Tier-0 inference under 20 ms at p95, and no raw text, content
embedding, or stable identifier leaving the device. Under them we build a
retrieval path over a 6,020-leaf commercial taxonomy: a static
embedding table distilled from a Korean sentence
transformer, quantized to 4 bits, no inference runtime. Our main result is where that constraint costs accuracy. On real Korean commerce text labelled by others (22,900 AI-Hub shopping reviews), mid-category top-5 on real product names is 75.0% against an 18.4% permutation baseline, but splits on one observable: a query containing some leaf name as a substring scores 83.5%, one containing none 45.2%. A generic 196.6x larger teacher seemed to localize the gap (+20.1 pp without an anchor, +0.1 with). That null was two effects cancelling: the same teacher fine-tuned on the student's own contrastive pairs reaches 0.8586 and beats the pure-encoder student by +10.6 pp with an anchor and +20.9 pp without. The cost is not uniform, but it is not free anywhere; where the anchor is absent, task adaptation buys the teacher nothing, so what the constrained encoder lacks there is capacity. The expensive regime is detectable on-device from the ranker's own score margin: declining the least confident fifth lifts the rest to 0.8296. A second axis we first reported, a manufacturer model code, does not survive source-category fixed effects (-4.0 pp, p=0.51); the anchor does (+13.0 pp). Payload is 2,942,652 bytes, all three library links measured. Tier-0 p95 is 4.431 and 3.670 ms on two iPhones (A14, A16) and 5.080 ms on a budget Android tablet (Snapdragon 695), all slower than three server CPUs on the same code. Taxonomy supervision is mostly synthetic Korean utterances.
Read the full abstract ▾
Computation and Language
FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speech
Puneet Mathur and Dinesh Manocha
Natural turn-taking in full-duplex voice interaction requires determining from partial
speech whether a pause reflects hesitation or a completed conversational intent. Acoustic voice activity detection lacks this semantic information, while cascaded ASR-based endpointing introduces transcription dependence and additional processing stages. We formulate semantic endpoint detection as a causal audio-
language reasoning task and introduce FD-VAD, an ASR-free streaming endpointer that maps bounded causal audio windows directly to Continue/Stop decisions. FD-VAD combines a frozen
speech encoder with a lightweight modality adapter and a parameter-efficiently adapted
language model, using a last-chunk training objective for streaming inference. We further introduce confidence-gated endpoint commitment to control interruption versus delay and boundary-focused hard-negative sampling to improve decisions around ambiguous turn boundaries. Across in-domain and conversational evaluations, FD-VAD outperforms strong streaming and non-streaming semantic turn classifiers, and achieves the highest EOT recall among qualifying systems on TurnBench dev set $0.853$ (at FP<=0.10) in a zero-shot setting. These results show that semantic endpointing can be performed directly from streaming audio without intermediate ASR or dialogue state tracking.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.LG
Sage: Formalization with Semantic Correction
Thomas Hirtz, Farzad Jafarrahmani, Abdelmouksit Sagueni et al. (+3 more)
While neural
theorem provers have achieved impressive milestones in formal mathematics, they largely operate on the assumption that faithful Lean 4 formal statements are already provided. Translating informal natural
language into a formal
language is a critical data bottleneck plagued by an "illusion of rigor": standard type-checkers accept statements that compile but drop hypotheses, introduce vacuous truths, or subtly alter mathematical bounds. To resolve this, we introduce Sage (Semantic
Agent-Guided Formalization Engine), an agentic framework that replaces monolithic translation with a four-stage decomposed generation pipeline coupled with a dual-signal semantic correction loop. By pairing Lean 4 compiler diagnostics with multi-dimensional semantic feedback, our correction loop enforces mathematical fidelity alongside syntactic validity. By explicitly accounting for the gap between open-ended queries and declarative formal targets, our pipeline prevents models from achieving high formalization rates by guessing unverified answers (exhibiting a 70.9% answer leakage rate in monolithic baselines). Consequently, Sage suppresses leakage to 2.7% while achieving 73.3% pass@4 joint compilation and semantic fidelity on the Omni-MATH without proofs (compared to 42.0% for a fine-tuned Goedel-Formalizer-V2 baseline). Finally, on IMO-Unformalized, a novel frontier of 175 unformalized International Mathematical Olympiad problems, Sage demonstrates effective zero-shot generalization with 87.4% pass@4 verified fidelity compared to just 19.4% for the baseline, winning over 79% of blind pairwise evaluations.
Read the full abstract ▾
All authors Thomas Hirtz, Farzad Jafarrahmani, Abdelmouksit Sagueni, Xiang Zhou, Wenping Deng, Liang Zhang
Computation and Language
Don't Repeat Yourself: Self-Supervised Fine-Tuning for Coverage
Eric Fithian, Kirill Skobelev and X.Y. Han
In verifiable domains such as math and coding, finding one correct solution among many attempts can matter more than the pass rate of each attempt. Post-training can concentrate large
language model outputs around a few modes, while increasing sampling temperature has limited effectiveness. We introduce Don't Repeat Yourself Supervised Fine-Tuning (DRY-SFT), a post-training method that increases output diversity and coverage: the probability of at least one correct solution among many attempts. DRY-SFT has two stages. First, for each problem, sequentially generate K solutions, showing the model all prior attempts and asking for a different solution. Second, fine-tune on each attempt independently, removing prior attempts from the context. The process uses no reward, verifier, or correctness filter. On HumanEval+, MBPP+, and DS-1000, DRY-SFT raises pass@100 by 10.8, 12.5, and 12.4 percentage points, respectively, at a small cost to pass@1. Structural diversity, measured by abstract syntax tree edit distance among passing solutions, rises significantly on all three
benchmarks. DRY-SFT also solves 244 of 600 problems that the base model did not solve in the same 200 attempts. Across nine open-weight models, lower structural diversity of the base model significantly predicts larger DRY-SFT gains, indicating that the method is especially effective on more mode-collapsed models.
Read the full abstract ▾
Computation and Language
Verification of PETSc with CIVL using LLM-generated ACSL contracts and deterministic driver generation
Hansol Suh, Jan Hückelheim and Stephen Siegel
Parallel numerical libraries such as PETSc are widely used in science and engineering applications where wrong results can have costly consequences. Despite this, numerical libraries are rarely formally verified. One of the challenges is the need for an expert to hand-write a specification and manually apply a verification tool, often requiring the development of a harness or driver, all of which can contain additional bugs that lead to false positives or false negatives during verification. With recent advancements in large
language models (LLMs), it is tempting to generate such drivers and reference models automatically, but one-shot generation based on a simple
prompt is brittle and leads to additional unverified code that needs to be audited. In this paper, we present an approach to use LLMs in a limited setting to generate a small, human-certifiable ACSL contract from the function's documentation, combined with a deterministic toolchain that supports a restricted ACSL profile and generates a driver that uses the CIVL verifier to check the implementation against the contract and, when available, an existing reference model. We demonstrate the pipeline end-to-end on three PETSc functions: MatAXPY (reference model already exists), MatAYPX (no reference model, so the certified contract is the sole oracle), and the non-compressing mode of MatFilter (no reference model, with more complex, conditional behavior). With this pipeline, we were able to discover a bug in PETSc's MatAYPX function that was previously undiscovered and had been present in the code since 1997.
Read the full abstract ▾
Computation and Language
The Temporal Tug-of-War: Visualizing and Detecting RAG Conflicts in Diffusion Models via Trajectory Variance
Sravan Karthick T, Pranav Darshan, Pranav A et al. (+2 more)
Retrieval-Augmented Generation (RAG) introduces a specific failure mode in discrete
diffusion language models: when retrieved context contradicts parametric knowledge, the iterative denoising process becomes a visible battleground between competing knowledge sources. We identify temporal semantic divergence as an observable for detecting these conflicts and introduce the Trajectory Variance Score (TVS), a simple and interpretable measure of this divergence. TVS computes the mean pairwise cosine distance of answer
embeddings across independent
stochastic denoising trajectories, capturing the temporal tug of war between parametric and contextual attractors. Requiring as few as two parallel inference runs, TVS is computationally lightweight. Across four diverse
datasets (Synthetic, SciQ, PopQA, and CounterFact), a simple Logistic Regression classifier using TVS achieves $70.10\%$ accuracy and $0.7647$ AUROC on LLaDA. On Dream 7B, increasing the number of trajectories from two to five improves accuracy from $63.91\%$ to $69.62\%$. More complex sequential models provide only marginal improvements over the linear classifier. Evaluation across LLaDA and Dream 7B demonstrates that conflict-induced trajectory dynamics and their key properties transfer across distinct
diffusion architectures.
Read the full abstract ▾
All authors Sravan Karthick T, Pranav Darshan, Pranav A, Minal Moharir, Ivan P. Yamshchikov
Computation and Language · Cross-listed from cs.LG
When Keywords Drop but Classifiers Hold: Soft Refusals under KV Cache Compression
Kang Chen, Xiuze Zhou, Hong Chen et al. (+1 more)
KV cache compression is widely used for long context LLM inference under
memory constraints, while deployed systems typically score refusals after generation with keyword filters or learned classifiers. Such monitors are intended to indicate whether a model declined a harmful request under the serving regime actually used. However, it remains unclear whether matched compression that preserves task accuracy also preserves agreement between lightweight lexical monitors and stronger refusal classifiers. We study this with a paired protocol on n=200 harmful
prompts with a long filler context: each
prompt is answered once under full retention and once under matched eviction after a shared prefill, and the same replies are scored by keyword heuristics, the HarmBench Llama-2-13B classifier, an auxiliary LLM judge, and humans on disagreements. On Qwen2.5-3B, keyword refusal falls from 98.0% to 80.5% (McNemar p~1e-8) while classifier refusal stays near ceiling (99.0%-99.5%) and MMLU accuracy is unchanged (50.0%); human labels predominantly follow the classifier, consistent with soft refusals. The gap is not universal and weakens under short fillers and paired SnapKV, so safety auditing under compression should rely on several judges matched to the serving context rather than on keyword rates alone.
Read the full abstract ▾
All authors Kang Chen, Xiuze Zhou, Hong Chen, Yuanguo Lin
Computation and Language
An Evaluation of AI-Supported Evidence-Based Learning for Public Speaking Skill Development
Sashini Hettiarachchi, Shahbaz Siddeeq, Mika Saari et al. (+1 more)
Public speaking is an essential skill in academic and professional contexts, but it often causes anxiety. Although several AI-based
speech coaching tools exist, they typically provide generic feedback and lack model
speeches for learning. This study addresses these gaps by developing a system that incorporates
speech context into evaluation criteria, enabling more tailored feedback. It also generates improved model
speeches in both text and audio formats to support example-based learning. The system was evaluated in a pilot study with seven undergraduate students over one to three weeks. Participants used the system repeatedly and practiced the same
speech at least three times. Results showed improvements in public speaking performance for all participants. Filler word usage decreased, and anxiety levels dropped by 15.3% to 34.4%. Participants found the context-aware feedback and revised
speeches useful and confidence-building. These findings suggest that context-aware feedback and model
speeches can enhance public speaking skills and reduce anxiety. Future studies could explore
video-based analysis of physical behavior during
speeches.
Read the full abstract ▾
All authors Sashini Hettiarachchi, Shahbaz Siddeeq, Mika Saari, Pekka Abrahamsson
Computation and Language
A Benchmark Framework for Screening Automation in Systematic Reviews
Gauransh Kumar, Luciano Marchezan, Guillaume Genois et al. (+2 more)
Systematic reviews (SR) are essential for evidence-based research, but their screening phase is highly time-consuming and labor-intensive. Large
language models (LLMs) offer a promising opportunity to reduce this workload by assisting with article relevance classification. However, existing evaluation approaches often rely on traditional metrics that may be misleading for highly imbalanced SR screening
datasets. This paper presents a
benchmark dataset of $45\,064$ labeled entries for evaluating LLM performance in SR screening across 32 curated secondary studies. It proposes an evaluation framework that accounts for class imbalance, i.e., the natural prevalence of excluded articles relative to included articles in SRs. It also introduces PromptSR, a tool designed to support
prompt experimentation, experiment management, and result analysis for LLM-based screening. We also present a use case demonstrating the application of SRBench and PromptSR.
Read the full abstract ▾
All authors Gauransh Kumar, Luciano Marchezan, Guillaume Genois, Kévin Delcourt, Eugene Syriani
Computation and Language
Bootstrapping Conversational Recommendation Agents At Spotify: Synthetic Data Generation and Self-Improvement Loops
Enrico Palumbo, Alexandre Tamborrino, Victor Ode et al. (+15 more)
Conversational recommendation
agents are a new paradigm for content discovery, enabling users to express complex intents through natural
language (e.g., "recommend Italian indie artists I haven't heard before"). A central challenge in building such
agents is optimizing
agent planning – deciding how to select, sequence, and invoke tools – particularly in cold-start settings where real user interactions are not yet available. We introduce a pipeline for multi-turn synthetic data generation and a self-improvement loop to address this challenge. The synthetic data pipeline transforms single-turn
prompts into realistic multi-turn conversations, enabling systematic evaluation before launch. The self-improvement loop combines variance-based contrastive optimization with iterative refinement through a coding
agent, automatically identifying and fixing planning and tool-use errors. Our approach improves quality by +8% on top of a highly optimized manual
prompt. The system has been productionized and significantly accelerated iteration cycles for the launch of a conversational recommendation
agent at Spotify. Online A/B tests demonstrate its effectiveness, with +14% user listening, +5% increase in weekly active users, and a 5% reduction in skip rate compared to a prior experience supporting only session refinement. This work provides a practical framework for accelerating the development of conversational recommendation
agents in industry.
Read the full abstract ▾
All authors Enrico Palumbo, Alexandre Tamborrino, Victor Ode, Ben Lacker, Adrià Casas Escoda, Jeremy Hopple, Marcus Better, James Leoni, Hugo Galvão, Hugues Bouchard, Mounia Lalmas, José Luis Redondo García, Abenezer Abebe, Ann Clifton, Anton Blomberg, Henrik Lindström, Dani Doro, Christine Doig Cardet
Computation and Language
Understanding Reliability in LLM-based Human Behavior Simulation
Pei Wang, Lei Wang, Yuanzi Li et al. (+1 more)
Large
language models (LLMs) are increasingly used to simulate human survey responses and behavioral reactions, yet unreliable simulations can mislead social science conclusions. However, existing evaluations focus on end-to-end scores, leaving it unclear how different aspects of the simulation process interact to determine reliability. We propose ReliMap, which decomposes LLM-based human behavior simulation into three structured layers and evaluates reliability at both the individual level (R1) and population level (R2) across three configuration dimensions: model capacity, profile completeness, and population coverage. Through experiments across four simulation tasks and eleven LLMs, we find that all models exhibit substantial distributional bias without profile conditioning. Profile conditioning reduces this bias with diminishing returns. Larger models benefit more, and attribute informativeness matters more than quantity. Critically, R1 gains do not reliably transfer to R2–individual and population-level reliability can move in opposite directions. At the population layer, increasing coverage reduces variance but not systematic bias, with R2 stabilizing at around 50-100 individuals. These findings highlight that reliable simulation cannot be achieved by optimizing any single layer in isolation, but requires coordinated improvement across all three.
Read the full abstract ▾
All authors Pei Wang, Lei Wang, Yuanzi Li, Xu Chen
Computation and Language · Cross-listed from cs.LG
BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence
Chuxuan Hu, Yeye He, Penny Zhou et al. (+3 more)
Business intelligence (BI) is a cornerstone of enterprise decision-making and is widely used by enterprise users in software such as Power BI and Tableau. In traditional BI workflows, users need to prepare data by (1) identifying relevant tables, (2) performing data transformations, and (3) building join relationships, before they can (4) answer their business questions. These steps can be complex and time-consuming, making BI challenging. Given the strong capabilities of large
language models (LLMs) in working with data, we study their ability to answer BI questions end-to-end, without requiring users to manually perform the tedious preparation steps. To do this, we harvest a large collection of real-world BI projects from public sources, and manually extract pairs of (questions, ground-truth answers) from real user dashboards. The resulting
benchmark, BI-Bench, is the first
benchmark to systematically study LLMs' ability on end-to-end BI. We find that even frontier LLMs perform poorly on BI-Bench, with less than 50% accuracy. To address their limitations, we design a tool-augmented BI-
Agent that decomposes BI workflows into subtasks on structured data, such as search, join, and transform, and orchestrates specialized data management methods across BI stages. Furthermore, we develop a post-training framework that synthesizes training trajectories from real BI projects, enabling BI-
Agent to be further post-trained using both supervised fine-tuning (SFT) and reinforcement learning (RL). BI-
Agent achieves substantial accuracy gains of up to 40 percentage points with vanilla LLMs, and post-trained BI-
Agent yields gains of up to 30 points. Our results highlight the importance of combining tool-augmented
reasoning with domain-specific post-training in complex BI workflows, and point to promising directions for future research.
Read the full abstract ▾
All authors Chuxuan Hu, Yeye He, Penny Zhou, Wee Hyong Tok, Daniel Kang, Surajit Chaudhuri
Computation and Language · Cross-listed from eess.AS
Cross-Lingual Parkinson's Disease Severity Assessment Using Pre-trained Speech Embeddings: A Multi-Class Evaluation
Simon Pals and Cristian Tejedor-Garcia
Parkinson's disease (PD) often manifests through
speech impairments, facilitating accessible, non-invasive, and cost-effective severity assessment for early diagnosis and progression tracking. Despite advances in
speech foundation models (SFMs), their cross-lingual generalization for PD severity multi-class classification remains underexplored due to limited labeled data, a lack of explainable methods and variability across
languages and
datasets. In this work, we evaluate pre-trained
embeddings from four state-of-the-art open-source SFMs across three
datasets in zero-shot and k-shot cross-lingual settings for multi-class PD severity assessment. Our results show that pre-trained
speech embeddings enable meaningful cross-lingual transfer, although performance is sensitive to
dataset properties, preprocessing, and adaptation strategy. Misclassifications under these conditions related to inter-speaker variability and atypical
speech patterns highlight the need for more robust feature extraction and modeling for PD severity assessment while emphasizing the importance of explainability for reliable clinical insights.
Read the full abstract ▾
Computation and Language
For Your Eyes Only: Evaluating Coordination Between Isolated Language Model Instances
Alexander Shirnin and Aleksey Kudelya
As model-generated content is increasingly consumed by other model instances in automated workflows, a practically important question arises: can a model embed a signal in natural
language that an independent instance of the same model can detect, relying only on shared pre-training and task instructions, without any shared
memory or coordination-specific training? We introduce For Your Eyes Only, a cooperative signalling game designed to evaluate this directly. A Sender produces free-form descriptions for two words, one of which is a hidden target; an isolated Receiver must identify it. We evaluate seven contemporary models from four architectural families on 300 word pairs from established psycholinguistic corpora, using the Double-Pass Success Rate to control for output biases. We find that most models struggle to maintain coordination once they are required to avoid detectable signals, while one frontier model retains near-perfect performance even after such filtering. We further show that models can direct this capability toward deliberate misdirection, and that coordination is consistently weaker across architectures than within them.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.AI
Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models
Alizishaan Khatri, Chiquita Prabhu and Omkar Neogi
Autonomous systems increasingly rely on Large
Language Models (LLMs) yet the safety infrastructure surrounding these models introduces latency and compute overhead. This limits utility in resource-constrained, time-critical deployments. Existing external guardrail models remain blind to the model's internal workings, creating a fundamental assurance gap. We ask: does the model already know when the content is harmful? We extract activations from LLaMA-3.1-8B and train lightweight MLP classifier probes (12.6M parameters) to detect harmful
prompts. Evaluated on WildJailbreak, Beavertails, and AEGIS 2.0, our probes achieve F1 scores of 99%, 83%, and 84%, respectively competitive with 1000x larger guard models while cutting latency and compute costs.
Read the full abstract ▾
Computation and Language · Cross-listed from cs.MM
From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning
Pan Wang, Siwei Song, Hui Ji et al. (+12 more)
The rapid expansion of
multimodal models has surfaced formidable bottlenecks in computation,
memory, and deployment, catalyzing the rise of Efficient
Multimodal Learning (EML) as a pivotal research frontier. Despite intensive progress, a cohesive understanding of what, how, and where efficiency is manifested across the learning stack remains fragmented. This survey systematizes the EML landscape by introducing the first structured, model-to-system taxonomy. We distill insights from over 300 seminal works into three hierarchical levels–model,
algorithm, and system–addressing architectural parsimony, execution refinement, and hardware-aware orchestration, respectively. Moving beyond a purely categorical review, we offer a methodological synthesis of the vertical synergies between these layers, elucidating how cross-layer co-design contributes to the fundamental "Efficiency-Utility-
Privacy" trade-off. Through an integrative case study of
Multimodal Large
Language Models (MLLMs), we trace the field's evolutionary trajectory from initial structural adjustments to modern full-stack resource orchestration. Furthermore, we provide a holistic discussion and application-specific optimization blueprints for diverse domains and posit a paradigm shift toward self-regulating intelligence, where efficiency is an intrinsic, emergent property of the model's fundamental design rather than a post-hoc constraint. Finally, we present open challenges and future directions that will define the trajectory of EML research. This survey establishes a structured framework for
multimodal systems that are not only high-performing and generalizable but natively efficient and ready for ubiquitous deployment. A continuously updated version is available at https://github.com/pwang322/Efficient-
Multimodal-Learning-Survey.
Read the full abstract ▾
All authors Pan Wang, Siwei Song, Hui Ji, Siqi Cao, Heng Yu, Zhijian Liu, Huanrui Yang, Yingyan Celine Lin, Beidi Chen, Mohit Bansal, Xiaoming Liu, Pengfei Zhou, Ming-Hsuan Yang, Tianlong Chen, Jingtong Hu
Computation and Language · Cross-listed from cs.CY
BurnRiSc: Toward Non-Invasive Burnout Screening in Open Source from Public Repository Signals
Timofey Sanko, Yuan Tian and Mariam Guizani
Burnout is a chronic occupational syndrome, and open source is close to a worst case for it: maintainers absorb unbounded demand with no manager to reallocate work and no organization to notice decline. The cost is not only personal. Burnout precedes withdrawal, and in projects sustained by a handful of maintainers, one departure can break infrastructure that thousands of downstream systems depend on. Yet the field has no way to see it coming: self-report inventories, the only existing measure, miss exactly the contributors most in need of detection and cannot be applied retroactively, so the field cannot even ask how common burnout is or what helps. We present BurnRiSc, a framework that operationalizes the Oldenburg Burnout Inventory's two dimensions, exhaustion and disengagement, as 14 behavioral and linguistic signals computed from GitHub activity and scored against each contributor's own history. The signals aggregate into two weighted dimension scores, with weights learned from labeled cases, and average into a monthly Burnout Risk Score (BRS). In a preliminary evaluation across 68 contributors in ten repositories (ten disclosed burnout cases, twelve comparable-volume collapses, and 46 comparison contributors), sustained BRS elevation precedes 6 of 10 disclosures by 6-15 months, 8 of 10 when adding peak BRS as a second criterion, and 10 of 10 over any prior time frame. We thus present BurnRiSc as evidence that burnout is screenable from public data.
Read the full abstract ▾
Computation and Language
Less Is More: Graph-free Multimodal RAG via Multi-signal Late Fusion
Tithi Rakshit, Hongkuan Zhou, Lavdim Halilaj et al. (+1 more)
Graph-based
retrieval-augmented generation (RAG) is widely used for
multimodal, cross-document question answering. However, building corpus-level graphs is expensive, slow to query, and difficult to maintain. We present TrioRAG, a graph-free
multimodal framework that integrates evidence from three complementary signals: the question, the anchor image, and a VLM-enhanced query generated from both. Each signal retrieves independently over a shared multi-vector index of page text and page images, and the results are combined through late fusion. Further, we introduce AutoQA, a
multimodal automotive
benchmark whose questions are grounded in noisy, web-sourced images rather than clean document-sourced figures. Its questions require
reasoning across manuals. We position it as a model-curated testbed rather than a human-validated gold standard. Across three
benchmarks, TrioRAG matches or outperforms graph-based systems while reducing total cost and accelerating per-query inference by 1.6-2.3 times. By construction, AutoQA grounds its questions in out-of-corpus web images. In this setting image
retrieval reaches only 19.3% document-level recall, while text-derived signals, especially the VLM-enhanced query, keep
retrieval robust.
Read the full abstract ▾
All authors Tithi Rakshit, Hongkuan Zhou, Lavdim Halilaj, Yuqicheng Zhu
Computation and Language · Cross-listed from cs.SD
A Cross-Lingual Acoustic Disease-Alignment Framework for Respiratory Health Assessment from Spontaneous Speech
Roksana Khanom, Raghib Asfak Tasnim, Bodrun Nahar Bithi et al. (+4 more)
Spontaneous
speech offers a scalable, noninvasive signal for respiratory health assessment, yet interpretable models that generalize across
languages remain challenging because disease-related acoustic changes are confounded by
language-specific phonetic variation. We present CL-DAF, a Cross-Lingual Disease-
Alignment Framework that identifies acoustic dimensions whose disease effects remain consistent across
languages. Using 201 English and 75 newly collected Bangla speakers, we construct a common 272-dimensional acoustic representation and quantify disease
alignment using signed rank-biserial effects and the
Language Invariance Score. We first show that spontaneous Bangla
speech separates COPD from controls (AUC 0.85); however, 133 features reverse their disease direction across
languages and the full representation transfers poorly (AUC 0.49 from Bangla to English). CL-DAF isolates 26 disease-aligned features that raise AUCs to 0.825 and 0.722 from English to Bangla and Bangla to English, respectively. These findings provide a foundation for multilingual clinical
speech models emphasizing pathology over
language-dependent variation.
Read the full abstract ▾
All authors Roksana Khanom, Raghib Asfak Tasnim, Bodrun Nahar Bithi, Shafia Shirin Supty, Saiful Islam Raju, Ashok Agrawala, Nirupam Roy
Computation and Language · Cross-listed from cs.CV
Riemannian–Lorentz Fusion of Vision Transformers and State-Space Models
Badri N. Patro and Vijay S. Agneeswaran
Scaling deep learning faces critical bottlenecks: data exhaustion, exponential training costs, and resource concentration. Model merging combines pre-trained checkpoints without gradient descent, offering orders-of-magnitude savings versus retraining. Combining independently trained
vision models is difficult when their architectures and parameter shapes differ. Existing weight-space merging methods generally assume aligned, shape-compatible checkpoints, whereas a
Vision Transformer (ViT) and a state-space model (SSM) implement
token mixing with different operators. We study a hybrid Heterogeneous merging setting that retains both architectures while aligning parameter groups by semantic role. Our proposed Riemannian–Lorentz Parameter Fusion (RLPF) method projects aligned groups to common coordinates, lifts selected coordinates to the Lorentz hyperboloid model of
hyperbolic space, computes a regularized geodesic barycenter, and decodes the result into the two branches. A learned gate then combines branch logits for each input. Component groups use fixed
curvature values, with normalization parameters treated as Euclidean. In the results available in this manuscript, the fine-tuned system obtains 82.37% on CIFAR-10, 75.04% on Oxford-IIIT Pet, and 78.58% top-1 accuracy on ImageNet-1K; the corresponding best-parent accuracies are 76.54%, 71.42%, and 76.42%. On ImageNet-1K, the reported pre-fine-tuning initialization reaches 77.80%. These results support further study of geometry-aware heterogeneous fusion, but not a training-free single-checkpoint merge: RLPF is a two-branch hybrid whose gate and reported final models are trained.
Read the full abstract ▾
Computation and Language
The Role of Fine-grained Harm Signals in LLM Safety
Soyeon Park, Seogyeong Jeong, Sunwoo Kim et al. (+1 more)
Prior work has shown that internal harmfulness representations in large
language models vary across risk categories, while sharing a common general harm representation component. This raises a question about the role of the category-specific component beyond general harm representation in LLM safety. To answer this question, we isolate the category-specific component by removing shared general harmfulness representation from each categorical harmfulness representation, yielding a category residual that is orthogonal to general harmfulness at every layer. Using activation steering with category residuals across 11 risk categories in 3 instruction-tuned LLMs, we find that whether category residuals encode harmfulness varies across categories, and that this category-wise pattern is similar across models. Whether category residuals induce refusal also varies across categories, but this category-wise pattern is more model-dependent. We also find that category residuals increase LLMs' downstream internal
alignment with shared general harmfulness representation. Together, these findings demonstrate that more fine-grained category residuals should also be considered beyond shared general harmfulness representation to fully understand LLM safety. More broadly, our findings show that even a direction orthogonal to a concept at one layer can contribute to the concept's downstream amplification.
Read the full abstract ▾
All authors Soyeon Park, Seogyeong Jeong, Sunwoo Kim, Alice Oh
Computation and Language
A frontend-backend architecture for tool calls in full-duplex speech models
Ke Hu, Slyne Deng, Chen Chen et al. (+10 more)
Full-duplex
speech-to-
speech (S2S) models provide natural, low-latency conversational interaction and would benefit from the ability to use external tools and complete voice-
agent tasks. We propose a frontend-backend architecture where a duplex
speech-to-text frontend learns to emit a delegation
token and forwards streaming ASR transcripts to a text-based backend LLM for tool calls. Tool-call results from the backend are injected back into the frontend through a lightweight prefill-and-repeat mechanism and then synthesized using streaming TTS to the user. Our approach largely preserves regular duplex turn-taking, interruption handling, and low-latency interaction as it requires minimal modifications to the frontend model. In a single-turn tool-call evaluation, our system achieves 92-97% tool-call recall, competitive tool-call prediction performance, and 81.2% accuracy in rejecting irrelevant calls. When equipped with a larger backend (e.g., Qwen3-235B-A22B), our system achieves competitive results on Full-Duplex-Bench-V3 compared to open and closed source models, and significantly outperforms GPT-realtime-mini and Qwen3-Omni-30B-A3B-Instruct on EVA-Bench. These results demonstrate that backend delegation is an effective and modular approach for combining natural duplex
speech interaction with strong agentic tool-call capabilities.
Read the full abstract ▾
All authors Ke Hu, Slyne Deng, Chen Chen, Elena Rastorgueva, Edresson Casanova, Punit Kumar, Dharmendra Choudhary, Nikhil Srihari, Ameya Sunil Mahabaleshwarkar, Viet Anh Trinh, Slim Essid, Oluwatobi Olabiyi, Zhehuai Chen