Artificial Intelligence · Cross-listed from cs.CL
NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities
Jagadeesh Balam, Travis Bartley, Edresson Casanova et al. (+51 more)
We introduce NemotronLabs VoiceChat, an open full-duplex
speech-to-
speech model with native tool-calling capabilities. NemotronLabs VoiceChat combines a streaming
speech encoder and decoder-only
language model with parallel specialized output streams for
agent text and structured function calls, an auxiliary RNN-T branch for incremental user transcription, and a streaming TTS decoder. This design enables the model to listen, transcribe, reason, invoke tools, and speak within a unified streaming architecture while preserving the temporal behavior required for natural conversation. On Full-Duplex-Bench 1.0, NemotronLabs VoiceChat achieves the lowest pause-handling takeover rates among evaluated open-weight systems, 100% takeover following user interruptions, and a 4.33/5 post-interruption response-quality score. On Full-Duplex-Bench 1.5, it resumes its response after user backchannels in 93% of cases. NemotronLabs VoiceChat obtains a 55.1 normalized average on VoiceBench and, on Full-Duplex-Bench 3.0 (FDB 3.0), achieves 82.5% tool-selection F1, while argument accuracy and end-to-end tool execution remain areas for improvement. These results demonstrate that full-duplex interaction,
speech recognition and generation, general
language capabilities, and external tool use can be integrated in a single open
speech-to-
speech model without sacrificing real-time conversational behavior.
Read the full abstract ▾
All authors Jagadeesh Balam, Travis Bartley, Edresson Casanova, Sanjay Chauhan, Chen Chen, Zhehuai Chen, Zijia Chen, Francesco Ciannella, Shalini De Mello, Slyne Deng, Mikyas Desta, Harishchandra Dubey, Slim Essid, Nourchene Ferchichi, Boris Ginsburg, Mariana Graterol Fuenmayor, Negar Habibi, Kevin Hu, Anand Joseph, Viraj Karandikar, Myungjong Kim, Viacheslav Klimkov, Seelan Lakshmi Narasimhan, Lily Lee, Jason Li, Eileen Long, Ameya Mahabaleshwarkar, Aditya Malte, Adi Margolin, Amrita Mazumdar, Sasha Meister, Valentin Mendelev, Koki Nagano, Oluwatobi Olabiyi, Seonwook Park, Ankita Pasad, Yifan Peng, Elena Rastorgueva, Jayda Ritchie, Jason Roche, Rajarshi Roy, Nikhil Srihari, Yuanhang Su, Yoshi Suhara, Viet Anh Trinh, Jinhan Wang, Piotr Zelasko, Hui Wang, Puhui Meng, Chaosen Zhang, Yunsheng Liu, Shawn Wang, Wenjing Li, Zhonglei He
Artificial Intelligence
Learning Cardiac Features: ECG Biometrics Across Time and~Exercise
Luca Thiebaud, Paul Chauchat, Mustapha Ouladsine et al. (+1 more)
Electrocardiograms (ECGs) carry subject-specific patterns enabling reliable individual discrimination, forming the basis of ECG biometrics. Beyond authentication, this paradigm holds significant potential to secure sensitive cardiac data and to serve as a pretext task in self-supervised learning. Yet, most studies remain confined to singlesession, resting data, leaving robustness to temporal and physiological variations largely untested. We address this gap by evaluating ECG biometrics under realistic conditions involving exercise-induced stress and cross-session variability. A Siamese ResNet with late multi-lead fusion strategy is trained on a large ECG
dataset extracted from cardiopulmonary exercise tests and evaluated with a exercise-and time-aware protocol, as well as on public
benchmarks. This first extensive assessment of ECG biometrics under combined physiological and temporal variability achieves an intra-session rest-to-peak EER of 1.7% and stateof-the-art 3.9% on the CYBHi
dataset. Findings support the presence of an intrinsic cardiac signature resilient to physiological and temporal drift.
Read the full abstract ▾
All authors Luca Thiebaud, Paul Chauchat, Mustapha Ouladsine, Stéphane Delliaux
Artificial Intelligence · Cross-listed from cs.RO
When Should a Failing Robot Ask? Initiating Corrective Human-Robot Dialogue from Audited Sensor Evidence
Eshika Pathak and Leela Krishna
A
robot that fails at a task faces the first decision in corrective dialogue: act on its own diagnosis, consult another onboard sensor, or interrupt a person. Choosing well requires knowing how much the
robot's sensors reveal about the cause and how reliable the
robot's own diagnosis is. We build a simulated
benchmark in which every failure's true cause is known, because we injected it, and measure what each sensor reveals, with explicit checks against data leakage. Some failures are diagnosable from camera images; others only from the
robot's force data (0.99 from force data, no image method above 0.55). We then test six open
vision-
language models. Their behavior tracks the surface of the
prompt, not the evidence: moving the refusal option from last to first in the answer list collapses refusal rates from 78-100% to 0-6% in three of the six swept model-and-family pairs. Accuracy from frames stays at or below a majority-class baseline under every
prompt variant, with or without worked examples, and stated confidence carries no information about correctness. Handing the same models the force data as ten lines of text produces the first above-baseline diagnoses, in four of the six models: much of the failure reflects missing sensor data, not missing ability. We pose the choice as a three-action decision problem, act, consult your own sensors, or ask a human, whose optimal policy follows from measured accuracy. The models do not follow it, and their ask rates ignore a fourfold change in question cost. One question to a human still lifts them from that baseline to roughly the answerer's own reliability (0.70-0.81 when they ask). The decision to ask should be tied to measured accuracy and stated costs, not to the model's confidence.
Read the full abstract ▾
Artificial Intelligence
AutoViewMem: Self-Configuring Orthogonal Views for Conversational Long-Term Memory
Zijie Cao, Xijun Qu, Zhicheng Gu et al. (+7 more)
Long-term
memory is essential for large
language model (LLM)
agents to maintain consistency and personalization over extended interactions. Existing
memory systems typically rely on fixed granularities or static schemas, but these designs struggle when heterogeneous information, such as preferences, events, constraints, and temporal updates, is embedded in a single mixed representation. The resulting semantic interference makes top-K
retrieval sensitive to noise and often leaves relevant evidence poorly ranked. We present AutoViewMem, a data-driven framework that organizes long-term conversational
memory into self-configuring, low-overlap semantic views before indexing. AutoViewMem discovers candidate views from interaction traces, selects a compact complementary view set, and uses these views to guide write-time structured extraction of provenance-grounded
memories. This representation-first design moves semantic disentanglement from
retrieval time to write time, allowing standard top-K similarity search to retrieve focused evidence without explicit routing or iterative
retrieval. We further apply offline consolidation to improve
memory compactness and consistency. Experiments on the LoCoMo and PersonaMem
benchmarks, under both Qwen3-8B and Qwen3-14B backbones, show that AutoViewMem improves long-horizon question answering and personalization over strong
memory baselines while preserving a simple inference pipeline.
Read the full abstract ▾
All authors Zijie Cao, Xijun Qu, Zhicheng Gu, Xiaoshu Chen, Duanyang Yuan, Yanning Hou, Sihang Zhou, Jianxing Gong, Jian Huang, Yang Mei
Artificial Intelligence
What Should We Ask Next? Retrieval-Aware Question Learning for Interactive ReID
Lyucheng Qian, John Yuehan Zhang and Pingyu Wang
Interactive
retrieval with partial evidence constitutes a sequential information-acquisition problem: an
agent must choose questions that acquire useful evidence for the next
retrieval update. Existing systems train this decision by imitating an offline ordering of candidate QA pairs. However, a question's value depends on the response it elicits and its downstream effect on
retrieval. We establish that candidate discriminativeness and perceived usefulness provide weak supervision for this objective, then introduce RAVEL, a
retrieval-aware online reinforcement learning framework for interactive person re-identification. RAVEL initializes from supervised question generation, observes the current Top-4 candidates directly, and optimizes the question policy with rank feedback from the full question-answer-
retrieval loop. Experiments on Interactive-PEDES show that RAVEL delivers progressively stronger
retrieval performance across five interaction rounds. Further analysis shows that RAVEL allocates more of its interaction budget to localized open-ended
prompts targeting specific attributes.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Detecting Pretraining Data in Large Language Models from a Free-Energy Perspective
Chenye Ke, Zirui Liu, Qi Liu et al. (+4 more)
Detecting pretraining data in large
language models is challenging because high likelihood can reflect either training exposure or strong generalization. In the joint space of prediction loss and predictive
entropy, a likelihood-only detector uses a horizontal boundary and can mistake predictable non-members for members. Motivated by this, we introduce an inclined boundary that evaluates prediction loss relative to predictive
entropy. Our analysis shows that
entropy correction can preserve the expected membership signal while reducing its variance, thereby improving standardized member–non-member separation. We further extend the mean–variance analysis to the more general setting with a nonzero mean
entropy gap. Interestingly, this
entropy-adjusted score admits a Helmholtz free-energy interpretation, leading to Energy Transfer Detection (ETD), which views pretraining data detection from a macroscopic residual free-energy transfer perspective. Extensive experiments show that ETD achieves the best average detection performance, improving average AUROC by up to 3.5% and TPR@5%FPR by up to 5.1%, while remaining robust across diverse settings.
Read the full abstract ▾
All authors Chenye Ke, Zirui Liu, Qi Liu, Yan Zhuang, Jintao Zhang, Zhenya Huang, Shijin Wang
Artificial Intelligence · Cross-listed from cs.CV
Benchmarking the Explanatory Quality of Open-Weight Vision-Language Models in Face Recognition
Laurent Colbois and Sébastien Marcel
Vision-
Language Models (VLMs) have recently been proposed as promising tools for face recognition, as they can produce natural
language explanations alongside similarity scores. This capability is considered appealing for face comparisons in forensic contexts, which require decisions to be transparent and auditable. However, existing evaluations of VLMs for that use case focus mostly on recognition accuracy, while the validity of generated explanations remains unquantified. In this work, we introduce a benchmarking framework for VLM-based face recognition that treats explanation quality as a core evaluation axis. We propose two criteria that explanations should satisfy: relevance, i.e., reliance on identity-stable facial features; and faithfulness, i.e.,
alignment with the visible image content without hallucinated features. We jointly develop a methodology enabling the quantification of relevance and faithfulness of evaluated models, based on constraining model outputs to a structured explanation format that supports automated querying and
auditing. Using this framework, we
benchmark several families of open-weight VLMs, jointly evaluating face verification accuracy and explanation quality. Our results highlight remaining shortcomings of produced explanations, and emphasize the need for such explanation quality metrics to get a complete picture of model performance. The proposed
benchmark and open-source evaluation harness provide a foundation for proper benchmarking and future fine-tuning of explainable face recognition systems.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Neural Cellular Automata Learn General Features in their Hidden Channels
Etienne Guichard and Stefano Nichele
Modern deep learning models achieve impressive generalization through over-parameterization, but this paradigm often struggles with overfitting and memorization in few-shot regimes. Neural Cellular Automata (NCAs) offer a highly parameter-efficient alternative, yet research has focused primarily on their output, leaving the role of their internal hidden channels largely unexplored. In this paper, we investigate the internal dynamics of NCA hidden channels and introduce a novel transfer-learning mechanism that injects a pretrained teacher's hidden states into a student model to guide early optimization. Evaluated on few-shot and scale-variant MNIST
benchmarks, NCAs outperform comparable recurrent and feed-forward architectures, demonstrating superior generalization with a minimal parameter budget (~9,800 parameters). Mechanistic analysis reveals that the hidden channels decouple feature extraction from uniform classification consensus by absorbing morphological complexity and converging to mutually orthogonal states. Furthermore, we demonstrate that these hidden channels capture general, scale-invariant topological primitives rather than class-specific templates. This allows a student model to achieve strong few-shot performance on unseen classes using features transferred from a teacher trained only on a subset of digits (0-5). Our results highlight the potential of utilizing hidden-state dynamics as a robust, decentralized computational substrate for parameter-efficient transfer learning
Read the full abstract ▾
Artificial Intelligence
AutoRecLab: Describe the Experiment, Get the Code!
Moritz Baumgart, Philipp Meister, Justus Krell et al. (+3 more)
Empirical evaluation is central to recommender-systems (RecSys) research, but turning experimental designs into executable code remains a manual and error-prone task. We present AutoRecLab, a Python-based autonomous RecSys lab that automates RecSys experiments from natural-
language prompts. Given a research idea, AutoRecLab derives explicit experiment requirements, builds and validates a prototype, and iteratively expands it into the requested full experiment. The workflow combines
retrieval-augmented generation (RAG) for documentation lookup, static type verification, and execution-steered tree search. In our demonstration, AutoRecLab autonomously implements an explicit-to-implicit feedback conversion study. In a baseline comparison across six
algorithms and three
datasets, 8 of 9 runs succeed at an average cost of approx- imately $1 per run with GPT-5.4-mini.
Read the full abstract ▾
All authors Moritz Baumgart, Philipp Meister, Justus Krell, Michael Schmidt, Bela Gipp, Joeran Beel
Artificial Intelligence · Cross-listed from cs.CL
Do Personality-Tuned LLMs Make Better Social Agents?
Tim Krabbe and Xiaodan Shi
LLMs are increasingly used in social simulations for socially interactive
agents and
robots, offering more flexibility than rule-based systems. However, even though they mimic human behaviour very well, there is a persistent alienness to them. This work investigates whether personality-aware fine-tuning can reduce this gap by improving the consistency and controllability of personality-conditioned dialogue generation compared with instruction prompting alone. We fine-tune two small open-weight LLMs, Qwen2.5-7B-Instruct and Ministral-8B-Instruct, using a corpus that combines personality-labelled social media posts and dialogues to create a personality-based dialogue engine for social simulation. The resulting models are evaluated across multiple social interaction scenarios using three independent LLM judges, which assess personality fidelity and provide evidence-based behavioral interpretations. We additionally quantify inter-rater agreement and lexical characteristics of the generated dialogue. Results indicate that fine-tuned models are not better at role-playing different personalities than their respective baseline models. However, low inter-rater agreement limits the confidence with which these results can be interpreted. Concerning the quality of generated texts, fine-tuned models are mostly comparable to the baselines, with fine-tuning improving the linguistic diversity of the Qwen models. While the results appear generally usable and the baseline models offer the best overall performance, future studies should place greater emphasis on the quality and domain
alignment of training data for accurate personality role-playing.
Read the full abstract ▾
Artificial Intelligence
EnterpriseVal: Quantifying the Efficacy, Reliability and Value of Generative AI in the Enterprise
Abbas Raza Ali, Muhammad Ajmal Siddiqui and Moona Zahid
Frontier
language models now produce professional deliverables that expert graders judge to match human work on a substantial share of economically valuable tasks, yet most enterprise GenAI initiatives fail to show a measurable business effect and a large fraction of agentic projects are expected to be cancelled. We argue that this is substantially a measurement problem: public
benchmarks answer "what can the model do?", whereas a deployment decision requires "is this workflow fit, reliable, safe and worth scaling - here, on our data, under our controls?". We present EnterpriseVal, a use-case-level evaluation system that closes this gap. It comprises (i) a formal specification of the use case and of the frozen socio-technical configuration under test, model,
prompts,
retrieval, tools, guardrails and human oversight, with an autonomy level and consequence tier that jointly set the required evaluation intensity; (ii) a metric catalogue spanning fidelity, utility, efficiency, reliability, assurance and oversight; (iii) a grading protocol that scales blinded expert judgement with calibrated LLM-as-judge scoring through prediction-powered inference; (iv) a two-tier threshold gate, stated as an executable
algorithm, that maps metric vectors with confidence bounds to REJECT/CONDITIONAL/SCALE decisions; and (v) a value-and-risk model in which the reviewer catch rate is a measured parameter. We report a pilot across three workflows in a global bank. In credit-memo drafting, human-graded citation precision reached 88% and
hallucination rate 1.6% for the best model against gates of 70% and 5%; in procedure transformation, analyst refinement effort fell from an estimated 27.4 to 2.9 hours per document. We separate established results, documented pilot evidence, the proposed system and open hypotheses, and specify the experiments required for full validation
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
Federated Deep Clustering Networks for High-Dimensional and Heterogeneous Data
Morris Stallmann, Charalampos S. Kouzinopoulos, Marcin Pietrasik et al. (+1 more)
Clustering high-dimensional data is a fundamental task in unsupervised machine learning with applications to a variety of domains. In the centralized data scenario, this task is commonly solved using deep clustering methods that utilize deep neural network architectures to learn clustering-friendly latent space representations. In
Federated Learning, where data is distributed between clients and is private, deep clustering methods are less explored. In particular, recently introduced
federated deep clustering methods, despite showing very promising performance, still fall short in reliably providing good performance if data across clients are non-identically-independently distributed. In this work, we introduce a generalization of Deep Clustering Networks to the
federated scenario, named FedDCN, that simultaneously optimizes a reconstruction loss and a clustering loss. To ensure robustness and latent space
alignment in non-identically-independently distributed data scenarios, FedDCN generates synthetic data augmentations, and its learning objective includes a
geometric regularization for latent space
alignment. Through experimental evaluation, the effectiveness of the approach under IID and non-IID assumptions is demonstrated, and future research directions are identified.
Read the full abstract ▾
All authors Morris Stallmann, Charalampos S. Kouzinopoulos, Marcin Pietrasik, Anna Wilbik
Artificial Intelligence · Cross-listed from cs.HC
Touvigation: Embodied Adaptive Object Acquisition for Blind and Low-Vision Users in Unfamiliar Indoor Environments
George Xi Wang, Xiangyu Li, Shaoyue Wen et al. (+8 more)
Blind and low-
vision users often face challenges when locating and physically acquiring objects in unfamiliar indoor environments. Existing
vision-
language-model-based assistants can provide semantic descriptions but may introduce latency,
hallucinations, and guidance that is poorly aligned with
embodied action. We present Touvigation, a hands-free object acquisition system that combines
vision-
language understanding with persistent local spatial modeling to provide low-latency, body-relative guidance. Drawing on formative interviews with eight blind and low-
vision participants, we design a multi-stage guidance framework that adapts spatial references as users transition from orienting, to walking, to reaching and tactile verification. We evaluated Touvigation with 12 blind and low-
vision participants against a
multimodal large-
language-model assistant and unassisted search. Touvigation achieved 100% task success, compared with 58% for the
multimodal assistant and 85% for unassisted search, while reducing completion time and cognitive workload. Our findings demonstrate how persistent spatial grounding and adaptive
embodied guidance can improve object acquisition for blind and low-
vision users.
Read the full abstract ▾
All authors George Xi Wang, Xiangyu Li, Shaoyue Wen, Jiaqian Hu, Junan Xie, Yupeng Wang, Ziyue Shi, Qijun Chen, Maaike Bouwmeester, Yuhua Jin, Jing Qian
Artificial Intelligence · Cross-listed from cs.LG
Matrix AdaGrad: Row-wise and Column-wise Adaptive Subgradient Methods
Wenpeng Zhang, Runsheng Yu and Peilin Zhao
Adaptive optimization methods such as AdaGrad and Adam are widely used in modern deep neural network training, but their adaptive scaling is primarily designed for vector-valued parameters and does not explicitly exploit
matrix structure. Recent
matrix-aware optimizers demonstrate the benefits of structured optimization, yet a general theoretical framework for deriving
matrix-aware adaptivity comparable to that of AdaGrad remains lacking. In this work, we develop an Online Mirror Descent framework with adaptive proximal functions for
matrix-valued parameters, providing a principled methodology for deriving
matrix-aware adaptive optimization through online regret minimization. By introducing row-wise and column-wise
matrix proximal functions, our framework explicitly reveals the trade-off governing adaptive scaling: increasing the scaling factors reduces the gradient-dependent dual norm term while increasing the cost of evolving the proximal geometry. In the row-wise setting, this trade-off becomes separable under diagonal parameterization, allowing the adaptive scaling for each row to be derived independently by minimizing its corresponding row-wise regret bound. The column-wise counterpart follows directly by applying the row-wise construction to the transposed
matrix. This framework yields Row-wise
Matrix AdaGrad and Column-wise
Matrix AdaGrad as concrete instantiations, with regret guarantees that are strictly tighter than those of entry-wise AdaGrad under row-sparse or column-sparse gradient structures. Experiments on
matrix factorization and stacked deep MLP training further demonstrate the benefits of
matrix-aware adaptive scaling, yielding improved optimization performance in both settings and enhanced optimization stability and trainability at larger learning rates and greater network depths in the latter.
Read the full abstract ▾
Artificial Intelligence
MIST: Multimodal Survival Prediction with Genomic-Guided Histology Attention
Muhammet Sami Yavuz, Sabri Mustafa Kahya, Richard R. Chen et al. (+2 more)
Multimodal survival models can combine complementary prognostic information from whole-slide images and genomic profiles, but effective fusion remains challenging amid external cohort shift and computational complexity. To address these challenges, we propose MIST,
multimodal survival prediction with genomic-guided histology
attention. MIST represents genomic features as
tokens and allows them to query compact foundation-model-derived histology context
tokens before survival prediction. This design enriches molecular information with histology context rather than merging separately encoded modalities only at the final stage. Training combines discrete-time survival prediction with genomic feature masking, WSI dropout, and paired WSI-genomics contrastive
alignment. Across four external evaluations in colon, renal, lung, and glioblastoma cohorts, MIST improves external C-index over standard fusion baselines in the primary comparisons. These results support genomic-guided histology
attention as a compact and effective strategy for
multimodal oncology outcome prediction. Our code is available at https://github.com/samiyavuuz/MIST .
Read the full abstract ▾
All authors Muhammet Sami Yavuz, Sabri Mustafa Kahya, Richard R. Chen, Jana Lipkova, Benedikt Wiestler
Artificial Intelligence · Cross-listed from cs.HC
An Agentic Just-in-Time Adaptive Intervention System for Personalized Sleep Support: Proof-of-Concept Study with N of 1 Data
Nick Rezaee and Chelsea Boccagno
Background: Just-in-time adaptive interventions (JITAIs) can use behavioral data to adapt support to changing contexts, but many rely on predefined rules and manual configuration. Objective: We developed a proof-of-concept sleep JITAI using an AI
agent to review personal data, evaluate reminders, adapt interventions, and record decisions for human review. Methods: Running in Home Assistant on a configurable schedule, the
agent follows a reusable skill file to review 30 days of sleep and behavioral data, including physical activity, smartphone use, and bedtime routines, to identify patterns and create or update automated reminders. Results: Initial runs demonstrated technical feasibility, successfully completing data review and intervention decisions while limiting reminders to three per day and saving decision records. Conclusions: Agentic AI may enable flexible, adaptive sleep JITAIs. The architecture supports future comparison with fixed or rulebased interventions, requires human oversight, and could extend to other health behaviors.
Read the full abstract ▾
Artificial Intelligence
LLM-Generated Feature Pools for Time Series Anomaly Detection
Youssef Attia El Hili, Malik Tiomoko and Corinne Ancourt
We study how far a simple statistical pipeline can go on univariate time series anomaly detection under a strict selection protocol. The method extracts a small pool of statistics over sliding windows, scores each window with a transductive robust (MAD) model, and selects a feature subset per domain on a held-out tuning split. On TSB-AD-U it reaches $0.529$ per-series VUS-PR, above the best neural ($0.45$) and statistical ($0.44$) entries on the public leaderboard and within $0.06$ of the strongest pretrained foundation model, several of which use more supervision than ours. Ablations locate the cause: across three selection strategies and a hindsight oracle the score moves by $0.031$, and across the aggregation grid by $0.096$, while changing the candidate pool moves it by $0.226$. The candidate pool sets the ceiling; the search over it is second-order. We therefore generate a pool per domain by prompting a
multimodal LLM with in-context example windows from that domain. The generated pools match the hand-crafted one under matched selection, and the two cover different domains: selecting over their union improves on the generated pool in all twelve generator-seed pairs and lifts the pipeline to $0.588$, matching the performance of the best entry on the leaderboard.
Read the full abstract ▾
Artificial Intelligence
ECG Mirage: Revealing and Mitigating the Underutilisation of ECGs in Vision-Language Models for Clinical Prediction
Jinning Liang, Mingcheng Zhu and Tingting Zhu
Emergency department (ED) decision-making relies on heterogeneous clinical information, including patient history, vital signs, laboratory results, and electrocardiograms (ECGs).
Vision–
language models (VLMs) can jointly process these modalities, but strong predictive performance does not necessarily imply meaningful use of the correct patient's ECG. We term this failure mode ECG Mirage: apparent
multimodal capability without useful dependence on patient-specific ECG information. We distinguish two forms: ECG neglect, where ECGs provide little predictive benefit, and ECG confusion, where matched ECGs outperform no-image inputs but not mismatched ECGs. To evaluate these behaviours, we compare predictions obtained with matched ECGs, outcome-discordant mismatched ECGs, and no-image inputs while holding the clinical text and prediction targets fixed. Across four VLMs on MDS-ED, matched ECGs provide no consistent advantage for either ICU admission or clinical deterioration prediction. We then train four restricted visual
prompts using supervised learning followed by conditional direct preference optimisation, while keeping the VLM backbone frozen. The resulting models achieve balanced accuracies of 70.6% for ICU admission and 67.5% for deterioration and increase the matched-versus-mismatched performance gap to approximately 16.5 and 5.5 percentage points, respectively. Overall, our study identifies ECG Mirage in
multimodal clinical prediction and introduces visual
prompt tuning as an efficient mitigation strategy.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.RO
ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction
Tim Engelbracht, René Zurbrügg, Mayank Mittal et al. (+4 more)
Manipulating objects requires understanding not only their motion, but also the physical properties that determine it. For articulated objects, these include inertia, friction, and mechanisms such as springs or door closers, whose effects can vary with configuration and velocity. Such properties are not directly observable from appearance: visually identical doors may require very different effort to manipulate. Existing digital-twin pipelines recover primarily kinematics or assign static physical parameters from visual and
language priors, which can yield physically implausible estimates. As a result, state-dependent mechanism dynamics remain unidentified and are not represented in standard asset formats. We present ForceTwin, a system for identifying physics-informed digital twins of articulated objects from instrumented human interaction. A person probes an object using a handheld force-sensing gripper, providing synchronized poses and interaction forces from which we estimate the articulation, parametric dynamics including inertia, Coulomb friction, viscous damping, and a structured neural residual capturing state-dependent mechanism forces. ForceTwin nearly halves the inertial-parameter error of a VLM prior. As a feedforward dynamics model for impedance control on a Spot and a Franka FR3, ForceTwin achieves 87% goal completion across nine object-embodiment pairs, compared with 60% using VLM-prior and 57% using kinematics-only twins, with the largest gains on objects whose strong mechanisms cause both baselines to stall. We further use the identified twins to train whole-body door-traversal policies and deploy them in the real world. Project Page: https://timengelbracht.github.io/forcetwin-website/
Read the full abstract ▾
All authors Tim Engelbracht, René Zurbrügg, Mayank Mittal, Marco Hutter, Marc Pollefeys, Hermann Blum, Zuria Bauer
Artificial Intelligence
Pierre Beckmann, Matthieu Queloz and Andre Freitas
Behavioral failures can make a
transformer appear to lack a world model even when it has learned faithful representations of its environment. We demonstrate this in TaxiGPT, a
transformer trained on random walks through Manhattan whose failures have been interpreted as evidence of an incoherent internal map. Through mechanistic analysis and causal interventions, we show that the model represents intersections and streets, tracks its position, and uses a goal compass to navigate. We trace its failures to interference between superposed intersection features, which disrupts localization within the internal map. Affordance packing, which groups representations of intersections with the same legal moves, helps limit the consequences of these errors. Finally, we propose mechanistic indicators that we use to compare models and show that world-modeling capacities emerge at different stages of training. Our findings motivate a shift from asking whether a model has a world model to mechanistically studying its world modeling: the interacting capacities through which it represents its environment and uses those representations to guide behavior.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
Balanced Prompt Adaptation against Entropy-Induced Collapse for Test-Time Binary Segmentation
Zhengshan Wang, Joshua Charles Webster-Ford, Yifei Tian et al. (+3 more)
Entropy minimization is a standard objective for test-time adaptation (TTA), but it can fail in imbalanced binary
segmentation. Unlike image classification, dense
segmentation aggregates thousands of pixel predictions, allowing the larger predicted class to dominate the update, pull minority predictions toward itself, and produce a degenerate mask as predictions saturate and their
entropy gradients vanish. We theoretically establish this collapse in a shared-shift model. This analysis motivates Balanced-Anchor
Prompt Adaptation (BAPA), which combines two complementary modules. The Class-Balanced Anchors (CBA) module selects high-confidence anchors separately from each predicted class and gives foreground and background equal total loss weight, preventing the larger region from dominating the update. Dynamic
Prompt Adaptation (DPA) refreshes these anchors after each prediction update and optimizes only text-side
prompt residuals while keeping the
vision-
language encoders frozen. This
prompt-only update refines the foreground-background decision boundary without altering the pretrained dense visual representation. Across experiments from four domains, BAPA achieves the highest mean Dice among the evaluated methods. Factorized ablations further validate the complementary roles of CBA and DPA, supporting balanced
prompt adaptation as an effective alternative to
entropy minimization for test-time binary
segmentation.
Read the full abstract ▾
All authors Zhengshan Wang, Joshua Charles Webster-Ford, Yifei Tian, Xinxin Wang, Long Chen, Weiping Ding
Artificial Intelligence · Cross-listed from cs.CL
CIBuzzBench: A Benchmark for Cross-Lingual Understanding of Chinese Internet Buzzwords
Yifan Wang, Junyu Lu, Qifan Wang et al. (+6 more)
Chinese social media has generated a vast and continually evolving lexicon of internet buzzwords whose meanings are often non-literal and deeply rooted in local cultural and pragmatic contexts. Existing research has primarily focused on interpreting these buzzwords within Chinese, leaving largely unexplored whether LLMs can transfer such culturally grounded knowledge across
languages and accurately convey the intended meanings in English. This cross-lingual capability is also critical for safety, as harmful expressions may obscure their offensive content through culture-specific homophony, euphemism, irony, or coded
language. In this paper, we investigate the ability of advanced LLMs to understand Chinese internet buzzwords across
languages. To this end, we introduce CIBuzzBench, the first
benchmark for cross-lingual Chinese-to-English understanding of Chinese internet buzzwords. CIBuzzBench comprises 3,001 Chinese internet buzzwords annotated with English meaning explanations, English equivalents, category labels, and harmfulness labels. Based on these annotations, we design three evaluation tasks: Meaning Explanation, Cross-lingual Equivalent Matching, and Culturally Grounded Harmfulness Detection. We evaluate representative state-of-the-art proprietary and Chinese LLMs under both English- and Chinese-prompting settings. Our results show that LLMs continue to struggle with the cross-lingual understanding of Chinese internet buzzwords, particularly in fine-grained non-literal interpretation, robust equivalent matching under option perturbations, and calibrated harmfulness detection. These findings highlight the persistent challenges posed by culturally grounded
language phenomena for
multilingual LLMs and safety-oriented evaluation. The
dataset and code are available at https://github.com/SuperYFan/CIBuzzBench.
Read the full abstract ▾
All authors Yifan Wang, Junyu Lu, Qifan Wang, Shun Zhang, Chaozhuo Li, Jiahao Liu, Zhijun Cao, Lingbin Bu, Fanliang Bu
Artificial Intelligence · Cross-listed from cs.CR
TERMon: Detecting Persistent Behavioral Threats in Edge AI via Hardware-Native Ternary Runtime Monitor
Arish Sateesan and Edlira Dushku
Edge AI accelerators are increasingly deployed in safety-critical environments, where model outputs may control physical actuators, make access-control decisions, or trigger alarms. In these settings, runtime failures often remain undetected because model corruption, distribution shift, and
adversarial inputs can still produce well-formed, confident predictions. This paper presents TERMon, a lightweight hardware runtime monitor that detects such anomalies by observing inference behavior rather than re-executing or formally verifying the model. TERMon represents class-conditional trusted behavior as hardware-efficient ternary patterns that are matched in parallel against a thermometer-encoded fingerprint. The ternary encoding reproduces the corresponding unquantized range decision exactly. TERMon detects harmful weight corruptions in proportion to their behavioral impact, while out-of-distribution and
adversarial inputs are largely not separable using the monitored features at a strict false-positive operating point. We implemented TERMon on a PYNQ-Z2 FPGA, and the pipelined design requires no on-chip block RAM or DSPs and has a two-cycle decision latency.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
CIPL: A Channel-Aware Framework for Recoverable Privacy Leakage in LLM Agents
Tao Huang, Guosen Wu, Guolong Zheng et al. (+5 more)
Privacy leakage in LLM
agents is commonly evaluated within individual components such as
memory,
retrieval, or tool-use pipelines, which makes it difficult to distinguish internal exposure from information that an external observer can actually recover. We present CIPL (Channel Inversion for
Privacy Leakage), a channel-aware evaluation framework for black-box
privacy leakage in LLM
agents. CIPL represents a target through sensitive source, selection, assembly, execution, observation, and extraction stages and evaluates the transition from selected sensitive units to attacker-recoverable output under a shared protocol. Experiments across
memory-based,
retrieval-mediated, and tool-mediated targets, together with a BrowserUse live-
agent case study, show that storage labels alone do not determine recoverability.
Memory targets form a near-saturated reference case,
retrieval-mediated leakage is frequently partial, and tool-mediated and live-
agent leakage varies strongly with observation surface,
prompt-to-channel
alignment,
retrieval depth, and provider behavior. A stratified semantic audit further identifies attacker-useful disclosures that canonical exact matching misses. CIPL therefore provides a common framework for comparing how internal sensitive dependence is realized as externally recoverable leakage across heterogeneous
agent pipelines.
Read the full abstract ▾
All authors Tao Huang, Guosen Wu, Guolong Zheng, Jiayang Meng, Chen Hou, Xu Yang, Xuechao Yang, Feng Xia
Artificial Intelligence
Listen Before You Speak: Response Planning from Listener Facial Reactions for Conversational Speech Generation
Yunji Chu
Conversational
speech depends on dialogue context and the listener's immediately preceding behavior. We propose ReACT-TTS, a two-stage framework that uses a one-second pre-response listener facial sequence to plan the next utterance's
emotion and prosody before
speech realization. On a strict dyadic MELD protocol, Temporal conditioning yields higher mean macro-F1 and VAD concordance than Text-only across ten seeds, while accuracy remains essentially unchanged. Ablations show that temporal modeling performs best among the visual variants and that an explicit early-to-late difference is unnecessary; correct listener reactions also outperform cyclic mismatches on average. In a contextual-appropriateness study with 20
speech researchers, 76% of judgments prefer Temporal, 9% Text-only, and 15% report no preference. We further connect the predicted response style to a Grad-TTS backbone for end-to-end
speech realization. Overall, the results support pre-response listener dynamics as complementary cues for conversational response planning. The source code is available at https://github.com/CYJ1/ReACT-TTS_public.
Read the full abstract ▾
Artificial Intelligence
GUARD: Natural Forgetting in Large Reasoning Models via Guided Answer-Reasoning Distillation
Zeyu Yan, Guanghao Zhou, Minghui Qiu et al. (+2 more)
Recent advances in large
reasoning models (LRMs) have made machine unlearning more challenging, as protected facts or unsafe rationales may surface in intermediate chain-of-thought (CoT) traces before the final answer is produced. Existing unlearning objectives typically suppress the target content or redirect internal representations, but they never specify how the post-forgetting trajectory should continue, which can lead to hallucinated substitutes, malformed boundaries, or repetitive outputs. We argue that LRM unlearning should instead learn a natural forgetting trajectory: a coherent non-disclosing CoT followed by a stable refusal-style answer that replace the original disclosure. To this end, we propose Guided Answer-
Reasoning Distillation (GUARD), which converts model-generated unsafe disclosures into safe-exit trajectories, aligns a frozen LRM via guidance
tokens, and distills the guided behavior into model parameters. To address the lack of metrics for replacement quality beyond leakage, we further introduce Natural Forgetting
Reasoning Score (NFRS), which captures structural stability, fluency, and unsupported substitutes in forgotten outputs. Extensive experiments on R-TOFU and a
STAR-1-derived harmful-intent setting show that GUARD substantially reduces unsafe and
privacy disclosures across two widely adopted distilled LRMs while preserving
reasoning utility.
Read the full abstract ▾
All authors Zeyu Yan, Guanghao Zhou, Minghui Qiu, Ming Gao, Cen Chen
Artificial Intelligence
Accelerating Dense LLMs via L0-regularized Mixture-of-Experts
Zhenyu Zhang, Jiudong Yang, Zhaowen Tao et al. (+1 more)
Large
language models (LLMs) achieve strong performance but suffer from slow and costly inference. Existing acceleration methods often lead to noticeable performance degradation, while Mixture-of-Experts (MoE) models require extensive computational resources. In this paper, we propose L0-MoE, a lightweight MoE approach using L0-regularization to accelerate dense LLMs nearly without performance loss. Our method introduces a cluster confusion
matrix for domain-aware
dataset curation and applies dynamic batching for efficient training. Experiments show that L0-MoE achieves up to 2.5x speedup over dense models while maintaining competitive performance, outperforming existing LLM acceleration baselines.
Read the full abstract ▾
All authors Zhenyu Zhang, Jiudong Yang, Zhaowen Tao, Meng Chen
Artificial Intelligence · Cross-listed from cs.DL
From Code Archival to Knowledge Graph: Bridging Software Heritage, COAR Notify and Wikidata
Camillo Carlo Pellizzari di San Girolamo and Francesco Tosoni
Software is a first-class scientific object, yet validated links between source code and the scholarly record remain largely absent from the Linked Open Data (LOD) cloud, isolating archived artefacts from semantic discovery. This paper presents an end-to-end reconciliation pipeline that harvests, validates, and models publication-to-repository pairs from sources where the link between a paper and its source code is explicit and editorially verified: the software-centric journals JOSS, SoftwareX, and IPOL, together with the reproducibility reports of the SIGMOD Availability and Reproducibility Initiative (ARI). This yields a curated corpus of 4,397 $\langle$DOI, repository-URL$\rangle$ pairs. We design two distinct application profiles grounded in Wikidata classes (one for scholarly articles, one for software instances) aligned with the schema.org and CodeMeta vocabularies. This architectural separation enables rule-based reconciliation at two granularities: lightweight, inline publication references or standalone, first-class Wikidata software nodes equipped with SWHIDs, Software Heritage's content-addressed identifiers. A read-only lookup against Wikidata shows that only 82 of the harvested repositories were already modelled there; human-reviewed batches have since created 4{,}182 new software items cross-linked to their articles. We further show that payloads of the emerging COAR Notify protocol, an external effort we do not develop, map natively onto our input format, so the same backend could later serve a live enrichment stream. Our core contribution is a pair of application profiles that turn Wikidata into a connector between the scholarly record and archived source code; we openly release all code, application profiles, and harvested
datasets.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from eess.AS
Samsone: A Family of Open Small Audio Language Models for On-Device Inference
Piotr Masztalski, Michał K. Grzeszczyk and Olaf Sikorski
The success of Large Audio
Language Models has driven the development of massive
multimodal networks exceeding billions of parameters. However, the demand for
privacy-preserving, low-latency processing has shifted focus toward Small Audio
Language Models (SALMs) capable of on-device execution. In this paper, we introduce Samsone, a family of SALMs designed for edge computing. Our core model, Samsone-134M, establishes a new state-of-the-art for its size class across multiple
benchmarks. We further explore the scaling laws of SALMs by introducing Samsone-99M and Samsone-356M. Despite their compact footprint, the Samsone family delivers performance competitive with models orders of magnitude larger. To foster open research and reproducibility, we train Samsone on publicly available data. We release the training code, model weights, mobile-optimized checkpoints and provide an open-source Android application to demonstrate real-time on-device inference of Samsone.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
When Steering Fails in Latent Reasoning: A Latent-to-Language Transition Gap
Gaoxiang Huang and Lei Qi
Activation steering has become a widely used approach for controlling
language models during explicit chain-of-thought (CoT)
reasoning, motivating its extension to latent CoT. However, we find that steering continuous thoughts produces substantially weaker effects on subsequent
language generation than steering explicit CoT, even when the hidden representations are moved by comparable amounts. We first show that task information remains identifiable in continuous thoughts. Hence, we hypothesize a latent-to-
language transition gap, in which an intervention effect in latent space fails to transfer to
language generation. Two further results support this hypothesis: the output distribution changes abruptly at the transition boundary, and task-related directions exert much weaker bidirectional control in latent CoT than in explicit CoT. These findings identify the transition interface as a central target for evaluating and designing future latent-steering methods.
Read the full abstract ▾