Audio and Speech Processing
MAESTRO: a Multimodal Auditory-attention Egocentric Speech-TRacking Open corpus
K M Naimul Hassan, Ali Alavi and Donald S. Williamson
Humans rely on gaze, head movements, and visual cues to attend to speakers in noisy environments, yet auditory
attention decoding (AAD) has been studied primarily using electroencephalography (
EEG). We introduce the
Multimodal Auditory-
attention Egocentric
Speech-TRacking Open (MAESTRO) corpus, the first AAD
dataset to simultaneously record
EEG, eye gaze, pupillometry, egocentric
video, and head inertial measurement unit (IMU) data. MAESTRO includes four competing speakers and background noise across multiple signal-to-noise ratio (SNR) conditions, enabling
attention decoding under realistic listening scenarios. Through a four-speaker
attention decoding
benchmark, we show that combining behavioral and physiological signals improves decoding performance over
EEG-only approaches, enabling future advances in
multimodal auditory
attention decoding. These findings open the door to new applications, analyses, and methodological advances in
multimodal AAD. The complete
dataset is publicly available at https://huggingface.co/
datasets/aspire-osu/maestro-
eeg-
dataset . The official code repository is available at https://github.com/ASPIRE-OSU/MAESTRO .
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.SD
NVAlign: Direct-Gradient Optimization for Non-Verbal Control in Continuous Autoregressive Flow Matching Text-to-Speech
Qiaolin Wang, Pedro Sandoval-Segura, Anunaya Joshi et al. (+2 more)
While modern text-to-
speech (TTS) systems generate highly natural
speech and support inline non-verbal vocalization (NVV) tags, accurate control over these events remains challenging. A key gap is the lack of established post-training methods for non-verbal control in continuous autoregressive flow-matching TTS. To this end, we present NVAlign, a direct-gradient post-training framework for NVV tag-following in this architecture. We first perform supervised fine-tuning (SFT) of TTS models and an NVV-aware automatic
speech recognition (NV-ASR) model on NVV-annotated
speech, then freeze the NV-ASR model to serve as the reward model for post-training. A two-step gradient surrogate enables efficient reward backpropagation through the flow-matching sampler to jointly update the autoregressive backbone and acoustic flow head. Fidelity penalties and reference-velocity regularization help preserve speaker similarity and
speech quality. Results from NVV-SuperBench and human listening evaluations show that NVAlign improves tag-following accuracy over SFT and Flow-GRPO baselines. These findings demonstrate that direct reward-gradient optimization can improve non-verbal control in continuous autoregressive flow-matching TTS. Audio samples are available at https://nvalign.github.io/.
Read the full abstract ▾
All authors Qiaolin Wang, Pedro Sandoval-Segura, Anunaya Joshi, Edvardas Jurkonis, Jake Downie
Audio and Speech Processing · Cross-listed from cs.SD
CORD-KWS: Calibrated, Order-Aware Detection for Open-Vocabulary Keyword Spotting
Ramesh Gundluru, Adarsh Arigala and Sri Rama Murty Kodukula
Open-vocabulary keyword spotting (KWS) must detect arbitrary keywords without retraining. Cross-
attention-based models achieve state-of-the-art performance but require pairwise interaction between the audio and each keyword, recomputing the fused representation for every audio–keyword pair.
Embedding-based models avoid this computational cost through independent encoding and similarity scoring, yet remain inferior on challenging
benchmarks such as LibriPhrase-hard. We show that this gap is primarily a training issue rather than a limitation of model capacity. While contrastive learning encourages correct keywords to rank above negatives, it does not explicitly calibrate absolute similarity scores, which are critical for applying a fixed threshold across keyword detection. We propose Calibrated, Order-aware Detection KWS (CORD-KWS), a training framework that introduces two complementary objectives while retaining the same encoders, cosine scoring, and O(1) keyword enrollment: a calibrated detection head that supervises absolute scores, and a frame-level CTC objective that preserves phonetic order information. CORD-KWS achieves EERs of 0.43% and 9.64% on LibriPhrase-easy and LibriPhrase-hard, respectively, outperforming existing
embedding- and cross-
attention-based models.
Read the full abstract ▾
Systems and Control
An Exact Lyapunov Characterization of Regional Rate Performance for Rational Stability
Declan S. Jagt and Matthew M. Peet
Rational stability is a quantitative stability notion for nonlinear ordinary differential equations (ODEs), under which convergence is characterized by a rationally-in-time decaying bound on solutions. Although Lyapunov characterizations of rational stability have been proposed, no existing condition has been shown to exactly characterize rational rate performance. The paper resolves this issue by providing an exact converse Lyapunov characterization of regional rational stability with prescribed rate performance. To test the resulting converse Lyapunov conditions numerically, a tiered set of convex relaxations is proposed which can be enforced using Sum-of-Squares (SOS) programming. A conjectured conservatism scaling factor in rate performance is associated with each tier and validated numerically. The first proposed tier appears to be novel and is shown to outperform classical SOS-based approaches. Numerical examples are used to validate the results and compute nested regions of state-space on which prescribed levels of performance can be guaranteed.
Read the full abstract ▾
Image and Video Processing
Awaken, Then Scale: Tiny Adaptation for MRI Reconstruction
Mohammed Wattad, Tamir Shor and Alexander M. Bronstein
Dormant Awakening (DA) specializes a
magnetic resonance imaging (MRI) reconstruction model by fitting a small set of zero weights to one labeled slice. Across fourteen U-shaped convolutional network (U-Net) and
vision transformer (ViT) sources, fitting 1,004 or 10,560 weights gives mean peak signal-to-noise ratio (PSNR) gains of .430 and .509 dB. We analyze how scaling the fitted output correction changes evaluation PSNR, then use the calibration correction size to cap changes on new images. The cap bounds changes in root mean squared error (RMSE) relative to the source without an evaluation reference. At 10% adaptation budget, we compare fixed halving and dynamic capping for DA, unrestricted sparse adaptation and low-rank adaptation (LoRA). The controls increase both measured calibration-evaluation correlations across all six tested architecture-adapter settings. Experiments concern one constructed fastMRI-to-M4Raw shift with fixed source checkpoints and participant panels.
Read the full abstract ▾
Image and Video Processing
Beyond Sparsity: Weight Location and Network Context in Pruned MRI Reconstruction
Mohammed Wattad, Tamir Shor and Alexander M. Bronstein
Pruning reduces the number of weights in
magnetic resonance imaging (MRI) reconstruction networks. Equal sparsity, however, can retain weights with different computational roles and different compatibility with the trained network. We study these effects across 120 convolutional U-Net and
vision transformer models using controlled edits evaluated before retraining. In U-Net, preserving high-resolution computation improves reconstruction at equal deletion counts across all tested sparsity levels. Equal-operation controls reveal additional sensitivity of the first convolution, which operation count alone cannot explain. In
transformers, retained pretrained weight energy correlates with quality within sparsity levels, yet the same masks reverse their reconstruction ranking when the surrounding trained weights change. At 90% sparsity, matching intermediate output scales largely removes a high-energy-mask penalty while retaining an advantage for the network's original mask. Thus sparsity alone does not characterize reconstruction quality: architecture-specific descriptors are informative, but mask quality can still depend on the surrounding trained network.
Read the full abstract ▾
Image and Video Processing
MammoClaw: Towards Skill-Evolving Agent Harness for Breast Cancer Mammography Analysis
Krishna Kanth Nakka
In this work, we explore MammoClaw, a training-free
agent framework that leverages frozen MLLMs for mammography analysis. To support agentic investigation, we equip the
agent with lightweight mammography-specific tools for targeted image analysis, including ROI, paired-view, and contralateral-breast examination. MammoClaw iteratively gathers evidence through these tools, while skill
evolution enables non-parametric adaptation by transforming failed trajectories into reusable guidance for later runs. We evaluate the framework on BI-RADS assessment and breast density estimation tasks. In our experiments, we find that tools alone do not reliably improve performance, whereas evolved skills can improve tool-use behavior and performance in some settings. Beyond these results, MammoClaw enables transparent inspection of evidence acquisition, tool interactions, and failure modes, facilitating the analysis and auditing of
agent behavior. We view this work as an exploratory study of training-free, self-evolving agentic approaches for mammography and hope it provides a concrete starting point for future work on mammography-specific tools and self-
evolution mechanisms. We release our code at https://krishnakanthnakka.github.io/mammoclaw.
Read the full abstract ▾
Signal Processing
Stacked Fluid Metasurfaces: Mutual-Coupling-Aware Modeling and Optimization
Giovanni Iacovelli, Chandan Kumar Sheemar and Symeon Chatzinotas
Stacked intelligent metasurfaces process the transmitted field layer by layer, and fluid
antennas make the position of every radiator a design variable. Combined, they pack radiators at sub-wavelength spacings within and across layers, where mutual coupling governs the physics that current models omit. This paper develops a coupling-consistent model of a fluid transmit layer illuminating a stack of passive fluid beyond-diagonal layers. The multiport impedance
matrix of any port constellation is shown to be the impedance kernel sampled at the three-dimensional separations, in one closed spherical-Hankel form whose in-plane restriction is intra-layer coupling and whose axial restriction replaces the scalar Rayleigh–Sommerfeld propagator. Only the transmitter is driven: the layer currents are induced, and the familiar cascade is the single-pass limit of one
matrix inversion. In the wavenumber domain the light circle still separates radiation from reaction, now with a plane-wave propagator attached, every
Fourier mode sees a transmission line loaded by the layers, and an efficiency identity shows that what the stack costs in efficiency is set by its total induced-current norm, not by its layer count. Because the load is never assumed layer-diagonal, interconnections may join atoms on different sheets, and vertical networks repeated at every site are spectrally local. Alternating optimization designs precoders, positions, and loads with closed-form gradients. Numerically, fluid ports on a $\lambda/8$ grid perform within a few per cent of continuously movable ones, while a design based on the conventional cascade model can do worse than open-circuiting the stack.
Read the full abstract ▾
Audio and Speech Processing
RePlay: Retrieval-Based Voice Playback for Multi-Turn spoken dialogue
Sathvik Udupa, Naveen Kumar and Ryan Folmsbee
Many voice interaction applications require exact control over both the content and delivery of responses, typically using pre-recorded lines. Recent full-duplex models respond with low latency but cannot guarantee exact content or reproduce a specific recorded performance, while cascaded systems can be constrained to predefined responses at the cost of additional latency. We propose RePlay, a spoken dialogue system adapted from PersonaPlex that handles multi-turn conversations by retrieving and playing pre-recorded lines. Using probing, we identify the layer and frame at which the upcoming response becomes recoverable, and use this hidden state as the
retrieval query. RePlay retains only the layers up to that point and replaces text and
speech generation with lightweight turn-taking and
retrieval heads. In simulated multi-turn interviews, RePlay reaches a median latency of 383 ms, 3 to 7 times lower than ASR-LLM cascades of comparable dialogue quality, at the cost of lower exact-line accuracy. In a user study, participants preferred RePlay in 63% of ratings versus 12% for a fast cascade with a small LLM (p = 0.008), and showed a non-significant preference (46% vs. 21%) over a slower cascade with a stronger LLM.
Read the full abstract ▾
Signal Processing · Cross-listed from cs.LG
Online Learning via Learned Latent Bayesian Tracking
Guy Gerson, Tomer Raviv, Nir Shlezinger et al. (+2 more)
Online learning in non-stationary environments requires models to adapt rapidly from streaming data under strict computational constraints. A principled approach casts online learning as
Bayesian state tracking, where model parameters are updated sequentially via
Bayesian filtering. However, applying
Bayesian filters directly to modern deep models is computationally prohibitive due to the high dimensionality of parameter space, forcing existing methods to rely on restrictive approximations or manually designed low-dimensional subspaces. In this work, we identify the absence of a suitable low-dimensional dynamical representation as the core bottleneck in
Bayesian filtering-based online learning. Accordingly, we propose Adaptive Update through Representation Adaptation (AURA), a meta-learning framework that learns offline a low-dimensional latent state-space model governing the
evolution of optimal model parameters under distribution shift. Online adaptation is then performed via extended Kalman filtering in this learned latent space followed by reconstruction of the full model parameters through a learned lifting map, enabling efficient single-step online adaptation while preserving model expressiveness. Evaluated on online adaptation of neural wireless receivers under time-varying channels and on non-stationary image classification, AURA shows substantial improvements in adaptation speed, accuracy, and computational efficiency over existing online learning and
Bayesian filtering baselines, demonstrating that an adaptation-aware latent geometry is beneficial for effective
Bayesian online learning in high-dimensional models.
Read the full abstract ▾
All authors Guy Gerson, Tomer Raviv, Nir Shlezinger, Tirza Routtenberg, Osvaldo Simeone
Audio and Speech Processing · Cross-listed from cs.SD
TinyAudio: Compact and Efficient Text-to-Audio Generation for Low-Resource Deployment
Junxi Liu, Xiquan Li, Wenhao Guan et al. (+5 more)
Text-to-audio (TTA) generation has advanced rapidly in generation quality and instruction following. However, representative systems often require around a billion parameters, limiting deployment on resource-constrained devices. This paper introduces TinyAudio, a compact flow-matching-based TTA model for low-resource deployment. At its core, TinyAudio uses TA-DiT, a 35M single-stream flow-matching
Transformer. TinyAudio also includes TA-CLAP, a 32M audio-aligned text encoder, and TA-VAE, whose 20M decoder reconstructs 44.1 kHz audio from compressed latents. TinyAudio has only 87M parameters in total, over 90% fewer than representative billion-parameter pipelines, and uses 0.48 GB peak GPU
memory. TinyAudio achieves competitive generation quality on AudioCaps and TTA-Bench. We further introduce TinyAudio-MF, a MeanFlow-accelerated model that enables real-time generation with a four-core CPU quota. Our results demonstrate a practical quality-footprint trade-off for low-resource TTA deployment.
Read the full abstract ▾
All authors Junxi Liu, Xiquan Li, Wenhao Guan, Yifan Duan, Zhikang Niu, Yanru Huo, Ziyang Ma, Xie Chen
Audio and Speech Processing
Assessing a Mathematical Model of Syllable Production via CTW Alignment with EMA Data
Frédéric Berthommier
This study evaluates a mathematical model of syllable production using an EMA
dataset from a recently published study, which proved highly compatible with the model's architecture. The data set consists of regularly structured French phrases of equal duration and reduced phonetic and syllabic complexity. A dedicated procedure was used to transform EMA recordings into Maeda parameters. The Model-generated trajectories were then realigned with these transformed data using Canonical Time Warping (CTW). Statistical validation was conducted via a permutation test, comparing
alignment quality against a surrogate condition with phonetically incongruent model outputs. The results show significantly better
alignment for phonetically congruent pairings, revealing a strong structural correspondence between the model and the articulatory data. These findings provide experimental support for the representational validity of the model.
Read the full abstract ▾
Signal Processing · Cross-listed from cs.IT
Fundamental Limits of Sequence Reconstruction Problems in Immunogenomics
Jaswanthi Mandalapu, Deepak Charan, V. Arvind Rameshwar et al. (+1 more)
The goal of personalized immunogenomics is to recover an individual's germline immunoglobulin gene segments from 'repertoire sequences' altered by trimming, extension, and mutation. In this work, we study the fundamental trace complexity (or the number of samples/traces required for accurate reconstruction) of D-gene reconstruction under three biologically motivated trace-generation models introduced by Bhardwaj et al. (2021) and develop practical
algorithms for reconstruction problems left open in the original work. First, for the TrimSuffixAndExtend model, we establish the optimal trace complexity to be {\Theta(n)}, and develop a low-complexity Prefix-Filtered Mode (PFM) decoder that achieves this scaling. Second, for the closely related two-sided TrimAndExtend model, we show that the optimal trace complexity is instead {\Theta(n^2)}, and is achieved by the simple Bit-Wise Mode (BWM) decoder. Third, for the SuffixExtend-t(TrimSuffix) model, we establish polynomially separated lower and upper bounds on trace complexity. Our results follow from information-theoretic lower bounds coupled with tight analyses of the proposed reconstruction
algorithms.
Read the full abstract ▾
All authors Jaswanthi Mandalapu, Deepak Charan, V. Arvind Rameshwar, Nir Weinberger
Audio and Speech Processing
SAID: Semantic Acoustic Imaging Detector for Sound Event Localization and Detection
Runbang Wang, Zining Liang, Yin Cao et al. (+1 more)
In daily life, people hear
speech, footsteps, and music around them. We can often recognize these sounds and judge where they come from. Each sound source can be shown on a separate acoustic map, a rectangular image covering $360^{\circ}$ horizontally and $180^{\circ}$ vertically. The map shows the directions occupied by the source as a region and the sound energy within that region. A class label identifies the sound. Predicting these labeled acoustic maps from audio is called semantic acoustic imaging. Such maps could help
robots perceive their surroundings and allow augmented reality displays to show sound regions and classes over the real world. Existing models can recognize sound classes and estimate a direction for each source. However, a direction alone does not describe the source region or its energy. Acoustic imaging must also distinguish sound sources in nearby directions, while the number of active sources and the regions they occupy can change over time. We therefore propose the Semantic Acoustic Imaging Detector (SAID), which predicts a separate labeled acoustic map for each active source from audio. First, we pretrain Audio2Sph, SAID's audio encoder, through sound energy estimation across directions without class labels. Then, we train the complete SAID model to predict source regions, energy, and classes together. We also develop a pipeline that generates simulated recordings for pretraining and supports fine-tuning on real recordings. On the official DCASE2026 Task 3 Track A evaluation set, our submitted system ranks first with 0.1080 macro-averaged mean average precision (Macro mAP) and 0.3962 Macro Pearson $r$. Demos and code are provided at https://github.com/IN03X/SAID.
Read the full abstract ▾
All authors Runbang Wang, Zining Liang, Yin Cao, Qiuqiang Kong
Systems and Control · Cross-listed from cs.RO
ExoLaN: Physics-Consistent Context-Aware Dynamics Learning for Exoskeletons
Lucas Schulze, Maximilian Schwarz, Jona Hoppe et al. (+2 more)
Task-agnostic assistive exoskeleton control based on human intention offers greater flexibility than conventional approaches that rely on predefined tasks or motion patterns. Human joint torque estimation enables task-agnostic assistance by characterizing user actions. Physics-consistent methods such as Deep Lagrangian Networks (DeLaN) have been applied to estimate the human torques in multi-user settings, but existing approaches cannot adapt to a specific user without retraining, and do not account for intermittent contacts during locomotion. We propose ExoLaN, a Context-Aware DeLaN for human-exoskeleton interaction that learns the full coupled system dynamics while adapting to changes in interaction context. ExoLaN combines temporal context with partial contact-force measurements from force-sensitive insoles to infer latent dynamics
embeddings and estimate generalized contact torques. On seven unseen users performing 21 unseen tasks, ExoLaN reduces torque estimation MSE by 7% compared to a black-box baseline. Beyond inverse dynamics, ExoLaN serves as a unified model that also enables accurate forward prediction: training with a multi-step prediction loss reduces acceleration MSE by 59% and long-horizon position and velocity errors by 60% and 93%, respectively, compared with a single-step loss. Moreover, the learned latent context captures task information without explicit task labels, making it a promising signal for task-aware assistive control.
Read the full abstract ▾
All authors Lucas Schulze, Maximilian Schwarz, Jona Hoppe, Jan Peters, Oleg Arenz
Signal Processing
Wireless Evidence Acquisition for Multimodal Inference Constrained by Temporal Windows
Alessandro Compagnoni, Anup Mishra, Carla Fabiana Chiasserini et al. (+2 more)
Multimodal inference in safety-critical applications, such as autonomous driving and
robot navigation, requires heterogeneous sensor observations to reach an edge server within a task-prescribed temporal window. Referenced to the event or state being inferred, this window ends at the latest time an observation remains useful for the current decision. Since sensor readiness times, data volumes, wireless channel conditions, and task relevance vary across sensors, maximising network throughput does not necessarily minimise the fused prediction error when the window closes. We cast
multimodal uplink scheduling as sequential wireless evidence acquisition and, for a linear minimum mean-square error (LMMSE) fusion model, derive a conditional evidence gain metric from the reduction in residual-error volume. Defined through second-order statistics, this metric applies beyond Gaussian models and coincides with conditional mutual information when the target and prediction errors are jointly Gaussian. We then develop MIRA, a greedy task-driven maximum information-rate allocation policy that combines conditional evidence gain with each sensor's channel state information and remaining data volume, while updating sensor relevance as evidence is acquired. Experiments on synthetic classification, human activity recognition, and vehicle-trajectory regression show that MIRA outperforms both relevance-only and channel-only scheduling. Relative to the former, MIRA improves classification accuracy by up to 70% and reduces regression mean-square error (MSE) by up to 5.5%. Relative to the latter, it requires 58% less acquisition time to attain an 80% target accuracy, while achieving up to 95% higher accuracy and 9% lower regression MSE. These gains are achieved without maximising received-data volume.
Read the full abstract ▾
All authors Alessandro Compagnoni, Anup Mishra, Carla Fabiana Chiasserini, Elad Michael Schiller, Petar Popovski
Signal Processing
Variable Dimension IMM and GPB2 Filters for Tracking in Multiple Model Systems
Roberto Pérez-Pérez and Ángel F. García-Fernández
This paper presents mathematically principled filters for multiple-model systems with states of different dimensionality, specifically the variable-dimension interacting multiple model (VD-IMM) filter and the variable-dimension generalised pseudo-
Bayesian filter of order 2 (VD-GPB2). To do so, we first provide a
Bayesian modelling of a variable dimensional dynamic system, and its measurements. Then, for variable dimensional linear Gaussian dynamic and measurement models, the VD-IMM filter is derived by assuming a posterior that has a Gaussian density for each mode, and then performing a Kullback-Leibler Divergence (KLD) minimisation after each prediction step to keep the Gaussian density form for each mode. Subsequently, the
Bayesian update step is performed, keeping a Gaussian density for each mode. The VD-GPB2 filter considers the same model as the VD-IMM filter and also assumes that the posterior for each mode is Gaussian. In contrast, the VD-GPB2 filter propagates a Gaussian mixture for each mode in the prediction step. In the update step, the VD-GPB2 filter performs a KLD minimisation to have a Gaussian density for each mode. Simulation results show the benefits of the VD-IMM and VD-GPB2 filters compared to previous alternatives.
Read the full abstract ▾
Signal Processing · Cross-listed from cs.SD
AFA-Net: A Differential Attention Approach for Auditory Attention Detection
Philip H. Lee, Shreeram Suresh Chandra, Karan Thakkar et al. (+1 more)
Auditory
Attention Detection (AAD) utilizes electroencephalographic (
EEG) signals to identify a target speaker in a multi-speaker environment. Despite considerable progress, existing deep learning architectures often lack explicit mechanisms for handling noisy
EEG data. To address this limitation, we propose Auditory Focus
Attention Networks (AFA-Net), a machine learning framework that replaces vanilla
attention with a simple yet flexible differential
attention mechanism to help focus on task-relevant neural activity. AFA-Net achieves an upward accuracy of 96.8% at the 2s decision window, while using substantially fewer parameters than most existing methods. To the best of our knowledge, AFA-Net is among the first frameworks to explicitly try to combat
EEG noise to improve AAD.
Read the full abstract ▾
All authors Philip H. Lee, Shreeram Suresh Chandra, Karan Thakkar, John H.L. Hansen
Signal Processing · Cross-listed from cs.LG
Differential Attention Unlocks Complementary EEG and Speech Fusion for Emotion Recognition
Philip H. Lee, Shreeram Suresh Chandra and John H.L. Hansen
Multimodal emotion recognition (MER) increasingly pairs
EEG with
speech, treating internal neural signals and external vocal expression as informative views of affect. In practice, naive fusion underperforms the stronger single modality, because
EEG artifacts inject noise that corrupts the shared representation. We introduce EmoSpeechBrain, a
multimodal framework built on the insight that noise suppression is a precondition for effective fusion. Its
EEG encoder uses differential
attention, taking the difference between two
attention maps to cancel shared noise and isolate discriminative neural activity. An
attention-based gating adapter aligns both modalities in a shared space and weights each one's contribution to the prediction. On two
datasets - PME4 and EAV, EmoSpeechBrain improves MER accuracy by up to 12.9% over other state-of-the-art (SOTA)
EEG encoders, and surpasses unimodal
speech and
EEG baselines by up to 13.1% and 23.1%. These results show that once
EEG noise is suppressed, fusion delivers gains that naive combination cannot.
Read the full abstract ▾
Image and Video Processing
Towards Whole-Study Screening for Congenital Heart Disease in Fetal Ultrasound Using Multiple Instance Learning
Mohamed Azzam, Ruobing Liu, Esther C. Ugwueke et al. (+8 more)
Congenital heart disease (CHD) is the most common birth defect, yet a large fraction of cases remain undetected on prenatal ultrasound, in part because current artificial-intelligence methods assume that the key diagnostic frames have already been isolated from a study, by a clinician or by a view classifier. We remove that assumption and address CHD screening directly at the level of the whole ultrasound study. We propose a two-stage framework that first learns transferable frame representations by self-supervised masked-autoencoder pre-training on unlabeled fetal ultrasound, then identifies cardiac frames with a disease-robust module and aggregates them with a
transformer-based multiple instance learning (MIL) model that produces a case-level diagnosis from study-level labels alone. The model further returns its highest-scoring frames for clinician review, and a hierarchical head separates critical from non-critical CHD. On the internal test set of our multi-source development cohort (FUSE), the proposed cardiac-gated MIL model reaches an area under the curve (AUC) of 0.985 with a specificity of 0.990, outperforming the reproduced NATMED ensemble (AUC 0.861, specificity 0.600) and the FetalCLIP foundation model (AUC 0.867, specificity 0.710). On an independent external cohort, all models initially perform near chance, but label-free CORAL adaptation raises the proposed model from an AUC of 0.513 to 0.944, whereas whole-study and view-dependent baselines do not recover. These results indicate that whole-study MIL with disease-robust cardiac-frame identification is an accurate and deployable route to prenatal CHD screening.
Read the full abstract ▾
All authors Mohamed Azzam, Ruobing Liu, Esther C. Ugwueke, Ziyang Xu, Shibiao Wan, Alex Foy, Abraham Zabih, Jason Christensen, Neil Hamill, Ling Li, Jieqiong Wang
Systems and Control · Cross-listed from stat.ML
Equation discovery with Bayesian tree-adjoining grammars
Christopher A. Lindley, Nikolaos Dervilis and Keith Worden
Tree-Adjoining Grammars (TAGs) have recently been introduced to Nonlinear System Identification (NLSI) as a means of encoding an entire model class as a finite set of grammatical rules, from which candidate models are assembled as trees. Existing TAG-based identifiers rely on evolutionary optimisation and return point estimates of the model structure. This paper instead proposes the TAG framework within a
Bayesian setting. A generative prior is defined over tree structures and their parameters, and a Reversible-Jump MCMC sampler with structure-preserving tree moves is used to infer the joint posterior over model structure, parameters and predictions. Two training objectives are considered; that is, a one-step-ahead objective with conjugate parameter proposals, and a simulation-based objective handled by likelihood-free inference. The approach is validated on a simulated
polynomial NARX system, the Silverbox
benchmark, and wave-loading data from the Christchurch Bay Tower, where
embedding Morison's equation as a fixed initial tree yields a grey-box model that outperforms the physics-driven baseline. The results demonstrate that
Bayesian TAGs are well suited to quantifying uncertainty in equation discovery for dynamical systems and to fitting physics-informed models.
Read the full abstract ▾
Signal Processing
Resolving the Dual-Timescale Forecasting Dilemma in High-Frequency Cyber-Physical Telemetry via Physics-Grounded Dynamic Asymptotic Decomposition
Avik Kumar Das
High-frequency industrial cyber-physical systems (CPS) stream continuous sensor telemetry at sub-10-second intervals. Multi-horizon forecasting over hours is required for predictive maintenance and operational control. However, deploying machine learning models over extended horizons (H = 2,160 steps at dt = 5 s) encounters the Dual-Timescale Forecasting Dilemma: the trade-off between short-horizon kinetic momentum and long-horizon diurnal thermodynamic
equilibrium. Autoregressive rollouts suffer from compounding error drift, while direct multi-output projectors exhibit variance explosion and noise extrapolation. In this work, we prove that autoregressive mutual information decays exponentially toward zero over extended lead times, and that the optimal minimum-variance
estimator converges asymptotically to the periodic diurnal baseline (Lemma 1 and
Theorem 1). Guided by these bounds, we propose the Dynamic Asymptotic Decomposition (DAD) framework. The architecture couples a continuous sigmoidal authority transition schedule \(\alpha(h)\) with monotonic L2 regularization scaling \(\lambda(h)\), transferring prediction authority from kinetic autoregression to diurnal
equilibrium while enforcing physical non-negativity. Evaluated across a 30-day industrial telemetry stream (N = 518,400 steps) over 16 walk-forward test windows, DAD achieves an overall Mean Absolute Error (MAE) of 0.1764, representing a 78.3% error reduction over Deep LSTM (0.8143), 60.2% over PatchTST (0.4431), and 20.8% over DLinear (0.2227). Computational profiling demonstrates edge execution in 0.72 ms with 0.203 MFLOPs and 72k parameters on a single CPU core (over 3400x faster than
Transformer baselines), validated by an automated dual-stage CI/CD quality assurance harness and physical fail-safe architecture.
Read the full abstract ▾
Audio and Speech Processing
Why Alzheimer's Speech Screening Fails to Generalize: Bridging the Deployment Gap via Cross-Corpus Evidence Anchoring
Zijian Lu, Sizhe Liu, Yin Zhang et al. (+6 more)
Speech-based screening is a promising, non-invasive approach for detecting Alzheimer's disease and related cognitive risks. However, models trained on a single domain often generalize poorly to unseen
languages, tasks, or recording protocols. This paper investigates this deployment gap using a leave-one-corpus-out evaluation across four distinct
datasets. Among 70 interpretable
speech and
language features, 59 exhibit direction conflicts between healthy control and cognitive risk groups across corpora, with pause, silence, and
speech rate showing high protocol sensitivity. Furthermore, while the XLM-R text baseline achieves strong average performance, its Area Under the ROC Curve (AUC) drops to 0.520 on the weakest held-out domain. A standard GroupDRO baseline reaches a 0.766 mean speaker AUC and a 0.504 worst-domain AUC under the same protocol. To address this, we propose a fusion method that integrates XLM-R text baseline scores with evidence anchors selected during training. Balanced fusion achieves a 0.785 mean speaker AUC, while anchor-heavy fusion raises the worst-case speaker AUC to 0.615. This work highlights the need to audit feature transferability and report worst-case domain robustness in cognitive
speech screening.
Read the full abstract ▾
All authors Zijian Lu, Sizhe Liu, Yin Zhang, Jixuan Deng, Xinrong Lin, Xinchen Yuan, Chicheng Jin, Yiping Zuo, Yuanchao Li
Systems and Control
Sensitivity-driven Personalization of a Glucoregulatory Model for Digital Twin Therapeutics in Type 1 Diabetes
Clara Escorihuela-Altaba, Vihangkumar V. Naik, Eleonora Manzoni et al. (+1 more)
Digital twins are increasingly used in diabetes research, but reproducing individual glucose dynamics requires accurate identification of glucoregulatory model parameters. Traditional sensitivity analysis can identify influential parameters, yet a ranking based on limited conditions may miss parameters that matter during specific disturbances or for particular individuals. We therefore examine both the magnitude and timing of parameter influence across dynamic input-output conditions and assess whether a common ranking holds across participants. We analyze the Hovorka glucoregulatory model using data from 192 participants receiving automated insulin delivery therapy in the Type 1 Diabetes and Exercise Initiative
dataset. We extend Sobol sensitivity analysis to time series and rank parameter influence under four conditions: full-day profiles, isolated meal disturbances, insulin bolus injections, and postprandial responses. We combine the condition-specific results into a global ranking and use it to select parameters for participant-specific identification. Compared with population parameters, identification restricted to the sensitivity-derived subset reduces the root mean square error of 60-minute glucose predictions by 60%, to approximately 31 mg/dL. These findings suggest that a global ranking can capture parameter influence across individuals and dynamic conditions. By narrowing the parameters requiring identification, this approach reduces computational cost and could accelerate the development of personalized diabetes digital twins.
Read the full abstract ▾
All authors Clara Escorihuela-Altaba, Vihangkumar V. Naik, Eleonora Manzoni, Jose Garcia-Tirado
Signal Processing
Scalable Variable Selection under Predictor Dependence with Adaptive Virtual Dummies
Taulant Koka and Michael Muma
Reliable high-dimensional variable selection requires scalable error-controlling methods. The Terminating-Random Experiments (T-Rex) selector estimates the false discovery rate (FDR) by aggregating early-terminated forward-selection paths in which predictors compete with synthetic dummies. We address two remaining challenges: i) predictor dependence can bias dummy-predictor competition; ii) computation is wasted on recomputing terms shared across experiments. Building on
memory-efficient virtual dummies, which sequentially sample projections from their exact conditional law, we estimate the conditional Gaussian law of inactive predictors and map draws onto the remaining sphere radius. A shared lazy Gram cache computes the response product and each requested Gram column once across experiments. Simulations across three covariance structures show that uniform spherical dummies can exceed the target FDR, whereas the proposed method empirically controls FDR. Caching yields more than a sixfold speedup, and the method remains feasible with 100000 predictors, where competing FDR-controlling methods become computationally impractical.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.SD
Acoustic-to-Text KV Compression for Full-Duplex Speech Models
Yejin Lee, Seungbeom Kim, Yongha Lee et al. (+1 more)
Full-duplex
speech language models continuously accumulate acoustic key-value (KV) states, making long-running interactions
memory-intensive. During listening, the model can finish processing an audio unit before the next arrives; we term the remaining interval listening-time slack. We propose acoustic-to-text KV compression, which introduces a transcription side channel to convert incoming
speech into compact textual
memory within this interval. When the cache exceeds a target budget during inference, older acoustic states are evicted while transcripts and recent acoustic context remain. We train the side channel with LoRA using cross-
entropy on transcription segments. To preserve listening and speaking behavior, we apply knowledge
distillation to the original model's
token-level output distributions at native prediction positions. On ten-minute LongSpeech sessions, our MiniCPM-o 4.5 implementation reduces peak streaming KV-cache size by 64.6% compared with the same model without eviction. The proposed method also improves transcription, temporal question answering, and summarization over the baseline. Full-Duplex-Bench evaluations further show comparable pause-handling, turn-taking, and interruption performance.
Read the full abstract ▾
All authors Yejin Lee, Seungbeom Kim, Yongha Lee, Kyuhong Shim
Signal Processing
POPT: Physics-Informed Deep Learning for Phase-Only Positioning in Distributed MIMO
Fatih Ayten, Ossi Kaltiokallio, Jukka Talvitie et al. (+6 more)
This article develops a physics-informed deep learning framework for single-snapshot 3D phase-only positioning in narrowband phase-coherent distributed multiple-input multiple-output (D-
MIMO) networks under two-ray propagation. Unlike prior phase-coherent D-
MIMO localization methods that largely assume line-of-sight (LoS)-only channels, we consider a LoS path and a specular ground reflection. Since the narrowband single-snapshot observation cannot resolve these components in delay, their coherent superposition induces structured perturbations in the carrier phase measurements. To address this, we develop a Gaussian process (GP)-based model that learns the quasi-periodic phase distortion caused by ground reflection from carrier phase measurements collected at a small number of training locations and uses it to generate high-quality synthetic samples. Building on these samples, we propose the Phase-Only Positioning
Transformer (POPT), an encoder-only
transformer that captures inter-
antenna point (AP) phase relationships without solving the highly non-convex maximum-likelihood (ML) problem underlying model-based
estimators. We also derive the fundamental position error bound (PEB) and develop maximum-likelihood estimation (MLE) for the considered multipath phase-only D-
MIMO positioning problem. Numerical results show that, with only 50 GP training locations, the proposed method achieves near-PEB accuracy. The analytical MLE is highly sensitive to relative permittivity mismatch, whereas the proposed method mitigates this dependence by learning the phase perturbation directly from measurement data. Compared with MLE, the proposed approach also reduces floating-point operation (FLOP) complexity and inference time by about 1.7 and 3.6 orders of magnitude, respectively.
Read the full abstract ▾
All authors Fatih Ayten, Ossi Kaltiokallio, Jukka Talvitie, Akshay Jain, Mehmet C. Ilter, Musa Furkan Keskin, Elena Simona Lohan, Henk Wymeersch, Mikko Valkama
Image and Video Processing
Enhancing Photogrammetric Digital Surface Models with Pretrained Diffusion Models and Multimodal Conditioning
Antoine Lorentz, Stéphane May, Valentine Bellet et al. (+2 more)
Large-scale Digital Surface Models (DSMs) can be produced cost-effectively from satellite images via stereo-photogrammetry. However, the resulting 3D maps are often contaminated by noise, outliers, and voids. On the other hand, aerial
LiDAR provides high-accuracy elevation measurements at a substantially higher cost. In this work, we study
diffusion models conditioned both on photogrammetric DSMs and Pléiades imagery to refine vertically co-registered DSMs. We introduce a modified Stable
Diffusion 3 architecture with a pruned text stream and a patch-wise normalization strategy, enabling stable training on
LiDAR data and transfer from natural images to elevation maps. Experiments in French cities demonstrate that
multimodal conditioning improves elevation accuracy, reducing Dense Urban RMSE from 6.00 to 3.45 m in the in-context cities and from 4.16 to 2.77 m in the held-out city of Bordeaux.
Read the full abstract ▾
All authors Antoine Lorentz, Stéphane May, Valentine Bellet, Dawa Derksen, Bastien Nespoulous
Signal Processing
Automated Artifact Removal in EEG Age Prediction: A systematic comparison
Davoud Hajhassani, Paul-Adrien Graignic, Bruno Aristimunha et al. (+8 more)
Automated
EEG artifact removal may improve downstream analysis but can also alter predictive information. We benchmarked nine automated artifact removal methods against a common no-artifact-removal baseline for cross-
dataset EEG age prediction. We introduce Signal Quality Index (SQI)-guided GEDAI, which leverages local signal-quality assessment to restrict correction to the channel–epoch pairs requiring intervention. Three deep neural architectures were trained on TUEG and evaluated without target-domain fitting on ds005385, LEMON, and TDBRAIN. Across this setting, GEDAI and SQI-guided GEDAI were the only methods with consistent gains over baseline in age prediction performance across all
datasets and architectures ($\Delta$MAE $=-0.77/-0.64$ years, $\Delta R^2=+0.083/+0.072$, respectively). The remaining methods were neutral or detrimental on average ($\Delta$MAE $=+0.22\pm0.16$ years, $\Delta R^2=-0.023\pm0.016$ across methods). The two GEDAI-based methods achieved closely matched performance, while SQI guidance reduced the median modification ratio from $74.78\%$ to $42.80\%$. These findings show that curation benefits are method-dependent and establish SQI guidance as a more selective operating point, leaving more of the original
EEG unchanged and limiting the potential loss of neural activity while retaining most of GEDAI's predictive benefit.
Read the full abstract ▾
All authors Davoud Hajhassani, Paul-Adrien Graignic, Bruno Aristimunha, Apolline Mellot, Tom Mariani, Clément Nober, Bruna J. Lopes, Léo Burgund, Lionel Kusch, Thomas Semah, Arnault H. Caillet
Audio and Speech Processing · Cross-listed from cs.CV
Who Says What: Symbolic Trimodal Binding Mechanisms in Audio-Visual LLMs
Jihoo Jung, Youngjoon Jang and Joon Son Chung
Current Audio-Visual LLMs (AVLLMs) struggle with
reasoning over
videos featuring multi-speaker dialogues. In such
videos, resolving "who says what" is crucial, which necessitates trimodal (text-audio-visual) binding. Motivated by these challenges, we systematically investigate how this trimodal binding is achieved in AVLLMs. Specifically, we identify emergent symbolic trimodal binding mechanisms in AVLLMs that utilize modality-specific symbolic variables. By encoding auditory and visual components into symbolic variables-capturing temporal utterance sequences and spatial entity coordinates, respectively-the model establishes cross-modal linking within this abstract space. Crucially, we reveal that when trimodal binding fails, the breakdown predominantly stems from misaligned audio-visual connections. To overcome this bottleneck, we introduce an audio-visual prompting method utilizing an off-the-shelf Active Speaker Detection (ASD) model. By simply overlaying visual bounding boxes on active speakers, this training-free approach yields immediate performance gains across four conversation-centric
benchmarks. Moreover, lightweight fine-tuning of fewer than 300 steps on these ASD-prompted-
videos extends these gains to three general AV
benchmarks, suggesting the generalizability of our method.
Read the full abstract ▾