Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science

eess Watch Electrical Engineering and Systems Science

Monday 26 January

Audio and Speech Processing

Noise-Robust AV-ASR Using Visual Features Both in the Whisper Encoder and Decoder

Zhengyang Li, Thomas Graave, Björn Möller et al. (+3 more)

In audiovisual automatic speech recognition (AV-ASR) systems, information fusion of visual features in a pre-trained ASR has been proven as a promising method to improve noise robustness. In this work, based on the prominent Whisper ASR, first, we propose a simple and effective visual fusion method – use of visual features both in encoder and decoder (dual-use) – to learn the audiovisual interactions in the encoder and to weigh modalities in the decoder. Second, we compare visual fusion methods in Whisper models of various sizes. Our proposed dual-use method shows consistent noise robustness improvement, e.g., a 35% relative improvement (WER: 4.41% vs. 6.83%) based on Whisper small, and a 57% relative improvement (WER: 4.07% vs. 9.53%) based on Whisper medium, compared to typical reference middle fusion in babble noise with a signal-to-noise ratio (SNR) of 0dB. Third, we conduct ablation studies examining the impact of various module designs and fusion options. Fine-tuned on 1929 hours of audiovisual data, our dual-use method using Whisper medium achieves 4.08% (MUSAN babble noise) and 4.43% (NoiseX babble noise) average WER across various SNRs, thereby establishing a new state-of-the-art in noisy conditions on the LRS3 AV-ASR benchmark. Our code is at https://github.com/ifnspaml/Dual-Use-AVASR

Read the full abstract

All authors Zhengyang Li, Thomas Graave, Björn Möller, Zehang Wu, Matthias Franz, Tim Fingscheidt

Audio and Speech Processing · Cross-listed from cs.SD

OCR-Enhanced Multimodal ASR Can Read While Listening

Junli Chen, Changli Tang, Yixuan Li et al. (+2 more)

Visual information, such as subtitles in a movie, often helps automatic speech recognition. In this paper, we propose Donut-Whisper, an audio-visual ASR model with dual encoder to leverage visual information to improve speech recognition performance in both English and Chinese. Donut-Whisper combines the advantage of the linear and the Q-Former-based modality alignment structures via a cross-attention module, generating more powerful audio-visual features. Meanwhile, we propose a lightweight knowledge distillation scheme showcasing the potential of using audio-visual models to teach audio-only models to achieve better performance. Moreover, we propose a new multilingual audio-visual speech recognition dataset based on movie clips containing both Chinese and English partitions. As a result, Donut-Whisper achieved significantly better performance on both English and Chinese partition of the dataset compared to both Donut and Whisper large V3 baselines. In particular, an absolute 5.75% WER reduction and a 16.5% absolute CER reduction were achieved on the English and Chinese sets respectively compared to the Whisper ASR baseline.

Read the full abstract

All authors Junli Chen, Changli Tang, Yixuan Li, Guangzhi Sun, Chao Zhang

Signal Processing

Integrated Channel Estimation and Sensing for Near-Field ELAA Systems via Low-Rank Tensor Decomposition

Jionghui Wang, Jun Fang, Hongbin Li et al. (+1 more)

In this paper, we study the problem of uplink channel estimation for near-filed orthogonal frequency division multiplexing (OFDM) systems, where a base station (BS), equipped with an extremely large-scale antenna array (ELAA), serves multiple users over the same time-frequency resource block. A non-orthogonal pilot transmission scheme is considered to accommodate a larger number of users that can be supported by ELAA systems without incurring an excessive amount of training overhead. To facilitate efficient multi-user channel estimation, we express the received signal as a third-order low-rank tensor, which admits a canonical polyadic decomposition (CPD) model for line-of-sight (LoS) scenarios and a block term decomposition (BTD) model for non-line-of-sight (NLoS) scenarios. An alternating least squares (ALS) algorithm and a non-linear least squares (NLS) algorithm are employed to perform CPD and BTD, respectively. Channel parameters are then efficiently extracted from the recovered factor matrices. By exploiting the geometry of the propagation paths in the estimated channel, users' positions can be precisely determined in LoS scenarios. Moreover, our uniqueness analysis shows that the proposed tensor-based joint multi-user channel estimation framework is effective even when the number of pilot symbols is much smaller than the number of users, revealing its potential in training overhead reduction. Simulation results demonstrate that the proposed method achieves markedly higher channel estimation accuracy than compressed sensing (CS)-based approaches.

Read the full abstract

All authors Jionghui Wang, Jun Fang, Hongbin Li, Boyu Ning

Audio and Speech Processing

Residual Learning for Neural Ambisonics Encoders

Thomas Deppisch, Yang Gao, Manan Mittal et al. (+4 more)

Emerging wearable devices such as smartglasses and extended reality headsets demand high-quality spatial audio capture from compact, head-worn microphone arrays. Ambisonics provides a device-agnostic spatial audio representation by mapping array signals to spherical harmonic (SH) coefficients. In practice, however, accurate encoding remains challenging. While traditional linear encoders are signal-independent and robust, they amplify low-frequency noise and suffer from high-frequency spatial aliasing. On the other hand, neural network approaches can outperform linear encoders but they often assume idealized microphones and may perform inconsistently in real-world scenarios. To leverage their complementary strengths, we introduce a residual-learning framework that refines a linear encoder with corrections from a neural network. Using measured array transfer functions from smartglasses, we compare a UNet-based encoder from the literature with a new recurrent attention model. Our analysis reveals that both neural encoders only consistently outperform the linear baseline when integrated within the residual learning framework. In the residual configuration, both neural models achieve consistent and significant improvements across all tested metrics for in-domain data and moderate gains for out-of-domain data. Yet, coherence analysis indicates that all neural encoder configurations continue to struggle with directionally accurate high-frequency encoding.

Read the full abstract

All authors Thomas Deppisch, Yang Gao, Manan Mittal, Benjamin Stahl, Christoph Hold, David Alon, Zamir Ben-Hur

Systems and Control

Convex Chance-Constrained Stochastic Control under Uncertain Specifications with Application to Learning-Based Hybrid Powertrain Control

Teruki Kato, Ryotaro Shima and Kenji Kashima

This paper proposes a strictly convex chance-constrained stochastic control framework that accounts for uncertainty in control specifications such as reference trajectories and operational constraints. Control inputs and risk allocation are jointly optimized under general, possibly non-Gaussian, uncertainties through sufficient deterministic conditions for probabilistic constraint satisfaction. A decreasing risk regularizer ensures existence and full use of the allowable risk. Under the stated regularity assumptions, the joint solution is unique and locally Lipschitz-continuous, supporting stable numerical optimization. The formulation is extended to learning-based exactly linearizable (EL) models through monotone state and input transformations. A hybrid powertrain example compares deterministic control, equal risk allocation, and optimized risk allocation with and without regularization in terms of charge sustaining, constraint violations, closed-loop cost, and solver time.

Read the full abstract

Systems and Control · Cross-listed from cs.AI

A Generative AI-Driven Reliability Layer for Action-Oriented Disaster Resilience

Geunsik Lim

As climate-related hazards intensify, conventional early warning systems (EWS) disseminate alerts rapidly but often fail to trigger timely protective actions, leading to preventable losses and inequities. We introduce Climate RADAR (Risk-Aware, Dynamic, and Action Recommendation system), a generative AI-based reliability layer that reframes disaster communication from alerts delivered to actions executed. It integrates meteorological, hydrological, vulnerability, and social data into a composite risk index and employs guardrail-embedded large language models (LLMs) to deliver personalized recommendations across citizen, volunteer, and municipal interfaces. Evaluation through simulations, user studies, and a municipal pilot shows improved outcomes, including higher protective action execution, reduced response latency, and increased usability and trust. By combining predictive analytics, behavioral science, and responsible AI, Climate RADAR advances people-centered, transparent, and equitable early warning systems, offering practical pathways toward compliance-ready disaster resilience infrastructures.

Read the full abstract

Signal Processing · Cross-listed from cs.IT

A Heterogeneous Massive MIMO Technique for Uniform Service in Cellular Networks

Wei Jiang and Hans D. Schotten

Traditional cellular networks struggle with poor quality of service (QoS) for cell-edge users, while cell-free (CF) systems offer uniform QoS but incur high roll-out costs due to acquiring numerous access point (AP) sites and deploying a large-scale optical fiber network to connect them. This paper proposes a cost-effective heterogeneous massive MIMO architecture that integrates centralized co-located antennas at a cell-center base station with distributed edge APs. By strategically splitting massive antennas between centralized and distributed nodes, the system maintains high user fairness comparable to CF systems but reduces infrastructure costs substantially, by minimizing the required number of AP sites and fronthaul connections. Numerical results demonstrate its superiority in balancing performance and costs compared to cellular and CF systems.

Read the full abstract

Audio and Speech Processing

Noise-Robust Contrastive Learning with an MFCC-Conformer For Coronary Artery Disease Detection

Milan Marocchi, Matthew Fynn and Yue Rong

Cardiovascular diseases (CVD) are the leading cause of death worldwide, with coronary artery disease (CAD) comprising the largest subcategory of CVDs. Recently, there has been increased focus on detecting CAD using phonocardiogram (PCG) signals, with high success in clinical environments with low noise and optimal sensor placement. Multichannel techniques have been found to be more robust to noise; however, achieving robust performance on real-world data remains a challenge. This work utilises a novel multichannel energy-based noisy-segment rejection algorithm, using heart and noise-reference microphones, to discard audio segments with large amounts of nonstationary noise before training a deep learning classifier. This conformer-based classifier takes mel-frequency cepstral coefficients (MFCCs) from multiple channels, further helping improve the model's noise robustness. The proposed method achieved 78.4% accuracy and 78.2% balanced accuracy on 297 subjects, representing improvements of 4.1% and 4.3%, respectively, compared to training without noisy-segment rejection.

Read the full abstract

Systems and Control

Reinforcement Learning with Distributed MPC for Fuel-Efficient Platoon Control with Discrete Gear Transitions

Samuel Mallick, Gianpietro Battocletti, Dimitris Boskos et al. (+2 more)

Cooperative control of groups of autonomous vehicles (AVs), i.e., platoons, is a promising direction to improving the efficiency of autonomous transportation systems. In this context, distributed co-optimization of both vehicle speed and gear position can offer benefits for fuel-efficient driving. To this end, model predictive control (MPC) is a popular approach, optimizing the speed and gear-shift schedule while explicitly considering the vehicles' dynamics over a prediction window. However, optimization over both the vehicles' continuous dynamics and discrete gear positions is computationally intensive, and may require overly long sample times or high-end hardware for real-time implementation. This work proposes a reinforcement learning (RL)-based distributed MPC approach to address this issue. For each vehicle in the platoon, a policy is trained to select and fix the gear positions across the prediction window of a local MPC controller, leaving a significantly simpler continuous optimization problem to be solved as part of a distributed MPC scheme. In order to reduce the computational cost of training and facilitate the scalability of the proposed approach to large platoons, the policies are parameterized such that the emergent multi-agent RL problem can be decoupled into single-agent learning tasks. In addition, a recurrent neural-network (RNN) architecture is proposed for the gear selection policy, such that the learning is scalable even as the number of possible gear-shift schedules grows exponentially with the MPC prediction horizon. In highway-driving simulations, the proposed approach is shown to have a significantly lower computation burden and a comparable performance in terms of fuel-efficient platoon control, with respect to pure MPC-based co-optimization.

Read the full abstract

All authors Samuel Mallick, Gianpietro Battocletti, Dimitris Boskos, Azita Dabiri, Bart De Schutter

Systems and Control

Validation of a Software-Defined 100-Gb/s RDMA Streaming Architecture for Ultrafast Optoacoustic and Ultrasound Imaging

Federico Villani, Christian Vogt, Luca Specht et al. (+5 more)

Optoacoustic (OA) imaging has emerged as a powerful investigation tool, with demonstrated applicability in oncology, neuroscience, and cardiovascular biology. However, its clinical translation is limited with the existing OA systems, which often rely on bulky and expensive acquisition hardware mainly optimized for pulse-echo ultrasound (US) imaging. Despite the fact that OA imaging has different requirements for receive bandwidths and timing synchronization with external laser sources, there is a strong need for unified OA-US imaging platforms, as pulse-echo US remains the standard tool for visualizing soft tissues. To address these challenges, we propose a new data acquisition architecture for ultrafast OA and US imaging that fully covers the requirements for large channel counts, wide bandwidth, and software-defined operation. LtL combines state-of-the-art wideband analog front-ends, a Zynq UltraScale+ MPSoC integrating FPGA fabric with an Application Processing Unit, and a 100 GbE Remote Direct Memory Access (RDMA) backend enabling raw-data streaming at up to 95.6 Gb/s. The architecture avoids local buffers followed by burst transfers, which commonly constrain sustainable frame rate and recording intervals, thus achieving true continuous and sustained streaming of raw data. We validate the core elements of the LtL architecture using a 16-channel demonstration system built from commercial evaluation boards. We further verify the signal chain for up to 256-channel scalability, confirming the wide bandwidth capabilities to support state-of-the-art data transmission speeds.

Read the full abstract

All authors Federico Villani, Christian Vogt, Luca Specht, Jero Schmid, Xiang Liu, Andrea Cossettini, Daniel Razansky, Luca Benini

Audio and Speech Processing

Efficient Rehearsal for Continual Learning in ASR via Singular Value Tuning

Steven Vander Eeckt and Hugo Van hamme

Continual Learning (CL) in Automatic Speech Recognition (ASR) suffers from catastrophic forgetting when adapting to new tasks, domains, or speakers. A common strategy to mitigate this is to store a subset of past data in memory for rehearsal. However, rehearsal-based methods face key limitations: storing data is often costly, infeasible with pre-trained models, or restricted by privacy regulations. Running existing rehearsal-based methods with smaller memory sizes to alleviate these issues usually leads to degraded performance. We propose a rehearsal-based CL method that remains effective even with minimal memory. It operates in two stages: first, fine-tuning on the new task; second, applying Singular Value Decomposition (SVD) to the changes in linear layers and, in a parameter-efficient manner, retraining only gating vectors on the singular values, which control to extent to which updates from the first stage are accepted, using rehearsal. We extensively test and analyze our method on two monolingual and two multilingual benchmarks. Our method reduces forgetting and outperforms state-of-the-art CL approaches for ASR, even when limited to a single utterance per previous task.

Read the full abstract

Audio and Speech Processing · Cross-listed from cs.SD

LLM-ForcedAligner: A Non-Autoregressive and Accurate LLM-Based Forced Aligner for Multilingual and Long-Form Speech

Bingshen Mu, Xian Shi, Xiong Wang et al. (+3 more)

Forced alignment (FA) predicts start and end timestamps for words or characters in speech, but existing methods are language-specific and prone to cumulative temporal shifts. The multilingual speech understanding and long-sequence processing abilities of speech large language models (SLLMs) make them promising for FA in multilingual, crosslingual, and long-form speech settings. However, directly applying the next-token prediction paradigm of SLLMs to FA results in hallucinations and slow inference. To bridge the gap, we propose LLM-ForcedAligner, reformulating FA as a slot-filling paradigm: timestamps are treated as discrete indices, and special timestamp tokens are inserted as slots into the transcript. Conditioned on the speech embeddings and the transcript with slots, the SLLM directly predicts the time indices at slots. During training, causal attention masking with non-shifted input and label sequences allows each slot to predict its own timestamp index based on itself and preceding context, with loss computed only at slot positions. Dynamic slot insertion enables FA at arbitrary positions. Moreover, non-autoregressive inference is supported, avoiding hallucinations and improving speed. Experiments across multilingual, crosslingual, and long-form speech scenarios show that LLM-ForcedAligner achieves a 69%~78% relative reduction in accumulated averaging shift compared with prior methods. Checkpoint and inference code are available at https://github.com/QwenLM/Qwen3-ASR.

Read the full abstract

All authors Bingshen Mu, Xian Shi, Xiong Wang, Hexin Liu, Jin Xu, Lei Xie

Audio and Speech Processing · Cross-listed from cs.SD

VIBEVOICE-ASR Technical Report

Zhiliang Peng, Jianwei Yu, Yaoyao Chang et al. (+21 more)

This report presents VibeVoice-ASR, a general-purpose speech understanding framework built upon VibeVoice, designed to address the persistent challenges of context fragmentation and multi-speaker complexity in long-form audio (e.g., meetings, podcasts) that remain despite recent advancements in short-form speech recognition. Unlike traditional pipelined approaches that rely on audio chunking, VibeVoice-ASRsupports single-pass processing for up to 60 minutes of audio. It unifies Automatic Speech Recognition, Speaker Diarization, and Timestamping into a single end-to-end generation task. In addition, VibeVoice-ASR supports over 50 languages, requires no explicit language setting, and natively handles code-switching within and across utterances. Furthermore, we introduce a prompt-based context injection mechanism that allows users to supply customized conetxt, significantly improving accuracy on domain-specific terminology and polyphonic character disambiguation.

Read the full abstract

All authors Zhiliang Peng, Jianwei Yu, Yaoyao Chang, Zilong Wang, Li Dong, Yingbo Hao, Yujie Tu, Chenyu Yang, Wenhui Wang, Songchen Xu, Yutao Sun, Hangbo Bao, Weijiang Xu, Yi Zhu, Zehua Wang, Ting Song, Yan Xia, Zewen Chi, Shaohan Huang, Liang Wang, Chuang Ding, Shuai Wang, Xie Chen, Furu Wei

Systems and Control · Cross-listed from cs.AI

Success Conditioning as Policy Improvement: The Optimization Problem Solved by Imitating Success

Daniel Russo

A widely used technique for improving policies is success conditioning, in which one collects trajectories, identifies those that achieve a desired outcome, and updates the policy to imitate the actions taken along successful trajectories. This principle appears under many names – rejection sampling with SFT, goal-conditioned RL, Decision Transformers – yet what optimization problem it solves, if any, has remained unclear. We prove that success conditioning exactly solves a trust-region optimization problem, maximizing policy improvement subject to a $\chi^2$ divergence constraint whose radius is determined automatically by the data. This yields an identity: relative policy improvement, the magnitude of policy change, and a quantity we call action-influence – measuring how random variation in action choices affects success rates – are exactly equal at every state. Success conditioning thus emerges as a conservative improvement operator. Exact success conditioning cannot degrade performance or induce dangerous distribution shift, but when it fails, it does so observably, by hardly changing the policy at all. We apply our theory to the common practice of return thresholding, showing this can amplify improvement, but at the cost of potential misalignment with the true objective.

Read the full abstract

Audio and Speech Processing

OneVoice: One Model, Triple Scenarios-Towards Unified Zero-shot Voice Conversion

Zhichao Wang, Tao Li, Wenshuo Ge et al. (+3 more)

Recent progress of voice conversion~(VC) has achieved a new milestone in speaker cloning and linguistic preservation. But the field remains fragmented, relying on specialized models for linguistic-preserving, expressive, and singing scenarios. We propose OneVoice, a unified zero-shot framework capable of handling all three scenarios within a single model. OneVoice is built upon a continuous language model trained with VAE-free next-patch diffusion, ensuring high fidelity and efficient sequence modeling. Its core design for unification lies in a Mixture-of-Experts (MoE) designed to explicitly model shared conversion knowledge and scenario-specific expressivity. Expert selection is coordinated by a dual-path routing mechanism, including shared expert isolation and scenario-aware domain expert assignment with global-local cues. For precise conditioning, scenario-specific prosodic features are fused into each layer via a gated mechanism, allowing adaptive usage of prosody information. Furthermore, to enable the core idea and alleviate the imbalanced issue (abundant speech vs. scarce singing), we adopt a two-stage progressive training that includes foundational pre-training and scenario enhancement with LoRA-based domain experts. Experiments show that OneVoice matches or surpasses specialized models across all three scenarios, while verifying flexible control over scenarios and offering a fast decoding version as few as 2 steps. Audio samples are available on demo page.

Read the full abstract

All authors Zhichao Wang, Tao Li, Wenshuo Ge, Zihao Cui, Shilei Zhang, Junlan Feng

Audio and Speech Processing · Cross-listed from cs.SD

From Human Speech to Ocean Signals: Transferring Speech Large Models for Underwater Acoustic Target Recognition

Mengcheng Huang, Xue Zhou, Chen Xu et al. (+1 more)

Underwater acoustic target recognition (UATR) plays a vital role in marine applications but remains challenging due to limited labeled data and the complexity of ocean environments. This paper explores a central question: can speech large models (SLMs), trained on massive human speech corpora, be effectively transferred to underwater acoustics? To investigate this, we propose UATR-SLM, a simple framework that reuses the speech feature pipeline, adapts the SLM as an acoustic encoder, and adds a lightweight classifier.Experiments on the DeepShip and ShipsEar benchmarks show that UATR-SLM achieves over 99% in-domain accuracy, maintains strong robustness across variable signal lengths, and reaches up to 96.67% accuracy in cross-domain evaluation. These results highlight the strong transferability of SLMs to UATR, establishing a promising paradigm for leveraging speech foundation models in underwater acoustics.

Read the full abstract

All authors Mengcheng Huang, Xue Zhou, Chen Xu, Dapeng Man

Signal Processing · Cross-listed from physics.optics

Physics-Integrated Inference for Signal Recovery in Non-Gaussian Regimes

Mohamed A. Mousa, Leif Bauer, Ziyi Yang et al. (+3 more)

High-performance room-temperature sensing is often limited by non-stationary $1/f$ fluctuations and non-Gaussian stochasticity. In spintronic devices, thermally activated Néel switching creates heavy-tailed noise that masks weak signals, defeating linear filters optimized for Gaussian statistics. Here, we introduce a physics-integrated inference framework that decouples signal morphology from stochastic transients using a hierarchical 1D CNN-GRU topology. By learning the temporal signatures of Néel relaxation, this architecture reduces the Noise Equivalent Differential Temperature (NEDT) of spintronic Poisson bolometers by a factor of six (233.78 mK to 40.44 mK), effectively elevating room-temperature sensitivity toward cryogenic limits. We demonstrate the framework's universality across the electromagnetic and biological spectrum, achieving a 9-fold error suppression in Radar tracking, a 40% uncertainty reduction in LiDAR, and a 15.56 dB SNR enhancement in ECG. This hardware-inference coupling recovers deterministic signals from fluctuation-dominated regimes, enabling near-ideal detection limits in noisy edge environments.

Read the full abstract

All authors Mohamed A. Mousa, Leif Bauer, Ziyi Yang, Utkarsh Singh, Angshuman Deka, Zubin Jacob

Sunday 25 January

Systems and Control

Energy-Aware Reinforcement Learning for Robotic Manipulation of Articulated Components in Infrastructure Operation and Maintenance

Xiaowen Tao, Yinuo Wang, Haitao Ding et al. (+2 more)

With the growth of intelligent civil infrastructure and smart cities, operation and maintenance (O&M) increasingly requires safe, efficient, and energy-conscious robotic manipulation of articulated components, including access doors, service drawers, and pipeline valves. However, existing robotic approaches either focus primarily on grasping or target object-specific articulated manipulation, and they rarely incorporate explicit actuation energy into multi-objective optimisation, which limits their scalability and suitability for long-term deployment in real O&M settings. Therefore, this paper proposes an articulation-agnostic and energy-aware reinforcement learning framework for robotic manipulation in intelligent infrastructure O&M. The method combines part-guided 3D perception, weighted point sampling, and PointNet-based encoding to obtain a compact geometric representation that generalises across heterogeneous articulated objects. Manipulation is formulated as a Constrained Markov Decision Process (CMDP), in which actuation energy is explicitly modelled and regulated via a Lagrangian-based constrained Soft Actor-Critic scheme. The policy is trained end-to-end under this CMDP formulation, enabling effective articulated-object operation while satisfying a long-horizon energy budget. Experiments on representative O&M tasks demonstrate 16%-30% reductions in energy consumption, 16%-32% fewer steps to success, and consistently high success rates, indicating a scalable and sustainable solution for infrastructure O&M manipulation.

Read the full abstract

All authors Xiaowen Tao, Yinuo Wang, Haitao Ding, Yuanyang Qi, Ziyu Song

Signal Processing · Cross-listed from cs.LG

Scaled Dot-Product Attention implements projection of inputs onto a common surface

Terence D Sanger

Scaled dot-product attention (SDPA) is a fundamental component responsible for the success of large-language models and other nonlinear signal processing applications. The rationale for SDPA has been based upon "query, key, value" concepts borrowed from database theory, but these concepts are difficult to reconcile with standard methods in mathematical signal processing. We show that SDPA can be rewritten in a different but mathematically equivalent form as a projection of the input vectors onto a common surface determined by the inputs themselves. Therefore SDPA discovers nonlinear dependencies in the input that are time-dependent and context-dependent. The rewritten form of SDPA permits increased speed of both feedforward and learning algorithms, but more importantly suggests potential extensions. In the context of language, we re-interpret the role of SDPA as finding a time-dependent contextual meaning determined by the surface on which the set of input vectors lies. Input token embeddings are then modified by the local context surface. This interpretation differs substantially from the concept of "self-attention", and provides a strong justification for the use of SDPA for time-series data with time-varying local nonlinear dependencies.

Read the full abstract

Audio and Speech Processing

RIR-Mega-Speech: A Reverberant Speech Corpus with Comprehensive Acoustic Metadata and Reproducible Evaluation

Mandip Goswami

Despite decades of research on reverberant speech, comparing methods remains difficult because most corpora lack per-file acoustic annotations or provide limited documentation for reproduction. We present RIR-Mega-Speech, a corpus of approximately 117.5 hours created by convolving LibriSpeech utterances with roughly 5,000 simulated room impulse responses from the RIR-Mega collection. Every file includes RT60, direct-to-reverberant ratio (DRR), and clarity index ($C_{50}$) computed from the source RIR using clearly defined, reproducible procedures. We also provide scripts to rebuild the dataset and reproduce all evaluation results. Using Whisper small on 1,500 paired utterances, we measure 5.20% WER (95% CI: 4.69–5.78) on clean speech and 7.70% (7.04–8.35) on reverberant versions, corresponding to a paired increase of 2.50 percentage points (2.06–2.98). This represents a 48% relative degradation. WER increases monotonically with RT60 and decreases with DRR, consistent with prior perceptual studies. While the core finding that reverberation harms recognition is well established, we aim to provide the community with a standardized resource where acoustic conditions are transparent and results can be verified independently. The repository includes one-command rebuild instructions for both Windows and Linux environments.

Read the full abstract

Signal Processing · Cross-listed from cs.LG

How Much Temporal Modeling is Enough? A Systematic Study of Hybrid CNN-RNN Architectures for Multi-Label ECG Classification

Alireza Jafari and Fatemeh Jafari

Accurate multi-label classification of electrocardiogram (ECG) signals remains challenging due to the coexistence of multiple cardiac conditions, pronounced class imbalance, and long-range temporal dependencies in multi-lead recordings. Although recent studies increasingly rely on deep and stacked recurrent architectures, the necessity and clinical justification of such architectural complexity have not been rigorously examined. In this work, we perform a systematic comparative evaluation of convolutional neural networks (CNNs) combined with multiple recurrent configurations, including LSTM, GRU, Bidirectional LSTM (BiLSTM), and their stacked variants, for multi-label ECG classification on the PTB-XL dataset comprising 23 diagnostic categories. The CNN component serves as a morphology-driven baseline, while recurrent layers are progressively integrated to assess their contribution to temporal modeling and generalization performance. Experimental results indicate that a CNN integrated with a single BiLSTM layer achieves the most favorable trade-off between predictive performance and model complexity. This configuration attains superior Hamming loss (0.0338), macro-AUPRC (0.4715), micro-F1 score (0.6979), and subset accuracy (0.5723) compared with deeper recurrent combinations. Although stacked recurrent models occasionally improve recall for specific rare classes, our results provide empirical evidence that increasing recurrent depth yields diminishing returns and may degrade generalization due to reduced precision and overfitting. These findings suggest that architectural alignment with the intrinsic temporal structure of ECG signals, rather than increased recurrent depth, is a key determinant of robust performance and clinically relevant deployment.

Read the full abstract

Image and Video Processing

OCTA-Based Biomarker Characterization in nAMD

MAria Simona Tivadar, Ioana Damian, Adrian Groza et al. (+1 more)

We aim to enhance ophthalmologists' decision-making when diagnosing the Neovascular Age-Related Macular Degeneration (nAMD). We developed three tools to analyze Optical Coherence Tomography Angiography images: (1) extracting biomarkers such as mCNV area and vessel density using image processing; (2) generating a 3D visualization of the neovascularization for a better view of the affected regions; and (3) applying an ensemble of three white box machine learning algorithms (decision tree, support vector machines and DL-Learner) for nAMD diagnosis. The learned expressions reached 100% accuracy for the training data and 68% accuracy in testing. The main advantage is that all the learned models white-box, which ensures explainability and transparency, allowing clinicians to better understand the decision-making process.

Read the full abstract

All authors MAria Simona Tivadar, Ioana Damian, Adrian Groza, Simona Delia Nicoara

Audio and Speech Processing

SpatialEmb: Extract and Encode Spatial Information for 1-Stage Multi-channel Multi-speaker ASR on Arbitrary Microphone Arrays

Yiwen Shao, Yong Xu, Sanjeev Khudanpur et al. (+1 more)

Spatial information is a critical clue for multi-channel multi-speaker target speech recognition. Most state-of-the-art multi-channel Automatic Speech Recognition (ASR) systems extract spatial features only during the speech separation stage, followed by standard single-channel ASR on the separated speech. This approach results in an inefficient, lengthy pipeline and sub-optimal ASR performance due to the accumulated errors from preprocessing modules. Furthermore, most spatial feature extraction methods depend on the knowledge of speaker positions and microphone topology, making the systems reliant on specific settings and challenging to adapt to new equipment. In this work, we propose a solution to these issues with a lightweight embedding module named SpatialEmb, which extracts and encodes spatial information directly for the ASR model, supporting both fixed and arbitrary microphone topology. We conduct comprehensive experiments on AliMeeting, a real meeting corpus, to determine the optimal model design for SpatialEmb in terms of both performance and efficiency. Our best model trained with 105 hours Train-Ali-far achieves 17.04% and 20.32% character error rates (CER) on the Eval and Test sets, establishing a new state-of-the-art result with the same training data.

Read the full abstract

All authors Yiwen Shao, Yong Xu, Sanjeev Khudanpur, Dong Yu

Image and Video Processing

Dominant Sets Based Band Selection in Hyperspectral Imagery

Onur Haliloğlu, Ufuk Sakarya, B. Uğur Töreyin et al. (+1 more)

Hyperspectral imagery is composed of huge amount of data which creates significant transmission latencies for communication systems. It is vital to decrease the huge data size before transmitting the Hyperspectral imagery. Besides, large data size leads to processing problems, especially in practical applications. Moreover, due to the lack of sufficient training samples, Hughes phenomena occur with huge amount of data. Feature selection can be used in order to get rid of huge data problems. In this paper, a band selection framework is introduced to reduce the data size and to find out the most proper spectral bands for a specific application. The method is based on finding "dominant sets" in hyperspectral data, so that spectral bands are clustered. From each cluster, the band that reflects the cluster behavior the most is selected to form the most valuable band set in the spectra for a specific application. The proposed feature selection method has low computational complexity since it performs on a small size of data when realizing the feature selection. The aim of the study is to find out a general framework that can define required bands for classification without requiring to perform on the whole data set. Results on Pavia and Salinas datasets show that the proposed framework performs better than the state-of-the-art feature selection methods in terms of classification accuracy.

Read the full abstract

All authors Onur Haliloğlu, Ufuk Sakarya, B. Uğur Töreyin, Orhan Gazi

Signal Processing · Cross-listed from nlin.AO

Neural-Inspired Multi-Agent Molecular Communication Networks for Collective Intelligence

Boran A. Kilic and Ozgur B. Akan

Molecular Communication (MC) is a pivotal enabler for the Internet of Bio-Nano Things (IoBNT). However, current research often relies on super-capable individual agents with complex transceiver architectures that defy the energy and processing constraints of realistic nanomachines. This paper proposes a paradigm shift towards collective intelligence, inspired by the cortical networks of the biological brain. We introduce a decentralized network architecture where simple nanomachines interact via a diffusive medium using a threshold-based firing mechanism modeled by Greenberg-Hastings (GH) cellular automata. We derive fixed-point equations for steady-state populations via mean-field analysis and validate them against stochastic simulations. We demonstrate that the network undergoes a second-order phase transition at a specific activation threshold. Crucially, we show that both pairwise and collective mutual information peak exactly at this critical transition point, confirming that the system maximizes information propagation and processing capacity at the edge of chaos.

Read the full abstract

Audio and Speech Processing

AmbER$^2$: Dual Ambiguity-Aware Emotion Recognition Applied to Speech and Text

Jingyao Wu, Grace Lin, Yinuo Song et al. (+1 more)

Emotion recognition is inherently ambiguous, with uncertainty arising both from rater disagreement and from discrepancies across modalities such as speech and text. There is growing interest in modeling rater ambiguity using label distributions. However, modality ambiguity remains underexplored, and multimodal approaches often rely on simple feature fusion without explicitly addressing conflicts between modalities. In this work, we propose AmbER$^2$, a dual ambiguity-aware framework that simultaneously models rater-level and modality-level ambiguity through a teacher-student architecture with a distribution-wise training objective. Evaluations on IEMOCAP and MSP-Podcast show that AmbER$^2$ consistently improves distributional fidelity over conventional cross-entropy baselines and achieves performance competitive with, or superior to, recent state-of-the-art systems. For example, on IEMOCAP, AmbER$^2$ achieves relative improvements of 20.3% on Bhattacharyya coefficient (0.83 vs. 0.69), 13.6% on R$^2$ (0.67 vs. 0.59), 3.8% on accuracy (0.683 vs. 0.658), and 4.5% on F1 (0.675 vs. 0.646). Further analysis across ambiguity levels shows that explicitly modeling ambiguity is particularly beneficial for highly uncertain samples. These findings highlight the importance of jointly addressing rater and modality ambiguity when building robust emotion recognition systems.

Read the full abstract

All authors Jingyao Wu, Grace Lin, Yinuo Song, Rosalind Picard

Systems and Control · Cross-listed from stat.ML

A Cherry-Picking Approach to Large Load Shaping for More Effective Carbon Reduction

Bokan Chen, Raiden Hasegawa, Adriaan Hilbers et al. (+7 more)

Shaping multi-megawatt loads, such as data centers, impacts generator dispatch on the electric grid, which in turn affects system CO2 emissions and energy cost. Substantiating the effectiveness of prevalent load shaping strategies, such as those based on grid-level average carbon intensity, locational marginal price, or marginal emissions, is challenging due to the lack of detailed counterfactual data required for accurate attribution. This study uses a series of calibrated granular ERCOT day-ahead direct current optimal power flow (DC-OPF) simulations for counterfactual analysis of a broad set of load shaping strategies on grid CO2 emissions and cost of electricity. In terms of annual grid level CO2 emissions reductions, LMP-based shaping outperforms other common strategies, but can be significantly improved upon. Examining the performance of practicable strategies under different grid conditions motivates a more effective load shaping approach: one that "cherry-picks" a daily strategy based on observable grid signals and historical data. The cherry-picking approach to power load shaping is applicable to any large flexible consumer on the electricity grid, such as data centers, distributed energy resources and Virtual Power Plants (VPPs).

Read the full abstract

All authors Bokan Chen, Raiden Hasegawa, Adriaan Hilbers, Ross Koningstein, Ana Radovanović, Utkarsh Shah, Gabriela Volpato, Mohamed Ahmed, Tim Cary, Rod Frowd

Systems and Control

Data-driven nonparametric Li-ion battery ageing model aiming at learning from real operation data – Part B: Cycling operation

Lucu M., Martinez-Laserna E., Gandiaga I. et al. (+4 more)

Conventional Li-ion battery ageing models, such as electrochemical, semi-empirical and empirical models, require a significant amount of time and experimental resources to provide accurate predictions under realistic operating conditions. At the same time, there is significant interest from industry in the introduction of new data collection telemetry technology. This implies the forthcoming availability of a significant amount of real-world battery operation data. In this context, the development of ageing models able to learn from in-field battery operation data is an interesting solution to mitigate the need for exhaustive laboratory testing. In a series of two papers, a data-driven ageing model is developed for Li-ion batteries under the Gaussian Process framework. A special emphasis is placed on illustrating the ability of the Gaussian Process model to learn from new data observations, providing more accurate and confident predictions, and extending the operating window of the model. The first paper of the series focussed on the systematic modelling and experimental verification of cell degradation through calendar ageing. Conversantly, this second paper addresses the same research challenge when the cell is electrically cycled. A specific covariance function is composed, tailored for use in a battery ageing application. Over an extensive dataset involving 124 cells tested during more than three years, different training possibilities are contemplated in order to quantify the minimal number of laboratory tests required for the design of an accurate ageing model. A model trained with only 26 tested cells achieves an overall mean-absolute-error of 1.04% in the capacity curve prediction, after being validated under a broad window of both dynamic and static cycling temperatures, Depth-of-Discharge, middle-SOC, charging and discharging C-rates.

Read the full abstract

All authors Lucu M., Martinez-Laserna E., Gandiaga I., Liu K., Camblong H., Widanage W.D., Marco J

Systems and Control · Cross-listed from math.OC

On maximum hands-off restricted hybrid control for discrete-time switched linear systems

Darsana U and Atreyee Kundu

This paper deals with design of maximum hands-off hybrid control sequences for discrete-time switched linear systems. It is a sparsest combination of a discrete control sequence (i.e. the switching sequence) and a continuous control sequence, both satisfying pre-specified restrictions on the admissible actions, that steers a given initial state of the switched system to the origin of the state-space in a pre-specified duration of time. Given the subsystems dynamics, the sets of admissible continuous and discrete control, the initial state and the time horizon, we present a new algorithm that, under certain conditions on the subsystems dynamics and the admissible control, designs maximum hands-off hybrid control sequences for the resulting switched system. The key apparatuses for our analysis are graph theory and linear algebra. Numerical examples are presented to demonstrate our results.

Read the full abstract

Systems and Control

Data-driven nonparametric Li-ion battery ageing model aiming at learning from real operation data – Part A: Storage operation

Lucu M., Martinez-Laserna E., Gandiaga I. et al. (+4 more)

Conventional Li-ion battery ageing models, such as electrochemical, semi-empirical and empirical models, require a significant amount of time and experimental resources to provide accurate predictions under realistic operating conditions. At the same time, there is significant interest from industry in the introduction of new data collection telemetry technology. This implies the forthcoming availability of a significant amount of real-world battery operation data. In this context, the development of ageing models able to learn from in-field battery operation data is an interesting solution to mitigate the need for exhaustive laboratory testing. In a series of two papers, a data-driven ageing model is developed for Li-ion batteries under the Gaussian Process framework. A special emphasis is placed on illustrating the ability of the Gaussian Process model to learn from new data observations, providing more accurate and confident predictions, and extending the operating window of the model. This first paper focusses on the systematic modelling and experimental verification of cell degradation through calendar ageing. A specific covariance function is composed, tailored for use in a battery ageing application. Over an extensive dataset involving 32 cells tested during more than three years, different training possibilities are contemplated in order to quantify the minimal number of laboratory tests required for the design of an accurate ageing model. A model trained with only 18 tested cells achieves an overall mean-absolute-error of 0.53% in the capacity curves prediction, after being validated under a broad window of both dynamic and static temperature and SOC storage conditions.

Read the full abstract

All authors Lucu M., Martinez-Laserna E., Gandiaga I., Liu K., Camblong H., Widanage W.D., Marco J