Peer Revenue Play credits. Real papers.

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science

eess Watch Electrical Engineering and Systems Science (sign in to use)

Friday 24 April

Signal Processing

A Survey on the Symbiosis of Artificial Intelligence and Terahertz Networks

The Terahertz (THz) band (0.1-10 THz) has emerged as a critical frontier for future communication systems, offering ultra-wide bandwidths that enable Terabits-per-second (Tbps) wireless links and high-precision sensing and imaging. However, practical deployment of THz systems is hindered by unique challenges, including intricate channel characteristics, high-dimensional and large-scale optimization problems, and highly dynamic network environments. Artificial Intelligence (AI) serves as a transformative enabler to address these challenges, providing robust capabilities for precise modeling, advanced signal processing, complex optimization, real-time decision-making, and prediction, among others. Reciprocally, the unprecedented bandwidth and high-resolution sensing capabilities of THz networks provide a promising physical infrastructure for AI, facilitating training, inference, and data collection. This survey presents a systematic and comprehensive overview of AI-driven solutions across the entire THz communication network and the symbiosis of AI and THz networks. To begin with, a foundational overview of AI technologies tailored for wireless communications is presented. Subsequently, AI-based innovations are investigated, spanning from hardware design, channel modeling, physical layer optimization, up to higher-layer network protocols and advanced THz services, including mobile edge computing and sensing-empowered applications. In parallel, the capacity of THz networks to serve AI is examined, underscoring a profound paradigm shift towards a mutual symbiosis where AI and THz co-evolve and empower each other. Finally, by synthesizing these state-of-the-art advancements and identifying open research directions, this survey highlights the potential of AI in copilot with development of THz communication systems.
Read the full abstract

All authors Chong Han, Jingting Jiang, Meixia Tao, Wenjun Zhang

Signal Processing

Time-Localized Parametric Decomposition of Respiratory Airflow for Sub-Breath Analysis

Respiratory airflow signals provide critical insight into breathing mechanics, yet conventional analysis methods remain limited in their ability to characterize the internal structure of individual breaths. Traditional approaches treat airflow as a quasi-periodic signal and rely on global descriptors such as tidal volume or peak flow, obscuring sub-breath events that reflect neuromuscular coordination and compensatory breathing strategies. This study introduces a parametric framework for decomposing inspiratory airflow into a small number of time-localized components with explicit amplitude, onset time, and duration parameters. Unlike spectral or data-adaptive methods, the proposed approach employs physiologically grounded basis functions, Half-Sine, Gaussian, and Beta, to represent intrabreath waveform morphology through constrained nonlinear optimization. Evaluation across 8,276 breaths demonstrates high reconstruction accuracy (mean squared error $<$ 0.001 for four-component models) and robust parameter precision under moderate noise. Component-derived features describing sub-breath timing and coordination improved classification of cognitive fatigue states arising from cognitive-respiratory competition by up to 30.7% in Matthews correlation coefficient compared with classical respiratory metrics. These results establish that modeling airflow as a sum of parameterized, time-localized primitives provides an interpretable and precise foundation for quantifying intrabreath organization, compensatory breathing dynamics, and respiratory motor control adaptation under cognitive-respiratory dual-task demands.
Read the full abstract

Signal Processing

Mobility Aware Power Control for VCSEL Based Indoor OWC

Optical wireless communication (OWC) is a promising technology for supporting data intensive services in indoor environments due to its large unregulated spectrum, high spatial reuse, and potential for multigigabit data rates. In particular, vertical cavity surface emitting laser (VCSEL) based systems enable highly directional transmission, allowing efficient spatial separation of users and improved link performance. However, the use of narrow optical beams also makes system performance highly sensitive to user mobility and device orientation, as movement directly affects beam alignment and optical channel gain. Consequently, power allocation strategies that ignore mobility dynamics often provision excess optical power to maintain reliable connectivity, resulting in inefficient energy use. In this work, a power control framework for dynamic indoor OWC networks that explicitly accounts for mobility driven channel variation is developed. It uses a hybrid Gauss Markov and learning based approach that captures both user movement continuity and behaviour driven orientation changes. The mobility states are then used to guide power allocation decisions. Simulation results show that incorporating mobility aware channel prediction enables more accurate power allocation, and improves energy efficiency compared with conventional power control schemes in dynamic indoor environments.
Read the full abstract

All authors Walter Zibusiso Ncube, Ahmad Adnan Qidan, Taisir El-Gorashi, Jaafar M. H. Elmirghani

Systems and Control · Cross-listed from math.OC

Compositional Online Learning for Multi-Objective System Co-Design

Many engineered systems must balance competing objectives, such as performance and safety, cost and reliability, or efficiency and sustainability, and are naturally modeled as compositions of interacting subsystems. We study online multi-objective decision-making in monotone co-design, where functionalities and resources are partially ordered, and the goal is to identify the target-feasible antichain of non-dominated trade-offs using few expensive evaluations. We introduce optimistic evaluators: history-dependent bounds on functionality and resource mappings that enable safe elimination of implementations before full evaluation. Based on these evaluators, we develop an elimination-based rejection-sampling algorithm, prove its soundness, and show that the admissible region shrinks monotonically as information accumulates. We instantiate the framework under monotonicity, Lipschitz continuity, and linear-parametric structure. For compositional co-design problems modeled by multigraphs, we show how local optimistic certificates propagate through the tractable remainder of the graph to yield system-level optimistic feasibility and resource bounds. Experiments on multi-robot fleet design, intermodal mobility systems, and synthetic monotone and Lipschitz benchmarks show substantial sample-efficiency gains over uniform sampling, Bayesian optimization, and multi-objective evolutionary algorithms.
Read the full abstract

Signal Processing · Cross-listed from math.RA

On the rank of quaternion Hankel matrices

This paper discusses the left and right ranks of quaternion matrices with Hankel structure. While they are in general different for arbitrary quaternion matrices, we show that the left and right ranks of quaternion Hankel matrices are equal. Moreover, we establish the relation between Hankel matrices and the existence of linear recurrence relations with quaternion coefficients and discuss some practical implications for computational methods relying on low-rank properties of quaternion Hankel matrices.
Read the full abstract

Image and Video Processing

Useful nonrobust features are ubiquitous in biomedical images

We study whether deep networks for medical imaging learn useful nonrobust features - predictive input patterns that are not human interpretable and highly susceptible to small adversarial perturbations - and how these features impact test performance. We show that models trained only on nonrobust features achieve well above chance accuracy across five MedMNIST classification tasks, confirming their predictive value in-distribution. Conversely, adversarially trained models that primarily rely on robust features sacrifice in-distribution accuracy but yield markedly better performance under controlled distribution shifts (MedMNIST-C). Overall, nonrobust features boost standard accuracy yet degrade out-of-distribution performance, revealing a practical robustness-accuracy trade-off in medical imaging classification tasks that should be tailored to the requirements of the deployment setting.
Read the full abstract

All authors Coenraad Mouton, Randle Rabe, Niklas C. Koser, Nicolai Krekiehn, Christopher Hansen, Jan-Bernd Hövener, Claus-C. Glüer

Image and Video Processing

Are Natural-Domain Foundation Models Effective for Accelerated Cardiac MRI Reconstruction?

The emergence of large-scale pretrained foundation models has transformed computer vision, enabling strong performance across diverse downstream tasks. However, their potential for physics-based inverse problems, such as accelerated cardiac MRI reconstruction, remains largely underexplored. In this work, we investigate whether natural-domain foundation models can serve as effective image priors for accelerated cardiac MRI reconstruction, and compare the performance obtained against domain-specific counterparts such as BiomedCLIP. We propose an unrolled reconstruction framework that incorporates pretrained, frozen visual encoders, such as CLIP, DINOv2, and BiomedCLIP, within each cascade to guide the reconstruction process. Through extensive experiments, we show that while task-specific state-of-the-art reconstruction models such as E2E-VarNet achieve superior performance in standard in-distribution settings, foundation-model-based approaches remain competitive. More importantly, in challenging cross-domain scenarios, where models are trained on cardiac MRI and evaluated on anatomically distinct knee and brain datasets–foundation models exhibit improved robustness, particularly under high acceleration factors and limited low-frequency sampling. We further observe that natural-image-pretrained models, such as CLIP, learn highly transferable structural representations, while domain-specific pretraining (BiomedCLIP) provides modest additional gains in more ill-posed regimes. Overall, our results suggest that pretrained foundation models offer a promising source of transferable priors, enabling improved robustness and generalization in accelerated MRI reconstruction.
Read the full abstract

All authors Anam Hashmi, Mayug Maniparambil, Julia Dietlmeier, Kathleen M. Curran, Noel E. O'Connor

Image and Video Processing

MTT-Bench: Predicting Social Dominance in Mice via Multimodal Large Language Models

Understanding social dominance in animal behavior is critical for neuroscience and behavioral studies. In this work, we explore the capability of Multimodal Large Language Models(MLLMs) to analyze raw behavioral video of mice and predict their dominance hierarchy. We introduce MTT-Bench, a novel benchmark comprising annotated videos of pairwise mouse interactions for Mouse Tube Test analysis. Building on existing MLLM architectures, we fine-tune these models to perform zero-shot inference on unseen behavioral sequences, predicting social dominance without explicit labels during testing. Our framework demonstrates promising results, showing high agreement with tube test rankings. This work opens a new direction for applying foundation models to ethology and social behavior analysis, without the need to design domain-specific models.
Read the full abstract

Image and Video Processing · Cross-listed from cs.CV

Improving Driver Drowsiness Detection via Personalized EAR/MAR Thresholds and CNN-Based Classification

Driver drowsiness is a major cause of traffic accidents worldwide, posing a serious threat to public safety. Vision-based driver monitoring systems often rely on fixed Eye Aspect Ratio (EAR) and Mouth Aspect Ratio (MAR) thresholds; however, such fixed values frequently fail to generalize across individuals due to variations in facial structure, illumination, and driving conditions. This paper proposes a personalized driver drowsiness detection system that monitors eyelid movements, head position, and yawning behavior in real time and provides warnings when signs of fatigue are detected. The system employs driver-specific EAR and MAR thresholds, calibrated before driving, to improve classical metric-based detection. In addition, deep learning-based Convolutional Neural Network (CNN) models are integrated to enhance accuracy in challenging scenarios. The system is evaluated using publicly available datasets as well as a custom dataset collected under diverse lighting conditions, head poses, and user characteristics. Experimental results show that personalized thresholding improves detection accuracy by 2-3% compared to fixed thresholds, while CNN-based classification achieves 99.1% accuracy for eye state detection and 98.8% for yawning detection, demonstrating the effectiveness of combining classical metrics with deep learning for robust real-time driver monitoring.
Read the full abstract

All authors Gökdeniz Ersoy, Mehmet Alper Tatar, Eray Tonbul, Serap K{\i}rb{\i}z

Signal Processing

Time-Frequency Pilot Sequence Design and LoS Delay-Doppler Estimation

We present a novel framework for line-of-sight (LoS) delay-Doppler (DD) estimation in dense scattering propagation environments. We present two time-frequency (TF) domain pilot sequences inspired by the Zadoff-Chu sequence that exhibit desirable autocorrelation properties. Further, we present a twisted convolution-based approach for LoS DD estimation directly from the TF-domain received signal, avoiding an additional TF to DD transformation, which is commonly found in literature. Numerical results from simulations demonstrate that the proposed framework significantly outperforms traditional single-carrier Zadoff-Chu sequences in both delay and Doppler estimation over a wide range of Rician fading factor and SNR values.
Read the full abstract

Signal Processing

The manifold of unitary and symmetric matrices: characterization, Riemannian optimization and application to BD-RIS design

This paper proposes and analyzes Riemannian optimization algorithms on the manifold of unitary and symmetric matrices, denoted ${\cal {U}}_s$, which naturally models the scattering matrices of passive and reciprocal devices such as beyond-diagonal reconfigurable intelligent surfaces (BD-RISs). Despite its relevance, the geometry of ${\cal {U}}_s$ has remained largely unexplored, and existing BD-RIS optimization methods either ignore the symmetry constraint or rely on costly Takagi-based parameterizations. We first provide a rigorous geometric characterization of ${\cal {U}}_s$, deriving its tangent space, a simple retraction, and closed-form expressions for geodesics. Building on these results, we develop two Riemannian manifold optimization (MO) algorithms tailored to ${\cal {U}}_s$: a line-search (LS) based scheme and a phase-optimization (PO) update along geodesics. We then apply the proposed framework to BD-RIS-assisted multiple-input multiple-output (MIMO) links, addressing sum-gain maximization, rate maximization, and minimum mean-square error problems, where they outperform existing approaches. Furthermore, we show that when the number of BD-RIS elements exceeds the total number of antennas, the optimal scattering matrix is low-rank, which motivates and enables efficient low-rank variants of the proposed algorithms.
Read the full abstract

All authors Ignacio Santamaria, Carlos Beltrán, Eduard Jorswieck, Mohammad Soleymani, Jesus Gutiérrez

Audio and Speech Processing

DM-ASR: Diarization-aware Multi-speaker ASR with Large Language Models

Multi-speaker automatic speech recognition (ASR) aims to transcribe conversational speech involving multiple speakers, requiring the model to capture not only what was said, but also who said it and sometimes when it was spoken. Recent Speech-LLM approaches have shown the potential of unified modeling for this task, but jointly learning speaker attribution, temporal structure, and lexical recognition remains difficult and data-intensive. At the current stage, leveraging reliable speaker diarization as an explicit structural prior provides a practical and efficient way to simplify this task. To effectively exploit such priors, we propose DM-ASR, a diarization-aware multi-speaker ASR framework that reformulates the task as a multi-turn dialogue generation process. Given an audio chunk and diarization results, DM-ASR decomposes transcription into a sequence of speaker- and time-conditioned queries, each corresponding to one speaker in one time segment. This formulation converts multi-speaker recognition into a series of structured sub-tasks, explicitly decoupling speaker-temporal structure from linguistic content and enabling effective integration of diarization cues with the reasoning capability of large language models. We further introduce an optional word-level timestamp prediction mechanism that interleaves word and timestamp tokens, yielding richer structured outputs and better transcription quality. Our analysis shows that diarization systems provide more reliable speaker identities and segment-level boundaries, while LLMs excel at modeling linguistic content and long-range dependencies, demonstrating their complementary strengths. Experiments on Mandarin and English benchmarks show that the proposed approach achieves strong performance with relatively small models and training data, while remaining competitive with or outperforming existing unified approaches.
Read the full abstract

All authors Li Li, Ming Cheng, Weixin Zhu, Yannan Wang, Juan Liu, Ming Li

Signal Processing · Cross-listed from cs.IT

Multi-User ISAC with Heterogeneous Unknown Parameters: Optimal Beamforming based on Distribution Information

This paper studies an integrated sensing and communication (ISAC) system where a multi-antenna base station (BS) communicates with multiple single-antenna users in the downlink and senses the unknown and random angle information of a target based on its prior distribution information and the received echo signals. We focus on a challenging scenario with heterogeneous unknown parameters where the target's reflection coefficient is also unknown with no prior information. We consider a general transmit beamforming structure with both communication beams and dedicated sensing beams, where the communication users can cancel the interference caused by the pre-determined sensing signals. By adopting the periodic posterior Cramer-Rao bound (PCRB) to quantify a lower bound of the mean-cyclic error (MCE) for sensing the periodic angle parameter, we optimize the transmit beamforming to minimize the periodic PCRB, subject to individual communication user rate constraints, which is a non-convex problem. By leveraging the semi-definite relaxation (SDR) technique and Lagrange duality theory, we derive the optimal solution and prove that at most one dedicated sensing beam is needed. Numerical results validate our analysis and effectiveness of the proposed beamforming design.
Read the full abstract

Image and Video Processing

Selective Depthwise Separable Convolution for Lightweight Joint Source-Channel Coding in Wireless Image Transmission

Depthwise separable convolutional (DSConv) layers have been successfully applied to deep learning (DL)-based joint source-channel coding (JSCC) schemes to reduce computational complexity. However, a systematic investigation of the layerwise and ratio-wise replacement of standard convolutional (Conv) layers with DSConv layers in JSCC systems for wireless image transmission remains largely unexplored. In this letter, we propose a configurable lightweight JSCC framework that incorporates a selective replacement strategy, enabling flexible Conv-to-DSConv replacement at different replacement ratios and positions. By varying the replacement ratio, we obtain models with different computational complexities and analyze their impact on reconstruction performance. Furthermore, we investigate how replacements at different encoder and decoder depths influence reconstruction quality under a fixed replacement ratio. Our results show that Conv-to-DSConv replacement at the intermediate layers of the encoder and decoder achieves a favorable complexity-performance trade-off, revealing layer-wise redundancy in DL-based JSCC systems. Extensive experiments further demonstrate that the proposed framework achieves substantial parameter reduction with only slight performance degradation, enabling flexible complexity-performance trade-offs for resource-constrained edge devices.
Read the full abstract

All authors Ming Ye, Kui Cai, Cunhua Pan, Zhen Mei, Wanting Yang, Chunguo Li

Systems and Control

Multi-robot obstacle-aware shepherding of non-cohesive target agents

This paper presents a novel control strategy for multi-agent shepherding of non-cohesive targets in obstacle-rich environments. Unlike previous approaches that assume cohesive flocking behavior, our method handles targets that interact only with nearby herders through repulsive forces and exhibit no inter-target coordination. Each herder employs a hybrid control policy that combines direct goal-oriented steering with obstacle-tangent maneuvering, enabling targets to circumnavigate obstacles while being guided toward a goal region. The herder dynamics integrate three key behaviors: return-to-goal motion when idle, target steering with adaptive directional control, and obstacle avoidance using both normal and tangential force components. Numerical simulations demonstrate superior performance compared to existing shepherding methods, achieving higher target confinement rates in cluttered environments. Experimental validation using TurtleBot4 herders and Osoyoo target robots in an indoor arena confirms the practical effectiveness of the proposed approach.
Read the full abstract

All authors Cinzia Tomaselli, Stefano Covone, Andreagiovanni Reina, Mario di Bernardo

Signal Processing

Fundamental Theorems on Controllability in Wave-domain Processing for Holographic MIMO

Wave-domain processing is an emerging paradigm where signal processing operations are partially shifted from the digital to the electromagnetic (EM) domain. Leveraging reconfigurable EM devices, this approach aims to reduce complexity, energy consumption, and latency in next-generation wireless systems employing holographic MIMO. This paper establishes fundamental theorems on the controllability of generic reconfigurable EM devices, where wave processing is achieved through the dynamic configuration of passive scatterers. Specifically, we derive necessary and sufficient conditions for controllability as a function of geometry and mutual coupling between elements. Finally, we provide a detailed discussion and numerical results characterizing the interplay between the number of elements, physical size, degrees of freedom, and directivity.
Read the full abstract

Systems and Control · Cross-listed from math.OC

Strategically Robust Linear Quadratic Dynamic Games

We study linear quadratic dynamic games where players are uncertain about each other's control policies or goals and consequently seek to be strategically robust. Building on recent work on strategically robust and risk-averse game theory, we first formalize the problem of strategically robust linear quadratic dynamic games. We show that these can be rewritten as simple transformations of linear quadratic games in which each player chooses a controller in a fictitious game in which they are faced with an adversary who is penalized for deviating from the other players' policies. This formulation naturally induces a novel notion of dynamic equilibrium, which we call a strategically robust dynamic equilibrium. We establish existence and uniqueness of such equilibria and furthermore show that the equilibrium policies are Markovian, linear, and can be efficiently computed via coupled backward Riccati equations. Through numerical simulations, including experiments in a network game, we illustrate the benefits of strategic robustness in designing robust and resilient decentralized control schemes. Our experiments also expose a "free-lunch" phenomenon in games in which robustness does not incur a corresponding loss in performance but can yield improvements in players' utilities and social welfare.
Read the full abstract

Systems and Control

Control of Multi-agent Systems under STL Specifications based on Prescribed Performance Observers

This paper addresses decentralized control of large-scale heterogeneous multi-agent systems subject to bounded external disturbances and limited communication, with the objective of satisfying cooperative Signal Temporal Logic (STL) specifications. The considered specifications involve spatiotemporal tasks that require collaboration among multiple agents, including agents beyond direct communication neighborhoods. To address the communication constraints, a $k$-hop Prescribed Performance State Observer ($k$-hop PPSO) is designed to enable each agent to estimate the states of agents up to $k$ communication hops away using only information from $1$-hop neighbors, while guaranteeing predefined performance bounds on the estimation errors. The estimation error bounds are explicitly incorporated into a reformulation of the spatial robustness of the STL specifications, yielding robustness measures that account for worst-case estimation uncertainty. Based on the modified robustness, a decentralized continuous-time feedback control law is designed to guarantee satisfaction of the STL specifications in the presence of bounded disturbances and estimation errors. The proposed framework provides formal correctness guarantees using only local information and limited communication. Numerical simulations illustrate the theoretical results.
Read the full abstract

Audio and Speech Processing · Cross-listed from cs.SD

Transformer-Based Rhythm Quantization of Performance MIDI Using Beat Annotations

Rhythm transcription is a key subtask of notation-level Automatic Music Transcription (AMT). While deep learning models have been extensively used for detecting the metrical grid in audio and MIDI performances, beat-based rhythm quantization remains largely unexplored. In this work, we introduce a novel deep learning approach for quantizing MIDI performances using a priori beat information. Our method leverages the transformer architecture to effectively process synchronized score and performance data for training a quantization model. Key components of our approach include dataset preparation, a beat-based pre-quantization method to align performance and score times within a unified framework, and a MIDI tokenizer tailored for this task. We adapt a transformer model based on the T5 architecture to meet the specific requirements of rhythm quantization. The model is evaluated using a set of score-level metrics designed for objective assessment of quantization performance. Through systematic evaluation, we optimize both data representation and model architecture. Additionally, we apply performance and score augmentations, such as transposition, note deletion, and performance-side time jitter, to enhance the model's robustness. Finally, a qualitative analysis compares our model's quantization performance against state-of-the-art probabilistic and deep-learning models on various example pieces. Our model achieves an onset F1-score of 97.3% and a note value accuracy of 83.3% on the ASAP dataset. It generalizes well across time signatures, including those not seen during training, and produces readable score output. Fine-tuning on instrument-specific datasets further improves performance by capturing characteristic rhythmic and melodic patterns. This work contributes a robust and flexible framework for beat-based MIDI quantization using transformer models.
Read the full abstract

Audio and Speech Processing

Audio Effect Estimation with DNN-Based Prediction and Search Algorithm

Audio effects play an essential role in sound design. This research addresses the task of audio effect estimation, which aims to estimate the configuration of applied effects from a wet signal. Existing approaches to this problem can be categorized into predictive approaches, which use models pre-trained in a data-driven manner, and search-based approaches, which are based on wet signal reconstruction. In this study, we propose a novel approach that integrates these approaches: first, DNNs predict the dry signal and effect configuration, and then a search is performed based on wet signal reconstruction using these predictions. By estimating the dry signal in the prediction stage, it becomes possible to complement or improve the predictions using reconstruction similarity as an objective function. The experimental evaluation showed that methods based on the proposed approach outperformed the method solely based on the predictive approach. Furthermore, the findings suggest that the task division of predicting the effect type combination followed by the search-based estimation of order and parameters was the most effective across various metrics.
Read the full abstract

Signal Processing

A General EM-Based Channel Model for Reconfigurable Antenna Systems

Reconfigurable antenna systems (RASs), such as fluid antennas and movable antennas, are poised to play a pivotal role in sixth-generation (6G) systems by dynamically adapting the antenna elements for system performance enhancement. However, unlocking their full potential requires channel models that accurately capture the influence of antenna configurations on the radiation, propagation, and reception of signals. Existing channel models suffer from several limitations, such as neglecting polarization effects, being restricted to specific antenna types, or relying on oversimplified assumptions. In this paper, we propose a general electromagnetic (EM)-based channel model grounded in spherical vector wave expansion (SVWE). The proposed EM-based channel model captures the impact of antenna position and orientation on the channel gain, thereby making it particularly well-suited for RASs. The effectiveness and accuracy are validated through comparisons with commercial simulation software, demonstrating excellent agreement in predicted channel gains. Moreover, it is shown that antenna orientation is a critical factor governing communication performance, and that dynamically adjusting the antenna orientation yields up to 70% improvement in achievable communication rate compared to a fixed-antenna configuration.
Read the full abstract

Audio and Speech Processing

Listening with Time: Precise Temporal Awareness for Long-Form Audio Understanding

While Large Audio Language Models (LALMs) achieve strong performance on short audio, they degrade on long-form inputs. This degradation is more severe in temporal awareness tasks, where temporal alignment becomes increasingly inaccurate as audio duration grows. We attribute these limitations to the lack of data, benchmarks, and modeling approaches tailored for long-form temporal awareness. To bridge this gap, we first construct LAT-Chronicle, a 1.2k hour long-form audio dataset with temporal annotations across real-world scenarios. We further develop LAT-Bench, the first human-verified benchmark supporting audio up to 30 minutes while covering three core tasks: Dense Audio Caption, Temporal Audio Grounding, and Targeted Audio Caption. Leveraging these resources, we propose LAT-Audio, formulating temporal awareness as a progressive global-to-local reasoning paradigm. A global timeline is first constructed as an aligned temporal-semantic context,and the Think-With-Audio Chain-of-Thought (TWA-CoT) is then introduced to perform iterative reasoning by incorporating local audio information via tool use. Experiments show that LAT-Audio surpasses existing models on long-form audio temporal awareness tasks and improves robustness to input duration. We release the dataset, benchmark, and model to facilitate future research at https://github.com/alanshaoTT/LAT-Audio-Repo.
Read the full abstract

All authors Mingchen Shao, Hang Su, Wenjie Tian, Bingshen Mu, Zhennan Lin, Lichun Fan, Zhenbo Luo, Jian Luan, Lei Xie

Audio and Speech Processing · Cross-listed from cs.CL

TTS-PRISM: A Perceptual Reasoning and Interpretable Speech Model for Fine-Grained Diagnosis

While generative text-to-speech (TTS) models approach human-level quality, monolithic metrics fail to diagnose fine-grained acoustic artifacts or explain perceptual collapse. To address this, we propose TTS-PRISM, a multi-dimensional diagnostic framework for Mandarin. First, we establish a 12-dimensional schema spanning stability to advanced expressiveness. Second, we design a targeted synthesis pipeline with adversarial perturbations and expert anchors to build a high-quality diagnostic dataset. Third, schema-driven instruction tuning embeds explicit scoring criteria and reasoning into an efficient end-to-end model. Experiments on a 1,600-sample Gold Test Set show TTS-PRISM outperforms generalist models in human alignment. Profiling six TTS paradigms establishes intuitive diagnostic flags that reveal fine-grained capability differences. TTS-PRISM is open-source, with code and checkpoints at https://github.com/xiaomi-research/tts-prism.
Read the full abstract

All authors Xi Wang, Jie Wang, Xingchen Song, Baijun Song, Jingran Xie, Jiahe Shao, Zijian Lin, Di Wu, Meng Meng, Jian Luan, Zhiyong Wu

Image and Video Processing

Multimodal Diffusion to Mutually Enhance Polarized Light and Low Resolution EBSD Data

In spite of the utility of 3-D electron back-scattered diffraction (EBSD) microscopy, the data collection process can be time-consuming with serial-sectioning. Hence, it is natural to look at other modalities, such as polarized light (PL) data, to accelerate EBSD data collection, supplemented with shared information. Complementarily, features in chaotic PL data could even be enriched with a handful of EBSD measurements. To inherently learn the complex dynamics between EBSD and PL to solve these inverse problems, we use an unconditional multimodal diffusion model, motivated by progress in diffusion models for inverse problems. Although trained solely on synthetic data once, our model has strong generalizable capabilities on real data which can be low-resolution, noisy, corrupted, and misregistered. With inference-time scaling, we show gains in performance on a variety of objectives including grain boundary prediction, super-resolution, and denoising. With our model, we demonstrate that there is little difference from full resolution performance with only 25% (1/4 the resolution) of EBSD data and corrupted PL data.
Read the full abstract

All authors Harry Dong, Timofey Efimov, Megna Shah, Jeff Simmons, Sean Donegan, Marc De Graef, Yuejie Chi

Audio and Speech Processing

UniSonate: A Unified Model for Speech, Music, and Sound Effect Generation with Text Instructions

Generative audio modeling has largely been fragmented into specialized tasks, text-to-speech (TTS), text-to-music (TTM), and text-to-audio (TTA), each operating under heterogeneous control paradigms. Unifying these modalities remains a fundamental challenge due to the intrinsic dissonance between structured semantic representations (speech/music) and unstructured acoustic textures (sound effects). In this paper, we introduce UniSonate, a unified flow-matching framework capable of synthesizing speech, music, and sound effects through a standardized, reference-free natural language instruction interface. To reconcile structural disparities, we propose a novel dynamic token injection mechanism that projects unstructured environmental sounds into a structured temporal latent space, enabling precise duration control within a phoneme-driven Multimodal Diffusion Transformer (MM-DiT). Coupled with a multi-stage curriculum learning strategy, this approach effectively mitigates cross-modal optimization conflicts. Extensive experiments demonstrate that UniSonate achieves state-of-the-art performance in instruction-based TTS (WER 1.47%) and TTM (SongEval Coherence 3.18), while maintaining competitive fidelity in TTA. Crucially, we observe positive transfer, where joint training on diverse audio data significantly enhances structural coherence and prosodic expressiveness compared to single-task baselines. Audio samples are available at https://qiangchunyu.github.io/UniSonate/.
Read the full abstract

All authors Chunyu Qiang, Xiaopeng Wang, Kang Yin, Yuzhe Liang, Yuxin Guo, Teng Ma, Ziyu Zhang, Tianrui Wang, Cheng Gong, Yushen Chen, Ruibo Fu, Chen Zhang, Longbiao Wang, Jianwu Dang

Audio and Speech Processing

Advancing automatic speech recognition using feature fusion with self-supervised learning features: A case study on Fearless Steps Apollo corpus

Using self-supervised learning (SSL) models has significantly improved performance for downstream speech tasks, surpassing the capabilities of traditional hand-crafted features. This study investigates the amalgamation of SSL models, with the aim to leverage both their individual strengths and refine extracted features to achieve improved speech recognition models for naturalistic scenarios. Our research investigates the massive naturalistic Fearless Steps (FS) APOLLO resource, with particular focus on the FS Challenge (FSC) Phase-4 corpus, providing the inaugural analysis of this dataset. Additionally, we incorporate the CHiME-6 dataset to evaluate performance across diverse naturalistic speech scenarios. While exploring previously proposed Feature Refinement Loss and fusion methods, we found these methods to be less effective on the FSC Phase-4 corpus. To address this, we introduce a novel deep cross-attention (DCA) fusion method, designed to elevate performance, especially for the FSC Phase-4 corpus. Our objective is to foster creation of superior FS APOLLO community resources, catering to the diverse needs of researchers across various disciplines. The proposed solution achieves an absolute +1.1% improvement in WER, providing effective meta-data creation for the massive FS APOLLO community resource.
Read the full abstract

Signal Processing

Enabling Flexible AFDM-ISAC Design: When Ambiguity Shaping Meets PAPR Control

Affine frequency division multiplexing (AFDM) has emerged as an enabling waveform for integrated sensing and communication (ISAC) due to its intrinsic chirp signaling nature. Nevertheless, the practicality of AFDM-ISAC systems needs to address two major technical challenges, i.e., high ambiguity function (AF) sidelobes and high peak-to-average power ratio (PAPR). By exploiting the reserved chirp-subcarrier (RCS) symbols and per-subcarrier pre-chirp parameters, we develop a flexible and unified AFDM waveform design framework for AF shaping and PAPR control. The proposed framework supports three tunable modes: AF shaping via weighted integrated sidelobe level (ISL) minimization, PAPR minimization, and joint AF shaping and PAPR control under a prescribed PAPR constraint. To solve the formulated nonconvex problem and to accommodate the discrete-phase constraints on the pre-chirp parameters, a joint ISL-PAPR discrete-phase majorization-minimization (JIPD-MM) algorithm is developed. Simulation results verify the effectiveness of the proposed framework under all the three design modes, with the benchmark comparisons conducted at similar effective spectral efficiencies. It is shown that the resulting weighted-ISL and PAPR gains lead to improved weak-target detectability in multi-target scenarios and lower bit error rate (BER) under power-amplifier (PA) nonlinearity.
Read the full abstract

All authors Lingsheng Meng, Yong Liang Guan, Zilong Liu, Yirui Luo, Pingzhi Fan

Systems and Control · Cross-listed from cs.LG

Optimal sequential decision-making for error propagation mitigation in digital twins

Here, we explore the problem of error propagation mitigation in modular digital twins as a sequential decision process. Building on a companion study that used a Hidden Markov Model (HMM) to infer latent error regimes from surrogate-physics residuals, we develop a Markov Decision Process (MDP) in which the inferred regimes serve as states, corrective interventions serve as actions, and a scalar reward that takes into consideration the cost-benefit tradeoff between system fidelity and maintenance expense. The baseline transition matrix is extracted from the HMM-learned parameters. We then extend the formulation to a Partially Observable MDP (POMDP) that accounts for the imperfect nature of regime classification by maintaining a belief distribution updated via Bayesian filtering, with the HMM confusion matrix serving as the observation model. Both formulations are solved via dynamic programming and validated through Gillespie stochastic simulation. We then benchmark two model-free reinforcement learning algorithms, Q-learning and REINFORCE, to assess whether effective policies can be learned without explicit model knowledge. A systematic comparison of different intervention policies demonstrates that the MDP policy achieves the highest cumulative reward and fraction of time in nominal operation, while the POMDP recovers approximately 95% of MDP performance under realistic observation noise. Sensitivity analyses across observation quality, repair probability, and discount factor confirm the robustness of these conclusions, and the major gaps in the policy hierarchy are statistically significant at $p < 0.001$. The gap between MDP and POMDP performance quantifies the value of information providing a principled criterion for investing in improved classification accuracy.
Read the full abstract

Systems and Control

Sampling-Based Safety Filter with Probabilistic Restrictiveness Guarantee

Ensuring safety is a critical requirement for autonomous systems, yet providing formal guarantees for nominal controllers remains a significant challenge. In this paper, we propose a modular sampling-based safety filter to ensure the safety of arbitrary nominal control inputs. At each timestep, the filter evaluates the safety of the nominal input by leveraging control sequence samples generated via Stein Variational Model Predictive Control (SV-MPC). This approach approximates a safety-conditioned posterior distribution over control sequences, enabling the filter to effectively capture multimodal safe regions in complex, non-convex environments. The filter guarantees safety by overriding the nominal input when all sampled control sequence candidates are deemed unsafe. By leveraging the scenario approach, the proposed method provides a probabilistic guarantee on its restrictiveness. We validate the filter through collision avoidance tasks in both single- and multi-vehicle settings, demonstrating its efficacy in navigating cluttered environments where nominal controllers may fail.
Read the full abstract

Systems and Control · Cross-listed from math.OC

Training Dynamics and Induced Preconditioning of ReLU Policy Gradient for Scalar LQR

Policy gradient is not invariant to reparameterization; the training dynamics of the weights of a neural controller differ from those of the induced gains. We make that difference exact for deterministic scalar discounted linear-quadratic regulation with bias-free one-hidden-layer ReLU policies. We show that policy gradient on the weights induces preconditioning on the gains' dynamics, alongside an explicit quadratic finite-step correction and an activation-group error. Moreover, we identify explicit conditions on the step sizes and initialization parameters that guarantee that the sequence of generated controllers remains stabilizing throughout training and converges to the optimal gains with high probability. For a given system and cost parameters, the sufficient step-size conditions imply contraction bounds independent of the network width. Numerical analysis illustrates how the neural realization changes the induced gain dynamics while maintaining stability across training.
Read the full abstract