Systems and Control · Cross-listed from cs.RO
UAVGENT: A Language-Guided Distributed Control Framework
Ziyi Zhang, Xiyu Deng, Guannan Qu et al. (+1 more)
We study
language-in-the-loop control for multi-drone systems that execute evolving, high-level missions while retaining formal robustness guarantees at the physical layer. We propose a three-layer architecture in which (i) a human operator issues natural-
language instructions, (ii) an LLM-based supervisor periodically interprets, verifies, and corrects the commanded task in the context of the latest state and target estimates, and (iii) a distributed inner-loop controller tracks the resulting reference using only local relative information. We derive a theoretical guarantee that characterizes tracking performance under bounded disturbances and piecewise-smooth references with discrete jumps induced by LLM updates. Overall, our results illustrate how centralized
language-based task
reasoning can be combined with distributed feedback control to achieve complex behaviors with provable robustness and stability.
Read the full abstract ▾
All authors Ziyi Zhang, Xiyu Deng, Guannan Qu, Yorie Nakahira
Systems and Control
Optimal Control-Based Falsification of Learnt Dynamics via Neural ODEs and Symbolic Regression
Lasse Kötz, Jonas Sjöberg and Knut Åkesson
We present a falsification framework that integrates learned surrogate dynamics with optimal control to efficiently generate counterexamples for cyber-physical systems specified in signal temporal logic (STL). The unknown system dynamics are identified using neural ODEs, while known a-priori structure is embedded directly into the model, reducing data requirements. The learned neural ODE is converted into an analytical form via symbolic regression, enabling fast and interpretable trajectory optimization. Falsification is cast as minimizing STL robustness over input trajectories; negative robustness yields candidate counterexamples, which are validated on the original system. Spurious traces are iteratively used to refine the surrogate, while true counterexamples are returned as final results. Experiments on ARCH-COMP 2024
benchmarks show that this method requires orders of magnitude fewer experiments of the system under test than optimization-based approaches that do not model system dynamics.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.CL
Guided by the Plan: Enhancing Faithful Autoregressive Text-to-Audio Generation with Guided Decoding
Juncheng Wang, Zhe Hu, Chao Xu et al. (+5 more)
Autoregressive (AR) models excel at generating temporally coherent audio by producing
tokens sequentially, yet they often falter in faithfully following complex textual
prompts, especially those describing complex sound events. We uncover a surprising capability in AR audio generators: their early prefix
tokens implicitly encode global semantic attributes of the final output, such as event count and sound-object category, revealing a form of implicit planning. Building on this insight, we propose Plan-Critic, a lightweight auxiliary model trained with a Generalized Advantage Estimation (GAE)-inspired objective to predict final instruction-following quality from partial generations. At inference time, Plan-Critic enables guided exploration: it evaluates candidate prefixes early, prunes low-fidelity trajectories, and reallocates computation to high-potential planning seeds. Our Plan-Critic-guided sampling achieves up to a 10-point improvement in CLAP score over the AR baseline-establishing a new state of the art in AR text-to-audio generation-while maintaining computational parity with standard best-of-N decoding. This work bridges the gap between causal generation and global semantic
alignment, demonstrating that even strictly autoregressive models can plan ahead.
Read the full abstract ▾
All authors Juncheng Wang, Zhe Hu, Chao Xu, Siyue Ren, Yuxiang Feng, Yang Liu, Baigui Sun, Shujun Wang
Systems and Control
Resilient Interval Observer-Based Control for Cooperative Adaptive Cruise Control under FDI Attack
Parisa Ansari Bonab, Elisabeth Andarge Gedefaw and Mohammad Khajenejad
Connectivity in connected and autonomous vehicles (CAVs) introduces vulnerability to cyber threats such as false data injection (FDI) attacks, which can compromise system reliability and safety. To ensure resilience, this paper proposes a control framework combining a nonlinear controller with an interval observer for robust state estimation under measurement noise. The observer bounds leader's states, while a neural network-based
estimator estimates the unknown FDI attacks in real time. These estimates are then used to mitigate FDI attack effects maintaining safe inter-vehicle spacing. The proposed approach leverages an idea of interval observer-based estimation and merges model-based and learning-based methods to achieve accurate estimations and real-time performance. MATLAB/Simulink results confirm resilient tracking, precise FDI attack estimation, and robustness to noise, demonstrating potential for real-world CACC applications under cyberattacks, disturbance, and bounded measurement noise.
Read the full abstract ▾
Systems and Control
Allocating Corrective Control to Mitigate Multi-agent Safety Violations Under Private Preferences
Johnathan Corbin, Sarah H.Q. Li and Jonathan Rogers
We propose a novel framework that computes the corrective control efforts to ensure joint safety in multi-
agent dynamical systems. This framework efficiently distributes the required corrective effort without revealing individual
agents' private preferences. Our framework integrates high-order control barrier functions (HOCBFs), which enforce safety constraints with formal guarantees of safety for complex dynamical systems, with a
privacy-preserving resource allocation mechanism based on the progressive second price (PSP) auction. When a joint safety constraint is violated,
agents iteratively bid on new corrective efforts via 'avoidance credits' rather than explicitly solving for feasible corrective efforts that remove the safety violation. The resulting correction, determined via a second price payment rule, coincides with the socially optimal safe distribution of corrective actions. Critically, the bidding process achieves this optimal allocation efficiently and without revealing private preferences of individual
agents. We demonstrate this method through multi-
robot hardware experiments on the Robotarium platform.
Read the full abstract ▾
Systems and Control
HERMES: A Unified Open-Source Framework for Realtime Multimodal Physiological Sensing, Edge AI, and Intervention in Closed-Loop Smart Healthcare Applications
Maxim Yudayev, Juha Carlon, Diwas Lamsal et al. (+2 more)
Intelligent assistive technologies are increasingly recognized as critical daily-use enablers for people with disabilities and age-related functional decline. Longitudinal studies, curation of quality
datasets, live monitoring in activities of daily living, and intelligent intervention devices, share the largely unsolved need in reliable high-throughput
multimodal sensing and processing. Streaming large heterogeneous data from distributed sensors, historically closed-source environments, and limited prior works on realtime closed-loop AI methodologies, inhibit such applications. To accelerate the emergence of clinical deployments, we deliver HERMES - an open-source high-performance Python framework for continuous
multimodal sensing and AI processing at the edge. It enables synchronized data collection, and realtime streaming inference with user PyTorch models, on commodity computing devices. HERMES is applicable to fixed-lab and free-living environments, of distributed commercial and custom sensors. It is the first work to offer a holistic methodology that bridges cross-disciplinary gaps in real-world implementation strategies, and guides downstream AI model development. Its application on the closed-loop intelligent prosthesis use case illustrates the process of suitable AI model development from the generated constraints and trade-offs. Validation on the use case, with 4 synchronized hosts cooperatively capturing 18 wearable and off-body modalities, demonstrates performance and relevance of HERMES to the trajectory of the intelligent healthcare domain.
Read the full abstract ▾
All authors Maxim Yudayev, Juha Carlon, Diwas Lamsal, Vayalet Stefanova, Benjamin Filtjens
Audio and Speech Processing · Cross-listed from cs.SD
SSVD-O: Parameter-Efficient Fine-Tuning with Structured SVD for Speech Recognition
Pu Wang, Shinji Watanabe and Hugo Van hamme
Parameter-efficient fine-tuning (PEFT) is a scalable approach for adapting large
speech foundation models to new domains. While methods such as LoRA and its state-of-the-art variants reduce adaptation costs, they typically allocate parameters uniformly across model subspaces, which limits their efficiency and scalability in
speech applications. Building on our prior work, this paper introduces SSVD-Outer (SSVD-O), an extension of the structured SVD-guided (SSVD) fine-tuning method. SSVD-O combines input acoustic feature space-associated inner transformations with output semantic feature space-associated outer transformations to enable scalable and balanced adaptation. We conduct the first systematic analysis of parameter budget allocation across model subspaces in PEFT for automatic
speech recognition (ASR), and investigate the trade-off between learning and forgetting under constrained resources. SSVD-O is benchmarked against LoRA, DoRA, PiSSA, and SSVD on domain-shifted ASR tasks, including child
speech and regional accents, across model scales from 0.1B to 2B within the ESPnet framework. Experimental results show that SSVD-O consistently narrows the performance gap to full fine-tuning while improving generalization and mitigating catastrophic forgetting.
Read the full abstract ▾
Audio and Speech Processing
SLAP: Scalable Language-Audio Pretraining with Variable-Duration Audio and Multi-Objective Training
Xinhao Mei, Gael Le Lan, Haohe Liu et al. (+5 more)
Contrastive
language-audio pretraining (CLAP) has achieved notable success in learning semantically rich audio representations and is widely adopted for various audio-related tasks. However, current CLAP models face several key limitations. First, they are typically trained on relatively small
datasets, often comprising a few million audio samples. Second, existing CLAP models are restricted to short and fixed duration, which constrains their usage in real-world scenarios with variable-duration audio. Third, the standard contrastive training objective operates on global representations, which may hinder the learning of dense, fine-grained audio features. To address these challenges, we introduce Scalable
Language-Audio Pretraining (SLAP), which scales
language-audio pretraining to 109 million audio-text pairs with variable audio durations and incorporates multiple training objectives. SLAP unifies contrastive loss with additional self-supervised and captioning losses in a single-stage training, facilitating the learning of richer dense audio representations. The proposed SLAP model achieves new state-of-the-art performance on audio-text
retrieval and zero-shot audio classification tasks, demonstrating its effectiveness across diverse
benchmarks.
Read the full abstract ▾
All authors Xinhao Mei, Gael Le Lan, Haohe Liu, Zhaoheng Ni, Varun Nagaraja, Yang Liu, Yangyang Shi, Vikas Chandra
Audio and Speech Processing · Cross-listed from cs.SD
SmoothCLAP: Soft-Target Enhanced Contrastive Language\–Audio Pretraining for Affective Computing
Xin Jing, Jiadong Wang, Andreas Triantafyllopoulos et al. (+4 more)
The ambiguity of human
emotions poses several challenges for machine learning models, as they often overlap and lack clear delineating boundaries. Contrastive
language-audio pretraining (CLAP) has emerged as a key technique for generalisable
emotion recognition. However, as conventional CLAP enforces a strict one-to-one
alignment between paired audio-text samples, it overlooks intra-modal similarity and treats all non-matching pairs as equally negative. This conflicts with the fuzzy boundaries between different
emotions. To address this limitation, we propose SmoothCLAP, which introduces softened targets derived from intra-modal similarity and paralinguistic features. By combining these softened targets with conventional contrastive supervision, SmoothCLAP learns
embeddings that respect graded emotional relationships, while retaining the same inference pipeline as CLAP. Experiments on eight affective computing tasks across English and German demonstrate that SmoothCLAP is consistently achieving superior performance. Our results highlight that leveraging soft supervision is a promising strategy for building
emotion-aware audio-text models.
Read the full abstract ▾
All authors Xin Jing, Jiadong Wang, Andreas Triantafyllopoulos, Maurice Gerczuk, Shahin Amiriparian, Jun Luo, Björn Schuller
Signal Processing
Automated Angular Received-Power Characterization of Embedded mmWave Transmitters Using Geometry-Calibrated Spatial Sampling
Maaz Qureshi, Mohammad Omid Bagheri, Abdelrahman Elbadrawy et al. (+2 more)
This paper presents an automated measurement methodology for angular received-power characterization of embedded millimeter-wave transmitters using geometry-calibrated spatial sampling. Characterization of integrated mmWave transmitters remains challenging due to limited angular coverage and
alignment variability in conventional probe-station techniques, as well as the impracticality of anechoic-chamber testing for platform-mounted active modules. To address these challenges, we introduce RAPTAR, an autonomous measurement system for angular received-power acquisition under realistic installation constraints. A collaborative
robot executes geometry-calibrated, collision-aware hemispherical trajectories while carrying a calibrated receive probe, enabling controlled and repeatable spatial positioning around a fixed device under test. A spectrum-analyzer-based receiver chain acquires amplitude-only received power as a function of angle and distance following quasi-static pose stabilization. The proposed framework enables repeatable angular received-power mapping and power-domain comparison against idealized free-space references derived from full-wave simulation. Experimental results for a 60-GHz radar module demonstrate a mean absolute received-power error below 2 dB relative to simulation-derived references and a 36.5 % reduction in error compared to manual probe-station measurements, attributed primarily to reduced
alignment variability and consistent spatial sampling. The proposed method eliminates the need for coherent field measurements and near-field transformations, enabling practical power-domain characterization of embedded mmWave modules. It is well suited for angular validation in real-world platforms where conventional anechoic measurements are impractical.
Read the full abstract ▾
All authors Maaz Qureshi, Mohammad Omid Bagheri, Abdelrahman Elbadrawy, William Melek, George Shaker
Image and Video Processing · Cross-listed from cs.CV
PISE: Physics-Anchored Semantically-Enhanced Deep Computational Ghost Imaging for Robust Low-Bandwidth Machine Perception
Tong Wu
We propose PISE, a physics-informed deep ghost imaging framework for low-bandwidth edge perception. By combining adjoint operator initialization with semantic guidance, PISE improves classification accuracy by 2.57% and reduces variance by 9x at 5% sampling.
Systems and Control
An Experimental Comparison of Sliding Mode and Immersion and Invariance Adaptive Controllers forPosition-feedback Tracking of a Simple Mechanical System with Friction
Luis Cervantes-Pérez, Víctor Santibáñez, Jesús Sandoval et al. (+2 more)
The purpose of this paper is to illustrate, in an experimental facility consisting of a simple pendular device, the performance of a sliding mode adaptive position-feedback tracking controller of mechanical systems with friction reported in the literature. To put this experimental evidence in perspective, we compare the performance of the sliding mode scheme with the one obtained by an adaptive controller designed following the well-known immersion and invariance technique.
Read the full abstract ▾
All authors Luis Cervantes-Pérez, Víctor Santibáñez, Jesús Sandoval, Romeo Ortega, Jose Guadalupe Romero
Image and Video Processing
Deep Lightweight Unrolled Network for High Dynamic Range Modulo Imaging
Brayan Monroy and Jorge Bacca
Modulo-Imaging (MI) offers a promising alternative for expanding the dynamic range of images by resetting the signal intensity when it reaches the saturation level. Subsequently, high-dynamic range (HDR) modulo imaging requires a recovery process to obtain the HDR image. MI is a non-convex and ill-posed problem where recent recovery networks suffer in high-noise scenarios. In this work, we formulate the HDR reconstruction task as an optimization problem that incorporates a deep prior and subsequently unrolls it into an optimization-inspired deep neural network. The network employs a lightweight convolutional denoiser for fast inference with minimal computational overhead, effectively recovering intensity values while mitigating noise. Moreover, we introduce the Scaling Equivariance term that facilitates self-supervised fine-tuning, thereby enabling the model to adapt to new modulo images that fall outside the original training distribution. Extensive evaluations demonstrate the superiority of our method compared to state-of-the-art recovery
algorithms in terms of performance and quality.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.SD
Multi-Task Instruction Tuning via Data Scheduling for Low-Resource Arabic SpeechLLMs
Hunzalah Hassan Bhatti, Firoj Alam and Shammur Absar Chowdhury
Audio large
language models (LLMs) enable unified
speech understanding and generation, but adapting them to linguistically complex and dialect-rich settings such as Arabic-English remains challenging. We present a controlled study of multi-task instruction tuning for an Arabic-centric audio LLM across generative tasks, including automatic
speech recognition (ASR) and
speech and text summarization, as well as discriminative tasks, including dialect identification (DID) and
speech emotion recognition (SER), in a resource-constrained setting. To support end-to-end Arabic
speech summarization, we introduce AraMega-SSum, the first Arabic
speech summarization
dataset designed for training and benchmarking Arabic-centric audio LLMs. We compare four training strategies: (i) Uniform Mixing (UM), (ii) Task-Progressive Curriculum (TPC), (iii) Aligner-Based Diverse Sampling (ADS) for training-time batch construction, and (iv) a two-stage TPC->ADS strategy. Our results reveal a clear efficiency-robustness trade-off. TPC achieves the strongest performance on generative tasks, including ASR and summarization. ADS improves paralinguistic tasks but reduces generative stability when used alone. The two-stage TPC->ADS strategy provides the best overall balance, achieving the strongest DID and SER performance while outperforming large proprietary models such as Gemini-2.5-Pro on discriminative tasks. We will publicly release AraMega-SSum together with all experimental resources to support future research in Arabic
speech understanding.
Read the full abstract ▾
Audio and Speech Processing
Robust Online Overdetermined Independent Vector Analysis Based on Bilinear Decomposition
Kang Chen, Xianrui Wang, Yichen Yang et al. (+6 more)
Online blind source separation is essential for both
speech communication and human-machine interaction. Among existing approaches, overdetermined independent vector analysis (OverIVA) delivers strong performance by exploiting the statistical independence of source signals and the orthogonality between source and noise subspaces. However, when applied to large microphone arrays, the number of parameters grows rapidly, which can degrade online estimation accuracy. To overcome this challenge, we propose decomposing each long separation filter into a bilinear form of two shorter filters, thereby reducing the number of parameters. Because the two filters are closely coupled, we design an alternating iterative projection
algorithm to update them in turn. Simulation results show that, with far fewer parameters, the proposed method achieves improved performance and robustness.
Read the full abstract ▾
All authors Kang Chen, Xianrui Wang, Yichen Yang, Andreas Brendel, Gongping Huang, Zbyněk Koldovský, Jingdong Chen, Jacob Benesty, Shoji Makino
Signal Processing
The Effect of Noise Correlation on MMSE Channel Estimation in One-Bit Quantized Systems
Minhua Ding, Prathapasinghe Dharmawansa, Italo Atzeni et al. (+1 more)
This paper analyzes the impact of spatially correlated additive noise on the minimum mean-square error (MMSE) estimation of multiple-input multiple-output (
MIMO) channels from one-bit quantized observations. Although additive noise can be correlated in practical scenarios, e.g., due to jamming, clutter, or other external disturbances, the effect of such correlation on the MMSE channel
estimator in this setting remains unexplored in prior work. Against this backdrop, we derive a novel analytical expression for the general
MIMO MMSE channel
estimator, which is inherently nonlinear in one-bit observations, and accommodates arbitrary channel and noise correlation structures. To further characterize the impact of noise correlation, we subsequently specialize the general MMSE expression to certain tractable multi
antenna configurations in which both the channel and the noise assume single-parameter constant correlation structures. Our analyses reveal nontrivial, noise-correlation-induced scenarios in which the
estimator remains linear despite non-zero channel and noise correlation parameters. Moreover, the results indicate that, at low-to-medium signal-to-noise ratio, noise correlation improves the MMSE performance when channels are uncorrelated, but degrades performance when channels are strongly correlated.
Read the full abstract ▾
All authors Minhua Ding, Prathapasinghe Dharmawansa, Italo Atzeni, Antti Tölli
Audio and Speech Processing · Cross-listed from cs.SD
A Unified Neural Codec Language Model for Selective Editable Text to Speech Generation
Hanchen Pei, Shujie Liu, Yanqing Liu et al. (+5 more)
Neural codec
language models achieve impressive zero-shot Text-to-
Speech (TTS) by fully imitating the acoustic characteristics of a short
speech prompt, including timbre, prosody, and paralinguistic information. However, such holistic imitation limits their ability to isolate and control individual attributes. In this paper, we present a unified codec
language model SpeechEdit that extends zero-shot TTS with a selective control mechanism. By default, SpeechEdit reproduces the complete acoustic profile inferred from the
speech prompt, but it selectively overrides only the attributes specified by explicit control instructions. To enable controllable modeling, SpeechEdit is trained on our newly constructed LibriEdit
dataset, which provides delta (difference-aware) training pairs derived from LibriHeavy. Experimental results show that our approach maintains naturalness and robustness while offering flexible and localized control over desired attributes. Audio samples are available at https://
speech-editing.github.io/
speech-editing/.
Read the full abstract ▾
All authors Hanchen Pei, Shujie Liu, Yanqing Liu, Jianwei Yu, Yuanhang Qian, Gongping Huang, Sheng Zhao, Yan Lu
Audio and Speech Processing
Purification Before Fusion: Toward Mask-Free Speech Enhancement for Robust Audio-Visual Speech Recognition
Linzhi Wu, Xingyu Zhang, Hao Yuan et al. (+5 more)
Audio-visual
speech recognition (AVSR) typically improves recognition accuracy in noisy environments by integrating noise-immune visual cues with audio signals. Nevertheless, high-noise audio inputs are prone to introducing adverse interference into the feature fusion process. To mitigate this, recent AVSR methods often adopt mask-based strategies to filter audio noise during feature interaction and fusion, yet such methods risk discarding semantically relevant information alongside noise. In this work, we propose an end-to-end noise-robust AVSR framework coupled with
speech enhancement, eliminating the need for explicit noise mask generation. This framework leverages a Conformer-based bottleneck fusion module to implicitly refine noisy audio features with
video assistance. By reducing modality redundancy and enhancing inter-modal interactions, our method preserves
speech semantic integrity to achieve robust recognition performance. Experimental evaluations on the public LRS3
benchmark suggest that our method outperforms prior advanced mask-based baselines under noisy conditions.
Read the full abstract ▾
All authors Linzhi Wu, Xingyu Zhang, Hao Yuan, Yakun Zhang, Changyan Zheng, Liang Xie, Tiejun Liu, Erwei Yin
Signal Processing
Temporal Data and Short-Time Averages Improve Multiphase Mass Flow Metering
Amanda Nyholm, Yessica Arellano, Jinyu Liu et al. (+2 more)
Reliable flow measurements are essential in many industries, but current instruments often fail to accurately estimate multiphase flows, which are frequently encountered in real-world operations. Combining machine learning (ML)
algorithms with accurate single-phase flowmeters has therefore received extensive research
attention in recent years. The Coriolis mass flowmeter is a widely used single-phase meter that provides direct mass flow measurements, which ML models can be trained to correct, thereby reducing measurement errors in multiphase conditions. This paper demonstrates that preserving temporal information significantly improves model performance in such scenarios. We compare a multilayer perceptron, a windowed multilayer perceptron, and a convolutional neural network (CNN) on three-phase air-water-oil flow data from 342 experiments. Whereas prior work typically compresses each experiment into a single averaged sample, we instead compute short-time averages from within each experiment and train models that preserve temporal information at several downsampling intervals. The CNN performed best at 0.25 Hz with approximately 95 % of relative errors below 13 %, a normalized root mean squared error of 0.03, and a mean absolute percentage error of approximately 4.3 %, clearly outperforming the best single-averaged model and demonstrating that short-time averaging within individual experiments is preferable. Results are consistent across multiple data splits and random seeds, demonstrating robustness.
Read the full abstract ▾
All authors Amanda Nyholm, Yessica Arellano, Jinyu Liu, Damian Krakowiak, Pierluigi Salvo Rossi
Signal Processing
RIS-Enhanced Information-Decoupled Symbiotic Radio Over Broadcasting Signals
Shu Cai, Ya-Feng Liu, Jun Zhan et al. (+1 more)
This paper studies a reconfigurable intelligent surface (RIS)-enhanced decoupled symbiotic
radio (SR) system in which a primary transmitter delivers common data to multiple primary receivers (PRs), while a RIS-based backscatter device sends secondary data to a backscatter receiver (BRx). Unlike conventional SR, the BRx performs energy detection and never decodes the primary signal, thereby removing ambiguity and preventing exposure of the primary payload to unintended receivers. In this paper, we formulate the problem as the minimization of the transmit power subject to a common broadcast rate constraint across all PRs and a bit error rate (BER) constraint at the BRx. The problem is nonconvex due to the unit-modulus RIS constraint and coupled quadratic forms. Leveraging a rate-balanced reformulation and a monotonic BER ratio characterization, we develop a low-complexity penalty-based block coordinate descent
algorithm with closed-form updates. Numerical results show fast convergence of the proposed
algorithm and reduced power consumption of the considered RIS-enhanced information-decoupled SR system over conventional SR baselines.
Read the full abstract ▾
All authors Shu Cai, Ya-Feng Liu, Jun Zhan, Qi Zhang