Audio and Speech Processing · Cross-listed from cs.SD
Unsupervised Speech Enhancement via Drifting
Diego Caviedes-Nozal, Liang Xu, Rasmus Kongsgaard Olsson et al. (+1 more)
This paper addresses unsupervised
speech enhancement in the unpaired setting using drifting methods, where training relies on separate collections of degraded and clean audio without corresponding pairs. While recent drifting approaches enable unpaired training, they do so at a heavy cost: because the objective optimizes only a marginal prior over clean
speech, the enhancer gradually loses the input's linguistic content and speaker identity. To fix this, we introduce input-conditioned drifting. We preserve the pull of the clean corpus while re-tethering the output to the degraded input via two mechanisms: an anchor encoder supplies the missing likelihood by pulling toward the input's features, and a key encoder conditions the prior by re-weighting retrieved frames. Neither requires labels or paired data. Using a training-free encoder selection criterion, Word Error Rate on VoiceBank-DEMAND falls to 10.1% (unprocessed: 11.7%), speaker similarity recovers from 0.490 to 0.879, and the recipe transfers in part to dereverberation on WSJ0-REVERB: content improves, rendering quality does not.
Read the full abstract ▾
All authors Diego Caviedes-Nozal, Liang Xu, Rasmus Kongsgaard Olsson, W. Bastiaan Kleijn
Signal Processing
Nonnegative DAG Learning via Concomitant Estimation
Madeline Navarro, Gonzalo Mateos and Samuel Rey
We study the problem of learning directed acyclic graphs (DAGs) with nonnegative edge weights from observational data. We propose the Nonnegative and Concomitant (NoCo) DAG
estimator, which jointly recovers the weighted graph structure and the exogenous noise variances in the linear structural equation model for the observations. Different from prior art, this noise-adaptive formulation blends a smoothed concomitant lasso criterion with a simpler log-determinant acyclicity constraint that exploits nonnegativity and yields a more benign optimization landscape. Specifically, nonnegative weights allow us to impose acyclicity directly on the adjacency
matrix without elementwise squaring of its entries, thus avoiding the well-documented degeneracy of the Karush-Kuhn-Tucker conditions. Computationally, we develop a method of multipliers'
algorithm that accommodates both homoscedastic and heteroscedastic noise profiles. Within each iteration, we use block successive convex approximation to minimize the augmented Lagrangian, alternating between proximal gradient steps for the adjacency
matrix and closed-form updates for the noise scales. Simulated experiments demonstrate NoCo's improved structural and edge-weight recovery relative to competing methods in a variety of settings, highlighting the benefits of exploiting nonnegativity along with noise adaptivity.
Read the full abstract ▾
Systems and Control
On the Achievable Inertia Constant of Inverter Based Resources
Christos M. Nikolakakos, Hassan Haes Alhelou and Nikos Hatziargyriou
As inverter-based resources (IBRs) displace synchronous generators, the inertia they can actually contribute to the grid becomes a critical planning parameter. Unlike synchronous machines, this contribution is bounded simultaneously by the available energy reserve, the converter power rating combined with voltage ride-through (VRT) obligations, and the inertia-emulation control scheme with its activation delay. This paper derives each bound in closed form and combines them into a unified envelope $\Heff(t,\lambda,V_g)=\min(\HE,\HP,\HC)$, which uses parameters accessible to the system operator to quantify how much inertia a plant can provide at a given loading $\lambda$, grid voltage $V_g$, and time $t$ after a disturbance. The analysis shows that below a loading-dependent critical voltage, VRT reactive-current priority does not leave active-current headroom for inertial power injection; that the converter overload ratio $\kappa$ matters mainly during voltage dips; and that de-loading yields a linear, quantifiable inertia gain, while collocated energy storage (ESS) removes the source-coupling constraint, making grid-following (GFL) and grid-forming (GFM) plants with storage equivalent post-activation, up to the GFL activation-delay discount within short RoCoF measurement windows. Simulations on the IEEE 9-bus system confirm the framework and motivate a two-dimensional inertia capability curve $H(\lambda,V_{g,\min})$ as a grid-code instrument analogous to the P–Q diagram of synchronous generators.
Read the full abstract ▾
Systems and Control
Robust Variable-Horizon MPC for Landing a Multirotor UAV on a Moving Platform
Sander Doodeman, Niels Berkers, Paula Chanfreut et al. (+2 more)
Landing a multirotor Unmanned Aerial Vehicle (
UAV) on a moving platform is challenging because a
UAV is underactuated and the desired landing state is generally a non-
equilibrium state. Shrinking- and variable-horizon approaches are promising for reaching such non-
equilibrium targets, but often lack robustness to disturbances. Robust Variable-Horizon Model Predictive Control (VH-MPC) addresses this limitation but is computationally complex for a high-dimensional system such as a multirotor
UAV. This paper presents a computationally efficient robust Variable-Horizon MPC method for reaching non-
equilibrium targets under bounded disturbances. The online horizon search is restricted to a neighborhood of the previously selected horizon, while maintaining recursive feasibility under bounded disturbances. A fixed robust positively invariant tube provides horizon-independent constraint tightening. By exploiting the differential flatness property of a
UAV, this approach is applied to a decoupled linearized system, enabling real-time implementation on an onboard Raspberry Pi 5 at 20 Hz. Simulations show that restricting the horizon search substantially reduces computation with limited impact on the objective, while real-time Gazebo simulations demonstrate the robustness of the method. Physical experiments demonstrate successful landing of a multirotor
UAV on a moving Stewart platform.
Read the full abstract ▾
All authors Sander Doodeman, Niels Berkers, Paula Chanfreut, Elena Torta, Duarte Antunes
Signal Processing
Joint DNN Partitioning and Resource Allocation for Satellite-Terrestrial Collaborative Inference Systems
Wenyu Liang, Zesong Fei, Peng Liu et al. (+2 more)
This letter investigates a satellite-terrestrial collaborative inference system for remote sensing information processing, where a ground station (GS) serves multiple low Earth orbit (LEO) satellites running heterogeneous deep neural network (DNN) inference tasks. By partitioning the DNN between the satellite and the GS, each satellite executes the front-end layers locally, transmits the resulting intermediate features to the GS, thereby offloading the remaining layers to the GS for inference completion. We jointly optimize DNN partitioning, satellite and GS computing resources, satellite-ground bandwidth allocation, and satellite transmit power to minimize the average task completion latency under per-satellite energy constraints. To address the coupling between discrete partitioning decisions and continuous communication-computing variables, we propose a two-layer optimization
algorithm. The inner layer performs closed-form updates and efficient nested bisection searching to update the continuous resource variables, while the outer layer employs random-restart coordinate descent to refine the partitioning decisions. Simulation results show that the proposed
algorithm effectively reduces task completion latency compared with conventional binary offloading.
Read the full abstract ▾
All authors Wenyu Liang, Zesong Fei, Peng Liu, Xinyi Wang, Ming Zeng
Systems and Control
Frequency Measurement Practices for Inertia Assessment in Inverter Based Resources Dominated Power Systems
Christos M. Nikolakakos, Hassan Haes Alhelou and Nikos Hatziargyriou
Transmission system operators and protective relays compute the rate of change of frequency (RoCoF) from frequency averaged over tens to hundreds of milliseconds, as standards and grid codes specify. Any response delivered inside that window lowers the measured RoCoF, so the inferred inertia depends on the window and the responses in service. This paper defines the windowed inertia estimate and derives an exact identity the system inertia divided by one minus the fraction of the disturbance energy delivered inside the window and a closed-form prediction that applies the same
estimator to the trajectory of a linear response model. Eleven IEEE 9-bus RMS runs synchronous, grid-following and grid-forming configurations, disturbance sizes and parameter sweeps are reproduced by that model, and the identity closes on the measured response powers. Fast frequency response inflates the estimate most: at 500 ms a grid-following system with 20 % less synchronous inertia reports the same value as the all-synchronous system, the grid-forming system a higher one. On a reduced Greek transmission system the 500ms estimate exceeds the inertia by 17-26%, and a scenario with 10% less synchronous inertia reports more than the all-synchronous scenario's inertia. Operators can thus compute what their measurement reports.
Read the full abstract ▾
Audio and Speech Processing
Measurement-Based Bitrate-Energy-Quality Analysis of Neural Audio Codec Decoders on Laptop and Phone Platforms
Seunghyeon Shin and Seokjin Lee
Neural audio codecs can achieve similar objective quality at lower bitrates than conventional codecs, but their decoder-side computational cost may offset this bitrate advantage on battery-powered client devices. This paper presents a measurement-based rate–energy–quality analysis of four neural audio codecs–EnCodec, DAC, HILCodec, and SNAC–and two conventional baselines, AAC-LC and Opus, on laptop and phone platforms.
Speech and music are evaluated separately using the original-reference ViSQOL protocol. For the main comparison, operating points are matched by selecting the measured point nearest to the midpoint of the common overlap in treatment-level mean ViSQOL. A secondary analysis includes only explicitly evaluated bitrate settings. Decoder energy is measured as idle-subtracted energy per second of audio (J/s), and results are summarized as the median of three repeated runs for execution-valid runtime and device paths. Pairwise break-even transmission-energy thresholds are then derived analytically from the measured decoder energy and bitrate. At the matched-quality operating points, the evaluated neural codecs achieved similar ViSQOL scores at lower bitrates but generally required more decoder-side energy than the applicable conventional codecs. EnCodec produced the lowest neural break-even thresholds in both matched-quality cohorts on both platforms. By contrast, several DAC and SNAC comparisons on the Phone XNNPACK CPU path exceeded 200 mJ/kbit, and their full-band Phone configurations required more than 1 s to decode 1 s of audio. These results show that lower bitrate alone does not guarantee an energy benefit: deployment efficiency also depends on decoder complexity, the effective runtime and device mapping, execution validity, accelerator availability, and the transmission-energy coefficient.
Read the full abstract ▾
Audio and Speech Processing
CharDuplex: Building Character-Consistent Full-Duplex Spoken Dialogue Models
Donghang Wu, Yisi Liu, Chen Chen et al. (+2 more)
Full-duplex
speech models are moving voice interaction beyond conventional turn-taking, yet natural conversation is shaped not only by when an
agent speaks, but also by how it behaves as a conversational character. We present CharDuplex, a character-driven full-duplex
speech model that combines real-time spoken interaction with persona-conditioned behavior. We first adapt GLM-4-Voice to an always-on dual-stream architecture and train the model for full-duplex conversation. Then a fully automated pipeline constructs character-conditioned dialogue data from open-source character descriptions for character-conditioned supervised fine-tuning. The model is further refined with the proposed FDGym, where an LLM-simulated user dynamically interacts with the model, enabling reinforcement learning over evolving multi-turn interactions. On SpeechRole-Eval, CharDuplex achieves the highest average score among the evaluated open-source models, while remaining competitive with closed-source systems. It also demonstrates competitive general
speech intelligence and strong full-duplex interaction capabilities. CharDuplex demonstrates a practical training recipe for building full-duplex voice assistants that are not only interactive, but also character-consistent.
Read the full abstract ▾
All authors Donghang Wu, Yisi Liu, Chen Chen, Hexin Liu, Eng Siong Chng
Audio and Speech Processing
Beyond Textual Chain-of-Thought: JEPA-Conditioned Latent Reasoning for Large Audio Language Models
Donghang Wu, Haoyang Zhang, Yizhou Peng et al. (+6 more)
Explicit textual Chain-of-Thought (CoT) has improved the
reasoning ability of large audio
language models (LALMs). However, textual CoTs are often constructed from text captions of audio and provide limited access to the acoustic evidence, which can introduce problems like
hallucination. To address this modality-gap issue, we introduce JELAR, a Joint-
Embedding Predictive Architecture (JEPA)-based latent
reasoning framework that conditions latent
reasoning supervision on acoustic representations learned from raw waveforms. During training, a frozen WavJEPA model provides representations learned directly from raw waveforms. A non-causal expert first constructs answer-aware queries, which cross-attend to WavJEPA
embeddings to produce latent
reasoning targets. The LALM is trained to predict these targets before generating its response. Experimental results show that JELAR improves the Audio-Reasoner baseline by 2.70 and 9.10 absolute percentage points on MMAU-mini and MMAR, respectively, demonstrating the effectiveness of JEPA-conditioned latent
reasoning as an alternative to explicit textual CoT supervision.
Read the full abstract ▾
All authors Donghang Wu, Haoyang Zhang, Yizhou Peng, Shreyas Gopal, Yi-Wen Chao, Chen Chen, Hexin Liu, William Tjhi, Eng-Siong Chng
Audio and Speech Processing · Cross-listed from cs.CV
Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomy
Abhinav Sharma, Sai Karthik Navuluru, Wang Wei et al. (+21 more)
Video and audio are perceived together, yet most generative models treat them in isolation. We examine methods that model the two modalities jointly, generate one from the other, or edit them in a coupled manner, organized around a single question: how is the output kept coherent across modalities in time and semantics? A unified formulation casts joint generation, cross-modal generation, and joint editing as three problems defined on a single distribution over audio-visual pairs, and a taxonomy compares methods along five design axes. To our knowledge, this is the first overview to systematically taxonomize joint audio-visual editing, which we map as nine edit categories spanning 28 edit types. We describe methods,
datasets, and metrics for each setting and close with the open problems we view as most consequential.
Read the full abstract ▾
All authors Abhinav Sharma, Sai Karthik Navuluru, Wang Wei, Daksh Dangi, Xiangbo Gao, Li Li, Bo Ni, Vardhan Dongre, Junda Wu, Xiyang Hu, Jiuxiang Gu, Seunghyun Yoon, Tong Yu, Chien Van Nguyen, Mohamed Elmoghany, Nedim Lipka, Hoda Eldardiry, Hongjie Chen, Tyler Derr, Thien Huu Nguyen, Zhengzhong Tu, Nesreen K. Ahmed, Franck Dernoncourt, Ryan A. Rossi
Systems and Control
Derivative-Free Generalized Multivariable Super-Twisting Control for Constrained Euler-Lagrange Systems
Chidre Shravista Kashyap and Jishnu Keshavan
Robotic manipulators performing payload lift-and-transfer must track prescribed trajectories under abrupt load changes, with limited actuation and inaccurate plant knowledge. Such plants are inherently described by multivariable Euler–Lagrange~(EL) dynamics with cross-coupling established through inertial and Coriolis terms. The super-twisting
algorithm~(STA) is the standard approach to regulating a first-order sliding mode~(FOSM) in such settings, yet existing designs are largely restricted to scalar sliding variables, and rest on a disturbance-derivative assumption that EL systems violate. In particular, the FOSM dynamics depend on the disturbance and become discontinuous under the step changes in load produced by payload engagement and release, rendering that derivative ill-defined. This study resolves both with a derivative-free generalized multivariable super-twisting framework. The discontinuous sign-function integral term is replaced with a continuous fractional-power term whose dynamics are independent of the disturbance derivative, while the known state-dependent scaling of the perturbation is incorporated into both the control law and the adaptive gain update. A time-shifting barrier function guarantees prescribed-time convergence to a user-specified bound independently of initial conditions, with the design relying only on mass-
matrix norm bounds. Simulation and experimental results on a 7-DoF manipulator under actuator saturation are used to validate the proposed scheme.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.SD
SAIL: Spatial Audio Intelligence with Large Language Models via Disentangled Acoustic-Spatial Encoding and Dual-Stream Q-Former
Zhengding Luo, Jinyang Wu, Haozhe Ma et al. (+3 more)
Spatial audio large
language models (LLMs) enable embodied
agents, wearable assistants, and immersive systems to recognize sound events, localize sources, and reason about their spatial relationships. However, existing spatial audio LLMs often rely on early fusion of acoustic and spatial features and source-agnostic
token representations. These designs make it difficult to preserve the correspondence between individual sound events and their spatial attributes, particularly in multi-source scenes. To address this limitation, we propose SAIL, a Spatial Audio Intelligence framework with LLMs that preserves acoustic-spatial structure and source-level correspondence from audio encoding to LLM
alignment. SAIL introduces a Disentangled Spatial Audio
Transformer that represents Mel-spectrogram and interaural phase difference features as separate acoustic and spatial streams. Source-discriminative task queries further learn event, direction, and distance information for each source. A Dual-Stream Q-Former then aligns the two streams with the LLM using acoustic and spatial queries organized by source slots. Compared with the early-fusion baseline, SAIL achieves consistent improvements in dual-source sound event detection, direction and distance estimation, and spatial
reasoning. These results demonstrate the importance of structured, source-discriminative audio representations for multi-source spatial understanding and
reasoning.
Read the full abstract ▾
All authors Zhengding Luo, Jinyang Wu, Haozhe Ma, Yanghao Zhou, Woon-Seng Gan, Wenwu Wang
Audio and Speech Processing
Audio Tokens as a Budgeted Resource: Marginal-Utility Allocation for Scalable Audio Representations
Mingyu Zhao, Jinchao Zhang and Zhiyong Wu
Discrete audio
tokens are widely used as a representation interface, yet fixed-depth RVQ tokenizers allocate equal capacity to every frame despite varying refinement value. We introduce UniAdapt, which learns marginal utility of RVQ refinements on a frozen codec and allocates them under exact serialized-bit budgets. A rate-independent causal controller predicts acoustic utility, while an optional semantic head supports
speech-only utterance-level allocation; measured acoustic and semantic marginal gains on
speech have a correlation of 0.42. For causal allocation, a primal-dual allocator selects prefix-valid depths, while an exact guard constrains each sequence prefix to its matched fixed-depth serialized budget. Under utterance-level allocation, UniAdapt reduces Log-STFT distortion by 1.07-4.39 percent across
speech, music, and environmental audio without larger budgets. Causally, it improves three of four
speech rates with zero violations across 800 utterance-rate evaluations and runs faster than real time. A 20-listener utterance-level MUSHRA study shows a significant 3.52-point
speech improvement, with no significant differences on music or environmental audio. These results support separating utility prediction from budget enforcement for scalable, budget-conditioned audio representations.
Read the full abstract ▾
Signal Processing
Stacked Intelligent Metasurface-Diffractive Deep Neural Networks for Onboard Terrain Classification from SAR Level-0 Raw Data
Mengbing Liu, Xin Li, Jiancheng An et al. (+1 more)
Real-time terrain classification directly from Level-0 raw Synthetic Aperture Radar (SAR) data remains restricted by traditional digital-centric paradigms, where the processing of noisy, high-dimensional patches is hindered by computationally intensive processors and significant downlink latency. To address these fundamental limitations, this work establishes a new research paradigm for autonomous on-board sensing by proposing a Stacked Intelligent Metasurface-Diffractive Deep Neural Network (SIM-D$^2$NN). This architecture leverages the physical wave-propagation medium to offload inference tasks from digital processors to a physical device. By executing in-wave feature mapping, the SIM-D$^2$NN facilitates a move toward an integrated `compute-while-transmitting' framework, providing an alternative to the traditional `digitize-then-process' sequence. The multi-layer metasurface is positioned at the forefront of the satellite communication module. The initial layer modulates the raw SAR data through both amplitude and phase adjustments, where a 90$^\circ$ phase rotation is introduced as a lightweight but effective augmentation strategy to enhance robustness against noise and Doppler distortions. Subsequent layers learn variable phase shifts, enabling advanced feature mapping for the classification task. The classification results at the terrestrial station can be directly obtained based on the signal amplitude received at each
antenna. This design reduces reliance on downlink bandwidth and high-power terrestrial computing, achieving performance around 90% in the binary task directly from real raw SAR data in terms of accuracy, precision, recall, and F1 Score. Therefore, our method helps bridge the gap between next-generation remote sensing tasks and in-orbit processing needs, paving the way for computationally efficient remote sensing applications.
Read the full abstract ▾
All authors Mengbing Liu, Xin Li, Jiancheng An, Chau Yuen
Systems and Control · Cross-listed from cs.RO
When World Models Lie: Adaptive Safety Analysis Under Wrong Imaginations
John Cao and Somil Bansal
World models offer a powerful substrate for safety
reasoning in high-dimensional robotic systems, but they are also fallible: their predictions can be biased, miscalibrated, or confidently wrong. This creates a central challenge for latent-space safety filters, which often learn Hamilton-Jacobi safety value functions on the dynamics of a world model. If the world model is incorrect, the resulting value function can inherit its errors and produce overconfident safety estimates. Existing latent safety filters often rely on auxiliary signals such as ensemble disagreement or value-target consistency residuals for adaptation, but these signals can remain small even when the world model's predictions deviate from observations. We propose an adaptive latent safety filter that calibrates safety
reasoning using directly observed world-model error. Our method uses Adaptive Conformal Inference to construct online uncertainty sets from discrepancies between predicted and observation-inferred latent states, then evaluates safety pessimistically by minimizing the learned value function over these sets. This allows the filter to remain minimally conservative when the world model is accurate, while becoming more cautious when observations reveal model mismatch. We provide a finite-time coverage guarantee for the adaptive uncertainty radius. Through simulation and hardware experiments, we show that our method significantly reduces failures relative to state-of-the-art latent safety filters while preserving task completion.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.CL
SALMONN-duo: Adaptive Dual-System Coordination for Full-Duplex Voice Agents
Wenyi Yu, Siyin Wang, Terumi Chiba et al. (+5 more)
Full-duplex
speech large
language models (LLMs) enable low-latency, natural voice interaction. However, real-world
agents must also use tools and perform deliberative
reasoning-operations whose variable latency and computational cost conflict with the stringent timing requirements of real-time conversation. To reconcile these demands, we propose SALMONN-duo, an adaptive dual-system voice
agent inspired by dual-process theories of cognition. SALMONN-duo separates real-time interaction from deliberative computation by pairing an always-on, fast-thinking full-duplex
speech LLM (system 1) with a powerful asynchronous slow-thinking LLM
agent (system 2). Beyond handling real-time interaction, system 1 learns when to answer directly and when to delegate, remaining responsive during backend execution and seamlessly integrating returned information into the ongoing dialogue without exposing tool traces or losing conversational context. Evaluations on single-turn spoken question answering (QA) and multi-turn conversations demonstrate that adaptive delegation substantially improves accuracy on knowledge-intensive and multi-hop
reasoning questions, while knowledge-boundary-aware training avoids unnecessary system 2 invocations. On a customized version of $\tau$-Voice, SALMONN-duo further demonstrates its ability to complete environment-grounded, policy-constrained tasks through multi-turn interactions in realistic business scenarios. Finally, cost-aware reinforcement learning further enhances the trade-off between task performance and backend usage across the QA and conversation tasks, while improving task success and response safety on $\tau$-Voice with an acceptable increase in the delegation rate.
Read the full abstract ▾
All authors Wenyi Yu, Siyin Wang, Terumi Chiba, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Lu Lu, Chao Zhang
Audio and Speech Processing
Explainable and Generalisable LLM-based Cognitive Decline Detection with Spontaneous Speech
Ziyun Cui, Wen Wu, Chuan Shi et al. (+10 more)
Alzheimer's disease (AD) and mild cognitive impairment (MCI), which may precede AD, manifest early through subtle linguistic and acoustic alterations. Traditional diagnostics, however, are often resource-intensive and lack scalability for mass screening. To address these challenges, we introduce a novel bilingual
speech large
language model framework for automated, explainable cognitive screening. Unlike conventional pipelines that rely on error-prone automatic
speech recognition, our system directly processes raw
speech to learn joint acoustic-semantic representations, preserving critical prosodic cues often lost in transcription. Utilising our newly collected PUTH-AD
dataset alongside multiple open-source corpora, we implemented a multi-task learning objective that simultaneously performs cognitive status classification and generates clinician-understandable natural
language explanations. Our system achieved the highest average accuracy and AUROC across six
dataset/task conditions, comparing three representative baselines. The system demonstrated cross-task transfer to held-out PUTH-AD task subsets, maintaining classification accuracy on an entirely unseen cognitive task without task-specific fine-tuning. Furthermore, clinician evaluation confirms that the generated explanations are both clinically relevant and largely consistent with the underlying
speech evidence, supporting their potential utility in clinical interpretation. This study provides a scalable, objective, and explainable framework for
speech-based cognitive screening, combining cognitive status classification with natural
language explanations that clinicians can assess and verify, bridging the gap between advanced AI and clinical utility.
Read the full abstract ▾
All authors Ziyun Cui, Wen Wu, Chuan Shi, Shuguang Yang, Xueying Gui, Yan Zheng, Qiong Yang, Haiyan Zhao, Wei-Qiang Zhang, Ji Wu, Yelei Li, Nan Li, Chao Zhang
Audio and Speech Processing
GAMF: Learned and Analytical Array Transfer Function Matching for Array-Generic Direction-of-Arrival Estimation
Zhiheng Jin, Shichao Hu, Chunyang Xu et al. (+1 more)
Microphone positional encoding supports cross-array direction-of-arrival (DOA) estimation, but coordinates alone cannot fully describe device shadowing or microphone directivity. We propose a Generalizable ATF Matching Framework (GAMF) for DOA estimation across array geometries and microphone counts, using array transfer functions (ATFs) as acoustic descriptors. The learned branch incorporates ATF
embeddings into geometry-conditioned neural estimation to match acoustic observations with candidate directions. The analytical branch performs normalized ATF matching adapted from generalized steered response power. A hybrid configuration combines their scores through adaptive gating. Simulations across array configurations show that both learned and hybrid configurations outperform a representative positional-encoding-based neural baseline, remain competitive with analytical ATF matching in clean, low-reverberation scenes, and substantially improve upon it under stronger noise or reverberation. On eight-microphone LOCATA Task 1 recordings, the hybrid configuration outperforms the evaluated state-of-the-art baselines for three-dimensional DOA estimation, achieving mean errors of $3.76^\circ$ for three-dimensional DOA and $2.87^\circ$ for azimuth.
Read the full abstract ▾
All authors Zhiheng Jin, Shichao Hu, Chunyang Xu, Mengyao Zhu
Systems and Control
A Weak Notion of Symmetry for Control Systems
Jake Welde, Riley Link and Pieter van Goor
Symmetry (or invariance) is a powerful structural property that enables efficient, effective solutions for estimation and control. However, the constraints imposed on a system's dynamics by classical invariance make
symmetry a very rigid property, which may be broken by external forces or confined to only a portion of the overall system. Seeking greater flexibility, this work introduces a novel relaxed notion of
symmetry, termed “weak invariance”, in which the non-symmetric part of the dynamics (the “residual”) can be captured entirely by another control system evolving on the
symmetry group. Weakly invariant systems are strictly more general than classical invariant systems, but they nonetheless enjoy many similar favorable properties. In particular, we prove that any weakly invariant system admits a cascade decomposition in which the driven subsystem is group affine, showing that weak
symmetry generalizes not only classical
symmetry, but also the (thus far distinct) class of group affine systems. We also show that a weak
symmetry with autonomous residual can be factored out of the system's error dynamics, enabling yet a greater reduction of dimensionality as compared to classical
symmetries. Finally, we study the example of an aerial vehicle under the influence of
gravity, for which we propose a nine-dimensional weak
symmetry (strictly containing the system's familiar four-dimensional classical
symmetry). Weak invariance thus generalizes classical
symmetry while also preserving key structural properties, thereby laying a foundation for more flexible methods of
symmetry-informed control.
Read the full abstract ▾
Audio and Speech Processing
SPEAR-Gen: Generation-Aware Pre-training for Unified Speech Representations
Xiaoyu Yang, Arthur Hinsvark, Antonios Alexos et al. (+3 more)
Speech understanding and generation place different demands on
speech representations, and existing models are typically optimised towards one capability or the other. To reduce this gap, we introduce SPEAR-Gen, a
speech representation model that learns a single representation for both capabilities. Task-aligned feature aggregation consolidates complementary linguistic and paralinguistic information across a frozen encoder into discrete targets for masked prediction, while a coarse-to-fine objective combines log-Mel reconstruction with residual flow matching to preserve spectral structure and fine-grained acoustic variation. Experiments on SUPERB and
speech resynthesis show that SPEAR-Gen maintains strong understanding performance while substantially improving resynthesis quality and speaker preservation. These results demonstrate that a single
speech representation can effectively support both understanding and generation.
Read the full abstract ▾
All authors Xiaoyu Yang, Arthur Hinsvark, Antonios Alexos, Osama Hanna, Philip C. Woodland, Yiting Lu
Systems and Control
Integrated Thermal and Power Management for Wave-Powered Subsea Data Centers via Nonlinear Model Predictive Control
Wanqun Yang and Jun Chen
This paper develops an integrated modeling and nonlinear model predictive control (NMPC) framework for coordinating thermal management, flexible workload scheduling, wave-power utilization, and battery operation in a wave-powered subsea data center. Realistic data center workloads are constructed from job-level CPU,
memory, and GPU measurements from the MIT Supercloud
dataset and divided into interactive and delay-tolerant flexible jobs. Thermal behavior is represented by a three-node lumped model of the IT equipment, recirculating nitrogen, and pressure hull with surrounding seawater as the thermal boundary. The NMPC jointly optimizes the flexible workload power budget and cooling command subject to thermal, battery, and workload constraints. Closed-loop simulations under different workload, thermal, battery, and renewable-generation conditions demonstrate that the proposed framework maintains thermal safety while adapting cooling operation and flexible workload execution to wave-power availability and battery state-of-charge. The parametric studies show that battery capacity and wave-generation capacity strongly affect battery availability and flexible-workload queue accumulation, while excessive renewable generation capacity may lead to increased energy curtailment. Monte Carlo and distance-correlation analyses further show that flexible-job delay is relatively insensitive to the investigated system parameters, whereas terminal battery state-of-charge is primarily influenced by battery energy capacity and wave generation capacity.
Read the full abstract ▾
Signal Processing
Certificate-Governed CRT Sparse FFT: Verified Global-Label Candidate Construction under Explicit Decoding Models
Aaron R. Flouro and Shawn P. Chadwick
We present a deterministic CRT-based sparse
Fourier architecture for exact recovery of noiseless, on-grid, at-most-\(k\)-sparse spectra with an engineered transform length. The transform length \(N\) is selected as an exact product of small pairwise-coprime stage
primes, allowing each stage to use transforms whose size scales with the sparsity rather than the ambient dimension. Three co-
prime-increment time shifts provide a one-sided singleton-consistency screen: every genuine singleton passes, while passing collision bins remain provisional candidates. Each candidate bin is mapped by a total label-emission rule to at most one global frequency label, and per-stage label sets are intersected, yielding a deterministic \(O(k)\) candidate bound for every input. Candidate construction uses \(O(k\log N)\) samples. In a comparison real-RAM model with exact arithmetic and exact phase evaluation but without unit-cost root-index decoding, it requires \(O(k\log^2 N/\log k)\) arithmetic operations; in a stronger root-index-oracle model the arithmetic cost is \(O(k\log N)\). Under all-stage genuine-singleton survival, the candidate set contains the true support. Exactness, however, does not depend on that survival condition: a consecutive-sample residual verifier accepts a sparse reconstruction only when it is exact, and all failures route to a dense FFT. The resulting hybrid
algorithm therefore returns the exact spectrum for every input in the stated noiseless, on-grid, at-most-\(k\)-sparse model, with \(O(N\log N)\) worst-case runtime. The verified sparse path additionally incurs an \(O(k^2)\) verifier cost in the root-index-oracle model. Neither computational model is a bit-complexity model.
Read the full abstract ▾