Audio and Speech Processing · Cross-listed from cs.MM
FORTE: FOL-guided Optimal Refinement for Text-audio rEtrieval
Arghya Pal and Sailaja Rajanala
Text-to-audio
retrieval has made significant progress with shared
embedding models such as CLAP and Pengi, yet they often struggle with fine-grained semantic
alignment due to the inherent modality gap between text and audio. In this work, we propose FORTE, a unified framework that integrates structured logical
reasoning with parameter-efficient cross-modal
alignment to improve
retrieval precision. Our approach first transforms queries into first-order logic and refines them via a constrained search that preserves semantic invariance while introducing discriminative attributes. The refined representation is then aligned with audio
embeddings using a lightweight projection module, followed by a predicate-aware re-ranking step that enforces logical consistency at inference. Extensive experiments on AudioCaps and Clotho demonstrate consistent improvements over strong baselines, particularly in challenging fine-grained scenarios. Our results highlight the effectiveness of combining symbolic
reasoning with representation learning for cross-modal
retrieval.
Read the full abstract ▾
Systems and Control
Efficient Multi-Agent Optimization of Optical Power in S+C+L-Band Systems
Junzhe Xiao, Kaida Chen, Cong Wang et al. (+4 more)
We propose an AI
Agent tailored for link power management in multi-band systems. In S+C+L band span-level study, the
agent efficiently solves various optimization objectives. In network-wide evaluation, it delivers 689.0 Tbps gain in total allocated traffic with merely 303 average interactions per power profile.
Read the full abstract ▾
All authors Junzhe Xiao, Kaida Chen, Cong Wang, Zekun Niu, Minghui Shi, Yanhan Zhou, Lilin Yi
Systems and Control
Physics-Informed Graph Learning Acceleration for Large-Scale AC-OPF with Topology Changes
Keunju Song, Kyungnam Park, Sua Choi et al. (+5 more)
In power systems, alternating current optimal power flow (AC-OPF) has been a challenging problem for decades due to its nonconvexity, but fast and efficient solutions are even more needed because of high penetration of large scale renewable generation and load growth. Recently, neural networks (NN) have gained
attention in solving AC-OPF, but it is still in an early stage to be applicable for real and large-scale power system operation with
topology-changing characteristics. To end this, we propose a novel framework called GraphOPF that considers
topology-adaptability, scalability, NN training time, self-supervision, and feasibility altogether. Extensive experiments show that the proposed framework against the baselines is up to 200 times faster in NN training and up to 66 times faster in solving AC-OPF for large-scale power systems including the real Korean power system, while achieving more than 99% feasibility.
Read the full abstract ▾
All authors Keunju Song, Kyungnam Park, Sua Choi, Seunguk Kim, Tae-un Kim, Youngmin Choi, Sang-Won Min, Hongseok Kim
Audio and Speech Processing
M2S-AVSR: Modality-aware Multi-view Self-supervised Representation for Robust Audio-Visual Speech Recognition
Fei Su, Cancan Li, Ming Li et al. (+1 more)
Audio-Visual
Speech Recognition (AVSR) enhances
speech recognition robustness by leveraging visual cues, while real-world scenarios remain challenging due to viewpoint variation, audio distortion, and visual occlusion, which degrade modality quality and increase audio-visual asynchrony. In this paper, we propose a novel Modality-aware Multi-view Self-supervised representation framework for robust Audio-Visual
Speech Recognition (M2S-AVSR). First, we introduce a multi-view representation learning encoder to learn view-invariant visual
speech representations. Next, we employ a modality-aware module that explicitly models modality quality and cross-modal synchrony to perform fine-grained modality-aware fusion, enabling fine-grained visual information injection during decoding. In addition, we release AISHELL8-RealScene, a public multi-scenario, multi-view conversational audio-visual
dataset recorded in real-world environments, and establish a
speech recognition
benchmark on it. Experiments on English and Mandarin
benchmarks demonstrate the effectiveness of the proposed method under challenging conditions. On LRS3, M2S-AVSR achieves up to 29.4% relative improvement under viewpoint perturbation and visual degradation settings. Our method also achieves new state-of-the-art performance on the MISP2021-AVSR test set. On AISHELL8-RealScene, it achieves the best result in outdoor scenes. The proposed method and
dataset provide useful support for future research on robust
speech and
multimodal tasks under realistic conditions.
Read the full abstract ▾
All authors Fei Su, Cancan Li, Ming Li, Juan Liu
Audio and Speech Processing · Cross-listed from cs.SD
SagnacAssisted Enhanced OTDR for Distributed Acoustic Sensing: A Standardized Benchmark and Engineering Evaluation Framework
Weiguang Wang, Fugen Wu, Hailing Wang et al. (+4 more)
Phase-sensitive optical time-domain reflectometry ($\phi$-OTDR) is widely used in large-scale distributed acoustic sensing (DAS) because it provides distributed spatiotemporal monitoring over long sensing distances. Its field performance can still deteriorate because of polarization-induced fading (PIF), local signal degradation, and strong environmental interference. This study develops a Sagnac-assisted enhanced $\phi$-OTDR sensing architecture and a standardized
benchmark framework for engineering-oriented DAS event recognition. The Sagnac interferometer provides a continuous phase response that supplements fading-prone observations in the $\phi$-OTDR channel, and heterogeneous signal
alignment is achieved using a cross-correlation procedure implemented on an FPGA platform. The
benchmark protocol compares conventional feature-engineering methods, probabilistic shallow classifiers, single-branch deep models, and dual-branch fusion models under consistent data partitioning, preprocessing, and metric definitions. Experiments on a 10-km sensing fiber with six representative acoustic event classes show that the dual-branch fusion model provides the most favorable trade-off among the evaluated methods, reaching 89.79% accuracy, 89.83% macro-F1, and a nuisance alarm rate of 5.00% on the balanced test set. The results also show that channel grouping strongly affects dual-branch evaluation, indicating that deployment-oriented conclusions should be based on accuracy, macro-F1, nuisance alarm rate, false negative rate, and latency rather than accuracy alone. This work provides a physically motivated enhancement strategy for $\phi$-OTDR-based DAS and a reproducible
benchmark protocol for future fusion-oriented sensing research. The implementation and scripts for reproducing the DAS event-recognition experiments are publicly available at https://github.com/wawa-abc/das.
Read the full abstract ▾
All authors Weiguang Wang, Fugen Wu, Hailing Wang, Xuechen Liang, Xiaobin Li, Ru Han, Tianchang Xie
Audio and Speech Processing · Cross-listed from cs.SD
Do speech foundation models perceive speaker similarity as humans do?
Minoru Kishi, Hayato Yagi, Shinnosuke Takamichi et al. (+1 more)
This study presents a comparative analysis between the speaker
embeddings of
speech foundation models and human subjective perception of speaker similarity. Human listeners have the ability to judge speaker similarity on a continuous scale discerning how similar two voices are. In contrast,
speech foundation models embed speaker characteristics into numerical representation. However, a question remains: does the numerical distance between speaker
embeddings in these models truly align with the similarity perceived by humans? To address this, we conduct a comprehensive investigation using more than 40 models to compare model-derived distances with human-perceived similarity scores. Furthermore, we identify which factors in model configuration contribute most to a speaker
embedding that mirrors human perception. Our findings provide insights for the development of more perceptually grounded
speech foundation models.
Read the full abstract ▾
All authors Minoru Kishi, Hayato Yagi, Shinnosuke Takamichi, Yuki Saito
Audio and Speech Processing
Enhancing Audio Captioning with Auxiliary AudioSet Semantics
Shubham Gupta, Adarsh Arigala and Sri Rama Murty Kodukula
Automatic Audio Captioning (AAC) seeks to generate natural
language descriptions of complex acoustic scenes, bridging auditory perception and
language understanding. However, word-selection indeterminacy and increasing reliance on large-scale sequence-to-sequence or LLM-based models limit practical deployment. We propose a resource-efficient AAC framework that explicitly grounds caption generation in auxiliary AudioSet semantics. Frame-level acoustic representations extracted using a ConvNeXt encoder are augmented with top-$K$ predicted AudioSet keywords, providing structured contextual cues for decoding. A compact six-layer BART-style decoder conditions on this joint acoustic-semantic representation, enabling caption generation without LLM-scale decoding. The proposed design balances semantic grounding and computational efficiency within a compact architecture. Evaluations on Clotho V2 and AudioCaps confirm competitive caption quality under practical deployment constraints.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.MM
Beyond Generative Decoding: Discriminative Hidden-State Readout from a Native Omni-Modal LLM for Multimodal Sentiment Analysis
Bin Wen and Tien-Ping Tan
Multimodal sentiment analysis (MSA) infers human affect from
language, acoustic, and visual signals. Recent methods increasingly adapt large
multimodal models (LMMs) via generative readout: prompting the model to emit a sentiment score as a text string. While convenient, this ties continuous regression to discrete autoregressive decoding, incurring unmeasured costs. We revisit this readout mechanism and propose a discriminative formulation built on the Thinker module of a native omni-modal LLM (Qwen2.5-Omni-7B). Instead of text decoding, we map the final-layer hidden state of the last non-padding
token to a continuous score via a lightweight regression head in a single forward pass. Using 4-bit quantization and low-rank adaptation (QLoRA), the entire 7B pipeline – including
video and audio processing – trains on a single consumer GPU (RTX 5090, 32 GB) with 10-21 GB peak
memory and 1.14% trainable parameters. Through a controlled comparison fixing the backbone, data, and LoRA configuration, we isolate the impact of the readout. On CMU-MOSI and CMU-MOSEI, our discriminative readout reaches state-of-the-art accuracy without task-specific feature engineering (MOSI: MAE 0.551, Corr 0.888; MOSEI: MAE 0.506, Corr 0.790) and exhibits strong multi-seed stability. In contrast, the generative readout – even after equivalent supervised training – more than doubles the mean absolute error, yields unparsable or out-of-range outputs (2.8% zero-shot), and suffers from higher latency. Modality ablations reveal a text-dominant regime on CMU-MOSI. Our findings indicate that how an LMM is read out is as consequential as how it is trained, demonstrating that a discriminative readout offers a more accurate, efficient, and reliable alternative for continuous MSA.
Read the full abstract ▾
Systems and Control
Tracking Control for a Dynamic Model of an Underwater Submersible
Matthew Hampsey, Pieter van Goor, Ravi Banavar et al. (+1 more)
Underwater vehicles are naturally modelled as rigid bodies on SE(3) subjected to added mass effects. The passivity of the
Hamiltonian structure of the system can be exploited to design energy-based stabilising controllers, however, the extension of these control designs to tracking control is not trivial since the error system for the classical error formulations is not itself
Hamiltonian. In this paper, we show that a novel choice of error function leads to error dynamics that are
Hamiltonian. We go on to derive an energy-based tracking control for a fully coupled model of a submersible vehicle. Asymptotic convergence of the control scheme is proved and the control is demonstrated in a simulation study of the Blue Robotics BlueROV2 Heavy submersible.
Read the full abstract ▾
All authors Matthew Hampsey, Pieter van Goor, Ravi Banavar, Robert Mahony
Systems and Control · Cross-listed from cs.RO
Accelerating and Scaling MPC-Guided Reinforcement Learning for Humanoid Locomotion and Manipulation
Junheng Li, Liang Wu, Sergio A. Esteban et al. (+3 more)
In
humanoid motion control, model predictive control (MPC) offers physically grounded prediction and constraint handling, while reinforcement learning (RL) enables robust whole-body skills through large-scale simulation. However, using MPC inside RL often requires time-consuming problem construction or excessive training overhead, making such frameworks difficult to justify in practice. This work studies efficient training-time MPC guidance for
humanoid locomotion and manipulation, termed MPC-RL. We introduce a centroidal-dynamics MPC reward formulation that leverages guidance from MPC trajectories in training time. To make this practical in massively parallel RL, we develop $\pi^n$MPC, a parallel-in-horizon and construction-free batched GPU MPC solver that operates directly on time-varying dynamics to avoid high
memory usage and pre-compilation. Through a variety of comparative studies and hardware validations, we have found that MPC-RL achieves superior performance in locomotion and manipulation skills. The code base is available at https://github.com/junhengl/mpc-rl.
Read the full abstract ▾
All authors Junheng Li, Liang Wu, Sergio A. Esteban, Lizhi Yang, Ján Drgoňa, Aaron D. Ames
Systems and Control · Cross-listed from cs.RO
Dynamic Multi-Agent Pickup and Delivery in Robotic Cellular Warehousing Systems
Cheng Ren, Ming Li, Xinping Guan et al. (+1 more)
Robotic cellular warehousing systems (RCWS) give rise to multi-
agent pickup and delivery (MAPD) processes in which
robots sequentially collect multiple stock-keeping units (SKUs) for each order. Unlike classical MAPD formulations that assume static tasks, real warehouse operations often involve dynamic order
evolution, where new SKUs may be appended to an order while it is being executed. Motivated by this practical requirement, this letter formulates the Dynamic-MAPD problem considering internal order
evolution for the first time. Building on the
token passing (TP) mechanism, we propose two event-triggered online replanning
algorithms. The two strategies target different
robot-resource configurations, depending on whether additional robotic resources are available for cooperative assistance. The first, Dynamic-TP, enables an event-triggered dynamic response by allowing
robots to replan from their current execution states through priority-aware
token acquisition after order updates. The second, Cooperative-TP, further enables reserved
robots to assist newly added SKUs while preserving the original order ownership. Simulation results demonstrate that the proposed methods significantly reduce order flowtime compared with static and non-cooperative baselines, thereby improving the order fulfillment efficiency in RCWS.
Read the full abstract ▾
All authors Cheng Ren, Ming Li, Xinping Guan, George Q. Huang
Systems and Control
Development of a Structured Approach for Establishing Mission Engineering Requirements
Taylor C. Fazzini and Daniel R. Herber
This paper addresses the question: How can mission effectiveness be systematically defined or approximated in the absence of customer requirements? Legacy requirements engineering frameworks presuppose customer input to define specifications but leave a gap in the process when stakeholder input is ill-defined or missing. Rapid build and development programs (such as military acquisition, space assets, infrastructure projects, etc.) often see requirement and objective evolutions throughout the proposal process, so a more adaptive method is needed. To address this gap, a structured approach is proposed that decomposes mission intent into mission context, functions, constraints, critical dimensions, effectiveness attributes, and architecture alternatives. This method conducts a mission feasibility assessment, prioritizes mission-critical dimensions using Best-Worst Scaling, and introduces a mission complexity factor to quantitatively understand the impacts of external mission difficulties, technology maturity, evidence and confidence standards, and mission utility. The resulting method provides a traceable basis for deriving Tier 1 and 2 requirements. The approach is structured to support future Unified Architecture Framework (UAF) and Systems Modeling
Language (SysML) artifact integration. The proposed framework is demonstrated using a notional close air support mission example.
Read the full abstract ▾
Signal Processing
3D Spherical Fluid Antennas for Spatially Reconfigurable Communications
Wenchi Cheng, Hongyun Jin and Zhuohui Yao
As sixth-generation (6G) wireless systems evolve toward higher frequency bands, large-scale
antenna arrays, and intelligent interaction with the wireless environment, conventional fixed-position
antennas (FPAs) are increasingly constrained by limited spatial degrees of freedom and insufficient hardware-level adaptability. Fluid
antenna systems (FAS) provide new physical-layer flexibility by dynamically reconfiguring
antenna ports, geometries, and radiation characteristics. However, existing studies have mainly focused on one- or two-dimensional apertures, leaving the spatial reconfigurability required for complex three-dimensional (3D) propagation environments insufficiently exploited. In this article, we present a 3D spherical fluid
antenna system (3D SFAS) architecture for flexible spatially reconfigurable communications. By activating radiating elements in different spherical regions, 3D SFAS realizes array-level spatial reconfiguration through flexible region switching. Within the selected regions, element-level reconfiguration further adjusts the effective aperture size, array
topology, and radiation characteristics. This joint framework enables flexible beamforming, concurrent multi-region transmission, blockage-adaptive aperture switching, effective-aperture reconfiguration, and high-resolution 3D aperture control. We also discuss its potential applications in space-air-ground integrated networks, high-mobility communications, integrated sensing and communication systems, and emergency communications. Numerical results demonstrate the potential of 3D SFAS to improve wireless communication performance through flexible spatial reconfiguration. Overall, 3D SFAS extends FAS design beyond 2D position switching toward comprehensive 3D spatial reconfigurability.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.SD
SB-RF: Schrödinger Bridge Rectified Flow for One-Step Robust Speech Enhancement
Caixia Lu, Xueyang Lv, Penglong Hu et al. (+1 more)
Generative models have shown promising results for
speech enhancement (SE), but they often rely on multi-step inference, limiting low-latency deployment. We propose SB-RF, a one-step generative framework that integrates Rectified Flow (RF) with Schrödinger Bridge (SB) theory. During training, SB-RF samples intermediate states from an SB time marginal and trains a conditional velocity field with the RF velocity-matching objective. At inference, SB-RF starts from the noisy observation and applies a single Euler update. Experiments show that SB-RF achieves competitive performance among generative methods on the VoiceBank-DEMAND
benchmark. To further assess performance beyond this standard setting, we evaluate SB-RF on a simulated low signal-to-noise ratio test set using an expanded training
dataset. Under these conditions, SB-RF achieves superior performance over the compared baselines, supporting its potential for real-world applications.
Read the full abstract ▾
All authors Caixia Lu, Xueyang Lv, Penglong Hu, Jiaming Xu
Audio and Speech Processing · Cross-listed from cs.SD
Sound Effects Dataset Unification With the Universal Category System
Jun Woo Beck and Alexander Lerch
Sound effects (SFX)
datasets and libraries often employ distinct tagging schemes, taxonomies, and metadata structures. This creates challenges for research on SFX classification and generation because incompatible taxonomies lead to siloed
datasets that might require individualized approaches, result in non-comparable outcomes, and prevent data merging strategies. We propose a modular
dataset relabeling framework that adopts the Universal Category System (UCS), an industry-standard hierarchical taxonomy for sound effects, as a shared structural foundation. This open-source framework enables us (i) to convert tags of existing
datasets to UCS with a rule-based multi-stage pipeline and conflict resolution to achieve high automatic conversion rates, (ii) to suggest a stratified
dataset split for the new labels, and (iii) to combine multiple
datasets. To showcase the practical utility, we introduce the EnvSound-UCS
dataset, a publicly available unified UCS-compliant
dataset of environmental sounds with 58,057 sound clips from three sources: AudioSet, FSD50K, and ESC-50.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.CL
Domain-Aware Mispronunciation Detection and Diagnosis Using Language-Specific Statistical Graphs
Huu Tuong Tu, Hanh Nguyen, Thien Van Luong et al. (+3 more)
Mispronunciation Detection and Diagnosis (MDD) has gained increasing importance in computer-assisted
language learning and
speech technology in recent years. In this paper, we propose a method for constructing statistical graphs that enable models to learn phoneme confusion patterns represented as directed graphs. Furthermore, we introduce a
language-specific strategy to capture systematic pronunciation differences across various native
language (L1) backgrounds. The effectiveness of our approach is demonstrated through extensive experiments on the L2-ARCTIC
benchmark, where it achieves an F1-score of 59.52%, outperforming several competitive baselines.
Read the full abstract ▾
All authors Huu Tuong Tu, Hanh Nguyen, Thien Van Luong, Nguyen Tien Cuong, Vu Huan, Nguyen Thi Thu Trang
Audio and Speech Processing · Cross-listed from cs.SD
Probing Spatial Structure in Pretrained Audio Representations
Chuyang Chen, Sivan Ding, Adrian S. Roman et al. (+1 more)
Pretrained spatial audio encoders are increasingly used as general-purpose representations for perceptual tasks, yet their spatial encoding capabilities remain poorly understood. We introduce the Spatial Audio Representation Learning (SARL)
benchmark, a controlled framework for evaluating spatial information in pretrained audio models. SARL probes source-level factors (azimuth, elevation, distance, class) and room-level factors (RT60, volume, shape). Experiments across diverse encoders reveal three patterns: input configuration and training paradigm shape spatial encoding; source factors are consistently easier to decode than room factors; and sensitivity analysis under controlled perturbations shows heterogeneous responses to source and room variation. These results reveal systematic biases in current pretrained audio representations. SARL is released as an open-source
benchmark for reproducible evaluation of spatial audio representations.
Read the full abstract ▾
All authors Chuyang Chen, Sivan Ding, Adrian S. Roman, Juan P. Bello
Systems and Control
Learning-Assisted Day-Ahead Energy Scheduling for Frequency-Secure Inverter-Dominated Grids with Grid-Forming Battery Energy Storage Systems
Fan Jiang and Xingpeng Li
As grid-forming (GFM) battery energy storage systems (BESS) are increasingly deployed to enhance power system inertial response and frequency stability, incorporating their frequency support capabilities into day-ahead energy scheduling (DAES) is essential for achieving both frequency security and operational efficiency. However, accurately determining frequency metrics in grids with coexisting GFM inverters and synchronous generators requires electromagnetic transient (EMT) simulations, which are computationally prohibitive for direct
embedding in grid operational optimization models. To bridge the gap between modeling accuracy and computational efficiency, a learning-assisted DAES (LA-DAES) framework is proposed in this work. By leveraging a surrogate model to represent the frequency support dynamics of GFM BESS, the proposed framework ensures frequency security with a reasonable solve time. Comparative results demonstrate that, relative to analytical frequency-constrained DAES, the proposed LA-DAES framework more accurately captures grid frequency metrics and improves the utilization of GFM BESS.
Read the full abstract ▾
Systems and Control
Constrained Deep Reinforcement Learning for Cognitive Radar Resource Management
Ziyang Lu, M. Cenk Gursoy, Chilukuri K. Mohan et al. (+1 more)
In this paper, multi-target tracking and scanning are considered in a radar system operating in the track-while-scan mode. Specifically, time allocation for radar scanning and tracking of multiple maneuvering targets under a time budget constraint is addressed, aiming to jointly optimize the performance of both tracking and scanning in a cognitive radar. We first present the details of the model for tracking and scanning and formulate the time management task as a constrained optimization problem. Subsequently, we design a \gls{cdrl} framework to find the time allocation strategy for the problem. In the proposed \gls{cdrl} framework, the parameters of the neural networks and the dual variable are learned simultaneously. The deep deterministic policy gradient (DDPG)
algorithm is introduced to tackle continuous action space and its performance is compared with deep Q-learning, heuristic approaches, and an optimization-based approach. Numerical results show that the radar with the proposed \gls{cdrl} framework can autonomously allocate more time to the tracking task that requires greater
attention while providing time for scanning and also constraining the total time budget below the predefined threshold.
Read the full abstract ▾
All authors Ziyang Lu, M. Cenk Gursoy, Chilukuri K. Mohan, Pramod K. Varshney