Peer Revenue Now accepting positions

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science

eess Watch Electrical Engineering and Systems Science (sign in to use)

Thursday 22 January

Audio and Speech Processing · Cross-listed from cs.SD

Sink or SWIM: Tackling Real-Time ASR at Scale

Real-time automatic speech recognition systems are increasingly integrated into interactive applications, from voice assistants to live transcription services. However, scaling these systems to support multiple concurrent clients while maintaining low latency and high accuracy remains a major challenge. In this work, we present SWIM, a novel real-time ASR system built on top of OpenAI's Whisper model that enables true model-level parallelization for scalable, multilingual transcription. SWIM supports multiple concurrent audio streams without modifying the underlying model. It introduces a buffer merging strategy that maintains transcription fidelity while ensuring efficient resource usage. We evaluate SWIM in multi-client settings – scaling up to 20 concurrent users – and show that it delivers accurate real-time transcriptions in English, Italian, and Spanish, while maintaining low latency and high throughput. While Whisper-Streaming achieves a word error rate of approximately 8.2% with an average delay of approximately 3.4 s in a single-client, English-only setting, SWIM extends this capability to multilingual, multi-client environments. It maintains comparable accuracy with significantly lower delay – around 2.4 s with 5 clients – and continues to scale effectively up to 20 concurrent clients without degrading transcription quality and increasing overall throughput. Our approach advances scalable ASR by improving robustness and efficiency in dynamic, multi-user environments.
Read the full abstract

All authors Federico Bruzzone, Walter Cazzola, Matteo Brancaleoni, Dario Pellegrino

Systems and Control · Cross-listed from cs.GT

Game-to-Real Gap: Quantifying the Effect of Model Misspecification in Network Games

Game-theoretic models and solution concepts provide rigorous tools for predicting collective behavior in multi-agent systems. In practice, however, different agents may rely on different game-theoretic models to design their strategies. As a result, when these heterogeneous models interact, the realized outcome can deviate substantially from the outcome each agent expects based on its own local model. In this work, we introduce the game-to-real gap, a new metric that quantifies the impact of such model misspecification in multi-agent environments. The game-to-real gap is defined as the difference between the utility an agent actually obtains in the multi-agent environment (where other agents may have misspecified models) and the utility it expects under its own game model. Focusing on quadratic network games, we show that misspecifications in either (i) the external shock or (ii) the player interaction network can lead to arbitrarily large game-to-real gaps. We further develop novel network centrality measures that allow exact evaluation of this gap in quadratic network games. Our analysis reveals that standard network centrality measures fail to capture the effects of model misspecification, underscoring the need for new structural metrics that account for this limitation. Finally, through illustrative numerical experiments, we show that existing centrality measures in network games may provide a counterintuitive understanding of the impact of model misspecification.
Read the full abstract

All authors Bryce L. Ferguson, Chinmay Maheshwari, Manxi Wu, Shankar Sastry

Image and Video Processing

Experience with Single Domain Generalization in Real World Medical Imaging Deployments

A desirable property of any deployed artificial intelligence is generalization across domains, i.e. data generation distribution under a specific acquisition condition. In medical imagining applications the most coveted property for effective deployment is Single Domain Generalization (SDG), which addresses the challenge of training a model on a single domain to ensure it generalizes well to unseen target domains. In multi-center studies, differences in scanners and imaging protocols introduce domain shifts that exacerbate variability in rare class characteristics. This paper presents our experience on SDG in real life deployment for two exemplary medical imaging case studies on seizure onset zone detection using fMRI data, and stress electrocardiogram based coronary artery detection. Utilizing the commonly used application of diabetic retinopathy, we first demonstrate that state-of-the-art SDG techniques fail to achieve generalized performance across data domains. We then develop a generic expert knowledge integrated deep learning technique DL+EKE and instantiate it for the DR application and show that DL+EKE outperforms SOTA SDG methods on DR. We then deploy instances of DL+EKE technique on the two real world examples of stress ECG and resting state (rs)-fMRI and discuss issues faced with SDG techniques.
Read the full abstract

Audio and Speech Processing

TidyVoice: A Curated Multilingual Dataset for Speaker Verification Derived from Common Voice

The development of robust, multilingual speaker recognition systems is hindered by a lack of large-scale, publicly available and multilingual datasets, particularly for the read-speech style crucial for applications like anti-spoofing. To address this gap, we introduce the TidyVoice dataset derived from the Mozilla Common Voice corpus after mitigating its inherent speaker heterogeneity within the provided client IDs. TidyVoice currently contains training and test data from over 212,000 monolingual speakers (Tidy-M) and around 4,500 multilingual speakers (Tidy-X) from which we derive two distinct conditions. The Tidy-M condition contains target and non-target trials from monolingual speakers across 81 languages. The Tidy-X condition contains target and non-target trials from multilingual speakers in both same- and cross-language trials. We employ two architectures of ResNet models, achieving a 0.35% EER by fine-tuning on our comprehensive Tidy-M partition. Moreover, we show that this fine-tuning enhances the model's generalization, improving performance on unseen conversational interview data from the CANDOR corpus. The complete dataset, evaluation trials, and our models are publicly released to provide a new resource for the community.
Read the full abstract

All authors Aref Farhadipour, Jan Marquenie, Srikanth Madikeri, Eleanor Chodroff

Signal Processing · Cross-listed from cs.LG

Efficient Gaussian process learning via subspace projections

We propose a novel training objective for GPs constructed using lower-dimensional linear projections of the data, referred to as projected likelihood (PL). We provide a closed-form expression for the information loss related to the PL and empirically show that it can be reduced with random projections on the unit sphere. We show the superiority of the PL, in terms of accuracy and computational efficiency, over the exact GP training and the variational free energy approach to sparse GPs over different optimisers, kernels and datasets of moderately large sizes.
Read the full abstract

Systems and Control · Cross-listed from cs.NI

Multi-User Content Diversity in Wireless Networks

Immersive applications such as eXtended Reality (XR), cloud gaming, and real-time video streaming are central to the vision of 6G networks. These applications require not only low latency and high data rates, but also consistent and high-quality User Experience (UX). Traditional rate allocation and congestion control mechanisms in wireless networks treat users uniformly based on channel conditions, rely only on network-centric Key Performance Indicators (KPIs), and ignore the content diversity, which can lead to inefficient resource utilization and degraded UX. In this paper, we introduce the concept of Multi-User Content Diversity, which recognizes that different users concurrently consume media with varying complexity, and therefore have different bitrate requirements to achieve satisfactory UX. We propose multiple different frameworks that exploit multi-user content diversity and lead to overall network-wide gains in terms of UX. For each framework, we demonstrate the required information exchange between Application Servers (ASs), Application Clients (ACs), and the network, and the algorithms that run in each of these components to optimize a network-wide UXbased objective. Simulation results demonstrate that exploiting multi-user content diversity leads to significant gains in UX capacity, UX fairness, and network utilization, when compared to conventional rate control methods. These findings highlight the potential of content-aware networking as a key enabler for emerging wireless systems.
Read the full abstract

All authors Belal Korany, Peerapol Tinnakornsrisuphap, Saadallah Kassir, Prashanth Hande, Hyun Yong Lee, Thomas Stockhammer, Hemanth Sampath

Audio and Speech Processing

EdgeSpot: Efficient and High-Performance Few-Shot Model for Keyword Spotting

We introduce an efficient few-shot keyword spotting model for edge devices, EdgeSpot, that pairs an optimized version of a BC-ResNet-based acoustic backbone with a trainable Per-Channel Energy Normalization frontend and lightweight temporal self-attention. Knowledge distillation is utilized during training by employing a self-supervised teacher model, optimized with Sub-center ArcFace loss. This study demonstrates that the EdgeSpot model consistently provides better accuracy at a fixed false-alarm rate (FAR) than strong BC-ResNet baselines. The largest variant, EdgeSpot-4, improves the 10-shot accuracy at 1% FAR from 73.7% to 82.0%, which requires only 29.4M MACs with 128k parameters.
Read the full abstract

Systems and Control

Optimal County-Level Siting of Data Centers in the United States

Data centers are growing rapidly, creating the pressing need for the development of critical infrastructure build out to support these resource-intensive large loads. Their immense consumption of electricity and, often, freshwater, continues to stress an already constrained and aging power grid and water resources. This paper presents a comprehensive modeling approach to determine the optimal locations to construct such facilities by quantifying their resource use and minimizing associated costs. The interdisciplinary modeling approach incorporates a number of factors including the power grid, telecommunications, climate, water use, and collocated generation potential. This work establishes the base model whose functionality is shown through several test cases focusing on carbon-free generation collocation on a county-level in the United States. The results suggest that while capital costs are the biggest driver, having a longer future outlook and allowing more variable generation collocation influences the model to choose sites with higher renewable potential.
Read the full abstract

All authors Maria Vabson, Muhy Eddin Zater, Amir Sajadi, Kyri Baker, Bri-Mathias Hodge

Signal Processing

Angle of Arrival Estimation for Gesture Recognition from reflective body-worn tags

We investigate hand gesture recognition by leveraging passive reflective tags worn on the body. Considering a large set of gestures, distinct patterns are difficult to be captured by learning algorithms using backscattered received signal strength (RSS) and phase signals. This is because these features often exhibit similarities across signals from different gestures. To address this limitation, we explore the estimation of Angle of Arrival (AoA) as a distinguishing feature, since AoA characteristically varies during body motion. To ensure reliable estimation in our system, which employs Smart Antenna Switching (SAS), we first validate AoA estimation using the Multiple SIgnal Classification (MUSIC) algorithm while the tags are fixed at specific angles. Building on this, we propose an AoA tracking method based on Kalman smoothing. Our analysis demonstrates that, while RSS and phase alone are insufficient for distinguishing certain gesture data, AoA tracking can effectively differentiate them. To evaluate the effectiveness of AoA tracking, we implement gesture recognition system benchmarks and show that incorporating AoA features significantly boosts their performance. Improvements of up to 15% confirm the value of AoA-based enhancement.
Read the full abstract

All authors Sahar Golipoor, Reza Ghazalian, Ines Lobato Mesquita, Stephan Sigg

Signal Processing

Gesture Recognition from body-Worn RFID under Missing Data

We explore hand-gesture recognition through the use of passive body-worn reflective tags. A data processing pipeline is proposed to address the issue of missing data. Specifically, missing information is recovered through linear and exponential interpolation and extrapolation. Furthermore, imputation and proximity-based inference are employed. We represent tags as nodes in a temporal graph, with edges formed based on correlations between received signal strength (RSS) and phase values across successive timestamps, and we train a graph-based convolutional neural network that exploits graph-based self-attention. The system outperforms state-of-the-art methods with an accuracy of 98.13% for the recognition of 21 gestures. We achieve 89.28% accuracy under leave-one-person-out cross-validation. We further investigate the contribution of various body locations on the recognition accuracy. Removing tags from the arms reduces accuracy by more than 10%, while removing the wrist tag only reduces accuracy by around 2%. Therefore, tag placements on the arms are more expressive for gesture recognition than on the wrist.
Read the full abstract

All authors Sahar Golipoor, Richard T. Brophy, Ying Liu, Reza Ghazalian, Stephan Sigg

Systems and Control · Cross-listed from cs.MA

AMBER: A Columnar Architecture for High-Performance Agent-Based Modeling in Python

Python is widely used for agent-based modelling because it is accessible and has a mature scientific ecosystem, but object-per-agent execution incurs interpreter overhead that restricts the population sizes feasible in interactive modelling, calibration, and parameter sweeps. This paper presents AMBER, a Python framework that stores agent state in a Polars-backed columnar table and exposes population operations through a compact view API. The framework preserves conventional model and agent abstractions while translating common population updates into compiled column operations; behaviours that do not vectorise remain expressible through a buffered object-oriented path. We evaluate AMBER on wealth transfer, random walk, and spatial SIR benchmarks against Mesa, AgentPy, SimPy, Melodie, Agents.jl, and AMBER's own loop path, using invariant checks to verify comparable model outputs before timing. Across the tested workloads, AMBER has the lowest execution time among Python-hosted implementations and achieves speedups of up to $1118\times$ over Mesa; on the largest SIR benchmark it is also faster than the Julia-based Agents.jl implementation.
Read the full abstract

Systems and Control · Cross-listed from math.OC

Maximizing Reach-Avoid Probabilities for Linear Stochastic Systems via Control Architectures

The maximization of reach-avoid probabilities for stochastic systems is a central topic in the control literature. Yet, the available methods are either restricted to low-dimensional systems or suffer from conservative approximations. To address these limitations, we propose control architectures that combine the flexibility of Markov Decision Processes with the scalability of Model Predictive Controllers. The Model Predictive Controller tracks reference signals while remaining agnostic to the stochasticity and reach-avoid objective. Instead, the reach-avoid probability is maximized by optimally updating the controller's reference online. To achieve this, the closed-loop system, consisting of the system and Model Predictive Controller, is abstracted as a Markov Decision Process in which a new reference can be chosen at every time-step. A feedback policy generating optimal references is then computed via Dynamic Programming. If the state space of the system is continuous, the Dynamic Programming algorithm must be executed on a finite system approximation. Modifications to the Model Predictive Controller enable a computationally efficient robustification of the Dynamic Programming algorithm to approximation errors, preserving bounds on the achieved reach-avoid probability. The approach is validated on a perturbed 12D quadcopter model in cluttered reach-avoid environments proving its flexibility and scalability.
Read the full abstract

All authors Niklas Schmid, Jaeyoun Choi, Oswin So, Chuchu Fan

Systems and Control

BESTOpt: A Modular, Physics-Informed Machine Learning based Building Modeling, Control and Optimization Framework

Modern buildings are increasingly interconnected with occupancy, heating, ventilation, and air-conditioning (HVAC) systems, distributed energy resources (DERs), and power grids. Modeling, control, and optimization of such multi-domain systems play a critical role in achieving building-sector decarbonization. However, most existing tools lack scalability and physical consistency for addressing these complex, multi-scale ecosystem problems. To bridge this gap, this study presents BESTOpt, a modular, physics-informed machine learning (PIML) framework that unifies building applications, including benchmarking, evaluation, diagnostics, control, optimization, and performance simulation. The framework adopts a cluster-domain-system/building-component hierarchy and a standardized state-action-disturbance-observation data typology. By embedding physics priors into data-driven modules, BESTOpt improves model accuracy and physical consistency under unseen conditions. Case studies on single-building and cluster scenarios demonstrate its capability for multi-level centralized and decentralized control. Looking ahead, BESTOpt lays the foundation for an open, extensible platform that accelerates interdisciplinary research toward smart, resilient, and decarbonized building ecosystems.
Read the full abstract

All authors Zixin Jiang, Ruizhi Song, Guowen Li, Yuhang Zhang, Zheng O'Neill, Xuezheng Wang, Judah Goldfeder, Bing Dong

Audio and Speech Processing · Cross-listed from cs.SD

The CMU-AIST submission for the ICME 2025 Audio Encoder Challenge

This technical report describes our submission to the ICME 2025 audio encoder challenge. Our submitted system is built on BEATs, a masked speech token prediction based audio encoder. We extend the BEATs model using 74,000 hours of data derived from various speech, music, and sound corpora and scale its architecture upto 300 million parameters. We experiment with speech-heavy and balanced pre-training mixtures to study the impact of different domains on final performance. Our submitted system consists of an ensemble of the Dasheng 1.2 billion model with two custom scaled-up BEATs models trained on the aforementioned pre-training data mixtures. We also propose a simple ensembling technique that retains the best capabilities of constituent models and surpasses both the baseline and Dasheng 1.2B. For open science, we publicly release our trained checkpoints via huggingface at https://huggingface.co/shikhar7ssu/OpenBEATs-ICME-SOUND and https://huggingface.co/shikhar7ssu/OpenBEATs-ICME.
Read the full abstract

All authors Shikhar Bharadwaj, Samuele Cornell, Kwanghee Choi, Hye-jin Shim, Soham Deshmukh, Satoru Fukayama, Shinji Watanabe

Systems and Control

Stochastic Control Barrier Functions under State Estimation: From Euclidean Space to Lie Groups

Ensuring safety for autonomous systems under uncertainty remains challenging, particularly when safety of the true state is required despite the true state not being fully known. Control barrier functions (CBFs) have become widely adopted as safety filters. However, standard CBF formulations do not explicitly account for state estimation uncertainty and its propagation, especially for stochastic systems evolving on manifolds. In this paper, we propose a safety-critical control framework with a provable bound on the finite-time safety probability for stochastic systems under noisy state information. The proposed framework explicitly incorporates the uncertainty arising from both process and measurement noise, and synthesizes controllers that adapt to the level of uncertainty. The framework admits closed-form solutions in linear settings, and experimental results demonstrate its effectiveness on systems whose state spaces range from Euclidean space to Lie groups.
Read the full abstract

Systems and Control · Cross-listed from cs.MA

Average Unfairness in Routing Games

We propose average unfairness as a new measure of fairness in routing games, defined as the ratio between the average latency and the minimum latency experienced by users. This measure is a natural complement to two existing unfairness notions: loaded unfairness, which compares maximum and minimum latencies of routes with positive flow, and user equilibrium (UE) unfairness, which compares maximum latency with the latency of a Nash equilibrium. We show that the worst-case values of all three unfairness measures coincide and are characterized by a steepness parameter intrinsic to the latency function class. We show that average unfairness is always no greater than loaded unfairness, and the two measures are equal only when the flow is fully fair. Besides that, we offer a complete comparison of the three unfairness measures, which, to the best of our knowledge, is the first theoretical analysis in this direction. Finally, we study the constrained system optimum (CSO) problem, where one seeks to minimize total latency subject to an upper bound on unfairness. We prove that, for the same tolerance level, the optimal flow under an average unfairness constraint achieves lower total latency than any flow satisfying a loaded unfairness constraint. We show that such improvement is always strict in parallel-link networks and establish sufficient conditions for general networks. We further illustrate the latter with numerical examples. Our results provide theoretical guarantees and valuable insights for evaluating fairness-efficiency tradeoffs in network routing.
Read the full abstract

All authors Pan-Yang Su, Arwa Alanqary, Bryce L. Ferguson, Manxi Wu, Alexandre M. Bayen, Shankar Sastry

Systems and Control

Interconnection-based Model Reduction for Linear Hybrid Systems

In this paper, we address the model reduction problem for linear hybrid systems via the interconnection-based technique called moment matching. We consider two classical interconnections, namely the direct and swapped interconnections, in the hybrid setting, and we present families of reduced-order models for each interconnection via a hybrid characterisation of the steady-state responses. By combining the results for each interconnection, the design of a reduced-order model that achieves moment matching simultaneously for both interconnections is studied. In addition, we show that the presented results have simplified counterparts when the jumps of the hybrid system are periodic. A numerical simulation is finally given to illustrate the results.
Read the full abstract

Systems and Control · Cross-listed from cs.RO

Improve the autonomy of the SE2(3) group based Extended Kalman Filter for Integrated Navigation: Application

One of the core advantages of SE2(3) Lie group framework for navigation modeling lies in the autonomy of error propagation. In the previous paper, the theoretical analysis of autonomy property of navigation model in inertial, earth and world frames was given. A construction method for SE2(3) group navigation model is proposed to improve the non-inertial navigation model toward full autonomy. This paper serves as a counterpart to previous paper and conducts the real-world strapdown inertial navigation system (SINS)/odometer(ODO) experiments as well as Monte-Carlo simulations to demonstrate the performance of improved SE2(3) group based high-precision navigation models.
Read the full abstract

All authors Maosong Wang, Jiarui Cui, Wenqi Wu, Peiqi Li, Xianfei Pan

Audio and Speech Processing

Loose coupling of spectral and spatial models for multi-channel diarization and enhancement of meetings in dynamic environments

Sound capture by microphone arrays opens the possibility to exploit spatial, in addition to spectral, information for diarization and signal enhancement, two important tasks in meeting transcription. However, there is no one-to-one mapping of positions in space to speakers if speakers move. Here, we address this by proposing a novel joint spatial and spectral mixture model, whose two submodels are loosely coupled by modeling the relationship between speaker and position index probabilistically. Thus, spatial and spectral information can be jointly exploited, while at the same time allowing for speakers speaking from different positions. Experiments on the LibriCSS data set with simulated speaker position changes show great improvements over tightly coupled subsystems.
Read the full abstract

All authors Adrian Meise, Tobias Cord-Landwehr, Christoph Boeddeker, Marc Delcroix, Tomohiro Nakatani, Reinhold Haeb-Umbach

Image and Video Processing

Phi-SegNet: Phase-Integrated Supervision for Medical Image Segmentation

Deep learning has substantially advanced medical image segmentation, yet achieving robust generalization across diverse imaging modalities and anatomical structures remains a major challenge. A key contributor to this limitation lies in how existing architectures, ranging from CNNs to Transformers and their hybrids, primarily encode spatial information while overlooking frequency-domain representations that capture rich structural and textural cues. Although recent studies have begun exploring spectral information at the feature level, supervision-level integration of frequency cues-crucial for fine-grained object localization-remains largely untapped. To this end, we propose Phi-SegNet, a CNN-based architecture that incorporates phase-aware information at both architectural and optimization levels. The network integrates Bi-Feature Mask Former (BFMF) modules that blend neighboring encoder features to reduce semantic gaps, and Reverse Fourier Attention (RFA) blocks that refine decoder outputs using phase-regularized features. A dedicated phase-aware loss aligns these features with structural priors, forming a closed feedback loop that emphasizes boundary precision. Evaluated on five public datasets spanning X-ray, US, histopathology, MRI, and colonoscopy, Phi-SegNet consistently achieved state-of-the-art performance, with an average relative improvement of 1.54+/-1.26% in IoU and 0.98+/-0.71% in F1-score over the next best-performing model. In cross-dataset generalization scenarios involving unseen datasets from the known domain, it also exhibits robust and superior performance, highlighting its adaptability and modality-agnostic design. These findings demonstrate the potential of leveraging spectral priors in both feature representation and supervision, paving the way for generalized segmentation frameworks that excel in fine-grained object localization. Our code is available on GitHub.
Read the full abstract

Systems and Control · Cross-listed from cs.RO

Improve the autonomy of the SE2(3) group based Extended Kalman Filter for Integrated Navigation: Theoretical Analysis

One of core advantages of the SE2(3) Lie group framework for navigation modeling lies in the autonomy of error propagation. Current research on Lie group based extended Kalman filters has demonstrated that error propagation autonomy holds in low-precision applications, such as in micro electromechanical system (MEMS) based integrated navigation without considering earth rotation and inertial device biases. However, in high-precision navigation state estimation, maintaining autonomy is extremely difficult when considering with earth rotation and inertial device biases. This paper presents the theoretical analysis on the autonomy of SE2(3) group based high-precision navigation models under inertial, earth and world frame respectively. Through theoretical analysis, we find that the limitation of the traditional, trivial SE2(3) group navigation modeling method is that the presence of Coriolis force terms introduced by velocity in non-inertial frame. Therefore, a construction method for SE2(3) group navigation models is proposed, which brings the navigation models closer to full autonomy.
Read the full abstract

All authors Jiarui Cui, Maosong Wang, Wenqi Wu, Peiqi Li, Xianfei Pan

Systems and Control

Dynamic Tactile Sensing System and Soft Actor Critic Reinforcement Learning for Inclusion Characterization

This paper presents the Dynamic Tactile Sensing System that utilizes robotic tactile sensing in conjunction with reinforcement learning to locate and characterize embedded inclusions. A dual arm robot is integrated with an optical Tactile Imaging Sensor that utilizes the Soft Actor Critic Algorithm to acquire tactile data based on a pixel intensity reward. A Dynamic Interrogation procedure for tactile exploration is developed that enables the robot to first localize inclusion and refine their positions for precise imaging. Experimental validation conducted on Polydimethylsiloxane phantoms demonstrates that the robot using the Tactile Soft Actor Critic Model was able to achieve size estimation errors of 2.61% and 5.29% for soft and hard inclusions compared to 7.84% and 6.87% for expert human operators. Results also show that Dynamic Tactile Sensing System was able to locate embedded inclusions and autonomously determine their mechanical properties, useful in applications such as breast tumor characterization.
Read the full abstract

Signal Processing

Hybrid Channel Estimation with Quantized Phase Feedback for Over-the-Air Computation

To reduce the signaling overhead of over-the-air computation, a hybrid channel estimation scheme is proposed, where reciprocity-based and feedback-based channel estimation are combined. In particular, the impact of quantized phase-feedback is studied while the amplitude is assumed estimated exactly. The scheme enables selecting the estimation precision of amplitude and phase separately, depending on the importance of each. Two variants of the scheme are proposed: As shown through simulations and theory, the second variant with reciprocity-based estimation of the channel phase, and optimal quantization of phase feedback, can outperform the first variant estimating the phase by feedback only.
Read the full abstract

Audio and Speech Processing

Timbre-Aware LLM-based Direct Speech-to-Speech Translation Extendable to Multiple Language Pairs

Direct Speech-to-Speech Translation (S2ST) has gained increasing attention for its ability to translate speech from one language to another, while reducing error propagation and latency inherent in traditional cascaded pipelines. However, existing direct S2ST systems continue to face notable challenges, including instability in semantic-acoustic alignment when parallel speech data is scarce, difficulty in preserving speaker identity, and limited multilingual scalability. In this work, we introduce DS2ST-LM, a scalable, single-stage direct S2ST framework leveraging a multilingual Large Language Model (LLM). The architecture integrates a Whisper speech encoder, a learnable projection module, a Qwen2-0.5B LLM, and a timbre-controlled vocoder. We construct GigaS2S-1000, a 1000-hour bilingual corpus by extending the GigaST dataset with high-fidelity synthetic target speech, and show that this synthetic data alleviates data scarcity to some extent. We investigate two semantic token generation strategies: speech-derived S3 tokens and text-derived tokens generated by a pre-trained LLM, and analyze their impact on training stability and semantic consistency. We further evaluate three projection architectures (Linear, Conv1D-Linear, and Q-Former) and observe that while higher-capacity projectors converge faster, the simple Linear projector achieves higher performance. Extensive experiments demonstrate that DS2ST-LM outperforms traditional cascaded and ST (Qwen-Audio) + TTS baselines across both lexical (BLEU, METEOR) and semantic (BLEURT, COMET) metrics, while extending to multiple language pairs, including French, Spanish, German, Hindi, Bengali, and Urdu. Furthermore, we incorporate timbre-aware speech synthesis to preserve speaker information, enabling DS2ST-LM to surpass prior direct S2ST systems in both speaker similarity and perceptual naturalness.
Read the full abstract

All authors Lalaram Arya, Mrinmoy Bhattacharjee, Adarsh C. R., S. R. Mahadeva Prasanna

Systems and Control

Stability Analysis of Power-Electronics-Dominated Grids Using Scaled Relative Graphs

This paper presents a novel approach to stability analysis for grid-connected converters utilizing Scaled Relative Graphs (SRG). Our method effectively decouples grid and converter dynamics, thereby establishing a comprehensive and efficient framework for evaluating closed-loop stability. Our analysis accommodates both linear and non-linear loads, enhancing its practical applicability. Furthermore, we demonstrate that our stability assessment remains unaffected by angular variations resulting from dq-frame transformations, significantly increasing the method's robustness and versatility. The effectiveness of our approach is validated in several simulation case studies, which illustrate its broad applicability in modern power systems.
Read the full abstract

Signal Processing

Low-Complexity Sparse Superimposed Coding for Ultra Reliable Low Latency Communications

Sparse superimposed coding (SSC) has emerged as a promising technique for short-packet transmission in ultra-reliable low-latency communication scenarios. However, conventional SSC schemes often suffer from high encoding and decoding complexity due to the use of dense codebook matrices. In this paper, we propose a low-complexity SSC scheme by designing a sparse codebook structure, where each codeword contains only a small number of non-zero elements. The decoding is performed using the traditional multipath matching pursuit algorithm, and the overall complexity is significantly reduced by exploiting the sparsity of the codebook. Simulation results show that the proposed scheme achieves a favorable trade-off between BLER performance and computational complexity, and exhibits strong robustness across different transmission block lengths.
Read the full abstract

All authors Yanfeng Zhang, Xi'an Fan, Xu Zhu, Jinkai Zheng, Hui Liang, Weiwei Yang, Tom H. Luan

Image and Video Processing

THOR: A Versatile Foundation Model for Earth Observation Climate and Society Applications

Current Earth observation foundation models are architecturally rigid, struggle with heterogeneous sensors and are constrained to fixed patch sizes. This limits their deployment in real-world scenarios requiring flexible computeaccuracy trade-offs. We propose THOR, a "computeadaptive" foundation model that solves both input heterogeneity and deployment rigidity. THOR is the first architecture to unify data from Copernicus Sentinel-1, -2, and -3 (OLCI & SLSTR) satellites, processing their native 10 m to 1000 m resolutions in a single model. We pre-train THOR with a novel randomized patch and input image size strategy. This allows a single set of pre-trained weights to be deployed at inference with any patch size, enabling a dynamic trade-off between computational cost and feature resolution without retraining. We pre-train THOR on THOR Pretrain, a new, large-scale multi-sensor dataset and demonstrate state-of-the-art performance on downstream benchmarks, particularly in data-limited regimes like the PANGAEA 10% split, validating that THOR's flexible feature generation excels for diverse climate and society applications.
Read the full abstract

All authors Theodor Forgaard, Jarle H. Reksten, Anders U. Waldeland, Valerio Marsocci, Nicolas Longépé, Michael Kampffmeyer, Arnt-Børre Salberg

Systems and Control · Cross-listed from math.OC

Time-Optimal Switching Surfaces for Triple Integrator under Full Box Constraints

Time-optimal control for triple integrator under full box constraints is a fundamental problem in the field of optimal control, which has been widely applied in the industry. However, scenarios involving asymmetric constraints, non-stationary boundary conditions, and active position constraints pose significant challenges. This paper provides a complete characterization of time-optimal switching surfaces for the problem, leading to novel insights into the geometric structure of the optimal control. The active condition of position constraints is derived, which is absent from the literature. An efficient algorithm is proposed, capable of planning time-optimal trajectories under asymmetric full constraints and arbitrary boundary states, with a 100% success rate. Computational time for each trajectory is within approximately 10$\mu$s, achieving a 5-order-of-magnitude reduction compared to optimization-based baselines.
Read the full abstract

Signal Processing

A Covariance Matching Approach to Graph Topology Identification

Graph topology identification (GTI) is a central challenge in networked systems, where the underlying structure is often hidden, yet nodal data are available. Conventional solutions to address these challenges rely on probabilistic models or complex optimization formulations, commonly suffering from non-convexity or requiring restrictive assumptions on acyclicity or positivity. In this paper, we propose a novel covariance matching (CovMatch) framework that directly aligns the empirical covariance of the observed data with the theoretical covariance implied by an underlying graph. We show that as long as the data-generating process permits an explicit covariance expression, CovMatch offers a unified route to topology inference. We showcase our methodology on linear structural equation models (SEMs), showing that CovMatch naturally handles both undirected and general sparse directed graphs - whether acyclic or positively weighted - without explicit knowledge of these structural constraints. Through appropriate reparameterizations, CovMatch simplifies the graph learning problem to either a conic mixed integer program for undirected graphs or an orthogonal matrix optimization for directed graphs. Numerical results confirm that, even for relatively large graphs, our approach efficiently recovers the true topology and outperforms standard baselines in accuracy. These findings highlight CovMatch as a powerful alternative to log-determinant or Bayesian methods for GTI, paving the way for broader research on learning complex network topologies with minimal assumptions.
Read the full abstract

Signal Processing

Performance Scaling Laws for PD Array-based Receivers in IM/DD Optical Wireless Communication Systems

We study the performance scaling laws for electrical-domain combining in photodetector (PD) array-based receivers employing intensity modulation and direct detection, taking into account the inherent square-law relationship between the optical and electrical received powers. The performance of PD array-based systems is compared, in terms of signal-to-noise ratio (SNR) and achievable rate, to that of a reference receiver employing a single PD. Analytical and numerical results show that PD arrays provide performance gains for sufficiently narrow beams and above an SNR threshold. Furthermore, increasing the number of PDs alone does not enhance performance, and joint optimization of beam pattern, transverse electromagnetic mode, received power, and PD positions is necessary. Our model and derived insights provide practical guidelines and highlight the trade-offs for the design of next-generation high-bandwidth PD array receivers.
Read the full abstract