Signal Processing · Cross-listed from cs.CV
Resolving Multi-Target Association in OFDM-based ISAC via Vision-aided Multi-Modal Learning
Meng Hua, Chenghong Bian and Deniz Gunduz
Orthogonal frequency division multiplexing (OFDM)-based integrated sensing and communication (
ISAC) systems commonly extract target parameters by peak-searching a delay-Doppler map (DDM) constructed from reflected pilots. In multi-target scenarios, this results in ambiguity: the DDM does not reveal which physical target produced which peak, and two targets within the same delay-Doppler resolution
cell cannot be separated. We propose a
vision-assisted OFDM-
ISAC framework that resolves both limitations by fusing wireless and visual modalities. The transmitter encodes an onboard street-view image with deep joint source-channel coding (DeepJSCC) and transmits it over the same OFDM waveform used for sensing; the receiver reconstructs the image, runs a fine-tuned YOLOv5 detector and fuses the resulting per-target features (bounding-box coordinates and class labels) with the DDM and transmitter-receiver geometry through a learned multi-modal network. To stabilize training of the high dimensional delay and Doppler classifiers, we introduce a Kullback Leibler loss against triangular soft labels centered on the ground-truth bin. On a Blender-rendered vehicular testbed, the proposed framework achieves a 16 cm localization root mean square error (RMSE) and a 10.8 ns delay RMSE. An ablation study confirms that removing the visual modality causes a 60x degradation in localization. These results highlight the potential of
vision to overcome the data-association and resolution limits of single-modality
ISAC.
Read the full abstract ▾
Audio and Speech Processing
DSSCNet: A Transfer Learning Framework for Cross-Corpus Dysarthric Speech Severity Classification
Arnab Kumar Roy, Hemant Kumar Kathania, Paban Sapkota et al. (+2 more)
Dysarthric
speech severity classification is challenging due to speaker variability, class imbalance, and limited
datasets. This study introduces DSSCNet, a deep learning model that employs transfer learning and multi-corpus learning to enhance speaker-independent classification. By pre-training on one dysarthric
speech corpus and fine-tuning on another, DSSCNet achieves improved feature extraction and cross-corpus generalization. Experimental results demonstrate that DSSCNet outperforms state-of-the-art models for speaker-independent severity classification, achieving 75.80% accuracy on TORGO and 68.25% on UA-
Speech, significantly reducing misclassification errors. The findings confirm that leveraging knowledge transfer between
datasets improves model robustness, making DSSCNet well-suited for automated dysarthria assessment. This research contributes to the development of more effective assistive
speech technologies for individuals with
speech impairments.
Read the full abstract ▾
All authors Arnab Kumar Roy, Hemant Kumar Kathania, Paban Sapkota, Sudarsana Reddy Kadiri, Shrikanth Narayanan
Audio and Speech Processing
How Well Do Self-Supervised Speech Models Encode Age and Gender in Children's Speech? A Layer-Wise Analysis Across Multiple Architectures
Abhijit Sinha, Hemant Kumar Kathania, Mohit Joshi et al. (+3 more)
Self-supervised learning (SSL) models have become a central component of modern
speech processing systems, as they enable the learning of rich acoustic representations without reliance on labeled data. Despite their success on adult
speech, it remains unclear how effectively these models capture speaker-related attributes such as age and gender in children's
speech, which differs substantially from adult
speech due to ongoing physiological and cognitive development. Higher pitch, increased articulatory variability, and age-dependent acoustic changes make children's
speech a particularly challenging domain. In this work, we present a comprehensive analysis of how age and gender information is encoded across layers of four widely used SSL models: Wav2Vec2, HuBERT, Data2Vec, and WavLM. Layer-wise features are extracted and evaluated using a lightweight CNN on two
benchmark children's
speech corpora, PFSTAR and CMU Kids. To analyze feature compactness and redundancy, PCA is applied to identify redundancy and highlight the dimensions that contribute most to classification performance. Experimental results show that age- and gender-related information is unevenly distributed across SSL layers, with early to mid-level layers encoding the strongest paralinguistic cues. HuBERT achieves the best overall performance for age classification, while Wav2Vec2 and HuBERT lead gender classification on PFSTAR and CMU Kids, respectively. Beyond single-split evaluation, we further demonstrate that these findings remain stable under speaker-wise cross-validation, layer aggregation, and cross-database evaluation, indicating robustness to data imbalance and domain mismatch. Finally, we show that reliable age and gender classification is achievable even from short
speech segments of 1–3 seconds.
Read the full abstract ▾
All authors Abhijit Sinha, Hemant Kumar Kathania, Mohit Joshi, Harishankar Kumar, Shrikanth Narayanan, Sudarsana Reddy Kadiri
Signal Processing
RemoteRF: An Open-Source Platform to Democratize Access to Software-Defined Radios in Wireless Research and Education
Ethan Y. Ge and Ian P. Roberts
Software-defined
radios (SDRs) are powerful tools for research and education in wireless communications, but their cost and complexity put them out of reach for many universities and researchers worldwide. To address this, we introduce RemoteRF, a platform for creating large-scale testbeds of distributed SDRs that are centrally managed by a single server. These SDRs can be remotely accessed by users over the internet, allowing them to conduct wireless experiments at any time from virtually anywhere, as long as they have a network connection. When used in research, RemoteRF can be used to develop and experimentally evaluate new communication techniques or to collect real-world data to train and test machine learning models. When used in education, RemoteRF can allow students in virtually any sized class to share a handful of SDRs to complete active learning lab exercises that parallel course lectures. In an effort to democratize access to SDRs across the globe, the software powering RemoteRF has been made open-source and is extensively documented, allowing anyone to deploy their own instance today in a matter of minutes. Over the past year or so, RemoteRF has been used in both teaching and research at UCLA, where it has logged nearly 4,000 hours of use by more than 200 students and researchers to date.
Read the full abstract ▾
Systems and Control · Cross-listed from math.OC
Information Design under Uncertain Utilities: Probabilistic and CVaR Approaches
Furkan Sezer
This paper studies information design when the designer lacks precise knowledge of
agents' payoff coefficients. The Calibrated Bayes Correlated
Equilibrium (Cal-BCE) is introduced as a solution concept that augments the Bayes correlated
equilibrium with a corrector policy preserving incentive compatibility under the designer's structural uncertainty, adapting its revelation principle to this setting. The design problem is nonconvex in general, but under a linear-quadratic-Gaussian structure it admits convex second-order cone and semidefinite reformulations under two-sided probabilistic and conditional value-at-risk (CVaR) constraints, with feasibility guaranteed by a Hadamard invertibility condition. A joint decentralization
theorem shows that both designs cap cross-
agent action covariances, the CVaR design more tightly at a common tolerance; but because the formulations operate at design-specific feasibility thresholds, the realized ordering is calibration-dependent. Experiments on fifteen sector ETFs confirm the trade-off: the probabilistic design attains higher mean welfare and the CVaR design better tail protection, with neither dominating outright.
Read the full abstract ▾
Image and Video Processing · Cross-listed from cs.SE
Failure Analysis in Transition: An Industry Survey of Challenges, Priorities, and Standardization Needs in Advanced Packaging and Heterogeneous Integration
Himanandhan Reddy Kottur, Nusra Akter Takia, Mahamudul Hassan Fuad et al. (+3 more)
Failure analysis is being reshaped by heterogeneous integration, chiplet-based architectures, hybrid bonding, backside technologies, & increasingly buried package structures. To examine how practitioners view this transition, an anonymous survey was distributed across a broad set of organizations involved in
semiconductor design, packaging, systems, tools, & failure analysis. The survey collected approximately one hundred responses & probed organizational background, supported product domains, future priorities in failure analysis, critical bottlenecks, sample preparation challenges, emerging architecture specific pain points, & perceived needs for workflow acceleration & data standardization. The results show that heterogeneous integration, chiplet, and three-dimensional products dominate the respondent base at 69%, while package & heterogeneous integration failure analysis received the highest importance rating at 7.92 out of 10. Hybrid bonding emerged as the most difficult new architecture to analyze at 54%, higher-resolution non-destructive imaging ranked as the most important future accelerator at 8.18 out of 10, and 83% of respondents supported formalized data standardization frameworks. The complete survey data are provided in Appendix A (Table II) to improve transparency & support future benchmarking.
Read the full abstract ▾
All authors Himanandhan Reddy Kottur, Nusra Akter Takia, Mahamudul Hassan Fuad, Istiaq Firoz Shiam, Matthew Walsh, Navid Asadizanjani
Systems and Control · Cross-listed from cs.RO
Zero-shot Transfer of Reinforcement Learning Control Policies for the Swing-Up and Stabilization of a Cart-Pole System
Nikki Xu and Hien Tran
Reinforcement learning (RL) is a powerful and convenient tool to modernize controller design. In this work, we study the zero-shot transfer of RL-based control policies from simulation to hardware for cart-pole swing-up and stabilization. The two policies are trained independently, and the handoff is implemented in Simulink via switching logic. We apply a first-order action smoothing filter to prevent hardware damage from high-frequency oscillatory actuation. Pairing this bandwidth-aware filtering with sensitivity-guided domain randomization (DR) and a simple linear curriculum learning (CL) schedule, we obtain a swing-up policy that in all of our experiments injects sufficient energy for handoff into the stabilizer's region of attraction. The stabilization policy rejects disturbances within the tested range, and the swing-up policy can re-engage after larger perturbations and restores the pendulum to the inverted position.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.RO
Durability-Aware Multi-Objective Optimization of the Jansen Linkage: Trading Gait Quality Against Joint Wear
Jichao Wang
The Jansen linkage is a single-degree-of-freedom planar leg mechanism whose eleven "holy numbers" were evolved by Theo Jansen to optimize the foot-path gait alone, with no regard for the wear of its revolute joints. This paper introduces a durability objective into the design of the Jansen leg. A parametric forward-kinematic model (two-circle-intersection solver), an inverse-dynamic model (constraint-Jacobian / Lagrange-multiplier formulation of a seven-body, ten-joint system, independently cross-verified by a reduced-DOF energy method), and an Archard wear model are coupled to evaluate, for any set of link lengths, both gait quality and the per-cycle sliding wear at every pin. Because the wear is computed on ideal, clearance-free revolute joints, the resulting wear figures are a relative comparative ranking rather than an absolute life prediction. A bi-objective problem – composite gait error versus total joint wear, subject to step-length, ground-clearance, duty-factor and assembly constraints – is solved with NSGA-II. Under the adopted gait metric the classical Jansen design is Pareto-dominated: for a representative design, link-length adjustments within +/-29% simultaneously flatten the stance (-28%), smooth the stance velocity (-58%) and reduce total joint wear by ~56%. A sensitivity study shows the wear advantage is robust across a crank-speed x payload envelope (48%-56%) and identifies the link lengths that most strongly govern wear. A variance-based global (Sobol) analysis confirms that two link lengths dominate the wear variance, and a Monte-Carlo manufacturing-tolerance study shows the wear advantage degrades gracefully under realistic fabrication error. The framework provides a practical route to longer-lived walking linkages and a baseline for future wear-clearance-impact coupled studies.
Read the full abstract ▾
Signal Processing
A Novel Grant Prediction Method for 5G NR Terminals
Chenhao Wu, Xiaojiang Xu, Yuxuan Li et al. (+3 more)
5G NR user equipment suffers from high power consumption due to continuous PDCCH monitoring. Predictive dynamic power management (DPM) can save energy by
forecasting data grants, but accurate prediction is challenging due to unobservable scheduling states and bursty grant patterns. This paper proposes IOHMM-BO, a high-order input-output hidden Markov model with
Bayesian optimization. Based on real 5G NR traces, we capture long-range dependencies via a compound state and jointly optimize model order and listening window using
Bayesian optimization. Experiments on real traces show that IOHMM-BO achieves 45.3% accuracy, 5.0% false negative rate, and 43% energy saving with low computational overhead. The method provides a balanced trade-off between reliability and energy efficiency.
Read the full abstract ▾
All authors Chenhao Wu, Xiaojiang Xu, Yuxuan Li, Yuanhao Xu, Wenhui Xiong, Xiaoyu Fu
Systems and Control
On the connection between input-output resonances and internal modes of linear time-invariant systems
Bassam Bamieh
It is shown that in general, there is no connection between the location of the internal modes of a Linear Time-Invariant (LTI) system and the shape of its input-output frequency response. In particular, it is shown that resonance peaks of the frequency response do not necessarily correspond to under-damped internal modes. This phenomenon, though rare, can occur in high (or infinite) dimensional LTI systems. In the Single Input Single Output (SISO) case, this phenomenon can be attributed to the location of system zeros, while in certain Multi Input Multi Output (
MIMO) cases without system zeros, it can be attributed to the non-normality of the
matrix generating the internal dynamics.
Read the full abstract ▾
Systems and Control · Cross-listed from math-ph
Electromagnetic Characterization of Magnetic Bar: Case of Square Cross-Section Shape
Taha El Hajji, Bruno Ricardo Marques and Lars Sjöberg
This paper presents a complete two-dimensional theoretical model for the electromagnetic behavior of square-section solid
magnetic bars under sinusoidal loading. Through the application of Maxwell's equations within a Cartesian coordinate system and the integration of complex permeability, exact mathematical expressions are derived for mutual impedance, internal
magnetic fields, flux, and core losses.
Hyperbolic functions are utilized to separate the variables, enabling the accurate representation of edge flux accumulation and the 2D skin effect. In addition to mathematically decoupling eddy current and hysteresis losses, this formulation yields a new apparent permeability parameter. This parameter establishes a fast, reliable method for
magnetic steel characterization that bypasses the extensive processing times associated with Finite Element Analysis (FEA). Numerical results over 1 Hz-1 MHz show the apparent relative permeability decreasing from 500 to 300 and a characteristic resistance peak near 700 kHz, marking the transition from volumetric to surface-dominated loss regimes.
Read the full abstract ▾
Systems and Control
Reinforcement Learning-Based Traffic Signal Control for IoT-Enabled Intersections
Yousef AlSaqabi
Urban traffic congestion remains a persistent challenge in car-dependent cities, imposing significant economic and societal costs. Traffic signal systems are increasingly deployed as networked cyber-physical components within smart-city infrastructures, where distributed sensing and edge intelligence enable adaptive traffic management. This paper investigates reinforcement learning (RL) as an edge-intelligent approach for adaptive traffic signal operation at a signalized urban intersection in Kuwait. A Proximal Policy Optimization (PPO)-based controller is developed to dynamically allocate green-phase durations using locally observed traffic states, without relying on future demand information or centralized coordination. The controller is evaluated in a realistic simulation environment informed by real-world hourly traffic volume data from Kuwait, and is compared against both conventional fixed-time control and a vehicle-actuated controller representing the current state of practice, using average vehicle delay, queue length, and emissions as performance metrics. Under nominal conditions, the proposed controller reduces average vehicle delay by 46% relative to fixed-time control and 34% relative to actuated control, while also lowering per-vehicle CO2 emissions by approximately 23%. These performance gains persist under demand perturbations of +/-15%, generalize from weekday to weekend traffic patterns, and are corroborated by a reward function ablation; low variance across five random seeds confirms their statistical reliability. These findings demonstrate the practicality of learning-based edge traffic signal control as a building block for IoT-enabled smart-city transportation systems, and as a deployable precursor toward fully connected, Internet of Vehicles (IoV)-based urban mobility.
Read the full abstract ▾
Systems and Control
A Pre-Dispatch Resonance Safety Criterion for AI Training Clusters
Chandan Chaudhary, Abanish Tiwari, Yansong Pei et al. (+2 more)
Hyperscale AI training clusters operate under the Bulk Synchronous Parallel protocol, which impose a periodic power swing on the transmission grid. Every GPU in the job transitions between compute and idle in lockstep, so the aggregate power traces a square wave at the training iteration period. Production iteration periods of one to ten seconds place the forcing frequency within the inter-area electromechanical mode band of large interconnections, where a training schedule can drive a mode at resonance. This paper derives a closed-form pre-dispatch safety criterion that bounds the maximum cluster size a grid can absorb at any proposed iteration period. The derivation inverts the steady-state forced two-area swing equations. The criterion defines a danger band of iteration periods, extends to the square-wave harmonics, and parameterizes the modal response from planning-study eigenanalysis and the forcing amplitude from GPU specifications. Applied to the IEEE 39-bus system at a production-representative duty cycle, the criterion shows that the maximum safe cluster at resonance is $66\,900$ GPUs under light damping. Rescheduling the same job less than one second away from resonance reduces the deviation $7.4\times$ with no hardware change. These results establish the training iteration period as a controllable grid-safety parameter and supply the analytic screening tool that reliability directives on current large loads lack.
Read the full abstract ▾
All authors Chandan Chaudhary, Abanish Tiwari, Yansong Pei, Mohammed Ben-Idris, Joydeep Mitra
Signal Processing
Anticipating the Optimism Gap: Predicting Distribution-Shift Degradation of RF-Impairment Detectors from In-Distribution Statistics
Chakshu Baweja
Detectors for GNSS
radio-frequency impairments (jamming, spoofing, multipath) are usually reported with a single AUC measured on the distribution they were tuned on. That number falls once conditions move, and the size of the drop is rarely known in advance because labelled field data is scarce. We ask whether this optimism can be predicted before any out-of-distribution data is seen. On an open, parameter-grounded synthetic testbed with a tunable severity shift, we evaluate thirteen detectors (five physics baselines, full-feature logistic regression and multilayer perceptrons, and single-feature learned controls) across four impairment classes. The optimism gap, the difference between in-distribution and shifted AUC, grows monotonically as the shift deepens (mean Spearman correlation 0.50). It is driven by how many observables a detector uses rather than by whether it is learned, and it varies systematically by class. Centrally, a ridge model built only from in-distribution score statistics predicts the gap for a detector it has never seen (R^2 = 0.47) and for an impairment class it has never seen (R^2 = 0.46); both are significant against a 2000-fold permutation null (p < 0.001) and survive removing the feature that is, by construction, part of the target. The headline findings are synthetic. We then run the pre-registered protocol on three open field corpora: on Jammertest 2024 the cross-detector prediction holds (R^2 = 0.11, p = 0.009), and on SatGrid, whose spoofer power sweep gives a calibrated severity axis, in-distribution AUC overstates higher-severity AUC by up to 0.22 and to the point of sign inversion, with in-distribution AUC and realised gap perfectly rank-correlated (Spearman rho = 1.0). The mechanism survives contact with real data, at smaller magnitude than in simulation. We release the testbed, a software-receiver front end, the ingest adapters and the protocol.
Read the full abstract ▾
Signal Processing
Full-Domain Coupler: A Wireless Native Neural Backbone for Channel Representation and Deduction
Zirui Chen, Ziqing Xing, Zhaoyang Zhang et al. (+5 more)
Data representation is a fundamental issue in deep learning. However, as wireless data scales and deeply couples across many physical domains such as time, space, and frequency, existing wireless artificial intelligence (AI) technologies lack dedicated representation solutions. Instead, they mainly rely on stitching general-purpose networks, a tool-driven paradigm that inevitably results in structural redundancy and bottlenecks in information flow. To fill this gap, this paper proposes Coupler, a wireless native-AI neural backbone designed for representation learning of channel state information (CSI)–the pivotal data in wireless systems. Leveraging the revealed physical insights of channel tensors, Coupler decomposes representation learning into individual domains on a layer-by-layer basis, and then couples the learned domain-specific features through a dimension-staggered cascade. This full-domain interleaved learning architecture enables superior parameter efficiency and fine-grained multi-domain feature fusion. Based on this backbone, we use the complex-domain multilayer perceptrons (CMLPs) as spatial and frequency domain learners, while employing three optional mechanisms–convolution,
attention, or gating–to capture temporal dependencies. This results in a series of efficient channel learning schemes with diverse functionalities and extreme lightweights, showcasing the compactness, versatility and flexibility of Coupler. We evaluate these schemes on channel deduction, a general representation task encompassing channel estimation, interpolation, prediction, and feedback. Extensive experimental evaluations validate their significant performance gains and robust applicability even for real-world measured data, demonstrating the potential of Coupler as a promising basic architecture in the design of wireless foundation models.
Read the full abstract ▾
All authors Zirui Chen, Ziqing Xing, Zhaoyang Zhang, Hongning Ruan, Yuzhi Yang, Zhaohui Yang, Chongwen Huang, Merouane Debbah
Audio and Speech Processing
Using Phonological-Level Wav2Vec2 for Mandarin Automatic Mispronunciation Detection and Diagnosis
Jinghao Chen, Mostafa Shahin and Beena Ahmed
Automatic mispronunciation detection and diagnosis (MDD) plays a crucial role in L2 Mandarin pronunciation learning. While end-to-end (E2E) based MDD methods have substantially improved phoneme-level detection accuracy, diagnostic feedback remains limited, as segmental and tonal errors are not explicitly separated. In this paper, we propose a phonological feature-based MDD framework that models both segmental and tonal attributes within a unified Wav2Vec2 CTC architecture. Experimental results show that the proposed method reduces the False Acceptance Rate (FAR) by 10.1% and the Diagnostic Error Rate (DER) by 23.6% compared with the phoneme-only baseline system. By decomposing phonemes into low-level phonological components, the proposed approach enables more detailed and interpretable diagnostic feedback for L2 learners.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.CL
Benchmarking Large Language Models for Grapheme-to-Phoneme Conversion: A Japanese Case Study
Tomoki Koriyama
Grapheme-to-phoneme (G2P) conversion is essential for controllable and robust text-to-
speech, and large
language models (LLMs), with broad linguistic knowledge, offer a promising approach. We benchmarked over 30 LLMs on Japanese G2P, comparing them with conventional morphological analyzers on 3000 manually annotated sentences. We evaluated two prompting strategies: a parse mode, where the LLM performs morphological analysis followed by rule-based kana conversion, and a direct mode, where the LLM directly predicts kana readings. The results show that model size, version, and Japanese-specialized training are key factors, with the best LLMs achieving kana character error rate below 0.52% vs. the best conventional tool (1.03%). Parse mode outperforms direct mode for most models, as rule-based post-processing relieves the LLM of handling complex pronunciation rules. We also show that feeding LLM-predicted kana into a kana-input TTS yields better pronunciation than end-to-end TTS.
Read the full abstract ▾
Systems and Control
TSO-DSO Coordination for Flexibility Management Across Voltage Levels
Gustavo Valverde, Anibal Sanjab, Florin Capitanescu et al. (+4 more)
Several sources of flexibility in transmission and, especially, distribution networks are being unlocked by advances in information and communication technologies, aggregators, and new flexibility
markets. However, maximizing benefits for both transmission and distribution system operators in a coordinated way requires new
algorithms, modeling tools, and modernization of regulatory frameworks. Such approaches must account for uncertainties, the physical and operational constraints of flexibility providers and the grid itself, constraints on information exchange, and scalability, including computational requirements and time constraints. Given the diverse contexts and jurisdictions around the world, there is no single recipe for achieving coordination, but important trends and shared challenges are emerging. This paper surveys the complexities of coordination from technical,
market, and technological perspectives, and outlines current practices, proposed approaches, and future research directions to effectively manage, coordinate, model, and leverage flexibility across voltage levels.
Read the full abstract ▾
All authors Gustavo Valverde, Anibal Sanjab, Florin Capitanescu, Christian Rehtanz, Gianluigi Migliavacca, Zhengshuo Li, Innocent Kamwa
Signal Processing
XMDCA-TL: An Explainable Multi-Domain Channel Attention Transfer Learning Framework for Fault Diagnosis in Industrial Gas Turbines
Amir Jahangard Takaloo, Mahdi Aliyari Shoorehdeli and Ehsan Mohammadi
In this study, a multi-task, interpretable transfer learning framework, XMDCA-TL, is proposed for fault diagnosis in industrial gas turbines. In the proposed method, the vibration time waveform is first converted into a multi-domain RGB representation comprising time, frequency, and time-frequency domains. A ConvNeXtV2-based encoder then processes these images, and the Multi-Domain Channel
Attention (MDCA) mechanism is applied to its deep layers to model interactions among different domains and complementary dependencies in the signals. To improve the quality of the learned representations and enhance the model's robustness to noise, a self-supervised strategy based on hybrid masking, along with a UNet-based decoder to reconstruct the masked regions, has been designed. To overcome the limitation of labeled data in industrial environments, transfer learning was employed to transfer knowledge from laboratory data to real-world data from a 42.2 MW MGT-40 gas turbine at the Zahedan power plant. Additionally, a comprehensive Explainable Artificial Intelligence (XAI) framework was developed to analyze decision-making regions, evaluate domain importance, examine the flow of
attention between domains, and assess reconstruction uncertainty. The results showed that XMDCA-TL, while achieving satisfactory fault diagnosis performance, possesses domain adaptability and robustness to noise and provides a physical interpretation of the model's decision-making process.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.CL
Adding Robust Code-Switching Capabilities to High Performance Multilingual ASR
Enes Yavuz Ugan and Alexander Waibel
Code-switching (CSW) remains challenging for large multi-lingual ASR systems in real-world deployment. While fine-tuning on synthetic CSW data is possible, it generally degrades strong monolingual baselines. Our goal is to preserve these capabilities while extending models to handle complex code-switching, including morphological variations across
languages. We propose
Bayesian factorized adaptation, which learns to efficiently integrate switching-relevant knowledge into strong pretrained models without overwriting existing capabilities. Requiring only a small amount of synthetic data, our approach reduces transcription errors by 32.87% on code-switched words while improving overall WER by 5.31%, all while maintaining mono-lingual performance. Our results demonstrate that effective CSW adaptation depends more on knowledge integration than data complexity.
Read the full abstract ▾
Systems and Control
Uncertainty-Disentangled Probabilistic Stability Analysis in Wind Power Integrated Weak Grids
Samson S. Yu and Yinsong Chen
Conventional probabilistic small-signal stability analysis (PSSSA) propagates a single forecast distribution, conflating irreducible weather randomness (aleatoric) with reducible forecast-model uncertainty (epistemic). This letter propagates a second-order renewable forecast through the modal-stability map via an independent germ variable, separating the two contributions exactly in closed form by a disentangled
polynomial chaos expansion (d-PCE). The split underpins a forecast-aware $(\alpha,\beta,\gamma)$ stability certificate whose conservative branch converges to its irreducible aleatoric limit at $O(N^{-1/2})$ – making a failed certificate diagnostic: epistemic-dominated risk recovers with better data; aleatoric-dominated risk needs improvements of the physical control system.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.HC
Integrating Facial Generation into Full-Duplex Spoken Dialogue Systems
Jingjing Jiang, Atsumoto Ohashi and Ryuichiro Higashinaka
Full-duplex spoken dialogue models, such as Moshi, enable natural, low-latency voice conversations. However, they remain limited to the audio modality, lacking the facial expressions that are integral to human communication. We present Moshi-Face, the first full-duplex dialogue model that jointly processes the user's audio and facial input while simultaneously generating
speech and facial motion. We first construct a vector-quantized variational autoencoder (VQ-VAE) as a face codec that encodes 3D head meshes extracted from facial
videos into compact discrete
tokens, referred to as face
tokens, and conversely reconstructs 3D meshes from these
tokens. We then extend Moshi with a Face
Transformer module that generates face
tokens non-autoregressively, enabling Moshi-Face to produce synchronized audio and face
tokens in real time. Experiments show that Moshi-Face achieves audiovisual
alignment at low latency while preserving the dialogue quality of the original audio-only model.
Read the full abstract ▾
Systems and Control
Joint Visibility Analysis of RIS in Non-Terrestrial Networks through Stochastic Geometry
Ashutosh Balakrishnan, Junse Lee and Francois Baccelli
Non-Terrestrial networks (NTNs) are a key theme in upcoming 6G communications, especially for ubiquitous coverage. Urban environments, comprising of high rise buildings often result in blocking the line of sight (LoS) path between the user equipment (UE) and the NTN base station (NTN-BS). In this paper we investigate the situation where reconfigurable intelligent surfaces (RIS) are deployed on the building roof-tops to ensure multi-hop connectivity between the UE and the NTN-BS. In such a scenario, it becomes crucial to statistically study the LoS visibility of the RIS from the UE as well as from the NTN-BS, hence termed as joint visibility. In this work, accounting for the dual stochasticity arising from the locations of the RIS deployed buildings and the respective random building heights, we statistically study the probability of joint RIS visibility in a two-dimensional (2D) scenario considering a deterministic location of the NTN-BS. Further, we study the joint RIS visibility statistics conditional on the UE-NTN link being LoS or non-LoS. For the RISs deployed as a point point process (PPP) having exponentially distributed heights, the expected RISs jointly visible under the unconditional and conditional
geometric settings are derived in closed form. Interestingly, in the 2D setting, the maximum expected RISs jointly visible, unconditionally, is twice the Basel number $(\pi^2/ 6)$. The simulated results are analyzed over building density, average building height, the altitude and position of the NTN-BS. We also illustrate probability heatmaps, demonstrating the strongest chance to have a RIS used conditioned on the system geometry. This study is expected to be useful in planning the deployment of RIS in urban areas, improving the signal and for assessing economic aspects.
Read the full abstract ▾
Systems and Control
DC Link Capacitor Ripple Constraints Limit the Benefits of Utility-Owned Four-Wire Power Converters
Matthew Deakin, Xu Deng, Shafiq Odhano et al. (+1 more)
Utilities are increasingly interested in power converters to increase the headroom of their assets by actively controlling power flows on their network. In this work we demonstrate that thermal limits of dc link capacitors can result in substantially diminished benefits of these converters under unbalanced operation, due to constraints on neutral current and double-line frequency power ripple. Considering nine voltage source converter
topologies with varying ripple capabilities, the upper bound (in terms of additional headroom released) increases by more than 80% compared to a no-ripple case for the application of phase current unbalance mitigation.
Read the full abstract ▾
All authors Matthew Deakin, Xu Deng, Shafiq Odhano, Rahmat Heidari
Audio and Speech Processing · Cross-listed from cs.SD
AugCodec: A Low-Bitrate Disentangled Neural Speech Codec via Data Augmentation
Dongmei Wang, Xiaohang Sun, Yang Liu et al. (+10 more)
We propose AugCodec, a low-bitrate disentangled neural
speech codec that leverages data augmentation to decompose
speech into three distinct components: semantic, speaker, and prosody
tokens. Specifically, we employ tailored augmenta tion strategies to transform
speech into distinct variants, each serving as input for extracting
tokens that preserve the target attribute while suppressing others. This disentanglement strategy enables substantial reduction in
token rate. Further more, we introduce an augmentation loss that aligns semantic encoder outputs between source and voice-converted
speech, encouraging speaker-agnostic
embeddings while mitigating the acoustic mismatch induced by voice conversion. Experiments on LibriSpeech test-clean demonstrate that AugCodec significantly outperforms state-of-the-art methods in both reconstruction quality and disentanglement, while operating at only 12.5Hz with three
token streams.
Read the full abstract ▾
All authors Dongmei Wang, Xiaohang Sun, Yang Liu, Fanjie Kong, Abhishek Yanamandra, Abhinav Jain, Daniel Tompkins, Woohyun Kang, Najmeh Sadoughi, Sunil Hadap, Xiang Hao, Zhu Liu, Caren Chen
Audio and Speech Processing
ProsoCodec: Prosody-Oriented Speech Codec for Voice Conversion
Jeongsoo Choi, Ji-Hoon Kim, Shujie Hu et al. (+1 more)
Neural
speech codecs efficiently compress
speech and have become a foundation for
speech generation, but they are typically learned as holistic representations that intertwine linguistic content, speaker identity, and prosody. While this design is effective for zero-shot voice cloning, it hinders downstream tasks that require prosody preservation or transfer, such as voice conversion. To address this, we introduce ProsoCodec, a prosody-oriented
speech codec that models prosody as a conditional residual rather than as a disentangled stream. Specifically, by conditioning both the encoder and decoder on text and speaker
embeddings as prefix
tokens, the discrete bottleneck is encouraged to capture prosodic variation not explained by content and speaker. To further preserve prosody, we use the low-frequency mel band and train the model on paired same-speaker utterances. Experiments on voice conversion show improved prosody preservation and reduced source-timbre leakage.
Read the full abstract ▾
All authors Jeongsoo Choi, Ji-Hoon Kim, Shujie Hu, Joon Son Chung
Audio and Speech Processing · Cross-listed from cs.SD
Improving Engine Sound Analysis in Hot-Test Environments via a RAB-U-Net (Residual Attention Block U-Net) Noise Removal Method
Raheleh Mohseni and Mahdi Aliyari Shoorehdeli
During hot tests on a production line, engine-sound analysis is crucial to ensuring product quality and performance. However, background noise often interferes with accurate sound analysis, leading to potential errors in engine diagnostics. Traditionally, skilled technicians listen to engine sounds to assess engine health, but this is prone to significant inaccuracies. This study presents an innovative deep learning-based approach to address this issue by removing background noise from engine sound recordings using a U-Net neural network structure enhanced with Residual
Attention Blocks (RAB-U-Net). Our intelligent noise removal system significantly improves the accuracy of engine noise detection, outperforming traditional techniques and providing a robust solution for real-time applications in production line environments. This study proposes a novel system for engine noise detection in production lines, marking a valuable advancement for the automotive industry in applying deep learning methods to improve the quality of engine diagnostics.
Read the full abstract ▾
Signal Processing · Cross-listed from cs.IT
One-Bit Clustering for Two Component Sub-Gaussian Mixture Models
Junren Chen and Yun Yang
Clustering is a fundamental problem in statistics and machine learning. We propose the first one-bit clustering method for two-component sub-Gaussian mixture models. The method uses only one bit per entry of each sample obtained via a dithered quantizer. Under a mild non-spikiness condition on the cluster centers, we show that a variant of Lloyd's
algorithm achieves a misclassification rate that decays exponentially with a signal-to-noise ratio comparable to that in the unquantized setting. This result further implies exact recovery under an explicit separation condition, which exceeds the optimal threshold for unquantized data by only a logarithmic factor. When the dimension $p$ is sufficiently large, the non-spikiness condition can be enforced by applying a random rotation using a Haar distributed
matrix prior to quantization. In particular, it holds with high probability when $p \gtrsim 1$ for partial recovery and $p \gtrsim \log n \log\log n$ for exact recovery, where $n$ is the sample size. We also establish a minimax lower bound, showing that the misclassification rate and separation condition exhibit sharp constants in general. Numerical results are provided to corroborate the theory and demonstrate the efficacy of the proposed method.
Read the full abstract ▾
Audio and Speech Processing
ESPnet3: Infrastructure for Scalable Speech and Audio Research in the Foundation Model Era
Masao Someki, Alexander Polok, Carlos Carvalho et al. (+14 more)
Recent
speech research involves increasingly large
datasets, complex models, and diverse experimental workflows. However, existing frameworks require substantial engineering effort to support such experiments. We present ESPnet3, a
speech and audio research framework built on a modular system architecture with configuration-driven
dataset composition and unified Python-based workflows. ESPnet3 introduces a DataOrganizer abstraction for flexible
dataset integration and
dataset sharding for
memory-efficient large-scale training, while allowing recipe-specific logic through lightweight stage overrides. In OWSM pre-training experiments, ESPnet3 reduces per-epoch training time by 21.1 minutes compared to ESPnet2 and achieves >80% GPU utilization in multi-node training. Fine-tuning experiments show that new models and
datasets can be integrated with around 46 lines of additional code. ESPnet3 will be publicly released with model checkpoints and training logs.
Read the full abstract ▾
All authors Masao Someki, Alexander Polok, Carlos Carvalho, Chyi-Jiunn Lin, Da-Hee Yang, Jiatong Shi, Jinchuan Tian, Nelson Enrique Yalta Soplin, Samuele Cornell, Siddhant Arora, Francisco Teixeira, Wei Wang, William Chen, Alberto Abad, Chenda Li, Shinji Watanabe, Wangyou Zhang
Signal Processing
Improving Doppler Resilience of OFDM through Delay-Doppler Sensing
Danish Nisar, Saif Khan Mohammed, Muhammad Ubadah et al. (+2 more)
The performance of traditional CP-OFDM degrades severely in doubly-spread wireless channels due to inter-carrier interference (ICI). In this paper, we propose DD domain sensing based CP-OFDM where we transmit a Zadoff-Chu (ZC) pilot signal overlaid on CP-OFDM data carriers. At the receiver, DD domain signal processing is used to acquire the effective DD domain channel filter which is stationary in the DD domain. From this DD domain estimate, we derive the complete frequency domain (FD) input-output (I/O) relation between CP-OFDM carriers, acquiring which is otherwise difficult with traditional time-frequency signal processing. Using this FD I/O relation, we estimate the received FD pilot signal which is then canceled from the received FD signal, resulting in a data-only signal. Joint detection of all CP-OFDM data carriers from this data-only signal equalizes the effect of ICI. Numerical simulations of the standardized 3GPP TDL-C channel shows that in high mobility scenarios, the proposed DD domain sensing based CP-OFDM achieves significantly better spectral efficiency when compared to that achieved by traditional CP-OFDM.
Read the full abstract ▾
All authors Danish Nisar, Saif Khan Mohammed, Muhammad Ubadah, Ronny Hadani, Robert Calderbank