Peer Revenue Fully funded

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Electrical Engineering and Systems Science

eess Watch Electrical Engineering and Systems Science (sign in to use)

Friday 25 September

Systems and Control · Cross-listed from cs.DC

Job Class Thermal Intent Aware Liquid Cooling Allocation for AI Data Centers

GPU-dense AI data centers need to run on liquid cooling as air simply cannot shed the heat at these power densities. Yet the cooling loops themselves are blind to what workloads are about to run; they crank up flow only after a sensor catches a temperature climb, which can take 30 to 50 seconds. We built Job-Class Thermal Intent (JCTI) to close that window. The scheduler already knows a job is coming and what class it belongs to; JCTI feeds that information straight to the cooling controller so it can stage coolant before the heat shows up. We pulled the thermal signatures for each job class out of MLPerf GPU power traces and tuned arrival patterns against Alibaba cluster data. Over 120 paired Monte Carlo trials the numbers come out to 56.4% fewer thermal violations and 60.2% less cumulative overshoot than a straight PI loop. As AI data centers evolving towards gigawatt grid loads with highly fluctuating power swings, thermally-aware scheduling reduces sudden demand and improves load prediction in grid side. Cooling and scheduling have been running as two separate systems for years despite each one knowing something the other needs, JCTI wires them together.
Read the full abstract

Audio and Speech Processing · Cross-listed from cs.SD

Symbiotic Architecture for Post-Hoc Audio Extension of Frozen Language Models

This paper proposes an architecture for equipping large language models (LLMs) with audio-understanding capabilities without fine-tuning their weights. The proposed symbiotic architecture employs an injector module that writes audio-conditioned vectors directly into the target LLM's short-term memory, i.e., the key-value (KV) cache, enabling the LLM to behave as an audio language model (ALM). The architectural advantages are twofold. First, it improves the scalability of ALMs: because the proposed method bypasses the LLM during audio injection, the injection cost is governed by the injector width rather than the backbone width, and can therefore scale more slowly than the cost of full-backbone prefilling. Second, since the training scheme does not update the LLM weights, the original capabilities of the LLM are preserved without the risk of degradation from fine-tuning. The effectiveness of the proposed method is evaluated on both audio-understanding tasks (automatic speech recognition, audio question answering, and acoustic scene classification) and text-only tasks. We confirm that, while activating fewer parameters during audio prefilling, our architecture outperforms the conventional method with a frozen LLM and approaches the performance of a fine-tuned ALM, all while preserving the backbone LLM's original text-only task performance by construction.
Read the full abstract

Systems and Control · Cross-listed from cs.RO

Moving Horizon Estimation for Quadrotors: An $\mathcal{L}_1$ Adaptive Optimizer Approach

Moving Horizon Estimation (MHE) is a state estimation method based on finite-horizon optimization that can offer higher accuracy at the cost of increased computation compared to Kalman filter-based approaches. We present a linear smoothing MHE formulation as a dense Quadratic Program (QP), and a solver consisting of a continuous-time Newton's method augmented with the $\mathcal{L}_1$ Adaptive Optimizer ($\mathcal{L}_1$-AO). While MHE is inherently time-varying, conventional approaches treat it as a sequence of independent, time-invariant problems and employ iterative solvers at each time step, which can be both inaccurate and computationally burdensome. In contrast, time-varying solvers track the optimal solution with fewer iterations by exploiting the temporal evolution of the problem, thereby reducing the computational load. In this research, we enhance both the performance and efficiency of MHE through a time-varying solver with an $\mathcal{L}_1$-AO augmentation that compensates for the prediction inaccuracy, which is common in practice due to noisy sensors and the lack of prior knowledge of the system. Simulation results on a quadrotor platform show that the $\mathcal{L}_1$-AO-augmented approach solves the MHE optimization problem more efficiently than the baseline time-invariant solver and achieves higher estimation accuracy under challenging conditions, compared with both the Extended Kalman Filter and the standard MHE.
Read the full abstract

All authors Thinh Nguyen, Minkyung Kim, Sandeep Banik, Jinrae Kim, Naira Hovakimyan

Audio and Speech Processing · Cross-listed from cs.SD

Dialogue-Based Streaming Audio-Visual Target Speaker Extraction with Predictive Dialogue Information

In face-to-face, real-time communication, a talking agent must track the target speaker through natural pauses, turn-taking, and backchannels, often amid background cross-talk. Most target speaker extraction (TSE) studies, however, rely on simulated mixtures with full or sparse overlap and ignore the turn-taking of real conversations. We therefore introduce, to our knowledge, the first benchmark for online audio-visual TSE (AV-TSE), built from intact dyadic interactions with independent third-party interference. Observing that anticipating upcoming activity from semantic, acoustic, and facial cues benefits online AV-TSE, we propose an LLM-based target-speaker voice activity projection (TS-VAP) module. Unlike conventional VAP with separated speaker channels, it forecasts the future activity of the target and conversational partner directly from the overlapping mixture, drawing on the linguistic and conversational knowledge of a speech-LLM, and uses this prediction to guide a low-latency separator. We further combine this predictive context with historical and synchronous speaker context. Experiments show that TS-VAP consistently improves streaming extraction across multiple AV-TSE backbones, and that further combining historical, synchronous, and predictive context yields nearly 1 dB gain on real AV conversations. Project page: https://jjjjiaozi.github.io/TS-VAP/.
Read the full abstract

Audio and Speech Processing · Cross-listed from cs.CL

Learning Natural Conversational Behavior in Tandem Speech-to-Speech Models with Randomized Guidance

Tandem speech-to-speech architectures couple a responsive speech frontend with an asynchronous text backend. In KAME, a large language model (LLM) serves as the backend, supplying candidate responses as guidance to the speech frontend while the user is still speaking. Ordinary conversation recordings capture the eventual response but not the guidance the backend would supply during the user's utterance. Generating the missing guidance with a simulator LLM adds substantial data-preparation overhead when training on real conversations. We propose randomized intermediate guidance, which derives guidance directly from the conversation corpus rather than simulating backend LLM behavior. During training, target responses provide informative guidance, while randomly sampled responses provide potentially irrelevant updates during the utterance. This combination aims to teach the frontend to use backend information selectively. On synthetic dialogues, KAME trained with this recipe achieves response quality comparable to that of the LLM-generated and similarity-based baselines. Training on 3.8k hours of real conversations improves smooth turn-taking and audio-judge naturalness over synthetic-data KAME while retaining a response-quality advantage over Moshi. These results show that randomized guidance offers a practical route to combining the response-quality benefits of tandem models with natural conversational behavior learned from real speech.
Read the full abstract

All authors Manato Yaguchi, Yotaro Kubo, Hikaru Asano, So Kuroki

Image and Video Processing · Cross-listed from cs.CV

Learning Polarization Image Restoration with General Restoration Priors

Polarization imaging captures distinctive surface and geometric cues that benefit a wide range of vision tasks. However, real-world polarization acquisition is often affected by multiple coupled degradations, making image restoration essential for practical polarization vision. Existing methods are largely tailored to specific degradations and remain constrained by the limited scale and quality of polarization data. To address these limitations, we develop an all-in-one polarization restoration framework for diverse and composite degradations. We first study the impact of different polarization representations on restoration performance and identify the normalized Stokes representation as an effective choice for separating intensity and polarization information. Accordingly, we devise a dual-branch architecture that separates intensity and polarization modeling. To overcome the limitations of polarization-specific training, the intensity branch leverages pretrained general restoration priors and a mixture-of-experts extension for composite degradations, while its restoration knowledge is adaptively distilled into the symmetric polarization branch via a cross-domain feature transform. In addition, we establish a composite-degradation polarization benchmark to support all-in-one restoration research. Extensive experiments on public datasets and our proposed benchmark demonstrate the effectiveness of the proposed method.
Read the full abstract

All authors Chenggong Li, Jinhao Liu, Caiyun Wu, Yidong Luo, Junchao Zhang, Degui Yang

Image and Video Processing

Deep Pseudo-Proximal Map: A Self-Supervised Data-Fitting Agent for Iterative Reconstruction

Iterative algorithms for inverse imaging problems split reconstruction into alternating subproblems, one of which requires evaluating the data-fitting proximal map. In many practical cases, this proximal map has no analytic solution and so must be approximated by an inner loop solver at every iteration, compounding the cost of the outer reconstruction loop. To address this, we propose the pseudo-proximal map (PPM), a reformulation of the data-fitting proximal map as the minimum mean square error estimate of a synthetic probabilistic model. We implement the deep PPM as a self-supervised neural network trained only on sampled Gaussian noise, requiring no ground-truth training images. The deep PPM can be trained for any operator for which the forward model $A$ and its transpose $A^T$ can be evaluated, with provable equivalence to the proximal map when $A$ is linear. We validate the deep PPM on Gaussian deblurring and 4x super-resolution, where the proximal map has an analytic solution, and on X-ray computed tomography (XCT), where no analytic solution exists. Used as the data-fitting agent within an iterative reconstruction method, the deep PPM reproduces the reference reconstruction to within 1% NRMSE for all three operators, and for XCT, it replaces the inner conjugate-gradient loop with a single network evaluation that is 18x faster.
Read the full abstract

All authors Haley Duba-Sullivan, Emma J. Reid, Charles A. Bouman, Gregery T. Buzzard

Systems and Control · Cross-listed from cs.RO

Learning Vision-Based Agile Gap Traversal: Differentiable Simulation with a Warm-Started Critic

Traversing narrow gaps is challenging for autonomous quadrotors, especially when control commands come directly from high-dimensional visual observations. Existing end-to-end methods often rely on behavior cloning or full-rollout backpropagation through time (BPTT) via differentiable simulation, which can limit policy performance or incur high training costs. We propose a two-stage reinforcement learning framework for more efficient ego-centric visuomotor gap-traversal policy training, leveraging quasi-analytical policy gradients (QPG) via differentiable simulation and critic warm-starting. The framework utilizes QPG to avoid backpropagation through visual rendering, reducing computation and memory costs while improving sample efficiency. In the first stage, an expert actor and critic are trained using privileged observations, including gap geometry. Unlike prior gap-traversal approaches, our training utilizing QPG does not require resetting the agent along optimized reference trajectories. In the second stage, a visual policy is trained using binary gap masks from two ego-centric cameras and low-dimensional observations, while its privileged critic is warm-started from the first stage. This substantially improves training efficiency and traversal success compared with cold-starting the critic or using full-rollout BPTT. Our framework does not require retraining the expert actor when system parameters change, enabling more efficient generalization across drone platforms than state-of-the-art visual gap-traversal methods based on action supervision. The learned visual policy also generalizes to gaps with unseen shapes. Extensive real-world experiments further demonstrate robust gap traversal using binary masks rendered online. Beyond gap traversal, the proposed framework is generic and can be extended to other visuomotor robot learning tasks.
Read the full abstract

All authors Nuthasith Gerdpratoom, Tianchen Sun, Yichao Gao, Lin Zhao

Audio and Speech Processing · Cross-listed from cs.SD

Training-Free Contextual ASR via SpeechLLM-Based Error-Aware Selective Retrieval

Recognition of domain-specific and low-frequency terms remains challenging for automatic speech recognition (ASR). Although contextual biasing can improve their recognition, directly providing a large terminology dictionary introduces many irrelevant biasing terms. Retrieval-based contextual biasing addresses this issue by selecting candidate terms from an external dictionary, but querying many recognized words requires numerous dictionary lookups and may yield poorly targeted candidates. We propose a training-free contextual ASR framework in which a pretrained speech large language model (SpeechLLM) jointly generates an ASR hypothesis and localizes error spans likely to involve domain-specific terms. Only the localized spans are used to retrieve phonologically similar terms from an external terminology dictionary. The same SpeechLLM then re-recognizes the audio conditioned on the first-pass hypothesis and the retrieved terms, without task-specific model training. To assess applicability across domains, we evaluate the framework on medical, air traffic control, and financial speech. The proposed method substantially reduces dictionary queries while improving the recall and ranking of relevant terminology candidates and second-pass ASR performance across all three domains.
Read the full abstract

All authors Natsuo Yamashita, Ai Nemoto, Ryosuke Koichi, Masaaki Yamamoto

Signal Processing · Cross-listed from cs.IT

Threat-Aware Energy-Efficient Deployment for Dynamic UAV Networks: A Multi-Agent RL Approach

Ensuring operational safety in threat-prone environments remains a critical challenge for multi-UAV networks serving as aerial base stations. This paper proposes an efficient framework to maximize global energy efficiency (EE) while promoting safe operation through threat-aware clustering and reward-based safety enforcement. The proposed framework is executed in three steps. First, a threat-aware K-means (TAKM) algorithm determines the minimum required UAVs and computes safe initial placements. Second, an optimal matching stage assigns physical UAVs to these centroids to minimize energy expenditure. Third, a threat-aware multi-agent twin delayed deep deterministic policy gradient (MATD3) algorithm dynamically optimizes trajectories, power, and user associations. Simulation results show that the proposed framework achieves zero observed safety violations in the considered scenarios while achieving superior EE and faster convergence than other learning methods and non-clustering baselines. Compared to heuristic optimization, the proposed framework outperforms the greedy particle swarm optimization (GPSO) and achieves performance comparable to that of the optimized PSO (OPSO), while incurring significantly lower online deployment computational complexity. Furthermore, the proposed framework demonstrates effective generalization to unseen user distributions, large UAV fleets, and different threat geometries, while maintaining zero safety violations.
Read the full abstract

All authors Faisal Al-Kamali, Hussein A. Ammar, Francois Chan, James H. Bayes, Yasser Gadallah, Mohamed H. Ahmed

Signal Processing

A Vehicle-Integrated Approach to Digital Twin Deployment for Bridges

The project A Vehicle Integrated Approach to Digital Twin Deployment for Bridges is developing a scalable framework for bridge condition assessment using vibration measurements from a sensorised inspection vehicle. It combines vehicle bridge interaction modelling, state estimation, machine learning and surrogate modelling to reduce reliance on permanent sensor networks. The Old Ada Bridge in Japan is the principal case study because it provides direct and vehicle-based field measurements across multiple experimentally introduced damage states. Progress has been made across three streams. First, forward and inverse Fourier Neural Operator (FNO) models have been demonstrated on a benchmark beam, enabling rapid response prediction and identification of damage location and severity. Second, an inspection vehicle optimisation framework has been developed to select vehicle mass and tyre suspension stiffness that maximise separation between healthy and damaged responses, integrating contact-point reconstruction, damage assessment, Kriging and particle swarm optimisation. Third, an Augmented Kalman Filter-based virtual sensing framework has been developed to estimate moving loads and reconstruct displacement, velocity and acceleration at unmeasured locations from sparse sensors. The next phase will extend and integrate these methods using the Old Ada Bridge model and field data. FNO models will be transferred to the truss structure and evaluated using intact and damaged measurements. Vehicle optimisation will be tested for transferability, while virtual sensing will be extended to the coupled vehicle bridge system to estimate road roughness, vehicle parameters, moving forces and structural responses. Ultimately, these components will form an integrated vehicle-driven digital twin for rapid simulation, response reconstruction and damage assessment under realistic conditions.
Read the full abstract

All authors Mehri Alamdari, Zihao Liu, Jiaji Wang, Kawabe Daigo, Chul Woo Kim

Image and Video Processing

Image Reconstruction from Phase with Untrained Neural Priors

Fourier phase encodes important spatial image structure, but recovering an image without measured spectral magnitude requires additional constraints and leaves absolute intensity ambiguous. We propose a projection-based two-stage framework that combines Fourier-phase and spatial-support constraints with an image-specific neural prior. The first stage alternates constraint enforcement with regularized neural-prior updates, while the second performs phase/support refinement alone with guaranteed convergence. We evaluate two neural-prior implementations on the same 77 microscopy images and compare them with a constraint-only baseline. After 500 final refinement passes, the best-performing variant achieves 31.41 dB pooled PSNR, 35.75 dB mean PSNR, and 0.9531 mean SSIM, improving pooled PSNR by 1.51~dB and reducing pooled MSE by 29.3% relative to the baseline. The results demonstrate the benefit of combining neural guidance with explicit constraint refinement at the evaluated iteration budget, while showing that lower phase residual alone does not guarantee greater reconstruction accuracy.
Read the full abstract

Systems and Control

Certificate-Carrying Distributed Model Predictive Control on Product Manifolds with $\mathrm{SO}(3)$

This paper studies constraint certification in synchronous distributed model predictive control (DMPC) when neighboring predictions change between sampling instants. Before the parallel local solves, each agent communicates a shifted prediction and an announced update budget. A hard trajectory trust region makes that budget enforceable, while an edge-wise feasibility cap computed from the shifted packets keeps the fallback feasible without using any current optimizer output. Distance and relative-attitude constraints are tightened with explicit Lipschitz constants and two budget layers: one accounts for the simultaneous neighbor update and the other retains a checkable shift reserve. We prove hard pairwise constraint satisfaction and recursive feasibility under stated nominal-execution and terminal assumptions, give the additional residual caused by execution error, and derive a local practical value-decrease bound. A spacecraft formation example uses hard terminal and pairwise constraints, a geodesic relative- attitude constraint on $\SO$, and reproducible terminal-set checks. Comparisons with fixed, trajectory-only, and windowed online margins show that the proposed budget reduces conservatism while preserving a positive shifted-feasibility margin.
Read the full abstract

All authors Shengjun Zhang, Tingyi Liu, Lei Xu, Tao Yang

Thursday 24 September

Systems and Control · Cross-listed from cs.LG

HydroSphere: A Framework for Governed, Self-Healing Wastewater Infrastructure

Rapid industrialization and urban growth are increasing pressure on water quality and wastewater treatment systems, while conventional treatment plants often rely on static monitoring and control strategies that cannot easily adapt to changing pollutant conditions. This paper presents HydroSphere, a governed, data-driven framework for real-time water quality monitoring, forecasting, treatment optimization, and fault recovery. HydroSphere is evaluated using 2.82 million water-quality measurements collected between 1940 and 2023. The framework integrates three main components. First, a hybrid TCN-LSTM model performs multi-step forecasting across seven water-quality parameters, achieving an RMSE of 0.1417, MAE of 0.1047, and R2 of 0.3596. Second, the Adaptive Dosage Optimization Module uses PPO reinforcement learning to adjust chemical dosing, achieving a mean step reward of 1.059 compared with 1.017 for a fixed-dose baseline. The results also show that unconstrained reward optimization can lead to excessive dosing, demonstrating the need for explicit operational safeguards. Third, the SHADE anomaly detection module uses a deep autoencoder to identify sensor and process anomalies, achieving an F1 score of 0.651 under controlled fault injection. HydroSphere combines these capabilities with tiered governance, deterministic safety bounds, and human oversight to support safer and more adaptive water infrastructure. The framework provides a scalable foundation for intelligent wastewater management and supports the objectives of UN Sustainable Development Goals 6 and 13.
Read the full abstract

All authors Prabu, Fancy C, Suresh A, Srini Ramaswamy

Systems and Control · Cross-listed from cs.LG

Bounded Autonomy and Verifiable Safety for Agentic AI Enabled Automation

Agentic AI-enabled automation cannot be safely deployed in high-stakes environments on probabilistic reasoning alone. A recurring risk is epistemic drift: as reasoning deepens, system behavior may move away from subject-matter-expert constraints for safe operation. This paper presents BRaVeS, a bounded reasoning and safety-governance framework termed the Defensible Next-Gen Reasoning System (DNRS). BRaVeS encodes SME-defined constraints as invariant anchors, proposes MoDA-Style (Mixture of Depths Attention) depth-aware access as a candidate mechanism for keeping these anchors visible during inference, and uses a state hierarchy (SMARtAutonomy) to reduce autonomy as epistemic risk increases. To formalize bounded recovery, we introduce the Lyapunov-Bounded Consensus Framework (LBCF), which maps continuous epistemic-risk signals into a finite K-bag abstraction and applies shielded state transitions that enforce Lyapunov-style energy descent or route the system to a human-mediated terminal state. The formal convergence result applies to the finite LBCF abstraction under fixed thresholds and feasible-shield assumptions; it does not prove safety of the full continuous neural activation space. We evaluate the framework through a discrete event Monte Carlo simulation using HAI 22.04 industrial-control-system time-series data with synthetic noise and sensor-degradation regimes. Across the tested parameter-grouping strategies and thresholds, the LBCF process achieved finite-step convergence and no safety-guard violations. These results provide simulation-based evidence that bounded governance behavior can be enforced under the stated abstraction, while motivating future work on deployed transformer implementations, live human-in-the-loop validation, and broader adversarial settings.
Read the full abstract

Systems and Control

Vulnerability-Weighted Routing of Timing-Critical Nets for Configuration-Upset-Resilient SRAM-Based FPGAs

Conventional FPGA routing optimizes timing, congestion, and routability but does not distinguish routes with similar nominal performance and substantially different susceptibility to configuration-induced delay degradation. This paper presents a vulnerability-weighted routing methodology for SRAM-based field-programmable gate arrays (FPGAs) that incorporates predicted routing-fault severity directly into the routing objective. A continuous vulnerability cost relates delay perturbations caused by electrically attachable dormant routing resources to the available downstream timing slack, while a complementary configuration-concentration term discourages excessive localization of vulnerable resources. To limit implementation disruption, only the highest-risk nets are selectively ripped up and rerouted while unaffected routes remain fixed. The method is implemented on a Zynq UltraScale+ XCZU7EV using a Vivado/RapidWright-based flow and evaluated across four routed benchmarks against commercial timing-driven routing, vulnerability-agnostic rerouting, and binary vulnerable-resource avoidance. Controlled configuration-equivalent perturbations provide hardware-level validation. The proposed method reduces aggregate configuration-induced timing vulnerability by 41.7% with approximately 1.0% nominal timing degradation and captures 85.8% of the vulnerability reduction obtained at the expanded routing budget by rerouting only the highest-risk 5% of eligible nets. The results demonstrate that continuous vulnerability information can improve configuration-upset resilience with limited impact on nominal routing quality.
Read the full abstract

Audio and Speech Processing

When Intent Arrives Late: A Benchmark for Full-Duplex Speech Models under Delayed Intent Revelation

Native full-duplex speech models can respond before a user finishes speaking, making the behavior of the model depend on both the final utterance and when intent-defining information arrives. Existing benchmarks primarily evaluate turn-taking mechanics, whereas safety evaluations typically assume that the complete request is observed prior to response generation. We introduce LateIntent-Bench, a matched-pair benchmark for delayed intent revelation. A shared ambiguous prefix precedes either a benign or a harmful continuation, using a controlled pause to delay when the branches become distinguishable. We define the Premature Response Rate (PRR) to measure whether response onset precedes the revelation of intent. Joint evaluation of harmful and benign engagement distinguishes changes in safety selectivity from general losses in responsiveness. Across 3,136 sessions, four native full-duplex models exhibit distinct response-timing patterns under delayed intent. Three models show increased harmful engagement while maintaining benign responsiveness, whereas one loses engagement with both branches. Inserting a 1.5s silence after intent revelation keeps PRR near the no-pause baseline and substantially reduces changes in harmful engagement. These results demonstrate that evaluating fully specified requests alone fails to capture emerging timing patterns, highlighting the necessity of assessing delayed intent revelation in full-duplex models. The code will be publicly released soon.
Read the full abstract

All authors Yang Xiao, Tianyi Peng, Hanyu Meng, Ting Dang

Systems and Control

ThermE: Predictive Management of Shared Thermal Headroom for Sustained LLM Inference on Thermally Constrained Edge SoCs

Compact edge system-on-chip (SoC) platforms increasingly run sustained LLM inference under thermal constraints, while their CPU, GPU, and RAM share a cooling path. Prefill and decode therefore consume shared, time-varying thermal headroom, yet vendor governors react only near hardware throttling thresholds without knowledge of request state or upcoming work. A control decision that improves current performance can thus consume headroom too quickly and degrade subsequent service. In this paper, we present ThermE, a runtime system that predicts and jointly manages shared thermal headroom for sustained LLM inference on edge SoCs. Its Fast LLM-to-Heat Compiler maps the model, requests, runtime state, and candidate actions to domain heats without executing LLMs. A partial differential equation (PDE)-constrained Headroom Predictor uses ThermPINN for offline thermal identification and a Reduced Headroom Predictor (RHP) for low-overhead online uncertainty-calibrated headroom forecasts. An Uncertainty-Aware Action Scheduler then selects actions that balance serving quality and future headroom. We implement ThermE atop vLLM and evaluate it across four LLM inference workloads. The results show that ThermE reduces TTFT and TPOT by 40.55% and 12.17%, respectively, relative to vLLM. It achieves a 5.70% SLO violation rate, compared with 12.30% for the strongest baseline, while its predictor obtains a 1.94 $^\circ$C MAE with 11.50 ms overhead.
Read the full abstract

Audio and Speech Processing

Optimal transport meets speech: a tutorial review

Optimal Transport (OT) provides a principled framework for comparing and transforming probability distributions while preserving geometric structure. Recently, OT has gained significant attention in machine learning due to its ability to measure discrepancies between distributions, even when their supports do not overlap, making it effective for tasks such as generative modeling, domain adaptation, and transfer learning. Despite its success in fields such as computer vision and natural language processing, OT remains relatively underexplored in speech research. Speech signals present unique challenges, including temporal dynamics, speaker variability, noise, reverberation, and heterogeneous multimodal representations involving audio, text, and visual information. These factors often lead to distribution mismatches, where OT offers a natural framework for alignment and interpretation. This work aims to promote broader adoption of OT in speech processing by: (1) reviewing OT foundations through intuitive physical interpretations and highlighting connections to modern generative models; (2) presenting computational algorithms suitable for deep learning frameworks; and (3) demonstrating OT applications in cross-domain and cross-modal speech tasks, including speech enhancement, automatic speech recognition, language and speaker recognition, and audio spoof detection. We highlight OT's strong potential for addressing distributional variations in real-world speech applications.
Read the full abstract

Audio and Speech Processing

Cross-Modal Knowledge Distillation for Acoustic Pedestrian Detection

Audio-only pedestrian detection is attractive for urban sensing but limited by weak acoustic cues. An appealing strategy is cross-modal knowledge distillation, in which a video teacher supervises the audio student during training so that the deployed model runs on audio alone. Under this task's severe class imbalance and wide video-audio modality gap, however, what such distillation contributes is unclear. We introduce Trust-Filtered Distillation (TFD), which selectively suppresses teacher supervision on pedestrian samples, and interpret its logit formulation as conditional label smoothing under a shared temperature. Across ten distillation configurations under five-fold cross-session validation on ASPED, most methods yield modest gains in macro accuracy accompanied by small changes in PR-AUC. The main effect is higher no-pedestrian accuracy at the cost of lower pedestrian recall. Adding TFD to logit distillation strengthens this trade-off without improving mean PR-AUC. These findings clarify the benefits and limitations of selective cross-modal supervision by distinguishing operating-point shifts from discrimination gains in imbalanced acoustic detection.
Read the full abstract

All authors Yonghyun Kim, Chaeyeon Han, Sancho Gatungay, Subhrajit Guhathakurta, Alexander Lerch

Image and Video Processing

Beyond MSE: Rician Likelihood Denoising for Self-Supervised Cardiac $T2$ and $T1\rho$ MRI

Magnetic resonance imaging involves an inherent trade-off among spatial resolution, acquisition time, and noise. This trade-off contributes to long scan times and high cost. Deep learning has improved image denoising, but cardiac MRI remains difficult because high-resolution, rapid acquisitions generally lack corresponding low-noise ground truth. Self-supervised denoising offers a potential solution by learning from noisy image pairs or even single noisy acquisitions. However, we show that Noise2Void-style blind-spot denoising, which uses a mean squared error (MSE) loss and assumes zero-mean, independent and identically distributed (i.i.d.) noise, is poorly suited to MR magnitude images. When applied to short-axis $T2$-weighted and $T1\rho$-weighted cardiac MRI with synthetic Rician noise, it produces biased denoised images and biased parametric maps of $T2$ and $T1\rho$. To address this limitation, we formulate self-supervised denoising as maximum likelihood estimation under a known Rician noise model. This yields unbiased denoisers that are competitive with supervised baselines.
Read the full abstract

All authors Nicholas A. Jacobs, Jason Mendes, Ravi Ranjan, Edward DiBella, Shireen Elhabian

Audio and Speech Processing

PRIME-ANC: Path-Ratio-Informed Modeling for Efficient Neural Filter Synthesis in Active Noise Control

Changes in listener acoustics require active noise control (ANC) filters to be redesigned for new acoustic paths. We introduce PRIME-ANC, a shared neural synthesizer that learns a bounded, path-dependent logmagnitude correction to a regularized path-ratio base. Minimum-phase reconstruction and truncation produce finite-impulse-response (FIR) filters; training optimizes their noise-control performance. Across ten random training/test splits per dataset, PRIME-ANC achieves average held-out one-third-octave reductions of 18.81 and 17.76 dB over 50 Hz-5 kHz on a ten-path dataset and a public earphone database, respectively. On original earphone measurements, it improves upon the path-ratio base by 7.89 dB. Ablation studies support the contributions of both the analytic base and the path-dependent correction. Given calibrated paths for a held-out listener condition, PRIME-ANC generates a path-specific FIR without iterative optimization. With three Gauss-Newton updates, it reaches 21.43 dB reduction, approaching direct weighted least-squares design while producing lower amplification and root-mean-square control output.
Read the full abstract

All authors Yaokun Huang, Chunyang Xu, Haowen Hua, Sen Lin, Shichao Hu, Mengyao Zhu

Signal Processing · Cross-listed from cs.IT

Litter-Masked Block z-Channel: Designing Cover Distributions Against Public-Design Observers

Networking defenses against traffic analysis fill idle slots with cover traffic, yet the physical layer carrying it still hands upper layers the erasure-only model of a silent-idle link, leaving the cover content unexploited. We shape that content: the idle-slot waveform is drawn from the codebook complement under a design distribution, the litter distribution, chosen to defeat a public-design passive observer while the upper layer keeps its block z-channel model. The maximum a posteriori (MAP) receiver reduces to a single threshold on the log-likelihood ratio between the best codeword and the prior-weighted litter aggregate; its errors are silent corruptions the upper layer cannot detect and erasures that retransmission absorbs. Receiver design is thus one-dimensional, minimizing erasure under a cap on silent corruption, and the joint design a Stackelberg game against a Kullback-Leibler (KL) indistinguishability metric, solved by an alternating convex relaxation and corroborated by an independent reinforcement-learning policy. On a rate-$3/4$ $(16,12)$ parity-check (LDPC) code against an observer who knows her own channel but whose channel the transmitter knows only statistically, shaped litter cuts the observer's Stein detection exponent by $41$ to $48\%$ relative to uniform cover, flat over a $10$~dB span and in the activity rate. Below the point where the legitimate link turns reliable, no cover distribution admits an operating point; at that point shaping costs about one percent of throughput, and nothing measurable $2$~dB above it.
Read the full abstract

All authors Bin Han, Wenwen Chen, Yao Zhu, Rafael F. Schaefer, Giuseppe Caire, Hans D. Schotten

Audio and Speech Processing

Oracle Complementarity Is Not Realizable Complementarity in Frozen-Encoder Audio-Visual Emotion Recognition

Complementarity analyses of multimodal systems commonly report an oracle ceiling (the fraction of examples on which at least one unimodal branch is correct) and interpret the gap between it and realized fusion accuracy as recoverable headroom. We show that this ceiling is not a fusion target. Using frozen self-supervised audio and visual encoders with a trained head, we measure the ceiling and realized gain on CREMA-D under two audio encoders (one whose fine-tuning lineage includes CREMA-D, one clean) and on EAV under the contaminated encoder. Replacing the fine-tuned encoder with a self-supervised one triples the headroom, from +0.055to +0.186. A second clean encoder and a controlled degradation of the contaminated one fall on the same curve, so provenance moves the headroom by changing audio-branch accuracy. Concatenation realizes at most about half of the headroom (converted fraction 0.55), and on EAV the converted fraction is indistinguishable from zero despite larger headroom. A learned router is beaten by plain concatenation everywhere. Oracle headroom measures branch disagreement, not a fusion budget.
Read the full abstract

Audio and Speech Processing

Adapting Personalized Speech Enhancement for Low-Latency Audio-Visual Target-Speaker Extraction

Online audio-visual target-speaker extraction aims to remove competing voices while preserving speech quality and bounding lookahead. Existing extractors are built and evaluated for separation on synthetic mixtures, leaving listening quality and meeting behavior largely untested. We introduce Audio-Visual Personalized Voice Quality Enhancement (AV-PVQE), which approaches these requirements from the other direction. We start from a personalized speech enhancement model that reconstructs a requested voice at high quality but confuses the target in 46% of two-speaker mixtures despite clean enrollment. Adding mouth features at its speaker-conditioning input and jointly fine-tuning the visual and reconstruction networks reduces this rate to 1.6%, with no future frames and 20 ms of algorithmic delay. Compared with an online autoregressive audio-visual extractor, AV-PVQE yields separation gains on two synthetic benchmarks and larger gains on recorded meetings, and keeps its advantage on excerpts with more speakers than the fine-tuning mixtures. In personalized P.835 listening tests on two meeting corpora, it improves overall quality over this extractor by 0.57 and 0.63 MOS, with similar mean rating relative to the starting model. Preservation and rejection tests show that it keeps the target intact when no competing voice is present and suppresses competing speech when the target is absent.
Read the full abstract

Signal Processing

FRESHLATENT: Channel-Aware Latent Adaptation for Resource-Constrained Embodied VLM Perception

Mission-critical UAVs increasingly rely on split vision-language model (VLM) perception under tight onboard-resource and wireless-communication constraints. However, corruption of transmitted intermediate features creates a deployment mismatch for clean-trained split interfaces, while stronger channel-aware codecs can impose substantial onboard cost. We present FreshLatent, a lightweight channel-aware latent adapter that trains a power-normalized encoder-decoder through wireless corruption while keeping the surrounding VLM frozen. We formulate deployment around a mission-conditioned perception requirement and embedded interface cost, linking channel quality and communication budget to the operating conditions under which perception remains usable. At 0 dB and the tightest communication budget, FreshLatent improves gIoU and cIoU over clean split compression by 20.79 and 20.87 points, respectively. At the most adverse evaluated SNR (0 dB), across all three communication budgets, FreshLatent recovers 63.5-69.1% of the gIoU improvement achieved by a much heavier, range-trained feature-JSCC codec. On an NVIDIA Jetson AGX Xavier in 10-W mode, FreshLatent uses 37-40x fewer encoder parameters, 7.7-9.9x lower edge-interface latency, and 8.8-10.0x lower edge-interface energy than the heavier codec. Together, these results show that lightweight channel-aware adaptation can recover a substantial fraction of the robustness of a much larger communication interface while broadening quality-valid operation under constrained wireless conditions.
Read the full abstract

All authors Rajat Bhattacharjya, Minwoo Kim, Arnab Sarkar, Tamoghno Das, Sing-Yao Wu, Eli Bozorgzadeh, Marco Levorato, Nikil Dutt

Systems and Control

Resilience Enhancement of Distribution Grids Through a Three-Stage Framework for Scheduling Mobile Energy Storage Systems

As high-impact, low-probability disturbances such as extreme weather events increase in frequency, enhancing power grid resilience has become a critical priority. Mobile Energy Storage Systems (MESSs) have strong potential for helping with this purpose due to their high operational flexibility and fast deployability. They can be rapidly relocated to supply critical loads, support islanded operation, and adapt to changing conditions of the grid, making them a promising addition or alternative to conventional resilience-providing methods. This paper proposes a three-stage framework for scheduling MESS units. In the first stage (Normal Operation), charging/discharging scheduling of MESSs at buses is optimized for economic arbitrage. Upon receiving an early warning signal, the second stage (Proactive Positioning) shifts the focus to resilience. Here, the model determines the optimal staging points within the transportation network to minimize the expected time of arrival at critical loads, considering the probability distribution of line outages. The third stage (Dynamic Relocation) addresses the post-event restoration phase, in which MESSs are relocated in response to grid updates, such as subsequent line failures. Furthermore, a graph neural network-based solution approach is proposed to further enhance computational efficiency in determining the optimal placing of MESS units during the pre-allocation and relocation stages. The simulation results demonstrate that the proposed framework reduces expected energy not served and improves the restoration speed.
Read the full abstract

Systems and Control

Decentralized Gain Learning for Voltage Control via Local Trajectory Convolution

Fast and spatially heterogeneous fluctuations from distributed energy resources call for voltage regulation that is both fast and responsive to changing operating conditions. Local linear Volt/VAR control offers the advantage of promptly regulating voltages without real-time communication, but its performance depends critically on the choice of control gains. Optimizing these gains for a system-level objective, however, typically requires an accurate network model or centralized communication. In this work, we uncover a structural property that enables such optimization using only local measurements. Specifically, by exploiting the self-adjoint structure of the closed-loop voltage dynamics, we show that each component of the system-level objective gradient can be exactly recovered from a self-convolution of the corresponding bus's voltage deviation trajectory. Building on this result, we develop a simple decentralized gain optimization method in which each bus updates its control gain using only its local data trajectory. Simulation results demonstrate that the proposed approach effectively updates the Volt/VAR control gains in response to changing operating conditions while requiring only local information.
Read the full abstract

Signal Processing

Coordinate-conditioned detector-atom network for montage-agnostic EEG channel completion: zero-shot transfer and a regime map

EEG electrode layouts differ across devices, from consumer headsets to 128-channel nets, and learned methods for completing missing electrodes assume a fixed layout and within-dataset training, so they cannot be applied to unseen devices. We propose the coordinate-conditioned detector-atom network (CC-DAN), a self-supervised convolutional dictionary model whose atoms are spherical-harmonic functions of electrode coordinates and whose detector pools observed electrodes with coordinate-dependent weights, making it independent of electrode number, order, and names. In a plug-and-play protocol, a single model pretrained on five public datasets (132 subjects) is applied zero-shot to nine held-out sets with 8 to 128 channels and compared with nearest-neighbor and spherical spline interpolation (SSI), a within-subject oracle, and learned baselines on signal error, band-wise error, and downstream tasks. With five or fewer observed electrodes, sparse real layouts, and hemispheric dropout, CC-DAN beats both interpolations in nearly all subjects and approaches the oracle; on the EGI net and 8- to 16-channel devices it keeps this advantage up to 15 electrodes, whereas SSI is superior with 15 or more electrodes on dense 10-05 montages. CC-DAN is also the most robust to electrode-position errors. Mean-squared-error gains do not carry over to downstream accuracy, and the amplitude shrinkage of MSE-optimal prediction degrades power-based measures, which a variance-matching loss and gain calibration mitigate. The resulting regime map tells practitioners where learned completion pays off and where classical interpolation remains the method of choice, and the plug-and-play protocol lets one pretrained model connect an unseen device to a fixed-montage pipeline without retraining.
Read the full abstract

Signal Processing

Low-Latency Coded Tensor–Matrix Multiplication for Distributed Signal Processing Systems

Large-scale signal processing systems, including aeronautical and aerospace platforms, increasingly rely on tensor operations, whose distributed execution is constrained by communication, memory, and latency bottlenecks. In this paper, we propose a tensor-aware coded computation framework for distributed mode-1 tensor–matrix multiplication that operates directly on tensor subtensors, avoiding explicit unfolding and preserving multi-dimensional structure. For a fixed partitioning configuration, the proposed tensor-PolyDot scheme achieves the same recovery threshold, communication cost, worker-side computation, and memory requirements as conventional matrix-based PolyDot schemes. However, by leveraging multivariate encoding, it replaces high-degree univariate interpolation at the fusion node with structured low-dimensional decoding. This leads to a substantial reduction in decoding complexity and latency, without affecting any other system metric. Additionally, the tensor-based approach improves memory locality and enables parallel decoding across tensor modes, making it well-suited to modern hardware architectures. Numerical results show decoding speedups proportional to the tensor partitioning factor, reaching up to an order-of-magnitude improvement in practical settings. These advantages make the proposed framework particularly relevant to latency-critical aeronautical and aerospace distributed signal processing systems.
Read the full abstract