Audio and Speech Processing
EarLog: Opportunistic Ear Physiology Sensing toward Personalized Hearing Risk Awareness
Yongzhi Huang, Jiayi Zhao and Kaishun Wu
Many headphone users risk irreversible hearing damage during high-load listening, yet current protection still relies on sound-pressure limits. Clinical tests detect damage after it becomes measurable, leaving the risk-accumulating process unobserved. The acoustic reflex (AR), an involuntary response to high acoustic load, marks when the ear enters a protective state. AR traces are weak, and existing measurements require dedicated probes and controlled conditions that are difficult to maintain during everyday audio. We present EarLog, which turns playback, the dominant interference, into an opportunistic probe. EarLog introduces content-conditioned residual sensing to suppress variations caused by nonstationary content, individual ear acoustics, and device response. It identifies weak AR through loudness-dependent changes and localized spectral irregularities in the residual. A resonance-constrained sparse finite impulse response (FIR) model captures the short-window response. Dual-microphone observations, mild-distortion recovery, and cross-session reference updates keep detection usable under low signal-to-noise ratio (SNR), fit changes, and environmental drift. EarLog runs in real time on an embedded controller using a low-cost dual-microphone add-on. We evaluate it with 81 participants, 16 headphone models, and more than 400 hours of recordings. EarLog achieves 99.0% sensitivity and 98.7% specificity on a clinically referenced
benchmark. EarLog records listener-specific AR events during everyday listening, providing data for future longitudinal studies of hearing risk.
Read the full abstract ▾
Audio and Speech Processing
Acoustic Biomarkers for Alzheimer's Disease Detection from Spontaneous Speech
Anh L Tran, Arjee Javellana Restar and Bereket Kindo
Alzheimer's disease (AD) remains a major public health challenge, and early identification is crucial for timely intervention and care planning. Recent work has shown that
speech-based digital biomarkers can capture subtle changes in vocal production, prosody, and timing that often precede or accompany cognitive decline. In this study, we investigate whether acoustic features extracted from spontaneous
speech can reliably distinguish individuals with AD from healthy controls. Using a structured
speech dataset derived from the Cookie Theft picture-description task, we extract clinically informative low-level descriptors spanning prosodic, spectral, and temporal domains, and evaluate a range of machine learning classifiers. Our results indicate that acoustic biomarkers carry strong discriminative value, with the best-performing models achieving clinically meaningful separation between diagnostic groups. These findings support the use of non-invasive, low-cost audio-based screening tools and motivate further work on
multimodal and longitudinal assessment frameworks for dementia risk detection.
Read the full abstract ▾
Image and Video Processing · Cross-listed from q-bio.QM
Optimization of Deep Learning Model for Denoising Dose Profiles in Proton Therapy
Junaid Jawaid, Matteo Spezialetti, Gabriele Libardi et al. (+2 more)
Proton therapy offers a dosimetric advantage over conventional radiotherapy through the Bragg peak, enabling precise dose delivery to the tumour while sparing surrounding tissue. Monte Carlo (MC) simulation provides the most accurate dose calculation but is computationally expensive for clinical workflows. A promising compromise is to simulate with a reduced number of primary histories and recover a high-quality dose distribution through a deep learning denoising step. In this work, we take a 3D multiscale U-Net from recent literature as a case study and investigate two improvements critical for clinical deployment. First, we replace the standard MSE training objective with a differentiable $\gamma$-index-based loss, extended to native 3D operation, and demonstrate that directly optimising the clinical evaluation metric yields a mean Gamma Passing Rate of 99.47% at 2mm/2% and 95.67% at 1mm/1% on the test set, compared to 92.10% and 71.13% for MSE. Second, we apply L1-norm structured filter
pruning to reduce the model's
memory footprint, achieving a 13.3% parameter reduction with negligible performance loss under partial
pruning, and 20% reduction under full-block
pruning after fine-tuning.
Read the full abstract ▾
All authors Junaid Jawaid, Matteo Spezialetti, Gabriele Libardi, Giovanni Stilo, Filippo Mignosi
Audio and Speech Processing
Intonation Perception in Real and Synthetic Speech across Varying Familiarity Levels: A Pilot Study of Equivalence Assessment
Hanrui Zhou, Gaoyuan Zhang, Yixiang Chen et al. (+4 more)
Language training relies on a corpus constructed by a large number linguistic materials. AI-powered voice clones provide a way to construct the corpus with relatively low cost. Singing voice conversion (SVC) model is used to generate synthetic voices. This study compares participants' performances on natural and synthetic
speech in two experiments, similarity perception and intonation recognition. In the accuracy of similarity perception task, a significant interaction between
speech type and intonation is found, suggesting that question may serve as a cue for speaker identification but may be influenced by synthetic features. In the accuracy of intonation recognition task, a significant interaction between
speech type and familiarity is observed, indicating that
speech type affects how much familiarity contributes to voice processing.
Read the full abstract ▾
All authors Hanrui Zhou, Gaoyuan Zhang, Yixiang Chen, Yujie Xing, Feng Xu, Xurong Xie, Hui Chen
Signal Processing
PI-AMFM: Permutation-Invariant Learning for Variable-Cardinality AM-FM Mode Decomposition in Biomedical Signal Analysis
Youngsun Kong and Ki H. Chon
Physiological recordings often contain nonstationary oscillatory components whose number and dynamics vary across signals. Amplitude- and frequency-modulated (AM-FM) representations are well suited to characterizing such dynamics and have shown broad utility in biomedical signal analysis. Recent approaches have incorporated neural networks to learn mode decomposition patterns from data, but component cardinality is often predefined or determined through separate stopping or selection mechanisms. We propose a permutation-invariant neural framework for variable-cardinality AM-FM mode decomposition (PI-AMFM). PI-AMFM combines a multiscale temporal encoder, Mamba backbone, and component-presence estimation, with permutation-invariant Hungarian matching during training. On synthetic AM-FM signals, PI-AMFM achieved lower decomposition, instantaneous-frequency, reconstruction, and mode-count errors than the compared methods while preserving the overall trajectory pattern in a crossing-chirp example. On photoplethysmographic recordings, recovered modes captured cardiac and respiratory dynamics despite training only on synthetic signals. These results support the feasibility of PI-AMFM for variable-cardinality decomposition of nonstationary biomedical signals.
Read the full abstract ▾
Image and Video Processing
Spatially Gated Diffusion for Localized Counterfactual Chest Radiograph Editing
Kamran Ullah Afaq and Basit Raza
Editing a chest radiograph requires completing the requested change while preserving unrelated content. We study a latent
diffusion editor with an instruction-independent source trajectory and an instruction-conditioned editing trajectory. A learned gate mixes their post-sampler candidates at each executed step, and a separate image-space mask composites the decoded proposal with the source. On 2,400 MIMIC-derived requests, 2,244 outputs met the joint target, preservation, quality, and coverage rubric (93.5%), and target completion was 97.4%. Joint validity exceeded a matched composition-only control by 1.7 percentage points (paired patient-cluster 95% interval, 0.6–2.8). At a fixed learned mask, the learned-gate proposal improved joint validity by 1.6 points (0.8–2.4); at a fixed learned proposal, the learned mask improved it by 2.6 points (1.7–3.5). Across three training seeds, mean joint validity was 93.5% with a 0.3-point sample standard deviation. A blinded 240-request assessment yielded adjudicated joint validity of 93.3% for the full editor and 91.7% for composition only. Protected-region mean absolute error decreased from 0.0190 in the raw proposal to 0.0075 after composition. These findings distinguish recurrent-gating effects on the proposal from preservation through final composition in the assessed cohort.
Read the full abstract ▾
Systems and Control
Distributed Adaptive Neural Interval Observers for Unknown Nonlinear Systems
Tien Dat Vu, My Nguyen Bach, Phuoc Vinh Nguyen et al. (+1 more)
This paper develops a distributed adaptive neural interval observer for unknown nonlinear systems with locally incomplete measurements. Each sensor node constructs lower and upper state estimates using its local output and neighboring observer information, while unknown nonlinear dynamics are approximated by adaptive neural models. A distributed adaptation mechanism guarantees bounded estimation and weight errors, whereas a cooperative realization preserves the componentwise interval property. To enhance neural-weight convergence without requiring persistent excitation, a finite experience-replay integral concurrent-learning mechanism is incorporated into the adaptation law. For non-Metzler error dynamics, a Sylvester-based coordinate transformation is introduced to recover a Hurwitz–Metzler distributed realization. The theoretical developments are validated through a nonlinear distributed estimation example.
Read the full abstract ▾
All authors Tien Dat Vu, My Nguyen Bach, Phuoc Vinh Nguyen, Minh Doan
Systems and Control
Event-Triggered Practical Fixed-Time Integral Reinforcement Learning for Unknown Nonlinear Systems
Tien Dat Vu, My Nguyen Bach and Minh Doan
This paper develops an event-triggered fixed-time integral reinforcement learning framework for optimal control of unknown nonlinear systems. An integral data-driven identifier is first used to reconstruct the unknown dynamics, after which an inverse-optimal formulation is employed to construct a fixed-time running cost. A learning law satisfying the practical fixed-time property is then derived. Previously collected data, or data obtained during a finite excitation interval, are stored in an experience-replay buffer and incorporated into the weight-update law. This avoids the persistent-excitation condition, which is often difficult to satisfy in practical operation. To reduce communication and control updates, an event-triggered mechanism is introduced. The paper shows that, under the event-triggered implementation, the closed-loop system still achieves practical fixed-time stability, while the proposed triggering rule guarantees the exclusion of Zeno behavior. Finally, a nonlinear example is presented to verify the theoretical results developed in the paper.
Read the full abstract ▾
Signal Processing · Cross-listed from quant-ph
Real-Time Adaptive Filtering and the Boxcar Limit in Superconducting Qubit Readout
Hans Johnson, Tanay Roy, Leonardo Bove et al. (+3 more)
Control and readout systems for superconducting
quantum computing support real-time control and high-fidelity
qubit-state discrimination. We examine when additional digital processing improves readout fidelity. A common baseline for dispersive readout uses a boxcar averager to compute a uniformly weighted average over a fixed window, followed by threshold-based state assignment. In this paper, we evaluated an adaptive finite-impulse-response (FIR) filter trained in real time using the least-mean-squares (LMS)
algorithm on a field-programmable gate array (FPGA)
quantum controller. We also analyzed measured readout shots offline to compare the frozen filter with boxcar averaging and to evaluate per-sample weighting and sequential detection. For the fixed FIR filters tested, we show summing the filtered samples gives the boxcar sum multiplied by one complex number, apart from a small edge correction. We call this the kernel collapse. The multiplication scales the state separation and noise equally without improving state discrimination, and we call this the boxcar limit. The frozen LMS filter reduces trace noise, but same-shot offline comparisons show no resolved improvement in readout fidelity at the tested operating points. On a 2D transmon, tuning the integration window accounts for much of the observed improvement over full-window integration, highlighting the importance of an optimized boxcar baseline when evaluating per-sample weighting. We then examine how changing the decision method can reduce calibration effort and decision time. On a higher-fidelity 3D ancilla readout, model-based template estimation reaches the same fidelity target as direct averaging with about 30 rather than 300 labeled shots per state. Offline sequential replay assigns most shots before the acquisition window ends, with contrast fidelity within an absolute +/-0.5% tolerance of the boxcar on the primary...
Read the full abstract ▾
All authors Hans Johnson, Tanay Roy, Leonardo Bove, David van Zanten, Silvia Zorzetti, Jafar Saniie
Audio and Speech Processing
MAV-C: A Framework for the Joint Objective Estimation of Audio-Visual Complexity in Immersive Virtual Environments
Luca Resti, Amelia Gully, Michael McLoughlin et al. (+2 more)
We present the Motion-Aware Audio-Visual Complexity Metric (MAV-C), a reference-free framework for the joint objective estimation of audio-visual complexity. The metric combines
entropy-based audio features (temporal, spectral, and spatial) with visual features (Sobel gradient magnitude, chromatic uniqueness, and optical flow) via a parametric fusion stage, producing a continuous joint complexity score CAV (t) [0,1]. We validate MAV-C on two
datasets: a controlled synthetic corpus (SYN) of stimuli with known signal characteristics and a naturalistic gameplay corpus (GAM) of 60 clips drawn from the SAFEPLAY-X
dataset. On SYN, the metric exhibits strong validity: the audio score CA and visual score CV are each insensitive to changes in the opposite modality (CoV < 0.003), the joint score CAV spans [0.00,0.90] across all parameter combinations, and single-axis feature sweeps produce monotone trajectories (Spearman up to 0.995). On GAM, CV differs significantly across content categories (Kruskal-Wallis p = 0.021) while CA does not, and the two sub-scores are uncorrelated (r = 0.03), confirming they operate on independent signal dimensions. OFAT sensitivity analysis identifies a two-tier parameter hierarchy, with modality balance (wa) and visual regularization (v) as most significant tunable parameters. Full subjective calibration is planned as future work.
Read the full abstract ▾
All authors Luca Resti, Amelia Gully, Michael McLoughlin, Gavin Kearney, Alena Denisova
Signal Processing
ARCTAN: Arbitrary RF Containment Using Tactical Aerial Networks and Differentiable Ray Tracing
Samuel Rivera, Zhihui Gao, Yiming Li et al. (+1 more)
Aerial base stations (ABSs) can rapidly establish connectivity in ad hoc, infrastructure-deprived environments, but their broadcast, line-of-sight transmissions leak far beyond the intended service area, exposing communications to passive eavesdropping and interference. Prior physical layer defenses based on cooperative jamming typically assume known eavesdropper locations, simplified statistical channels, or continuously repositioned jammers. We instead pose the problem as a
radio frequency (RF) containment: confining usable signal to a user-defined, arbitrarily-shaped target zone while denying it elsewhere independent of eavesdropper location. We present ARCTAN, a gradient-based optimization framework that jointly optimizes the position, orientation, and transmit power of stationary ABSs and cooperative jammers (CJs) by backpropagating through site-specific, differentiable 3D ray traced channels. Evaluated in a high-fidelity digital twin across three target zone geometries, ARCTAN achieves a mean in-zone SINR of approximately 10 dB while reducing mean out-of-zone SINR from 13-16 dB to -3-5 dB, and suppressing signal-leakage ratios from over 93% to below 46% requiring at most 10 of 12 candidate CJs.
Read the full abstract ▾
All authors Samuel Rivera, Zhihui Gao, Yiming Li, Tingjun Chen
Image and Video Processing
Synthetic-to-Real Transfer in Cerebral Microbleed Generation and Segmentation
To-Liang Hsu, Ting-Yu Lai, Ching-Ting Lin et al. (+2 more)
The development of automated cerebral microbleed (CMB) detection models is hindered by the low prevalence of CMBs and the high cost of expert annotation. To address this limitation, we developed a synthetic CMB generation pipeline and investigated the effectiveness of synthetic lesions for training deep learning detectors. Models trained solely on synthetic data achieved substantial detection performance, reaching approximately 87% of the lesion sensitivity of their real-trained counterparts. We further investigated the complementary roles of synthetic and real data under different training paradigms, revealing that a significant performance gap remains. Moreover, we found that successful synthetic-to-real transfer is strongly dependent on the downstream detection architecture, providing new insight into both the potential and limitations of synthetic data for CMB detection.
Read the full abstract ▾
All authors To-Liang Hsu, Ting-Yu Lai, Ching-Ting Lin, Chun-Hao Huang, Wei-Chun Wang
Audio and Speech Processing
Articulatory Source-Filter TTS: Physically Grounded Control through Vocal Tract Kinematics
Jesuraj Bandekar, Shinji Watanabe and Prasanta Kumar Ghosh
Modern neural text-to-
speech (TTS) systems achieve remarkable acoustic fidelity but act as black boxes, offering little interpretable control over the vocal tract filter. We propose a controllable source-filter TTS architecture grounded in articulatory kinematics. An Acoustic-to-Articulatory Inversion (AAI) model, enhanced by large-scale pretrained representations, generates kinematic pseudo-trajectories for a large TTS corpus. These trajectories condition the filter response, while predicted pitch and energy contours parameterise the glottal source. Source and filter are predicted independently, and the source is refined by an Optimal Transport Conditional Flow Matching (OT-CFM) module before recombination into the final spectrogram. Our model achieves intelligibility and naturalness competitive with similarly sized baselines, with only a modest spectral fidelity cost in exchange for explicit control. Evaluations reveal clear source-filter disentanglement, enabling stable prosodic scaling and cross-speaker source/filter recombination, where F0 remains tied to the source speaker while vocal-tract characteristics follow the filter speaker. Finally, direct manipulation of articulatory trajectories enables fine-grained control, such as accent modification, offering a new direction for interpretable
speech synthesis. Audio samples: https://coding-phoenix-12.github.io/ArticulatorySFTTS/
Read the full abstract ▾
Audio and Speech Processing
Frequency-Weighted Soft-Constrained Spatially Selective Active Noise Control for Open-Fitting Hearables
Tong Xiao, Reinhild Roden, Matthias Blau et al. (+1 more)
Recently, soft-constrained spatially selective active noise control (SSANC) for open-fitting hearables has been proposed to enable a flexible trade-off between noise reduction and desired-
speech preservation using a scalar trade-off parameter across all frequencies. This paper generalizes soft-constrained SSANC by introducing frequency-dependent weighting of the desired-
speech preservation error while maintaining low-delay time-domain active control. Frequency-dependent weightings based on the
speech intelligibility index (SII), the modified intermediate reference system (MIRS), the long-term average
speech spectrum (LTASS), and a signal-dependent oracle are compared with scalar weighting using measured acoustic impulse responses in a multi-speaker scenario. The results show that LTASS and oracle weightings provide the largest
speech-intelligibility improvements and comparable
speech-quality improvements at substantially lower distortion levels than those obtained with SII and MIRS weightings. While the oracle weighting provides the best overall perceptual trade-off, LTASS weighting provides a comparable perceptual trade-off without requiring the clean desired-
speech signal.
Read the full abstract ▾
All authors Tong Xiao, Reinhild Roden, Matthias Blau, Simon Doclo
Systems and Control · Cross-listed from cs.AI
Meta-Multi-Agent Reinforcement Learning for Fast Adaptation of Interactive Policies with Applications to Autonomous Driving
Huiwen Yan, Kyriakos G. Vamvoudakis and Mushuang Liu
This paper develops a meta-multi-
agent reinforcement learning (meta-MARL) framework to enable fast adaptation of interactive policies in a multi-
agent system (MAS). Meta-reinforcement learning (meta-RL) enables
agents to rapidly adapt to new tasks/environments using a bi-level optimization mechanism. However, existing meta-RL generally focuses on single-
agent systems. Extending these frameworks and
algorithms to multi-
agent systems poses additional challenges, as tasks are characterized by not only the environment but also
agents' strategic interactions. To address these challenges, we model multi-
agent reinforcement learning (MARL) problems as Markov games (MGs) and develop a meta-MARL framework for rapid interactive policy adaptation across a distribution of MGs. A new concept, called meta-NE, is defined to describe the desired solution concept in a meta-MARL problem. Sufficient conditions for the equivalence between a meta-NE and a stationary point of the gradient-play-based meta-MARL
algorithm are established. Our evaluation on autonomous-driving tasks demonstrates that the proposed meta-MARL method achieves faster adaptation than pretrained MARL baselines, validating the effectiveness of our framework.
Read the full abstract ▾
Image and Video Processing
Nonparametric Distribution Matching for Self-Supervised Whole-Slide Image Condensation
Duong M. Nguyen, Trong Nghia Hoang, Hang Thi Nguyen et al. (+3 more)
Histological whole-slide images (WSIs) are central to computational pathology but pose severe computational challenges due to their extremely high resolution, often spanning several gigabytes per slide. To enable scalable learning, existing methods apply self-supervised data condensation to reduce computational cost, but typically rely on heuristic prototype learning and do not explicitly preserve learning-relevant feature distributions for downstream tasks. In response, we introduce a principled reformulation of WSI condensation as a distribution-matching problem under a fixed representational lens, and develop NICER, a tractable approximation framework based on a nonparametric prior with slide-adaptive capacity. Experiments on five histopathology
datasets, together with clinical evaluation from a board-certified pathologist, show that NICER consistently outperforms prior methods, achieving an average accuracy improvement of 7.44% while offering improved efficiency-accuracy trade-offs, highlighting the benefits of principled, distribution-aware condensation for scalable histological representation learning. Source codes are available in https://github.com/nmduonggg/NICER.
Read the full abstract ▾
All authors Duong M. Nguyen, Trong Nghia Hoang, Hang Thi Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Minh N. Do
Audio and Speech Processing
Silence-the-Mimic: Accelerating Imperceptible Perturbation Generation Against Voice Cloning
Runqiu Xu
Deep neural network-based Voice Conversion (VC) and Text-to-
Speech (TTS) models have rapidly advanced, enabling realistic voice cloning with minimal input data. Such capabilities raise serious concerns over unauthorized cloning of speaker identities and the associated
privacy and security risks. Current imperceptible
adversarial protection methods rely on quality control losses that are highly sensitive to hyperparameter tuning and computationally expensive due to lengthy optimization. To address these limitations, we propose a fast protection method that generates perceptually constrained perturbations in the frequency domain under a psychoacoustic masking-based constraint. Our approach strictly enforces perceptibility bounds during
adversarial training, eliminating the need for iterative quality balancing and significantly reducing computational cost. Experiments on multiple state-of-the-art VC and TTS models show that STM achieves competitive or superior protection performance with substantially better perceptual quality and up to $45.3\times$ speedup over existing white-box baselines. These results demonstrate the effectiveness of frequency-domain perturbations with perceptual constraints as a practical paradigm for protecting against voice cloning.
Read the full abstract ▾
Audio and Speech Processing
Interpretable Destination-Aware Synthesizer Modulation Recovery
David Liu, Giulio Cengarle, David Cooper et al. (+2 more)
Synthesizer programming is a challenging task, particularly in configuring modulation: the time-varying control of parameters such as pitch, filter cutoff, and oscillator level by modulators such as Low-Frequency Oscillators (LFOs). Prior work has shown that modulation can be reconstructed from the clean audio through curve matching, yet the recovered parameters are often untransferable to modern synthesizers. We present an interpretable destination-aware modulation and waveform recovery pipeline that mirrors how musicians often recreate sounds on a synthesizer. First, the system identifies the modulated destinations; then it recovers the LFO shape associated with each destination, and estimates the oscillator waveform to match the source timbre. The training is supported by a differentiable synthesizer that incorporates a noise oscillator, and more modulation options than previous work. We carefully train the models for optimal perceptual quality, with extensive exploration of perceptual losses and
adversarial training. Through objective and subjective evaluation, we show that predicting the destinations correctly is essential, our model excels in the modulation focused inverse synthesis tasks, and our Gammatone loss and CQT discriminator significantly outperform the traditionally-used MSS loss in improving perceptual quality. We provide audio samples from the subjective test.
Read the full abstract ▾
All authors David Liu, Giulio Cengarle, David Cooper, Mark Vinton, Haici Yang
Systems and Control · Cross-listed from cs.LG
Learning Linear Systems under Heavy-Tailed Noise: A Non-Asymptotic Analysis from A Single Trajectory
Xiaomian Yang and Sungho Shin
We establish non-asymptotic sample complexity bounds for the least-squares estimation of vector autoregressive models for exponentially stable systems with heavy-tailed noise based on a single observed trajectory. By assuming i.i.d. noise, bounded noise covariance, and persistent excitation, we show that the estimation error is $\widetilde{\mathcal{O}}(r^{1/2}T^{-1/2+1/p})$ under bounded $p$th moment for $p > 2$, where $T$ is the number of samples, $r$ is the noise dimension, and $\widetilde{\mathcal{O}}(\cdot)$ hides logarithmic terms. We also introduce a unifying approach to sample complexity analysis applicable to broad classes of noise distributions and showcase this by deriving error bounds for sub-exponential and sub-Gaussian noise distributions. Finally, we specialize our analysis to autoregressive models with exogenous inputs and show that the dimension factor of the error bound is independent of the model order.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.SD
PLACE: Positional Latent Adaptation via Conditioned Embeddings for Binaural Audio Generation
Tiernon Riesenmy, You Zhang, Gautam Bhattacharya et al. (+1 more)
We present PLACE, a method that extends the pretrained any-to-audio model AudioX for binaural generation from arbitrary combinations of text,
video, and optional audio
prompts. PLACE augments
video conditioning with Perception Encoder Core features, aligns text and
video representations to derive spatial cues, and applies a conditioning-dependent low-rank transformation to the generated latent. The adapter is supervised via decoded-audio interaural level and time difference objectives. Trained on MRSAudio, PLACE improves most metrics over ViSAGe on FAIR-Play and achieves an improved SpatialCLAP score over SpatialSonic on the BEWO-1M Single Static test split. Listener evaluations favor PLACE for
video-to-audio and out-of-distribution text-to-audio generation, demonstrating flexible
multimodal control and improved spatial consistency.
Read the full abstract ▾
All authors Tiernon Riesenmy, You Zhang, Gautam Bhattacharya, Andrea Fanelli
Systems and Control · Cross-listed from cs.AI
Spatial Strategies, Not Actions: Vector-Quantized Geodesics as Tools for LLM-Driven Agents
Gabriel Turinici
Large
language model (LLM) based
agents are often criticized for lacking spatial understanding and mainly exploiting statistical text patterns. We investigate their spatial comprehension through an architecture combining geometrical tools with a LLM serving as a high-level orchestrator in grid-world environments. The
agent first collects geodesic trajectories, which are then vector-quantized to extract a representative subset. Offline, the LLM associates a natural
language description of the underlying behavioral patterns to each selected trajectory, making it a tool. Online, the LLM chooses the appropriate tool conditioned on the current state and goal. Low-level control is handled by primitive actions that execute the trajectory associated with the tool. From an agentic AI perspective, this approach separates learning into two levels: tool discovery is handled through unsupervised quantization of trajectories, while
reasoning and decision-making are handled by the LLM. We test the approach in a partially observable dynamic 2D grid environment with an open
vision-
language model (Qwen3.6-35B-A3B). Pairing the geometry-derived tool library with an
agent-centered zoom tool and a collision detection tool lets a fast, non-
reasoning configuration match the goal-reaching rate of a much more costly chain-of-thought version, while cutting the cost of a decision from minutes to seconds.
Read the full abstract ▾
Audio and Speech Processing
End-to-End Historical Music Restoration in Latent Space
Steven Cho, Junghyun Koo, Raphael Lafargue et al. (+3 more)
Historical music restoration (HMR) has almost exclusively focused on constrained problems such as Super-Resolution or the restoration of solo pieces, under-exploring the general task of restoring orchestral historical music, which has multiple instruments. This under-exploration is largely because the HMR domain, early-20th-century recordings, has no pre-degradation ground-truth pairs, making the restoration task unsupervised and more challenging. This paper presents a supervised end-to-end orchestral HMR
benchmark by exploring both the synthetic degradation functions and the end-to-end generative deep-learning restoration methods. We simulate the historical recording degradation chain more faithfully than prior work, which makes orchestral restoration into a tractable supervised problem. A latent flow-matching model trained on the resulting synthetic pairs outperforms existing HMR baselines on intrusive, non-intrusive, and subjective evaluations. We also curate and release a 9.3-hour license-free, unpaired, historical classical-music test set, along with code and audio demos.
Read the full abstract ▾
All authors Steven Cho, Junghyun Koo, Raphael Lafargue, Tushar Dhyani, Eloi Moliner, Yuki Mitsufuji
Systems and Control
Aging-Aware Online Distributed Scheduling for Lifecycle Carbon Reduction in Geo-Distributed Data Centers
Junyu Lin, Wenjie Liu, Shunbo Lei et al. (+3 more)
The rapid proliferation of data centers (DCs), driven by cloud computing and artificial intelligence (AI), has led to massive energy demand and carbon emissions, posing significant sustainability challenges. Carbon-aware optimization in geographically distributed data centers has been widely studied. Most existing approaches mainly focus on operational carbon emissions from server usage. However, existing literature often ignores workload-induced thermal stress, which accelerates nonlinear hardware degradation. This leads to more frequent server replacements and ultimately increases embodied carbon emissions. To address these limitations, we propose a comprehensive carbon life-cycle modeling framework for distributed data centers. Apart from operational carbon emissions, this work combines workload scheduling with a utilization-dependent exponential aging model to evaluate long-term carbon costs from server degradation. In order to solve the proposed optimization model in an online and
privacy-preserving manner, an enhanced Lyapunov framework with time-varying queue shifting (TVQS) is first introduced to handle system uncertainties. Then, a zero-sum perturbation-based alternating direction method of multipliers (ZSP-ADMM) framework is developed to enable distributed coordination across geographically separated data centers while protecting locally exchanged workload information. Simulation results demonstrate that the proposed approach achieves up to 13.0% lower carbon emissions and 12.6% lower operational costs compared with
benchmarks.
Read the full abstract ▾
All authors Junyu Lin, Wenjie Liu, Shunbo Lei, Wentian Lu, Jianhui Wang, Junhong Liu
Image and Video Processing
CPathOGen: Spatially and Morphologically Controlled H&E Counterfactuals for Probing Pathology Models
Samarth Singhal and Varang Rai
Computational pathology models infer biologically and clinically meaningful outcomes from histology, but their predictions are shaped by complex, intertwined tissue signals whose roles are important to understand. Common pixel- and feature-space perturbations can produce implausible tissue, making model responses difficult to interpret. We introduce CPathOGen, a conditional latent-
diffusion framework for generating paired H&E counterfactuals with explicit controls over cellular spatial organization, nuclear morphology, and stain appearance. Cellular maps condition spatial structure through a spatial encoder, while a morphology/appearance vector modulates denoising through blockwise feature-wise linear modulation (FiLM). On held-out H&E tiles, CPathOGen generates visually plausible tissue with improved distributional agreement after spatially guided selection, as reflected by lower Fréchet Inception Distance (FID) and Kernel Inception Distance (KID); generated
cells track requested abundance and position, and measured morphology and color vary monotonically with their controls. We use these verified interventions to probe pathology encoders with endpoint heads, task-specific classifiers, and survival models. Responses are quantified using total variation distance and prediction-flip rate. We further introduce the Biology-Nuisance Sensitivity Ratio, a metric that contrasts model sensitivity to biologically motivated morphology and spatial factors with sensitivity to non-biological, stain-related nuisance variation. CPathOGen provides a practical, fidelity-audited framework for evaluating robustness and controlled feature sensitivity in computational pathology model. \href{https://github.com/a12dongithub/PathOGen}{GitHub} and \href{https://huggingface.co/a12donhf/CPathOGen}{Hugging~Face}.
Read the full abstract ▾
Signal Processing
Mean Spatial Frequency Decoupling for Learning-Based Uplink-to-Downlink Covariance Conversion in FDD Massive MIMO
Melih Can Zerin
In frequency division duplexing (FDD) massive multiple-input multiple-output (
MIMO) systems, the uplink (UL)-to-downlink (DL) channel covariance
matrix (CCM) conversion problem is studied to relieve the heavy burden of DL training and feedback required for channel estimation. Learning- based methods perform well up to a certain array size, but for a fixed
dataset size their accuracy deteriorates with the number of
antennas, to the point where simple model-based methods outperform them. This paper identifies a key cause of this behavior and addresses it. The mean angle of arrival (AoA) induces a phase ramp along the lags of the CCM. Since the oscillation rate of this ramp grows with the number of
antennas, a
dataset of fixed size becomes increasingly sparse relative to the variation that must be captured. We propose estimating the slope of this ramp from the UL CCM separately and mapping it to the DL band in closed form, leaving the learner with a residual that is largely insensitive to the mean AoA, which substantially reduces the performance degradation with an increasing number of
antennas. The proposed scheme, termed deramping, is a combination of pre- and post-processing steps that applies to learning-based conversion methods without altering their internal structure, as demonstrated on three structurally different learners. Simulation results show that deramping reduces the covariance estimation error of all three learners under uniform,
Laplacian, and Gaussian angular power spectra,keeps the interpolation-based learners ahead of a model-based
benchmark at large array sizes, and improves downlink channel estimation.
Read the full abstract ▾
Image and Video Processing
Hetero-modal learning and corruption-resistant hetero-modal inference for joint segmentation of white matter hyperintensities and ischaemic stroke lesions in MRI
Jesse Phitidis, William N. Whiteley, Joanna M. Wardlaw et al. (+3 more)
White matter hyperintensities (WMH) and ischaemic stroke lesions (ISL) are visually confounding, co-occurring pathologies that require large, diverse
datasets for robust deep learning segmentation. However, assembling such
datasets is hindered by cohort samples that lack reference segmentations from both features and complete sets of MRI structural sequences (i.e., "modalities"). To maximise data utility, we investigate hetero-modal learning using a
dataset of 206 vascular disease patients across four MRI sequences (T1-weighted, T2-weighted, fluid-attenuated inversion recovery, and
diffusion-weighted imaging) with expert annotations of both WMH and ISL. We demonstrate that hetero-modal learning outperforms models trained using a single imaging modality in scenarios with substantial missing data, including a split where only 10% of the training data contains all four modalities while the remainder is uni-modal, and a split relying on a single shared "anchor" modality with zero overlap between the remaining modalities. Furthermore, models trained under this second split successfully perform inference on unseen combinations of modalities. Yet, while standard hetero-modal networks handle missing sequences, clinical deployment introduces the additional challenge of silent data degradation - where modalities are present but severely corrupted. To bridge this gap, we introduce the
Multimodal Attention Router (MMAR) block. Our experiments demonstrate that, when trained with a "corruption augmentation" strategy, the MMAR effectively dynamically weights the encoded features of each modality, maintaining strong performance during hetero-modal inference even in the presence of unflagged catastrophically corrupted modalities.
Read the full abstract ▾
All authors Jesse Phitidis, William N. Whiteley, Joanna M. Wardlaw, Miguel O. Bernabeu, Francesco Dalla Serra, Maria del C. Valdés Hernández
Audio and Speech Processing
Multi-agent Auditory Scene Analysis: Improved Localization Speed and Robustness by Multi-beamformed Speech Quality Feedback
Caleb Rascon
A real-time auditory scene analyzer (ASA) aims to carry out the tasks of locating, separating and classifying the sound sources present in a given acoustic environment. Recently, an effort has been made into modelling an ASA as a multi-
agent system, with each one of its
agents performing one of the aforementioned tasks and communicating their results to the rest of their peer
agents. These communication routes are used as feedback loops to fix local errors at a global level, providing robustness while reducing local complexity. An example of the benefits of this approach is the optimization of
speech quality by correcting in real-time the estimated location of the
speech source of interest. However, their optimization speed has been shown to be considerably slow. One possible reason is that it solely relies on a series of single quality estimations (provided by a reference-free quality
estimator model) that vary considerably from one window to the next, which results in a difficult search space to optimize. In this work, a new optimization mechanism is proposed that instead relies on a series of sets of quality estimations over a range of locations, providing a clearer view of the search space, simplifying its optimization. The proposed ASA now has a considerably smaller optimization time, is more accurate, and is more stable when being evaluated in real-life acoustic scenarios to correct higher levels of localization errors, all while being less complex than previous efforts. The only trade-off is that there is an increase in the response time of the quality estimation
agent, but the complete ASA is still able to run in real-time. The performance shown in this work again shows the benefits of modelling an ASA as a multi-
agent system.
Read the full abstract ▾