Audio and Speech Processing
Training-Free Speech-Centric Omni Understanding with Frozen VLMs
Ankan Deria, Hanoona Rasheed, Xilin He et al. (+2 more)
Audio-visual understanding remains challenging because models must jointly interpret spoken content, visual events, and their temporal relationships. Existing omni models typically introduce dedicated audio encoders and rely on expensive audio-
video-text training, tightly coupling omni capability to specific VLM backbones and potentially weakening their existing visual and
reasoning abilities. This raises three questions: whether native omni training is necessary for every new VLM, whether
speech-centric omni capability can be added while preserving the original backbone, and where richer acoustic representations remain essential. We introduce Training-Free Omni (TFO), a plug-and-play framework that converts a frozen VLM into a
speech-centric omni model without architectural modification, or
multimodal re-
alignment. TFO uses Whisper to extract confidence-filtered, timestamped transcripts and routes them through the VLM's existing
language interface, while leaving its visual pathway unchanged. Across matched comparisons with native omni models on 56
benchmarks and 21
languages, TFO is competitive on audio-visual understanding, improves average audio-only performance across all five model settings, and achieves substantial multilingual
speech gains. Freezing the VLM also generally preserves stronger image/
video understanding, visual grounding, coding, mathematical
reasoning, and medical question answering than the corresponding native omni checkpoints. These results show that strong
speech-centric omni understanding can often be obtained through modular audio-to-
language routing rather than costly backbone-specific training.
Read the full abstract ▾
All authors Ankan Deria, Hanoona Rasheed, Xilin He, Fahad Shahbaz Khan, Salman Khan
Image and Video Processing
LoRCA: LoRA Cycle Adaptation for Histology to HiP-CT Translation with DINOv3
Yang Zhou, Edoardo Occhipinti, Banboye Kidzeru Elvis et al. (+11 more)
Hierarchical Phase-Contrast
Tomography (HiP-CT) is a synchrotron based X-ray imaging technique that enables non-destructive, volumetric imaging of intact organs with multi-resolutions bridging 20 $\mu m$/voxel for whole organs to near-cellular resolution ($\sim$0.8 $\mu m$/voxel) in local regions. This offers the opportunity to bring volumetric whole-organ context to histology. However, nonlinear registration between H&E histology and HiP-CT volumes is challenging due to the differences in feature representations of different colour spaces. Synthesis-before-registration methods have shown strong results in histology-to-MRI and histology-to-CT
alignment. However, existing approaches either rely on manual anatomical contours or are trained from scratch without semantic constraints, limiting their generalisability to soft tissue organs and novel modalities. We propose LoRCA (LoRA Cycle Adaptation), a cycle consistent style translation framework built on a shared frozen DINOv3 with modality-specific LoRA adapters, learning modality-specific representations that are decoded and adversarially trained. LoRCA enables structure-preserving translation without requiring paired training data. The frozen backbone is intended to be a structural anchor that prevents content drift by preserving pretrained semantic-extraction capability. We evaluate translation quality using Fréchet Inception Distance (FID) and structural fidelity via mutual information and Canny edge preservation. LoRCA outperforms CycleGAN in both translation quality and structural consistency. As a preliminary indicator of downstream registration utility, we find that style-translated images yield increased feature correspondences under MatchAnything on manually aligned HiP-CT and histology test pairs, suggesting that LoRCA-style translation is a promising step towards 2D histological sections to 3D HiP-CT volumes registration.
Read the full abstract ▾
All authors Yang Zhou, Edoardo Occhipinti, Banboye Kidzeru Elvis, Jishizhan Chen, Stathis Megas, Joseph Brunet, Joanna Purzycka, Theresa Urban, Hector Dejea, Sarah Amalia Teichmann, Menna R Clatworthy, Paul Tafforeau, Peter D Lee, Claire L Walsh
Image and Video Processing
Implicit representations are dead. Long live explicit primitives!
Nil Stolt-Ansó, Maik Dannecker, Wenqi Huang et al. (+2 more)
Continuous parameterization of medical data has emerged as a powerful paradigm for resolution-independent image representation. While Implicit Neural Representations offer high fidelity and compact storage, their reliance on global Multi-Layer Perceptrons incurs sizeable computational costs, large
memory requirements, and extensive optimization times. As medical imaging trends towards ever-more detailed, high-resolution volumes, these costs impose significant bottlenecks in the applicability of implicit approaches. Recently, explicit Gaussian-based primitives have revolutionized the representation learning paradigm by trading deep network evaluations for localized, rasterization-friendly primitives. In this paper, we present a comprehensive, cross-dimensional evaluation of Gaussian representations against implicit approaches for medical imaging applications. First, we outline a theoretical overview on the mathematical properties offered by explicit primitives beyond what is capable under the implicit neural paradigm. Subsequently, we
benchmark the computational performance on two demanding image
datasets: 2D microscopy histology and 3D lung Computed
Tomography (CT). Our experiments demonstrate that Gaussian representations consistently match or surpass reconstruction metrics compared to implicit methods across all compression factors, while displaying significantly lower optimization times, and
memory requirements. Together with the compelling mathematical properties offered by explicit primitives, these findings motivate the wider adoption of Gaussian representations and position them as an attractive direction for future research in medical imaging.
Read the full abstract ▾
All authors Nil Stolt-Ansó, Maik Dannecker, Wenqi Huang, Andras Jakab, Daniel Rueckert
Image and Video Processing
Pre- to Post-Contrast Synthesis of Breast DCE-MRI using Latent Bridge Matching
Sina Amirrajab, Zohaib Salahuddin, Henry C Woodruff et al. (+1 more)
Dynamic contrast-enhanced
magnetic resonance imaging (DCE-MRI) is central to breast cancer imaging, but gadolinium administration increases scan burden and motivates contrast-reduced alternatives, including synthetic contrast generation. We propose a latent bridge matching (LBM) framework for synthesizing peak-enhanced breast DCE-MRI from pre-contrast images in the MAMA-SYNTH challenge setting. Instead of starting from Gaussian noise as in conventional latent
diffusion models (LDMs), the proposed model learns a conditional bridge between paired pre-contrast and peak-enhanced VAE latents. A latent UNet predicts the remaining correction from intermediate bridge states to the peak-enhanced latent, enabling iterative refinement while keeping the trajectory anchored to patient-specific anatomy. We evaluated two LBM conditioning variants on 91 DUKE validation cases. For the tumor-conditioned variant, tumor masks were used as conditioning inputs. Tumor-conditioning improved performance compared with pre-contrast conditioning, reducing MSE from 1.023 to 0.940 and FRD from 7.523 to 4.716, while increasing tumor SSIM from 0.355 to 0.429. The tumor-conditioned LBM also outperformed the evaluated LDM baseline on this validation cohort. These results suggest that latent bridge matching is a promising pre-contrast-anchored formulation for virtual contrast enhancement, while further work is needed to validate generalization and remove dependence on ground-truth tumor masks at inference.
Read the full abstract ▾
All authors Sina Amirrajab, Zohaib Salahuddin, Henry C Woodruff, Philippe Lambin
Image and Video Processing
Robustness of transferability estimation metrics for medical imaging
Niclas Claßen, Théo Sourget, Dovile Juodelyte et al. (+2 more)
In transfer learning, the choice of source model largely influences the performance on a target
dataset. Still, selecting a fitting source remains a challenging task, especially in medical imaging where one has to decide between models pre-trained on off-the-shelf options, such as ImageNet, and domain specific
datasets. Transferability estimation (TE) metrics address this problem by aiming to predict the best performing source model in a computationally cost effective way. However, previous work has reported conflicting TE metric performances due to differences in experimental setups. Moreover, most TE metrics are designed for and evaluated on natural images, while being optimized for accuracy, whereas in medical imaging metrics that are more robust to class imbalance are typically used. We study the impact of varying the target
dataset as an isolated factor, by constructing miniature populations of different sample sizes and random seeds. In addition, we investigate the influence of the evaluation metric used to obtain the reference ranking. We find that small modifications to the target
dataset change the rankings. Furthermore, we show that the choice of evaluation metric affects the reference rankings and therefore the evaluation of TE metrics. Overall, we observe a low agreement between rankings from TE metrics and reference. The code, model checkpoints and data splits used in this work are available through https://github.com/niclasclassen/robustness-of-transferability-estimation-metrics-for-medical-imaging.
Read the full abstract ▾
All authors Niclas Claßen, Théo Sourget, Dovile Juodelyte, Rob van der Goot, Veronika Cheplygina
Image and Video Processing
Energy and Performance Benchmarking of Deep Learning Models for Breast Cancer Detection
Samar Garrab and Ghada Achour
Recent advances in machine learning have greatly improved breast cancer detection, enabling more accurate and timely diagnosis. Deep learning (DL) models show strong potential for medical image analysis; however, as their architectural complexity increases, their environmental impacts are becoming a growing concern. In this paper, we present a comparative analysis of seven DL models for breast cancer detection on two medical
datasets: Breast Ultrasound and BreakHis 400X. The evaluated architectures range from Convolutional Neural Networks (CNNs) and
transformers to hybrid models. In addition to performance metrics, we assess CO2 emissions during both training and inference. Our results show that EfficientNet and ResNet consistently deliver strong performance, although with higher CO2 emissions. The selected
transformers, such as DeiT-Tiny, perform competitively on both
datasets, whereas DenseNet121 achieves lower accuracy. On the Breast Ultrasound
Dataset, DeiT provides the most favourable balance between accuracy and energy consumption, whereas on the BreakHis
dataset, the ViT and Swin models achieve the best results. Overall, our findings indicate that no single architecture category from the evaluated ones consistently dominates across the two selected
datasets. Our results highlight the importance of jointly considering performance, emissions, and
dataset characteristics when selecting models for medical applications.
Read the full abstract ▾
Image and Video Processing
Structural Guidance for Unified Joint Demosaicing and Denoising
Qixin Zheng, Ping Chen, Qiangqiang Shen et al. (+1 more)
Joint demosaicing and denoising is a fundamental step in camera image signal processing, yet remains challenging because different Bayer-like color filter arrays (CFAs) and sensor noise jointly corrupt both color sampling and image content. Existing unified restoration networks explicitly model CFA geometry but are still driven primarily by pixel-level supervision, making them prone to structural degradation around edges, repetitive textures, and moiré patterns where local evidence is unreliable. We attribute this limitation partly to the absence of explicit structural guidance beyond pixel-level reconstruction supervision. Motivated by this observation, we propose a structural-guided unified restoration framework that injects pretrained structural knowledge into CFA-aware image restoration. Our model receives a unified five-channel observation consisting of the raw mosaic, CFA masks, and a noise-level map. A SwinIR restoration branch reconstructs pixel details under CFA-conditioned modulation, while a parallel structural
reasoning branch extracts complementary structural cues from a sparse pseudo-RGB observation. To bridge the substantial domain gap between sparse noisy sensor data and the natural-image pretraining domain of the structural encoder, we introduce a lightweight trainable adapter before residually fusing structural and restoration features. A shared decoder jointly predicts the restored RGB image and an auxiliary clean mosaic, providing supervision in both image and sensor domains. Extensive experiments across multiple CFA patterns and noise levels demonstrate consistent improvements over state-of-the-art unified and CFA-specific methods, indicating that adapted structural priors can enhance robust camera image restoration. The source codes and
dataset are provided in the supplementary material.
Read the full abstract ▾
All authors Qixin Zheng, Ping Chen, Qiangqiang Shen, Haijin Zeng
Image and Video Processing
SpecF2M: A Spectral-Aware Multi-task Network Estimating Axial Length and Refractive Error from Pediatric Fundus Photographs
Mengxian He, Xinyue Liu, Yunyun Sun et al. (+5 more)
Spherical Equivalent Refraction (SER) and Axial Length (AL) are core indicators for pediatric myopia screening, yet their measurements require dedicated biometry and cycloplegic refraction. Fundus photography offers an accessible imaging modality, as myopia-related posterior-pole changes are visible in 45$^\circ$ fundus images. However, these cues are often low-contrast, spatially diffuse, and multi-scale. Moreover, AL, Sphere (SPH), and Cylinder (CYL) share partially overlapping but non-identical anatomical correlates. We propose SpecF2M, a spectral-aware multi-task network for estimating AL and SER components from pediatric fundus photographs. SpecF2M integrates a deterministic anatomy-guided enhancement module, a hybrid spatial–spectral backbone combining MixCNN and Hybrid Spectral Learning (HSL) blocks, and an expert-routing head for component-level estimation of AL, SPH, and CYL. On a pediatric cohort of 4,359 eligible child visits and 6,966 fundus images, SpecF2M outperforms controlled CNN/ViT baselines for AL and SPH estimation, achieving MAEs of 0.5347 mm and 0.7062 D, respectively. Component-level analysis further reveals asymmetric task coupling, where CYL exhibits weaker association with fundus-derived myopic patterns than AL/SPH. These results support fundus-based, screening-oriented estimation of pediatric myopia indicators, while external validation remains necessary before deployment.
Read the full abstract ▾
All authors Mengxian He, Xinyue Liu, Yunyun Sun, Wei Hao, Minqing Zhang, Lichun Wang, Shunyi Zhang, Wu Yuan
Image and Video Processing
APCReg: Anatomical-Prior-Guided Coarse-to-Fine CBCT–IOS Registration via Multi-View Projection and Reliability-Controlled Residual Correction
Xincan Zheng, Yaqi Wang, Zhi Li et al. (+4 more)
Registration between cone-beam computed
tomography (CBCT) and intraoral scans (IOS) is essential for patient-specific surgical planning. However, disparate imaging modalities, limited overlap, and large pose offsets make automated registration unreliable. Consequently, clinical registration remains dependent on conventional geometry pipelines and manual clinician adjustment. To address these challenges, we propose APCReg, an anatomical-prior-guided coarse-to-fine framework for global registration and reliability-controlled residual correction. Specifically, multi-view anatomical coarse registration (MACR) performs ordered orthogonal projection
alignment (buccal, proximal, and occlusal) to decompose the six-degree-of-freedom search before three-dimensional refinement. Overlap-aware residual registration (OARR) combines shared KPConv features, a folded arch-length cue, overlap-gated cross-
attention, and Sinkhorn matching. Finally, dental-arch-structured hypothesis selection evaluates diverse poses on held-out reliable correspondences, while a ground-truth-free coarse-retention guard conditionally retains a geometrically reliable coarse pose. On 60 held-out jaw pairs, APCReg achieves a submillimeter mean Chamfer distance of 0.87 mm and a Hausdorff distance of 2.92 mm under this evaluation protocol, and ranks first across the six reported metrics among the evaluated open-source baselines.
Read the full abstract ▾
All authors Xincan Zheng, Yaqi Wang, Zhi Li, Jiahao Bao, Lan Feng, Yiru Xia, Shuai Wang
Image and Video Processing
Knowledge-Guided 3D CT Generation: A Conditioning-Centric Taxonomy
Francesca Pia Panaccione, Eugenio Lomurno and Matteo Matteucci
Controllable generation guided by external knowledge is a key requirement in modern generative deep learning applications, enabling the synthesis of samples with explicit constraints on semantic content, structural properties, and variability. In 3D Computed
Tomography (CT), such control is essential for clinical applications, including data augmentation,
privacy-preserving data sharing, and the simulation of specific anatomical or pathological scenarios. While research on conditional 3D CT generation has expanded rapidly, the diversity of existing approaches makes systematic comparison difficult and obscures fundamental design choices. In this survey, we propose a conditioning-centric taxonomy that organizes the literature along three orthogonal dimensions: the type of external knowledge (K), the knowledge integration paradigm (I), and the generative architecture (A). This factorization defines an explicit design space (K x I x A) that provides a unified perspective on prior work. Using this framework, we systematize existing methods, identify dominant trends and recurring design patterns, and highlight underexplored regions of the design space that point toward promising directions for future research.
Read the full abstract ▾
Image and Video Processing
Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRI
Fidel Omar Tito Cruz, Neda Ghafouri, Zengyan Wang et al. (+3 more)
Pathologic complete response (pCR) is an important endpoint in neoadjuvant chemotherapy (NAC) for breast cancer, and predicting pCR from imaging during treatment could support treatment response assessment. Many existing imaging-based approaches rely on a single static timepoint, which fails to capture changes that occur during treatment. In this work, we present a longitudinal framework that combines a frozen 3D foundation encoder (Pillar-0) with our Temporal Dynamics Network (TDN) to predict treatment response from serial Dynamic Contrast-Enhanced (DCE) MRI acquired across four clinical timepoints from pre-treatment to pre-surgery. The TDN combines time-aware volumetric
embeddings with clinical and treatment data to predict pCR. Evaluated on 982 patients from the combined I-SPY2 and ACRIN-6698 cohort, the proposed model achieves strong performance across all reported metrics when longitudinal 3D imaging is fused with clinical data (test AUROC: 73.6%, balanced accuracy: 69.1%). While clinical variables provide the strongest individual predictive signal, longitudinal 3D imaging contributes complementary information when fused with clinical data, improving pCR prediction. Our source code is available at: https://github.com/omarftt/longitudinal_temporal_pillar.
Read the full abstract ▾
All authors Fidel Omar Tito Cruz, Neda Ghafouri, Zengyan Wang, Pegah Khosravi, Yu Tian, Chen Chen
Systems and Control · Cross-listed from math.OC
Universal Birkhoff Method for Computing Extremals in the Elliptic Restricted Three-Body Problem
Michael J. Dixon and Isaac M. Ross
The computation of finite-thrust extremal arcs in the elliptic restricted three-body trajectory optimization problem is considered. Libration-point orbits are approximated to near-machine precision using a fast
Fourier transform of the sampled values of the state vector at Chebyshev-Gauss-Lobatto points. Checkable optimality conditions are derived by applying Pontryagin's principle to minimum-time and time-constrained minimum-propellant problems. These necessary conditions include criteria for optimal departure and arrival points. For propellant consumption, a recently developed computational model is employed. This model is agnostic to the specific impulse of the propellant and varies as the inverse quadratic of a cosine term. Candidate optimal solutions are generated by combining the universal Birkhoff theory for trajectory optimization with the fast, guess-free spectral
algorithm. The extremality of the Birkhoff-computed solution is validated against the
Hamiltonian minimization condition and the transversality conditions. It is shown that the Birkhoff-theoretic spectral
algorithm can generate verifiable extremals without any assistance or initialization from dynamical systems theory.
Read the full abstract ▾
Image and Video Processing
Integrating spectral and morphological plant features with decision-tree models for early-season cotton biomass and nitrogen status estimation from multi-year UAV data
Vaishali Swaminathan, Nithya Rajan, J Alex Thomasson et al. (+4 more)
Precision nitrogen (N) management (PNM) for cotton requires in-season monitoring of crop growth parameters and N status indicators to decide fertilizer timing, placement, and application rates for optimal canopy development and yield. This study developed remote sensing and machine learning-based methods to estimate cotton dry biomass weight (DBW), plant N uptake (PNU), plant N concentration (PNC), critical N dilution (Nc), and nitrogen nutrition index (NNI) to support PNM. To achieve this, a three-year field-based N-management study was conducted and unmanned aerial vehicle (
UAV)-based multispectral images were acquired between early vegetative growth and flowering stages, critical for fertilizer applications. Spatiotemporally consistent spectral and morphological plant features, including plant height (PH) and fractional canopy cover (FCC), provided reliable model training inputs. DBW, PNU, and PNC estimates from simple regression using vegetation indices (VIs), multiple linear regression (MLR) combining VIs, PH, and FCC, and decision-tree models, random forest regression (RFR) and extreme gradient boosting (XGB), combining spectral reflectance, PH, and FCC were evaluated using trial-held-out (THO) and leave-one-year-out (LOYO) validation methods. The best validation accuracies were from RFRTHO (R2 = 0.88 and MAPE = 23.14% for DBW; R2 = 0.84 and MAPE = 20.61% for PNU; R2 = 0.85 and MAPE = 7.82% for PNC) and XGBTHO (R2 = 0.87 and MAPE = 21.91% for DBW; R2 = 0.81 and MAPE = 21.40% for PNU; R2 = 0.86 and MAPE = 7.66% for PNC). Nc was calculated from model estimated DBW and PNC for high-yielding, medium-to-tall cotton varieties grown in the Texas Coastal Plains and validated using ground-truth biomass measurements. NNI derived from XGBTHO outputs performed marginally better than NNI from RFRTHO in identifying N-deficient plots and multi-level N-stress categorization.
Read the full abstract ▾
All authors Vaishali Swaminathan, Nithya Rajan, J Alex Thomasson, Amrit Shrestha, Karem Meza Capcha, Robert Hardin, Pramod Pokhrel
Image and Video Processing
Bit Allocation Transfer for Perceptual Quality Enhancement of Traditional Video Codecs
Runyu Yang and Ivan V. Bajić
Traditional block-based
video codecs, such as H.264/AVC, H.265/HEVC and H.266/VVC, rely on hand-crafted Rate-Distortion Optimization (RDO) processes that primarily minimize Mean Squared Error (MSE), which correlates poorly with human perceptual quality. While neural
video compression methods can easily optimize perceptually aligned metrics like MS-SSIM, their high computational complexity limits practical deployment. This paper proposes a novel bit allocation transfer framework that bridges these two paradigms to enhance the perceptual quality of conventional
video codecs. Specifically, we train a quantization step generation model using a perceptual loss within a neural
video compression framework (DCVC-FM). The model takes the original frame and a motion-compensated prediction as input and outputs a quantization step map. We then derive a block-wise bit ratio from this map and convert it into a Quantization Parameter (QP) map for a traditional
video codec. Experimental results on the HEVC B$\sim$D
dataset demonstrate that our method achieves 20.20%, 8.25%, and 8.37% bitrate savings in terms of MS-SSIM compared with the standard reference software JM-19.0, HM-16.20, and VTM-23.0, respectively, with additional gains when utilizing predicted frames. Our approach effectively transfers the implicit perceptual importance learned by neural
video compression models to guide block-level bit allocation in traditional
video codecs without modifying their core decoding syntax.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.NI
Coordinated Spectrum Coexistence Across Heterogeneous Commercial and Federal Services
Minh Dat Nguyen, Paolo Testolina, Pedram Johari et al. (+2 more)
Future wireless networks are expected to support the coexistence of cellular communications,
radio frequency (RF) sensing, radionavigation, and radiolocation radar-among others-over congested federal sub-6 GHz spectrum under heterogeneous Quality of Service (QoS) requirements, driven by escalating cellular traffic demand, the proliferation of 6G sensing and positioning services, and regulatory pressure to repurpose federal bands without displacing incumbents. We develop a unified framework in which all four services dynamically share a common Physical Resource Block (PRB) pool under centralized coordination, formulating weighted cellular sum-rate maximization subject to duty-cycle, orthogonality, sensing signal-to-noise ratio (SNR), and Cramer-Rao-based positioning constraints. The resulting Mixed-Integer Nonlinear Program (MINLP) is solved by alternating optimization across PRB assignment, scheduling, and successive convex approximation for power allocation, complemented by a low-complexity QoS-aware greedy heuristic. Site-specific ray-tracing simulations on the BostonTwin urban digital twin show substantial gains in spectrum efficiency and cellular throughput while strictly meeting sensing and positioning QoS, establishing coordinated multi-service sharing as a viable architecture for cellular-federal-radar coexistence in next-generation military and civilian networks.
Read the full abstract ▾
All authors Minh Dat Nguyen, Paolo Testolina, Pedram Johari, Michele Polese, Tommaso Melodia
Systems and Control
Generalized Nyquist Criterion Limitations and Misconceptions for Frequency Domain Stability Analysis of Inverter-based Resources Integrated Power Grids
Hassan Yazdani and Saeed Lotfifard
This paper presents theoretical and numerical studies demonstrating the limitations of the Generalized Nyquist Criterion (GNC) in assessing the small-signal stability of inverter-based resources (IBRs) modeled as multi-input, multi-output (
MIMO) systems. GNC leads to unnecessary computational burden and increased analytical complexity for nominal stability analysis. The paper demonstrates that the GNC framework is not reliable for robust-stability analysis and may lead to misleading results when applied to
MIMO IBR systems. To demonstrate these limitations and enable a proper
MIMO robust stability assessment of IBRs using mu-analysis, the paper develops a model-uncertainty-augmented representation of an IBR-integrated power grid. The proposed model explicitly captures the structural characteristics and spatial distribution of various uncertainty sources, such as parametric variations, unmodeled dynamics, and measurement errors. Since these uncertainties naturally occur in the physical three-phase (abc) systems, they are systematically transformed into the dq0 frame within the developed model.
Read the full abstract ▾
Audio and Speech Processing
Mitigating Over-Suppression in Speech Enhancement via Inference-Time Rethink-and-Refine Correction Module
Mike Qu, Yu-Wen Chen and Julia Hirschberg
We present a rethink-and-refine correction module that addresses over-suppression, a common failure mode of
speech enhancement (SE) models, where
speech cues are suppressed alongside noise. Our method operates entirely in the inference stage without additional training, allowing seamless integration with diverse SE models. Given noisy and enhanced signals, we obtain word- or phoneme-level
alignments using an automatic
speech recognition model and identify intervals where enhancement is unreliable. These intervals are then selectively remixed through convex interpolation, with per-segment weights optimized to maximize a composite objective balancing perceptual quality and
speech preservation. Experiments on the URGENT 2024 and 2025, VCTK-DEMAND, and MSP-PODCAST
datasets show consistent improvements in perceptual quality, intelligibility, and downstream performance compared to conventional SE alone, demonstrating the benefit of rethink-and-refine framework for robust
speech processing.
Read the full abstract ▾
Systems and Control
Information-Aware Model Predictive Control for Satellite Inspection
Sarah E. Clees, Sean Phillips and Christopher Petersen
Autonomous spacecraft inspection requires trajectories that satisfy safety and control constraints while enabling the collection of informative measurements about a target spacecraft. Traditional guidance and control methods typically decouple estimation from control, resulting in trajectories that do not explicitly optimize sensing geometry. This work presents a model predictive control (MPC) framework that incorporates estimation covariance in the control objective using a formulation inspired by dual control and covariance steering. The estimation covariance evolves according to a linear Kalman filter, and the measurement model depends on the relative geometry between the
agent spacecraft and the target. By
embedding the covariance dynamics within the MPC problem, the resulting trajectories account for measurement quality, actively reduce uncertainty, and improve observability in the estimated features of the target. The problem is formulated using relative motion dynamics via the Hill-Clohessy-Wiltshire equations with constraints on control input, relative distance, and terminal maximum covariance. Numerical simulations demonstrate that the proposed framework generates feasible inspection trajectories that actively reduce estimation covariance of points of interest on a target while satisfying input and safety constraints of the
agent. A mesh analysis of initial conditions further illustrates how feasibility and the value function of the trajectory depend on the initial conditions and constraint activity.
Read the full abstract ▾
Signal Processing
CFD-Guided Detection of Concept Drift in Multimodal Physiologic Signals
Farouk Ganiyu Adewumi, Timothy Oladunni, Rochak Ghimire et al. (+3 more)
Cardiovascular AI models can classify clean elec- trocardiogram (ECG) signals, but real wearable signals change because of motion, breathing, posture, sensor contact, and true clinical deterioration. This paper asks when a model should keep its prediction, change it, or flag uncertainty. We propose a physiologic stability framework, called PECS, that compares changes inside the model with measurable changes in the signal. ECG is treated as the main cardiac signal, photoplethysmography (PPG) adds pulse and vascular information, and respiration is used only when ECG and PPG disagree. We test the framework on PTB-XL at pilot and full scales and on synchronized BIDMC and MIMIC waveform cohorts. The PTB-XL pilot and full- scale analyses selected different domain pairs, and the strongest cross-modal pair also changed across BIDMC and MIMIC, showing that adding every available signal is not always the best choice. PECS outperformed the evaluated drift-detection baseline implementations, reaching drift classification accuracy (DCA) of 0.8786 on expanded BIDMC and 0.9560 on MIMIC. The MIMIC results also showed that respiration can help during disagreement cases, but it should be used selectively rather than as an automatic override. Overall, the results support PECS as a candidate monitoring framework for wearable cardiovascular AI while highlighting the need for scale-aware domain selection and interpretable trust routing
Read the full abstract ▾
All authors Farouk Ganiyu Adewumi, Timothy Oladunni, Rochak Ghimire, Kosisochukwu Ogbuanya, Sanaa Reeves, Sandy Akoy
Systems and Control · Cross-listed from cs.RO
Enhancing Autonomous Vehicle Navigation with a Clothoid-Based Lateral Controller
Aashish Shaju, Steve Southward and Mehdi Ahmadian
This study introduces an advanced lateral control strategy for autonomous vehicles using a clothoid-based approach integrated with an adaptive lookahead mechanism. The primary focus is on enhancing lateral stability and path-tracking accuracy through the application of Euler spirals for smooth
curvature transitions, thereby reducing passenger discomfort and the risk of vehicle rollover. An innovative aspect of our work is the adaptive adjustment of lookahead distance based on real-time vehicle dynamics and road geometry, which ensures optimal path following under varying conditions. A quasi-feedback control
algorithm constructs optimal clothoids at each time step, generating the appropriate steering input. A lead filter compensates for the vehicle's lateral dynamics lag, improving control responsiveness and stability. The effectiveness of the proposed controller is validated through a comprehensive co-simulation using TruckSim and Simulink, demonstrating significant improvements in lateral control performance across diverse driving scenarios. Future directions include scaling the controller for higher-speed applications and further optimization to minimize off-track errors, particularly for articulated vehicles.
Read the full abstract ▾
Systems and Control
Lost Opportunity Costs Under Ramp Stress: A Comparison of Ramp-Product Dispatch and Look-Ahead Economic Dispatch
Aidan Looney, Qian Zhang and Le Xie
This paper studies whether ramp-product settlement compensates generators that absorb intertemporal ramp scarcity as effectively as look-ahead economic dispatch. We evaluate this question by comparing the lost opportunity cost (LOC) induced by each approach. Using rolling-horizon simulations on a 10-generator system and a modified RTS-GMLC system, we compare ramp-product settlement (RP-LMP), look-ahead settlement (LA-LMP), and temporal locational marginal pricing (TLMP). In the featured deterministic ramp stressed cases, aggregate LOC is higher under RP-LMP than under LA-LMP. The main contribution is a generator-level critical assessment for ramp-product settlement: we identify which units are left with uncompensated intertemporal opportunity cost under RP-LMP and how that burden changes under LA-LMP. RP-LMP LOC is concentrated on units with repeated ramp-binding exposure, while LA-LMP mainly relieves those same units and leaves smaller residual LOC on other units. Multi-day tests preserve this ordering under perfect foresight in the larger test system, but show that it need not hold under forecast error.
Read the full abstract ▾
Systems and Control
Design and Analysis of a Programmable Ramp Generator for Frequency Synthesizers
Pranjal Mahajan, Hrishikesh Milind Gawas, Ishan Acharyya et al. (+5 more)
This paper presents a fully digital Programmable Ramp Generator (PRG) for reconfigurable frequency-ramp synthesizers used in Frequency-Modulated Continuous-Wave radar, spread-spectrum clock generation, and test and measurement applications. A novel modular architecture is proposed, integrating a phase-accumulator-based frequency sweep generator and Delta-Sigma Modulator (DSM) that drives a Multi-Modulus Divider (MMD), giving independent digital control over ramp slope, bandwidth, and duration. While prior works demonstrate chirp synthesis, the
circuit-level constraints governing reliable PRG operation have not been jointly analyzed. This work identifies and analyzes three such constraints: (a) MMD octave-boundary behavior under dynamic updates, (b) divider-synchronous timing, and (c) DSM limits on step duration and achievable slope. Together, these constraints define the PRG design space, enabling a systematic
algorithm that maps application-level specifications directly to hardware parameters. Following this methodology, a 16-bit PRG implemented in TSMC 65 nm CMOS is verified across 18 ramp profiles and hardware prototyping on a Xilinx Spartan-7 FPGA confirms real-time operation.
Read the full abstract ▾
All authors Pranjal Mahajan, Hrishikesh Milind Gawas, Ishan Acharyya, Umesh Khetan, Navya Valmeekam, Mohd Wajid, Hemant Kumar Meena, Abhishek Srivastava
Image and Video Processing · Cross-listed from q-bio.QM
JUMP-lite: Compact, reproducible benchmarking of cell representations
Alán F. Muñoz, Johan Fredin Haslum, Runxi Shen et al. (+2 more)
Image-based profiling captures rich phenotypic signatures for drug discovery and functional genomics. Large public
datasets like JUMP
Cell Painting now provide millions of images for systematic study. However, JUMP alone occupies 115 TB, and fragmented evaluation practices make systematic comparisons of representation methods impractical for many researchers. Here we present Nahual, an open-source framework for reproducible model deployment, and JUMP-lite, a 92.0 GB subset of JUMP that is approximately 1,250-fold smaller, selected to cover genetic modalities and compound annotations and reduced via lossy JPEG XL compression. Using these resources, we
benchmark five representation methods, including classical features (CellProfiler) and deep learning models (MorphEM, OpenPhenom, SubCell, DINOv2). Moderate compression broadly retains signal relative to uncompressed images. Standardized phenotypic activity and consistency metrics reveal meaningful performance differences across methods. Together, JUMP-lite and Nahual provide a foundation for accessible, reproducible benchmarking of image-based
cell representations.
Read the full abstract ▾
All authors Alán F. Muñoz, Johan Fredin Haslum, Runxi Shen, Anne E. Carpenter, Shantanu Singh
Audio and Speech Processing
SemBridge: Semantic Token Anchoring for Continuous-Latent Autoregressive Speech Generation
Hanke Xie, Haopeng Lin, Jiale Qian et al. (+13 more)
Continuous-latent autoregressive
speech generation has emerged as a promising alternative to discrete-
token modeling by avoiding quantization loss and preserving richer acoustic information. However, continuous acoustic targets do not expose linguistic structure as explicit
token-level prediction targets. Consequently, the autoregressive
language model (LM) must acquire linguistic structure indirectly through acoustic prediction, which can compromise the content fidelity of generated
speech. We propose SemBridge, a training-only semantic-
token anchoring framework for continuous-latent autoregressive
speech generation. SemBridge uses discrete semantic
tokens to directly supervise autoregressive LM states and employs a Semantic-Aligned Acoustic VAE to organize the continuous target space under the same semantic reference. The semantic supervision is used only during training, while inference remains entirely continuous. We evaluate SemBridge on zero-shot text-to-
speech (TTS) and score-conditioned singing voice synthesis (SVS). Across multiple
benchmarks, SemBridge improves content accuracy, as measured by word and character error rates (WER/CER), while maintaining competitive speaker similarity and perceptual quality. Experimental results demonstrate that explicit semantic-
token supervision for autoregressive state learning is an effective and general direction for continuous
speech generation. The model code and checkpoints will be available at https://github.com/ASLP-lab/SemBridge
Read the full abstract ▾
All authors Hanke Xie, Haopeng Lin, Jiale Qian, Dake Guo, Yuepeng Jiang, Zhichao Wang, Wenxiao Cao, Jingbin Hu, Guobin Ma, Wenhao Li, Huakang Chen, Chengyou Wang, Ming Tao, Zhonghua Fu, Lei Xie, Xinsheng Wang
Signal Processing
RIS-Aided mmWave Localization Under Cross-Link Interference via Beam-Domain ML Fingerprinting
Md Tarek Hassan, Dmitry Zelenchuk and Muhammad Ali Babar Abbasi
Accurate user equipment (UE) localization is critical for beam management in reconfigurable intelligent surface (RIS)-assisted millimeter-wave (mmWave) based sixth-generation (6G) networks, especially if the direct base-station-UE links are unavailable. This paper proposes a beam-domain fingerprint framework that maps the received signal-to-noise ratio (SNR) across a small set of predefined RIS reflection states to the UE azimuth angle and range, without requiring channel state information (CSI). Crucially, we extend the framework to a realistic interference-impaired scenario in which a nearby cross-link interferer (CLI) corrupts the clean SNR fingerprint, yielding a signal-to-interference-plus-noise ratio (SINR) fingerprint; an interference-to-noise ratio (INR)-constrained calibration strategy keeps the interference level physically interpretable. Four machine-learning (ML) regressors are evaluated under both conditions. Simulation results at 28 GHz with a 20x20 RIS show that k-nearest neighbors (KNN) achieves the lowest angle MAE of 0.37 degrees and range MAE of 4 cm under clean conditions, rising to 1.4 degrees and 7.6 cm under interference. A key finding is that interference degrades angle estimation substantially more than range estimation across all models, a consequence of the asymmetric encoding of location information in the beam-domain fingerprint.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.SD
Cloud-Boosted Low-Compute Multi-Channel Speech Enhancement
Xulin Fan, Juan Azcarreta, Ashutosh Pandey et al. (+5 more)
Low-latency, low-compute
speech enhancement is essential for wearable devices with real-time communication requirements, but strict computational constraints significantly limit on-device performance. Knowledge Boosting has been proposed as an effective approach to improve edge model performance by leveraging a more capable server-side model, but performance gains for
speech enhancement have been limited. We propose a collaborative framework incorporating three techniques: (1) delayed server output as additional input, (2) layerwise feature boosting that transfers intermediate server representations to guide edge inference, and (3) collaborative multichannel Wiener filtering, which fuses weighted covariance
matrices estimated from both server and edge models for improved beamforming. Experimental results demonstrate that the proposed collaborative framework significantly outperforms the edge-only baseline with minimal additional computational overhead.
Read the full abstract ▾
All authors Xulin Fan, Juan Azcarreta, Ashutosh Pandey, Jesus Alvarez, Ke Tan, Jacob Donley, Ritwik Giri, Buye Xu
Signal Processing · Cross-listed from cs.IT
Efficient Discrete Position Design for Movable Antenna Systems: Low Complexity and Robustness
Haonan Wang, Xianghao Yu, Rui Wang et al. (+2 more)
Building on advances in reconfigurable
antenna techniques, movable
antennas (MAs) can dynamically reshape
antenna arrays and introduce additional spatial degrees of freedom (DoFs), thereby further improving communication performance. Despite these benefits, existing MA design
algorithms often entail prohibitively high computational complexity from discrete positioning selection, which prevents practical implementations of MAs. In this paper, we investigate efficient solutions for the mutual information (MI) maximization problem of a multi-user multiple-input multiple-output (MU-
MIMO) uplink communication system aided by discrete MAs. To this end, we first formulate the discrete MA positioning problem with the assumption of perfect channel state information (CSI). Then, we prove that the design problem falls into the category of monotone submodular maximization subject to a 2-system constraint. Accordingly, we propose a low-complexity distance-constrained submodular position search
algorithm, which is theoretically shown to achieve at least 1/3 of the optimum. Furthermore, we extend our approach to scenarios with imperfect CSI, and show that the proposed submodular optimization-based design remains robust against channel estimation errors. Numerical results demonstrate that the proposed scheme can achieve at least 90% of the optimal solution's MI gain under both perfect and imperfect CSI assumptions. Remarkably, the
algorithm achieves orders-of-magnitude complexity reduction (e.g., 34.4x faster than the branch-and-bound approach) while maintaining significant MI gains.
Read the full abstract ▾
All authors Haonan Wang, Xianghao Yu, Rui Wang, Ang Li, Ying-Jun Angela Zhang
Systems and Control
Topology Inference for Immune System Networks by Using Cell Amount Data
Yushan Li, Rikard Forlin, Dimos V. Dimarogonas et al. (+1 more)
Recent years have witnessed the advanced development of
topology inference research, which helps elucidate the interaction relationships of components in many biological networks. This paper focuses on inferring the
topology of a group of immune
cells, based on the collected data from
cell-depletion based experiments. The problem is very challenging due to i) the lack of standard analytical models for the
cell interactions, and ii) the restrictive data availability determined by the huge experiment and time costs. To address these issues, we first leverage certain common knowledge and observations on the experiments to characterize three properties on the
cell amounts during the interaction process: state non-negativity, ratio-based convergence, and triple signs of
topology weights. Then, we construct a new model with simple structure and analytical convenience, and obtain sufficient conditions for the model to accommodate all three properties. Finally, based on the constructed model, we propose a constrained quadratic programming method to infer the
topology from limited number of data pairs. Validation on experiment data demonstrate the effectiveness of the proposed method.
Read the full abstract ▾
All authors Yushan Li, Rikard Forlin, Dimos V. Dimarogonas, Petter Brodin
Signal Processing · Cross-listed from cs.LG
FedDOSE: Federated Learning Framework Decomposing Site Effects for Modeling Brain Dynamic Functional Connectivity
Deepank Girish, Yi Hao Chan, Yubin Zheng et al. (+2 more)
Functional
Magnetic Resonance Imaging ( fMRI ) data are often pooled into collaborative multi-site consortia, as deep learning models for analyses require large
datasets to generalize well. While
Federated Learning (FL) offers a
privacy-preserving paradigm for collaborative training, standard approaches continue to struggle with statistical heterogeneity. In particular, site differences pose a key challenge in multi-site data settings. Additionally, existing FL approaches for fMRI rely on static Functional Connectivity ( FC), omitting dynamic information in
brain networks. To address this, we propose FedDOSE, a novel framework that explicitly decomposes site differences for analysis of dynamic FC (dFC). FedDOSE introduces a Modularity-Guided Tucker Decomposition block to encode high-dimensional dFC tensors and capture modular-level spatio-temporal patterns efficiently. Class-specific prototypes are generated across all sites and subsequently aligned at the global level by using a combination of Optimal Transport (OT) barycenter formulation and Procrustes analysis. Extensive experiments for diagnosing Autism Spectrum Disorder (ASD) and
Attention-Deficit Hyperactivity Disorder (ADHD) on three multi-site resting-state fMRI
datasets: ABIDE-I, ABIDE-II, and ADHD-200, demonstrate that FedDOSE outperforms state-of-the-art methods in ASD and ADHD detection. Our results highlight its effectiveness in learning robust representations from multi-site
datasets for reliable analysis.
Read the full abstract ▾
All authors Deepank Girish, Yi Hao Chan, Yubin Zheng, Sukrit Gupta, Jagath C. Rajapakse
Audio and Speech Processing · Cross-listed from cs.LG
Omni-modal decomposition autoencoders learn full-stack wearable disentangled representations
Ioannis Ziogas, Ensieh Khazaei, Bilal Taha et al. (+4 more)
Learning disentangled representations is a key requirement for developing versatile, general-purpose, and sustainable models in multi-modal wearable computing. However, existing approaches do not operate as full-stack wearable processors, i.e., they do not simultaneously address task-specific classification performance, disentangled and interpretable representation learning, fusion, and generative modeling of highly heterogeneous multi-modal time series. To address this gap, we introduce Omni-modal Variational Decomposition Autoencoders (OmniDecVAEs), a framework that efficiently learns multi-purpose representations in a unified and scalable manner from arbitrarily many modalities. OmniDecVAEs extend DecVAEs by learning modality-conditioned time-frequency latent subspaces through a multi-view self-supervised decomposition loss and a shared asymmetric autoencoder (AE) architecture. Results on a challenging omni-modal human activity recognition (HAR) setting with up to thirty modalities, demonstrate the ability of OmniDecVAEs to learn full-stack wearable representations. When compared to
transformer-based and VAE-based methods, OmniDecVAEs full-stack disentangled representation properties lead to accuracy improvements of 1.01% and 6.75% in activity and identity recognition, respectively. Furthermore, OmniDecVAEs synthesize realistic omni-modal time-frequency data that manifest with enhanced reconstructions (mean absolute error improves by 76.84%) and distributional similarity between real and synthetic data (maximum mean discrepancy improves by 13.85%). Our results highlight OmniDecVAEs potential as a lightweight model suitable for intelligent edge wearables and clinical healthcare, unifying processing requirements and abilities in a single model, through its enhanced representational capacity, modality-invariant spatial complexity (4.1M parameters), and real-time latency.
Read the full abstract ▾
All authors Ioannis Ziogas, Ensieh Khazaei, Bilal Taha, Aamna Al Shehhi, Ahsan H. Khandoker, Leontios J. Hadjileontiadis, Dimitrios Hatzinakos