Audio and Speech Processing · Cross-listed from cs.SD
RA-QA: A Benchmarking System for Respiratory Audio Question Answering Under Real-World Heterogeneity
Gaia A. Bertolino, Yuwei Zhang, Tong Xia et al. (+2 more)
As conversational
multimodal AI tools are increasingly adopted to process patient data for health assessment, robust
benchmarks are needed to measure progress and expose failure modes under realistic conditions. Despite the importance of respiratory audio for mobile health screening, respiratory audio question answering remains underexplored, with existing studies evaluated narrowly and lacking real-world heterogeneity across modalities, devices, and question types. We hence introduce the Respiratory-Audio Question-Answering (RA-QA)
benchmark, including a standardized data generation pipeline, a comprehensive
multimodal QA collection, and a unified evaluation protocol. RA-QA harmonizes public RA
datasets into a collection of 9 million format-diverse QA pairs covering diagnostic and contextual attributes. We
benchmark general audio-
language models as well as domain-specific architectures, establishing reproducible reference points and showing how current approaches fail under heterogeneity.
Read the full abstract ▾
All authors Gaia A. Bertolino, Yuwei Zhang, Tong Xia, Domenico Talia, Cecilia Mascolo
Signal Processing
Newton-Direction-Based ReLU-Thresholding Methods for Nonnegative Sparse Signal Recovery
Ning Bian, Zhong-Feng Sun, Yun-Bin Zhao et al. (+2 more)
Nonnegative sparse signal recovery has been extensively studied due to its broad applications. Recent work has integrated rectified linear unit (ReLU) techniques to enhance existing recovery
algorithms. We merge Newton-type thresholding with ReLU-based approaches to propose two
algorithms: Newton-Direction-Based ReLU-Thresholding (NDRT) and its enhanced variant, Newton-Direction-Based ReLU-Thresholding Pursuit (NDRTP). Theoretical analysis iindicates that both
algorithms can guarantee exact recovery of nonnegative sparse signals when the measurement
matrix satisfies a certain condition.. Numerical experiments demonstrate NDRTP achieves competitive performance compared to several existing methods in both noisy and noiseless scenarios.
Read the full abstract ▾
All authors Ning Bian, Zhong-Feng Sun, Yun-Bin Zhao, Jin-Chuan Zhou, Nan Meng
Signal Processing
Combining scEEG and PPG for reliable sleep staging
Jiawei Wang, Liang Xu, Shuntian Zheng et al. (+7 more)
Reliable sleep staging remains challenging for lightweight wearable devices such as single-channel electroencephalography (scEEG) or photoplethysmography (PPG). scEEG offers direct measurement of cortical activity and serves as the foundation for sleep staging, yet exhibits limited performance on light sleep stages. PPG provides a low-cost complement that captures autonomic signatures effective for detecting light sleep. However, prior PPG-based methods rely on full night recordings (8 - 10 hours) as input context, which is less practical to provide timely feedback for sleep intervention. In this work, we investigate scEEG-PPG fusion for 4-class sleep staging under short-window (30 s - 30 min) constraints. First, we evaluate the temporal context required for each modality, to better understand the relationship of sleep staging performance with respect to monitoring window. Second, we investigate three fusion strategies: score-level fusion, cross-
attention fusion enabling feature-level interactions, and Mamba-enhanced fusion incorporating temporal context modeling. Third, we train and evaluate on the Multi-Ethnic Study of Atherosclerosis (MESA)
dataset and perform cross-
dataset validation on the Cleveland Family Study (CFS) and the Apnea, Bariatric surgery, and CPAP (ABC)
datasets. The Mamba-enhanced fusion achieves the best performance on MESA (Cohen's Kappa $\kappa$ = 0.798, Acc = 86.9%), with particularly notable improvement in light sleep classification (F1-score: 85.63% vs. 77.76%, recall: 82.85% vs. 69.95% for scEEG alone), and generalizes well to CFS and ABC
datasets with different populations. These findings suggest that scEEG-PPG fusion is a promising approach for lightweight wearable based sleep monitoring, offering a pathway toward more accessible sleep health assessment. Source code of this project can be found at: https://github.com/DavyWJW/scEEG-PPGFusion
Read the full abstract ▾
All authors Jiawei Wang, Liang Xu, Shuntian Zheng, Yu Guan, Kaichen Wang, Ziqing Zhang, Chen Chen, Laurence T. Yang, Sai Gu, M. Jamal Deen
Image and Video Processing
Deep Learning CNN for Pneumonia Detection: Advancing Digital Health in Society 5.0
Hadi Almohab
Pneumonia is a serious global health problem, contributing to high morbidity and mortality, especially in areas with limited diagnostic tools and healthcare resources. This study develops a Convolutional Neural Network (CNN) based on deep learning to automatically detect pneumonia from chest X-ray images. The method involves training the model on labeled
datasets with preprocessing techniques such as normalization, data augmentation, and image quality enhancement to improve robustness and generalization. Testing results show that the optimized model achieves 91.67% accuracy, ROC-AUC of 0.96, and PR-AUC of 0.95, demonstrating strong performance in distinguishing pneumonia from normal images. In conclusion, this CNN model has significant potential as a fast, consistent, and reliable diagnostic aid, supporting Society 5.0 by integrating artificial intelligence to improve healthcare services and public well-being.
Read the full abstract ▾
Image and Video Processing · Cross-listed from cs.CV
SOAR: Regression-based LiDAR Relocalization for UAVs
Hengyu Mu, Jianshi Wu, Yuxin Guo et al. (+5 more)
Regression-based
LiDAR relocalization has recently emerged as a promising solution for high-precision positioning in GNSS-denied environments. However, these methods are primarily tailored to autonomous driving, exhibiting significantly degraded accuracy in unmanned aerial vehicle (
UAV) scenarios due to arbitrary pose variations and irregular flight paths. In this paper, we propose SOAR, a regression-based
LiDAR relocalization framework for
UAVs. Specifically, we introduce a locality-preserving sliding window
attention module with locally invariant positional encoding to capture discriminative
geometric structures robust to viewpoint changes. A coordinate-independent feature initialization module is further designed to eliminate sensitivity to global transformations. Furthermore, most existing
UAV datasets are limited to evaluate
LiDAR relocalization in real-world, due to the lack of synchronized
LiDAR scans, accurate 6-DoF poses, or multiple traversals. Thus, we construct a large-scale
UAV LiDAR localization
dataset with 4 scenes and 13 irregular paths exhibiting rotation and altitude variations, providing a more realistic
benchmark for
UAVs. Extensive experiments demonstrate that our method achieves state-of-the-art performance, improving the localization success rate by 40% and reducing mean error over 10m on UAVLoc. Our code and
dataset will be released soon.
Read the full abstract ▾
All authors Hengyu Mu, Jianshi Wu, Yuxin Guo, XianLian Lin, Qingyong Hu, Sheng Ao, Chenglu Wen, Cheng Wang
Signal Processing
SIM-assisted Secure Mobile Communications via Enhanced Proximal Policy Optimization Algorithm
Wenxuan Ma, Bin Lin, Hongyang Pan et al. (+4 more)
With the development of sixth-generation (6G) wireless communication networks, the security challenges are becoming increasingly prominent, especially for mobile users (MUs). As a promising solution, physical layer security (PLS) technology leverages the inherent characteristics of wireless channels to provide security assurance. Particularly, stacked intelligent metasurface (SIM) directly manipulates electromagnetic waves through their multilayer structures, offering significant potential for enhancing PLS performance in an energy efficient manner. Thus, in this work, we investigate an SIM-assisted secure communication system for MUs under the threat of an eavesdropper, addressing practical challenges such as channel uncertainty in mobile environments, multiple MU interference, and residual hardware impairments. Consequently, we formulate a joint power and phase shift optimization problem (JPPSOP), aiming at maximizing the achievable secrecy rate (ASR) of all MUs. Given the non-convexity and dynamic nature of this optimization problem, we propose an enhanced proximal policy optimization
algorithm with a bidirectional long short-term
memory mechanism, an off-policy data utilization mechanism, and a policy feedback mechanism (PPO-BOP). Through these mechanisms, the proposed
algorithm can effectively capture short-term channel fading and long-term MU mobility, improve sample utilization efficiency, and enhance exploration capabilities. Extensive simulation results demonstrate that PPO-BOP significantly outperforms
benchmark strategies and other deep reinforcement learning
algorithms in terms of ASR.10.1109/TWC.2026.3658332
Read the full abstract ▾
All authors Wenxuan Ma, Bin Lin, Hongyang Pan, Geng Sun, Enyu Shi, Jiancheng An, Chau Yuen
Signal Processing · Cross-listed from cs.LG
AmbientFM: A Foundation Model for Ambient Sensing
Guozhen Zhu, Yuqian Hu, Sakila Jayaweera et al. (+6 more)
Ambient intelligence seeks to continuously understand human presence, activity, and physiology in physical spaces, enabling smart environments, health monitoring, and human-computer interaction. WiFi infrastructure offers a ubiquitous, always-on, and
privacy-preserving sensing substrate across billions of IoT devices. Yet ambient sensing with WiFi remains largely fragmented, with most systems relying on task-specific models, labeled data collection, and customized training pipelines. We present AmbientFM, a foundation model for ambient sensing through ubiquitous WiFi signals. AmbientFM is pre-trained on 9.2 million unlabeled Channel State Information (CSI) samples collected over 439 days from 20 commercial device types deployed in real homes, the largest real-world WiFi CSI pre-training corpus reported to date. It learns transferable wireless representations through contrastive learning, masked reconstruction, and physics-informed objectives tailored to wireless signals. With lightweight adaptation, a single backbone transfers to 9 downstream tasks, achieving above 0.90 AUROC on all classification
benchmarks and supporting dense spatial reconstruction. AmbientFM reduces the need for labeled data and task-specific design, enabling scalable ambient intelligence on existing wireless infrastructure.
Read the full abstract ▾
All authors Guozhen Zhu, Yuqian Hu, Sakila Jayaweera, Weihang Gao, Wei-Hsiang Wang, Jiaxuan Zhang, Beibei Wang, Chenshu Wu, K. J. Ray Liu
Audio and Speech Processing
Beyond the Utterance: An Empirical Study of Very Long Context Speech Recognition
Robert Flynn and Anton Ragni
Automatic
speech recognition (ASR) models are normally trained to operate over single utterances, with a short duration of less than 30 seconds. This choice has been made in part due to computational constraints, but also reflects a common, but often inaccurate, modelling assumption that treats utterances as independent and identically distributed samples. When long-format audio recordings are available, to work with such systems, these recordings must first be segmented into short utterances and processed independently. In this work, we show that due to recent algorithmic and hardware advances, this is no longer necessary, and current
attention-based approaches can be used to train ASR systems that operate on sequences of over an hour in length. Therefore, to gain a better understanding of the relationship between the training/evaluation sequence length and performance, we train ASR models on large-scale data using 10 different sequence lengths from 10 seconds up to 1 hour. The results show a benefit from using up to 21.8 minutes of context, with up to a 14.2% relative improvement from a short context baseline in our primary experiments. Through modifying various architectural components, we find that the method of encoding positional information and the model's width/depth are important factors when working with long sequences. Finally, a series of evaluations using synthetic data are constructed to help analyse the model's use of context. From these results, it is clear that both linguistic and acoustic aspects of the distant context are being used by the model.
Read the full abstract ▾
Audio and Speech Processing
Windowed SummaryMixing: An Efficient Fine-Tuning of Self-Supervised Learning Models for Low-resource Speech Recognition
Aditya Srinivas Menon, Kumud Tripathi, Raj Gohil et al. (+1 more)
Self-supervised learning (SSL) has advanced
speech processing but suffers from quadratic complexity due to self-
attention. To address this, SummaryMixing (SM) has been proposed as a linear-time alternative that summarizes entire utterances using mean pooling but lacks sufficient local context. In this work, we introduce Windowed SummaryMixing (WSM), which enhances SM by integrating local neighborhood summaries alongside the global summary, maintaining efficiency while improving temporal dependencies. Additionally, we introduce a selective fine-tuning approach, replacing self-
attention layers in SSL models with WSM blocks and fine-tuning only these blocks in low-resource settings. Our approach improves ASR performance while reducing peak VRAM usage by 40% in the SSL models. WSM blocks have linear-time complexity with enhanced context awareness. Selectively replacing some
attention layers reduces compute,
memory, and latency, making it ideal for low-resource
speech recognition.
Read the full abstract ▾
All authors Aditya Srinivas Menon, Kumud Tripathi, Raj Gohil, Pankaj Wasnik
Audio and Speech Processing · Cross-listed from cs.SD
The SJTU X-LANCE Lab System for MSR Challenge 2025
Jinxuan Zhu, Hao Qiu, Haina Zhu et al. (+3 more)
This report describes the system submitted to the music source restoration (MSR) Challenge 2025. Our approach is composed of sequential BS-RoFormers, each dealing with a single task including music source separation (MSS), denoise and dereverb. To support 8 instruments given in the task, we utilize pretrained checkpoints from MSS community and finetune the MSS model with several training schemes, including (1) mixing and cleaning of
datasets; (2) random mixture of music pieces for data augmentation; (3) scale-up of audio length. Our system achieved the first rank in all three subjective and three objective evaluation metrics, including an MMSNR score of 4.4623 and an FAD score of 0.1988. We have open-sourced all the code and checkpoints at https://github.com/ModistAndrew/xlance-msr.
Read the full abstract ▾
All authors Jinxuan Zhu, Hao Qiu, Haina Zhu, Jianwei Yu, Kai Yu, Xie Chen
Image and Video Processing
MTS-CSNet: Multiscale Tensor Factorization for Deep Compressive Sensing on RGB Images
Mehmet Yamac, Lei Xu, Serkan Kiranyaz et al. (+1 more)
Deep learning based compressive sensing (CS) methods typically learn sampling operators using convolutional or block wise fully connected layers, which limit receptive fields and scale poorly for high dimensional data. We propose MTSCSNet, a CS framework based on Multiscale Tensor Summation (MTS) factorization, a structured operator for efficient multidimensional signal processing. MTS performs mode-wise linear transformations with multiscale summation, enabling large receptive fields and effective modeling of cross-dimensional correlations. In MTSCSNet, MTS is first used as a learnable CS operator that performs linear dimensionality reduction in tensor space, with its adjoint defining the initial back-projection, and is then applied in the reconstruction stage to directly refine this estimate. This results in a simple feed-forward architecture without iterative or proximal optimization, while remaining parameter and computation efficient. Experiments on standard CS
benchmarks show that MTSCSNet achieves state-of-the-art reconstruction performance on RGB images, with notable PSNR gains and faster inference, even compared to recent
diffusion-based CS methods, while using a significantly more compact feed-forward architecture.
Read the full abstract ▾
All authors Mehmet Yamac, Lei Xu, Serkan Kiranyaz, Moncef Gabbouj
Image and Video Processing · Cross-listed from cs.CV
Markerless Head Tracking for Accurate and Accessible Neuronavigation
Ziye Xie, Oded Schlesinger, Raj Kundu et al. (+7 more)
Neuronavigation is widely used in biomedical research and interventions to guide the precise placement of instruments around the head to support procedures such as transcranial
magnetic stimulation. Traditional systems, however, rely on subject-mounted markers that require manual registration, may shift during procedures, and can cause discomfort. We introduce and evaluate markerless approaches that replace expensive hardware and physical markers with low-cost visible and infrared light cameras incorporating stereo and depth sensing, combined with algorithmic modeling of the facial geometry. Validation with 50 human subjects yielded a median tracking discrepancy of only 2.32 mm and 2.01$^\circ$ for the best markerless
algorithm compared to a conventional marker-based system, which indicates sufficient accuracy for transcranial
magnetic stimulation and a substantial improvement over prior markerless results. The study also suggests that integration of the data from the various camera sensors can improve the overall accuracy further. The proposed markerless neuronavigation methods can reduce setup cost and complexity, improve patient comfort, and expand access to neuronavigation in clinical and research settings.
Read the full abstract ▾
All authors Ziye Xie, Oded Schlesinger, Raj Kundu, Jessica Y. Choi, Pablo Iturralde, Dennis A. Turner, Stefan M. Goetz, Guillermo Sapiro, Angel V. Peterchev, J. Matias Di Martino
Systems and Control
Trojan Attacks on Neural Network Controllers for Robotic Systems
Farbod Younesi, Walter Lucia and Amr Youssef
Neural network controllers are increasingly deployed in robotic systems for tasks such as trajectory tracking and pose stabilization. However, their reliance on potentially untrusted training pipelines or supply chains introduces significant security vulnerabilities. This paper investigates
backdoor (Trojan) attacks against neural controllers, using a differential-drive mobile
robot platform as a case study. In particular, assuming that the
robot's tracking controller is implemented as a neural network, we design a lightweight, parallel Trojan network that can be embedded within the controller. This malicious module remains dormant during normal operation but, upon detecting a highly specific trigger condition defined by the
robot's pose and goal parameters, compromises the primary controller's wheel velocity commands, resulting in undesired and potentially unsafe
robot behaviours. We provide a proof-of-concept implementation of the proposed Trojan network, which is validated through simulation under two different attack scenarios. The results confirm the effectiveness of the proposed attack and demonstrate that neural network-based robotic control systems are subject to potentially critical security threats.
Read the full abstract ▾
Systems and Control
GPU-to-Grid: Voltage Regulation via GPU Utilization Control
Zhirui Liang, Jae-Won Chung, Mosharaf Chowdhury et al. (+2 more)
While the rapid expansion of data centers poses challenges for power grids, it also offers new opportunities as flexible loads. Existing power system research often abstracts data centers as aggregate resources, while computer system research focuses on GPU energy efficiency and largely ignores grid impacts. To bridge this gap, we develop a GPU-to-Grid framework that couples device-level GPU control with power system objectives. We study distribution-level voltage regulation enabled by LLM inference flexibility, using batch size as a data-center-side control knob that trades off GPU power consumption, inference latency, and
token throughput. We first formulate the problem as an optimization problem and then realize it as an online feedback optimization controller, implemented by the data center operator using its own empirical GPU power-performance model and real-time measurements from both the GPU and grid systems. Our key insight is that reducing GPU power alleviates lower-voltage violations, while increasing GPU power mitigates upper-voltage violations; this challenges the common belief that minimizing GPU power is always beneficial to power grids.
Read the full abstract ▾
All authors Zhirui Liang, Jae-Won Chung, Mosharaf Chowdhury, Jiasi Chen, Vladimir Dvorkin
Image and Video Processing
Personalized White Matter Bundle Segmentation for Early Childhood
Elyssa M. McMaster, Michael E. Kim, Nancy R. Newlin et al. (+12 more)
White matter
segmentation methods from
diffusion magnetic resonance imaging range from streamline clustering-based approaches to bundle mask delineation, but none have proposed a pediatric-specific approach. We hypothesize that a deep learning model with a similar approach to TractSeg will improve similarity between an
algorithm-generated mask and an expert-labeled ground truth. Given a cohort of 56 manually labelled white matter bundles, we take inspiration from TractSeg's 2D UNet architecture, and we modify inputs to match bundle definitions as determined by pediatric experts, evaluation to use k fold cross validation, the loss function to masked Dice loss. We evaluate Dice score, volume overlap, and volume overreach of 16 major regions of interest compared to the expert labeled
dataset. To test whether our approach offers statistically significant improvements over TractSeg, we compare Dice voxels, volume overlap, and adjacency voxels with a Wilcoxon signed rank test followed by false discovery rate correction. We find statistical significance across all bundles for all metrics with one exception in volume overlap. After we run TractSeg and our model, we combine their output masks into a 60 label atlas to evaluate if TractSeg and our model combined can generate a robust, individualized atlas, and observe smoothed, continuous masks in cases that TractSeg did not produce an anatomically plausible output. With the improvement of white matter pathway
segmentation masks, we can further understand neurodevelopment on a population level scale, and we can produce reliable estimates of individualized anatomy in pediatric white matter diseases and disorders.
Read the full abstract ▾
All authors Elyssa M. McMaster, Michael E. Kim, Nancy R. Newlin, Gaurav Rudravaram, Adam M. Saunders, Aravind R. Krishnan, Jongyeon Yoon, Ji S. Kim, Bryce L. Geeraert, Meaghan V. Perdue, Catherine Lebel, Daniel Moyer, Kurt G. Schilling, Laurie E. Cutting, Bennett A. Landman
Systems and Control
Learning Nonlinear Continuous-Time Systems for Formal Uncertainty Propagation and Probabilistic Evaluation
Peter Amorese and Morteza Lahijanian
Nonlinear ordinary differential equations (ODEs) are powerful tools for modeling real-world dynamical systems. However, propagating initial state uncertainty through nonlinear dynamics, especially when the ODE is unknown and learned from data, remains a major challenge. This paper introduces a novel continuum dynamics perspective for model learning that enables formal uncertainty propagation by constructing Taylor series approximations of probabilistic events. We establish sufficient conditions for the soundness of the approach and prove its asymptotic convergence. Empirical results demonstrate the framework's effectiveness, particularly when predicting rare events.
Read the full abstract ▾
Image and Video Processing · Cross-listed from cs.CV
Food Portion Estimation: From Pixels to Calories
Gautham Vinod and Fengqing Zhu
Reliance on images for dietary assessment is an important strategy to accurately and conveniently monitor an individual's health, making it a vital mechanism in the prevention and care of chronic diseases and obesity. However, image-based dietary assessment suffers from estimating the three dimensional size of food from 2D image inputs. Many strategies have been devised to overcome this critical limitation such as the use of auxiliary inputs like depth maps, multi-view inputs, or model-based approaches such as template matching. Deep learning also helps bridge the gap by either using
monocular images or combinations of the image and the auxillary inputs to precisely predict the output portion from the image input. In this paper, we explore the different strategies employed for accurate portion estimation.
Read the full abstract ▾
Signal Processing
Phase-Only Positioning in Distributed MIMO Under Phase Impairments: AP Selection Using Deep Learning
Fatih Ayten, Musa Furkan Keskin, Akshay Jain et al. (+5 more)
Carrier phase positioning (CPP) can enable cm-level accuracy in next-generation wireless systems, while recent literature shows that accuracy remains high using phase-only measurements in distributed
MIMO (D-
MIMO). However, the impact of phase synchronization errors on such systems remains insufficiently explored. To address this gap, we first show that the proposed hyperbola intersection method achieves highly accurate positioning even in the presence of phase synchronization errors, when trained on appropriate data reflecting such impairments. We then introduce a deep learning (DL)-based D-
MIMO antenna point (AP) selection framework that ensures high-precision localization under phase synchronization errors. Simulation results show that the proposed framework improves positioning accuracy compared to prior-art methods, while reducing inference complexity by approximately 19.7%.
Read the full abstract ▾
All authors Fatih Ayten, Musa Furkan Keskin, Akshay Jain, Mehmet C. Ilter, Ossi Kaltiokallio, Jukka Talvitie, Elena Simona Lohan, Mikko Valkama
Systems and Control · Cross-listed from math.OC
Banach Control Barrier Functions for Large-Scale Swarm Control
Xuting Gao, Guillem Pascual, Scott Brown et al. (+1 more)
This paper studies the safe control of very large multi-
agent systems via a generalized framework that employs so-called
Banach Control Barrier Functions (B-CBFs). Modeling a large swarm as probability distribution over a spatial domain, we show how B-CBFs can be used to appropriately capture a variety of macroscopic constraints that can integrate with large-scale swarm objectives. Leveraging this framework, we define stable and filtered gradient flows for large swarms, paying special
attention to optimal transport
algorithms. Further, we show how to derive
agent-level, microscopical
algorithms that are consistent with macroscopic counterparts in the large-scale limit. We then identify conditions for which a group of
agents can compute a distributed solution that only requires local information from other
agents within a communication range. Finally, we showcase the theoretical results over swarm systems in the simulations section.
Read the full abstract ▾
All authors Xuting Gao, Guillem Pascual, Scott Brown, Sonia Martínez