Image and Video Processing
Non-contact, Real-time, Heart-rate Measurement using Image Processing with Commodity Cameras and AI Agents
Kelly Li and Fulu Li
Heart rate measurement is one of the key requirements for real-time health monitoring, in particular for health caring of elderly people. Traditional heart rate measurement relies on contact sensing mechanisms such as some heart rate measurement devices at medical hospitals or some wearable devices with embedded sensors such as Apple Watch, etc. In this paper, we develop a system for non-contact, real-time, heart rate measurement using image processing with commodity cameras such as an embedded camera on a laptop, where we use an innovative
algorithm to capture the relevant signals for the computation of heart rate in a time series in real life environments. The presented heart rate computation (HRC) process is composed with four major steps: (a) identify frames per second of the camera in use, i.e., 30 frames per second for a given camera, (b) face detection (FD) with shape predictor of 68 face landmarks using deep learning (DL) method, (c) time sliding window (TSW)
algorithm to de-noise the signal by smoothing out the noise, and (d) compute heart rate based on identified signal periodicity. We test and analyze the developed prototypes against heart rate results by Apple Watch and check the difference range in multiple rounds and compute the mean of the difference for the measurement values of the heart rate of the same person at the same time. We will do further tuning and optimization of the present methods and deploy the system as a personal AI
agent [6] for health monitoring as our future directions.
Read the full abstract ▾
Image and Video Processing
Reconfigurable Radiology Labels Without Relabeling
Jean-Benoit Delbrouck, Dave Van Veen, Akash Pattnaik et al. (+4 more)
Public chest-radiograph (CXR)
datasets are typically released with small, fixed label schemas such as CheXpert-14. However, the underlying free-text reports describe far more findings – and which findings matter depends on the task, site, and reader. We release a pipeline that converts free-text reports into multi-label
matrices and then reconfigures the label schema through dictionary edits rather than new inference passes, i.e., without relabeling the corpus. After this one-time pass, reconfiguring MIMIC-CXR (223K reports) from cached annotations takes 196 seconds with no API cost, compared to \$6.6K for an equivalent relabeling pass with Claude Opus 4.7. Using a 58-label taxonomy, we show that 43% of CXR studies contain at least one finding outside CheXpert-14. Image probes trained on these labels match CheXpert-14 probes on shared targets while also reaching 0.78 AUROC on expert-reviewed long-tail labels that CheXpert-14 cannot represent. These results suggest a different unit of work for radiology labeling: once reports are structured, the label schema becomes a configuration to edit, not a corpus to relabel.
Read the full abstract ▾
All authors Jean-Benoit Delbrouck, Dave Van Veen, Akash Pattnaik, Kalina Slavkova, Javid Abderezaei, Harris Bergman, Khan Siddiqui
Systems and Control
Input-to-State Stability Implications in Contraction Theory
Yu Kawano and Francesco Bullo
For nonlinear control systems on normed vector spaces, we characterize an incremental input-to-state stability (ISS) type property in which the overshoot constant multiplies both the initial-condition and the input terms. Working through the associated variational system, we show that two properties are equivalent: an ISS-type bound on the variational system, and the incremental ISS-type bound on the original system. We further establish the equivalence between an infinitesimal contraction condition, expressed through a Lyapunov-type function, and an incremental Lyapunov condition. Each of these equivalent conditions yields a necessary condition and a sufficient condition for the ISS-type bounds, differing only in the input Lipschitz constant of the vector field. When the overshoot constant equals one, the infinitesimal contraction condition reduces to the standard norm-based contraction conditions. We establish these implications under mere continuous differentiability of the vector field, and we illustrate the results through sensitivity
matrices and Lyapunov characteristic exponents. Moreover, we develop similar implications for discrete-time systems.
Read the full abstract ▾
Signal Processing
Sensor-Adaptive Infrared Spectral Reconstruction with Plug-and-Play Diffusion Priors
Alireza Siyavashi, Jon Schlipf, Sebastian Reiter et al. (+3 more)
Hyperspectral sensing enables material identification; however, state-of-the-art spectrometers are costly and bulky, which limits their use in mobile applications. We address this by proposing sparse spectrum reconstruction from narrowband photocurrents using a pseudoinverse-guided
diffusion model ({\Pi}GDM). With {\Pi}GDM we use a denoising
diffusion probabilistic model (DDPM) to reconstruct the spectrum, which is trained on a large public spectral
dataset to learn realistic spectral priors, eliminating the need for paired sensor measurements. At inference, {\Pi}GDM alternates reverse-
diffusion denoising steps with pseudoinverse projection to enforce consistency with measured photocurrents via the calibrated responsivity
matrices of sensors. Consequently, our method is sensor-adaptive: when detector arrays change, we simply substitute the responsivity
matrix in the pseudoinverse projection without retraining of the
diffusion model. The resulting computational spectrometer achieves 1.502% average estimation error, outperforming Tikhonov, Gaussian, compressive-sensing, and multilayer perceptron (MLP) baselines, while providing calibrated uncertainty estimates via Monte Carlo sampling from different random initializations of {\Pi}GDM. Summarizing, our approach offers an accurate, compact alternative for spectral recovery on resource-constrained platforms.
Read the full abstract ▾
All authors Alireza Siyavashi, Jon Schlipf, Sebastian Reiter, Inga Fischer, Christian Wenger, Christian Herglotz
Signal Processing
Unexpected Far-Near-Far Transition in Mobile Near Field Terahertz Communications
Peng Zhang, Hanmei Yuan, Zhe Wang et al. (+2 more)
At THz frequencies, the radiative near-field distance can be sufficiently large to matter in real deployments. Existing near-field formulas are often understood in a simple way: as the link distance decreases, the propagation regime is expected to change only once, i.e., from far field to near field. This paper shows that this intuition can fail for an elevated access point with downward tilt serving a ground user moving along the ground. Along such a path, the link distance and the viewing angle change together, so the near-field to far-field transition may take place more than once, creating an unexpected far-near-far transition. In this paper, we derive analytical conditions for when this transition occurs for tilted ULA-to-point and UPA-to-point scenarios and compute the corresponding transition point(s) on the ground. Numerical results validate the analysis and further show that this behavior depends strongly on the deployment geometry and can also arise at lower frequencies.
Read the full abstract ▾
All authors Peng Zhang, Hanmei Yuan, Zhe Wang, Vitaly Petrov, Emil Björnson
Audio and Speech Processing · Cross-listed from cs.CL
Revisiting the Relation Between Language Model Perplexity and ASR Word Error Rate for Modern End-to-End Speech Recognition
Mohammad Zeineldeen, Albert Zeyer, Haoran Zhang et al. (+3 more)
Language model (LM) perplexity (PPL) has historically been used as a proxy for automatic
speech recognition (ASR) word error rate (WER), with prior work reporting an approximately linear relation in log-log space. Modern end-to-end ASR systems challenge this assumption because they already contain internal
language modeling capacity, are often evaluated without external
language models, and can now be combined with neural LMs and large
language models (LLMs) through different recognition strategies. This paper revisits the relation between PPL and WER for modern ASR systems. We study whether external LMs still improve current end-to-end ASR systems, whether the PPL-WER relation remains linear in log-log space, how encoder context length affects this relation, and how LLM perplexities fit into the trend observed for standard neural LMs. We further investigate internal
language modeling (ILM) in
attention-based encoder-decoder systems and show that ILM subtraction changes the observed PPL-WER relation, indicating that the decoder's internal LM must be considered when interpreting the effect of external LM quality.
Read the full abstract ▾
All authors Mohammad Zeineldeen, Albert Zeyer, Haoran Zhang, Robin Schmitt, Ralf Schlüter, Hermann Ney
Systems and Control
Reachability Analysis for Power Systems with Heterogeneous Resources via Jordan Transformation
Damola Ajeyemi, Antonin Colot, Sairaj Dhople et al. (+2 more)
This paper develops a computationally efficient framework for reachability analysis of transmission-level power system dynamics with synchronous generators, grid-forming and grid-following inverters, and uncertain power injections/withdrawals. Starting from reduced-order device models and a frequency-divider representation, we derive a linear ordinary-differential-equation model suitable for efficient reachable-set computation under bounded disturbances across network buses. The proposed reachability method combines interval reachability and contraction-based bounds to construct certified over-approximations for the linear ordinary-differential-equation model. A real Jordan transformation separates non-oscillatory modes, handled through a linear
embedding system, from oscillatory modes, enclosed using contraction-based ball bounds. Numerical experiments on a modified IEEE 39-bus system validate the reachable tubes against high-fidelity electromagnetic-transient (EMT) simulations, and demonstrate multi-second reachable sets computed in sub-second time.
Read the full abstract ▾
All authors Damola Ajeyemi, Antonin Colot, Sairaj Dhople, Emiliano Dall'Anese, Saber Jafarpour
Signal Processing
DBNN: Neural Spike Classification Using a Deep Binarized Neural Network
Binyi Ren, Luca M. Meyer and Majid Zamani
Implantable
brain-computer interfaces require on-node spike sorting to reduce telemetry bandwidth and power while maintaining reliable neural decoding. This paper presents a hardware-oriented deep binarized neural network (DBNN) spike-sorting system with two binarized hidden layers with 256
neurons and a fixed-point output layer to enable multiplier-free inference dominated by sign-controlled accumulation and bit-wise logic. The proposed classifier operates on compact 16-sample spike waveforms to reduce the implementation cost (16-256-256-3) and achieves a median classification accuracy of 98.7% on both synthetic and in-vivo
datasets. An FPGA prototype on a Cyclone V device operates at 50 MHz and requires 528 cycles per spike, corresponding to a 0.01 ms compute latency, while consuming 828 ALMs and 1023 registers with zero DSP blocks. For ASIC feasibility, the DBNN is implemented using FreePDK45-based flow; synthesis in Synopsys Design Compiler indicates an estimated silicon area of 0.014 mm2 and an operating power of 122 nW at 20 kHz under a 1.1 V supply. These results demonstrate that the proposed DBNN spike sorter offers a favorable trade-off between accuracy and implementation cost, supporting low-power, implantable neural interfaces. Overall, the proposed DBNN spike sorter achieves high accuracy (98.7%) with extremely low hardware cost (0.014 mm2, 122 nW at 20 kHz) and multiplier-free operation, making it suitable for low-power, implantable neural interfaces. This paper introduces the first DBNN designed for real-time neural spike sorting, striking an excellent balance between input data size and network complexity.
Read the full abstract ▾
Audio and Speech Processing
Distributed Multichannel Wiener Filtering for Topology-Unconstrained Wireless Acoustic Sensor Networks
Paul Didier, Pourya Behmandpoor, Henri Gode et al. (+4 more)
This paper introduces the
topology-independent distributed multichannel Wiener filter (TI-dMWF), a novel
algorithm for distributed node-specific signal estimation in wireless acoustic sensor networks (WASNs) with unconstrained
topologies. The TI-dMWF enables each node in the network to compute its centralized multichannel Wiener filter solution by exchanging only low-dimensional fused signals, without requiring iterative estimation, unlike state-of-the-art approaches such as the
topology-independent distributed adaptive node-specific signal estimation (TI-DANSE)
algorithm. The TI-dMWF is proven optimal when each source is observed by either all nodes or only one node. Theoretical analysis and numerical simulations confirm that it achieves centralized estimation performance in a single run. Its latency as a function of the pruned-tree depth and its computational complexity are also analyzed. Its robustness is assessed in reverberant-room simulations under estimated second-order statistics, various network
topologies, and deviations from the assumed observability model.
Read the full abstract ▾
All authors Paul Didier, Pourya Behmandpoor, Henri Gode, Toon van Waterschoot, Simon Doclo, Jörg Bitzer, Marc Moonen
Systems and Control · Cross-listed from cs.LG
Federated Physics-Grounded Reinforcement Learning for Distributed Stability Control in Smart Grids
Omar Al-Refai, Ibrahim Shahbaz, Adam Ali Husseinat et al. (+1 more)
Transient stability control in smart grids requires rapid post-fault damping of generator frequency and rotor angle deviations to prevent cascading failures. This paper proposes FedPPO-PG, a
Federated Multi-
Agent Proximal Policy Optimization framework with Physics-Grounded neighborhoods, which reformulates transient stability control as a cooperative multi-
agent reinforcement learning problem optimized directly against closed-loop stability objectives. Each generator hosts an independent local actor augmented with the frequency deviations of its two most strongly coupled electrical neighbors, identified from the post-fault Kron-reduced susceptance
matrix. A guided policy initialization phase warm-starts all actors from the classical decentralized controller, while a centralized critic guides advantage estimation under the centralized training–decentralized execution (CTDE) paradigm. Evaluated on a simulation of the IEEE 39-bus
benchmark system across five training and three unseen fault contingencies, FedPPO-PG achieves 100% stabilization in all 24 trials, reduces mean stability time by 72.4%, and cuts the control power by 7-14 times compared to the centralized baseline. Each actor executes independently with no central coordinator at deployment, and the per-actor inference latency satisfies the IEEE/IEC 60255-118-1-2018 real-time reporting requirements.
Read the full abstract ▾
All authors Omar Al-Refai, Ibrahim Shahbaz, Adam Ali Husseinat, Eman Hammad
Signal Processing
Replicating the Signature: Unsupervised Targeted Impersonation Attack on RF Fingerprinting
Haytham Albousayri and Bechir Hamdaoui
This paper presents a novel impersonation attack framework that aims to fool RF Fingerprinting (RFFP) identification systems by synthesizing signals that replicate the hardware-specific impairments of a target device. Our framework leverages unsupervised learning to enable accurate impairment estimation, combined with signal processing-based generation to synthesize high-fidelity
adversarial signals. Unlike prior works that assume full access to the legitimate (victim) RFFP classifier, we consider a more realistic attack strategy where the adversary performs the attack from a completely different transceiver hardware. We further evaluate our proposed attack under realistic and challenging deployment settings, including over-the-air transmission in both Line-of-Sight (LoS) and Non-Line-of-Sight (NLoS) scenarios. Extensive experiments conducted on a Bluetooth Low Energy (BLE) device testbed demonstrate that our attacks remain highly effective even under severe access constraints, significantly outperforming existing baselines in terms of targeted attack success rates by over 80%. We additionally analyze the effects of cross-domain generalization, signal representation mismatch, and classifier diversity, highlighting the robustness and
Read the full abstract ▾
Image and Video Processing · Cross-listed from cs.CV
A Task-Driven Evaluation of UAV Detection and Tracking under Synthetic Fog
Amir Pouladi, Vesal Ahsani, Haijun Li et al. (+2 more)
Fog severely degrades the visibility of small unmanned aerial vehicles (
UAVs) in skydominant, long-range imagery, reducing the reliability of downstream detection and tracking. This paper presents a task-driven evaluation framework that links depth-aware synthetic fog generation, image restoration, object detection, and tracking within a unified pipeline. Given the practical difficulty of collecting and annotating foggy
UAV scenes, synthetic fog is generated from real clear-weather outdoor images containing
UAV targets using
monocular depth estimation and the atmospheric scattering model. Representative restoration methods from classical, convolutional neural network (CNN)-based, and
transformer-based families are first compared, after which the selected restoration model is integrated into the downstream perception pipeline. Detection is evaluated under both clean-only and fog-inclusive training regimes using multiple detector variants, while tracking-by-detection is assessed on clean, foggy, and restored
video sequences. Beyond image-level restoration metrics, the study evaluates how fog and restoration affect detection robustness and tracking performance. The results show that fog substantially degrades both detection and tracking, primarily through increased missed detections. Fog-inclusive training provides the most consistent improvement in robustness, whereas test-time restoration is most beneficial when the detector has been trained only on clean imagery. These findings show that restoration quality does not necessarily translate into proportional gains in downstream perception and therefore should be evaluated jointly with detection and tracking performance.
Read the full abstract ▾
All authors Amir Pouladi, Vesal Ahsani, Haijun Li, Homayoun Najjaran, Afzal Suleman
Audio and Speech Processing · Cross-listed from cs.CL
SPEARBench: A Benchmark for Naturalness Evaluation in Streaming Speech-to-Speech Language Models
Thomas Thebaud, Yuzhe Wang, Hao Zhang et al. (+5 more)
Streaming
speech-to-
speech language models aim to answer spoken queries directly with synthetic
speech. However, standard
speech and text
benchmarks do not capture whether these systems behave naturally in conversations, where timing, turn-taking, prosody, interpersonal stance,
language and dialect consistency, and relationship-aware appropriateness jointly shape perceived quality. We introduce SPEARBench, a
benchmark for evaluating naturalness in
speech-to-
speech language models from question-answer interactions. SPEARBench constructs controlled dialogue
prompts from the Seamless Interaction corpus, runs inference across multiple models, and evaluates generated answers using a multidimensional protocol that covers response latency, interruptions,
speech quality, ASR robustness,
language and dialect consistency, emotional naturalness, interpersonal stance, and explainable distributional baselines. The
benchmark includes original human answers as a reference condition and reports results for several contemporary models. Results show that current models can achieve high signal-level quality and low ASR error while still differing from human conversational behavior in latency, overlap, dialect preservation, emotional adaptation, and interpersonal stance dynamics.
Read the full abstract ▾
All authors Thomas Thebaud, Yuzhe Wang, Hao Zhang, Sathvik Manikantan Napa Ugandhar, Ashish Hallur, Georgi Tinchev, Venkatesh Ravichandran, Laureano Moro-Velazquez
Signal Processing
Wavelet Scattering Transform for Interpretable Schizophrenia Biomarker Discovery and Classification from Resting-State EEG
Md. Taksimul Ahsan Tawhid, Nasif Ahmed Rafe, Alif Tahmid Priyom et al. (+1 more)
Schizophrenia is a debilitating neuropsychiatric disorder characterized by profound cortical network dysregulation, for which objective, clinically translatable
EEG based biomarkers remain underdeveloped. Existing automated classification pipelines rely predominantly on static power spectral density features inherently blind to amplitude modulation dynamics and cross-frequency coupling, phenomena central to schizophrenia pathophysiology, while adopting epoch level cross validation strategies that introduce temporal data leakage, artificially inflate reported performance. This study introduces a mathematically principled diagnostic framework integrating the multi-order Wavelet Scattering Transform(WST), strict Leave One Subject Out (LOSO) cross-validation, and SHAP explainability for simultaneous
EEG classification and biomarker discovery. Hierarchical WST coefficients capturing multi-scale amplitude modulation structure were extracted from resting state multichannel
EEG. Subject-level ANOVA with Benjamini Hochberg false discovery rate correction identified significant biomarkers, with Random Forest and SVM classifiers evaluated under strict LOSO cross validation and subject-level majority voting. Second-order scattering coefficients encoding cross frequency coupling dominated the discriminative biomarker set, with gamma-band features most prevalent, demonstrating that temporal amplitude modulation constitutes the primary electrophysiological signature of schizophrenia. Electrode P3 was identified as the single most discriminative site. Under rigorous subject independent evaluation, the Random Forest achieved 90.48% accuracy (AUC = 0.9339; sensitivity = 95.56%). The proposed WST framework establishes a rigorous, interpretable standard for
EEG-driven psychiatric biomarker discovery that can also be applicable in the detection of schizophrenia subtypes in the future.
Read the full abstract ▾
All authors Md. Taksimul Ahsan Tawhid, Nasif Ahmed Rafe, Alif Tahmid Priyom, K. M. Mustafizur Rahman
Audio and Speech Processing
ProPS: Prompted Profile Synthesis for Natural Language-Conditioned Speaker Embedding Distributions
Thomas Thebaud, Junhyeok Lee, Laureano Moro-Velazquez et al. (+2 more)
Speaker
embeddings, or x-vectors, are widely used to represent speaker identity and speaker-related attributes, but existing
embedding extractors are typically descriptive rather than generative: they map an observed
speech segment to an x-vector, which is then used for downstream applications. We introduce ProPS, Prompted Profile Synthesis, a framework for generating distributions of speaker
embeddings conditioned on natural
language prompts such as "a thirties male speaker with an Indian accent". ProPS converts human-written profile descriptions into sentence
embeddings and uses a mixture density network trained on a large-scale
dataset to predict a Gaussian mixture model in the x-vector space. The model is trained by maximizing the likelihood that real speaker
embeddings match the requested profile, and its generated distributions are evaluated by negative log-likelihood on held-out x-vectors and by attribute classification accuracies on sampled synthetic x-vectors. Experiments show that ProPS produces profile-conditioned distributions and generates x-vectors that preserve requested speaker attributes such as age, gender, accent, and prosodic characteristics. This design enables controllable speaker-profile synthesis for
speech generation systems like Text-To-
Speech (TTS) or Voice Conversion (VC) while anchoring generated distributions in observed speaker-
embedding structure.
Read the full abstract ▾
All authors Thomas Thebaud, Junhyeok Lee, Laureano Moro-Velazquez, Jesus Villalba Lopez, Najim Dehak
Signal Processing
Optimal Base Station Placement for Beyond 5G Networks with Non-Convex Topology
Mohamed Shalma, Amr Mansour and Ahmed El-Mahdy
This paper investigates the optimal placement of a millimeter-wave (mmWave) base station (BS) within a realistic U-shaped environment with non-convex
topology. The problem is challenging and NP-hard due to the non-convex
topology and the non-convex objective functions which are the sum-rate maximization and max-min
fairness, the latter being additionally non-smooth. To address this challenge, the BS placement is formulated as a Markov Decision Process (MDP). Then, we propose two deep reinforcement learning (DRL) techniques: First, the deployment area is discretized into a grid and optimized using a Deep Q-Network (DQN). Second, the U-shaped region is partitioned into continuous subspaces, where a Deep Deterministic Policy Gradient (DDPG)
agent is dedicated to each subspace then the best BS placement is selected among partitions. Results demonstrate that optimal placement achieves full coverage and yields a Jain index of 0.99. Furthermore, the proposed partitioned multi-space DDPG achieves better solution than DQN with lower complexity.
Read the full abstract ▾
Systems and Control
Shadow Pricing of Static Voltage Stability Services within Unit Commitment for Inverter-Dominated Power Systems
Peng Wang and Luis Badesa
Modern power systems are increasingly dominated by Inverter-Based Resources (IBR), most of which work in Grid-following (GFL) mode. This implies that they do not directly control their terminal voltage, so the static voltage stability at these buses may be compromised, especially under constant-power-factor operation that lacks voltage-adaptive reactive support. In addition, weather-driven IBR are often installed in electrically remote areas with low Short-
Circuit Ratio (SCR), further exacerbating voltage issues. To address this challenge, grid-forming control can be utilized to enhance low-SCR buses, while GFL-IBR could be explicitly required to provide voltage support through grid codes. As an alternative, a
market mechanism could be devised that incentivizes relevant generators to proactively adjust their operating points as a service to maintain voltage stability, while the theoretical framework for such a
market has not been developed. To fill this gap, this work adopts a second-order cone-based static voltage stability constraint for GFL-IBR buses within a unit commitment problem, and proposes a mechanism to assign shadow prices to this ancillary service. To determine appropriate price values under non-convex conditions, different pricing schemes are assessed. Using a modified IEEE 30-bus system, we demonstrate that both the dispatchable and restricted pricing methods can yield revenue-adequate service prices, though the former may deliver less efficient price signals and the latter may require well-defined uplift payments. This implies that, given differentiated pricing mechanisms and price signals, operators need to select a suitable pricing method in accordance with actual system conditions and
market rules.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.CL
Unified Audio Intelligence Without Regressing on Text Intelligence
Zhifeng Kong, Sang-gil Lee, Jaehyeon Kim et al. (+17 more)
Audio intelligence involves understanding,
reasoning about, and generating both audio and
speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM. Audex adopts a simple unified design with a single
Transformer decoder: audio inputs are encoded and projected into the text
embedding space, while text
tokens and quantized audio output
tokens are treated uniformly during generation. This architecture enables strong audio-text fusion, seamless
multimodal generation, and compatibility with standard LLM training and inference infrastructure. For training, we meticulously curate audio-text
datasets comprising 157.4B audio
tokens and 320.5B text
tokens. We apply multi-stage supervised training on these
datasets, followed by text-only Cascade RL and multi-domain on-policy
distillation. Audex delivers state-of-the-art audio understanding,
speech recognition and translation, text-to-
speech, audio generation, and
speech-to-
speech generation, while preserving very compelling
reasoning,
alignment, knowledge, long-context, and agentic capabilities of its text-only LLM backbone with marginal or no regression. We release the model checkpoints to facilitate open research.
Read the full abstract ▾
All authors Zhifeng Kong, Sang-gil Lee, Jaehyeon Kim, Boxin Wang, Zihan Liu, Sungwon Kim, Yang Chen, Arushi Goel, Rajarshi Roy, Wenliang Dai, Zhuolin Yang, Yangyi Chen, Dongfu Jiang, Sreyan Ghosh, Tuomas Rintamaki, Andrew Tao, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
Systems and Control · Cross-listed from cs.RO
VLM-CASE: Vision-Language Model Enabled Context-Adaptive Safety Envelopes for Anticipatory Safe Autonomous Driving
Tianjia Yang, Ke Li, Ruwen Qin et al. (+1 more)
Adverse driving conditions, such as bad weather, remain a principal barrier to autonomous driving because they degrade two things at once: what the vehicle can perceive and what it can physically do. Human drivers cope by anticipation,
reasoning about the scene and re-budgeting speed, following distance, and steering before grip or sight is lost, whereas current autonomous driving systems at best react after the fact. This paper proposes VLM-CASE, a framework that gives an autonomous vehicle this anticipatory capacity while keeping its motion bounded by a formal safety model at all times. A
vision-
language model (VLM), fine-tuned with low-rank adaptation (LoRA), reasons about the scene from the front-camera image and reports the road surface and visibility conditions. This output parametrizes a context-adaptive safety envelope (CASE), derived from physical limits and the guarantees of responsibility-sensitive safety, that couples braking and steering through a shared friction budget. A model predictive controller then drives freely within the envelope, while the VLM runs asynchronously so it never blocks the real-time control loop. We validate the framework in closed-loop CARLA simulation on tasks that demand both lateral and longitudinal control, across a range of weather, road-surface, and lighting conditions. The resulting controller, VLM-CASE-MPC, completes all trials, outperforming a conventional MPC baseline and a state-of-the-art VLM-integrated controller. Ablations confirm that the gains come from context adaptation, with the friction and visibility adaptations proving complementary. Furthermore, the framework is controller-agnostic and pairs with almost any low-level controller, offering a promising direction for safe autonomous driving. The
dataset and supplementary materials for VLM-CASE are available at https://github.com/ytj254/VLM-CASE.
Read the full abstract ▾
All authors Tianjia Yang, Ke Li, Ruwen Qin, Xianbiao Hu
Systems and Control
Model-Guided Local Bayesian Optimization for Tuning of Interpretable Controllers in Injection Molding
Jens Ahlers, Robert Göllinger, Xu Chen et al. (+2 more)
Advanced control methods have proven effective for controlling cavity pressure, a key determinant of part-quality attributes, in the plastics injection molding process. However, the abstract nature of the resulting control laws makes them difficult to interpret in a production environment, thereby limiting adoption in industrial applications. Additionally, controller optimization poses a severe challenge due to the diversity of mold geometries and materials. We propose a method to automatically optimize interpretable controllers during manufacturing while being cycle-efficient and risk-aware. The approach uses a Physics-Inspired Neural Mixture-of-Local-Experts model of the injection molding dynamics and augments its simulated closed-loop costs with a residual Gaussian Process, enabling Local
Bayesian Optimization of controller parameters. We
benchmark the
algorithm against Vanilla
Bayesian Optimization (BO) in simulation, using three controllers with parameter counts ranging from 1 to 30. Using the local method, we identify controller parameters that yield costs comparable to or lower than those of global BO over 20 optimization iterations, while mitigating high-cost excursions during tuning.
Read the full abstract ▾
All authors Jens Ahlers, Robert Göllinger, Xu Chen, Heike Vallery, Sebastian Stemmler
Signal Processing · Cross-listed from cs.NI
A Modular O-RAN Testbed Based on SRS Open Source O-CU/O-DU and Massive Beams Modular O-RU
Fabian Göttsch, Oriol Font-Bach, Andreas Benzin et al. (+5 more)
In this paper, we present a modular open
radio access network (O-RAN) consisting of the 5G Core, a central (O-CU) and distributed unit (O-DU) by Software
Radio Systems (SRS) and an O-RAN
radio unit (O-RU), MODRAD-SC, by Massive Beams (MB). OCUDU provides an open source 5G-compliant O-CU and O-DU solution developed by SRS, while MB's
radio unit is a fully O-RAN compliant category A O-RU. According to O-RAN split 7.2a, OCUDU performs higher layer functions up to the high physical (PHY) layer, while the O-RU handles low PHY and RF functions. This results in an O-RAN-compliant 5G gNodeB. In an alternative configuration, OCUDU and MODRAD-SC operate in a software-defined
radio fashion corresponding to split 8, facilitating non-real-time experiments among others. In both cases, the system provides full control over O-CU, O-DU, and O-RU. In addition, we will discuss the possibility to attach an analog beamformer to the O-RU, enabling hybrid digital-analog beamforming. The flexibility and modularity offered by OCUDU and MODRAD-SC enable the practical realization of a multitude of applications, ranging from 5G demonstrators to pre-6G experiments. The system addresses the requirements of academia and industry and is well-suited as an easy-to-use platform for experimental and practical deployments.
Read the full abstract ▾
All authors Fabian Göttsch, Oriol Font-Bach, Andreas Benzin, Dennis Osterland, Andre Puschmann, Felix-Christopher Lutz, Wilhelm Keusgen, Giuseppe Caire
Signal Processing
Multiuser MIMO-AFDM Beamforming for ISAC in Doubly Dispersive Channels
Rang Liu, Ming Li, A. Lee Swindlehurst et al. (+2 more)
Integrated sensing and communication (
ISAC) in high-mobility channels requires waveform and beamforming designs that are robust to delay-Doppler dispersion. With this in mind, in this paper we study a monostatic multiuser multiple-input multiple-output (
MIMO) affine frequency division multiplexing (AFDM) downlink system. We develop a discrete affine
Fourier transform (DAFT)-domain model that preserves Doppler-induced inter-bin coupling and derive a data-aided delay-Doppler detector. The expected matched-bin detector signal-to-noise ratio (SNR) is shown to be proportional to a transmit-covariance beampattern, which leads to a detector-SNR-based sector-illumination constraint. The resulting sensing-constrained weighted sum-rate maximization problem is solved using a combined weighted minimum mean squared error (WMMSE) and majorization-minimization (MM) formulation. Simulations show that the proposed AFDM design outperforms its orthogonal frequency division multiplexing (OFDM) counterpart in terms of the rate-sensing tradeoff, robustness to Doppler, and delay-Doppler sensing quality.
Read the full abstract ▾
All authors Rang Liu, Ming Li, A. Lee Swindlehurst, Norman Franchi, Robert Schober
Signal Processing
An Ambiguity-Function-Assisted Newtonized Channel Estimation Method for Pulse-Shaped AFDM Under Fractional Delay and Doppler
Yuanfang Ma, Zulin Wang, Yuanhan Ni et al. (+2 more)
Accurate channel estimation for pulse-shaped AFDM systems over doubly selective channels with fractional normalized delay and Doppler remains challenging. This paper proposes a low-complexity ambiguity-function-assisted newtonized channel (AFNC) estimation method. Specifically, we first present a closed-form input-output relation for pulse-shaped affine frequency division multiplexing (AFDM) under fractional normalized delay and Doppler. As a further step, we demonstrate that the input-output relation admits a low-complexity representation by offline precomputing and storing the discretized ambiguity function of the shaping pulse, followed by tailored cyclic-shift and stacking operations. Building on this representation, AFNC performs fractional delay-Doppler channel estimation through Newtonized refinement, where the required Jacobian and Hessian updates are computed efficiently using the low-complexity input-output representation. Simulation results confirm the effectiveness of the proposed approach.
Read the full abstract ▾
All authors Yuanfang Ma, Zulin Wang, Yuanhan Ni, Tao Luo, Peng Yuan
Systems and Control
Surrogate-based prioritization of sub-problems for Benders decomposition in energy planning
Wanhong Yu, Boyung Jürgens and Leonard Göke
Benders decomposition solves optimization problems by separating the first-stage master problem from one or more second-stage sub-problems. While the standard Benders decomposition solves all sub-problems in each iteration, solving only selected sub-problems still guarantees convergence and can reduce solution time, but raises the question of how to select. In this work, we introduce surrogate-based prioritization of sub-problems. The method leverages surrogates to estimate the sub-problems' objectives, assess the current error of the cutting-plane
estimator, and then prioritize the sub-problem with the largest error. We implement surrogate-based prioritization within sequential and asynchronous Benders decomposition. Both these
algorithms also leverage the surrogate to trigger convergence checks and implement regularization.
Benchmarks for an energy planning problem with a few large sub-problems show that the applied prioritization strategy works. The reduction in solution time correlates with the surrogate's accuracy. In our case,
geometric interpolation-based surrogates are more accurate than machine learning methods. As a result, prioritization consistently and significantly outperforms the standard
algorithm in sequential Benders decomposition. The speed-up increases with the number of scenarios, reaching 33% with four scenarios and 55% with ten scenarios. In the case of asynchronous parallelization, the impact on performance is less clear, and the average speed-up from prioritization is 19%.
Read the full abstract ▾
Audio and Speech Processing
Saba Tabatabaee, Mark Tiede, Suzanne Boyce et al. (+2 more)
Characteristic timing patterns are reflected in the acoustic
speech signal, encompassing both vocal tract configuration and acoustic excitation. Previous studies have demonstrated that
speech inversion (SI) systems can recover these timing patterns from
speech, including oral tract variables (tongue and lip constrictions) and source information such as periodic and aperiodic energies and fundamental frequency. In this study, we develop an SI system that simultaneously estimates oral tract variables and three source information parameters trained on co-recorded American English
speech audio and articulatory kinematics and investigate cross-linguistic generalizability by evaluating performance on previously unseen
languages. Pearson product-moment correlation scores of 0.83 and 0.74 were achieved on untrained French and Russian respectively, across oral tract variables and source information when comparing estimated data with ground-truth measurements.
Read the full abstract ▾
All authors Saba Tabatabaee, Mark Tiede, Suzanne Boyce, Liran Oren, Carol Espy-Wilson
Signal Processing
A Body-of-Revolution Human Model for RF Sensing with Measurement-Driven Calibration for Indoor Environments
Haoqing Wen, Michele D'Amico, Matteo Oldoni et al. (+5 more)
Model training for Device-Free Localization (DFL) and
Radio-Frequency (RF) sensing systems heavily relies on large-scale
datasets, which are costly and time-consuming to obtain through measurements across different environments and sensing configurations. Lightweight yet physically consistent propagation models are therefore critical for efficient generation of realistic RF sensing data. This paper presents an RF sensing prediction approach for indoor environments based on a Body of Revolution (BoR) human model. A fast 2.5-Dimensional Finite Element Method (2.5-D FEM) is proposed for computing the scattering fields of a human-like BoR model under the excitation of a vertical polarized dipole. Through comparisons, the proposed BoR model is shown to preserve scattering characteristics close to 3-D human bodies while yielding a smaller computational cost compared to a simple cylindrical model. A measurement-driven background-field modeling approach is further introduced for practical indoor applications, accounting for the complex propagation effects of indoor environments implicitly. Comparing with measurements of a typical indoor DFL scenario, the proposed approach achieves approximately 85% prediction accuracy and reproduces the spatial Received Signal Strength Indicator (RSSI) variations observed in practice, proving its potential for RF sensing prediction and large-scale database generation at a fraction of the computational cost required for full-wave simulations.
Read the full abstract ▾
All authors Haoqing Wen, Michele D'Amico, Matteo Oldoni, Federica Fieramosca, Vittorio Rampa, Stefano Savazzi, Qi Wu, Gian Guido Gentili
Systems and Control
Double interior-point regularization for large-scale capacity expansion
Leonard Göke and Giovanni Sansavini
Capacity expansion is a key tool for planning future energy systems. However, weather-dependent generation and long-duration storage result in problem sizes that exceed the computational limits of conventional interior-point solvers, making it impossible to plan renewable systems that are cost-efficient and reliable across a wide range of weather conditions. To tackle such large problems, this paper introduces the double interior-point regularization (DIP-set) for Benders Decomposition, combining the advantages of traversing the interior of the solution space while remaining close to a reference solution. We
benchmark the method on a power-sector problem and an energy-system problem, varying problem size and the level of foresight during operations. Results demonstrate that DIP-set outperforms competing regularizations in all test cases. The speed-up increases with size, reaching 30-50% for the largest problems, which are the most critical for planning renewable systems and are too large for state-of-the-art methods. The key benefit of DIP-set is its ability to mitigate the sharp decrease in convergence as BD approaches the optimal solution.
Read the full abstract ▾
Systems and Control
Weighted Phase Volume Method in Stability Analysis: Integral Criteria and Ellipsoidal Reachable Sets
Igor B. Furtat
A method for analysing the stability of dynamical systems is proposed, based on the introduction of a weighted phase volume and time rescaling by a positive function. The advantage of the method is the ability to set the contraction properties of the phase volume by choosing the weighting function and the scaling factor, while preserving the
topology of the phase portrait. Integral dissipativity conditions are derived, leading to new definitions of integral stability, asymptotic stability, and exponential stability. For quadratic weighting functions, covering and inner ellipsoids are constructed, providing
geometric estimates of reachable sets. The connection between the proposed approach and classical Lyapunov stability is established. The efficiency of the method is demonstrated through numerical examples.
Read the full abstract ▾
Systems and Control
Pathological Regimes of Closed-Loop Recommendation Systems over Social Networks
Mariano Simone and Frasca Paolo
This paper addresses the problem of designing recommendation systems for social networks and e-commerce platforms from a control-theoretic perspective. We formulate recommendation design as an infinite-horizon state-feedback optimal control problem whose performance index rewards
alignment/engagement while penalizing polarization, large deviations from an uncontrolled baseline, recommendation mismatch, control effort, and exposure disagreement across neighboring users. We derive explicit spectral conditions under which the reduced quadratic stage cost is strictly positive-definite, and we show that the failure of these conditions makes the resulting recommendation design exhibit pathological behaviors, such as unstable free modes, non-attainment of the infimum, or failure of the stationary affine synthesis.
Read the full abstract ▾
Image and Video Processing
Reduced-complexity Adaptive Loop Filtering via Input-dependent Graph Filters
Wen-Yang Lu, Eduardo Pavez, Antonio Ortega et al. (+2 more)
Adaptive Loop Filtering is an important tool for suppressing compression artifacts in modern
video codecs. In the enhanced compression model (ECM), a software test model used for experimenting with
video coding tools beyond Versatile
Video Coding, fixed filters are trained offline and achieve high signal adaptivity via a fine-grained gradient-based classifier, resulting in a large number of fixed filters that introduce redundancy and increased implementation complexity. Reducing this redundancy without compromising artifact suppression, therefore, remains a key challenge. This paper proposes an alternative graph-based fixed-filtering framework for adaptive loop filtering. By using a graph to encode pixel-intensity relationships, our approach captures local structural information more effectively than gradient-based classification alone. Fixed filters are learned as
polynomial graph filters, enabling structurally similar local patterns to share common filtering behavior. Experimental results demonstrate that the proposed approach achieves a comparable performance to the ECM baseline while reducing the number of required filters by an order of magnitude.
Read the full abstract ▾
All authors Wen-Yang Lu, Eduardo Pavez, Antonio Ortega, Roman Chernyak, Shan Liu