Audio and Speech Processing
In-Sync: Adaptation of Speech Aware Large Language Models for ASR with Word Level Timestamp Predictions
Xulin Fan, Vishal Sunder, Samuel Thomas et al. (+3 more)
Recent advances in
speech-aware
language models have coupled strong acoustic encoders with large
language models, enabling systems that move beyond transcription to produce richer outputs. Among these, word-level timestamp prediction is critical for applications such as captioning, media search, and
multimodal synchronization, yet it is often handled by external
alignment tools. In this work, we extend an existing
speech-aware
language model to predict timestamps directly alongside transcripts. We introduce a set of novel lightweight training strategies that improve
alignment robustness while preserving recognition quality. Experiments across multiple
datasets show that these strategies not only enhance timestamp accuracy, but also yield gains in overall ASR performance. Together, they demonstrate an efficient and unified approach to
speech recognition with precise timestamp prediction.
Read the full abstract ▾
All authors Xulin Fan, Vishal Sunder, Samuel Thomas, Mark Hasegawa-Johnson, Brian Kingsbury, George Saon
Signal Processing
Applied AI-Enhanced RF Interference Rejection
Rahul Jain, Pierre Trepagnier, Rick Gentile et al. (+2 more)
AI-enhanced interference rejection in
radio frequency (RF) transmissions has recently attracted interest because deep learning approaches trained on both the signal of interest (SOI) and the signal mixture (SOI plus interference) can outperform traditional approaches which only consider the SOI. The goal is to detect, demodulate, and decode signals over a range of signal-to-interference-plus-noise (SINR) levels without having a detailed, design-level knowledge of the interfering signal or the propagation conditions. Our present AI interference suppression results are based on Autoregressive
Transformer Decoder models which exhibit orders of magnitude faster throughput at inference time than WaveNet models developed in earlier work. As a specific example, we investigate an analog FM "Walkie Talkie"
radio signal of interest in the presence of an Orthogonal Frequency-Division Multiplexing (OFDM) interferer. This type of interferer is near-ubiquitous in the current RF landscape. Our results clearly show the benefits of
transformer-based interference mitigation in tactical settings. We show that unintelligible transmissions become intelligible via metrics such as Perceptual Evaluation of
Speech Quality (PESQ), while overall latency is kept to a minimum using readily available lightweight GPUs such as a Jetson AGX Orin. We believe these same techniques can also be applied to a broader set of national security scenarios, as well as having commercial applications.
Read the full abstract ▾
All authors Rahul Jain, Pierre Trepagnier, Rick Gentile, Joey Botero, Alexia Schulz
Systems and Control · Cross-listed from physics.soc-ph
On Common Misconceptions in Classical Vehicle Dynamics
Massimo Guiggiani
Classical vehicle dynamics contains several widely adopted misconceptions that, while intuitively appealing, may lead to inconsistencies when examined under a rigorous mechanical framework. This paper revisits a number of such misconceptions, clarifying their domains of validity and highlighting potential sources of misunderstanding. The aim is not to dismiss established models, but to promote a more precise and scientifically grounded interpretation of key concepts.
Read the full abstract ▾
Signal Processing
An AI-Ready Pipeline for Impedance-Resolved QCM Biosensor: Interpretable Line-Shape Features, Redundancy Control, and Robust Regression
Ceyhun Kirimli, Elcim Elgun and Yagmur Tugtag
Accurate inference from quartz
crystal microbalance (QCM) measurements in
liquids is often limited by reducing resonance behavior to two scalar endpoints (frequency and dissipation shifts, $\Delta f$ and $\Delta D$) or by relying on single-equation analytical models (Kanazawa-model). We propose an AI-ready, impedance-resolved workflow that preserves full resonance line-shape information and converts it into compact, physically interpretable features for supervised regression. A passive microfluidic mixer generates glycerol–water concentration gradients under a constant total flow rate (50~$\mu$L/min), while complex impedance spectra of a 10~MHz AT-cut quartz
crystal are recorded in real time. Each sweep of nine spectra is parameterized via constrained Gaussian/Lorentzian models to yield 52 line-shape descriptors spanning extrema of $|Z|$, $X$, and $B$ and peaks of $R$, phase, and $G$. The pipeline integrates consensus outlier handling, redundancy-aware feature ranking (mRMR), and cross-validated regression across linear, kernel, and ensemble models. Compared with the classical Kanazawa baseline, the impedance line-shape approach reduces concentration-prediction error from 0.456 to 0.148~%v/v RMSE (3.09$\times$ lower). The results demonstrate that impedance-resolved line-shape features provide a robust and interpretable basis for machine-learning-assisted QCM inference and illustrate a generalizable pattern for AI-enabled spectral sensing.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.CE
The Elusive Nature of Roughness: Linking Hydraulics and Graph Theory for Water Distribution Networks Model Calibration
Karol Dykiert, Mateusz Stolarski, Michał Czuba et al. (+2 more)
Accurate pipe roughness estimation in large-scale water distribution networks is often hindered by the high cost of traditional field methods. This study investigates whether network partitioning, by utilizing hydraulic and graph-derived attributes, can enhance the calibration of these parameters. Using a high-fidelity model of a real network as a
benchmark, we evaluate density-based clustering, and
topology-driven grouping strategies. Optimization experiments demonstrate that attribute-based grouping yields stable, repeatable results comparable to manual calibration for hydraulically significant pipes. While hydraulic attributes generate more distinct cluster structures, the inclusion of graph-based data improves calibration robustness by stabilizing the optimization process. Notably, density-based clustering achieves similar accuracy to k-means while reducing computational effort in specific configurations. Although the method does not eliminate all sources of uncertainty, results suggest that
topology-informed grouping provides a systematic, reproducible, and computationally efficient alternative to manual heuristics, highlighting the critical role of network structure in reliable parameter estimation.
Read the full abstract ▾
All authors Karol Dykiert, Mateusz Stolarski, Michał Czuba, Wojciech Cieżak, Piotr Bródka
Image and Video Processing · Cross-listed from cs.CV
FreqFormer: Hierarchical Frequency-Domain Attention with Adaptive Spectral Routing for Long-Sequence Video Diffusion Transformers
Haopeng Jin
Long-sequence
video diffusion transformers hit a quadratic self-
attention cost that dominates runtime and
memory for very long
token sequences. Most efficient
attention methods use one approximation everywhere, yet
video features are spectrally structured: low frequencies carry global layout and coarse motion; high frequencies carry texture and fine detail. We present FreqFormer, a frequency-aware heterogeneous
attention framework.
Token features are split into spectral bands with different operators: dense global
attention on compressed low-frequency content, structured block-sparse
attention on mid frequencies, and sliding-window local
attention on high frequencies. A lightweight spectral routing network allocates heads across bands using layer statistics and the
diffusion timestep, shifting compute toward global structure early in denoising and detail later. Cross-band summary
tokens provide cheap residual exchange. FreqFormer is paired with a fused GPU execution plan that co-schedules dense, sparse, and local branches to cut kernel launches and
memory traffic. We give a consistent complexity model, an orthonormal-decomposition view of approximation, and simulation-based systems numbers (throughput, arithmetic intensity,
memory traffic, duration scaling). In simulations from 64K to 1M
tokens, FreqFormer substantially reduces estimated
attention FLOPs and KV-related
memory traffic versus dense
attention while keeping a hardware-friendly pattern, supporting spectrally structured heterogeneous
attention as a practical direction for long-
video diffusion transformers.
Read the full abstract ▾
Systems and Control · Cross-listed from math.OC
Sliced Wasserstein Steering between Gaussian Measures
Kaito Ito and Anqi Dong
Optimal transport with quadratic cost provides a
geometric framework for steering an ensemble, modeled by a probability law, with minimal effort. Yet ambient-space formulations become unwieldy in high dimensions, and sensing or actuation in practice often reveals only linear views of the state – camera silhouettes,
LiDAR beams, tomographic slices. We develop a sliced feedback controller for distribution steering: the evolving law is projected onto one-dimensional directions on the sphere, the optimal one-dimensional velocity is synthesized in each projection, and these velocities are averaged to produce a feedback control in the ambient space. The construction reduces to the Benamou–Brenier problem in one dimension. In addition, it is invariant under orthogonal transforms, nonexpansive under projections, and well posed on $\mathcal{P}_2(\mathbb{R}^n)$. Computation proceeds by sampling directions on the sphere and solving independent one-dimensional subproblems, yielding a scalable method aligned with partial observations. In the Gaussian setting, we show that the developed sliced controller steers the law to the prescribed target. Furthermore, we derive an identity relating the energy consumption incurred by the controller to the sliced Wasserstein distance.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.CV
See No Evil: Semantic Context-Aware Privacy Risk Detection for AR
Jialu Liu, Yao Li, Zhuoheng Li et al. (+2 more)
Augmented reality (AR) systems pose unique
privacy risks due to their continuous capture of visual data. Existing AR
privacy frameworks lack semantic understanding of visual content, limiting their effectiveness in detecting context-dependent
privacy risks. We propose PrivAR, which leverages
vision language models (VLMs) with chain-of-thought prompting for contextual
privacy risk detection in AR environments. PrivAR uses visual scene cues to infer potential sensitive information types, such as identifying password notes in office environments through contextual
reasoning. PrivAR detects and obfuscates textual content, preventing exposure of sensitive information while preserving contextual cues necessary for VLM inference. Additionally, we investigate contextually-informed warning interfaces to enhance user
privacy awareness. Experiments on a real-world AR
dataset show that PrivAR achieves superior accuracy (81.48%) and F1-score (84.62%) compared to baselines, while reducing
privacy leakage rate to 17.58%. User studies evaluating contextually-informed warning interfaces provide insights into effective
privacy-aware AR design.
Read the full abstract ▾
All authors Jialu Liu, Yao Li, Zhuoheng Li, Huining Li, Ying Chen
Image and Video Processing · Cross-listed from quant-ph
Magnitude Is All You Need? Rethinking Phase in Quantum Encoding of Complex SAR Data
Sakthi Prabhu Gunasekar and Prasanna Kumar Rangarajan
Synthetic Aperture Radar (SAR) data is inherently complex-valued, while
quantum machine learning (QML) models operate in complex
Hilbert spaces. This similarity suggests that using both the magnitude and phase of SAR data in
quantum encoding should help automatic target recognition in SAR images. In this study, we test this assumption by comparing five encoding strategies for
quantum models: magnitude-only encoding, joint magnitude-phase encoding, in-phase and quadrature encoding, preprocessed phase encoding, and a purely
quantum architecture. All approaches are evaluated under a unified experimental setup on the MSTAR
benchmark dataset. Surprisingly, we find that magnitude-only encoding performs better than phase-inclusive encodings in hybrid
quantum-classical models. It achieves 99.57 percent accuracy on the 3-class task and 71.19 percent accuracy on the 8-class task, outperforming complex-valued alternatives under the same framework. Adding phase information provides little or no improvement and can sometimes degrade performance. However, in purely
quantum models with only 184 to 224 trainable parameters and no classical neural-network layers, phase information becomes much more important, improving accuracy by up to 21.65 percentage points. These findings show that the usefulness of phase information depends not only on the data, but also on the architecture used to process it. Hybrid models can compensate for missing phase information through their classical components, while pure
quantum models rely more strongly on phase information for class discrimination. The results provide practical guidance for encoding complex-valued data in
quantum machine learning and highlight the importance of jointly designing encoding strategies and model architectures for current
quantum systems.
Read the full abstract ▾
Signal Processing · Cross-listed from cs.LG
BioTrain: Sub-MB, Sub-50mW On-Device Fine-Tuning for Edge-AI on Biosignals
Run Wang, Victor J. B. Jung, Philip Wiese et al. (+5 more)
Biosignals exhibit substantial cross-subject and cross-session variability, inducing severe domain shifts that degrade post-deployment performance for small, edge-oriented AI models. On-device adaptation is therefore essential to both preserve user
privacy and ensure system reliability. However, existing sub-100 mW MCU-based wearable platforms can only support shallow or sparse adaptation schemes due to the prohibitive
memory footprint and computational cost of full backpropagation (BP). In this paper, we propose BioTrain, a framework enabling full-network fine-tuning of state-of-the-art biosignal models under milliwatt-scale power and sub-megabyte
memory constraints. We validate BioTrain using both offline and on-device
benchmarks on
EEG and EOG
datasets, covering Day-1 new-subject calibration and longitudinal adaptation to signal drift. Experimental results show that full-network fine-tuning achieves accuracy improvements of up to 35% over non-adapted baselines and outperforms last-layer updates by approximately 7% during new-subject calibration. On the GAP9 MCU platform, BioTrain enables efficient on-device training throughput of 17 samples/s for
EEG and 85 samples/s for EOG models within a power envelope below 50 mW. In addition, BioTrain's efficient
memory allocator and network
topology optimization enable the use of a large batch size, reducing peak
memory usage. For fully on-chip BP on GAP9, BioTrain reduces the
memory footprint by 8.1x, from 5.4 MB to 0.67 MB, compared to conventional full-network fine-tuning using batch normalization with batch size 8.
Read the full abstract ▾
All authors Run Wang, Victor J. B. Jung, Philip Wiese, Sebastian Frey, Giusy Spacone, Francesco Conti, Alessio Burrello, Luca Benini
Systems and Control · Cross-listed from math.OC
Network Epidemic Control via Model Predictive Control: Extended Version
Mahtab Talaei, Alex Olshevsky, Laura F. White et al. (+1 more)
Balancing the societal costs of non-pharmaceutical interventions with epidemic suppression requires adaptive feedback control. Rather than relying on state-dependent operational caps, we formulate an infinite-horizon optimal control problem for a networked SIQR model that strictly enforces suppression via a hard spectral constraint on the transmission dynamics. We derive a safety-critical Model Predictive Control (MPC) approximation that embeds this spectral certificate stage-wise, yielding a tunable exponential decay rate. Furthermore, we construct a terminal set ensuring recursive feasibility and a feasible continuation that decays globally, proving positive invariance directly via the physical depletion of susceptibles rather than standard quadratic Lyapunov functions. To handle prediction uncertainty, we develop a robust counterpart that replaces nominal constraints by upper-envelope versions, recovering recursive feasibility and finite-horizon realized decay. We conclude by validating our approaches using simulation studies that leverage public data from counties in the state of Massachusetts.
Read the full abstract ▾
All authors Mahtab Talaei, Alex Olshevsky, Laura F. White, Ioannis Ch. Paschalidis
Systems and Control
Digital Twin for Real-Time Security Assessment and Flexibility Activation in the Bornholm Distribution System
Anosh Arshad Sundhu, Aysegül Kahraman and Spyros Chatzivasileiadis
The increasing penetration of distributed energy resources (DERs) is transforming distribution networks into actively managed systems, introducing challenges related to voltage regulation, thermal loading limits, and operational security. This paper presents the development and implementation of a real-time Digital Twin (DT) for security assessment and coordinated flexibility activation in active distribution networks, demonstrated on the Bornholm Island system using real measurement data. The implemented DT integrates network
topology and smart meter measurements to perform security assessment under normal operation and N-1 contingencies, and to determine corrective and preventive flexibility actions using an optimization-based approach. Results show that load variation and contingency scenarios introduce operational limit violations, primarily driven by voltage magnitude constraints. The implemented flexibility strategy effectively mitigates these violations through coordinated active and reactive power control, enhancing system security and operational efficiency. The findings highlight the potential of DT-based approaches for reliable and flexible operation of future distribution networks.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.RO
Boundary Sampling to Learn Predictive Safety Filters via Pontryagin's Maximum Principle
James Dallas, Thomas Lew, John Talbot et al. (+3 more)
Safety filters provide a practical approach for enforcing safety constraints in autonomous systems. While learning-based tools scale to high-dimensional systems, their performance depends on informative data that includes states likely to lead to constraint violation, which can be difficult to efficiently sample in complex, high-dimensional systems. In this work, we characterize trajectories that barely avoid safety violations using the Pontryagin Maximum Principle. These boundary trajectories are used to guide data collection for learned Hamilton-Jacobi Reachability, concentrating learning efforts near safety-critical states to improve efficiency. The learned Control Barrier Value Function is then used directly for safety filtering. Simulations and experimental validation on a shared-control automotive racing application demonstrate PMP sampling improves learning efficiency, yielding faster convergence, reduced failure rates, and improved safe set reconstruction, with wall times around 3ms.
Read the full abstract ▾
All authors James Dallas, Thomas Lew, John Talbot, Jonathan DeCastro, Somil Bansal, John Subosits
Systems and Control
Path Integral Control in Gaussian Belief Space for Partially Observed Systems
Goutam Das and Takashi Tanaka
This paper extends path integral control (PIC) to partially observed systems by formulating the problem in Gaussian belief space. PIC uses the matching condition, which requires the
diffusion and control channels to be proportional, to linearize the Hamilton–Jacobi–Bellman equation through the Cole–
Hopf transform. We show that the belief
diffusion satisfies this condition only when the observation function is affine. Thus, the standard Cole–
Hopf linearization does not apply to non-affine observation models. Restricting the problem to Gaussian beliefs provides a finite-dimensional approximation with deterministic covariance
evolution and reduces the problem to
stochastic control of the belief mean. We derive necessary and sufficient conditions for matching in this reduced space, obtain an exact Cole–
Hopf linearization and a Feynman–Kac representation, and develop the MPPI-Belief
algorithm. Numerical experiments on a
navigation task with state-dependent observation noise demonstrate the effectiveness of MPPI-Belief compared with certainty-equivalent and particle-filter-based methods.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.CR
Threat Modeling and Attack Surface Analysis of IoT-Enabled Controlled Environment Agriculture Systems
Andrii Vakhnovskyi
The United States designates Food and Agriculture as one of sixteen critical infrastructure sectors, yet no mandatory cybersecurity requirements exist for agricultural operations and no formal threat model has been published for Controlled Environment Agriculture (CEA) systems. This paper presents the first comprehensive threat model for IoT-enabled CEA, applying STRIDE analysis, MITRE ATT&CK for ICS mapping, and IEC 62443 zone-and-conduit decomposition to a production platform deployed across 30+ commercial facilities in 8 U.S.
climate zones. We enumerate 123 unique threats across 25 data-flow-diagram elements spanning 15 communication protocols, 10 of which operate with zero authentication or encryption by design. We identify five novel attack classes unique to AI-driven CEA: stealth destabilization of neural-network-tuned PID controllers, baseline drift poisoning of anomaly detectors, cross-facility propagation via
federated transfer learning,
adversarial agronomic schedules that exploit crop biology rather than computational models, and reward poisoning of reinforcement-learning energy optimizers. Physical impact analysis quantifies crop loss timelines from minutes (aeroponics) to days, including worker safety hazards from CO2 injection manipulation. A survey of 10 commercial CEA vendors reveals only one CVE ever issued, zero bug bounty programs, and zero IEC 62443 certifications. We propose a defense-in-depth countermeasure framework and recommend Security Level 2 as a minimum baseline.
Read the full abstract ▾
Systems and Control
Structure- and Stability-Preserving Learning of Port-Hamiltonian Systems
Binh Nguyen, Nam T. Nguyen and Truong X. Nghiem
This paper investigates the problem of data-driven modeling of port-
Hamiltonian systems while preserving their intrinsic
Hamiltonian structure and stability properties. We propose a novel neural-network-based port-
Hamiltonian modeling technique that relaxes the convexity constraint commonly imposed by neural network-based
Hamiltonian approximations, thereby improving the expressiveness and generalization capability of the model. By removing this restriction, the proposed approach enables the use of more general non-convex
Hamiltonian representations to enhance modeling flexibility and accuracy. Furthermore, the proposed method incorporates information about stable
equilibria into the learning process, allowing the learned model to preserve the stability of multiple isolated
equilibria rather than being restricted to a single
equilibrium as in conventional methods. Two numerical experiments are conducted to validate the effectiveness of the proposed approach and demonstrate its ability to achieve more accurate structure- and stability-preserving learning of port-
Hamiltonian systems compared with a baseline method.
Read the full abstract ▾
Image and Video Processing · Cross-listed from cs.CV
DroneScan-YOLO: Redundancy-Aware Lightweight Detection for Tiny Objects in UAV Imagery
Yann V. Bellec
Aerial object detection in
UAV imagery presents unique challenges due to the high prevalence of tiny objects, adverse environmental conditions, and strict computational constraints. Standard YOLO-based detectors fail to address these jointly: their minimum detection stride of 8 pixels renders sub-32px objects nearly undetectable, their CIoU loss produces zero gradients for non-overlapping tiny boxes, and their architectures contain significant filter redundancy. We propose DroneScan-YOLO, a holistic system contribution that addresses these limitations through four coordinated design choices: (1) increased input resolution of 1280x1280 to maximize spatial detail for tiny objects, (2) RPA-Block, a dynamic filter
pruning mechanism based on lazy cosine-similarity updates with a 10-epoch warm-up period, (3) MSFD, a lightweight P2 detection branch at stride 4 adding only 114,592 parameters (+1.1%), and (4) SAL-NWD, a hybrid loss combining Normalized Wasserstein Distance with size-adaptive CIoU weighting, integrated into YOLOv8's TaskAligned assignment pipeline. Evaluated on VisDrone2019-DET, DroneScan-YOLO achieves 55.3% mAP@50 and 35.6% mAP@50-95, outperforming the YOLOv8s baseline by +16.6 and +12.3 points respectively, improving recall from 0.374 to 0.518, and maintaining 96.7 FPS inference speed with only +4.1% parameters. Gains are most pronounced on tiny object classes: bicycle AP@50 improves from 0.114 to 0.328 (+187%), and awning-tricycle from 0.156 to 0.237 (+52%).
Read the full abstract ▾
Systems and Control · Cross-listed from math.OC
A Momentum-based Stochastic Algorithm for Linearly Constrained Nonconvex Optimization
Chenyang Qiu, Mihitha Maithripala and Zongli Lin
This paper studies a
stochastic algorithm for linearly constrained nonconvex optimization, where the objective function is smooth but only unbiased
stochastic gradients with bounded variance are available. We propose a momentum-based augmented Lagrangian method that employs a Polyak-type gradient
estimator and requires only one
stochastic gradient evaluation per iteration. Under the standard
stochastic oracle model and the smoothness condition of the expected objective, we establish a convergence guarantee in terms of the first-order KKT residual of the original constrained problem. In particular, the proposed method computes an $\epsilon$-stationary solution in expectation within $O(\epsilon^{-4})$
stochastic gradient evaluations. Numerical experiments further show that the proposed method achieves competitive iteration complexity and improved wall-clock efficiency compared with representative recursive-momentum baselines.
Read the full abstract ▾
Systems and Control
Dynamic Regret in Time-varying MDPs with Intermittent Information
Negin Musavi and Melkior Ornik
We study sequential decision-making in time-varying Markov decision processes (TVMDPs) under limited update rates, where the decision-maker observes the system and updates its model only intermittently. Such settings arise in applications with sensing, communication, or computational constraints that preclude continuous adaptation. Our goal is to understand how the performance of an
agent, which learns and plans using receding-horizon control under these information constraints, degrades as a function of the update rate. We propose a skip-update learning and planning framework that combines likelihood-based estimation of time-varying transition kernels with finite-horizon planning and executes policies between updates using stale information. We analyze its performance via dynamic regret relative to an oracle policy with full knowledge of the dynamics and continuous observations. Our main result establishes a dynamic regret bound that explicitly quantifies the impact of intermittent updates, decomposing regret into contributions from update times and skip intervals and revealing its dependence on temporal variation, estimation uncertainty, and the duration of intervals without updates. In particular, the dominant contribution from skip intervals admits a linear dependence on the interval length and the rate of temporal variation, while its effect is mitigated by mixing-induced contraction.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.RO
Capability-Aware Heterogeneous Control Barrier Functions for Decentralized Multi-Robot Safe Navigation
Joonkyung Kim, Yanze Zhang, Wenhao Luo et al. (+1 more)
Safe
navigation for multi-
robot systems requires enforcing safety without sacrificing task efficiency under decentralized decision-making. Existing decentralized methods often assume
robot homogeneity, making shared safety requirements non-uniformly interpreted across heterogeneous
agents with structurally different dynamics, which could lead to avoidance obligations not physically realizable for some
robots and thus cause safety violations or deadlock. In this paper, we propose Capability-Aware Heterogeneous Control Barrier Function (CA-HCBF), a decentralized framework for consistent safety enforcement and capability-aware coordination in heterogeneous
robot teams. We derive a canonical second-order control-affine representation that unifies holonomic and nonholonomic
robots under acceleration-level control via canonical transformation and backstepping, preserving forward invariance of the safe set while avoiding relative-degree mismatch across heterogeneous dynamics. We further introduce a support-function-based directional capability metric that quantifies each
robot's ability to follow its motion intent, deriving a pairwise responsibility allocation that distributes the safety burden proportionally to each
robot's motion capability. A feasibility-aware clipping mechanism further constrains the allocation to each
agent's physically achievable range, mitigating infeasible constraint assignments common in dense decentralized CBF settings. Simulations with up to 30 heterogeneous
robots and a physical multi-
robot demonstration show improved safety and task efficiency over baselines, validating real-world applicability across
robots with distinct kinematic constraints.
Read the full abstract ▾
All authors Joonkyung Kim, Yanze Zhang, Wenhao Luo, Yiwei Lyu
Image and Video Processing · Cross-listed from cs.CV
SemiFA: An Agentic Multi-Modal Framework for Autonomous Semiconductor Failure Analysis Report Generation
Shivam Chand Kaushik
Semiconductor failure analysis (FA) requires engineers to examine inspection images, correlate equipment telemetry, consult historical defect records, and write structured reports, a process that can consume several hours of expert time per case. We present SemiFA, an agentic multi-modal framework that autonomously generates structured FA reports from
semiconductor inspection images in under one minute. SemiFA decomposes FA into a four-
agent LangGraph pipeline: a DefectDescriber that classifies and narrates defect morphology using DINOv2 and LLaVA-1.6, a RootCauseAnalyzer that fuses SECS/GEM equipment telemetry with historically similar defects retrieved from a Qdrant vector database, a SeverityClassifier that assigns severity and estimates yield impact, and a RecipeAdvisor that proposes corrective process adjustments. A fifth node assembles a PDF report. We introduce SemiFA-930, a
dataset of 930 annotated
semiconductor defect images paired with structured FA narratives across nine defect classes, drawn from procedural synthesis, WM-811K, and MixedWM38. Our DINOv2-based classifier achieves 92.1% accuracy on 140 validation images (macro F1 = 0.917), and the full pipeline produces complete FA reports in 48 seconds on an NVIDIA A100-SXM4-40 GB GPU. A GPT-4o judge ablation across four modality conditions demonstrates that multi-modal fusion improves root cause
reasoning by +0.86 composite points (1-5 scale) over an image-only baseline, with equipment telemetry as the more load-bearing modality. To our knowledge, SemiFA is the first system to integrate SECS/GEM equipment telemetry into a
vision-
language model pipeline for autonomous FA report generation.
Read the full abstract ▾
Audio and Speech Processing
ProSDD: Learning Prosodic Representations for Speech Deepfake Detection against Expressive and Emotional Attacks
Aurosweta Mahapatra, Ismail Rasim Ulgen, Kong Aik Lee et al. (+2 more)
Speech deepfake detection (SDD) systems perform well on standard
benchmarks datasets but often fail to generalize to expressive and emotional spoofing attacks. Many methods rely on spoof-heavy training data, learning
dataset-specific artifacts rather than transferable cues of natural
speech. In contrast, humans internalize variability in real
speech and detect fakes as deviations from it. We introduce ProSDD, a two-stage framework that enriches model
embeddings through supervised masked prediction of speaker-conditioned prosodic variation based on pitch, voice activity, and energy. Stage I learns prosodic variability from real
speech, and Stage II jointly optimizes this objective with spoof classification. ProSDD consistently outperforms baselines under both ASVspoof 2019 and 2024 training, reducing ASVspoof 2024 EER from 25.43% to 16.14% (2019-trained) and from 39.62% to 7.38% (2024-trained), while achieving 50% relative reductions on EmoFake and EmoSpoof-TTS.
Read the full abstract ▾
All authors Aurosweta Mahapatra, Ismail Rasim Ulgen, Kong Aik Lee, Nicholas Andrews, Berrak Sisman
Systems and Control
A Control Co-Design Framework to Achieve Solution Feasibility in Energy System Optimization Problems
Tania Rifat Jahan and Donald J. Docimo
This work explores methods to identify energy system designs for infeasible control co-design optimization problems. Control co-design, or CCD, has been recognized as a powerful tool to maximize energy system capabilities through simultaneous determination of plant and controller parameters. However, due to the inherent nonlinearities, complexity, and conflicting criteria of energy systems, CCD optimization problems are susceptible to infeasibility and can lack potential solutions. While transforming the optimization problem by relaxing constraints has been developed for optimal control infeasibility challenges, solution feasibility for CCD is relatively unexplored. This paper proposes a framework to convert infeasible optimization problems into solvable forms for a class of CCD problems. The framework introduces a procedure to rank metric bounds from least likely to most likely to cause infeasibility. This provides guidance to algorithmically relax a limited number of constraints, leaving others intact. The proposed framework is applied to a CCD problem for designing a battery within a microgrid. Comparison against a baseline approach for relaxing optimization problems shows the framework requires only a reduced number of iterations to determine a solution.
Read the full abstract ▾
Systems and Control
Synthesis and Deployment of Maximal Robust Control Barrier Functions through Adversarial Reinforcement Learning
Donggeon David Oh, Duy P. Nguyen, Haimin Hu et al. (+1 more)
Robust control barrier functions (CBFs) provide a principled mechanism for smooth safety enforcement under worst-case disturbances. However, existing approaches typically rely on explicit, closed-form structure in the dynamics (e.g., control-affine) and uncertainty models. This has led to limited scalability and generality, with most robust CBFs certifying only conservative subsets of the maximal robust safe set. In this paper, we introduce a new robust CBF framework for general nonlinear systems under bounded uncertainty. We first show that the safety value function solving the dynamic programming Isaacs equation is a valid robust discrete-time CBF that enforces safety on the maximal robust safe set. We then adopt the key reinforcement learning (RL) notion of quality function (or Q-function), which removes the need for explicit dynamics by lifting the barrier certificate into state-action space and yields a novel robust Q-CBF constraint for safety filtering. Combined with
adversarial RL, this enables the synthesis and deployment of robust Q-CBFs on general nonlinear systems with black-box dynamics and unknown uncertainty structure. We validate the framework on a canonical inverted pendulum
benchmark and a 36-D quadruped simulator, achieving substantially less conservative safe sets than barrier-based baselines on the pendulum and reliable safety enforcement even under
adversarial uncertainty realizations on the quadruped.
Read the full abstract ▾
All authors Donggeon David Oh, Duy P. Nguyen, Haimin Hu, Jaime Fernández Fisac