Audio and Speech Processing · Cross-listed from cs.SD
Generalizable Prompt Tuning for Audio-Language Models via Semantic Expansion
Jaehyuk Jang, Wonjun Lee, Kangwook Ko et al. (+1 more)
Prompt tuning has achieved remarkable progress in
vision-
language models (VLMs) and is recently being adopted for audio-
language models (ALMs). However, its generalization ability in ALMs remains largely underexplored. We observe that conventional
prompt tuning for ALMs also suffers from the Base-New Tradeoff, and we identify that this issue stems from the disrupted semantic structure of the
embedding space. To address this issue, we propose Semantically Expanded
Prompt Tuning (SEPT)-a plug-and-play framework that explicitly regularizes the
prompt embedding space by incorporating semantic neighbors generated by large
language models. SEPT introduces a novel semantic expansion loss with margin constraints that promote intra-class compactness and inter-class separability, thereby enhancing the semantic structure of the
prompt embedding space. For comprehensive evaluation, we establish the first
benchmark setup for
prompt generalization in ALMs, covering both base-to-new generalization and cross-
dataset transferability. Extensive experiments demonstrate that SEPT consistently improves generalization performance across multiple
prompt tuning baselines, while maintaining computational cost during inference.
Read the full abstract ▾
All authors Jaehyuk Jang, Wonjun Lee, Kangwook Ko, Changick Kim
Signal Processing
Millimeter-Wave Gesture Recognition in ISAC: Does Reducing Sensing Airtime Hamper Accuracy?
Jakob Struye, Nabeel Nisar Bhat, Siddhartha Kumar et al. (+2 more)
Most Integrated Sensing and Communications (
ISAC) systems require dividing airtime across their two modes. However, the specific impact of this decision on sensing performance remains unclear and underexplored. In this paper, we therefore investigate the impact on a gesture recognition system using a Millimeter-Wave (mmWave)
ISAC system. With our
dataset of power per beam pair gathered with two mmWave devices performing constant beam sweeps while test subjects performed distinct gestures, we train a gesture classifier using Convolutional Neural Networks. We then subsample these measurements, emulating reduced sensing airtime, showing that a sensing airtime of 25 % only reduces classification accuracy by 0.15 percentage points from full-time sensing. Alongside this high-quality sensing at low airtime, mmWave systems are known to provide extremely high data throughputs, making mmWave
ISAC a
prime enabler for applications such as truly wireless Extended Reality.
Read the full abstract ▾
All authors Jakob Struye, Nabeel Nisar Bhat, Siddhartha Kumar, Mohammad Hossein Moghaddam, Jeroen Famaey
Systems and Control
Developing a Quantitative Resiliency Approach
Vincent P. Paglioni, Graeme Troxell, Aaron Brown et al. (+2 more)
Resiliency has garnered
attention in the management of critical infrastructure as a metric of system performance, but there are significant roadblocks to its implementation in a realistic decision-making framework. Contrasted to risk and reliability, which have robust quantification approaches and undergird many regulatory approaches to system safety (e.g., "risk-informed decision-making"), resiliency is a diffuse, qualitatively-understood characteristic, often treated differently or distinctly. However, in the emerging context of highly-complex, highly-interdependent critical systems, the idea of reliability (as the probability of non-failure) may not be an appropriate metric of system health. As a result, focus is shifting towards resiliency-centered approaches that value the response to failure as much as the avoidance of failure. Supporting this approach requires a robustly-defined, quantitative understanding of resiliency. In this paper, we explore the foundations of reliability and resiliency engineering, and propose an approach to resiliency-informed decision-making bolstered by a quantitative understanding of resiliency.
Read the full abstract ▾
All authors Vincent P. Paglioni, Graeme Troxell, Aaron Brown, Steve Conrad, Mazdak Arabi
Signal Processing
Energy Harvesting in High Altitude Platform Station Enabled Sensor Networks
Melek Tuylu and Eylem Erdogan
High altitude platform station (HAPS) systems are becoming crucial facilitators for future wireless communication networks, enhancing connectivity across all vertical communication layers, including small Internet of Things (IoT) sensors and devices, terrestrial users, and aerial devices. In the context of the widely recognized vertical heterogeneous network (VHetNet) architecture, HAPS systems can provide service to both aerial and ground users. However, integrating HAPS systems as a core element in the VHetNet architecture presents a considerable energy challenge, marking a prominent constraint for their operation. Driven by this challenge, we introduce an energy harvesting (EH) strategy tailored for HAPS systems, enabling a HAPS system to gather energy from another HAPS system, which is not constrained by energy limitations. To assess the performance capabilities of the proposed model, we derive outage probability (OP), ergodic capacity (EC) and verify them by using Monte Carlo (MC) simulations. Moreover, we explore the system in terms of throughput. The findings reveal that harnessing full potential of EH stands as a viable approach to meet the energy demands of HAPS systems.
Read the full abstract ▾
Systems and Control
Policy Synthesis for Interval MDPs via Polyhedral Lyapunov Functions
Negar Monir and Sadegh Soudjani
Decision-making under uncertainty is central to many safety-critical applications, where decisions must be guided by probabilistic modeling formalisms. This paper introduces a novel approach to policy synthesis in multi-objective interval Markov decision processes using polyhedral Lyapunov functions. Unlike previous Lyapunov-based methods that mainly rely on quadratic functions, our method utilizes polyhedral functions to enhance accuracy in managing uncertainties within value iteration of dynamic programming. We reformulate the value iteration
algorithm as a switched affine system with interval uncertainties and apply control-theoretic stability principles to synthesize policies that guide the system toward a desired target set. By constructing an invariant set of attraction, we ensure that the synthesized policies provide convergence guarantees while minimizing the impact of transition uncertainty in the underlying model. Our methodology removes the need for computationally intensive Pareto curve computations by directly determining a policy that brings objectives within a specified range of their target values. We validate our approach through numerical case studies, including a recycling
robot and an electric vehicle battery, demonstrating its effectiveness in achieving policy synthesis under uncertainty.
Read the full abstract ▾
Audio and Speech Processing
Discriminating real and synthetic super-resolved audio samples using embedding-based classifiers
Mikhail Silaev, Konstantinos Drossos and Tuomas Virtanen
Generative
adversarial networks (GANs) and
diffusion models have recently achieved state-of-the-art performance in audio super-resolution (ADSR), producing perceptually convincing wideband audio from narrowband inputs. However, existing evaluations primarily rely on signal-level or perceptual metrics, leaving open the question of how closely the distributions of synthetic super-resolved and real wideband audio match. Here we address this problem by analyzing the separability of real and super-resolved audio in various
embedding spaces. We consider both middle-band ($4\to 16$~kHz) and full-band ($16\to 48$~kHz) upsampling tasks for
speech and music, training linear classifiers to distinguish real from synthetic samples based on multiple types of audio
embeddings. Comparisons with objective metrics and subjective listening tests reveal that
embedding-based classifiers achieve near-perfect separation, even when the generated audio attains high perceptual quality and state-of-the-art metric scores. This behavior is consistent across
datasets and models, including recent
diffusion-based approaches, highlighting a persistent gap between perceptual quality and true distributional fidelity in ADSR models.
Read the full abstract ▾
Systems and Control
Local Updates in Distributed Optimization: Provable Acceleration and Topology Effects
Zuang Wang and Yongqiang Wang
Inspired by the success of performing multiple local optimization steps between communication rounds in
federated learning, incorporating such local updates into distributed optimization has recently attracted growing interest. However, unlike
federated learning, where local updates can accelerate training by reducing gradient estimation error under minibatch settings, it remains unclear whether similar benefits persist when exact gradients are available. Moreover, existing theoretical results typically require reducing the step size when multiple local updates are employed, which can entirely offset any potential benefit of these additional local updates. In this paper, we focus on the classic DIGing
algorithm and leverage the tight performance bounds provided by Performance Estimation Problems (PEP) to show that incorporating local updates can indeed accelerate distributed optimization. To the best of our knowledge, this is the first rigorous demonstration of such acceleration for a broad class of objective functions. Our analysis further reveals that, under an appropriate step size, performing only two local updates is sufficient to achieve the maximal possible improvement, and that additional local updates provide no further gains. Because more updates increase computational cost, these findings offer practical guidance for efficient implementation. We also show that these speed gains depend critically on the network structure, with sparser or less connected graphs, characterized by the spectral properties of the mixing
matrix, yielding smaller improvements. Extensive experiments on both synthetic and real-world
datasets corroborate the theoretical findings.
Read the full abstract ▾
Signal Processing
Dual-Transformer Aided Hierarchical Deep Reinforcement Learning for Robust RIS-Assisted Near-Field Communications
Mohammad Ghassemi, Han Zhang, Ali Afana et al. (+2 more)
The deployment of extremely large aperture arrays (ELAAs) in sixth-generation (6G) networks is expected to shift communications into the near-field regime, where spherical-wave propagation enables distance-aware beamfocusing but remains highly vulnerable to physical blockages that cause non-line-of-sight (NLoS) conditions. To resolve this inherent vulnerability, reconfigurable intelligent surfaces (RIS) can be utilized to circumvent these blockages and effectively establish reliable NLoS communication links. In envisioned deployment scenarios, accurately acquiring instantaneous CSI and predicting sudden blockages is profoundly challenging due to the prohibitive pilot overhead associated with massive passive arrays and the unpredictable mobility of environmental scatterers. To address this, we propose the Dual-
Transformer Hierarchical Deep Reinforcement Learning (DT-HDRL) framework, which integrates two specialized
transformer models with a two-timescale hierarchical control
agent. The first
transformer integrates a ray-tracing digital twin prior with distance-aware
geometric correction features to yield rapid and precise CSI estimates, while a complementary
vision transformer (ViT) processes sequential camera frames to forecast impending blockages prior to link degradation. These predictive outputs are then fed directly into the hierarchical control
agent. Within this architecture, a high-level controller processes the slow-timescale blockage predictions to jointly dictate the user transmission path (line-of-sight (LoS) or RIS-assisted NLoS) and RIS active/sleep scheduling, whereas a low-level controller employs the fast-timescale CSI estimates to perform joint base station (BS) beamfocusing and RIS phase-shift optimization.
Read the full abstract ▾
All authors Mohammad Ghassemi, Han Zhang, Ali Afana, Akram Bin Sediq, Melike Erol-Kantarci
Systems and Control · Cross-listed from cs.LG
Sensor to Pixels: Decentralized Swarm Gathering via Image-Based Reinforcement Learning
Yigal Koifman, Eran Iceland, Erez Koifman et al. (+2 more)
This study highlights the potential of image-based reinforcement learning methods for addressing swarm-related tasks. In multi-
agent reinforcement learning, effective policy learning depends on how
agents sense, interpret, and process inputs. Traditional approaches often rely on handcrafted feature extraction or raw vector-based representations, which limit the scalability and efficiency of learned policies concerning input order and size. In this work we propose an image-based reinforcement learning method for decentralized control of a multi-
agent system, where observations are encoded as structured visual inputs that can be processed by Neural Networks, extracting its spatial features and producing novel decentralized motion control rules. We evaluate our approach on a multi-
agent convergence task of
agents with limited-range and bearing-only sensing that aim to keep the swarm cohesive during the aggregation. The
algorithm's performance is evaluated against two
benchmarks: an analytical solution proposed by Bellaiche and Bruckstein, which ensures convergence but progresses slowly, and VariAntNet, a neural network-based framework that converges much faster but shows medium success rates in hard constellations. Our method achieves high convergence, with a pace nearly matching that of VariAntNet. In some scenarios, it serves as the only practical alternative.
Read the full abstract ▾
All authors Yigal Koifman, Eran Iceland, Erez Koifman, Ariel Barel, Alfred M. Bruckstein
Image and Video Processing · Cross-listed from cs.LG
Inferring Clinically Relevant Molecular Subtypes of Pancreatic Cancer from Routine Histopathology Using Deep Learning
Abdul Rehman Akbar, Alejandro Levya, Ashwini Esnakula et al. (+12 more)
Molecular subtyping of PDAC into basal-like and classical has established prognostic and predictive value. However, its use in clinical practice is limited by cost, turnaround time, and tissue requirements, thereby restricting its application in the management of PDAC. We introduce PanSubNet, an interpretable deep learning framework that predicts therapy-relevant molecular subtypes directly from standard H&E-stained WSIs. PanSubNet was developed using data from 1,055 patients across two multi-institutional cohorts (PANCAN, n=846; TCGA, n=209) with paired histology and RNA-seq data. Ground-truth labels were derived using the validated Moffitt 50-gene signature refined by GATA6 expression. The model employs dual-scale architecture that fuses cellular-level morphology with tissue-level architecture, leveraging
attention mechanisms for multi-scale representation learning and transparent feature attribution. On internal validation within PANCAN using five-fold cross-validation, PanSubNet achieved mean AUC of 88.5% with balanced sensitivity and specificity. External validation on the independent TCGA cohort without fine-tuning demonstrated robust generalizability (AUC 84.0%). PanSubNet preserved and, in metastatic disease, strengthened prognostic stratification compared to RNA-seq based labels. Prediction uncertainty linked to intermediate transcriptional states, not classification noise. Model predictions are aligned with established transcriptomic programs, differentiation markers, and DNA damage repair signatures. By enabling rapid, cost-effective molecular stratification from routine H&E-stained slides, PanSubNet offers a clinically deployable and interpretable tool for genetic subtyping. We are gathering data from two institutions to validate and assess real-world performance, supporting integration into digital pathology workflows and advancing precision oncology for PDAC.
Read the full abstract ▾
All authors Abdul Rehman Akbar, Alejandro Levya, Ashwini Esnakula, Elshad Hasanov, Anne Noonan, Lingbin Meng, Susan Tsai, Vaibhav Sahai, Midhun Malla, Sarbajit Mukherjee, Upender Manne, Anil Parwani, Wei Chen, Ashish Manne, Muhammad Khalid Khan Niazi
Image and Video Processing
Edit2Restore:Few-Shot Image Restoration via Parameter-Efficient Adaptation of Pre-trained Editing Models
Mustafa Ak{\i}n Y{\i}lmaz, Ahmet Bilican, Burak Can Biner et al. (+1 more)
Image restoration has traditionally required training specialized models on thousands of paired examples per degradation type. Large pre-trained text-conditioned image editing models encode rich priors about image structure, quality, and degradation, yet we find that this knowledge does not, on its own, make them restorers: state-of-the-art editing models largely fail at restoration in the zero-shot regime. We show that what these priors lack is not capability but direction, and that a small amount of parameter-efficient adaptation supplies it. Fine-tuning LoRA adapters on FLUX.1 Kontext, a 12B-parameter flow matching model for image-to-image translation, with only 32–128 paired images per task and guided by simple text
prompts, we turn a mediocre zero-shot editor into a competitive restorer. A single unified adapter, conditioned on task-specific
prompts, handles five diverse degradations. Despite using three to four orders of magnitude less data, our few-shot model surpasses a recent restoration baseline trained on over a million curated pairs on the majority of perceptual and distribution-level metrics, on which we evaluate in keeping with our focus on perceptual rather than pixel-fidelity quality. Through comprehensive studies, we analyze the impact of training-set size, the trade-off between task-specific and unified multi-task adapters, the effect of text encoder adaptation, and zero-shot baseline performance, establishing pre-trained editing models as a compelling, data-efficient foundation for few-shot,
prompt-guided image restoration.
Read the full abstract ▾
All authors Mustafa Ak{\i}n Y{\i}lmaz, Ahmet Bilican, Burak Can Biner, Ahmet Murat Tekalp
Signal Processing
SEP Analysis of a Low-Resolution SIMO System with M-PSK over Fading Channels
Amila Ravinath, Minhua Ding, Bikshapathi Gouda et al. (+2 more)
In this paper, the average symbol error probability (SEP) of a phase-quantized single-input multiple-output (SIMO) system with M-ary phase-shift keying (PSK) modulation is analyzed under Rayleigh fading and additive white Gaussian noise. By leveraging a novel method, we derive exact SEP expressions for a quadrature PSK (QPSK)-modulated n-bit phase-quantized SIMO system with maximum ratio combining (SIMO-MRC), along with the corresponding high signal-to-noise ratio (SNR) characterizations in terms of diversity and coding gains. For a QPSK-modulated 2-bit phase-quantized SIMO system with selection combining, the diversity and coding gains are further obtained for an arbitrary number of receive
antennas, complementing existing results. Interestingly, the proposed method also reveals a duality between a SIMO-MRC system and a phase-quantized multiple-input single-output (MISO) system with maximum ratio transmission, when the modulation order, phase-quantization resolution,
antenna configuration, and the channel state information (CSI) conditions are reciprocal. This duality enables direct inference to obtain the diversity of a general M-PSK-modulated n-bit phase-quantized SIMO-MRC system, and extends the results to its MISO counterpart. All the above results have been obtained assuming perfect CSI at the receiver (CSIR). Finally, the SEP analysis of a QPSK-modulated 2-bit phase-quantized SIMO system is extended to the limited CSIR case, where the CSI at each receive
antenna is represented by only 2 bits of channel phase information. In this scenario, the diversity gain is shown to be further halved in general.
Read the full abstract ▾
All authors Amila Ravinath, Minhua Ding, Bikshapathi Gouda, Italo Atzeni, Antti Tölli
Systems and Control
Modeling and Control for UAV with Off-center Slung Load
Zongyang Lv, Yanmei Jia, Yongqing Liu et al. (+3 more)
Unmanned aerial vehicle (
UAV) with slung load system is a classic air transportation system. In practical applications, the suspension point of the slung load does not always align with the center of mass (CoM) of the
UAV due to mission requirements or mechanical interference. This offset creates coupling in the system's nonlinear dynamics which leads to a complicated motion control problem. In existing research, modeling of the system are performed about the
UAV's CoM. In this work we use the point of suspension instead. Based on the new model, a cascade control strategy is developed. In the middle-loop controller, the acceleration of the suspension point is used to regulate the swing angle of the slung load without the need for considering the coupling between the slung load and the
UAV. An inner-loop controller is designed to track the
UAV's attitude without the need of simplification on the coupling effects. We prove local exponential stability of the closed-loop using Lyapunov approach. Finally, simulations and experiments are conducted to validate the proposed control system.
Read the full abstract ▾
All authors Zongyang Lv, Yanmei Jia, Yongqing Liu, Alan F. Lynch, Qing Zhao, Yuhu Wu
Systems and Control · Cross-listed from cs.RO
Revisiting Continuous-Time Trajectory Estimation via Gaussian Processes and the Magnus Expansion
Timothy Barfoot, Cedric Le Gentil and Sven Lilge
Continuous-time state estimation has been shown to be an effective means of (i) handling asynchronous and high-rate measurements, (ii) introducing smoothness to the estimate, (iii) post hoc querying the estimate at times other than those of the measurements, and (iv) addressing certain observability issues related to scanning-while-moving sensors. A popular means of representing the trajectory in continuous time is via a Gaussian process (GP) prior, with the prior's mean and covariance functions generated by a linear time-varying (LTV)
stochastic differential equation (SDE) driven by white noise. When the state comprises elements of Lie groups, previous works have resorted to a patchwork of local GPs each with a linear time-invariant SDE kernel, which while effective in practice, lacks theoretical elegance. Here we revisit the full LTV GP approach to continuous-time trajectory estimation, deriving a global GP prior on Lie groups via the Magnus expansion, which offers a more elegant and general solution. We provide a numerical comparison between the two approaches and discuss their relative merits.
Read the full abstract ▾
Systems and Control · Cross-listed from math.DS
Nonlinear Spectral Modeling and Control of Soft-Robotic Muscles from Data
Leonardo Bettini, Amirhossein Kazemipour, Robert K. Katzschmann et al. (+1 more)
Artificial muscles are essential for compliant musculoskeletal robotics but complicate control due to nonlinear multiphysics dynamics. Hydraulically amplified electrostatic (HASEL) actuators, a class of soft artificial muscles, offer high performance but exhibit
memory effects and hysteresis. Here we present a data-driven reduction and control strategy grounded in spectral submanifold (SSM) theory. In the adiabatic regime, where inputs vary slowly relative to intrinsic transients, trajectories rapidly converge to a low-dimensional slow
manifold. We learn an explicit input-to-output map on this
manifold from forced-response trajectories alone, avoiding decay experiments that can trigger hysteresis. We deploy the SSM-based model for real-time control of an antagonistic HASEL-clutch joint. This approach yields a substantial reduction in tracking error compared to feedback-only and feedforward-only baselines under identical settings. This record-and-control workflow enables rapid characterization and high-performance control of soft muscles and muscle-driven joints without detailed physics-based modeling.
Read the full abstract ▾
All authors Leonardo Bettini, Amirhossein Kazemipour, Robert K. Katzschmann, George Haller