Image and Video Processing · Cross-listed from cs.CV
FlatLands: Generative Floormap Completion From a Single Egocentric View
Subhransu S. Bhattacharjee, Dylan Campbell and Rahul Shome
A single
egocentric image typically captures only a small portion of the floor, yet a complete metric traversability map of the surroundings would better serve applications such as indoor
navigation. We introduce FlatLands, a
dataset and
benchmark for single-view bird's-eye view (BEV) floor completion. The
dataset contains 270,575 observations from 17,656 real metric indoor scenes drawn from six existing
datasets, with aligned observation, visibility, validity, and ground-truth BEV maps, and the
benchmark includes both in- and out-of-distribution evaluation protocols. We compare training-free approaches, deterministic models, ensembles, and
stochastic generative models. Finally, we instantiate the task as an end-to-end
monocular RGB-to-floormaps pipeline. FlatLands provides a rigorous testbed for uncertainty-aware indoor mapping and generative completion for
embodied navigation.
Read the full abstract ▾
Audio and Speech Processing
AILive Mixer: A Deep Learning based Zero Latency Automatic Music Mixer for Live Music Performances
Devansh Zurale, Iris Lorente, Michael Lester et al. (+1 more)
In this work, we present a deep learning-based automatic multitrack music mixing system catered towards live performances. In a live performance, channels are often corrupted with acoustic bleeds of co-located instruments. Moreover, audio-visual synchronization is of critical importance thus putting a tight constraint on the audio latency. In this work we primarily tackle these two challenges of handling bleeds in the input channels to produce the music mix with zero latency. Although there have been several developments in the field of automatic music mixing in recent times, most or all previous works focus on offline production for isolated instrument signals and to the best of our knowledge, this is the first end-to-end deep learning system developed for live music performances. Our proposed system currently predicts mono gains for a multitrack input, but its design along with the precedent set in past works, allows for easy adaptation to future work of predicting other relevant music mixing parameters.
Read the full abstract ▾
All authors Devansh Zurale, Iris Lorente, Michael Lester, Alex Mitchell
Systems and Control
The Geometry of Transmission Zeros in Distance-Based Formations
Solomon Goldgraber Casspi and Daniel Zelazo
This letter presents a
geometric input-output analysis of distance-based formation control, focusing on the phenomenon of steady-state signal blocking between actuator and sensor pairs. We characterize steady-state multivariable transmission zeros, where fully excited rigid-body and deformational modes destructively interfere at the measured output. By analyzing the DC gain transfer
matrix of the linearized closed-loop dynamics, we prove that for connected, flexible frameworks, structural transmission zeros are strictly non-generic; the configuration-dependent cross-coupling required to induce them occupies a proper algebraic set of measure zero. However, because extracting actionable sensor-placement rules from these complex algebraic varieties is analytically intractable, we restrict our focus to infinitesimally rigid formations. For these baselines, we prove that the absence of internal flexes forces the zero-transmission condition to collapse into an explicit affine hyperplane defined by the actuator and the global formation geometry, which we term the spatial locus of transmission zeros. Finally, we introduce the global transmission polygon–a convex polytope constructed from the intersection of these loci. This construct provides a direct
geometric synthesis rule for robust sensor allocation, guaranteeing full-rank steady-state transmission against arbitrary single-node excitations.
Read the full abstract ▾
Audio and Speech Processing
Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech
Jaesung Bae, Xiuwen Zheng, Minje Kim et al. (+2 more)
Dysarthric
speech quality assessment (DSQA) is critical for clinical diagnostics and inclusive
speech technologies. However, subjective evaluation is costly and difficult to scale, and the scarcity of labeled data limits robust objective modeling. To address this, we propose a three-stage framework that leverages unlabeled dysarthric
speech and large-scale typical
speech datasets to scale training. A teacher model first generates pseudo-labels for unlabeled samples, followed by weakly supervised pretraining using a label-aware contrastive learning strategy that exposes the model to diverse speakers and acoustic conditions. The pretrained model is then fine-tuned for the downstream DSQA task. Experiments on five unseen
datasets spanning multiple etiologies and
languages demonstrate the robustness of our approach. Our Whisper-based baseline significantly outperforms SOTA DSQA predictors such as SpICE, and the full framework achieves an average SRCC of 0.761 across unseen test
datasets.
Read the full abstract ▾
All authors Jaesung Bae, Xiuwen Zheng, Minje Kim, Chang D. Yoo, Mark Hasegawa-Johnson
Systems and Control
Demand Response Under Stochastic, Price-Dependent User Behavior
Guido Cavraro, Andrey Bernstein and Emiliano Dall'Anese
This paper focuses on price-based residential demand response implemented through dynamic adjustments of electricity prices during DR events. It extends existing DR models to a
stochastic framework in which customer response is represented by price-dependent random variables, leveraging models and tools from the theory of
stochastic optimization with decision-dependent distributions. The inherent epistemic uncertainty in the customers' responses renders open-loop, model-based DR strategies impractical. To address this challenge, the paper proposes to employ
stochastic, feedback-based pricing strategies to compensate for estimation errors and uncertainty in customer response. The paper then establishes theoretical results demonstrating the stability and near-optimality of the proposed approach and validates its effectiveness through numerical simulations.
Read the full abstract ▾
Image and Video Processing
Standardizing Medical Images at Scale for AI
Callen MacPhee, Yiming Zhou, Koichiro Kishima et al. (+1 more)
Deep learning has achieved remarkable success in medical image analysis, yet its performance remains highly sensitive to the heterogeneity of clinical data. Differences in imaging hardware, staining protocols, and acquisition conditions produce substantial domain shifts that degrade model generalization across institutions. Here we present a physics-based data preprocessing framework based on the PhyCV (Physics-Inspired Computer
Vision) family of
algorithms, which standardizes medical images through deterministic transformations derived from optical physics. The framework models images as spatially varying optical fields that undergo a virtual diffractive propagation followed by coherent phase detection. This process suppresses non-semantic variability such as color and illumination differences while preserving diagnostically relevant texture and structural features. When applied to histopathological images from the Camelyon17-WILDS
benchmark, PhyCV preprocessing improves out-of-distribution breast-cancer classification accuracy from 70.8% (Empirical Risk Minimization baseline) to 90.9%, matching or exceeding data-augmentation and domain-generalization approaches at negligible computational cost. Because the transform is physically interpretable, parameterizable, and differentiable, it can be deployed as a fixed preprocessing stage or integrated into end-to-end learning. These results establish PhyCV as a generalizable data refinery for medical imaging-one that harmonizes heterogeneous
datasets through first-principles physics, improving robustness, interpretability, and reproducibility in clinical AI systems.
Read the full abstract ▾
All authors Callen MacPhee, Yiming Zhou, Koichiro Kishima, Bahram Jalali
Systems and Control
Time-Transformation-Based Analysis of Systems with Periodic Delay via Perturbative Expansion
Jungbae Chun, Sengiyumva Kisole, Matthew M. Peet et al. (+1 more)
It is difficult to analyze the stability of systems with time-varying delays. One approach is to construct a time-transformation that converts the system into a form with a constant delay but with a time-varying scalar appearing in the system
matrices. The stability of this transformed system can then be analyzed using methods to bound the effect of the time-varying scalar. One issue is that this transformation is non-unique and requires the solution of an Abel equation. A specific time-transformation typically must be computed numerically. We address this issue by computing an explicit, although approximate, time-transformation for systems where the delay has a constant plus small periodic term. We use a perturbative expansion to construct our explicit solutions. We provide a simple numerical example to illustrate the approach. We also demonstrate the use of this time-transformation to analyze stability of the system with this class of periodic delays.
Read the full abstract ▾
All authors Jungbae Chun, Sengiyumva Kisole, Matthew M. Peet, Peter Seiler
Systems and Control · Cross-listed from math.OC
Parameterization of Seed Functions for Equivalent Representations of Time-Varying Delay Systems
Sengiyumva Kisole, Jungbae Chun, Peter Seiler et al. (+1 more)
Abel's classic transformation shows that any well-posed system with time-varying delay is equivalent to a parameter-varying system with fixed delay. The existence of such a parameter-varying constant delay representation then simplifies the problems of stability analysis and optimal control. Unfortunately, the method for construction of such transformations has been ad-hoc – requiring an iterative time-stepping approach to constructing the transformation beginning with a seed function subject to boundary-value constraints. Moreover, a poor choice of seed function often results in a constant delay representation with large time-variations in system parameters – obviating the benefits of such a representation. In this paper, we show how the set of all feasible seed functions can be parameterized using a basis for $L_2$. This parameterization is then used to search for seed functions for which the corresponding time-transformation results in smaller parameter variation. The parameterization of admissible seed functions is illustrated with numerical examples that contrast how well-chosen and poorly chosen seed functions affect the boundedness of a time transformation.
Read the full abstract ▾
All authors Sengiyumva Kisole, Jungbae Chun, Peter Seiler, Matthew M. Peet
Systems and Control · Cross-listed from math.OC
Fast Relax-and-Round Unit Commitment with Economic Horizons
Shaked Regev, Evan J. R. Brody, Charles Foltz et al. (+2 more)
The US energy system is under increasing pressure to serve expanding data centers loads and to accommodate a larger number of generating units with varying technologies and ownership structures. Therefore, developing new unit commitment (UC) methods remains a priority for reliable and affordable grid operations. We expand our novel computational method for UC to include ramping constraints and long-horizon planning, and provide a theoretical bound on its accuracy. We solve problems with thousands of generators at 5-minute
market intervals. We show that our method can solve UC problems with over 20,000 generators in approximately 10 seconds on commodity hardware and that an increased planning horizon leads to sizable op- erational cost savings. This is twice as fast as the leading commercial solver, Gurobi. The gap increases when adding more constraints. Our method can be implemented using existing continuous optimization solvers and adapted for different applications. We prove a bound on the error of these solvers and show that it vanishes (in relative terms) as the problem becomes larger. Our novel
algorithm could allow an operator to make horizon-aware economic decisions for large systems.
Read the full abstract ▾
All authors Shaked Regev, Evan J. R. Brody, Charles Foltz, Eve Tsybina, Slaven Peles
Systems and Control
Adaptive Tube MPC: Beyond a Common Quadratically Stabilizing Feedback Gain
Anchita Dey and Shubhendu Bhasin
This paper proposes an adaptive tube framework for model predictive control (MPC) of discrete-time linear time-invariant systems subject to parametric uncertainty and additive disturbances. In contrast to conventional tube-based MPC schemes that employ fixed tube geometry and constraint tightening designed for worst-case uncertainty, the proposed approach incorporates online parameter learning to progressively refine the parametric uncertainty set and update the parameter estimates. These updates are used to adapt the components of the MPC optimization problem, including the prediction model, feedback gain, terminal set, and tube cross-sections. As the uncertainty set contracts, the required amount of constraint tightening reduces and the tube shrinks accordingly, yielding less conservative control actions. Recursive feasibility, robust constraint satisfaction, and closed-loop stability are formally established. Furthermore, the framework does not require the existence of a common quadratically stabilizing linear feedback gain for the entire parametric uncertainty set, thereby relaxing a standard assumption in existing tube-based MPC formulations. Numerical examples illustrate the effectiveness of the proposed approach.
Read the full abstract ▾
Signal Processing
Experimental Demonstration of Snapshot Differential Positioning with LEO Satellites
Soham Desai and Dave Cade
Positioning using Global
Navigation Satellite Systems (GNSS) typically requires several seconds of continuous signal reception from satellites in Medium Earth Orbit (MEO). This requirement poses challenges for applications where receivers can only capture signals intermittently or operate under constrained power and visibility conditions. In such scenarios, maintaining continuous tracking or reliable line-of-sight to GNSS satellites may be difficult, and conventional GNSS frequencies may also be vulnerable to interference or jamming. Low Earth Orbit (LEO) satellite constellations provide an attractive alternative due to their lower orbital altitudes, which result in higher received signal strengths, as well as their operation across a wide range of spectrum including Mobile-Satellite Service (MSS) and terrestrial L and S bands. These characteristics make LEO signals promising for
navigation in challenging environments. This work presents a snapshot-based differential positioning framework that leverages signals from LEO satellites. In the proposed approach, a receiver collects signals for short durations (5-10 seconds) before entering a low-power state, enabling positioning with intermittent observations. Doppler measurements from multiple satellites are combined with a differential measurement model using a fixed reference receiver to mitigate common errors such as satellite clock bias and ephemeris uncertainty. Experimental results demonstrate that the proposed differential Doppler framework operates effectively within the constraints of snapshot-based reception. The method achieves a position error reduction of approximately 47% even when only three satellites are simultaneously visible to both the rover and the reference station.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.LG
Game-Theory-Assisted Reinforcement Learning for Border Defense: Early Termination based on Analytical Solutions
Goutam Das, Michael Dorothy, Kyle Volle et al. (+1 more)
Game theory provides the gold standard for analyzing
adversarial engagements, offering strong optimality guarantees. However, these guarantees often become brittle when assumptions such as perfect information are violated. Reinforcement learning (RL), by contrast, is adaptive but can be sample-inefficient in large, complex domains. This paper introduces a hybrid approach that leverages game-theoretic insights to improve RL training efficiency. We study a border defense game with limited perceptual range, where defender performance depends on both search and pursuit strategies, making classical differential game solutions inapplicable. Our method employs the Apollonius Circle (AC) to compute
equilibrium in the post-detection phase, enabling early termination of RL episodes without learning pursuit dynamics. This allows RL to concentrate on learning search strategies while guaranteeing optimal continuation after detection. Across single- and multi-defender settings, this early termination method yields 10-20% higher rewards, faster convergence, and more efficient search trajectories. Extensive experiments validate these findings and demonstrate the overall effectiveness of our approach.
Read the full abstract ▾
All authors Goutam Das, Michael Dorothy, Kyle Volle, Daigo Shishika
Systems and Control
Rethinking Frequency Control in Power Systems
Taulant Kerci, Angel Vaca, Andrew Groom et al. (+2 more)
Frequency control in power systems is implemented in a hierarchical structure traditionally known as primary frequency control (PFC), secondary frequency control (SFC) and tertiary control reserve (TCR) and, some jurisdictions, include time error control (TEC) as well. This hierarchical structure has been designed around a century ago based on timescales separation, that is, approximately an order of magnitude difference between each control structure. This paper argues, based on real-world observations as well as detailed dynamic simulations on a model of the All-Island power system (AIPS) of Ireland, that this frequency control structure is not necessary in current and future converter-dominated power grids. The paper proposes to redesign this structure by removing the SFC and TCR and rely on PFC and a real-time energy
market. The PFC is responsible for addressing fast power imbalances in timescales of tens of ms to few minutes (e.g., 100 ms to 5 minutes) while the real-time energy
market is responsible for addressing longer imbalances in timescales of minutes to hours (e.g., 5 minutes to 1 hour). TEC, on the other hand, is considered as optional.
Read the full abstract ▾
All authors Taulant Kerci, Angel Vaca, Andrew Groom, Julia Matevosyan, Federico Milano
Systems and Control
Two-Phase Cell Switching in 6G vHetNets: Sleeping-Cell Load Estimation and Renewable-Aware Switching Toward NES
Maryam Salamatmoghadasi, Metin Ozturk and Halim Yanikomeroglu
This paper proposes a two phase framework to improve the sustainability in vertical heterogeneous networks that integrate various types of base stations~(BSs), including terrestrial macro BSs~(MBSs), small BSs~(SBSs), and a high altitude platform station super MBS (HAPS SMBS). In Phase I, we address the critical and often overlooked challenge of estimating the traffic load of sleeping SBSs, a prerequisite for practical
cell switching, by introducing three methods with varying data dependencies: (i) a distance based
estimator (no historical data), (ii) a multi level clustering (MLC)
estimator (limited historical data), and (iii) a long short term
memory~(LSTM) based temporal predictor (full historical data). In Phase II, we incorporate the most accurate estimation results from Phase I into a renewable energy aware
cell switching strategy, explicitly modeling
solar powered SBSs in three operational scenarios that reflect realistic hybrid grid renewable deployments. This flexible design allows the framework to adapt switching strategies based on renewable availability and storage conditions, making it more practical and robust for real world networks. Using a real call detail record
dataset from Milan, simulation results show that the LSTM method achieves a mean absolute percentage error (MAPE) below 1% in Phase I, while in Phase II, the threshold based
solar integration scenario achieves up to 23% network energy saving (NES) relative to conventional
cell switching. Overall, the proposed framework bridges the gap between theoretical
cell switching models and practical, sustainable 6G
radio access network~(RAN) operation, enabling significant energy saving without compromising quality of service.
Read the full abstract ▾
Systems and Control · Cross-listed from math.OC
Reachability Analysis for Design Optimization
Steven Nguyen, Jorge Cortés and Boris Kramer
We present an approach to approximate reachable sets for linear systems with bounded L-infinity controls in finite time. Our first approach investigates the boundaries of these sets and reveals an exact characterization for single-input, planar systems with real, distinct
eigenvalues. The second approach leverages convergence of the Lp-norms to L-infinity and uses Lp-norm reachable sets as an approximation of the L-infinity-norm reachable sets. Our optimal control results yield insights that make computational approximations of the Lp-norm reachable sets more tractable, and yield exact characterizations for L-infinity with the previous assumptions on the system. As an example, we incorporate our reachability analysis into the design optimization of a highly-maneuverable aircraft. Introducing constraints based on reachability allow us to factor physical limitations to desired flight maneuvers into the design process.
Read the full abstract ▾
Signal Processing
Multi-Year Spectral Structure of 6G Candidate Bands at 2.7 GHz and 4.4 GHz
Amir Hossein Fahim Raouf and Ismail Guvenc
Mid-band spectrum between 2 and 8 GHz is a critical resource for sixth-generation (6G) systems as it uniquely balances favorable propagation characteristics with scalable bandwidth. Recent U.S. policy highlights candidate bands near 2.7, 4.4, and 7.1 GHz, all of which host substantial federal and non-federal incumbency, including high-power radiolocation and aeronautical telemetry systems. Although these segments are being considered for potential relocation of federal incumbents to enable commercial use, their long-term viability depends on the structural integrity of the spectrum. In such environments, the practical value of spectrum depends on the reliability and contiguity of available spectrum opportunities. This paper presents a measurement-driven feasibility analysis of two representative segments, 2.69-2.9 GHz and 4.4-4.94 GHz, using Software-Defined
Radio (SDR) measurements collected during Packapalooza campaigns from 2022 to 2025. Deployment-oriented metrics are introduced to quantify scan-window reliability (SWR), altitude-dependent usable spectrum availability ratio (USAR), largest contiguous clean bandwidth (LCCB), spectral fragmentation, and extreme interference excursions. The results reveal significant year-to-year structural variability. In the 2.69-2.9 GHz band, USAR remains near unity in 2022 and 2023, but drops to approximately 0.65 in 2024 and 0.8 in 2025, accompanied by fragmentation and limited contiguous bandwidth across altitudes. The 4.4-4.94 GHz band exhibits a similar temporal pattern, but with smaller reliability degradation and larger contiguous support, often exceeding several hundred megahertz even during incumbent-dominant periods. The results highlight that wideband feasibility in these candidate bands depends strongly on spectral contiguity and structural stability rather than nominal bandwidth alone.
Read the full abstract ▾
Signal Processing · Cross-listed from physics.geo-ph
High-Resolution Trans-Oceanic Distributed Acoustic Sensing Enabled by a Bi-Directional Sensor Implementation
Mikael Mazur, Nicolas K. Fontaine, Roland Ryf et al. (+14 more)
We demonstrate continuous distributed acoustic sensing over a 4400km long undersea cable. Bi-directional operation improves the strain signal-to-noise rate by >20dB, enabling 88000 50-m-spaced measurement points at a nominal telecom launch power.
Read the full abstract ▾
All authors Mikael Mazur, Nicolas K. Fontaine, Roland Ryf, Martin Karrenbach, Kristopher McBrian, Keith McLaughlin, Brian Sperry, Anuar Butler, Valey Kamalov, Lauren Dallachiesa, Ells Burrows, David Winter, Haoshuo Chen, Jeewan Naik, Kishore Padmaraju, Ajay Mistry, David Neilson
Systems and Control
Solar Daylighting to Offset LED Lighting in Vertical Farming: A Techno-Economic Study of Light Pipes
Francesco Ceccanti, Aldo Bischi, Marco Antonelli et al. (+1 more)
Vertical farming is a controlled-environment agriculture (CEA) approach in which crops are grown in stacked layers under regulated
climate and lighting, enabling predictable production but requiring high electricity input. This study quantifies the techno-economic impact of roof-mounted daylighting in a three-tier container vertical farm using a light-pipe (LP) system that delivers sunlight to the upper tier. The optical chain, comprising a straight duct and a tilting aluminum-coated mirror within a rotating dome, was modelled in Tonatiuh to estimate crop-level
photon delivery and
solar gains. These outputs were coupled with a transient AGRI-Energy model to perform year-round simulations for Dubai. Tier-3 strategies were compared against a fully LED
benchmark, including daylight-only operation, on/off supplementation, PWM dimming, UV-IR filtering, variable-transmittance control, and simple glazing. Ray-tracing predicted an overall LP optical efficiency of 45%-75%, depending on
solar position, quantifying the fraction of incident daylight at the collector aperture delivered to the target growing zone. Daylight-only operation reduced the total three-tier yield by 17% and was not economically viable despite 27-29% electricity savings. Hybrid daylight-LED strategies preserved
benchmark yield while reducing electricity use. PWM dimming combined with UV-IR filtering achieved the lowest specific electricity energy consumption (6.32 kWh/kg), 14% below the
benchmark. Overall, viability remains CAPEX-limited because achievable electricity savings are insufficient to offset the added investment and thus improves mainly under high electricity and carbon-price contexts, although the LP system delivers a 15-38% lower light cost than an optical-fiber reference under identical incident daylight.
Read the full abstract ▾
All authors Francesco Ceccanti, Aldo Bischi, Marco Antonelli, Andrea Baccioli
Image and Video Processing · Cross-listed from cs.AR
GLANCE: Gaze-Led Attention Network for Compressed Edge-inference
Neeraj Solanki, Hong Ding, Sepehr Tabrizchi et al. (+4 more)
Real-time object detection in AR/VR systems faces critical computational constraints, requiring sub-10\,ms latency within tight power budgets. Inspired by biological foveal
vision, we propose a two-stage pipeline that combines differentiable weightless neural networks for ultra-efficient gaze estimation with
attention-guided region-of-interest object detection. Our approach eliminates arithmetic-intensive operations by performing gaze tracking through
memory lookups rather than multiply-accumulate computations, achieving an angular error of $8.32^{\circ}$ with only 393 MACs and 2.2 KiB of
memory per frame. Gaze predictions guide selective object detection on attended regions, reducing computational burden by 40-50% and energy consumption by 65%. Deployed on the Arduino Nano 33 BLE, our system achieves 48.1% mAP on COCO (51.8% on attended objects) while maintaining sub-10\,ms latency, meeting stringent AR/VR requirements by improving the communication time by $\times 177$. Compared to the global YOLOv12n baseline, which achieves 39.2%, 63.4%, and 83.1% accuracy for small, MEDium, and LARGE objects, respectively, the ROI-based method yields 51.3%, 72.1%, and 88.1% under the same settings. This work shows that
memory-centric architectures with explicit
attention modeling offer better efficiency and accuracy for resource-constrained wearable platforms than uniform processing.
Read the full abstract ▾
All authors Neeraj Solanki, Hong Ding, Sepehr Tabrizchi, Ali Shafiee Sarvestani, Shaahin Angizi, David Z. Pan, Arman Roohi
Systems and Control · Cross-listed from math.OC
Saddle Point Evasion via Curvature-Regularized Gradient Dynamics
Liraz Mudrik, Isaac Kaminer, Sean Kragelund et al. (+1 more)
Nonconvex optimization underlies many modern machine learning and control tasks, where saddle points pose the dominant obstacle to reliable convergence in high-dimensional settings. Escaping these saddle points deterministically using continuous-time optimization remains an open challenge: gradient descent is blind to
curvature,
stochastic perturbation methods lack deterministic guarantees, and Newton-type approaches suffer from Hessian singularity. Adopting the perspective of viewing optimization
algorithms as dynamical systems, we present
Curvature-Regularized Gradient Dynamics (CRGD), which augments the objective with a smooth penalty on the negative Hessian
eigenvalues, yielding an augmented cost that serves as an optimization Lyapunov function with user-selectable convergence rates to second-order stationary points. Numerical experiments confirm that CRGD converges to second-order stationary points, even in regimes where gradient descent fails.
Read the full abstract ▾
All authors Liraz Mudrik, Isaac Kaminer, Sean Kragelund, Abram H. Clark
Audio and Speech Processing · Cross-listed from cs.SD
AC-Foley: Reference-Audio-Guided Video-to-Audio Synthesis with Acoustic Transfer
Pengjun Fang, Yingqing He, Yazhou Xing et al. (+3 more)
Existing
video-to-audio (V2A) generation methods predominantly rely on text
prompts alongside visual information to synthesize audio. However, two critical bottlenecks persist: semantic granularity gaps in training data, such as conflating acoustically distinct sounds under coarse labels, and textual ambiguity in describing micro-acoustic features. These bottlenecks make it difficult to perform fine-grained sound synthesis using text-controlled modes. To address these limitations, we propose AC-Foley, an audio-conditioned V2A model that directly leverages reference audio to achieve precise and fine-grained control over generated sounds. This approach enables fine-grained sound synthesis, timbre transfer, zero-shot sound generation, and improved audio quality. By directly conditioning on audio signals, our approach bypasses the semantic ambiguities of text descriptions while enabling precise manipulation of acoustic attributes. Empirically, AC-Foley achieves state-of-the-art performance for Foley generation when conditioned on reference audio, while remaining competitive with state-of-the-art
video-to-audio methods even without audio conditioning. Code and demo are available at: https://ff2416.github.io/AC-Foley-Page
Read the full abstract ▾
All authors Pengjun Fang, Yingqing He, Yazhou Xing, Qifeng Chen, Ser-Nam Lim, Harry Yang
Systems and Control
Switching-Reference Voltage Control for Distribution Systems with AI-Training Data Centers
Mingyuan Yan, Trager Joswig-Jones, Baosen Zhang et al. (+2 more)
Large-scale AI training workloads in data centers exhibit rapid and periodic power swings that can induce voltage deviations in power distribution systems. Existing voltage controllers treat these swings as a generic disturbance, leading to high control effort but still large voltage violations. However, such emerging loads are not random: they alternate between two distinct operating phases. This paper exploits this structure with a decentralized switching-reference voltage control framework. By switching each voltage reference in step with the workload phases, the controller cancels the phase-induced voltage shift, holding the voltage within limits with low control effort. Because real-time communication between buses is not always available, the controller is designed to infer the reference from local voltage measurements. This paper further proves convergence under deadband and saturation. In case studies on real AI training power traces, the switching reference suppresses voltage violations, sometimes eliminating them entirely, while reducing the control effort by approximately an order of magnitude compared with conventional droop control. Further experiments confirm that it remains effective with multiple data centers and internal load smoothing.
Read the full abstract ▾
All authors Mingyuan Yan, Trager Joswig-Jones, Baosen Zhang, Yize Chen, Wenqi Cui
Systems and Control · Cross-listed from cs.AI
Computational Concept of the Psyche
Anton Kolonin and Vladimir Krykov
This article presents an overview of approaches to modeling the human psyche in the context of constructing an artificial one. Based on this overview, a concept of cognitive architecture is proposed, in which the psyche is viewed as the operating system of a living or artificial subject, comprising a space of states, including the state of needs that determine the meaning of a subject's being in relation to stimuli from the external world, and intelligence as a decision-making system regarding actions in this world to satisfy these needs. Based on this concept, a computational formalization is proposed for creating artificial general intelligence systems for an
agent through experiential learning in a state space that includes
agent's needs, taking into account their biological or existential significance for the intelligent
agent, along with
agent's sensations and actions. Thus, the problem of constructing artificial general intelligence is formalized as a system for making optimal decisions in the space of specific
agent needs under conditions of uncertainty, maximizing success in achieving goals, minimizing existential risks, and maximizing energy efficiency. A minimal experimental implementation of the model is presented.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.AR
Co-Design of Memory-Storage Systems for Workload Awareness with Interpretable Models
Jay Sarkar, Vamsi Pavan Rayaprolu and Abhijeet Bhalerao
Solid-state storage architectures based on NAND or emerging
memory devices (SSD), are fundamentally architected and optimized for both reliability and performance. Achieving these simultaneous goals requires co-design of
memory components with firmware-architected Error Management (EM)
algorithms for density- and performance-scaled
memory technologies. We describe a Machine Learning (ML) for systems methodology and modeling for co-designing the EM subsystem together with the natural variance inherent to scaled silicon process of
memory components underlying SSD technology. The modeling analyzes NAND
memory components and EM
algorithms interacting with comprehensive suite of synthetic (stress-focused and JEDEC) and emulation (YCSB and similar) workloads across Flash Translation abstraction layers, by leveraging a statistically interpretable and intuitively explainable ML
algorithm. The generalizable co-design framework evaluates several thousand datacenter SSDs spanning multiple generations of
memory and storage technology. Consequently, the modeling framework enables continuous, holistic, data-driven design towards generational architectural advancements. We additionally demonstrate that the framework enables Representation Learning of the EM-workload domain for enhancement of the architectural design-space across broad spectrum of workloads.
Read the full abstract ▾
Systems and Control · Cross-listed from cs.SE
Lore: Repurposing Git Commit Messages as a Structured Knowledge Protocol for AI Coding Agents
Ivan Stetsenko
As AI coding
agents become both primary producers and consumers of source code, the software industry faces an accelerating loss of institutional knowledge. Each commit captures a code diff but discards the
reasoning behind it - the constraints, rejected alternatives, and forward-looking context that shaped the decision. I term this discarded
reasoning the Decision Shadow. This paper proposes Lore, a lightweight protocol that restructures commit messages - using native git trailers - into self-contained decision records carrying constraints, rejected alternatives,
agent directives, and verification metadata. Lore requires no infrastructure beyond git, is queryable via a standalone CLI tool, and is discoverable by any
agent capable of running shell commands. The paper formalizes the protocol, compares it against five competing approaches, stress-tests it against its strongest objections, and outlines an empirical validation path.
Read the full abstract ▾
Audio and Speech Processing
spINAch: A Diachronic Corpus of French Broadcast Speech Controlled for Speakers' Age and Gender
Simon Devauchelle, David Doukhan, Rémi Uro et al. (+7 more)
We present spINAch, a large diachronic corpus of French
speech from
radio and television archives, balanced by speakers' gender, age (20-95 years old), and spanning 60 years from 1955 to 2015. The
dataset includes over 320 hours of recordings from more than two thousand speakers. The methodology for building the corpus is described, focusing on the quality of collected samples in acoustic terms. The data were automatically transcribed and phonetically aligned to allow studies at a phonemic level. More than 3 million oral vowels have been analyzed to propose their fundamental frequency and formants. The corpus, available to the community for research purposes, is valuable for describing the
evolution of Parisian French through the representation of gender and age. The presented analyses also demonstrate that the diachronic nature of the corpus allows the observation of various phonetic phenomena, such as the
evolution of voice pitch over time (which does not differ by gender in our data) and the neutralization of the /a/-/$a$/ opposition in Parisian French during this period.
Read the full abstract ▾
All authors Simon Devauchelle, David Doukhan, Rémi Uro, Lucas Ondel Yang, Valentin Pelloin, Olympia Imbert-Brégégère, Véronique Lefort, Kévin Picard, Emeline Seignobos, Albert Rilliard
Image and Video Processing · Cross-listed from cs.CV
Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation
Yuanfan Zheng, Kunyu Peng, Xu Zheng et al. (+1 more)
Cross-domain panoramic semantic
segmentation has attracted growing interest as it enables comprehensive 360{\deg} scene understanding for real-world applications. However, it remains particularly challenging due to severe
geometric Field of View (FoV) distortions and inconsistent open-set semantics across domains. In this work, we formulate an open-set domain adaptation setting, and propose Extrapolative Domain Adaptive Panoramic
Segmentation (EDA-PSeg) framework that trains on local perspective views and tests on full 360{\deg} panoramic images, explicitly tackling both
geometric FoV shifts across domains and semantic uncertainty arising from previously unseen classes. To this end, we propose the Euler-Margin
Attention (EMA), which introduces an angular margin to enhance viewpoint-invariant semantic representation, while performing amplitude and phase modulation to improve generalization toward unseen classes. Additionally, we design the Graph Matching Adapter (GMA), which builds high-order graph relations to align shared semantics across FoV shifts while effectively separating novel categories through structural adaptation. Extensive experiments on four
benchmark datasets under camera-shift, weather-condition, and open-set scenarios demonstrate that EDA-PSeg achieves state-of-the-art performance, robust generalization to diverse viewing geometries, and resilience under varying environmental conditions. The code is available at https://github.com/zyfone/EDA-PSeg.
Read the full abstract ▾
All authors Yuanfan Zheng, Kunyu Peng, Xu Zheng, Kailun Yang
Signal Processing · Cross-listed from cs.RO
On the Derivation of Tightly-Coupled LiDAR-Inertial Odometry with VoxelMap
Zhihao Zhan
This note presents a concise mathematical formulation of tightly-coupled
LiDAR-Inertial Odometry within an iterated error-state Kalman filter framework using a VoxelMap representation. Rather than proposing a new
algorithm, it provides a clear and self-contained derivation that unifies the
geometric modeling and probabilistic state estimation through consistent notation and explicit formulations. The document is intended to serve both as a technical reference and as an accessible entry point for a foundational understanding of the system architecture and estimation principles.
Read the full abstract ▾
Signal Processing · Cross-listed from cs.IT
DMD Prediction of MIMO Channel Using Tucker Decomposition
Irina Kopnina, Dmitry Artemasov and Sergey Matveev
Accurate channel state information (CSI) prediction is crucial for next-generation multiple-input multiple-output (
MIMO) communication systems. Classical prediction methods often become inefficient for high-dimensional and rapidly time-varying channels. To improve prediction efficiency, it is essential to exploit the inherent low-rank tensor structure of the
MIMO channel. Motivated by this observation, we propose a dynamic mode decomposition (DMD)-based prediction framework operating on the low-dimensional core tensors obtained via a Tucker decomposition. The proposed method predicts reduced-order channel cores, significantly lowering computational complexity. Simulation results demonstrate that the proposed approach preserves the dominant channel dynamics and achieves high prediction accuracy.
Read the full abstract ▾
Audio and Speech Processing · Cross-listed from cs.SD
Music Genre Classification: A Comparative Analysis of Classical Machine Learning and Deep Learning Approaches
Sachin Prajuli, Abhishek Karna and OmPrakash Dhakl
Automatic music genre classification is a long-standing challenge in Music Information
Retrieval (MIR); work on non-Western music traditions remains scarce. Nepali music encompasses culturally rich and acoustically diverse genres–from the call-and-response duets of Lok Dohori to the rhythmic poetry of Deuda and the distinctive melodies of Tamang Selo–that have not been addressed by existing classification systems. In this paper, we construct a novel
dataset of approximately 8,000 labeled 30-second audio clips spanning eight Nepali music genres and conduct a systematic comparison of nine classification models across two paradigms. Five classical machine learning classifiers (Logistic Regression, SVM, KNN, Random Forest, and XGBoost) are trained on 51 hand-crafted audio features extracted via Librosa, while four deep learning architectures (CNN, RNN, parallel CNN-RNN, and sequential CNN followed by RNN) operate on Mel spectrograms of dimension 640 x 128. Our experiments reveal that the sequential Convolutional Recurrent Neural Network (CRNN)–in which convolutional layers feed into an LSTM–achieves the highest accuracy of 84%, substantially outperforming both the best classical models (Logistic Regression and XGBoost, both at 71%) and all other deep architectures. We provide per-class precision, recall, F1-score, confusion
matrices, and ROC analysis for every model, and offer a culturally grounded interpretation of misclassification patterns that reflects genuine overlaps in Nepal's musical traditions.
Read the full abstract ▾