Image and Video Processing · Cross-listed from cs.GR
How to Build Digital Humans? From Priors to Photorealistic Avatars
Wojciech Zielonka, Tobias Kirschstein, Timo Bolkart et al. (+8 more)
This state-of-the-art report provides an overview of controllable 3D human avatar creation. We describe current 3D avatar systems, which typically consist of three stages: (i) learning priors of human appearance and motion, (ii) creating a personalized avatar, and (iii) animating the avatar. To limit the scope, we focus on the prior learning and avatar creation stages. We define current avatar representations and introduce a taxonomy that categorizes existing work along multiple axes, including body regions and employed priors. We review methods for full-body and head avatars, as well as layered representations that decompose the body into components such as hands, hair, and garments. Finally, we outline common underlying principles, reference key literature for newcomers, and discuss open challenges and future research directions.
Read the full abstract ▾
All authors Wojciech Zielonka, Tobias Kirschstein, Timo Bolkart, Simon Giebenhain, Vanessa Sklyarova, Xiang Deng, Donglai Xiang, Shunsuke Saito, Yebin Liu, Matthias Niessner, Justus Thies
Audio and Speech Processing · Cross-listed from cs.SD
Doppelganger: Sound Effects and Their Synthetic Twins
Elliott Ash
Audio-conditioned generators now produce synthetic sound effects from real recordings, so the real and synthetic versions of an event increasingly coexist in sound libraries and in the corpora used to train audio models – yet no
benchmark measures whether a representation can match a synthetic clip to the specific real recording it was generated from. I introduce Doppelganger, a
benchmark for matching sound effects across the synthetic-real boundary, pairing 10,420 real clips across 34 everyday sound events each with an audio-conditioned synthetic twin, alongside a controlled 7-class corpus. Off-the-shelf audio encoders do not cross the boundary cleanly. Making the
embedding ignore the boundary the standard way – training it on sound-event labels – works on familiar sounds but backfires on new ones, dropping below the untrained encoder. Training on the pairs instead – a clip and its own synthetic twin – generalizes. On sound events held out of training, it recovers the exact real source about 80% of the time (up from 61% untrained; chance 0.03%), whereas no objective meaningfully improves category-level recognition on those unseen events. The learned matching is specific to one generator – it survives changes to that generator's settings but not a switch to a different generator, and collapses for the text-only generators tested. A human annotation baseline (49 listeners) lands well above chance but below the models on the same trials. Synthetic twins fool people into calling them real about 29% of the time, yet a generator-specific detector separates these audio-conditioned twins from real recordings perfectly.
Read the full abstract ▾
Audio and Speech Processing
When and why do handcrafted cues help self-supervised anti-spoofing? A causal and faithfulness analysis
Yugwon Won
Most spoofing countermeasures now place a light classifier on top of a self-supervised (SSL)
speech encoder. A growing line of work adds handcrafted acoustic features and fuses them by cross-
attention, partly because the
attention weights appear to explain which cues the model relies on. Two things about such fusion remain untested: whether the handcrafted features contribute anything once a strong SSL encoder is already in place, and whether the
attention map faithfully reflects what the classifier actually uses. We study both questions with MOSAIC, a single model that handles logical access (LA) and physical access (PA) attacks by projecting a 152-dimensional biophonetic vector into six query
tokens attending over thirteen intermediate layers of WavLM-Large. Rather than pursuing state-of-the-art accuracy, we treat the model itself as the object of analysis and apply inference-time interventions that zero out selected components. Removing the handcrafted branch lowers the EER by 0.67 percentage points under LA, so WavLM alone is sufficient there, but raises it by 0.76 points under PA, where the branch supplies replay channel evidence the encoder does not carry. The 6x13
attention map is highly stable under resampling (bootstrap rank correlation 0.99), yet its faithfulness varies by domain: layer
attention weights predict the causal importance of each layer under PA (rho = +0.62) but not under LA (rho = -0.28). The handcrafted branch and its
attention-based explanation are therefore trustworthy for replay attacks and not for synthetic ones. On standard
benchmarks the model is mid-range on LA and ahead of the official baselines on cross-source deepfakes (6.21% EER on ASVspoof 2021 DF). The contribution is not a new architecture but a checking procedure: verify handcrafted cues and
attention maps by intervention before trusting either.
Read the full abstract ▾
Systems and Control
Certified Breathing Stability Regions in Nonlinear Dynamical Systems: Composite Lyapunov Certificates, M-Matrix Conditions, and a Resilience-Fragility Correspondence
Marian Mester
We develop a unified, certified lower bound on the time-to-boundary margin M for transient stability of interconnected dissipative systems under slow parameter drift. The companion work establishes M as the first-passage time of the joint state-parameter motion to the synchronism boundary and proves M = CCT exactly on the one-machine-infinite-bus reduction, while leaving the multimachine certified margin open. Here a composite (mixed-region) Lyapunov function, formed by absorbing the restoring intra-group coupling into group energy functions and treating only the residual cross-cut coupling through the comparison principle, yields a positively invariant inner estimate of the region of attraction whenever an associated test
matrix is a nonsingular M-
matrix. The certified region breathes with the drift: its size is governed by a single critical synchronising stiffness k_c (lambda), and as k_c -> 0 at the boundary the region breathes shut and the certified margin M_low <= M_true vanishes. We give a nonlinear sector form of the construction, a domain-neutral resilience-fragility reading in which the coupling that certifies order is the one whose growth certifies collapse, and a constructive control corollary establishing a sharp dichotomy between damping injection and structural action. The mechanism is demonstrated identically on the WSCC nine-bus power system and on an inertial Kuramoto network, whose normalised breathing curves collapse, to leading order, onto a single profile. We present this collapse as numerical evidence for a conjectured universal form; a normal-form proof is identified as the precise open step.
Read the full abstract ▾
Signal Processing
Cognitive Digital Twins for Self-Aware Channel Estimation
Afan Ali, Ali Arshad Nasir and Daniel Benevides da Costa
Artificial intelligence (AI) and machine learning (ML)-based channel
estimators silently degrade when propagation conditions drift from their training distributions. This letter proposes a model-agnostic cognitive digital twin (CDT) framework that combines a variational autoencoder (VAE) with latent activation monitoring to detect distribution drift and autonomously execute continue, update, or retire lifecycle actions without requiring ground-truth channel knowledge. The proposed framework is fully compatible with the AI-native lifecycle management envisioned in 3rd Generation Partnership Project (3GPP). Simulations over various channels demonstrate accurate drift detection and robust channel estimation, consistently outperforming conventional offline-trained deep learning
estimators under moderate and severe channel drift.
Read the full abstract ▾
Signal Processing · Cross-listed from stat.ME
Beyond Tensor Probabilistic Independent Component Analysis – Putting Block-Term Decomposition and Independent Vector Analysis Together
Eleftherios Kofidis
Tensor probabilistic independent component analysis (TPICA) is a popular approach to analyzing functional
magnetic resonance imaging (fMRI) data, which draws its popularity from its ability to enrich the advantages of the statistics-based ICA with the awareness of the multi-way nature of these data, brought about and exploited via a deterministic 3-way (time $\times$ space $\times$ subjects) tensor decomposition (Canonical Polyadic Decomposition (CPD)) model. It has, however, received critique concerning its robustness in realistic fMRI unmixing scenarios, notably those involving sources that are strongly overlapped in space. Such cases may not meet the assumption of statistical independence required in ICA. They can instead be better described as independent vectors (or subspaces) of dependent components, pointing to the adoption of alternative statistical approaches, notably independent vector analysis (IVA). On the other hand, on the deterministic side, CPD is often restrictive and is outperformed by the more flexible block-term decomposition (BTD) model, also in the fMRI source unmixing context. Given the above, plus strong evidence of links between IVA and BTD, it is deemed worthwhile to consider the possibilities of generalizing TPICA to a BTD-based “TPIVA" extension, which would more successfully combine the power of statistics and tensor decomposition. This could also entail a generalization of the BTD model, where (non)collinearity would be replaced by statistical (in)dependence. This note aims to outline the state-of-the-art and the above ideas in more detail, serving as a preliminary, motivating step in this research direction.
Read the full abstract ▾
Systems and Control
Towards Effcient Low Altitude Sensing: A Dual Heterogeneous Graph Learning Method for UAV Task Allocation
Guangyu Lei, Tianhao Liang, Bingyan Xie et al. (+1 more)
With the development of low altitude intelligent systems, multiple unmanned aerial vehicles (
UAVs) can collaboratively execute more complex tasks. Conventional task allocation methods usually regard tasks and
UAVs as isolated entities, making it difficult to capture task dependencies and
UAV communication relationships. To address this issue, this paper proposes a dual heterogeneous graph learning based
UAV task allocation method. A directed task graph is constructed to represent task dependencies and encode task resource requirements, while an undirected
UAV communication graph is built to model communication relationships and encode
UAV resource states. The task allocation problem is formulated as a structural matching problem between the task graph and the
UAV communication graph. A graph
attention network based feature extraction method is introduced to learn structural representations from both graphs through message passing. A cross
attention mechanism is further integrated with proximal policy optimization to optimize the matching between task nodes and
UAV nodes for task allocation. Simulation results demonstrate that the proposed method achieves a higher task completion rate and shorter task completion time than
benchmark methods under different evaluation settings. Furthermore, a
UAV sensing and computing application is developed on the AirSim simulation platform. A large
language model is employed to convert natural
language task requirements into a structured task graph for autonomous
UAV task execution, demonstrating the potential of the proposed framework for natural
language driven
UAV mission planning and execution.
Read the full abstract ▾
All authors Guangyu Lei, Tianhao Liang, Bingyan Xie, Tingting Zhang
Systems and Control · Cross-listed from cs.IT
Orchestrating Communication, Computing, and Energy Transfer for Wireless-Powered 6G Closed-Loop Controls
Chengleyang Lei, Wei Feng, Yanmin Wang et al. (+4 more)
Future sixth generation (6G) communications are expected to support robotic control tasks in applications such as industrial automation and emergency response, where sensors, computing units, and
robots are interconnected via nervous system-like networks to form sensing-communication-computing-control (SC3) closed loops. However, the limited battery capacities of devices within these SC3 loops constrain operational duration and degrade control efficiency, particularly in remote or post-disaster scenarios. To address this challenge, wireless power transfer (WPT) can be leveraged to provide continuous energy supply for SC3 closed loops. In this paper, we investigate a wireless-powered SC3 system, where a satellite transfers energy via
radio frequency (RF) signals to support the communication and computing processes of multiple SC3 closed loops. By accounting for the intricate coupling among computing, communication, and energy transfer, we propose a holistic design framework to enhance overall control performance. Specifically, we adopt the linear quadratic regulator (LQR) cost as the performance metric and formulate a sum LQR cost minimization problem. The uplink/downlink transmit power, bandwidth allocation, computing capability, communication/computing time allocation, and WPT power allocation are jointly optimized. We recast the problem into a more tractable form and develop an iterative
algorithm to solve it. For the special case of a single loop, we further analyze the properties of optimal solutions in energy-limited scenarios to provide insights for practical parameter configuration. Simulation results demonstrate the performance gains of the proposed scheme.
Read the full abstract ▾
All authors Chengleyang Lei, Wei Feng, Yanmin Wang, Yunfei Chen, Xiaoyu Liu, Liuguo Yin, Ning Ge
Signal Processing
AI-RAN on NPUs: Baseband Processing Without Baseband Chips
Shilong Zhang, Luping Xiang, Jienan Chen et al. (+1 more)
AI-RAN aims to unify artificial intelligence and
radio access network workloads on a shared compute substrate. While this paradigm has so far been demonstrated primarily on Graphics Processing Units (GPUs), it remains unclear whether Neural Processing Units (NPUs), which are AI accelerators optimized for inference, can also support wireless baseband processing. Here, we provide the first affirmative answer by resolving the fundamental mismatch between baseband workloads and NPU architecture. A computational isomorphism exists:
matrix and vector engines NPUs dedicate to inference inherently cover physical-layer operations. Yet NPU architectures are natively shaped for dense-tensor AI inference, not baseband. This architectural mismatch surfaces as opposing optimization objectives: traditional baseband minimizes arithmetic operations, whereas NPU performance demands maximizing engine utilization. We close this gap by reconstructing communication
algorithms onto AI compute primitives, prioritizing engine utilization over arithmetic count. We validate this with a complete OFDM transceiver on an Ascend 310B1 edge NPU, demonstrating end-to-end over-the-air transmission via USRP X300 at 3.0 GHz.
Read the full abstract ▾
All authors Shilong Zhang, Luping Xiang, Jienan Chen, Kun Yang
Systems and Control · Cross-listed from math.OC
Robust Receding Horizon Games with Additive Uncertainty
Dinesh Patra, Tanish Jain and Ashish R. Hota
We study a receding horizon game in which multiple
agents drive linear systems subject to additive disturbances, private state and input constraints, and shared coupling constraints. We propose a robust game-theoretic control framework that combines tube-based constraint tightening with a finite-horizon generalized Nash
equilibrium problem (GNEP), equipped with a discrete algebraic Riccati equation (DARE)-based terminal cost and a decoupled positively invariant terminal set. The framework guarantees recursive feasibility for every bounded disturbance realization. Exploiting the potential-game structure induced by tracking costs, we further establish asymptotic convergence of each
agent's nominal state to a steady-state variational generalized Nash
equilibrium (vGNE), and show that each
agent's actual state converges to a neighborhood of the vGNE determined by the minimal robust positively invariant set.
Read the full abstract ▾
Signal Processing
Light Coils: MRI with Fully Optical Data and Power Transmission
Zining Liu, Morteza Teymoori, Jakob Gerlach et al. (+5 more)
In MRI, dense receiver coil arrays with a high number of coil elements are used to efficiently detect and encode the signal. Further increasing the number of coils is hampered by electrical cabling and massive electronics that introduce electromagnetic coupling, integration complexity and even safety constraints. Here we introduce the novel Light Coils concept, a fully optical MRI receive architecture in which data transmission, front-end power delivery, and coil detuning are all implemented optically, thereby reducing the massive galvanic cabling to a few optical fibers. For signal encoding, Mach-Zehnder modulators (MZM) are used to convert the MR signal from each coil onto a C-band optical carrier. The preamplifiers are driven via a power-over-fiber (PoF) system that uses a high-efficiency photovoltaic (PV)
cell for optical-to-electrical power conversion. A pulse-sequence-triggered optical path controls active detuning. Jointly optimizing modulator bias, optical power and front-end gain under realistic receiver chain conditions, Light Coils can match the signal-to-noise ratio (SNR) of conventional RF coil systems with galvanic cables at MZM input powers of 5-10mW and photonic power converter inputs of 80-100mW. At a clinical 3T MRI system, we show in vivo human
brain imaging with a single-channel Light Coil element with an image quality and SNR comparable to a conventional coaxial readout using the identical coil element. Extending the concept to a four-channel array using dense wavelength-division multiplexing over a single fiber, we demonstrate wavelength-selective routing with inter-channel optical isolation exceeding 28dB, reduced noise correlation compared with the galvanic reference, and parallel imaging. These results establish a scalable route towards lightweight, modular, and potentially ultra-dense MRI receive arrays based on integrated photonics and power-over-fiber.
Read the full abstract ▾
All authors Zining Liu, Morteza Teymoori, Jakob Gerlach, Reza Aghabagheri, Henning Helmers, Michael Bock, Caglar Ataman, Ali Oezen
Audio and Speech Processing
Noisy Environment Adaptation of Neural Speech Codec via Focal Mask and Noise Feature Separation
Shaokai Li, Weiping Tu and Yuhong Yang
Neural
speech codec has attracted extensive
attention for high-quality reconstruction at low-bitrate. However, real-world noise severely degrades its performance and hinders high-quality clean
speech reconstruction. To tackle this problem, we propose FocalSE, a novel
speech enhancement method that performs feature denoising, noise feature separation and noise recognition in the continuous
embedding space of neural
speech codecs. Specifically, we develop focal modulation-based compression and decompression to capture global context and local mutual information, and generate focal masks to recover clean feature
embeddings. We then separate noise
embeddings from noisy
embeddings to improve denoising performance. Finally, we use ResNet1D-18 to recognize noise categories for better separation effectiveness. Extensive experiments on two standard
datasets, LibriTTS and ESC50, demonstrate that our method outperforms state-of-the-art approaches under low-bitrate and low-SNR conditions.
Read the full abstract ▾
Image and Video Processing
Non-Line-of-Sight imaging using raster scanning at NIR wavelength
Mohammad Roueinfar and Mahdi Salmanian
Non-line-of-sight (NLOS) imaging is an emerging technique with transformative potential, enabling the visualization of hidden objects through indirect light reflection. This paper presents a NLOS imaging method operating in the near-infrared (NIR) wavelengths, specifically employing a raster scanning technique with a pan-tilt device. The NIR
laser, operating at a wavelength of 808 nm and an output power of 500 mW, illuminates a hidden target occluded by an obstacle. The imaging process involves three bounces: the
laser beam first strikes a relay wall, then reflects off the hidden target, returns to the relay wall, and subsequently reaches the NIR camera. This study systematically evaluates the effectiveness of the proposed method across three distinct targets, demonstrating the capability to recover high-quality images from non-line-of-sight scenarios. The obtained images of the hidden targets are compared with their ground truth images, and the error in the obtained images is assessed based on the criteria of Mean Squared Error (MSE) and Root Mean Square Error (RMSE).
Read the full abstract ▾
Systems and Control · Cross-listed from cs.IT
Lower Bound of Networked Control with Multiple Sensors and One Controller And The Application to Tracking Gaussian-Markov Source
Sijie Li, Takashi Tanaka and Hyeji Kim
This paper investigates the causal rate-distortion function for networked control systems with multiple encoders and a single decoder, a longstanding open problem in information and control theory. While previous work has explored the causal rate-distortion function for single-encoder and feedback-enabled networked settings, the case of networks without feedback remains unaddressed. We establish a novel directed information lower bound, the first derived for the networked control setting. We further demonstrate the optimality of linear, independent encoders and linear decoders for optimizing this lower bound for Linear Quadratic Gaussian (LQG) plant and quadratic cost, with the condition that the full plant state is observed when sensors are sitting together. By reducing the original infinite-dimensional optimization problem to a finite-dimensional one, our approach simplifies the analysis. Additionally, our directed information lower bound provides an alternate proof for the sufficiency of linear encoders in the single encoder and single decoder setting with side information, extending prior results in the literature. We present Semidefinite Programming formulations for the causal rate distortion function of Gaussian-Markov sources with linear side information and the singular noise
matrix.
Read the full abstract ▾
Systems and Control
Scenario-based Data-Enabled Predictive Control: Robustification via the Scenario Approach
Sebastian Zieglmeier, Nikolas Recke and Mathias Hudoba de Badyn
This paper proposes Scenario-Based Data-Enabled Predictive Control (Scenario-DeePC), which integrates the scenario optimization framework into Data-enabled Predictive Control (DeePC) to provide probabilistic guarantees on constraint satisfaction under uncertainty. In contrast to existing methods, the uncertainty is characterized directly from data by constructing empirical disturbance scenarios from observed prediction errors, keeping the method fully consistent with the data-driven philosophy of DeePC and free of distributional assumptions. We establish the supporting theory, including a distribution-free probabilistic guarantee on constraint satisfaction and recursive feasibility of the receding-horizon scheme. An adaptive extension collects scenarios online, enabling the controller to adjust to changing noise characteristics, disturbances, and operating-point-dependent model mismatches. The approach is demonstrated on a linear Boeing 747 model and a nonlinear two-tank system, showing a significant reduction in constraint violations compared to standard DeePC, while maintaining comparable tracking performance in nominal conditions and improving tracking accuracy in the nonlinear setting.
Read the full abstract ▾
Signal Processing
Optimal Uplink Pinching-Antenna Activation
Zhenqiao Cheng and Chongjun Ouyang
An uplink multiuser pinching-
antenna system (PASS) is considered, where multiple dielectric waveguides are deployed at the base station and one pinching
antenna (PA) is activated on each waveguide. For practical implementation, each PA is restricted to a finite number of preconfigured locations. The resulting uplink sum-rate maximization problem is represented as a layered tree search. Three
algorithms are then developed: a greedy search (GS), a beam search (BeS), and an optimal branch-and-bound (BnB) search. In GS, the locally best branch is selected through efficient
matrix-inverse updates. In BeS, several promising partial paths are retained to provide a tunable performance-complexity tradeoff. In BnB, noncompetitive subtrees are pruned through a monotonic transformed objective without loss of optimality. Substantial gains over a conventional fixed array are demonstrated by numerical results. Near-optimal performance is also achieved by GS and moderate-width BeS at a lower computational co t than BnB.
Read the full abstract ▾
Image and Video Processing
FedProIn: Mitigating Client Drift for Learnable Prototypes in Federated Medical Imaging
Harsh Kumar, Tarun Kumar Garg and Vaanathi Sundaresan
Federated learning (FL) is severely hindered by statistical heterogeneity due to variations in scanners, acquisition protocols, and patient populations. Such non-IID data induces client drift during local optimization, leading to unstable convergence and suboptimal global models when parameter-based aggregation is applied. We propose a prototype-based, influence-aware
federated learning framework (FedProIn) that uses multiple learnable class prototypes to capture shared semantic structures across heterogeneous clients. We introduce feature divergence loss and prototype contrastive loss to mitigate client drift by decomposing it into feature drift and prototype drift. In addition, we propose a normalized influence aggregation strategy that adaptively weights client prototypes according to their contribution to the global representation, reducing the impact of biased or low-quality updates. Experimental results on two publicly available medical
datasets, HAM10000 and Matek-19, demonstrate that FedProIn achieves accuracies of (83.5% IID, 81.1% non-IID) on HAM10000 and (96.2% IID, 95.8% non-IID) on Matek-19, respectively, outperforming existing baselines in both conditions. Our code is available at https://github.com/harsh-kmr/FedProIn.
Read the full abstract ▾
Audio and Speech Processing
DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speech
Junwon Moon, Yejin Lee, Seungbeom Kim et al. (+4 more)
Autoregressive (AR) text-to-
speech (TTS) models generate discrete
speech tokens sequentially, which makes inference slow and can degrade robustness, since local errors propagate to later positions and can escalate into
hallucination. This limitation stems from their left-to-right AR commitment: each
token must be determined before future
speech-
token context is available. However, such ordering is not an inherent requirement for TTS, since the model receives the full input text before synthesis. In this paper, we introduce DELTA-TTS, a lightweight LoRA-based adaptation framework that converts a pretrained AR TTS model into a discrete
diffusion language model (dLLM) for confidence-ordered
speech-
token decoding. To better capture the local structure of
speech, DELTA-TTS incorporates a convolution module that injects local acoustic context, together with a 1/t-weighted training objective and a time-shifted inference schedule that together defer low-confidence positions to later steps. Trained on only 585 hours of LibriTTS, DELTA-TTS achieves a 1.75% WER on Seed-TTS test-en, outperforming its AR backbone while generating
tokens 3.3 times faster. Further analysis shows that DELTA-TTS produces sharper text-
speech alignment, increases overall decoding confidence, and mitigates the
hallucinations observed in AR generation. Audio samples, inference code, and model weights are available at https://slpr-lab.github.io/DELTA-TTS.
Read the full abstract ▾
All authors Junwon Moon, Yejin Lee, Seungbeom Kim, Hoseong Ahn, Sewoong Park, Heeseung Kim, Kyuhong Shim
Audio and Speech Processing · Cross-listed from cs.CL
Speaker-Disentangled Chunk-Wise Regression for Syllabic Tokenization
Ryota Komatsu, Kota Kawakita, Takuma Okamoto et al. (+1 more)
Unsupervised syllabic tokenization aims to learn discrete syllabic
tokens that capture latent linguistic content-related structure from raw
speech. Recent syllabic tokenization methods employ teacher-student
distillation of the pretrained HuBERT to organize latent
speech frame representations into syllabic segments. However, when trained with an utterance-level cross-
entropy objective, the model predicts speaker identity rather than linguistic content, thereby compromising the purity of syllabic
tokens. To address this problem, we propose a speaker-disentangled syllabic tokenizer that regresses speaker-perturbed student representations toward clean teacher targets within fixed-length chunks. Experimental results demonstrate that our proposed method achieves state-of-the-art performance in syllable boundary detection and syllabic segment clustering. Moreover, a
speech language model trained on our syllabic
tokens achieves a 7% relative improvement in syntactic and semantic understanding over the phone-level SpiRit-LM.
Read the full abstract ▾
All authors Ryota Komatsu, Kota Kawakita, Takuma Okamoto, Takahiro Shinozaki