Computer Vision
Ngoc Duy Tran, Kien Do, Duc Thanh Nguyen et al. (+1 more)
One-step
diffusion distillation achieves strong performance on small
datasets but remains insufficient for high-resolution or complex text-to-image settings. Extending to multistep
distillation often requires training data, suffers from early-step degradation, inconsistent outputs across sampling budgets, and incurs high computational cost. We identify that these limitations stem from simultaneous training, where a single adapted network approximates multiple score functions of different student distributions at once, causing biased distribution matching and unstable optimization. Therefore, we propose Progressive Multistep Data-free
Distillation (PMDD), which assigns a dedicated adapted network per student distribution while avoiding the
memory overhead of maintaining multiple adapted networks. Combined with a stage-wise unforget loss to preserve earlier-step generations, PMDD reduces optimization bias in the student, provides faster convergence, and a single unified model that supports multiple inference budgets. Extensive experiments show that PMDD matches or outperforms data-free few-step baselines at every multistep budget, reaches FID 1.99 on ImageNet 64x64 in two steps, below its 512-NFE teacher (2.32), and achieves FID 7.76 on zero-shot COCO 512x512, while requiring 4-5x less training compute than data-free DMDv2 to support three inference budgets.
Read the full abstract ▾
All authors Ngoc Duy Tran, Kien Do, Duc Thanh Nguyen, Truyen Tran
Computer Vision
Kandinsky S2V 1.0: Audio-Driven Avatar Video Generation
Nikolay Semenov, Artem Riabinin, Sergey Bassenko et al. (+8 more)
We present Kandinsky S2V 1.0, an open family of
speech-to-
video models for controllable audio-driven human animation, comprising Kandinsky S2V 1.0 Lite (2B parameters) and Kandinsky S2V 1.0 Pro (21B parameters). From a single reference image, a
speech waveform, and a text
prompt, the models generate long-form 25 FPS
video in which lip motion, expression, pose, and gesture follow the
speech. The architecture draws text and audio conditioning from one Qwen2.5-Omni model and adds a reference-aware three-scale visual
memory. Training starts from the Kandinsky 5.0
Video foundation models [1] and combines mixed reference-
memory layouts with an area-adaptive face loss. The training corpus contains 1.17M single-speaker clips selected by active-speaker detection, with captions generated directly in English and Russian. A side-by-side human evaluation against the strongest open-source baseline and two proprietary commercial models shows that Kandinsky S2V 1.0 Pro performs comparably to or better than these models on average and is also preferred for lip synchronization. On common
speech to
video benchmarks, Kandinsky S2V 1.0 Lite and Kandinsky S2V 1.0 Pro rank among the top performing models according to mean rank over five reported metrics. Beyond the base models, we provide {\pi}-Flow distilled variants of both Kandinsky S2V 1.0 Lite and Kandinsky S2V 1.0 Pro. The distilled Kandinsky S2V 1.0 Pro model generates
video in 16 sampling steps without classifier-free guidance, achieving a 5.4 times end-to-end speedup over the base model, while matching or exceeding it on nearly all reported
benchmark metrics. To accelerate open research and deployment in
speech-to-
video generation, we release the code, model checkpoints, and diffusers integration under the MIT license.
Read the full abstract ▾
All authors Nikolay Semenov, Artem Riabinin, Sergey Bassenko, Anna Averchenkova, Julia Agafonova, Anastasia Aliaskina, Kirill Kozlov, Lev Novitskiy, Nikita Samsonov, Nikolai Gerasimenko, Denis Dimitrov
Computer Vision · Cross-listed from eess.IV
Deep Learning for Atmospheric Turbulence Mitigation: A Comparison Between GANs and Transformers
Juan S. Carrillo and Jérôme Gilles
Atmospheric
turbulence creates complex spatial and temporal distortions that significantly reduce image quality. This work compares two deep learning methods for
turbulence reduction: Generative
Adversarial Networks (GANs) and
Transformer-based models. Both methods are trained and tested on the SOTIS
dataset, which includes multi-frame sequences under different
turbulence conditions with ground truth references. Our results show an important finding: GAN-based models perform better in severely degraded, low-quality situations, while
Transformer-based models achieve better results in moderate to high-quality settings, especially in correcting
geometric distortions. We also enhance the
Turbulence Mitigation
Transformer (TMT) by introducing a refined training strategy, an augmented input design that includes a temporally averaged frame as prior structural information, and a modified loss function that incorporates the Structural Similarity Index (SSIM). These improvements lead to consistent performance gains across various
turbulence conditions. Overall, this study provides an in-depth evaluation of deep learning methods for
turbulence reduction and offers practical guidance for selecting models in different imaging environments.
Read the full abstract ▾
Computer Vision
LAMP: 3D Source-Aware Lighting Estimation at Multiple Points
Yanqiao Wang and Jin Sun
Existing illumination estimation methods primarily recover lighting distributions for rendering, but typically leave the contributions of individual physical light sources at specific scene locations implicit. We address this gap by leveraging the scene understanding capabilities of
vision-
language models (VLMs) to connect local illumination with its physical sources. We introduce LAMP, a framework for point-conditioned, source-aware illumination
reasoning from a single RGB image with marked query points. LAMP constructs a shared inventory of active light sources, ranks contributing sources for each query, and predicts their 3D query-to-source directions. We supervise these relationships using procedurally controlled indoor scenes, reusing independently rendered source responses to derive contribution targets across lighting configurations and obtaining exact directions from source and query positions. On the Infinigen test set, fine-tuning Qwen3-VL-8B increases dominant-source 3D PCK@0.5m from 0% to 70.6%, with 94.5% of directions within 12.5 degrees of ground truth among correctly localized sources. Combined with a lightweight network predicting dense RGB irradiance, LAMP achieves probe-rendering quality comparable to prior lighting-estimation methods while additionally localizing individual physical sources in 3D and explicitly attributing local illumination to them.
Read the full abstract ▾
Computer Vision
Long-horizon Video Generation with Slot Memory Addressed By Region Tokens
Eren Cetin, Julius Erbach, Vukasin Bozic et al. (+6 more)
Memory for long-horizon
video generation must satisfy two competing requirements: it must remain bounded so generation can continue indefinitely, while still recovering what was observed at a place when the camera returns, regardless of how long ago it was seen. Reconstruction-based
memories provide spatial recall by caching and reprojecting explicit geometry, but require growing storage and depend on estimated 3D structure. Recurrent
memories keep a constant-size state, but retention is governed by temporal recurrence, causing old observations to fade with time. We introduce SMART, a slot
memory addressed by region
tokens, which combines bounded storage with spatially addressed recall. SMART stores content in a constant-size slot recurrence whose writes are addressed by the observing camera and whose reads are queried by the rendering camera, without constructing an explicit 3D representation or maintaining a growing cache. Consequently, recall is conditioned on inter-view geometry rather than on recency. Each read additionally provides a confidence estimate that conditions a chunk-causal
diffusion transformer, allowing it to selectively trust retrieved content while relying on its generative prior elsewhere. On long trajectories in
Memory Maze and Minecraft, SMART maintains high-fidelity recall far beyond its training horizon while using constant
memory and per-step cost.
Read the full abstract ▾
All authors Eren Cetin, Julius Erbach, Vukasin Bozic, Hyojun Go, Zixiang Zhao, Nikolai Kalischek, Federico Tombari, Konrad Schindler, Dominik Narnhofer
Computer Vision · Cross-listed from cs.LG
Riding the Fixed-Point Curve: Image Classification in the Continuous Thought Machine
Thomas Hannagan
The Continuous Thought Machine (CTM) is a recurrent network that attends to inputs and makes predictions based on the synchronization between its
neurons. On tasks spanning many domains, it displays emergent thought processes that have not been explained. We report a number of findings about how the CTM classifies images, focusing on ImageNet. Each thinking step is an attracting but strongly non-normal map exhibiting balanced amplification (Murphy and Miller, 2009); while attraction is already present at CTM initialization, training highly promotes non-normality. By Banach's
theorem, the neural state has a locally unique attracting fixed point, which moves at every step, and the thought trajectory is the curve these fixed points trace, followed one tick behind. Most of that curve's motion is shared across images, and visible as a previously reported traveling wave of activity across
neurons. The
attention schedule along the thought is largely generic: shuffling queries across images at every tick costs only 7 percent of classification accuracy, while image-independent increments predict the next query with R^2 = 0.98 (held out). Our results suggest that the CTM has found a single strategy to process all ImageNet images.
Read the full abstract ▾
Computer Vision
A Gradient-Regularized Deep Learning Method for Restoring Image in Myopic Vision
Kebin Contreras, Luis Toscano-Palomino, Rafael Torres et al. (+1 more)
Myopic
vision degrades the quality of human
vision due to spherical eye defocus aberrations that impair the perception of fine spatial details. To address this limitation, this work proposes a deep learning-based approach to computationally design transformed images that allow individuals with myopia in fixed distance to perceive scenes with enhanced clarity. The method trains a neural network to counteract defocus aberrations modeled using the second-order Zernike
polynomial $Z_2^0$. The approach employs a U-Net architecture trained with a loss function combining reconstruction error minimization with spatial difference structural preservation through $\ell_1$ regularization. Training is carried out exclusively on the KITTI
dataset, while testing is performed on DIV2K and ImageNet
datasets, demonstrating generalization across diverse image distributions. Under myopic conditions modeled by $Z_2^0$, baseline performance shows visual degradation with PSNR values ranging from 19.47 to 27.42 dB across
datasets and severity levels. The proposed gradient-regularized method achieves consistent improvements, delivering PSNR gains ranging from 1.48 to 7.34 dB and significantly lower perceptual compared to classical baselines.
Read the full abstract ▾
All authors Kebin Contreras, Luis Toscano-Palomino, Rafael Torres, Jorge Bacca
Computer Vision
Decoupled Drift-Diffusion World Models for Consistent Future Frame Prediction
Anton Arapin, Aditya Singh, Yixin Wang et al. (+2 more)
World models of visual information predict future image frames conditioned on given action sequences. This predicted scene
evolution is driven by deterministic changes such as controlled motion on a
robot platform, as well as
stochastic changes such as rendering of manipulated objects or previously occluded regions. Our key insight is that the spatial location of these
stochastic changes is itself learnable. We leverage this insight by jointly regressing both the deterministic next-state prediction and the spatial uncertainty map, then using the uncertainty map to guide a conditional
diffusion process. We propose Decoupled Drift-
Diffusion World Model (D3WM), which uses a
transformer trained with a Gaussian negative-log-likelihood objective to produce both the deterministic next-state prediction and a spatial map of heteroscedastic aleatoric uncertainty. The learned uncertainty map then provides a structured spatial gate for a conditional
diffusion model: deterministic regions are anchored to the predicted state, while uncertain regions, such as disocclusions or dynamic textures, are sampled by the
diffusion process. We frame this decomposition under a variance-gated
stochastic differential equation that separates deterministic drift from
stochastic diffusion. D3WM shows improved results over competitive baselines in two distinct types of action spaces: robotic tabletop manipulation and camera motion.
Read the full abstract ▾
All authors Anton Arapin, Aditya Singh, Yixin Wang, Jun Gao, Bernadette Bucher
Computer Vision
Into the LidarVerse: A Real-Time Unified LiDAR World Model Across Tasks and Sensors
Henry Che, Xuanchi Ren, Sanja Fidler et al. (+3 more)
We present LidarVerse, a real-time unified
LiDAR world model for sensor simulation across tasks and sensors. Existing
LiDAR simulators largely follow one of two complementary paradigms: reconstruction-based methods preserve the geometry of a captured scene but require lengthy per-scene optimization, whereas generative methods amortize learning across data but lack the consistency needed for long-horizon rollouts. LidarVerse brings both paradigms into a single, unified conditional generative framework. A universal tokenizer maps
LiDAR range sequences into a shared latent space, where a bidirectional teacher learns a common interface for conditioning on a
LiDAR prefix, an HD-Map, a persistent spatial cache, and optionally, text. Varying the available scene evidence turns the same model into a generator of new worlds, a point-cloud forecaster, or a generalizable resimulator of observed scenes without per-scene optimization. We distill the teacher into a four-step autoregressive causal student and maintain both a rolling temporal K/V cache and a world-aligned spatial cache, enabling persistent long-horizon rollouts with real-time inference at 11 FPS on a single H100 GPU. LidarVerse achieves state-of-the-art performance on most reported metrics across all three evaluation protocols, while its tokenizer supports 32-, 64-, and 128-beam
LiDAR and generalizes to an unseen scanning pattern. These results demonstrate that generative and reconstruction-grounded
LiDAR simulation can be realized within one model, providing the core capabilities for future closed-loop sensor simulation.
Read the full abstract ▾
All authors Henry Che, Xuanchi Ren, Sanja Fidler, Shenlong Wang, Laura Leal-Taixé, Shengyu Huang
Computer Vision
Agronomic Characterization and Time-Series UAV Phenotyping to Support Field Pea Breeding in Iowa
Yanben Shen, Joscif Raigne, Asheesh Singh et al. (+1 more)
Field pea has gained interest in the Midwest U.S. and has the potential to benefit Iowa cropping systems. This study evaluated 17 accessions and 7 Canadian varieties for agronomic and seed yield performance. This study found that PI accession trials exhibit genetic diversity in phenology and canopy traits, whereas the Canadian elite varieties demonstrated great yield potential under Iowa conditions. The accession genotypes matured between 75 DAP (late June) and 95 DAP (late July) across six site-years. The top-yielding Canadian varieties averaged 1961 kg/ha across two site-years. To comprehensively assess field pea performance, this study extracted time-series Unmanned Aerial Vehicle(
UAV)-derived traits, assessed
UAV-derived traits, and evaluated their correlations with multiple agronomic traits and yield. The results found that
UAV-derived time-series canopy area was the most yield-relevant trait (r = 0.63) among all
UAV-derived canopy features, vegetation indices, and manually collected measurements. Compared with yield,
UAV-derived canopy area showed higher selection reliability (H2 = 0.87, CV = 13.46%), making it suitable for phenotypic selection and accelerating Midwest field pea breeding programs. This study identifies useful
UAV-derived features for pea breeding and demonstrates the value of integrating diverse germplasm, elite commercial varieties, and time-series
UAV phenotyping. Combining genetic variation from the accession trial with the yield potential of high-performing commercial varieties provides valuable breeding materials and phenotyping strategies for developing locally adapted, high-yielding field pea varieties for future production in the Midwest.
Read the full abstract ▾
All authors Yanben Shen, Joscif Raigne, Asheesh Singh, Arti Singh
Computer Vision · Cross-listed from eess.IV
DoReMe: Dual optimization of Reconstruction and Motion estimation for self-navigated free-breathing abdominal MRI using Gaussian Splatting
Majd Helo, Dominik Nickel and Thomas Kuestner
Purpose: Free-breathing abdominal MRI is limited by respiratory motion, which introduces image artifacts and k-space inconsistencies. Existing reconstruction approaches often depend on pre-estimated motion fields or external respiratory surrogate signals. We propose a coupled reconstruction-motion estimation
algorithm for self-navigated free-breathing acquisition, in which motion is represented by compact Gaussian primitives and jointly refined with the image estimates during iterative reconstruction. Methods: An iterative reconstruction that alternates between data consistency updates and motion modelling was proposed. Respiratory motion was represented using Gaussian primitives. A Gaussian splatting was integrated into the reconstruction
algorithm. The method was evaluated on self-navigated free-breathing Cartesian gradient-echo (GRE) acquisition in four healthy volunteers, including retrospective scan shortening experiments, and was compared with regularized compressed sensing (CS) reconstruction. Results: Respiratory motion states were reconstructed in approximately 1.5 min. Compared with CS, the proposed method improved visual image quality, particularly in coronal liver views, and better preserved anatomical structures in retrospectively shortened acquisitions. These findings provide proof of concept for the proposed method. Conclusion: The proposed coupled reconstruction-motion estimation
algorithm improved visual image quality compared with CS alone. Gaussian primitives provided a compact and sufficiently flexible representation of nonrigid respiratory motion with practical runtime, supporting further scanner integration and validation in larger cohorts.
Read the full abstract ▾
Computer Vision
Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models
Mahdieh Mallahnezhad, Masoud Jafaripour, Mohamadali Tofigh et al. (+2 more)
Multimodal Large
Language Models (MLLMs) have been widely applied in embodied
agents and robotics for
vision tasks. However, MLLMs suffer from limitations in spatial
reasoning and perception. Post-training techniques have shown promise for improving these capabilities; yet, there are limited data-driven surveys of such approaches. In this survey, we consider approaches to fine-tuning pretrained MLLMs for spatial
reasoning tasks, ranging from supervised to reinforcement learning, with an emphasis on the
datasets used. We investigate themes of
dataset construction, data modality and annotation, and evaluation of the resulting models. Based on these themes, we categorize research into three groups:
dataset design,
dataset construction and evaluation. We find four main methods for constructing fixed spatial annotations: rules and templates, LLM prompting, procedural generation, and human-in-the-loop. We also observe substantial variation in evaluation approaches, including different scoring methods, tolerance metrics, and
benchmark types. Finally, we identify opportunities for future research in unified dynamic annotation, efficient spatial
reasoning, standardization of evaluation, expanded spatial range, and human-centered spatial interactions.
Read the full abstract ▾
All authors Mahdieh Mallahnezhad, Masoud Jafaripour, Mohamadali Tofigh, Sheila Schoepp, Osmar R. Zaiane
Computer Vision
The Gap Is in the Geometry: A Convergent Meta-Analysis of Multi-Facet Aerial and Remote Sensing Video Understanding
Georgios Angelidis, Aggelos Psiris, Maria Vakalopoulou et al. (+3 more)
Aerial and remote sensing
video is increasingly evaluated like web
video, by posing multiple-choice questions to
multimodal large
language models (MLLMs) over sampled frames and reporting accuracy. This paper argues that this recipe measures the wrong aspect, and supports the claim with a convergent meta-analysis of a broad set of published results, organized in four evidence tiers, in which a claim becomes a position only when collected evidence agrees. Parameter scale exhibits a median of +3.4 accuracy points per tenfold increase on aerial
benchmarks, far below a human gap of at least 19 points, whereas acquisition conditions control perception, as 315 of 318 evaluations in 19 within-paper contrasts lose accuracy when conditions such as altitude, view, object size or time horizon become harder. No aerial
benchmark in the corpus combines
video input with per-task human and random baselines, competence collapses from answering to acting in closed-loop navigation, and the same MLLMs score higher on the aerial
benchmark than on ground spatial ones, but lower on studies that test cross-view geometry. From these findings the current study derives five falsifiable principles and proposes the AVU-Card, a reporting card for aerial
video benchmarks
Read the full abstract ▾
All authors Georgios Angelidis, Aggelos Psiris, Maria Vakalopoulou, Georgios Th. Papadopoulos, Giorgos Sfikas, Iraklis Varlamis
Computer Vision · Cross-listed from cs.AI
GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning
Qingsong Xie, Jiahao Tian, Jiacheng Tang et al. (+5 more)
Constructing robust GUI
agents relies on
multimodal visual grounding: the ability to precisely map natural
language intents to actionable on-screen components. Despite abundant desktop, web, and mobile
benchmarks, high-fidelity resources tailored for diverse environments are surprisingly scarce. To bridge this gap, we introduce the Screen Referring Grounding
Dataset (ScreenRef), a comprehensive GUI grounding
dataset consisting of GUI screenshots paired with natural
language referring expressions and box or point annotations. Spanning desktop, web, and mobile domains, ScreenRef comprises 586K screenshots and over 633K referring expression instances, covering tasks such as text recognition, visual matching, spatial understanding, and refusal scenarios. To boost the model's grounding performance, we propose the
Distillation-to-Reinforcement Learning (D2RL) method to address the limited generalization of supervised fine-tuning and the sparse reward problem in reinforcement learning. Utilizing ScreenRef, we train Qwen3-VL-8B-Instruct via sequential SFT and D2RL training, yielding GroundGUI-8B. GroundGUI-8B achieves 95.2% on ScreenSpot-V2, 63.9% on ScreenSpot-Pro, 65.4% on OSWorld-G, 86.8% on MMBench-GUI L2, and 38.3% on UI-
Vision. Among 8B-scale models, it produces substantial gains over models trained on open-source
datasets, with performance comparable or even superior to models trained on closed-source data. Code, models, and
datasets are available at https://github.com/PrentisAI/GroundGUI.
Read the full abstract ▾
All authors Qingsong Xie, Jiahao Tian, Jiacheng Tang, Zhifeng Li, Madalina Ciobanu, Navan Preet Singh, Qingqing Mao, Ritankar Das
Computer Vision
MonoMIP: Multi-Height Inverse Perspective Mapping for Monocular 3D Object Detection
Emir Ulurak and Furkan K{\i}raç
Monocular 3D object detection methods increasingly rely on depth-guided
transformers, but they operate entirely in the perspective image plane where spatial relationships are distorted by foreshortening. We propose MonoMIP, a dual-decoder framework that adds complementary 3D
geometric reasoning through Bird's-Eye View (BEV) features constructed via Multi-Height Inverse Perspective Mapping. We perform IPM at K elevation planes spanning ground to rooftop, producing an ordered stack of height-conditioned metric BEV features that reduces above-ground misprojection and exposes depth-dependent appearance variation across planes. A BEV decoder with multi-scale deformable
attention aggregates this multi-height evidence per query, while a depth-guided 3D decoder provides complementary perspective-space predictions. Without extra training data, MonoMIP attains 25.84% Moderate AP3D|R40 on KITTI val and 21.66% on the official test set for Car, outperforming recent published monocular methods on both the KITTI and Waymo
benchmarks.
Read the full abstract ▾
Computer Vision
WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models
Nan Zhou, Jianxiong Gao, Qiuyu Wang et al. (+10 more)
As
video models enable longer and more realistic interactive experiences, maintaining a world consistent with its history becomes essential. This requires both preserving previously observed content and ensuring coherent world
evolution over time. World
memory is a critical capability to evaluate alongside visual fidelity and
video duration. We introduce WorldMemBench, a
benchmark for systematically evaluating world
memory in
video world models. Across 260 test cases, it assesses static
memory through the retention of human identity and appearance, object appearance and geometry, and environmental content and geometry, and dynamic
memory through event progression and state persistence. Using controlled camera departures and multi-revisit trajectories, the
benchmark evaluates static content against fixed initial references and dynamic events against expected state transitions. It combines appearance and geometry metrics with event-specific assessments, and reports
video quality and camera-following accuracy as complementary diagnostics. Evaluation of nine representative models shows that strong static retention does not imply reliable dynamic state
evolution, while repeated revisits reveal
memory degradation beyond the first return. Human studies support the
alignment between our automatic evaluations and human judgments. WorldMemBench provides a systematic testbed for diagnosing
memory failures and guiding the development of persistent, temporally coherent
video worlds.
Read the full abstract ▾
All authors Nan Zhou, Jianxiong Gao, Qiuyu Wang, Wen Wang, Jiahao Wang, Yufeng Yuan, Shuaiting Li, Hao Ouyang, Yujun Shen, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
Computer Vision
Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment
Xiao Yang, Zekun Zhang, Yuxuan Fan et al. (+5 more)
Video-based remote physiological measurement (RPM) has broad applications, while
Vision-
Language Models (VLMs) provide a promising opportunity to enable end-to-end physiological monitoring for future human-aware intelligent systems. However, existing RPM methods rely on task-specific models and cannot be seamlessly integrated with the
multimodal understanding and interaction capabilities of VLMs. We revisit RPM with VLMs and provide empirical evidence for two major bottlenecks: generic visual semantic representations fail to preserve fine-grained physiological cues; while discrete
token prediction in autoregressive generation is inherently misaligned with continuous numerical measurement. To address these challenges, we propose VITAL (Visual-physiological
Token Alignment for
Language Models), a visual-physiological
token alignment framework for RPM and preserving general
vision tasks. VITAL retains the original visual pathway for semantic understanding and introduces an independent physiological pathway to jointly model semantic and physiological information. Moreover, we propose a structured numerical
alignment approach to improve the reliability of generative heart rate estimation by enhancing numerical relationship modeling. Extensive experiments show that VITAL improves cross-domain estimation performance on multiple rPPG
datasets while preserving the general visual understanding capability of VLMs, demonstrating its effectiveness as a unified visual-physiological model. Code will be released.
Read the full abstract ▾
All authors Xiao Yang, Zekun Zhang, Yuxuan Fan, Duoxun Tang, Ziyang Liu, Dengbo He, Kaishun Wu, Jiyao Wang
Computer Vision · Cross-listed from eess.IV
Focus, Align, and Diffuse: Time-Series-Aware Keyframe Diffusion for Cardiac Dynamic Synthesis from Sparse Observations
Junkai Liu, Haofan Wu, Nay Aung et al. (+3 more)
Dense cardiac dynamics are essential for assessing cardiac function, yet in practice clinicians often observe only extremely sparse measurements. However, recovering full dynamics from such sparsity is a highly ill-posed problem due to the lack of temporal cues. While auxiliary time-series signals like ECG provide physiological guidance, effectively exploiting them is challenging due to the cross-modal representation gap and phase misalignment. In this work, we present CardioFAD, a Focus–Align–Diffuse paradigm that reformulates sparse cardiac dynamic synthesis as a recoverability–alignability anchor discovery problem. Our core insight is that resolving extreme temporal ambiguity requires a compact set of anchor states that are both sufficient to recover the visual trajectory and reliable for ECG–visual
alignment. We therefore learn motion-salient keyframes under two complementary principles: trajectory sufficiency, which preserves trajectory-critical intra-modal motion content, and anchor
alignment, which promotes consistent inter-modal correspondence at these anchors. We implement the Focus and Align modules optimized by the two aspects. Diffuse then performs two-stage generation by first synthesizing keyframes and subsequently interpolating the remaining frames. Extensive experiments on cardiac MRI and echocardiography
videos demonstrate superior visual quality, temporal coherence, and improved functional consistency under the extreme one-shot regime.
Read the full abstract ▾
All authors Junkai Liu, Haofan Wu, Nay Aung, Joao A. C. Lima, Steffen E. Petersen, Le Zhang
Computer Vision · Cross-listed from cs.RO
RingNav: Task-Shaped Circular Memory for Visual Heading
Naitri Rajyaguru, Aryan Tajne, Stephen Jardim et al. (+2 more)
Tiny
robots have limited computation and
memory, yet visual navigation systems often maintain more state than a task requires. We ask whether persistent
memory can instead be shaped by the task. We present RingNav, a \(10.6\)k–parameter visual heading
estimator that constrains its
memory to the circular
manifold \(S^1\). RingNav learns only the turn between consecutive images, while heading is composed directly on the circle. On the TartanAir Visual SLAM Challenge, RingNav reduces mean absolute turn error by \(27\%\) and accumulated heading error over $4$ sec by \(15\%\) relative to a similarly sized learned-
memory model. The structured update also gives exactly zero turn for identical frames and exact cancellation under sequence reversal. Without fine-tuning, RingNav transfers to real-world drone experiments and continues estimating heading under severe visual degradation. RingNav runs at $27.3$Hz on Raspberry Pi Zero 2W and can be deployed on an ultra-low power GAP8 processor at $33.4$Hz.
Read the full abstract ▾
All authors Naitri Rajyaguru, Aryan Tajne, Stephen Jardim, Cornelia Fermuller, Yiannis Aloimonos
Computer Vision
How Far Can Coding Agents Go in Visual Generation?
Hengguang Zhou, Yuanhao Ban, Jiaqi Feng et al. (+3 more)
Modern image generation largely relies on specialized generative models that leave the process of image construction implicit in learned parameters. We explore an alternative: general-purpose coding
agents that construct images through executable programs. Our study of zero-shot drawing on AgentDrawBench, a 200-
prompt benchmark, reveals a distinctive capability profile.
Agents excel at preserving structure and satisfying precise visual constraints, achieving competitive results in domains where compositional accuracy matters more than photorealistic appearance, while remaining limited in perceptual richness. Their visual expressiveness varies with the drawing interface, and a qualitative study on reconstruction shows how procedural skills extracted from human demonstrations support different artistic media. We then examine code as an external visual
language of thought: a structured, executable representation that
agents can construct and inspect. We test it in visual question answering task and find reconstructed visual code retains substantial information for question answering and, in some domains, supports higher accuracy than the original image. These abilities raise a further question: do
agents draw on a shared internal representation of scene content beyond a particular drawing program? We explore an internal visual
language-of-thought hypothesis, in which a shared, compositionally structured representation supports visual operations across drawing
languages. Cross-harness drawings and images assembled from independent single-pixel queries exhibit related layouts and tones.
Agents also maintain self-generated room arrangements across viewpoints after visible drawings are removed. These results demonstrate that the coding
agent would maintain a consistent and coherent world inside its imagine even before the harness gets involved and the generation starts.
Read the full abstract ▾
All authors Hengguang Zhou, Yuanhao Ban, Jiaqi Feng, Xiaohuan Pei, Justin Cui, Cho-Jui Hsieh
Computer Vision
Action Jets: Learning the Local Shape of the Action-conditioned Data Manifold
Anton Arapin, Chahyon Ku and Bernadette Bucher
World models are a powerful means of
reasoning and planning, but many of these action-conditioned predictive models are high dimensional and computationally expensive. These challenges limit the number of action candidates that can be evaluated by a planner. We show that this cost is avoidable for planners over short horizons, which only require local information with respect to the current state. For a fixed state, the frames reachable by varying a low-dimensional action (a camera movement or an end-effector command) lie on a low-dimensional
manifold. We estimate the local shape of that
manifold directly via a network trained to estimate how each image region moves per unit of each action axis supervised by data collected from samples by the generator. A correction head fit to trajectory data estimates the shape of this
manifold region up to a third-order Taylor
polynomial. We call this network estimating the Taylor
polynomial describing the
manifold shape an Action Jet and compute its radius of validity in closed form. By traversing this approximation of the local
manifold shape, we only have to perform inference once per state for any number of candidate actions. Scoring sixteen candidates from a single extraction is over 300x faster than with either source generator, and this gap widens with the number of candidates a planner evaluates. We measure against state-of-the-art action-conditioned generators, IRASim and WorldGym on BridgeData V2, a real 7-DoF manipulation
benchmark, and FrameCrafter on DL3DV scenes, where we chart from the novel-view generators DepthSplat and LagerNVS. Because only the tangent comes from the generator and every other term from the data, the Action Jet also predicts held-out frames more accurately than IRASim, the generator that supplied its tangent, at short horizons and at over 100x less compute.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
$\lambda_0$: Scaling Open-World Egocentric Human Data for Humanoid Loco-Manipulation
Lambda-humanoid Team
Humanoid loco-manipulation requires coordinated whole-body interaction with the physical world, yet collecting
robot demonstrations at scale remains costly. Human demonstrations offer a scalable alternative, but existing egocentric human supervision often omits locomotion and posture, while embodiment differences and controller dynamics hinder accurate end-effector execution. We introduce HumanVerse, a 500-hour
dataset of open-world human loco-manipulation pairing egocentric
video with synchronized body and hand motion. Our human-to-humanoid pipeline aligns camera viewpoints and arm appearance, and refines motion references and controller inputs offline to reduce end-effector errors while preserving whole-body coordination. We validate the pipeline through zero-shot transfer to Unitree G1 on representative whole-body manipulation and long-range loco-manipulation tasks, without target-task
robot demonstrations. Building on this pipeline, we develop $\lambda_0$, a humanoid foundation model for whole-body loco-manipulation, with
robot-aligned human demonstrations providing supervision for mid-training. Increasing whole-body human data consistently improves downstream task progress. $\lambda_0$ achieves 90.0% success on SIMPLE and 62.5% across real-world loco-manipulation tasks. We expect this study to guide scalable whole-body human data collection toward humanoid foundation models for richer interaction with the physical world. Homepage: https://lambdahumanoid.github.io/
Read the full abstract ▾
Computer Vision · Cross-listed from cs.SD
An Initial Study of Physiological Markers for Deepfake Speech Detection
Vojtěch Staněk, Michal Frič, Kamil Malinka et al. (+1 more)
Conventional deepfake
speech detectors primarily target synthesis artifacts and often degrade under unseen attacks or recording conditions. Human
speech, however, contains physiological markers associated with respiration, phonation, and temporal coordination arising from the
speech-production system that synthesizers may not reproduce faithfully. This motivates the hypothesis that deepfake
speech can be detected by identifying inconsistencies in these patterns, which we test using 36 handcrafted physiological markers organized into respiratory, vocal, and temporal categories. We evaluate the marker vectors with logistic regression and SVM classifiers on pooled English recordings. The best in-domain system achieves a 21.40% equal error rate (EER), while logistic regression achieves 32.77% on the out-of-domain
dataset pair. Since the markers describe specific
speech-production properties, the logistic-regression coefficients indicate whether higher marker values push the classifier toward bona fide or deepfake
speech. These results provide initial evidence that physiological markers offer a useful, interpretable signal for deepfake
speech detection.
Read the full abstract ▾
All authors Vojtěch Staněk, Michal Frič, Kamil Malinka, Anton Firc
Computer Vision · Cross-listed from cs.RO
ChunkAct: Action Chunking for Agentic Robotics
Bingxin Xu, Wenrui Bao, Xinxin Liu et al. (+3 more)
Agentic robotics places a large
language model (LLM)
agent in the
robot control loop, where it composes perception, motion, and
vision-
language-action (VLA) tools under the guidance of a
memory and thereby generalizes to new tasks without retraining. This generality comes at a steep price in efficiency. Every primitive requires a round of LLM
reasoning over the full interaction history, so the cost of an episode grows with the number of primitives. We find this
reasoning largely redundant: once a sub-goal is chosen, its primitives follow a routine already prescribed by
memory. We present ChunkAct, which introduces action chunking to agentic robotics. Online, a single call launches a chunk of chained primitives, which executes autonomously and returns control only upon failure. Offline, ChunkAct compiles the
memory into these chunks, so execution is
memory-free at test time and the planner is reserved for decisions that require
reasoning. LLM computation therefore scales with the number of decisions rather than the number of primitives. On LIBERO-Pro, ChunkAct improves task success by 5.0 points over the state of the art in agentic robotics while requiring 2.9x fewer LLM calls, 4.4x fewer input
tokens, and 2.3x fewer output
tokens.
Read the full abstract ▾
All authors Bingxin Xu, Wenrui Bao, Xinxin Liu, Jiawei Zhou, Yuzhang Shang, Emilio Ferrara
Computer Vision · Cross-listed from cs.RO
LoopWAM: Closed-Loop Generalist Policy Improvement from Deployment Rollouts
Wenxuan Guo, Yingping Liang, Zhaoguo Wang et al. (+11 more)
Scaling expert demonstrations alone does not enable generalist
robot policies to improve from deployment. Real-world rollouts contain successes, suboptimal behavior, and failures: imitating every recorded action can reinforce poor behavior, while filtering out failures discards evidence of physical interactions and their consequences. We introduce LoopWAM, a closed-loop world-action model that learns directly from this full quality spectrum, using rollouts from both itself and other deployed policies. Source-dependent noise sampling unifies behavior learning and action-conditioned world modeling, while quality steering and quality-weighted supervision retain full
video supervision across qualities and regulate action imitation.
Video and action
tokens interact throughout parallel denoising, allowing learned consequences to inform action generation directly. On RoboChallenge Table30-V2, three deployment-and-training rounds raise success from 23.67% to 43.33%, a 19.7-percentage-point gain, with a final score of 56.80. LoopWAM also achieves 94.5% average success on RoboTwin 2.0 and 49.5% overall success on RoboCasa365 without additional embodied pretraining, obtaining the best overall results among the compared methods across all three
benchmarks. On challenging shirt folding task, two rounds of deployment-driven improvement after single-task post-training raise success from 13.20% to 55.56%. These results demonstrate closed-loop improvement from mixed-quality experience under both multi-task learning and single-task post-training. Project page: https://loop-wam.github.io/
Read the full abstract ▾
All authors Wenxuan Guo, Yingping Liang, Zhaoguo Wang, Ziyuan Li, Angyuan Ma, Zhenyu Wu, Chengkun Wang, Bingyao Yu, Ze Chen, Borui Zhang, Xiuwei Xu, Meng Zhang, Erjin Zhou, Jiwen Lu
Computer Vision · Cross-listed from cs.RO
EgoExplorer: A Vision-Motion-Touch Observatory for Human Loco-Manipulation In Situ
Yunsong Zhou, Xuekun Jiang, Haiqi Shen et al. (+11 more)
Learning humanoid loco-manipulation from human demonstrations rests on observing how locomotion, posture, and object interaction evolve together as tasks unfold. Scaling collection across diverse environments, in turn, favors lightweight, portable capture systems. Yet, existing
datasets provide only partial views of this coordination. Thus, we introduce EgoExplorer, a 100-hour
vision-motion-touch observatory of human loco-manipulation built with a wearable system. With an emphasis on professional activities, its continuous recordings align egocentric
video with reconstructed whole-body motion, fine-grained finger articulation, and tactile observations, accompanied by step-level descriptions. To achieve this coverage with sparse body sensing, we develop the Calibrate-Complete-Constrain pipeline. Concretely, reusable calibration fixes participant geometry and sensor attachments before motion inference to disentangle mounting offsets from articulation. A temporal prior then completes unobserved joint motion, while layered constraints promote measurement agreement, temporal continuity, and
geometric consistency, properties that motion plausibility alone does not guarantee. Against tracker-fitted references, the method reduces body-point error by 71.9% relative to the strongest baseline; matched evaluations also show gains in hand stability and contact estimation. Preliminary studies explore
multimodal generation and conditioning alongside humanoid retargeting with whole-body control in simulation.
Read the full abstract ▾
All authors Yunsong Zhou, Xuekun Jiang, Haiqi Shen, Hangxu Liu, Chaojian Shi, Yujie Liu, Jingyuan Li, Haoran Yang, Jiafei Cao, Dongzhan Zhou, Dong Wang, Yao Mu, Chunhua Shen, Weinan Zhang
Computer Vision
Cepheid: Decoupling Scale from Structure for Cross-Family Motion Completion
Chih-Hai Su, Tsung-Wei Ke and Yu-Lun Liu
Sparse-to-dense motion completion recovers dense mesh motion from sparse vertex displacements. Residual priors trained on one geometry family frequently fail to beat parameter-free harmonic extension off-family. Moreover, cross-family training meets a scaling barrier: residual amplitude varies by more than an order of magnitude across geometry families and collapses as observations become denser. Since this scale depends on unknown target motion, it cannot be measured directly at test time. We decouple residual structure from physical scale by learning a dimensionless prior, estimating amplitude from observables and a lightweight correction fitted only on training sources. Our unified model surpasses harmonic extension across all evaluated families, including unseen cloth and legged bodies, without skeletons, registration, or cross-mesh correspondence. The same prior applies without retraining to point-tracked animal
video and a
robot grasping a soft body. Project page: https://su-terry.github.io/cepheid-website/
Read the full abstract ▾
Computer Vision
MiniDog: Dog-Breed Text-to-image Flow Matching from Pretraining to Fine-Tuning on 4 Consumer GPUs
Reyhaneh Esmailizadeh, Xingjian Leng, Zhanhao Liang et al. (+1 more)
We generally perceive that training text-to-image
diffusion models requires heavy compute: it is true if we aim at a comprehensive frontier model. This understanding prevents people from training their own models when getting started in the field. We develop MiniDog, a framework that enables pretraining
diffusion transformers in 55 minutes and fine-tuning in another 15 minutes on four 3090 GPUs. For data, we use 26k dog images from ImageNet and caption them for pretraining and use 2k images generated by FLUX.1-schnell for fine-tuning. We precompute the image and text features and cache them for efficiency. For architecture, we use E2E-INVAE for its semantic latent space, a small
transformer with 22M parameters, a small Qwen model for text encoding, and in-context conditioning. For training, we use iREPA and an EUPE perception encoder for acceleration. For evaluation, our repository supports FID, HPSv2 score, PickScore, and various VAE reconstruction metrics. The project is open-sourced at https://github.com/Reyhaneesmailizadeh/MiniDog. We do not claim technical novelty.
Read the full abstract ▾
All authors Reyhaneh Esmailizadeh, Xingjian Leng, Zhanhao Liang, Liang Zheng
Computer Vision · Cross-listed from cs.CL
Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards
Wenbo Hu, Shraman Pramanick, Krishna Kumar Singh et al. (+3 more)
Long-horizon
agents have drawn growing
attention for their ability to work autonomously – from hours to days – on economically valuable tasks, a capability essential to automated discovery and recursive self-improvement (RSI). Yet open-source models still struggle on long-horizon tasks, largely for lack of both training data and a working recipe, and scaling both is hard: long, complex tasks are difficult to curate and verify, and their rewards are sparse. We address both. First, we introduce Tev-Gym, the first long-horizon
agent training
dataset with verifiable, dense, progressive rewards – 1,170 verified task environments across 7 domains. It is built on a latent-world-first approach that first constructs a latent world: a complete and correct hidden specification of the task domain, workflow, and expected state after each step. Each task is then progressively rendered from this specification into an
agent-facing environment, enabling controllable complexity and a natural progress metric. Second, we turn this metric into a training recipe: via progress-potential reward shaping (PPRS), we convert progress into a dense per-step reward that improves training efficiency and makes otherwise-intractable tasks trainable. Training with this recipe yields Tev-27B, which achieves substantial gains on the
Agents' Last Exam computing and math sciences domains (ALE-CM), matching GPT-6 Astra and Claude Opus 5 at Pass@5. Despite training only on ALE-related data, it generalizes to the Terminal-Bench (TB) 3.0 and 4.0 subsets, surpassing GLM 5.2 and Gemini 3.7 Flash, and reaches Claude Opus 4.8-level performance on middle-horizon tasks such as TB 2.1.
Read the full abstract ▾
All authors Wenbo Hu, Shraman Pramanick, Krishna Kumar Singh, Nanyun Peng, Kai-Wei Chang, Yong Jae Lee
Computer Vision
PLAYGROUND: Progressive Layout Generation with Render-Grounded Planning
Jaeho Lee, Junhwan Heo, Junghyun Park et al. (+5 more)
Generating editable graphic designs requires arranging visual and textual elements into a coherent composition while preserving their content and editability. Although recent MLLM-based approaches employ iterative generation and visual refinement to reflect the real-world design workflows, their high-level plans are predetermined prior to execution, limiting their adaptability as the design evolves. To address this limitation, we propose PlayGround, a progressive layout generation framework that dynamically adapts to the current rendered canvas. PlayGround establishes a progressive design loop comprising a Progressive Planner, a Layout Generator, and a Visual Verifier, guided by a Structure-Aware Retriever. Given an initial set of unordered elements, the Structure-Aware Retriever initially infers a coarse layout structure to retrieve structurally similar reference designs. Subsequently, at each turn, the Progressive Planner examines the current rendered canvas alongside the remaining elements, forms the next functional group, and plans its composition. The Layout Generator then predicts editable layout attributes, while the Visual Verifier evaluates rendered candidates and provides feedback when refinement is necessary. To train the Progressive Planner with step-wise supervision, we construct a two-stage data generation pipeline that extracts functional groups and turn-level planning trajectories from completed designs. Experiments on Crello and LICA demonstrate the effectiveness of PlayGround over existing methods. Our code and weights will be publicly released.
Read the full abstract ▾
All authors Jaeho Lee, Junhwan Heo, Junghyun Park, WonJun Moon, Eunju Yang, Seungho Jang, Seongchan Kim, Seungryong Kim