Computer Vision · Cross-listed from cs.LG
Kernelized Parameter Sensitivity Atlases for Amortized Conditional Interpretability
Davide Wiest, Ahanaf Hasan Ariq and Abdul Monaf Chowdhury
Gradient-based attribution provides input-specific information about model parameters, but computing it via backpropagation for many samples is expensive, while static importance scores discard input dependence. We derive KPSA (Kernelized Parameter Sensitivity Atlas) from a block decomposition of the empirical neural tangent kernel. It measures normalized parameter-group sensitivity, embeds those sources in an explicit query space, and uses a lightweight kernel to estimate sensitivities for unseen inputs. We derive bounds that separate finite-source from irreducible representation error and give margin-based top-$k$ recovery. This paper studies two interpretability applications. Firstly, KPSA identifies parameter groups with high susceptibility to perturbations across
vision, forecasting, and
protein-modeling setups. Across a fivefold range of parameter budgets ($0.02$-$0.1\%$), its selected groups attain $39$-$85\%$ of the perturbation susceptibility of a first-order backpropagation oracle. In
vision and forecasting, these parameter groups form sparse sensitivity
circuits that extend across depth and vary systematically with the input. Secondly, we use KPSA on an activation-attribution problem for MLP features in
transformers. It retains $81\%$ of the gap between a backpropagation oracle and a constant-sensitivity baseline on
vision while increasing batched query-time throughput from 551 to 2,041 images/s and reducing peak GPU
memory from 4.55 to 0.44GB. On the
protein-modeling task, it recovers $68\%$ of this gap. An ablation of KPSA's kernel aggregation shows that nearest-neighbor
retrieval in KPSA space can remain competitive. The empirics show that compact kernel atlases can store useful input-dependent parameter sensitivity while amortizing repeated gradient attribution, a step toward scalable conditional interpretability. Code can be found at https://github.com/Axym-Labs/kpsa .
Read the full abstract ▾
Computer Vision
Reference-Free Food Volume Estimation from Egocentric Stereo Video
Vivan Kumar, Gautham Vinod and Fengqing Zhu
Current stereo
vision-based approaches for food portion estimation are hindered by the user burden of using a mobile phone to capture multiple images or placing a reference object in the scene for scale calibration. As a step towards passive dietary intake monitoring, we introduce a method that uses stereo
video from a wearable egocentric device to reconstruct food items in true physical scale to accurately estimate volume. By utilizing FoundationStereo for zero-shot stereo matching, SAM 2 for food segmentation, and TSDF fusion over a set of keyframes, our method yields a visually and metrically accurate 3D mesh of any plated food item from a single stereo recording. We leverage the unique offerings of wearables, such as a factory-calibrated stereo baseline and camera poses estimated on-device via visual-inertial odometry (VIO), to perform fully automatic reconstruction without a scale reference object in the scene. We show strong initial volume estimation results across five recorded food items, achieving a mean absolute percentage error of 9.1%.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
StereoEngine: Scaling Stereo Bimanual Robot Data without Stereo Cameras
Jason Chen, I-Chun Arthur Liu, Gaurav Sukhatme et al. (+1 more)
Stereo
vision is increasingly used for robotic manipulation, with a growing body of work incorporating it into
Vision-
Language-Action (VLA) models. However, the scarcity of large-scale, real-world stereo
datasets remains a fundamental bottleneck for scaling such policies. Recent generative monocular-to-stereo methods target settings such as VR content and autonomous driving. No prior work produces realistic synthetic stereo data for robotic manipulation, where stereo supervision has come predominantly from simulation. We propose StereoEngine, which introduces two complementary methods to synthesize stereo data for robotic manipulation. In the first, we warp a monocular (left)
video into the right view using estimated depth and inpaint the disoccluded regions with a
video diffusion model, lifting monocular demonstrations into stereo pairs without training on real stereo. In the second, we render a stereo pair in simulation and jointly generate both photorealistic views, producing consistent left-right data without any real
video as input. StereoEngine outperforms existing baselines in generated stereo quality, and using its synthetic data for policy mid-training or task-specific fine-tuning substantially improves downstream bimanual manipulation. Project website: https://stereoengine.github.io
Read the full abstract ▾
All authors Jason Chen, I-Chun Arthur Liu, Gaurav Sukhatme, Daniel Seita
Computer Vision
Zebra3D: Efficient Hybrid Foundation Models For Scalable 3D Reconstruction
Utkarsh Saxena, Ruijing Yang, Mehdi Rezagholizadeh et al. (+3 more)
Feed-forward 3D foundation models reconstruct scenes from unposed images, but the quadratic cost of global
attention limits their scalability to large view sets. We present Zebra3D, a layerwise sparse-linear hybrid
attention framework that improves the efficiency of pretrained 3D foundation models without training from scratch. Across VGGT, HunyuanWorld-Mirror, and $\pi^3$, we observe a broadly consistent depth-wise pattern: early stages are relatively tolerant to linear replacement, while sensitive middle-to-late stages tend to exhibit more concentrated
attention. Zebra3D exploits this structure by assigning linear-centered
attention to tolerant stages and tile-sparse
attention to sensitive stages, using landmark summaries to preserve unselected context. Closed-form affine fitting enables training-free conversion, while optional post-conversion training combines task supervision with output and relational feature
distillation. Across all three backbones, Zebra3D achieves strong reconstruction and camera-pose accuracy at high
attention sparsity, with training-free variants remaining close to their dense counterparts. End-to-end speedups increase with input view count, reaching up to $7.1\times$ at 1,024 views. After post-conversion training, Zebra3D-VGGT reaches 80.5% RealEstate10K AUC@30 at 94% mean
attention sparsity, within 1.1 percentage points of dense VGGT fine-tuned on the same data and schedule.
Read the full abstract ▾
All authors Utkarsh Saxena, Ruijing Yang, Mehdi Rezagholizadeh, Akash Haridas, Vikram Appia, Emad Barsoum
Computer Vision · Cross-listed from cs.CL
Unsupervised Multimodal Reward Modeling
Yueying Li, Jingxuan Fan, Kianté Brantley et al. (+2 more)
Vision-
language reward models are trained on annotated preference pairs, which are expensive to collect, can disagree, and are ranked by
benchmark accuracy. We build one with no human preference label: a linear head on an intermediate layer of a frozen Qwen3-VL backbone, trained to score a web caption above a perturbed version of the same caption; labeled data enters only to select the layer. The layer is the decisive choice, worth 5.8 points over the last layer, and one recipe transferred without re-tuning beats or at least matches GPT-4o on three
benchmarks at 32B, 6.6 points ahead on VL-RewardBench, from a few thousand parameters at about a thousandth of fine-tuning compute. At an equal pair budget, perturbations teach a frozen head 3 to 5 points more than curated RLAIF-V pairs. Because the backbone is frozen, the objective is convex, and the perturbation sampler specifies the reward, which the
benchmark does not identify: two heads trained on disjoint halves of the same perturbations tie on the
benchmark and differ by 6.4 points in
hallucination judging. Retraining the head on the policy's own verifier-labeled candidates cuts caption
hallucination by 5.1 CHAIR points and lifts image-generation pass rate by 5.4, while scoring lower on every reward
benchmark. Together, an intermediate-layer probe, free same-image perturbations, and a verifier-driven bootstrap turn give a recipe for
multimodal reward modeling that needs no annotator, costs a fraction of fine-tuning, and is specified by the comparisons it is trained on rather than by a
benchmark rank.
Read the full abstract ▾
All authors Yueying Li, Jingxuan Fan, Kianté Brantley, Sham M. Kakade, Hanlin Zhang
Computer Vision
Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation
Marco Stefani, Edoardo Parpaiola, Matteo Bino et al. (+5 more)
High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation across Convolutional Neural Networks (CNNs),
Vision Transformers (ViTs), and
Vision Mamba (ViM). Unlike predominantly qualitative studies, we quantify the
alignment between model
attention and human visual perception using eye-tracking ground truth. We formalize this comparison via a standardized normalization pipeline mapping post-hoc attribution maps and gaze heatmaps onto spatial probability distributions.
Alignment is evaluated across fine-grained classification, medical imaging, and emotion recognition using architecture-specific XAI methods. We measure fidelity using Jensen-Shannon similarity, Chi-square similarity, and Pearson correlation, three complementary
estimators capturing support overlap, mass discrepancy, and structural co-variation. Our results reveal distinct
attention dynamics: CNNs tend to exhibit a more global focus, with spatially more diffuse saliency maps that correlate well with human perception in general contexts, while ViTs demonstrate fine-grained, patch-level
attention, often focused on discriminative microfeatures. The Mamba architecture shows biased behavior: it achieves strong explainability in medical tasks but exhibits divergent intensity distributions in natural images; our analytical characterization of the metrics makes this dissociation explicit, since near-unit support overlap is attained simultaneously with a vanishing linear correlation. We further analyze the effect of backbone freezing and ensemble strategies on model interpretability. The results highlight how architectural and training choices influence the
alignment between model saliency and human visual
attention.
Read the full abstract ▾
All authors Marco Stefani, Edoardo Parpaiola, Matteo Bino, Gianluca Caregnato, Francesco Roncolato, Daniel Fusaro, Loris Nanni, Alberto Pretto
Computer Vision
WoodBR-Cross: A public wood-image dataset and individual-tree-disjoint benchmark for species identification
Daniel Casanova, Alexandre Bahia Gontijo, Arnaldo Candido Junior et al. (+2 more)
Wood-image
benchmarks can yield optimistic estimates when images from the same tree occur in both training and test sets. WoodBR-Cross contains 4,875 transverse-surface images in 38 operational taxonomic classes, linked to 303 distinct trees identified by numeric catalogue witness codes (codigos-testemunho) in the Brazilian Forest Service collection. A species-level subset comprises 3,406 images, 20 classes and 214 trees. Three tasks compare duplicate-aware image partitions for all classes (A), image partitions for the species subset (B1), and individual-tree-disjoint partitions for that subset (B2). In B2, all images from each tree are kept together, so no tree is shared across training, validation or test within a fold. ResNet-50, ConvNeXt-Tiny and ViT-B/16 were evaluated with three outer folds, internal validation and five training seeds. Handcrafted and frozen-feature baselines complement border-only and randomized-label controls. Validation-selected ConvNeXt-Tiny achieved B2 tree-level macro-F1 of 0.8621 +/- 0.0317; frozen DINOv2 features yielded 0.9000 +/- 0.0722. Values are means and standard deviations across three seed-averaged fold means. Results estimate performance on previously unseen trees from this collection.
Read the full abstract ▾
All authors Daniel Casanova, Alexandre Bahia Gontijo, Arnaldo Candido Junior, Silvana Nisgoski, Pedro Luiz de Paula Filho
Computer Vision
Feature Competition and Its Representational Origins in Vision Language Models: Evidence from the Stroop Paradigm
Nurbüke Teker, Rui Xiao, Zeynep Akata et al. (+1 more)
When you ask a
vision-
language model (VLM) a question with an image, it answers your question based on the image. But to act means to choose: saying one thing means choosing to respond not say another. This becomes an issue when the image contains multiple features that equally relate to the question at hand: how would VLMs choose to respond? This problem is precisely studied by the classical Stroop task in psychology: among a set of deliberately designed images whose text and color conflicts (e.g., \textcolor{blue}{RED}), participants are instructed to report the color. We adapt this psychological paradigm to a
benchmark and prompted a range of contrastive and generative VLMs on their behavior. Across models, we observe a persistent bias toward the feature of text in Stroop images: when text and color disagree, models preferentially generate responses centered to the text rather than color. We then opened the black box and examined the source of this bias and why certain features influences the model's behavior more than others. In the
embedding space, we found that textual cues are represented more strongly and separably than color cues. This representational asymmetry predicts intervention success: steering models toward text-related directions is substantially easier than steering them toward color-related directions. Using a classical psychological paradigm, we demonstrate, at both behavioral and representational levels, that distinct features within the same image can exert unequal influence on VLM behavior. Our findings suggest that VLMs assign different degrees of saliency to different features within the same image, highlighting the need to understand how cue-specific saliency shapes their performance in widely used downstream applications.
Read the full abstract ▾
All authors Nurbüke Teker, Rui Xiao, Zeynep Akata, Shuchen Wu
Computer Vision
ForcingBooster: Adaptive Patch-wise Denoising for Fast Autoregressive Video Diffusion
Zekai Gu, Peng Wang, Jiahao Lu et al. (+3 more)
Autoregressive
video diffusion models have become a leading approach to real-time interactive
video generation and
video world modeling. However, existing methods typically allocate computation uniformly across space when generating the next
video chunk, updating all patches with the same denoising schedule and number of function evaluations. This ignores the spatial heterogeneity of
video content: static, simple, or recurring regions are often easier to denoise, while other regions require further refinement or additional context. A uniform budget thus leads to redundant computation on easy patches, whereas reducing the fixed step count can leave difficult patches insufficiently refined. We introduce \methodname{}, a general framework that dynamically allocates denoising steps to individual patches: easy patches stop early to improve efficiency, while difficult patches receive more refinement steps. This design accelerates generation with negligible quality degradation. Our method is compatible with most existing few-step autoregressive
diffusion models, requiring only a lightweight routing module with modest training. Experiments demonstrate that \methodname{} reduces the total number of denoising steps applied across
video patches by about 50% while maintaining visual quality comparable to the four-step baseline.
Read the full abstract ▾
All authors Zekai Gu, Peng Wang, Jiahao Lu, Yongsen Mao, Cheng Lin, Yuan Liu
Computer Vision
MemReg: Streaming Outdoor LiDAR Point Cloud Registration with Hybrid Memory Buffers
Jiuming Liu, Mengmeng Liu, Guangming Wang et al. (+6 more)
Prior outdoor
LiDAR registration works are commonly limited by the pair-wise input paradigm, which neglects temporal correlations inherent within streaming sequences. In this paper, we propose a novel multi-frame outdoor point cloud registration network for streaming
LiDAR scans enhanced with pose-related
memory buffers. The key observation is that long-term temporal
LiDAR sequences can provide rich global contextual information to complete sparse measurements, filter outliers, and address low-overlap problems, thereby boosting registration performance. Specifically, two hybrid
memory buffers are designed, including an implicit
memory feature buffer and an explicit
memory pose buffer, to store and dynamically update pose-related temporal features. Moreover, a novel dynamic history weighting module is developed to adaptively fuse current and history pose-related features. Extensive experiments on three outdoor
datasets, including KITTI, nuScenes, and Apollo-Southbay, demonstrate state-of-the-art performance of MemReg, surpassing all previous pair-wise methods and also multi-frame SLAM systems. Our method also generalizes surprisingly well to multiview indoor registration scenarios with rather competitive performance on 3DMatch, 3DLoMatch, and ScanNet. Code will be released at https://github.com/IRMVLab/MemReg.
Read the full abstract ▾
All authors Jiuming Liu, Mengmeng Liu, Guangming Wang, Lihao Liu, Michael Ying Yang, Yunpeng Zhang, Francesco Nex, Hao Cheng, Hesheng Wang
Computer Vision
Mind the Temporal Gap: Evaluating Video Models for Temporal Understanding in Driving
Shiva Sreeram, Erfan Aasi, Tsun-Hsuan Wang et al. (+2 more)
Modern
video-
language models are increasingly treated as general-purpose
video reasoners, yet it remains unclear whether they robustly capture temporal dynamics in safety-critical settings such as driving. We study this question through controlled ego-motion tasks that require identifying actions from driving
videos. Across
video-
language models, we find that standard prompting, visual prompting, few-shot prompting, and chain-of-thought prompting remain unreliable and highly task- and model-dependent, with persistent class biases, although newer models show substantial task-specific few-shot gains. In contrast, lightweight LoRA finetuning and simple probes trained on intermediate visual features recover strong motion-discriminative performance, indicating that motion-relevant cues are often present inside the models but are not reliably used by the final
video-to-
language pathway. These results suggest that, in driving, current
video-
language models remain unreliable temporal reasoners and often behave more like frame-level, bias-driven recognizers than dependable temporal world models.
Read the full abstract ▾
All authors Shiva Sreeram, Erfan Aasi, Tsun-Hsuan Wang, Alaa Maalouf, Daniela Rus
Computer Vision
Scaling Multi-Teacher Distillation for Digital Pathology
Sofiène Boutaj, Pierre Marza, Varun Belagali et al. (+3 more)
Multi-teacher
distillation allows transferring knowledge from multiple "teacher" networks to a single "student" network. This method is promising in fields such as digital pathology, where many powerful foundation models were proposed recently. However, as we show in this paper, the standard multi-teacher
distillation approach reacts poorly to an increase in the number of teachers used to train a student encoder. We demonstrate that learning teacher-specific representations is a key to scaling in MTD. Importantly, different design choices, such as learnable teacher
tokens, a tailored
attention scheme, additional mixture-of-experts layers and a contrastive loss, are proposed to better learn such teacher-specific representations. We show that our method better scales with respect to the number of considered teachers, allowing us to train compact student encoders, up to 10x more efficient than larger teacher foundation models, while matching their performance and even outperforming them on a large set of tile-level and slide-level tasks. We conduct a thorough empirical validation, evaluating more than 10 foundation models on 39 tasks spanning tile and slide levels. As a result, we release a new collection of strong and efficient foundation models, named OMNI, trained from the knowledge of 10 state-of-the-art foundation models. We finally conduct an analysis of the learned teacher-specific representations, highlighting their complementarity and explaining why they can be easily aggregated at downstream time.
Read the full abstract ▾
All authors Sofiène Boutaj, Pierre Marza, Varun Belagali, Dimitris Samaras, Maria Vakalopoulou, Stergios Christodoulidis
Computer Vision
MIRAGE: Metric Inference and Reliability-Aware Gaussian Estimation from Feed-Forward Vision
Xinzhe Wang, Changjian Jiang, Kaiwen Song et al. (+8 more)
Feed-forward 3D priors enable high-quality reconstruction from image streams but often lack reliable metric scale and suffer from long-term trajectory drift. In contrast,
LiDAR–inertial estimation provides metrically accurate motion but lacks the image-space consistency needed for high-fidelity reconstruction. We propose $MIRAGE$, which anchors feed-forward visual geometry to
LiDAR–IMU metric motion and regulates online pose corrections according to correspondence support and cross-modal agreement, jointly enabling accurate metric pose estimation and high-fidelity Gaussian reconstruction. Experiments on Oxford Spires, M2DGR and INS show that MIRAGE achieves accurate metric trajectory estimation, outperforming COLMAP, a widely used but computationally expensive reference for
vision-only
benchmarks. Motivated by the scarcity of long-horizon monocular
benchmarks for large-scale indoor reconstruction, we further build a
multimodal data-collection platform and use MIRAGE to establish accurate 6-DoF reference trajectories for a new
vision-only
benchmark. We
benchmark recent monocular large-scale reconstruction methods on this
dataset, providing a systematic evaluation of their performance in long-horizon indoor scenes.
Read the full abstract ▾
All authors Xinzhe Wang, Changjian Jiang, Kaiwen Song, Xudong Li, Kerui Ren, Ran Yi, Lizhuang Ma, Chunhua Shen, Linning Xu, Tao Lu, Mulin Yu
Computer Vision · Cross-listed from cs.RO
CASTER: Formalizing Zero-Shot Manipulation from Synthetic Videos as Constrained Optimization
Yinlong Dai, Homanga Bharadhwaj and Dylan P. Losey
Prior research on learning from humans often requires real human demonstrations. But advances in
video generation models offer a new paradigm: we can now
prompt these generators to make synthetic
videos of the task. On the plus side, these synthetic
videos take place in the
robot's current environment, and do not require actual human teaching. However, while the generated
videos appear to show plausible motions, they lack both physical consistency and
robot grounding, and thus the
robot cannot directly roll out the trajectory shown by the
video. In this paper we seek to formalize the process of going from synthetic
video generation to real-world
robot trajectories for zero-shot manipulation tasks. We recognize that this problem is an instance of constrained optimization. The object trajectories extracted from the synthetic
video provide an initial guess at the correct motion, and the
robot must refine these trajectories (for instance, within a digital twin) while enforcing physical feasibility and semantic consistency. Our optimization-based formalism provides insights into the
video generation properties necessary for success. Specifically, we show that the
video generation must probabilistically seed the search within a semantic basin of attraction where the trajectory can be locally modified without losing its task
alignment. Our analysis further reveals that state-of-the-art approaches can be viewed as instances of our unified formalism with key simplifications. In our experiments we remove these simplifications to more closely match the underlying optimization, leading to more robust zero-shot trajectories from synthetic
videos and improving average task success by 34.1% compared to the best-performing baseline. See
videos at our project website: https://collab.me.vt.edu/CASTER
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Fly-by-Code: Embodied Coding Agents for Aerial Manipulation with Active Visual and Physical Feedback
Jaewoo Lee, Jeongyeon Seo, Sihyun Cho et al. (+10 more)
Embodied coding
agents construct
robot policies from natural-
language instructions by generating executable programs and refining them from execution feedback. This refinement assumes that execution feedback reveals whether the task succeeded and, after a failure, provides sufficient evidence to guide program revision. Aerial manipulation violates this assumption in two ways. Payload constraints restrict onboard camera coverage and leave task-critical outcomes outside the
robot's view (partial observability), while strong dynamic coupling and limited control margins make physical interactions fragile (limited interaction robustness) and leave failure causes unresolved. Re-examining the same feedback cannot recover observations that were never collected or separate failure causes that the available evidence does not distinguish. To acquire this missing evidence, we introduce Active Feedback (AF), an active "View and Probe" framework. When execution feedback leaves unresolved questions, the
agent synthesizes a separate diagnostic program that runs in simulation to inspect unseen outcomes from informative viewpoints (View) or test hypotheses about failure causes through targeted physical interactions (Probe). Across six simulated aerial manipulation tasks with 60 trials per condition, AF reduces false completion declarations from 25 to 5 and raises final task success from 31.7% to 75% compared with terminal feedback under the same coding-
agent configuration and task-execution limit. Using shared
robot APIs, task programs refined with AF in simulation run on a physical aerial manipulator without task-program source-code modification and complete multi-stage manipulation tasks in the real world. These results show that executable diagnosis fills gaps in execution feedback for embodied coding
agents, improving both completion assessment and program refinement in aerial manipulation.
Read the full abstract ▾
All authors Jaewoo Lee, Jeongyeon Seo, Sihyun Cho, Gyeongrak Choe, Yutong Wang, Bavin Saravanan, Jia-Bin Huang, Furong Huang, Sebastian Scherer, Guanya Shi, H. Jin Kim, Seungjae Lee, Dongjae Lee
Computer Vision · Cross-listed from cs.RO
Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration
Jusuk Lee, Sungha Kim, Yeonsoo Park et al. (+8 more)
While learning dexterous manipulation from a single human
video offers a promising alternative to costly
robot demonstrations, many recent methods predominantly imitate demonstrated motions. Such strict motion matching often limits generalization to initial object poses, goal poses, and grasps not shown in the
video. Alternatively, discovering a policy via reinforcement learning (RL) allows for broad generalization, but without prior guidance, it struggles with high-dimensional exploration in complex, multi-stage tasks. To address these coupled generalization and exploration challenges, we present Dex-One2Many, a real-to-sim-to-real framework that learns a generalizable dexterous manipulation policy from a single human
video. Our key insight is to abstract the
video into sequential scene graphs that guide RL, enabling efficient exploration while preserving broad generalizability. The graphs serve as generative constraints for sampling diverse reset states and provide dense rewards for each stage. Because the graphs constrain relations rather than exact poses, these reset states cover object poses and grasps beyond the
video, while initializing each stage from them with dense rewards keeps exploration short and guided. Trained entirely in simulation, Dex-One2Many transfers zero-shot to a real multi-fingered hand. Across five tool-use and manipulation tasks, Dex-One2Many exceeds baselines by 6.5% in seen configurations, while its robust generalization widens this gap to 71% in unseen scenarios.
Read the full abstract ▾
All authors Jusuk Lee, Sungha Kim, Yeonsoo Park, Jonguk Cheon, Yoonkyo Jung, Yongjun You, H. Jin Kim, Jia-Bin Huang, Furong Huang, Youngseok Jang, Seungjae Lee
Computer Vision
Rubric-CEPR: Self-Evolving Image Editing via Reward-Verified Self-Distillation
Ritesh Thawkar, Shubham Patle, Shravan Venkatraman et al. (+1 more)
Instruction-guided image editors have become highly capable, yet improving them further still depends on human-edited training pairs or external reward models. Such supervision is costly to obtain and can reward plausible failures: a realistic output may leave the requested change undone or alter content that should be preserved. In this work, we strive to improve a pretrained image editor using only its own generations, without human-edited targets or an external training-time reward model. To this end, we propose a self-evolving framework, named Rubric-CEPR, that verifies the editor's own samples with its internal representations through a rubric-augmented Contrastive Edit-Preservation Reward (CEPR). A Planner proposes structured edit instructions from unlabeled images, the Editor samples multiple candidate edits, and a frozen Critic scores each candidate with decomposed rubric checks for edit realization, removal of the old state, and content preservation, using features already exposed by the editor. Non-compensatory gates reject infeasible candidates, and the best verified candidate is distilled into the editor through lightweight adapter training. On Qwen-Image-Edit, Rubric-CEPR improves ImgEdit from 4.36 to 4.60 (+5.5%), with a +24.9% gain on object isolation, and transfers to GEdit-Bench and Complex-Edit. The same procedure also improves Step1X-Edit by +7.8% on ImgEdit. We hope our approach will serve as a solid baseline for image editors that improve themselves from their own verified samples. Our code is publicly available at https://github.com/riteshthawkar/Rubric-CEPR
Read the full abstract ▾
All authors Ritesh Thawkar, Shubham Patle, Shravan Venkatraman, Rao Muhammad Anwer
Computer Vision · Cross-listed from cs.RO
DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training
Junyan Li, Ruizhi Li, Yu Liu et al. (+6 more)
We present DreamTrue, a multi-view, cross-embodiment
robot world model for action-faithful and physically plausible
video prediction. Training such a model on existing
robot datasets faces two obstacles: imprecise calibration can impair action following, while limited coverage of unsuccessful interactions can bias predictions toward successful outcomes. To improve action following across embodiments, we render action trajectories into image-space conditions and introduce offline
geometric calibration to align these conditions with the target
videos. To broaden interaction coverage, we introduce counterfactual post-training, modifying recorded action trajectories and generating future
videos under a wider range of actions and contact configurations. To provide feedback on these predictions without paired ground-truth futures, we construct a human-annotated
video dataset covering
robot, object, and interaction defects and use it to train an embodied
video reward model. Its scores guide reinforcement-learning post-training toward more physically plausible interaction outcomes. On AgiBot, DreamTrue attains state-of-the-art action following, while reducing the human-assessed interaction defect rate from from 48.12% to 6.25%. Notably, our model ranks first in the world model track of the AgiBot World Challenge 2026. The project page can be found at https://brave-eai.github.io/DreamTrue.
Read the full abstract ▾
All authors Junyan Li, Ruizhi Li, Yu Liu, Xiangshuo Liu, Mingchao Sun, Hongyu Pan, Mu Xu, Lue Fan, Zhaoxiang Zhang
Computer Vision
What 30,000 Hours of Ego-centric Video Does Not Teach
Jiahua Dong, Anurag Bagchi, Yash Jangir et al. (+7 more)
World models offer a promising alternative to physics-based simulators, yet remain far from practical deployment. We ask how far scaling ego-centric human
video takes them, using a
dataset of 30,000 hours spanning over 1,000 scene types and 14,000 contributors. Rather than relying on opaque downstream metrics, we directly evaluate
agent and object-interaction fidelity on a challenging out-of-distribution
benchmark. Increasing training data by 100x improves both, but unevenly: the
agent is modeled well, while object fidelity remains far lower and improves slowly. We show that the
agent gains need not come from data, and a careful visual conditioning design saturates fidelity with a fraction of it, which lets us measure object fidelity on its own and discover its saturation point. We then introduce a supervision scheme that shifts capacity from scene appearance toward object dynamics, improving object fidelity though a substantial gap remains. Finally, our conclusions transfer to downstream humanoid modeling. Overall, our results suggest that scaling ego-centric data brings
agent modeling close to its limit while leaving its effects on the world far behind, and that closing this gap will depend on how models are trained, not only on how much data they see.
Read the full abstract ▾
All authors Jiahua Dong, Anurag Bagchi, Yash Jangir, Muhammad Zubair Irshad, Sergey Zakharov, Martial Hebert, Homanga Bharadhwaj, Yu-Xiong Wang, Vitor Campagnolo Guizilini, Pavel Tokmakov
Computer Vision
OuroWorld: Bringing Any 3D World Alive as Diverse, Endlessly Looping 3D Cinemagraphs
You-Zhe Xie, Ting-Wei Chou, Yu-Hsuan Li et al. (+3 more)
Recent 3D world models generate photorealistic, explorable scenes that remain frozen in time. OuroWorld is a mask-free framework that turns any static 3D Gaussian Splatting scene into a 3D cinemagraph: a dynamic scene with vivid, diverse motion looping seamlessly from any viewpoint. A
vision-
language model infers plausible dynamics and guides a
video model to synthesize a reference
video, which we lift and complete into multi-view
videos. To learn from this imperfect supervision, we propose Inconsistency-Robust Periodic 4DGS: a
Fourier-series deformation field guarantees looping by construction, while a Grounded Drift Field anchored at the reference view absorbs cross-view inconsistency. Unlike prior Eulerian methods limited to fluid-like motion, we capture general deformation, object motion, and illumination change. We introduce a ground-truth-free evaluation covering vividness, naturalness, loop seam coherence, and scene quality. On 39 reconstructed and generated scenes, OuroWorld outperforms all baselines and wins 70.8%-99.0% of user-study comparisons. Project page: https://ouroworld.userwei.com
Read the full abstract ▾
All authors You-Zhe Xie, Ting-Wei Chou, Yu-Hsuan Li, Kaipeng Zhang, Zhixiang Wang, Yu-Lun Liu
Computer Vision
WorldGuide: Goal-Directed Video World Model for Procedural Task Execution
Ankan Deria, Komal Kumar, Hisham Cholakkal et al. (+2 more)
Video generators and
video-based world models can synthesize plausible visual trajectories, but long-horizon procedural tasks require generation to adapt to what has actually been produced. A model must determine the next action from its generated state, execute that action, and recognize when the task is complete. Open-loop generation cannot adapt to execution outcomes, while existing closed-loop systems often rely on pretrained executors or indirect verification. This leaves a gap between deciding an action and successfully realizing it. We formulate procedural
video generation as closed-loop task execution in visual world space and introduce WorldGuide. Given only an initial image and a task goal, WorldGuide predicts an atomic action, generates its corresponding
video clip, and uses the generated result to select the next action or terminate. The Planner and Executor are trained on the same step-level procedural demonstrations: the Planner learns to predict the next atomic action or task completion from visual progress, while the Executor is directly trained to realize the predicted actions. Hierarchical visual
memory maintains state across long-horizon execution with bounded history
token cost. Due to the lack of step-level action-
video supervision for joint planner-executor training, we introduce WorldGuide Bench: approximately 59K step-annotated
videos across 245 tasks and 27 procedural categories. WorldGuide achieves a 33.33% Task Success on WorldGuide-Bench, compared with 29.90% for the strong recent
video model MiniMax-H3, even though MiniMax-H3 receives reference action plans, and achieves 47.69% on VideoCraft-Bench compared with 32.73% for MiniMax-H3 under goal-only conditioning. These results demonstrate the importance of coupling planning with learned execution for goal-directed procedural
video generation.
Read the full abstract ▾
All authors Ankan Deria, Komal Kumar, Hisham Cholakkal, Fahad Shahbaz Khan, Salman Khan
Computer Vision
OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning
Zhongyu Yang, Jiale Tao, Ruitao Chen et al. (+9 more)
Multimodal large
language models (MLLMs) are rapidly evolving toward continuous audio–visual
reasoning, creating an urgent need for evaluations that expose their capability limits. Audio–visual captioning is an ideal diagnostic task, yet current
benchmarks face a coupled trade-off: whole-caption scores provide coverage without localization, local probes provide localization without coverage, and unconstrained LLM judges introduce instability. We introduce OmniCapBench (Omni-
Video Caption
Benchmark), a
benchmark that reframes audio–visual caption evaluation as a deep-structured diagnostic framework. OmniCapBench shifts the prediction target from free-form text to sets of atomic, verifiable evaluation units across three tracks: entity references, visual shots, and audio events, enabling reliable scoring with deterministic constraint checks and localized LLM-based semantic comparisons. With 786 densely annotated
videos, OmniCapBench effectively distinguishes MLLM perception errors, including temporal grounding failures, identity drift, cross-modal misalignment, and hallucinated descriptions. Evaluating frontier MLLMs reveals strong local perception but weak long-horizon audio–visual
reasoning, particularly in identity drift and cross-modal misalignment, providing a fine-grained roadmap for omnimodal development.
Read the full abstract ▾
All authors Zhongyu Yang, Jiale Tao, Ruitao Chen, Zuhao Yang, Yingfang Yuan, Xueliang Zhao, Auden, Kai Wang, Shuai Shao, Biao Wang, Steve Yves, Qinglin Lu
Computer Vision · Cross-listed from cs.HC
Hybrid Cinematography: Previsualizing and Managing Hallucination Risk in Generative Video Reshooting
Nhan Tran, Neal Wadhwa, Abe Davis et al. (+1 more)
On a film set, the camera move is committed during a take. Generative
video reshooting lets filmmakers change it afterward, but may require hallucinating unrecorded content, a gap sometimes discovered only after leaving the set. We present Hybrid Cinematography, a workflow that bridges physical capture and generative reshooting to manage
hallucination risk while filmmakers can still act on it. Using an editable 3D shot plan and a proxy of the take, our previsualization evaluates
hallucination risk in real time. Seeing where the take lacks support, filmmakers can iteratively adjust the plan, explore moves that balance capture and generation, shoot guided pickups, or knowingly accept
hallucination. We demonstrate the workflow through a mobile augmented reality application for on-set planning, capture, and review, and an offline pipeline for existing
video. A study with experienced filmmakers reveals how previsualizing risk informs camera decisions and exposes tensions between creative intent and generative
hallucination.
Read the full abstract ▾
All authors Nhan Tran, Neal Wadhwa, Abe Davis, Stefan Stojanov
Computer Vision · Cross-listed from cs.AI
BrickBench: Evaluating Agentic Brick Design
Peter Kulits, Yiqing Xu, R. Kenny Jones et al. (+2 more)
We propose BrickBench, a
benchmark for agentic text-conditioned LEGO-set design. Given a
prompt, an
agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. To do so, it must select parts from a discrete library and reason jointly about local and global constraints. We score validity,
alignment, and design across three settings that vary in scale and part availability. We provide BrickAgent, an environment for coding
agents to construct, inspect, and validate their designs. We find that leading
agents largely satisfy verifiable physical and semantic requirements, but fall short of human designs. We release our
benchmark and environment at http://www.brickben.ch
Read the full abstract ▾
All authors Peter Kulits, Yiqing Xu, R. Kenny Jones, Cordelia Schmid, Jiajun Wu
Computer Vision
VersaCamVLA: Camera-Configurable VLA Policies for Robotic Manipulation
Boyao Han, Chen Shi, Jingjing Qian et al. (+2 more)
Vision-
Language-Action (VLA) models have emerged as powerful foundations for robotic manipulation, but their reliance on fixed camera configurations during training makes them brittle to changes in camera count or pose during deployment. To overcome these limitations, we propose VersaCamVLA, a camera-configurable framework that decouples camera-set representation from action learning. VersaCamVLA learns a unified scene-
token interface that maps an arbitrary, variable set of posed RGB views into fixed-size latent scene
tokens. This is achieved via multi-signal target-view prediction and Wrist-Augmented Pose Sampling (WAPS), which leverages natural wrist-camera motion for free pose diversity. At deployment, a lightweight spatial encoder injects these compact scene
tokens into a pretrained base VLA as a supplementary visual condition, requiring no explicit 3D sensing or novel-view rendering. Experiments on RoboTwin, LIBERO, and a real-
robot platform demonstrate that VersaCamVLA consistently outperforms prior VLA methods and direct multi-view baselines, maintaining robust performance across varying camera counts and unseen camera poses.
Read the full abstract ▾
All authors Boyao Han, Chen Shi, Jingjing Qian, ZhuoTan Tian, Li Jiang
Computer Vision
One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts
Adrian Bulat, Yassine Ouali and Georgios Tzimiropoulos
In this work, we show that a single
Transformer block, applied recurrently, can match the accuracy of a full-depth
vision encoder at comparable inference FLOPs without intermediate feature
distillation. reViT restores depth-specific transformations by representing the FFN at each recurrent depth as a convex combination of a small shared expert bank. A continuous normalized-depth coordinate programs this mixture, defining a resampleable trajectory through FFN parameter space. We evaluate this design in two regimes: supervised ImageNet-1k training and
distillation from a DINOv2 teacher. Across both regimes, controlled adaptations identify weight-space merging as the strongest tested MoE family at a matching one-FFN budget, ahead of the
token-dispatch and output-mixture alternatives. Trained from scratch, reViT-B/16 attains DeiT III accuracy with about 70% fewer stored parameters. An 8-experts model distilled using only the teacher's output features retains nearly all of its DINOv2 teacher's linear-probe accuracy and transfers across classification, segmentation, and depth prediction. Elastic-depth training allows one checkpoint (trained model) to operate at multiple tested depths by resampling the same normalized coordinate interval. For fixed-depth deployment, the recurrent block can be materialized as a conventional dense graph, removing online routing and merging without changing the one-FFN-per-depth compute but expanding deployment storage.
Read the full abstract ▾
Computer Vision
LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation
Suhwan Cho, Yonwoo Choi, Soongjin Kim et al. (+2 more)
Generating an egocentric
video from a single exocentric recording is a challenging case of novel view synthesis, as the two cameras share little overlap and much of the target view is unobserved. Current state-of-the-art methods reconstruct the scene explicitly by estimating depth, lifting the
video into a point cloud, and re-rendering it from the egocentric camera to condition a
video diffusion model. This deterministic mapping assigns each pixel to a single reprojected location, which preserves texture but translates depth errors into misplaced content. We ask what a
video diffusion model should receive as its condition and propose a lifting-free answer: a learned view synthesizer, an LVSM-style
transformer fine-tuned to render the egocentric view directly without depth, point clouds, or reprojection, resolving cross-view correspondence internally. In contrast, its probabilistic mapping averages each region over candidate source locations according to a learned correspondence distribution, preserving structure while fine texture is averaged away. We argue that this trade-off suits a
diffusion generator, whose denoising training excels at restoring detail, so an effective condition should prioritize structural
alignment over sharpness. This distribution's concentration also yields a per-region confidence, used both to mask low-confidence regions and to guide the generator toward high-confidence areas during early layout-forming denoising steps. Our approach consistently outperforms the state-of-the-art explicit pipeline and generalizes to other
datasets without retraining. The synthesizer thus supplies view structure, and the
diffusion model its detail.
Read the full abstract ▾
All authors Suhwan Cho, Yonwoo Choi, Soongjin Kim, Jicheol Park, Taegyu Lim
Computer Vision
FastBench: Can Streaming VLMs Perceive High-Dynamic Real-World Streams?
Yuxuan Hu, Weikang Shi, Yang Bo et al. (+10 more)
Streaming
Video Large
Language Models (VLMs) enable continuous
video understanding, yet existing
benchmarks focus on low-dynamic scenarios. Under bounded context budgets, models must balance temporal history, spatial resolution, and temporal granularity; sparse sampling at 1–2 FPS misses fast events. We introduce FastBench to evaluate high-dynamic perception in real-world
video streams. Its trajectory-grounded pipeline combines QA generation from high-FPS clips, filtering of questions answerable at 2 FPS, answer verification using SAM3 and CoTracker3 trajectories, and three rounds of human inspection. FastBench contains 306 QA pairs across eight domains, six capabilities, and forward, instant, and backward temporal scopes, with human-annotated evidence intervals. We also present ProactiveFrame, a training-free baseline that adjusts incoming frame rates through text
tokens. A dual-tier sliding window retains recent high-FPS observations while downsampling older ones into sparse history. Experiments reveal substantial limitations: the strongest model, Gemini-3.5-Flash, scores only 50.7%. Denser sampling improves Qwen3-VL-8B from 32.9% at 2 FPS to 44.6% at 24 FPS, but gains saturate as history is compressed. ProactiveFrame outperforms sparse uniform sampling by 5.4 and 1.5 percentage points, yet remains well below oracle-guided focusing, showing that current VLMs struggle to determine from the stream alone when finer temporal perception is needed. FastBench provides a testbed for high-dynamic streaming
video understanding. Code and data: https://github.com/Ashone3/FastBench.
Read the full abstract ▾
All authors Yuxuan Hu, Weikang Shi, Yang Bo, Xudong Lu, Xintong Guo, Shuhan Li, Yuyang He, Huankang Guan, Peiwen Sun, Yunqiao Yang, Wenbo Li, Rui Liu, Hongsheng Li
Computer Vision
Pumpire: Unified Benchmark for Metric Distance Estimation
Siyu Chen, Zehan Wang, Jiayang Xu et al. (+6 more)
We present Pumpire, a unified
benchmark for evaluating metric point-pair distance estimation capability of both image- and
video-level 3D foundation models, with or without depth priors. In contrast to previous approaches that normally evaluate depth and camera intrinsics separately or evaluate point-clouds with
geometric similarity metrics, which cannot directly reflect models' point-to-point distance estimation capability, Pumpire directly assesses point-to-point distances from the reconstructed geometry. To this end, we collect a large-scale and diverse
dataset (pumpire-6k) comprising 100 real-world scenes, each annotated with physically measured point-pair distances and containing 64 frames, for a total of 6,400 frames. Building on this
dataset, we establish a holistic evaluation protocol that covers both image- and
video-level 3D foundation models and enables direct assessment of point-pair distance errors and cross-setting comparison. We conduct extensive experiments across 29 baseline configurations of representative 3D foundation models and provide a comprehensive analysis of the results. By offering this
benchmark, we target the more fundamental ability to perceive and estimate physical scale in the reconstructed 3D space, which prior evaluation protocols have largely overlooked. The project page can be found at https://pumpire.github.io/
Read the full abstract ▾
All authors Siyu Chen, Zehan Wang, Jiayang Xu, Yihan Wu, Jialei Wang, Junming Chen, Ziang Zhang, Yutong Ying, Zhou Zhao
Computer Vision
Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matching
Luping Liu, Bingyi Kang, Yifan Wang et al. (+1 more)
Dense correspondence matching has historically been bounded by simplifying spatio-temporal priors, such as smooth motion and rigid geometry. While effective for classical tasks, these assumptions break down in image editing and reference-guided generation (IEG), where transformations can preserve visual identity while breaking physical continuity. To establish identity-preserving correspondence across such transformations, we introduce FreeMatching, a generalizable framework combining generative and semantic foundation representations with heterogeneous supervision from classical
datasets, tracked
videos, and synthetic scenes. Teacher-guided iterative refinement further improves correspondence in IEG without dense correspondence annotations. Experimentally, a single FreeMatching model substantially improves correspondence quality on challenging IEG image pairs while retaining competitive performance on classical
benchmarks. Furthermore, we demonstrate its utility as a quantitative metric for evaluating identity preservation, with scores that correlate with human judgment. The code is available at https://github.com/luping-liu/FreeMatching.
Read the full abstract ▾
All authors Luping Liu, Bingyi Kang, Yifan Wang, Dong Xu