Computer Vision
VLMs are Good Teachers for Video Reasoning via Adaptive Test-Time Optimization
Junhao Cheng, Liang Hou, Tianxiong Zhong et al. (+4 more)
The recent "
Reasoning with
Video" paradigm utilizes
Video Generation Models (VGMs) to generate temporally coherent visual trajectories to complete
reasoning tasks. Although state-of-the-art VGMs excel at visual quality, they often struggle to understand and follow task-specific rules, leading to logical failures across diverse
reasoning scenarios. Existing efforts try to utilize
Vision-
Language Models (VLMs) as problem pre-solvers to produce or refine textual guidance for the VGM. However, textual descriptions fail to capture intricate spatiotemporal details, and VGMs often struggle to faithfully execute fine-grained or long-tail instructions even with a valid plan. While VLMs struggle as solvers, they possess strong perception capabilities to evaluate process-constraint satisfaction and final-goal achievement. Leveraging this strength, we introduce a paradigm shift that transitions the role of VLMs to "teachers". Specifically, a VLM teacher extracts task-specific rules to formulate differentiable rewards, guiding a VGM Reasoner via test-time online optimization of a lightweight LoRA module. This strategy enables adaptive test-time optimization and extends the
reasoning capabilities beyond the VGM's intrinsic boundaries. Evaluations on symbolic (VBVR-Bench) and general-purpose (RULER-Bench)
video reasoning benchmarks show that the proposed method yields a 16.7-point average performance gain, outperforming the VLM-as-Solver paradigm (+0.4 points) and Best-of-N scaling (+2.2 points) by a large margin at comparable test-time cost. These findings reveal that integrating VLMs as test-time teachers offers a promising paradigm for achieving generalizable
video reasoning. Project Page: https://VLM-as-Teacher.github.io/
Read the full abstract ▾
All authors Junhao Cheng, Liang Hou, Tianxiong Zhong, Xin Tao, Pengfei Wan, Kun Gai, Jing Liao
Computer Vision
LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation
Qixin Hu, Shuai Yang, Wei Huang et al. (+2 more)
Autoregressive (AR)
video diffusion enables variable-length synthesis, but long-horizon generation often suffers from accumulated errors and identity drift. For efficiency, existing methods commonly adopt sliding-window
attention during generation. This creates an irreversible generation trajectory: once the active window accumulates appearance errors, subsequent generations can only condition on this degraded trajectory and drift further away. We address this limitation by formulating long
video generation as a
retrieval-augmented generation (RAG) problem. Rather than relying solely on the recent window, we treat previously generated latents as a dynamic, searchable history. We propose LongLive-RAG, a general
retrieval framework for AR
video generation. At each new block, LongLive-RAG uses a query
embedding to retrieve relevant historical latents. This lightweight
retrieval step adds only a small overhead relative to generation and lets the generator condition on non-local context instead of only the recent window. To make
retrieval more discriminative, we introduce the Window Temporal Delta Loss that suppresses redundant local similarity and encourages
embeddings to capture meaningful temporal changes. Together, these components help reduce error accumulation caused by sliding-window
attention. Experiments across multiple AR backbones and generation lengths show improved long-
video quality and the best average VBench-Long rank. To our knowledge, among open-ended AR long
video generation methods, LongLive-RAG is the first to formulate self-generated latent history as content-addressable
retrieval memory. Code is available at https://github.com/qixinhu11/LongLive-RAG.
Read the full abstract ▾
All authors Qixin Hu, Shuai Yang, Wei Huang, Song Han, Yukang Chen
Computer Vision
Modeling Depth Ambiguity: A Mixture-Density Representation for Flying-Point-Free Depth Estimation
Siyuan Bian, Congrong Xu and Jun Gao
Despite advances in depth estimation, flying points remain a persistent failure mode: near object boundaries, depth
estimators often predict spurious 3D points in the empty space between foreground and background surfaces. We trace this artifact to a standard modeling choice: assigning each pixel a single depth hypothesis. At boundaries, a pixel can straddle a foreground and a background surface, so its true depth is ambiguous between the two. A model that predicts a single depth cannot keep both possibilities, so training instead pulls the prediction toward an intermediate depth that lies on neither surface. We address this with MDA, a mixture-density representation that lets the model predict multiple depth hypotheses and their associated probabilities for each pixel. Near boundaries, different hypotheses can align with different surfaces, and the decoded depth is selected from one of these hypotheses rather than placed in the empty space between them. Across different architectures, MDA substantially improves boundary reconstruction over each base model and largely removes flying-point artifacts even under severe input blur, while adding negligible runtime overhead. The same mixture-density framework naturally extends to transparent objects, where it predicts multiple depth layers at transparent pixels, and to sky regions, where a dedicated component separates the unbounded sky from finite-depth regions, producing flying-point-free skylines.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
AFUN: Towards an Affordance Foundation Model for Functionality Understanding
Zhaoning Wang, Yi Zhong, Jiawei Fu et al. (+2 more)
Affordance understanding bridges visual perception and physical action, serving as an explainable interface for
robot manipulation in open and unstructured real-world environments. Yet, building an affordance foundation model that not only understands where and how the interaction should happen, but also generalizes across diverse environments, objects, and tasks, remains a long-standing research challenge. Existing methods typically address only part of this challenge, either localizing task-relevant regions without specifying executable motion, or predicting motion but with limited scalability. In this paper, we present ourmodel, a step towards an affordance foundation model for functionality understanding. From a single RGB-D observation and a
language task description, ourmodel predicts a task-conditional functional mask (where to interact) and a 3D post-contact motion curve (how to interact). To support open-world generalization, we build a large-scale standardized data pipeline that converts heterogeneous
robot, human, simulation, and real-world scan data into a shared affordance schema with
language, masks, and object-centric 3D motion labels. We evaluate ourmodel from three aspects: for affordance
segmentation, ourmodel outperforms all baselines by a large margin across 8 test sets from 4
benchmarks, improving mean gIoU/cIoU by +23.9/+26.3; for contact-point prediction, it predicts substantially more accurate points, with a 12.7–61.3% hit-rate gain over the best baseline; and for 3D motion, it achieves the best performance on all three test sets. ourmodel can be deployed for real-world
robot manipulation without finetuning for
robot embodiment or using task-specific heuristics, demonstrating the ability to adapt to open-world affordance tasks. Project page: https://www.zhaoningwang.com/AFUN
Read the full abstract ▾
All authors Zhaoning Wang, Yi Zhong, Jiawei Fu, Henrik I. Christensen, Jun Gao
Computer Vision
LL-Bench: Rethinking Low-Level Vision Evaluation in the Era of Large-Scale Generative Models
Lu Liu, Huiyu Duan, Chenxin Zhu et al. (+6 more)
Large-scale generative models have demonstrated remarkable capabilities across image generation and editing tasks. However, their performance in low-level
vision tasks, which require pixel-wise control, remains insufficiently studied. To address this gap, we introduce LL-Bench, a comprehensive
Benchmark for evaluating the capabilities of large-scale generative models on Low-Level
vision tasks. The
benchmark comprises 2,469 real-world degraded images covering 16 low-level degradation tasks, and 28,919 restored images produced by 10 state-of-the-art large-scale generative models and 21 conventional restoration models, which are annotated with 152,020 expert-level pairwise human preferences and 28,334 quality scores. Built upon LL-Bench, we present a systematic diagnosis that reveals the performance boundaries and unique failure modes of large-scale generative models across diverse low-level
vision tasks, compared with conventional representative restoration approaches. Moreover, we investigate the effectiveness of current quality evaluation metrics on LL-Bench, which exhibit significant discrepancy with human ratings. To better align restored-image quality assessment with human preferences, we further propose LL-Score, an MLLM-based evaluator that captures both restoration quality and
hallucination existence. Extensive experiments demonstrate that LL-score not only outperforms existing image quality assessment metrics, but also serves as a promising reward model for training generative models on low-level
vision tasks.
Read the full abstract ▾
All authors Lu Liu, Huiyu Duan, Chenxin Zhu, Jintong Lu, Haoyun Jiang, Liu Yang, Qiang Hu, Guangtao Zhai, Xiaoyun Zhang
Computer Vision
Improving Combined Detection and Classification of TEM Defects via Mask-Conditioned Latent Diffusion Augmentation
Ni Li, Nuohao Liu, Ryan Jacobs et al. (+5 more)
Analyzing microstructural defects in transmission electron microscopy (TEM) images, particularly in irradiated metal
alloys, is often limited by the availability of high-quality, labeled data. To address this, we introduce a generative data augmentation approach using a mask-conditioned latent
diffusion model (LDM) for synthesizing realistic TEM images with controllable, automatically labeled multi-class defect masks. Without requiring manual annotations for generation, our method enables the creation of synthetic image-mask pairs by sampling distributions learned from experimental masks. These generated data were used to augment small experimental
datasets of varying sizes (10, 50, and 100 labeled experimental images) to train a Mask Regional Convolutional Neural Network (R-CNN) model for defect detection and classification. Our results show that generative augmentation yields small overall model performance improvements, with up to a 0.02 gain in the harmonic mean of detection and classification F1 scores. However, we also find that the relative contributions to detection and classification improvement depend on the specific train/test data split. These findings highlight the potential of targeted generative models to enhance deep learning performance in data-scarce microscopy-based image quantification tasks.
Read the full abstract ▾
All authors Ni Li, Nuohao Liu, Ryan Jacobs, Ajay Annamareddy, Maciej P. Polak, Kevin Field, Izabela Szlufarska, Dane Morgan
Computer Vision
Why Not Hyperparameter-Friendly Optimisation? A Monotonic Adaptive Norm Rescaling Approach For Long-Tailed Recognition
Shuo Zhang, Chenqi Li and Tingting Zhu
Long-tailed recognition poses a significant challenge for deep learning. The two-stage decoupling paradigm, which separates representation learning from classifier retraining, offers a promising solution. During the classifier retraining stage, adaptive norm rescaling is a popular technique. It adjusts the per-class weight norms via parameter regularization, which inevitably introduces hyperparameters. However, many studies report that long-tailed recognition is sensitive to these hyperparameters, as their setup significantly impacts performance. In this paper, we first provide a class-conditional distribution perspective to support norm rescaling methods. Furthermore, we propose a simple but effective approach called Self-Adaptive Monotonic Normalization (SAMN). SAMN avoids the need for parameter regularization. It directly enforces monotonicity on per-class weight norms using the Pool Adjacent Violators
Algorithm, making the method hyperparameter-friendly. SAMN is a universal strategy that integrates seamlessly with other methods for enhanced performance. Experiments on
benchmark datasets demonstrate that our method significantly boosts long-tailed recognition performance, often achieving state-of-the-art results.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
FigSIM: A Dataset for Fine-grained Suicide Severity and Figurative Language in Suicide Memes
Liuliu Chen, Elise R. Carrotte, Brian E. Chapman et al. (+2 more)
Suicide memes are memes used to express suicide-related thoughts or comment on suicide-related issues. Suicide memes are increasingly common on social media, yet remain poorly understood and potentially harmful. There is an urgent need to better understand their characteristics and to develop appropriate content moderation strategies that limits users' exposure to potentially harmful content. Currently, the absence of annotated
datasets of suicide memes remains a key barrier to developing and evaluating automated moderation approaches. In this paper, we introduce FigSIM, the first
dataset designed for fine-grained analysis of suicide memes. The
dataset consists of 1049 memes, each annotated for (1) fine-grained suicide severity levels, (2) figurative phenomena (e.g., metaphors), and (3) suicide-related content (e.g., suicide method depiction). We
benchmark 16 unimodal and
multimodal models across three tasks: figurative
language, suicide severity, and suicide-related content detection. Overall, FigSIM demonstrates that suicide memes pose unique challenges for both modeling and content moderation. Analysis revealed biases, such as underprediction of higher suicide severity levels, especially for figurative memes. The
dataset (including splits used for analyses) is publicly available. Content Warning: This paper contains suicide-related content that may be triggering.
Read the full abstract ▾
All authors Liuliu Chen, Elise R. Carrotte, Brian E. Chapman, Jo Robinson, Mike Conway
Computer Vision
Moment-Video: Diagnosing Temporal Fidelity of Video MLLMs on Momentary Visual Events
Xiaolin Liu, Yilun Zhu, Xiangyu Zhao et al. (+9 more)
Video multimodal large
language models (MLLMs) have made rapid progress in general and long-form
video understanding, yet their ability to capture and use brief, answer-critical visual evidence remains underexplored. Momentary actions or state transitions may determine an answer while occupying only a small fraction of the
video. Performance on broad
video understanding tasks alone does not establish whether models can reliably identify and use this temporally localized evidence. We introduce Moment-
Video, a
benchmark for evaluating temporal fidelity through momentary visual event understanding. Questions are constructed around visually observable events or event sequences, requiring models to notice, count, describe, or reason about transient evidence. The
benchmark contains 1,000 human-verified
video-QA pairs across seven domains and 25 subcategories, covering four tasks: Temporal Occurrence, Temporal Counting, Action Description, and Temporal
Reasoning. We evaluate 34 proprietary and open-source MLLMs. Under the main evaluation settings, the best-performing model, Seed-2.0-Pro, achieves 39.6% overall accuracy, while most open-source models remain below 25%. Diagnostic analyses show that denser frame sampling improves some models, but substantial errors persist even when sampled frames cover all annotated necessary event intervals in an audited subset. Performance also declines on longer
videos, suggesting additional temporal-localization challenges. By exposing these limitations, Moment-
Video provides a diagnostic
benchmark to guide the evaluation and development of
video MLLMs that more reliably understand brief, answer-critical visual events.
Read the full abstract ▾
All authors Xiaolin Liu, Yilun Zhu, Xiangyu Zhao, Xuehui Wang, Yan Li, Xin Li, Haoyu Cao, Xing Sun, Shaofeng Zhang, Xu Yang, Zhihang Zhong, Xue Yang
Computer Vision · Cross-listed from cs.LG
Drifting Preference Optimization for One-Step Generative Models
Zhou Jiang, Yandong Wen and Zhen Liu
One-step text-to-image generators are attractive for deployment because they generate an image with a single forward pass, but preference finetuning them remains difficult: standard
alignment methods often rely on policy likelihoods, denoising trajectories, differentiable reward gradients, or test-time optimization. We propose Drifting Preference Optimization (DrPO), an online preference-finetuning method for deterministic one-step generators. For each
prompt, DrPO samples candidates from the current generator, ranks them with a target reward, and uses high- and low-scoring samples to synthesize a feature-space update direction. The update is a non-parametric dipole preference field plus a reference drift estimated from the frozen base generator, and is optimized through a detached feature-space regression target. The target reward is used only for ranking, so DrPO can train with large, black-box, or non-differentiable rewards while inference remains a single generator call. We evaluate DrPO on SD-Turbo and SDXL-Turbo with multiple target rewards and
benchmarks, including HPSv3 and GenEval. DrPO improves
alignment over reward-gradient-free one-step preference baselines and reduces HPSv3 training computation by $3.51\times$ under the matched effective-batch setting by removing reward-model backpropagation. Initial offline experiments suggest that sample-based gradient synthesis can also be used beyond online reward ranking.
Read the full abstract ▾
Computer Vision
ToolFG: Towards Well-Grounded Fine-Grained Image Classification
Yu Xue, Haoxuan Qu, Zhuoling Li et al. (+4 more)
Fine-grained image classification (FGIC) has broad applications and has attracted significant research
attention. In this paper, we explore a novel paradigm for solving FGIC by proposing ToolFG, the first tool-integrated MLLM-based framework tailored to FGIC. ToolFG enables MLLMs to autonomously and flexibly use external tools during the
reasoning process, actively interact with images, and collect verifiable visual cues for distinguishing highly similar categories in a more reliable and well-grounded manner. To equip the model with such tool-use ability, we design a novel MCTS-guided tool-use knowledge
distillation mechanism, which effectively mines tool-use- and FGIC-relevant knowledge from advanced proprietary MLLMs for model training. Furthermore, we propose a model-tool co-
evolution mechanism that jointly refines the toolset and the model's tool-use policy, driving them toward a mutually adapted and FGIC-specialized state. Extensive experiments demonstrate the effectiveness of our framework.
Read the full abstract ▾
All authors Yu Xue, Haoxuan Qu, Zhuoling Li, Yihang Lou, Yan Bai, Hossein Rahmani, Jun Liu
Computer Vision
Not All Points Are Equal: Uncertainty-Aware 4D LiDAR Scene Synthesis
Xiang Xu, Alan Liang, Youquan Liu et al. (+5 more)
Constructing faithful 4D worlds from
LiDAR-acquired sequences is crucial for
embodied AI, yet current generative frameworks apply uniform modeling capacity across all spatial regions. This ignores that perceptual difficulty varies dramatically within a single scan: distant surfaces, occluded boundaries, and small-scale objects carry far higher uncertainty than well-observed structures. We present U4D, a new framework that explicitly leverages spatial uncertainty to guide
LiDAR scene generation in a "hard-to-easy" schedule. U4D derives per-point uncertainty maps via Shannon
Entropy from a pretrained segmentor, then applies an unconditional
diffusion stage to synthesize high-
entropy areas with precise geometry, followed by a conditional completion stage that fills in the remaining regions using these structures as priors. A MoST (Mixture of Spatio-Temporal) block further maintains cross-frame coherence by dynamically balancing spatial detail and temporal continuity. Extensive experiments on nuScenes and SemanticKITTI demonstrate state-of-the-art scene fidelity, temporal consistency, and downstream performance.
Read the full abstract ▾
All authors Xiang Xu, Alan Liang, Youquan Liu, Xian Sun, Linfeng Li, Lingdong Kong, Ziwei Liu, Qingshan Liu
Computer Vision
Question-Aware Evidence Ledgers for Video Relational Reasoning
Yilin Ou, Mengshi Qi and Huadong Ma
The VRR-QA challenge evaluates visual relational
reasoning in
videos, where answers often depend on implicit spatial relations, event boundaries, target identity, and dialogue context rather than a single salient frame. We present a test-time
reasoning pipeline built around a strong GPT-5.5
video QA solver and a set of question-aware evidence ledgers. The initial solver answers each question from a uniform
video representation, while routed ledgers are prompted to make the required targets, count units, reference frames, and temporal or spatial scope explicit for counting, spatial, endpoint, viewpoint, and dialogue
reasoning. External tools such as open-vocabulary detection, depth cues, pair crops, ASR, and scene-graph ledgers are used only as evidence sources. A conservative gate keeps the current answer unless independent evidence uniquely supports a different option. The final evidence-gated pipeline achieves 92.95% overall accuracy and 93.79% macro accuracy on the challenge test split.
Read the full abstract ▾
Computer Vision
GloResNet: A lightweight 3D CNN with global topological features for preterm brain injury prediction
Boyu Yuan, Jiamiao Lu, Weichuan Zhang et al. (+5 more)
This study introduces an automated deep learning framework for predicting
brain injury (BI) in preterm infants from T2-weighted MRI (dHCP
dataset). We propose GloResNet, a lightweight 3D CNN based on ResNet-10, pretrained on MedicalNet to address data scarcity. A global
manifold mapping strategy first resamples each 3D volume to 128x128x128 and then applies subject-wise z-score intensity normalization, thereby preserving global
topology while standardizing appearance. Training integrates mixup, class weighting, and test-time augmentation for robustness. In 5-fold cross-validation, GloResNet achieved 75.18% average accuracy (peak 81.82%), with specificity 0.81 and sensitivity 0.76. Results demonstrate that a
topology-aware lightweight CNN has the capability to effectively predict neonatal BI, offering a non-invasive screening tool. The source code of this paper can be obtained from the GitHub repository: https://github.com/ICL-SUST/GloResNet-Preterm-
Brain
Read the full abstract ▾
All authors Boyu Yuan, Jiamiao Lu, Weichuan Zhang, Benqing Wu, Tuo Wang, Changshan Wang, Changming Sun, Liang Guo
Computer Vision
MORPHOS: Autoregressive 4D Generation with Temporal Structured Latents
Minkyung Kwon, Jinhyeok Choi, Youngjin Shin et al. (+3 more)
We present MORPHOS, a novel autoregressive framework that generates dynamic 3D assets from
videos across diverse representations, including meshes, 3D Gaussians, and radiance fields. Existing methods are typically limited to a single representation, struggle to model topological changes, or fail to maintain temporal consistency over long
videos. To address these limitations, we introduce the Temporal Structured Latents (T-SLAT), a unified 4D representation that jointly encodes geometry and appearance along the temporal dimension. Leveraging T-SLAT, MORPHOS autoregressively generates dynamic 3D assets via causal
attention, conditioning each frame on its preceding history to ensure temporal consistency while handling evolving
topologies. We also propose a temporal-structural augmentation to mitigate error accumulation in autoregressive generation. MORPHOS achieves state-of-the-art performance in appearance and competitive results in geometry across multiple
benchmarks, demonstrating superior generalization across various representations and robustness in long-horizon generation.
Read the full abstract ▾
All authors Minkyung Kwon, Jinhyeok Choi, Youngjin Shin, Jaeyeong Kim, JongMin Lee, Seungryong Kim
Computer Vision
X-Stream: Exploring MLLMs as Multiplexers for Multi-Stream Understanding
Peiwen Sun, Xudong Lu, Huadai Liu et al. (+10 more)
While
video streaming understanding has made significant strides, real-world applications, such as live sports broadcasting, autonomous driving, and multi-screen collaboration, inherently demand continuous, multi-stream interactions. However, existing
benchmarks are confined to single-stream paradigms, leaving a critical gap in evaluating online, cross-stream
reasoning. To bridge this, we introduce X-Stream, the first
benchmark dedicated to multi-stream streaming understanding. Comprising 4,220 rigorously curated QA pairs across 932
videos, X-Stream evaluates 11 subtasks across multi-window, multi-view, and multi-device scenarios. Crucially, our
dataset is constructed using a novel dual-verification pipeline that prevents over-reliance on a single stream. Furthermore, we pioneer the conceptualization of multi-modal large
language models (MLLMs) as naive multiplexers, systematically evaluating their performance through the lens of Signal Multiplexing Theory. Our extensive online inference experiments reveal a stark reality: state-of-the-art MLLMs struggle significantly with concurrent streams, achieving only about 50% score and exhibiting poor proactive ability. Ultimately, X-Stream exposes the trade-off of current multiplexing schemes, providing both a practical evaluation protocol and empirical guidance for next-generation multi-stream
agents.
Read the full abstract ▾
All authors Peiwen Sun, Xudong Lu, Huadai Liu, Yang Bo, Dongming Wu, Huankang Guan, Minghong Cai, Jinpeng Chen, Xintong Guo, Shuhan Li, Fang Liu, Rui Liu, Xiangyu Yue
Computer Vision
Places in the Wild: A Large, High-Resolution RAW Photograph Dataset for Ecologically Valid Vision Research
Michelle R. Greene
Large image
datasets have accelerated progress in cognitive neuroscience and computer
vision. However, most
datasets are low-resolution, internet-sourced JPEGs with unknown capture conditions and limited spatial context. Places in the Wild is a
dataset of 67,574 high-resolution photographs collected in situ across 810 physical locations spanning 260 basic-level scene categories, including indoor, urban, and natural environments. At each location, a 45-megapixel Canon EOS R5 mounted on a panoramic tripod captured 72 images at 5-degree horizontal intervals plus 12 images at varying elevations, yielding dense 360-degree viewpoint sampling. All images were recorded simultaneously as 14-bit RAW (CR3) files and compressed JPEGs, preserving sensor-level detail for analyses of luminance, contrast, color, and other image statistics. The
dataset is accompanied by complete EXIF metadata and a suite of image-quality metrics. Places in the Wild supports research on viewpoint-dependent recognition in humans and models, training and evaluation of scene-understanding systems under realistic conditions, characterization of natural scene statistics, and experiments requiring near-full-field visual displays.
Read the full abstract ▾
Computer Vision
Retrieve What's Missing: Coverage-Maximizing Retrieval for Consistent Long Video Generation
Minseok Joo, Dogyun Park, Taehoon Lee et al. (+2 more)
Maintaining long-term
geometric consistency remains challenging for long-horizon autoregressive
video generation.
Memory-augmented generative models address this by retrieving historical frames, but their effectiveness depends on two key design choices: what 3D-
geometric evidence should represent past observations, and how
memory frames should be selected from this evidence. Existing methods often rely on camera poses or field-of-view overlap, which are lightweight but too coarse to reason about pixel-wise visibility, or use explicit 3D reconstruction, which provides fine-grained evidence but is costly to maintain over long rollouts. We propose Coverage-Maximizing
Retrieval-Augmented Generation (COVRAG), a depth-based
memory retrieval framework that uses pretrained 3D priors to construct a target-view coverage map as lightweight 3D
memory evidence. For frame selection, COVRAG maximizes residual coverage gain, iteratively retrieving frames that explain target-view regions not covered by the current context or previously selected
memories. To improve scalability in long-
video generation, we introduce sliding-window depth caching for efficient geometry estimation. Experiments on RealEstate10K and DL3DV10K show that COVRAG improves long-horizon
geometric consistency while maintaining low latency compared to baselines.
Read the full abstract ▾
All authors Minseok Joo, Dogyun Park, Taehoon Lee, Kyujin Lee, Hyunwoo J. Kim
Computer Vision
MASER: Modality-Adaptive Specialist Routing for Embodied 3D Spatial Intelligence
Hilton Raj and Vishnuram AV
In 3D environments,
Embodied Agents answer spatially relevant questions through
reasoning from a mixture of modalities including natural
language, RGB images, point clouds, depth maps and camera poses. Existing
Vision-
Language models (VLMs) are fine-tuned over a single modality. This completely ignores the question semantics which may favor a different modality than the finetuned modality. To address this, we propose MASER (Modality-Adaptive SpEcialist Routing), a lightweight framework that trains five different modality adapters of a shared VLM backbone and learns a neural routing policy that selects the best adapter based on the question during inference. We encode each question with a frozen sentence
transformer and pass the
embedding through a small Multi-layer Perceptron (MLP) trained on oracle adapter-accuracy labels. We evaluate our methodology over the Open3D-VQA
benchmark and our evaluations show that no single modality is universally optimal – point-cloud answers are best in 51.5% of cases. MASER routes with 51.3% oracle agreement, outperforming a Random-Forest ablation (43.5%), with only a single adapter call per question.
Read the full abstract ▾
Computer Vision
Active Exploring like a Pigeon: Reinforcing Spatial Reasoning via Agentic Vision-Language Models
Wei Deng, Xianlin Zhang and Mengshi Qi
Enabling
Vision-
Language Models (VLMs) to perform spatial
reasoning remains challenging. Existing approaches treat VLMs as passive observers, which is difficult for real-world applications. Moreover, reinforcement learning methods rely on sparse rewards, limiting their effectiveness for complex
reasoning tasks. Inspired by pigeons' building and exploiting cognitive maps for
navigation, we propose a novel agentic pipeline for spatial
reasoning. First, we introduce a new dynamic cognitive map parameterizing scene layout as object positions and orientations, serving as persistent
memory for new observations. Second, we propose a novel Spatial Assertion Codes (SAC), Python expressions programmatically describing spatial relationships. By collaborating with the dynamic cognitive map, SAC enables verification of intermediate
reasoning steps, providing dense reward signals. We optimize the model via supervised and reinforcement finetuning. Experiments on the MindCube
benchmark demonstrate state-of-the-art performance with 80.5% overall accuracy, outperforming the best current method by 29.5 accuracy points (a relative improvement of 53.2%) on the challenging Rotation subset. Our code and data are open-sourced at https://github.com/dw-dengwei/active-spatial-
reasoning.git.
Read the full abstract ▾
Computer Vision
Initialization is Half the Battle: Generating Diverse Images from a Guidance Potential Posterior
Xiang Li, Dianbo Liu and Kenji Kawaguchi
Despite the remarkable fidelity of generative models, they frequently suffer from mode collapse. Existing strategies for enhancing diversity predominantly focus on intervening during the generation trajectory. We identify a critical oversight that the standard Gaussian initialization often causes trajectories to collapse into dominant modes because it is agnostic to the guidance potential landscape. In this work, we formulate selecting the initial noise from a guidance potential posterior, which effectively re-weights the prior towards diversity-rich regions. To sample from this distribution efficiently, we introduce Diversity-inducing Initialization (DivIn), which leverages Langevin dynamics to actively navigate the initialization landscape, steering initial noise away from collapsing regions while anchoring them to the valid data
manifold. Our method serves as an inference-time diversity enhancement compatible with both
diffusion and flow matching models. Extensive experiments show that DivIn exhibits a superior performance in both class-to-image and text-to-image scenarios. Furthermore, we highlight that as DivIn is orthogonal to trajectory-based methods, combining them significantly expands the diversity-quality Pareto frontier beyond what either achieves in isolation.
Read the full abstract ▾
Computer Vision
Reason-Then-Retrieve for CoVR-R with Structured Edit Prompts and Dense-Sparse Fusion
Dongqing Liu, Mengshi Qi and Hongwei Ji
CoVR-R studies reason-aware composed
video retrieval: given a reference
video and an edit instruction, the system must retrieve the target
video that satisfies the edit. The main difficulty is that the target is not described directly; it must be inferred from fine-grained changes in object identity, action order, final state, hand interaction, and scene transition. We build a zero-shot reason-then-retrieve pipeline around Qwen3.5-27B. For each gallery
video, the model generates a
retrieval-oriented structured description and a dense
embedding by pooling generated-
token hidden states with
token-dependent weights. For each query, the model first performs edit
reasoning over the reference
video and instruction, then generates a target-
video description whose hidden states serve as the query
embedding. We complement dense
retrieval with a TF-IDF branch over the generated texts and fuse the two rankings with split-specific weights. On validation, the current best submission reaches 80.81 at R@1, 94.86 at R@5, 97.11 at R@10, and 98.59 at R@50. On the blind test split, it reaches 89.73 at R@1, 95.79 at R@5, 96.63 at R@10, and 97.98 at R@50.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
HLL: Can Agents Cross Humanity's Last Line of Verification?
Xinhao Song, Su Su, Sirui Song et al. (+6 more)
Multimodal agents are increasingly expected to operate interfaces on behalf of users, raising a central deployment question: can they truly substitute for humans in workflows that services deliberately protect against automation? CAPTCHA verification makes this question concrete. It is not merely a visual puzzle, but a human-verification boundary placed before account creation, content access, form submission, and other protected actions. We introduce Humanity's Last Line of Verification (HLL), a controlled
benchmark that uses interactive CAPTCHA verification to evaluate whether
agents can cross this boundary through grounded, human-like interaction rather than recognition alone. HLL covers diverse CAPTCHA interactions and exposes
agents to controlled realism stressors, including cluttered webpages, harder task variants, and trace-conditioned validation of the solving process. We evaluate eight frontier
multimodal agents in a closed-loop GUI environment. The results show that current
agents remain brittle at this human-substitution boundary: performance varies sharply across verification types, degrades under realistic interface conditions, and drops further when correct answers must be supported by valid action traces. By exposing gaps in localization, action calibration, state tracking, and process consistency, HLL provides a concrete testbed for measuring how close
multimodal agents are to acting as human substitutes in protected real-world workflows. Our code is available at https://github.com/XinhaoS0101/HLL
Read the full abstract ▾
All authors Xinhao Song, Su Su, Sirui Song, Hongliang Wu, Wen Shen, Zhihua Wei, Gongshen Liu, Linfeng Zhang, Dongrui Liu
Computer Vision · Cross-listed from cs.CL
PaSBench-Video: A Streaming Video Benchmark for Proactive Safety Warning
Yusong Zhao, Yuejin Xie, Youliang Yuan et al. (+4 more)
Between the first visible sign of danger and the moment an accident occurs, there is often a window where intervention remains possible.
Video-capable
multimodal large
language models (MLLMs) could serve as always-on safety monitors that issue warnings during this window. Yet current
benchmarks do not test this ability: they rely on static inputs, ignore timing precision, and omit false-positive measurement on safe scenes. We present PaSBench-
Video, a 740-
video benchmark with 481 risk and 259 no-risk
videos across four domains: driving, healthcare, daily life, and industrial production. Risk
videos are annotated with frame-level risk onset and accident boundaries. A model must observe the
video causally and produce a warning that is both temporally calibrated and content-correct. Testing 13 MLLMs, we find that no model exceeds 20.0% on our strictest metric, and recall is tightly coupled with false-positive rate, with Pearson correlation 0.64: higher detection comes only at the cost of triggering warnings on the majority of safe clips. Performance splits sharply by domain: models achieve moderate recall at low false-positive rates in daily life, where risks are inherently anomalous, yet fire indiscriminately in driving, where routine and hazardous scenes look alike. These results indicate that current models rely on scene-level activity cues rather than
reasoning about emerging harm.
Read the full abstract ▾
All authors Yusong Zhao, Yuejin Xie, Youliang Yuan, Junjie Hu, Jitian Guo, Yujiu Yang, Pinjia He
Computer Vision
Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation
Yuheng Chen, Teng Hu, Yuji Wang et al. (+3 more)
Identity-preserving
video generation (IPVG) aims to synthesize high-fidelity
videos that follow text
prompts while faithfully preserving a reference identity. Despite recent progress, existing IPVG methods still struggle to balance high-level semantic control and low-level identity fidelity. To bridge this gap, we propose ST-DRC, an effective Spatial-Temporal Decoupled Reference Conditioning framework for identity-preserving text-to-
video generation. At the framework level, ST-DRC performs latent in-context feature injection by encoding the reference image with the
video VAE and concatenating it with noisy
video latents, enabling rich low-level identity details to be accessed without additional adapters. To separate identity-aware reference
retrieval from appearance copying, we introduce TASS-RoPE, a Temporal-Adjacent Spatial-Shifted RoPE scheme that places reference
tokens near the
video sequence in time but shifts them in space, allowing reference information to flow through spatio-temporal
attention while suppressing pixel-level copy-paste shortcuts. To further prevent shortcut learning and strengthen the otherwise diluted identity supervision in the
diffusion objective, we combine appearance-invariant reference augmentation with face-guided identity objectives, encouraging the model to preserve identity under variations in color, pose, and layout. At inference time, we introduce a three-stream reference classifier-free guidance strategy that independently controls text adherence and reference fidelity. Experiments demonstrate that ST-DRC achieves strong identity preservation,
prompt alignment, temporal consistency, and
video quality with a lightweight design built on LTX-2.3. Our method ranks among the top submissions in the facial identity-preserving
video generation track, validating the effectiveness of spatial-temporal decoupled reference conditioning.
Read the full abstract ▾
All authors Yuheng Chen, Teng Hu, Yuji Wang, Qingdong He, Lizhuang Ma, Jiangning Zhang
Computer Vision
Geometry-Aware Implicit Memory for Video World Models
Zhengxuan Wei, Xu Guo, Xinghui Li et al. (+8 more)
Video world models aim to simulate controllable visual environments, but long-horizon rollouts depend on what the model remembers after observations leave its native context window. Explicit
memories retain frames or online 3D reconstructions, which can suffer from heuristic
retrieval errors, redundant appearance storage, or reconstruction artifacts. Implicit
memories compress history into a compact state, but existing designs are not explicitly constrained to encode cross-view scene geometry. We propose GIM-World, a geometry-aware implicit
memory framework for
video world models. A lightweight
transformer encoder compresses variable-length history into fixed-size
memory tokens, a camera-queryable geometry head distills 3D scene structure from a frozen foundation model into the
memory during training, and an information-guided
pruning rule keeps encoding cost bounded as history grows. The geometry teacher is discarded at inference, leaving a lightweight
memory module. Experiments on MIND show that GIM-World better preserves long-horizon
geometric and visual consistency than both explicit- and implicit-
memory baselines.
Read the full abstract ▾
All authors Zhengxuan Wei, Xu Guo, Xinghui Li, Xunzhi Xiang, Min Wei, Yiran Zhu, Qiulin Wang, Xintao Wang, Pengfei Wan, Xiangwang Hou, Qi Fan
Computer Vision
GC-MoE: Genomics-Guided Cell-Type-Specific Mixture of Experts for Histology-Based Single-Cell Spatial Transcriptomics
Kaito Shiku, Ahtisham Fazeel Abbasi, Ryoma Bise et al. (+4 more)
Histology-based single-
cell spatial transcriptomics (ST) estimation aims to predict gene expression for individual
cells from histopathological images and
cell locations, reducing the need for costly single-
cell ST measurements. Unlike existing histology-to-ST methods that mainly predict spot-level profiles for local regions containing multiple
cells, this task requires modeling
cell-to-
cell expression variability, which is strongly structured by
cell type. We propose Genomics-Guided
Cell-Type-Specific Mixture-of-Experts (GC-MoE), which estimates
cell-type probabilities with a routing network and softly combines
cell-type-specific experts for gene expression prediction. To further encode
cell-type-dependent gene programs, we introduce the
Cell-Type-Specific Co-Expression-Aware Predictor (CAP), together with a lightweight
Cell-to-
Cell Interaction
Attention (C2CA) module for neighboring-
cell context. Experiments and ablations on public single-
cell ST
datasets show consistent improvements over existing single-
cell and adapted spot-level baselines.
Read the full abstract ▾
All authors Kaito Shiku, Ahtisham Fazeel Abbasi, Ryoma Bise, Yuichiro Iwashita, Kazuya Nishimura, Andreas Dengel, Muhammad Nabeel Asim
Computer Vision
Edge Prediction for Roof Wireframe Reconstruction with Transformers
Gustav Hanning, Ludvig Dillén, Jonathan Astermark et al. (+2 more)
This paper presents a competitive solution to the S23DR Challenge 2026, which aims to reconstruct 3D house roof wireframe models from sparse SfM point clouds and ground-level semantic
segmentations and depth maps. Our proposed method utilizes an end-to-end
Transformer encoder-decoder architecture inspired by DETR. To effectively process the
geometric and semantic data, the sparse SfM point cloud input is dynamically subsampled based on semantic priority and augmented with Gestalt and ADE20k class features. To further increase
segmentation context, we fuse the point features with additional Gestalt feature encodings which are obtained by projecting the points into latent feature maps produced by a frozen autoencoder. Learned query
embeddings are then decoded directly into 3D wireframe edges via cross-
attention mechanisms. Evaluated on the "HoHo 22k"
dataset, our approach significantly outperforms both handcrafted and learned baselines, achieving a Hybrid Structure Score (HSS) of 0.6476 and securing the second-highest position on the challenge's private leaderboard.
Read the full abstract ▾
All authors Gustav Hanning, Ludvig Dillén, Jonathan Astermark, Johanna Lidholm, Viktor Larsson
Computer Vision
Explainable Forensics of Manipulated Segments in Untrimmed Long Videos
Yue Feng, Jingjing Li, Qijia Lu et al. (+9 more)
The rapid advancement of AI-driven
video generation has transformed content creation, while simultaneously increasing the risk of misinformation through localized manipulations in long-form
videos. Existing
video forensic methods predominantly operate on short, independent clips, and thus fail to capture realistic scenarios where AI-generated content is sparsely embedded within otherwise authentic footage. To bridge this gap, we formulate the task of Temporal AI-Generated Segment Localization and Explanation, which targets authenticity detection, temporal localization, and interpretable analysis of manipulated segments in untrimmed long
videos. We further introduce TASLE, a large-scale
benchmark comprising 12,472 untrimmed
videos with diverse manipulation patterns and rich annotation signals, including temporal boundaries, authenticity labels, and segment-level rationales. In addition, we propose MSLoc, a coarse-to-fine forensic baseline that combines a boundary-sensitive proposal generation module for efficient long-
video scanning with an MLLM-based refinement module for precise boundary localization and interpretable
reasoning. Experiments validate the effectiveness of the proposed baseline, highlighting the importance of segment-level explainable forensics for long-form AI-generated
video analysis. Our
dataset and code are publicly available at https://debby-0527.github.io/TASLE.
Read the full abstract ▾
All authors Yue Feng, Jingjing Li, Qijia Lu, Wei Ji, Jingrou Zhang, Fei Shen, Xiao Li, Yizhen Jia, Qiang Chen, Limin Wang, Wentong Li, Jie Qin
Computer Vision
Honey, I Shrunk the Arc de Triomphe!
Yuanbo Xiangli, Hanyu Chen, Xueqing Tsang et al. (+1 more)
Metric scale
monocular geometry estimation has seen significant progress through large-scale data aggregation, yet current foundation models suffer from a persistent ”scale-collapse” phenomenon: distant landmarks and vast landscapes are metrically underestimated. We hypothesize that this performance gap stems from a training data bottleneck, where existing metric-scale
datasets are hardware-constrained to homogenous vehicle-captured
LiDAR or short-range indoor scans, or consist of synthetic data that lacks the semantic complexity of the physical world. To bridge this gap, we curate a new metrically-grounded, in-the-wild
dataset that we call MetricScenes, gathered from a variety of sources including Internet photo collections and stereo imagery. We estimate camera poses and initial depth maps for each scene using off-the-shelf methods, and recover absolute scale from geo-tagged metadata as well as known stereo camera baselines. We also improve the quality of depth maps derived from MetricScenes via a new two-stage Poisson completion method. Fine-tuning MoGe-2 on our
dataset significantly mitigates scale-collapse and achieves superior metric accuracy in unconstrained, open-domain scenes while maintaining state-of-the-art performance on standard
benchmarks.
Read the full abstract ▾
All authors Yuanbo Xiangli, Hanyu Chen, Xueqing Tsang, Noah Snavely