Computer Vision
LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation
Bo Miao, Weijia Liu, Jun Luo et al. (+8 more)
Language-conditioned goal
navigation requires
embodied agents to locate user-specified targets without step-by-step guidance. However, existing
benchmarks largely focus on category-level goals or rely on instance descriptions generated by
vision-
language models, which often contain ambiguities and semantic errors, limiting systematic and reliable evaluation. We introduce HieraNav, an open-vocabulary
navigation task with goals at four hierarchical semantic levels: scene, room, region, and instance. We present
Language as a Map (LangMap), the first goal
navigation benchmark to enrich real-world indoor 3D scans with human-verified semantic annotations supporting tasks across all four goal levels. Built on HM3D using a contrastive annotation protocol that compares same-scene regions and instances, LangMap provides region labels and discriminative region and instance descriptions covering 414 object categories and contains over 18K tasks. Each target has concise and detailed descriptions, enabling evaluation across instruction styles. Automated and human evaluations validate our annotation quality: our descriptions improve text-to-view matching accuracy over GOAT-Bench's by 23 points on all shared annotated instances, and an independent human audit yields 92.5% unique-and-correct matches. We also propose PlaNaVid, an RGB-only baseline that combines Bounded Diverse
Memory with high-level planning to
prime a reactive policy for multi-goal
navigation, achieving top-tier success rates without depth, 3D scene representations, or object masks. Further analyses reveal that exploration and hierarchical disambiguation failures become more prominent at finer goal levels, while long-tail categories, small objects, distant targets, timely stopping, and multi-goal completion remain challenging.
Benchmark and code: https://bo-miao.github.io/LangMap
Read the full abstract ▾
All authors Bo Miao, Weijia Liu, Jun Luo, Lachlan Shinnick, Jian Liu, Thomas Hamilton-Smith, Yuhe Yang, Zijie Wu, Vanja Videnovic, Feras Dayoub, Anton van den Hengel
Computer Vision
Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation
Hongzhou Zhu, Min Zhao, Guande He et al. (+3 more)
To achieve real-time interactive
video generation, current methods distill pretrained bidirectional
video diffusion models into few-step autoregressive (AR) models, facing an architectural gap when full
attention is replaced by causal
attention. However, existing approaches do not bridge this gap theoretically. They initialize the AR student via ODE
distillation, which requires frame-level injectivity, where each noisy frame must map to a unique clean frame under the PF-ODE of an AR teacher. Distilling an AR student from a bidirectional teacher violates this condition, preventing recovery of the teacher's flow map and instead inducing a conditional-expectation solution, which degrades performance. To address this issue, we propose Causal Forcing, which uses an autoregressive teacher for ODE initialization to bridge the architectural gap, and then applies the same DMD procedure as in Self Forcing. Empirical results show that our method outperforms all baselines across all metrics, surpassing the SOTA Self Forcing by 19.3% in Dynamic Degree, 8.7% in VisionReward, and 16.7% in Instruction Following. Project page: \href{https://thu-ml.github.io/CausalForcing.github.io/}{https://thu-ml.github.io/CausalForcing.github.io/}; the code: \href{https://github.com/thu-ml/Causal-Forcing}{https://github.com/thu-ml/Causal-Forcing}.
Read the full abstract ▾
All authors Hongzhou Zhu, Min Zhao, Guande He, Hang Su, Chongxuan Li, Jun Zhu
Computer Vision
3A-VLA: Abstraction-Aligned Action Learning for Vision-Language Agents in 3D Game Worlds
Zheyuan Zhou, Liang Du, Zixun Sun et al. (+5 more)
Despite advances in
vision-
language-action (VLA) models, they remain limited in highly dynamic game settings such as 3D open worlds and competitive player versus player (PvP) settings, where
agents must efficiently extract sparse, actionable signals from dense visual input while integrating
multimodal cues to track off-screen high-value targets in real time. To address this, we introduce 3A-VLA (Abstraction-Aligned Action VLA), a framework that grounds action learning in explicit intention and environment abstractions rather than superficial pattern matching. We introduce dual task-agnostic abstractions: the intention abstraction (IA), which condenses verbose instructions and
reasoning into explicit semantic primitives, and the environment semantics abstraction (ESA), which structures dense visual streams into a spatial-functional affordance representation to guide grounded actions. We further propose an abstraction
alignment reweighting (AAR) module that adaptively reweights the action imitation loss. A continuous intention-environment
alignment signal is used to emphasize reliable action supervision when abstractions agree and reduce the influence of ambiguous demonstrations when they diverge, thereby learning a policy that balances fine-grained control and high-level
reasoning without the need for manual rules. Extensive experiments show that 3A-VLA performs competitively among the evaluated baselines in both open-world (Minecraft) and competitive PvP (Game for Peace) settings. It also demonstrates zero-shot combat transfer from Game for Peace to high-fidelity games, including Valorant, CS2, GTA V, Elden Ring, and Mount & Blade.
Read the full abstract ▾
All authors Zheyuan Zhou, Liang Du, Zixun Sun, Xiaoyu Zhou, Ruimin Ye, Qihao Chen, Yinda Chen, Lemiao Qiu
Computer Vision
SSI-DM: Singularity Skipping Inversion of Diffusion Models
Chen Min, Enze Jiang, Jishen Peng et al. (+1 more)
Inverting real images into the noise space is essential for editing tasks using
diffusion models, yet existing methods produce non-Gaussian noise with poor editability due to the inaccuracy in early noising steps. We identify the root cause: a mathematical singularity that renders inversion fundamentally ill-posed. We propose Singularity Skipping Inversion of
Diffusion Models (SSI-DM), which bypasses this singular region by adding small noise before standard inversion. This simple approach produces inverted noise with natural Gaussian properties while maintaining reconstruction fidelity. As a plug-and-play technique compatible with general
diffusion models, our method achieves superior performance on public image
datasets for reconstruction and interpolation tasks, providing a principled and efficient solution to
diffusion model inversion.
Read the full abstract ▾
All authors Chen Min, Enze Jiang, Jishen Peng, Zheng Ma
Computer Vision
Learning Topology-Aware Implicit Field for Unified Pulmonary Tree Modeling with Incomplete Topological Supervision
Ziqiao Weng, Jiancheng Yang, Kangxian Xie et al. (+2 more)
Pulmonary trees extracted from CT images frequently exhibit topological incompleteness, such as missing or disconnected branches, which substantially degrades downstream anatomical analysis and limits the applicability of existing pulmonary tree modeling pipelines. Current approaches typically rely on dense volumetric processing, explicit graph
reasoning, or generic point cloud completion priors, leading to limited efficiency, weak structural awareness, and reduced robustness under realistic structural corruption. We propose TopoField, a
topology-aware implicit modeling framework that treats
topology repair as a first-class modeling problem and enables unified multi-task inference for pulmonary tree analysis. TopoField represents pulmonary anatomy using sparse surface and skeleton point clouds and learns a continuous implicit field that supports
topology repair without relying on complete or explicit disconnection annotations, by training on synthetically introduced structural disruptions over already incomplete trees. Building upon the repaired implicit representation, anatomical labeling and lung segment reconstruction are jointly inferred through task-specific implicit functions within a single forward pass. Extensive experiments on the Lung3D+
dataset demonstrate that TopoField consistently improves topological completeness and achieves accurate anatomical labeling and lung segment reconstruction under challenging incomplete scenarios. We further validate TopoField on real incomplete outputs from an external
segmentation model, demonstrating its applicability to realistic
segmentation pipelines. Owing to its implicit formulation, TopoField attains high computational efficiency, completing all tasks in just over one second per case, highlighting its practicality for large-scale and time-sensitive clinical applications.
Read the full abstract ▾
All authors Ziqiao Weng, Jiancheng Yang, Kangxian Xie, Bo Zhou, Weidong Cai
Computer Vision
Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Models
Yu Zeng, Wenxuan Huang, Zhen Fang et al. (+14 more)
Multimodal Large
Language Models (MLLMs) have advanced VQA and now support
Vision-DeepResearch systems that use search engines for complex visual-textual fact-finding. However, evaluating these visual and textual search abilities is still difficult, and existing
benchmarks have two major limitations. First, existing
benchmarks are not visual search-centric: answers that should require visual search are often leaked through cross-textual cues in the text questions or can be inferred from the prior world knowledge in current MLLMs. Second, overly idealized evaluation scenario: On the image-search side, the required information can often be obtained via near-exact matching against the full image, while the text-search side is overly direct and insufficiently challenging. To address these issues, we construct the
Vision-DeepResearch
benchmark (VDR-Bench) comprising 2,000 VQA instances. All questions are created via a careful, multi-stage curation pipeline and rigorous expert review, designed to assess the behavior of
Vision-DeepResearch systems under realistic real-world conditions. Moreover, to address the insufficient visual
retrieval capabilities of current MLLMs, we propose a simple multi-round cropped-search workflow. This strategy is shown to effectively improve model performance in realistic visual
retrieval scenarios. Overall, our results provide practical guidance for the design of future
multimodal deep-research systems. The code will be released in https://github.com/Osilly/
Vision-DeepResearch.
Read the full abstract ▾
All authors Yu Zeng, Wenxuan Huang, Zhen Fang, Shuang Chen, Yufan Shen, Yishuo Cai, Xiaoman Wang, Zhenfei Yin, Lin Chen, Zehui Chen, Shiting Huang, Yiming Zhao, Xu Tang, Yao Hu, Philip Torr, Wanli Ouyang, Shaosheng Cao
Computer Vision
CIEC: Coupling Implicit and Explicit Cues for Multimodal Weakly Supervised Manipulation Localization
Xinquan Yu, Wei Lu, Xiangyang Luo et al. (+1 more)
To mitigate the threat of misinformation,
multimodal manipulation localization has garnered growing
attention. Consider that current methods rely on costly and time-consuming fine-grained annotations, such as patch/
token-level annotations. This paper proposes a novel framework named Coupling Implicit and Explicit Cues (CIEC), which aims to achieve
multimodal weakly-supervised manipulation localization for image-text pairs utilizing only coarse-grained image/sentence-level annotations. It comprises two branches, image-based and text-based weakly-supervised localization. For the former, we devise the Textual-guidance Refine Patch Selection (TRPS) module. It integrates forgery cues from both visual and textual perspectives to lock onto suspicious regions aided by spatial priors. Followed by the background silencing and spatial contrast constraints to suppress interference from irrelevant areas. For the latter, we devise the Visual-deviation Calibrated
Token Grounding (VCTG) module. It focuses on meaningful content words and leverages relative visual bias to assist
token localization. Followed by the asymmetric sparse and semantic consistency constraints to mitigate label noise and ensure reliability. Extensive experiments demonstrate the effectiveness of our CIEC, yielding results comparable to fully supervised methods on several evaluation metrics.
Read the full abstract ▾
All authors Xinquan Yu, Wei Lu, Xiangyang Luo, Rui Yang
Computer Vision
Lung Nodule Image Synthesis Driven by Two-Stage Generative Adversarial Networks
Lu Cao, Xiquan He, Junying Zeng et al. (+2 more)
The limited sample size and insufficient diversity of lung nodule CT
datasets severely restrict the performance and generalization ability of detection models. Existing methods generate images with insufficient diversity and controllability, suffering from issues such as monotonous texture features and distorted anatomical structures. Therefore, we propose a two-stage generative
adversarial network (TSGAN) to enhance the diversity and spatial controllability of synthetic data by decoupling the morphological structure and texture features of lung nodules. In the first stage, StyleGAN is used to generate semantic
segmentation mask images, encoding lung nodules and tissue backgrounds to control the anatomical structure of lung nodule images; The second stage uses the DL-Pix2Pix model to translate the mask map into CT images, employing local importance
attention to capture local features, while utilizing dynamic weight multi-head window
attention to enhance the modeling capability of lung nodule texture and background. Compared to the original
dataset, the accuracy improved by 4.6% and mAP by 4% on the LUNA16
dataset. Experimental results demonstrate that TSGAN can enhance the quality of synthetic images and the performance of detection models.
Read the full abstract ▾
All authors Lu Cao, Xiquan He, Junying Zeng, Chaoyun Mai, Min Luo
Computer Vision · Cross-listed from eess.SP
Real-Time 2D LiDAR Object Detection Using Three-Frame RGB Scan Encoding
Soheil Behnam Roudsari, Alexandre S. Brandão and Felipe N. Martins
Indoor service
robots need perception that is robust, more
privacy-friendly than RGB
video, and feasible on embedded hardware. We present a camera-free 2D
LiDAR object detection pipeline that encodes short-term temporal context by stacking three consecutive scans as RGB channels, yielding a compact YOLOv8n input without occupancy-grid construction while preserving angular structure and motion cues. Evaluated in Webots across 160 randomized indoor scenarios with strict scenario-level holdout, the method achieves 98.4% mAP@0.5 (0.778 mAP@0.5:0.95) with 94.9% precision and 94.7% recall on four object classes. On a Raspberry Pi 5, it runs in real time with a mean post-warm-up end-to-end latency of 47.8ms per frame, including scan encoding and postprocessing. Relative to a closely related occupancy-grid
LiDAR-YOLO pipeline reported on the same platform, the proposed representation is associated with substantially lower reported end-to-end latency. Although results are simulation-based, they suggest that lightweight temporal encoding can enable accurate and real-time
LiDAR-only detection for embedded indoor robotics without capturing RGB appearance.
Read the full abstract ▾
Computer Vision
Reg4Pru: Regularisation Through Random Token Routing for Token Pruning
Julian Wyatt, Ronald Clark and Irina Voiculescu
Transformers are widely adopted in modern
vision models due to their strong ability to scale with
dataset size and generalisability. However, this comes with a major drawback: computation scales quadratically to the total number of
tokens. Numerous methods have been proposed to mitigate this. For example, we consider
token pruning with reactivating
tokens from preserved representations, but the increased computational efficiency of this method results in decreased stability from the preserved representations, leading to poorer dense prediction performance at deeper layers. In this work, we introduce Reg4Pru, a training regularisation technique that mitigates
token-
pruning performance loss for
segmentation. We compare our models on the FIVES blood vessel
segmentation dataset and find that Reg4Pru improves average precision by an absolute 46% compared to the same model trained without routing. This increase is observed using a configuration that achieves a 29% relative speedup in wall-clock time compared to the non-pruned baseline. These findings indicate that Reg4Pru is a valuable regulariser for
token reduction strategies.
Read the full abstract ▾
Computer Vision
LoopViT: Scaling Visual ARC with Looped Transformers
Wen-Jie Shu, Xuerui Qiu, Rui-Jie Zhu et al. (+3 more)
Recent advances in visual
reasoning have leveraged
vision transformers to tackle the ARC-AGI
benchmark. However, we argue that the feed-forward architecture, where computational depth is strictly bound to parameter size, falls short of capturing the iterative, algorithmic nature of human induction. In this work, we propose a recursive architecture called Loop-ViT, which decouples
reasoning depth from model capacity through weight-tied recurrence. Loop-ViT iterates a weight-tied Hybrid Block, combining local convolutions and global
attention, to form a latent chain of thought. Crucially, we introduce a parameter-free Dynamic Exit mechanism based on predictive
entropy: the model halts inference when its internal state “crystallizes" into a low-uncertainty attractor. Empirical results on the ARC-AGI-1
benchmark validate this perspective: our 18M model achieves 65.8% accuracy, outperforming massive 73M-parameter ensembles. These findings demonstrate that adaptive iterative computation offers a far more efficient scaling axis for visual
reasoning than simply increasing network width. The code is available at https://github.com/WenjieShu/LoopViT.
Read the full abstract ▾
All authors Wen-Jie Shu, Xuerui Qiu, Rui-Jie Zhu, Harold Haodong Chen, Yexin Liu, Harry Yang
Computer Vision
Deep learning enables urban change profiling through alignment of historical maps
Sidi Wu, Yizi Chen, Maurizio Gribaudi et al. (+4 more)
Prior to modern Earth observation technologies, historical maps provide a unique record of long-term urban transformation and offer a lens on the evolving identity of cities. However, extracting consistent and fine-grained change information from historical map series remains challenging due to spatial misalignment, cartographic variation, and degrading document quality, limiting most analyses to small-scale or qualitative approaches. We propose a deep-learning-based framework for fine-grained urban change analysis from large collections of historical maps, built on a modular design that integrates spatial map
alignment, multi-temporal object extraction, and change estimation. This framework shifts the analysis of historical maps from ad hoc visual comparison toward systematic, quantitative characterization of urban change. Experiments demonstrate the robust performance of the proposed
alignment and object extraction methods. Applied to Paris between 1868 and 1937, the framework reveals the spatial and temporal heterogeneity in urban transformation, suggesting its potential relevance for research in the social sciences and humanities. The modular design of our framework provides flexibility for future adaptation to different cartographic contexts and downstream applications.
Read the full abstract ▾
All authors Sidi Wu, Yizi Chen, Maurizio Gribaudi, Konrad Schindler, Clément Mallet, Julien Perret, Lorenz Hurni
Computer Vision · Cross-listed from cs.RO
FD-VLA: Force-Distilled Vision-Language-Action Model for Contact-Rich Manipulation
Ruiteng Zhao, Wenshuo Wang, Yicheng Ma et al. (+4 more)
Force sensing is a crucial modality for
Vision-
Language-Action (VLA) frameworks, as it enables fine-grained perception and dexterous manipulation in contact-rich tasks. We present Force-Distilled VLA (FD-VLA), a novel framework that integrates force awareness into contact-rich manipulation without relying on physical force sensors. The core of our approach is a Force
Distillation Module (FDM), which distills force by mapping a learnable query
token, conditioned on visual observations and
robot states, into a predicted force
token aligned with the latent representation of actual force signals. During inference, this distilled force
token is injected into the pretrained VLM, enabling force-aware
reasoning while preserving the integrity of its
vision-
language semantics. This design provides two key benefits: first, it allows practical deployment across a wide range of
robots that lack expensive or fragile force-torque sensors, thereby reducing hardware cost and complexity; second, the FDM introduces an additional force-
vision-state fusion prior to the VLM, which improves cross-modal
alignment and enhances perception-action robustness in contact-rich scenarios. Surprisingly, our physical experiments show that the distilled force
token outperforms direct sensor force measurements as well as other baselines, which highlights the effectiveness of this force-distilled VLA approach.
Read the full abstract ▾
All authors Ruiteng Zhao, Wenshuo Wang, Yicheng Ma, Xiaocong Li, Francis E.H. Tay, Marcelo H. Ang Jr., Haiyue Zhu
Computer Vision
Eliminating Registration Bias in Synthetic CT Generation: A Physics-Based Simulation Framework
Lukas Zimmermann, Michael Rauter, Maximilian Schmid et al. (+2 more)
Supervised synthetic CT generation from CBCT requires registered training pairs, yet perfect registration between separately acquired scans remains unattainable. This registration bias propagates into trained models and corrupts standard evaluation metrics. This may suggest that superior
benchmark performance indicates better reproduction of registration artifacts rather than anatomical fidelity. We propose physics-based CBCT simulation to provide geometrically aligned training pairs by construction, combined with evaluation using
geometric alignment metrics against input CBCT rather than biased ground truth. On two independent pelvic
datasets, models trained on synthetic data achieved superior
geometric alignment (Normalized Mutual Information: 0.31 vs 0.22) despite lower conventional intensity scores. Intensity metrics showed inverted correlations with clinical assessment for deformably registered data, while Normalized Mutual Information consistently predicted observer preference across registration methodologies (rho = 0.31, p < 0.001). Clinical observers preferred synthetic-trained outputs in 87% of cases, demonstrating that
geometric fidelity, not intensity agreement with biased ground truth, aligns with clinical requirements.
Read the full abstract ▾
All authors Lukas Zimmermann, Michael Rauter, Maximilian Schmid, Dietmar Georg, Barbara Knäusl
Computer Vision
Toxicity Assessment in Preclinical Histopathology via Class-Aware Mahalanobis Distance for Known and Novel Anomalies
Olga Graf, Dhrupal Patel, Peter Groß et al. (+3 more)
Drug-induced toxicity is a leading cause of preclinical and early-clinical failure, making early detection critical. Histopathology is the gold standard for toxicity assessment but relies on expert pathologists, creating a bottleneck for large-scale screening. We introduce an AI-based anomaly detection framework for whole-slide images (WSIs) of rodent liver that identifies healthy tissue and known pathologies (anomalies) and flags samples without training data as out-of-distribution (OOD). We evaluate OOD detection on two held-out categories: apoptosis (single-
cell, near-OOD) and staining/processing artifacts (heterogeneous, far-OOD). We build a novel pixelwise-annotated
dataset and fine-tune a pre-trained
Vision Transformer (DINOv2) via Low-Rank Adaptation (LoRA) for
segmentation, then use the Mahalanobis distance for OOD detection with class-specific thresholds. Optimizing the false positive rate subject to a predefined constraint on the false negative rate yields only 0.16% of pathological tissue classified as healthy and 0.35% of healthy tissue classified as pathological. Our false negative rate does not penalise cross-type errors, reflecting the safety-first objective of never overlooking a lesion; under the stricter correct-class criterion our method assigns 93.93% of ID and 89.38% of OOD findings to their own class. The study demonstrates technical feasibility of pixel-level anomaly detection for mouse liver histopathology, indicating possible applications in improving preclinical workflows and drug development efficiency.
Read the full abstract ▾
All authors Olga Graf, Dhrupal Patel, Peter Groß, Charlotte Lempp, Matthias Hein, Fabian Heinemann
Computer Vision
MLV-Edit: Towards Consistent and Highly Efficient Editing for Minute-Level Videos
Yangyi Cao, Yuanhang Li, Lan Chen et al. (+1 more)
We propose MLV-Edit, a training-free, flow-based framework that address the unique challenges of minute-level
video editing. While existing techniques excel in short-form
video manipulation, scaling them to long-duration
videos remains challenging due to prohibitive computational overhead and the difficulty of maintaining global temporal consistency across thousands of frames. To address this, MLV-Edit employs a divide-and-conquer strategy for segment-wise editing, facilitated by two core modules: Velocity Blend rectifies motion inconsistencies at segment boundaries by aligning the flow fields of adjacent chunks, eliminating flickering and boundary artifacts commonly observed in fragmented
video processing; and
Attention Sink anchors local segment features to global reference frames, effectively suppressing cumulative structural drift. Extensive quantitative and qualitative experiments demonstrate that MLV-Edit consistently outperforms state-of-the-art methods in terms of temporal stability and semantic fidelity.
Read the full abstract ▾
All authors Yangyi Cao, Yuanhang Li, Lan Chen, Qi Mao
Computer Vision
Enhancing Diffusion-Based Quantitatively Controllable Image Generation via Matrix-Form EDM and Adaptive Vicinal Training
Xin Ding, Yun Chen, Sen Zhang et al. (+5 more)
Continuous Conditional
Diffusion Model (CCDM) is a
diffusion-based framework designed to generate high-quality images conditioned on continuous regression labels. Although CCDM has demonstrated clear advantages over prior approaches across a range of
datasets, it still exhibits notable limitations and has recently been surpassed by a GAN-based method, namely CcGAN-AVAR. These limitations mainly arise from its reliance on an outdated
diffusion framework and its low sampling efficiency due to long sampling trajectories. To address these issues, we propose an improved CCDM framework, termed iCCDM, which incorporates the more advanced Elucidated
Diffusion Model (EDM) framework with substantial modifications to improve both generation quality and sampling efficiency. Specifically, iCCDM introduces a novel
matrix-form EDM formulation together with an adaptive vicinal training strategy. Extensive experiments on four
benchmark datasets, spanning image resolutions from $64\times64$ to $256\times256$, demonstrate that iCCDM consistently outperforms existing methods, including state-of-the-art large-scale text-to-image
diffusion models (e.g., Stable
Diffusion 3, FLUX.1, and Qwen-Image), achieving higher generation quality while significantly reducing sampling cost.
Read the full abstract ▾
All authors Xin Ding, Yun Chen, Sen Zhang, Kao Zhang, Nenglun Chen, Peibei Cao, Yongwei Wang, Fei Wu
Computer Vision · Cross-listed from cs.LG
An Empirical Study of World Model Quantization
Zhongqian Fu, Tianyi Zhao, Kai Han et al. (+3 more)
World models learn an internal representation of environment dynamics, enabling
agents to simulate and reason about future states within a compact latent space for tasks such as planning, prediction, and inference. However, running world models rely on hevay computational cost and
memory footprint, making model quantization essential for efficient deployment. To date, the effects of post-training quantization (PTQ) on world models remain largely unexamined. In this work, we present a systematic empirical study of world model quantization using DINO-WM as a representative case, evaluating diverse PTQ methods under both weight-only and joint weight-activation settings. We conduct extensive experiments on different visual planning tasks across a wide range of bit-widths, quantization granularities, and planning horizons up to 50 iterations. Our results show that quantization effects in world models extend beyond standard accuracy and bit-width trade-offs: group-wise weight quantization can stabilize low-bit rollouts, activation quantization granularity yields inconsistent benefits, and quantization sensitivity is highly asymmetric between encoder and predictor modules. Moreover, aggressive low-bit quantization significantly degrades the
alignment between the planning objective and task success, leading to failures that cannot be remedied by additional optimization. These findings reveal distinct quantization-induced failure modes in world model-based planning and provide practical guidance for deploying quantized world models under strict computational constraints. The code will be available at https://github.com/huawei-noah/noah-research/tree/master/QuantWM.
Read the full abstract ▾
All authors Zhongqian Fu, Tianyi Zhao, Kai Han, Hang Zhou, Xinghao Chen, Yunhe Wang
Computer Vision
Teacher-Guided Student Self-Knowledge Distillation Using Diffusion Model
Yu Wang, Chuanguang Yang, Zhulin An et al. (+6 more)
Existing Knowledge
Distillation (KD) methods often align feature information between teacher and student by exploring meaningful feature processing and loss functions. However, due to the difference in feature distributions between the teacher and student, the student model may learn incompatible information from the teacher. To address this problem, we propose teacher-guided student
Diffusion Self-KD, dubbed as DSKD. Instead of the direct teacher-student
alignment, we leverage the teacher classifier to guide the sampling process of denoising student features through a light-weight
diffusion model. We then propose a novel locality-sensitive hashing (LSH)-guided feature
distillation method between the original and denoised student features. The denoised student features encapsulate teacher knowledge and could be regarded as a teacher role. In this way, our DSKD method could eliminate discrepancies in mapping manners and feature distributions between the teacher and student, while learning meaningful knowledge from the teacher. Experiments on visual recognition tasks demonstrate that DSKD significantly outperforms existing KD methods across various models and
datasets. Our code is attached in supplementary material.
Read the full abstract ▾
All authors Yu Wang, Chuanguang Yang, Zhulin An, Weilun Feng, Jiarui Zhao, Chengqing Yu, Libo Huang, Boyu Diao, Yongjun Xu
Computer Vision
FSVideo: Fast Speed Video Diffusion Model in a Highly-Compressed Latent Space
FSVideo Team, Qingyu Chen, Zhiyuan Fang et al. (+17 more)
We introduce FSVideo, a fast speed
transformer-based image-to-
video (I2V)
diffusion framework. We build our framework on the following key components: 1.) a new
video autoencoder with highly-compressed latent space ($64\times64\times4$ spatial-temporal downsampling ratio), achieving competitive reconstruction quality; 2.) a
diffusion transformer (DIT) architecture with a new layer
memory design to enhance inter-layer information flow and context reuse within DIT, and 3.) a multi-resolution generation strategy via a few-step DIT upsampler to increase
video fidelity. Our final model, which contains a 14B DIT base model and a 14B DIT upsampler, achieves competitive performance against other popular open-source models, while being an order of magnitude faster. We discuss our model design as well as training strategies in this report.
Read the full abstract ▾
All authors FSVideo Team, Qingyu Chen, Zhiyuan Fang, Haibin Huang, Xinwei Huang, Tong Jin, Minxuan Lin, Bo Liu, Celong Liu, Chongyang Ma, Xing Mei, Xiaohui Shen, Yaojie Shen, Fuwen Tan, Angtian Wang, Xiao Yang, Yiding Yang, Jiamin Yuan, Lingxi Zhang, Yuxin Zhang
Computer Vision
UrbanGS: A Scalable and Efficient Architecture for Geometrically Accurate Large-Scene Reconstruction
Changbai Li, Haodong Zhu, Hanlin Chen et al. (+6 more)
While 3D Gaussian Splatting (3DGS) enables high-quality, real-time rendering for bounded scenes, its extension to large-scale urban environments gives rise to critical challenges in terms of
geometric consistency,
memory efficiency, and computational scalability. To address these issues, we present UrbanGS, a scalable reconstruction framework that effectively tackles these challenges for city-scale applications. First, we propose a Depth-Consistent D-Normal Regularization module. Unlike existing approaches that rely solely on
monocular normal
estimators, which can effectively update rotation parameters yet struggle to update position parameters, our method integrates D-Normal constraints with external depth supervision. This allows for comprehensive updates of all
geometric parameters. By further incorporating an adaptive confidence weighting mechanism based on gradient consistency and inverse depth deviation, our approach significantly enhances multi-view depth
alignment and
geometric coherence, which effectively resolves the issue of
geometric accuracy in complex large-scale scenes. To improve scalability, we introduce a Spatially Adaptive Gaussian
Pruning (SAGP) strategy, which dynamically adjusts Gaussian density based on local
geometric complexity and visibility to reduce redundancy. Additionally, a unified partitioning and view assignment scheme is designed to eliminate boundary artifacts and optimize computational load. Extensive experiments on multiple urban
datasets demonstrate that UrbanGS achieves superior performance in rendering quality,
geometric accuracy, and
memory efficiency, providing a systematic solution for high-fidelity large-scale scene reconstruction.
Read the full abstract ▾
All authors Changbai Li, Haodong Zhu, Hanlin Chen, Xiuping Liang, Tongfei Chen, Shuwei Shao, Linlin Yang, Huobin Tan, Baochang Zhang
Computer Vision
Multi-View Stenosis Classification Leveraging Transformer-Based Multiple-Instance Learning Using Real-World Clinical Data
Nikola Cenikj, Özgün Turgut, Alexander Müller et al. (+6 more)
Coronary artery stenosis is a leading cause of cardiovascular disease, diagnosed by analyzing the coronary arteries from multiple angiography views. Although numerous deep-learning models have been proposed for stenosis detection from a single angiography view, their performance heavily relies on expensive view-level annotations, which are often not readily available in hospital systems. Moreover, these models fail to capture the temporal dynamics and dependencies among multiple views, which are crucial for clinical diagnosis. To address this, we propose SegmentMIL, a
transformer-based multi-view multiple-instance learning framework for patient-level stenosis classification. Trained on a real-world clinical
dataset, using patient-level supervision and without any view-level annotations, SegmentMIL jointly predicts the presence of stenosis and localizes the affected anatomical region, distinguishing between the right and left coronary arteries and their respective segments. SegmentMIL obtains high performance on internal and external evaluations and outperforms both view-level models and classical MIL baselines, underscoring its potential as a clinically viable and scalable solution for coronary stenosis diagnosis. Our code is available at https://github.com/NikolaCenic/mil-stenosis.
Read the full abstract ▾
All authors Nikola Cenikj, Özgün Turgut, Alexander Müller, Alexander Steger, Jan Kehrer, Marcus Brugger, Daniel Rueckert, Eimo Martens, Philip Müller
Computer Vision
Beyond Bag-of-Words: Diagnosing Compositional Binding Failures in Vision-Language Models
Cristian Sbrolli, Toshihiko Yamasaki and Matteo Matteucci
Modern
vision-
language models struggle with basic compositional
reasoning, failing to bind attributes to objects or relations to their referents. Existing
benchmarks either rely on noisy real images that conflate confounding visual variables with the
reasoning failure, or use simplistic synthetic scenes lacking the realism modern VLMs are tuned for. We introduce Auto-Comp, a fully automated, concept-driven pipeline that bridges this gap by generating photorealistic compositional
benchmarks at scale. Its core innovation is a parallel A/B construction: for each concept, the pipeline emits a Minimal sample (template caption, isolated objects on a white background) and a Contextual sample (LLM-rewritten caption, objects embedded in a realistic scene), isolating core binding ability from visio-linguistic complexity. We instantiate four task families spanning the two canonical axes of compositional binding: Color and Shape-Color (attribute binding), and Position and Relative Size (relational binding). We evaluate over 25 VLMs spanning CLIP, SigLIP, hard-negative-trained, and frontier generative models. The findings are consistent across architectures and scales: every model exhibits a large Swap-vs-Confusion gap, with low-
entropy distractors (e.g., repeated objects or colors) exposing failures beyond the known bag-of-words limitations. We further uncover a task-dependent trade-off: visio-linguistic context aids relational
reasoning but hinders attribute binding through visual clutter. We publicly release the pipeline and
benchmarks.
Read the full abstract ▾
Computer Vision
One Size, Many Fits: Aligning Diverse Group-Wise Click Preferences in Large-Scale Advertising Image Generation
Shuo Lu, Haohan Wang, Wei Feng et al. (+14 more)
Advertising image generation has increasingly focused on online metrics like Click-Through Rate (CTR), yet existing approaches adopt a “one-size-fits-all" strategy that optimizes for overall CTR while neglecting preference diversity among user groups. This leads to suboptimal performance for specific groups, limiting targeted marketing effectiveness. To bridge this gap, we present One Size, Many Fits (OSMF), a unified framework that aligns diverse group-wise click preferences in large-scale advertising image generation. OSMF begins with product-aware adaptive grouping, which dynamically organizes users based on their attributes and product characteristics, representing each group with rich collective preference features. Building on these groups, preference-conditioned image generation employs a Group-aware
Multimodal Large
Language Model (G-MLLM) to generate tailored images for each group. The G-MLLM is pre-trained to simultaneously comprehend group features and generate advertising images. Subsequently, we fine-tune the G-MLLM using our proposed Group-DPO for group-wise preference
alignment, which effectively enhances each group's CTR on the generated images. To further advance this field, we introduce the Grouped Advertising Image Preference
Dataset (GAIP), the first large-scale public
dataset of group-wise image preferences, including around 600K groups built from 40M users. Extensive experiments demonstrate that our framework achieves the state-of-the-art performance in both offline and online settings. Our code and
datasets will be released at https://github.com/JD-GenX/OSMF.
Read the full abstract ▾
All authors Shuo Lu, Haohan Wang, Wei Feng, Weizhen Wang, Shen Zhang, Yaoyu Li, Ao Ma, Zheng Zhang, Jingjing Lv, Junjie Shen, Ching Law, Bing Zhan, Yuan Xu, Huizai Yao, Yongcan Yu, Chenyang Si, Jian Liang
Computer Vision
Rethinking Genomic Modeling Through Optical Character Recognition
Hongxin Xiang, Pengsen Ma, Yunkang Cao et al. (+4 more)
Recent genomic foundation models largely adopt large
language model architectures that treat DNA as a one-dimensional
token sequence. However, exhaustive sequential reading is structurally misaligned with sparse and discontinuous genomic semantics, leading to wasted computation on low-information background and preventing understanding-driven compression for long contexts. Here, we present OpticalDNA, a
vision-based framework that reframes genomic modeling as Optical Character Recognition (OCR)-style document understanding. OpticalDNA renders DNA into structured visual layouts and trains an OCR-capable
vision–
language model with a visual DNA encoder and a document decoder, where the encoder produces compact, reconstructible visual
tokens for high-fidelity compression. Building on this representation, OpticalDNA defines
prompt-conditioned objectives over core genomic primitives-reading, region grounding, subsequence
retrieval, and masked span completion-thereby learning layout-aware DNA representations that retain fine-grained genomic information under a reduced effective
token budget. Across diverse genomic
benchmarks, OpticalDNA consistently outperforms recent baselines; on sequences up to 450k bases, it achieves the best overall performance with nearly 20$\times$ fewer effective
tokens, and surpasses models with up to 985$\times$ more activated parameters while tuning only 256k trainable parameters.
Read the full abstract ▾
All authors Hongxin Xiang, Pengsen Ma, Yunkang Cao, Di Yu, Haowen Chen, Xinyu Yang, Xiangxiang Zeng
Computer Vision
Gongli Xi, Kun Wang, Zeming Gao et al. (+5 more)
Large
multimodal reasoning models solve challenging visual problems via explicit long-chain inference: they gather visual clues from images and decode clues into textual
tokens. Yet this capability also increases
hallucinations, where the model generates content that is not supported by the input image or the question. To understand this failure mode, we identify
reasoning drift: during clue gathering, the model over-focuses on question-irrelevant entities, diluting focus on task-relevant cues and gradually decoupling the
reasoning trace from visual grounding. As a consequence, many inference-time localization or intervention methods developed for non-
reasoning models fail to pinpoint the true clues in
reasoning settings. Motivated by these insights, we introduce ClueRecall, a metric for assessing visual clue
retrieval, and present ClueTracer, a training-free, parameter-free, and architecture-agnostic plugin for
hallucination suppression. ClueTracer starts from the question and traces how key clues propagate along the model's
reasoning pathway (question $\rightarrow$ outputs $\rightarrow$ visual
tokens), thereby localizing task-relevant patches while suppressing spurious
attention to irrelevant regions. Remarkably, without any additional training, ClueTracer improves all
reasoning architectures (including R1-OneVision, Ocean-R1, MM-Eureka, etc.) by $\mathbf{1.21\times}$ on
reasoning benchmarks
Read the full abstract ▾
All authors Gongli Xi, Kun Wang, Zeming Gao, Huahui Yi, Haolang Lu, Qianshan Wei, Ye Tian, Wendong Wang
Computer Vision
UniDriveDreamer: A Single-Stage Multimodal World Model for Autonomous Driving
Guosheng Zhao, Yaozeng Wang, Xiaofeng Wang et al. (+10 more)
World models have demonstrated significant promise for data synthesis in autonomous driving. However, existing methods predominantly concentrate on single-modality generation, typically focusing on either multi-camera
video or
LiDAR sequence synthesis. In this paper, we propose UniDriveDreamer, a single-stage unified
multimodal world model for autonomous driving, which directly generates
multimodal future observations without relying on intermediate representations or cascaded modules. Our framework introduces a
LiDAR-specific variational autoencoder (VAE) designed to encode input
LiDAR sequences, alongside a
video VAE for multi-camera images. To ensure cross-modal compatibility and training stability, we propose Unified Latent Anchoring (ULA), which explicitly aligns the latent distributions of the two modalities. The aligned features are fused and processed by a
diffusion transformer that jointly models their
geometric correspondence and temporal
evolution. Additionally, structured scene layout information is projected per modality as a conditioning signal to guide the synthesis. Extensive experiments demonstrate that UniDriveDreamer outperforms previous state-of-the-art methods in both
video and
LiDAR generation, while also yielding measurable improvements in downstream
Read the full abstract ▾
All authors Guosheng Zhao, Yaozeng Wang, Xiaofeng Wang, Zheng Zhu, Tingdong Yu, Guan Huang, Yongchen Zai, Ji Jiao, Changliang Xue, Xiaole Wang, Zhen Yang, Futang Zhu, Xingang Wang
Computer Vision
SurfSplat: Conquering Feedforward 2D Gaussian Splatting with Surface Continuity Priors
Bing He, Jingnan Gao, Yunuo Chen et al. (+5 more)
Reconstructing 3D scenes from sparse images remains a challenging task due to the difficulty of recovering accurate geometry and texture without optimization. Recent approaches leverage generalizable models to generate 3D scenes using 3D Gaussian Splatting (3DGS) primitive. However, they often fail to produce continuous surfaces and instead yield discrete, color-biased point clouds that appear plausible at normal resolution but reveal severe artifacts under close-up views. To address this issue, we present SurfSplat, a feedforward framework based on 2D Gaussian Splatting (2DGS) primitive, which provides stronger anisotropy and higher
geometric precision. By incorporating a surface continuity prior and a forced alpha blending strategy, SurfSplat reconstructs coherent geometry together with faithful textures. Furthermore, we introduce High-Resolution Rendering Consistency (HRRC), a new evaluation metric designed to evaluate high-resolution reconstruction quality. Extensive experiments on RealEstate10K, DL3DV, and ScanNet demonstrate that SurfSplat consistently outperforms prior methods on both standard metrics and HRRC, establishing a robust solution for high-fidelity 3D reconstruction from sparse inputs. Project page: https://hebing-sjtu.github.io/SurfSplat-website/
Read the full abstract ▾
All authors Bing He, Jingnan Gao, Yunuo Chen, Ning Cao, Gang Chen, Zhengxue Cheng, Li Song, Wenjun Zhang
Computer Vision
Localized Control in Diffusion Models via Latent Vector Prediction
Pablo Domingo-Gregorio and Javier Ruiz-Hidalgo
Diffusion models emerged as a leading approach in text-to-image generation, producing high-quality images from textual descriptions. However, attempting to achieve detailed control to get a desired image solely through text remains a laborious trial-and-error endeavor. Recent methods have introduced image-level controls alongside with text
prompts, using prior images to extract conditional information such as edges,
segmentation and depth maps. While effective, these methods apply conditions uniformly across the entire image, limiting localized control. In this paper, we propose a novel methodology to enable precise local control over user-defined regions of an image, while leaving to the
diffusion model the task of autonomously generating the remaining areas according to the original
prompt. Our approach introduces a new training framework that incorporates masking features and an additional loss term, which leverages the prediction of the initial latent vector at any
diffusion step to enhance the correspondence between the current step and the final sample in the latent space. Extensive experiments demonstrate that our method effectively synthesizes high-quality images with controlled local conditions.
Read the full abstract ▾
Computer Vision
Enhancing Multi-Image Understanding through Delimiter Token Scaling
Minyoung Lee, Yeji Park, Dongjun Hwang et al. (+3 more)
Large
Vision-
Language Models (LVLMs) achieve strong performance on single-image tasks, but their performance declines when multiple images are provided as input. One major reason is the cross-image information leakage, where the model struggles to distinguish information across different images. Existing LVLMs already employ delimiter
tokens to mark the start and end of each image, yet our analysis reveals that these
tokens fail to effectively block cross-image information leakage. To enhance their effectiveness, we propose a method that scales the hidden states of delimiter
tokens. This enhances the model's ability to preserve image-specific information by reinforcing intra-image interaction and limiting undesired cross-image interactions. Consequently, the model is better able to distinguish between images and reason over them more accurately. Experiments show performance gains on multi-image
benchmarks such as Mantis, MuirBench, MIRB, and QBench2. We further evaluate our method on text-only tasks that require clear distinction. The method improves performance on multi-document and multi-table understanding
benchmarks, including TQABench, MultiNews, and WCEP-10. Notably, our method requires no additional training or inference cost.
Read the full abstract ▾
All authors Minyoung Lee, Yeji Park, Dongjun Hwang, Yejin Kim, Seong Joon Oh, Junsuk Choe