Computer Vision
GibbsLens: Training-Free Camera Control via Block-Conditional Gibbs Refinement
Yuzhu Wang, Xi Ye, Duo Su et al. (+2 more)
Training-free camera control for pretrained flow-matching
video generators is a partial-observation inverse problem: a depth-warped guidance
video supplies noisy evidence on a subset of latent sites, which the sampler must reconcile with the pretrained prior. Existing methods struggle to balance the trade-off between trajectory adherence and visual quality and the heuristic guidance-strength tuning lacks robustness. We propose GibbsLens, which resolves this dilemma through a structural change to the sampler: each outer hard-replacement guidance step is augmented with an inner-loop block-conditional pseudo-Gibbs refinement on the unobserved complement at the same noise level, with provable convergence to the partial-observation conditional data law. To accelerate convergence on high-dimensional
video latents, we exploit their conditional locality, partitioning the unobserved complement into 3D patches, each tracked by a custom mixing indicator that adaptively freezes converged patches. On RealEstate10K and DAVIS, GibbsLens attains the best FVD against all seven training-free and training-based competitors, outperforming every training-free baseline on every reported metric. Project page: https://xiye20.github.io/projects/GibbsLens/.
Read the full abstract ▾
All authors Yuzhu Wang, Xi Ye, Duo Su, Yangyang Xu, Jun Zhu
Computer Vision
FIS-DiT: Breaking the Few-Step Video Inference Barrier via Training-Free Frame Interleaved Sparsity
Jian Tang, Jiawei Fan, Qingbin Liu et al. (+3 more)
While the overall inference latency of
Video Diffusion Transformers (DiTs) can be substantially reduced through model
distillation, per-step inference latency remains a critical bottleneck. Existing acceleration paradigms primarily exploit redundancy across the denoising trajectory; however, we identify a limitation where these step-wise strategies encounter diminishing returns in few-step regimes. In such scenarios, fewer intermediate states can limit opportunities for feature reuse or predictive modeling, motivating complementary forms of acceleration. To overcome this, we propose Frame Interleaved Sparsity DiT (FIS-DiT), a training-free and operator-agnostic framework that shifts the optimization focus from the temporal trajectory to the latent frame dimension. Our approach is motivated by an intrinsic duality within this dimension: the existence of frame-wise sparsity that permits reduced computation, coupled with a structural consistency that calls for maintaining coverage of frame positions in the global spatiotemporal context. Leveraging this insight, we implement Frame Interleaved Sparsity (FIS) as an execution strategy that manipulates frame subsets across the model hierarchy, refreshing all latent positions without requiring full-scale block computation. Empirical evaluations on Wan 2.2 and HunyuanVideo 1.5 demonstrate that FIS-DiT reaches 2.4$\times$ speedup while maintaining comparable performance on VBench, providing a scalable and robust pathway toward real-time high-definition
video generation.
Read the full abstract ▾
All authors Jian Tang, Jiawei Fan, Qingbin Liu, Zheng Wei, Jiang Bian, Zang Li
Computer Vision
When Brains Disagree: Biological Ambiguity Underlies the Challenge of Amyloid PET Synthesis from Structural MRI
Louise E. G. Baron, Ross Callaghan, David M. Cash et al. (+3 more)
Structural MRI-to-amyloid PET synthesis has been proposed as a non-invasive alternative for amyloid assessment in Alzheimer's disease (AD). However, reported performance of identical models varies widely across studies, and increasingly complex architectures have not led to consistent gains. This inconsistency is thought to be caused by a fundamental biological ambiguity: MRI captures neurodegeneration, while PET measures amyloid pathology - two processes that are often temporally decoupled in AD. As a result, similar MRI patterns may correspond to different amyloid states, creating ambiguous one-to-many mappings. MRI-to-amyloid PET synthesis may therefore be intrinsically ill-posed; however, this idea has yet to be tested scientifically. The aim of this work is to test this hypothesis through two controlled experiments. We first control the training distribution by stratifying paired MRI-PET data by amyloid and neurodegeneration status. Using two standard synthesis models under a controlled design, we show that biologically unambiguous mappings are learnable in isolation, but performance collapses when data ambiguity is introduced. This demonstrates that ambiguity in the data distribution, rather than architectural capacity, constrains performance. Second, we show that introducing orthogonal biological information in the form of
plasma biomarkers resolves this ambiguity. When
multimodal inputs are incorporated, performance improves and stability is restored. Together, these findings suggest that limited and inconsistent performance in MRI-to-amyloid PET synthesis is explained by intrinsic biological ambiguity, and that stable, meaningful progress requires
multimodal integration rather than architectural complexity.
Read the full abstract ▾
All authors Louise E. G. Baron, Ross Callaghan, David M. Cash, Philip S. J. Weston, Hojjat Azadbakht, Hui Zhang
Computer Vision · Cross-listed from cs.IR
Very Efficient Listwise Multimodal Reranking for Long Documents
Yiqun Sun, Pengfei Wei and Lawrence B. Hsieh
Listwise reranking is a key yet computationally expensive component in
vision-centric
retrieval and
multimodal retrieval-augmented generation (M-RAG) over long documents. While recent VLM-based rerankers achieve strong accuracy, their practicality is often limited by long visual-
token sequences and multi-step autoregressive decoding. We propose ZipRerank, a highly efficient listwise
multimodal reranker that directly addresses both bottlenecks. It reduces input length via a lightweight query-image early interaction mechanism and eliminates autoregressive decoding by scoring all candidates in a single forward pass. To enable effective learning, ZipRerank adopts a two-stage training strategy: (i) listwise pretraining on large-scale text data rendered as images, and (ii)
multimodal finetuning with VLM-teacher-distilled soft-ranking supervision. Extensive experiments on the MMDocIR
benchmark show that ZipRerank matches or surpasses state-of-the-art
multimodal rerankers while reducing LLM inference latency by up to an order of magnitude, making it well-suited for latency-sensitive real-world systems. The code is available at https://github.com/dukesun99/ZipRerank.
Read the full abstract ▾
Computer Vision
GATA2Floor: Graph attention for floor counting in street-view facades
Ngoc Tan Le, Tzoulio Chamiti, Eirini Papagiannopoulou et al. (+1 more)
Automated analysis of building facades from street-level imagery has great potential for urban analytics, energy assessment, and emergency planning. However, it requires
reasoning over spatially arranged elements rather than solely isolated detections. In this work, we model each facade as a graph over window/door detections with a vertical prior on edges. Additionally, we introduce GATA2Floor, a multi-head Graph
Attention v2 (GATv2) based model that predicts the global floor count of a building and, via learnable cross-
attention queries, softly assigns elements to latent floor slots, yielding interpretable outputs and robustness to irregular designs. To mitigate the lack of labeled
datasets, we demonstrate that the proposed graph-based
reasoning can be applied without annotations by leveraging a lightweight label-free proposal mechanism based on self-supervised features and
vision-
language scoring. Our approach demonstrates the value of graph-
attention-based relational
reasoning for facade understanding.
Read the full abstract ▾
All authors Ngoc Tan Le, Tzoulio Chamiti, Eirini Papagiannopoulou, Nikos Deligiannis
Computer Vision
UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
Houcheng Jiang, Jiajun Fu, Junfeng Fang et al. (+4 more)
Multimodal large
language models are increasingly expected to perform thinking with images, yet existing visual latent
reasoning methods still rely on explicit textual chain-of-thought interleaved with visual latent
tokens. This interleaved design limits efficiency and keeps
reasoning fragmented across separate text and
vision channels. We propose UniVLR, a unified visual latent
reasoning framework that treats textual
reasoning and auxiliary visual evidence as a shared visual workspace. Instead of preserving text CoT as an independent inference-time path, UniVLR renders
reasoning traces together with auxiliary images and learns to compress this unified representation into compact visual latent
tokens. At inference time, the model reasons only through visual latents and directly decodes the final answer, avoiding both external tool calls and verbose text
reasoning. Experiments on real-world perception and visual
reasoning tasks show that UniVLR outperforms prior visual latent
reasoning methods while using substantially fewer generated
reasoning tokens, suggesting a more unified and efficient paradigm for visual thinking in MLLMs.
Read the full abstract ▾
All authors Houcheng Jiang, Jiajun Fu, Junfeng Fang, Chen Gao, Xiang Wang, Xiangnan He, Yong Li
Computer Vision
Selection, Not Fusion: Radar-Modulated State Space Models for Radar-Camera Depth Estimation
Zhangcheng Hou and Tomoaki Ohtsuki
Radar camera depth estimation turns a very sparse but metric radar signal into a dense depth map. Existing methods combine radar and image features outside the sequence operator of the backbone, through concatenation, confidence gating, sparse supervision, or graph based extraction. Mamba variants for
multimodal fusion also leave the selection mechanism itself unimodal. We argue that selection is the right place for radar to enter. We introduce Radar-Modulated Selection (RMS), which injects radar inside the selective scan of Mamba. Radar adds learned perturbations, initialised at zero, to the step size $\Delta$ and the readout $\mathbf{C}$. The input projection $\mathbf{B}$ and the state
matrix $\mathbf{A}$ never see radar. At initialisation the block equals a pretrained image Mamba, so radar gains influence only where it improves accuracy. The construction also couples the two modalities at linear cost and falls back to the image backbone when radar is missing. We deploy RMS in a Multi-View Scan Pyramid (MVSP) that matches the fusion operator to the reach of radar at each scale. On nuScenes, SemoDepth reduces MAE by 34.0%, 29.9%, and 29.9% over the previous best within 50, 70, and 80 m. It also has the lowest single frame latency among methods with a public release (26.8 ms). A feature blend placed before the scan adds no accuracy on top of RMS, which supports the view that selection inside the scan can replace fusion outside it. When twice as many spurious radar returns as true ones are injected, the MAE@80 of SemoDepth rises by 1.4%, against 33.6% for a Mamba baseline that fuses radar before the scan.
Read the full abstract ▾
Computer Vision
REFNet++: Multi-Task Efficient Fusion of Camera and Radar Sensor Data in Bird's-Eye Polar View
Kavin Chandrasekaran, Sorin Grigorescu, Gijs Dubbelman et al. (+1 more)
A realistic view of the vehicle's surroundings is generally offered by camera sensors, which is crucial for environmental perception. Affordable radar sensors, on the other hand, are becoming invaluable due to their robustness in variable weather conditions. However, because of their noisy output and reduced classification capability, they work best when combined with other sensor data. Specifically, we address the challenge of
multimodal sensor fusion by aligning radar and camera data in a unified domain, prioritizing not only accuracy, but also computational efficiency. Our work leverages the raw range-Doppler (RD) spectrum from radar and front-view camera images as inputs. To enable effective fusion, we employ a variational encoder-decoder architecture that learns the transformation of front-view camera data into the Bird's-Eye View (BEV) polar domain. Concurrently, a radar encoder-decoder learns to recover the angle information from the RD data that produce Range-Azimuth (RA) features. This
alignment ensures that both modalities are represented in a compatible domain, facilitating robust and efficient sensor fusion. We evaluated our fusion strategy for vehicle detection and free space
segmentation against state-of-the-art methods using the RADIal
dataset.
Read the full abstract ▾
All authors Kavin Chandrasekaran, Sorin Grigorescu, Gijs Dubbelman, Pavol Jancura
Computer Vision
RevealLayer: Disentangling Hidden and Visible Layers via Occlusion-Aware Image Decomposition
Binhao Wang, Shihao Zhao, Bo Cheng et al. (+6 more)
Recent
diffusion-based approaches have made substantial progress in image layer decomposition. However, accurately decomposing complex natural images remains challenging due to difficulties in occlusion completion, robust layer disentanglement, and precise foreground boundaries. Moreover, the scarcity of high-quality multi-layer natural image
datasets limits advancement. To address these challenges, we propose RevealLayer, a
diffusion-based framework that decomposes an RGB image into multiple RGBA layers, enabling precise layer separation and reliable recovery of occluded content in natural images. RevealLayer incorporates three key components: (1) a Region-Aware
Attention module to disentangle hidden and visible layers; (2) an Occlusion-Guided Adapter to leverage contextual information to enhance overlapping regions; and (3) a composite loss to enforce sharp alpha boundaries and suppress residual artifacts. To support training and evaluation, we introduce RevealLayer-100K, a high-quality multi-layer natural image constructed through a collaboration between automated
algorithms and human annotation, and further establish RevealLayerBench for benchmarking layer decomposition in general natural scenes. Extensive experiments demonstrate that RevealLayer consistently outperforms existing approaches in layer decomposition.
Read the full abstract ▾
All authors Binhao Wang, Shihao Zhao, Bo Cheng, Qiuyu Ji, Yuhang Ma, Liebucha Wu, Shanyuan Liu, Dawei Leng, Yuhui Yin
Computer Vision · Cross-listed from cs.RO
See What Matters: Differentiable Grid Sample Pruning for Generalizable Vision-Language-Action Model
Yixu Feng, Zinan Zhao, Yanxiang Ma et al. (+4 more)
Vision-
Language-Action (VLA) models have shown remarkable promise in robotics manipulation, yet their high computational cost hinders real-time deployment. Existing
token pruning methods suffer from a fundamental trade-off: aggressive compression using
pruning inevitably discards critical
geometric details like contact points, leading to severe performance degradation. This forces a compromise, limiting the achievable compression rate and thus the potential speedup. We argue that breaking this trade-off requires rethinking compression as a geometry-aware, continuous
token resampling in the
vision encoder. To this end, we propose the Differentiable Grid Sampler (GridS), a plug-and-play module that performs task-aware, continuous resampling of visual
tokens in VLA. By adaptively predicting a minimal set of salient coordinates and extracting features via differentiable interpolation, GridS preserves essential spatial information while achieving drastic compression (with fewer than 10% original visual
tokens). Experiments on both LIBERO
benchmark and a real robotic platform demonstrate that validating the lowest feasible visual
token count reported to date, GridS achieves a 76% reduction in FLOPs with no degradation in the success rate. The code is available at https://github.com/Fediory/Grid-Sampler.
Read the full abstract ▾
All authors Yixu Feng, Zinan Zhao, Yanxiang Ma, Chenghao Xia, Chengbin Du, Yunke Wang, Chang Xu
Computer Vision
Mitigating Action-Relation Hallucinations in LVLMs via Relation-aware Visual Enhancement
Zhenxin Qin, Qiang Li, Qingzhuo Wang et al. (+3 more)
Large
Vision-
Language Models (LVLMs) have achieved remarkable performance on diverse
vision-
language tasks. However, LVLMs still suffer from
hallucinations, generating text that contradicts the visual input. Existing research has primarily focused on mitigating object
hallucinations, but often overlooks more complex relation
hallucinations, particularly action relations involving interactions between objects. In this study, we empirically observe that the primary cause of action-relation
hallucinations in LVLMs is the insufficient
attention allocated to visual information. Thus, we propose a framework to locate action-relevant image regions and enhance the LVLM's
attention to those regions. Specifically, we define the Action-Relation Sensitivity (ARS) score to identify
attention heads that are most sensitive to action-relation changes, thereby localizing action-relevant image regions that contain key visual cues. Then, we propose the Relation-aware Visual Enhancement (RVE) method to enhance the LVLM's
attention to these action-relevant image regions. Extensive experiments demonstrate that, compared to existing baselines, our method achieves superior performance in mitigating action-relation
hallucinations with negligible additional inference cost. Furthermore, it effectively generalizes to spatial-relation
hallucinations and object
hallucinations.
Read the full abstract ▾
All authors Zhenxin Qin, Qiang Li, Qingzhuo Wang, Ruiyang Qin, Zhihua Wei, Wen Shen
Computer Vision · Cross-listed from cs.LG
Stop Marginalizing My Dreams: Model Inversion via Laplace Kernel for Continual Learning
Patryk Krukowski, Jacek Tabor, Przemysław Spurek et al. (+2 more)
Data-free continual learning (DFCIL) relies on model inversion to synthesize pseudo-samples and mitigate catastrophic forgetting. However, existing inversion methods are fundamentally limited by a simplifying assumption: they model feature distributions using diagonal covariance, effectively ignoring correlations that define the geometry of learned representations. As a result, synthesized samples often lack fidelity, limiting knowledge retention. In this work, we show that modeling feature dependencies is a key ingredient for effective DFCIL. We introduce REMIX, a structured covariance modeling framework that enables scalable full-covariance modeling without the prohibitive cost of dense
matrix inversion and log-determinant computation. By leveraging a Laplace kernel parameterization, REMIX captures structured feature dependencies using
memory that scales linearly with the feature dimensionality, while requiring only an additional logarithmic factor in computation. Modeling these correlations produces more coherent synthetic samples and consistently improves performance across standard DFCIL
benchmarks. Our results demonstrate that moving beyond diagonal assumptions is essential for effective and scalable data-free continual learning. Our code is available at https://github. com/pkrukowski1/REMIX-Model-Inversion-via-Laplace-Kernel.
Read the full abstract ▾
All authors Patryk Krukowski, Jacek Tabor, Przemysław Spurek, Marek Śmieja, Łukasz Struski
Computer Vision
OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models
Minseok Kang, Minhyeok Lee, Jungho Lee et al. (+6 more)
As
Video Large
Language Models (
Video-LLMs) scale to longer and more complex
videos, their inference cost grows rapidly due to the large volume of visual
tokens accumulated across frames. Training-free
token compression has emerged as a practical solution to this bottleneck. However, existing temporal compression methods rely primarily on cross-frame
token similarity or
segmentation heuristics, overlooking each
token's semantic role within its frame and failing to adapt compression strength to the compressibility of each frame pair. In this work, we propose OTT-Vid, a transport-derived allocation framework for temporal
token compression. Our approach consists of two stages: spatial
pruning identifies representative content within each frame, and optimal transport (OT) is then solved between neighboring frames to estimate temporal compressibility. We formulate this OT with non-uniform
token mass, which protects semantically important
tokens from aggressive compression, and a locality-aware cost that captures both feature and spatial disparities. The resulting transport plan jointly balances
token importance and matching cost, while its total cost defines the transport difficulty of each frame pair, which we use to allocate compression budgets dynamically. Experiments on six
benchmarks spanning
video question answering and temporal grounding show that OTT-Vid preserves 95.8% of VQA and 73.9% of VTG performance while retaining only 10% of
tokens, consistently outperforming existing state-of-the-art training-free compression methods.
Read the full abstract ▾
All authors Minseok Kang, Minhyeok Lee, Jungho Lee, Minjung Kim, Donghyeong Kim, Dayeon Lee, Heeseung Choi, Ig-jae Kim, Sangyoun Lee
Computer Vision
SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions
Markus Essl, Marta Moscati, Mubashir Noman et al. (+4 more)
Multimodal sensor fusion has demonstrated remarkable performance improvements over unimodal approaches in 3D object detection for autonomous vehicles. Typically, existing methods transform
multimodal data from independent sensors, such as camera and
LiDAR, into a unified bird's-eye view (BEV) representation for fusion. Although effective in ideal conditions, this strategy suffers from substantial performance deterioration when camera or
LiDAR data are missing, corrupted, or noisy. To address this vulnerability, we develop a framework-agnostic fusion module for camera and
LiDAR data that allows for handling cases when one of the two modalities is missing or corrupted. To demonstrate the effectiveness of our module, we instantiate it in BEVFusion [1], a well-established framework to combine camera and
LiDAR data for 3D object detection. By means of quantitative experiments on the MultiCorrupt
dataset, we demonstrate that our module achieves favorable performance improvements under scenarios of missing and corrupted modalities, substantially outperforming existing unified representation approaches across a wide range of sensor deterioration scenarios and reaching state-of-the-art performance in scenarios of corrupted modality due to extreme weather conditions and sensor failure.
Read the full abstract ▾
All authors Markus Essl, Marta Moscati, Mubashir Noman, Muhammad Zaigham Zaheer, Usman Naseem, Shah Nawaz, Markus Schedl
Computer Vision
Urban Risk-Aware Navigation via VQA-Based Event Maps for People with Low Vision
Antoni Valls and Jordi Sanchez-Riera
Visual impairment affects hundreds of millions of people worldwide, severely limiting their ability to navigate urban environments safely and independently. While wearable assistive devices offer a promising platform for real-time hazard detection, existing approaches rely on task-specific
vision pipelines that lack flexibility and generalizability. In this work, we propose an event map framework based on visual question answering that leverages
Vision-
Language Models (VLMs) for pedestrian scene description and hazard identification across diverse real-world environments, using a three-level hierarchical query structure to enable fine-grained scene understanding without task-specific retraining. Model responses are aggregated into a weighted risk scoring system that maps street segments into four discrete safety categories, producing navigable risk-aware event maps for route planning. To support evaluation and future research, we introduce a geographically diverse
dataset spanning 20 cities across six continents, comprising over 800 annotated images and 18,000 answered questions. We
benchmark four VQA architectures -ViLT, LLaVA, InstructBLIP, and Qwen-VL- and find that generative
Multimodal Large
Language Models (MLLMs) substantially outperform classification-based approaches, with Qwen-VL achieving the best overall balance of precision and recall. These results demonstrate the viability of MLLMs as a flexible and generalizable foundation for assistive
navigation systems for visually impaired people.
Read the full abstract ▾
Computer Vision
Revisiting Shadow Detection from a Vision-Language Perspective
Yonghui Wang, Shaokai Liu, Wengang Zhou et al. (+2 more)
Shadow detection is commonly formulated as a
vision-driven dense prediction problem, where models rely primarily on pixel-wise visual supervision to distinguish shadows from non-shadow regions. However, this formulation can become unreliable in visually ambiguous cases, where similar dark regions may correspond either to cast shadows or to intrinsically dark surfaces, making visual evidence alone insufficient for establishing a stable decision rule. In this work, we revisit shadow detection from a
vision–
language perspective and argue that robust prediction benefits from an explicit semantic reference beyond visual cues alone. We propose SVL, a Shadow
Vision–
Language framework that uses
language as an explicit semantic reference to disambiguate shadows from visually similar dark regions. SVL aligns global image representations with shadow-related text
embeddings through scene-level shadow ratio regression, and transfers this semantic guidance to dense prediction via global-to-local coupling and local patch-level constraints. Built on a frozen DINOv3 image encoder, SVL learns only lightweight projection and decoding modules, yielding a parameter-efficient design with less than $1\%$ trainable parameters. Extensive experiments on multiple shadow detection
benchmarks, including dedicated hard-case evaluations, suggest strong overall performance and improved robustness under visually ambiguous conditions. Code is available at https://github.com/harrytea/SVL.
Read the full abstract ▾
All authors Yonghui Wang, Shaokai Liu, Wengang Zhou, Hao Feng, Houqiang Li
Computer Vision
M$^4$-SAM: Multi-Modal Mixture-of-Experts with Memory-Augmented SAM for RGB-D Video Salient Object Detection
Jiyuan Liu, Jia Lin, Xiaofei Zhou et al. (+3 more)
The Segment Anything Model 2 (SAM2) has emerged as a foundation model for universal
segmentation. Owing to its generalizable visual representations, SAM2 has been successfully applied to various downstream tasks. However, extending SAM2 to the RGB-D
video salient object detection (RGB-D VSOD) task encounters three challenges including limited spatial modeling of linear LoRA, insufficient employment of SAM's multi-scale features, and dependence of initialization on explicit
prompts. To address the issues, we present Multi-Modal Mixture-of-Experts with
Memory-Augmented SAM (M$^4$-SAM), which equips SAM2 with modality-related PEFT, hierarchical feature fusion, and
prompt-free
memory initialization. Firstly, we inject Modality-Aware MoE-LORA, which employs convolutional experts to encode local spatial priors and introduces a modality dispatcher for efficient multi-modal fine-tuning, into SAM2's encoder. Secondly, we deploy Gated Multi-Level Feature Fusion, which hierarchically aggregates multi-scale encoder features with an adaptive gating mechanism, to balance spatial details and semantic context. Finally, to conduct zero-shot VSOD without manual
prompts, we utilize a Pseudo-Guided Initialization, where a coarse mask is regarded as a pseudo prior and used to bootstrap the
memory bank. Extensive experiments demonstrate that M$^4$-SAM achieves the state-of-the-art performance across all evaluation metrics on three public RGB-D VSOD
datasets.
Read the full abstract ▾
All authors Jiyuan Liu, Jia Lin, Xiaofei Zhou, Runmin Cong, Deyang Liu, Zhi Liu
Computer Vision · Cross-listed from eess.IV
Rezkellah Noureddine Khiati, Pierre-Yves Brillet and Catalin Fetita
Unsupervised
segmentation of pulmonary pathologies in CT remains an open challenge due to the absence of annotated multi pathology cohorts and the failure of existing
diffusion-based methods to exploit the quantitative Hounsfield Unit (HU) signal that physically distinguishes tissue classes. To address this, we propose DiffSegLung,a framework that introduces
Diffusion Radiomic
Distillation, in which handcrafted radiomic descriptors serve as a physics grounded teacher to shape the bottleneck of a 3D
diffusion U-Net via a contrastive objective, transferring pathology discriminative structure into the learned representation without any annotations. At inference, the teacher is discarded and multitimestep bottleneck features are clustered by a Gaussian Mixture Model with HU-guided label assignment, followed by Sobel
Diffusion Fusion for boundary refinement. Evaluated on 190 expert annotated axial slices drawn from four heterogeneous CT cohorts, Diff-SegLung improves
segmentation across all four pathology classes over unsupervised baselines and improves generation fidelity over prior CT
diffusion models.
Read the full abstract ▾
Computer Vision
Focusable Monocular Depth Estimation
Yuxin Du, Tao Lin, Zile Zhong et al. (+7 more)
Monocular depth foundation models generalize well across scenes, yet they are typically optimized with uniform pixel-wise objectives that do not distinguish user-specified or task-relevant target regions from the surrounding context. We therefore introduce Focusable
Monocular Depth Estimation (FDE), a region-aware depth estimation task in which, given a specified target region, the model is required to prioritize foreground depth accuracy, preserve sharp boundary transitions, and maintain coherent global scene geometry. To prioritize task-critical region modeling, we propose FocusDepth, a
prompt-conditioned
monocular relative depth estimation framework that guides depth modeling to focus on target regions via box/text
prompts. The core Multi-Scale Spatial-Aligned Fusion (MSSA) in FocusDepth spatially aligns multi-scale features from Segment Anything Model 3 to the Depth Anything family and injects them through scale-specific, gated conditional fusion. This enables dense
prompt cue injection without disrupting
geometric representations, thereby endowing the depth estimation model with focused perception capability. To study FDE, we establish FDE-Bench, a target-centric
monocular relative depth
benchmark built from image-target-depth triplets across five
datasets, containing 252.9K/72.5K train/val triplets and 972 categories spanning real-world and
embodied simulation environments. On FDE-Bench, FocusDepth consistently improves over globally fine-tuned DA2/DA3 baselines under both box and text
prompts, with the largest gains appearing in target boundary and foreground regions while preserving global scene geometry. Ablations show that MSSA's spatial
alignment is the key design factor, as disrupting
prompt-geometry correspondence increases AbsRel by up to 13.8%.
Read the full abstract ▾
All authors Yuxin Du, Tao Lin, Zile Zhong, Runting Li, Xiyao Chen, Jiting Liu, Chenglin Liu, Ying-Cong Chen, Yuqian Fu, Bo Zhao
Computer Vision · Cross-listed from cs.LG
One-Step Generative Modeling via Wasserstein Gradient Flows
Jiaqi Han, Puheng Li, Qiushan Guo et al. (+3 more)
Diffusion models and flow-based methods have shown impressive generative capability, especially for images, but their sampling is expensive because it requires many iterative updates. We introduce W-Flow, a framework for training a generator that transforms samples from a simple reference distribution into samples from a target data distribution in a single step. This is achieved in two steps: we first define an
evolution from the reference distribution to the target distribution through a Wasserstein gradient flow that minimizes an energy functional; second, we train a static neural generator to compress this
evolution into one-step generation. We instantiate the energy functional with the Sinkhorn divergence, which yields an efficient optimal-transport-based update rule that captures global distributional discrepancy and improves coverage of the target distribution. We further prove that the finite-sample training dynamics converge to the continuous-time distributional dynamics under suitable assumptions. Empirically, W-Flow sets a new state of the art for one-step ImageNet 256$\times$256 generation, achieving 1.29 FID, with improved mode coverage and domain transfer. Compared to multi-step
diffusion models with similar FID scores, our method yields approximately 100$\times$ faster sampling. These results show that Wasserstein gradient flows provide a principled and effective foundation for fast and high-fidelity generative modeling.
Read the full abstract ▾
All authors Jiaqi Han, Puheng Li, Qiushan Guo, Renyuan Xu, Stefano Ermon, Emmanuel J. Candès
Computer Vision · Cross-listed from cs.RO
DreamAvoid: Critical-Phase Test-Time Dreaming to Avoid Failures in VLA Policies
Xianzhe Fan, Yuxiang Lu, Shenyuan Gao et al. (+4 more)
Vision-
Language-Action (VLA) models are often brittle in fine-grained manipulation, where minor action errors during the critical phases can rapidly escalate into irrecoverable failures. Since existing VLA models rely predominantly on successful demonstrations for training, they lack an explicit awareness of failure during these critical phases. To address this, we propose DreamAvoid, a critical-phase test-time dreaming framework that enables VLA models to anticipate and avoid failures. We also introduce an autonomous boundary learning paradigm to refine the system's understanding of the subtle boundary between success and failure. Specifically, we (1) utilize a Dream Trigger to determine whether the execution has entered a critical phase, (2) sample multiple candidate action chunks from the VLA via an Action Proposer, and (3) employ a Dream Evaluator, jointly trained on mixed data (success, failure, and boundary cases), to "dream" the short-horizon futures corresponding to the candidate actions, evaluate their values, and select the optimal action. We conduct extensive evaluations on real-world manipulation tasks and simulation
benchmarks. The results demonstrate that DreamAvoid can effectively avoid failures, thereby improving the overall task success rate. Across four real-world tasks, DreamAvoid achieves 72.5% success, compared with 48.8% for the base policy and 54.4% for GPC-RANK. Our code is available at https://github.com/XianzheFan/DreamAvoid.
Read the full abstract ▾
All authors Xianzhe Fan, Yuxiang Lu, Shenyuan Gao, Xiaoyang Wu, Ruihua Han, Manling Li, Hengshuang Zhao
Computer Vision
BronchoLumen: Analysis of recent YOLO-based architectures for real-time bronchial orifice detection in video bronchoscopy
Yongchao Li and Marian Himstedt
Bronchoscopy is routinely conducted in pulmonary clinics and intensive care units, but navigating the complex branching of the respiratory tract remains challenging. This paper introduces BronchoLumen, a real-time YOLO-based system for detecting bronchial orifices in
video bronchoscopy, aiming to assist
navigation and CAD systems. The paper investigates if bronchial orifices can be robustly detected across image domains using state-of-the-art object detection and a limited set of public image data. The study includes the description and comparison of YOLOv8, a widely adopted architecture, and YOLOv12, a more recent architecture integrating
attention-based modules to improve spatial
reasoning. Both models are trained and tested solely on publicly available
datasets comprising different image domains. A comparison of both models is conducted based on the common metrics mAP@0.5 and mAP@0.5:0.9 with the latter emphasizing localization accuracy. For YOLOv8 we obtained a mAP@0.5 of 0.91 on an in-domain and 0.68 on a cross-domain test set. YOLOv12 achieved 0.84 and 0.68 respectively with slightly better localization accuracy with mAP@0.5:0.9 of 0.48 and 0.26 compared to YOLOv8 with 0.45 and 0.25. Challenges like motion blur and low contrast occasionally entailed uncertainties but the system demonstrated overall robustness in most scenarios. BronchoLumen is an open-weight, YOLO-based solution for bronchial orifice detection offering high accuracy and efficiency across multiple image domains. While the more recent YOLOv12 achieves better localization accuracy, we observed a slightly worse precision. The models have been made publicly available to foster further research in bronchoscopy
navigation.
Read the full abstract ▾
Computer Vision
WorldComp2D: Spatio-semantic Representations of Object Identity and Location from Local Views
SeongMin Jin and Doo Seok Jeong
Learning latent representations that capture both semantic and spatial information is central to efficient spatio-semantic
reasoning. However, many existing approaches rely on implicit latent structures combined with dense feature maps or task-specific heads, limiting computational efficiency and flexibility. We propose WorldComp2D, a novel lightweight representation learning framework that explicitly structures latent space geometry according to object identity and spatial proximity using multiscale local receptive fields. This framework consists of (i) a proximity-dependent encoder that maps a given observation into a spatio-semantic latent space and (ii) a localizer that infers the coordinates of objects in the input from the resulting spatio-semantic representation. Using facial landmark localization as a proof-of-concept, we show that, compared to SoTA lightweight models, WorldComp2D reduces the numbers of parameters and FLOPs by up to 4.0X and 2.2X, respectively, while maintaining real-time performance on CPU. These results demonstrate that explicitly structured latent spaces provide an efficient and general foundation for spatio-semantic
reasoning. This framework is open-sourced at https://github.com/JinSeongmin/WorldComp2D.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Allegory of the Cave: Measurement-Grounded Vision-Language Learning
Kepeng Xu, Li Xu, Gang He et al. (+1 more)
Vision-
language models typically reason over post-ISP RGB images, although RGB rendering can clip, suppress, or quantize sensor evidence before inference. We study whether grounding improves when the visual interface is moved closer to the underlying camera measurement. We formulate measurement-grounded
vision-
language learning and instantiate it as PRISM-VL, which combines RAW-derived Meas.-XYZ inputs, camera-conditioned grounding, and Exposure-Bracketed Supervision Aggregation for transferring supervision from RGB proxies to measurement-domain observations. Using a quality-controlled 150K instruction-tuning set and a held-out
benchmark targeting low-light, HDR, visibility-sensitive, and
hallucination-sensitive cases, PRISM-VL-8B reaches 0.6120 BLEU, 0.4571 ROUGE-L, and 82.66% LLM-Judge accuracy, improving over the RGB Qwen3-VL-8B baseline by +0.1074 BLEU, +0.1071 ROUGE-L, and +4.46 percentage points. These results suggest that part of VLM grounding error arises from information lost during RGB rendering, and that preserving measurement-domain evidence can improve
multimodal reasoning.
Read the full abstract ▾
All authors Kepeng Xu, Li Xu, Gang He, Wenxin Yu
Computer Vision
CaC: Advancing Video Reward Models via Hierarchical Spatiotemporal Concentrating
Jiyuan Wang, Huan Ouyang, Jiuzhou Lin et al. (+10 more)
In this paper, we propose Concentrate and Concentrate (CaC), a coarse-to-fine anomaly reward model based on
Vision-
Language Models. During inference, it first conducts a global temporal scan to anchor anomalous time windows, then performs fine-grained spatial grounding within the localized interval, and finally derives robust judgments via structured spatiotemporal Chain-of-Thought
reasoning. To equip the model with these capabilities, we construct the first large-scale generated
video anomaly
dataset with per-frame bounding-box annotations, temporal anomaly windows, and fine-grained attribution labels. Building on this
dataset, we design a three-stage progressive training paradigm. The model initially learns spatial and temporal anchoring through single- and multi-frame supervised fine-tuning, and then is optimized by a reinforcement learning strategy based on two-turn Group Relative Policy Optimization (GRPO). Beyond conventional accuracy rewards, we introduce Temporal and Spatial IoU rewards to supervise the intermediate localization process, effectively guiding the model toward more grounded and interpretable spatiotemporal
reasoning. Extensive experiments demonstrate that CaC can stably concentrate on subtle anomalies, achieving a 25.7% accuracy improvement on fine-grained anomaly
benchmarks and, when used as a reward signal, CaC reduces generated-
video anomalies by 11.7% while improving overall
video quality.
Read the full abstract ▾
All authors Jiyuan Wang, Huan Ouyang, Jiuzhou Lin, Chunyu Lin, Dewen Fan, Boheng Zhang, Haonan Fan, Honglie Wang, Yiyang Fan, Zhenlong Yuan, Yongrui Heng, Guosheng Lin, Fan Yang
Computer Vision
EPIC: Efficient Predicate-Guided Inference-Time Control for Compositional Text-to-Image Generation
Sunung Mun, Sunghyun Cho and Jungseul Ok
Recent text-to-image (T2I) generators can synthesize realistic images, but still struggle with compositional
prompts involving multiple objects, counts, attributes, and relations. We introduce EPIC (Efficient Predicate-Guided Inference-Time Control), a training-free inference-time refinement framework for compositional T2I generation. EPIC casts refinement as predicate-guided search: it parses the original
prompt once into a visual program of object variables and typed predicates, then verifies each candidate image against this program using visual evidence extracted from that image. A candidate is judged to satisfy the
prompt when all predicates pass; otherwise, unmet predicates guide the next step, routing localized failures to targeted editing and layout-level or persistent failures to resampling. On GenEval2, EPIC improves the
prompt-level Soft-TIFA GM score from 34.16% for single-pass generation to 71.46%. With the same generator, editor, and image-model execution budget, EPIC outperforms the strongest baseline by 19.23 points. It also uses 31% fewer image-model executions, 72% fewer
multimodal large
language model (MLLM) calls, and 81% fewer MLLM
tokens per
prompt.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Unlocking Compositional Generalization in Continual Few-Shot Learning
Phu-Quy Nguyen-Lam, Phu-Hoa Pham, Dao Sy Duy Minh et al. (+3 more)
Object-centric representations promise a key property for few-shot learning: Rather than treating a scene as a single unit, a model can decompose it into individual object-level parts that can be matched and compared across different concepts. In practice, this potential is rarely realized. Continual learners either collapse scenes into global
embeddings, or train with part-level matching objectives that tie representations too closely to seen patterns, leaving them unable to generalize to truly novel concepts. In this paper, we identify this fundamental structural conflict and pioneer a new paradigm that strictly decouples representation learning from compositional inference. Leveraging the inherent patch-level semantic geometry of self-supervised
Vision Transformers (ViTs), our framework employs a dual-phase strategy. During training, slot representations are optimized entirely toward holistic class identity, preserving highly generalizable, object-level geometries. At inference, preserved slots are dynamically composed to match novel scenes. We demonstrate that this paradigm offers dual structural benefits: The frozen backbone naturally prevents representation drift, while our lightweight, holistic optimization preserves the features' capacity for novel-concept transfer. Extensive experiments validate this approach, achieving state-of-the-art unseen-concept generalization and minimal forgetting across standard continual learning
benchmarks.
Read the full abstract ▾
All authors Phu-Quy Nguyen-Lam, Phu-Hoa Pham, Dao Sy Duy Minh, Chi-Nguyen Tran, Huynh Trung Kiet, Long Tran-Thanh
Computer Vision
CAST: Collapse-Aware multi-Scale Topology Fusion for Multimodal Coreset Selection
Boran Zhao, Hetian Liu, Zhenxian Hu et al. (+3 more)
The training of large
multimodal models fundamentally relies on massive image-text
datasets, which inevitably incur prohibitive computational overhead.
Dataset selection offers a promising paradigm by identifying a highly informative coreset. However, existing approaches suffer from two critical limitations: (i) single-modality-dominated sampling methods, which ignore the fine-grained cross-modal information imbalance inherent in
multimodal datasets and thus lead to semantic loss in the other modality; and (ii) coarse-grained sample-scoring-based sampling methods, where the selected coreset tends to be biased toward the scoring model, making it difficult to guarantee distributional equivalence between the coreset and the original
dataset. Meanwhile, existing distribution matching and discrete sampling strategies often fail to jointly account for global semantic structure, local fine-grained details, and redundancy-aware coverage in dense regions. To this end, we propose CAST, a Collapse-Aware multi-Scale
Topology fusion framework for
multimodal coreset selection. We first construct image- and text-modality
topologies, and derive a unified
topology via local-collapse-aware refinement and cross-modal fusion. We then introduce a multi-scale distribution matching criterion in the
diffusion wavelet domain, encouraging the coreset to approximate the original
dataset at multiple scales. Finally, we introduce a local soft relational coverage mechanism that extends pure
geometric coverage to relation-aware indirect coverage, penalizing redundant selections in dense clusters. Extensive experiments on Flickr30K and MS-COCO show that CAST outperforms existing
dataset selection baselines, showcasing great superiority in cross-architecture generalization and energy efficiency over state-of-the-art
multimodal synthesis methods.
Read the full abstract ▾
All authors Boran Zhao, Hetian Liu, Zhenxian Hu, Yuqing Yuan, Yu Yan, Pengju Ren
Computer Vision
ScaleMoGen: Autoregressive Next-Scale Prediction for Human Motion Generation
Inwoo Hwang, Hojun Jang, Bing Zhou et al. (+3 more)
We present ScaleMoGen, a scale-wise autoregressive framework for text-driven human motion generation. Unlike conventional autoregressive approaches that rely on standard next-
token prediction, ScaleMoGen frames motion generation as a coarse-to-fine process. We quantize 3D motions into compositional discrete
tokens across multiple skeletal-emporal scales of increasing granularity, learning to generate motion by autoregressively predicting next-scale
token maps. To maintain structural integrity, our motion tokenizers and quantizers are explicitly designed so that discrete
tokens at every scale strictly preserve the skeletal hierarchy. Additionally, we employ bitwise quantization and prediction, which efficiently scale up the tokenizer vocabulary to preserve motion details and stabilize optimization. Extensive experiments demonstrate that ScaleMoGen achieves state-of-the-art performance, establishing an FID of 0.030 (vs. 0.045 for MoMask) on HumanML3D and a CLIP Score of 0.693 (vs. 0.685 for MoMask++) on the SnapMoGen
dataset. Furthermore, we demonstrate that our skeletal-temporal multi-scale representation naturally facilitates training-free, text-guided motion editing.
Read the full abstract ▾
All authors Inwoo Hwang, Hojun Jang, Bing Zhou, Jian Wang, Young Min Kim, Chuan Guo
Computer Vision
WildRelight: A Real-World Benchmark and Physics-Guided Adaptation for Single-Image Relighting
Lezhong Wang, Mehmet Onurcan Kaya, Siavash Bigdeli et al. (+1 more)
Recent single-image relighting methods, powered by advanced generative models, have achieved impressive photorealism on synthetic
benchmarks. However, their effectiveness in the complex visual landscape of the real world remains largely unverified. A critical gap exists, as current
datasets are typically designed for multi-view reconstruction and fail to address the unique challenges of single-image relighting. To bridge this synthetic-to-real gap, we introduce WildRelight, the first in-the-wild
dataset specifically created for evaluating single-image relighting models. WildRelight features a diverse collection of high-resolution outdoor scenes, captured under strictly aligned, temporally varying natural illuminations, each paired with a high-dynamic-range environment map. Using this data, we establish a rigorous
benchmark revealing that state-of-the-art models trained on synthetic data suffer from severe domain shifts. The strictly aligned temporal structure of WildRelight enables a new paradigm for domain adaptation. We demonstrate this by introducing a physics-guided inference framework that leverages the captured natural light
evolution as a self-supervised constraint. By integrating
Diffusion Posterior Sampling (DPS) with temporal Sampling-Aware Test-Time Adaptation (TTA), we show that the
dataset allows synthetic models to align with real-world statistics on-the-fly, transforming the intractable sim-to-real challenge into a tractable self-supervised task. The
dataset and code will be made publicly available to foster robust, physically-grounded relighting research.
Read the full abstract ▾
All authors Lezhong Wang, Mehmet Onurcan Kaya, Siavash Bigdeli, Jeppe Revall Frisvad