Computer Vision
Think While You Map: Asynchronous Vision-Language Agents for Incremental 3D Scene Graphs
Deniz Bickici, Michael Pabst, Shohei Mori et al. (+1 more)
Open-vocabulary 3D scene graph methods typically operate in two stages: first reconstruct, then enrich with
vision-
language models, leaving the graph unqueryable during exploration. We argue that this sequential coupling is unnecessary and propose an asynchronous architecture in which lightweight online mapping runs concurrently with heavyweight semantic refinement. A probabilistic voxel-based backbone maintains stable object identities incrementally, while background VLM
agents progressively enrich the graph. This framework resolves duplicate object tracks through semantic loop closure, attaches fine-grained visual attributes and derives spatial relations between objects. A multi-target frame scheduler amortizes VLM cost by selecting a small set of informative frames that jointly cover multiple targets. The resulting scene graph is queryable during exploration and grows in semantic richness over time. Our method matches or outperforms existing open-vocabulary 3D scene graph methods on semantic
segmentation (ScanNet, Replica) and surpasses the prior state-of-the-art across three visual grounding
benchmarks (Sr3D+, Nr3D, ScanRefer) by 15.3 to 18.8 A@0.25. Project page: https://denizbickici.github.io/thinkgraphs/
Read the full abstract ▾
All authors Deniz Bickici, Michael Pabst, Shohei Mori, Dieter Schmalstieg
Computer Vision
AeroVerse-SatAgent: UAV-Satellite Collaborative Spatial Reasoning Inspired by the Dual Visual Pathway Theory of Cognitive Neuroscience
Wenyi Zhang, Fanglong Yao, Youzhi Liu et al. (+5 more)
With the rapid advancement of aerospace
embodied intelligence, enabling Unmanned Aerial Vehicles (
UAVs) to autonomously understand and reason about complex environments has become increasingly important. However, existing
UAV-based spatial
reasoning approaches face critical limitations: single-view perception renders them vulnerable to occlusions and perspective distortions, while most VLMs lack explicit
geometric modeling, relying on semantic cues and yielding inconsistent
reasoning under viewpoint and scale variations. To address these challenges, we propose SatAgent, a
UAV-Satellite collaborative spatial
reasoning model inspired by the dual-pathway mechanism of the human visual system. By jointly leveraging satellite and
UAV perspectives, SatAgent enables robust, accurate
reasoning in complex urban environments. We first introduce a
Geometric-Aware 3D Reconstruction Encoder that elevates 2D
UAV features into explicit 3D spatial representations. Next, we design a multi-view
topology-semantic
alignment module integrating cross-view features within a unified BEV coordinate system. We further introduce a multi-view consistency loss encouraging viewpoint-invariant representations. Finally, we construct SatAgent-SR130K, the first large-scale
UAV-Satellite collaborative multi-view spatial
reasoning dataset. Experiments show SatAgent outperforms state-of-the-art general-purpose foundation models and specialized spatial
reasoning models by 25.91% and 11.69%, respectively, across diverse tasks, achieving particularly high accuracy in complex
geometric relationship
reasoning.
Read the full abstract ▾
All authors Wenyi Zhang, Fanglong Yao, Youzhi Liu, Peng Hu, Zhengqiu Zhu, Chen Gao, Xian Sun, Kun Fu
Computer Vision
Towards a foundational model for recognising diastematic Gregorian notation
Daniel Kurek and Jan Hajič jr
Optical recognition of Gregorian notation has recently been attempted with end-to-end methods, with four
datasets introduced. However, each of these
datasets is in a different encoding. We design a common encoding based on the S-GABC proposal, convert all four
datasets to this common encoding, and train a shared end-to-end foundational model for diastematic Gregorian notation that establishes a new state of the art across all four
datasets.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Revising RVL-CDIP: Quantifying Errors and Test-Train Overlap
Stefan Larson, Attila Nagy, Sam Desai et al. (+8 more)
RVL-CDIP is a popular
dataset for benchmarking document classifiers. However, the
dataset contains ample amounts of label errors as well as non-trivial amounts of test-train overlap, both of which may impact model performance metrics. In this paper, we address these two problems by (1) finding and fixing label errors, and (2) detecting and addressing test-train overlap. We produce several variations of RVL-CDIP with label error and test-train overlap fixes, and
benchmark document classification performance on these new RVL-CDIP variations. Our rigorous analysis of RVL-CDIP finds that the corpus contains 12% label error and approximately 35% test-train duplication. Remediation sees improvements in classification accuracy when errors are removed, but sees decreases in accuracy when duplicates are removed. We additionally evaluate models on RVL-CDIP-N, an out-of-distribution
benchmark, finding that training on error-corrected data substantially improves OOD generalization, with supervised models gaining an average of 8.1 percentage points in accuracy and improvements as large as 14 percentage points.
Read the full abstract ▾
All authors Stefan Larson, Attila Nagy, Sam Desai, Cyrus Desai, Nicole C. Lima, Yixin Yuan, Siddharth Betala, Kaushal K. Prajapati, Jamiu T. Suleiman, Sharad Duwal, Kevin Leach
Computer Vision
Temporal Training Strategies for Left Atrium and Left Atrial Appendage Segmentation in Dynamic Contrast 4DCT
David Montalvo-García, Lauren Severance, Elliot R. McVeigh et al. (+1 more)
Dynamic contrast-enhanced cardiac CT enables time-resolved analysis of contrast filling and washout in the left atrium (LA) and left atrial appendage (LAA), with potential applications for assessing blood stasis in atrial fibrillation (AF). Accurate
segmentation across all frames is required for such analysis but is challenging due to large temporal contrast variations and the use of a single annotation per registered sequence. This creates a trade-off between training for robustness and limiting label noise. In this study, we investigate how temporal training-set design affects nnUNet-based
segmentation of the LA and LAA in dynamic 4DCT. We compare training using a minimal two-frame
dataset reflecting standard clinical practice, a physiologically selected subset of frames, and the full 27-frame sequence. We further evaluate the impact of foreground-based normalization. Training with all frames yielded the best performance in early low-contrast phases. However, the physiologically selected subset achieved comparable performance from the filling phase onward. Applying normalization parameters derived from the full
dataset improved performance of reduced
datasets in low-contrast frames, but did not fully close the gap. These findings highlight the importance of temporal diversity in training data for robust
segmentation in dynamic CT, while indicating that carefully selected frame subsets may provide an effective trade-off between performance and efficiency for downstream applications.
Read the full abstract ▾
All authors David Montalvo-García, Lauren Severance, Elliot R. McVeigh, María J. Ledesma-Carbayo
Computer Vision
No Prompt, No Leaks: A Robust Generative Steganography Framework via Prompt-Free Diffusion
Jingwen Cai, Fen Xiao, Shuhua Deng et al. (+1 more)
Generative image steganography synthesizes stego images directly from secret information to achieve inherent security advantages. Latent
Diffusion Models (LDMs) have recently emerged as a fundamental image steganography framework that modulates secret latent representations with text
prompts. Limited by the inflexibility of text
prompts, these methods still struggle to generate high-quality stego images and accurately recover secret images. In this work, we propose a
prompt-free
diffusion image steganography framework that integrates style semantic priors to control more robust and reliable stego image generation. Specifically, a Cascaded Affine Coupling Module (CACM) establishes a bijective, deterministic mapping between a secret image and its latent representation. Then, style semantics are integrated into the
diffusion process to control latent representation and ensure visual imperceptibility in the generated stego images. To mitigate trajectory deviations stemming from the unconditioned reverse process, a predictor-corrector mechanism is introduced to iteratively refine the generation trajectory via feedback from the current and predicted next states. Extensive experimental results show that the proposed method achieves competitive performance compared to state-of-the-art methods in terms of security, secret image reconstruction accuracy and controllability.
Read the full abstract ▾
All authors Jingwen Cai, Fen Xiao, Shuhua Deng, Xieping Gao
Computer Vision
Temporal Preservation over Processing: Diagnosing and Designing Spatiotemporal Single-Stage Video Detectors
Karam Tomotaki-Dawoud, Anna Hilsmann, Peter Eisert et al. (+1 more)
Single-stage
video object detectors are increasingly deployed in time-critical applications, yet it remains unclear whether these models genuinely reason over temporal context or merely exploit a single informative frame-a gap hidden by standard metrics, which reward correct predictions regardless of how they are reached. We address this from two complementary directions: first, we propose TemporalLens, a model-agnostic diagnostic framework probing temporal dependence through controlled perturbations, structured occlusions, temporal shuffling, redundancy injection, and resolution degradation, revealing whether a detector actually uses information across time. Applied to stacked-frame 2D detectors and our YOLO-3D architecture, it exposes behavioural differences invisible to mAP: stacked 2D models collapse when the target frame is removed, while spatiotemporal models recover predictions from earlier frames, a signature of real temporal reliance. Second, we detail YOLO-3D, a modular real-time spatiotemporal detector built on YOLOv8, and show that simply preserving temporal depth through the backbone is the dominant performance driver (+3.7 pp mAP@50 at 32 frames averaged across scales). Together, the diagnostics and architecture turn "does this detector reason over time?" into a measurable, actionable question.
Read the full abstract ▾
All authors Karam Tomotaki-Dawoud, Anna Hilsmann, Peter Eisert, Sebastian Bosse
Computer Vision
Visual Semantic Entropy: Do Vision Language Models Recognize Visual Ambiguity?
Ta Duc Huy, Trang Nguyen, Townim Chowdhury et al. (+5 more)
Vision-
language models can produce confident answers on visually ambiguous inputs, resulting in biased predictions. Common
entropy-based methods, such as Semantic
Entropy (SE), rely on output diversity. Yet our analysis shows that overconfident visual
embeddings suppress output diversity under
stochastic decoding, causing SE to underestimate uncertainty in such cases. Recent methods instead probe output diversity through input perturbations, including textual paraphrasing or joint text-image perturbations, and show improved performance. We study these approaches and reveals that the resulting variability is often dominated by textual changes rather than visual evidence, causing uncertainty estimates to reflect
prompt sensitivity rather than visual ambiguity. We therefore propose Visual Semantic
Entropy (VSE), which perturbs only the image to probe nearby visual variations while keeping the text query fixed. VSE measures uncertainty by clustering generated answers into semantic prototypes and computing the mass-weighted dispersion among them. Extensive evaluation across five modern
vision-
language models and five diverse VQA
benchmarks demonstrates that VSE effectively captures visual ambiguity, establishing a new state-of-the-art for VLM uncertainty estimation.
Read the full abstract ▾
All authors Ta Duc Huy, Trang Nguyen, Townim Chowdhury, Ankit Yadav, Minh-Son To, Zhibin Liao, Johan W. Verjans, Vu Minh Hieu Phan
Computer Vision · Cross-listed from cs.LG
Resolving superposition in AI for interpretability and cross-modal alignment in patient-neuronal images
Jisung Park, Seohyeon Kang, Daeun Yoo et al. (+8 more)
Artificial intelligence is transforming our capability to solve biological challenges. In dimensionality bottleneck regimes exacerbated by high-dimensional biological data, neural networks force distinct concepts into the lower dimensions known as superposition. Although this superposition is widely known to hinder interpretability, its impact on corrupting the geometry of latent spaces remains critically overlooked. Here, we utilized sparse autoencoders (SAEs) trained on over 100,000 multiplexed images of patient-derived Parkinson's disease and healthy
neurons to resolve superposition. This approach bypasses the mathematical non-uniqueness of feature attribution by shifting to interpretable latent representation analysis. We theoretically and empirically demonstrate that superposition contaminates representational metric spaces, and thereby SAEs successfully recover
geometric fidelity. By treating these geometrically purified representations as single-
cell state vectors, we adapted single-
cell RNA sequencing (scRNA-seq) data analysis methodologies directly to the image domain. Finally, we introduce GW-map, utilizing Gromov-Wasserstein optimal transport to align these image representations with authentic scRNA-seq data de novo. This coupling reconstructs hierarchical neuronal pathology pathways such as Calcium-AIS scaffold, without reference spatial transcriptomics, establishing a scalable foundation for spatial biology. Code is available at https://github.com/jijihihi/Bio\_superposition
Read the full abstract ▾
All authors Jisung Park, Seohyeon Kang, Daeun Yoo, Eunsu Lee, Seoin Cho, Wooyeop Choi, Ian Choi, James R. Evan, Daesoo Kim, Sonia Gandhi, Minee L. Choi
Computer Vision
One Video, One World: Turning Monocular Video into Physical 4D Scenes
Junhao Chen, Boran Zhang, Mingjin Chen et al. (+7 more)
We introduce OVOW, the first training-free system that reconstructs instance-level, simulation-ready 4D mesh scenes from a single
monocular video. Recent 4D reconstruction achieves impressive rendering quality, but its outputs (\eg, implicit fields, Gaussian primitives, or point clouds) lack the watertight
topology, instance separation, and standardized physical interfaces required by physics simulators and
embodied AI. OVOW closes this gap with a four-stage pipeline: a
vision-
language model discovers, labels, and motion-classifies all instances; category-aware reconstruction yields per-instance meshes for rigid objects and
topology-consistent mesh sequences for deformable ones; an iterative render-match-optimize procedure recovers metric scale and 6-DoF pose trajectories; and physics-grounded assembly enforces ground contact and inter-object support. Crucially, we model all motion, rigid and non-rigid, through direct vertex deformation without category-specific priors or skeleton rigging, producing watertight mesh scenes ready for downstream physics simulation and editing. We further establish the first
benchmark for structured
Video-to-4D evaluation, with metrics for
geometric correctness, instance separation, and physical plausibility beyond visual fidelity; the same pipeline doubles as a scalable engine for synthesizing paired
video-to-4D simulation data for future 4D world models and
embodied AI. Across two synthetic
benchmarks (static and 4D), OVOW attains the best overall layout and geometry accuracy and the lowest photometric and semantic error among all baselines, and on
monocular video runs one to two orders of magnitude faster than the baselines, while downstream physics simulation confirms its physical stability.
Read the full abstract ▾
All authors Junhao Chen, Boran Zhang, Mingjin Chen, Henghaofan Zhang, Saining Zhang, Congcong Zhu, Hao Zhao, Ruqi Huang, Zhihao Li, Yufei Wang
Computer Vision
MS-Resampler: Multi-Scope Visual Resampling for Efficient Multimodal LLMs
Zhongyang Li, Yaqian Li, Faming Fang et al. (+6 more)
Multimodal large
language models (MLLMs) typically employ resampling-based projectors to transform dense visual features into a compact
token sequence for
language modeling. Most existing resamplers adopt a single, fixed aggregation scope via global cross-
attention, which can blur fine-grained local evidence and limit the ability to capture both local details and global context within a fixed
token budget. In this work, we propose MS-Resampler, a multi-scope visual resampling framework for MLLMs. MS-Resampler instantiates multiple scope-specific resamplers by injecting explicit spatial scope priors into the resampling
attention, enabling each branch to aggregate visual information at a particular granularity from local to global. The outputs of these scope-specific resamplers are then adaptively fused to produce the final visual representations for
language modeling. Extensive experiments on ten public
multimodal benchmarks show that MS-Resampler consistently improves visual understanding and
multimodal reasoning over conventional single-scope resamplers, while introducing only minimal computational overhead.
Read the full abstract ▾
All authors Zhongyang Li, Yaqian Li, Faming Fang, Rinyoichi Takezoe, Zi-Hao Bo, Cheng Qian, Mo Guang, Guixu Zhang, Kaiwen Long
Computer Vision
MAPE: Defending Against Transferable Adversarial Attacks Using Multi-Source Adversarial Perturbations Elimination
Xinlei Liu, Jichao Xie, Tao Hu et al. (+4 more)
Neural networks are vulnerable to meticulously crafted
adversarial examples, leading to high-confidence misclassifications in image classification tasks. Due to their consistency with regular input patterns and the absence of reliance on the target model and its output information, transferable
adversarial attacks exhibit a notably high stealthiness and detection difficulty, making them a significant focus of defense. In this work, we propose a deep learning defense known as multi-source
adversarial perturbations elimination (MAPE) to counter diverse transferable attacks. MAPE comprises the single-source
adversarial perturbation elimination (SAPE) mechanism and the pre-trained models probabilistic scheduling
algorithm (PPSA). SAPE utilizes a thoughtfully designed channel-
attention U-Net as the defense model and employs
adversarial examples generated by a pre-trained model (e.g., ResNet) for its training, thereby enabling the elimination of known
adversarial perturbations. PPSA introduces model difference quantification and negative momentum to strategically schedule multiple pre-trained models, thereby maximizing the differences among
adversarial examples during the defense model's training and enhancing its robustness in eliminating
adversarial perturbations. MAPE effectively eliminates
adversarial perturbations in various
adversarial examples, providing a robust defense against attacks from different substitute models. In a black-box attack scenario utilizing ResNet-34 as the target model, our approach achieves average defense rates of over 95.1% on CIFAR-10 and over 71.5% on Mini-ImageNet, demonstrating state-of-the-art performance.
Read the full abstract ▾
All authors Xinlei Liu, Jichao Xie, Tao Hu, Peng Yi, Yuxiang Hu, Shumin Huo, Zhen Zhang
Computer Vision
Domain Adaptive Object Detection via Dual-Stream Bilevel-Cycle Optimization
Yannan Chen, Wei Wang, Wenqiang Wang et al. (+5 more)
Cycle self-training (CST) breaks the shared classifier assumption of the standard self-training framework, which is effective for unsupervised domain adaptation and exploits unlabeled target data by training with target pseudo-labels. CST introduces a target classifier and employs an inner-outer loop updating strategy, addressing the issue of unreliable pseudo-labels and enabling pseudo-labels to generalize across domains. Despite its success in image classification, extending CST to object detection faces three main challenges. First, the upper bound of CST in object detection is constrained by three types of unreliable pseudo-labels, such as classification error alone, localization error alone, and their combination. Second, since object detection involves detecting multiple target objects, directly applying CST leads to training insta bility. Third, a wider numerical range of regression coordinates leads to exploding losses. To this end, we apply CST to both classification and regression and propose the Dual-Stream Bilevel-Cycle Optimization framework. Specifically, we construct CST upon Mean Teacher to prevent training instability and use extra normalization to map the regression bounding box into a standardized space, effectively addressing exploding losses. Also, we provide a theoretical derivation of the regression bound. Extensive experiments across four cross domain standard scenarios demonstrate that our framework achieves considerable results.
Read the full abstract ▾
All authors Yannan Chen, Wei Wang, Wenqiang Wang, Ruoyu Chen, Jiancheng Wang, Mingbo Yang, Yaowei Wang, Xiaochun Cao
Computer Vision · Cross-listed from cs.MM
Evidence Triangulation for Multimodal Fact-Checking in the Wild
Stefanos-Iordanis Papadopoulos, Zacharias Chrysidis, Christos Koutlis et al. (+2 more)
The proliferation of multimedia content on social platforms has fueled
multimodal misinformation, where images are used to reinforce false claims. Consequently,
Multimodal Fact-Checking (MFC) has emerged as an increasingly important research area. However, current progress is hindered by a reliance on synthetic training data and curated
benchmarks that fail to capture the complexity of in-the-wild data. Furthermore, existing detection models rely on restricted intra-modality consistency or unconstrained all-to-all fusion, failing to capture nuanced relations between posts and external evidence. To address these limitations, we introduce X-POSE, a
benchmark of real-world, community-annotated
multimodal posts from X (formerly Twitter), augmented with full-length news articles retrieved via VLM-optimized search. Additionally, we propose TRENT, a novel MFC model that performs evidence triangulation using three parallel cross-
attention streams alongside a relational fusion mechanism that explicitly models entailment and contradiction. Extensive evaluations demonstrate that TRENT consistently outperforms state-of-the-art specialized models and commercial VLMs. The code,
prompt templates, and
dataset are available at https://github.com/stevejpapad/evidence-triangulation
Read the full abstract ▾
All authors Stefanos-Iordanis Papadopoulos, Zacharias Chrysidis, Christos Koutlis, Symeon Papadopoulos, Panagiotis C. Petrantonakis
Computer Vision
Language-Assisted Super-Resolution from Real-World Low-Resolution Patches
Joonkyu Park and Kyoung Mu Lee
Single image super-resolution aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs. Training SR models typically requires paired HR-LR data, which is difficult to obtain in reality. As a result, most methods synthesize LR images by artificially degrading HR images with handcrafted kernels or camera ISP adjustments. However, these synthetic degradations fail to capture the complexity of real LR images, leading to poor generalization in practice. To address this, we observe that even within a single high-quality image, regions at different depths exhibit varying resolutions, where distant regions act as LR patches and closer ones as HR patches. This allows the extraction of real, degradation-induced LR patches from real images. Since these LR patches lack paired HR counterparts, we propose LA-SR (
Language Assistant for SR), a novel framework for unpaired SR. The key idea of LA-SR is to redefine unpaired SR in the
language space, using
vision-
language models to bridge the LR-HR gap. LA-SR projects images into a semantically rich space representing both content and quality, and applies two
language-guided losses: linguistic content loss to preserve semantic fidelity, and linguistic quality loss to enhance perceptual realism. With this
alignment, LA-SR effectively super-resolves real LR inputs, producing realistic outputs that overcome the limitations of synthetic-data-trained methods.
Read the full abstract ▾
Computer Vision
RCL-Mamba: A Dual-domain State Space Model for Measurement-oriented Image Restoration in Rotational Sparse-View Scanning Computed Laminography
Xuyang Duan, Genyuan Zhang, Zhenjiang Dong et al. (+4 more)
Rotational Scanning Computed Laminography (RCL) is widely utilized for the Non-Destructive Testing (NDT) of large planar components. However, to facilitate rapid inspection, continuous sparse-view scanning is often employed, where the angular integration effect during exposure induces rotational blur in the projection domain. Furthermore, the data incompleteness inherent in sparse sampling manifests as sparse artifacts in the reconstructed image domain. To address these cross-domain degradations, this paper proposes RCL-Mamba, a measurement-oriented dual-domain State Space Model (SSM)-based image restoration network. The framework adopts a cascaded joint processing strategy: it first corrects the rotational blur in the projection domain and subsequently suppresses the sparse artifacts in the image domain. Additionally, we design a Mamba-CNN dual-branch module to adaptively balance large-scale blur correction with local detail recovery. Evaluations on both simulated
datasets and real-world Printed
Circuit Board (PCB) scans demonstrate that RCL-Mamba outperforms existing baselines in blur removal, artifact suppression, and structural preservation. Line-profile-based structural measurement further verifies that the proposed method better preserves via/pad boundaries and slender trace profiles. Crucially, by reducing the required scanning views from 512 to 64, our method enhances inspection efficiency by approximately 8-fold without compromising reconstruction quality, offering a robust measurement-oriented restoration solution for high-throughput RCL inspection with improved structural measurement fidelity.
Read the full abstract ▾
All authors Xuyang Duan, Genyuan Zhang, Zhenjiang Dong, Chuandong Tan, Zihao Wang, Junyao Wang, Fenglin Liu
Computer Vision
Patient-Level Elbow Abnormality Detection: Leakage-Aware Evaluation of Learned Preprocessing, Calibration, and Triage-Oriented Operating Points
Ahmed Sallam and Ahmet Kaplan
In this study, we examine learned preprocessing pipelines in the context of triage-oriented orthopedic abnormality detection task using elbow radiographs from MURA
dataset. The evaluation focuses on patient-level detection of musculoskeletal abnormalities under a leakage-aware protocol. We compare multiple preprocessing pipelines, with and without a lightweight DnCNN module as a learned preprocessing component, to assess their impact on discrimination and calibration. Performance is assessed using discrimination metrics (AUROC, PR-AUC), calibration measures (ECE, Brier score), and validation-selected operating point analysis targeting high specificity. Results show that differences across preprocessing strategies are modest and configuration-dependent, with no consistent discrimination advantage over the raw-input DenseNet121 baseline. The raw and diverse inputs combined with the DnCNN front-end showed reduced ECE and Brier score, while CLAHE combined with DnCNN did not improve calibration. Overall, the results suggest that under patient-level evaluation, preprocessing gains are modest and configuration-dependent; the raw-input DenseNet121 baseline remains competitive throughout, and no tested preprocessing strategy produced a consistent discrimination advantage across all metrics.
Read the full abstract ▾
Computer Vision
Bridging Video Understanding and Generation in a Unified Framework
Yuqi Wang, Runyi Li, Ruoyu Feng et al. (+3 more)
Recently, unified image generation and understanding have been extensively explored. However, extending such unified modeling paradigms to the
video domain remains largely underexplored. A central challenge is that
video understanding favors compact, discriminative semantic representations, whereas
video generation requires dense signals that preserve visual details and temporal coherence.
Videos naturally capture both spatial semantics and temporal dynamics, making them a more suitable modality for unified
multimodal modeling compared to static images. In this paper, we propose Vega, a unified framework that bridges
video understanding and generation. Vega leverages a shared vocabulary to jointly model text and visual representations and employs a hybrid architecture combining autoregressive (AR) prediction with
diffusion-based rendering. Specifically, the AR model focuses on predicting semantically meaningful visual
tokens for keyframes, providing a structured representation that guides the
diffusion module in rendering dense, high-resolution
video frames. Extensive experiments demonstrate that Vega achieves strong performance on
video generation
benchmarks such as VBench and
video understanding
benchmarks like VideoMME.
Read the full abstract ▾
All authors Yuqi Wang, Runyi Li, Ruoyu Feng, Renjie Chen, Wenfeng Lin, Mingyu Guo
Computer Vision
Accelerated Likelihood Maximization for Diffusion-based Versatile Content Generation
Hyunsoo Lee, Inwoo Hwang and Young Min Kim
Generating diverse, coherent, and plausible content from partially given inputs remains a fundamental challenge for
diffusion models. Existing approaches face clear limitations: training-based approaches offer strong task-specific results but require costly computation, and they generalize poorly across tasks. Training-free approaches offer better efficiency, but they do not explicitly optimize over unobserved variables, leading to globally inconsistent results. To address these limitations, we introduce Accelerated Likelihood Maximization (ALM), a novel training-free sampling strategy integrated into the reverse
diffusion process that significantly extends the applicability of
diffusion models beyond simple generation tasks. Unlike previous methods that implicitly influence missing regions through pre-generated region constraints, we directly optimize the unobserved region during the sampling process, enabling globally coherent and plausible generation. Furthermore, we incorporate an acceleration strategy that significantly improves computational efficiency without sacrificing performance. Experimental results demonstrate that ALM consistently outperforms state-of-the-art methods in various data domains and tasks, establishing a powerful paradigm for versatile content generation.
Read the full abstract ▾
Computer Vision
Wavelet-Optimized Pseudo-3D Accelerated Diffusion Model for Truncated Computed Laminography
Genyuan Zhang, Junyao Wang, Chuandong Tan et al. (+2 more)
Computed Laminography (CL) is a key technology for the nondestructive testing of large plate-shaped objects. However, field-of-view (FOV) limitations inevitably lead to truncation of projected data, an ill-posed inverse problem that causes severe reconstruction artifacts. Existing deep learning methods typically rely on 2D architectures that lack rigorous data consistency constraints. Furthermore, they conventionally confine artifact removal strictly to the FOV, discarding potentially recoverable information outside it. To overcome these limitations, we first introduce a comprehensive CL FOV analysis, categorizing the space into data-complete, data-incomplete, and data-free regions. By extending our reconstruction target to encompass the data-incomplete region, we significantly expand the effective imaging range and enhance scanning efficiency. To achieve this, we propose a novel wavelet-optimized pseudo-3D accelerated
diffusion model for CL truncation reconstruction (CL-DM). Our method utilizes a standard 2D
diffusion model for slice aggregation, combined with a 3D model-based iterative reconstruction (MBIR) method to ensure strict data consistency. To mitigate inter-slice discontinuities, we introduce wavelet regularization along the z-direction, paired with a translation-invariant (TI) mechanism and a low-frequency preservation strategy. Finally, we introduce a 3D fast sampling architecture, significantly accelerating inference speed. Extensive simulations and real-world experiments demonstrate that CL-DM is superior in effectively eliminating truncation artifacts and restoring high-fidelity, continuous 3D structures.
Read the full abstract ▾
All authors Genyuan Zhang, Junyao Wang, Chuandong Tan, Fenglin Liu, Yongning Zhou
Computer Vision
Deep Spectral Models for Robust Dental Shape Generation
Tibor Kubík, François Guibault, Michal Španěl et al. (+1 more)
Accurate modeling of dental crown morphology is fundamental for diagnosis, orthodontic planning, and computer-aided restoration design. However,
datasets suitable for training such models are typically limited in size. We present ToothForge, a deep spectral generative framework that models dental crown geometries from compact, intrinsic representations. By operating in the spectral domain, ToothForge learns a latent
manifold of 3D tooth shapes through synchronized spectral
embeddings, ensuring consistent modeling across samples with varying connectivity. Spectral synchronization mitigates the instability of Laplace-Beltrami eigenbases and enables efficient learning in a low-dimensional space. The framework is thoroughly evaluated through robustness analysis, ablation studies, and benchmarking against PCA-based statistical shape models and point-based generative frameworks. Results show that synchronized spectral modeling achieves reconstruction and generative performance comparable to or exceeding spatial approaches, while maintaining compactness and
geometric interpretability. Together, the compact synchronized coefficients and low-dimensional learning space make the framework particularly suitable for limited
datasets, as often encountered in dental and medical domains, and applicable in real-world scenarios where guaranteeing consistent connectivity across shapes from various clinics is unrealistic.
Read the full abstract ▾
All authors Tibor Kubík, François Guibault, Michal Španěl, Hervé Lombaert
Computer Vision
Editing Everything Everywhere All at Once
Fabio Quattrini, Carmine Zaccagnino, Enis Simsar et al. (+4 more)
Editing multiple elements of an image in a single forward pass is a practical alternative to multi-turn image manipulation, offering improved efficiency and potentially better harmonization. However, when several instructions target different regions, semantic interference often leads to attribute leakage and poor edit disentanglement, especially as the number of edits increases. In this work, we propose MICE (Multi-Instance Concurrent Editing), a training-free strategy for scalable multi-instance image editing with
Multimodal Diffusion Transformers. MICE modifies the additive bias of joint
attention to regulate interactions between instance-specific edit instructions, latent, and context
tokens identified via user-provided
segmentation masks. Specifically, MICE allows intra-instance
attention, penalizes interactions between neighboring region
tokens, and suppresses unrelated cross-instance
attention. As a result, our method enforces attribute binding while preserving global visual consistency. We evaluate MICE on LoMOE-Bench and introduce MICE-Bench, a more challenging
benchmark with an average of 8.5 concurrent edits per image. The experiments demonstrate that our approach outperforms strong baselines and recent competitors in terms of visual quality preservation and faithfulness to the editing instructions.
Read the full abstract ▾
All authors Fabio Quattrini, Carmine Zaccagnino, Enis Simsar, Marta Tintoré Gazulla, Rita Cucchiara, Alessio Tonioni, Silvia Cascianelli
Computer Vision
CLIMB: Centroid-Based Hierarchical Memory for Online Continual Self-Supervised Learning
Julien Lefebvre, Stefan Duffner and Mathieu Lefort
Online Continual Self-Supervised Learning (OCSSL) aims to learn representations from a continuous stream of unlabeled data, without knowledge of task boundaries and under
memory constraints. Existing methods rely either on replay buffers that exploit latent space structure, or on regularization alone. We present CLIMB (Continual Learning with Intelligent
Memory Bank), which combines both simultaneously. Our method introduces a hierarchical centroid-based
memory, bounded in total number of stored images, combined with knowledge
distillation on replayed examples to limit representation drift. The
memory groups similar images into centroids, providing hard-to-discriminate examples for contrastive learning while covering the diversity of observed distributions. Experiments on Split CIFAR-100 and Split ImageNet-100, on standard
benchmarks from the state-of-the-art as well as a new protocol with irregular task distributions show that CLIMB outperforms state-of-the-art OCSSL methods.
Read the full abstract ▾
Computer Vision
Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents
Xueqiao Sun, Xiaohan Wang, Ludwig Schmidt et al. (+2 more)
Computer-use
agents, which leverage
multimodal large
language models (MLLMs) to operate computers and complete tasks, have attracted significant
attention for their utility and versatility. A major challenge in developing these
agents is collecting large-scale, high-quality trajectories. The standard approach generates synthetic data through a self-improving loop: an
agent is placed in a verifiable environment and iteratively fine-tuned on its successful trajectories. Despite its effectiveness, this paradigm exploits only successful trajectories and discards the failed ones, even though failures carry rich information about a model's weaknesses. In this work, we explore a complementary failure-driven self-improvement loop, a data-centric paradigm that turns failed trajectories into
agent improvements. Specifically, we employ an LLM to diagnose failure modes, propose inference-time solutions, and generate code patches – lightly verified by humans – that upgrade the
agent. We validate this approach with the state-of-the-art OpenCUA-72B model on the OSWorld
benchmark, improving the success rate from 42.3% to 48.9%, a gain of 6.6 percentage points, without any additional training cost and with only modest inference overhead. Our results demonstrate that failure-driven self-improvement is a viable complement to success-based pipelines, enabling more efficient
agent improvement.
Read the full abstract ▾
All authors Xueqiao Sun, Xiaohan Wang, Ludwig Schmidt, Serena Yeung-Levy, Yuhui Zhang
Computer Vision
WarpHammer: Densifying Scene Warps with 3D Object Priors for Extreme View Synthesis
Michael Green, Gavriel Habib, Dvir Samuel et al. (+4 more)
Projection-conditioned novel view synthesis (NVS) warps an explicit 3D reconstruction of the input view into the target camera and conditions a generator on the warped rendering. This works well for small viewpoint changes but degrades sharply under large orbital motion: the warp becomes sparse around the orbited object, where hidden surfaces dominate the new view and mirror-like artifacts emerge, causing the generator to lose both pixel content and the implicit camera cue carried by the warp. We introduce WarpHammer, a training-free framework that resolves this failure mode by augmenting the warped scene with an explicit 3D reconstruction of the object obtained from a native 3D generative prior (e.g., SAM3D). The reconstructed object adds missing foreground surfaces and occludes background points that should no longer be visible, restoring both appearance and camera cues without fine-tuning the base model. The same explicit object representation further unlocks a capability current NVS pipelines do not support: incorporating auxiliary views of the object from sources outside the target scene, for example, a casual snapshot of a car paired with a manufacturer studio shot of the same model. We process the reference and auxiliary images jointly with a pretrained multi-view geometry foundation model, which predicts a unified point cloud that we fuse into the 3D object reconstruction. This yields substantially more faithful geometry than single-image reconstruction, without requiring user-provided camera poses for the auxiliary views. On five
benchmarks, WarpHammer produces stable novel views at viewpoint deviations where strong baselines collapse, and is the first scene-level NVS method that can naturally fuse auxiliary, pose-unknown object views from an external source.
Read the full abstract ▾
All authors Michael Green, Gavriel Habib, Dvir Samuel, Tal Berkovitz Shalev, Issar Tzachor, Rami Ben-Ari, Or Litany
Computer Vision
Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoning
Chih-Ting Liao, Fei Shen, Xin Cao et al. (+1 more)
The standard way to read latent knowledge out of a model, a linear probe confirmed by a steering recovery, can systematically overstate what a
vision-
language model (VLM) actually grounds in the image. We show this on spatial
reasoning, where the error is invisible to both probing and steering yet exposed by a one-line causal control: replacing the image with a gray blank. Probes decode the within-axis answer at 73–97% across axes, and a training-free projection lifts a near-chance axis from 59% to 79%, exactly the signature of unlocking latent knowledge. The blank-image arbiter refutes it, revealing three grounding regimes that probing conflates: an axis can be grounded (
vision-dependent, correct), a prior (
vision-independent, with its decode and its apparent recovery a directional default rather than perception), or, surprisingly, inverted: decodable, causally controllable, but deployed with the wrong sign, so the model scores below chance and the error requires looking. The taxonomy holds across the studied VLMs: in fourteen models spanning six
language-model families and 2B–27B, horizontal is grounded, vertical is a prior, and depth is inverted, with the inversion emerging at scale within families. The decode-versus-deploy inversion replicates on seven of eight models across five families, and the minimal edit that re-deploys it varies with geometry: a training-free rotation matches a trained edit on the cleanest model, while distributed inversions need a trained low-rank edit, tracing a per-model correction-complexity spectrum. The cheap, self-calibrating arbiter cleanly separates grounded perception, inverted perception, and prior substitution; we argue it should be a default control for latent-knowledge and steering claims in VLMs.
Read the full abstract ▾
All authors Chih-Ting Liao, Fei Shen, Xin Cao, Tat-Seng Chua
Computer Vision
Rethinking the Role of Feature Engineering and Learning Strategies in Few-Shot Hidden Emotion Recognition
Xiaochuan Guo, Jihao Gu, Haixu Liu et al. (+6 more)
In this paper, we present the solution developed by our team, XInsight Lab, which achieved first place in Track 3 of the 4th EI-MIGA-IJCAI Challenge with a test accuracy of 0.76923. To address the challenge of weak and sparse implicit
emotion evidence in long
videos, this paper extends the winning solution from the previous competition and proposes a compact multi-modal temporal modeling framework. The framework integrates and evaluates the effects of multi-source features, including 2D/3D skeletons, facial expression Blendshapes, DINOv2/v3
vision foundation models, X-CLIP
video features, and Gemini semantic priors. Architecturally, we propose a cross-
attention mechanism that utilizes static pose features, denoted as Base, as the Query and dynamic micro-motion differential features, denoted as Offset, as the Key and Value. By capturing local relative velocities, this mechanism eliminates static biases related to individual body shape and identity. Concurrently, an adaptive pooling method based on Multiple Instance Learning is employed to extract instantaneous
emotions while suppressing background noise in long sequences. Finally, the paper reveals the representation collapse phenomenon of general
vision foundation models in micro-dynamic tasks, and analyzes the underlying mechanisms where networks fall into public-leaderboard-driven pseudo-generalization due to shortcut learning and rote memorization.
Read the full abstract ▾
All authors Xiaochuan Guo, Jihao Gu, Haixu Liu, Yuxin Liu, Qi Wang, Yufei Wang, Fei Wang, Kun Li, Dan Guo
Computer Vision
Yaojun Hu, Kun Yuan, Nassir Navab et al. (+3 more)
Surgical vision-
language foundation models typically adopt educational materials, such as
surgical lecture
videos, to transfer
surgical knowledge encoded in
language into visual representations. These knowledge are multi-dimensional and hierarchical: fine-grained action cues appear in narration, mid-level key steps are summarized in subsection headings, and global procedural context, such as patient history and
surgical strategy, is described in abstract texts. Prior work largely collapses these heterogeneous signals into a single flat
embedding space, implicitly assuming independence across hierarchy levels. However, this is suboptimal because it ignores cross-level semantic containment, e.g., actions belong to steps, steps compose phases, weakens long-range dependency modeling. To this end, we propose a
hyperbolic surgical video-
language pre-training framework that explicitly preserves the hierarchical structure by mitigating structural false negatives induced by procedural context and enforcing semantic consistency between parent phases and their constituent child steps. Extensive experiments on multiple
surgical benchmarks show consistent gains in zero- and few-shot phase recognition across procedures and institutions.
Read the full abstract ▾
All authors Yaojun Hu, Kun Yuan, Nassir Navab, Haochao Ying, Jian Wu, Nicolas Padoy
Computer Vision
UHD-MFF: Shattering Barriers in Multi-Focus Ultra-High-Definition Image Fusion via Learnable Lookup Tables
Yibing Zhang, Xunpeng Yi, Qinglong Yan et al. (+3 more)
With the advancement of imaging technology, ultra-high-definition images have become increasingly essential in modern visual applications. However, existing multi-focus image fusion remains largely confined to low-resolution images and faces three major barriers in UHD scenarios, namely data availability, model adaptability, and deployment feasibility, which severely hinder its practical application. To shatter these barriers, first, we propose the UHD-MFF
dataset, the first large-scale ultra-high-resolution multi-focus fusion
dataset. Second, we propose a scale-specialized lookup-table framework tailored for ultra-high-resolution images, termed as UMF-LUT. It consists of Coarse-Region Lookup Table (C-LUT) and Detail-Edge Lookup Table (D-LUT). Specifically, C-LUT performs joint queries of multiple gradient cues and semantic cues at low-resolution scales to enable region-level decision-making. Also, D-LUT operates at high-resolution scales, leveraging efficient
Laplacian cues to provide complementary edge-level decision information. Such a design makes the model particularly well-suited for ultra-high-resolution multi-focus image fusion. Finally, it offers strong deployability with minimal computational overhead, enabling real-time 4K multi-focus fusion and showing promising potential for smartphone. Extensive experiments demonstrate that it outperforms SOTA methods in both visual fidelity and quantitative metrics. It effectively advances the development of multi-focus image fusion toward ultra-high-resolution imaging scenarios. The code is available at https://github.com/zyb5/UHD-MFF.
Read the full abstract ▾
All authors Yibing Zhang, Xunpeng Yi, Qinglong Yan, Yeda Wang, Han Xu, Jiayi Ma
Computer Vision
ForgeDrive: Bidirectional Cross-Conditioning for Unified Visual-Action Generation in Autonomous Driving
Xuchang Zhong, He Zheng, Chenxu Zhao et al. (+9 more)
World-model-based autonomous driving endows the model with the ability to understand scene
evolution. Yet this promise is undermined by the prevailing imagine-then-act paradigm, which allows errors from the more challenging visual generation stage to cascade into action planning. We introduce ForgeDrive, a unified autoregressive
diffusion framework with visual-action cross-conditioning that closes this gap through act-then-imagine paradigm. ForgeDrive factorizes the future as a sequence of per-timestep frame-action pairs, intertwining each action with its corresponding visual observation. During training, we decouple the
diffusion timesteps of the two modalities and introduce a UniDiffuser-style noise scheduler to get the ability to infer either modality from its counterpart and deepen understanding of relationships between images and actions. At inference, we propose a novel act-then-imagine inference paradigm, and find that at each step, action generation is a capability internalized during training, requiring no clean future frame as a prerequisite at inference time; instead, the generated action can improve the accuracy of future frame generation, which in turn enhances the quality of the next action. Additionally, we augment each step with future ego-status prediction, further sharpening planning ability. Extensive experiments on NAVSIM demonstrate that ForgeDrive not only unifies driving simulation, planning, and visual odometry into a single model, but also outperforms existing strong planners without any post-training strategy.
Read the full abstract ▾
All authors Xuchang Zhong, He Zheng, Chenxu Zhao, Tianxiong Lv, Hangqi Fan, Bohua Wang, Yushan Liu, Li Gao, Zhihao Liao, Leigang Luo, Congyang Zhao, Yang Cai