Computer Vision
RIRF: Reasoning Image Restoration Framework
Wending Yan, Rongkai Zhang, Kaihua Tang et al. (+2 more)
Universal image restoration (UIR) aims to recover clean images from diverse and unknown degradations using a unified model. Existing UIR methods primarily focus on pixel reconstruction and often lack explicit diagnostic
reasoning over degradation composition, severity, and scene semantics prior to restoration. We propose Reason and Restore (R&R), a novel framework that integrates structured Chain-of-Thought (CoT)
reasoning into the image restoration pipeline. R&R introduces an explicit reasoner, implemented by fine-tuning Qwen3-VL, to diagnose degradation types, quantify degradation severity, infer key degradation-related factors, and describe relevant scene and object semantics. The resulting structured
reasoning provides interpretable and fine-grained diagnostic priors for the restorer. To further improve restoration quality, the quantified degradation severity produced by the reasoner is leveraged as reinforcement learning (RL) signals to guide and strengthen the restorer. Unlike existing
multimodal LLM-based agentic systems that decouple
reasoning from low-level
vision tasks, R&R tightly couples semantic diagnostic
reasoning with pixel-level restoration in a unified framework. Extensive experiments across diverse UIR
benchmarks demonstrate that R&R achieves state-of-the-art performance while offering unique interpretability into the restoration process.
Read the full abstract ▾
All authors Wending Yan, Rongkai Zhang, Kaihua Tang, Yu Cheng, Qiankun Liu
Computer Vision
VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning
Yucheng Shen, Jiulong Wu, Jizhou Huang et al. (+3 more)
Visual
Retrieval-Augmented Generation (VRAG) empowers
Vision-
Language Models to retrieve and reason over visually rich documents. To tackle complex queries requiring multi-step
reasoning, agentic VRAG systems interleave
reasoning with iterative
retrieval. However, existing agentic VRAG faces two critical bottlenecks. (1) Visual Evidence Sparsity: key evidence is scattered across pages yet processed in isolation, hindering cross-page
reasoning; moreover, fine-grained intra-image evidence often requires precise visual actions, whose misuse degrades
retrieval quality; (2) Search Drift in Long Horizons: the accumulation of visual
tokens across retrieved pages dilutes context and causes cognitive overload, leading
agents to deviate from their search objective. To address these challenges, we propose VISOR (Visual
Retrieval-Augmented Generation via Iterative Search and Over-horizon
Reasoning), a unified single-
agent framework. VISOR features a structured Evidence Space for progressive cross-page
reasoning, coupled with a Visual Action Evaluation and Correction mechanism to manage visual actions. Additionally, we introduce a Dynamic Trajectory with Sliding Window and Intent Injection to mitigate search drift. They anchor the evidence space while discarding earlier raw interactions, preventing context from being overwhelmed by visual
tokens. We train VISOR using a Group Relative Policy Optimization-based Reinforcement Learning (GRPO-based RL) pipeline with state masking and credit assignment tailored for dynamic context reconstruction. Extensive experiments on ViDoSeek, SlideVQA, and MMLongBench demonstrate that VISOR achieves state-of-the-art performance while maintaining reasonable and controllable computational costs for long-horizon visual
reasoning tasks. The source code is available at https://github.com/syc1336/VISOR.
Read the full abstract ▾
All authors Yucheng Shen, Jiulong Wu, Jizhou Huang, Dawei Yin, Lingyong Yan, Min Cao
Computer Vision
Online3R: Online Learning for Consistent Sequential Reconstruction Based on Geometry Foundation Model
Shunkai Zhou, Zike Yan, Fei Xue et al. (+3 more)
We present Online3R, a new sequential reconstruction framework that is capable of adapting to new scenes through online learning, effectively resolving inconsistency issues. Specifically, we introduce a set of learnable lightweight visual
prompts into a pretrained, frozen geometry foundation model to capture the knowledge of new environments while preserving the fundamental capability of the foundation model for geometry prediction. To solve the problems of missing groundtruth and the requirement of high efficiency when updating these visual
prompts at test time, we introduce a local-global self-supervised learning strategy by enforcing the local and global consistency constraints on predictions. The local consistency constraints are conducted on intermediate and previously local fused results, enabling the model to be trained with high-quality pseudo groundtruth signals; the global consistency constraints are operated on sparse keyframes spanning long distances rather than per frame, allowing the model to learn from a consistent prediction over a long trajectory in an efficient way. Our experiments demonstrate that Online3R outperforms previous state-of-the-art methods on various
benchmarks. Project page: https://shunkaizhou.github.io/online3r-1.0/
Read the full abstract ▾
All authors Shunkai Zhou, Zike Yan, Fei Xue, Dong Wu, Yuchen Deng, Hongbin Zha
Computer Vision
Incremental Semantics-Aided Meshing from LiDAR-Inertial Odometry and RGB Direct Label Transfer
Muhammad Affan, Ville Lehtola and George Vosselman
Geometric high-fidelity mesh reconstruction from
LiDAR-inertial scans remains challenging in large, complex indoor environments – such as cultural buildings – where point cloud sparsity,
geometric drift, and fixed fusion parameters produce holes, over-smoothing, and spurious surfaces at structural boundaries. We propose a modular, incremental RGB+
LiDAR pipeline that generates incremental semantics-aided high-quality meshes from indoor scans through scan frame-based direct label transfer. A
vision foundation model labels each incoming RGB frame; labels are incrementally projected and fused onto a
LiDAR-inertial odometry map; and an incremental semantics-aware Truncated Signed Distance Function (TSDF) fusion step produces the final mesh via marching cubes. This frame-level fusion strategy preserves the
geometric fidelity of
LiDAR while leveraging rich visual semantics to resolve
geometric ambiguities at reconstruction boundaries caused by
LiDAR point-cloud sparsity and
geometric drift. We demonstrate that semantic guidance improves
geometric reconstruction quality; quantitative evaluation is therefore performed using
geometric metrics on the Oxford Spires
dataset, while results from the NTU VIRAL
dataset are analyzed qualitatively. The proposed method outperforms state-of-the-art
geometric baselines ImMesh and Voxblox, demonstrating the benefit of semantics-aided fusion for
geometric mesh quality. The resulting semantically labelled meshes are of value when reconstructing Universal Scene Description (USD) assets, offering a path from indoor
LiDAR scanning to XR and digital modeling.
Read the full abstract ▾
Computer Vision
Realizing Immersive Volumetric Video: A Multimodal Framework for 6-DoF VR Engagement
Zhengxian Yang, Shengqi Wang, Shi Pan et al. (+8 more)
Fully immersive experiences that tightly integrate 6-DoF visual and auditory interaction are essential for virtual and augmented reality. While such experiences can be achieved through computer-generated content, constructing them directly from real-world captured
videos remains largely unexplored. We introduce Immersive Volumetric
Videos, a new volumetric media format designed to provide large 6-DoF interaction spaces, audiovisual feedback, and high-resolution, high-frame-rate dynamic content. To support IVV construction, we present ImViD, a multi-view, multi-modal
dataset built upon a space-oriented capture philosophy. Our custom capture rig enables synchronized multi-view
video-audio acquisition during motion, facilitating efficient capture of complex indoor and outdoor scenes with rich foreground–background interactions and challenging dynamics. The
dataset provides 5K-resolution
videos at 60 FPS with durations of 1-5 minutes, offering richer spatial, temporal, and
multimodal coverage than existing
benchmarks. Leveraging this
dataset, we develop a dynamic light field reconstruction framework built upon a Gaussian-based spatio-temporal representation, incorporating flow-guided sparse initialization, joint camera temporal calibration, and multi-term spatio-temporal supervision for robust and accurate modeling of complex motion. We further propose, to our knowledge, the first method for sound field reconstruction from such multi-view audiovisual data. Together, these components form a unified pipeline for immersive volumetric
video production. Extensive
benchmarks and immersive VR experiments demonstrate that our pipeline generates high-quality, temporally stable audiovisual volumetric content with large 6-DoF interaction spaces. This work provides both a foundational definition and a practical construction methodology for immersive volumetric
videos.
Read the full abstract ▾
All authors Zhengxian Yang, Shengqi Wang, Shi Pan, Hongshuai Li, Haoxiang Wang, Lin Li, Guanjun Li, Zhengqi Wen, Borong Lin, Jianhua Tao, Tao Yu
Computer Vision · Cross-listed from eess.IV
DSVTLA: Deep Swin Vision Transformer-Based Transfer Learning Architecture for Multi-Type Cancer Histopathological Cancer Image Classification
Muazzem Hussain Khan, Tasdid Hasnain, Md. Jamil khan et al. (+4 more)
In this study, we proposed a deep Swin-
Vision Transformer-based transfer learning architecture for robust multi-cancer histopathological image classification. The proposed framework integrates a hierarchical Swin
Transformer with ResNet50-based convolution features extraction, enabling the model to capture both long-range contextual dependencies and fine-grained local morphological patterns within histopathological images. To validate the efficiency of the proposed architecture, an extensive experiment was executed on a comprehensive multi-cancer
dataset including Breast Cancer, Oral Cancer, Lung and Colon Cancer, Kidney Cancer, and Acute Lymphocytic Leukemia (ALL), including both original and segmented images were analyzed to assess model robustness across heterogeneous clinical imaging conditions. Our approach is benchmarked alongside several state-of-the-art CNN and transfer models, including DenseNet121, DenseNet201, InceptionV3, ResNet50, EfficientNetB3, multiple ViT variants, and Swin
Transformer models. However, all models were trained and validated using a unified pipeline, incorporating balanced data preprocessing, transfer learning, and fine-tuning strategies. The experimental results demonstrated that our proposed architecture consistently gained superior performance, reaching 100% test accuracy for lung-colon cancer, segmented leukemia
datasets, and up to 99.23% accuracy for breast cancer classification. The model also achieved near-perfect precision, f1 score, and recall, indicating highly stable scores across divers cancer types. Overall, the proposed model establishes a highly accurate, interpretable, and also robust multi-cancer classification system, demonstrating strong
benchmark for future research and provides a unified comparative assessment useful for designing reliable AI-assisted histopathological diagnosis and clinical decision-making.
Read the full abstract ▾
All authors Muazzem Hussain Khan, Tasdid Hasnain, Md. Jamil khan, Ruhul Amin, Md. Shamim Reza, Md. Al Mehedi Hasan, Md Ashad Alam
Computer Vision
AsymLoc: Towards Asymmetric Feature Matching for Efficient Visual Localization
Mohammad Omama, Gabriele Berton, Eric Foxlin et al. (+1 more)
Precise and real-time visual localization is critical for applications like AR/VR and robotics, especially on resource-constrained edge devices such as smart glasses, where battery life and heat dissipation can be a primary concerns. While many efficient models exist, further reducing compute without sacrificing accuracy is essential for practical deployment. To address this, we propose asymmetric visual localization: a large Teacher model processes pre-mapped database images offline, while a lightweight Student model processes the query image online. This creates a challenge in matching features from two different models without resorting to heavy, learned matchers. We introduce AsymLoc, a novel
distillation framework that aligns a Student to its Teacher through a combination of a geometry-driven matching objective and a joint detector-descriptor
distillation objective, enabling fast, parameter-less nearest-neighbor matching. Extensive experiments on HPatches, ScanNet, IMC2022, and Aachen show that AsymLoc achieves up to 95% of the teacher's localization accuracy using an order of magnitude smaller models, significantly outperforming existing baselines and establishing a new state-of-the-art efficiency-accuracy trade-off.
Read the full abstract ▾
All authors Mohammad Omama, Gabriele Berton, Eric Foxlin, Yelin Kim
Computer Vision
SCoRe: Clean Image Generation from Diffusion Models Trained on Noisy Images
Yuta Matsuzaki, Seiichi Uchida and Shumpei Takezaki
Diffusion models trained on noisy
datasets often reproduce high-frequency training artifacts, significantly degrading generation quality. To address this, we propose SCoRe (Spectral Cutoff Regeneration), a training-free, generation-time spectral regeneration method for clean image generation from
diffusion models trained on noisy images. Leveraging the spectral bias of
diffusion models, which infer high-frequency details from low-frequency cues, SCoRe suppresses corrupted high-frequency components of a generated image via a frequency cutoff and regenerates them via SDEdit. Crucially, we derive a theoretical mapping between the cutoff frequency and the SDEdit initialization timestep based on Radially Averaged Power Spectral Density (RAPSD), which prevents excessive noise injection during regeneration. Experiments on synthetic (CIFAR-10) and real-world (SIDD) noisy
datasets demonstrate that SCoRe substantially outperforms post-processing and noise-robust baselines, restoring samples closer to clean image distributions without any retraining or fine-tuning.
Read the full abstract ▾
Computer Vision
Rays as Pixels: Learning A Joint Distribution of Videos and Camera Trajectories
Wonbong Jang, Shikun Liu, Soubhik Sanyal et al. (+6 more)
Recovering camera parameters from images and rendering scenes from novel viewpoints have been treated as separate tasks in computer
vision and graphics. This separation breaks down when image coverage is sparse or poses are ambiguous, since each task depends on what the other produces. We propose Rays as Pixels, a
Video Diffusion Model (VDM) that learns a joint distribution over
videos and camera trajectories. To our knowledge, this is the first model to predict camera poses and do camera-controlled
video generation within a single framework. We represent each camera as dense ray pixels (raxels), a pixel-aligned encoding that lives in the same latent space as
video frames, and denoise the two jointly through a Decoupled Self-Cross
Attention mechanism. A single trained model handles three tasks: predicting camera trajectories from
video, generating
video from input images along a pre-defined trajectory, and jointly synthesizing
video and trajectory from input images. We evaluate on pose estimation and camera-controlled
video generation, and introduce a closed-loop self-consistency test showing that the model's predicted poses and its renderings conditioned on those poses agree. Ablations against Plücker
embeddings confirm that representing cameras in a shared latent space with
video is subtantially more effective.
Read the full abstract ▾
All authors Wonbong Jang, Shikun Liu, Soubhik Sanyal, Juan Camilo Perez, Kam Woh Ng, Sanskar Agrawal, Juan-Manuel Perez-Rua, Yiannis Douratsos, Tao Xiang
Computer Vision
Sambit Ghosh, R. Venkatesh Babu and Chirag Agarwal
Vision Language Models (VLMs) have achieved remarkable success by integrating visual encoders with large
language models (LLMs). While VLMs process dense image
tokens across deep
transformer stacks (incurring substantial computational overhead), it remains fundamentally unclear whether sustained image-
token processing is necessary for their performance or visual representations meaningfully evolve from early to later layers. In this work, we systematically investigate the functional role of image
tokens in VLMs and show that visual representations rapidly converge to a bounded-complexity regime, \ie their
entropy stabilizes, intrinsic dimensionality compresses, and trajectory
curvature approaches a near-constant profile. In contrast, textual representations continue to undergo substantial restructuring across depth. Once stabilized, visual representations become largely interchangeable between layers, indicating limited additional transformation in deeper stages. Further, depth-wise visual truncation reveals that the necessity of visual processing is task-dependent, where single-
token predictions remain comparatively robust to truncated visual depth, but multi-
token generation require sustained access to visual representations. Under deterministic decoding, reducing visual depth perturbs intermediate
reasoning trajectories more strongly than final outputs, suggesting that image
tokens influence the structure of
reasoning more than the ultimate conclusions. Collectively, these findings question the assumption that deeper visual processing is uniformly essential in VLMs, challenging the current paradigm of
multimodal LLM architectures.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Multi-task Just Recognizable Difference for Video Coding for Machines: Database, Model, and Coding Application
Junqi Liu, Yun Zhang, Xiaoxia Huang et al. (+2 more)
Just Recognizable Difference (JRD) boosts coding efficiency for machine
vision through visibility threshold modeling, but is currently limited to a single-task scenario. To address this issue, we propose a Multi-Task JRD (MT-JRD)
dataset and an Attribute-assisted MT-JRD (AMT-JRD) model for
Video Coding for Machines (VCM), enhancing both prediction accuracy and coding efficiency. First, we construct a
dataset comprising 27,264 JRD annotations from machines, supporting three representative tasks including object detection, instance
segmentation, and keypoint detection. Secondly, we propose the AMT-JRD prediction model, which integrates Generalized Feature Extraction Module (GFEM) and Specialized Feature Extraction Module (SFEM) to facilitate joint learning across multiple tasks. Thirdly, we innovatively incorporate object attribute information into object-wise JRD prediction through the Attribute Feature Fusion Module (AFFM), which introduces prior knowledge about object size and location. This design effectively compensates for the limitations of relying solely on image features and enhances the model's capacity to represent the perceptual mechanisms of machine
vision. Finally, we apply the AMT-JRD model to VCM, where the accurately predicted JRDs are applied to reduce the coding bit rate while preserving accuracy across multiple machine
vision tasks. Extensive experimental results demonstrate that AMT-JRD achieves precise and robust multi-task prediction with a mean absolute error of 3.781 and error variance of 5.332 across three tasks, outperforming the state-of-the-art single-task prediction model by 6.7% and 6.3%, respectively. Coding experiments further reveal that compared to the baseline VVC and JPEG, the AMT-JRD-based VCM improves an average of 3.861% and 7.886% Bjontegaard Delta-mean Average Precision (BD-mAP), respectively.
Read the full abstract ▾
All authors Junqi Liu, Yun Zhang, Xiaoxia Huang, Long Xu, Weisi Lin
Computer Vision
PhysInOne: Visual Physics Learning and Reasoning in One Suite
Siyuan Zhou, Hejun Wang, Hu Cheng et al. (+36 more)
We present PhysInOne, a large-scale synthetic
dataset addressing the critical scarcity of physically-grounded training data for AI systems. Unlike existing
datasets limited to merely hundreds or thousands of examples, PhysInOne provides 2 million
videos across 153,810 dynamic 3D scenes, covering 71 basic physical phenomena in mechanics, optics, fluid dynamics, and magnetism. Distinct from previous works, our scenes feature multiobject interactions against complex backgrounds, with comprehensive ground-truth annotations including 3D geometry, semantics, dynamic motion, physical properties, and text descriptions. We demonstrate PhysInOne's efficacy across four emerging applications: physics-aware
video generation, long-/short-term future frame prediction, physical property estimation, and motion transfer. Experiments show that fine-tuning foundation models on PhysInOne significantly enhances physical plausibility, while also exposing critical gaps in modeling complex physical dynamics and estimating intrinsic properties. As the largest
dataset of its kind, orders of magnitude beyond prior works, PhysInOne establishes a new
benchmark for advancing physics-grounded world models in generation, simulation, and
embodied AI.
Read the full abstract ▾
All authors Siyuan Zhou, Hejun Wang, Hu Cheng, Jinxi Li, Dongsheng Wang, Junwei Jiang, Yixiao Jin, Jiayue Huang, Shiwei Mao, Shangjia Liu, Yafei Yang, Hongkang Song, Shenxing Wei, Zihui Zhang, Peng Huang, Shijie Liu, Zhengli Hao, Hao Li, Yitian Li, Wenqi Zhou, Zhihan Zhao, Zongqi He, Hongtao Wen, Shouwang Huang, Peng Yun, Bowen Cheng, Pok Kazaf Fu, Wai Kit Lai, Jiahao Chen, Kaiyuan Wang, Zhixuan Sun, Ziqi Li, Haochen Hu, Di Zhang, Chun Ho Yuen, Bing Wang, Zhihua Wang, Chuhang Zou, Bo Yang
Computer Vision
SynFlow: Scaling Up LiDAR Scene Flow Estimation with Synthetic Data
Qingwen Zhang, Xiaomeng Zhu, Chenhan Jiang et al. (+1 more)
Reliable 3D dynamic perception requires models that can anticipate motion beyond predefined categories, yet progress is hindered by the scarcity of dense, high-quality motion annotations. While self-supervision on unlabeled real data offers a path forward, empirical evidence suggests that scaling unlabeled data fails to close the performance gap due to noisy proxy signals. In this paper, we propose learning robust real-world motion priors entirely from scalable simulation. We introduce SynFlow, a data generation pipeline for large-scale synthetic
LiDAR scene flow. Unlike prior works that prioritize sensor-specific realism, SynFlow employs a motion-oriented strategy to synthesize diverse kinematic patterns across 4,000 sequences ($\sim$940k frames), termed SynFlow-4k. This represents a $34\times$ scale-up in annotated volume over existing real-world
benchmarks. Our experiments demonstrate that SynFlow-4k provides a highly domain-invariant motion prior. In a zero-shot regime, models trained only on our synthetic data generalize across multiple real-world
benchmarks, comparable to in-domain supervised baselines on nuScenes and outperforming state-of-the-art methods on TruckScenes by 31.8%. Furthermore, SynFlow-4k serves as a label-efficient foundation: fine-tuning with only 5% of real-world labels surpasses models trained from scratch on the full available budget. We open-source the pipeline and
dataset to facilitate research in generalizable 3D motion estimation. More detail can be found at https://kin-zhang.github.io/SynFlow.
Read the full abstract ▾
All authors Qingwen Zhang, Xiaomeng Zhu, Chenhan Jiang, Patric Jensfelt
Computer Vision
EGLOCE: Training-Free Energy-Guided Latent Optimization for Concept Erasure
Junyeong Ahn, Seojin Yoon and Sungyong Baik
As text-to-image
diffusion models grow increasingly prevalent, the ability to remove specific concepts-mostly explicit content and many copyrighted characters or styles-has become essential for safety and compliance. Existing unlearning approaches often require costly re-training, modify parameters at the cost of degradation of unrelated concept fidelity, or depend on indirect inference-time adjustment that compromise the effectiveness of concept erasure. Inspired by the success of energy-guided sampling for preservation of the condition of
diffusion models, we introduce Energy-Guided Latent Optimization for Concept Erasure (EGLOCE), a training-free approach that removes unwanted concepts by re-directing noisy latent during inference. Our method employs a dual-objective framework: a repulsion energy that steers generation away from target concepts via gradient descent in latent space, and a retention energy that preserves semantic
alignment to the original
prompt. Combined with previous approaches that either require erroneous modified model weights or provide weak inference-time guidance, EGLOCE operates entirely at inference and enhances erasure performance, enabling plug-and-play integration. Extensive experiments demonstrate that EGLOCE improves concept removal while maintaining image quality and
prompt alignment across baselines, even with
adversarial attacks. To the best of our knowledge, our work is the first to establish a new paradigm for safe and controllable image generation through dual energy-based guidance during sampling.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Efficient Unlearning through Maximizing Relearning Convergence Delay
Khoa Tran and Simon S. Woo
Machine unlearning poses challenges in removing mislabeled, contaminated, or problematic data from a pretrained model. Current unlearning approaches and evaluation metrics are solely focused on model predictions, which limits insight into the model's true underlying data characteristics. To address this issue, we introduce a new metric called relearning convergence delay, which captures both changes in weight space and prediction space, providing a more comprehensive assessment of the model's understanding of the forgotten
dataset. This metric can be used to assess the risk of forgotten data being recovered from the unlearned model. Based on this, we propose the Influence Eliminating Unlearning framework, which removes the influence of the forgetting set by degrading its performance and incorporates weight decay and injecting noise into the model's weights, while maintaining accuracy on the retaining set. Extensive experiments show that our method outperforms existing metrics and our proposed relearning convergence delay metric, approaching ideal unlearning performance. We provide theoretical guarantees, including exponential convergence and upper bounds, as well as empirical evidence of strong retention and resistance to relearning in both classification and generative unlearning tasks.
Read the full abstract ▾
Computer Vision
Region-Constrained Group Relative Policy Optimization for Flow-Based Image Editing
Zhuohan Ouyang, Zhe Qian, Wenhuo Cui et al. (+1 more)
Instruction-guided image editing requires balancing target modification with non-target preservation. Recently, flow-based models have emerged as a strong and increasingly adopted backbone for instruction-guided image editing, thanks to their high fidelity and efficient deterministic ODE sampling. Building on this foundation, GRPO-based reward-driven post-training has been explored to directly optimize editing-specific rewards, improving instruction following and editing consistency. However, existing methods often suffer from noisy credit assignment: global exploration also perturbs non-target regions, inflating within-group reward variance and yielding noisy GRPO advantages. To address this, we propose RC-GRPO-Editing, a region-constrained GRPO post-training framework for flow-based image editing under deterministic ODE sampling. It suppresses background-induced nuisance variance to enable cleaner localized credit assignment, improving editing region instruction adherence while preserving non-target content. Concretely, we localize exploration via region-decoupled initial noise perturbations to reduce background-induced reward variance and stabilize GRPO advantages, and introduce an
attention concentration reward that aligns cross-
attention with the intended editing region throughout the rollout, reducing unintended changes in non-target regions. Experiments on CompBench show consistent improvements in editing region instruction adherence and non-target preservation.
Read the full abstract ▾
All authors Zhuohan Ouyang, Zhe Qian, Wenhuo Cui, Chaoqun Wang
Computer Vision · Cross-listed from q-bio.QM
Cluster-First Labelling: An Automated Pipeline for Segmentation and Morphological Clustering in Histology Whole Slide Images
Muhammad Haseeb Ahmad, Sharmila Rajendran, Damion Young et al. (+1 more)
Labelling tissue components in histology whole slide images (WSIs) is prohibitively labour-intensive: a single slide may contain tens of thousands of structures–
cells,
nuclei, and other morphologically distinct objects–each requiring manual boundary delineation and classification. We present a cloudnative, end-to-end pipeline that automates this process through a cluster-first paradigm. Our system tiles WSIs, filters out tiles deemed unlikely to contain valuable information, segments tissue components with Cellpose-SAM (including
cells,
nuclei, and other morphologically similar structures), extracts neural
embeddings via a pretrained ResNet-50, reduces dimensionality with UMAP, and groups morphologically similar objects using DBSCAN clustering. Under this paradigm, a human annotator labels representative clusters rather than individual objects, reducing annotation effort by orders of magnitude. We evaluate the pipeline on 3,696 tissue components across 13 diverse tissue types from three species (human, rat, rabbit), measuring how well unsupervised clusters align with independent human labels via per-tile Hungarian-
algorithm matching. Our system achieves a weighted cluster-label
alignment accuracy of 96.8%, with 7 of 13 tissue types reaching perfect agreement. The pipeline, a companion labelling web application, and all evaluation code are released as open-source software.
Read the full abstract ▾
All authors Muhammad Haseeb Ahmad, Sharmila Rajendran, Damion Young, Jon Mason
Computer Vision · Cross-listed from cs.MM
Through Their Eyes: Fixation-aligned Tuning for Personalized User Emulation
Lingfeng Huang, Huizhong Guo, Tianjun Wei et al. (+2 more)
Large
language model (LLM)
agents are increasingly deployed as scalable user simulators for recommender system evaluation. Yet existing simulators perceive recommendations through text or structured metadata rather than the visual interfaces real users browse-a critical gap, since
attention over recommendation layouts is both visually driven and highly personalized. We investigate whether aligning a
vision-
language model's (VLM's) visual
attention with user-specific gaze patterns can improve simulation fidelity. Analysis of a real-world eye-tracking
dataset collected in a carousel-based recommendation setting reveals that users exhibit stable individual gaze patterns strongly predictive of click behavior. Building on this finding, we propose Fixation-Aligned Tuning for user Emulation (FixATE). Our approach first probes the VLM's internal visual
attention via interpretability operators to obtain a slot-level relevance distribution comparable with human fixation, and then learns personalized soft
prompts to steer the model's
attention toward each user's characteristic fixation pattern. Experiments across three interpretability-based probing operators and two architecturally distinct VLM backbones demonstrate consistent improvements in both
attention alignment and click prediction accuracy. These results suggest that making the model "see like the user" is a viable path toward simulators that more faithfully reproduce how users perceive and act in recommendation interfaces.
Read the full abstract ▾
All authors Lingfeng Huang, Huizhong Guo, Tianjun Wei, Yingpeng Du, Zhu Sun
Computer Vision
EpiAgent: An Agent-Centric System for Ancient Inscription Restoration
Shipeng Zhu, Ang Chen, Na Nie et al. (+3 more)
Ancient inscriptions, as repositories of cultural
memory, have suffered from centuries of environmental and human-induced degradation. Restoring their intertwined visual and textual integrity poses one of the most demanding challenges in digital heritage preservation. However, existing AI-based approaches often rely on rigid pipelines, struggling to generalize across such complex and heterogeneous real-world degradations. Inspired by the skill-coordinated workflow of human epigraphers, we propose EpiAgent, an
agent-centric system that formulates inscription restoration as a hierarchical planning problem. Following an Observe-Conceive-Execute-Reevaluate paradigm, an LLM-based central planner orchestrates collaboration among
multimodal analysis, historical experience, specialized restoration tools, and iterative self-refinement. This
agent-centric coordination enables a flexible and adaptive restoration process beyond conventional single-pass methods. Across real-world degraded inscriptions, EpiAgent achieves superior restoration quality and stronger generalization compared to existing methods. Our work marks an important step toward expert-level
agent-driven restoration of cultural heritage. The code is available at https://github.com/blackprotoss/EpiAgent.
Read the full abstract ▾
All authors Shipeng Zhu, Ang Chen, Na Nie, Pengfei Fang, Min-Ling Zhang, Hui Xue
Computer Vision
Robust 4D Visual Geometry Transformer with Uncertainty-Aware Priors
Ying Zang, Yidong Han, Chaotao Ding et al. (+8 more)
Reconstructing dynamic 4D scenes is an important yet challenging task. While 3D foundation models like VGGT excel in static settings, they often struggle with dynamic sequences where motion causes significant
geometric ambiguity. To address this, we present a framework designed to disentangle dynamic and static components by modeling uncertainty across different stages of the reconstruction process. Our approach introduces three synergistic mechanisms: (1)
Entropy-Guided Subspace Projection, which leverages information-theoretic weighting to adaptively aggregate multi-head
attention distributions, effectively isolating dynamic motion cues from semantic noise; (2) Local-Consistency Driven Geometry Purification, which enforces spatial continuity via radius-based neighborhood constraints to eliminate structural outliers; and (3) Uncertainty-Aware Cross-View Consistency, which formulates multi-view projection refinement as a heteroscedastic maximum likelihood estimation problem, utilizing depth confidence as a probabilistic weight. Experiments on dynamic
benchmarks show that our approach outperforms current state-of-the-art methods, reducing Mean Accuracy error by 13.43% and improving
segmentation F-measure by 10.49%. Our framework maintains the efficiency of feed-forward inference and requires no task-specific fine-tuning or per-scene optimization.
Read the full abstract ▾
All authors Ying Zang, Yidong Han, Chaotao Ding, Yuanqi Hu, Deyi Ji, Qi Zhu, Xuanfu Li, Jin Ma, Lingyun Sun, Tianrun Chen, Lanyun Zhu
Computer Vision
Arbitration Failure, Not Perceptual Blindness: How Vision-Language Models Resolve Visual-Linguistic Conflicts
Farhad Nooralahzadeh, Omid Rohanian, Yi Zhang et al. (+2 more)
When a
Vision-
Language Model (VLM) sees a blue banana and answers "yellow", is the problem of perception or arbitration? We explore the question in ten VLMs with various sizes and reveal an Encoding-Grounding Dissociation: models that fail to report what they see (and thus provide a wrong answer) still encode the visual evidence as strongly as models that provide the correct answer. Using
Multimodal Arbitration Crossover (MAC) analysis with layer-by-layer Logit Lens probing, we track the competition between visual and prior signals across every layer of each model. We show that visual attributes can be linearly decodable from early layers (AUC > 0.86). The accuracy remains nearly identical for both successful and failed samples. However, the gap in the final-layer logit - not the strength of encoding - better predicts grounding outcomes with a correlation of $\rho=$ 0.847. After having studied when VLMs base their answers on image clues rather than prior knowledge, we want to understand the causal relationships. We establish causality through full-sequence activation patching. The standard last-
token interventions in LLM interpretability do not affect VLMs. In contrast, replacing the full
token sequence at layers identified by MAC alters 60 to 84% of outputs. Partial-
token decomposition shows that image
tokens carry almost all of the causal impact, while text
tokens have none. Scaling addresses the remaining architectural differences to achieve perfect retention. Moving from diagnosis to intervention, we show that training-free activation steering - both linear and sparse autoencoder-guided - in early layers can improve visual grounding by up to +3.8% with degrading performance in some setups. Overall, these findings lead to a clear conclusion: VLMs already see well, but the challenge is acting on what they see. Targeted interventions can help to bridge this gap.
Read the full abstract ▾
All authors Farhad Nooralahzadeh, Omid Rohanian, Yi Zhang, Jonathan Fürst, Kurt Stockinger
Computer Vision
LuMon: A Comprehensive Benchmark and Development Suite with Novel Datasets for Lunar Monocular Depth Estimation
Aytaç Sekmen, Fatih Emre Gunes, Furkan Horoz et al. (+9 more)
Monocular Depth Estimation (MDE) is crucial for autonomous lunar rover
navigation using electro-optical cameras. However, deploying terrestrial MDE networks to the Moon brings a severe domain gap due to harsh shadows, textureless regolith, and zero atmospheric scattering. Existing evaluations rely on analogs that fail to replicate these conditions and lack actual metric ground truth. To address this, we present LuMon, a comprehensive benchmarking framework to evaluate MDE methods for lunar exploration. We introduce novel
datasets featuring high-quality stereo ground truth depth from the real Chang'e-3 mission and the CHERI dark analog
dataset. Utilizing this framework, we conduct a systematic zero-shot evaluation of state-of-the-art architectures across synthetic, analog, and real
datasets. We rigorously assess performance against mission critical challenges like craters, rocks, extreme shading, and varying depth ranges. Furthermore, we establish a sim-to-real domain adaptation baseline by fine tuning a foundation model on synthetic data. While this adaptation yields drastic in-domain performance gains, it exhibits minimal generalization to authentic lunar imagery, highlighting a persistent cross-domain transfer gap. Our extensive analysis reveals the inherent limitations of current networks and sets a standard foundation to guide future advancements in extraterrestrial perception and domain adaptation.
Read the full abstract ▾
All authors Aytaç Sekmen, Fatih Emre Gunes, Furkan Horoz, Hüseyin Umut Iş{\i}k, Mehmet Alp Ozaydin, Onur Altay Topaloglu, Şahin Umutcan Üstündaş, Yurdasen Alp Yeni, Halil Ersin Soken, Erol Sahin, Ramazan Gokberk Cinbis, Sinan Kalkan
Computer Vision
Visually-Guided Policy Optimization for Multimodal Reasoning
Zengbin Wang, Feng Xiong, Liang Lin et al. (+5 more)
Reinforcement learning with verifiable rewards (RLVR) has significantly advanced the
reasoning ability of
vision-
language models (VLMs). However, the inherent text-dominated nature of VLMs often leads to insufficient visual faithfulness, characterized by sparse
attention activation to visual
tokens. More importantly, our empirical analysis reveals that temporal visual forgetting along
reasoning steps exacerbates this deficiency. To bridge this gap, we propose Visually-Guided Policy Optimization (VGPO), a novel framework to reinforce visual focus during policy optimization. Specifically, VGPO initially introduces a Visual
Attention Compensation mechanism that leverages visual similarity to localize and amplify visual cues, while progressively elevating visual expectations in later steps to counteract visual forgetting. Building on this mechanism, we implement a dual-grained advantage re-weighting strategy: the intra-trajectory level highlights
tokens exhibiting relatively high visual activation, while the inter-trajectory level prioritizes trajectories demonstrating superior visual accumulation. Extensive experiments demonstrate that VGPO achieves better visual activation and superior performance in mathematical
multimodal reasoning and visual-dependent tasks. The code has been released at https://github.com/wzb-bupt/VGPO.
Read the full abstract ▾
All authors Zengbin Wang, Feng Xiong, Liang Lin, Xuecai Hu, Yong Wang, Yanlin Wang, Man Zhang, Xiangxiang Chu
Computer Vision · Cross-listed from cs.RO
VAG: Dual-Stream Video-Action Generation for Embodied Data Synthesis
Xiaolei Lang, Yang Wang, Yukun Zhou et al. (+10 more)
Recent advances in
robot foundation models trained on large-scale human teleoperation data have enabled
robots to perform increasingly complex real-world tasks. However, scaling these systems remains difficult because collecting task-specific demonstrations is expensive and labor-intensive. Synthetic data, especially generated
videos, offer a promising direction, but existing World Models (WMs) are not directly suitable for policy learning since they do not provide paired action trajectories. World-Action (WA) models partially address this by predicting actions with visual outputs, yet often lack strong
video-action
alignment, while two-stage pipelines that generate
video first and then infer actions introduce inefficiency and error accumulation. To address these limitations, we propose VAG, a unified flow-matching-based dual-stream framework that jointly generates
video and action under visual and
language conditioning. By synchronizing denoising in both branches and using an adaptive 3D pooling mechanism to transfer compact global
video context to the action branch, VAG improves cross-modal consistency during generation. Across both simulated and real-world settings, VAG produces aligned
video-action pairs with competitive prediction quality, supports executable trajectory replay, and provides useful synthetic pretraining data that improves downstream policy generalization, indicating its potential as a practical world-action model for
embodied data synthesis.
Read the full abstract ▾
All authors Xiaolei Lang, Yang Wang, Yukun Zhou, Chaojun Ni, Kerui Li, Jiagang Zhu, Tianze Liu, Jiajun Lv, Xingxing Zuo, Yun Ye, Guan Huang, Xiaofeng Wang, Zheng Zhu
Computer Vision
From Frames to Events: Rethinking Evaluation in Human-Centric Video Anomaly Detection
Narges Rashvand, Shanle Yao, Armin Danesh Pazho et al. (+2 more)
Pose-based
Video Anomaly Detection (VAD) has gained significant
attention for its
privacy-preserving nature and robustness to environmental variations. However, traditional frame-level evaluations treat
video as a collection of isolated frames, fundamentally misaligned with how anomalies manifest and are acted upon in the real world. In operational surveillance systems, what matters is not the flagging of individual frames, but the reliable detection, localization, and reporting of a coherent anomalous event, a contiguous temporal episode with an identifiable onset and duration. Frame-level metrics are blind to this distinction, and as a result, they systematically overestimate model performance for any deployment that requires actionable, event-level alerts. In this work, we propose a shift toward an event-centric perspective in VAD. We first audit widely used VAD
benchmarks, including SHT[19], CHAD[6], NWPUC[4], and HuVAD[25], to characterize their event structure. We then introduce two strategies for temporal event localization: a score-refinement pipeline with hierarchical Gaussian smoothing and adaptive binarization, and an end-to-end Dual-Branch Model that directly generates event-level detections. Finally, we establish the first event-based evaluation standard for VAD by adapting Temporal Action Localization metrics, including tIoU-based event matching and multi-threshold F1 evaluation. Our results quantify a substantial performance gap: while all SoTA models achieve frame-level AUC-ROC exceeding 52% on the NWPUC[4], their event-level localization precision falls below 10% even at a minimal tIoU=0.2, with an average event-level F1 of only 0.11 across all thresholds. The code base for this work is available at https://github.com/TeCSAR-UNCC/EventCentric-VAD.
Read the full abstract ▾
All authors Narges Rashvand, Shanle Yao, Armin Danesh Pazho, Babak Rahimi Ardabili, Hamed Tabkhi
Computer Vision · Cross-listed from cs.RO
Multimodal Anomaly Detection for Human-Robot Interaction
Guilherme Ribeiro, Iordanis Antypas, Leonardo Bizzaro et al. (+2 more)
Ensuring safety and reliability in human-
robot interaction (HRI) requires the timely detection of unexpected events that could lead to system failures or unsafe behaviours. Anomaly detection thus plays a critical role in enabling
robots to recognize and respond to deviations from normal operation during collaborative tasks. While reconstruction models have been actively explored in HRI, approaches that operate directly on feature vectors remain largely unexplored. In this work, we propose MADRI, a framework that first transforms
video streams into semantically meaningful feature vectors before performing reconstruction-based anomaly detection. Additionally, we augment these visual feature vectors with the
robot's internal sensors' readings and a Scene Graph, enabling the model to capture both external anomalies in the visual environment and internal failures within the
robot itself. To evaluate our approach, we collected a custom
dataset consisting of a simple pick-and-place robotic task under normal and anomalous conditions. Experimental results demonstrate that reconstruction on
vision-based feature vectors alone is effective for detecting anomalies, while incorporating other modalities further improves detection performance, highlighting the benefits of
multimodal feature reconstruction for robust anomaly detection in human-
robot collaboration.
Read the full abstract ▾
All authors Guilherme Ribeiro, Iordanis Antypas, Leonardo Bizzaro, João Bimbo, Nuno Cruz Garcia
Computer Vision
Structure-Aware Fine-Grained Gaussian Splatting for Expressive Avatar Reconstruction
Yuze Su, Hongsong Wang, Jie Gui et al. (+1 more)
Reconstructing photorealistic and
topology-aware human avatars from
monocular videos remains a significant challenge in the fields of computer
vision and graphics. While existing 3D human avatar modeling approaches can effectively capture body motion, they often fail to accurately model fine details such as hand movements and facial expressions. To address this, we propose Structure-aware Fine-grained Gaussian Splatting (SFGS), a novel method for reconstructing expressive and coherent full-body 3D human avatars from a
monocular video sequence. The SFGS use both spatial-only triplane and time-aware hexplane to capture dynamic features across consecutive frames. A structure-aware gaussian module is designed to capture pose-dependent details in a spatially coherent manner and improve pose and texture expression. To better model hand deformations, we also propose a residual refinement module based on fine-grained hand reconstruction. Our method requires only a single-stage training and outperforms state-of-the-art baselines in both quantitative and qualitative evaluations, generating high-fidelity avatars with natural motion and fine details. The code is on Github: https://github.com/Su245811YZ/SFGS
Read the full abstract ▾
All authors Yuze Su, Hongsong Wang, Jie Gui, Liang Wang
Computer Vision · Cross-listed from eess.IV
UHD Low-Light Image Enhancement via Real-Time Enhancement Methods with Clifford Information Fusion
Xiaohan Wang, Chen Wu, Dawei Zhao et al. (+8 more)
Considering efficiency, ultra-high-definition (UHD) low-light image restoration is extremely challenging. Existing methods based on
Transformer architectures or high-dimensional complex convolutional neural networks often suffer from the "
memory wall" bottleneck, failing to achieve millisecond-level inference on edge devices. To address this issue, we propose a novel real-time UHD low-light enhancement network based on
geometric feature fusion using Clifford
algebra in 2D Euclidean space. First, we construct a four-layer feature pyramid with gradually increasing resolution, which decomposes input images into low-frequency and high-frequency structural components via a Gaussian blur kernel, and adopts a lightweight U-Net based on depthwise separable convolution for dual-branch feature extraction. Second, to resolve structural information loss and artifacts from traditional high-low frequency feature fusion, we introduce spatially aware Clifford
algebra, which maps feature tensors to a multivector space (scalars, vectors, bivectors) and uses Clifford similarity to aggregate features while suppressing noise and preserving textures. In the reconstruction stage, the network outputs adaptive Gamma and Gain maps, which perform physically constrained non-linear brightness adjustment via Retinex theory. Integrated with FP16 mixed-precision computation and dynamic operator fusion, our method achieves millisecond-level inference for 4K/8K images on a single consumer-grade device, while outperforming state-of-the-art (SOTA) models on several restoration metrics.
Read the full abstract ▾
All authors Xiaohan Wang, Chen Wu, Dawei Zhao, Guangwei Gao, Dianjie Lu, Guijuan Zhang, Linwei Fan, Xu Lu, Shuai Wu, Hang Wei, Zhuoran Zheng
Computer Vision · Cross-listed from eess.IV
Compositional-Degradation UAV Image Restoration: Conditional Decoupled MoE Network and A Benchmark
Jinquan Yan, Zhicheng Zhao, Zhengzheng Tu et al. (+3 more)
UAV images are critical for applications such as large-area mapping, infrastructure inspection, and emergency response. However, in real-world flight environments, a single image is often affected by multiple degradation factors, including rain, haze, and noise, undermining downstream task performance. Current unified restoration approaches typically rely on implicit degradation representations that entangle multiple factors into a single condition, causing mutual interference among heterogeneous corrections. To this end, we propose DAME-Net, a Degradation-Aware Mixture-of-Experts Network that decouples explicit degradation perception from degradation-conditioned reconstruction for compositional
UAV image restoration. Specifically, we design a Factor-wise Degradation Perception module(FDPM) to provide explicit per-factor degradation cues for the restoration stage through multi-label prediction with label-similarity-guided soft
alignment, replacing implicit entangled conditions with interpretable and generalizable degradation descriptions. Moreover, we develop a Conditioned Decoupled MoE module(CDMM) that leverages these cues for stage-wise conditioning, spatial-frequency hybrid processing, and mask-constrained decoupled expert routing, enabling selective factor-specific correction while suppressing irrelevant interference. In addition, we construct the Multi-Degradation
UAV Restoration
benchmark (MDUR), the first large-scale
UAV benchmark for compositional
UAV image restoration, with 43 degradation configurations from single degradations to four-factor composites and standardized seen/unseen splits.Extensive experiments on MDUR demonstrate consistent improvements over representative unified restoration methods, with greater gains on unseen and higher-order composite degradations. Downstream experiments further validate benefits for
UAV object detection.
Read the full abstract ▾
All authors Jinquan Yan, Zhicheng Zhao, Zhengzheng Tu, Chenglong Li, Jin Tang, Bin Luo
Computer Vision
VAGNet: Vision-based Accident Anticipation with Global Features
Vipooshan Vipulananthan and Charith D. Chitraranjan
Traffic accidents are a leading cause of fatalities and injuries across the globe. Therefore, the ability to anticipate hazardous situations in advance is essential. Automated accident anticipation enables timely intervention through driver alerts and collision avoidance maneuvers, forming a key component of advanced driver assistance systems. In autonomous driving, such predictive capabilities support proactive safety behaviors, such as initiating defensive driving and human takeover when required. Using dash-cam
video as input offers a cost-effective solution, but it is challenging due to the complexity of real-world driving scenes. Accident anticipation systems need to operate in real-time. However, current methods involve extracting features from each detected object, which is computationally intensive. We propose VAGNet, a deep neural network that learns to predict accidents from dash-cam
video using global features of traffic scenes without requiring explicit object-level features. The network consists of
transformer and graph modules, and we use the
vision foundation model VideoMAE-V2 for global feature extraction. Experiments on four
benchmark datasets (DAD, DoTA, DADA, and Nexar) show that our method anticipates accidents with higher average precision while being computationally more efficient compared to existing methods. It also achieves higher mean time-to-accident scores in most cases, while being competitive in others.
Read the full abstract ▾