Computer Vision · Cross-listed from eess.IV
Unsupervised Deep Image Prior for Sparse-View and Limited-Angle Electron Tomography
Serge Brosset, Daniel del Pozo Bueno, Thomas David et al. (+3 more)
Electron
tomography (ET) plays an important role in the three-dimensional (3D) characterization of nanomaterials. However, under limited-angle and sparse-view conditions, conventional
algorithms produce degraded reconstructions, which compromise the quality and interpretability of resulting 3D data. In this paper, we present deep image prior (DIP), an unsupervised deep learning (DL) approach, for highly degraded
tomography acquisitions and demonstrate, using simulated data, that its performance is comparable to that of supervised approaches requiring training
datasets, even for tilt ranges as limited as 60{\deg} and tilt increments of 10{\deg}. We then apply it to experimental data and show that it enables reliable 3D quantification under both sparse-view and limited-angle conditions, highlighting its potential for a wide range of materials and acquisition modalities.
Read the full abstract ▾
All authors Serge Brosset, Daniel del Pozo Bueno, Thomas David, Laure Guetaz, Philippe Ciuciu, Zineb Saghi
Computer Vision
Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation
Joseph Hoche, David Brellmann and Gianni Franchi
Uncertainty quantification (UQ) remains a critical challenge in Large
Vision Language Models (LVLMs) for reliable predictions and real-world deployment. However, most existing methods are adapted from the LLM literature and primarily focus on the
language modality, leaving the contribution of visual information to LVLM uncertainty largely underexplored. In this paper, we investigate how LVLMs process visual information and whether this process can be used to improve uncertainty estimation. By analyzing hidden representations after the integration of visual features during the generation process, we observe that high-confidence predictions rely more heavily on visual content than uncertain ones. Building on this insight, we propose Visual-Grounded
Token UQ (VIG-TUQ), a training-free framework that explicitly incorporates visual grounding into uncertainty estimation by weighting
token-level
language uncertainty with visual grounding scores. We evaluate VIG-TUQ on multiple
datasets and across diverse LVLM architectures, including early-fusion, late-fusion, and native-fusion models. Results indicate that our method often improves upon existing
token-level uncertainty approaches. Code and data will be made available upon acceptance.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CR
Do Modern Post-Hoc Watermarking Methods Beat Broken-Arrows?
Enoal Gesny and Eva Giboulot
With the rapid proliferation of generative models, such as
diffusion models, digital watermarking has emerged as a crucial solution for identifying AI-generated images. Modern post-hoc watermarking schemes use neural networks to achieve an extremely low false-alarm rate while remaining robust to common image transformations. However, there is a lack of comparison between these modern methods and classic ones, particularly in real-world scenarios where robustness and security take precedence over achieving an extremely low false-alarm probability. In this paper, we propose a fair comparison of robustness and security between modern and classic post-hoc watermarking across various types of classic augmentations and recent sophisticated attacks. Our experiments show that, in a realistic scenario, classic watermarking outperforms modern techniques in terms of security while maintaining robustness.
Read the full abstract ▾
Computer Vision
Image Thresholding: Understanding Bias of Evaluation Metrics towards Specific Evaluation Functions
Eslam Hegazy and Mohamed Gabr
Multilevel image thresholding is widely used for
segmentation in applications ranging from medical imaging to remote sensing. Classical objective functions, such as Otsu's between-class variance and Kapur's
entropy, are often optimized using metaheuristic
algorithms, with performance evaluated via metrics like Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR). These evaluations implicitly assume that SSIM and PSNR provide unbiased measures of
segmentation quality. In this study, we examine this assumption by analyzing the correlation between thresholding objective functions and quality metrics across all possible thresholds for images in the BSDS500
dataset. Results show that Otsu's criterion consistently exhibits high correlation with both SSIM and PSNR, while Kapur's
entropy demonstrates weaker and more variable correlation. Otsu outperforms Kapur in correlation with PSNR for all images and with SSIM for over 91%. Our findings reveal an inherent metric-objective-function bias. This work highlights the need for more neutral evaluation frameworks and motivates extending the analysis to additional thresholding criteria and domains. Source code of this paper can be found at https://w3id.org/met-dp/icpr26-95
Read the full abstract ▾
Computer Vision
YOLO26-RipeLoc Lite: A lightweight architecture for tomato ripeness detection and picking point localization in greenhouse robotic harvesting
Rajmeet Singh, Manveen Kaur, Shahpour Alirezaee et al. (+1 more)
In greenhouse tomato production, automated harvesting requires accurate detection of ripe tomatoes, ripeness classification, and precise picking-point localization for robotic end-effectors. This paper proposes YOLO26-RipeLoc Lite, a lightweight deep learning architecture based on YOLO26 for simultaneous detection, ripeness classification, and center-point localization of greenhouse tomatoes. The model introduces three modifications: (1) a Lightweight Feature Pyramid Network (LFPN) with depthwise separable convolutions for efficient multi-scale fusion, (2) a Ripeness-Aware
Attention Module (RAAM) with dual pooling and a learnable ripeness bias vector for enhanced color-texture discrimination, and (3) a Compact Detection Head (CDH) with shared convolutions and an integrated center-point regression branch for direct grasp planning. The model is evaluated on a custom
dataset of 1,500 images with 6,227 instances (3,566 ripe, 2,661 unripe) from the SILAL greenhouse, Abu Dhabi, UAE. YOLO26-RipeLoc Lite achieves mAP@0.5 of 92.9% (95.2% ripe, 90.6% unripe) with the highest precision (95.2%) among all evaluated architectures using only 2.38M parameters. Post-training BatchNorm
pruning at 30% reduces parameters to ~1.8M with negligible accuracy loss. Ablation studies confirm that greenhouse-aware HSV augmentation provides the largest improvement (+2.02 pp mAP@50), backbone freezing achieves peak precision (93.8%), and 3-phase progressive unfreezing yields the best localization quality (mAP@50:95 of 64.6%). Comparisons with YOLOv8n/s, YOLO11n/s, YOLO12n/s, and YOLO26s confirm superior accuracy-efficiency: 2.9 pp higher precision than YOLO12n with 7.0% fewer parameters and integrated center-point localization for robotic end-effector guidance.
Read the full abstract ▾
All authors Rajmeet Singh, Manveen Kaur, Shahpour Alirezaee, Irfan Hussain
Computer Vision
PILOT: A Data-Free Continual Learning Approach for Real-Time Semantic Segmentation via Boundary Guidance
Yujing Zhou, Prashant Shekhar, Thomas Yang et al. (+1 more)
Real-time semantic
segmentation models offer an excellent balance between accuracy and inference speed. However, deploying these models in dynamic real world environments often requires the ability to learn novel classes incrementally without retraining on the entire
dataset. This capability is known as continual learning. In this regard, the standard fine-tuning methods in deep learning often fail due to catastrophic forgetting, where the model learns new information but forgets previously trained and learned classes. Contributing to this crucial domain, the current paper proposes a novel continual learning framework tailored for PIDNet, which is a widely cited state-of-the-art real-time semantic
segmentation model. Our method, PILOT(Parallel Incremental Learning Over Time), introduces a real-time and lightweight strategy by implementing a parallel Derivative-branch (D-branch) designed to capture the high frequency boundary information of novel classes while freezing the trained parameters of the original
segmentation network. This novel setup allows the model to adapt to new semantic categories while preserving the knowledge of previously learned classes. By using only data associated with the new class, our model significantly reduces training overhead. Experimental results demonstrate that our approach successfully segments new classes while maintaining high mean Intersection over Union (mIoU) on the original base classes, thereby comfortably outperforming all major continual learning approaches in this domain. Overall, PILOT is shown to effectively mitigate catastrophic forgetting with minimal impact on inference latency, thus maintaining real-time performance.
Read the full abstract ▾
All authors Yujing Zhou, Prashant Shekhar, Thomas Yang, Yongxin Liu
Computer Vision
COVD: Continual Open-Vocabulary Object Detection with Novel Concept Injection
Yupeng Zhang, Ruize Han, Yuzhong Feng et al. (+3 more)
Open-vocabulary object detection (OVD) has made significant progress, enabling detectors to generalize from seen to unseen categories. However, real-world category spaces continually evolve, and existing OVD models still struggle with newly emerging concepts, while repeated full retraining is prohibitively expensive. To this end, we introduce a new task setting, termed Continual OVD with Novel Concept Injection (COVD), where models sequentially learn incoming novel concept groups while preserving prior concepts and original open-vocabulary knowledge, along with a new
benchmark, Novel-114. Our key observation is that pretrained visual encoders often already perceive and represent many novel concepts, and the main bottleneck lies in the lack of stable semantic
alignment between visual representations and textual concepts. Based on this, we propose NoIn-Det, an efficient continual injection framework without additional parameters. NoIn-Det freezes the visual encoder, preserves the text representation space using only texts of common concepts and previously injected concepts, and injects novel concepts by updating only a small subset of text-branch parameters beneficial to novel concept learning. Extensive experiments show that NoIn-Det effectively learns novel concepts, preserves old knowledge, and consistently outperforms existing continual learning methods for VLMs without introducing additional parameters.Novel-114 and the code will be released.
Read the full abstract ▾
All authors Yupeng Zhang, Ruize Han, Yuzhong Feng, Zixin Ren, Yuntong Tian, Liang Wan
Computer Vision
Equivalent Flows, Unequal Learning: Clean-Latent Prediction in Transformers
Funing Fu, Tenghui Wang, Guanyu Zhou et al. (+2 more)
Flow samplers consume velocity, but the neural network can predict the clean endpoint and convert it to velocity through a fixed affine readout. We study this choice with JLT, a latent
Transformer in a frozen variational autoencoder (VAE) representation. For squared error, the optimal clean and velocity predictors are algebraically equivalent; a finite
Transformer assigns different computation to its learned output under the two interfaces. A local Gaussian analysis identifies a known residual response supplied by the readout and isotropic target variance added by velocity prediction. Measured FLUX.2 channel spectra support this
geometric distinction: 90% of target variance occupies 83 of 128 clean directions versus 109 velocity directions. Under a matched velocity objective, clean prediction improves ImageNet FID-50K from 6.56 to 2.70 at Base scale and from 2.12 to 1.47 at Large scale, with lower FID at every measured Large checkpoint. Scaling clean prediction to 951M parameters reaches FID-50K 1.19 and IS 271.96. In addition, an objective ablation at Base scale shows that direct clean regression reaches FID-50K 2.38 without time-dependent error weighting. These results show how moving known computation outside the network changes learning under algebraically equivalent flow interfaces. Code: https://github.com/akatsuki-neo/JLT/blob/main/README.md
Read the full abstract ▾
All authors Funing Fu, Tenghui Wang, Guanyu Zhou, Junyong Cen, Qichao Zhu
Computer Vision
Pop-Up Distractions Reveal Bag-of-Events Behavior in Video Large Language Models
Oscar Chew, Serhii Honcharenko, Qian-Hui Chen et al. (+4 more)
A key capability for
video understanding is reliably linking subjects to events across time, yet whether
Video Large
Language Models (VideoLLMs) actually achieve this remains unclear. In this work, we introduce DistractionBench to evaluate whether VideoLLMs can robustly link subjects and events in the presence of unrelated
video segments. Through controlled interventions, such as inserting short advertisement clips into longer
videos, we show that VideoLLMs frequently hallucinate interactions between entities from different segments, incorrectly attributing actions from injected advertisements to subjects in the main
video. We characterize this systematic
hallucination as bag-of-events (BoE) behavior, where models process
videos as collections of events rather than temporally structured sequences. Evaluating 11 popular VideoLLMs, we find that all models exhibit substantial BoE behavior. Our findings suggest that VideoLLMs lack reliable mechanisms for temporal grounding and motivate the development of models with more robust subject-event association.
Read the full abstract ▾
All authors Oscar Chew, Serhii Honcharenko, Qian-Hui Chen, Patricia Lu, Dishant Zaveri, Khoa D. Doan, Kuan-Hao Huang
Computer Vision
Semi-Supervised Gaze Estimation via Disentangled Subspace Contrastive Learning
Qida Tan, Hongyu Yang and Wenchao Du
Appearance-based gaze estimation always suffers from poor generalization due to limited annotated samples and insufficient
dataset diversity. Leading approaches adopt weakly supervised learning to generate large-scale pseudo-labeled data from unconstrained real-world scenarios, aiming to mitigate the domain shifts. In this work, we devise a simple yet effective semi-supervised learning architecture that leverages unlabeled data to enhance domain generalization, thereby reducing reliance on labor-intensive manual annotations. Our key insight is to impose Jacobian regularization to disentangle feature representations into discriminative subspaces dedicated to specific gaze components, such as pitch and yaw angles. We further exploit the intrinsic ordinal ranking within each subspace for contrastive learning, enabling the model to learn robust gaze representations from a small set of labeled samples and an abundance of unlabeled ones. This ultimately yields our Disentangled Subspace Contrastive Learning (DSCL) framework. Extensive experiments on multiple
benchmarks verify that the proposed DSCL is plug-and-play, achieving competitive performance using only 20%, 10%, and even 5% of the annotated data under both in-domain and cross-domain evaluation settings. The public code is available at \href{https://github.com/da60266/DSCL}{https://github.com/da60266/DSCL}.
Read the full abstract ▾
Computer Vision
BeCARE: Budgeted Cache Refresh for Diffusion Transformer Acceleration
Yuhang Zhang, Junxiang Qiu, Huixia Ben et al. (+4 more)
Training-free feature caching accelerates
diffusion transformer (DiT) inference by reusing or
forecasting intermediate features. However, fixed schedules make compute predictable but ignore
prompt- and timestep-dependent risk, whereas hand-tuned error thresholds adapt locally but leave realized compute difficult to control. To address these limitations, we present Budgeted Cache Refresh (BeCARE), a training-free framework that, given a user-specified cap on full computations, adaptively determines when to refresh the cache during accelerated inference. Specifically, we first derive an error-amplification profile from the sampler's noise schedule to characterize the varying impact of approximation errors across denoising timesteps. We then combine this profile with
prompt-specific extrapolation residuals and cache age to form an accumulated risk score, triggering a refresh when continued caching becomes harmful. Meanwhile, we use the same profile to construct an analytic spending reference and adjust the refresh threshold through feedback, thereby distributing the limited refresh budget over the sampling trajectory. Finally, structural safeguards (a fixed warmup, a budget-derived late-stage reserve, and a maximum cache-reuse length) prevent unreliable extrapolation and premature budget exhaustion. These designs enable
prompt-adaptive refresh placement and allow a single calibration to transfer across budget tiers without per-tier tuning. Experiments on FLUX.1-dev with 200
prompts and measured FLOPs show that our method consistently outperforms representative training-free caching baselines across speedups from 3x to 6x. At the main operating point (3.3x acceleration), it improves paired PSNR by 2.7 dB over the strongest baseline using no more compute. On SD3.5 Large, it also substantially improves the same Taylor cache engine over fixed-interval scheduling at matched compute.
Read the full abstract ▾
All authors Yuhang Zhang, Junxiang Qiu, Huixia Ben, Zhenhua Tang, Lin Liu, Shuo Wang, Yanbin Hao
Computer Vision
IPIBench: Evaluating Interactive Proactive Intelligence of MLLMs under Continuous Streams
Jinzhao Li, Yinuo Chen, Wenxuan Song et al. (+5 more)
Recent
multimodal large
language models (MLLMs) achieve strong performance on reactive question answering, but real-world streaming assistants require proactive
reasoning over continuous visual inputs. Existing
benchmarks mainly study reactive or proactive interactions in isolated single-turn settings, overlooking dynamic multi-turn scenarios where users may add, modify, or cancel proactive requests alongside interleaved reactive queries. To address this gap, we introduce IPIBench, the first
benchmark for evaluating Interactive Proactive Intelligence of MLLMs under streaming
video settings. IPIBench covers proactive monitoring, proactive task management, and interleaved reactive-proactive requests. Evaluations on representative MLLMs reveal two major limitations: unstable proactive triggering and weak coordination between reactive and proactive behaviors. We further propose IPI-
Agent, a training-free agentic framework with an interaction-control policy and a temporal-gating mechanism for stabilizing proactive triggering and coordinating multi-turn interactions. Experiments show that IPI-
Agent consistently improves existing MLLMs across all
benchmark settings.
Read the full abstract ▾
All authors Jinzhao Li, Yinuo Chen, Wenxuan Song, Yijia Lei, Yichi Zhang, Honglei Yan, Panwang Pan, Miao Liu
Computer Vision
BEAT: Rhythm-Elastic Alignment for Agentic Music-guided Movie Trailer Generation
Yutong Wang, Yunke Wang, Xinyuan Chen et al. (+1 more)
Automatic movie trailer generation must select shots from a full-length film and synchronize them with background music. Existing methods either relegate music
alignment to post-processing or enforce rigid one-to-one shot-music mappings, overlooking that professional editing rhythm is elastic: rapid cuts accompany high-energy passages while sustained shots span quieter bars. We introduce BEAT, a framework that addresses this gap with two core components: MuVA, a compact music-visual
alignment encoder trained with Sinkhorn-regularized two-stage learning, and Bar-DP, an energy-adaptive dynamic programming
algorithm that produces elastic many-to-one
alignments following musical dynamics. These components are integrated into a five-phase agentic pipeline that grounds the core
alignment in learned cross-modal features while coordinating higher-level creative decisions through structured text signals. To support comprehensive evaluation, we also introduce TrailerArena, a
benchmark with 20+ metrics across four complementary dimensions. On TrailerArena, BEAT achieves state-of-the-art performance across shot selection, ordering, and perceptual quality, while producing fully composed trailers end-to-end.
Read the full abstract ▾
All authors Yutong Wang, Yunke Wang, Xinyuan Chen, Chang Xu
Computer Vision
SCKAN: Structural Consensus-based KAN Prototype Learning for Semi-Supervised Pancreas Segmentation
Yuqi Liu, Yufei Chen, Wei Fu et al. (+2 more)
Accurate pancreas
segmentation is critical for early cancer diagnosis, where annotation scarcity necessitates Semi-Supervised Learning (SSL). However, due to significant inter-sample morphological variability, existing SSL methods face severe generalizability limitations under sparse supervision, leading to the Supervision Bias problem. To address this, we propose Structural Consensus-based KAN Prototype Learning (SCKAN), which constructs the first cross-sample structural consensus learning with Kolmogorov-Arnold Networks (KANs), to achieve more generalizable and accurate
segmentation. Specifically, SCKAN contains two key designs: Structure-constrained Prototype Consistency Learning (SPCL), which
prompts unbiased structural representation by enforcing cross-sample consistency via prototype-level contrastive optimization, and Consensus-based Kolmogorov-Arnold Fusion (CKaF), which reduces morphology-specific bias by aggregating stable consensus and filtering sample-wise noise via KAN's adaptive B-spline nonlinearity. Extensive experiments on two public pancreas
datasets demonstrate the effectiveness of SCKAN. Code is at https://github.com/rhodaliu17/SCKAN.
Read the full abstract ▾
All authors Yuqi Liu, Yufei Chen, Wei Fu, Xiaodong Yue, Shuo Li
Computer Vision
NeR-SC: Adapting Neural Video Representation to Screen Content
Ruohan Shi, Jiaoyan Zhao and Haogang Feng
Implicit neural representations have emerged as a promising paradigm for
video compression, with recent methods achieving competitive performance on natural
video. However, screen content
video – common in remote desktop, online education, and cloud gaming – exhibits distinct statistics: sharp edges, limited color palettes, and strong temporal redundancy. Existing neural representation methods, designed for natural scenes, lack mechanisms to exploit these properties, leaving substantial room for improvement. In this paper, we propose NeR-SC, a neural representation framework tailored for screen content
video. Building on the SNeRV backbone, NeR-SC introduces three screen-content-specific modules: (i) a learnable color palette that models the discrete color structure of screen content by restricting the low-frequency sub-band to a learned color set; (ii) a multi-gate dense fusion module that replaces sequential feature fusion with dense,
attention-gated cross-stage interaction; and (iii) an
embedding-level frame skip strategy that bypasses redundant decoder invocations for static frames, with zero training overhead. Experiments on DSCVC and VCD show that NeR-SC achieves 40.32~dB and 41.73~dB average PSNR, outperforming representative neural
video representation methods and, at low bitrates, surpassing H.264 and H.265. The skip strategy enables real-time decoding with no loss in quality.
Read the full abstract ▾
Computer Vision
Black-box Membership Inference Attacks on the Pre-training Data of Image-generation Models
Tao Qi, Huili Wang, Yuanhong Huang et al. (+6 more)
The rapid advancement of
diffusion-based image generation models has raised serious concerns regarding potential copyright and
privacy infringements involving human-created data. Membership inference attacks (MIAs) have emerged as a promising tool for identifying unauthorized data usage during model training. Existing methods typically assess the ability of model to denoise perturbed suspect images as an indicator of membership status. However, the discriminative power of such features is highly dependent on the degree of model memorization and deteriorates significantly when applied to less exposed data (e.g., pre-training data). Although several methods attempt to enhance detection by leveraging internal model features, these features are generally inaccessible in mainstream closed-source image generation platforms, limiting their practicality. In this paper, we demonstrate that analyzing how a black-box
diffusion model denoises a target image and corresponding perturbed textual instructions can reveal more distinctive membership cues. Based on this insight, we propose a black-box membership inference attack framework (named SD-MIA) that leverages a cross-modal data perturbation mechanism to detect pre-training data in
diffusion models. We conduct extensive experiments on both a public
benchmark dataset and a newly constructed
dataset, each comprising pre-training membership and non-membership samples with identical distributions. Experimental results demonstrate that SD-MIA achieves superior performance compared to existing baselines, including those with the unfair advantage of accessing internal model features.
Read the full abstract ▾
All authors Tao Qi, Huili Wang, Yuanhong Huang, Wendan Wang, Lianchao Zhao, Jinrui Wang, Zichen Qin, Shangguang Wang, Yongfeng Huang
Computer Vision
Timestep-Aware SVDQuant-GPTQ for W4A4 Quantization of Wan2.2-I2V
Junhao Wu, Dezhong Yao and Hai Jin
W4A4 quantization of large
video diffusion Transformers offers substantial
memory savings but is hindered by two main challenges: sparse large-magnitude activation outliers, and strongly timestep-dependent activation distributions across the multi-step denoising trajectory. These difficulties are compounded by Wan2.2-I2V's two-expert Mixture-of-Experts DiT design, whose high-noise and low-noise experts exhibit distinct quantization sensitivities that a single global calibration policy cannot capture. We propose a post-training quantization framework combining SVDQuant-based low-rank outlier compensation, GPTQ-based reconstruction-aware residual weight quantization, and timestep-bin-wise per-layer activation clipping-ratio search conducted independently for each expert. On the OpenS2V-Eval
benchmark, our method reduces peak GPU
memory by 59.3% relative to the BF16 baseline while incurring only a 0.9% drop in VBench average score and a 2.3% drop in Imaging Quality, demonstrating that expert- and timestep-aware calibration is essential for high-fidelity W4A4 inference on MoE
video DiTs.
Read the full abstract ▾
Computer Vision
ChartAct: A Benchmark for Dynamic Chart Understanding
Muye Huang, Lin Wu, Lingling Zhang et al. (+5 more)
Charts are widely used to present complex data for analysis and decision making. Existing chart understanding
benchmarks mainly focus on static charts, but real-world charts are often dynamic and interactive. Key information may only appear after actions such as hovering, clicking, zooming, or dragging. Dynamic chart understanding therefore requires models to identify visible content, choose proper interactions, and reason over changing chart states. To evaluate this ability, we propose ChartAct, an interactive
benchmark for dynamic chart understanding. ChartAct collects and filters 673 dynamic charts from 8 real chart websites, covers 7 common chart types, and constructs 1,440 high-quality question-answer samples. Each sample is instantiated in two environments, Dynamic Chart and Dashboard Chart, to evaluate dynamic chart understanding under different contexts. Based on ChartAct, we systematically evaluate 11 advanced
multimodal models and GUI
agents. Experimental results show that existing models still have clear limitations in dynamic chart understanding. The strongest model, Claude-Opus-4.7, achieves an average success rate of 84.5%, while most models remain below 60%. We also conduct detailed failure attribution and case analysis. ChartAct provides a new
benchmark for studying chart understanding in real interactive environments. Codes at https://github.com/wulin-wulin/OSWorld_Chart
Read the full abstract ▾
All authors Muye Huang, Lin Wu, Lingling Zhang, Hang Yan, Zhiyuan Wang, Yumeng Fu, Zesheng Yang, Jun Liu
Computer Vision
On the Robustness of Machine Unlearning for Vision-Language Models
Yujie Lin, Kaidi Jia, Jiayao Ma et al. (+2 more)
Vision-
language models (VLMs) may memorize undesirable information from training data, motivating growing interest in machine unlearning. In this work, we present the first systematic survey and robustness analysis of VLM unlearning. We provide a comprehensive taxonomy and review of existing VLM unlearning methods, together with unified evaluations under multiple
prompt settings. We then propose three attack paradigms to examine whether forgotten
multimodal knowledge can be reactivated through contextual prompting or downstream retraining. Extensive experiments show that many existing methods remain vulnerable under these attacks, indicating that current approaches often hide rather than fully remove target knowledge. Our study provides new insights into the robustness and limitations of current VLM unlearning methods and highlights the need for more reliable
multimodal unlearning strategies. Code is available at https://github.com/XMUDeepLIT/VLM-UnL-Attack.
Read the full abstract ▾
All authors Yujie Lin, Kaidi Jia, Jiayao Ma, Chengyi Yang, Jinsong Su
Computer Vision
CodecCap: High-Fidelity Codec-Inspired Residual Modeling for Dense Video Captioning
Zihan Lin, Songhe Deng, Shuwei He et al. (+6 more)
Existing
video captioning methods struggle to balance visual fidelity and redundancy: holistic captions are compact but lose fine-grained evidence, whereas segment-wise captions improve coverage but introduce heavy redundancy. We propose CodecCap, a codec-inspired framework for high-fidelity dense
video captioning. Analogous to
video codecs, CodecCap represents
videos using keyframe and residual captions. Keyframe captions exhaustively encode stable visual context, while residual captions capture temporally only localized actions, motions and changes. This effectively preserves fine-grained visual evidence while reducing redundant descriptions. To quantify the fidelity of captions, we introduce VidCapQA, a caption-then-QA
benchmark with 1,000 questions across 14 capability dimensions. Results on VidCapQA show that captions directly generated by strong VLMs still miss many visual details, highlighting caption representation as a critical bottleneck. Experiments show that CodecCap significantly surpasses direct captioning with the same underlying VLMs, suggesting keyframe-residual captioning a way for high-fidelity
video-
language supervision. We further use CodecCap to construct CodecVDC-100K, a large-scale dense captioning
dataset with anchor, residual, scene-level, and
video-level supervision.
Read the full abstract ▾
All authors Zihan Lin, Songhe Deng, Shuwei He, Danxiang Zhu, Dan Zhang, Yishu Lei, Xianlong Luo, Shikun Feng, Rui Liu
Computer Vision
DinoComplete: 3D Shape Completion with Distilled Semantic Priors and State Space Models
Furkan Mert Algan and Eckehard Steinbach
3D shape completion from partial scans remains challenging for unseen categories and noisy real-world observations, where geometry alone is often insufficient for inferring missing structure. We present DinoComplete, a deterministic and efficient shape completion framework that augments
geometric reconstruction with voxel-aligned semantic priors distilled from DINO features. First, we construct multi-view DINO feature volumes aligned with ShapeNet data and train a student network to predict dense semantic features directly from incomplete shapes. These predicted features capture global structure and part-aware semantic context while remaining aligned with the underlying geometry. We then integrate these distilled features into a completion network, where
geometric and semantic voxel representations are fused through voxel state-space modeling. To enable efficient long-range
reasoning without sacrificing resolution, we introduce a multi-scale voxel Mamba module that refines the fused features by combining full-grid and chunk-wise sequence modeling. Experiments on unseen ShapeNet categories and ScanNet objects show that DinoComplete achieves stronger completion quality than prior deterministic and generative based completion methods while using fewer parameters, requiring lower
memory, and achieving faster inference. Our results demonstrate that distilling semantic priors from visual foundation models improves generalization and robustness in 3D shape completion.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Object Pose and Shape Estimation for Grasping: Does it Work?
Pavan Karke, Kushal Shah, Gaurav Singh et al. (+3 more)
The problem of object pose and shape estimation has seen key advancements lately. Encoder-decoder (e.g., SAM3D, LRM, CRISP) and
diffusion-based models (e.g., InstantMesh, Zero123, SceneComplete) have shown category-agnostic shape encoding capacity and open-set generalizability. In this work, we ask the question: Are the object pose and shape estimation methods mature enough, such that when used with antipodal grasp sampling, can outperform the end-to-end grasp synthesis methods? We explore this question in detail by scoping our study to parallel jaw grippers, 7-DoF grasps, and single-view RGB(-D) image as input. We implement and compare a state-of-the-art, end-to-end grasp synthesis method and three modular methods, which first estimate the object pose and shape for all objects in the scene, and generate grasps using antipodal sampling. We observe that the modular methods outperform the end-to-end method in all our experiments. The modular methods are able to synthesize plenty of grasps, even for small objects, where the end-to-end methods fail. The effectiveness of the modular methods is contingent on the accuracy of the pose and shape estimation, and suffers partial degradation in cluttered scenes - a limitation of the existing pose and shape estimation methods. We also analyze the failure modes and run-times for the three modular methods, which use two different ways of object pose and shape estimation: one based on an encoder-decoder model, while another a
diffusion model. Finally, we demonstrate that the single-view object pose and shape estimation methods can be augmented with
vision-
language models to yield
language-conditioned grasps from just single-view RGB-D image as input. We notice comparable performance to the state-of-the-art LERF-TOGO baseline.
Read the full abstract ▾
All authors Pavan Karke, Kushal Shah, Gaurav Singh, Md Faizal Karim, K Madhava Krishna, Rajat Talak
Computer Vision
Leveraging Text-to-Image Diffusion Models for Unsupervised Visual Object Tracking
Zhengbo Zhang, Zhigang Tu, Junsong Yuan et al. (+2 more)
Unsupervised visual object tracking is a challenging task that requires following arbitrary targets in
videos without training on ground-truth annotations. Despite considerable progress, existing state-of-the-art unsupervised trackers often struggle in scenarios that demand fine-grained understanding of semantic and visual structural information within
video frames. Text-to-image
diffusion models are well known for their ability to generate images that accurately reflect the semantics and structures described in the input
prompt, demonstrating a strong grasp of visual semantics and structures. Building on this capability, we approach the unsupervised tracking from a new perspective by exploiting the rich semantic knowledge encoded in pretrained text-to-image
diffusion models. To adapt the
diffusion models, which are originally developed for image generation, to the tracking task, we reinterpret the models as a bridge between text and image modalities. This connection is realized through the cross-
attention mechanism: when both text and an image are input into the models, they highlight the regions of the image that are semantically aligned with the text in the cross-
attention maps. We therefore learn a
prompt that represents the tracking target and activates its corresponding region in the cross-
attention map for each frame, which enables object tracking with the
diffusion model. Specifically, our method Diff-Tracking is composed of two main components: an initial
prompt learner and an online
prompt updater. The initial
prompt learner generates a
prompt that captures the target object in the first frame, allowing the
diffusion model to identify the target. The online
prompt updater refines the
prompt based on motion information, enabling consistent tracking across
video frames. We evaluate our approach on six challenging tracking
datasets demonstrate the effectiveness of our approach.
Read the full abstract ▾
All authors Zhengbo Zhang, Zhigang Tu, Junsong Yuan, De Wen Soh, Bo Du
Computer Vision
Similarity-based representation factorization for revealing interpretable dimensions in representational data
Florian P. Mahner, Ka Chun Lam, Francisco Pereira et al. (+1 more)
The study of representations is widespread across fields, including neuroscience, psychology, and artificial intelligence. While representations are often studied and compared through similarities between stimuli, current methods provide only limited access to the dimensions that shape these representations and are often limited in interpretability. To overcome these challenges, here we introduce Similarity-Based Representation Factorization (SRF), a general computational method for recovering low-dimensional, non-negative, interpretable
embeddings from similarity
matrices derived from measured data. Across simulations and many neural, behavioral, and computational
datasets, SRF recovers interpretable dimensions from diverse forms of representational data, even for very sparsely sampled, incomplete data. The dimensions derived from these
datasets match those obtained by task-specific models, predict independent behavioral properties, improve exploratory analysis, and offer higher power for confirmatory hypothesis testing than comparing similarity
matrices. Together, these results establish SRF as a general-purpose method with broad applications for uncovering, understanding, and using the dimensions underlying representations.
Read the full abstract ▾
All authors Florian P. Mahner, Ka Chun Lam, Francisco Pereira, Martin N. Hebart
Computer Vision
I2PRef: Image-Driven Point Completion with Iterative Refinement
Azhar Hussian, Marina Ritthaler, André Kaup et al. (+1 more)
We present an image-conditioned point cloud completion approach that treats images as the primary
geometric source rather than a secondary guide. To this end, we introduce an Image-to-Point (I2P) module that can reconstruct complete point clouds directly from a single RGB image, with no need for 3D inputs. Additionally, we introduce a
transformer-based Point-to-Point (P2P) refinement module that uses self- and cross-
attention between point
tokens and image features to iteratively refine the coarse I2P output. The I2P module enables the image encoder to learn rich
geometric representations, while the P2P module progressively recovers fine-grained details. Unlike existing
multimodal methods that rely on auxiliary losses or fusion modules, our explicit I2P task provides a strong, geometry-aware prior based on images alone. Extensive experiments on ShapeNet-ViPC demonstrate state-of-the-art completion performance with a 12.3% relative Chamfer Distance improvement over prior methods. Code is available at: https://github.com/AzharSindhi/I2PRef.git
Read the full abstract ▾
All authors Azhar Hussian, Marina Ritthaler, André Kaup, Vasileios Belagiannis
Computer Vision
SIMPC: Learning Self-Induced Mirror-Point Consistency for Unsupervised Point Cloud Denoising
Chengwei Zhang, Xueyi Zhang, Tao Jiang et al. (+4 more)
In point clouds, noise directly perturbs point coordinates that encode both spatial location and geometry, making one-to-one correspondence construction more challenging than in images. Existing methods impose statistical mappings across noisy variants via noise or optimal transport, but suffer from correspondence ambiguity. In this work, we propose Self-Induced Mirror-Point Consistency (SIMPC) to learn deterministic correspondences between points and the underlying surface in an unsupervised manner. For each noisy point, SIMPC generates a mirror-point on the opposite side of the underlying surface, guided by
geometric priors during the denoising process. By encouraging consistency between the denoising targets of the original point and its mirror counterpart, SIMPC effectively localizes the position of underlying surface. Extensive experiments on synthetic and real-world
datasets demonstrate that SIMPC significantly outperforms state-of-the-art unsupervised methods and surpasses several strong supervised counterparts.
Read the full abstract ▾
All authors Chengwei Zhang, Xueyi Zhang, Tao Jiang, Xinhao Xu, Wenjie Li, Fubo Zhang, Longyong Chen
Computer Vision
Small Object Detection in Industrial Recycling: A New Dataset and YOLO Performance Evaluation
Oussama Messai, Abbass Zein-Eddine, Abdelouahid Bentamou et al. (+4 more)
In this paper, we address the problem of detecting small, dense, and overlapping objects, a major challenge in computer
vision. Our focus is on reviewing proposed methods based on deep learning supervised approaches. We provide a detailed comparison of these systems on a new
dataset of more than 10k images and 120k instances, highlighting their performance, accuracy, and computational efficiency in the industrial recycling process use case. Through this comparative analysis, we identify the most reliable systems currently available and the specific challenges they are designed to tackle. Furthermore, we explore the benefits of data augmentation and synthetic images. Based on our analysis, we also propose potential future directions and innovative solutions that could enhance the effectiveness of small, dense and overlapped object detection systems. The scope of our investigations encompasses object detection, length measurement, and anomaly detection within the context of the recycling process. The anomaly detection strategy is robust against variations in image resolution and zoom levels, ensuring reliable performance in industrial applications. The repository of the proposed
dataset, methods and evaluation codes can be found at: https://github.com/o-messai/SDOOD
Read the full abstract ▾
All authors Oussama Messai, Abbass Zein-Eddine, Abdelouahid Bentamou, Mickael Picq, Nicolas Duquesne, Stéphane Puydarrieux, Yann Gavet
Computer Vision
Natural Human Motion Recovery by Aligning High-Order Temporal Dynamics from Monocular Videos
Dingkun Wei, Zehong Shen, Yan Xia et al. (+3 more)
Human motion recovered from
monocular videos often appears overly smooth or dynamically inconsistent, even when joint positions are numerically accurate. We observe that this limitation stems from the absence of reliable high-order temporal cues – velocity and acceleration – which are essential for reconstructing motion that exhibits realistic momentum, timing, and high-frequency detail. We introduce HTD-Refine, a post-processing framework that augments existing Human Motion Recovery (HMR) pipelines using explicitly estimated high-order temporal dynamics. At the core of our system is PVA-Net, a temporal
transformer that infers per-joint 2D positions, 3D velocities, and 3D accelerations directly from a
monocular video. These predicted dynamics serve as soft yet informative constraints in a global optimization procedure that refines world-space trajectories, significantly reducing jitter, suppressing over-smoothing, and restoring physically plausible motion. Extensive experiments on challenging in-the-wild
benchmarks show that HTD-Refine consistently improves state-of-the-art HMR methods, yielding more accurate global trajectories and substantially more natural motion dynamics. Our results highlight the critical role of high-order temporal modeling in advancing
monocular human motion recovery.
Read the full abstract ▾
All authors Dingkun Wei, Zehong Shen, Yan Xia, Georgios Pavlakos, Yujun Shen, Xiaowei Zhou
Computer Vision
RoadGIE: Towards A Global-Scale Aerial Benchmark for Generalizable Interactive Road Extraction
Chenxu Peng, Chenxu Wang, Yimian Dai et al. (+3 more)
Accurate road
segmentation from aerial imagery is fundamental to many geospatial applications. However, existing
datasets often suffer from limited scene diversity, low semantic granularity, and poor structural continuity, restricting their generalization across environments. To address these challenges, we introduce WorldRoadSeg-360K, the largest and most diverse road
segmentation dataset to date, comprising 366,947 high-resolution images collected from 38 countries and 223 cities across various terrains and continents. WorldRoadSeg-360K serves as a comprehensive
benchmark and reveals key challenges in handling diverse and structurally complex scenes. Automated approaches often struggle to preserve road connectivity, while current interactive methods lack efficient,
topology-sensitive tools for real-world road editing. To this end, we present RoadGIE, establishing a novel interactive paradigm for road extraction in remote sensing. Unlike prior point- or box-based prompting strategies, RoadGIE supports connectivity-aware
prompts, including clicks and scribbles, which inherently align with the
topology of road networks. To improve structural consistency and mitigate performance degradation during iterative interactions, RoadGIE integrates an expert-guided prompting strategy and adapts the skeleton-based recall loss for interactive scenarios. RoadGIE achieves state-of-the-art performance in both
segmentation accuracy and topological consistency on WorldRoadSeg-360K and other
benchmarks, while maintaining efficient operation with only 3.7M parameters. The code are publicly available at: https://github.com/chaineypung/RoadGIE
Read the full abstract ▾
All authors Chenxu Peng, Chenxu Wang, Yimian Dai, Yongxiang Liu, Ming-Ming Cheng, Xiang Li
Computer Vision
REVERSE: Reinforcing Evidence Verification and Search for Agentic Image geo-localization
Yong Li, Furong Jia, Dacheng Yin et al. (+4 more)
Image geo-localization aims to determine where a photograph was taken, a task that often requires more than recognizing visible landmarks. Human experts typically solve it through an iterative workflow: they inspect informative regions, form location hypotheses, seek external evidence, and revise their judgments as new clues appear. Existing methods only partially capture this process: direct prediction methods bypass evidence acquisition altogether, while
retrieval-augmented methods introduce external evidence but usually provide limited supervision on the intermediate decisions of where to search, how to query, and how to filter noisy results. We present REVERSE, a framework that reinforces the interplay between evidence search and verification to enable multi-turn agentic
reasoning. REVERSE teaches three intermediate decisions: where to look, what to query, and what evidence to trust. To support this, we construct tool-grounded trajectories with annotated region selections, search observations, and geo-informative evidence labels, and introduce process rewards for visual grounding, query utility, and evidence discrimination. An offline search cache makes
retrieval observations stable and reusable during reinforcement learning, enabling dense supervision over noisy search results. With a 4B model, REVERSE outperforms strong
retrieval-augmented baselines and rivals substantially larger models on Im2GPS3k and YFCC4k. Code is available at https://github.com/yonglleee/REVERSE.
Read the full abstract ▾
All authors Yong Li, Furong Jia, Dacheng Yin, Kang Rong, Fengyun Rao, Jing Lyu, Fan Zhang