Computer Vision
SegPAR: Class-Centric Decision-Based Sparse Attack for Semantic Segmentation
Dongsu Song, DaeYun GO, Boseung Seo et al. (+1 more)
Despite the practical relevance of sparse decision-based black-box threats, they have received limited
attention in semantic
segmentation. To bridge this gap, we adapt the most representative decision-based black-box sparse attacks from the classification domain to serve as baselines, establishing a rigorous
benchmark for this underexplored setting. In this context, we demonstrate that one of the existing methods suffers from severe query inefficiency due to its image-centric pixel accumulation, which rapidly exhausts query budgets across the vast image space. To overcome this, we propose SegPAR, a novel decision-based framework that shifts to a class-centric exploration paradigm. Furthermore, to eliminate the misleading feedback generated by standard decision rewards during pixel accumulation, we introduce a novel discrepancy reward. Extensive experiments show that SegPAR significantly outperforms black-box baselines in sparsity efficiency and MIoU reduction, while remaining competitive with white-box sparse attacks. Code is available at \href{https://github.com/KAU-QuantumAILab/SegPAR}{https://github.com/KAU-QuantumAILab/SegPAR}.
Read the full abstract ▾
All authors Dongsu Song, DaeYun GO, Boseung Seo, Jay Hoon Jung
Computer Vision · Cross-listed from eess.IV
Uncertainty-Aware and Explainable Ensemble Deep Learning Framework for Multi-Class Skin Lesion Classification
Rofiqul Islam and Lilatul Ferdouse
Skin cancer diagnosis from dermoscopic images remains challenging due to high intra-class variability, inter-class similarity, class imbalance, and the limited interpretability of deep learning models. This paper proposes an uncertainty-aware and explainable deep learning framework for multi-class skin lesion classification. The framework combines a
vision transformer model (MaxViT-Tiny) with CNN-based models (ConvNeXt-Tiny and EfficientNetV2-B0) through deep ensemble learning. Monte Carlo (MC) Dropout estimates predictive uncertainty and identifies unreliable predictions, while Grad-CAM++, an explainable AI (XAI) technique, provides visual explanations by highlighting lesion regions that influence model decisions. Evaluated on the HAM10000
dataset, the framework achieves 96% accuracy and 99% ROC-AUC under uncertainty-aware filtering (
entropy < 1.0, confidence >= 0.7), with macro-average precision, recall, and F1-score of 94%, 95%, and 95%, respectively, and 96% weighted-average scores across all three metrics. The results demonstrate accurate, interpretable, and uncertainty-aware skin lesion classification for trustworthy computer-aided diagnosis.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Geometry-Based Compression of Plenoptic Point Clouds
Davi R. Freitas, Gustavo L. Sandri and Ricardo L. de Queiroz
Plenoptic point clouds (PPC) are novel data structures that represent the light from different viewing directions in order to provide a higher degree of realism to regular point clouds. This is achieved by associating each point to multiple colors instead of a single one. Here, we present a method to efficiently compress the attributes of a PPC, consisting of a Karhunen-Loève transform over the color attributes followed by multiple attribute coders with intra prediction capability. This compression scheme can be incorporated within the MPEG's geometry-based PCC (G-PCC) standard, using any of G-PCC's existing solutions for attribute coding. Compression performance assessment using PPCs of different spatial resolutions reveals competitive results in comparison to existing methods, such as RAHT-based or
video-based PCC solutions. We believe our coder to be the new state of the art.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
A Geodesic Cut-Cell Prior for Neural Skinning
Wenchao Ma, Surya Dwarakanath, Yizhak Ben-Shabat et al. (+4 more)
We introduce cut-
cell skinning, a
geometric prior designed to augment data-driven skinning weight generation. While data-driven methods show promise in producing high-quality skinning weights, they often lack the generalizability of classic
geometric approaches. To bridge this gap, we propose a
geometric prior that can be robustly computed for in-the-wild meshes and is efficient for large-scale machine learning workflows. The key idea of our cut-
cell skinning is a fast graph-based approximation of the volumetric geodesics distances, motivated by their importance in classic skinning weight computation. Our method achieves orders of magnitude speedup compared to optimization-based solvers and remains resilient to topological artifacts common in cage- or voxel-based alternatives. We demonstrate the efficacy of the cut-
cell skinning prior by integrating it into recent neural skinning models, showing consistent improvements across existing methods and achieving state-of-the-art results. Project page: https://wenchao-m.github.io/CutCell.github.io/
Read the full abstract ▾
All authors Wenchao Ma, Surya Dwarakanath, Yizhak Ben-Shabat, Dario Kneubühler, Haomiao Jiang, Sharon X. Huang, Hsueh-Ti Derek Liu
Computer Vision
AdvFD: Boosting Visual Generation via Adversarial Fr'echet Distance Loss
Mingju Gao, Jingkai Zhou, Kun Gai et al. (+2 more)
Fréchet distance has recently emerged as an effective distribution-level objective for generator post-training, complementing the conventional sample-level
diffusion and flow-matching losses. However, directly optimizing Fréchet objectives can cause Fréchet hacking. The target metrics keep improving, but visual quality and Fréchet
alignment in other feature spaces may stagnate or deteriorate. We attribute this failure to the static pretrained feature spaces used by existing Fréchet losses. These feature spaces provide incomplete and fixed views of the differences between real and generated distributions. To address this limitation, we propose
Adversarial Fréchet Distance (AdvFD), which complements the static representation targets in FD-Loss with a calibrated adversarially learned representation. AdvFD augments the original static Fréchet objective with a learnable representation that adversarially maximizes the Fréchet discrepancy between real and generated samples, while the generator minimizes the same discrepancy in the resulting adaptive feature space. To prevent the
adversarial representation from trivially increasing the objective through feature amplification, we further introduce real-feature whitening, which normalizes its scale and covariance geometry and stabilizes the min–max optimization. Extensive experiments show that AdvFD consistently improves one-step generator post-training across both JiT and pMF backbones and across different model scales.
Read the full abstract ▾
All authors Mingju Gao, Jingkai Zhou, Kun Gai, Changqian Yu, Hao Tang
Computer Vision · Cross-listed from cs.RO
Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning
Wenrui Bao, Tianyun Jiang, Zhiben Chen et al. (+3 more)
Learning reliable
surgical manipulation policies is bottlenecked by the scarcity of action-labeled demonstrations: teleoperated
surgical robot (e.g., dVRK) trajectories with synchronized kinematics are costly to collect, while
surgical tasks demand precise contact handling, long-horizon
reasoning, and bimanual coordination. Endoscopic
video is comparatively inexpensive and abundant relative to synchronized
video–kinematics trajectories, and a natural way to exploit it is to learn world models of
surgical scenes. However, existing
surgical world models use
video primarily for simulation or policy evaluation, and rarely translate the learned dynamics into closed-loop control. This gap raises our central question: under a fixed budget of action-labeled demonstrations, does action-free
video pretraining improve closed-loop
surgical manipulation? To answer it, we introduce the
Surgical World-Action Model (
Surgical WAM), a unified generative model built on Cosmos Policy that jointly predicts future endoscopic observations and executable
surgical robot action chunks.
Surgical WAM first learns
surgical visual dynamics from action-free
video and is then fine-tuned on the fixed action-labeled budget; at deployment, it acts as a closed-loop, receding-horizon controller that executes a short prefix of each predicted action chunk and replans from the resulting observation. On a suite of four simulated
surgical manipulation tasks,
video pretraining improves the average success rate from 63.5% to 77.8%, including an absolute gain of 20 percentage points on PegTransfer, with the largest improvements on contact-rich and bimanual tasks. These results demonstrate that action-free
video provides transferable visual dynamics priors for learning
surgical robot control with limited action supervision, positioning data-efficient
video pretraining as a practical path toward scaling up
surgical robot learning.
Read the full abstract ▾
All authors Wenrui Bao, Tianyun Jiang, Zhiben Chen, Ser-Nam Lim, Peter D. Peng, Yuzhang Shang
Computer Vision
Capturing Uncertainty in Human Motion for Representation Learning in Soccer
Yizhou Xu, Lars Bretzner, Tiesheng Wang et al. (+1 more)
This paper presents a self-supervised representation learning framework for understanding 3D skeleton-based human motion in soccer, using future motion prediction as the learning objective. Since human motion is inherently uncertain, accounting for multiple plausible futures is essential for capturing the underlying motion dynamics and learning effective representations. To this end, we introduce a conditioning module for motion prediction that models a probabilistic distribution over discretized future motions in 3D Euclidean space, learning multimodality with explicit supervision from future trajectories. Experiments on large-scale soccer player tracking data show that our approach substantially improves motion prediction accuracy. Moreover, the learned representations effectively transfer to multiple soccer downstream applications, demonstrating strong cross-task generalization.
Read the full abstract ▾
All authors Yizhou Xu, Lars Bretzner, Tiesheng Wang, Atsuto Maki
Computer Vision
VidForensics-M1: Meta-Detection Reinforcement Learning with Verifiable Temporal Grounding for AI-Generated Video Forensics
Bowei Liu, Zheng Lu, Yuhan Bian et al. (+8 more)
Recent advances in
video generation models have significantly improved the realism of synthetic
videos, blurring the boundary between generated and authentic content and raising concerns about misinformation. Existing MLLM-based detectors mainly rely on supervised fine-tuning or label-level reinforcement learning, where coarse supervision limits generalization to unseen scenarios and emerging
video generators. To overcome these limitations, we are the first to introduce meta-detection into AI-generated
video detection, enabling reliable forgery detection by jointly optimizing predicted labels and supporting evidence within reinforcement learning. This paradigm requires reliable evidence signals and effective mechanisms to integrate them into label-level optimization. Textual rationales provide semantic descriptions of forgery artifacts, but their generation and verification depend on external models, making supervision vulnerable to
hallucinations and semantic biases. In contrast, temporal grounding provides more objective and verifiable evidence, as manipulated intervals can be precisely controlled during forgery construction. Based on this insight, we propose an automated data construction pipeline that generates paired real-fake
videos by replacing temporal segments with boundary-frame-conditioned
video generation models. Furthermore, we introduce Evidence-Guided Reward Redistribution, which performs evidence-aware credit assignment by redistributing rewards among label-correct responses according to evidence quality. This preserves reliable label supervision while encouraging detectors to acquire fine-grained and verifiable forgery localization capabilities. Extensive experiments demonstrate that VidForensics-M1 effectively leverages verifiable temporal evidence to achieve robust and generalizable AI-generated
video detection.
Read the full abstract ▾
All authors Bowei Liu, Zheng Lu, Yuhan Bian, Xinchen Zhang, Xingming Shui, Yuesheng Huang, Xuhuan Li, Zihao Liu, Yifan Yang, Jun Zhou, Xiu Li
Computer Vision
Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation
Shiyu Xuan and Zechao Li
GUI Visual Grounding is a fundamental capability for GUI
agents. Existing models typically freeze their parameters after deployment, limiting their ability to adapt to unseen interfaces. Although recent methods attempt to adapt models via test-time reinforcement learning, they cannot reflect upon failed exploration. To overcome this, we propose a Test-Time Self-Evolving framework that enables models to improve after deployment without human-annotated ground truth. It constructs a closed-loop of Exploration, Evaluation, Reflection, and Internalization. Specifically, the
agent first explores unseen interfaces by predicting grounding coordinates for given instructions. To evaluate these explorations, we introduce an MLLM-based Reflector to assess the generated results and provide the corresponding
reasoning reflections. To internalize reflection knowledge into the model weights, we propose Reflection-Guided On-Policy Self-
Distillation, which translates high-level
reasoning into dense
token-level supervision via a conditioned self-teacher. Furthermore, we design a Contrastive Calibration method to prevent incorrect auto-regressive prefixes from corrupting the supervisory signals during failed explorations. Extensive experiments across six
benchmarks demonstrate our framework's effectiveness, achieving an average accuracy improvement of 7.4% over the base model. To the best of our knowledge, this is the first work to successfully exploit on-policy self-
distillation for test-time adaptation in GUI visual grounding. By filling the gap in post-deployment adaptation, our framework completes the self-evolving capability of GUI
agents. The code will be released.
Read the full abstract ▾
Computer Vision
MultiModal Code-Switching: Interleaving Visual Objects into Language for Explicit Object-Level Alignment
Changhao Xiang, Shangyu Xing, Zhen Wu et al. (+2 more)
Existing
Multimodal Large
Language Models (MLLMs) predominantly rely on image-text pairs for modality
alignment pretraining, mapping global image representations to long textual descriptions. However, this image-level
alignment suffers from referential ambiguity: models struggle to infer the correspondences between multiple visual objects and textual entities from the global representation, leading to data inefficiency and suboptimal semantic grounding. To address this, we propose
MultiModal Code-Switching (MMCS), a novel pretraining paradigm that provides explicit object-level supervision. Inspired by the linguistic phenomenon of code-switching, MMCS interleaves
vision and
language by replacing textual entities with their corresponding visual objects, enforcing local
vision-
language grounding. We further develop a scalable data synthesis pipeline to generate a pretraining
dataset of 773K samples with accurate object-entity correspondences. Experiments show that MMCS is highly data-efficient: with only 50K samples, it matches or surpasses models trained on 600K image-text pairs. Furthermore, MMCS consistently improves visual grounding and perception capabilities across varying model scales.
Read the full abstract ▾
All authors Changhao Xiang, Shangyu Xing, Zhen Wu, Jianbing Zhang, Xinyu Dai
Computer Vision
CausalSplat: Towards Comprehensive Hierarchical Reasoning in 3D Gaussian Splatting
Jiayu Ding, Meilu Song, Yun Chen et al. (+2 more)
While 3D Gaussian Splatting (3DGS) has advanced open vocabulary scene understanding, existing methods remain confined to explicit queries. They struggle to interpret implicit intents, complex spatial constraints, and commonsense
reasoning required for practical
embodied interactions. To address this gap, we introduce the task of
reasoning 3D Gaussian
segmentation and construct two
benchmarks, Causal-LERF and Causal-ScanNet. These
benchmarks systematically evaluate commonsense, spatial, affordance, and counterfactual
reasoning. Evaluations reveal that current state of the art methods perform poorly on these
reasoning challenges. Therefore, we propose CausalSplat, a framework that integrates
vision-
language models with 3D scene graphs to disentangle explicit structural perception from implicit logical inference. Extensive experiments demonstrate that CausalSplat achieves state of the art performance on our
reasoning benchmarks while showing strong generalizability on standard referring and open vocabulary 3D
segmentation tasks. Project Page: https://jiayuding031020.github.io/CausalSplat
Read the full abstract ▾
All authors Jiayu Ding, Meilu Song, Yun Chen, Wei Gao, Ge Li
Computer Vision
PRMU: A Corpus-Free Benchmark for Person-Centric Knowledge Unlearning in Multimodal Large Language Models
Huafeng Chen, Yueming Lyu, Ziyuan Chen et al. (+4 more)
Multimodal large
language models (MLLMs) have demonstrated remarkable capabilities in storing and recalling rich person-related knowledge, raising increasing concerns about reliable knowledge removal. However, existing machine unlearning approaches for MLLMs typically assume access to original forget and retain corpora, which are often unavailable in realistic deletion scenarios. To address this limitation, we introduce PRMU, a
benchmark for evaluating corpus-free
multimodal unlearning under realistic person-centric deletion requests. PRMU focuses on naturally acquired person-related knowledge and evaluates whether models can remove target knowledge while preserving related knowledge through diverse textual and visual probes, including
adversarial evaluation and fine-grained locality analysis. To facilitate research in this setting, we further introduce Similarity-Gated Projection Editing (SGPE), a lightweight corpus-free unlearning baseline with knowledge displacement, protected parameter-space editing, and locality-aware
multimodal control. Extensive experiments on representative MLLMs reveal that existing unlearning methods often suffer from unfavorable forgetting-locality trade-offs, with significant locality degradation under aggressive forgetting settings, and remain vulnerable to
multimodal knowledge reactivation. Meanwhile, SGPE provides a competitive trade-off between target forgetting, locality preservation, and general
multimodal utility. We hope PRMU can facilitate future research toward realistic and scalable
multimodal machine unlearning. Code and
dataset will be released at https://github.com/2231122/PRMU.
Read the full abstract ▾
All authors Huafeng Chen, Yueming Lyu, Ziyuan Chen, Wenda Tan, Chenyang Si, Liucheng Guo, Caifeng Shan
Computer Vision
SAR2Agri: Learning SAR Intensity Representations for Agricultural Monitoring
Moti Rattan Gupta and Anupam Sobti
Agricultural monitoring faces unique challenges, arising from the landscape's complex temporal, phenological, and
climate dynamics, yet monitoring them is critical for ensuring food security. Synthetic Aperture Radar (SAR) satellites offer all-weather day-night imaging capability supporting key monitoring tasks including crop type mapping, yield prediction and phenological event detection. Existing
multimodal remote sensing foundation models including TerraMind and CopernicusFM learn SAR representations by grounding them in optical imagery using joint encoding and contrastive learning techniques, while SAR-specific foundation models such as SAR-JEPA, SARMAE, and SAR-W-MixMAE primarily focus on target detection, flood mapping, and land cover classification applications. Recent work has introduced phenology inspired temporal pretext tasks with optical imagery which has shown strong performance on agricultural downstream tasks. In this work, we propose the first self-supervised learning pipeline focused on using only SAR intensity imagery for agricultural applications. We improve the temporal pretext tasks through masking and curriculum learning to enhance the pretraining pipeline's ability to capture phenological features from SAR. On the SICKLE
benchmark, our final model achieves 84.9% IoU on crop type mapping, outperforming optical baselines (by 15.3 pt) and existing SAR baselines (by 2.2 pt), demonstrating the effectiveness of our proposed pipeline for pretraining SAR intensity encoders for agricultural monitoring.
Read the full abstract ▾
Computer Vision
Is There Really a Camouflaged Object? Towards Realistic Camouflaged Object Detection
Huafeng Chen, Yueming Lyu, Chenyang Si et al. (+3 more)
Camouflaged object detection (COD) aims to segment objects that are visually concealed in their surroundings and has attracted increasing
attention in recent years. However, most existing COD methods are developed under a closed-world assumption, where each input image is assumed to contain a camouflaged object. This assumption ignores realistic scenarios with pure backgrounds or non-camouflaged objects, causing existing models to produce severe false positives when deployed in open-world environments. To address this limitation, we propose OPC16K, a large-scale
benchmark for realistic COD. OPC16K contains 16,245 images from 14 sources and is carefully organized into camouflaged-object images, pure background images, and non-camouflaged-object images, enabling comprehensive evaluation of both
segmentation quality and negative-sample rejection. Based on this
benchmark, we further propose OPCNet, a presence-aware camouflage network that reformulates COD from a pure
segmentation task into a joint problem of object localization and camouflage existence
reasoning. Specifically, OPCNet introduces hierarchical existence
reasoning to distinguish CO, BG, and NOCOD scenarios, similarity-aware camouflage relation modeling to capture foreground-background camouflage cues, and existence-aware feature refinement to regulate
segmentation features with existence predictions. Extensive experiments on OPC16K demonstrate that OPCNet achieves superior performance under the proposed realistic COD evaluation protocol, significantly reducing false positives on negative samples while maintaining accurate camouflaged-object
segmentation. Code and
dataset will be released at https://github.com/2231122/OPCOD.
Read the full abstract ▾
All authors Huafeng Chen, Yueming Lyu, Chenyang Si, Wende Tan, Liucheng Guo, Caifeng Shan
Computer Vision
AlbumentationsX: One Augmentation Pipeline for Images and Related Annotations
Vladimir Iglovikov
Augmentation can corrupt a training example when an image and its annotations receive different random changes. A crop must use the same coordinates for the image, mask, boxes, keypoints, stereo views,
video frames, or volume. Code paths that choose these values separately can silently misalign the data. AlbumentationsX keeps the transform list, probabilities, annotation settings, and random seed in one Compose object. Each call chooses random values once and applies them to every supported part of the training example. The library keeps each object's mask, box, and label together and lets projects add their own transforms. It can also save the pipeline definition, show what happened in one call, and run that call again. The examples place Compose after files have been decoded into arrays and before PyTorch groups examples into a batch. AlbumentationsX executes the declared transforms. Practitioners still decide whether a flip, crop, color change, or other operation preserves the correct label for their task.
Read the full abstract ▾
Computer Vision
Every Packet Counts: Dispersing Information for Loss-Resilient Learned Image Compression
Yuhang Wei, Chuqin Zhou, Yibo Shi et al. (+2 more)
Learned image compression (LIC) has achieved impressive rate-distortion performance. However, existing methods remain highly vulnerable to packet loss, a common challenge in satellite and emergency communications. This vulnerability stems from non-uniform information distribution at the packetization stage and sequential decoding dependencies at the
entropy coding stage. We propose an end-to-end loss-resilient image compression scheme that addresses both. Before packetization, we introduce an Inter-Channel Redistribution (ICR) mechanism to redistribute channel energy, preventing critical information concentrating in a small subset of channels. Then, an Interleaved Channel Grouping (ICG) strategy partitions latent channels in a strided manner to disperse information across packets, with each packet kept within constrained sizes. To limit cascading errors from lost packets, we adopt a two-layer dual-branch autoregressive structure to shorten the dependency chain. Extensive experiments demonstrate that our method consistently outperforms existing approaches in both reconstruction quality and stability. At 20% packet loss, it achieves an average PSNR gain of 1.84 dB over LossResilientLIC while reducing PSNR variance by an order of magnitude. Notably, trained under uniform random loss only, our model generalizes to bursty loss modeled by the Gilbert-Elliott channel, outperforming methods explicitly trained for such conditions.
Read the full abstract ▾
All authors Yuhang Wei, Chuqin Zhou, Yibo Shi, Jing Wang, Guo Lu
Computer Vision · Cross-listed from cs.LG
Cross-View Feature Matching: Survey, Benchmarking, and Foundation-Model Perspectives
Songlin Du, Xiaoyong Lu, Zeyu Wu et al. (+5 more)
Cross-view feature matching aims to establish reliable correspondences across images with large viewpoint variations. Over the past decade, the field has evolved from task-specific models toward increasingly unified and generalizable correspondence models, with recent progress further driven by the emergence of
vision foundation models (VFMs). Despite these advances, existing studies remain highly diverse in their problem formulations, model architectures, training paradigms, and evaluation protocols, making it difficult to obtain a unified understanding of the field. In this survey, we present a unified review of cross-view feature matching. We first introduce a structured taxonomy covering feature extraction, single-type feature matcher, multi-type feature matcher, VFMs based methods, training strategy and robust estimation, providing a coherent framework for analysis and comparison. We further examine recent advances, distilling key design principles and highlighting the shift toward unified and generalizable correspondence models. We also provide a unified experimental benchmarking of representative state-of-the-art methods under consistent protocols, enabling fair and comprehensive performance comparisons. In addition, we discuss open challenges and future directions, including efficiency, robustness under extreme conditions, and cross-domain generalization. This survey aims to provide a comprehensive and structured reference for understanding the
evolution, current landscape, and future development of cross-view feature matching in the era of
vision foundation models.
Read the full abstract ▾
All authors Songlin Du, Xiaoyong Lu, Zeyu Wu, Xiaobo Lu, Guobao Xiao, Bin Fan, Jiayi Ma, Takeshi Ikenaga
Computer Vision
Learning Gaussian Structure: Intervention-Guided Density Control for Feed-Forward Driving Reconstruction
Hang Li, Jiahe Li, Meiying Gu et al. (+3 more)
Feed-forward Gaussian reconstruction has recently emerged as an efficient approach for driving scene reconstruction. However, prevailing
LiDAR-based methods preserve the initial correspondence between observed points and Gaussian primitives, treating the initialized primitive set as the final representation. Unlike optimization-based 3DGS, these methods cannot accumulate gradients during training to determine how the scenes representation should be densified. Meanwhile, the shared sparse backbone only fuses observations from different timestamps implicitly, without explicitly aggregating cross-time evidence for individual primitives. In this paper, we present Learning Gaussian Structure (LGS), a framework that enhances both Gaussian structure and primitive attributes. Our key observation is that changes in local gradient responses induced by a prune or add intervention reveal whether the corresponding structural adjustment benefits reconstruction. Based on this observation, our Gaussian Densify Policy learns a Densify Map comprising Prune and Addition Scores from controlled interventions, and directly adjusts the Gaussian structure during inference. We further develop a compact Cross-Time Point Query that explicitly retrieves and aggregates neighboring features from Gaussian primitives at other timestamps for reliable attribute prediction. Extensive experiments on the Waymo Open
Dataset and PandaSet demonstrate that LGS consistently outperforms existing methods.
Read the full abstract ▾
All authors Hang Li, Jiahe Li, Meiying Gu, Jin Zheng, Lina Yu, Xiao Bai
Computer Vision
Foundation Model-Enabled Efficient Data Sampling (FEEDS): A label-efficient training strategy for pan-cancer, multi-tracer PET/CT datasets
Biratal Raj Wagle, Bashirul Azam Biswas, Grant Chau et al. (+5 more)
Automated lesion
segmentation in whole-body PET/CT imaging can assist clinicians with cancer detection, staging, and treatment planning across radiotracers and cancer types. However, training lesion
segmentation models that capture variations in lesion size, distribution, and appearance requires large annotated
datasets, whose creation is both time- and expertise-intensive. As a result, models trained on limited labeled PET/CT data often lack the accuracy and generalizability needed for clinical use. We present FEEDS (Foundation model-Enabled Efficient Data Sampling), a label- and compute-efficient learning strategy that uses
vision foundation model
embeddings to select the most informative and diverse unlabeled cases for expert annotation. Unlike unsupervised, semi-supervised, and active learning approaches, FEEDS is a one-step training paradigm requiring only a limited, representative training set, making it label- and compute-efficient. We train and validate FEEDS using the AutoPET-III
dataset. We test its accuracy and generalizability on three held-out sets: AutoPET-III, DeepPSMA, and an internal Dartmouth-Hitchcock Medical Center
dataset. We evaluate clinical utility at the voxel, lesion, and anatomic region level to assess performance in high-risk areas and treatment planning utility. FEEDS outperforms random-sampling-based labeling, pseudolabel-based semi-supervised learning, and training with limited labeled data alone. It generalizes across all three test sets, FDG and PSMA tracers, and multiple diseases, matching fully-labeled (100%) training performance with 70% less annotation burden. FEEDS addresses the challenge of label scarcity in an automatic lesion
segmentation framework by providing a practical approach for constructing representative and diverse annotation queues from large, unannotated clinical repositories.
Read the full abstract ▾
All authors Biratal Raj Wagle, Bashirul Azam Biswas, Grant Chau, Matthew E. Maeder, Muhammad Azeem Arshad, Michael S. Leapman, James B. Yu, Indrani Bhattacharya
Computer Vision
Static in Frames, Dynamic in Events: Rethinking Features in Event Cameras as Motion Cues
Hesam Araghi, Jan van Gemert and Nergis Tomen
Event cameras capture intensity changes asynchronously with high temporal resolution, requiring novel preprocessing methods for downstream tasks. Unlike static intensity snapshots, event data inherently encode information about scene dynamics and object motion, meaning that features derived from events can exhibit behaviors with no direct analogue in frame-based
vision. In this paper, we analyze two features used in event-based corner detection—the
eigenvalues of the structure tensor and the spatiotemporal density values—and show that they are motion cues. We hypothesize that these features, combined with local
geometric information, can enhance motion estimation tasks. To validate this, we first theoretically analyze how the
eigenvalues of the structure tensor at moving corner points relate to the direction of motion. We then design controlled experiments on a synthetic
dataset, confirming that extending local
geometric features with
eigenvalues and density values provides complementary motion information and is robust to texture and shot noise. Finally, we integrate the proposed features into a state-of-the-art event-based optical flow network and evaluate on the real-world DSEC
benchmark, where the added features consistently improve accuracy, with the largest gains in data-scarce scenarios and for lower-capacity models. The code for this paper can be found at: \href{https://github.com/hesamaraghi/static-in-frames-dynamic-in-events}{https://github.com/hesamaraghi/static-in-frames-dynamic-in-events}.
Read the full abstract ▾
Computer Vision
CapProbe: Evaluating Detailed Image Captions via Full-Scene Dense Question Answering
Mouxiao Huang, Qiangyu Yan, Borui Jiang et al. (+1 more)
Evaluating detailed image captions from
Vision-
Language Models (VLMs) requires going beyond surface-level semantic similarity. Reference-based metrics (e.g., CIDEr and SPICE) and LLM-as-scorer protocols struggle to verify dense factual claims, while existing QA-based alternatives generally offer lower probe density, narrower domain coverage, or no explicit
alignment between individual questions and segmented image regions. We introduce CapProbe, a full-scene dense QA
benchmark that turns detailed caption evaluation into region-aligned factual checking. Each image is decomposed into coarse semantic regions covering both foreground and background elements; for every retained region, we generate multiple-choice questions spanning 10 semantic categories, forming a dense checklist of probed visual facts. Guided by a two-tier taxonomy of 37 L1 domains and 219 L2 sub-domains, CapProbe comprises 346 images, 1,868 regions, and 25,650 questions, averaging 74 QA pairs per image. A
language judge answers from the caption alone; an Uncertain option and Effective Accuracy provide a judge-dependent proxy for distinguishing unanswered probes from incorrectly resolved ones, while density-based metrics penalize verbose yet uninformative captions. The protocol is cost-effective: by converting unconstrained scalar scoring into structured MCQ reading, it reduces open-ended scoring bias while remaining judge-conditioned and yields relatively stable model rankings under a fixed reader. Experiments on 13 VLMs show large Coverage gaps across models, a clear competency-efficiency trade-off, and failure modes that sparse or overlap-based evaluation often misses. The
benchmark data, annotations, and evaluation code will be released soon.
Read the full abstract ▾
All authors Mouxiao Huang, Qiangyu Yan, Borui Jiang, Han Shu
Computer Vision
Entropy-Centric Explainable AI for Remote Sensing Image Segmentation
Ali Saleh, Abdul Karim Gizzini, Mohamad Ghassany et al. (+1 more)
Artificial intelligence (AI) has become a powerful approach to solving complex problems in critical domains. Many concerns arise regarding the decision-making process of its models, mainly due to deep neural networks outperforming their peers at the cost of ambiguity in feature extraction and prediction. Consequently, in critical domains such as remote sensing, where high-resolution imagery must be analyzed using black-box models, the lack of transparency limits trust in these models and, thus, their adoption. In light of this reality, explaining and understanding the complex decision-making process of AI models has become essential. Explainable AI (XAI) aims to bridge this gap by providing insights into how and why certain decisions are made. While significant progress has been achieved in explaining image classification tasks, image
segmentation still offers considerable room for improvement. In this context, this paper proposes an
entropy-centric XAI method for semantic
segmentation. Moreover, a new XAI evaluation methodology is proposed to efficiently measure the relevance of the regions highlighted by the proposed XAI method. Experimental results demonstrate the superiority of the proposed XAI method compared with recently adapted XAI methods for semantic
segmentation.
Read the full abstract ▾
All authors Ali Saleh, Abdul Karim Gizzini, Mohamad Ghassany, Ali J. Ghandour
Computer Vision
A Comparative Evaluation of Deep Learning Object Detection Models on a Real-World Multi-Plant Dataset from Africa
Ismail Ismail Tijjani, Sunusi Muhammad Ibrahim, Amina Ibrahim Khaleel et al. (+5 more)
The application of computer
vision in agriculture has shown significant potential for improving crop monitoring and precision farming. However, many existing approaches rely on controlled
datasets that do not adequately represent realworld farming conditions, particularly in underrepresented regions such as Africa. This study presents a comparative evaluation of six object detection models YOLOv5, YOLOv8, YOLO11, YOLO26, Faster R-CNN, and RT-DETR using a real-world
dataset, AgriAISeg 1 , collected manually from Nigerian farms. AgriAISeg comprises 3,382 images of sesame, cabbage, and tomato crops captured under varying environmental conditions, including changes in illumination, occlusion, and viewing perspectives. Models were trained, and performance was assessed using precision, recall, mAP@0.5, and mAP@0.5:0.95. The results show that RT-DETR achieved the highest overall performance with a precision of 0.768 and mAP@0.5:0.95 of 0.624, while YOLOv8 and YOLO11 also demonstrated strong and consistent performance. In contrast, Faster R-CNN recorded significantly lower accuracy, with an overall mAP@0.5 of 0.466, indicating reduced effectiveness under complex field conditions. In addition, YOLO-based models exhibited superior training efficiency compared to Faster R-CNN.These findings demonstrate that modern one-stage and
transformer-based detectors provide more reliable and efficient solutions for plant detection in realworld agricultural environments.
Read the full abstract ▾
All authors Ismail Ismail Tijjani, Sunusi Muhammad Ibrahim, Amina Ibrahim Khaleel, Lanre Olusegun Akinola, Fatima Isa Jibrin, Muhammad Bashir Aliyu, Abdullahi Abdussalam Dalhat, Abdullahi Suiudeen
Computer Vision
HUI360: A 360{\deg} Egocentric Dataset and Baselines for Human-Robot Interaction Anticipation
Raphael Lorenzo-Louis, Fabio Amadio, Bertrand Luvison et al. (+1 more)
As
robots increasingly operate in human-populated environments, anticipating human intentions is essential for enabling proactive and socially aware behavior. Automatic anticipation of human-
robot interactions is thus emerging as a crucial perception challenge for
embodied agents. To this end, we introduce HUI360, the largest
dataset for human-
robot interaction anticipation in the wild and its set of baselines. The
dataset was collected from a mobile
robot, in the wild, over multiple days within a 3-month period, and in several environments, capturing natural, spontaneous behaviors from both passersby and users, and encompassing a diverse range of individuals. This variety enables evaluating and improving the generalization capabilities of interaction anticipation models. We designed a pipeline and share code for automatic interaction annotation in arbitrary 360-degree equirectangular
videos, along with interfaces for manual refinement. Using this pipeline, we release the HUI360 open set of 1M pre-processed annotations, including detailed 2D poses, facial keypoints, and
segmentation masks, obtained using state-of-the-art computer
vision methods and manually curated to ensure high-quality tracking and interaction annotation. Additionally, we release the raw panoptic 360-degree images captured from the
robot's
egocentric viewpoint (on demand, for research purpose only in compliance with GDPR). Finally, we establish
benchmark baselines for interaction anticipation, including the first cross-
dataset evaluations for this task: to this end, we also release 6M annotations for another existing in-the-wild outdoor
dataset collected from a mobile
robot (SSUP-HRI).
Dataset and code can be found at https://hucebot.github.io/hui360.
Read the full abstract ▾
All authors Raphael Lorenzo-Louis, Fabio Amadio, Bertrand Luvison, Serena Ivaldi
Computer Vision
3D Weighted Geometric Graph Neural Networks for Sheep Facial Pain Assessment
Alam Noor, Luis Almeida and Mohamed Daoudi
Deep learning systems perform mainly within the 2D for a single image domain and take the face as a single-dimension representation, losing sight of the 3D anatomy of sheep and cross-landmark spatial relationships that are intrinsic to the clinically proven Sheep Pain Facial Expression Scale (SPFES). This paper presents the 3D Sheep Pain Facial Expression System (3D-SPFES), a novel,
monocular depth-aware
geometric graph neural network system that integrates each SPFES facial landmark, such as the ears, eyes, and nose, into 3D Euclidean space estimated from a single RGB camera by using VideoDepthAnything, thus preventing the need for specialized depth hardware. Each landmark node includes a feature vector containing its 3D spatial coordinates, estimated surface normal, and facial attribute class
embedding. Edges linked to nodes are assigned weights based on an aggregate metric that combines both Euclidean distance and surface co-planarity in a 3D space. A Weighted
Geometric Graph Neural Network (WG-GNN) studies this graph using $\mathcal{K} = 3$ geometry-aware message-passing layers enhanced by a scaled dot-product
attention method that selectively enhances anatomically relevant inter-landmark messages. The resultant node
embeddings are combined into $\mathcal{O} = 3$ pain-level clusters and integrated into a Normalized Pain Score (NPS) within the range of $[0, 100%]$ a confidence-weighted, SPFES-derived scoring method.
Read the full abstract ▾
Computer Vision
Multi-Level Evidence Aggregation for Robust Facial Phenotype Retrieval in Rare Genetic Disorder Prioritization
Alexander Hustinx, Carolin Kaffiné, Behnam Javanmardi et al. (+2 more)
AI-assisted facial phenotyping supports rare genetic disorder prioritization by retrieving visually similar diagnosed cases from facial image reference databases such as the GestaltMatcher Database (GMDB). Existing GestaltMatcher-based
retrieval frameworks compare each test image with individual gallery images in a facial phenotype
embedding space. However, this pointwise formulation does not fully exploit available evidence, because patients may have multiple images and disorders may be represented by multiple diagnosed gallery patients. We propose an inference-time multi-level evidence aggregation framework that improves facial phenotype
retrieval without modifying the underlying GestaltMatcher-Arc encoder. The framework combines
embedding-level patient aggregation of multiple images from the same individual, patient-weighted disorder centroids, and hybrid individual-centroid scoring to integrate test-patient observations, disorder-level gallery evidence, and local nearest-neighbor evidence. We evaluated the approach on GMDB v1.1.4 across disorders represented during training (GMDB-Freq), unseen disorders (GMDB-Rare), and multi-image patient subsets, using a unified gallery containing both GMDB-Freq and GMDB-Rare disorders. Multi-level evidence aggregation improved mean per-disorder top-$N$
retrieval accuracy across all evaluation subsets. Top-1 accuracy increased from 38.52% to 48.82% on GMDB-Freq and from 19.38% to 23.79% on GMDB-Rare. On multi-image subsets, top-1 accuracy increased from 46.12% to 60.94% on GMDB-Multi-Freq and from 18.54% to 26.71% on GMDB-Multi-Rare. These findings show that inference-time aggregation can improve next-generation facial phenotype
retrieval without retraining the encoder, supporting a shift from isolated single-image matching toward multi-level aggregation of patient and disorder evidence for rare-disorder prioritization.
Read the full abstract ▾
All authors Alexander Hustinx, Carolin Kaffiné, Behnam Javanmardi, Tzung-Chien Hsieh, Peter Krawitz
Computer Vision
When Visual Signals Mislead: A Mechanistic Study of Attribute Hallucination in Vision-Language Models
Yufei Zhang, Chenlu Zhan and Hongwei Wang
Attribute
hallucination—where
vision-
language models (VLMs) correctly identify an object but mischaracterize its properties—is prevalent yet mechanistically poorly understood. The dominant explanation,
language-prior dominance, has motivated prior-suppression methods, but this explanation has not been directly tested at the attribute level. We present VISOR (Visual-Operational Remediation), a unified framework that couples null-image-based diagnosis with routed remediation. Its VSNR diagnostic decomposes each prediction into a visual logit signal and a
language-prior signal. Across 10,791 negative-ground-truth samples from three VLM families and three attribute types, the visual signal strongly predicts false positives, whereas the
language-prior signal is near chance. VISOR uses this diagnosis to separate two failure modes: low-margin but directionally correct visual signals in color/state attributes, and low-SNR or misaligned visual signals in material attributes. The same diagnosis routes each query to the appropriate operator: calibration for threshold-placement errors, abstention for training-free low-SNR handling, or targeted visual adaptation for material failures that prior suppression cannot correct. Across Qwen, InternVL, and LLaVA, VISOR reduces attribute false positives without relying on the prior-dominance assumption.
Read the full abstract ▾
Computer Vision
R4DSG: Relative 4D Scene Graph Memory for Object-Centric Question Answering in Long Egocentric Video
Ke Ma, Yamin Mao, Weiming Li et al. (+5 more)
Long-horizon
egocentric video is a rich substrate for wearable AI assistants, but object-centric questions such as where an item was moved, when it last changed state, or why it was relocated remain difficult because caption- and transcript-based
memories rarely preserve persistent object identity or structured spatial change. Existing long-
video QA methods mainly emphasize temporal grounding and clip
retrieval, while prior 3D scene-graph methods typically assume stronger geometry than free-motion wearable RGB
video provides, including point clouds, RGB-D input, posed views, sparse reconstruction, or reconstructed scenes. R4DSG introduces a relative 4D scene graph
memory for long
egocentric video. Instead of storing raw graph sequences, R4DSG converts
video into compact queryable
memory entries indexed by time, place, persistent objects, anchor-relative change, and local interaction context. The main idea is to separate stable anchors from dynamic objects, maintain persistent object identity across frames, and represent object state through anchor-relative transitions rather than a globally aligned world model. Built on recent RGB-only advances in promptable
video segmentation, temporal propagation, and relative 3D lifting, the method produces a
retrieval-ready
memory directly usable for long-horizon question answering. Evaluation on a 255-question object-related subset from EgoLifeQA shows, under question-only
retrieval, a 6.7-point overall gain over EgoRAG-Text and a 12.5-point gain on when questions, which highlights the value of temporally organized object
memory. These results position relative 4D scene graphs as a practical
memory substrate for wearable assistants, AR systems, and
embodied multimedia
agents. GitHub Page: https://dualtransparency.github.io/R4DSG/.
Read the full abstract ▾
All authors Ke Ma, Yamin Mao, Weiming Li, Shuai Tan, Yijie Zhong, Hao Chen, Haofen Wang, Meng Wang
Computer Vision
Watching Synthetic Videos: Aligning Cross-modal Representations with Visual Synthesis for Zero-shot Video Captioning
Liangyu Fu, Junbo Wang, Yuke Li et al. (+3 more)
Text-only training is a popular paradigm in zero-shot
video captioning, where the
video distribution is not available to the model during training, leading to a cross-modal gap between the training (text-only) and the inference (
video-only). Previous works attempt to bridge the gap through simple linear transformations. However, the inherent gap between text and
video makes cross-modal representation space
alignment insufficient, resulting in inaccurate sentences. To address this issue, we propose a novel zero-shot
video captioning framework (WSV) consisting of two training stages, which first generates corresponding synthetic
video latent representations via a pretrained text-to-
video generation model. To strengthen the fidelity of the latent representations, we propose a polisher capable of bridging the gap between real and synthetic
video distributions. Subsequently, we design a prompter that conditions GPT-2 on the polished latent representations to generate the captions in the second training stage. During inference, an input
video is encoded by a pretrained 3D Causal VAE and then fed directly into the prompter, which in turn guides GPT-2 to produce the final caption. Experimental results conducted on MSVD, MSR-VTT, and VATEX
datasets demonstrate that our proposed method achieves scores of 52 and 95.7 on the B@4 and CIDEr metrics, respectively.
Read the full abstract ▾
All authors Liangyu Fu, Junbo Wang, Yuke Li, Ya Jing, Xuecheng Wu, Zhiyong Wang
Computer Vision
HNDiff: Haze-Noise Diffusion for Image Dehazing
Jin-Ting He, Fu-Jen Tsai, Yan-Tsung Peng et al. (+3 more)
Existing
diffusion-based methods have recently made significant progress in image dehazing. However, they typically neglect the physics of haze formation and reconstruct clean images from pure Gaussian noise, thereby limiting their restoration potential. To address this issue, we propose Haze-Noise
Diffusion (HNDiff), a novel
diffusion framework that embeds the atmospheric scattering model as an inductive bias. By grounding
diffusion in physical principles, HNDiff ensures that the restoration aligns more closely with underlying mechanisms of haze formation. In its forward process, we introduce joint haze-noise
diffusion with a haze-aware noise scheduler, which progressively adds both haze and noise to an image. Essentially, the scheduler adapts noise levels according to haze density, meaning that regions with heavier haze receive stronger noise injection to encourage content generation, while clearer regions receive lighter noise to better preserve details, which directly links the forward degradation process with the physics of haze. In the reverse process, we then derive a physically consistent dehazing-denoising process that simultaneously removes haze and noise to restore a clean image in a manner aligned with the forward degradation process. To further enhance practicality, we propose Latent HNDiff, which compiles clean latent priors that can be seamlessly integrated into existing dehazing networks to boost performance. Extensive experiments show that our work significantly improves leading dehazing backbones and achieves state-of-the-art results on
benchmark datasets. The project page is available at https://jin-ting-he.github.io/HNDiff .
Read the full abstract ▾
All authors Jin-Ting He, Fu-Jen Tsai, Yan-Tsung Peng, Min-Hung Chen, Chia-Wen Lin, Yen-Yu Lin