Computer Vision
Geometry-Guided Self-Supervision for Ultra-Fine-Grained Recognition with Limited Data
Shijie Wang, Yadan Luo, Zijian Wang et al. (+3 more)
This paper investigates the intrinsic geometrical features of highly similar objects and introduces a general self-supervised framework called the
Geometric Attribute Exploration Network (GAEor), which is designed to address the ultra-fine-grained visual categorization (Ultra-FGVC) task in data-limited scenarios. Unlike prior work that often captures subtle yet critical distinctions, GAEor generates
geometric attributes as novel alternative recognition cues. These attributes are determined by various details within the object, aligned with its
geometric patterns, such as the intricate vein structures in soybean leaves. Crucially, each category exhibits distinct
geometric descriptors that serve as powerful cues, even among objects with minimal visual variation – a factor largely overlooked in recent research. GAEor discovers these
geometric attributes by first amplifying geometry-relevant details via visual feedback from a backbone network, then
embedding the relative polar coordinates of these details into the final representation. Extensive experiments demonstrate that GAEor significantly sets new state-of-the-art records in five widely-used Ultra-FGVC
benchmarks.
Read the full abstract ▾
All authors Shijie Wang, Yadan Luo, Zijian Wang, Haojie Li, Zi Huang, Mahsa Baktashmotlagh
Computer Vision
Divide-and-Conquer Approach to Holistic Cognition in High-Similarity Contexts with Limited Data
Shijie Wang, Zijian Wang, Yadan Luo et al. (+3 more)
Ultra-fine-grained visual categorization (Ultra-FGVC) aims to classify highly similar subcategories within fine-grained objects using limited training samples. However, holistic yet discriminative cues, such as leaf contours in extremely similar cultivars, remain under-explored in current studies, thereby limiting recognition performance. Though crucial, modeling holistic cues with complex morphological structures typically requires massive training samples, posing significant challenges in data-limited scenarios. To address this challenge, we propose a novel Divide-and-Conquer Holistic Cognition Network (DHCNet) that implements a divide-and-conquer strategy by decomposing holistic cues into spatially-associated subtle discrepancies and progressively establishing the holistic cognition process, significantly simplifying holistic cognition while reducing dependency on training data. Technically, DHCNet begins by progressively analyzing subtle discrepancies, transitioning from smaller local patches to larger ones using a self-shuffling operation on local regions. Simultaneously, it leverages the unaffected local regions to potentially guide the perception of the original topological structure among the shuffled patches, thereby aiding in the establishment of spatial associations for these discrepancies. Additionally, DHCNet incorporates the online refinement of these holistic cues discovered from local regions into the training process to iteratively improve their quality. As a result, DHCNet uses these holistic cues as supervisory signals to fine-tune the parameters of the recognition model, thus improving its sensitivity to holistic cues across the entire objects. Extensive evaluations demonstrate that DHCNet achieves remarkable performance on five widely-used Ultra-FGVC
datasets.
Read the full abstract ▾
All authors Shijie Wang, Zijian Wang, Yadan Luo, Haojie Li, Zi Huang, Mahsa Baktashmotlagh
Computer Vision
Silicon Aware Neural Networks
Sebastian Fieldhouse and Kea-Tiong Tang
Recent work in the machine learning literature has demonstrated that deep learning can train neural networks made of discrete logic gate functions to perform simple image classification tasks at very high speeds on CPU, GPU and FPGA platforms. By virtue of being formed by discrete logic gates, these Differentiable Logic Gate Networks (DLGNs) lend themselves naturally to implementation in custom silicon - in this work we present a method to map DLGNs in a one-to-one fashion to a digital CMOS standard
cell library by converting the trained model to a gate-level netlist. We also propose a novel loss function whereby the DLGN can optimize the area, and indirectly power consumption, of the resulting
circuit by minimizing the expected area per
neuron based on the area of the standard
cells in the target standard
cell library. Finally, we also show for the first time an implementation of a DLGN as a silicon
circuit in simulation, performing layout of a DLGN in the SkyWater 130nm process as a custom hard macro using a Cadence standard
cell library and performing post-layout power analysis. We find that our custom macro can perform classification on MNIST with 97% accuracy 41.8 million times a second at a power consumption of 83.88 mW.
Read the full abstract ▾
Computer Vision
PLaMo 2.1-VL Technical Report
Tommi Kerola, Yuya Masuda, Takashi Masuko et al. (+5 more)
We introduce PLaMo 2.1-VL, a lightweight
Vision Language Model (VLM) for autonomous devices, available in 8B and 2B variants and designed for local and edge deployment with Japanese-
language operation. Focusing on Visual Question Answering (VQA) and Visual Grounding as its core capabilities, we develop and evaluate the models for two real-world application scenarios: factory task analysis via tool recognition, and infrastructure anomaly detection. We also develop a large-scale synthetic data generation pipeline and comprehensive Japanese training and evaluation resources. PLaMo 2.1-VL outperforms comparable open models on Japanese and English
benchmarks, achieving 61.5 ROUGE-L on JA-VG-VQA-500 and 85.2% accuracy on Japanese Ref-L4. For the two application scenarios, it achieves 53.9% zero-shot accuracy on factory task analysis, and fine-tuning on power plant data improves anomaly detection bbox + label F1-score from 39.7 to 64.9.
Read the full abstract ▾
All authors Tommi Kerola, Yuya Masuda, Takashi Masuko, Toshiki Nakanishi, Daisuke Nishino, Kuniyuki Takahashi, Hanqin Wang, Yoshihiro Yamada
Computer Vision · Cross-listed from cs.LG
Concept Inconsistency in Dermoscopic Concept Bottleneck Models: A Rough-Set Analysis of the Derm7pt Dataset
Gonzalo Nápoles, Isel Grau and Yamisleydi Salgueiro
Concept Bottleneck Models (CBMs) route predictions exclusively through a clinically grounded concept layer, binding interpretability to concept-label consistency. When a
dataset contains concept-level inconsistencies, identical concept profiles mapped to conflicting diagnosis labels create an unresolvable bottleneck that imposes a hard ceiling on achievable accuracy. In this paper, we apply rough set theory to the Derm7pt dermoscopy
benchmark and characterize the full extent and clinical structure of this inconsistency. Among 305 unique concept profiles formed by the 7 dermoscopic criteria of the 7-point melanoma checklist, 50 (16.4%) are inconsistent, spanning 306 images (30.3% of the
dataset). This yields a theoretical accuracy ceiling of 92.1%, independent of backbone architecture or training strategy for CBMs that exclusively operate with hard concepts. In addition, we characterize the conflict-severity distribution, identify the clinical features most responsible for boundary ambiguity, and evaluate two filtering strategies with quantified effects on
dataset composition and CBM interpretability. Symmetric removal of all boundary-region images yields Derm7pt+, a fully consistent
benchmark subset of 705 images with perfect quality of classification and no hard accuracy ceiling. Building on this filtered
dataset, we present a hard CBM evaluated across 19 backbone architectures from the EfficientNet, DenseNet, ResNet, and Wide ResNet families. Under symmetric filtering, explored for completeness, EfficientNet-B5 achieves the best label F1 score (0.85) and label accuracy (0.90) on the held-out test set, with a concept accuracy of 0.70. Under asymmetric filtering, EfficientNet-B7 leads across all four metrics, reaching a label F1 score of 0.82 and concept accuracy of 0.70. These results establish reproducible baselines for concept-consistent CBM evaluation on dermoscopic data.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models
Yusuf Çelebi, Yağ{\i}z Asker, Özay Ezerceli et al. (+4 more)
Fine-tuning Large
Language Models (LLMs) remains structurally uncertain despite parameter-efficient methods such as Low-Rank Adaptation (LoRA), as the layer-specific roles of internal representations are poorly understood, leading to heuristic decisions about where adaptation should be applied. We model the
evolution of hidden states as a high-dimensional
geometric trajectory and propose using the Ramer-Douglas-Peucker (RDP)
algorithm, a parameter-free and training-free polygon simplification method that preserves global structural transitions while eliminating locally redundant changes, to identify critical breakpoints along the representation path. Crucially, we use these
geometric pivots not merely for analysis, but as a direct decision signal for determining which layers should be adapted during parameter-efficient fine-tuning. By integrating this geometry-aware layer selection strategy into LoRA fine-tuning of Qwen3-8B-Base, we achieve superior performance on MMLU-Math using only 13 RDP-selected layers (81.67%), significantly outperforming both full 36-layer adaptation (79.32%) and random 13-layer selection (75.56%), as well as the baseline Qwen3-8B-Base model (74.25%). These results demonstrate that leveraging the intrinsic geometry of representation trajectories provides a robust, interpretable, and training-free signal for optimizing layer selection during model adaptation.
Read the full abstract ▾
All authors Yusuf Çelebi, Yağ{\i}z Asker, Özay Ezerceli, Mahmoud ElHussieni, Selva Taş, Reyhan Bayraktar, Fatma Betül Terzioğlu
Computer Vision
Multi-view Crowd Tracking Transformer with View-Ground Interactions Under Large Real-World Scenes
Qi Zhang, Jixuan Chen, Kaiyi Zhang et al. (+3 more)
Multi-view crowd tracking estimates each person's tracking trajectories on the ground of the scene. Recent research works mainly rely on CNNs-based multi-view crowd tracking architectures, and most of them are evaluated and compared on relatively small
datasets, such as Wildtrack and MultiviewX. Since these two
datasets are collected in small scenes and only contain tens of frames in the evaluation stage, it is difficult for the current methods to be applied to real-world applications where scene size and occlusion are more complicated. In this paper, we propose a
Transformer-based multi-view crowd tracking model, MVTrackTrans, which adopts interactions between camera views and the ground plane for enhanced multi-view tracking performance. Besides, for better evaluation, we collect and label two large real-world multi-view tracking
datasets, MVCrowdTrack and CityTrack, which contain a much larger scene size over a longer time period. Compared with existing methods on the two large and new
datasets, the proposed MVTrackTrans model achieves better performance, demonstrating the advantages of the model design in dealing with large scenes. We believe the proposed
datasets and model will push the frontiers of the task to more practical scenarios, and the
datasets and code are available at: https://github.com/zqyq/MVTrackTrans.
Read the full abstract ▾
All authors Qi Zhang, Jixuan Chen, Kaiyi Zhang, Xinquan Yu, Antoni B. Chan, Hui Huang
Computer Vision
Framelet-Based Blind Image Restoration with Minimax Concave Regularization
Heng Zhang, Reza Parvaz and Rui Yang
Recovering corrupted images is one of the most challenging problems in image processing. Among various restoration tasks, blind image deblurring has been extensively studied due to its practical importance and inherent difficulty. In this problem, both the point spread function (PSF) and the underlying latent sharp image must be estimated simultaneously. This problem cannot be solved directly due to its ill-posed nature. One powerful tool for solving such problems is total variation (TV) regularization. The $\ell_0$-norm regularization within the TV framework has been widely adopted to promote sparsity in image gradients or transform domains, leading to improved preservation of edges and fine structures. However, the use of the $\ell_0$-norm results in a highly nonconvex and computationally intractable optimization problem, which limits its practical applicability. To overcome these difficulties, we employ the minimax concave penalty (MCP), which promotes enhanced sparsity and provides a closer approximation to the $\ell_0$-norm. In addition, a reweighted $\ell_1$-norm regularization is incorporated to further reduce estimation bias and improve the preservation of fine image details and textures. After introducing the proposed model, a numerical
algorithm is developed to solve the resulting optimization problem. The effectiveness of the proposed approach is then demonstrated through experimental evaluations on several test images.
Read the full abstract ▾
Computer Vision
Shengqin Wang, Wentao Yan, Huichi Zhou et al. (+4 more)
Agentic
multimodal models have garnered significant
attention for their ability to leverage external tools to tackle complex tasks. However, it is observed that such
agents often meet premature interaction collapse, caused by two primary reasons: 1) the terminal reward often appending on the last
token prevents the advantage from distinguishing trajectories with exploratory behavior; 2) excessively redundant context hinders the
agent from absorbing useful feedback. To address these issues, we propose the Deepening
Reasoning MMSearchAgent, the framework leverages the structural proximity to derive advantage signals from the whole rollout trajectories in an entire batch, such that trajectories of different lengths are further encouraged to be generated, even when containing the same correct answer. Additionally, differentiated gaussian rewards are employed to dynamically calibrate interaction tolerance, thereby ensuring information reliability and reduce redundancy. To support multi-turn interaction training, we have constructed a multi-step deep-
reasoning dataset including 3602 high-quality QA pair with at least 3 reasonning steps. Extensive experiments demonstrate that our method achieves state-of-the-art performance, outperforming the MMSearch-R1 by 8.4$\%$ on FVQA-test.
Read the full abstract ▾
All authors Shengqin Wang, Wentao Yan, Huichi Zhou, Yihang Chen, Kun Shao, Zhizhong Zhang, Yuan Xie
Computer Vision
Feature Perturbation Pool-based Fusion Network for Unified Multi-Class Industrial Defect Detection
Yuanchan Xu, Wenjun Zang and Ying Wu
Multi-class defect detection constitutes a critical yet challenging task in industrial quality inspection, where existing approaches typically suffer from two fundamental limitations: (i) the necessity of training separate models for each defect category, resulting in substantial computational and
memory overhead, and (ii) degraded robustness caused by inter-class feature perturbation when heterogeneous defect categories are jointly modeled. In this paper, we present FPFNet, a Feature Perturbation Pool-based Fusion Network that synergistically integrates a
stochastic feature perturbation pool with a multi-layer feature fusion strategy to address these challenges within a unified detection framework. The feature perturbation pool enriches the training distribution by randomly injecting diverse noise patterns – including Gaussian noise, F-Noise, and F-Drop – into the extracted feature representations, thereby strengthening the model's robustness against domain shifts and unseen defect morphologies. Concurrently, the multi-layer feature fusion module aggregates hierarchical feature representations from both the encoder and decoder through residual connections and normalization, enabling the network to capture complex cross-scale relationships while preserving fine-grained spatial details essential for precise defect localization. Built upon the UniAD architecture~\cite{you2022unified}, our method achieves state-of-the-art performance on two widely adopted
benchmarks: 97.17% image-level AUROC and 96.93% pixel-level AUROC on MVTec-AD, and 91.08% image-level AUROC and 99.08% pixel-level AUROC on VisA, surpassing existing methods by notable margins while introducing no additional learnable parameters or computational complexity.
Read the full abstract ▾
Computer Vision
Unposed-to-3D: Learning Simulation-Ready Vehicles from Real-World Images
Hongyuan Liu, Bochao Zou, Qiankun Liu et al. (+10 more)
Creating realistic and simulation-ready 3D assets is crucial for autonomous driving research and virtual environment construction. However, existing 3D vehicle generation methods are often trained on synthetic data with significant domain gaps from real-world distributions. The generated models often exhibit arbitrary poses and undefined scales, resulting in poor visual consistency when integrated into driving scenes. In this paper, we present Unposed-to-3D, a novel framework that learns to reconstruct 3D vehicles from real-world driving images using image-only supervision. Our approach consists of two stages. In the first stage, we train an image-to-3D reconstruction network using posed images with known camera parameters. In the second stage, we remove camera supervision and use a camera prediction head that directly estimates the camera parameters from unposed images. The predicted pose is then used for differentiable rendering to provide self-supervised photometric feedback, enabling the model to learn 3D geometry purely from unposed images. To ensure simulation readiness, we further introduce a scale-aware module to predict real-world size information, and a harmonization module that adapts the generated vehicles to the target driving scene with consistent lighting and appearance. Extensive experiments demonstrate that Unposed-to-3D effectively reconstructs realistic, pose-consistent, and harmonized 3D vehicle models from real-world images, providing a scalable path toward creating high-quality assets for driving scene simulation and digital twin environments.
Read the full abstract ▾
All authors Hongyuan Liu, Bochao Zou, Qiankun Liu, Haochen Yu, Qi Mei, Jianfei Jiang, Chen Liu, Cheng Bi, Zhao Wang, Xueyang Zhang, Yifei Zhan, Jiansheng Chen, Huimin Ma
Computer Vision
Allo{SR}$^2$: Rectifying One-Step Super-Resolution to Stay Real via Allomorphic Generative Flows
Zihan Wang, Xudong Huang, Junbo Qiao et al. (+4 more)
Real-world image super-resolution (Real-SR) has been revolutionized by leveraging the powerful generative priors from
Diffusion Models (DMs) and Flow Matching (FM). However, existing one-step methods typically replace Gaussian noise with degraded low-resolution (LR) latents at initialization, introducing a substantial distribution shift that further leads to trajectory deviation and prior collapse under extreme acceleration. To overcome these limitations, we propose Allo{SR}$^2$, a novel FM-based framework that rectifies one-step SR flows via allomorphic generative flows to maintain high-fidelity generative realism. Specifically, we utilize SNR-Guided Trajectory Initialization to identify a statistically aligned intermediate state along the pre-trained path to integrate LR representations into the generative flow. To ensure a stable, low-
curvature path for one-step inference, we propose Flow-Anchored Trajectory Consistency (FATC), which explicitly regularizes the velocity field of the underlying probability flow. Furthermore, we develop Allomorphic Trajectory Matching (ATM), a self-
adversarial distillation strategy that jointly models the SR flow and the generative flow within a unified velocity field, enabling one-step Real-SR while preserving the generative prior. Extensive experiments on both synthetic and real-world
benchmarks demonstrate that Allo{SR}$^2$ achieves state-of-the-art performance in one-step Real-SR, offering a superior balance between fidelity and realism while maintaining extreme efficiency.
Read the full abstract ▾
All authors Zihan Wang, Xudong Huang, Junbo Qiao, Wei Li, Jie Hu, Xinghao Chen, Shaohui Lin
Computer Vision
Learning to Credit the Right Steps: Objective-aware Process Optimization for Visual Generation
Rui Li, Ke Hao, Yuanzhi Liang et al. (+4 more)
Reinforcement learning, particularly Group Relative Policy Optimization (GRPO), has emerged as an effective framework for post-training visual generative models with human preference signals. However, its effectiveness is fundamentally limited by coarse reward credit assignment. In modern visual generation, multiple reward models are often used to capture heterogeneous objectives, such as visual quality, motion consistency, and text
alignment. Existing GRPO pipelines typically collapse these rewards into a single static scalar and propagate it uniformly across the entire
diffusion trajectory. This design ignores the stage-specific roles of different denoising steps and produces mistimed or incompatible optimization signals. To address this issue, we propose Objective-aware Trajectory Credit Assignment (OTCA), a structured framework for fine-grained GRPO training. OTCA consists of two key components. Trajectory-Level Credit Decomposition estimates the relative importance of different denoising steps. Multi-Objective Credit Allocation adaptively weights and combines multiple reward signals throughout the denoising process. By jointly modeling temporal credit and objective-level credit, OTCA converts coarse reward supervision into a structured, timestep-aware training signal that better matches the iterative nature of
diffusion-based generation. Extensive experiments show that OTCA consistently improves both image and
video generation quality across evaluation metrics.
Read the full abstract ▾
All authors Rui Li, Ke Hao, Yuanzhi Liang, Haibin Huang, Chi Zhang, Yun Gu, Xuelong Li
Computer Vision
Adaptive Slicing-Assisted Hyper Inference for Enhanced Small Object Detection in High-Resolution Imagery
Francesco Moretti, Yi Jin and Guiqin Mario
Deep learning-based object detectors have achieved remarkable success across numerous computer
vision applications, yet they continue to struggle with small object detection in high-resolution aerial and satellite imagery, where dense object distributions, variable shooting angles, diminutive target sizes, and substantial inter-class variability pose formidable challenges. Existing slicing strategies that partition high-resolution images into manageable patches have demonstrated promising results for enlarging the effective receptive field of small targets; however, their reliance on fixed slice dimensions introduces significant redundant computation, inflating inference cost and undermining detection speed. In this paper, we propose Adaptive Slicing-Assisted Hyper Inference (ASAHI), a novel slicing framework that shifts the paradigm from prescribing a fixed slice size to adaptively determining the optimal number of slices according to image resolution, thereby substantially mitigating redundant computation while preserving beneficial overlap between adjacent patches. ASAHI integrates three synergistic components: (1)an adaptive resolution-aware slicing
algorithm that dynamically generates 6 or 12 overlapping patches based on a learned threshold, (2)a slicing-assisted fine-tuning (SAF) strategy that constructs augmented training data comprising both full-resolution and sliced image patches, and (3)a Cluster-DIoU-NMS (CDN) post-processing module that combines the
geometric merging efficiency of Cluster-NMS with the center-distance-aware suppression of DIoU-NMS to achieve robust duplicate elimination in crowded scenes. Extensive experiments on VisDrone2019 and xView, demonstrate that ASAHI achieves state-of-the-art performance with 56.8% on VisDrone2019-DET-val and 22.7% on xView-test, while reducing inference time by 20-25% compared to the baseline SAHI method.
Read the full abstract ▾
Computer Vision
Thinking Before Matching: A Reinforcement Reasoning Paradigm Towards General Person Re-Identification
Quan Zhang, Jingze Wu, Jialong Wang et al. (+3 more)
Learning identity-discriminative representations with multi-scene generality has become a critical objective in person re-identification (ReID). However, mainstream perception-driven paradigms tend to identify fitting from massive annotated data rather than identity-causal cues understanding, which presents a fragile representation against multiple disruptions. In this work, ReID-R is proposed as a novel
reasoning-driven paradigm that achieves explicit identity understanding and
reasoning by incorporating chain-of-thought into the ReID pipeline. Specifically, ReID-R consists of a two-stage contribution: (i) Discriminative
reasoning warm-up, where a model is trained in a CoT label-free manner to acquire identity-aware feature understanding; and (ii) Efficient reinforcement learning, which proposes a non-trivial sampling to construct scene-generalizable data. On this basis, ReID-R leverages high-quality reward signals to guide the model toward focusing on ID-related cues, achieving accurate
reasoning and correct responses. Extensive experiments on multiple ReID
benchmarks demonstrate that ReID-R achieves competitive identity discrimination as superior methods using only 14.3K non-trivial data (20.9% of the existing data scale). Furthermore, benefit from inherent
reasoning, ReID-R can provide high-quality interpretation for results.
Read the full abstract ▾
All authors Quan Zhang, Jingze Wu, Jialong Wang, Xiaohua Xie, Jianhuang Lai, Hongbo Chen
Computer Vision
Attention-based Multi-modal Deep Learning Model of Spatio-temporal Crop Yield Prediction with Satellite, Soil and Climate Data
Gopal Krishna Shyam and Ila Chandrakar
Crop yield prediction is one of the most important challenge, which is crucial to world food security and policy-making decisions. The conventional
forecasting techniques are limited in their accuracy with reference to the fact that they utilize static data sources that do not reflect the dynamic and intricate relationships that exist between the variables of the environment over time [5,13]. This paper presents
Attention-Based Multi-Modal Deep Learning Framework (ABMMDLF), which is suggested to be used in high-accuracy spatio-temporal crop yield prediction. The model we use combines multi-year satellite imagery, high-resolution time-series of meteorological data and initial soil properties as opposed to the traditional models which use only one of the aforementioned factors [12, 21]. The main architecture involves the use of Convolutional Neural Networks (CNN) to extract spatial features and a Temporal
Attention Mechanism to adaptively weight important phenological periods targeted by the
algorithm to change over time and condition on spatial features of images and
video sequences. As can be experimentally seen, the proposed research work provides an R^2 score of 0.89, which is far better than the baseline models do.
Read the full abstract ▾
Computer Vision
An Object-Centered Data Acquisition Method for 3D Gaussian Splatting using Mobile Phones
Yuezhe Zhang, Luqian Bai, Mengting Yu et al. (+3 more)
Data acquisition through mobile phones remains a challenge for 3D Gaussian Splatting (3DGS). In this work we target the object-centered scenario and enable reliable mobile acquisition by providing on-device capture guidance and recording onboard sensor signals for offline reconstruction. After the calibration step, the device orientations are aligned to a baseline frame to obtain relative poses, and the optical axis of the camera is mapped to an object-centered spherical grid for uniform viewpoint indexing. To curb polar sampling bias, we compute area-weighted spherical coverage in real-time and guide the user's motion accordingly. We compare the proposed method with RealityScan and the free-capture strategy. Our method achieves superior reconstruction quality using fewer input images compared to free capture and RealityScan. Further analysis shows that the proposed method is able to obtain more comprehensive and uniform viewpoint coverage during object-centered acquisition.
Read the full abstract ▾
All authors Yuezhe Zhang, Luqian Bai, Mengting Yu, Lei Wei, Shuai Wan, Yifan Zhang
Computer Vision
When Can We Trust Deep Neural Networks? Towards Reliable Industrial Deployment with an Interpretability Guide
Hang-Cheng Dong, Yuhao Jiang, Yibo Jiao et al. (+5 more)
The deployment of AI systems in safety-critical domains, such as industrial defect inspection, autonomous driving, and medical diagnosis, is severely hampered by their lack of reliability. A single undetected erroneous prediction can lead to catastrophic outcomes. Unfortunately, there is often no alternative but to place trust in the outputs of a trained AI system, which operates without an internal safeguard to flag unreliable predictions, even in cases of high accuracy. We propose a post-hoc explanation-based indicator to detect false negatives in binary defect detection networks. To our knowledge, this is the first method to proactively identify potentially erroneous network outputs. Our core idea leverages the difference between class-specific discriminative heatmaps and class-agnostic ones. We compute the difference in their intersection over union (IoU) as a reliability score. An
adversarial enhancement method is further introduced to amplify this disparity. Evaluations on two industrial defect detection
benchmarks show our method effectively identifies false negatives. With
adversarial enhancement, it achieves 100% recall, albeit with a trade-off for true negatives. Our work thus advocates for a new and trustworthy deployment paradigm: data-model-explanation-output, moving beyond conventional end-to-end systems to provide critical support for reliable AI in real-world applications.
Read the full abstract ▾
All authors Hang-Cheng Dong, Yuhao Jiang, Yibo Jiao, Lu Zou, Kai Zheng, Bingguo Liu, Dong Ye, Guodong Liu
Computer Vision · Cross-listed from cs.GR
SketchFaceGS: Real-Time Sketch-Driven Face Editing and Generation with Gaussian Splatting
Bo Li, Jiahao Kang, Yubo Ma et al. (+4 more)
3D Gaussian representations have emerged as a powerful paradigm for digital head modeling, achieving photorealistic quality with real-time rendering. However, intuitive and interactive creation or editing of 3D Gaussian head models remains challenging. Although 2D sketches provide an ideal interaction modality for fast, intuitive conceptual design, they are sparse, depth-ambiguous, and lack high-frequency appearance cues, making it difficult to infer dense, geometrically consistent 3D Gaussian structures from strokes - especially under real-time constraints. To address these challenges, we propose SketchFaceGS, the first sketch-driven framework for real-time generation and editing of photorealistic 3D Gaussian head models from 2D sketches. Our method uses a feed-forward, coarse-to-fine architecture. A
Transformer-based UV feature-prediction module first reconstructs a coarse but geometrically consistent UV feature map from the input sketch, and then a 3D UV feature enhancement module refines it with high-frequency, photorealistic detail to produce a high-fidelity 3D head. For editing, we introduce a UV Mask Fusion technique combined with a layer-by-layer feature-fusion strategy, enabling precise, real-time, free-viewpoint modifications. Extensive experiments show that SketchFaceGS outperforms existing methods in both generation fidelity and editing flexibility, producing high-quality, editable 3D heads from sketches in a single forward pass.
Read the full abstract ▾
All authors Bo Li, Jiahao Kang, Yubo Ma, Feng-Lin Liu, Bin Liu, Fang-Lue Zhang, Lin Gao
Computer Vision
Benchmarking Vision Foundation Models for Domain-Generalizable Face Anti-Spoofing
Mika Feng, Pierre Gallin-Martel, Koichi Ito et al. (+1 more)
Face Anti-Spoofing (FAS) remains challenging due to the requirement for robust domain generalization across unseen environments. While recent trends leverage
Vision-
Language Models (VLMs) for semantic supervision, these
multimodal approaches often demand prohibitive computational resources and exhibit high inference latency. Furthermore, their efficacy is inherently limited by the quality of the underlying visual features. This paper revisits the potential of
vision-only foundation models to establish a highly efficient and robust baseline for FAS. We conduct a systematic benchmarking of 15 pre-trained models, such as supervised CNNs, supervised ViTs, and self-supervised ViTs, under severe cross-domain scenarios including the MICO and Limited Source Domains (LSD) protocols. Our comprehensive analysis reveals that self-supervised
vision models, particularly DINOv2 with Registers, significantly suppress
attention artifacts and capture critical, fine-grained spoofing cues. Combined with Face Anti-Spoofing Data Augmentation (FAS-Aug), Patch-wise Data Augmentation (PDA) and
Attention-weighted Patch Loss (APL), our proposed
vision-only baseline achieves state-of-the-art performance in the MICO protocol. This baseline outperforms existing methods under the data-constrained LSD protocol while maintaining superior computational efficiency. This work provides a definitive
vision-only baseline for FAS, demonstrating that optimized self-supervised
vision transformers can serve as a backbone for both
vision-only and future
multimodal FAS systems. The project page is available at: https://gsisaoki.github.io/FAS-VFMbenchmark-CVPRW2026/ .
Read the full abstract ▾
All authors Mika Feng, Pierre Gallin-Martel, Koichi Ito, Takafumi Aoki
Computer Vision
Xiaotian Zhang, Jianhui Wei, Yuan Wang et al. (+9 more)
Despite remarkable progress toward general-purpose
video models, a critical question remains unanswered: how far are these models from achieving true
multimodal reasoning? Existing
benchmarks fail to address this question rigorously, as they remain constrained by straightforward task designs and fragmented evaluation metrics that neglect complex
multimodal reasoning. To bridge this gap, we introduce CLVG-Bench, an evaluation framework designed to probe
video models' zero-shot
reasoning capabilities via Context Learning in
Video Generation. CLVG-Bench comprises more than 1,000 high-quality, manually annotated metadata across 6 categories and 47 subcategories, covering complex scenarios including physical simulation, logical
reasoning, and interactive contexts. To enable rigorous and scalable assessment, we further propose an Adaptive
Video Evaluator (AVE) that aligns with human expert perception using minimal annotations, delivering interpretable textual feedback across diverse
video context tasks. Extensive experiments reveal a striking answer to our central question: while state-of-the-art (SOTA)
video models, such as Seedance 2.0, demonstrate competence on certain understanding and
reasoning subtasks, they fall substantially short with logically grounded and interactive generation tasks (achieving success rates <25% and ~0%, respectively), exposing
multimodal reasoning and physical grounding as critical bottlenecks. By systematically quantifying these limitations, the proposed method provides actionable feedbacks and a clear roadmap toward truly robust, general-purpose
video models. CLVG-Bench and code are released here.
Read the full abstract ▾
All authors Xiaotian Zhang, Jianhui Wei, Yuan Wang, Jie Tan, Yichen Li, Yan Zhang, Ziyi Chen, Daoan Zhang, Dezhi YU, Wei Xu, Songtao Jiang, Zuozhu Liu
Computer Vision
Towards Modality-Agnostic Medical Image Anomaly Detection: A Training-Free Manifold Refinement Approach
Pritam Kar, Gouri Lakshmi S and Saptarshi Bej
Deploying AI-based anomaly detection across diverse clinical imaging settings remains challenging because most existing methods rely on modality-specific architectures, anatomical priors, or extensive retraining, limiting their use as general-purpose screening tools. One-class classification (OCC) offers a label-efficient alternative by training exclusively on normal data, but conventional two-stage pipelines fit a density
estimator directly on raw pretrained
embeddings, leaving substantial discriminative structure in the latent space unexploited. We introduce a training-free, modality-agnostic framework that inserts an explicit
manifold-refinement stage between feature extraction and anomaly scoring. Empirical density weights, estimated via a UMAP-derived neighborhood graph, guide an iterative shift of
embeddings toward locally dense regions, compacting normal samples, leaving anomalies relatively isolated prior to Gaussian density estimation and Mahalanobis-based scoring. This refinement introduces no additional trainable parameters and no architectural modification, allowing it to be layered onto any pretrained encoder. Evaluated on the MedIAnomaly
benchmark across seven
datasets spanning five imaging modalities (X-ray, MRI, fundus, dermatoscopy, histopathology), the framework achieves the best AUC on four
datasets and the best Average Precision on five
datasets among methods evaluated in the
benchmark, outperforming specialized reconstruction and
diffusion-based methods with a single fixed hyperparameter configuration across all modalities. These results demonstrate that meaningful gains can be achieved through post-hoc
geometric refinement of existing representations rather than bespoke encoders, offering a practical and scalable AI screening framework for real-world, multi-modality clinical workflows where retraining and abnormal-case annotation are costly or infeasible.
Read the full abstract ▾
Computer Vision
MSDS: Deep Structural Similarity with Multiscale Representation
Danling Kang, Xue-Hua Chen, Bin Liu et al. (+3 more)
Deep-feature-based perceptual similarity models have demonstrated strong
alignment with human visual perception in Image Quality Assessment (IQA). However, most existing approaches operate at a single spatial scale, implicitly assuming that structural similarity at a fixed resolution is sufficient. The role of spatial scale in deep-feature similarity modeling thus remains insufficiently understood. In this letter, we isolate spatial scale as an independent factor using a minimal multiscale extension of DeepSSIM, referred to as Deep Structural Similarity with Multiscale Representation (MSDS). The proposed framework decouples deep feature representation from cross-scale integration by computing DeepSSIM independently across pyramid levels and fusing the resulting scores with a lightweight set of learnable global weights. Experiments on multiple
benchmark datasets demonstrate consistent and statistically significant improvements over the single-scale baseline, while introducing negligible additional complexity. The results empirically confirm spatial scale as a non-negligible factor in deep perceptual similarity, isolated here via a minimal testbed.
Read the full abstract ▾
All authors Danling Kang, Xue-Hua Chen, Bin Liu, Keke Zhang, Weiling Chen, Tiesong Zhao
Computer Vision
ST-Prune: Training-Free Spatio-Temporal Token Pruning for Vision-Language Models in Autonomous Driving
Lin Sha, Haiyun Guo, Tao Wang et al. (+4 more)
Vision-
Language Models (VLMs) have become central to autonomous driving systems, yet their deployment is severely bottlenecked by the massive computational overhead of multi-view camera and multi-frame
video input. Existing
token pruning methods, primarily designed for single-image inputs, treat each frame or view in isolation and thus fail to exploit the inherent spatio-temporal redundancies in driving scenarios. To bridge this gap, we propose ST-Prune, a training-free, plug-and-play framework comprising two complementary modules: Motion-aware Temporal
Pruning (MTP) and Ring-view Spatial
Pruning (RSP). MTP addresses temporal redundancy by encoding motion
volatility and temporal recency as soft constraints within the diversity selection objective, prioritizing dynamic trajectories and current-frame content over static historical background. RSP further resolves spatial redundancy by exploiting the ring-view camera geometry to penalize bilateral cross-view similarity, eliminating duplicate projections and residual background that temporal
pruning alone cannot suppress. These two modules together constitute a complete spatio-temporal
pruning process, preserving key scene information under strict compression. Validated across four
benchmarks spanning perception, prediction, and planning, ST-Prune establishes new state-of-the-art for training-free
token pruning. Notably, even at 90%
token reduction, ST-Prune achieves near-lossless performance with certain metrics surpassing the full-model baseline, while maintaining inference speeds comparable to existing
pruning approaches.
Read the full abstract ▾
All authors Lin Sha, Haiyun Guo, Tao Wang, Cong Zhang, Min Huang, Jinqiao Wang, Qinghai Miao
Computer Vision
Denoising, Fast and Slow: Difficulty-Aware Adaptive Sampling for Image Generation
Johannes Schusterbauer, Ming Gui, Yusong Li et al. (+3 more)
Diffusion- and flow-based models usually allocate compute uniformly across space, updating all patches with the same timestep and number of function evaluations. While convenient, this ignores the heterogeneity of natural images: some regions are easy to denoise, whereas others benefit from more refinement or additional context. Motivated by this, we explore patch-level noise scales for image synthesis. We find that naively varying timesteps across image
tokens performs poorly, as it exposes the model to overly informative training states that do not occur at inference. We therefore introduce a timestep sampler that explicitly controls the maximum patch-level information available during training, and show that moving from global to patch-level timesteps already improves image generation over standard baselines. By further augmenting the model with a lightweight per-patch difficulty head, we enable adaptive samplers that allocate compute dynamically where it is most needed. Combined with noise levels varying over both space and
diffusion time, this yields Patch Forcing (PF), a framework that advances easier regions earlier so they can provide context for harder ones. PF achieves superior results on class-conditional ImageNet, remains orthogonal to representation
alignment and guidance methods, and scales to text-to-image synthesis. Our results suggest that patch-level denoising schedules provide a promising foundation for adaptive image generation.
Read the full abstract ▾
All authors Johannes Schusterbauer, Ming Gui, Yusong Li, Pingchuan Ma, Felix Krause, Björn Ommer
Computer Vision
Diff-SBSR: Learning Multimodal Feature-Enhanced Diffusion Models for Zero-Shot Sketch-Based 3D Shape Retrieval
Hang Cheng, Fanhe Dong and Long Zeng
This paper presents the first exploration of text-to-image
diffusion models for zero-shot sketch-based 3D shape
retrieval (ZS-SBSR). Existing sketch-based 3D shape
retrieval methods struggle in zero-shot settings due to the absence of category supervision and the extreme sparsity of sketch inputs. Our key insight is that large-scale pretrained
diffusion models inherently exhibit open-vocabulary capability and strong shape bias, making them well suited for zero-shot visual
retrieval. We leverage a frozen Stable
Diffusion backbone to extract and aggregate discriminative representations from intermediate U-Net layers for both sketches and rendered 3D views.
Diffusion models struggle with sketches due to their extreme abstraction and sparsity, compounded by a significant domain gap from natural images. To address this limitation without costly retraining, we introduce a
multimodal feature-enhanced strategy that conditions the frozen
diffusion backbone with complementary visual and textual cues from CLIP, explicitly enhancing the ability of semantic context capture and concentrating on sketch contours. Specifically, we inject global and local visual features derived from a pretrained CLIP visual encoder, and incorporate enriched textual guidance by combining learnable soft
prompts with hard textual descriptions generated by BLIP. Furthermore, we employ the Circle-T loss to dynamically strengthen positive-pair attraction once negative samples are sufficiently separated, thereby adapting to sketch noise and enabling more effective sketch-3D
alignment. Extensive experiments on two public
benchmarks demonstrate that our method consistently outperforms state-of-the-art approaches in ZS-SBSR.
Read the full abstract ▾
Computer Vision
BALTIC: A Benchmark and Cross-Domain Strategy for 3D Reconstruction Across Air and Underwater Domains Under Varying Illumination
Michele Grimaldi, David Nakath, Oscar Pizarro et al. (+3 more)
Robust 3D reconstruction across varying environmental conditions remains a critical challenge for robotic perception, particularly when transitioning between air and water. To address this, we introduce BALTIC, a controlled
benchmark designed to systematically evaluate modern 3D reconstruction methods under variations in medium and lighting. The
benchmark comprises 13
datasets spanning two media (air and water) and three lighting conditions (ambient, artificial, and mixed), with additional variations in motion type, scanning pattern, and initialization trajectory, resulting in a diverse set of sequences. Our experimental setup features a custom water tank equipped with a
monocular camera and an HTC Vive tracker, enabling accurate ground-truth pose estimation. We further investigate cross-domain reconstruction by augmenting
underwater image sequences with a small number of in-air views captured under similar lighting conditions. We evaluate Structure-from-Motion reconstruction using COLMAP in terms of both trajectory accuracy and scene geometry, and use these reconstructions as input to Neural Radiance Fields and 3D Gaussian Splatting methods. The resulting models are assessed against ground-truth trajectories and in-air references, while rendered outputs are compared using perceptual and photometric metrics. Additionally, we perform a color restoration analysis to evaluate radiometric consistency across domains. Our results show that under controlled, texture-consistent conditions, Gaussian Splatting with simple preprocessing (e.g., white balance correction) can achieve performance comparable to specialized
underwater methods, although its robustness decreases in more complex and heterogeneous real-world environments
Read the full abstract ▾
All authors Michele Grimaldi, David Nakath, Oscar Pizarro, Jonatan Scharff Willners, Ignacio Carlucho, Yvan R. Petillot
Computer Vision
PortraitDirector: A Hierarchical Disentanglement Framework for Controllable and Real-time Facial Reenactment
Chaonan Ji, Jinwei Qi, Sheng Xu et al. (+2 more)
Existing facial reenactment methods struggle with a trade-off between expressiveness and fine-grained controllability. Holistic facial reenactment models often sacrifice granular control for expressiveness, while methods designed for control may struggle with fidelity and robust disentanglement. Instead of treating facial motion as a monolithic signal, we explore an alternative compositional perspective. In this paper, we introduce PortraitDirector, a novel framework that formulates face reenactment as a hierarchical composition task, achieving high-fidelity and controllable results. We employ a Hierarchical Motion Disentanglement and Composition strategy, deconstructing facial motion into a Spatial Layer for physical movements and a Semantic Layer for emotional content. The Spatial Layer comprises: (i) global head pose, managed via a dedicated representation and injection pathway; (ii) spatially separated local facial expressions, distilled from cropped facial regions and purged of emotional cues via
Emotion-Filtering Module leveraging an information bottleneck. The Semantic Layer contains a derived global
emotion. The disentangled components are then recomposed into an expressive motion latent. Furthermore, we engineer the framework for real-time performance through a suite of optimizations, including
diffusion distillation, causal
attention and VAE acceleration. PortraitDirector achieves streaming, high-fidelity, controllable 512 x 512 face reenactment at 20 FPS with a end-to-end 800 ms latency on a single 5090 GPU.
Read the full abstract ▾
All authors Chaonan Ji, Jinwei Qi, Sheng Xu, Peng Zhang, Bang Zhang
Computer Vision · Cross-listed from cs.LG
Robust Continual Unlearning against Knowledge Erosion and Forgetting Reversal
Eun-Ju Park, Youjin Shin and Simon S. Woo
As a means to balance the growth of the AI industry with the need for
privacy protection, machine unlearning plays a crucial role in realizing the “right to be forgotten” in artificial intelligence. This technique enables AI systems to remove the influence of specific data while preserving the rest of the learned knowledge. Although it has been actively studied, most existing unlearning methods assume that unlearning is performed only once. In this work, we evaluate existing unlearning
algorithms in a more realistic scenario where unlearning is conducted repeatedly, and in this setting, we identify two critical phenomena: (1) Knowledge Erosion, where the accuracy on retain data progressively degrades over unlearning phases, and (2) Forgetting Reversal, where previously forgotten samples become recognizable again in later phases. To address these challenges, we propose SAFER (StAbility-preserving Forgetting with Effective Regularization), a continual unlearning framework that maintains representation stability for retain data while enforcing negative logit margins for forget data. Extensive experiments show that SAFER mitigates not only knowledge erosion but also forgetting reversal, achieving stable performance across multiple unlearning phases.
Read the full abstract ▾
Computer Vision
Ruibing Hou, Mingyue Zhou, Yuwei Gui et al. (+5 more)
Faithfully modeling human behavior in dynamic environments is a foundational challenge for
embodied intelligence. While conditional motion synthesis has achieved significant advances,
egocentric motion generation remains largely underexplored due to the inherent complexity of first-person perception. In this work, we investigate
Egocentric Vision-
Language (Ego-VL) motion generation. This task requires synthesizing 3D human motion conditioned jointly on first-person visual observations and natural
language instructions. We identify a critical
reasoning-generation
entanglement challenge: the simultaneous optimization of semantic
reasoning and kinematic modeling introduces gradient conflicts. These conflicts systematically degrade the fidelity of
multimodal grounding and motion quality. To address this challenge, we propose a hierarchical generative framework EgoMotion. Inspired by the biological decoupling of cognitive
reasoning and motor control, EgoMotion operates in two stages. In the Cognitive
Reasoning stage, A
vision-
language model (VLM) projects
multimodal inputs into a structured space of discrete motion primitives. This forces the VLM to acquire goal-consistent representations, effectively bridging the semantic gap between high-level perceptual understanding and low-level action execution. In the Motion Generation stage, these learned representations serve as expressive conditioning signals for a
diffusion-based motion generator. By performing iterative denoising within a continuous latent space, the generator synthesizes physically plausible and temporally coherent trajectories. Extensive evaluations demonstrate that EgoMotion achieves state-of-the-art performance, and produces motion sequences that are both semantically grounded and kinematically superior to existing approaches.
Read the full abstract ▾
All authors Ruibing Hou, Mingyue Zhou, Yuwei Gui, Mingshuang Luo, Bingpeng Ma, Hong Chang, Shiguang Shan, Xilin Chen