Computer Vision
Yanpeng Zhao, Wentao Ding, Hongtao Li et al. (+2 more)
A recent trend in
vision-
language models (VLMs) has been to enhance their spatial cognition for
embodied domains. Despite progress, existing evaluations have been limited both in paradigm and in coverage, hindering rapid, iterative model development. To address these limitations, we propose ESPIRE, a diagnostic
benchmark for
embodied spatial
reasoning. ESPIRE offers a simulated world that physically grounds VLMs and evaluates them on spatial-
reasoning-centric robotic tasks, thus narrowing the gap between evaluation and real-world deployment. To adapt VLMs to robotic tasks, we decompose each task into localization and execution, and frame both as generative problems, in stark contrast to predominant discriminative evaluations (e.g., via visual-question answering) that rely on distractors and discard execution. This decomposition further enables a fine-grained analysis beyond passive spatial
reasoning toward
reasoning to act. We systematically design ESPIRE both at the instruction level and at the environment level, ensuring broad coverage of spatial
reasoning scenarios. We use ESPIRE to diagnose a range of frontier VLMs and provide in-depth analysis of their spatial
reasoning behaviors.
Read the full abstract ▾
All authors Yanpeng Zhao, Wentao Ding, Hongtao Li, Baoxiong Jia, Zilong Zheng
Computer Vision
Multimodal OCR: Parse Anything from Documents
Handong Zheng, Yumeng Li, Kaile Zhang et al. (+22 more)
We present
Multimodal OCR (MOCR), a document parsing paradigm that jointly parses text and graphics into unified textual representations. Unlike conventional OCR systems that focus on text recognition and leave graphical regions as cropped pixels, our method, termed dots.mocr, treats visual elements such as charts, diagrams, tables, and icons as first-class parsing targets, enabling systems to parse documents while preserving semantic relationships across elements. It offers several advantages: (1) it reconstructs both text and graphics as structured outputs, enabling more faithful document reconstruction; (2) it supports end-to-end training over heterogeneous document elements, allowing models to exploit semantic relations between textual and visual components; and (3) it converts previously discarded graphics into reusable code-level supervision, unlocking
multimodal supervision embedded in existing documents. To make this paradigm practical at scale, we build a comprehensive data engine from PDFs, rendered webpages, and native SVG assets, and train a compact 3B-parameter model through staged pretraining and supervised fine-tuning. We evaluate dots.mocr from two perspectives: document parsing and structured graphics parsing. On document parsing
benchmarks, it ranks second only to Gemini 3 Pro on our OCR Arena Elo leaderboard, surpasses existing open-source document parsing systems, and sets a new state of the art of 83.9 on olmOCR Bench. On structured graphics parsing, our model achieves higher reconstruction quality than Gemini 3 Pro across image-to-SVG
benchmarks, demonstrating strong performance on charts, UI layouts, scientific figures, and chemical diagrams. These results show a scalable path toward building large-scale image-to-code corpora for
multimodal pretraining. Code and models are publicly available at https://github.com/rednote-hilab/dots.mocr.
Read the full abstract ▾
All authors Handong Zheng, Yumeng Li, Kaile Zhang, Liang Xin, Guangwei Zhao, Hao Liu, Jiayu Chen, Jie Lou, Qi Fu, Rui Yang, Shuo Jiang, Weijian Luo, Weijie Su, Weijun Zhang, Xingyu Zhu, Yabin Li, Yiwei ma, Yu Chen, Yuqiu Ji, Zhaohui Yu, Guang Yang, Colin Zhang, Lei Zhang, Yuliang Liu, Xiang Bai
Computer Vision
SortScrews: A Dataset and Baseline for Real-time Screw Classification
Tianhao Fu, Bingxuan Yang, Juncheng Guo et al. (+2 more)
Automatic identification of screw types is important for industrial automation, robotics, and inventory management. However, publicly available
datasets for screw classification are scarce, particularly for controlled single-object scenarios commonly encountered in automated sorting systems. In this work, we introduce $\textbf{SortScrews}$, a
dataset for casewise visual classification of screws. The
dataset contains 560 RGB images at $512\times512$ resolution covering six screw types and a background class. Images are captured using a standardized acquisition setup and include mild variations in lighting and camera perspective across four capture settings. To facilitate reproducible research and
dataset expansion, we also provide a reusable data collection script that allows users to easily construct similar
datasets for custom hardware components using inexpensive camera setups. We establish baseline results using transfer learning with EfficientNet-B0 and ResNet-18 classifiers pretrained on ImageNet. In addition, we conduct a well-explored failure analysis. Despite the limited
dataset size, these lightweight models achieve strong classification accuracy, demonstrating that controlled acquisition conditions enable effective learning even with relatively small
datasets. The
dataset, collection pipeline, and baseline training code are publicly available at https://github.com/ATATC/SortScrews.
Read the full abstract ▾
All authors Tianhao Fu, Bingxuan Yang, Juncheng Guo, Shrena Sribalan, Yucheng Chen
Computer Vision
SAW: Toward a Surgical Action World Model via Controllable and Scalable Video Generation
Sampath Rapuri, Lalithkumar Seenivasan, Dominik Schneider et al. (+9 more)
A
surgical world model capable of generating realistic
surgical action
videos with precise control over tool-tissue interactions can address fundamental challenges in
surgical AI and simulation – from data scarcity and rare event synthesis to bridging the sim-to-real gap for
surgical automation. However, current
video generation methods, the very core of such
surgical world models, require expensive annotations or complex structured intermediates as conditioning signals at inference, limiting their scalability. Other approaches exhibit limited temporal consistency across complex laparoscopic scenes and do not possess sufficient realism. We propose
Surgical Action World (SAW) – a step toward
surgical action world modeling through
video diffusion conditioned on four lightweight signals:
language prompts encoding tool-action context, a reference
surgical scene, tissue affordance mask, and 2D tool-tip trajectories. We design a conditional
video diffusion approach that reformulates
video-to-
video diffusion into trajectory-conditioned
surgical action synthesis. The backbone
diffusion model is fine-tuned on a custom-curated
dataset of 12,044 laparoscopic clips with lightweight spatiotemporal conditioning signals, leveraging a depth consistency loss to enforce
geometric plausibility without requiring depth at inference. SAW achieves state-of-the-art temporal consistency (CD-FVD: 199.19 vs. 546.82) and strong visual quality on held-out test data. Furthermore, we demonstrate its downstream utility for (a)
surgical AI, where augmenting rare actions with SAW-generated
videos improves action recognition (clipping F1-score: 20.93% to 43.14%; cutting: 0.00% to 8.33%) on real test data, and (b)
surgical simulation, where rendering tool-tissue interaction
videos from simulator-derived trajectory points toward a visually faithful simulation engine.
Read the full abstract ▾
All authors Sampath Rapuri, Lalithkumar Seenivasan, Dominik Schneider, Roger Soberanis-Mukul, Yufan He, Hao Ding, Jiru Xu, Chenhao Yu, Chenyan Jing, Pengfei Guo, Daguang Xu, Mathias Unberath
Computer Vision · Cross-listed from eess.IV
Accelerating Stroke MRI with Diffusion Probabilistic Models through Large-Scale Pre-training and Target-Specific Fine-Tuning
Yamin Arefeen, Sidharth Kumar, Steven Warach et al. (+2 more)
Purpose: To develop a data-efficient strategy for accelerated MRI reconstruction with
Diffusion Probabilistic Generative Models (DPMs) that enables faster scan times in clinical stroke MRI when only limited fully-sampled data samples are available. Methods: Our simple training strategy, inspired by the foundation model paradigm, first trains a DPM on a large, diverse collection of publicly available
brain MRI data in fastMRI and then fine-tunes on a small
dataset from the target application using carefully selected learning rates and fine-tuning durations. The approach is evaluated on controlled fastMRI experiments and on clinical stroke MRI data with a blinded clinical reader study. Results: DPMs pre-trained on approximately 4000 subjects with non-FLAIR contrasts and fine-tuned on FLAIR data from only 20 target subjects achieve reconstruction performance comparable to models trained with substantially more target-domain FLAIR data across multiple acceleration factors. Experiments reveal that moderate fine-tuning with a reduced learning rate yields improved performance, while insufficient or excessive fine-tuning degrades reconstruction quality. When applied to clinical stroke MRI, a blinded reader study involving two neuroradiologists indicates that images reconstructed using the proposed approach from $2 \times$ accelerated data are non-inferior to standard-of-care in terms of image quality and structural delineation. Conclusion: Large-scale pre-training combined with targeted fine-tuning enables DPM-based MRI reconstruction in data-constrained, accelerated clinical stroke MRI. The proposed approach substantially reduces the need for large application-specific
datasets while maintaining clinically acceptable image quality, supporting the use of foundation-inspired
diffusion models for accelerated MRI in targeted applications.
Read the full abstract ▾
All authors Yamin Arefeen, Sidharth Kumar, Steven Warach, Hamidreza Saber, Jonathan Tamir
Computer Vision
A Closed-Form Solution for Debiasing Vision-Language Models with Utility Guarantees Across Modalities and Tasks
Tangzheng Lian, Guanyu Hu, Yijing Ren et al. (+2 more)
While
Vision-
Language Models (VLMs) have achieved remarkable performance across diverse downstream tasks, recent studies have shown that they can inherit social biases from the training data and further propagate them into downstream applications. To address this issue, various debiasing approaches have been proposed, yet most of them aim to improve
fairness without having a theoretical guarantee that the utility of the model is preserved. In this paper, we introduce a debiasing method that yields a closed-form solution in the cross-modal space, achieving Pareto-optimal
fairness with bounded utility losses. Our method is training-free, requires no annotated data, and can jointly debias both visual and textual modalities across downstream tasks. Extensive experiments show that our method outperforms existing methods in debiasing VLMs across diverse
fairness metrics and
datasets for both group and intersectional
fairness in downstream tasks such as zero-shot image classification, text-to-image
retrieval, and text-to-image generation while preserving task performance.
Read the full abstract ▾
All authors Tangzheng Lian, Guanyu Hu, Yijing Ren, Dimitrios Kollias, Oya Celiktutan
Computer Vision · Cross-listed from cs.LG
Deconstructing the Failure of Ideal Noise Correction: A Three-Pillar Diagnosis
Chen Feng, Zhuo Zhi, Zhao Huang et al. (+5 more)
Statistically consistent methods based on the noise transition
matrix ($T$) offer a theoretically grounded solution to Learning with Noisy Labels (LNL), with guarantees of convergence to the optimal clean-data classifier. In practice, however, these methods are often outperformed by empirical approaches such as sample selection, and this gap is usually attributed to the difficulty of accurately estimating $T$. The common assumption is that, given a perfect $T$, noise-correction methods would recover their theoretical advantage. In this work, we put this longstanding hypothesis to a decisive test. We conduct experiments under idealized conditions, providing correction methods with a perfect, oracle transition
matrix. Even under these ideal conditions, we observe that these methods still suffer from performance collapse during training. This compellingly demonstrates that the failure is not fundamentally a $T$-estimation problem, but stems from a more deeply rooted flaw. To explain this behaviour, we provide a unified analysis that links three levels: macroscopic convergence states, microscopic optimisation dynamics, and information-theoretic limits on what can be learned from noisy labels. Together, these results give a formal account of why ideal noise correction fails and offer concrete guidance for designing more reliable methods for learning with noisy labels.
Read the full abstract ▾
All authors Chen Feng, Zhuo Zhi, Zhao Huang, Jiawei Ge, Ling Xiao, Nicu Sebe, Georgios Tzimiropoulos, Ioannis Patras
Computer Vision
Test-Time Attention Purification for Backdoored Large Vision Language Models
Zhifang Zhang, Bojun Yang, Shuo He et al. (+5 more)
Despite the strong
multimodal performance, large
vision-
language models (LVLMs) are vulnerable during fine-tuning to
backdoor attacks, where adversaries insert trigger-embedded samples into the training data to implant behaviors that can be maliciously activated at test time. Existing defenses typically rely on retraining backdoored parameters (e.g., adapters or LoRA modules) with clean data, which is computationally expensive and often degrades model performance. In this work, we provide a new mechanistic understanding of
backdoor behaviors in LVLMs: the trigger does not influence prediction through low-level visual patterns, but through abnormal cross-modal
attention redistribution, where trigger-bearing visual
tokens steal
attention away from the textual context - a phenomenon we term
attention stealing. Motivated by this, we propose CleanSight, a training-free, plug-and-play defense that operates purely at test time. CleanSight (i) detects poisoned inputs based on the relative visual-text
attention ratio in selected cross-modal fusion layers, and (ii) purifies the input by selectively
pruning the suspicious high-
attention visual
tokens to neutralize the
backdoor activation. Extensive experiments show that CleanSight significantly outperforms existing pixel-based purification defenses across diverse
datasets and
backdoor attack types, while preserving the model's utility on both clean and poisoned samples.
Read the full abstract ▾
All authors Zhifang Zhang, Bojun Yang, Shuo He, Weitong Chen, Wei Emma Zhang, Olaf Maennel, Lei Feng, Miao Xu
Computer Vision
Fair Lung Disease Diagnosis from Chest CT via Gender-Adversarial Attention Multiple Instance Learning
Aditya Parikh and Aasa Feragen
We present a
fairness-aware framework for multi-class lung disease diagnosis from chest CT volumes, developed for the Fair Disease Diagnosis Challenge at the PHAROS-AIF-MIH Workshop (CVPR 2026). The challenge requires classifying CT scans into four categories – Healthy, COVID-19, Adenocarcinoma, and Squamous
Cell Carcinoma – with performance measured as the average of per-gender macro F1 scores, explicitly penalizing gender-inequitable predictions. Our approach addresses two core difficulties: the sparse pathological signal across hundreds of slices, and a severe demographic imbalance compounded across disease class and gender. We propose an
attention-based Multiple Instance Learning (MIL) model on a ConvNeXt backbone that learns to identify diagnostically relevant slices without slice-level supervision, augmented with a Gradient Reversal Layer (GRL) that adversarially suppresses gender-predictive structure in the learned scan representation. Training incorporates focal loss with label smoothing, stratified cross-validation over joint (class, gender) strata, and targeted oversampling of the most underrepresented subgroup. At inference, all five-fold checkpoints are ensembled with horizontal-flip test-time augmentation via soft logit voting and out-of-the-fold threshold optimization for robustness. Our model achieves a mean validation competition score of 0.685 (std - 0.030), with the best single fold reaching 0.759. All training and inference code is publicly available at https://github.com/ADE-17/cvpr-fair-chest-ct
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
SCOPE: Scalable Training-Free Semantic Coreset Selection for Long-Tailed Federated Learning
Md Anwar Hossen, Nathan R. Tallent, Luanzheng Guo et al. (+1 more)
Scientific discovery increasingly relies on
federated learning over data from high-resolution instruments, and these data are strongly non-IID and long-tailed. Coreset selection could shorten training, but existing
federated coreset methods either prune with local heuristics that ignore the global class distribution, or first train a warm-up model at every client and upload class centroids of its features. This makes coreset selection a distributed bottleneck that consumes GPU time at every client, grows with the model and the number of clients, and must be repeated for every architecture. We introduce SCOPE, named for Semantic Coreset using Orthogonal Projection
Embeddings, a scalable, training-free
federated coreset framework for long-tailed data. Each client scores its samples once with a frozen, compact
vision-
language model and uploads scalar statistics per class. The server merges these into a global class profile, which lets each client remove anomalies and prune redundant samples only from classes that are locally abundant but globally common. SCOPE therefore requires no warm-up training, needs a single exchange with the server, and uploads a payload that depends only on the number of classes. On GPU nodes of an HPC cluster, with four
datasets, including scientific micrographs, and six baselines, SCOPE selects its coreset up to 7.7x faster and uploads up to 293x less data than the feature-based baseline. As the number of clients grows from 10 to 500, SCOPE's time before training grows by fewer seconds than any baseline's, and at 500 clients the baselines need 1.7 to 3.6x SCOPE's time. Trained on the coreset, the global model reaches 98% of the full-data accuracy in 18 to 42% fewer global rounds than full-data training, with 8 to 21% less client computation over 200 rounds.
Read the full abstract ▾
All authors Md Anwar Hossen, Nathan R. Tallent, Luanzheng Guo, Ali Jannesary
Computer Vision · Cross-listed from eess.IV
Reinforcing the Weakest Links: Modernizing SIENA with Targeted Deep Learning Integration
Riccardo Raciti, Lemuel Puglisi, Francesco Guarnera et al. (+2 more)
Percentage
Brain Volume Change (PBVC) derived from
Magnetic Resonance Imaging (MRI) is a widely used biomarker of
brain atrophy, with SIENA among the most established methods for its estimation. However, SIENA relies on classical image processing steps, particularly skull stripping and tissue
segmentation, whose failures can propagate through the pipeline and bias atrophy estimates. In this work, we examine whether targeted deep learning substitutions can improve SIENA while preserving its established and interpretable framework. To this end, we integrate SynthStrip and SynthSeg into SIENA and evaluate three pipeline variants on the ADNI and PPMI longitudinal cohorts. Performance is assessed using three complementary criteria: correlation with longitudinal clinical and structural decline, scan-order consistency, and end-to-end runtime. Replacing the skull-stripping module yields the most consistent gains: in ADNI, it substantially strengthens associations between PBVC and multiple measures of disease progression relative to the standard SIENA pipeline, while across both
datasets it markedly improves robustness under scan reversal. The fully integrated pipeline achieves the strongest scan-order consistency, reducing the error by up to 99.1%. In addition, GPU-enabled variants reduce execution time by up to 46% while maintaining CPU runtimes comparable to standard SIENA. Overall, these findings show that deep learning can meaningfully strengthen established longitudinal atrophy pipelines when used to reinforce their weakest image processing steps. More broadly, this study highlights the value of modularly modernizing clinically trusted neuroimaging tools without sacrificing their interpretability. Code is publicly available at https://github.com/Raciti/Enhanced-SIENA.git.
Read the full abstract ▾
All authors Riccardo Raciti, Lemuel Puglisi, Francesco Guarnera, Daniele Ravì, Sebastiano Battiato
Computer Vision
Thinking in Streaming Video
Zikang Liu, Longteng Guo, Handong Li et al. (+7 more)
Real-time understanding of continuous
video streams is essential for interactive assistants and
multimodal agents operating in dynamic environments. However, most existing
video reasoning approaches follow a batch paradigm that defers
reasoning until the full
video context is observed, resulting in high latency and growing computational cost that are incompatible with streaming scenarios. In this paper, we introduce ThinkStream, a framework for streaming
video reasoning based on a Watch–Think–Speak paradigm that enables models to incrementally update their understanding as new
video observations arrive. At each step, the model performs a short
reasoning update and decides whether sufficient evidence has accumulated to produce a response. To support long-horizon streaming, we propose
Reasoning-Compressed Streaming
Memory (RCSM), which treats intermediate
reasoning traces as compact semantic
memory that replaces outdated visual
tokens while preserving essential context. We further train the model using a Streaming Reinforcement Learning with Verifiable Rewards scheme that aligns incremental
reasoning and response timing with the requirements of streaming interaction. Experiments on multiple streaming
video benchmarks show that ThinkStream significantly outperforms existing online
video models while maintaining low latency and
memory usage. Code, models and data will be released at https://github.com/johncaged/ThinkStream
Read the full abstract ▾
All authors Zikang Liu, Longteng Guo, Handong Li, Ru Zhen, Xingjian He, Ruyi Ji, Xiaoming Ren, Yanhao Zhang, Haonan Lu, Jing Liu
Computer Vision
SGMatch: Semantic-Guided Non-Rigid Shape Matching with Flow Regularization
Tianwei Ye, Xiaoguang Mei, Yifan Xia et al. (+3 more)
Establishing accurate point-to-point correspondences between non-rigid 3D shapes remains a critical challenge, particularly under non-isometric deformations and topological noise. Existing functional map pipelines suffer from ambiguities that
geometric descriptors alone cannot resolve, and spatial inconsistencies inherent in the projection of truncated spectral bases to dense pointwise correspondences. In this paper, we introduce SGMatch, a learning-based framework that couples 3D-lifted semantic cues with trajectory-level feature transport regularization. Specifically, we design a Semantic-Guided Local Cross-
Attention module that integrates semantic features from
vision foundation models into
geometric descriptors while preserving local structural continuity. Furthermore, we adapt conditional flow matching as a time-conditioned feature transport regularizer that promotes spatially coherent point-wise recovery. Experimental results on multiple
benchmarks demonstrate that SGMatch achieves competitive performance across near-isometric settings and consistent improvements under non-isometric deformations and topological noise.
Read the full abstract ▾
All authors Tianwei Ye, Xiaoguang Mei, Yifan Xia, Fan Fan, Jun Huang, Jiayi Ma
Computer Vision · Cross-listed from cs.RO
MotionAnymesh: Physics-Grounded Articulation for Simulation-Ready Digital Twins
WenBo Xu, Liu Liu, Li Zhang et al. (+2 more)
Converting static 3D meshes into interactable articulated assets is crucial for
embodied AI and robotic simulation. However, existing zero-shot pipelines struggle with complex assets due to a critical lack of physical grounding. Specifically, ungrounded
Vision-
Language Models (VLMs) frequently suffer from kinematic
hallucinations, while unconstrained joint estimation inevitably leads to catastrophic mesh inter-penetration during physical simulation. To bridge this gap, we propose MotionAnymesh, an automated zero-shot framework that seamlessly transforms unstructured static meshes into simulation-ready digital twins. Our method features a kinematic-aware part
segmentation module that grounds VLM
reasoning with explicit SP4D physical priors, effectively eradicating kinematic
hallucinations. Furthermore, we introduce a geometry-physics joint estimation pipeline that combines robust type-aware initialization with physics-constrained trajectory optimization to rigorously guarantee collision-free articulation. Extensive experiments demonstrate that MotionAnymesh significantly outperforms state-of-the-art baselines in both
geometric precision and dynamic physical executability, providing highly reliable assets for downstream applications.
Read the full abstract ▾
All authors WenBo Xu, Liu Liu, Li Zhang, Dan Guo, RuoNan Liu
Computer Vision
Rethinking VLMs for Image Forgery Detection and Localization
Shaofeng Guo, Jiequan Cui and Richang Hong
With the rapid rise of Artificial Intelligence Generated Content (AIGC), image manipulation has become increasingly accessible, posing significant challenges for image forgery detection and localization (IFDL). In this paper, we study how to fully leverage
vision-
language models (VLMs) to assist the IFDL task. In particular, we observe that priors from VLMs hardly benefit the detection and localization performance and even have negative effects due to their inherent biases toward semantic plausibility rather than authenticity. Additionally, the location masks explicitly encode the forgery concepts, which can serve as extra priors for VLMs to ease their training optimization, thus enhancing the interpretability of detection and localization results. Building on these findings, we propose a new IFDL pipeline named IFDL-VLM. To demonstrate the effectiveness of our method, we conduct experiments on 9 popular
benchmarks and assess the model performance under both in-domain and cross-
dataset generalization settings. The experimental results show that we consistently achieve new state-of-the-art performance in detection, localization, and interpretability.Code is available at: https://github.com/sha0fengGuo/IFDL-VLM.
Read the full abstract ▾
Computer Vision
VIRD: View-Invariant Representation through Dual-Axis Transformation for Cross-View Pose Estimation
Juhye Park, Wooju Lee, Dasol Hong et al. (+4 more)
Accurate global localization is critical for autonomous driving and robotics, but GNSS-based approaches often degrade due to occlusion and multipath effects. As an emerging alternative, cross-view pose estimation predicts the 3-DoF camera pose corresponding to a ground-view image with respect to a geo-referenced satellite image. However, existing methods struggle to bridge the significant viewpoint gap between the ground and satellite views mainly due to limited spatial correspondences. We propose a novel cross-view pose estimation method that constructs view-invariant representations through dual-axis transformation (VIRD). VIRD first applies a polar transformation to the satellite view to facilitate horizontal correspondence, then uses context-enhanced positional
attention on the ground and polar-transformed satellite features to mitigate vertical misalignment, explicitly bridging the viewpoint gap. To further strengthen view invariance, we introduce a view-reconstruction loss that encourages the derived representations to reconstruct the original and cross-view images. Experiments on the KITTI and VIGOR
datasets demonstrate that VIRD outperforms the state-of-the-art methods without orientation priors, reducing median position and orientation errors by 50.7% and 76.5% on KITTI, and 18.0% and 46.8% on VIGOR, respectively.
Read the full abstract ▾
All authors Juhye Park, Wooju Lee, Dasol Hong, Changki Sung, Youngwoo Seo, Dongwan Kang, Hyun Myung
Computer Vision
Stake the Points: Structure-Faithful Instance Unlearning
Kiseong Hong, JungKyoo Shin and Eunwoo Kim
Machine unlearning (MU) addresses
privacy risks in pretrained models. The main goal of MU is to remove the influence of designated data while preserving the utility of retained knowledge. Achieving this goal requires preserving semantic relations among retained instances, which existing studies often overlook. We observe that without such preservation, models suffer from progressive structural collapse, undermining both the deletion-retention balance. In this work, we propose a novel structure-faithful framework that introduces stakes, i.e., semantic anchors that serve as reference points to maintain the knowledge structure. By leveraging these anchors, our framework captures and stabilizes the semantic organization of knowledge. Specifically, we instantiate the anchors from
language-driven attribute descriptions encoded by a semantic encoder (e.g., CLIP). We enforce preservation of the knowledge structure via structure-aware
alignment and regularization: the former aligns the organization of retained knowledge before and after unlearning around anchors, while the latter regulates updates to structure-critical parameters. Results from image classification,
retrieval, and face recognition show average gains of 32.9%, 22.5%, and 19.3% in performance, balancing the deletion-retention trade-off and enhancing generalization.
Read the full abstract ▾
Computer Vision
FedBPrompt: Federated Domain Generalization Person Re-Identification via Body Distribution Aware Visual Prompts
Xin Xu, Weilong Li, Wei Liu et al. (+5 more)
Federated Domain Generalization for Person Re-Identification (FedDG-ReID) learns domain-invariant representations from decentralized data. While
Vision Transformer (ViT) is widely adopted, its global
attention often fails to distinguish pedestrians from high similarity backgrounds or diverse viewpoints – a challenge amplified by cross-client distribution shifts in FedDG-ReID. To address this, we propose
Federated Body Distribution Aware Visual
Prompt (FedBPrompt), introducing learnable visual
prompts to guide
Transformer attention toward pedestrian-centric regions. FedBPrompt employs a Body Distribution Aware Visual
Prompts Mechanism (BAPM) comprising: Holistic Full Body
Prompts to suppress cross-client background noise, and Body Part
Alignment Prompts to capture fine-grained details robust to pose and viewpoint variations. To mitigate high communication costs, we design a
Prompt-based Fine-Tuning Strategy (PFTS) that freezes the ViT backbone and updates only lightweight
prompts, significantly reducing communication overhead while maintaining adaptability. Extensive experiments demonstrate that BAPM effectively enhances feature discrimination and cross-domain generalization, while PFTS achieves notable performance gains within only a few aggregation rounds. Moreover, both BAPM and PFTS can be easily integrated into existing ViT-based FedDG-ReID frameworks, making FedBPrompt a flexible and effective solution for
federated person re-identification. The code is available at https://github.com/leavlong/FedBPrompt.
Read the full abstract ▾
All authors Xin Xu, Weilong Li, Wei Liu, Wenke Huang, Zhixi Yu, Bin Yang, Xiaoying Liao, Kui Jiang
Computer Vision · Cross-listed from cs.LG
DirPA: Addressing Prior Shift in Imbalanced Few-shot Crop-type Classification
Joana Reuss, Ekaterina Gikalo and Marco Körner
Real-world agricultural monitoring is often hampered by severe class imbalance and high label acquisition costs, resulting in significant data scarcity. In few-shot learning (FSL) – a framework specifically designed for data-scarce settings – , training sets are often artificially balanced. However, this creates a disconnect from the long-tailed distributions observed in nature, leading to a distribution shift that undermines the model's ability to generalize to real-world agricultural tasks. We previously introduced Dirichlet Prior Augmentation (DirPA; Reuss et al., 2026a) to proactively mitigate the effects of such label distribution skews during model training. In this work, we extend the original study's geographical scope. Specifically, we evaluate this extended approach across multiple countries in the European Union (EU), moving beyond localized experiments to test the method's resilience across diverse agricultural environments. Our results demonstrate the effectiveness of DirPA across different geographical regions. We show that DirPA not only improves system robustness and stabilizes training under extreme long-tailed distributions, regardless of the target region, but also substantially improves individual class-specific performance by proactively simulating priors.
Read the full abstract ▾
Computer Vision
Spectral-Geometric Neural Fields for Pose-Free LiDAR View Synthesis
Yinuo Jiang, Jun Cheng, Yiran Wang et al. (+1 more)
Neural Radiance Fields (NeRF) have shown remarkable success in image novel view synthesis (NVS), inspiring extensions to
LiDAR NVS. However, most methods heavily rely on accurate camera poses for scene reconstruction. The sparsity and textureless nature of
LiDAR data also present distinct challenges, leading to
geometric holes and discontinuous surfaces. To address these issues, we propose SG-NLF, a pose-free
LiDAR NeRF framework that integrates spectral information with
geometric consistency. Specifically, we design a hybrid representation based on spectral priors to reconstruct smooth geometry. For pose optimization, we construct a confidence-aware graph based on feature compatibility to achieve global
alignment. In addition, an
adversarial learning strategy is introduced to enforce cross-frame consistency, thereby enhancing reconstruction quality. Comprehensive experiments demonstrate the effectiveness of our framework, especially in challenging low-frequency scenarios. Compared to previous state-of-the-art methods, SG-NLF improves reconstruction quality and pose accuracy by over 35.8% and 68.8%. Our work can provide a novel perspective for
LiDAR view synthesis.
Read the full abstract ▾
All authors Yinuo Jiang, Jun Cheng, Yiran Wang, Cheng Cheng
Computer Vision
Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Models
David McAllister, Miika Aittala, Tero Karras et al. (+4 more)
Reinforcement learning (RL) has become a standard technique for post-training
diffusion-based image synthesis models, as it enables learning from reward signals to explicitly improve desirable aspects such as image quality and
prompt alignment. In this paper, we propose an online RL variant that reduces the variance in the model updates by sampling paired trajectories and pulling the flow velocity in the direction of the more favorable image. Unlike existing methods that treat each sampling step as a separate policy action, we consider the entire sampling process as a single action. We experiment with both high-quality
vision language models and off-the-shelf quality metrics for rewards, and evaluate the outputs using a broad set of metrics. Our method converges faster and yields higher output quality and
prompt alignment than previous approaches.
Read the full abstract ▾
All authors David McAllister, Miika Aittala, Tero Karras, Janne Hellsten, Angjoo Kanazawa, Timo Aila, Samuli Laine
Computer Vision
Forecasting Epileptic Seizures from Contactless Camera via Cross-Species Transfer Learning
Mingkai Zhai, Wei Wang, Zongsheng Li et al. (+1 more)
Epileptic seizure
forecasting is a clinically important yet challenging problem in epilepsy research. Existing approaches predominantly rely on neural signals such as electroencephalography (
EEG), which require specialized equipment and limit long-term deployment in real-world settings. In contrast,
video data provide a non-invasive and accessible alternative, yet existing
video-based studies mainly focus on post-onset seizure detection, leaving seizure
forecasting largely unexplored. In this work, we formulate a novel task of
video-based epileptic seizure
forecasting, where short pre-ictal
video segments (3-10 seconds) are used to predict whether a seizure will occur within the subsequent 5 seconds. To address the scarcity of annotated human epilepsy
videos, we propose a cross-species transfer learning framework that leverages large-scale rodent
video data for auxiliary pretraining. This enables the model to capture seizure-related behavioral dynamics that generalize across species. Experimental results demonstrate that our approach achieves over 70% prediction accuracy under a strictly
video-only setting and outperforms existing baselines. These findings highlight the potential of cross-species learning for building non-invasive, scalable early-warning systems for epilepsy.
Read the full abstract ▾
All authors Mingkai Zhai, Wei Wang, Zongsheng Li, Quanying Liu
Computer Vision
A protocol for evaluating robustness to H&E staining variation in computational pathology models
Lydia A. Schönpflug, Nikki van den Berg, Sonali Andani et al. (+5 more)
Sensitivity to staining variation remains a major barrier to deploying computational pathology (CPath) models as hematoxylin and eosin (H&E) staining varies across laboratories, requiring systematic assessment of how this variability affects model prediction. In this work, we developed a three-step protocol for evaluating robustness to H&E staining variation in CPath models. Step 1: Select reference staining conditions, Step 2: Characterize test set staining properties, Step 3: Apply CPath model(s) under simulated reference staining conditions. Here, we first created a new reference staining library based on the PLISM
dataset. As an exemplary use case, we applied the protocol to assess the robustness properties of 306 microsatellite instability (MSI) classification models on the unseen SurGen colorectal cancer
dataset (n=738), including 300
attention-based multiple instance learning models trained on the TCGA-COAD/READ
datasets across three feature extractors (UNI2-h, H-Optimus-1, Virchow2), alongside six public MSI classification models. Classification performance was measured as AUC, and robustness as the min-max AUC range across four simulated staining conditions (low/high H&E intensity, low/high H&E color similarity). Across models and staining conditions, classification performance ranged from AUC 0.769-0.911 ($\Delta$ = 0.142). Robustness ranged from 0.007-0.079 ($\Delta$ = 0.072), and showed a weak inverse correlation with classification performance (Pearson r=-0.22, 95% CI [-0.34, -0.11]). Thus, we show that the proposed evaluation protocol enables robustness-informed CPath model selection and provides insight into performance shifts across H&E staining conditions, supporting the identification of operational ranges for reliable model deployment. Code is available at https://github.com/CTPLab/staining-robustness-evaluation .
Read the full abstract ▾
All authors Lydia A. Schönpflug, Nikki van den Berg, Sonali Andani, Nanda Horeweg, Jurriaan Barkey Wolf, Tjalling Bosse, Viktor H. Koelzer, Maxime W. Lafarge
Computer Vision
TRACE: Structure-Aware Character Encoding for Robust and Generalizable Document Watermarking
Jiale Meng, Jie Zhang, Runyi Hu et al. (+3 more)
We propose TRACE, a structure-aware framework leveraging
diffusion models for localized character encoding to embed data. Unlike existing methods that rely on edge features or pre-defined codebooks, TRACE exploits character structures that provide inherent resistance to noise interference due to their stability and unified representation across diverse characters. Our framework comprises three key components: (1) adaptive
diffusion initialization that automatically identifies handle points, target points, and editing regions through specialized
algorithms including movement probability
estimator (MPE), target point estimation (TPE) and mask drawing model (MDM), (2) guided
diffusion encoding for precise movement of selected point, and (3) masked region replacement with a specialized loss function to minimize feature alterations after the
diffusion process. Comprehensive experiments demonstrate \name{}'s superior performance over state-of-the-art methods, achieving more than 5 dB improvement in PSNR and 5% higher extraction accuracy following cross-media transmission. \name{} achieves broad generalizability across multiple
languages and fonts, making it particularly suitable for practical document security applications.
Read the full abstract ▾
All authors Jiale Meng, Jie Zhang, Runyi Hu, Zhe-Ming Lu, Tianwei Zhang, Yiming Li
Computer Vision
Composing Driving Worlds through Disentangled Control for Adversarial Scenario Generation
Yifan Zhan, Zhengqing Chen, Qingjie Wang et al. (+7 more)
A major challenge in autonomous driving is the "long tail" of safety-critical edge cases, which often emerge from unusual combinations of common traffic elements. Synthesizing these scenarios is crucial, yet current controllable generative models provide incomplete or entangled guidance, preventing the independent manipulation of scene structure, object identity, and ego actions. We introduce CompoSIA, a compositional driving
video simulator that disentangles these traffic factors, enabling fine-grained control over diverse
adversarial driving scenarios. To support controllable identity replacement of scene elements, we propose a noise-level identity injection, allowing pose-agnostic identity generation across diverse element poses, all from a single reference image. Furthermore, a hierarchical dual-branch action control mechanism is introduced to improve action controllability. Such disentangled control enables
adversarial scenario synthesis-systematically combining safe elements into dangerous configurations that entangled generators cannot produce. Extensive comparisons demonstrate superior controllable generation quality over state-of-the-art baselines, with a 17% improvement in FVD for identity editing and reductions of 30% and 47% in rotation and translation errors for action control. Furthermore, downstream stress-testing reveals substantial planner failures: across editing modalities, the average collision rate of 3s increases by 173%.
Read the full abstract ▾
All authors Yifan Zhan, Zhengqing Chen, Qingjie Wang, Zhuo He, Muyao Niu, Xiaoyang Guo, Wei Yin, Weiqiang Ren, Qian Zhang, Yinqiang Zheng
Computer Vision
Wear Classification of Abrasive Flap Wheels using a Hierarchical Deep Learning Approach
Falko Kähler, Maxim Wille, Ole Schmedemann et al. (+1 more)
Abrasive flap wheels are common for finishing complex free-form surfaces due to their flexibility. However, this flexibility results in complex wear patterns such as concave/convex flap profiles or flap tears, which influence the grinding result. This paper proposes a novel,
vision-based hierarchical classification framework to automate the wear condition monitoring of flap wheels. Unlike monolithic classification approaches, we decompose the problem into three logical levels: (1) state detection (new vs. worn), (2) wear type identification (rectangular, concave, convex) and flap tear detection, and (3) severity assessment (partial vs. complete deformation). A custom-built
dataset of real flap wheel images was generated and a transfer learning approach with EfficientNetV2 architecture was used. The results demonstrate high robustness with classification accuracies ranging from 93.8% (flap tears) to 99.3% (concave severity). Furthermore, Gradient-weighted Class Activation Mapping (Grad-CAM) is utilized to validate that the models learn physically relevant features and examine false classifications. The proposed hierarchical method provides a basis for adaptive process control and wear consideration in automated flap wheel grinding.
Read the full abstract ▾
All authors Falko Kähler, Maxim Wille, Ole Schmedemann, Thorsten Schüppstuhl
Computer Vision
Team LEYA in 10th ABAW Competition: Multimodal Ambivalence/Hesitancy Recognition Approach
Elena Ryumina, Alexandr Axyonov, Dmitry Sysoev et al. (+4 more)
Ambivalence/hesitancy recognition in unconstrained
videos is a challenging problem due to the subtle,
multimodal, and context-dependent nature of this behavioral state. In this paper, a
multimodal approach for
video-level ambivalence/hesitancy recognition is presented for the 10th ABAW Competition. The proposed approach integrates four complementary modalities: scene, face, audio, and text. Scene dynamics are captured with a VideoMAE-based model, facial information is encoded through emotional frame-level
embeddings aggregated by statistical pooling, acoustic representations are extracted with EmotionWav2Vec2.0 and processed by a Mamba-based temporal encoder, and linguistic cues are modeled using fine-tuned
transformer-based text models. The resulting unimodal
embeddings are further combined using
multimodal fusion models, including prototype-augmented variants. Experiments on the BAH corpus demonstrate clear gains of
multimodal fusion over all unimodal baselines. The best unimodal configuration achieved an average MF1 of 70.02%, whereas the best
multimodal fusion model reached 83.25%. The highest final test performance, 71.43%, was obtained by an ensemble of five prototype-augmented fusion models. The obtained results highlight the importance of complementary
multimodal cues and robust fusion strategies for ambivalence/hesitancy recognition.
Read the full abstract ▾
All authors Elena Ryumina, Alexandr Axyonov, Dmitry Sysoev, Timur Abdulkadirov, Kirill Almetov, Yulia Morozova, Dmitry Ryumin
Computer Vision
Multimodal Protein Language Models for Enzyme Kinetic Parameters: From Substrate Recognition to Conformational Adaptation
Fei Wang, Xinye Zheng, Kun Li et al. (+5 more)
Predicting enzyme kinetic parameters quantifies how efficiently an enzyme catalyzes a specific substrate under defined biochemical conditions. Canonical parameters such as the turnover number ($k_\text{cat}$), Michaelis constant ($K_\text{m}$), and inhibition constant ($K_\text{i}$) depend jointly on the enzyme sequence, the substrate chemistry, and the conformational adaptation of the active site during binding. Many learning pipelines simplify this process to a static compatibility problem between the enzyme and substrate, fusing their representations through shallow operations and regressing a single value. Such formulations overlook the staged nature of catalysis, which involves both substrate recognition and conformational adaptation. In this regard, we reformulate kinetic prediction as a staged
multimodal conditional modeling problem and introduce the Enzyme-Reaction Bridging Adapter (ERBA), which injects cross-modal information via fine-tuning into
Protein Language Models (PLMs) while preserving their biochemical priors. ERBA performs conditioning in two stages: Molecular Recognition Cross-
Attention (MRCA) first injects substrate information into the enzyme representation to capture specificity; Geometry-aware Mixture-of-Experts (G-MoE) then integrates active-site structure and routes samples to pocket-specialized experts to reflect induced fit. To maintain semantic fidelity, Enzyme-Substrate Distribution
Alignment (ESDA) enforces distributional consistency within the PLM
manifold in a reproducing kernel
Hilbert space. Experiments across three kinetic endpoints and multiple PLM backbones, ERBA delivers consistent gains and stronger out-of-distribution performance compared with sequence-only and shallow-fusion baselines, offering a biologically grounded route to scalable kinetic prediction and a foundation for adding cofactors, mutations, and time-resolved structural cues.
Read the full abstract ▾
All authors Fei Wang, Xinye Zheng, Kun Li, Yanyan Wei, Yuxin Liu, Ganpeng Hu, Tong Bao, Jingwen Yang
Computer Vision
Hierarchical Dual-Change Collaborative Learning for UAV Scene Change Captioning
Fuhai Chen, Pengpeng Huang, Junwen Wu et al. (+4 more)
This paper proposes a novel task for
UAV scene understanding -
UAV Scene Change Captioning (
UAV-SCC) - which aims to generate natural
language descriptions of semantic changes in dynamic aerial imagery captured from a movable viewpoint. Unlike traditional change captioning that mainly describes differences between image pairs captured from a fixed camera viewpoint over time,
UAV scene change captioning focuses on image-pair differences resulting from both temporal and spatial scene variations dynamically captured by a moving camera. The key challenge lies in understanding viewpoint-induced scene changes from
UAV image pairs that share only partially overlapping scene content due to viewpoint shifts caused by camera rotation, while effectively exploiting the relative orientation between the two images. To this end, we propose a Hierarchical Dual-Change Collaborative Learning (HDC-CL) method for
UAV scene change captioning. In particular, a novel
transformer, i.e. Dynamic Adaptive Layout
Transformer (DALT) is designed to adaptively model diverse spatial layouts of the image pair, where the interrelated features derived from the overlapping and non-overlapping regions are learned within the flexible and unified encoding layer. Furthermore, we propose a Hierarchical Cross-modal Orientation Consistency Calibration (HCM-OCC) method to enhance the model's sensitivity to viewpoint shift directions, enabling more accurate change captioning. To facilitate in-depth research on this task, we construct a new
benchmark dataset, named
UAV-SCC
dataset, for
UAV scene change captioning. Extensive experiments demonstrate that the proposed method achieves state-of-the-art performance on this task. The
dataset and code will be publicly released upon acceptance of this paper.
Read the full abstract ▾
All authors Fuhai Chen, Pengpeng Huang, Junwen Wu, Hehong Zhang, Shiping Wang, Xiaoguang Ma, Xuri Ge
Computer Vision
coDrawAgents: A Multi-Agent Dialogue Framework for Compositional Image Generation
Chunhan Li, Qifeng Wu, Jia-Hui Pan et al. (+7 more)
Text-to-image generation has advanced rapidly, but existing models still struggle with faithfully composing multiple objects and preserving their attributes in complex scenes. We propose coDrawAgents, an interactive multi-
agent dialogue framework with four specialized
agents: Interpreter, Planner, Checker, and Painter that collaborate to improve compositional generation. The Interpreter adaptively decides between a direct text-to-image pathway and a layout-aware multi-
agent process. In the layout-aware mode, it parses the
prompt into attribute-rich object descriptors, ranks them by semantic salience, and groups objects with the same semantic priority level for joint generation. Guided by the Interpreter, the Planner adopts a divide-and-conquer strategy, incrementally proposing layouts for objects with the same semantic priority level while grounding decisions in the evolving visual context of the canvas. The Checker introduces an explicit error-correction mechanism by validating spatial consistency and attribute
alignment, and refining layouts before they are rendered. Finally, the Painter synthesizes the image step by step, incorporating newly planned objects into the canvas to provide richer context for subsequent iterations. Together, these
agents address three key challenges: reducing layout complexity, grounding planning in visual context, and enabling explicit error correction. Extensive experiments on
benchmarks GenEval and DPG-Bench demonstrate that coDrawAgents substantially improves text-image
alignment, spatial accuracy, and attribute binding compared to existing methods.
Read the full abstract ▾
All authors Chunhan Li, Qifeng Wu, Jia-Hui Pan, Ka-Hei Hui, Jingyu Hu, Yuming Jiang, Bin Sheng, Xihui Liu, Wenjuan Gong, Zhengzhe Liu