Computer Vision · Cross-listed from cs.LG
Friction-Augmented Drifting Models for Resource-Efficient Domain Translation
Arkadii Kazanskii, Tatiana Petrova, Andrey Ustyuzhanin et al. (+3 more)
Single-step generators promise high-fidelity synthesis at a fraction of the inference and training cost of ordinary differential equation (ODE)-based flow models, a central concern when compute is limited. Drifting Models (DMs) train a one-step generator by evolving samples under a kernel-based drift field, avoiding ODE integration entirely, but a two-particle surrogate of their iteration admits a locally repulsive regime in which repulsion can dominate the attraction to the target. We introduce DMF (Drifting Model with Friction), which scales the drift field by a linearly-scheduled coefficient $1-\gamma(i)$. A closed-form analysis of the surrogate gives a per-step contraction threshold and a finite-horizon bound on the error trajectory, suggesting why friction can halt the iteration before it relaxes to a spurious force-balance fixed point. On FFHQ latent-space domain translation, DMF significantly improves on the frictionless DM it extends in both Fréchet Inception Distance (FID; paired $p=0.019$, Cohen's $d=1.71$) and CLIP-MMD (CMMD; $p=0.005$, $d=2.55$) with no additional forward passes or parameters, and on a 2D task it sharply improves DM's Fréchet (moment-matching) error while remaining on par under the 2-Wasserstein distance. DMF also achieves FID and CMMD comparable to the far more expensive Optimal Flow Matching (OFM) in our runs, at roughly $29\times$ lower training wall-clock on identical hardware. DMF thus delivers these gains with a single scheduled scalar.
Read the full abstract ▾
All authors Arkadii Kazanskii, Tatiana Petrova, Andrey Ustyuzhanin, Konstantin Bagrianskii, Aleksandr Puzikov, Radu State
Computer Vision
Towards Multi-View Sign Language Understanding: A Benchmark Dataset and Baseline
Xu Wang, Shengeng Tang, Wan Jiang et al. (+3 more)
Most existing Sign
Language Understanding (SLU) methods are developed under fixed-view settings and remain vulnerable to viewpoint changes, while the scarcity of multi-view data hinders systematic research on viewpoint robustness. To address this gap, we introduce MVSign, a
benchmark spanning three sign
languages and seven viewpoints. MVSign combines original frontal recordings from existing
datasets with additional views synthesized via $3$D whole-body motion reconstruction and target-view rendering, while preserving the source splits and annotations. We further propose CanonSLU, a framework for multi-view SLU. This framework employs a canonical-view-guided strategy, using the frontal view as a semantic anchor. Specifically, CanonSLU incorporates Canonical-View Knowledge Transfer (CKT) to transfer canonical knowledge from the frontal view to non-frontal views, and Motion Relation Aggregation (MRA) to aggregate motion-related features across frames and strengthen temporal representations under viewpoint changes. Extensive experiments on MVSign demonstrate competitive performance with gains on non-frontal views, establishing CanonSLU as an effective baseline for multi-view SLU.
Read the full abstract ▾
All authors Xu Wang, Shengeng Tang, Wan Jiang, Yaxiong Wang, Lechao Cheng, Richang Hong
Computer Vision
Extending One-Step Image Generation from Class Labels to Text via Discriminative Text Representation
Chenxi Zhao, Chen Zhu, Xiaokun Feng et al. (+6 more)
Few-step generation has been a long-standing goal, with recent one-step generation methods exemplified by MeanFlow achieving remarkable results. Existing research on MeanFlow primarily focuses on class-to-image generation. However, an intuitive yet unexplored direction is to extend the condition from fixed class labels to flexible text inputs, enabling richer content creation. Compared to the limited class labels, text conditions pose greater challenges to the model's understanding capability, necessitating the effective integration of powerful text encoders into the MeanFlow framework. Surprisingly, although incorporating text conditions appears straightforward, we find that integrating powerful LLM-based text encoders using conventional training strategies results in unsatisfactory performance. To uncover the underlying cause, we conduct detailed analyses and reveal that, due to the extremely limited number of refinement steps in the MeanFlow generation, such as only one step, the text feature representations are required to possess sufficiently high discriminability. This also explains why discrete and easily distinguishable class features perform well within the MeanFlow framework. Guided by these insights, we leverage a powerful LLM-based text encoder validated to possess the required semantic properties and adapt the MeanFlow generation process to this framework, resulting in efficient text-conditioned synthesis for the first time. Furthermore, we validate our approach on the widely used
diffusion model, demonstrating significant generation performance improvements. We hope this work provides a general and practical reference for future research on text-conditioned MeanFlow generation. The code is available at https://github.com/AMAP-ML/EMF.
Read the full abstract ▾
All authors Chenxi Zhao, Chen Zhu, Xiaokun Feng, Aiming Hao, Jiashu Zhu, Jiachen Lei, Jiahong Wu, Xiangxiang Chu, Jufeng Yang
Computer Vision
Embedding Arithmetic: A Lightweight, Tuning-Free Framework for Post-hoc Bias Mitigation in Text-to-Image Models
Venkatesh Thirugnana Sambandham and Torsten Schön
Modern text-to-image (T2I) models amplify harmful societal biases, challenging their ethical deployment. We introduce an inference-time method that reliably mitigates social bias while keeping
prompt semantics and visual context (background, layout, and style) intact. This ensures context persistency and provides a controllable parameter to adjust mitigation strength, giving practitioners fine-grained control over
fairness-coherence trade-offs. Using
Embedding Arithmetic, we analyze how bias is structured in the
embedding space and correct it without altering model weights,
prompts, or
datasets. Experiments on FLUX 1.0-Dev and Stable
Diffusion 3.5-Large show that the conditional
embedding space forms a complex, entangled
manifold rather than a grid of disentangled concepts. To rigorously assess semantic preservation beyond the circularity and bias limitations of of CLIP scores, we propose the Concept Coherence Score (CCS). Evaluated against this robust metric, our lightweight, tuning-free method significantly outperforms existing baselines in improving diversity while maintaining high concept coherence, effectively resolving the critical
fairness-coherence trade-off. By characterizing how models represent social concepts, we establish
geometric understanding of latent space as a principled path toward more transparent, controllable, and fair image generation.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge
Sua Lee, Sanghee Park and Jinbae Im
Multimodal Large
Language Models (MLLMs) have been increasingly used as automatic evaluators-a paradigm known as MLLM-as-a-Judge. However, their reliability and vulnerabilities to biases remain underexplored. We find that many MLLM judges fail to reliably integrate key visual or textual cues, yielding unreliable evaluations when evidence is missing or mismatched, and exhibiting instability under semantically irrelevant perturbations. To address this, we systematically define Compositional Bias in MLLM-as-a-Judge systems and introduce MM-JudgeBias, a
benchmark for evaluating it. MM-JudgeBias introduces controlled perturbations across Query, Image, and Response, and evaluates model behavior via two complementary metrics: Bias-Deviation (BD) for sensitivity and Bias-Conformity (BC) for stability. Our
dataset of over 1,800 curated and refined
multimodal samples, drawn from 29 source
benchmarks, enables a fine-grained diagnosis of nine bias types across diverse tasks and domains. Experiments on 26 state-of-the-art MLLMs reveal systematic modality neglect and asymmetric evaluation tendencies, underscoring the need for more reliable judges.
Read the full abstract ▾
Computer Vision
AI-based Waste Mapping for Addressing Climate-Exacerbated Flood Risk
Steffen Knoblauch, Levi Szamek, Iddy Chazua et al. (+3 more)
Urban flooding is a growing
climate change-related hazard in rapidly expanding African cities, where inadequate waste management often blocks drainage systems and amplifies flood risks. This study introduces an AI-powered urban waste mapping workflow that leverages openly available aerial and street-view imagery to detect municipal solid waste at high resolution. Applied in Dar es Salaam, Tanzania, our approach reveals spatial waste patterns linked to informal settlements and socio-economic factors. Waste accumulation in waterways was found to be up to three times higher than in adjacent urban areas, highlighting critical hotspots for
climate-exacerbated flooding. Unlike traditional manual mapping methods, this scalable AI approach allows city-wide monitoring and prioritization of interventions. Crucially, our collaboration with local partners ensured culturally and contextually relevant data labeling, reflecting real-world reuse practices for solid waste. The results offer actionable insights for urban planning,
climate adaptation, and sustainable waste management in flood-prone urban areas.
Read the full abstract ▾
All authors Steffen Knoblauch, Levi Szamek, Iddy Chazua, Benedcto Adamu, Innocent Maholi, Alexander Zipf
Computer Vision
Ammar Bhilwarawala and Mainak Bandyopadhyay
Automated fetal head
segmentation in ultrasound images is critical for accurate biometric measurements in prenatal care. While existing deep learning approaches have achieved a reasonable performance, they struggle with issues like low contrast, noise, and complex anatomical boundaries which are inherent to ultrasound imaging. This paper presents
Attention-ResUNet. It is a novel architecture that synergistically combines residual learning with multi-scale
attention mechanisms in order to achieve enhanced fetal head
segmentation. Our approach integrates
attention gates at four decoder levels to focus selectively on anatomically relevant regions while suppressing the background noise, and complemented by residual connections which facilitates gradient flow and feature reuse. Extensive evaluation on the HC18 Challenge
dataset where n = 200 demonstrates that
Attention ResUNet achieves a superior performance with a mean Dice score of 99.30 +/- 0.14% against similar architectures. It significantly outperforms five baseline architectures including ResUNet (99.26%),
Attention U-Net (98.79%), Swin U-Net (98.60%), Standard U-Net (98.58%), and U-Net++ (97.46%). Through statistical analysis we confirm highly significant improvements (p < 0.001) with effect sizes that range from 0.230 to 13.159 (Cohen's d). Using Saliency map analysis, we reveal that our architecture produces highly concentrated, anatomically consistent activation patterns, which demonstrate an enhanced interpretability which is crucial for clinical deployment. The proposed method establishes a new state of the art performance for automated fetal head
segmentation whilst maintaining computational efficiency with 14.7M parameters and a 45 GFLOPs inference cost. Code repository: https://github.com/Ammar-ss
Read the full abstract ▾
Computer Vision
Region-Grounded Report Generation for 3D Medical Imaging: A Fine-Grained Dataset and Graph-Enhanced Framework
Cong Huy Nguyen, Son Dinh Nguyen, Guanlin Li et al. (+8 more)
Automated medical report generation for 3D PET/CT imaging is fundamentally challenged by the high-dimensional nature of volumetric data and a critical scarcity of annotated
datasets, particularly for low-resource
languages. Current black-box methods map whole volumes to reports, ignoring the clinical workflow of analyzing localized Regions of Interest (RoIs) to derive diagnostic conclusions. In this paper, we bridge this gap by introducing VietPET-RoI, the first large-scale 3D PET/CT
dataset with fine-grained RoI annotation for a low-resource
language, comprising 600 PET/CT samples and 1,960 manually annotated RoIs, paired with corresponding clinical reports. Furthermore, to demonstrate the utility of this
dataset, we propose HiRRA, a novel framework that mimics the professional radiologist diagnostic workflow by employing graph-based relational modules to capture dependencies between RoI attributes. This approach shifts from global pattern matching toward localized clinical findings. Additionally, we introduce new clinical evaluation metrics, namely RoI Coverage and RoI Quality Index, that measure both RoI localization accuracy and attribute description fidelity using LLM-based extraction. Extensive evaluation demonstrates that our framework achieves SOTA performance, surpassing existing models by 19.7% in BLEU and 4.7% in ROUGE-L, while achieving a remarkable 45.8% improvement in clinical metrics, indicating enhanced clinical reliability and reduced
hallucination. Our code and
dataset are available on GitHub.
Read the full abstract ▾
All authors Cong Huy Nguyen, Son Dinh Nguyen, Guanlin Li, Tuan Dung Nguyen, Aditya Narayan Sankaran, Mai Huy Thong, Thanh Trung Nguyen, Mai Hong Son, Reza Farahbakhsh, Phi Le Nguyen, Noel Crespi
Computer Vision
Xiao Lingao and Yang He
Large-scale
dataset distillation requires storing auxiliary soft labels that can be 30-40x larger on ImageNet-1K and 200x larger on ImageNet-21K than the condensed images, undermining the goal of
dataset compression. We identify two fundamental issues necessitating such extensive labels: (1) insufficient image diversity, where high within-class similarity in synthetic images requires extensive augmentation, and (2) insufficient supervision diversity, where limited variety in supervisory signals during training leads to performance degradation at high compression rates. To address these challenges, we propose Label
Pruning and Quantization for Large-scale
Distillation (LPQLD). We enhance image diversity via class-wise batching and batch-normalization supervision during synthesis. For supervision diversity, we introduce Label
Pruning with Dynamic Knowledge Reuse to improve label-per-augmentation diversity, and Label Quantization with Calibrated Student-Teacher
Alignment to improve augmentation-per-image diversity. Our approach reduces soft label storage by 78x on ImageNet-1K and 500x on ImageNet-21K while improving accuracy by up to 7.2% and 2.8%, respectively. Extensive experiments validate the superiority of LPQLD across different network architectures and
dataset distillation methods. Code is available at https://github.com/he-y/soft-label-
pruning-quantization-for-
dataset-
distillation.
Read the full abstract ▾
Computer Vision
Can LLM-Generated Text Empower Surgical Vision-Language Pre-training?
Chengan Che, Chao Wang, Jiayuan Huang et al. (+2 more)
Recent advancements in self-supervised learning have led to powerful
surgical vision encoders capable of spatiotemporal understanding. However, extending these visual foundations to multi-modal
reasoning tasks is severely bottlenecked by the prohibitive cost of expert textual annotations. To overcome this scalability limitation, we introduce LIME, a large-scale multi-modal
dataset derived from open-access
surgical videos using human-free, Large
Language Model (LLM)-generated narratives. While LIME offers immense scalability, unverified generated texts may contain errors, including
hallucinations, that could potentially lead to catastrophically degraded pre-trained medical priors in standard contrastive pipelines. To mitigate this, we propose SurgLIME, a parameter-efficient
Vision-
Language Pre-training (VLP) framework designed to learn reliable cross-modal
alignments using noisy narratives. SurgLIME preserves foundational medical priors using a LoRA-adapted dual-encoder architecture and introduces an automated confidence estimation mechanism that dynamically down-weights uncertain text during contrastive
alignment. Evaluations on the AutoLaparo and Cholec80
benchmarks show that SurgLIME achieves competitive zero-shot cross-modal
alignment while preserving the robust linear probing performance of the visual foundation model.
Dataset, code, and models are publicly available at https://github.com/visurg-ai/SurgLIME.
Read the full abstract ▾
All authors Chengan Che, Chao Wang, Jiayuan Huang, Xinyue Chen, Luis C. Garcia-Peraza-Herrera
Computer Vision · Cross-listed from cs.RO
Chatting about Conditional Trajectory Prediction
Yuxiang Zhao, Wei Huang, Haipeng Zeng et al. (+2 more)
Human behavior has the nature of mutual dependencies, which requires human-
robot interactive systems to predict surrounding
agents trajectories by modeling complex social interactions, avoiding collisions and executing safe path planning. While there exist many trajectory prediction methods, most of them do not incorporate the own motion of the ego
agent and only model interactions based on static information. We are inspired by the humans theory of mind during trajectory selection and propose a Cross time domain intention-interactive method for conditional Trajectory prediction(CiT). Our proposed CiT conducts joint analysis of behavior intentions over time, and achieves information complementarity and integration across different time domains. The intention in its own time domain can be corrected by the social interaction information from the other time domain to obtain a more precise intention representation. In addition, CiT is designed to closely integrate with robotic motion planning and control modules, capable of generating a set of optional trajectory prediction results for all surrounding
agents based on potential motions of the ego
agent. Extensive experiments demonstrate that the proposed CiT significantly outperforms the existing methods, achieving state-of-the-art performance in the
benchmarks.
Read the full abstract ▾
All authors Yuxiang Zhao, Wei Huang, Haipeng Zeng, Huan Zhao, Yujie Song
Computer Vision
Test-Time Perturbation Learning with Delayed Feedback for Vision-Language-Action Models
Zehua Zang, Xi Wang, Fuchun Sun et al. (+4 more)
Vision-
Language-Action models (VLAs) achieve remarkable performance in sequential decision-making but remain fragile to subtle environmental shifts, such as small changes in object pose. We attribute this brittleness to trajectory overfitting, where VLAs over-attend to the spurious correlation between actions and entities, then reproduce memorized action patterns. We propose Perturbation learning with Delayed Feedback (PDF), a verifier-free test-time adaptation framework that improves decision performance without fine-tuning the base model. PDF mitigates the spurious correlation through uncertainty-based data augmentation and action voting, while an adaptive scheduler allocates augmentation budgets to balance performance and efficiency. To further improve stability, PDF learns a lightweight perturbation module that retrospectively adjusts action logits guided by delayed feedback, correcting overconfidence issue. Experiments on LIBERO (+7.4% success rate) and Atari (+10.3 human normalized score) demonstrate consistent gains of PDF in task success over vanilla VLA and VLA with test-time adaptation, establishing a practical path toward reliable test-time adaptation in
multimodal decision-making
agents. The code is available at \href{https://github.com/zhoujiahuan1991/CVPR2026-PDF}{https://github.com/zhoujiahuan1991/CVPR2026-PDF}.
Read the full abstract ▾
All authors Zehua Zang, Xi Wang, Fuchun Sun, Xiao Xu, Lixiang Lium, Jiahuan Zhou, Jiangmeng Li
Computer Vision
Sehyeong Jo, Gangjae Jang and Haesol Park
Vision Transformers (ViTs) have become a dominant architecture in computer
vision, yet their prediction process remains difficult to interpret because information is propagated through complex interactions across layers and
attention heads. Existing
attention based explanation methods provide an intuitive way to trace information flow. However, they rely mainly on raw
attention weights, which do not explicitly reflect the final decision and often lead to explanations with limited class discriminability. In contrast, gradient based localization methods are more effective at highlighting class specific evidence, but they do not fully exploit the hierarchical
attention propagation mechanism of
transformers. To address this limitation, we propose Decision-Aware
Attention Propagation (DAP), an attribution method that injects decision-relevant priors into
transformer attention propagation. By estimating
token importance through gradient based localization and integrating it into layer wise
attention rollout, the method captures both the structural flow of
attention and the evidence most relevant to the final prediction. Consequently, DAP produces attribution maps that are more class sensitive, compact, and faithful than those generated by conventional
attention based methods. Extensive experiments across
Vision Transformer variants of different model scales show that DAP consistently outperforms existing baselines in both quantitative metrics and qualitative visualizations, indicating that decision aware propagation is an effective direction for improving ViT interpretability.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Culture-Aware Humorous Captioning: Multimodal Humor Generation across Cultural Contexts
Run Xu, Lu Li, Rongzhao Zhang et al. (+1 more)
Recent
multimodal large
language models have shown promising ability in generating humorous captions for images, yet they still lack stable control over explicit cultural context, making it difficult to jointly maintain image relevance, contextual appropriateness, and humor quality under a specified cultural background. To address this limitation, we introduce a new
multimodal generation task, culture-aware humorous captioning, which requires a model to generate a humorous caption conditioned on both an input image and a target cultural context. Captions generated under different cultural contexts are not expected to share the same surface form, but should remain grounded in similar visual situations or humorous rationales.To support this task, we establish a six-dimensional evaluation framework covering image relevance, contextual fit, semantic richness, reasonableness, humor, and creativity. We further propose a staged
alignment framework that first initializes the model with high-resource supervision under the Western cultural context, then performs multi-dimensional preference
alignment via judge-based GRPO with a Degradation-aware Prototype Repulsion Constraint to mitigate reward hacking in open-ended generation, and finally adapts the model to the Eastern cultural context with a small amount of supervision. Experimental results show that our method achieves stronger overall performance under the proposed evaluation framework, with particularly large gains in contextual fit and a better balance between image relevance and humor under cultural constraints.
Read the full abstract ▾
All authors Run Xu, Lu Li, Rongzhao Zhang, Jie Xu
Computer Vision
Autonomous Unmanned Aircraft Systems for Enhanced Search and Rescue of Drowning Swimmers: Image-Based Localization and Mission Simulation
Sascha Emanuel Zell, Toni Schneidereit, Armin Fügenschuh et al. (+1 more)
Drowning is an omnipresent risk associated with any activity on or in the water, and rescuing a drowning person is particularly challenging because of the time pressure, making a short response time important. Further complicating water rescue are unsupervised and extensive swimming areas, precise localization of the target, and the transport of rescue personnel. Technical innovations can provide a remedy: We propose an Unmanned Aircraft System (UAS), also known as a drone-in-a-box system, consisting of a fleet of Unmanned Aerial Vehicles (
UAVs) allocated to purpose-built hangars near swimming areas. In an emergency, the UAS can be deployed in addition to Standard Rescue Operation (SRO) equipment to locate the distressed person early by performing a fully automated Search and Rescue (S&R) operation and dropping a flotation device. In this paper, we address automatically locating distressed swimmers using the image-based object detection architecture You Only Look Once (YOLO). We present a
dataset created for this application and outline the training process. We evaluate the performance of YOLO versions 3, 5, and 8 and architecture sizes (nano, extra-large) using Mean Average Precision (mAP) metrics mAP@.5 and mAP@.5:.95. Furthermore, we present two Discrete-Event Simulation (DES) approaches to simulate response times of SRO and UAS-based water rescue. This enables estimation of time savings relative to SRO when selecting the UAS configuration (type, number, and location of
UAVs and hangars). Computational experiments for a test area in the Lusatian Lake District, Germany, show that UAS assistance shortens response time. Even a small UAS with two hangars, each containing one
UAV, reduces response time by a factor of five compared to SRO.
Read the full abstract ▾
All authors Sascha Emanuel Zell, Toni Schneidereit, Armin Fügenschuh, Michael Breuß
Computer Vision
Class-specific diffusion models improve military object detection in a low-data domain
Ella P. Fokkinga, Jan Erik van Woerden, Thijs A. Eker et al. (+4 more)
Diffusion-based image synthesis has emerged as a promising source of synthetic training data for AI-based object detection and classification. In this work, we investigate whether images generated with
diffusion can improve military vehicle detection under low-data conditions. We fine-tuned the text-to-image
diffusion model FLUX.1 [dev] using LoRA with only 8 or 24 real images per class across 15 vehicle categories, resulting in class-specific
diffusion models, which were used to generate new samples from automatically generated text
prompts. The same real images were used to fine-tune the RF-DETR detector for a 15-class object detection task. Synthetic
datasets generated by the
diffusion models were then used to further improve detector performance. Importantly, no additional real data was required, as the generative models leveraged the same limited training samples. FLUX-generated images improved detection performance, particularly in the low-data regime (up to +8.0% mAP$_{50}$ with 8 real samples). To address the limited
geometric control of text
prompt-based
diffusion, we additionally generated structurally guided synthetic data using ControlNet with Canny edge-map conditioning, yielding a FLUX-ControlNet (FLUX-CN)
dataset with explicit control over viewpoint and pose. Structural guidance further enhanced performance when data is scarce (+4.1% mAP$_{50}$ with 8 real samples), but no additional benefit was observed when more real data is available. This study demonstrates that object-specific
diffusion models are effective for improving military object detection in a low-data domain, and that structural guidance is most beneficial when real data is highly limited. These results highlight generative image data as an alternative to traditional simulation pipelines for the training of military AI systems.
Read the full abstract ▾
All authors Ella P. Fokkinga, Jan Erik van Woerden, Thijs A. Eker, Sebastiaan P. Snel, Elfi I.S. Hofmeijer, Klamer Schutte, Friso G. Heslinga
Computer Vision
Enhancing Continual Learning of Vision-Language Models via Dynamic Prefix Weighting
Hyeonseo Jang, Hyuk Kwon and Kibok Lee
We investigate recently introduced domain-class incremental learning scenarios for
vision-
language models (VLMs). Recent works address this challenge using parameter-efficient methods, such as prefix-tuning or adapters, which facilitate model adaptation to downstream tasks by incorporating task-specific information into input
tokens through additive vectors. However, previous approaches often normalize the weights of these vectors, disregarding the fact that different input
tokens require different degrees of adjustment. To overcome this issue, we propose Dynamic Prefix Weighting (DPW), a framework that dynamically assigns weights to prefixes, complemented by adapters. DPW consists of 1) a gating module that adjusts the weights of each prefix based on the importance of the corresponding input
token, and 2) a weighting mechanism that derives adapter output weights as a residual of prefix-tuning weights, ensuring that adapters are utilized only when necessary. Experimental results demonstrate that our method achieves state-of-the-art performance in domain-class incremental learning scenarios for VLMs. The code is available at: https://github.com/YonseiML/dpw.
Read the full abstract ▾
Computer Vision
INTENT: Invariance and Discrimination-aware Noise Mitigation for Robust Composed Image Retrieval
Zhiwei Chen, Yupeng Hu, Zhiheng Fu et al. (+4 more)
Composed Image
Retrieval (CIR) is a challenging image
retrieval paradigm that enables to retrieve target images based on
multimodal queries consisting of reference images and modification texts. Although substantial progress has been made in recent years, existing methods assume that all samples are correctly matched. However, in real-world scenarios, due to high triplet annotation costs, CIR
datasets inevitably contain annotation errors, resulting in incorrectly matched triplets. To address this issue, the problem of Noisy Triplet Correspondence (NTC) has attracted growing
attention. We argue that noise in CIR can be categorized into two types: cross-modal correspondence noise and modality-inherent noise. The former arises from mismatches across modalities, whereas the latter originates from intra-modal background interference or visual factors irrelevant to the coarse-grained modification annotations. However, modality-inherent noise is often overlooked, and research on cross-modal correspondence noise remains nascent. To tackle above issues, we propose the Invariance and discrimiNaTion-awarE Noise neTwork (INTENT), comprising two components: Visual Invariant Composition and Bi-Objective Discriminative Learning, specifically designed to handle the two-aspect noise. The former applies causal intervention on the visual side via Fast
Fourier Transform (FFT) to generate intervened composed features, enforcing visual invariance and enabling the model to ignore modality-inherent noise during composition. The latter adopts collaborative optimization with both positive and negative samples, and constructs a scalable decision boundary that dynamically adjusts decisions based on the loyalty degree, enabling robust correspondence discrimination. Extensive experiments on two widely used
benchmark datasets demonstrate the superiority and robustness of INTENT.
Read the full abstract ▾
All authors Zhiwei Chen, Yupeng Hu, Zhiheng Fu, Zixu Li, Jiale Huang, Qinlei Huang, Yinwei Wei
Computer Vision
GS-STVSR: Ultra-Efficient Continuous Spatio-Temporal Video Super-Resolution via 2D Gaussian Splatting
Mingyu Shi, Xin Di, Long Peng et al. (+8 more)
Continuous Spatio-Temporal
Video Super-Resolution (C-STVSR) aims to simultaneously enhance the spatial resolution and frame rate of
videos by arbitrary scale factors, offering greater flexibility than fixed-scale methods that are constrained by predefined upsampling ratios. In recent years, methods based on Implicit Neural Representations (INR) have made significant progress in C-STVSR by learning continuous mappings from spatio-temporal coordinates to pixel values. However, these methods fundamentally rely on dense pixel-wise grid queries, causing computational cost to scale linearly with the number of interpolated frames and severely limiting inference efficiency. We propose GS-STVSR, an ultra-efficient C-STVSR framework based on 2D Gaussian Splatting (2D-GS) that drives the spatiotemporal
evolution of Gaussian kernels through continuous motion modeling, bypassing dense grid queries entirely. We exploit the strong temporal stability of covariance parameters for lightweight intermediate fitting, design an optical flow-guided motion module to derive Gaussian position and color at arbitrary time steps, introduce a Covariance resampling
alignment module to prevent covariance drift, and propose an adaptive offset window for large-scale motion. Extensive experiments on Vid4, GoPro, and Adobe240 show that GS-STVSR achieves state-of-the-art quality across all
benchmarks. Moreover, its inference time remains nearly constant at conventional temporal scales (X2–X8) and delivers over X3 speedup at extreme scales X32, demonstrating strong practical applicability.
Read the full abstract ▾
All authors Mingyu Shi, Xin Di, Long Peng, Boxiang Cao, Anran Wu, Zhanfeng Feng, Jiaming Guo, Renjing Pei, Xueyang Fu, Yang Cao, Zhengjun Zha
Computer Vision
HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval
Zixu Li, Yupeng Hu, Zhiwei Chen et al. (+4 more)
Composed Image
Retrieval (CIR) is a flexible image
retrieval paradigm that enables users to accurately locate the target image through a
multimodal query composed of a reference image and modification text. Although this task has demonstrated promising applications in personalized search and recommendation systems, it encounters a severe challenge in practical scenarios known as the Noise Triplet Correspondence (NTC) problem. This issue primarily arises from the high cost and subjectivity involved in annotating triplet data. To address this problem, we identify two central challenges: the precise estimation of composed semantic discrepancy and the insufficient progressive adaptation to modification discrepancy. To tackle these challenges, we propose a cHrono-synergiA roBust progressIve learning framework for composed image
reTrieval (HABIT), which consists of two core modules. First, the Mutual Knowledge Estimation Module quantifies sample cleanliness by calculating the Transition Rate of mutual information between the composed feature and the target image, thereby effectively identifying clean samples that align with the intended modification semantics. Second, the Dual-consistency Progressive Learning Module introduces a collaborative mechanism between the historical and current models, simulating human habit formation to retain good habits and calibrate bad habits, ultimately enabling robust learning under the presence of NTC. Extensive experiments conducted on two standard CIR
datasets demonstrate that HABIT significantly outperforms most methods under various noise ratios, exhibiting superior robustness and
retrieval performance. Codes are available at https://github.com/Lee-zixu/HABIT
Read the full abstract ▾
All authors Zixu Li, Yupeng Hu, Zhiwei Chen, Shiqi Zhang, Qinlei Huang, Zhiheng Fu, Yinwei Wei
Computer Vision · Cross-listed from cs.CL
SignDPO: Multi-level Direct Preference Optimisation for Skeleton-based Gloss-free Sign Language Translation
Muxin Pu, Xiao-Ming Wu, Mei Kuan Lim et al. (+3 more)
We present SignDPO, a novel multi-level Direct Preference Optimisation (DPO) framework designed to enhance the
alignment of skeleton-based Sign
Language Translation. While current skeleton-based models have made significant progress using Maximum Likelihood Estimation, they are primarily constrained by an imitation-based paradigm that lacks discriminative sensitivity to the fine-grained spatio-temporal nuances of sign
language, often leading to semantic drift. To address this, SignDPO shifts the optimisation goal from simple sequence mimicry to structured preference
alignment across spatial, temporal, and linguistic dimensions. Our framework involves three key designs. First, we introduce a hierarchical perturbation strategy to construct spatial and temporal non-preferred samples at both global and local granularities automatically. Second, we propose a self-guiding mechanism that leverages decoder cross-
attention scores to identify and perturb semantically salient skeletal regions, forcing the model to distinguish genuine sign signals from structural distortions. Third, we establish an automated
language-level preference generator by fine-tuning a dedicated perturbation model, capturing complex output-level failure modes without manual annotation. Extensive experiments on three widely adopted
benchmarks, CSL-Daily, How2Sign, and OpenASL, demonstrate that SignDPO consistently outperforms state-of-the-art gloss-free methods and even rivals established gloss-based ones. Our results suggest that multi-level preference
alignment is a powerful paradigm for bridging the gap between high-
entropy skeletal trajectories and discrete linguistic semantics.
Read the full abstract ▾
All authors Muxin Pu, Xiao-Ming Wu, Mei Kuan Lim, Chun Yong Chong, Wei Li, Chen Change Loy
Computer Vision
CFSR: Geometry-Conditioned Shadow Removal via Physical Disentanglement
Pan Wang, Yihao Hu, Xiujin Liu et al. (+1 more)
Traditional shadow removal networks often treat image restoration as an unconstrained mapping, lacking the physical interpretability required to balance localized texture recovery with global illumination consistency. To address this, we propose CFSR, a multi-modal prior-driven framework that reframes shadow removal as a physics-constrained restoration process. By seamlessly integrating 3D
geometric cues with large-scale foundation model semantics, CFSR effectively bridges the 2D-3D domain gap. Specifically, we first map observations into a custom HVI color space to suppress shadow-induced noise and robustly fuse RGB data with estimated depth priors. At its core, our
Geometric & Semantic Dual Explicit Guided
Attention mechanism utilizes DINO features and 3D surface normals to directly modulate the
attention affinity
matrix, structurally enforcing physical lighting constraints. To recover severely degraded regions, we inject holistic priors via a frozen CLIP encoder. Finally, our Frequency Collaborative Reconstruction Module (FCRM) achieves an optimal synthesis by decoupling the decoding process. Conditioned on
geometric priors, FCRM seamlessly harmonizes the reconstruction of sharp high-frequency occlusion boundaries with the restoration of low-frequency global illumination. Extensive experiments demonstrate that CFSR achieves state-of-the-art performance across multiple challenging
benchmarks.
Read the full abstract ▾
All authors Pan Wang, Yihao Hu, Xiujin Liu, Hang Wang
Computer Vision
Multi-View Hierarchical Graph Neural Network for Sketch-Based 3D Shape Retrieval
Hang Cheng, Muyan He, Mingyu Fan et al. (+3 more)
Sketch-based 3D shape
retrieval (SBSR) aims to retrieve 3D shapes that are consistent with the category of the input hand-drawn sketch. The core challenge of this task lies in two aspects: existing methods typically employ simplified aggregation strategies for independently encoded 3D multi-view features, which ignore the
geometric relationships between views and multi-level details, resulting in weak 3D representation. Simultaneously, traditional SBSR methods are constrained by visible category limitations, leading to poor performance in zero-shot scenarios. To address these challenges, we propose Multi-View Hierarchical Graph Neural Network (MV-HGNN), a novel framework for SBSR. Specifically, we construct a view-level graph and capture adjacent
geometric dependencies and cross-view message passing via local graph convolution and global
attention. A view selector is further introduced to perform hierarchical graph coarsening, enabling a progressively larger receptive field for graph convolution and mitigating the interference of redundant views, which leads to more discriminate discriminative hierarchical 3D representation. To enable category agnostic
alignment and mitigate overfitting to seen classes, we leverage CLIP text
embeddings as semantic prototypes and project both sketch and 3D features into a shared semantic space. We use a two-stage training strategy for category-level
retrieval and a one-stage strategy for zero-shot
retrieval under the same model architecture. Under both category-level and zero-shot settings, extensive experiments on two public
benchmarks demonstrate that MV-HGNN outperforms state-of-the-art methods.
Read the full abstract ▾
All authors Hang Cheng, Muyan He, Mingyu Fan, Chengfeng Xie, Xi Cheng, Long Zeng
Computer Vision
Trustworthy Endoscopic Super-Resolution
Julio Silva-Rodríguez and Ender Konukoglu
Super-resolution (SR) models are attracting growing interest for enhancing minimally invasive surgery and diagnostic
videos under hardware constraints. However, valid concerns remain regarding the introduction of hallucinated structures and amplified noise, limiting their reliability in safety-critical settings. We propose a direct and practical framework to make SR systems more trustworthy by identifying where reconstructions are likely to fail. Our approach integrates a lightweight error-prediction network that operates on intermediate representations to estimate pixel-wise reconstruction error. The module is computationally efficient and low-latency, making it suitable for real-time deployment. We convert these predictions into operational failure decisions by constructing Conformal Failure Masks (CFM), which localize regions where the SR output should not be trusted. Built on conformal risk control principles, our method provides theoretical guarantees for controlling both the tolerated error limit and the miscoverage in detected failures. We evaluate our approach on image and
video SR, demonstrating its effectiveness in detecting unreliable reconstructions in endoscopic and robotic surgery settings. To our knowledge, this is the first study to provide a model-agnostic, theoretically grounded approach to improving the safety of real-time endoscopic image SR.
Read the full abstract ▾
Computer Vision
Yu Zhang, Chuyang Sun, Kehai Chen et al. (+3 more)
Large
Vision-
Language Models (LVLMs) still struggle with
vision hallucination, where generated responses are inconsistent with the visual input. Existing methods either rely on large-scale annotated data for fine-tuning, which incurs massive computational overhead, or employ static post-hoc strategies that overlook the dynamic nature of
hallucination emergence. To address these, we introduce a new self-rewarding framework, enabling dynamic
hallucination mitigation at inference time without external supervision. On the empirical side, we reveal that visual
hallucination exhibits phase-wise dynamic patterns, peaking at the onset of each semantic phase. Drawing on these insights, we propose PSRD (\textbf{Phase-wise Self-Reward Decoding) for online
hallucination correction guided by phase-wise self-reward signals. To reduce the cost of repeated self-evaluation during decoding, we distill the
hallucination guidance signal from LVLMs into a lightweight reward model. The reward model subsequently provides on-the-fly guidance for targeted intervention during the decoding process, enabling precise
hallucination suppression. The proposed PSRD significantly reduces the
hallucination rate of LLaVA-1.5-7B by 50.0% and consistently outperforms existing post-hoc methods across five
hallucination evaluation
benchmarks for four LVLMs. Further analysis confirms that PSRD effectively mitigates
hallucination propagation and achieves a highly controllable trade-off between strong performance and inference efficiency.
Read the full abstract ▾
All authors Yu Zhang, Chuyang Sun, Kehai Chen, Xuefeng Bai, Yang Xiang, Min Zhang
Computer Vision
Identifying Ethical Biases in Action Recognition Models
Ana Baltaretu, Pascal Benschop and Jan van Gemert
Human Action Recognition (HAR) models are increasingly deployed in high-stakes environments, yet their
fairness across different human appearances has not been analyzed. We introduce a framework for
auditing bias in HAR models using synthetic
video data, generated with full control over visual identity attributes such as skin color. Unlike prior work that focuses on static images or pose estimation, our approach preserves temporal consistency, allowing us to isolate and test how changes to a single attribute affect model predictions. Through controlled interventions using the BEDLAM simulation platform, we show whether some popular HAR models exhibit statistically significant biases on the skin color even when the motion remains identical. Our results highlight how models may encode unwanted visual associations, and we provide evidence of systematic errors across groups. This work contributes a framework for
auditing HAR models and supports the development of more transparent, accountable systems in light of upcoming regulatory standards.
Read the full abstract ▾
Computer Vision
E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes
Koya Sakamoto, Taiki Miyanishi, Daichi Azuma et al. (+6 more)
Visual search in 3D environments requires
embodied agents to actively explore their surroundings and acquire task-relevant evidence. However, existing visual search and
embodied AI
benchmarks, including EQA, typically rely on static observations or constrained
egocentric motion, and thus do not explicitly evaluate fine-grained viewpoint-dependent phenomena that arise under unrestricted 5-DoF viewpoint control in real-world 3D environments, such as visibility changes caused by vertical viewpoint shifts, revealing contents inside containers, and disambiguating object attributes that are only observable from specific angles. To address this limitation, we introduce {E3VS-Bench}, a
benchmark for
embodied 3D visual search where
agents must control their viewpoints in 5-DoF to gather viewpoint-dependent evidence for question answering. E3VS-Bench consists of 99 high-fidelity 3D scenes reconstructed using 3D Gaussian Splatting and 2,014 question-driven episodes. 3D Gaussian Splatting enables photorealistic free-viewpoint rendering that preserves fine-grained visual details (e.g., small text and subtle attributes) often degraded in mesh-based simulators, thereby allowing the construction of questions that cannot be answered from a single view and instead require active inspection across viewpoints in 5-DoF. We evaluate multiple state-of-the-art VLMs and compare their performance with humans. Despite strong 2D
reasoning ability, all models exhibit a substantial gap from humans, highlighting limitations in active perception and coherent viewpoint planning specifically under full 5-DoF viewpoint changes.
Read the full abstract ▾
All authors Koya Sakamoto, Taiki Miyanishi, Daichi Azuma, Shuhei Kurita, Shu Morikuni, Naoya Chiba, Motoaki Kawanabe, Yusuke Iwasawa, Yutaka Matsuo
Computer Vision
MU-GeNeRF: Multi-view Uncertainty-guided Generalizable Neural Radiance Fields for Distractor-aware Scene
Wenjie Mu, Zhan Li, Chuanzhou Su et al. (+8 more)
Generalizable Neural Radiance Fields (GeNeRFs) enable high-quality scene reconstruction from sparse views and can generalize to unseen scenes. However, in real-world settings, transient distractors break cross-view structural consistency, corrupting supervision and degrading reconstruction quality. Existing distractor-free NeRF methods rely on per-scene optimization and estimate uncertainty from per-view reconstruction errors, which are not reliable for GeNeRFs and often misjudge inconsistent static structures as distractors. To this end, we propose MU-GeNeRF, a Multi-view Uncertainty-guided distractor-aware GeNeRF framework designed to alleviate GeNeRF's robust modeling challenges in the presence of transient distractions. We decompose distractor awareness into two complementary uncertainty components: Source-view Uncertainty, which captures structural discrepancies across source views caused by viewpoint changes or dynamic factors; and Target-view Uncertainty, which detects observation anomalies in the target image induced by transient distractors.These two uncertainties address distinct error sources and are combined through a heteroscedastic reconstruction loss, which guides the model to adaptively modulate supervision, enabling more robust distractor suppression and
geometric modeling.Extensive experiments show that our method not only surpasses existing GeNeRFs but also achieves performance comparable to scene-specific distractor-free NeRFs.
Read the full abstract ▾
All authors Wenjie Mu, Zhan Li, Chuanzhou Su, Xuanyi Shen, Ziniu Liu, Fan Lu, Yujian Mo, Junqiao Zhao, Tiantian Feng, Chen Ye, Guang Chen
Computer Vision
DifFoundMAD: Foundation Models meet Differential Morphing Attack Detection
Lazaro J. Gonzalez-Soler, André Dörsch, Christian Rathgeb et al. (+1 more)
In this work, we introduce DifFoundMAD, a parameter-efficient D-MAD framework that exploits the generalisation capabilities of
vision foundation models (FM) to capture discrepancies between suspected morphs and live capture images. In contrast to conventional D-MAD systems that rely on face recognition
embeddings or handcrafted feature differences, DifFoundMAD follows the standard differential paradigm while replacing the underlying representation space with
embeddings extracted from FMs. By combining lightweight finetuning with class-balanced optimisation, the proposed method updates only a small subset of parameters while preserving the rich representational priors of the underlying FMs. Extensive cross-database evaluations on standard D-MAD
benchmarks demonstrate that DifFoundMAD achieves consistent improvements over state-of-the-art systems, particularly at the strict security levels required in operational deployments such as border control: The error rates reported in the current state-of-the-art were reduced from 6.16% to 2.17% for high-security levels using DifFoundMAD
Read the full abstract ▾
All authors Lazaro J. Gonzalez-Soler, André Dörsch, Christian Rathgeb, Christoph Busch
Computer Vision
Chatting about Upper-Body Expressive Human Pose and Shape Estimation
Yuxiang Zhao, Wei Huang, Yujie Song et al. (+2 more)
Expressive Human Pose and Shape Estimation (EHPS) plays a crucial role in various AR/VR applications and has witnessed significant progress in recent years. However, current state-of-the-art methods still struggle with accurate parameter estimation for facial and hand regions and exhibit limited generalization to wild images. To address these challenges, we present CoEvoer, a novel one-stage synergistic cross-dependency
transformer framework tailored for upper-body EHPS. CoEvoer enables explicit feature-level interaction across different body parts, allowing for mutual enhancement through contextual information exchange. Specifically, larger and more easily estimated regions such as the torso provide global semantics and positional priors to guide the estimation of finer, more complex regions like the face and hands. Conversely, the localized details captured in facial and hand regions help refine and calibrate adjacent body parts. To the best of our knowledge, CoEvoer is the first framework designed specifically for upper-body EHPS, with the goal of capturing the strong coupling and semantic dependencies among the face, hands, and torso through joint parameter regression. Extensive experiments demonstrate that CoEvoer achieves state-of-the-art performance on upper-body
benchmarks and exhibits strong generalization capability even on unseen wild images.
Read the full abstract ▾
All authors Yuxiang Zhao, Wei Huang, Yujie Song, Liu Wang, Huan Zhao