Computer Vision · Cross-listed from cs.AI
ReflectCAP: Detailed Image Captioning with Reflective Memory
Kyungmin Min, Minbeom Kim, Kang-il Lee et al. (+2 more)
Detailed image captioning demands both factual grounding and fine-grained coverage, yet existing methods have struggled to achieve them simultaneously. We address this tension with Reflective Note-Guided Captioning (ReflectCAP), where a multi-
agent pipeline analyzes what the target large
vision-
language model (LVLM) consistently hallucinates and what it systematically overlooks, distilling these patterns into reusable guidelines called Structured Reflection Notes. At inference time, these notes steer the captioning model along both axes – what to avoid and what to attend to – yielding detailed captions that jointly improve factuality and coverage. Applying this method to 8 LVLMs spanning the GPT-4.1 family, Qwen series, and InternVL variants, ReflectCAP reaches the Pareto frontier of the trade-off between factuality and coverage, and delivers substantial gains on CapArena-Auto, where generated captions are judged head-to-head against strong reference models. Moreover, ReflectCAP offers a more favorable trade-off between caption quality and compute cost than model scaling or existing multi-
agent pipelines, which incur 21–36% greater overhead. This makes high-quality detailed captioning viable under real-world cost and latency constraints.
Read the full abstract ▾
All authors Kyungmin Min, Minbeom Kim, Kang-il Lee, Seunghyun Yoon, Kyomin Jung
Computer Vision
OmniFood8K: Single-Image Nutrition Estimation via Hierarchical Frequency-Aligned Fusion
Dongjian Yu, Weiqing Min, Qian Jiang et al. (+3 more)
Accurate estimation of food nutrition plays a vital role in promoting healthy dietary habits and personalized diet management. Most existing food
datasets primarily focus on Western cuisines and lack sufficient coverage of Chinese dishes, which restricts accurate nutritional estimation for Chinese meals. Moreover, many state-of-the-art nutrition prediction methods rely on depth sensors, restricting their applicability in daily scenarios. To address these limitations, we introduce OmniFood8K, a comprehensive
multimodal dataset comprising 8,036 food samples, each with detailed nutritional annotations and multi-view images. In addition, to enhance models' capability in nutritional prediction, we construct NutritionSynth-115K, a large-scale synthetic
dataset that introduces compositional variations while preserving precise nutritional labels. Moreover, we propose an end-to-end framework for nutritional prediction from a single RGB image. First, we predict a depth map from a single RGB image and design the Scale-Shift Residual Adapter (SSRA) to refine it for global scale consistency and local structural preservation. Second, we propose the Frequency-Aligned Fusion Module (FAFM) to hierarchically align and fuse RGB and depth features in the frequency domain. Finally, we design a Mask-based Prediction Head (MPH) to emphasize key ingredient regions via dynamic channel selection for more accurate prediction. Extensive experiments on multiple
datasets demonstrate the superiority of our method over existing approaches. Project homepage: https://yudongjian.github.io/OmniFood8K-food/
Read the full abstract ▾
All authors Dongjian Yu, Weiqing Min, Qian Jiang, Xing Lin, Xin Jin, Shuqiang Jiang
Computer Vision
Combating Pattern and Content Bias: Adversarial Feature Learning for Generalized AI-Generated Image Detection
Haifeng Zhang, Qinghui He, Xiuli Bi et al. (+3 more)
In recent years, the rapid development of generative artificial intelligence technology has significantly lowered the barrier to creating high-quality fake images, posing a serious challenge to information authenticity and credibility. Existing generated image detection methods typically enhance generalization through model architecture or network design. However, their generalization performance remains susceptible to data bias, as the training data may drive models to fit specific generative patterns and content rather than the common features shared by images from different generative models (asymmetric bias learning). To address this issue, we propose a Multi-dimensional
Adversarial Feature Learning (MAFL) framework. The framework adopts a pretrained
multimodal image encoder as the feature extraction backbone, constructs a real-fake feature learning network, and designs an
adversarial bias-learning branch equipped with a multi-dimensional
adversarial loss, forming an
adversarial training mechanism between authenticity-discriminative feature learning and bias feature learning. By suppressing generation-pattern and content biases, MAFL guides the model to focus on the generative features shared across different generative models, thereby effectively capturing the fundamental differences between real and generated images, enhancing cross-model generalization, and substantially reducing the reliance on large-scale training data. Through extensive experimental validation, our method outperforms existing state-of-the-art approaches by 10.89% in accuracy and 8.57% in Average Precision (AP). Notably, even when trained with only 320 images, it can still achieve over 80% detection accuracy on public
datasets.
Read the full abstract ▾
All authors Haifeng Zhang, Qinghui He, Xiuli Bi, Bo Liu, Chi-Man Pun, Bin Xiao
Computer Vision
Fundus Image-based Glaucoma Screening via Retinal Knowledge-Oriented Dynamic Multi-Level Feature Integration
Chi Liu, Yuzhuo Zhou, Sheng Shen et al. (+9 more)
While deep learning has advanced automated glaucoma screening via color fundus photography, existing purely data-driven models often overfit to confounding imaging artifacts and struggle to capture unpredictable pathological cues located beyond predefined anatomical boundaries. To address these limitations, we propose a retinal knowledge-oriented framework that synergizes dynamic multi-scale feature learning with domain-specific anatomical priors. The proposed architecture adopts a tri-branch structure to jointly model the global retinal context, the structural characteristics of the optic cup and disc, and dynamically cropped pathological regions. Specifically, we devise a Dynamic Window Mechanism (DWM) that adaptively discovers diagnostically informative patches via image-level supervision. Furthermore, we introduce a Knowledge-Enhanced Convolutional Block
Attention Module (KE-CBAM) that explicitly incorporates retinal priors from a pre-trained foundation model RETFound to guide spatial
attention, preventing the network from assigning spurious weights to irrelevant background noise. Extensive evaluations on the large-scale AIROGS
dataset demonstrate that our method achieves a state-of-the-art AUC of $98.5\%$ and an accuracy of $94.6\%$. More importantly, the integration of anatomical priors effectively mitigates the inherent class imbalance, significantly improving the detection of referable glaucoma cases. Additional validations on the SMDG-19
benchmark further confirm its superior cross-domain generalization, indicating that our contribution provides a robust, interpretable, and scalable solution for real-world clinical glaucoma diagnosis. Our code is available at https://github.com/magiczhuo/Glaucoma-Detection
Read the full abstract ▾
All authors Chi Liu, Yuzhuo Zhou, Sheng Shen, Zongyuan Ge, Fengshi Jing, Shiran Zhang, Yu Jiang, Anli Wang, Wenjian Liu, Feilong Yang, Tianqing Zhu, Xiaotong Han
Computer Vision
Unlocking the Potential of Grounding DINO in Videos: Parameter-Efficient Adaptation for Limited-Data Spatial-Temporal Localization
Zanyi Wang, Fan Li, Dengyang Jiang et al. (+4 more)
Spatio-temporal
video grounding (STVG) aims to localize queried objects within dynamic
video segments. Prevailing fully-trained approaches are notoriously data-hungry. However, gathering large-scale STVG data is exceptionally challenging: dense frame-level bounding boxes and complex temporal
language alignments are prohibitively expensive to annotate, especially for specialized
video domains. Consequently, conventional models suffer from severe overfitting on these inherently limited
datasets, while zero-shot foundational models lack the task-specific temporal awareness needed for precise localization. To resolve this small-data challenge, we introduce ST-GD, a data-efficient framework that adapts pre-trained 2D visual-
language models (e.g., Grounding DINO) to
video tasks. To avoid destroying pre-trained priors on small
datasets, ST-GD keeps the base model frozen and strategically injects lightweight adapters (~10M trainable parameters) to instill spatio-temporal awareness, alongside a novel temporal decoder for boundary prediction. This design naturally counters data scarcity. Consequently, ST-GD excels in data-scarce scenarios, achieving highly competitive performance on the limited-scale HC-STVG v1/v2
benchmarks, while maintaining robust generalization on the VidSTG
dataset. This validates ST-GD as a powerful paradigm for complex
video understanding under strict small-data constraints.
Read the full abstract ▾
All authors Zanyi Wang, Fan Li, Dengyang Jiang, Liuzhuozheng Li, Yunhua Zhong, Guang Dai, Mengmeng Wang
Computer Vision
Detecting Precise Hand Touch Moments in Egocentric Video
Huy Anh Nguyen, Feras Dayoub and Minh Hoai
We address the challenging task of detecting the precise moment when hands make contact with objects in
egocentric videos. This frame-level detection is crucial for augmented reality, human-computer interaction, assistive technologies, and
robot learning applications, where contact onset signals action initiation or completion. Temporally precise detection is particularly challenging due to subtle hand motion variations near contact, frequent occlusions, fine-grained manipulation patterns, and the inherent motion dynamics of first-person perspectives. To tackle these challenges, we propose a Hand-informed Context Enhanced module (HiCE; pronounced `high-see') that leverages spatiotemporal features from hand regions and their surrounding context through cross-
attention mechanisms, learning to identify potential contact patterns. Our approach is further refined with a grasp-aware loss and soft label that emphasizes hand pose patterns and movement dynamics characteristic of touch events, enabling the model to distinguish between near-contact and actual contact frames. We also introduce TouchMoment, an
egocentric dataset containing 4,021
videos and 8,456 annotated contact moments spanning over one million frames. Experiments on TouchMoment show that, under a strict evaluation criterion that counts a prediction as correct only if it falls within a two-frame tolerance of the ground-truth moment, our method achieves substantial gains and outperforms state-of-the-art event-spotting baselines by 16.91% average precision.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CR
CoLA: A Choice Leakage Attack Framework to Expose Privacy Risks in Subset Training
Qi Li, Cheng-Long Wang, Yinzhi Cao et al. (+1 more)
Training models on a carefully chosen portion of data rather than the full
dataset is now a standard preprocess for modern ML. From
vision coreset selection to large-scale filtering in
language models, it enables scalability with minimal utility loss. A common intuition is that training on fewer samples should also reduce
privacy risks. In this paper, we challenge this assumption. We show that subset training is not
privacy free: the very choices of which data are included or excluded can introduce new
privacy surface and leak more sensitive information. Such information can be captured by adversaries either through side-channel metadata from the subset selection process or via the outputs of the target model. To systematically study this phenomenon, we propose CoLA (Choice Leakage Attack), a unified framework for analyzing
privacy leakage in subset selection. In CoLA, depending on the adversary's knowledge of the side-channel information, we define two practical attack scenarios: Subset-aware Side-channel Attacks and Black-box Attacks. Under both scenarios, we investigate two
privacy surfaces unique to subset training: (1) Training-membership MIA (TM-MIA), which concerns only the
privacy of training data membership, and (2) Selection-participation MIA (SP-MIA), which concerns the
privacy of all samples that participated in the subset selection process. Notably, SP-MIA enlarges the notion of membership from model training to the entire data-model supply chain. Experiments on
vision and
language models show that existing threat models underestimate subset-training
privacy risks: the expanded
privacy surface leaks both training and selection membership, extending risks from individual models to the broader ML ecosystem.
Read the full abstract ▾
All authors Qi Li, Cheng-Long Wang, Yinzhi Cao, Di Wang
Computer Vision
Bridging the Micro–Macro Gap: Frequency-Aware Semantic Alignment for Image Manipulation Localization
Xiaojie Liang, Zhimin Chen, Ziqi Sheng et al. (+1 more)
As generative image editing advances, image manipulation localization (IML) must handle both traditional manipulations with conspicuous forensic artifacts and
diffusion-generated edits that appear locally realistic. Existing methods typically rely on either low-level forensic cues or high-level semantics alone, leading to a fundamental micro–macro gap. To bridge this gap, we propose FASA, a unified framework for localizing both traditional and
diffusion-generated manipulations. Specifically, we extract manipulation-sensitive frequency cues through an adaptive dual-band DCT module and learn manipulation-aware semantic priors via patch-level contrastive
alignment on frozen CLIP representations. We then inject these priors into a hierarchical frequency pathway through a semantic-frequency side adapter for multi-scale feature interaction, and employ a prototype-guided, frequency-gated mask decoder to integrate semantic consistency with boundary-aware localization for tampered region prediction. Extensive experiments on OpenSDI and multiple traditional manipulation
benchmarks demonstrate state-of-the-art localization performance, strong cross-generator and cross-
dataset generalization, and robust performance under common image degradations.
Read the full abstract ▾
All authors Xiaojie Liang, Zhimin Chen, Ziqi Sheng, Wei Lu
Computer Vision
SynMulti: Synthetic-to-Real Learning for Multimodal Video Understanding
Tanzila Rahman, Renjie Liao and Leonid Sigal
Training
multimodal large
language models (MLLMs) for
video understanding requires large-scale annotated data spanning diverse tasks such as object counting, question answering, and
segmentation. However, collecting and annotating
multimodal video data in real-world is costly, slow, and inherently limited in diversity and coverage. To address this challenge, we propose~SynMulti
dataset, along with a unified synthetic data generation pipeline capable of automatically producing unlimited
multimodal video data with rich and diverse supervision. Our framework supports multiple task formats within a single pipeline, enabling scalable and consistent data creation across tasks. To further enhance
reasoning ability, we introduce a VQA-based fine-tuning strategy that trains models to answer structured questions about visual content rather than relying solely on captions or simple instructions. This formulation encourages deeper visual grounding and
reasoning. We evaluate our approach in three challenging tasks:
video object counting,
video-based visual question answering, and
video object
segmentation. Experimental results demonstrate that models trained predominantly on synthetic data generalize effectively to real-world
datasets, often outperforming traditionally trained counterparts. Our findings highlight the potential of unified synthetic data pipelines as a scalable alternative to expensive real-world annotation for
multimodal video understanding.
Read the full abstract ▾
Computer Vision
HyperLiDAR: Adaptive Post-Deployment LiDAR Segmentation via Hyperdimensional Computing
Ivannia Gomez Moreno, Yi Yao, Ye Tian et al. (+7 more)
LiDAR semantic
segmentation plays a pivotal role in 3D scene understanding for edge applications such as autonomous driving. However, significant challenges remain for real-world deployments, particularly for on-device post-deployment adaptation. Real-world environments can shift as the system navigates through different locations, leading to substantial performance degradation without effective and timely model adaptation. Furthermore, edge systems operate under strict computational and energy constraints, making it infeasible to adapt conventional
segmentation models (based on large neural networks) directly on-device. To address the above challenges, we introduce HyperLiDAR, the first lightweight, post-deployment
LiDAR segmentation framework based on Hyperdimensional Computing (HDC). The design of HyperLiDAR fully leverages the fast learning and high efficiency of HDC, inspired by how the human
brain processes information. To further improve the adaptation efficiency, we identify the high data volume per scan as a key bottleneck and introduce a buffer selection strategy that focuses learning on the most informative points. We conduct extensive evaluations on two state-of-the-art
LiDAR segmentation benchmarks and two representative devices. Our results show that HyperLiDAR outperforms or achieves comparable adaptation performance to state-of-the-art
segmentation methods, while achieving up to a 13.8x speedup in retraining.
Read the full abstract ▾
All authors Ivannia Gomez Moreno, Yi Yao, Ye Tian, Xiaofan Yu, Flavio Ponzina, Michael Sullivan, Jingyi Zhang, Mingyu Yang, Hun Seok Kim, Tajana Rosing
Computer Vision
Self-Adversarial One Step Generation via Condition Shifting
Deyuan Liu, Peng Sun, Yansen Han et al. (+3 more)
The push for efficient text to image synthesis has moved the field toward one step sampling, yet existing methods still face a three way tradeoff among fidelity, inference speed, and training efficiency. Approaches that rely on external discriminators can sharpen one step performance, but they often introduce training instability, high GPU
memory overhead, and slow convergence, which complicates scaling and parameter efficient tuning. In contrast, regression based
distillation and consistency objectives are easier to optimize, but they typically lose fine details when constrained to a single step. We present APEX, built on a key theoretical insight:
adversarial correction signals can be extracted endogenously from a flow model through condition shifting. Using a transformation creates a shifted condition branch whose velocity field serves as an independent
estimator of the model's current generation distribution, yielding a gradient that is provably GAN aligned, replacing the sample dependent discriminator terms that cause gradient vanishing. This discriminator free design is architecture preserving, making APEX a plug and play framework compatible with both full parameter and LoRA based tuning. Empirically, our 0.6B model surpasses FLUX-Schnell 12B (20$\times$ more parameters) in one step quality. With LoRA tuning on Qwen-Image 20B, APEX reaches a GenEval score of 0.89 at NFE=1 in 6 hours, surpassing the original 50-step teacher (0.87) and providing a 15.33$\times$ inference speedup. Code is available https://github.com/LINs-lab/APEX.
Read the full abstract ▾
All authors Deyuan Liu, Peng Sun, Yansen Han, Zhenglin Cheng, Chuyan Chen, Tao Lin
Computer Vision
Jianzhe Ma, Zhonghao Cao, Shangkui Chen et al. (+3 more)
While
video large
language models (
Video-LLMs) excel in understanding slow-paced, real-world
egocentric videos, their capabilities in high-velocity, information-dense virtual environments remain under-explored. Existing
benchmarks focus on daily activities, yet lack a rigorous testbed for evaluating fast, rule-bound
reasoning in virtual scenarios. To fill this gap, we introduce EgoEsportsQA, a pioneering
video question-answering (QA)
benchmark for grounding perception and
reasoning in expert esports knowledge. We curate 1,745 high-quality QA pairs from professional matches across 3 first-person shooter games via a scalable six-stage pipeline. These questions are structured into a two-dimensional decoupled taxonomy: 11 sub-tasks in the cognitive capability dimension (covering perception and
reasoning levels) and 6 sub-tasks in the esports knowledge dimension. Comprehensive evaluations of state-of-the-art
Video-LLMs reveal that current models still fail to achieve satisfactory performance, with the best model only 71.58%. The results expose notable gaps across both axes: models exhibit stronger capabilities in basic visual perception than in deep tactical
reasoning, and they grasp overall macro-progression better than fine-grained micro-operations. Extensive ablation experiments demonstrate the intrinsic weaknesses of current
Video-LLM architectures. Further analysis suggests that our
dataset not only reveals the connections between real-world and virtual
egocentric domains, but also offers guidance for optimizing downstream esports applications, thereby fostering the future advancement of
Video-LLMs in various
egocentric environments.
Read the full abstract ▾
All authors Jianzhe Ma, Zhonghao Cao, Shangkui Chen, Yichen Xu, Wenxuan Wang, Qin Jin
Computer Vision
RSGMamba: Reliability-Aware Self-Gated State Space Model for Multimodal Semantic Segmentation
Guoan Xu, Yang Xiao, Guangwei Gao et al. (+3 more)
Multimodal semantic
segmentation has emerged as a powerful paradigm for enhancing scene understanding by leveraging complementary information from multiple sensing modalities (e.g., RGB, depth, and thermal). However, existing cross-modal fusion methods often implicitly assume that all modalities are equally reliable, which can lead to feature degradation when auxiliary modalities are noisy, misaligned, or incomplete. In this paper, we revisit cross-modal fusion from the perspective of modality reliability and propose a novel framework termed the Reliability-aware Self-Gated State Space Model (RSGMamba). At the core of our method is the Reliability-aware Self-Gated Mamba Block (RSGMB), which explicitly models modality reliability and dynamically regulates cross-modal interactions through a self-gating mechanism. Unlike conventional fusion strategies that indiscriminately exchange information across modalities, RSGMB enables reliability-aware feature selection and enhancing informative feature aggregation. In addition, a lightweight Local Cross-Gated Modulation (LCGM) is incorporated to refine fine-grained spatial details, complementing the global modeling capability of RSGMB. Extensive experiments demonstrate that RSGMamba achieves state-of-the-art performance on both RGB-D and RGB-T semantic
segmentation benchmarks, resulting 58.8% / 54.0% mIoU on NYUDepth V2 and SUN-RGBD (+0.4% / +0.7% over prior best), and 61.1% / 88.9% mIoU on MFNet and PST900 (up to +1.6%), with only 48.6M parameters, thereby validating the effectiveness and superiority of the proposed approach.
Read the full abstract ▾
All authors Guoan Xu, Yang Xiao, Guangwei Gao, Dongchen Zhu, Guo-Jun Qi, Wenjing Jia
Computer Vision
Cell Instance Segmentation via Multi-Task Image-to-Image Schrödinger Bridge
Hayato Inoue, Shota Harada, Shumpei Takezaki et al. (+1 more)
Existing
cell instance
segmentation pipelines typically combine deterministic predictions with post-processing, which imposes limited explicit constraints on the global structure of instance masks. In this work, we propose a multi-task image-to-image Schrödinger Bridge framework that formulates instance
segmentation as a distribution-based image-to-image generation problem. Boundary-aware supervision is integrated through a reverse distance map, and deterministic inference is employed to produce stable predictions. Experimental results on the PanNuke
dataset demonstrate that the proposed method achieves competitive or superior performance without relying on SAM pre-training or additional post-processing. Additional results on the MoNuSeg
dataset show robustness under limited training data. These findings indicate that Schrödinger Bridge-based image-to-image generation provides an effective framework for
cell instance
segmentation.
Read the full abstract ▾
All authors Hayato Inoue, Shota Harada, Shumpei Takezaki, Ryoma Bise
Computer Vision
GTPBD-MM: A Global Terraced Parcel and Boundary Dataset with Multi-Modality
Zhiwei Zhang, Xingyuan Zeng, Xinkai Kong et al. (+7 more)
Agricultural parcel extraction plays an important role in remote sensing-based agricultural monitoring, supporting parcel surveying, precision management, and ecological assessment. However, existing public
benchmarks mainly focus on regular and relatively flat farmland scenes. In contrast, terraced parcels in mountainous regions exhibit stepped terrain, pronounced elevation variation, irregular boundaries, and strong cross-regional heterogeneity, making parcel extraction a more challenging problem that jointly requires visual recognition, semantic discrimination, and terrain-aware
geometric understanding. Although recent studies have advanced visual parcel
benchmarks and image-text farmland understanding, a unified
benchmark for complex terraced parcel extraction under aligned image-text-DEM settings remains absent. To fill this gap, we present GTPBD-MM, the first
multimodal benchmark for global terraced parcel extraction. Built upon GTPBD, GTPBD-MM integrates high-resolution optical imagery, structured text descriptions, and DEM data, and supports systematic evaluation under Image-only, Image+Text, and Image+Text+DEM settings. We further propose Elevation and Text guided Terraced parcel network (ETTerra), a
multimodal baseline for terraced parcel delineation. Extensive experiments demonstrate that textual semantics and terrain geometry provide complementary cues beyond visual appearance alone, yielding more accurate, coherent, and structurally consistent delineation results in complex terraced scenes.
Read the full abstract ▾
All authors Zhiwei Zhang, Xingyuan Zeng, Xinkai Kong, Kunquan Zhang, Haoyuan Liang, Bohan Shi, Juepeng Zheng, Jianxi Huang, Yutong Lu, Haohuan Fu
Computer Vision
Towards Realistic and Consistent Orbital Video Generation via 3D Foundation Priors
Rong Wang, Ruyi Zha, Ziang Cheng et al. (+3 more)
We present a novel method for generating geometrically realistic and consistent orbital
videos from a single image of an object. Existing
video generation works mostly rely on pixel-wise
attention to enforce view consistency across frames. However, such mechanism does not impose sufficient constraints for long-range extrapolation, e.g. rear-view synthesis, in which pixel correspondences to the input image are limited. Consequently, these works often fail to produce results with a plausible and coherent structure. To tackle this issue, we propose to leverage rich shape priors from a 3D foundational generative model as an auxiliary constraint, motivated by its capability of modeling realistic object shape distributions learned from large 3D asset corpora. Specifically, we
prompt the
video generation with two scales of latent features encoded by the 3D foundation model: (i) a denoised global latent vector as an overall structural guidance, and (ii) a set of latent images projected from volumetric features to provide view-dependent and fine-grained geometry details. In contrast to commonly used 2.5D representations such as depth or normal maps, these compact features can model complete object shapes, and help to improve inference efficiency by avoiding explicit mesh extraction. To achieve effective shape conditioning, we introduce a multi-scale 3D adapter to inject feature
tokens to the base
video model via cross-
attention, which retains its capabilities from general
video pretraining and enables a simple and model-agonistic fine-tuning process. Extensive experiments on multiple
benchmarks show that our method achieves superior visual quality, shape realism and multi-view consistency compared to state-of-the-art methods, and robustly generalizes to complex camera trajectories and in-the-wild images.
Read the full abstract ▾
All authors Rong Wang, Ruyi Zha, Ziang Cheng, Jiayu Yang, Pulak Purkait, Hongdong Li
Computer Vision
Boosting Robust AIGI Detection with LoRA-based Pairwise Training
Ruiyang Xia, Qi Zhang, Yaowen Xu et al. (+4 more)
The proliferation of highly realistic AI-Generated Image (AIGI) has necessitated the development of practical detection methods. While current AIGI detectors perform admirably on clean
datasets, their detection performance frequently decreases when deployed "in the wild", where images are subjected to unpredictable, complex distortions. To resolve the critical vulnerability, we propose a novel LoRA-based Pairwise Training (LPT) strategy designed specifically to achieve robust detection for AIGI under severe distortions. The core of our strategy involves the targeted finetuning of a visual foundation model, the deliberate simulation of data distribution during the training phase, and a unique pairwise training process. Specifically, we introduce distortion and size simulations to better fit the distribution from the validation and test sets. Based on the strong visual representation capability of the visual foundation model, we finetune the model to achieve AIGI detection. The pairwise training is utilized to improve the detection via decoupling the generalization and robustness optimization. Experiments show that our approach secured the 3th placement in the NTIRE Robust AI-Generated Image Detection in the Wild challenge
Read the full abstract ▾
All authors Ruiyang Xia, Qi Zhang, Yaowen Xu, Zhaofan Zou, Hao Sun, Zhongjiang He, Xuelong Li
Computer Vision · Cross-listed from eess.IV
CBAM-Enhanced DenseNet121 for Multi-Class Chest X-Ray Classification with Grad-CAM Explainability
Utsho Kumar Dey
Pneumonia remains a leading cause of childhood mortality worldwide, with a heavy burden in low-resource settings such as Bangladesh where radiologist availability is limited. Most existing deep learning approaches treat pneumonia detection as a binary problem, overlooking the clinically critical distinction between bacterial and viral aetiology. This paper proposes CBAM-DenseNet121, a transfer-learning framework that integrates the Convolutional Block
Attention Module (CBAM) into DenseNet121 for three-class chest X-ray classification: Normal, Bacterial Pneumonia, and Viral Pneumonia. We also conduct a systematic binary-task baseline study revealing that EfficientNetB3 (73.88%) underperforms even the custom CNN baseline (78.53%) – a practically important negative finding for medical imaging model selection. To ensure statistical reliability, all experiments were repeated three times with independent random seeds (42, 7, 123), and results are reported as mean +/- standard deviation. CBAM-DenseNet121 achieves 84.29% +/- 1.14% test accuracy with per-class AUC scores of 0.9565 +/- 0.0010, 0.9610 +/- 0.0014, and 0.9187 +/- 0.0037 for bacterial pneumonia, normal, and viral pneumonia respectively. Grad-CAM visualizations confirm that the model attends to anatomically plausible pulmonary regions for each class, supporting interpretable deployment in resource-constrained clinical environments.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.SD
CoSyncDiT: Cognitive Synchronous Diffusion Transformer for Movie Dubbing
Gaoxiang Cong, Liang Li, Jiaxin Ye et al. (+4 more)
Movie dubbing aims to synthesize
speech that preserves the vocal identity of a reference audio while synchronizing with the lip movements in a target
video. Existing methods fail to achieve precise lip-sync and lack naturalness due to explicit
alignment at the duration level. While implicit
alignment solutions have emerged, they remain susceptible to interference from the reference audio, triggering timbre and pronunciation degradation in in-the-wild scenarios. In this paper, we propose a novel flow matching-based movie dubbing framework driven by the Cognitive Synchronous
Diffusion Transformer (CoSync-DiT), inspired by the cognitive process of professional actors. This architecture progressively guides the noise-to-
speech generative trajectory by executing acoustic style adapting, fine-grained visual calibrating, and time-aware context aligning. Furthermore, we design the Joint Semantic and
Alignment Regularization (JSAR) mechanism to simultaneously constrain frame-level temporal consistency on the contextual outputs and semantic consistency on the flow hidden states, ensuring robust
alignment. Extensive experiments on both standard
benchmarks and challenging in-the-wild dubbing
benchmarks demonstrate that our method achieves the state-of-the-art performance across multiple metrics.
Read the full abstract ▾
All authors Gaoxiang Cong, Liang Li, Jiaxin Ye, Zhedong Zhang, Hongming Shan, Yuankai Qi, Qingming Huang
Computer Vision
LiveMoments: Reselected Key Photo Restoration in Live Photos via Reference-guided Diffusion
Clara Xue, Zizheng Yan, Zhenning Shi et al. (+5 more)
Live Photo captures both a high-quality key photo and a short
video clip to preserve the precious dynamics around the captured moment. While users may choose alternative frames as the key photo to capture better expressions or timing, these frames often exhibit noticeable quality degradation, as the photo capture ISP pipeline delivers significantly higher image quality than the
video pipeline. This quality gap highlights the need for dedicated restoration techniques to enhance the reselected key photo. To this end, we propose LiveMoments, a reference-guided image restoration framework tailored for the reselected key photo in Live Photos. Our method employs a two-branch neural network: a reference branch that extracts structural and textural information from the original high-quality key photo, and a main branch that restores the reselected frame using the guidance provided by the reference branch. Furthermore, we introduce a unified Motion
Alignment module that incorporates motion guidance for spatial
alignment at both the latent and image levels. Experiments on real and synthetic Live Photos demonstrate that LiveMoments significantly improves perceptual quality and fidelity over existing solutions, especially in scenes with fast motion or complex structures. Our code is available at https://github.com/OpenVeraTeam/LiveMoments.
Read the full abstract ▾
All authors Clara Xue, Zizheng Yan, Zhenning Shi, Yuhang Yu, Jingyu Zhuang, Qi Zhang, Jinwei Chen, Qingnan Fan
Computer Vision
MAST: Mask-Guided Attention Control for Training-Free Regional-Multi Style Transfer
Dongkyung Kang, Jaeyeon Hwang, Junseo Park et al. (+6 more)
Style transfer applies the appearance of a reference image to a content image while preserving its spatial structure. Recent
diffusion-based methods achieve strong stylization but typically assume a single global style. We instead consider regional-multi style transfer, which assigns multiple references to user-specified regions of a content image. Extending them to this setting reveals two coupled shared-
attention issues: ambiguous mass allocation among content and style partitions and degraded selectivity as more styles are jointly normalized, while style aggregation at the
attention output further suppresses fine details. We propose MAST (Mask-Guided
Attention Control for Training-Free Regional-Multi Style Transfer), a unified
attention-control framework for frozen
diffusion models. Logit-level
Attention Mass Allocation enforces mask-derived partition masses, Sharpness-aware Temperature Scaling adaptively restores selectivity, and Discrepancy-aware Detail Injection recovers high-frequency content. MAST jointly processes all style–mask pairs in a single denoising pass without training, optimization, or post-hoc composition. Across two to five styles, MAST achieves the best average ArtFID, FID, and R-FID among all baselines, demonstrating regional style fidelity, content preservation, and scalability.
Read the full abstract ▾
All authors Dongkyung Kang, Jaeyeon Hwang, Junseo Park, Minji Kang, Yeryeong Lee, Beomseok Ko, Hanyoung Roh, Jeongmin Shin, Hyeryung Jang
Computer Vision · Cross-listed from cs.LG
SubFlow: Sub-mode Conditioned Flow Matching for Diverse One-Step Generation
Yexiong Lin, Jia Shi, Shanshan Ye et al. (+3 more)
Flow matching has emerged as a powerful generative framework, with recent few-step methods achieving remarkable inference acceleration. However, we identify a critical yet overlooked limitation: these models suffer from severe diversity degradation, concentrating samples on dominant modes while neglecting rare but valid variations of the target distribution. We trace this degradation to averaging distortion: when trained with MSE objectives, class-conditional flows learn a frequency-weighted mean over intra-class sub-modes, causing the model to over-represent high-density modes while systematically neglecting low-density ones. To address this, we propose SubFlow, Sub-mode Conditioned Flow Matching, which eliminates averaging distortion by decomposing each class into fine-grained sub-modes via semantic clustering and conditioning the flow on sub-mode indices. Each conditioned sub-distribution is approximately unimodal, so the learned flow accurately targets individual modes with no averaging distortion, restoring full mode coverage in a single inference step. Crucially, SubFlow is entirely plug-and-play: it integrates seamlessly into existing one-step models such as MeanFlow and Shortcut Models without any architectural modifications. Extensive experiments on ImageNet-256 demonstrate that SubFlow yields substantial gains in generation diversity (Recall) while maintaining competitive image quality (FID), confirming its broad applicability across different one-step generation frameworks. Project page: https://yexionglin.github.io/subflow.
Read the full abstract ▾
All authors Yexiong Lin, Jia Shi, Shanshan Ye, Wanyu Wang, Yu Yao, Tongliang Liu
Computer Vision
DreamStereo: Towards Real-Time Stereo Inpainting for HD Videos
Yuan Huang, Sijie Zhao, Jing Cheng et al. (+2 more)
Stereo
video inpainting, which aims to fill the occluded regions of warped
videos with visually coherent content while maintaining temporal consistency, remains a challenging open problem. The regions to be filled are scattered along object boundaries and occupy only a small fraction of each frame, leading to two key challenges. First, existing approaches perform poorly on such tasks due to the scarcity of high-quality stereo inpainting
datasets, which limits their ability to learn effective inpainting priors. Second, these methods apply equal processing to all regions of the frame, even though most pixels require no modification, resulting in substantial redundant computation. To address these issues, we introduce three interconnected components. We first propose Gradient-Aware Parallax Warping (GAPW), which leverages backward warping and the gradient of the coordinate mapping function to obtain continuous edges and smooth occlusion regions. Then, a Parallax-Based Dual Projection (PBDP) strategy is introduced, which incorporates GAPW to produce geometrically consistent stereo inpainting pairs and accurate occlusion masks without requiring stereo
videos. Finally, we present Sparsity-Aware Stereo Inpainting (SASI), which reduces over 70% of redundant
tokens, achieving a 10.7x speedup during
diffusion inference and delivering results comparable to its full-computation counterpart, enabling real-time processing of HD (768 x 1280)
videos at 25 FPS on a single A100 GPU.
Read the full abstract ▾
All authors Yuan Huang, Sijie Zhao, Jing Cheng, Hao Xu, Shaohui Jiao
Computer Vision
Style-Decoupled Adaptive Routing Network for Underwater Image Enhancement
Hang Xu, Chen Long, Bing Wang et al. (+2 more)
Underwater Image Enhancement (UIE) is essential for robust visual perception in marine applications. However, existing methods predominantly rely on uniform mapping tailored to average
dataset distributions, leading to over-processing mildly degraded images or insufficient recovery for severe ones. To address this challenge, we propose a novel adaptive enhancement framework, SDAR-Net. Unlike existing uniform paradigms, it first decouples specific degradation styles from the input and subsequently modulates the enhancement process adaptively. Specifically, since
underwater degradation primarily shifts the appearance while keeping the scene structure, SDAR-Net formulates image features into dynamic degradation style
embeddings and static scene structural representations through a carefully designed training framework. Subsequently, we introduce an adaptive routing mechanism. By evaluating style features and adaptively predicting soft weights at different enhancement states, it guides the weighted fusion of the corresponding image representations, accurately satisfying the adaptive restoration demands of each image. Extensive experiments show that SDAR-Net achieves a new state-of-the-art (SOTA) performance with a PSNR of 25.72 dB on real-world
benchmark, and demonstrates its utility in downstream
vision tasks. Our code is available at https://github.com/WHU-USI3DV/SDAR-Net.
Read the full abstract ▾
All authors Hang Xu, Chen Long, Bing Wang, Hao Chen, Zhen Dong
Computer Vision
ARGen: Affect-Reinforced Generative Augmentation towards Vision-based Dynamic Emotion Perception
Huanzhen Wang, Ziheng Zhou, Jiaqi Song et al. (+4 more)
Dynamic facial expression recognition in the wild remains challenging due to data scarcity and long-tail distributions, which hinder models from effectively learning the temporal dynamics of scarce
emotions. To address these limitations, we propose ARGen, an Affect-Reinforced Generative Augmentation Framework that enables data-adaptive dynamic expression generation for robust
emotion perception. ARGen operates in two stages: Affective Semantic Injection (ASI) and Adaptive Reinforcement
Diffusion (ARD). The ASI stage establishes affective knowledge
alignment through facial Action Units and employs a
retrieval-augmented
prompt generation strategy to synthesize consistent and fine-grained affective descriptions via large-scale visual-
language models, thereby injecting interpretable emotional priors into the generation process. The ARD stage integrates text-conditioned image-to-
video diffusion with reinforcement learning, introducing inter-frame conditional guidance and a multi-objective reward function to jointly optimize expression naturalness, facial integrity, and generative efficiency. Extensive experiments on both generation and recognition tasks verify that ARGen substantially enhances synthesis fidelity and improves recognition performance, establishing an interpretable and generalizable generative augmentation paradigm for
vision-based affective computing.
Read the full abstract ▾
All authors Huanzhen Wang, Ziheng Zhou, Jiaqi Song, Li He, Yunshi Lan, Yan Wang, Wenqiang Zhang
Computer Vision
ArtifactWorld: Scaling 3D Gaussian Splatting Artifact Restoration via Video Generation Models
Xinliang Wang, Yifeng Shi and Zhenyu Wu
3D Gaussian Splatting (3DGS) delivers high-fidelity real-time rendering but suffers from
geometric and photometric degradations under sparse-view constraints. Current generative restoration approaches are often limited by insufficient temporal coherence, a lack of explicit spatial constraints, and a lack of large-scale training data, resulting in multi-view inconsistencies, erroneous
geometric hallucinations, and limited generalization to diverse real-world artifact distributions. In this paper, we present ArtifactWorld, a framework that resolves 3DGS artifact repair through systematic data expansion and a homogeneous dual-model paradigm. To address the data bottleneck, we establish a fine-grained phenomenological taxonomy of 3DGS artifacts and construct a comprehensive training set of 107.5K diverse paired
video clips to enhance model robustness. Architecturally, we unify the restoration process within a
video diffusion backbone, utilizing an isomorphic predictor to localize structural defects via an artifact heatmap. This heatmap then guides the restoration through an Artifact-Aware Triplet Fusion mechanism, enabling precise, intensity-guided spatio-temporal repair within native self-
attention. Extensive experiments demonstrate that ArtifactWorld achieves state-of-the-art performance in sparse novel view synthesis and robust 3D reconstruction. Code available at: https://github.com/fyting/ArtifactWorld.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Socrates Loss: Unifying Confidence Calibration and Classification by Leveraging the Unknown
Sandra Gómez-Gálvez, Tobias Olenyi, Gillian Dobbie et al. (+1 more)
Deep neural networks, despite their high accuracy, often exhibit poor confidence calibration, limiting their reliability in high-stakes applications. Current ad-hoc confidence calibration methods attempt to fix this during training but face a fundamental trade-off: two-phase training methods achieve strong classification performance at the cost of training instability and poorer confidence calibration, while single-loss methods are stable but underperform in classification. This paper addresses and mitigates this stability-performance trade-off. We propose Socrates Loss, a novel, unified loss function that explicitly leverages uncertainty by incorporating an auxiliary unknown class, whose predictions directly influence the loss function and a dynamic uncertainty penalty. This unified objective allows the model to be optimized for both classification and confidence calibration simultaneously, without the instability of complex, scheduled losses. We provide theoretical guarantees that our method regularizes the model to prevent miscalibration and overfitting. Across four
benchmark datasets and multiple architectures, our comprehensive experiments demonstrate that Socrates Loss consistently improves training stability while achieving more favorable accuracy-calibration trade-off, often converging faster than existing methods.
Read the full abstract ▾
All authors Sandra Gómez-Gálvez, Tobias Olenyi, Gillian Dobbie, Katerina Taškova
Computer Vision
Physics-Grounded Monocular Vehicle Distance Estimation Using Standardized License Plate Typography
Manognya Lokesh Reddy and Zheng Liu
Accurate inter-vehicle distance estimation is a cornerstone of Advanced Driver Assistance Systems (ADAS) and autonomous driving. While
LiDAR and radar provide high precision, their high cost prohibits widespread adoption in mass-
market vehicles.
Monocular camera-based estimation offers a low-cost alternative but suffers from fundamental scale ambiguity. Recent deep learning methods for
monocular depth achieve impressive results yet require expensive supervised training, suffer from domain shift, and produce predictions that are difficult to certify for safety-critical deployment. This paper presents a framework that exploits the standardized typography of United States license plates as passive fiducial markers for metric ranging, resolving scale ambiguity through explicit
geometric priors without any training data or active illumination. First, a four-method parallel plate detector achieves robust plate reading across the full automotive lighting range. Second, a three-stage state identification engine fusing optical character recognition text matching, multi-design color scoring, and a lightweight neural network classifier provides robust identification across all ambient conditions. Third, hybrid depth fusion with inverse-variance weighting and online scale
alignment, combined with a one-dimensional constant-velocity Kalman filter, delivers smoothed distance, relative velocity, and time-to-collision for collision warning. Baseline validation on a controlled static
dataset reproduces a 2.3% coefficient of variation in character height measurements and a 36% reduction in distance-estimate variance compared with plate-width methods from prior work.
Read the full abstract ▾
Computer Vision
BarbieGait: An Identity-Consistent Synthetic Human Dataset with Versatile Cloth-Changing for Gait Recognition
Qingyuan Cai, Saihui Hou, Xuecai Hu et al. (+1 more)
Gait recognition, as a reliable biometric technology, has seen rapid development in recent years while it faces significant challenges caused by diverse clothing styles in the real world. This paper introduces BarbieGait, a synthetic gait
dataset where real-world subjects are uniquely mapped into a virtual engine to simulate extensive clothing changes while preserving their gait identity information. As a pioneering work, BarbieGait provides a controllable gait data generation method, enabling the production of large
datasets to validate cross-clothing issues that are difficult to verify with real-world data. However, the diversity of clothing increases intra-class variance and makes one of the biggest challenges to learning cloth-invariant features under varying clothing conditions. Therefore, we propose GaitCLIF (Gait-oriented CLoth-Invariant Feature) as a robust baseline model for cross-clothing gait recognition. Through extensive experiments, we validate that our method significantly improves cross-clothing performance on BarbieGait and the existing popular gait
benchmarks. We believe that BarbieGait, with its extensive cross-clothing gait data, will further advance the capabilities of gait recognition in cross-clothing scenarios and promote progress in related research.
Read the full abstract ▾
All authors Qingyuan Cai, Saihui Hou, Xuecai Hu, Yongzhen Huang
Computer Vision
Ride the Wave: Precision-Allocated Sparse Attention for Smooth Video Generation
Wentai Zhang, Ronghui Xi, Shiyao Peng et al. (+4 more)
Video Diffusion Transformers have revolutionized high-fidelity
video generation but suffer from the massive computational burden of self-
attention. While sparse
attention provides a promising acceleration solution, existing methods frequently provoke severe visual flickering caused by static sparsity patterns and deterministic block routing. To resolve these limitations, we propose Precision-Allocated Sparse
Attention (PASA), a training-free framework designed for highly efficient and temporally smooth
video generation. First, we implement a
curvature-aware dynamic budgeting mechanism. By profiling the generation trajectory acceleration across timesteps, we elastically allocate the exact-computation budget to secure high-precision processing strictly during critical semantic transitions. Second, we replace global homogenizing estimations with hardware-aligned grouped approximations, successfully capturing fine-grained local variations while maintaining peak compute throughput. Finally, we incorporate a
stochastic selection bias into the
attention routing mechanism. This probabilistic approach softens rigid selection boundaries and eliminates selection oscillation, effectively eradicating the localized computational starvation that drives temporal flickering. Extensive evaluations on leading
video diffusion models demonstrate that PASA achieves substantial inference acceleration while consistently producing remarkably fluid and structurally stable
video sequences.
Read the full abstract ▾
All authors Wentai Zhang, Ronghui Xi, Shiyao Peng, Jiayu Huang, Haoran Luo, Zichen Tang, Haihong E