Computer Vision
Qwen-Image-Bench: From Generation to Creation in Text-to-Image Evaluation
Niantong Li, Guangzheng Hu, Weixu Qiao et al. (+35 more)
Text-to-Image generation has evolved from basic image synthesis into a frequently used core capability in professional creative workflows, where simple text-image
alignment can no longer satisfy users' pressing demands for faithful real-world reconstruction and genuine creative expression. Existing
benchmarks, however, remain anchored in these foundational criteria and do not yet capture the nuanced capabilities that matter in authentic artistic practice, making it difficult to reliably distinguish state-of-the-art T2I models. To address the gap, we introduce Qwen-Image-Bench, a creator-centric
benchmark co-designed with professional artists and grounded in real-world creation scenarios. Qwen-Image-Bench enriches conventional evaluation with two application-driven dimensions: Real-world Fidelity and Creative Generation. Drawing on the staged
reasoning inherent in professional artistic workflows, we organize these five pillars into a top-down hierarchical taxonomy that further decomposes into 23 second-level sub-capabilities and 56 third-level verifiable rubrics. To ensure broad coverage, we curate 1000 stratified
prompts with each
prompt jointly exercising more than four fine-grained facets across multiple pillars. We train a unified judge model Q-Judger based on Qwen3.6-27B, supervised by 80 professional annotators from global art academies under blind labeling and triple-review protocols, that scores every image across all 56 verifiable facets, producing fine-grained, rubric-grounded, and fully attributable diagnostics rather than a single opaque score. Empirically, Qwen-Image-Bench reliably distinguishes leading T2I models, achieving the greatest separation on the two application-driven dimensions of Real-world Fidelity and Creative Generation where existing
benchmarks provide little insight, while also providing a trustworthy optimization signal for production-level T2I development.
Read the full abstract ▾
All authors Niantong Li, Guangzheng Hu, Weixu Qiao, Ying Ba, Qichen Hong, Shijun Shen, Jinlin Wang, Fan Zhou, Jianye Kang, Xin Shang, Ziyi He, Wei Wang, Dalin Li, Jiahao Li, Jie Zhang, Kaiyuan Gao, Kun Yan, Lihan Jiang, Ningyuan Tang, Shengming Yin, Tianhe Wu, Xiao Xu, Xiaoyue Chen, Yuxiang Chen, Yan Shu, Yanran Zhang, Yilei Chen, Yixian Xu, Zekai Zhang, Zhendong Wang, Zihao Liu, Zikai Zhou, Hongzhu Shi, Yi Wang, Bing Zhao, Hu Wei, Lin Qu, Chenfei Wu
Computer Vision
VLA-Hijack: A Transferable Patch Attack against Vision-Language-Action Models via Visual Proprioception Hijacking
Jiyuan Fu, Kaixun Jiang, Jingkai Jia et al. (+7 more)
While
Vision-
Language-Action (VLA) models have emerged as powerful generalist policies, their severe vulnerability to
adversarial patches significantly hinders their deployment in safety-critical domains. Moreover, existing patch attacks primarily focus on white-box settings, heavily overfitting to the specific action output space of the target model, which results in poor cross-architecture transferability. To overcome this limitation, we propose VLA-Hijack, a unified
adversarial framework that breaks the transferability bottleneck by exploiting a fundamental vulnerability identified in this work: before planning any motion, a VLA model must first use visual information to locate its own robotic arm within the environment. Targeting this shared visual self-localization process, our approach concurrently optimizes
Attention-Guided Proprioceptive Suppression to inhibit the real robotic arm's features, and
Multimodal Proprioceptive Injection to establish the patch as a surrogate "phantom embodiment". By alternating between semantic concept anchoring and visual prototype projection, VLA-Hijack effectively severs the semantic relationship between the
agent's true embodiment and its control policy. Extensive experiments across diverse architectures (OpenVLA, UniVLA, and CronusVLA) demonstrate that VLA-Hijack achieves superior optimization efficiency in white-box settings and sets a new SOTA for cross-architecture and cross-domain black-box transferability.
Read the full abstract ▾
All authors Jiyuan Fu, Kaixun Jiang, Jingkai Jia, Zhaoyu Chen, Xueyao Chen, Lingyi Hong, Shuyong Gao, Chenzhi Tan, Dingkang Yang, Wenqiang Zhang
Computer Vision
CogPortrait: Fine-Grained Eye-Region Control in Portrait Animation via Hierarchical Agent Planning
He Feng, Yongjia Ma, Donglin Di et al. (+2 more)
Portrait animation methods have achieved substantial visual quality and lip synchronization, but fine-grained manipulation of the eye region still faces a trade-off between input granularity and motion accuracy. Existing methods using
emotion labels or coarse text
prompts are insufficient for describing subtle ocular dynamics, whereas approaches based on Action Units or driving
videos provide higher fidelity at the cost of a heavier input burden. These limitations are still restrictive for beyond-
emotion states (e.g., thinking) and drowsiness. In light of the above, we propose CogPortrait, a two-stage framework that generates portrait animations from high-level labels. In the first stage, three chain-of-thought
Multimodal Large
Language Models (MLLMs)
agents compile high-level labels into facial keypoints through temporal event planning, prototype
retrieval, and composition from a real-behavior library, and semantic-physiological constraint enforcement. In the second stage, a DiT-based
video generation backbone synthesizes the final animation conditioned on the keypoints, reference portrait, audio, and text
prompt, enhanced by a dynamic classifier-free guidance strategy with eye-region-aware reweighting and KTO-based refinement for boundary cases. We further introduce the EMH
benchmark covering diverse
emotions and beyond-
emotion categories with two AU-level metrics for evaluating fine-grained eye-region and head-motion control. Extensive experiments on HDTF and the EMH
benchmark demonstrate that CogPortrait achieves more precise eye-region control than existing methods while maintaining supe- rior visual quality and identity consistency
Read the full abstract ▾
All authors He Feng, Yongjia Ma, Donglin Di, Lei Fan, Tonghua Su
Computer Vision
Beyond Surrogate Gradients: Fully Differentiable Token Pruning for Vision-Language Models
Landi He, Mingde Yao, Shawn Young et al. (+1 more)
Visual
token pruning reduces the computational cost of
Vision-
Language Models (VLMs) by removing redundant visual
tokens. Existing methods typically rely on Gumbel-Softmax to approximate discrete selection during training. However, the optimization is driven by surrogate gradients rather than the true selection process, leading to unreliable learning of
token importance. In this paper, we propose DiffPrune, which reformulates
pruning as continuous control of
token information instead of discrete selection learning. Specifically, we introduce an Information Throttler that modulates each
token using variance-preserving noise conditioned on importance scores, where higher scores induce less information suppression during training. This design directly operates on
token representations, naturally providing a fully differentiable optimization path for learning
token importance. At inference,
tokens are removed via hard thresholding on the learned scores. Across ten VLM
benchmarks, DiffPrune retains 96.5% of full-model accuracy while accelerating LLM prefill by 2.85x, with only 0.69 ms of inference overhead.
Read the full abstract ▾
All authors Landi He, Mingde Yao, Shawn Young, Lijian Xu
Computer Vision
Stay Fair! Ensuring Group Fairness in Diffusion Models Across Guidance Scales
Myeongsoo Kim, Eunji Kim, Minwoo Chae et al. (+1 more)
Diffusion models steer conditional generation with a tunable guidance scale to trade off
prompt alignment and diversity. However, existing debiasing techniques are optimized for a single scale, degrading
fairness when users adjust this parameter. We trace this behavior to a previously overlooked source by decomposing total bias into two components: a model bias and a guidance bias. While prior work primarily targets the former, we show that the guidance bias varies with the guidance scale, persisting even in models that have already been debiased. To address this, we extend Strong Demographic Parity to guidance and derive a condition under which the target distribution retains its group ratio across guidance scales. Building on this condition, we propose StayFair, which designs fair guidance
algorithms in both regimes. For classifier guidance, it equalizes the classifier's output distributions across groups; for classifier-free guidance, it shifts the null
embedding by a
prompt-dependent offset. Because StayFair modifies only the guidance step, it is orthogonal to model debiasing and can be layered onto existing fair
diffusion models. Across class-conditional and text-to-image generation, StayFair decouples
fairness from the guidance scale without sacrificing image quality. Our decomposition also enables a more detailed analysis of the sources of bias in
diffusion models.
Read the full abstract ▾
All authors Myeongsoo Kim, Eunji Kim, Minwoo Chae, Sangwoo Mo
Computer Vision
VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning
Xingyu Lu, Jinpeng Wang, Yi-Fan Zhang et al. (+13 more)
Visual captioning requires models to capture visual content faithfully while minimizing both omission and
hallucination. As the dominant paradigm for captioning, MLLMs have achieved strong performance through scaling and high-quality data. Recently, RL has emerged as a key route to driving MLLMs toward higher precision and broader coverage, however, existing reward designs for captioning fail to provide fine-grained and reliable signals for factual verification, limiting their effectiveness. To address this, we propose VCap, a Witness-Adjudicator reward that pairs the reference caption (a witness) with the visual signal (an adjudicator). By explicitly verifying factual consistency between the reference and policy-generated captions grounded in the visual signal, VCap delivers a reward signal with hypergeometric-distribution-level precision for caption quality verification. This design enables effective learning even from imperfect references, facilitating weak-to-strong generalization in RL training. In our experiments, an 8B model trained with VCap outperforms open- and closed-source SOTA models on multiple image and
video captioning
benchmarks. Human evaluation further confirms its strong
alignment with factual correctness. Additionally, VCap improves MLLM perceptual capability, generalizes across tasks, and surpasses best-of-N
distillation, challenging prior assumptions about RLVR.
Read the full abstract ▾
All authors Xingyu Lu, Jinpeng Wang, Yi-Fan Zhang, Yankai Yang, Yancheng Long, Yiyang Fan, Xuanyu Zheng, Haonan Fan, Kaiyu Jiang, Tianke Zhang, Changyi Liu, Bin Wen, Fan Yang, Tingting Gao, Han Li, Chun Yuan
Computer Vision
Dual-branch Distilled Transformer for Efficient Asymmetric UAV Tracking
Hongtao Yang, Bineng Zhong, Qihua Liang et al. (+4 more)
Given the real-time demands of
UAV tracking, many methods simplify the backbone to reduce computation, but this often weakens feature representation and degrades performance in complex scenarios. To alleviate this issue, we propose EATrack, an efficient and asymmetric
UAV tracking framework centered around a teacher-guided dual-branch
distillation strategy that enhances the feature expressiveness of the lightweight student model. Specifically, EATrack investigates two complementary perspectives of knowledge transfer: spatially focused feature-level
distillation that compensates for weakened representations by guiding the student to learn strong target representations, and prediction-level
distillation that enhances spatial localization by learning the teacher's capability for accurate target localization. Furthermore, to enhance robustness against appearance variations, we introduce a fine-grained target-aware
distillation strategy that selectively transfers the teacher's target modeling capacity to the student. A temporal adaptation module is incorporated at inference to enhance robustness over time. Experiments on five
UAV benchmarks demonstrate that EATrack achieves a favorable balance between accuracy and speed. Code: https://github.com/GXNU-ZhongLab/EATrack
Read the full abstract ▾
All authors Hongtao Yang, Bineng Zhong, Qihua Liang, Yaozong Zheng, Xiantao Hu, Yuanliang Xue, Shuxiang Song
Computer Vision
Enhancing Ultra-low-field MRI with Segmentation-guided Adversarial Learning
James Grover, Andrew Phair, Michael Ferraro et al. (+1 more)
Ultra-low-field (ULF) MRI offers portable and low-cost imaging but suffers from poor image quality. To address this, we present our submission to the 2025 ULF Enhancement Challenge (ULF-EnC), where the goal is to synthesise high-field-like MRIs from 64 mT scans. Our pipeline enhances ULF MRI through a combination of anatomical conditioning and model ensembling. We first generate tissue
segmentation priors using a Swin UNETR trained solely on challenge-provided data. These priors condition two independent enhancement networks - a CycleGAN and a
transformer-based residual enhancement model (T-REX) - each trained to synthesise 3 T-like MRIs. Outputs from both models are combined using a weighted average. Our approach produces enhanced MRIs that were comparable to high-field scans both quantitatively and qualitatively.
Read the full abstract ▾
All authors James Grover, Andrew Phair, Michael Ferraro, David E.J. Waddington
Computer Vision
Automated Estimation of Impact Time, Impact Location, and Shuttlecock Speed in Badminton Smashes Using Event Cameras
Yudai Washida, Yuto Kase, Kai Ishibe et al. (+2 more)
Quantifying impact phenomena in badminton smashes is important for evaluating both athletic performance and equipment; however, conventional measurement systems involve trade-offs between temporal resolution, data efficiency, and preparation effort. This study proposes a measurement method using two synchronized event cameras to automatically estimate impact time, impact location on the racket face, and post-impact shuttlecock speed in an integrated manner within the same trial. The swing interval was detected from event rate statistics, impact time was estimated from the shuttlecock trajectory inflection in the lateral-view event data, impact location was determined by ellipse fitting to the racket face in the rear-view event image, and shuttlecock speed was calculated in the sagittal plane. To validate the proposed method, Bland-Altman analysis was performed against a high-speed camera-based reference method using 125 smash trials from five players. Impact time and shuttlecock speed were estimated in all 124 analyzable trials, and impact location was estimated in 93.5% (116/124). The bias (95% CI) for impact time, medio-lateral impact location, longitudinal impact location, and shuttlecock speed were 1.84 ms (1.45 to 2.23), 3.45 mm (2.18 to 4.72), -1.92 mm (-2.97 to -0.88), and -1.00 m/s (-2.46 to 0.46), respectively. No proportional bias was observed for any metric. These results suggest that the proposed method can serve as a useful tool for integrated assessment of badminton smash performance and equipment in practical settings.
Read the full abstract ▾
All authors Yudai Washida, Yuto Kase, Kai Ishibe, Ryoma Yasuda, Sakiko Hashimoto
Computer Vision · Cross-listed from cs.LG
Geometry-Correct Diffusion Posterior Sampling with Denoiser-Pullback Curvature Guidance and Manifold-Aligned Damping
Seunghyeok Shin, Minwoo Kim, Dabin Kim et al. (+1 more)
Diffusion posterior sampling conditions
diffusion priors on measurements, but data-consistency updates are typically scaled by hand-tuned guidance weights and can destabilize sampling under stiff, operator-dependent
curvature. We replace scalar guidance with a per-noise-level damped Gauss–Newton correction computed in
diffusion-state coordinates. The correction pulls likelihood gradients back through the denoiser, uses a one-sided
curvature model that avoids forward denoiser Jacobians, and applies
diffusion-calibrated rank-one damping aligned with the denoiser residual. Each correction is solved with
matrix-free GMRES using automatic differentiation, and sampling proceeds with a variance-preserving Langevin transition with a closed-form drift/noise split. On FFHQ and ImageNet across inverse problems, it achieves competitive PSNR/SSIM/LPIPS while running markedly faster than most of the compared baselines; on accelerated MRI reconstruction, it achieves the best PSNR/SSIM among the compared baselines.
Read the full abstract ▾
All authors Seunghyeok Shin, Minwoo Kim, Dabin Kim, Hongki Lim
Computer Vision
ABot-OCR Technical Report
Kaitao Jiang, Ruiyan Gong, Xiaolong Cheng et al. (+3 more)
We introduce ABot-OCR, an end-to-end
vision-
language model that transcribes a page image directly into clean Markdown in a single forward pass. By doing so, our approach completely eliminates the need for brittle modular orchestration. To maximize parsing fidelity, we develop a dedicated data engine to provide large-scale, structurally consistent supervision. Furthermore, we propose Decoupled Heterogeneous Document Optimization, a structure-constrained reinforcement learning method that sharpens textual accuracy and strictly enforces markup well-formedness beyond supervised fine-tuning alone. Extensive evaluations demonstrate the superior performance of our framework. On the OmniDocBench v1.5 and v1.6
benchmarks, ABot-OCR achieves state-of-the-art scores of 92.81 and 93.30 among all end-to-end systems, substantially narrowing the performance gap relative to strong pipeline baselines. Finally, comprehensive
multilingual text recognition across ten diverse
languages further confirms the robust generalizability of ABot-OCR.
Read the full abstract ▾
All authors Kaitao Jiang, Ruiyan Gong, Xiaolong Cheng, Kangning Niu, Tianlun Li, Mu Xu
Computer Vision
Bridging the Generalization Gap in Adverse Weather Segmentation: A Training Recipe Perspective
Cong Xu, Pu Luo, Yumei Li et al. (+1 more)
This paper describes our approach for the 8th UG2+ Workshop (CVPR 2026) Track~2, which targets semantic
segmentation of outdoor scenes degraded by five weather conditions: blur, darkness, snow, haze, and glare. A central challenge we observe is a severe generalization gap – models that perform well on the validation set often collapse on the test set. For instance, SegFormer-B5 drops 16.1 mIoU points from validation to test, suggesting that model capacity alone is insufficient for robustness. We investigate whether a carefully designed training recipe, rather than architectural complexity, can address this gap. Starting from a pre-trained SegMAN-S backbone, we systematically study the effects of domain-adaptive fine-tuning, multi-source data mixing, scene-balanced sampling, and synthetic degradation augmentation. Our final system achieves 59.9% mIoU on the official test set while maintaining a validation-test gap of only 6.5 points – less than half that of larger models. We analyze negative results from architectural modifications, loss function variants, and model scaling to provide practical insights for weather-robust
segmentation under limited data.
Read the full abstract ▾
All authors Cong Xu, Pu Luo, Yumei Li, Boyou Xue
Computer Vision
Mags-RL: Wearing Multimodal LLMs a Magnifying Glass via Agentic Reinforcement Learning For Complex Scene Reasoning
Xuanzhao Dong, Wenhui Zhu, Peijie Qiu et al. (+11 more)
Despite their popularity and success,
Multimodal Large
Language Models (MLLMs) often struggle to interpret images accurately, which limits their
reasoning capability in complex scenarios (e.g., high object density and complex background clutter). Prior work mainly addresses this limitation by incorporating explicit visual cues like bounding boxes that require extra annotations. In addition, the resulting low-resolution crops often miss fine-grained details that MLLMs require for accurate
reasoning. Therefore, we propose Mags-RL, an Agentic Reinforcement Learning (RL) framework that equips MLLMs with an external super-resolution "magnifying glass"
agent for high-resolution fine-grained inspection. Specifically, the model performs two-round
reasoning: in the first round, it generates an initial rationale and autonomously identifies regions of interest without relying on additional annotations; in the second round, it invokes a super-resolution
agent to crop and upscale those regions, then revisits and verifies its earlier
reasoning to produce the final answer. We also introduce a novel curriculum learning strategy that enables data-efficient RL training, needing as few as only 40 training samples to achieve reasonable performance. Experiments on VSR, TallyQA, and GQA subsets show its superior performance against recent strong competing methods, demonstrating high-quality
reasoning with precise visual grounding. Code and weights will be released soon.
Read the full abstract ▾
All authors Xuanzhao Dong, Wenhui Zhu, Peijie Qiu, Xiwen Chen, Xiaobing Yu, Xin Li, Zhipeng Wang, Shao Tang, Gen Li, Yujian Xiong, Hao Wang, Yanxi Chen, Prayag Tiwari, Yalin Wang
Computer Vision
ROVER: Routing Object-Centric Visual Evidence for Grounded Multi-Image Reasoning
Guannan Lv, Ren Nie, Hongjian Dou et al. (+1 more)
Multimodal Large
Language Models (MLLMs) have increasingly localized and interleaved visual evidence for deliberative
reasoning. Grounding-based approaches typically focus on regions of interest (RoIs) by injecting cropped image patches or RoI-specific features into the
reasoning context. However, such designs can weaken holistic scene understanding and inter-object relations, while incurring decoding costs that scale with the number and size of RoIs. Alternatively, adaptive visual feature selection often requires fine-grained supervision or complex heuristics. To address these limitations, we propose ROVER (Routing Object-centric Visual Evidence for grounded multi-image
Reasoning), a lightweight, learnable plugin for efficient global visual evidence routing. Upon each object grounding prediction, ROVER injects a step-specific
token triplet to synergistically: (i) aggregate the ongoing
reasoning context, (ii) distill intra-image cues into a visual working space via object-centric differential
attention, and (iii) route and integrate history-aware evidence across objects and images within this space for subsequent
reasoning. We integrate ROVER into Qwen2.5-VL-7B and develop an interleaved SFT-to-GRPO training pipeline. Strictly adhering to the original
datasets and evaluation protocols, our method achieves the best performance on MM-GCoT (+4.8% answer accuracy, +14.6% grounding accuracy) and VideoEspresso (+8.6% answer accuracy). The VideoEspresso-trained model demonstrates strong transferability, outperforming the base model by +4.7% on average across diverse
benchmarks.
Read the full abstract ▾
All authors Guannan Lv, Ren Nie, Hongjian Dou, Tingting Gao
Computer Vision
Con-DSO: Learning Short-Horizon Consistency Priors for RGB-D Direct Sparse Odometry
Haolan Zhang, Thanh Nguyen Canh, Chenghao Li et al. (+3 more)
RGB-D direct visual odometry (VO) benefits from metric depth measurements but often degrades in the presence of dynamic objects, occlusions, illumination changes, and unreliable depth, which violate the photometric and
geometric consistency assumptions of direct
alignment. We propose Con-DSO, a consistency-aware RGB-D direct sparse odometry framework that addresses these challenges through a unified learned uncertainty model. A dual-branch consistency network is trained on adjacent RGB-D frame pairs using flow-guided photometric errors and projective depth-consistency errors to predict pixel-level photometric and
geometric uncertainty. The predicted uncertainty is first converted into pairwise quality to guide support-pixel selection and is then fused across adjacent frame pairs to form a host-side quality prior for keyframe-based tracking. To account for the different roles of photometric and depth information in direct RGB-D optimization, the quality prior is incorporated through a decoupled photometric-
geometric weighting scheme, with the
geometric weight applied only to the translational component of pose estimation. Experiments on five public RGB-D
benchmarks demonstrate consistent improvements over direct RGB-D odometry baselines, achieving more than 20% reduction in absolute trajectory error on ICL-NUIM and approximately 50% to 80% reductions on RGB-D Scenes V2, TUM/BONN, and OpenLORIS. These results demonstrate that learned consistency-aware uncertainty can substantially improve the robustness of RGB-D direct visual odometry in challenging environments.
Read the full abstract ▾
All authors Haolan Zhang, Thanh Nguyen Canh, Chenghao Li, Ziyan Gao, Xiongwen Jiang, Nak Young Chong
Computer Vision
Evaluating the Feasibility of Inferring Dietary Behavior Change Receptivity from Egocentric Images of Eating Environment
Long Li, Yuning Huang, Heather A. Eicher-Miller et al. (+3 more)
Accurately assessing dietary behavior change receptivity is essential for designing effective just-in-time adaptive interventions (JITAIs) that promote healthier eating habits. However, self-report-based assessment of behavior change receptivity is sparse and delayed, limiting its practical use in continuous monitoring. To explore whether passive sensing may help address this challenge, this study conducts a pilot investigation of inferring participants' self-reported behavior change receptivity from
egocentric eating images collected by a wearable camera. We use pilot data obtained from free-living eating episodes using the Automatic Ingestion Monitor v2 (AIM-2). The data included
egocentric image sequences captured during eating and paired with responses to questions assessing specific dimensions of behavior change receptivity (awareness, interaction capability, and motivation). To examine whether visual information contained any relevancy to these responses, we evaluated a transfer-learning-assisted framework that combines a pre-trained Contrastive
Language-Image Pre-Training (CLIP)
vision encoder with a lightweight
transformer classifier. The model processes eating episode image sequences to extract potential semantic and temporal cues related to behavior change receptivity. Preliminary experimental results show promising improvements over simple baseline models for behavior change receptivity indicators. These early findings suggest that
egocentric eating episode images may contain cues related to dietary behavior change receptivity, and warrant further investigation with larger and more comprehensive
datasets.
Read the full abstract ▾
All authors Long Li, Yuning Huang, Heather A. Eicher-Miller, J.Graham Thomas, Fengqing Zhu, Edward Sazonov
Computer Vision
SEMAGIC: Learning Semantically Consistent Deformable 3D Representations from In-the-Wild Images
Sky Cen, Wufei Ma, Guofeng Zhang et al. (+2 more)
Learning deformable 3D object models from single-view in-the-wild images has enabled impressive 3D shape reconstruction without supervision. However, it remains unclear whether these models capture the semantic structure required for downstream tasks. We find that existing deformable reconstruction approaches, despite producing visually plausible geometry, yield unstable correspondences across instances and perform poorly on semantic correspondence
benchmarks. We introduce SEMAGIC, a framework for learning semantically consistent deformable 3D representations from single-view in-the-wild images. Rather than treating reconstruction as the end goal, SEMAGIC uses deformable modeling as a mechanism to discover category-level correspondences. Each category is represented by a canonical template mesh and a learned deformation field, functioning similarly to an autoencoder that reconstructs instance geometry from image features, enabling vertices to maintain consistent semantic meaning across instances. Semantic consistency is enforced during training through (i) a feature-level consistency loss aligning semantic features between canonical and deformed meshes, and (ii) vertex-index-conditioned deformation that preserves semantic correspondence across instances. By explicitly coupling
geometric deformation with semantic
alignment, SEMAGIC produces representations that maintain stable part correspondences across intra-category variation. Experiments demonstrate that SEMAGIC improves semantic correspondence of deformable models by +14.7 PCK@0.1 on SPair-71k, establishing deformable models as effective semantic 3D representations.
Read the full abstract ▾
All authors Sky Cen, Wufei Ma, Guofeng Zhang, Alan Yuille, Adam Kortylewski
Computer Vision
When Think-with-Image Meets Safety: What Determines Multimodal Jailbreak Robustness?
Yuan Tian, Bing Hu, Fang Wu et al. (+3 more)
Think-with-image
reasoning is emerging as a new inference paradigm for large
vision-
language models, but its safety implications remain poorly understood. Existing systems already span multiple process designs, including direct response generation, text-only prior turn, visual-state manipulation, and explicit external image-tool invocation. In this paper, we ask which of these evaluated paradigms improves
multimodal jailbreak robustness, and why. Across multiple
vision-
language models, explicit image-tool interaction yields the lowest attack success rates in our experiments, reducing jailbreak success by around 30% relative on average across the evaluated models. This finding is initially surprising: ASR remains low even when the returned image-tool output is manually overridden or itself unsafe-looking, but returns near direct-answering levels under text-only prior turn controls. These results indicate that the lower ASR is not explained by benign returned-image semantics or by the textual image-tool trace alone. To explain the pattern, we introduce an image-tool safety vector framework that models image-tool invocation as a residual shift in hidden representations toward a safety-relevant direction. Representation-level analyses and activation interventions support this account. Overall, our results suggest that explicit image-tool interaction is a promising design pattern for improving jailbreak robustness, while also motivating pipeline-specific safety evaluation.
Read the full abstract ▾
All authors Yuan Tian, Bing Hu, Fang Wu, Xiaomin Li, Binghang Lu, Neil Zhenqiang Gong
Computer Vision
Structure-Guided Visual Perturbation Neutralization for LVLMs
Yuanhe Zhang, Xueting Wang, YanBin Ren et al. (+6 more)
Image inputs enable Large
Vision Language Models (LVLMs) to perceive fine-grained visual information, but also introduce a pixel-level attack surface through which
adversarial perturbations can elicit unsafe model behaviors. However, most existing defenses are designed for traditional computer
vision settings and thus often overlook the cross-modal
alignment required by LVLMs, leading to degraded performance. Meanwhile, the limited defenses tailored to LVLMs often require substantial image modifications and introduce considerable computational overhead, thereby compromising inference quality and efficiency. To address these limitations, we propose Structure-Induced Guided Neutralization (SIGN), a lightweight, plug-and-play defense framework that improves LVLM compatibility via Prior Structural Extraction and achieves efficient perturbation suppression via Dynamic Guided Neutralization. Extensive experiments show that SIGN achieves over 87% defense success rate with only 0.5% pixel modification and 0.16 seconds per image, while nearly preserving original visual representations and benign task performance. Our work offers a lightweight alternative to defenses that require costly model training and highlights the potential of exploiting a
vision encoder for efficient
adversarial protection. Our code is open source on https://anonymous.4open.science/r/SIGN-BCB1.
Read the full abstract ▾
All authors Yuanhe Zhang, Xueting Wang, YanBin Ren, Haoran Gao, Xinhan Zheng, Zhenhong Zhou, Fanyu Meng, Li Sun, Sen Su
Computer Vision
SIGMA: Semantic-Difference Instruction-Grounding Mask Annotator for Text-Driven Image Manipulation Localization
Peiyu Zhuang, Jianquan Yang, Haodong Li et al. (+6 more)
Text-driven image editing has advanced rapidly, but reliably localizing these manipulations requires image manipulation localization (IML) models trained on large pixel-annotated
datasets, and there is still no low-cost way to obtain such training data at scale. We observe that these data already exist in disguise: public editing
datasets contain millions of structurally identical $\textit{(original, edited)}$ pairs to IML training samples, lacking only pixel-level masks. Recovering these masks automatically is non-trivial: pixel differencing is overwhelmed by
diffusion-induced perturbations across all pixels, and instruction-only grounding localizes only what the
prompt describes, missing unintended editor side-effects. We propose $\textbf{SIGMA}$ ($\textbf{S}$emantic-difference $\textbf{I}$nstruction-$\textbf{G}$rounding $\textbf{M}$ask $\textbf{A}$nnotator), which performs semantic-feature differencing in a
vision foundation backbone and injects an instruction-derived spatial prior into this visual stream via bidirectional cross-modal refinement, amplifying the difference signal at intended-edit regions when the editor faithfully realizes user intent. SIGMA is trained in two complementary stages: Stage I supervises on inpainting masks and Stage II closes the
diffusion-domain shift via VAE-roundtrip noise calibration, EMA self-training, and an edit-noise disentanglement loss. SIGMA outperforms existing automatic mask generators on five
benchmarks ($\textbf{+12.20\%}$ F1, $\textbf{+11.16\%}$ IoU). When applied to public editing corpora, it produces a $\sim$1.1M IML training set that improves four diverse detectors by $\textbf{+20.98\%}$ F1 across five
datasets, turning previously unused editing data into a model-agnostic supervisory resource for IML. For reproducibility, our project page is available at \url{https://zhuangpeiyu.github.io/projects/SIGMA/}.
Read the full abstract ▾
All authors Peiyu Zhuang, Jianquan Yang, Haodong Li, Zhuoying Cai, Ruitao Xie, Jishen Zeng, Baoying Chen, Jiwu Huang, Xiaochun Cao
Computer Vision
Do We Really Need Quantum Machine Learning?: A Multidimensional Empirical Study
Sudip Vhaduri, Ryan Gammon and Sayanton Dibbo
The rapid growth of computer
vision and increasingly complex image recognition tasks has exposed fundamental computational limitations of classical machine learning models, motivating the exploration of
quantum computing as an emerging new paradigm. This paper presents a comprehensive benchmarking study of classical and
quantum machine learning models for image recognition on the MNIST handwritten digit
dataset, evaluating both traditional models, a Classical Support Vector Machine (CSVM) and a
Quantum Support Vector Machine (QSVM), and deep neural network models, a Classical Convolutional Neural Network (CCNN) and a
Quantum Convolutional Neural Network (QCNN), across four performance dimensions: classification accuracy, computational runtime, parameter count, and
memory requirements. Experiments are conducted as functions of both feature dimensionality and sample size, and across CPU and GPU execution environments, providing a controlled, multidimensional comparison to address gaps in prior work. For the SVM-based models, QSVM consistently outperforms CSVM in accuracy, reaching $\sim$ 0.90 versus $\sim$ 0.85 at 1,000 samples, with a higher computational cost. A feature count of 10
qubits and a sample size in the range of 200 – 500 emerge as practical operating points that balance accuracy and runtime. For the neural network models, CCNN and QCNN achieve comparable classification accuracy, both exceeding 0.96 at 64 features and 60,000 samples, yet QCNN offers superior parameter and
memory efficiency at higher feature counts, while incurring higher runtime. Across both model families,
quantum models consistently outperform classical models by greater margins in accuracy as feature dimensionality or sample size increases.
Read the full abstract ▾
Computer Vision
Rethinking Video-Language Model from the Language Input Perspective
Xiang Fang, Wanlong Fang, Changshuo Wang et al. (+2 more)
Driven by the wave of large
language models,
Video-
Language Models (VLMs) have become a significant yet challenging technology to bridge the gap between
videos and texts. Although previous VLM works have made significant progress, almost all of them implicitly assume that all the texts are predefined by the specific template. In real-world applications, such a strict assumption is impossible to satisfy since 1) predefining all the texts is extremely time-consuming and labor-intensive. 2) these predefined text inputs are too restrictive and user-unfriendly, limiting their applications. It is observed that given a
video input, texts with similar semantics but different templates lead to various performances. To this end, in this paper, we propose a novel plug-and-play framework for various VLM-based methods to fully bridge
videos and texts. Specifically, we first generate positive and negative texts from the original ones to target specific text components. Then, we propose an attribute-based text
reasoning strategy to mine fine-grained textual semantics of generated texts. Finally, we utilize
videos as guidance to conduct cross-modal bridging by designing a self-weighted loss. Extensive experiments show that the proposed method can serve as the plug-and-play module to effectively improve the performance of state-of-the-art VLMs.
Read the full abstract ▾
All authors Xiang Fang, Wanlong Fang, Changshuo Wang, Xiaoye Qu, Daizong Liu
Computer Vision
OphIn-500K: Curating Web-Scale Visual Instructions for Scaling Ophthalmic Multimodal Large Language Models
Xuanzhao Dong, Wenhui Zhu, Xiwen Chen et al. (+13 more)
The advancement of general medical
Multimodal Large
Language Models (MLLMs) has shown great potential for building conversational assistants to support clinical diagnosis. However, their adaptation to highly specialized domains such as ophthalmology remains underexplored, primarily due to the scarcity of large-scale, domain-specific instruction-tuning data. Existing ophthalmic
datasets for conversational
agents are often limited in scale and largely rely on images from established public
benchmarks, limiting the scalability of ophthalmic MLLMs and their ability to capture real-world clinical complexity. To address this gap, we propose $\textbf{OphIn-Engine}$, an ophthalmology-specific instruction data curation pipeline that constructs high-quality instruction data from open-access ophthalmology web-scale
videos. The pipeline integrates
multimodal transcription for extracting image-transcript pairs, visual cue separation and scoring for identifying clinically relevant visual descriptions, and instruction synthesis with quality control for generating accurate and diverse clinical dialogues. Using this engine, we introduce $\textbf{OphIn-500K}$, a large-scale
multimodal ophthalmology instruction-tuning
dataset containing over 500,000 instruction instances and more than 151,000 unique images from over 29,000
video clips, formatted as visual question answering (VQA), multi-turn conversational interactions, and chain-of-thought (CoT)
reasoning. Built upon this
dataset, we further develop $\textbf{OphIn-VL}$, an ophthalmology-specific MLLM with advanced visual understanding and conversational capabilities. Comprehensive experiments and case studies demonstrate that OphIn-VL achieves superior performance compared with state-of-the-art general medical and domain-specific MLLMs.
Read the full abstract ▾
All authors Xuanzhao Dong, Wenhui Zhu, Xiwen Chen, Hao Wang, Xin Li, Yujian Xiong, Jiajun Cheng, Jingjing Wang, Xiaobing Yu, Haiyu Wu, Shao Tang, Zhipeng Wang, Langechuan Liu, Shan Lin, Oana Dumitrascu, Yalin Wang
Computer Vision
Decoupled Training with Local Reinforcement Fine-Tuning in Federated Learning
Yuting Ma, Lechao Cheng and Xiaohua Xu
Federated Learning (FL) with pre-trained
Vision-
Language Models (VLMs) has emerged as a promising paradigm for various downstream tasks. By leveraging its strong representations, recent studies improve task adaptation under insufficient local data while preserving generalization. However, these methods emphasize fully local optimization with simple parameter aggregation,which can amplify inter-client optimization inconsistency and intra-client over-specialization under heterogeneous and full-data FL settings, making it difficult to balance global task adaptation and generalization. To address these challenges, we propose FedDTL, a novel
federated VLM framework that decouples the image encoder and text encoder across clients and the server. Through decoupled encoder training with server-client modality
alignment, FedDTL promotes coherent global semantic update and reduces inter-client optimization inconsistency, improving global task adaptation.To further mitigate intra-client over-specialization,we introduce a two-stage local fine-tuning, where a supervised fine-tuning stage enables rapid and reliable warm-start, followed by a reinforcement learning stage that enhances generalization. Extensive experiments on multiple
benchmarks, including label skew and feature shift, demonstrate that FedDTL achieves an effective balance between global task adaptation and generalization under various FL data distributions in both few-shot and full-data regimes.
Read the full abstract ▾
Computer Vision
Towards Unified Vision-Language Models with Incomplete Multi-Modal Inputs
Xiang Fang, Wanlong Fang, Changshuo Wang et al. (+4 more)
Video-
Language Models (VLMs) have demonstrated impressive multi-modal
reasoning capabilities across diverse computer
vision applications. However, these VLMs are task-specific and assume that both
video and
language inputs are complete. However, real-world VLM applications might face challenges due to deactivated sensors (e.g., cameras are unavailable due to data
privacy), yielding modality-incomplete data and leading to inconsistency between training and testing data. While straightforward incomplete input can boast training generalization-ability and lead to training failure, its potential risks to VLMs regarding safety and trustworthiness have been largely neglected. To this end, we make the first attempt to propose a unified incomplete
video-
language model to process the incomplete multi-modal inputs. Extensive experimental results show that our method can serve as a plug-and-play module for previous works to improve their performance in various multi-modal tasks.
Read the full abstract ▾
All authors Xiang Fang, Wanlong Fang, Changshuo Wang, Keke Tang, Daizong Liu, Siyi Wang, Wei Ji
Computer Vision
SIGMA: Bridging Structural and Distributional Gaps for Vision Foundation Model Adaptation
Lingyu Xiong, Jinjin Shi, Xuran Xu et al. (+3 more)
Vision Foundation Models (VFMs) have demonstrated impressive representational capabilities. However, adapting them to downstream tasks via full fine-tuning incurs prohibitive computational and storage overhead. Parameter-Efficient Fine-Tuning (PEFT) has emerged as a compelling alternative, aiming to achieve performance parity with full fine-tuning at minimal training costs. Nonetheless, applying PEFT to VFMs for dense prediction tasks remains challenging due to the structural and distributional gaps. To bridge these gaps, we propose Scale-Integrated Global Modulation Adapter (SIGMA), a novel lightweight PEFT method, which consists of two modules: scale-adaptive fusion and semantic modulation. Specifically, the scale-adaptive fusion module is utilized to bridge structural gaps by enhancing the extraction of multi-granularity visual information. Furthermore, SIGMA introduces semantic modulation on the fusion features to perform global feature
alignment to further eliminate the distribution gap. This design facilitates unified spatial and distributional adaptation, requiring only 1.72% trainable parameters relative to the VFM backbone. Comprehensive experiments across various downstream dense tasks and multiple VFM backbones demonstrate that SIGMA achieves consistent and superior performance over state-of-the-art PEFT methods.
Read the full abstract ▾
All authors Lingyu Xiong, Jinjin Shi, Xuran Xu, Cong Luo, Runyu Shi, Ying Huang
Computer Vision
SmartDirector: Keyframe-Conditioned Cinematic Video Generation with Narrative Pacing Control
Zhida Zhang, Jie Ma, Zhan Peng et al. (+5 more)
The narrative quality of a
video fundamentally determines its perceptual value. Although existing
video generation methods can produce visually appealing content, they predominantly rely on sparse conditioning signals such as text
prompts or first/last frames, which limits precise control over narrative structure and temporal pacing. In this paper, we propose SmartDirector, a framework that enhances the narrative capacity of
video generation models through multiple keyframes. SmartDirector supports flexible generation scenarios including single-shot generation, multi-shot narrative synthesis, and
video extension. The framework operates in two stages: Director-Gen generates a low-resolution
video conditioned on the provided keyframes, and Director-SR refines the output by exploiting high-resolution keyframes as semantic anchors to recover fine-grained details. To enable robust multi-keyframe training, we construct a data pipeline that curates single-shot and multi-shot sequences from movies. Extensive experiments demonstrate that SmartDirector substantially outperforms existing state-of-the-art approaches. We will release the code to facilitate further research.
Read the full abstract ▾
All authors Zhida Zhang, Jie Ma, Zhan Peng, Haoxue Wu, Yang Han, Jun Liang, Jie Cao, Jing Li
Computer Vision
Reflective Dialogue between Teacher and Solver Agents for Video Question Answering
Takuya Murakawa and Toru Tamaki
Various approaches have been proposed to adapt
Vision-
Language Models (VLMs) to specialized domains for
Video Question Answering, including fine-tuning and in-context learning. However, acquiring task-specific knowledge at the inference phase from only a small labeled support set without fine-tuning remains a challenge. In this paper, we propose a method that achieves adaptation solely through inference-time context injection. Our method first constructs a Reflective Dialogue (RD) – a multi-turn conversation between two
agents, in which Teacher poses each support question and delivers correctness feedback, and Solver answers and provides visual grounding explanations (or reflections) for both correct and incorrect answers. This dialogue history is then used as context at the inference phase. Experiments on the EgoCross
benchmark demonstrate that our method outperforms both a baseline zero-shot setting and a standard in-context learning approach that passes support set examples directly, achieving 3rd place in the Open-source Track of the 1st Cross-Domain EgoCross Challenge at the CVPR 2026 EgoVis Workshop, for which this paper also serves as a technical report.
Read the full abstract ▾
Computer Vision
A Road-Conditioned Traffic Movie Prediction Network with Spatiotemporal and Structure-Consistent Learning
Joshua Kofi Asamoah, Blessing Agyei Kyem and Armstrong Aboah
City-wide traffic
forecasting is important for congestion management, route guidance, and intelligent transportation systems, but accurate prediction remains challenging when future traffic must be generated as spatial maps over an entire urban network. Existing traffic movie prediction methods have improved frame-level accuracy, yet many still treat
forecasting mainly as image reconstruction. This can produce traffic maps that are numerically close to the ground truth but weakly constrained by road layout, connectivity, travel direction, and congestion propagation, especially in cross-city settings where both traffic behavior and road structure change. To address this limitation, this study proposes RCSNet, a road-conditioned spatiotemporal network that reformulates traffic movie prediction as
topology-guided future-state generation. RCSNet extracts road-aware representations from static road maps, models multi-horizon traffic dynamics from historical observations, aligns directional traffic features with local road structure, and progressively generates future traffic maps for improved temporal consistency. A structure-consistent learning objective further encourages predictions to remain accurate, road-aligned, and spatially stable. Experiments across multiple cities show that RCSNet improves both
forecasting accuracy and structural consistency. In same-city
forecasting on Berlin, Antwerp, and Moscow, RCSNet reduces average MAE, MSE, and RMSE by 11.5%, 10.0%, and 5.1%, respectively, compared with the closest baseline. In cross-city testing on unseen Chicago and Bangkok, it reduces RMSE by 10.6% and 10.5% without target-city fine-tuning. Additional horizon-wise, road-structure, explainability, statistical, and efficiency analyses show that RCSNet produces more accurate, transferable, road-aligned, and computationally efficient traffic forecasts.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
ClothTransformer: Unified Latent-Space Transformers for Scalable Cloth Simulation
Yu Zhang, Yidi Shao, Wenqi Ouyang et al. (+5 more)
Unified and scalable
Transformers have recently achieved remarkable success in modeling diverse phenomena traditionally associated with computer graphics, such as 3D visual effects, rendering processes, and motion in
videos. In this work, we take a step further by investigating whether modern
Transformer techniques can tackle the challenging task of cloth simulation. To this end, we present ClothTransformer, a framework that reformulates cloth simulation as autoregressive sequence modeling in a learned latent space. Existing neural cloth simulators are largely specialized to single scenarios, intrinsically coupled to the mesh discretization, and lack robust collision handling. Our approach addresses these limitations through three contributions: (1) a unified
Transformer architecture that handles diverse scenarios – body-driven garments, robotic manipulation, and free-fall collisions – under a single model and achieves approximately $4$–$9{\times}$ lower error than prior state-of-the-art methods across all scenarios; (2) a scalable latent-space formulation that compresses arbitrary-resolution meshes into a fixed-size set of latent
tokens, making temporal dynamics computation independent of mesh resolution; and (3) a diverse-scenario high-fidelity penetration-free
dataset of ${\sim}$493.4k frames spanning all three settings, which enables a differentiable Continuous Collision Detection (CCD) module to suppress penetration artifacts. Project Page: https://yucrazing.github.io/clothtransformer/
Read the full abstract ▾
All authors Yu Zhang, Yidi Shao, Wenqi Ouyang, Yushi Lan, Zhexin Liang, Chengrui Wu, Xudong Xu, Xingang Pan