Computer Vision
DanQing: An Up-to-Date Large-Scale Chinese Vision-Language Pre-training Dataset
Hengyu Shen, Tiancheng Gu, Bin Qin et al. (+10 more)
Vision-
Language Pre-training (VLP) models have achieved remarkable success by leveraging large-scale image-text pairs. While English-centric models like CLIP and SigLIP benefit from massive
datasets (e.g., LAION-400M), the development of Chinese VLP remains bottlenecked by the lack of high-quality, large-scale open-source data. In this paper, we present DanQing, a large-scale Chinese cross-modal
dataset containing 100 million high-quality image-text pairs curated from Common Crawl. To ensure superior data quality, we develop an effective systematic pipeline comprising data source selection, text refinement, visual diversification, and cross-modal cross-batch filtering, thereby effectively mitigating the intrinsic noise prevalent in web data. Notably, DanQing incorporates data from 2024-2025, enabling models to capture contemporary semantic trends and emerging concepts. Extensive experiments via continued pretraining of SigLIP2 models demonstrate that DanQing consistently outperforms existing Chinese
datasets across diverse downstream tasks, including zero-shot classification, cross-modal
retrieval, and Chinese-centric large
multimodal model tasks. Furthermore, in-depth analysis of DanQing reveals that it exhibits a more balanced semantic distribution and superior scaling capability compared to existing
datasets. To facilitate further research in Chinese
vision-
language pre-training, we will open-source the DanQing
dataset under the Creative Common CC-BY-NC 4.0 license.
Read the full abstract ▾
All authors Hengyu Shen, Tiancheng Gu, Bin Qin, Lan Wu, Yuling Wu, Shuo Tan, Zelong Sun, Jun Wang, Nan Wu, Xiang An, Weidong Cai, Ziyong Feng, Kaicheng Yang
Computer Vision · Cross-listed from eess.IV
Cell Behavior Video Classification Challenge, a benchmark for computer vision methods in time-lapse microscopy
Raffaella Fiamma Cabini, Deborah Barkauskas, Guangyu Chen et al. (+19 more)
The classification of microscopy
videos capturing complex cellular behaviors is crucial for understanding and quantifying the dynamics of biological processes over time. However, it remains a frontier in computer
vision, requiring approaches that effectively model the shape and motion of objects without rigid boundaries, extract hierarchical spatiotemporal features from entire image sequences rather than static frames, and account for multiple objects within the field of view. To this end, we organized the
Cell Behavior
Video Classification Challenge (CBVCC), benchmarking 35 methods based on three approaches: classification of tracking-derived features, end-to-end deep learning architectures to directly learn spatiotemporal features from the entire
video sequence without explicit
cell tracking, or ensembling tracking-derived with image-derived features. We discuss the results achieved by the participants and compare the potential and limitations of each approach, serving as a basis to foster the development of computer
vision methods for studying cellular dynamics.
Read the full abstract ▾
All authors Raffaella Fiamma Cabini, Deborah Barkauskas, Guangyu Chen, Zhi-Qi Cheng, David E Cicchetti, Judith Drazba, Rodrigo Fernandez-Gonzalez, Raymond Hawkins, Yujia Hu, Jyoti Kini, Charles LeWarne, Xufeng Lin, Sai Preethi Nakkina, John W Peterson, Koert Schreurs, Ayushi Singh, Kumaran Bala Kandan Viswanathan, Inge MN Wortel, Sanjian Zhang, Rolf Krause, Santiago Fernandez Gonzalez, Diego Ulisse Pizzagalli
Computer Vision
Attend to what I say: Highlighting relevant content on slides
Megha Mariam K M and C. V. Jawahar
Imagine sitting in a presentation, trying to follow the speaker while simultaneously scanning the slides for relevant information. While the entire slide is visible, identifying the relevant regions can be challenging. As you focus on one part of the slide, the speaker moves on to a new sentence, leaving you scrambling to catch up visually. This constant back-and-forth creates a disconnect between what is being said and the most important visual elements, making it hard to absorb key details, especially in fast-paced or content-heavy presentations such as conference talks. This requires an understanding of slides, including text, graphics, and layout. We introduce a method that automatically identifies and highlights the most relevant slide regions based on the speaker's narrative. By analyzing spoken content and matching it with textual or graphical elements in the slides, our approach ensures better synchronization between what listeners hear and what they need to attend to. We explore different ways of solving this problem and assess their success and failure cases. Analyzing multimedia documents is emerging as a key requirement for seamless understanding of content-rich
videos, such as educational
videos and conference talks, by reducing cognitive strain and improving comprehension. Code and
dataset are available at: https://github.com/meghamariamkm2002/Slide_Highlight
Read the full abstract ▾
Computer Vision
Optimizing Multimodal LLMs for Egocentric Video Understanding: A Solution for the HD-EPIC VQA Challenge
Sicheng Yang, Yukai Huang, Shitong Sun et al. (+4 more)
Multimodal Large
Language Models (MLLMs) struggle with complex
video QA
benchmarks like HD-EPIC VQA due to ambiguous queries/options, poor long-range temporal
reasoning, and non-standardized outputs. We propose a framework integrating query/choice pre-processing, domain-specific Qwen2.5-VL fine-tuning, a novel Temporal Chain-of-Thought (T-CoT) prompting for multi-step
reasoning, and robust post-processing. This system achieves 41.6% accuracy on HD-EPIC VQA, highlighting the need for holistic pipeline optimization in demanding
video understanding. Our code, fine-tuned models are available at https://github.com/YoungSeng/
Egocentric-Co-Pilot.
Read the full abstract ▾
All authors Sicheng Yang, Yukai Huang, Shitong Sun, Weitong Cai, Jiankang Deng, Jifei Song, Zhensong Zhang
Computer Vision
Beyond Inpainting: Unleash 3D Understanding for Stable Camera-Controlled Video Generation
Dong-Yu Chen, Yixin Guo, Shuojin Yang et al. (+2 more)
Camera control has been extensively studied in conditioned
video generation; however, performing precisely altering the camera trajectories while faithfully preserving the
video content remains a challenging task. The mainstream approach to achieving precise camera control is warping a 3D representation according to the target trajectory. However, such methods fail to fully leverage the 3D priors of
video diffusion models (VDMs) and often fall into the Inpainting Trap, resulting in subject inconsistency and degraded generation quality. To address this problem, we propose DepthDirector, a
video re-rendering framework with precise camera controllability. By leveraging the depth
video from explicit 3D representation as camera-control guidance, our method can faithfully reproduce the dynamic scene of an input
video under novel camera trajectories. Specifically, we design a View-Content Dual-Stream Condition mechanism that injects both the source
video and the warped depth sequence rendered under the target viewpoint into the pretrained
video generation model. This
geometric guidance signal enables VDMs to comprehend camera movements and leverage their 3D understanding capabilities, thereby facilitating precise camera control and consistent content generation. Next, we introduce a lightweight LoRA-based
video diffusion adapter to train our framework, fully preserving the knowledge priors of VDMs. Additionally, we construct a large-scale multi-camera synchronized
dataset named MultiCam-WarpData using Unreal Engine 5, containing 8K
videos across 1K dynamic scenes. Extensive experiments show that DepthDirector outperforms existing methods in both camera controllability and visual quality. Our code and
dataset will be publicly available.
Read the full abstract ▾
All authors Dong-Yu Chen, Yixin Guo, Shuojin Yang, Tai-Jiang Mu, Shi-Min Hu
Computer Vision
ELITE: Efficient Gaussian Head Avatar from a Monocular Video via Learned Initialization and TEst-time Generative Adaptation
Kim Youwang, Lee Hyoseok, Subin Park et al. (+2 more)
We introduce ELITE, an Efficient Gaussian head avatar synthesis from a
monocular video via Learned Initialization and TEst-time generative adaptation. Prior works rely either on a 3D data prior or a 2D generative prior to compensate for missing visual cues in
monocular videos. However, 3D data prior methods often struggle to generalize in-the-wild, while 2D generative prior methods are computationally heavy and prone to identity
hallucination. We identify a complementary synergy between these two priors and design an efficient system that achieves high-fidelity animatable avatar synthesis with strong in-the-wild generalization. Specifically, we introduce a feed-forward Mesh2Gaussian Prior Model (MGPM) that enables fast initialization of a Gaussian avatar. To further bridge the domain gap at test time, we design a test-time generative adaptation stage, leveraging both real and synthetic images as supervision. Unlike previous full
diffusion denoising strategies that are slow and
hallucination-prone, we propose a rendering-guided single-step
diffusion enhancer that restores missing visual details, grounded on Gaussian avatar renderings. Our experiments demonstrate that ELITE produces visually superior avatars to prior works, even for challenging expressions, while achieving 60x faster synthesis than the 2D generative prior method.
Read the full abstract ▾
All authors Kim Youwang, Lee Hyoseok, Subin Park, Gerard Pons-Moll, Tae-Hyun Oh
Computer Vision
From Physical Degradation Models to Task-Aware All-in-One Image Restoration
Hu Gao, Xiaoning Lei, Xichen Xu et al. (+2 more)
All-in-one image restoration aims to adaptively handle multiple restoration tasks with a single trained model. Although existing methods achieve promising results by introducing
prompt information or leveraging large models, the added learning modules increase system complexity and hinder real-time applicability. In this paper, we adopt a physical degradation modeling perspective and predict a task-aware inverse degradation operator for efficient all-in-one image restoration. The framework consists of two stages. In the first stage, the predicted inverse operator produces an initial restored image together with an uncertainty perception map that highlights regions difficult to reconstruct, ensuring restoration reliability. In the second stage, the restoration is further refined under the guidance of this uncertainty map. The same inverse operator prediction network is used in both stages, with task-aware parameters introduced after operator prediction to adapt to different degradation tasks. Moreover, by accelerating the convolution of the inverse operator, the proposed method achieves efficient all-in-one image restoration. The resulting tightly integrated architecture, termed OPIR, is extensively validated through experiments, demonstrating superior all-in-one restoration performance while remaining highly competitive on task-aligned restoration.
Read the full abstract ▾
All authors Hu Gao, Xiaoning Lei, Xichen Xu, Xingjian Wang, Lizhuang Ma
Computer Vision
RAG-3DSG: Enhancing 3D Scene Graphs with Re-Shot Guided Retrieval-Augmented Generation
Yue Chang, Rufeng Chen, Zhaofan Zhang et al. (+3 more)
Open-vocabulary 3D Scene Graph (3DSG) can enhance various downstream tasks in robotics by leveraging structured semantic representations, yet current 3DSG construction methods suffer from semantic inconsistencies caused by noisy cross-image aggregation under occlusions and constrained viewpoints. To mitigate the impact of such inconsistency, we propose RAG-3DSG, which introduces re-shot guided uncertainty estimation. By measuring the semantic consistency between original limited viewpoints and re-shot optimal viewpoints, this method quantifies the underlying semantic ambiguity of each graph object. Based on this quantification, we devise an Object-level
Retrieval-Augmented Generation (RAG) that leverages low-uncertainty objects as semantic anchors to retrieve more reliable contextual knowledge, enabling a
Vision-
Language Model to rectify the predictions of uncertain objects and optimize the final 3DSG. Extensive evaluations across three challenging
benchmarks and real-world
robot trials demonstrate that RAG-3DSG achieves superior recall and precision, effectively mitigating semantic noise to provide highly reliable scene representations for robotics tasks.
Read the full abstract ▾
All authors Yue Chang, Rufeng Chen, Zhaofan Zhang, Yi Chen, Yifan Tian, Sihong Xie
Computer Vision
Chao Huang, Benfeng Wang, Wei Wang et al. (+5 more)
Recent progress in
reasoning capabilities of
Multimodal Large
Language Models(MLLMs) has highlighted their potential for performing complex
video understanding tasks. However, in the domain of
Video Anomaly Detection and Understanding (VAD&U), existing MLLM-based methods are largely limited to anomaly localization or post-hoc description, lacking explicit
reasoning processes, risk awareness, and decision-oriented interpretation. To address this gap, we define a new task termed
Video Anomaly
Reasoning (VAR), which elevates
video anomaly analysis from descriptive understanding to structured, multi-stage
reasoning. VAR explicitly requires models to perform progressive
reasoning over anomalous events before answering anomaly-related questions, encompassing visual perception, causal interpretation, and risk-aware decision making. To support this task, we present a new
dataset with 8,641
videos, where each
video is annotated with diverse question types corresponding to different
reasoning depths, totaling more than 50,000 samples, making it one of the largest
datasets for
video anomaly. The annotations are based on a structured Perception-Cognition-Action Chain-of-Thought (PerCoAct-CoT), which formalizes domain-specific
reasoning priors for
video anomaly understanding. This design enables systematic evaluation of multi-stage and adaptive anomaly
reasoning. In addition, we propose Anomaly-Aware Group Relative Policy Optimization to further enhance
reasoning reliability under weak supervision. Building upon the proposed task and
dataset, we develop an end-to-end MLLM-based VAR model termed Vad-R1-Plus, which supports adaptive hierarchical
reasoning and risk-aware decision making. Extensive experiments demonstrate that the proposed
benchmark and method effectively advance the
reasoning capabilities of MLLMs on VAR tasks, outperforming both open-source and proprietary baselines.
Read the full abstract ▾
All authors Chao Huang, Benfeng Wang, Wei Wang, Jie Wen, Li Shen, Wenqi Ren, Yong Xu, Xiaochun Cao
Computer Vision · Cross-listed from cs.AI
MHub.ai: A Simple, Standardized, and Reproducible Platform for AI Models in Medical Imaging
Leonard Nürnberg, Dennis Bontempi, Suraj Pai et al. (+17 more)
Artificial intelligence (AI) has the potential to transform medical imaging by automating image analysis and accelerating clinical research. However, research and clinical use are limited by the wide variety of AI implementations and architectures, inconsistent documentation, and reproducibility issues. Here, we introduce MHub$.$ai, an open-source, container-based platform that standardizes access to AI models with minimal configuration, promoting accessibility and reproducibility in medical imaging. MHub$.$ai packages models from peer-reviewed publications into standardized containers that support direct processing of DICOM and other formats, provide a unified application interface, and embed structured metadata. Each model is accompanied by publicly available reference data that can be used to confirm model operation. MHub$.$ai includes an initial set of state-of-the-art
segmentation, prediction, and feature extraction models for different modalities. The modular framework enables adaptation of any model and supports community contributions. We demonstrate the utility of the platform in a clinical use case through comparative evaluation of lung
segmentation models. To further strengthen transparency and reproducibility, we publicly release the generated
segmentations and evaluation metrics and provide interactive dashboards that allow readers to inspect individual cases and reproduce or extend our analysis. By simplifying model use, MHub$.$ai enables side-by-side benchmarking with identical execution commands and standardized outputs, and lowers the barrier to clinical translation.
Read the full abstract ▾
All authors Leonard Nürnberg, Dennis Bontempi, Suraj Pai, Curtis Lisle, Steve Pieper, Ron Kikinis, Sil van de Leemput, Rahul Soni, Gowtham Murugesan, Cosmin Ciausu, Miriam Groeneveld, Felix J. Dorfner, Jue Jiang, Aneesh Rangnekar, Harini Veeraraghavan, Joeran S. Bosma, Keno Bressem, Raymond Mak, Andrey Fedorov, Hugo JWL Aerts
Computer Vision
LaViT: Aligning Latent Visual Thoughts for Multi-modal Reasoning
Linquan Wu, Tianxiang Jiang, Yifei Dong et al. (+6 more)
Current
multimodal latent
reasoning often relies on external supervision (e.g., auxiliary images), ignoring intrinsic visual
attention dynamics. In this work, we identify a critical Perception Gap in
distillation: student models frequently mimic a teacher's textual output while attending to fundamentally divergent visual regions, effectively relying on
language priors rather than grounded perception. To bridge this, we propose LaViT, a framework that aligns latent visual thoughts rather than static
embeddings. LaViT compels the student to autoregressively reconstruct the teacher's visual semantics and
attention trajectories prior to text generation, employing a curriculum sensory gating mechanism to prevent shortcut learning. Extensive experiments show that LaViT significantly enhances visual grounding, achieving up to +16.9% gains on complex
reasoning tasks and enabling a compact 3B model to outperform larger open-source variants and proprietary models like GPT-4o.
Read the full abstract ▾
All authors Linquan Wu, Tianxiang Jiang, Yifei Dong, Haoyu Yang, Fengji Zhang, Shichaang Meng, Ai Xuan, Linqi Song, Jacky Keung
Computer Vision
VQ-Seg: Vector-Quantized Token Perturbation for Semi-Supervised Medical Image Segmentation
Sicheng Yang, Zhaohu Xing and Lei Zhu
Consistency learning with feature perturbation is a widely used strategy in semi-supervised medical image
segmentation. However, many existing perturbation methods rely on dropout, and thus require a careful manual tuning of the dropout rate, which is a sensitive hyperparameter and often difficult to optimize and may lead to suboptimal regularization. To overcome this limitation, we propose VQ-Seg, the first approach to employ vector quantization (VQ) to discretize the feature space and introduce a novel and controllable Quantized Perturbation Module (QPM) that replaces dropout. Our QPM perturbs discrete representations by shuffling the spatial locations of codebook indices, enabling effective and controllable regularization. To mitigate potential information loss caused by quantization, we design a dual-branch architecture where the post-quantization feature space is shared by both image reconstruction and
segmentation tasks. Moreover, we introduce a Post-VQ Feature Adapter (PFA) to incorporate guidance from a foundation model (FM), supplementing the high-level semantic information lost during quantization. Furthermore, we collect a large-scale Lung Cancer (LC)
dataset comprising 828 CT scans annotated for central-type lung carcinoma. Extensive experiments on the LC
dataset and other public
benchmarks demonstrate the effectiveness of our method, which outperforms state-of-the-art approaches. Code available at: https://github.com/script-Yang/VQ-Seg.
Read the full abstract ▾
Computer Vision
Beyond Single Prompts: Synergistic Fusion and Arrangement for VICL
Wenwen Liao, Jianbo Yu, Yuansong Wang et al. (+2 more)
Vision In-Context Learning (VICL) enables inpainting models to quickly adapt to new visual tasks from only a few
prompts. However, existing methods suffer from two key issues: (1) selecting only the most similar
prompt discards complementary cues from other high-quality
prompts; and (2) failing to exploit the structured information implied by different
prompt arrangements. We propose an end-to-end VICL framework to overcome these limitations. Firstly, an adaptive Fusion Module aggregates critical patterns and annotations from multiple
prompts to form more precise contextual
prompts. Secondly, we introduce arrangement-specific lightweight MLPs to decouple layout priors from the core model, while minimally affecting the overall model. In addition, an bidirectional fine-tuning mechanism swaps the roles of query and
prompt, encouraging the model to reconstruct the original
prompt from fused context and thus enhancing collaboration between the fusion module and the inpainting model. Experiments on foreground
segmentation, single-object detection, and image colorization demonstrate superior results and strong cross-task generalization of our method.
Read the full abstract ▾
All authors Wenwen Liao, Jianbo Yu, Yuansong Wang, Shifu Yan, Xiaofeng Yang
Computer Vision
Enhancing Visual In-Context Learning by Multi-Faceted Fusion
Wenwen Liao, Jianbo Yu, Yuansong Wang et al. (+2 more)
Visual In-Context Learning (VICL) has emerged as a powerful paradigm, enabling models to perform novel visual tasks by learning from in-context examples. The dominant "retrieve-then-
prompt" approach typically relies on selecting the single best visual
prompt, a practice that often discards valuable contextual information from other suitable candidates. While recent work has explored fusing the top-K
prompts into a single, enhanced representation, this still simply collapses multiple rich signals into one, limiting the model's
reasoning capability. We argue that a more multi-faceted, collaborative fusion is required to unlock the full potential of these diverse contexts. To address this limitation, we introduce a novel framework that moves beyond single-
prompt fusion towards an multi-combination collaborative fusion. Instead of collapsing multiple
prompts into one, our method generates three contextual representation branches, each formed by integrating information from different combinations of top-quality
prompts. These complementary guidance signals are then fed into proposed MULTI-VQGAN architecture, which is designed to jointly interpret and utilize collaborative information from multiple sources. Extensive experiments on diverse tasks, including foreground
segmentation, single-object detection, and image colorization, highlight its strong cross-task generalization, effective contextual fusion, and ability to produce more robust and accurate predictions than existing methods.
Read the full abstract ▾
All authors Wenwen Liao, Jianbo Yu, Yuansong Wang, Qingchao Jiang, Xiaofeng Yang
Computer Vision
MathDoc: Benchmarking Structured Extraction and Active Refusal on Noisy Mathematics Exam Papers
Chenyue Zhou, Jiayi Tuo, Shitong Qin et al. (+7 more)
The automated extraction of structured questions from paper-based mathematics exams is fundamental to intelligent education, yet remains challenging in real-world settings due to severe visual noise. Existing
benchmarks mainly focus on clean documents or generic layout analysis, overlooking both the structural integrity of mathematical problems and the ability of models to actively reject incomplete inputs. We introduce MathDoc, the first
benchmark for document-level information extraction from authentic high school mathematics exam papers. MathDoc contains 3,609 carefully curated questions with real-world artifacts and explicitly includes unrecognizable samples to evaluate active refusal behavior. We propose a multi-dimensional evaluation framework covering stem accuracy, visual similarity, and refusal capability. Experiments on SOTA MLLMs, including Qwen3-VL and Gemini-2.5-Pro, show that although end-to-end models achieve strong extraction performance, they consistently fail to refuse illegible inputs, instead producing confident but invalid outputs. These results highlight a critical gap in current MLLMs and establish MathDoc as a
benchmark for assessing model reliability under degraded document conditions. Our project repository is available at \href{https://github.com/winnk123/papers/tree/master}{GitHub repository}
Read the full abstract ▾
All authors Chenyue Zhou, Jiayi Tuo, Shitong Qin, Wei Dai, Mingxuan Wang, Ziwei Zhao, Duoyang Li, Shiyang Su, Yanxi Lu, Yanbiao Ma
Computer Vision
FlowAct-R1: Towards Interactive Humanoid Video Generation
Lizhen Wang, Yongming Zhu, Zhipeng Ge et al. (+15 more)
Interactive
humanoid video generation aims to synthesize lifelike visual
agents that can engage with humans through continuous and responsive
video. Despite recent advances in
video synthesis, existing methods often grapple with the trade-off between high-fidelity synthesis and real-time interaction requirements. In this paper, we propose FlowAct-R1, a framework specifically designed for real-time interactive
humanoid video generation. Built upon a MMDiT architecture, FlowAct-R1 enables the streaming synthesis of
video with arbitrary durations while maintaining low-latency responsiveness. We introduce a chunkwise
diffusion forcing strategy, complemented by a novel self-forcing variant, to alleviate error accumulation and ensure long-term temporal consistency during continuous interaction. By leveraging efficient
distillation and system-level optimizations, our framework achieves a stable 25fps at 480p resolution with a time-to-first-frame (TTFF) of only around 1.5 seconds. The proposed method provides holistic and fine-grained full-body control, enabling the
agent to transition naturally between diverse behavioral states in interactive scenarios. Experimental results demonstrate that FlowAct-R1 achieves exceptional behavioral vividness and perceptual realism, while maintaining robust generalization across diverse character styles.
Read the full abstract ▾
All authors Lizhen Wang, Yongming Zhu, Zhipeng Ge, Youwei Zheng, Longhao Zhang, Tianshu Hu, Shiyang Qin, Mingshuang Luo, Jiaxu Zhang, Xin Chen, Yulong Wang, Zerong Zheng, Jianwen Jiang, Chao Liang, Weifeng Chen, Xing Wang, Yuan Zhang, Mingyuan Gao
Computer Vision
InfoSculpt: Sculpting the Latent Space for Generalized Category Discovery
Wenwen Liao, Hang Ruan, Jianbo Yu et al. (+3 more)
Generalized Category Discovery (GCD) aims to classify instances from both known and novel categories within a large-scale unlabeled
dataset, a critical yet challenging task for real-world, open-world applications. However, existing methods often rely on pseudo-labeling, or two-stage clustering, which lack a principled mechanism to explicitly disentangle essential, category-defining signals from instance-specific noise. In this paper, we address this fundamental limitation by re-framing GCD from an information-theoretic perspective, grounded in the Information Bottleneck (IB) principle. We introduce InfoSculpt, a novel framework that systematically sculpts the representation space by minimizing a dual Conditional Mutual Information (CMI) objective. InfoSculpt uniquely combines a Category-Level CMI on labeled data to learn compact and discriminative representations for known classes, and a complementary Instance-Level CMI on all data to distill invariant features by compressing augmentation-induced noise. These two objectives work synergistically at different scales to produce a disentangled and robust latent space where categorical information is preserved while noisy, instance-specific details are discarded. Extensive experiments on 8
benchmarks demonstrate that InfoSculpt validating the effectiveness of our information-theoretic approach.
Read the full abstract ▾
All authors Wenwen Liao, Hang Ruan, Jianbo Yu, Yuansong Wang, Qingchao Jiang, Xiaofeng Yang
Computer Vision
V-Zero: Self-Improving Multimodal Reasoning with Zero Annotation
Han Wang, Yi Yang, Jingyuan Hu et al. (+2 more)
Recent advances in
multimodal learning have significantly enhanced the
reasoning capabilities of
vision-
language models (VLMs). However, state-of-the-art approaches rely heavily on large-scale human-annotated
datasets, which are costly and time-consuming to acquire. To overcome this limitation, we introduce V-Zero, a general post-training framework that facilitates self-improvement using exclusively unlabeled images. V-Zero establishes a co-evolutionary loop by instantiating two distinct roles: a Questioner and a Solver. The Questioner learns to synthesize high-quality, challenging questions by leveraging a dual-track
reasoning reward that contrasts intuitive guesses with reasoned results. The Solver is optimized using pseudo-labels derived from majority voting over its own sampled responses. Both roles are trained iteratively via Group Relative Policy Optimization (GRPO), driving a cycle of mutual enhancement. Remarkably, without a single human annotation, V-Zero achieves consistent performance gains on Qwen2.5-VL-7B-Instruct, improving visual mathematical
reasoning by +1.7 and general
vision-centric by +2.6, demonstrating the potential of self-improvement in
multimodal systems. Code is available at https://github.com/SatonoDia/V-Zero
Read the full abstract ▾
All authors Han Wang, Yi Yang, Jingyuan Hu, Minfeng Zhu, Wei Chen
Computer Vision
Difficulty-guided Sampling: Bridging the Target Gap between Dataset Distillation and Downstream Tasks
Mingzhuo Li, Guang Li, Linfeng Ye et al. (+4 more)
In this paper, we propose difficulty-guided sampling (DGS) to bridge the target gap between the
distillation objective and the downstream task, therefore improving the performance of
dataset distillation. Deep neural networks achieve remarkable performance but have time and storage-consuming training processes.
Dataset distillation is proposed to generate compact, high-quality distilled
datasets, enabling effective model training while maintaining downstream performance. Existing approaches typically focus on features extracted from the original
dataset, overlooking task-specific information, which leads to a target gap between the
distillation objective and the downstream task. We propose leveraging characteristics that benefit the downstream training into data
distillation to bridge this gap. Focusing on the downstream task of image classification, we introduce the concept of difficulty and propose DGS as a plug-in post-stage sampling module. Following the specific target difficulty distribution, the final distilled
dataset is sampled from image pools generated by existing methods. We also propose difficulty-aware guidance (DAG) to explore the effect of difficulty in the generation process. Extensive experiments across multiple settings demonstrate the effectiveness of the proposed methods. It also highlights the broader potential of difficulty for diverse downstream tasks.
Read the full abstract ▾
All authors Mingzhuo Li, Guang Li, Linfeng Ye, Jiafeng Mao, Takahiro Ogawa, Konstantinos N. Plataniotis, Miki Haseyama
Computer Vision
Thinking Like Van Gogh: Structure-Aware Style Transfer via Flow-Guided 3D Gaussian Splatting
Lebin Zhou, Jingchuan Xiao, Zhendong Wang et al. (+4 more)
In 1888, Vincent van Gogh wrote, "I am seeking exaggeration in the essential." This principle, amplifying structural form while suppressing photographic detail, lies at the core of Post-Impressionist art. However, most existing 3D style transfer methods invert this philosophy, treating geometry as a rigid substrate for surface-level texture projection. To authentically reproduce Post-Impressionist stylization,
geometric abstraction must be embraced as the primary vehicle of expression. We propose a flow-guided
geometric advection framework for 3D Gaussian Splatting (3DGS) that operationalizes this principle in a mesh-free setting. Our method extracts directional flow fields from 2D paintings and back-propagates them into 3D space, rectifying Gaussian primitives to form flow-aligned brushstrokes that conform to scene
topology without relying on explicit mesh priors. This enables expressive structural deformation driven directly by painterly motion rather than photometric constraints. Our contributions are threefold: (1) a projection-based, mesh-free flow guidance mechanism that transfers 2D artistic motion into 3D Gaussian geometry; (2) a luminance-structure decoupling strategy that isolates
geometric deformation from color optimization, mitigating artifacts during aggressive structural abstraction; and (3) a VLM-as-a-Judge evaluation framework that assesses artistic authenticity through aesthetic judgment instead of conventional pixel-level metrics, explicitly addressing the subjective nature of artistic stylization.
Read the full abstract ▾
All authors Lebin Zhou, Jingchuan Xiao, Zhendong Wang, Jinhao Wang, Rongduo Han, Nam Ling, Cihan Ruan
Computer Vision
ReaMIL: Reasoning- and Evidence-Aware Multiple Instance Learning for Whole-Slide Histopathology
Hyun Do Jung, Jungwon Choi and Hwiyoung Kim
We introduce ReaMIL (
Reasoning- and Evidence-Aware MIL), a multiple instance learning approach for whole-slide histopathology that adds a light selection head to a strong MIL backbone. The head produces soft per-tile gates and is trained with a budgeted-sufficiency objective: a hinge loss that enforces the true-class probability to be $\geq \tau$ using only the kept evidence, under a sparsity budget on the number of selected tiles. The budgeted-sufficiency objective yields small, spatially compact evidence sets without sacrificing baseline performance. Across TCGA-NSCLC (LUAD vs. LUSC), TCGA-BRCA (IDC vs. Others), and PANDA, ReaMIL matches or slightly improves baseline AUC and provides quantitative evidence-efficiency diagnostics. On NSCLC, it attains AUC 0.983 with a mean minimal sufficient K (MSK) $\approx 8.2$ tiles at $\tau = 0.90$ and AUKC $\approx 0.864$, showing that class confidence rises sharply and stabilizes once a small set of tiles is kept. The method requires no extra supervision, integrates seamlessly with standard MIL training, and naturally yields slide-level overlays. We report accuracy alongside MSK, AUKC, and contiguity for rigorous evaluation of model behavior on WSIs.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Comparative Evaluation of Deep Learning-Based and WHO-Informed Approaches for Sperm Morphology Assessment
Mohammad Abbadi
Assessment of sperm morphological quality remains a critical yet subjective component of male fertility evaluation, often limited by inter-observer variability and resource constraints. This study presents a comparative biomedical artificial intelligence framework evaluating an image-based deep learning model (HuSHeM) alongside a clinically grounded baseline derived from World Health Organization criteria augmented with the Systemic Inflammation Response Index (WHO(+SIRI)). The HuSHeM model was trained on high-resolution sperm morphology images and evaluated using an independent clinical cohort. Model performance was assessed using discrimination, calibration, and clinical utility analyses. The HuSHeM model demonstrated higher discriminative performance, as reflected by an increased area under the receiver operating characteristic curve with relatively narrow confidence intervals compared to WHO(+SIRI). Precision-recall analysis further indicated improved performance under class imbalance, with higher precision-recall area values across evaluated thresholds. Calibration analysis indicated closer agreement between predicted probabilities and observed outcomes for HuSHeM, while decision curve analysis suggested greater net clinical benefit across clinically relevant threshold probabilities. These findings suggest that image-based deep learning may offer improved predictive reliability and clinical utility compared with traditional rule-based and inflammation-augmented criteria. The proposed framework supports objective and reproducible assessment of sperm morphology and may serve as a decision-support tool within fertility screening and referral workflows. The proposed models are intended as decision-support or referral tools and are not designed to replace clinical judgment or laboratory assessment.
Read the full abstract ▾
Computer Vision
CoF-T2I: Video Models as Pure Visual Reasoners for Text-to-Image Generation
Chengzhuo Tong, Mingkun Chang, Shenglong Zhang et al. (+12 more)
Recent
video generation models have revealed the emergence of Chain-of-Frame (CoF)
reasoning, enabling frame-by-frame visual inference. With this capability,
video models have been successfully applied to various visual tasks (e.g., maze solving, visual puzzles). However, their potential to enhance text-to-image (T2I) generation remains largely unexplored due to the absence of a clearly defined visual
reasoning starting point and interpretable intermediate states in the T2I generation process. To bridge this gap, we propose CoF-T2I, a model that integrates CoF
reasoning into T2I generation via progressive visual refinement, where intermediate frames act as explicit
reasoning steps and the final frame is taken as output. To establish such an explicit generation process, we curate CoF-Evol-Instruct, a
dataset of CoF trajectories that model the generation process from semantics to aesthetics. To further improve quality and avoid motion artifacts, we enable independent encoding operation for each frame. Experiments show that CoF-T2I significantly outperforms the base
video model and achieves competitive performance on challenging
benchmarks, reaching 0.86 on GenEval and 7.468 on Imagine-Bench. These results indicate the substantial promise of
video models for advancing high-quality text-to-image generation.
Read the full abstract ▾
All authors Chengzhuo Tong, Mingkun Chang, Shenglong Zhang, Yuran Wang, Cheng Liang, Zhizheng Zhao, Ruichuan An, Bohan Zeng, Yang Shi, Yifan Dai, Ziming Zhao, Guanbin Li, Pengfei Wan, Yuanxing Zhang, Wentao Zhang
Computer Vision
Nick Truong, Pritam P. Karmokar and William J. Beksi
Underwater imaging is fundamentally challenging due to wavelength-dependent light attenuation, strong scattering from suspended particles, turbidity-induced blur, and non-uniform illumination. These effects impair standard cameras and make ground-truth motion nearly impossible to obtain. On the other hand, event cameras offer microsecond resolution and high dynamic range. Nonetheless, progress on investigating event cameras for
underwater environments has been limited due to the lack of
datasets that pair realistic
underwater optics with accurate optical flow. To address this problem, we introduce the first synthetic
underwater benchmark dataset for event-based optical flow derived from physically-based ray-traced RGBD sequences. Using a modern
video-to-event pipeline applied to rendered
underwater videos, we produce realistic event data streams with dense ground-truth flow, depth, and camera motion. Moreover, we
benchmark state-of-the-art learning-based and model-based optical flow prediction methods to understand how
underwater light transport affects event formation and motion estimation accuracy. Our
dataset establishes a new baseline for future development and evaluation of
underwater event-based perception
algorithms. The source code and
dataset for this project are publicly available at https://robotic-
vision-lab.github.io/ueof.
Read the full abstract ▾
Computer Vision
DiCo: Disentangled Concept Representation for Text-to-image Person Re-identification
Giyeol Kim and Chanho Eom
Text-to-image person re-identification (TIReID) aims to retrieve person images from a large gallery given free-form textual descriptions. TIReID is challenging due to the substantial modality gap between visual appearances and textual expressions, as well as the need to model fine-grained correspondences that distinguish individuals with similar attributes such as clothing color, texture, or outfit style. To address these issues, we propose DiCo (Disentangled Concept Representation), a novel framework that achieves hierarchical and disentangled cross-modal
alignment. DiCo introduces a shared slot-based representation, where each slot acts as a part-level anchor across modalities and is further decomposed into multiple concept blocks. This design enables the disentanglement of complementary attributes (e.g., color, texture, shape) while maintaining consistent part-level correspondence between image and text. Extensive experiments on CUHK-PEDES, ICFG-PEDES, and RSTPReid demonstrate that our framework achieves competitive performance with state-of-the-art methods, while also enhancing interpretability through explicit slot- and block-level representations for more fine-grained
retrieval results.
Read the full abstract ▾
Computer Vision
VERHallu: Evaluating and Mitigating Event Relation Hallucination in Video Large Language Models
Zefan Zhang, Kehua Zhu, Shijie Jiang et al. (+3 more)
Video Large
Language Models (VideoLLMs) exhibit various types of
hallucinations. Existing research has primarily focused on
hallucinations involving the presence of events, objects, and scenes in
videos, while largely neglecting event relation
hallucination. In this paper, we introduce a novel
benchmark for evaluating the
Video Event Relation
Hallucination, named VERHallu. This
benchmark focuses on causal, temporal, and subevent relations between events, encompassing three types of tasks: relation classification, question answering, and counterfactual question answering, for a comprehensive evaluation of event relation
hallucination. Additionally, it features counterintuitive
video scenarios that deviate from typical pretraining distributions, with each sample accompanied by human-annotated candidates covering both
vision-
language and pure
language biases. Our analysis reveals that current state-of-the-art VideoLLMs struggle with dense-event relation
reasoning, often relying on prior knowledge due to insufficient use of frame-level cues. Although these models demonstrate strong grounding capabilities for key events, they often overlook the surrounding subevents, leading to an incomplete and inaccurate understanding of event relations. To tackle this, we propose a Key-Frame Propagating (KFP) strategy, which reallocates frame-level
attention within intermediate layers to enhance multi-event understanding. Experiments show it effectively mitigates the event relation
hallucination without affecting inference speed.
Read the full abstract ▾
All authors Zefan Zhang, Kehua Zhu, Shijie Jiang, Hongyuan Lu, Shengkai Sun, Tian Bai
Computer Vision
DW-DGAT: Dynamically Weighted Dual Graph Attention Network for Neurodegenerative Disease Diagnosis
Chengjia Liang, Zhenjiong Wang, Chao Chen et al. (+5 more)
Parkinson's disease (PD) and Alzheimer's disease (AD) are the two most prevalent and incurable neurodegenerative diseases (NDs) worldwide, for which early diagnosis is critical to delay their progression. However, the high dimensionality of multi-metric data with diverse structural forms, the heterogeneity of neuroimaging and phenotypic data, and class imbalance collectively pose significant challenges to early ND diagnosis. To address these challenges, we propose a dynamically weighted dual graph
attention network (DW-DGAT) that integrates: (1) a general-purpose data fusion strategy to merge three structural forms of multi-metric data; (2) a dual graph
attention architecture based on
brain regions and inter-sample relationships to extract both micro- and macro-level features; and (3) a class weight generation mechanism combined with two stable and effective loss functions to mitigate class imbalance. Rigorous experiments, based on the Parkinson Progression Marker Initiative (PPMI) and Alzheimer's Disease Neuroimaging Initiative (ADNI) studies, demonstrate the state-of-the-art performance of our approach.
Read the full abstract ▾
All authors Chengjia Liang, Zhenjiong Wang, Chao Chen, Ruizhi Zhang, Songxi Liang, Hai Xie, Haijun Lei, Zhongwei Huang
Computer Vision · Cross-listed from cs.MM
EditEmoTalk: Controllable Speech-Driven 3D Facial Animation with Continuous Expression Editing
Diqiong Jiang, Kai Zhu, Dan Song et al. (+3 more)
Speech-driven 3D facial animation aims to generate realistic and expressive facial motions directly from audio. While recent methods achieve high-quality lip synchronization, they often rely on discrete
emotion categories, limiting continuous and fine-grained emotional control. We present EditEmoTalk, a controllable
speech-driven 3D facial animation framework with continuous
emotion editing. The key idea is a boundary-aware semantic
embedding that learns the normal directions of inter-
emotion decision boundaries, enabling a continuous expression
manifold for smooth
emotion manipulation. Moreover, we introduce an emotional consistency loss that enforces semantic
alignment between the generated motion dynamics and the target
emotion embedding through a mapping network, ensuring faithful emotional expression. Extensive experiments demonstrate that EditEmoTalk achieves superior controllability, expressiveness, and generalization while maintaining accurate lip synchronization. Code and pretrained models will be released.
Read the full abstract ▾
All authors Diqiong Jiang, Kai Zhu, Dan Song, Jian Chang, Chenglizhao Chen, Zhenyu Wu
Computer Vision
Yulin He, Wei Chen, Zhikang Jian et al. (+5 more)
Reasoning segmentation is an emerging
vision-
language task that requires
reasoning over intricate text queries to precisely segment objects. However, existing methods typically suffer from overthinking, generating verbose
reasoning chains that interfere with object localization in
multimodal large
language models (MLLMs). To address this issue, we propose DR$^2$Seg, a self-rewarding framework that improves both
reasoning efficiency and
segmentation accuracy without requiring extra thinking supervision. DR$^2$Seg employs a two-stage rollout strategy that decomposes
reasoning segmentation into
multimodal reasoning and referring
segmentation. In the first stage, the model generates a self-contained description that explicitly specifies the target object. In the second stage, this description replaces the original complex query to verify its self-containment. Based on this design, two self-rewards are introduced to mitigate overthinking and the associated
attention dispersion. Extensive experiments conducted on 3B and 7B variants of Qwen2.5-VL, as well as on both SAM2 and SAM3, demonstrate that DR$^2$Seg consistently improves
reasoning efficiency and overall
segmentation accuracy.
Read the full abstract ▾
All authors Yulin He, Wei Chen, Zhikang Jian, Tianhang Guo, Wenjuan Zhou, Minglong Li, Shaowu Yang, Wenjing Yang
Computer Vision
The Spatial Blindspot of Vision-Language Models
Nahid Alam, Leema Krishna Murali, Siddhant Bharadwaj et al. (+7 more)
Vision-
language models (VLMs) have advanced rapidly, but their ability to capture spatial relationships remains a blindspot. Current VLMs are typically built with contrastive
language-image pretraining (CLIP) style image encoders. The training recipe often flattens images into 1D patch sequences, discarding the 2D structure necessary for spatial
reasoning. We argue that this lack of spatial awareness is a missing dimension in VLM design and a bottleneck for applications requiring spatial grounding, such as robotics and
embodied AI. To address this, we investigate (i) image encoders trained with alternative objectives and (ii) 2D positional encodings. Our experiments show that these architectural choices can lead to improved spatial
reasoning on several
benchmarks.
Read the full abstract ▾
All authors Nahid Alam, Leema Krishna Murali, Siddhant Bharadwaj, Patrick Liu, Timothy Chung, Drishti Sharma, Akshata A, Kranthi Kiran, Wesley Tam, Bala Krishna S Vegesna