Computer Vision
Universality Reconsidered: Rethinking the Validation of Foundation Models for General-Purpose 3D Medical Segmentation
Yichi Zhang, Le Xue, Feiyang Xiao et al. (+6 more)
Foundation models have emerged as a transformative paradigm in 3D medical imaging, with the promise of unified quantitative analysis across diverse targets and imaging modalities. Yet the prevailing conception of universality remains incomplete. Current models are predominantly developed and evaluated on
datasets largely concentrated around a limited set of imaging modalities and anatomical regions. In this Perspective, we evaluate representative 3D
segmentation foundation models using paired whole-body structural and functional imaging data. Our analysis reveals a substantial gap between
benchmark-reported performance and real-world generalization, with marked degradation on previously unseen data and particularly severe failures on functional imaging modalities. These findings suggest that current foundation models remain far from achieving true universality. We argue that progress requires not only scaling models and
datasets, but also a reconsideration of how universality is defined and validated, extending evaluation beyond regional structural
benchmarks toward whole-body structural and functional imaging. Our observations highlight the need to distinguish
benchmark success from genuine clinical generalization. Bridging this gap will be essential for translating foundation models from controlled evaluation settings to real-world medical practice.
Read the full abstract ▾
All authors Yichi Zhang, Le Xue, Feiyang Xiao, Wenbo Zhang, Gang Feng, Chenguang Zheng, Yuan Qi, Yuan Cheng, Zixin Hu
Computer Vision
AD-MIR: Bridging the Gap from Perception to Persuasion in Advertising Video Understanding via Structured Reasoning
Binxiao Xu, Junyu Feng, Xiaopeng Lin et al. (+7 more)
Multimodal understanding of advertising
videos is essential for interpreting the intricate relationship between visual storytelling and abstract persuasion strategies. However, despite excelling at general search, existing
agents often struggle to bridge the cognitive gap between pixel-level perception and high-level marketing logic. To address this challenge, we introduce AD-MIR, a framework designed to decode advertising intent via a two-stage architecture. First, in the Structure-Aware
Memory Construction phase, the system converts raw
video into a structured database by integrating semantic
retrieval with exact keyword matching. This approach prioritizes fine-grained brand details (e.g., logos, on-screen text) while dynamically filtering out irrelevant background noise to isolate key protagonists. Second, the Structured
Reasoning Agent mimics a marketing expert through an iterative inquiry loop, decomposing the narrative to deduce implicit persuasion tactics. Crucially, it employs an evidence-based self-correction mechanism that rigorously validates these insights against specific
video frames, automatically backtracking when visual support is lacking. Evaluation on the AdsQA
benchmark demonstrates that AD-MIR achieves state-of-the-art performance, surpassing the strongest general-purpose
agent, DVD, by 1.8% in strict and 9.5% in relaxed accuracy. These results underscore that effective advertising understanding demands explicitly grounding abstract marketing strategies in pixel-level evidence. The code is available at https://github.com/Little-Fridge/AD-MIR.
Read the full abstract ▾
All authors Binxiao Xu, Junyu Feng, Xiaopeng Lin, Haodong Li, Zhiyuan Feng, Bohan Zeng, Shaolin Lu, Ming Lu, Qi She, Wentao Zhang
Computer Vision
HistoMet: A Pan-Cancer Deep Learning Framework for Prognostic Prediction of Metastatic Progression and Site Tropism from Primary Tumor Histopathology
Yixin Chen, Ziyu Su, Lingbin Meng et al. (+4 more)
Metastatic Progression remains the leading cause of cancer-related mortality, yet predicting whether a primary tumor will metastasize and where it will disseminate directly from histopathology remains a fundamental challenge. Although whole-slide images (WSIs) provide rich morphological information, prior computational pathology approaches typically address metastatic status or site prediction as isolated tasks, and do not explicitly model the clinically sequential decision process of metastatic risk assessment followed by downstream site-specific evaluation. To address this research gap, we present a decision-aware, concept-aligned MIL framework, HistoMet, for prognostic metastatic outcome prediction from primary tumor WSIs. Our proposed framework adopts a two-module prediction pipeline in which the likelihood of metastatic progression from the primary tumor is first estimated, followed by conditional prediction of metastatic site for high-risk cases. To guide representation learning and improve clinical interpretability, our framework integrates linguistically defined and data-adaptive metastatic concepts through a pretrained pathology
vision-
language model. We evaluate HistoMet on a multi-institutional pan-cancer cohort of 6504 patients with metastasis follow-up and site annotations. Under clinically relevant high-sensitivity screening settings (95 percent sensitivity), HistoMet significantly reduces downstream workload while maintaining high metastatic risk recall. Conditional on metastatic cases, HistoMet achieves a macro F1 of 74.6 with a standard deviation of 1.3 and a macro one-vs-rest AUC of 92.1. These results demonstrate that explicitly modeling clinical decision structure enables robust and deployable prognostic prediction of metastatic progression and site tropism directly from primary tumor histopathology.
Read the full abstract ▾
All authors Yixin Chen, Ziyu Su, Lingbin Meng, Elshad Hasanov, Wei Chen, Anil Parwani, M. Khalid Khan Niazi
Computer Vision
Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by Chain-of-Thought Reasoning
Hulingxiao He, Zijun Geng and Yuxin Peng
Any entity in the visual world can be hierarchically grouped based on shared characteristics and mapped to fine-grained sub-categories. While Multi-modal Large
Language Models (MLLMs) achieve strong performance on coarse-grained visual tasks, they often struggle with Fine-Grained Visual Recognition (FGVR). Adapting general-purpose MLLMs to FGVR typically requires large amounts of annotated data, which is costly to obtain, leaving a substantial performance gap compared to contrastive CLIP models dedicated for discriminative tasks. Moreover, MLLMs tend to overfit to seen sub-categories and generalize poorly to unseen ones. To address these challenges, we propose Fine-R1, an MLLM tailored for FGVR through an R1-style training framework: (1) Chain-of-Thought Supervised Fine-tuning, where we construct a high-quality FGVR CoT
dataset with rationales of "visual analysis, candidate sub-categories, comparison, and prediction", transition the model into a strong open-world classifier; and (2) Triplet Augmented Policy Optimization, where Intra-class Augmentation mixes trajectories from anchor and positive images within the same category to improve robustness to intra-class variance, while Inter-class Augmentation maximizes the response distinction conditioned on images across sub-categories to enhance discriminative ability. With only 4-shot training, Fine-R1 outperforms existing general MLLMs,
reasoning MLLMs, and even contrastive CLIP models in identifying both seen and unseen sub-categories, showing promise in working in knowledge-intensive domains where gathering expert annotations for all sub-categories is arduous. Code is available at https://github.com/PKU-ICST-MIPL/FineR1_ICLR2026.
Read the full abstract ▾
Computer Vision
TeleBoost: A Systematic Alignment Framework for High-Fidelity, Controllable, and Robust Video Generation
Yuanzhi Liang, Xuan'er Wu, Yirui Liu et al. (+12 more)
Post-training is the decisive step for converting a pretrained
video generator into a production-oriented model that is instruction-following, controllable, and robust over long temporal horizons. This report presents a systematical post-training framework that organizes supervised policy shaping, reward-driven reinforcement learning, and preference-based refinement into a single stability-constrained optimization stack. The framework is designed around practical
video-generation constraints, including high rollout cost, temporally compounding failure modes, and feedback that is heterogeneous, uncertain, and often weakly discriminative. By treating optimization as a staged, diagnostic-driven process rather than a collection of isolated tricks, the report summarizes a cohesive recipe for improving perceptual fidelity, temporal coherence, and
prompt adherence while preserving the controllability established at initialization. The resulting framework provides a clear blueprint for building scalable post-training pipelines that remain stable, extensible, and effective in real-world deployment settings.
Read the full abstract ▾
All authors Yuanzhi Liang, Xuan'er Wu, Yirui Liu, Yijie Fang, Yizhen Fan, Ke Hao, Rui Li, Ruiying Liu, Ziqi Ni, Peng Yu, Yanbo Wang, Haibin Huang, Qizhen Weng, Chi Zhang, Xuelong Li
Computer Vision
Automated rock joint trace mapping using a supervised learning model trained on synthetic data generated by parametric modelling
Jessica Ka Yi Chiu, Tom Frode Hansen, Eivind Magnus Paulsen et al. (+1 more)
This paper presents a geology-driven machine learning method for automated rock joint trace mapping from images. The approach combines geological modelling, synthetic data generation, and supervised image
segmentation to address limited real data and class imbalance. First, discrete fracture network models are used to generate synthetic jointed rock images at field-relevant scales via parametric modelling, preserving joint persistence, connectivity, and node-type distributions. Second,
segmentation models are trained using mixed training and pretraining followed by fine-tuning on real images. The method is tested in box and slope domains using several real
datasets. The results show that synthetic data can support supervised joint trace detection when real data are scarce. Mixed training performs well when real labels are consistent (e.g. box-domain), while fine-tuning is more robust when labels are noisy (e.g. slope-domain where labels can be biased, incomplete, and inconsistent). Fully zero-shot prediction from synthetic model remains limited, but useful generalisation is achieved by fine-tuning with a small number of real data. Qualitative analysis shows clearer and more geologically meaningful joint traces than indicated by quantitative metrics alone. The proposed method supports reliable joint mapping and provides a basis for further work on domain adaptation and evaluation.
Read the full abstract ▾
All authors Jessica Ka Yi Chiu, Tom Frode Hansen, Eivind Magnus Paulsen, Ole Jakob Mengshoel
Computer Vision
Wenjie Liu, Hao Wu, Xin Qiu et al. (+6 more)
Modern
multimodal large
language models (MLLMs) adopt a unified self-
attention design that processes visual and textual
tokens at every
Transformer layer, incurring substantial computational overhead. In this work, we revisit the necessity of such dense visual processing and show that projected visual
embeddings are already well-aligned with the
language space, while effective
vision-
language interaction occurs in only a small subset of layers. Based on these insights, we propose ViCA (
Vision-only Cross-
Attention), a minimal MLLM architecture in which visual
tokens bypass all self-
attention and feed-forward layers, interacting with text solely through sparse cross-
attention at selected layers. Extensive evaluations across three MLLM backbones, nine
multimodal benchmarks, and 26
pruning-based baselines show that ViCA preserves 98% of baseline accuracy while reducing visual-side computation to 4%, consistently achieving superior performance-efficiency trade-offs. Moreover, ViCA provides a regular, hardware-friendly inference pipeline that yields over 3.5x speedup in single-batch inference and over 10x speedup in multi-batch inference, reducing visual grounding to near-zero overhead compared with text-only LLMs. It is also orthogonal to
token pruning methods and can be seamlessly combined for further efficiency gains. Our code is available at https://github.com/EIT-NLP/ViCA.
Read the full abstract ▾
All authors Wenjie Liu, Hao Wu, Xin Qiu, Xudong Wang, Yingqi Fan, Yihan Zhang, Anhao Zhao, Yunpu Ma, Xiaoyu Shen
Computer Vision · Cross-listed from q-bio.NC
How does longer temporal context enhance multimodal narrative video processing in the brain?
Prachi Jindal, Anant Khandelwal, Manish Gupta et al. (+3 more)
Understanding how humans and artificial intelligence systems process complex narrative
videos is a fundamental challenge at the intersection of neuroscience and machine learning. This study investigates how the temporal context length of
video clips (3–24 s clips) and the narrative-task prompting shape
brain-model
alignment during naturalistic movie watching. Using fMRI recordings from participants viewing full-length movies, we examine how
brain regions sensitive to narrative context dynamically represent information over varying timescales and how these neural patterns align with model-derived features. We find that increasing clip duration substantially improves
brain alignment for
multimodal large
language models (MLLMs), whereas unimodal
video models show little to no gain. Further, shorter temporal windows align with perceptual and early
language regions, while longer windows preferentially align higher-order integrative regions, mirrored by a layer-to-cortex hierarchy in MLLMs. Finally, experiments with four narrative-task
prompts show that they elicit task-specific, region-dependent
brain alignment patterns and context-dependent shifts in clip-level tuning in higher-order regions. Our work positions long-form narrative movies as a principled testbed for studying long-timescale temporal integration in long-context MLLMs and its relationship to cortical responses during narrative comprehension.
Read the full abstract ▾
All authors Prachi Jindal, Anant Khandelwal, Manish Gupta, Bapi S. Raju, Subba Reddy Oota, Tanmoy Chakraborty
Computer Vision
Visualizing the Invisible: Enhancing Radiologist Performance in Breast Mammography via Task-Driven Chromatic Encoding
Hui Ye, Shilong Yang, Chulong Zhang et al. (+4 more)
Purpose:Mammography screening is less sensitive in dense breasts, where tissue overlap and subtle findings increase perceptual difficulty. We present MammoColor, an end-to-end framework with a Task-Driven Chromatic Encoding (TDCE) module that converts single-channel mammograms into TDCE-encoded views for visual augmentation. Materials and Methods:MammoColor couples a lightweight TDCE module with a BI-RADS triage classifier and was trained end-to-end on VinDr-Mammo. Performance was evaluated on an internal test set, two public
datasets (CBIS-DDSM and INBreast), and three external clinical cohorts. We also conducted a multi-reader, multi-case (MRMC) observer study with a washout period, comparing (1) grayscale-only, (2) TDCE-only, and (3) side-by-side grayscale+TDCE. Results:On VinDr-Mammo, MammoColor improved AUC from 0.7669 to 0.8461 (P=0.004). Gains were larger in dense breasts (AUC 0.749 to 0.835). In the MRMC study, TDCE-encoded images improved specificity (0.90 to 0.96; P=0.052) with comparable sensitivity. Conclusion:TDCE provides a task-optimized chromatic representation that may improve perceptual salience and reduce false-positive recalls in mammography triage.
Read the full abstract ▾
All authors Hui Ye, Shilong Yang, Chulong Zhang, Yexuan Xing, Juan Yu, Yaoqin Xie, Wei Zhang
Computer Vision
Cross-Camera Cow Identification via Disentangled Representation Learning
Runcheng Wang, Yaru Chen, Guiguo Zhang et al. (+2 more)
Precise identification of individual cows is a fundamental prerequisite for comprehensive digital management in smart livestock farming. While existing animal identification methods excel in controlled, single-camera settings, they face severe challenges regarding cross-camera generalization. When models trained on source cameras are deployed to new monitoring nodes characterized by divergent illumination, backgrounds, viewpoints, and heterogeneous imaging properties, recognition performance often degrades dramatically. This limits the large-scale application of non-contact technologies in dynamic, real-world farming environments. To address this challenge, this study proposes a cross-camera cow identification framework based on disentangled representation learning. This framework leverages the Subspace Identifiability Guarantee (SIG) theory in the context of bovine visual recognition. By modeling the underlying physical data generation process, we designed a principle-driven feature disentanglement module that decomposes observed images into multiple orthogonal latent subspaces. This mechanism effectively isolates stable, identity-related biometric features that remain invariant across cameras, thereby substantially improving generalization to unseen cameras. We constructed a high-quality
dataset spanning five distinct camera nodes, covering heterogeneous acquisition devices and complex variations in lighting and angles. Extensive experiments across seven cross-camera tasks demonstrate that the proposed method achieves an average accuracy of 86.0%, significantly outperforming the Source-only Baseline (51.9%) and the strongest cross-camera baseline method (79.8%). This work establishes a subspace-theoretic feature disentanglement framework for collaborative cross-camera cow identification, offering a new paradigm for precise animal monitoring in uncontrolled smart farming environments.
Read the full abstract ▾
All authors Runcheng Wang, Yaru Chen, Guiguo Zhang, Honghua Jiang, Yongliang Qiao
Computer Vision
Human Identification at a Distance: Challenges, Methods and Results on the Competition HID 2025
Jingzhe Ma, Meng Zhang, Jianlong Yu et al. (+34 more)
Human identification at a distance (HID) is challenging because traditional biometric modalities such as face and fingerprints are often difficult to acquire in real-world scenarios. Gait recognition provides a practical alternative, as it can be captured reliably at a distance. To promote progress in gait recognition and provide a fair evaluation platform, the International Competition on Human Identification at a Distance (HID) has been organized annually since 2020. Since 2023, the competition has adopted the challenging SUSTech-Competition
dataset, which features substantial variations in clothing, carried objects, and view angles. No dedicated training data are provided, requiring participants to train their models using external
datasets. Each year, the competition applies a different random seed to generate distinct evaluation splits, which reduces the risk of overfitting and supports a fair assessment of cross-domain generalization. While HID 2023 and HID 2024 already used this
dataset, HID 2025 explicitly examined whether algorithmic advances could surpass the accuracy limits observed previously. Despite the heightened difficulty, participants achieved further improvements, and the best-performing method reached 94.2% accuracy, setting a new
benchmark on this
dataset. We also analyze key technical trends and outline potential directions for future research in gait recognition.
Read the full abstract ▾
All authors Jingzhe Ma, Meng Zhang, Jianlong Yu, Kun Liu, Zunxiao Xu, Xue Cheng, Junjie Zhou, Yanfei Wang, Jiahang Li, Zepeng Wang, Kazuki Osamura, Rujie Liu, Narishige Abe, Jingjie Wang, Shunli Zhang, Haojun Xie, Jiajun Wu, Weiming Wu, Wenxiong Kang, Qingshuo Gao, Jiaming Xiong, Xianye Ben, Lei Chen, Lichen Song, Junjian Cui, Haijun Xiong, Junhao Lu, Bin Feng, Mengyuan Liu, Ji Zhou, Baoquan Zhao, Ke Xu, Yongzhen Huang, Liang Wang, Manuel J Marin-Jimenez, Md Atiqur Rahman Ahad, Shiqi Yu
Computer Vision
SIGMA: Selective-Interleaved Generation with Multi-Attribute Tokens
Xiaoyan Zhang, Zechen Bai, Haofan Wang et al. (+1 more)
Recent unified models such as Bagel demonstrate that paired image-edit data can effectively align multiple visual tasks within a single
diffusion transformer. However, these models remain limited to single-condition inputs and lack the flexibility needed to synthesize results from multiple heterogeneous sources. We present SIGMA (Selective-Interleaved Generation with Multi-Attribute
Tokens), a unified post-training framework that enables interleaved multi-condition generation within
diffusion transformers. SIGMA introduces selective multi-attribute
tokens, including style, content, subject, and identity
tokens, which allow the model to interpret and compose multiple visual conditions in an interleaved text-image sequence. Through post-training on the Bagel unified backbone with 700K interleaved examples, SIGMA supports compositional editing, selective attribute transfer, and fine-grained
multimodal alignment. Extensive experiments show that SIGMA improves controllability, cross-condition consistency, and visual quality across diverse editing and generation tasks, with substantial gains over Bagel on compositional tasks.
Read the full abstract ▾
All authors Xiaoyan Zhang, Zechen Bai, Haofan Wang, Yiren Song
Computer Vision
Francesco Taioli, Shiping Yang, Sonia Raychaudhuri et al. (+3 more)
Language-driven object
navigation requires
agents to interpret natural
language descriptions of target objects, which combine intrinsic and extrinsic attributes for instance recognition and commonsense
navigation. Existing methods either (i) use end-to-end trained models with
vision-
language embeddings, which struggle to generalize beyond training data and lack action-level explainability, or (ii) rely on modular zero-shot pipelines with large
language models (LLMs) and open-set object detectors, which suffer from error propagation, high computational cost, and difficulty integrating their
reasoning back into the
navigation policy. To this end, we propose a compact 3B-parameter
Vision-
Language-Action (VLA)
agent that performs human-like
embodied reasoning for both object recognition and action selection, removing the need for stitched multi-model pipelines. Instead of raw
embedding matching, our
agent employs explicit image-grounded
reasoning to directly answer "Is this the target object?" and "Why should I take this action?" The
reasoning process unfolds in three stages: "think", "think summary", and "action", yielding improved explainability, stronger generalization, and more efficient
navigation. Code and
dataset available upon acceptance.
Read the full abstract ▾
All authors Francesco Taioli, Shiping Yang, Sonia Raychaudhuri, Marco Cristani, Unnat Jain, Angel X Chang
Computer Vision
FlexID: Training-Free Flexible Identity Injection via Intent-Aware Modulation for Text-to-Image Generation
Guandong Li and Yijun Ding
Personalized text-to-image generation aims to seamlessly integrate specific identities into textual descriptions. However, existing training-free methods often rely on rigid visual feature injection, creating a conflict between identity fidelity and textual adaptability. To address this, we propose FlexID, a novel training-free framework utilizing intent-aware modulation. FlexID orthogonally decouples identity into two dimensions: a Semantic Identity Projector (SIP) that injects high-level priors into the
language space, and a Visual Feature Anchor (VFA) that ensures structural fidelity within the latent space. Crucially, we introduce a Context-Aware Adaptive Gating (CAG) mechanism that dynamically modulates the weights of these streams based on editing intent and
diffusion timesteps. By automatically relaxing rigid visual constraints when strong editing intent is detected, CAG achieves synergy between identity preservation and semantic variation. Extensive experiments on IBench demonstrate that FlexID achieves a state-of-the-art balance between identity consistency and text adherence, offering an efficient solution for complex narrative generation.
Read the full abstract ▾
Computer Vision
Revealing the Semantic Selection Gap in DINOv3 through Training-Free Few-Shot Segmentation
Hussni Mohd Zakir and Eric Tatt Wei Ho
Recent self-supervised
Vision Transformers (ViTs), such as DINOv3, provide rich feature representations for dense
vision tasks. This study investigates the intrinsic few-shot semantic
segmentation (FSS) capabilities of frozen DINOv3 features through a training-free baseline, FSSDINO, utilizing class-specific prototypes and Gram-
matrix refinement. Our results across binary, multi-class, and cross-domain (CDFSS)
benchmarks demonstrate that this minimal approach, applied to the final backbone layer, is highly competitive with specialized methods involving complex decoders or test-time adaptation. Crucially, we conduct an Oracle-guided layer analysis, identifying a significant performance gap between the standard last-layer features and globally optimal intermediate representations. We reveal a "Safest vs. Optimal" dilemma: while the Oracle proves higher performance is attainable, matching the results of compute-intensive adaptation methods, current unsupervised and support-guided selection metrics consistently yield lower performance than the last-layer baseline. This characterizes a "Semantic Selection Gap" in Foundation Models, a disconnect where traditional heuristics fail to reliably identify high-fidelity features. Our work establishes the "Last-Layer" as a deceptively strong baseline and provides a rigorous diagnostic of the latent semantic potentials in DINOv3.The code is publicly available at https://github.com/hussni0997/fssdino.
Read the full abstract ▾
Computer Vision
MUFASA: A Multi-Layer Framework for Slot Attention
Sebastian Bock, Leonie Schüßler, Krishnakant Singh et al. (+2 more)
Unsupervised object-centric learning (OCL) decomposes visual scenes into distinct entities. Slot
attention is a popular approach that represents individual objects as latent vectors, called slots. Current methods obtain these slot representations solely from the last layer of a pre-trained
vision transformer (ViT), ignoring valuable, semantically rich information encoded across the other layers. To better utilize this latent semantic information, we introduce MUFASA, a lightweight plug-and-play framework for slot-
attention-based approaches to unsupervised object
segmentation. Our model computes slot
attention across multiple feature layers of the ViT encoder, fully leveraging their semantic richness. We propose a fusion strategy to aggregate slots obtained on multiple layers into a unified object-centric representation. Integrating MUFASA into existing OCL methods improves their
segmentation results across multiple
datasets, setting a new state of the art while simultaneously improving training convergence with only minor inference overhead.
Read the full abstract ▾
All authors Sebastian Bock, Leonie Schüßler, Krishnakant Singh, Simone Schaub-Meyer, Stefan Roth
Computer Vision
LLM-Guided Diagnostic Evidence Alignment for Medical Vision-Language Pretraining under Limited Pairing
Huimin Yan, Liang Bai, Xian Yang et al. (+1 more)
Most existing CLIP-style medical
vision–
language pretraining methods rely on global or local
alignment with substantial paired data. However, global
alignment is easily dominated by non-diagnostic information, while local
alignment fails to integrate key diagnostic evidence. As a result, learning reliable diagnostic representations becomes difficult, which limits their applicability in medical scenarios with limited paired data. To address this issue, we propose an LLM-Guided Diagnostic Evidence
Alignment method (LGDEA), which shifts the pretraining objective toward evidence-level
alignment that is more consistent with the medical diagnostic process. Specifically, we leverage LLMs to extract key diagnostic evidence from radiology reports and construct a shared diagnostic evidence space, enabling evidence-aware cross-modal
alignment and allowing LGDEA to effectively exploit abundant unpaired medical images and reports, thereby substantially alleviating the reliance on paired data. Extensive experimental results demonstrate that our method achieves consistent and significant improvements on phrase grounding, image–text
retrieval, and zero-shot classification, and even rivals pretraining methods that rely on substantial paired data.
Read the full abstract ▾
All authors Huimin Yan, Liang Bai, Xian Yang, Long Chen
Computer Vision
Bi-temporal Image-driven Acute Stroke Evolution Analysis
Md Sazidur Rahman, Kjersti Engan, Kathinka Dæhli Kurz et al. (+1 more)
Acute ischemic stroke requires rapid treatment decisions that are strongly guided by emergency imaging. Admission computed
tomography perfusion (CTP) is commonly used to estimate the ischemic core, representing irreversibly damaged tissue, and the penumbra, representing hypoperfused but potentially salvageable tissue. Follow-up
diffusion-weighted MRI (DWI) is then used to define the final infarct. Existing imaging approaches primarily focus on core–penumbra
segmentation or final-infarct prediction, but provide limited insight into how heterogeneous penumbral tissue evolves after treatment. We propose a bi-temporal tissue phenotyping framework that links admission CTP signatures with follow-up DWI-defined tissue outcome using six outcome-aware region-of-interest classes. Admission tissue signatures are characterized using statistical, radiomic, and deep-learning features extracted from mJ-Net and nnU-Net representations. On an internal cohort (SUH), salvaged and infarcted penumbra showed consistent feature-space separation ($\tilde{\Delta}{\cos}=0.146$, $p<0.05$), while core tissue showed minimal separation by subsequent fate. The largest separation was observed between initially non-hypoperfused tissue that later infarcted and healthy contralateral tissue ($\tilde{\Delta}{\cos}=0.460$, $p<0.05$). Cross-
dataset evaluation on the publicly available ISLES'24
dataset showed similar trends, supporting the consistency of the observed feature-space patterns. These findings suggest that admission CTP contains outcome-associated tissue information beyond conventional core-penumbra delineation. The code is available at https://github.com/yokko123/bi-temporal-ctp-dwi-code.
Read the full abstract ▾
All authors Md Sazidur Rahman, Kjersti Engan, Kathinka Dæhli Kurz, Mahdieh Khanmohammadi
Computer Vision
Fine-Grained Cat Breed Recognition with Global Context Vision Transformer
Mowmita Parvin Hera, Md. Shahriar Mahmud Kallol, Shohanur Rahman Nirob et al. (+5 more)
Accurate identification of cat breeds from images is a challenging task due to subtle differences in fur patterns, facial structure, and color. In this paper, we present a deep learning-based approach for classifying cat breeds using a subset of the Oxford-IIIT Pet
Dataset, which contains high-resolution images of various domestic breeds. We employed the Global Context
Vision Transformer (GCViT) architecture-tiny for cat breed recognition. To improve model generalization, we used extensive data augmentation, including rotation, horizontal flipping, and brightness adjustment. Experimental results show that the GCViT-Tiny model achieved a test accuracy of 92.00% and validation accuracy of 94.54%. These findings highlight the effectiveness of
transformer-based architectures for fine-grained image classification tasks. Potential applications include veterinary diagnostics, animal shelter management, and mobile-based breed recognition systems. We also provide a hugging face demo at https://huggingface.co/spaces/bfarhad/cat-breed-classifier.
Read the full abstract ▾
All authors Mowmita Parvin Hera, Md. Shahriar Mahmud Kallol, Shohanur Rahman Nirob, Md. Badsha Bulbul, Jubayer Ahmed, M. Zhourul Islam, Hazrat Ali, Mohammmad Farhad Bulbul
Computer Vision
Evaluating Object-Centric Models beyond Object Discovery
Krishnakant Singh, Simone Schaub-Meyer and Stefan Roth
Object-centric learning (OCL) aims to learn structured scene representations that support compositional generalization and robustness to out-of-distribution (OOD) data. However, OCL models are often not evaluated regarding these goals. Instead, most prior work focuses on evaluating OCL models solely through object discovery and simple
reasoning tasks, such as probing the representation via image classification. We identify two limitations in existing
benchmarks: (1) They provide limited insights on the representation usefulness of OCL models, and (2) localization and representation usefulness are assessed using disjoint metrics. To address (1), we use instruction-tuned VLMs as evaluators, enabling scalable benchmarking across diverse VQA
datasets to measure how well VLMs leverage OCL representations for complex
reasoning tasks. To address (2), we introduce a unified evaluation task and metric that jointly assess localization (where) and representation usefulness (what), thereby eliminating inconsistencies introduced by disjoint evaluation. Finally, we include a simple multi-feature reconstruction baseline as a reference point.
Read the full abstract ▾
Computer Vision
CA-YOLO: Cross Attention Empowered YOLO for Biomimetic Localization
Zhen Zhang, Qing Zhao, Xiuhe Li et al. (+6 more)
In modern complex environments, achieving accurate and efficient target localization is essential in numerous fields. However, existing systems often face limitations in both accuracy and the ability to recognize small targets. In this study, we propose a bionic stabilized localization system based on CA-YOLO, designed to enhance both target localization accuracy and small target recognition capabilities. Acting as the "
brain" of the system, the target detection
algorithm emulates the visual focusing mechanism of animals by integrating bionic modules into the YOLO backbone network. These modules include the introduction of a small target detection head and the development of a Characteristic Fusion
Attention Mechanism (CFAM). Furthermore, drawing inspiration from the human Vestibulo-Ocular Reflex (VOR), a bionic pan-tilt tracking control strategy is developed, which incorporates central positioning, stability optimization, adaptive control coefficient adjustment, and an intelligent recapture function. The experimental results show that CA-YOLO outperforms the original model on standard
datasets (COCO and VisDrone), with average accuracy metrics improved by 3.94%and 4.90%, respectively.Further time-sensitive target localization experiments validate the effectiveness and practicality of this bionic stabilized localization system.
Read the full abstract ▾
All authors Zhen Zhang, Qing Zhao, Xiuhe Li, Cheng Wang, Guoqiang Zhu, Yu Zhang, Yining Huo, Hongyi Yu, Yi Zhang
Computer Vision
Adaptive Image Zoom-in with Bounding Box Transformation for UAV Object Detection
Tao Wang, Chenyu Lin, Chenwei Tang et al. (+5 more)
Detecting objects from
UAV-captured images is challenging due to the small object size. In this work, a simple and efficient adaptive zoom-in framework is explored for object detection on
UAV images. The main motivation is that the foreground objects are generally smaller and sparser than those in common scene images, which hinders the optimization of effective object detectors. We thus aim to zoom in adaptively on the objects to better capture object features for the detection task. To achieve the goal, two core designs are required: \textcolor{black}{i) How to conduct non-uniform zooming on each image efficiently? ii) How to enable object detection training and inference with the zoomed image space?} Correspondingly, a lightweight offset prediction scheme coupled with a novel box-based zooming objective is introduced to learn non-uniform zooming on the input image. Based on the learned zooming transformation, a corner-aligned bounding box transformation method is proposed. The method warps the ground-truth bounding boxes to the zoomed space to learn object detection, and warps the predicted bounding boxes back to the original space during inference. We conduct extensive experiments on three representative
UAV object detection
datasets, including VisDrone, UAVDT, and SeaDronesSee. The proposed ZoomDet is architecture-independent and can be applied to an arbitrary object detection architecture. Remarkably, on the SeaDronesSee
dataset, ZoomDet offers more than 8.4 absolute gain of mAP with a Faster R-CNN model, with only about 3 ms additional latency. The code is available at https://github.com/twangnh/zoomdet_code.
Read the full abstract ▾
All authors Tao Wang, Chenyu Lin, Chenwei Tang, Jizhe Zhou, Deng Xiong, Jianan Li, Jian Zhao, Jiancheng Lv
Computer Vision
IM-Animation: An Implicit Motion Representation for Identity-decoupled Character Animation
Zhufeng Xu, Xuan Gao, Feng-Lin Liu et al. (+6 more)
Recent progress in
video diffusion models has markedly advanced character animation, which synthesizes motioned
videos by animating a static identity image according to a driving
video. Explicit methods represent motion using skeleton, DWPose or other explicit structured signals, but struggle to handle spatial mismatches and varying body scales. %proportions. Implicit methods, on the other hand, capture high-level implicit motion semantics directly from the driving
video, but suffer from identity leakage and
entanglement between motion and appearance. To address the above challenges, we propose a novel implicit motion representation that compresses per-frame motion into compact 1D motion
tokens. This design relaxes strict spatial constraints inherent in 2D representations and effectively prevents identity information leakage from the motion
video. Furthermore, we design a temporally consistent mask
token-based retargeting module that enforces a temporal training bottleneck, mitigating interference from the source images' motion and improving retargeting consistency. Our methodology employs a three-stage training strategy to enhance the training efficiency and ensure high fidelity. Extensive experiments demonstrate that our implicit motion representation and the propose IM-Animation's generative capabilities are achieve superior or competitive performance compared with state-of-the-art methods.
Read the full abstract ▾
All authors Zhufeng Xu, Xuan Gao, Feng-Lin Liu, Haoxian Zhang, Zhixue Fang, Yu-Kun Lai, Xiaoqiang Liu, Pengfei Wan, Lin Gao
Computer Vision
Learning Brain Representation with Hierarchical Visual Embeddings
Jiawen Zheng, Haonan Jia, Ming Li et al. (+4 more)
Decoding visual representations from
brain signals has attracted significant
attention in both neuroscience and artificial intelligence. However, the degree to which
brain signals truly encode visual information remains unclear. Current visual decoding approaches explore various
brain-image
alignment strategies, yet most emphasize high-level semantic features while neglecting pixel-level details, thereby limiting our understanding of the human visual system. In this paper, we propose a
brain-image
alignment strategy that leverages multiple pre-trained visual encoders with distinct inductive biases to capture hierarchical and multi-scale visual representations, while employing a contrastive learning objective to achieve effective
alignment between
brain signals and visual
embeddings. Furthermore, we introduce a Fusion Prior, which learns a stable mapping on large-scale visual data and subsequently matches
brain features to this pre-trained prior, thereby enhancing distributional consistency across modalities. Extensive quantitative and qualitative experiments demonstrate that our method achieves a favorable balance between
retrieval accuracy and reconstruction fidelity.
Read the full abstract ▾
All authors Jiawen Zheng, Haonan Jia, Ming Li, Yuhui Zheng, Yufeng Zeng, Yang Gao, Chen Liang
Computer Vision
Thermal odometry and dense mapping using learned odometry and Gaussian splatting
Tianhao Zhou, Yujia Chen, Zhihao Zhan et al. (+2 more)
Thermal infrared sensors, with wavelengths longer than smoke particles, can capture imagery independent of darkness,
dust, and smoke. This robustness has made them increasingly valuable for motion estimation and environmental perception in robotics, particularly in adverse conditions. Existing thermal odometry and mapping approaches, however, are predominantly
geometric and often fail across diverse
datasets while lacking the ability to produce dense maps. Motivated by the efficiency and high-quality reconstruction ability of recent Gaussian Splatting (GS) techniques, we propose TOM-GS, a thermal odometry and mapping method that integrates learning-based odometry with GS-based dense mapping. TOM-GS is among the first GS-based SLAM systems tailored for thermal cameras, featuring dedicated thermal image enhancement and
monocular depth integration. Extensive experiments on motion estimation and novel-view rendering demonstrate that TOM-GS outperforms existing learning-based methods, confirming the benefits of learning-based pipelines for robust thermal odometry and dense reconstruction.
Read the full abstract ▾
All authors Tianhao Zhou, Yujia Chen, Zhihao Zhan, Yuhang Ming, Jianzhu Huai
Computer Vision
GlobalWasteData: A Large-Scale, Integrated Dataset for Robust Waste Classification and Environmental Monitoring
Misbah Ijaz, Saif Ur Rehman Khan, Abd Ur Rehman et al. (+4 more)
The growing amount of waste is a problem for the environment that requires efficient sorting techniques for various kinds of waste. An automated waste classification system is used for this purpose. The effectiveness of these Artificial Intelligence (AI) models depends on the quality and accessibility of publicly available
datasets, which provide the basis for training and analyzing classification
algorithms. Although several public waste classification
datasets exist, they remain fragmented, inconsistent, and biased toward specific environments. Differences in class names, annotation formats, image conditions, and class distributions make it difficult to combine these
datasets or train models that generalize well to real world scenarios. To address these issues, we introduce the GlobalWasteData (GWD) archive, a large scale
dataset of 89,807 images across 14 main categories, annotated with 68 distinct subclasses. We compile this novel integrated GWD archive by merging multiple publicly available
datasets into a single, unified resource. This GWD archive offers consistent labeling, improved domain diversity, and more balanced class representation, enabling the development of robust and generalizable waste recognition models. Additional preprocessing steps such as quality filtering, duplicate removal, and metadata generation further improve
dataset reliability. Overall, this
dataset offers a strong foundation for Machine Learning (ML) applications in environmental monitoring, recycling automation, and waste identification, and is publicly available to promote future research and reproducibility.
Read the full abstract ▾
All authors Misbah Ijaz, Saif Ur Rehman Khan, Abd Ur Rehman, Tayyaba Asif, Sebastian Vollmer, Andreas Dengel, Muhammad Nabeel Asim
Computer Vision
SpatialReward: Bridging the Perception Gap in Online RL for Image Editing via Explicit Spatial Reasoning
Yancheng Long, Yankai Yang, Hongyang Wei et al. (+12 more)
Online Reinforcement Learning (RL) offers a promising avenue for complex image editing but is currently constrained by the scarcity of reliable and fine-grained reward signals. Existing evaluators frequently struggle with a critical perception gap we term "
Attention Collapse," where models neglect cross-image comparisons and fail to capture fine-grained details, resulting in inaccurate perception and miscalibrated scores. To address these limitations, we propose SpatialReward, a reward model that enforces precise verification via explicit spatial
reasoning. By anchoring
reasoning to predicted edit regions, SpatialReward grounds semantic judgments in pixel-level evidence, significantly enhancing evaluative accuracy. Trained on a curated 260k spatial-aware
dataset, our model achieves state-of-the-art performance on MMRB2 and EditReward-Bench, and outperforms proprietary evaluators on our proposed MultiEditReward-Bench. Furthermore, SpatialReward serves as a robust signal in online RL, boosting OmniGen2 by +0.90 on GEdit-Bench–surpassing the leading discriminative model and doubling the gain of GPT-4.1 (+0.45). These results demonstrate that spatial
reasoning is essential for unlocking effective
alignment in image editing.
Read the full abstract ▾
All authors Yancheng Long, Yankai Yang, Hongyang Wei, Wei Chen, Tianke Zhang, Haonan fan, Changyi Liu, Kaiyu Jiang, Jiankang Chen, Kaiyu Tang, Bin Wen, Fan Yang, Tingting Gao, Han Li, Shuo Yang
Computer Vision
SoulX-FlashHead: Oracle-guided Generation of Infinite Real-time Streaming Talking Heads
Tan Yu, Qian Qiao, Le Shen et al. (+9 more)
Achieving a balance between high-fidelity visual quality and low-latency streaming remains a formidable challenge in audio-driven portrait generation. Existing large-scale models often suffer from prohibitive computational costs, while lightweight alternatives typically compromise on holistic facial representations and temporal stability. In this paper, we propose SoulX-FlashHead, a unified 1.3B-parameter framework designed for real-time, infinite-length, and high-fidelity streaming
video generation. To address the instability of audio features in streaming scenarios, we introduce Streaming-Aware Spatiotemporal Pre-training equipped with a Temporal Audio Context Cache mechanism, which ensures robust feature extraction from short audio fragments. Furthermore, to mitigate the error accumulation and identity drift inherent in long-sequence autoregressive generation, we propose Oracle-Guided Bidirectional
Distillation, leveraging ground-truth motion priors to provide precise physical guidance. We also present VividHead, a large-scale, high-quality
dataset containing 782 hours of strictly aligned footage to support robust training. Extensive experiments demonstrate that SoulX-FlashHead achieves state-of-the-art performance on HDTF and VFHQ
benchmarks. Notably, our Lite variant achieves an inference speed of 96 FPS on a single NVIDIA RTX 4090, facilitating ultra-fast interaction without sacrificing visual coherence.
Read the full abstract ▾
All authors Tan Yu, Qian Qiao, Le Shen, Ke Zhou, Jincheng Hu, Dian Sheng, Bo Hu, Haoming Qin, Jun Gao, Changhai Zhou, Shunshun Yin, Siyuan Liu
Computer Vision
PTB-XL-Image-17K: A Large-Scale Synthetic ECG Image Dataset with Comprehensive Ground Truth for Deep Learning-Based Digitization
Naqcho Ali Mehdi and Aamir Ali Drigh
Electrocardiogram (ECG) digitization-converting paper-based or scanned ECG images back into time-series signals-is critical for leveraging decades of legacy clinical data in modern deep learning applications. However, progress has been hindered by the lack of large-scale
datasets providing both ECG images and their corresponding ground truth signals with comprehensive annotations. We introduce PTB-XL-Image-17K, a complete synthetic ECG image
dataset comprising 17,271 high-quality 12-lead ECG images generated from the PTB-XL signal database. Our
dataset uniquely provides five complementary data types per sample: (1) realistic ECG images with authentic grid patterns and annotations (50% with visible grid, 50% without), (2) pixel-level
segmentation masks, (3) ground truth time-series signals, (4) bounding box annotations in YOLO format for both lead regions and lead name labels, and (5) comprehensive metadata including visual parameters and patient information. We present an open-source Python framework enabling customizable
dataset generation with controllable parameters including paper speed (25/50 mm/s), voltage scale (5/10 mm/mV), sampling rate (500 Hz), grid appearance (4 colors), and waveform characteristics. The
dataset achieves 100% generation success rate with an average processing time of 1.35 seconds per sample. PTB-XL-Image-17K addresses critical gaps in ECG digitization research by providing the first large-scale resource supporting the complete pipeline: lead detection, waveform
segmentation, and signal extraction with full ground truth for rigorous evaluation. The
dataset, generation framework, and documentation are publicly available at https://github.com/naqchoalimehdi/PTB-XL-Image-17K and https://doi.org/10.5281/zenodo.18197519.
Read the full abstract ▾
Computer Vision
Perspective-aware fusion of incomplete depth maps and surface normals for accurate 3D reconstruction
Ondrej Hlinka, Georg Kaniak and Christian Kapeller
We address the problem of reconstructing 3D surfaces from depth and surface normal maps acquired by a sensor system based on a single perspective camera. Depth and normal maps can be obtained through techniques such as structured-light scanning and photometric stereo, respectively. We propose a perspective-aware log-depth fusion approach that extends existing orthographic gradient-based depth-normals fusion methods by explicitly accounting for perspective projection, leading to metrically accurate 3D reconstructions. Additionally, the method handles missing depth measurements by leveraging available surface normal information to inpaint gaps. Experiments on the DiLiGenT-MV data set demonstrate the effectiveness of our approach and highlight the importance of perspective-aware depth-normals fusion.
Read the full abstract ▾