Computer Vision
Venus: Benchmarking and Empowering Multimodal Large Language Models for Aesthetic Guidance and Cropping
Tianxiang Du, Hulingxiao He and Yuxin Peng
The widespread use of smartphones has made photography ubiquitous, yet a clear gap remains between ordinary users and professional photographers, who can identify aesthetic issues and provide actionable shooting guidance during capture. We define this capability as aesthetic guidance (AG) – an essential but largely underexplored domain in computational aesthetics. Existing
multimodal large
language models (MLLMs) primarily offer overly positive feedback, failing to identify issues or provide actionable guidance. Without AG capability, they cannot effectively identify distracting regions or optimize compositional balance, thus also struggling in aesthetic cropping, which aims to refine photo composition through reframing after capture. To address this, we introduce AesGuide, the first large-scale AG
dataset and
benchmark with 10,748 photos annotated with aesthetic scores, analyses, and guidance. Building upon it, we propose Venus, a two-stage framework that first empowers MLLMs with AG capability through progressively complex aesthetic questions and then activates their aesthetic cropping power via CoT-based rationales. Extensive experiments show that Venus substantially improves AG capability and achieves state-of-the-art (SOTA) performance in aesthetic cropping, enabling interpretable and interactive aesthetic refinement across both stages of photo creation. Code is available at https://github.com/PKU-ICST-MIPL/Venus_CVPR2026.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.MM
MSVBench: Towards Human-Level Evaluation of Multi-Shot Video Generation
Haoyuan Shi, Yunxin Li, Nanhao Deng et al. (+5 more)
The
evolution of
video generation toward complex, multi-shot narratives has exposed a critical deficit in current evaluation methods. Existing
benchmarks remain anchored to single-shot paradigms, lacking the comprehensive story assets and cross-shot metrics required to assess long-form coherence and appeal. To bridge this gap, we introduce MSVBench, the first comprehensive
benchmark featuring hierarchical scripts and reference images tailored for Multi-Shot
Video generation. We propose a hybrid evaluation framework that synergizes the high-level semantic
reasoning of Large
Multimodal Models (LMMs) with the fine-grained perceptual rigor of domain-specific expert models. Evaluating 20
video generation methods across diverse paradigms, we find that current models–despite strong visual fidelity–primarily behave as visual interpolators rather than true world models. We further validate the reliability of our
benchmark by demonstrating a state-of-the-art Spearman's rank correlation of 94.4% with human judgments. Finally, MSVBench extends beyond evaluation by providing a scalable supervisory signal. Fine-tuning a lightweight model on its pipeline-refined
reasoning traces yields human-aligned performance comparable to commercial models like Gemini-2.5-Flash.
Read the full abstract ▾
All authors Haoyuan Shi, Yunxin Li, Nanhao Deng, Zhenran Xu, Xinyu Chen, Longyue Wang, Baotian Hu, Min Zhang
Computer Vision
SpikeTrack: A Spike-driven Framework for Efficient Visual Tracking
Qiuyang Zhang, Jiujun Cheng, Qichao Mao et al. (+5 more)
Spiking Neural Networks (SNNs) promise energy-efficient
vision, but applying them to RGB visual tracking remains difficult: Existing SNN tracking frameworks either do not fully align with spike-driven computation or do not fully leverage
neurons' spatiotemporal dynamics, leading to a trade-off between efficiency and accuracy. To address this, we introduce SpikeTrack, a spike-driven framework for energy-efficient RGB object tracking. SpikeTrack employs a novel asymmetric design that uses asymmetric timestep expansion and unidirectional information flow, harnessing spatiotemporal dynamics while cutting computation. To ensure effective unidirectional information transfer between branches, we design a
memory-
retrieval module inspired by neural inference mechanisms. This module recurrently queries a compact
memory initialized by the template to retrieve target cues and sharpen target perception over time. Extensive experiments demonstrate that SpikeTrack achieves the state-of-the-art among SNN-based trackers and remains competitive with advanced ANN trackers. Notably, it surpasses TransT on LaSOT
dataset while consuming only 1/26 of its energy. To our knowledge, SpikeTrack is the first spike-driven framework to make RGB tracking both accurate and energy efficient. The code and models are available at https://github.com/faicaiwawa/SpikeTrack.
Read the full abstract ▾
All authors Qiuyang Zhang, Jiujun Cheng, Qichao Mao, Cong Liu, Yu Fang, Yuhong Li, Mengying Ge, Shangce Gao
Computer Vision · Cross-listed from eess.IV
Extending 2D foundational DINOv3 representations to 3D segmentation of neonatal brain MR images
Annayah Usman, Behraj Khan and Tahir Qasim Syed
Precise volumetric delineation of hippocampal structures is essential for quantifying neurodevelopmental trajectories in pre-term and term infants, where subtle morphological variations may carry prognostic significance. While foundation encoders trained on large-scale visual data offer discriminative representations, their 2D formulation is a limitation with respect to the $3$D organization of
brain anatomy. We propose a volumetric
segmentation strategy that reconciles this tension through a structured window-based disassembly-reassembly mechanism: the global MRI volume is decomposed into non-overlapping 3D windows or sub-cubes, each processed via a separate decoding arm built upon frozen high-fidelity features, and subsequently reassembled prior to a ground-truth correspendence using a dense-prediction head. This architecture preserves constant a decoder
memory footprint while forcing predictions to lie within an anatomically consistent geometry. Evaluated on the ALBERT
dataset for hippocampal
segmentation, the proposed approach achieves a Dice score of 0.65 for a single 3D window. The method demonstrates that volumetric anatomical structure could be recovered from frozen 2D foundation representations through structured compositional decoding, and offers a principled and generalizable extension for foundation models for 3D medical applications.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Clinically-aligned ischemic stroke segmentation and ASPECTS scoring on NCCT imaging using a slice-gated loss on foundation representations
Hiba Azeem, Behraj Khan and Tahir Qasim Syed
Rapid infarct assessment on non-contrast CT (NCCT) is essential for acute ischemic stroke management. Most deep learning methods perform pixel-wise
segmentation without modeling the structured anatomical
reasoning underlying ASPECTS scoring, where basal ganglia (BG) and supraganglionic (SG) levels are clinically interpreted in a coupled manner. We propose a clinically aligned framework that combines a frozen DINOv3 backbone with a lightweight decoder and introduce a Territory-Aware Gated Loss (TAGL) to enforce BG-SG consistency during training. This anatomically informed supervision adds no inference-time complexity. Our method achieves a Dice score of 0.6385 on AISD, outperforming prior CNN and foundation-model baselines. On a proprietary ASPECTS
dataset, TAGL improves mean Dice from 0.698 to 0.767. These results demonstrate that integrating foundation representations with structured clinical priors improves NCCT stroke
segmentation and ASPECTS delineation.
Read the full abstract ▾
Computer Vision
Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought
Kesen Zhao, Beier Zhu, Junbao Zhou et al. (+3 more)
Recent MLLMs increasingly rely on visual chain-of-thought to perform region-grounded
reasoning over images. However, existing approaches ground regions via either textified coordinates (causing modality mismatch and semantic fragmentation) or fixed-granularity patches that both limit precise region selection and often require non-trivial architectural changes. In this paper, we propose Numerical Visual Chain-of-Thought (NV-CoT), a framework that enables MLLMs to reason over images using continuous numerical coordinates. NV-CoT expands the MLLM action space from discrete vocabulary
tokens to a continuous Euclidean space, allowing models to directly generate bounding-box coordinates as actions with only minimal architectural modification. The framework supports both supervised fine-tuning and reinforcement learning. In particular, we replace categorical
token policies with a Gaussian (or Laplace) policy over coordinates and introduce stochasticity via reparameterized sampling, making NV-CoT fully compatible with GRPO-style policy optimization. Extensive experiments on three
benchmarks against 11 representative visual
reasoning baselines demonstrate that NV-CoT significantly improves localization precision and final answer accuracy, while also accelerating training convergence, validating the effectiveness of continuous-action visual
reasoning in MLLMs. Project page: https://kesenzhao.github.io/NV-CoT/
Read the full abstract ▾
All authors Kesen Zhao, Beier Zhu, Junbao Zhou, Xingyu Zhu, Zhongqi Yue, Hanwang Zhang
Computer Vision
SwitchCraft: Training-Free Multi-Event Video Generation with Attention Controls
Qianxun Xu, Chenxi Song, Yujun Cai et al. (+1 more)
Recent advances in text-to-
video diffusion models have enabled high-fidelity and temporally coherent
videos synthesis. However, current models are predominantly optimized for single-event generation. When handling multi-event
prompts, without explicit temporal grounding, such models often produce blended or collapsed scenes that break the intended narrative. To address this limitation, we present SwitchCraft, a training-free framework for multi-event
video generation. Our key insight is that uniform
prompt injection across time ignores the correspondence between events and frames. To this end, we introduce Event-Aligned Query Steering (EAQS), which steers frame-level
attention to align with relevant event
prompts. Furthermore, we propose Auto-Balance Strength Solver (ABSS), which adaptively balances steering strength to preserve temporal consistency and visual fidelity. Extensive experiments demonstrate that SwitchCraft substantially improves
prompt alignment, event clarity, and scene consistency compared with existing baselines, offering a simple yet effective solution for multi-event
video generation.
Read the full abstract ▾
All authors Qianxun Xu, Chenxi Song, Yujun Cai, Chi Zhang
Computer Vision
GDA-YOLO11: Amodal Instance Segmentation for Occlusion-Robust Robotic Fruit Harvesting
Caner Beldek, Emre Sariyildiz, Son Lam Phung et al. (+1 more)
Occlusion remains a critical challenge in robotic fruit harvesting, as undetected or inaccurately localised fruits often results in substantial crop losses. To mitigate this issue, we propose a harvesting framework using a new amodal
segmentation model, GDA-YOLO11, which incorporates architectural improvements and an updated asymmetric mask loss. The proposed model is trained on a modified version of a public citrus
dataset and evaluated on both the base
dataset and occlusion-sensitive subsets with varying occlusion levels. Within the framework, full fruit masks, including invisible regions, are inferred by GDA-YOLO11, and picking points are subsequently estimated using the Euclidean distance transform. These points are then projected into 3D coordinates for robotic harvesting execution. Experiments were conducted using real citrus fruits in a controlled environment simulating occlusion scenarios. Notably, to the best of our knowledge, this study provides the first practical demonstration of amodal instance
segmentation in robotic fruit harvesting. GDA-YOLO11 achieves a precision of 0.844, recall of 0.846, mAP@50 of 0.914, and mAP@50:95 of 0.636, outperforming YOLO11n by 5.1%, 1.3%, and 1.0% in precision, mAP@50, and mAP@50:95, respectively. The framework attains harvesting success rates of 92.59%, 85.18%, 48.14%, and 22.22% at zero to high occlusion levels, improving success by 3.5% under medium and high occlusion. These findings demonstrate that GDA-YOLO11 enhances occlusion robust
segmentation and streamlines perception-to-action integration, paving the way for more reliable autonomous systems in agriculture.
Read the full abstract ▾
All authors Caner Beldek, Emre Sariyildiz, Son Lam Phung, Gursel Alici
Computer Vision
CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answering
Yuyang Hong, Jiaqi Gu, Yujin Lou et al. (+7 more)
Knowledge-based visual question answering (KB-VQA) demonstrates significant potential for handling knowledge-intensive tasks. However, conflicts arise between static parametric knowledge in
vision language models (VLMs) and dynamically retrieved information due to the static model knowledge from pre-training. The outputs either ignore retrieved contexts or exhibit inconsistent integration with parametric knowledge, posing substantial challenges for KB-VQA. Current knowledge conflict mitigation methods primarily adapted from
language-based approaches, focusing on context-level conflicts through engineered prompting strategies or context-aware decoding mechanisms. However, these methods neglect the critical role of visual information in conflicts and suffer from redundant retrieved contexts, which impair accurate conflict identification and effective mitigation. To address these limitations, we propose CC-VQA: a novel training-free, conflict- and correlation-aware method for KB-VQA. Our method comprises two core components: (1)
Vision-Centric Contextual Conflict
Reasoning, which performs visual-semantic conflict analysis across internal and external knowledge contexts; and (2) Correlation-Guided Encoding and Decoding, featuring positional encoding compression for low-correlation statements and adaptive decoding using correlation-weighted conflict scoring. Extensive evaluations on E-VQA, InfoSeek, and OK-VQA
benchmarks demonstrate that CC-VQA achieves state-of-the-art performance, yielding absolute accuracy improvements of 3.3% to 6.4% compared to existing methods. Code is available at https://github.com/cqu-student/CC-VQA.
Read the full abstract ▾
All authors Yuyang Hong, Jiaqi Gu, Yujin Lou, Lubin Fan, Qi Yang, Ying Wang, Kun Ding, Yue Wu, Shiming Xiang, Jieping Ye
Computer Vision
AHAP: Reconstructing Arbitrary Humans from Arbitrary Perspectives with Geometric Priors
Xiaozhen Qiao, Wenjia Wang, Zhiyuan Zhao et al. (+4 more)
Reconstructing 3D humans from images captured at multiple perspectives typically requires pre-calibration, like using checkerboards or MVS
algorithms, which limits scalability and applicability in diverse real-world scenarios. In this work, we present AHAP (Reconstructing Arbitrary Humans from Arbitrary Perspectives), a feed-forward framework for reconstructing arbitrary humans from arbitrary camera perspectives without requiring camera calibration. Our core lies in the effective fusion of multi-view geometry to assist human association, reconstruction and localization. Specifically, we use a Cross-View Identity Association module through learnable person queries and soft assignment, supervised by contrastive learning to resolve cross-view human identity association. A Human Head fuses cross-view features and scene context for SMPL prediction, guided by cross-view reprojection losses to enforce body pose consistency. Additionally, multi-view geometry eliminates the depth ambiguity inherent in
monocular methods, providing more precise 3D human localization through multi-view triangulation. Experiments on EgoHumans and EgoExo4D demonstrate that AHAP achieves competitive performance on both world-space human reconstruction and camera pose estimation, while being 180$\times$ faster than optimization-based approaches.
Read the full abstract ▾
All authors Xiaozhen Qiao, Wenjia Wang, Zhiyuan Zhao, Jiacheng Sun, Ping Luo, Hongyuan Zhang, Xuelong Li
Computer Vision
Micro-expression Recognition Based on Dual-branch Feature Extraction and Fusion
Mingjie Zhang, Bo Li, Wanting Liu et al. (+5 more)
Micro-expressions, characterized by transience and subtlety, pose challenges to existing optical flow-based recognition methods. To address this, this paper proposes a dual-branch micro-expression feature extraction network integrated with parallel
attention. Key contributions include: 1) a residual network designed to alleviate gradient anishing and network degradation; 2) an Inception network constructed to enhance model representation and suppress interference from irrelevant regions; 3) an adaptive feature fusion module developed to integrate dual-branch features. Experiments on the CASME II
dataset demonstrate that the proposed method achieves 74.67% accuracy, outperforming LBP-TOP (by 11.26%), MSMMT (by 3.36%), and other comparative methods.
Read the full abstract ▾
All authors Mingjie Zhang, Bo Li, Wanting Liu, Hongyan Cui, Yue Li, Qingwen Li, Hong Li, Ge Gao
Computer Vision
Dongxu Zhang, Yiding Sun, Pengcheng Li et al. (+12 more)
While
Multimodal Large
Language Models (MLLMs) demonstrate proficiency in 2D scenes, extending their perceptual intelligence to 3D point cloud understanding remains a significant challenge. Current approaches focus primarily on aligning 3D features with pre-trained models. However, they typically treat
geometric reasoning as an implicit mapping process. These methods bypass intermediate logical steps and consequently suffer from
geometric hallucinations. They confidently generate plausible responses that fail to ground in precise structural details. To bridge this gap, we present PointCoT, a novel framework that empowers MLLMs with explicit Chain-of-Thought (CoT)
reasoning for 3D data. We advocate for a Look, Think, then Answer paradigm. In this approach, the model is supervised to generate geometry-grounded rationales before predicting final answers. To facilitate this, we construct Point-Reason-Instruct, a large-scale
benchmark comprising $\sim$86k instruction-tuning samples with hierarchical CoT annotations. By leveraging a dual-stream multi-modal architecture, our method synergizes semantic appearance with
geometric truth. Extensive experiments demonstrate that PointCoT achieves state-of-the-art performance on complex
reasoning tasks.
Read the full abstract ▾
All authors Dongxu Zhang, Yiding Sun, Pengcheng Li, Yumou Liu, Hongqiang Lin, Haoran Xu, Xiaoxuan Mu, Liang Lin, Wenbiao Yan, Ning Yang, Chaowei Fang, Juanjuan Zhao, Jihua Zhu, Conghui He, Cheng Tan
Computer Vision · Cross-listed from cs.RO
Haoxuan Xu, Tianfu Li, Wenbo Chen et al. (+4 more)
Vision-
Language Navigation (VLN)
agents often struggle with long-horizon
reasoning in unseen environments, particularly when facing ambiguous, coarse-grained instructions. While recent advances use knowledge graph to enhance
reasoning, the potential of
multimodal event knowledge inspired by human episodic
memory remains underexplored. In this work, we propose an event-centric knowledge enhancement strategy for automated process knowledge mining and feature fusion to solve coarse-grained instruction and long-horizon
reasoning in VLN task. First, we construct YE-KG, the first large-scale
multimodal spatiotemporal knowledge graph, with over 86k nodes and 83k edges, derived from real-world indoor
videos. By leveraging
multimodal large
language models (i.e., LLaVa, GPT4), we extract unstructured
video streams into structured semantic-action-effect events to serve as explicit episodic
memory. Second, we introduce STE-VLN, which integrates the above graph into VLN models via a Coarse-to-Fine Hierarchical
Retrieval mechanism. This allows
agents to retrieve causal event sequences and dynamically fuse them with
egocentric visual observations. Experiments on REVERIE, R2R, and R2R-CE
benchmarks demonstrate the efficiency of our event-centric strategy, outperforming state-of-the-art approaches across diverse action spaces. Our data and code are available on the project website https://sites.google.com/view/y-event-kg/.
Read the full abstract ▾
All authors Haoxuan Xu, Tianfu Li, Wenbo Chen, Yi Liu, Xingxing Zuo, Yaoxian Song, Haoang Li
Computer Vision
Leveraging Geometric Prior Uncertainty and Complementary Constraints for High-Fidelity Neural Indoor Surface Reconstruction
Qiyu Feng, Jiwei Shan, Shing Shin Cheng et al. (+1 more)
Neural implicit surface reconstruction with signed distance function has made significant progress, but recovering fine details such as thin structures and complex geometries remains challenging due to unreliable or noisy
geometric priors. Existing approaches rely on implicit uncertainty that arises during optimization to filter these priors, which is indirect and inefficient, and masking supervision in high-uncertainty regions further leads to under-constrained optimization. To address these issues, we propose GPU-SDF, a neural implicit framework for indoor surface reconstruction that leverages
geometric prior uncertainty and complementary constraints. We introduce a self-supervised module that explicitly estimates prior uncertainty without auxiliary networks. Based on this estimation, we design an uncertainty-guided loss that modulates prior influence rather than discarding it, thereby retaining weak but informative cues. To address regions with high prior uncertainty, GPU-SDF further incorporates two complementary constraints: an edge distance field that strengthens boundary supervision and a multi-view consistency regularization that enforces
geometric coherence. Extensive experiments confirm that GPU-SDF improves the reconstruction of fine details and serves as a plug-and-play enhancement for existing frameworks. Source code will be available at https://github.com/IRMVLab/GPU-SDF
Read the full abstract ▾
All authors Qiyu Feng, Jiwei Shan, Shing Shin Cheng, Hesheng Wang
Computer Vision
Topology-Driven Transferability Estimation of Medical Foundation Models for Segmentation
Jiaqi Tang, Shaoyang Zhang, Xiaoqi Wang et al. (+3 more)
The advent of large-scale self-supervised learning (SSL) has produced a vast zoo of medical foundation models. However, selecting optimal medical foundation models for specific
segmentation tasks remains a computational bottleneck. Existing Transferability Estimation (TE) metrics, primarily designed for classification, rely on global statistical assumptions and fail to capture the topological complexity essential for dense prediction. We propose a novel
Topology-Driven Transferability Estimation framework that evaluates
manifold tractability rather than statistical overlap. Our approach introduces three components: (1) Global Representation
Topology Divergence (GRTD), utilizing Minimum Spanning Trees to quantify feature-label structural isomorphism; (2) Local Boundary-Aware Topological Consistency (LBTC), which assesses
manifold separability specifically at critical anatomical boundaries; and (3) Task-Adaptive Fusion, which dynamically integrates global and local metrics based on the semantic cardinality of the target task. Validated on the large-scale OpenMind
benchmark across diverse anatomical targets and SSL foundation models, our approach significantly outperforms state-of-the-art baselines by around 31% relative improvement in the weighted Kendall metric, providing a robust, training-free proxy for efficient model selection without the cost of fine-tuning. The code will be made publicly available upon acceptance.
Read the full abstract ▾
All authors Jiaqi Tang, Shaoyang Zhang, Xiaoqi Wang, Jiaying Zhou, Yang Liu, Qingchao Chen
Computer Vision
Half-Truths Break Similarity-Based Retrieval
Bora Kargi, Arnas Uselis and Seong Joon Oh
When a text description is extended with an additional detail, image-text similarity should drop if that detail is wrong. We show that CLIP-style dual encoders often violate this intuition: appending a plausible but incorrect object or relation to an otherwise correct description can increase the similarity score. We call such cases half-truths. On COCO, CLIP prefers the correct shorter description only 34.0% of the time, and performance drops to 27.8% when the added detail is a relation. We trace this vulnerability to weak supervision on caption parts: contrastive training aligns full sentences but does not explicitly enforce that individual entities and relations are grounded. We propose CS-CLIP (Component-Supervised CLIP), which decomposes captions into entity and relation units, constructs a minimally edited foil for each unit, and fine-tunes the model to score the correct unit above its foil while preserving standard dual-encoder inference. CS-CLIP raises half-truth accuracy to 76.4% and improves average performance on established compositional
benchmarks by 5.8 points, suggesting that reducing half-truth errors aligns with broader gains in compositional understanding.
Read the full abstract ▾
Computer Vision
SegMate: Asymmetric Attention-Based Lightweight Architecture for Efficient Multi-Organ Segmentation
Andrei-Alexandru Bunea, Dan-Matei Popovici and Radu Tudor Ionescu
State-of-the-art models for medical image
segmentation achieve excellent accuracy, but require substantial computational resources, limiting deployment in typical resource-constrained clinical settings. We present SegMate, an efficient 2.5D framework that achieves state-of-the-art accuracy, while considerably reducing computational requirements. Our efficient design is the result of meticulously integrating asymmetric architectures,
attention mechanisms, multi-scale feature fusion, slice-based positional conditioning, and multi-task optimization. We demonstrate the efficiency-accuracy trade-off of our framework across three modern backbones (EfficientNetV2-M, MambaOut-Tiny, FastViT-T12). We perform experiments on three
datasets: TotalSegmentator, SegTHOR and AMOS22. Compared with the vanilla models, SegMate reduces computation (GFLOPs) by up to 2.5x and
memory footprint (VRAM) by up to 2.1x, while generally registering performance gains of around 1%. On TotalSegmentator, we achieve a Dice score of 93.51% with only 295 MB peak GPU
memory. Zero-shot cross-
dataset evaluations on SegTHOR and AMOS22 demonstrate strong generalization, with Dice scores of up to 86.85% and 89.35%, respectively. SegMate variants in this study empower single GPU setups to process up to 10,000 volumes per hour with just 74 J per volume using batch inference, enabling population-scale CT processing on commodity hardware. We release our open-source code at https://github.com/andreibunea99/SegMate.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
ABPolicy: Asynchronous B-Spline Flow Policy for Real-Time and Smooth Robotic Manipulation
Fan Yang, Peiguang Jing, Kaihua Qu et al. (+2 more)
Robotic manipulation requires policies that are smooth and responsive to evolving observations. However, synchronous inference in the raw action space introduces several challenges, including intra-chunk jitter, inter-chunk discontinuities, and stop-and-go execution. These issues undermine a policy's smoothness and its responsiveness to environmental changes. We propose ABPolicy, an asynchronous flow-matching policy that operates in a B-spline control-point action space. First, the B-spline representation ensures intra-chunk smoothness. Second, we introduce bidirectional action prediction coupled with refitting optimization to enforce inter-chunk continuity. Finally, by leveraging asynchronous inference, ABPolicy delivers real-time, continuous updates. We evaluate ABPolicy across seven tasks encompassing both static settings and dynamic settings with moving objects. Empirical results indicate that ABPolicy reduces trajectory jerk, leading to smoother motion and improved performance. Project website: https://teee000.github.io/ABPolicy/.
Read the full abstract ▾
All authors Fan Yang, Peiguang Jing, Kaihua Qu, Ningyuan Zhao, Yuting Su
Computer Vision
Experience-Guided Self-Adaptive Cascaded Agents for Breast Cancer Screening and Diagnosis with Reduced Biopsy Referrals
Pramit Saha, Mohammad Alsharid, Joshua Strong et al. (+1 more)
We propose an experience-guided cascaded multi-
agent framework for Breast Ultrasound Screening and Diagnosis, called BUSD-
Agent, that aims to reduce diagnostic escalation and unnecessary biopsy referrals. Our framework models screening and diagnosis as a two-stage, selective decision-making process. A lightweight `screening clinic'
agent, restricted to classification models as tools, selectively filters out benign and normal cases from further diagnostic escalation when malignancy risk and uncertainty are estimated as low. Cases that have higher risks are escalated to the `diagnostic clinic'
agent, which integrates richer perception and radiological description tools to make a secondary decision on biopsy referral. To improve
agent performance, past records of pathology-confirmed outcomes along with image
embeddings, model predictions, and historical
agent actions are stored in a
memory bank as structured decision trajectories. For each new case, BUSD-
Agent retrieves similar past cases based on image, model response and confidence similarity to condition the
agent's current decision policy. This enables
retrieval-conditioned in-context adaptation that dynamically adjusts model trust and escalation thresholds from prior experiences without parameter updates. Evaluation across 10 breast ultrasound
datasets shows that the proposed experience-guided workflow reduces diagnostic escalation in BUSD-
Agent from 84.95% to 58.72% and overall biopsy referrals from 59.50% to 37.08%, compared to the same architecture without trajectory conditioning, while improving average screening specificity by 68.48% and diagnostic specificity by 6.33%.
Read the full abstract ▾
All authors Pramit Saha, Mohammad Alsharid, Joshua Strong, J. Alison Noble
Computer Vision
Ref-Adv: Exploring MLLM Visual Reasoning in Referring Expression Tasks
Qihua Dong, Kuo Yang, Lin Ju et al. (+6 more)
Referring Expression Comprehension (REC) links
language to region level visual perception. Standard
benchmarks (RefCOCO, RefCOCO+, RefCOCOg) have progressed rapidly with
multimodal LLMs but remain weak tests of visual
reasoning and grounding: (i) many expressions are very short, leaving little
reasoning demand; (ii) images often contain few distractors, making the target easy to find; and (iii) redundant descriptors enable shortcut solutions that bypass genuine text understanding and visual
reasoning. We introduce Ref-Adv, a modern REC
benchmark that suppresses shortcuts by pairing linguistically nontrivial expressions with only the information necessary to uniquely identify the target. The
dataset contains referring expressions on real images, curated with hard distractors and annotated with
reasoning facets including negation. We conduct comprehensive ablations (word order perturbations and descriptor deletion sufficiency) to show that solving Ref-Adv requires
reasoning beyond simple cues, and we evaluate a broad suite of contemporary
multimodal LLMs on Ref-Adv. Despite strong results on RefCOCO, RefCOCO+, and RefCOCOg, models drop markedly on Ref-Adv, revealing reliance on shortcuts and gaps in visual
reasoning and grounding. We provide an in depth failure analysis and aim for Ref-Adv to guide future work on visual
reasoning and grounding in MLLMs.
Read the full abstract ▾
All authors Qihua Dong, Kuo Yang, Lin Ju, Handong Zhao, Yitian Zhang, Yizhou Wang, Huimin Zeng, Jianglin Lu, Yun Fu
Computer Vision
SelfOccFlow: Towards end-to-end self-supervised 3D Occupancy Flow prediction
Xavier Timoneda, Markus Herb, Fabian Duerr et al. (+1 more)
Estimating 3D occupancy and motion at the vehicle's surroundings is essential for autonomous driving, enabling situational awareness in dynamic environments. Existing approaches jointly learn geometry and motion but rely on expensive 3D occupancy and flow annotations, velocity labels from bounding boxes, or pretrained optical flow models. We propose a self-supervised method for 3D occupancy flow estimation that eliminates the need for human-produced annotations or external flow supervision. Our method disentangles the scene into separate static and dynamic signed distance fields and learns motion implicitly through temporal aggregation. Additionally, we introduce a strong self-supervised flow cue derived from features' cosine similarities. We demonstrate the efficacy of our 3D occupancy flow method on SemanticKITTI, KITTI-MOT, and nuScenes.
Read the full abstract ▾
All authors Xavier Timoneda, Markus Herb, Fabian Duerr, Daniel Goehring
Computer Vision
Bowen Yang, Zishuo Li, Yang Sun et al. (+15 more)
Embodied foundation models require large-scale, high-quality real-world interaction data for pre-training and scaling. However, existing data collection methods suffer from high infrastructure costs, complex hardware dependencies, and limited interaction scope, making scalable expansion challenging. In fact, humans themselves are ideal physically
embodied agents. Therefore, obtaining
egocentric real-world interaction data from globally distributed "human
agents" offers advantages of low cost and sustainability. To this end, we propose the Always-on
Egocentric (AoE) data collection system, which aims to simplify hardware dependencies by leveraging humans themselves and their smartphones, enabling low-cost, highly efficient, and scene-agnostic real-world interaction data collection to address the challenge of data scarcity. Specifically, we first employ an ergonomic neck-mounted smartphone holder to enable low-barrier, large-scale
egocentric data collection through a cloud-edge collaborative architecture. Second, we develop a cross-platform mobile APP that leverages on-device compute for real-time processing, while the cloud hosts automated labeling and filtering pipelines that transform raw
videos into high-quality training data. Finally, the AoE system supports distributed Ego
video data collection by anyone, anytime, and anywhere. We evaluate AoE on data preprocessing quality and downstream tasks, demonstrating that high-quality
egocentric data significantly boosts real-world generalization.
Read the full abstract ▾
All authors Bowen Yang, Zishuo Li, Yang Sun, Changtao Miao, Yifan Yang, Man Luo, Xiaotong Yan, Feng Jiang, Jinchuan Shi, Yankai Fu, Ning Chen, Junkai Zhao, Pengwei Wang, Guocai Yao, Shanghang Zhang, Hao Chen, Zhe Li, Kai Zhu
Computer Vision
DACESR: Degradation-Aware Conditional Embedding for Real-World Image Super-Resolution
Xiaoyan Lei, Wenlong Zhang, Biao Luo et al. (+3 more)
Multimodal large models have shown excellent ability in addressing image super-resolution in real-world scenarios by leveraging
language class as condition information, yet their abilities in degraded images remain limited. In this paper, we first revisit the capabilities of the Recognize Anything Model (RAM) for degraded images by calculating text similarity. We find that directly using contrastive learning to fine-tune RAM in the degraded space is difficult to achieve acceptable results. To address this issue, we employ a degradation selection strategy to propose a Real
Embedding Extractor (REE), which achieves significant recognition performance gain on degraded image content through contrastive learning. Furthermore, we use a Conditional Feature Modulator (CFM) to incorporate the high-level information of REE for a powerful Mamba-based network, which can leverage effective pixel information to restore image textures and produce visually pleasing results. Extensive experiments demonstrate that the REE can effectively help image super-resolution networks balance fidelity and perceptual quality, highlighting the great potential of Mamba in real-world applications. The source code of this work will be made publicly available at: https://github.com/nathan66666/DACESR.git
Read the full abstract ▾
All authors Xiaoyan Lei, Wenlong Zhang, Biao Luo, Hui Liang, Weifeng Cao, Qiuting Lin
Computer Vision
Altitude-Adaptive UAV-to-Map Image Matching for Aerial Geo-Localization
Xingyu Shao, Mengfan He, Liangzheng Sun et al. (+2 more)
Matching downward-looking unmanned aerial vehicle (
UAV) images to georeferenced satellite or aerial map tiles supports map-based geo-localization when direct positioning signals are degraded or unavailable. Variations in
UAV altitude change the observed ground footprint and create a scale mismatch between query images and fixed-scale map references. We address this problem with an altitude-adaptive aerial geo-localization framework. The framework estimates relative altitude from a single image using a frequency-domain representation and regression-as-classification. The predicted altitude-class center defines a scale prior that guides query normalization toward the canonical map scale before full-gallery visual place recognition. A footprint-adaptive margin classifier (FAMC) adjusts training margins according to footprint overlap. Experiments on two synthetic and two real-flight
datasets show that, at the 100 m localization threshold, altitude adaptation improves average R@1 and R@5 by 50.68 and 47.20 percentage points, respectively, for our visual place recognition module compared with the same
retrieval pipeline without altitude normalization. These results show that a relative-altitude-derived scale prior improves cross-altitude
UAV-to-map image matching in the evaluated regions and camera settings.
Read the full abstract ▾
All authors Xingyu Shao, Mengfan He, Liangzheng Sun, Chunyu Li, Ziyang Meng
Computer Vision
Bandwidth-adaptive Cloud-Assisted 360-Degree 3D Perception for Autonomous Vehicles
Faisal Hawladera, Rui Meireles, Gamal Elghazaly et al. (+2 more)
A key challenge for autonomous driving lies in maintaining real-time situational awareness regarding surrounding obstacles under strict latency constraints. The high processing requirements coupled with limited onboard computational resources can cause delay issues, particularly in complex urban settings. To address this, we propose leveraging Vehicle-to-Everything (V2X) communication to partially offload processing to the cloud, where compute resources are abundant, thus reducing overall latency. Our approach utilizes
transformer-based models to fuse multi-camera sensor data into a comprehensive Bird's-Eye View (BEV) representation, enabling accurate 360-degree 3D object detection. The computation is dynamically split between the vehicle and the cloud based on the number of layers processed locally and the quantization level of the features. To further reduce network load, we apply feature vector clipping and compression prior to transmission. In a real-world experimental evaluation, our hybrid strategy achieved a 72 % reduction in end-to-end latency compared to a traditional onboard solution. To adapt to fluctuating network conditions, we introduce a dynamic optimization
algorithm that selects the split point and quantization level to maximize detection accuracy while satisfying real-time latency constraints. Trace-based evaluation under realistic bandwidth variability shows that this adaptive approach improves accuracy by up to 20 % over static parameterization with the same latency performance.
Read the full abstract ▾
All authors Faisal Hawladera, Rui Meireles, Gamal Elghazaly, Ana Aguiar, Raphaël Frank
Computer Vision
Open-Vocabulary Semantic Segmentation in Remote Sensing via Hierarchical Attention Masking and Model Composition
Mohammadreza Heidarianbaei, Mareike Dorozynski, Hubert Kanyamahanga et al. (+2 more)
In this paper, we propose ReSeg-CLIP, a new training-free Open-Vocabulary Semantic
Segmentation method for remote sensing data. To compensate for the problems of
vision language models, such as CLIP in semantic
segmentation caused by inappropriate interactions within the self-
attention layers, we introduce a hierarchical scheme utilizing masks generated by SAM to constrain the interactions at multiple scales. We also present a model composition approach that averages the parameters of multiple RS-specific CLIP variants, taking advantage of a new weighting scheme that evaluates representational quality using varying text
prompts. Our method achieves state-of-the-art results across three RS
benchmarks without additional training.
Read the full abstract ▾
All authors Mohammadreza Heidarianbaei, Mareike Dorozynski, Hubert Kanyamahanga, Max Mehltretter, Franz Rottensteiner
Computer Vision
NAU-QMUL: Utilizing BERT and CLIP for Multi-modal AI-Generated Image Detection
Xiaoyu Guo and Arkaitz Zubiaga
With the aim of detecting AI-generated images and identifying the specific models responsible for their generation, we propose a multi-modal multi-task model. The model leverages pre-trained BERT and CLIP
Vision encoders for text and image feature extraction, respectively, and employs cross-modal feature fusion with a tailored multi-task loss function. Additionally, a pseudo-labeling-based data augmentation strategy was utilized to expand the training
dataset with high-confidence samples. The model achieved fifth place in both Tasks A and B of the `CT2: AI-Generated Image Detection' competition, with F1 scores of 83.16% and 48.88%, respectively. These findings highlight the effectiveness of the proposed architecture and its potential for advancing AI-generated content detection in real-world scenarios. The source code for our method is published on https://github.com/xxxxxxxxy/AIGeneratedImageDetection.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Polarization Uncertainty-Guided Diffusion Model for Color Polarization Image Demosaicking
Chenggong Li, Yidong Luo, Junchao Zhang et al. (+1 more)
Color polarization demosaicking (CPDM) aims to reconstruct full-resolution polarization images of four directions from the color-polarization filter array (CPFA) raw image. Due to the challenge of predicting numerous missing pixels and the scarcity of high-quality training data, existing network-based methods, despite effectively recovering scene intensity information, still exhibit significant errors in reconstructing polarization characteristics (degree of polarization, DOP, and angle of polarization, AOP). To address this problem, we introduce the image
diffusion prior from text-to-image (T2I) models to overcome the performance bottleneck of network-based methods, with the additional
diffusion prior compensating for limited representational capacity caused by restricted data distribution. To effectively leverage the
diffusion prior, we explicitly model the polarization uncertainty during reconstruction and use uncertainty to guide the
diffusion model in recovering high error regions. Extensive experiments demonstrate that the proposed method accurately recovers scene polarization characteristics with both high fidelity and strong visual perception.
Read the full abstract ▾
All authors Chenggong Li, Yidong Luo, Junchao Zhang, Degui Yang
Computer Vision · Cross-listed from eess.IV
Revisiting Integration of Image and Metadata for DICOM Series Classification: Cross-Attention and Dictionary Learning
Tuan Truong, Melanie Dohmen, Sara Lorio et al. (+1 more)
Automated identification of DICOM image series is essential for large-scale medical image analysis, quality control, protocol harmonization, and reliable downstream processing. However, DICOM series classification remains challenging due to heterogeneous slice content, variable series length, and entirely missing, incomplete or inconsistent DICOM metadata. We propose an end-to-end
multimodal framework for DICOM series classification that jointly models image content and acquisition metadata while explicitly accounting for all these challenges. (i) Images and metadata are encoded with modality-aware modules and fused using a bi-directional cross-modal
attention mechanism. (ii) Metadata is processed by a sparse, missingness-aware encoder based on learnable feature dictionaries and value-conditioned modulation. By design, the approach does not require any form of imputation. (iii) Variability in series length and image data dimensions is handled via a 2.5D visual encoder and
attention operating on equidistantly sampled slices. We evaluate the proposed approach on the publicly available Duke Liver MRI
dataset and a large multi-institutional in-house cohort, assessing both in-domain performance and out-of-domain generalization. Across all evaluation settings, the proposed method consistently outperforms relevant image only, metadata-only and
multimodal 2D/3D baselines. The results demonstrate that explicitly modeling metadata sparsity and cross-modal interactions improves robustness for DICOM series classification.
Read the full abstract ▾
All authors Tuan Truong, Melanie Dohmen, Sara Lorio, Matthias Lenga
Computer Vision
APPO: Attention-guided Perception Policy Optimization for Video Reasoning
Henghui Du, Chang Zhou, Xi Chen et al. (+1 more)
Complex
video reasoning, actually, relies excessively on fine-grained perception rather than on expert (e.g., Ph.D, Science)-level
reasoning. Through extensive empirical observation, we have recognized the critical impact of perception. In particular, when perception ability is almost fixed, enhancing
reasoning from Qwen3-8B to OpenAI-o3 yields only 0.7% performance improvement. Conversely, even minimal change in perception model scale (from 7B to 32B) boosts performance by 1.4%, indicating enhancing perception, rather than
reasoning, is more critical to improve performance. Therefore, exploring how to enhance perception ability through
reasoning without the need for expensive fine-grained annotation information is worthwhile. To achieve this goal, we specially propose APPO, the
Attention-guided Perception Policy Optimization
algorithm that leverages
token-level dense rewards to improve model's fine-grained perception. The core idea behind APPO is to optimize those
tokens from different responses that primarily focus on the same crucial
video frame (called intra-group perception
tokens). Experimental results on diverse
video benchmarks and models with different scales (3/7B) demonstrate APPO consistently outperforms GRPO and DAPO (0.5%~4%). We hope our work provides a promising approach to effectively enhance model's perception abilities through
reasoning in a low-cost manner, serving diverse scenarios and demands.
Read the full abstract ▾
All authors Henghui Du, Chang Zhou, Xi Chen, Di Hu