Computer Vision
CARDIAG: A Dense Segment Classification Benchmark of Deep Learning Architectures for Coronary Angiography
Dominik Bernard Lau, Hubert Malinowski, Jerzy Szyjut et al. (+5 more)
Accurate pixel-level classification of coronary angiograms is critical for cardiovascular disease assessment, yet the field lacks standardized evaluation protocols. In this work we demonstrate a new
benchmark for the assessment of deep learning models which densely classify pixels of coronary angiograms to one of SYNTAX classes (or background). The evaluation covers 24 distinct architectures starting with classic convnets to recent state-space-based
vision algorithms. We release CARDIAG - a multi-center, multi-label
dataset which we carefully split to reliably compute metrics, accounting for diameter error, overlap, centerline quality and calibration. The data contains SYNTAX labels, binary, uncertainty and
segmentation masks as well as intermediate frames together with the selected non-sensitive DICOM metadata. From the multitude of
algorithms, we nominate ConvNeXt V2 encoder with DeepLab V3 Plus decoder as the best performing, achieving macro $F_1=0.456$, which we then ensemble with Mamba U-Net and Feature Pyramid Network, for an increased $F_1=0.479$. We demonstrate all the architectures to be well calibrated and determine the generalization of the top 5 methods, together with the data efficiency of these architectures. We highlight the importance of both high-resolution and low-resolution features in encoding. We also demonstrate the model correctness in the context of patient demographic, vessel sides and projection angle configurations. Overall the released
benchmark allows for future studies to robustly and rigorously assess the proposals, not only for SYNTAX
segmentation, but lesion detection and many more.
Read the full abstract ▾
All authors Dominik Bernard Lau, Hubert Malinowski, Jerzy Szyjut, Adam Brzeski, Tomasz Dziubich, Radosław Targoński, Tomasz Figatowski, Natalia Zielińska
Computer Vision
GLI-AL: A Multi-Modal Glioma MRI Label Resource with Unified Anatomy-Lesion Labels
Xingyu Xiang, Shuang Hao, Fan Wang et al. (+2 more)
Existing BraTS-GLI
datasets provide a widely used
benchmark for adult glioma MRI
segmentation, but their task definition focuses on tumor subregions and does not systematically represent coexisting white matter hyperintensities (WMH). In joint
segmentation settings, such unlabeled abnormalities introduce task-specific label noise by treating pathological regions as normal tissue. To address this limitation, we introduce BraTS-GLI Anatomy-Lesion, a controlled-access, labels-only derived resource built from the BraTS 2023-GLI training cohort. The resource provides 1,251 unified eight-class anatomy-lesion label sets aligned with the original four-modal MRI cases, including image-repair labels for 116 cases requiring repaired imaging inputs. The cohort is organized into a 394-case purified subset and an 857-case extended subset, with case-level metadata covering label source, image-repair requirements, quality-control status, access conditions, checksums, and release boundaries. Compared with the original BraTS-GLI annotations, the resource substantially expands foreground supervision by incorporating healthy
brain tissues and previously unlabeled coexisting abnormalities within a unified label space. A validation study using MedNeXt and T1/FLAIR inputs suggests that WMH-aware supervision preserves healthy-tissue
segmentation performance across both in-domain GLI and external WMH
datasets, while improving sensitivity to coexisting lesions relative to noisy-control training. The resource is intended for scientific research and supports joint anatomy-lesion supervision, label-noise analysis, and reproducible evaluation. Data are available at https://www.synapse.org/Synapse:syn75210889/wiki/, and code is available at https://github.com/xyx200/brats-gli-anatomy-lesion-code. The data resource DOI is https://doi.org/10.7303/SYN75210889.
Read the full abstract ▾
All authors Xingyu Xiang, Shuang Hao, Fan Wang, Jianhua Ma, Chunfeng Lian
Computer Vision
A Framework for Individual Tree Growth Reconstruction Using Multi-Platform Laser Scanning
Daniella Tavi, Valtteri Soininen, Lassi Ruoppa et al. (+2 more)
Accurate tree-level forest monitoring using
laser scanning data requires reliable tree delineation, consistent tree correspondence across multitemporal point clouds, and accurate estimation of tree attributes and their change. Reconstructing tree growth in boreal forests is challenging due to the scarcity of historical stem-level data, propagation of errors from older sensors into change estimation, and growth rates with a magnitude of measurement uncertainty. This study investigates a framework for estimating individual tree diameter at breast height (DBH) and stem volume growth using 136 point clouds acquired between 2014–2025 with 11 scanners on airborne (ALS), mobile (MLS), and terrestrial
laser scanning (TLS) platforms across boreal forest test sites. Trees were delineated from an MLS point cloud using deep learning-based
segmentation which was transferred to the remaining point clouds, resulting in reliable multitemporal tree correspondence. Stem curves were derived from MLS/TLS data, with ALS data used for height estimation, enabling DBH and volume estimation and time series. A height growth-based scaling model was used to reconstruct stem attributes across time and estimate growth. Results showed that modeled growth achieved higher agreement with manual growth estimates than differencing independently estimated attributes from point clouds. The modeled-manual 5- and 10-year growth RMSEs were 55–111% and 26–67% for DBH, and 31–87% and 21–67% for volume, respectively, depending on plot difficulty. The scaling model was temporally robust, with errors remaining stable or stabilizing after 5–6 years, reaching maximum RMSEs of 8–12% for DBH and 12–23% for volume after 12 years. Combining MLS/TLS-derived stem measurements with multitemporal ALS-derived heights provided a robust framework for individual tree growth estimation without requiring multiple under-canopy scans.
Read the full abstract ▾
All authors Daniella Tavi, Valtteri Soininen, Lassi Ruoppa, Jesse Muhojoki, Juha Hyyppä
Computer Vision
Projection Pursuit CPCANet for Domain Generalization
Yu-Hsi Chen and Abd-Krim Seghouane
Domain Generalization (DG) aims to learn representations robust to distribution shifts. Recent
geometric alignment methods, such as CPCANet, extract domain-invariant structures through batch-wise Common Principal Component Analysis (CPCA). However, CPCANet suffers from rank-deficient covariance estimation due to the small-sample-size issue in mini-batch training. To address this limitation, we propose Projection Pursuit CPCANet (PP-CPCANet), a covariance-free framework that learns a global orthogonal basis on the Stiefel
manifold and jointly optimizes it with network parameters via the Cayley transform. We further introduce a
symmetry-breaking detached-median PP dispersion objective to extract common principal components (CPCs) with dense and robust optimization signals. Experiments on four DG
benchmarks show that PP-CPCANet achieves SOTA performance while maintaining stable training.
Read the full abstract ▾
Computer Vision
InnoText: A Unified Model for Visual Text Generation and Editing
Haowei Liu, Runze He, Jian Lu et al. (+10 more)
Diffusion models have recently achieved remarkable success in high-fidelity image synthesis, yet their application to visual text generation and editing remains relatively underexplored. Unlike general image generation, visual text tasks demand precise structural regularity and legibility, which may pose additional challenges for small-scale text and non-Latin scripts such as Chinese. Existing UNet-based models often struggle to produce clear and coherent text, while DiT-based models, though more expressive, are typically limited to a single task, which may lead to redundant training pipelines, inconsistent visual styles, and reduced cross-task generalization. To address these challenges, we propose InnoText, a unified DiT-based framework capable of performing both text generation and editing within a single model. We introduce a Font Size-Aware Modulation (FSAM) module to enhance representations across font scales, a Small-Character Aware Augmentation strategy to improve fine-grained fidelity, and a Task-Specific Region Weighted Loss for adaptive optimization. To support training and evaluation, we also construct a high-quality bilingual (English-Chinese) visual text
dataset covering diverse fonts, sizes, and backgrounds. Experimental results demonstrate that our method achieves superior generation accuracy and editing quality, producing visually appealing and realistic text images.
Read the full abstract ▾
All authors Haowei Liu, Runze He, Jian Lu, Ao Ma, Run Ling, Ke Cao, Jiasong Feng, Wei Feng, Shuo Lu, Yexing Xu, Yun Wang, Jing Wang, Zhanjie Zhang
Computer Vision
Spectral Prior for Reducing Exposure Bias in Diffusion Models
Yuya Kobayashi, Masato Ishii, Yuhta Takida et al. (+2 more)
Diffusion models typically suffer from error accumulation during iterative sampling, commonly referred to as exposure bias. We reveal systematic frequency-dependent discrepancies between training and inference, which can be interpreted as frequency-dependent SNR error. Crucially, the direction of this mismatch varies across models and timesteps, indicating that fixed correction rules do not generalize. We propose Spectral
Alignment (SPA), a lightweight, guidance-based method that calibrates the power spectrum of intermediate predictions to a pre-computed prior. Our approach consists of two stages: (1) offline fitting of a parametric spectrum model from training data, and (2) inference-time guidance via efficient FFT-based gradient computation. SPA introduces minimal computational overhead (3-4%) and is complementary to Classifier-Free Guidance (CFG). We demonstrate consistent improvements across diverse architectures, from pixel-space models (DDPM, ADM) to latent
diffusion models (SD2.0, SDXL) and flow-matching models (SD3.5, FLUX). Our implementation is available at https://github.com/SonyResearch/SPA.
Read the full abstract ▾
All authors Yuya Kobayashi, Masato Ishii, Yuhta Takida, Takashi Shibuya, Yuki Mitsufuji
Computer Vision
FAIR: Feature-Augmented Implicit Regularization for AI-generated Fake Image Detection
Md Redwanul Haque, Manzur Murshed, Manoranjan Paul et al. (+1 more)
Generalization remains a critical bottleneck in AI-generated image detection. Because many modern generators are proprietary or adversarially modified, existing detectors overfit to the low-level textural patterns of accessible training data, resulting in severe failures on unseen domains. Conventional regularization techniques (e.g., $L_1$/$L_2$ norms, Dropout) apply indiscriminate parametric constraints and fail to provide the domain-invariant structure necessary for cross-generator robustness. To address this, we propose Feature-Augmented Implicit Regularization (FAIR). FAIR introduces an orthogonal, macro-structural prior, specifically, Scene Composition Structure (SCS), during training to geometrically constrain the model's optimization trajectory. By augmenting the primary feature space with domain-invariant SCS features, FAIR explicitly penalizes texture-biased shortcut learning. Crucially, this structural prior is entirely discarded at inference, yielding a smoothed, generalized decision boundary with zero architectural or computational overhead. Extensive evaluations across five massive
benchmarks demonstrate that integrating FAIR into state-of-the-art detectors significantly improves cross-generator generalization, boosting accuracy by up to 8.04% and establishing new state-of-the-art robustness in zero-shot transfer scenarios.
Read the full abstract ▾
All authors Md Redwanul Haque, Manzur Murshed, Manoranjan Paul, Tsz-Kwan Lee
Computer Vision
CommandLM: Data driven behavior level descriptor for ego vehicles
Boris Tokic, Constantin Selzer and Fabian B. Flohr
As autonomous driving systems move toward real-world deployment, interpretable, behavior-level decision-making is essential for safety, trust, and regulation. We introduce CommandLM, a
multimodal large
language model that generates concise, human-readable behavior descriptions for ego vehicles from fused multi-sensor data. Our model processes temporally fused bird's-eye view representations from
LiDAR and multi-camera inputs via a Q-Former adapter connected to a quantized, LoRA-fine-tuned large
language model. Trained on our CommandLM-nuScenes
dataset, CommandLM produces intent-aware, interpretable captions suitable for planner supervision and safety
auditing. Experiments demonstrate strong linguistic and behavioral
alignment, achieving CIDEr 0.67, and BERT-F1 0.88, substantially outperforming the BLIP-2 baseline (CIDEr 0.52, BERT-F1 0.86). In human evaluation, 58% of the generated descriptions were rated accurate, efficient and rule-compliant, confirming their real-world plausibility. While the remaining descriptions may not always select the most efficient, goal-oriented behavior, CommandLM's interpretable outputs enable downstream validation systems to identify and correct such cases, making it an effective tool for transparent behavior
auditing. These results show that integrating
multimodal fusion with
language reasoning yields efficient and transparent behavior-level understanding for autonomous driving. We release our code and
dataset at: https://github.com/b-tok/CommandLM
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
The Lift Spectrum: How Measurement-to-Space Adaptivity Shapes Robustness in Image-Free Single-Pixel Sensing
Yuyuan Han, Jingwei Li, Xiaoxia Zhang et al. (+9 more)
Single-pixel sensing encodes a scene as a short sequence of coded measurements, and image-free methods infer the task directly from that sequence. We show that removing image reconstruction relocates the central design problem to the lift: how 1D measurements become a 2D task representation. We organize this choice as a lift spectrum from a fixed-physics inverse, through a learned static projection, to content-adaptive
retrieval. These are not interchangeable forms of reconstruction: the fixed-physics route reconstructs an image consumed at inference, whereas our spatiotemporal soft-fusion (STSF) network lifts measurements directly into task features, and task-prioritized loss scheduling (TPLS) uses a separate learned reconstruction branch only as scheduled training supervision. A probe-selected recurrent encoder and a parameter-matched lift ablation identify the STSF design. In simulation, STSF+TPLS exceeds the prior image-free baseline on three
datasets at 3.13% sampling (+3.2 to +9.9 pp foreground mIoU) and remains competitive down to 0.39%. The strongest clean-trained reconstruct-then-segment baseline wins without measurement noise, but measurement noise reverses the ranking: the reconstructed task input carries a 20-70x larger normalized relative perturbation than the measurements themselves. Stressed to failure, the three lift regions exhibit distinct dominant signatures–collapse, imprinting, and coarsening. STSF+TPLS transfers without fine-tuning to a real single-pixel bench, where the reversal reappears as a proof of concept; inference takes about 14 ms per mask on an RTX 4090. Within the tested fixed-acquisition regime, measurement-to-space adaptivity therefore organizes both the clean-to-noisy operating envelope and the failure a system encounters. Code and pretrained weights: https://github.com/Hanyuyuan6/STSF-TPLS.
Read the full abstract ▾
All authors Yuyuan Han, Jingwei Li, Xiaoxia Zhang, Long Qiu, Chong Wang, Wenxuan Hao, Jiangyu Han, Xinyu Yao, Yuchen He, Hui Chen, Jianbin Liu, Huaibin Zheng
Computer Vision
Alleviating Regional Shortcuts for Few-Shot Class-Incremental Learning
Haichen Zhou, Yazhe Lyu, Yixiong Zou et al. (+2 more)
Few-shot class-incremental learning (FSCIL) aims to incrementally learn novel classes with only a few samples while avoiding forgetting base classes. However, current methods show a tendency to misclassify novel-class samples into base classes, which we find to be caused by the excessive focus on base-class-discriminative regions on novel-class samples. In this work, we aim to explore the underlying mechanism for an interpretation and solution. We first provide a compositional view to analyze the transferred and reused spatial patterns on novel-class samples. Then, through extensive experiments and theoretical analysis, we identify both empirically and theoretically that a shortcut exists in the model's base-class training, which naturally forms the excessive focus on only the most discriminative regions (primitives), which we term as the regional shortcut. Finally, based on this interpretation, to address this problem, we propose a compositional-learning-based method to learn two primitive sets (a common set and a discriminative set), which alleviates the regional shortcut by constraining the model to learn and utilize the common primitive set for base- and novel-class recognition. Extensive experiments on standard FSCIL
benchmarks demonstrate the effectiveness of our approach, yielding consistent improvements over existing state-of-the-art methods in both accuracy and interpretability.
Read the full abstract ▾
All authors Haichen Zhou, Yazhe Lyu, Yixiong Zou, Ruixuan Li, Yuhua Li
Computer Vision
ReCowGnition: A Realistic Biometric Benchmark for Cow Face Recognition
Marco Huber, Marco Kiesewalter, Judith Louise Pieper et al. (+2 more)
With the development of precision livestock farming and the advances in computer
vision, visual animal biometrics has gained
attention. Using biometric technologies that have been proven effective for humans to identify livestock can increase animal welfare as well as production efficiency. However, challenges such as complex scenarios, similar appearances, occlusions, and non-cooperative behavior, as well as the limited amount of publicly available labeled
datasets, remain. In this work, we contribute a novel, publicly available cow face
benchmark dataset that has been collected in a realistic automatic scenario with 6,838 images of 161 different cows at a dairy farm. In addition to the public
dataset, we define two verification and four identification evaluation protocols to foster comparable research in the cow recognition research field. Further, we provide evaluation results on our
dataset of six
benchmark models, which include models trained on limited data, cross-species fine-tuned models, and zero-shot foundation model approaches.
Read the full abstract ▾
All authors Marco Huber, Marco Kiesewalter, Judith Louise Pieper, Bastian Kubsch, Naser Damer
Computer Vision
Rethinking Multi-Branch and Cross-Backbone Fusion for Vehicle Re-Identification under Foundation-Model Pretraining
Yu Wang and Hongyu Yang
Multi-branch architectures and CNN-
Transformer fusion are widely believed to improve vehicle re-identification (Re-ID) by combining complementary representations. We revisit this for a DINOv3-pretrained backbone. A single DINOv3-pretrained ConvNeXt with a tuned recipe reaches 88.19 mAP on VeRi-Wild Small and 77.47 on Large from visual cues alone, within the combined evaluation and optimization noise of the strongest protocol-verified metadata-dependent multi-branch baseline, and 92.38/83.68 with training-free re-ranking. Using this baseline and
retrieval-level branch diagnostics, we ask whether representational diversity still pays at this scale. In our runs, it does not. Across both
benchmarks and every converged configuration, concatenating multiple heads over a shared backbone moves the best single head by under one mAP point in either direction while costing four times the
embedding dimension; 99.7% of the concatenation's variance lies in 512 principal components, so the heads not only duplicate one another but each occupies a quarter of its nominal 2048 dimensions. Pushing diversity to its architectural limit, CNN versus
Transformer, we grant fusion every advantage through an asymmetric frozen-anchor scheme. Every
Transformer configuration still lands at least 13 mAP below the ConvNeXt backbone (13-15 for the two strongest, up to 46 for the weakest), and a paired per-query bootstrap bounds the fusion gain at +0.11 mAP (95% CI) even for the most favourable snapshot we obtained. One strong backbone with the right recipe and re-ranking is the efficiency frontier. All results use single-seed training and one foundation-model family; differences of this size are therefore reported as bounds rather than orderings, and we list falsifiers.
Read the full abstract ▾
Computer Vision
A Smooth Phase-Separation Model for Weak-Boundary Segmentation of Homogeneous Structures
Zihan Li, Jiebao Sun, Fanghui Song et al. (+1 more)
Segmentation of adjacent structures with similar intensity distributions remains a challenging problem in image analysis, particularly when object boundaries are weak or ambiguous. Under such conditions, classical variational models may suffer from degenerated image-driven forces, leading to boundary leakage or undesired merging of neighboring regions. To address these limitations, we propose a smooth phase-separation variational model based on the Cahn–Hilliard equation for weak-boundary
segmentation of homogeneous-appearance structures. The proposed framework integrates softmax-based region fitting with Cahn–Hilliard phase-field regularization to maintain interface discrimination under weak image-driven forces. We further introduce a mixed $L^2-H^{-1}$ gradient flow, which preserves higher-order interfacial regularization while allowing adaptive changes of phase masses, establish the continuous energy dissipation law, and prove the existence and uniqueness of weak solutions in the natural solution class. For numerical computation, we develop a stabilized scalar auxiliary variable (SAV) scheme that is linear, FFT-based, and satisfies a modified discrete energy dissipation law. Numerical experiments on synthetic and medical images demonstrate that the proposed method effectively separates adjacent homogeneous structures across weak boundaries and achieves competitive
segmentation accuracy and improved boundary localization compared with representative variational, phase-field, and deep learning methods.
Read the full abstract ▾
All authors Zihan Li, Jiebao Sun, Fanghui Song, Zhichang Guo
Computer Vision · Cross-listed from cs.CL
Scaling Native Multimodal Pre-Training From Scratch
Haoyuan Wu, Aoqi Wu, Hai Wang et al. (+3 more)
Although large
language models (LLMs) exhibit remarkable
reasoning capabilities, their reliance on text-only pre-training restricts the perception of the
multimodal physical world. Native
multimodal pre-training avoids this limitation by training models from scratch on
multimodal inputs, thereby achieving deep cross-modal integration and mitigating optimization asymmetries inherent to traditional late-fusion architectures. Despite these advantages, the scaling properties of this paradigm remain incompletely characterized. To address this gap, we investigate the optimal model size and
token count for training a
Transformer-based
vision-
language model under a fixed computational budget. Our study demonstrates that minimal objective loss adheres to a predictable compute law, whereas compute-optimal model sizes and
token counts scale as power laws. Notably,
language and
multimodal objectives manifest distinct allocation trends. The
language allocation exponents lie in a similar range across the different data mixtures. The
multimodal model-allocation exponent decreases modestly with the
multimodal token ratio, indicating a relative shift toward
token allocation. Additionally, our scaling analysis yields a budget-compensation rule. Specifically, an additional
multimodal-
token budget can offset the text-efficiency penalty caused by incorporating visual information into native
multimodal pre-training under a fixed compute budget. Downstream evaluations further reveal that native
multimodal pre-training is associated with improved spatial
reasoning and
multimodal few-shot learning. Generally, this empirical research establishes the essential groundwork for predictably scaling
multimodal foundation models.
Read the full abstract ▾
All authors Haoyuan Wu, Aoqi Wu, Hai Wang, Jiajia Wu, Jinxiang Ou, Bei Yu
Computer Vision
Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradation
M M Asif Ferdous
Vision-
language models (VLMs) are increasingly deployed on consumer hardware where input images are degraded by compression, camera shake, and poor lighting. In such settings, a reliable uncertainty signal matters more than raw accuracy, because it determines when a system should defer rather than answer. We evaluate two small open-weight VLMs – Qwen2-VL-2B-Instruct and SmolVLM-Instruct – across six realistic photographic degradations at three severity levels, comparing two confidence signals: the confidence the model states in natural
language, and the model's own mean
token probability over its generated answer. Across 3,800 predictions, we find a large and consistent gap. Verbalized confidence in Qwen2-VL is almost constant (mean 0.87-0.90 across all conditions) and detects its own errors at chance level (AUROC 0.39-0.75, typically ~0.50), while internal
token probability from the same model separates correct from incorrect answers with AUROC 0.92-0.99. In SmolVLM, verbalized confidence proved largely unobtainable: across three
prompt templates, only one of five pilot attempts produced a parseable confidence value, while internal probability again yielded above-chance error detection (AUROC 0.54-0.92). Both models fail in the same place: under severe underexposure, accuracy collapses (0.99->0.22 for Qwen2-VL, 0.97->0.42 for SmolVLM) while both confidence signals barely move, and internal error-detection falls to chance. We conclude that small VLMs encode usable self-knowledge that their verbalized output does not express, that internal probability is therefore the better deferral signal in constrained deployment, and that neither signal should be trusted under severe low-light conditions.
Read the full abstract ▾
Computer Vision
EVL-MCoT: Enhanced Vision-Language Multi-CoT for Harmful Meme Detection
Hao Yang, Jin Wang and Xuejie Zhang
MEMEs are widely used on the internet and often carry strong elements of sarcasm or irony. Understanding their hidden meanings typically requires a joint interpretation of text and
vision. Existing methods focus on the dual-stream
vision-
language model to extract the visual and text simultaneously, which lacks background information and prior knowledge about the comprehensive explanation of MEME. One feasible option is to adopt chain-of-thought (CoT). However, the simple CoT approach lacks multi-perspective thinking, which may compromise the reliability of the resulting answers. Moreover, it often relies on shallow feature fusion, lacking the fusion of local details and fine-grained visual-
prompt text
alignment. This limitation prevents a deeper understanding of the intricate connections between the visual and the text. Herein, an enhanced
vision-
language multi-CoT (EVL-MCoT) approach is proposed to address these limitations. By promoting multi-CoT, EVL-MCoT enhances consistency and reduces bias in the decision-making process. Additionally, we design a prototype-guided and context-guided decoding framework, which incorporates visual prototypes to guide the fusion process and enables the model to align textual and visual information more precisely. We achieve promising results on the HatefulMemes and MultiOff
datasets. The source code has been publicly released and is available at https://github.com/BGWH123/EVL-MCoT.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Zero-Shot Mission-Level Evaluation for Aerial MLLM Agents
Suman Navaratnarajah, Taehyoung Kim, Jona Ruthardt et al. (+5 more)
Multimodal Large
Language Models (MLLMs) are emerging as core
reasoning modules for
embodied agents, yet it remains unclear how well general-purpose models can solve long-horizon
embodied tasks from a single high-level instruction. We introduce MissionBench, a
benchmark for mission-level evaluation of MLLMs in aerial 3D environments. It comprises 120 missions across five simulated 3D environments and four task families.
Agents must autonomously plan, navigate, and report outcomes using only
egocentric observations and its action history, without aerial-specific fine-tuning. Across 22 open- and closed-source MLLMs, the strongest model succeeds on fewer than 35% of missions compared to 84.4% human performance, highlighting the difficulty of multi-step
embodied tasks. Despite large variations between model families, we observe gains from scaling, indicating that larger general-purpose models possess stronger zero-shot
embodied capabilities. Our analysis shows that mission-level competence requires coordinating multiple capabilities beyond spatial perception, including multi-step planning and adaptive
reasoning. This motivates closed-loop evaluation and highlights both the promise and risk of scaling-driven improvements for
embodied AI.
Read the full abstract ▾
All authors Suman Navaratnarajah, Taehyoung Kim, Jona Ruthardt, Ishaan Bhimwal, Ryousuke Yamada, Yannik Blei, Wolfram Burgard, Yuki M Asano
Computer Vision
Hao Yang, Jin Wang and Xuejie Zhang
Multimodal chain-of-thought (CoT)
reasoning integrates visual and textual cues through step-by-step inference. In small models with limited
token budgets, modality-interaction fusion often suppresses tiny cross-modal differences. In particular,
multimodal CoT often struggles when different images pair with identical text or different texts pair with an identical image, making such inputs nearly indistinguishable after fusion. This study proposes Visual Saliency Steering
Distillation (VSSD). VSSD leverages the
attention maps of
multimodal large
language models to generate perturbed images that capture task-sensitive feature directions, and then applies singular value decomposition to extract dominant steering vectors to guide inter-layer
distillation. Experiments on ScienceQA and M$^3$CoT demonstrate that VSSD improves rationale generation and answer inference. The code is available at https://github.com/BGWH123/VSSD.
Read the full abstract ▾
Computer Vision
Medical-Checklist: Assessing the Comprehension of Medical Images by Multimodal Models
Bannapol Limanond, Masanori Suganuma and Takayuki Okatani
This paper introduces a new
benchmark test, Medical-Checklist, for assessing medical
multimodal models. The recent advancements in
multimodal models have demonstrated significant potential in the field of medical
vision-
language tasks. However, it is becoming increasingly clear that evaluating these models' performance, whether they are applied to natural or medical images, is challenging. The critical question is whether the models can accurately understand an input image while associating it with relevant input text. To address this, Medical-Checklist imposes a binary test on the models: they are given an image and two captions, where one is correct and the other incorrect, and the model must select the correct one. The incorrect caption contains a single medical concept (word or phrase) that is inaccurately substituted from the correct caption. Although the task is simple, this simplicity enables the unified assessment of diverse
multimodal models designed and learned on different principles. It also enables us to verify whether models correctly understand a wide range of medical concepts across various medical sub-domains. Medical-Checklist is designed to reduce potential biases in data and to enable evaluation of the models' ability to handle out-of-distribution inputs, which were difficult in existing
datasets. When evaluating four state-of-the-art medical
multimodal models with Medical-Checklist, it was revealed that despite their excellent performance in specific tasks such as Med-VQA, they may not correctly understand images, suggesting a long journey ahead for clinical application. The
dataset and code will be made public upon acceptance.
Read the full abstract ▾
Computer Vision
Rethinking Layer-Wise Information Allocation for Vision Foundation Model Adaptation
Yuqi Li, Xi Xiao, Yunbei Zhang et al. (+6 more)
Vision foundation models are increasingly reused as frozen backbones for downstream visual recognition, making parameter-efficient adaptation a central problem.
Prompt-based adaptation, including Visual
Prompt Tuning (VPT), provides a lightweight way to specialize these models, but its layer-wise behavior remains poorly understood: performance is sensitive to
prompt depth, placement, and task distribution, and gains on standard in-domain
benchmarks do not always translate into robust generalization. We argue that this limitation is not solely an optimization issue, but a layer-wise information allocation issue: existing
prompt-based methods lack principled control over what
prompt-conditioned representations should preserve, suppress, and propagate across depth. Inspired by the Information Bottleneck principle, we introduce Prompted Information Bottlenecks (PIB), a framework that regularizes layer-wise compression-sufficiency trade-offs and promotes a more coherent cross-layer information path. The key idea is that effective adaptation should be minimal yet sufficient, retaining task-relevant local evidence in earlier layers while progressively discarding nuisance factors and redundant details in deeper layers. Extensive experiments show that PIB achieves strong performance across 34
datasets, reaching 92.1% on FGVC, 93.01% on HTA, and 77.33% on VTAB-1k, while tuning only 0.35% parameters on average across the main settings. Beyond
benchmark accuracy, PIB helps explain the non-monotonic behavior of
prompt capacity scaling, reduces shortcut reliance, and improves robustness under distribution shift and fine-grained recognition settings. These results position PIB as both a practical method and an information-allocation perspective for adapting frozen
vision foundation models. Our code is available at https://github.com/itsnotacie/MM-26-PIB
Read the full abstract ▾
All authors Yuqi Li, Xi Xiao, Yunbei Zhang, Lin Zhao, Yu Li, Aiden Zhao, Tianyang Wang, Hao Xu, Yingli Tian
Computer Vision
TextSLIP: Text Self-Supervised CLIP for Medical Report Generation
Haoyu Jiang and Ziping Cong
Automating radiology report generation is important for improving reporting consistency and clinical workflows . While Contrastive
Language–Image Pretraining (CLIP) has advanced medical
vision language modeling, existing CLIP-style approaches may still provide insufficient fine-grained semantic supervision for complex report generation. Standard CLIP primarily optimizes cross-modal
alignment, without explicitly structuring the textual
embedding space that guides visual representation learning. To address this limitation, we propose TextSLIP, a general medical
vision-
language pretraining framework that augments CLIP with intra-modal text contrastive learning. By improving textual
embedding discriminability through self-supervised augmented text pairs, TextSLIP is designed to provide finer-grained linguistic supervision to the visual encoder. As an initial validation, we pretrain TextSLIP on a curated
dataset of 7 million
brain MRI image-text pairs and fine-tune the pretrained visual encoder within a report generation architecture. In controlled comparisons with CLIP-style baselines, TextSLIP shows consistent improvements on report generation metrics. Ablation studies further suggest that text-side self-supervision contributes to the observed gains. These results indicate that text-level contrastive learning is a promising direction for improving medical visual-textual
alignment, while broader validation across additional medical domains remains an important next step.
Read the full abstract ▾
Computer Vision
Low-Altitude Channel Multipath Prediction via Panoramic Perception and Vision-Language Model
Zihang Zeng, Shu Sun, Meixia Tao et al. (+3 more)
Unmanned aerial vehicle (
UAV) communication is expected to support a wide range of low-altitude applications in 6G mobile networks. However, traditional statistical channel models provide limited accuracy in specific environments, while deterministic methods such as ray tracing usually rely on accurate three-dimensional environment models and involve high computational complexity. Existing
multimodal channel prediction approaches mainly focus on large-scale metrics such as path loss, and remain insufficient for modeling small-scale parameters. To address these limitations, this paper proposes PanoLAMP, a Panoramic perception and
vision-
language model-based Low-Altitude Multipath Prediction framework. It adopts a pretrained
vision-
language model as the backbone and captures the propagation environment features through panoramic RGB-D observations collected at both the transmitter and receiver to predict the delay, power, azimuth angle, and zenith angle offset relative to the line-of-sight path. Experiments are conducted on a synthetic
dataset containing 18,949
UAV-vehicle links across seven
UAV altitudes. Experimental results show that the proposed method consistently outperforms representative baselines in both multipath parameters and statistical metrics, and demonstrates stronger generalization across different flight heights.
Read the full abstract ▾
All authors Zihang Zeng, Shu Sun, Meixia Tao, Zhiyong Chen, Jianhua Mo, Xiangwen Gu
Computer Vision
Diffusion Models in Medical Image Inpainting: Challenges, Solution Taxonomy, and Future Directions
Arthur Dantas Mangussi, Joana Cristo Santos, Ricardo Cardoso Pereira et al. (+3 more)
Image inpainting aims to reconstruct missing or corrupted regions of an image while preserving as much as possible, visual and semantic consistency. In medical imaging, this task is particularly important because artifacts, missing information, and pathological alterations can compromise diagnostic reliability and downstream clinical applications. Recently,
diffusion models have emerged as state-of-the-art generative approaches for medical image inpainting due to their ability to generate anatomically consistent reconstructions. This survey presents a systematic review of
diffusion-based methods for medical image inpainting, covering the main architectures, applications,
datasets, and evaluation strategies reported across 60 studies. In addition, we propose a taxonomy for
diffusion-based approaches. The analysis reveals a rapid growth of research interest in
diffusion-based medical image inpainting, with denoising
diffusion probabilistic models and latent
diffusion models emerging as the dominant architectures. The reviewed studies mainly focus on artifact removal, data augmentation, pseudo-healthy tissue reconstruction, and anomaly detection, particularly in
magnetic resonance imaging and computed
tomography imaging. Overall,
diffusion models demonstrate strong performance in producing anatomically plausible reconstructions and aiding downstream clinical tasks. However, the review also highlights important challenges, including the lack of standardized
benchmarks, limited
dataset diversity, and restricted validation procedures across diverse clinical applications and imaging scenarios.
Read the full abstract ▾
All authors Arthur Dantas Mangussi, Joana Cristo Santos, Ricardo Cardoso Pereira, Ana Carolina Lorena, Mário A. T. Figueiredo, Pedro Henriques Abreu
Computer Vision
ISPCloak: Weaponizing ISP for Optimization-Free Physical Camouflage against Deepfake Detectors
Jiale Zhao, Jiajun Wan, Lei Tang et al. (+3 more)
The rapid advancement of generative models has spurred the critical need to evaluate the worst-case robustness of deepfake detectors. In this paper, we reveal a fundamental blind spot in current forensic paradigms: while existing detectors excel at capturing digital synthesis artifacts, their effectiveness drops drastically when AI-generated content is cloaked in authentic physical imaging characteristics. We posit that genuine photographs inherently possess hardware-intrinsic statistical signatures, which are imperceptible footprints imprinted by optical sensors and Image Signal Processing (ISP) pipelines, and are fundamentally absent in purely data-driven generative models. Driven by this insight, we propose ISPCloak, a novel optimization-free
adversarial attack framework that explicitly weaponizes the ISP pipeline to mislead the judgment of deepfake detectors. Rather than relying on computationally expensive gradient perturbations, our method first employs an Invertible ISP network to project images into the RAW domain. Then, we seamlessly imprint the complex statistical priors of real cameras onto AI-generated images by injecting realistic Poisson-Gaussian sensor noise and conducting forward ISP reconstruction. Synergized with generative artifact suppression and adaptive masking, this streamlined physical simulation enables ultra-fast generation of
adversarial examples. Extensive experiments show that
embedding authentic physical perturbations fundamentally disrupts a broad range of current detection mechanisms, yielding universally evasive
adversarial examples with imperceptible visual alterations.
Read the full abstract ▾
All authors Jiale Zhao, Jiajun Wan, Lei Tang, Ye Qin, Kebing Jin, Jinghui Qin
Computer Vision
Learning Adaptive Semantic Gaussian Allocation for 3D Occupancy
Kanglin Ning, Yiran Zhao, Wenrui Li et al. (+4 more)
Semantic 3D Gaussians provide a compact representation for 3D semantic occupancy prediction by rendering semantic primitives into a voxel volume under voxel-wise supervision. Recent methods have improved the modeling ability and efficiency of this representation through more flexible primitive shapes, geometry-guided initialization, and progressive densification. However, these advances mainly determine how primitives are represented, initialized, or added, and do not explicitly address how to select the most useful Gaussians when their total number must be limited to control
memory and computation. This imbalance creates an allocation bottleneck: redundant Gaussians remain in simple regions, while difficult regions receive insufficient semantic support. We propose the Semantic Gaussian Allocation
Transformer (SAGFormer), which uses Gaussian attributes and local
geometric-semantic features to score candidates and select a fixed final Gaussian set. Experiments on nuScenes-SurroundOcc and SSCBench-KITTI-360 show that SAGFormer improves occupancy prediction under the evaluated protocols and yields more semantically consistent and better-utilized Gaussian representations. Under similar final counts and raw coverage, it reduces semantic mixing, strengthens class-consistent voxel support, and produces fewer unused Gaussians. The results indicate that explicit capacity allocation is a useful complement to Gaussian refinement for semantic occupancy prediction.
Read the full abstract ▾
All authors Kanglin Ning, Yiran Zhao, Wenrui Li, Houde Quan, Qifan Li, Xingtao Wang, Xiaopeng Fan
Computer Vision
Farmland Extent and Visible Boundary Mapping from 1 m NAIP Imagery Using Residual U-Net and Text-Prompted SAM 3 Refinement
Mohammadreza Narimani, Vikram Anand and Parastoo Farajpoor
Agricultural field maps are often proprietary, incomplete, or outdated, yet they provide the spatial framework for crop monitoring, production accounting, and land-conversion analysis. This study presents a reproducible workflow for mapping farmland extent and visible boundaries from 1 m NAIP RGB imagery. Thirty-seven scenes spanning open cropland, peri-urban interfaces, semi-arid irrigation geometries, and fragmented mosaics were annotated in CVAT and converted to binary masks. Non-overlapping 256 x 256 patches yielded 5,698 samples, split by source scene into 3,850 training, 770 validation, and 1,078 test patches. A residual U-Net (ResUNet) trained with a Dice-dominant loss, L = 2.5(1 - Dice) + BCE, achieved test accuracy 0.8808, IoU 0.8605, Dice 0.9234, precision 0.8766, and recall 0.9794. A frozen SAM 3 branch prompted with "agricultural farmland field" was fused with ResUNet by logical OR. On selected difficult patches, Dice improved from 0.858 to 0.955 (orchard rows) and from 0.804 to 0.903 (fragmented parcels). Sliding-window stitching produced coherent regional masks (example tile Dice 0.898 and 0.919). The product is a semantic farmland-extent layer, not a cadastral parcel map, and supports agricultural monitoring where current field layers are unavailable.
Read the full abstract ▾