Computer Vision
Irem Zeynep Alagöz, Nils Morbitzer, Andrea Ramazzina et al. (+3 more)
Several disruptive research directions have recently emerged in computer
vision, including foundation models achieving previously unseen zero-shot performance in scene understanding, even interactively, and generative models that synthesize extremely realistic images. The latter have also been shown to be highly effective in scene understanding tasks thanks to their rich priors. However, for promptable
segmentation, foundation models struggle with accurately segmenting an object's region, leading to false positives and over-
segmentation. Notably, early attempts that leverage generative priors use
prompts only during post-processing, yielding suboptimal segments because the process is agnostic to the user input. In this paper, we target these limitations with
Prompt2Seg, a spatial conditioning framework for
diffusion-based
segmentation.
Prompt2Seg augments a frozen
diffusion segmentation model with a conditioning branch. Our approach takes spatial
prompts, represented as 2D Gaussians or confidence maps, as explicit input signals, training the model to respond directly to user intent. Fine-tuned on a deliberately constrained set of object categories drawn from Hypersim and Virtual KITTI 2,
Prompt2Seg generalizes zero-shot to a wide range of unseen object types and visual domains. We evaluate on seven
datasets ranging from standard
benchmarks to more challenging domains, including paintings,
egocentric views, and X-ray data. Furthermore, we demonstrate that
Prompt2Seg consistently outperforms the underlying
diffusion segmentation backbone across all
benchmarks. Our results suggest that the rich priors encoded in generative pretraining, combined with principled spatial conditioning, offer a compelling path toward broadly generalizing interactive
segmentation without large-scale mask supervision.
Read the full abstract ▾
All authors Irem Zeynep Alagöz, Nils Morbitzer, Andrea Ramazzina, Nassir Navab, Federico Tombari, Stefano Gasperini
Computer Vision
MaRS: Robust Out-of-Distribution Detection via Mahalanobis Residual Scoring
Francesco Di Salvo, Sebastian Doerrich and Christian Ledig
Foundation models provide highly descriptive representations for medical images, yet their reliability degrades under distribution shifts arising from changes in patients, devices, or acquisition conditions. Reliable out-of-distribution (OOD) detection is therefore essential for safe deployment. Recent post-hoc detectors efficiently exploit frozen
embeddings (e.g., kNN), whereas reconstruction-based OOD detection in latent feature space has seen limited adoption due to inconsistent performance. In this work, we show that the limitation of reconstruction-based methods in latent space does not stem from poor reconstruction quality, but from how reconstruction errors are scored. Standard L2 residual norms collapse the anisotropic residual structure, thereby suppressing informative deviations. To address this limitation, we introduce MaRS (Mahalanobis Residual Scoring), a label-free OOD detector that learns an in-distribution
manifold using a lightweight autoencoder and measures deviation via a Mahalanobis distance on reconstruction residuals, yielding variance-aware OOD scores. Across three imaging modalities, multiple types of distribution shift, and different model families and scales, MaRS outperforms established confidence-, distance-, and reconstruction-based baselines, while remaining fully post-hoc and lightweight. The code is available at https://github.com/francescodisalvo05/mars.
Read the full abstract ▾
Computer Vision
Learning Entropy Signature for Image Representation and Classification
Jan Glaser, Ivo Bukovsky, Noriyasu Homma et al. (+1 more)
Learning
Entropy (LE) has recently been extended to image analysis through Spatial Learning
Entropy Maps (SLEMs), which are two-dimensional LE distributions that highlight unusually high learning activity across an image. Unlike conventional image descriptors, SLEMs are generated by incremental, sample-wise learning of a pretrained feedforward MLP network, where local pixel neighborhoods are presented sequentially in a fixed spatial order to predict the corresponding central pixels. Consequently, the learning activity at each image location depends not only on its local structure but also on the knowledge acquired from previously processed locations. This paper introduces Learning
Entropy Signatures (LES), an image descriptor derived from SLEM using the K largest LE locations. LES captures the spatial organization of learning-relevant image structures and provides a compact representation of image content based on learning weight behavior. Experimental evaluation on image classification tasks shows that a relatively small number of K largest LE locations preserve substantial discriminative information. The results indicate a close relationship between the learning of neural weights and information relevance, extending the role of Learning
Entropy from time series to images and, within images, from structural point extraction to compact image representation and classification.
Read the full abstract ▾
All authors Jan Glaser, Ivo Bukovsky, Noriyasu Homma, Marcel Jirina
Computer Vision
4DVLT: Dynamic Scene Understanding with Worldline-Centered Vision-Language Tracking
Boxue Yang, Chaoyue Li, Shengyao Zhou et al. (+4 more)
Language-guided dynamic scene understanding requires identifying one physical entity across time, metric 3D space, and multiple viewpoints, a setting not captured by current
multimodal reasoning or tracking paradigms. We introduce 4DVLT, an offline calibrated multi-view task that unifies grounding with metric 3D and synchronized 2D tracking through a worldline binding identity, motion, visibility, and observations. We construct Instruct-4D, with 129.4K QA pairs, 64.7K targets, 851 scenes, and 9
reasoning types, and propose 4DTrack for query-conditioned inference over a 4D state graph. 4DTrack reaches 55.18 WQS, exceeding the strongest adapted VLT baseline by 33.01 points while improving grounding and trajectory quality. The project page is available at https://github.com/mikubaka88/4DVLT.
Read the full abstract ▾
All authors Boxue Yang, Chaoyue Li, Shengyao Zhou, Rui Qian, Haoyang Wu, Shaoqiu Zhang, Linfeng Zhang
Computer Vision
DR-Mamba: Automatic Inference-Time Domain Adaptation for Document Image Binarization via Sample-Conditioned Detail-Background Suppression
Sheng-Wei Chan and Jen-Shiun Chiang
Degraded document image binarization is sensitive to domain shifts caused by paper aging, bleed-through, stains, shadows, and uneven illumination, and the foreground-background separation of recent learning-based methods can become unstable on unseen degradation domains. We propose DR-Mamba, a sample-conditioned detail-background suppression framework that performs automatic inference-time domain adaptation for document image binarization. Unlike test-time adaptation methods that require gradient updates or auxiliary data at inference, DR-Mamba adapts to each input document through input-dependent gates within a single forward pass, requiring no target-domain labels, no fine-tuning, and no test-time parameter updates. Instead of using Mamba-style selective scanning as a single generic feature path, DR-Mamba reinterprets it as fast-slow route modeling: a fast detail route captures local stroke structures, while a slow background route accumulates spatially persistent degradation responses. The two routes are integrated through an input-dependent subtractive gate that explicitly suppresses background interference rather than fusing features by addition or concatenation. We further add full-resolution detail-guided reconstruction and thin-stroke-aware supervision to recover fine strokes lost during downsampling. Evaluated under a leave-one-year-out protocol on DIBCO-style
benchmarks, where each held-out year is treated as an unseen degradation domain, DR-Mamba shows that per-document, per-location subtractive suppression improves cross-domain robustness, with particularly strong performance on the most severely degraded held-out fold.
Read the full abstract ▾
Computer Vision
OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMs
Hao Vo, Phu Loc Nguyen, Khoa Vo et al. (+7 more)
Multimodal Large
Language Models (MLLMs) have achieved remarkable performance on 2D visual tasks, yet enhancing their spatial intelligence for real-world applications such as Autonomous Vehicles (AV) remains an open challenge. Existing geometry-aware MLLMs typically rely on auxiliary 3D models at inference time, introducing pipeline complexity and the risk of cascading failures. In this paper, we present OmniSpace, a simple yet effective plug-and-play paradigm for geometry-aware spatial
reasoning from purely 2D observations. Motivated by our finding that current MLLMs are bottlenecked by weak cross-view correspondence and depth estimation, OmniSpace introduces a Camera Pose Injector, a Multi-view Epipolar
Attention module, and a 3D
Geometric Distillation objective that jointly address these two limitations by transferring
geometric knowledge into the model. Extensive experiments show that OmniSpace surpasses existing methods on planning
benchmarks (nuScenes, Bench2Drive), risk detection (nuInstruct),
language (Omnidrive), and generalization (DriveBench).
Read the full abstract ▾
All authors Hao Vo, Phu Loc Nguyen, Khoa Vo, Sieu Tran, Duc Minh Nguyen, Ngo Xuan Cuong, Nghi D. Q. Bui, Anh Nguyen, Duy Minh Ho Nguyen, Ngan Le
Computer Vision
Automated sign detection across the Electronic Babylonian Library: A large-scale dataset and end-to-end cuneiform OCR pipeline
Wentao Che, Esteban Garcés Arias, Asim Niaz et al. (+2 more)
Learning to read cuneiform tablets is an extremely demanding task; consequently, of the roughly half million excavated tablets, only a small fraction has been analysed by Assyriologists. Computer
vision offers a promising avenue for decipherment but requires large, densely annotated
datasets. To address this limitation, the largest annotated cuneiform sign
dataset to date is used, and a Deformable Detection
Transformer (DETR)-based object detection model is evaluated under two class granularities of 173 and 106 classes. The proposed system integrates automatic tablet-side extraction, heuristic line grouping, and n-gram-based textual similarity evaluation to bridge visual sign detection and textual structure, and achieves consistent improvements of up to 28-37% over prior work on COCO-style detection metrics. At inference, the method is applied to 87,668 tablet fragments from the Electronic Babylonian Library (eBL) corpus, producing nearly 2.9 million sign detections. Although the approach operates without linguistic priors and remains sensitive to tablet damage and layout variability, it provides a scalable and interpretable foundation for corpus-wide cuneiform analysis and supports future integration with
multimodal and linguistic modelling frameworks.
Read the full abstract ▾
All authors Wentao Che, Esteban Garcés Arias, Asim Niaz, Andreas Bender, Enrique Jiménez
Computer Vision
MapReason-OSM: Can Vision-Language Models Make Graph-Verifiable Mobility Decisions from Street Maps ?
Srinivas Venkatanarayanan and Clement Pakkam Isaac
Vision-
language models (VLMs) are increasingly used to read maps for logistics, delivery, and accessible
navigation, where the output is an actionable decision (a route, a pin, a parking choice) that must respect the road network. Yet most map
benchmarks grade free text or multiple-choice answers that cannot be verified against the underlying graph. We present MapReason-OSM, a
benchmark and evaluation harness for graph-verifiable mobility decisions on self-rendered OpenStreetMap panels. We render fixed-style maps for ten U.S. downtowns at two aligned zoom scales, overlay a consistent marker grammar, and pair each panel with a hidden street graph and exact oracles, yielding 6,000 instances (12,000 panels across the two zooms) over 12 routing, facility-location, and visual disambiguation tasks. Models return structured decisions that we snap back to the graph and score for validity, legality, optimality, and constraint satisfaction, plus cross-zoom consistency. Across seven VLMs, models read maps and route simply but fail at graph cost
reasoning (single-facility pin placement is near chance even for frontier
reasoning models), and are frequently scale-inconsistent. We release the
benchmark, harness, and deterministic generator. Code and data: https://github.com/Vi-Sri/mapreason-osm
Read the full abstract ▾
Computer Vision
The Power of Light: Improving Synthetic-to-Real Domain Adaptation through Physically-Based Indirect Illumination
Hooman Tavakoli Ghinani, Tatjana Legler and Martin Ruskowski
While synthetic data generation resolves the manual labeling bottleneck in computer
vision, minimizing the syn-to-real domain gap requires optimizing rendering variables. This paper presents a systematic study analyzing the impact of lighting configurations and background complexity on object detection performance. We introduce SmartSDG, an automated, reproducible pipeline built on NVIDIA Isaac Sim using Physically-Based Shading (PBS), alongside ILLUM\_INTRUCK, a new multi-object industrial
benchmark dataset. Through 18 controlled experiments utilizing a state-of-the-art YOLOv12 framework, we demonstrate that complex, indirect lighting configurations paired with domain-relevant background variability significantly increase visual cue richness. Our quantitative findings show that avoiding direct specular peaks preserves crucial surface textures, mitigates the domain gap, reduces false positives, and accelerates model convergence compared to using conventional direct-light synthetic data. Ultimately, we provide actionable virtual scene design guidelines to maximize object detection robustness in industrial automation.
Read the full abstract ▾
Computer Vision
SeFi-Image: A Text-to-Image Foundation Model with Semantic-First Diffusion
Ruoyu Feng, Jinming Liu, Yuqi Wang et al. (+7 more)
Training image generation foundation models consumes substantial resources. Previous methods have attempted to leverage semantic guidance to accelerate the training process, yet their experiments were only conducted on simple
datasets such as ImageNet, at low resolutions, and with small-scale models. In this paper, we propose SeFi-Image, a text-to-image foundation model built upon semantic-first
diffusion, a novel latent
diffusion modeling paradigm. We instantiate SeFi-Image at three model scales, 1B, 2B, and 5B parameters, enabling systematic study of scaling behavior and flexible deployment under varying compute budgets. Notably, our largest 5B model was trained with merely 125K A800 GPU hours, corresponding to roughly 10-20% of the training compute used by Z-Image. However, it achieves results comparable to or even superior to Qwen-Image and Z-Image. Despite this modest training compute, SeFi-Image achieves strong performance on a wide range of
benchmarks, including GenEval, DPG, LongTextBench, OneIG, and CVTG-2K. Moreover, we provide DMD2-distilled few-step turbo variants for each model scale to accommodate diverse hardware constraints and latency requirements. We publicly release our code, weights and hope this work offers the community useful insights into semantic-guided
diffusion modeling for T2I generation, while also providing practical and readily deployable model options.
Read the full abstract ▾
All authors Ruoyu Feng, Jinming Liu, Yuqi Wang, Xin Cheng, Boyuan Liu, Shanglin Li, Hanshen Zhu, Wenfeng Lin, Mingyu Guo, Xin Jin
Computer Vision · Cross-listed from cs.CL
Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do
Zhuoran Jin, Kejian Zhu, Hongbang Yuan et al. (+5 more)
Chain-of-Thought (CoT) has become a standard method for improving
reasoning capabilities in large
language models (LLMs) by eliciting step-by-step thinking, but its effectiveness in
multimodal tasks remains unclear. In this paper, we aim to systematically investigate the key question: What can
multimodal Chain-of-Thought
reasoning do, and where and why does it fall short? To this end, we evaluate 12
multimodal tasks across perception and
reasoning categories using both 14 non-
reasoning models and 8
reasoning models. Our analysis reveals several important findings: (1) CoT is not a free lunch and should be used selectively depending on the specific requirements of each task. For perception tasks, CoT can lead to undesirable side effects, such as reduced performance in visual grounding and object counting. In contrast, it proves effective for
reasoning tasks involving mathematical, scientific, and multi-image
reasoning; (2) Compared to original models, existing open-source
multimodal reasoning models often yield only marginal overall improvements, possibly due to an overemphasis on mathematical
reasoning at the expense of broader capabilities; (3) Visual
reasoning remains a key bottleneck for current
multimodal CoT, as models exhibit a Look Light, Think Heavy pattern where verbal reflection rises and falls during
reasoning, whereas visual reflection consistently diminishes. These findings suggest that while
multimodal CoT handles verbal reflection relatively well, it lacks the ability to maintain deep visual introspection throughout the
reasoning process.
Read the full abstract ▾
All authors Zhuoran Jin, Kejian Zhu, Hongbang Yuan, Yupu Hao, Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao
Computer Vision
HiMatch-AD: DINOv3-driven Hierarchical Matching for Training-free Medical Anomaly Detection
Jiayu Huo, Jingyuan Hong, Meng Zhou et al. (+2 more)
Anomaly detection is essential for medical image analysis, where pathological regions often appear as rare deviations from normal anatomical structures. While training-based methods have achieved promising performance, they require task-specific optimization and extensive normal data, which limits scalability across modalities and institutions. Training-free approaches offer greater flexibility by leveraging pretrained visual representations, yet existing methods typically rely on simple nearest-neighbor
retrieval and naive aggregation strategies, which may fail to capture hierarchical semantics and ignore the reliability of multiple anomaly responses. In this work, we propose HiMatch-AD, a DINOv3-driven hierarchical matching framework for training-free medical anomaly detection. Our method first retrieves semantically relevant normal references via dual-branch matching that jointly considers global CLS-
token similarity and patch-level representations. Hierarchical anomaly maps are then generated across multiple
transformer stages by comparing clustered normal features with query representations. To robustly aggregate anomaly responses, we introduce a unified uncertainty-based fusion mechanism that adaptively weights maps according to their reliability. The entire framework operates without any task-specific training. Extensive experiments on the BMAD
benchmark, including
brain MRI, liver CT, and retinal OCT
datasets, demonstrate that HiMatch-AD consistently outperforms both training-based and DINO-based state-of-the-art methods, which highlights the effectiveness of multi-level matching and uncertainty-aware fusion for scalable medical anomaly detection.
Read the full abstract ▾
All authors Jiayu Huo, Jingyuan Hong, Meng Zhou, Liyun Chen, Le Zhang
Computer Vision · Cross-listed from cs.LG
Mitigating Measurement-Induced Training Instability in Hybrid Quantum Neural Networks for Protein Classification
Milton Mondal, Sushovan Chanda, Mohamad Mahdi Alawieh et al. (+6 more)
Hybrid
Quantum Neural Network (QNN) classifiers produce logits as expectation values of
quantum measurement operators. For standard Pauli measurements, these outputs are intrinsically bounded to the interval [-1,1]. When such bounded logits are used directly with the cross-
entropy loss applied to softmax-normalized logits for multi-class classification, the loss function operates in a regime of weak sensitivity to logit differences. As a consequence, parameter gradients are suppressed, leading to unstable optimization in variational
quantum classifiers (VQCs). In this work, we identify this effect as measurement-induced logit contraction, a previously uncharacterized source of trainability degradation in hybrid QNNs. To address this limitation, we introduce a learnable scaling parameter, termed
Quantum Measurement Temperature (QMT), which rescales
quantum measurement outputs prior to the loss. Unlike post-hoc calibration, QMT acts during training and compensates for the physically imposed bounds on
quantum measurement outputs. This rescaling increases gradient magnitude and variance, thereby improving loss sensitivity. The proposed mechanism is architecture-agnostic and does not modify the
quantum ansatz,
circuit depth, or measurement operators. Experiments on fluorescence microscopy images and a six-class variant of Fashion MNIST demonstrate that QMT consistently enhances logit separation, strengthens gradients, stabilizes training across random initializations, and improves classification accuracy, relative to unscaled measurement readouts. These results demonstrate that QMT enables stable and reliable training of hybrid QNNs for practical applications.
Read the full abstract ▾
All authors Milton Mondal, Sushovan Chanda, Mohamad Mahdi Alawieh, Brijesh Sukhadiya, Donatus Krah, Clinton Gonsalves, Antonios Ntolkeras, Silvio O. Rizzoli, Ali H. Shaib
Computer Vision
Training-Free Semantic Correction for Autoregressive Visual Models
Junhao Chen, Chanyu Zhu, Zheqi Lv et al. (+2 more)
Autoregressive visual models (AVMs) based on next-scale prediction have emerged as a prominent paradigm for image and
video synthesis. However, decomposing the generation process into discrete scales with varying granularities in AVM makes semantic errors difficult to identify and correct, thereby undermining the quality of the final output. Prior efforts to enhance AVM can be categorized into training-based and training-free approaches. Although training-based efforts to enhance AVM generation quality come at substantial computational cost, existing training-free methods neglect intermediate generation states, leaving semantic errors undiagnosed and allowing them to accumulate into the final output. In this paper, we focus on training-free paradigms and propose Gazer, a framework that integrates
multimodal large
language model feedback into the AVM sampling loop for in-generation semantic correction. Concretely, Gazer operates via two cooperating stages: the Reflective Diagnosis stage diagnoses semantic errors from intermediate states, while the Semantic Correction stage rewinds and rectifies the generation trajectory to realign with the target
prompt. Experiments on compositional image and
video benchmarks demonstrate that Gazer improves semantic
alignment and compositional accuracy across multiple AVMs without additional training.
Read the full abstract ▾
All authors Junhao Chen, Chanyu Zhu, Zheqi Lv, Keting Yin, Shengyu Zhang
Computer Vision
Venice-H1: Failure-Aware Query Re-Ranking with Multi-Scale Grid Signatures for Referring Image Segmentation
Nicolò Savioli
Modern Referring Image
Segmentation (RIS) systems generate multiple candidate masks per expression but rely on a simple heuristic–typically the argmax detection score–to select the final output. We identify query selection as a failure-case bottleneck: although heuristic selection succeeds on 82-93% of samples, the residual 7-18% of failures dominate the error budget, leaving a best-query selection gap of 3-11% mIoU. We introduce Venice-H1, a lightweight, backbone-decoupled post-hoc re-ranking module that encodes each candidate through multi-scale grid signatures–compact spatial descriptors pooled onto 4x4, 8x8, and 16x16 grids–and feeds them to a
Transformer-based re-ranker with a Failure Gate (ROCAUC 0.78-0.82) that intervenes only when the default choice is likely suboptimal. Instantiated on DeRIS-L and DeRIS-B, Venice-H1 achieves delta_fail of +1.40 and +0.89 mIoU with strictly positive 95% CIs on all 16/16 (split, backbone) pairs and harmful-switch rates below 0.53%. Zero-shot transfer to medical referring
segmentation (MS-CXR, M3D-RefSeg-2D) yields +1.16 and +0.51 mIoU without RIS-backbone fine-tuning. The module adds approximately 11.3M parameters and under 1 ms latency.
Read the full abstract ▾
Computer Vision
MAPS: Multi-Anchor Projection Similarity for Joint Vision-Language Geo-Localization
Yutong Hu, Siyuan Tan, Shaocheng Yan et al. (+3 more)
Humans localize places by integrating perceptual cues from
vision with semantic
reasoning from
language, forming a scene understanding that is both intuitive and structured. Although existing geo-localization models have made substantial progress in cross-view and cross-modal settings, they are largely built upon point-to-point
alignment, which is insufficient for joint
vision-
language queries. In such queries, visual and textual cues do not simply act as independent references, but jointly define a semantic subspace for locating the target. In this paper, we formulate
vision-
language geo-localization (VLGL) with joint image-text queries as a multi-anchor
geometric alignment problem and propose a unified framework for this setting. To realize this formulation, we propose Multi-Anchor Projection Similarity (MAPS), a new metric which constructs an anchor plane from visual and textual query features in a high-dimensional space and measures similarity by the projection length of the target feature onto this plane. Unlike cosine similarity which evaluates isolated pairwise relations, MAPS captures the
geometric consistency between the target feature and the joint query subspace, providing a more discriminative ranking criterion during
retrieval. To make the learned representation consistent with this geometry, we further introduce a MAPS-based contrastive loss that drives target features toward the corresponding anchor plane. The proposed framework, similarity metric, and training objective jointly yield state-of-the-art performance in VLGL.
Read the full abstract ▾
All authors Yutong Hu, Siyuan Tan, Shaocheng Yan, Pengcheng Shi, Qingwu Hu, Jiayuan Li
Computer Vision
PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Models
Xianghui Wang, Feng Chen, Wenbo Zhang et al. (+4 more)
Vision-
Language-Action (VLA) models provide a unified paradigm for robotic manipulation, yet their real-world deployment is often bottlenecked by execution efficiency. While existing efforts predominantly focus on compute-centric efficiency to reduce per-step inference latency, the intrinsic policy efficiency of these models remains largely unexplored. Policy efficiency is fundamentally affected by two factors, namely the effective executable length of predicted action chunks and the total physical steps required to complete a task. These two factors jointly determine the total number of forward inference calls during execution. We observe that current VLA policies struggle with planning unreliability and action redundancy, suffering from severe prediction degradation at the tail of action chunks and tending to generate unnecessarily redundant physical steps. To address this, we propose PolicyTrim, a reinforcement learning-based post-training framework that extends the reliable action chunk length and reduces redundant physical steps. For reliable chunk extension, we employ a dynamic exploration strategy that explicitly rewards the successful completion of longer executable lengths, progressively pushing the trustworthy prediction horizon to its empirical limit. For step efficiency, we design a redundancy-aware reward that directly favors successful task completions with fewer steps while penalizing unreproducible shortcuts, effectively eliminating redundant physical actions. Extensive experiments across three
benchmarks and three VLA models demonstrate that PolicyTrim improves action chunk utilization by 3$\times$ and reduces physical execution steps by 51.4%. Ultimately, our framework delivers up to a 5.83$\times$ end-to-end deployment speedup without compromising task success rates.
Read the full abstract ▾
All authors Xianghui Wang, Feng Chen, Wenbo Zhang, Hua Yan, Zixuan Wang, Changsheng Li, Yinjie Lei
Computer Vision
NegAS: Negative Label Guided Attention and Scoring for Out-of-Distribution Object Detection with Vision-Language Models
Yingjie Zhang, Shuai Li and Peng Wang
Out-of-Distribution (OOD) detection is essential for ensuring the robustness and reliability of object detection systems deployed in safety-critical applications. While prior research has mainly focused on uni-modal detectors or
vision-
language model (VLM) based classifiers, the potential of VLM-based object detectors in OOD scenarios remains underexplored. In this work, we take the first step toward building OOD object detection methods upon VLMs. We identify two challenges specific to VLM detectors: (i) their text-guided
attention enhances foreground with ID labels but treats background uniformly, leaving potential OOD regions unexploited for separating in-distribution (ID) from OOD instances; and (ii) their sigmoid-based multi-label outputs are incompatible with softmax-based OOD scores, calling for scoring functions consistent with VLM probabilistic outputs. Hence, we introduce Negative Label Guided
Attention and Scoring (NegAS). To address (i), we propose a negative label guided
attention module (NegA), where LLM-generated, visually-similar but semantically-different negative labels are used to guide
attention toward potential OOD background regions. To address (ii), we introduce a novel sigmoid-based OOD scoring function (NegS) that leverages both ID and negative labels, producing strong responses for ID instances and suppressed responses for OOD ones. Extensive experiments demonstrate that our approach improves OOD detection performance by a large margin while maintaining ID accuracy, e.g., reducing the FPR95 by 11.4% on the COCO
dataset and 25.5% on the OpenImages
dataset compared to the baseline model. While initially designed for dense VLM detectors like YOLO-World, we successfully adapt NegAS to Grounding DINO, a query-based VLM
transformer and achieve significant improvements, demonstrating the generalizability of our framework.
Read the full abstract ▾
Computer Vision
Trajectory Forcing: Structure-First Generation with Controllable Semantic Trajectories
Merve Kocabas, Gege Gao, Bernhard Schölkopf et al. (+1 more)
Diffusion and flow-based generative models produce strong images, yet their controllability remains largely endpoint-centric: users specify conditions and receive final outputs, while the intermediate generative dynamics remain hidden. Recent methods have begun to exploit generation order and process decomposition to improve sample quality, but still treat intermediate states as internal computation rather than objects for interaction. We propose Trajectory Forcing (TF), a trajectory-centric framework that makes the generation path explicit, semantic, and editable. TF organizes synthesis as a sequence of semantically structured stages, progressing from global layout to object-, part-, and detail-level representations. Each stage produces a decodable latent state that can be inspected, evaluated, and locally edited before the next stage begins. To instantiate this path, we derive coarse-to-fine teacher hierarchies by clustering pretrained visual representations such as DINOv2, and train a hierarchy-conditioned one-step flow-matching model at each level. We further introduce trajectory-aware metrics that measure structural consistency and local controllability beyond endpoint quality metrics such as FID. Experiments show that TF achieves competitive sample quality while exposing coherent intermediate states and supporting localized edits across semantic levels. By shifting the focus from final images to the generative path itself, TF opens a route toward controllable, trajectory-aware image synthesis.
Read the full abstract ▾
All authors Merve Kocabas, Gege Gao, Bernhard Schölkopf, Andreas Geiger
Computer Vision
Projection-Volume Fidelity Divergence: Diagnosing and Controlling Optimization Drift in Sparse-View 3D Gaussian Tomography
Yikuang Yuluo, Ao Wang, Shen Kuan et al. (+6 more)
Sparse-view computed
tomography is a severely ill-posed inverse problem, where recent 3D Gaussian Splatting methods offer an efficient explicit representation for tomographic reconstruction. However, we find that projection-domain optimization can be misleading in this setting: the rendered projections may continue to improve while the reconstructed volume deteriorates. We identify this failure mode as Projection-Volume Fidelity Divergence (PVFD), a representation-level optimization drift caused by anisotropic Gaussian deformation and view-specific primitive co-adaptation under sparse Radon constraints. To characterize this behavior, we introduce geometry- and volume-level diagnostics that measure needle-like Gaussian degeneration and the stability of the voxelized density field. Based on these observations, we propose LADES, a ground-truth-free optimization controller for sparse-view Gaussian
tomography. LADES combines Linearly Annealed Dropout, which applies strong
stochastic masking in early training to disrupt premature primitive co-adaptation and gradually restores full capacity for structural consolidation, with Structure-Aware Early Stopping, which terminates densification according to the saturation of Gaussian population growth rather than validation PSNR. Experiments on sparse-view CT reconstruction show that LADES improves volumetric fidelity, suppresses structural degeneration, and substantially reduces training time while maintaining competitive projection accuracy. These results suggest that robust Gaussian-based
tomography requires monitoring and controlling volumetric structure, rather than optimizing projection fit alone.
Read the full abstract ▾
All authors Yikuang Yuluo, Ao Wang, Shen Kuan, Yujie Liu, Wang Liao, Ying Chen, Shuangyang Zhong, Yixing Huang, Fuquan Wang
Computer Vision · Cross-listed from cs.LG
The Scissors Effect: When Resize-Based Input Diversity Helps or Hurts Transfer Attacks
Yuhang Jiang and Xiaojing Chen
Input Diversity (DI), a random resize and pad applied at each attack iteration, is a near-default ingredient of transfer-based attacks, widely assumed to improve transferability. We show this assumption is regime-dependent and, for adversarially trained surrogates, often reversed. Holding the attack fixed and varying only the surrogate, raising the DI probability improves transfer from standard surrogates but degrades it from robust ones: the two response curves separate like a pair of scissors, a pattern we call the Scissors Effect. On ImageNet, blind DI costs a robust source 10.3 percentage points of attack success across four architecturally diverse targets; the effect is several times smaller at 32x32. Direct measurement supports a bias-variance account: DI displaces the gradient by a comparable amount on both groups but reduces its variance only where the gradient is noisy, and robust surrogates have little noise left to average away. A gradient-consistency probe, frozen and hashed before the runs, predicts the sign of the effect on seven unseen surrogates, and we report where it fails alongside where it works. The practical consequence holds independently of the mechanism: leaving DI enabled by default understates the attack a robust surrogate can mount, and so overstates the robustness of the model being evaluated. Code: https://github.com/Avalon-S/ScissorsEffect.
Read the full abstract ▾
Computer Vision
Biological Sex Determination in Cadavers Using Deep Learning Algorithms from Computed Tomography Images of Pelvis and Skull
Giovanna Herculano Tormena, Davi Nascimento Araújo, Germano Coimbra Soares de Carvalho et al. (+10 more)
Sexual identification of decomposed cadavers challenges traditional methods dependent on visual anthropological analysis. This study evaluates state-of-the-art deep learning (including YOLO26, YOLO11, ConvNeXt-Tiny, EfficientNetV2, ViT-B16, VGG16, and ResNet50) with transfer learning to automatically determine biological sex from forensic computed
tomography (CT) scans. We analyzed 141 autopsied cadavers from the Forensic Medical Institute of Goiânia-GO, including a broad age range and varying conditions of preservation. The three-dimensional reconstructions of the pelvis and skull were converted into standardized two-dimensional profile projections, contributing to the study of this new technical approach. Data augmentation techniques compensated for sample limitations. Two scenarios were validated: binary and quaternary classification (one class per sex vs. one class per anatomical region of each sex). The best-performing model achieved highly consistent results on the pelvis region and still satisfactory performance on the skull region, reaching an overall patient-level accuracy of 95.65%, recall of 92.86%, F1- score of 94.36%, and precision of 97.22%, maintaining consistent performance across the evaluated cases, including those with trauma-related artifacts. Results indicate the technical feasibility of the methodology, demonstrating that deep learning models can provide objective, high-speed skeletal analysis. Since the study was conducted using data from a single institution and a single computed
tomography scanner, further validation across multiple centers and scanners is required to assess the generalizability of the proposed approach
Read the full abstract ▾
All authors Giovanna Herculano Tormena, Davi Nascimento Araújo, Germano Coimbra Soares de Carvalho, Gustavo Bruno Centenaro, Rafael Janowski Pozzer, Rodrigo Akira Azevedo Kurosawa, Danilo Aires Alves, Filipe Thiago Xavier de Campos, Pedro Henrique Macedo dos Santos, Pedro Augusto Prado Mota, Ricardo V. Godoy, João Manoel Herrera Pinheiro, Marcelo Becker
Computer Vision
Benchmarking Vision-Language Models for Microscopic Plant Image Understanding
Tianqi Wei, Xin Yu, Zhi Chen et al. (+2 more)
Microscopic imaging provides essential visual evidence for studying plant biology and pathology at the cellular and subcellular levels. However, existing
benchmarks on
vision-
language models primarily focus on macroscopic plant imagery, while the microscopic domain remains underexplored. To address this gap, we present PlantMicro, a comprehensive
benchmark for evaluating
vision-
language models (VLMs) in microscopic plant imagery. PlantMicro integrates more than 5,000 images collected across diverse hosts, biological domains, and imaging modalities. Building on this diversity, we design a set of complementary tasks that capture different facets of microscopic image understanding. To support these tasks, we construct over 9,000 VQA pairs that systematically evaluate the capabilities of VLMs. Experiments on PlantMicro show that current VLMs struggle with fine-grained recognition and biologically grounded
reasoning. For example, GPT-5 achieves 34.93% accuracy on the pathogen classification task, which is only modestly above the random-guessing baseline. The results highlight a significant gap in current VLMs' ability to comprehend plant microscopic images. PlantMicro provides a standardized foundation for advancing VLMs toward reliable and comprehensive microscopy-level plant understanding.
Read the full abstract ▾
All authors Tianqi Wei, Xin Yu, Zhi Chen, Scott Chapman, Zi Huang
Computer Vision
FetSelect: Task-Specific Architectures and Self-Supervised Learning for Automated Fetal Ultrasound Frame Selection
Mahmood Alzubaidi, Raden Muaz, Uzair Shah et al. (+4 more)
Automated frame selection for fetal biometry remains under addressed, with most prior work targeting generic quality assessment or downstream measurement pipelines that assume suitable frames are available. We introduce FetSelect, a task-specific framework that pairs a frozen
vision foundation backbone with a hybrid multi-head design: a Task-Gated classification head and a Detection-derived quality head combined via learned fusion. We curate 6,486 expert-labeled frames across four targets: Crown-Rump Length (CRL), Nuchal Translucency (NT), Nasal Bone (NB), and Scalebar, and adapt the backbone with BYOL pretraining on 19,019 unlabeled images. On a held-out test set (974 frames), FetSelect achieves mean AUROC 0.956 and mean correlation 0.818 with expert quality annotations. Ablations confirm that hybrid fusion surpasses single-head variants, and ultrasound-specific self-supervision yields consistent gains. Evaluation on external clinical
videos and 509 external CRL images demonstrates task-specific discrimination.
Read the full abstract ▾
All authors Mahmood Alzubaidi, Raden Muaz, Uzair Shah, Mohammed Ammar, Khalid Alyafei, Mowafa Househ, Marco Agus
Computer Vision
Human and AI collaboration for pulmonary nodule segmentation
Hongqiao Dong, Wenhao Chi, Ruobing Liang et al. (+9 more)
Medical expert annotators are scarce, and blind reliance on artificial intelligence (AI) can be misleading, motivating approaches in which humans, particularly junior medical trainees or even non-medical personnel, collaborate with AI to achieve robust medical
segmentation. Although the Segment Anything Model (SAM) shows promise for general-purpose image
segmentation, its performance in human-AI collaboration for specialized medical tasks has not been thoroughly evaluated. Here we present Hi-Seg, a human-in-the-loop
segmentation framework for pulmonary nodules built on SAM. Humans iteratively refine
prompts through trial-and-error learning and semantic
reasoning, progressively guiding SAM toward higher-quality masks. Using chest CT scans from 1,179 patients across 12 centers, we conducted the first large-scale external validation of collaborative human-SAM
segmentation. Across all annotator groups, Hi-Seg achieved a mean Dice score of almost 85%, outperforming five state-of-the-art deep learning models by 10-22% and 13 SAM variants by 1-29%. Hi-Seg improved
segmentation accuracy while reducing annotation time for medical annotators, and briefly trained non-medical annotators achieved performance comparable to that of the junior medical student. These findings suggest that human-in-the-loop
segmentation can reduce clinician workload, enable scalable crowdsourced annotation, and transform clinical workflows by facilitating the safe and efficient integration of foundation models into routine clinical practice.
Read the full abstract ▾
All authors Hongqiao Dong, Wenhao Chi, Ruobing Liang, Xiaokui Yang, Wenhua Liang, Peng Hou, Wenjun Pu, Yipeng Zhao, Ping Chen, Haiping Liu, Jianxing He, Bo Liu
Computer Vision · Cross-listed from cs.GR
Lighting-Consistent Object Transfer Across Radiance Fields
Nicolás Violante, George Kopanas, Linus Franke et al. (+2 more)
3D Gaussian Splatting (3DGS) is widely used to capture and render real scenes. Compositing objects from one capture into another has applications in many domains, such as VFX, architecture and interior design, or marketing. However, extracting an object from a source scene and naively pasting it into a target scene will fail to produce realistic results due to the different lighting conditions between the two scenes. To address this problem, we introduce a
diffusion model that harmonizes naively composited images with inconsistent lighting. The model is trained with a heterogeneous
dataset of image pairs (inconsistent composite input, consistent output), combining synthetic, generated, and real data. Our complete 3D solution allows a user to extract an object from the source scene and composite it into the target scene. From this, the (inconsistent) views of the target scene with the composite object are rendered. Our
diffusion model harmonizes each one of these views, which are finally consolidated in a 3DGS representation with a post-optimization step. Our method provides visually compelling results, making object transfer between 3DGS easy to use and significantly improving quality compared to previous methods.
Read the full abstract ▾
All authors Nicolás Violante, George Kopanas, Linus Franke, Julien Philip, George Drettakis
Computer Vision
Physically-guided Image Generation for Multi-Projection Mapping
Xingyun Liu, Yuqi Li, Jinhui Xiang et al. (+2 more)
Projection Mapping (PM) enables seamless superimposition of digital content onto real-world 3D objects, serving as a fundamental technique for immersive visualization, digital twins, and interactive art. Although text-to-image
diffusion models have greatly facilitated customized content creation, directly integrating them into practical PM pipelines remains challenging due to the mismatch between idealized 2D generation and physical constraints. To bridge this gap, this paper formalizes two application-level generative paradigms: the cooperative paradigm (harmonizing generated semantics with physical attributes) and the
adversarial paradigm (eliminating surface interference via radiometric compensation). Based on this, we propose ConPhyG, a unified controllable physically-guided generative multi-projection mapping framework that enables creators to interactively adjust physical constraints and flexibly switch generative paradigms. In cooperative mode, multi-dimensional physical priors (per-pixel gamut, depth, and edges) are injected into the
diffusion process. In
adversarial mode, the framework releases the generative potential and applies bounded numerical optimization for multi-projector radiometric compensation. It allows users to dynamically switch constraints to balance artistic freedom with physical feasibility. Furthermore, we extend ConPhyG to 360-degree multi-view consistent PM using a sequential generation strategy. Quantitative and qualitative evaluations on a real-world four-projector setup demonstrate that ConPhyG significantly outperforms state-of-the-art methods in
geometric alignment, gamut utilization, and semantic fidelity.
Read the full abstract ▾
All authors Xingyun Liu, Yuqi Li, Jinhui Xiang, Pinyan Tang, Chong Wang
Computer Vision
CVSBench: A Comprehensive Benchmark for Cross-view Spatial Reasoning and Dreaming
Ruixun Liu, Lingyu Zhang, Lanxuan Xue et al. (+3 more)
Humans can effortlessly reason about scenes across different viewpoints, yet it remains unclear whether
Vision-
Language Models (VLMs) possess similar cross-view spatial abilities. Satellite-street scene pairs, with their complex contexts and extreme viewpoint variations, provide an ideal testbed. Motivated by this, we introduce CVSBench, a large-scale
benchmark for evaluating cross-view spatial
reasoning through satellite-street pairs. This
benchmark supports multiple tasks, including cross-view VQA, cross-view grounding, and viewpoint identification. CVSBench comprises 3,297 cross-view image groups with 9,468 object-level annotations and 40,679 question-answer (QA) pairs, enabling systematic and controlled evaluation of cross-view spatial
reasoning. Extensive evaluations reveal that advanced VLMs struggle to maintain object-level and layout consistency under drastic viewpoint changes. To bridge this gap towards human-like spatial cognition, we investigate two categories of approaches: spatially grounded
reasoning and the incorporation of cognitive map inputs. Our findings demonstrate that
language-only
reasoning yields marginal improvements, while incorporating visual spatial imagination via a 3D scene imagination pipeline substantially improves cross-view
reasoning. These results highlight the necessity of explicit visual-spatial representations for robust spatial cognition in VLMs. Our data and code are released at https://huggingface.co/
datasets/zlyzlyzly/CVSBench.
Read the full abstract ▾
All authors Ruixun Liu, Lingyu Zhang, Lanxuan Xue, Kaiyu Li, Bowen Fu, Xiangyong Cao
Computer Vision
DreamUV: Unwrap Artist-like UV by End-to-End Flow Matching
Quanyuan Ruan, Jiabao Lei, Xingyi Du et al. (+1 more)
UV parameterization is a fundamental step in 3D content creation, yet producing production-ready UV layouts remains challenging due to the gap between
geometric distortion objectives and the stylistic preferences of professional artists. While classical methods optimize handcrafted energy functions, artist-authored UVs exhibit structural patterns such as straightened seams, axis-aligned islands, and flexible interior deformation, properties that are difficult to explicitly formulate. In this work, we present DreamUV, an end-to-end learning framework that formulates UV unwrapping as a generative Flow Matching problem. Rather than predicting a single optimal parameterization, DreamUV learns a mesh-conditioned transport process that maps noise samples to a distribution of artist-like UV layouts. To reflect real-world authoring practices, we introduce a boundary-aware training strategy that prioritizes seam geometry, and a Model-in-the-Loop Finetuning(MITL) scheme that explicitly accounts for discretization errors during sampling and stabilizes transport dynamics under heterogeneous supervision. We evaluate DreamUV on a large-scale
dataset of professionally authored UV layouts. Experiments demonstrate that our method produces significantly straighter boundaries and tighter axis-aligned islands than both classical and learning-based baselines, while maintaining competitive distortion metrics. Qualitative results and a user study with professional artists further confirm that DreamUV generates UV layouts that are not only valid, but aligned with practical production requirements.
Read the full abstract ▾
All authors Quanyuan Ruan, Jiabao Lei, Xingyi Du, Xifeng Gao
Computer Vision
Curvature-aware 3D length estimation of greenhouse cucumbers using RGB-D imaging and cubic spline arc-length integration
Manveen Kaur, Rajmeet Singh, Saeed Mozaffri et al. (+1 more)
Commercial greenhouse cucumber production is graded by fruit length, which drives harvest scheduling, labour allocation, and logistics. Manual measurement with thread or caliper is accurate but infeasible at commercial scale. This paper presents CucumberVision, a non-contact length estimation framework using an Intel RealSense D435 RGB-D camera. A YOLO26n instance
segmentation model locates cucumbers, and SAM (ViT-B backbone) refines each detection to a pixel-precise mask. Five methods are evaluated under matched conditions: (M1) a dominant-axis skeleton scan-line baseline; (M2) PCA on the bounding-box depth point cloud; (M3) SAM mask with medial-axis skeletonisation; (M4) a hybrid keypoint-guided approach using a YOLO26-pose model predicting five anatomical landmarks (KP0–KP4) with piecewise 3D arc-length; and (M5) a novel medial arc spline method fitting a cubic spline through the 3D medial axis of the SAM mask and computing arc length by trapezoidal integration – the first such application to elongated vegetable measurement. All methods share five-frame burst depth averaging, colour-stream intrinsic
alignment, and adaptive method selection with cascading fallbacks ensuring 100% coverage. A
benchmark of 48 captures across seven cucumbers in three size categories (small ~8 cm, medium ~13 cm, large ~25 cm) with thread-based ground truth establishes a significant accuracy hierarchy: M1 (MAPE 9.68%) > M2 (5.31%) > M4 (5.51%) > M3 (5.82%) > M5 (4.13%). M5 significantly outperforms all competitors at Bonferroni-corrected alpha=0.0125. A secondary contribution is identifying a 12–18% length underestimation caused by using depth-stream rather than colour-stream intrinsics after rs.align(rs.stream.color) – an under-reported error source. The complete system is released open source and runs in real time on a single consumer-grade GPU.
Read the full abstract ▾
All authors Manveen Kaur, Rajmeet Singh, Saeed Mozaffri, Shahpour Alirezaee