Computer Vision · Cross-listed from cs.MM
Zihang Wang, Siyue Zhang, Yilun Zhao et al. (+4 more)
Embedding models are a fundamental component of modern AI systems such as semantic search and
retrieval-augmented generation. Recent advances in large foundation models have substantially accelerated the development of
embedding models, including those based on Large
Language Models (LLMs),
Vision Language Models (VLMs), and
Multimodal LLMs. More recently, Large
Diffusion Language Models (dLLMs) and
Multimodal dLLMs have emerged as competitive alternatives to autoregressive models, offering advantages such as bidirectional
attention and parallel generation. This progress naturally raises a critical yet unexplored question: can
Multimodal dLLMs serve as effective
multimodal embedding models? To answer this, we present the first systematic study of converting
Multimodal dLLMs into
embedding models. We evaluate state-of-the-art
Multimodal dLLMs and Autoregressive VLMs across three categories of
embedding tasks: classification, visual question answering, and information
retrieval. Our results show that
Multimodal dLLM
embeddings generally underperform their autoregressive VLM counterparts. The stronger
diffusion-based model, LaViDa, lags by only 3.5 points on classification, 2.5 points on VQA, and 4.4 points on
retrieval tasks, whereas the other
diffusion-based model, MMaDA, exhibits substantially larger performance gaps, exceeding 20 points across all tasks. Further analysis reveals insufficient image-text
alignment in
diffusion-based models, accounting for the observed limitations in their
embedding performance.
Read the full abstract ▾
All authors Zihang Wang, Siyue Zhang, Yilun Zhao, Jingyi Yang, Tingyu Song, Anh Tuan Luu, Chen Zhao
Computer Vision
Diagnosis Support of Sickle Cell Anemia by Classifying Red Blood Cell Shape in Peripheral Blood Images
Wilkie Delgado-Font, Miriela Escobedo-Nicot, Manuel González-Hidalgo et al. (+3 more)
Red blood
cell (RBC) deformation is the consequence of several diseases, including sickle
cell anemia, which causes recurring episodes of pain and severe pronounced anemia. Monitoring patients with these diseases involves the observation of peripheral blood samples under a microscope, a time-consuming procedure. Moreover, a specialist is required to perform this technique, and owing to the subjective nature of the observation of isolated RBCs, the error rate is high. In this paper, we propose an automated method for differentially enumerating RBCs that uses peripheral blood smear image analysis. In this method, the objects of interest in the image are segmented using a Chan-Vese active contour model. An analysis is then performed to classify the RBCs, also called erythrocytes, as normal or elongated or having other deformations, using the basic shape analysis descriptors: circular shape factor (CSF) and elliptical shape factor (ESF). To analyze
cells that become partially occluded in a cluster during sample preparation, an elliptical adjustment is performed to allow the analysis of erythrocytes with discoidal and elongated shapes. The images of patient blood samples used in the study were acquired by a clinical laboratory specialist in the Special Hematology Department of the “Dr. Juan Bruno Zayas” General Hospital in Santiago de Cuba. A comparison of the results obtained by the proposed method in our experiments with those obtained by some state-of-the-art methods showed that the proposed method is superior for the diagnosis of sickle
cell anemia. This superiority is achieved for evidenced by the obtained F-measure value (0.97 for normal
cells and 0.95 for elongated ones) and several overall multiclass performance measures. The results achieved by the proposed method are suitable for the purpose of clinical treatment and diagnostic support of sickle
cell anemia.
Read the full abstract ▾
All authors Wilkie Delgado-Font, Miriela Escobedo-Nicot, Manuel González-Hidalgo, Silena Herold-Garcia, Antoni Jaume-i-Capó, Arnau Mir
Computer Vision
Data-Efficient Meningioma Segmentation via Implicit Spatiotemporal Mixing and Sim2Real Semantic Injection
Yunhao Xu, Fuquan Zong, Yexuan Xing et al. (+5 more)
The performance of medical image
segmentation is increasingly defined by the efficiency of data utilization rather than merely the volume of raw data. Accurate
segmentation, particularly for complex pathologies like meningiomas, demands that models fully exploit the latent information within limited high-quality annotations. To maximize the value of existing
datasets, we propose a novel dual-augmentation framework that synergistically integrates spatial
manifold expansion and semantic object injection. Specifically, we leverage Implicit Neural Representations (INR) to model continuous velocity fields. Unlike previous methods, we perform linear mixing on the integrated deformation fields, enabling the efficient generation of anatomically plausible variations by interpolating within the deformation space. This approach allows for the extensive exploration of structural diversity from a small set of anchors. Furthermore, we introduce a Sim2Real lesion injection module. This module constructs a high-fidelity simulation domain by transplanting lesion textures into healthy anatomical backgrounds, effectively bridging the gap between synthetic augmentation and real-world pathology. Comprehensive experiments on a hybrid
dataset demonstrate that our framework significantly enhances the data efficiency and robustness of state-of-the-art models, including nnU-Net and U-Mamba, offering a potent strategy for high-performance medical image analysis with limited annotation budgets.
Read the full abstract ▾
All authors Yunhao Xu, Fuquan Zong, Yexuan Xing, Chulong Zhang, Guang Yang, Shilong Yang, Xiaokun Liang, Juan Yu
Computer Vision · Cross-listed from cs.RO
Event-based Heterogeneous Information Processing for Online Vision-based Obstacle Detection and Localization
Reza Ahmadvand, Sarah Safura Sharif and Yaser Mike Banad
This paper introduces a novel framework for robotic
vision-based
navigation that integrates Hybrid Neural Networks (HNNs) with Spiking Neural Network (SNN)-based filtering to enhance situational awareness for unmodeled obstacle detection and localization. By leveraging the complementary strengths of Artificial Neural Networks (ANNs) and SNNs, the system achieves both accurate environmental understanding and fast, energy-efficient processing. The proposed architecture employs a dual-pathway approach: an ANN component processes static spatial features at low frequency, while an SNN component handles dynamic, event-based sensor data in real time. Unlike conventional hybrid architectures that rely on domain conversion mechanisms, our system incorporates a pre-developed SNN-based filter that directly utilizes spike-encoded inputs for localization and state estimation. Detected anomalies are validated using contextual information from the ANN pathway and continuously tracked to support anticipatory
navigation strategies. Simulation results demonstrate that the proposed method offers acceptable detection accuracy while maintaining computational efficiency close to SNN-only implementations, which operate at a fraction of the resource cost. This framework represents a significant advancement in neuromorphic
navigation systems for
robots operating in unpredictable and dynamic environments.
Read the full abstract ▾
Computer Vision
Analyzing VLM-Based Approaches for Anomaly Classification and Segmentation
Mohit Kakda, Mirudula Shri Muthukumaran, Uttapreksha Patel et al. (+1 more)
Vision-
Language Models (VLMs), particularly CLIP, have revolutionized anomaly detection by enabling zero-shot and few-shot defect identification without extensive labeled
datasets. By learning aligned representations of images and text, VLMs facilitate anomaly classification and
segmentation through natural
language descriptions of normal and abnormal states, eliminating traditional requirements for task-specific training or defect examples. This project presents a comprehensive analysis of VLM-based approaches for anomaly classification (AC) and anomaly
segmentation (AS). We systematically investigate key architectural paradigms including sliding window-based dense feature extraction (WinCLIP), multi-stage feature
alignment with learnable projections (AprilLab framework), and compositional
prompt ensemble strategies. Our analysis evaluates these methods across critical dimensions: feature extraction mechanisms, text-visual
alignment strategies,
prompt engineering techniques, zero-shot versus few-shot trade-offs, computational efficiency, and cross-domain generalization. Through rigorous experimentation on
benchmarks such as MVTec AD and VisA, we compare classification accuracy,
segmentation precision, and inference efficiency. The primary contribution is a foundational understanding of how and why VLMs succeed in anomaly detection, synthesizing practical insights for method selection and identifying current limitations. This work aims to facilitate informed adoption of VLM-based methods in industrial quality control and guide future research directions.
Read the full abstract ▾
All authors Mohit Kakda, Mirudula Shri Muthukumaran, Uttapreksha Patel, Lawrence Swaminathan Xavier Prince
Computer Vision
SGW-GAN: Sliced Gromov-Wasserstein Guided GANs for Retinal Fundus Image Enhancement
Yujian Xiong, Xuanzhao Dong, Wenhui Zhu et al. (+3 more)
Retinal fundus photography is indispensable for ophthalmic screening and diagnosis, yet image quality is often degraded by noise, artifacts, and uneven illumination. Recent GAN- and
diffusion-based enhancement methods improve perceptual quality by aligning degraded images with high-quality distributions, but our analysis shows that this focus can distort intra-class geometry: clinically related samples become dispersed, disease-class boundaries blur, and downstream tasks such as grading or lesion detection are harmed. The Gromov Wasserstein (GW) discrepancy offers a principled solution by aligning distributions through internal pairwise distances, naturally preserving intra-class structure, but its high computational cost restricts practical use. To overcome this, we propose SGW-GAN, the first framework to incorporate Sliced GW (SGW) into retinal image enhancement. SGW approximates GW via random projections, retaining relational fidelity while greatly reducing cost. Experiments on public
datasets show that SGW-GAN produces visually compelling enhancements, achieves superior diabetic retinopathy grading, and reports the lowest GW discrepancy across disease labels, demonstrating both efficiency and clinical fidelity for unpaired medical image enhancement.
Read the full abstract ▾
All authors Yujian Xiong, Xuanzhao Dong, Wenhui Zhu, Xin Li, Oana Dumitrascu, Yalin Wang
Computer Vision
Diffusion Representations for Fine-Grained Image Classification: A Marine Plankton Case Study
A. Nieto Juscafresa, Á. Mazcuñán Herreros and J. Sullivan
Diffusion models have emerged as state-of-the-art generative methods for image synthesis, yet their potential as general-purpose feature encoders remains underexplored. Trained for denoising and generation without labels, they can be interpreted as self-supervised learners that capture both low- and high-level structure. We show that a frozen
diffusion backbone enables strong fine-grained recognition by probing intermediate denoising features across layers and timesteps and training a linear classifier for each pair. We evaluate this in a real-world plankton-monitoring setting with practical impact, using controlled and comparable training setups against established supervised and self-supervised baselines. Frozen
diffusion features are competitive with supervised baselines and outperform other self-supervised methods in both balanced and naturally long-tailed settings. Out-of-distribution evaluations on temporally and geographically shifted plankton
datasets further show that frozen
diffusion features maintain strong accuracy and Macro F1 under substantial distribution shift.
Read the full abstract ▾
Computer Vision
Using deep learning for predicting cleansing quality of colon capsule endoscopy images
Puneet Sharma, Kristian Dalsbø Hindberg, Benedicte Schelde-Olesen et al. (+3 more)
In this study, we explore the application of deep learning techniques for predicting cleansing quality in colon capsule endoscopy (CCE) images. Using a
dataset of 500 images labeled by 14 clinicians on the Leighton-Rex scale (Poor, Fair, Good, and Excellent), a ResNet-18 model was trained for classification, leveraging stratified K-fold cross-validation to ensure robust performance. To optimize the model, structured
pruning techniques were applied iteratively, achieving significant sparsity while maintaining high accuracy. Explainability of the pruned model was evaluated using Grad-CAM, Grad-CAM++, Eigen-CAM, Ablation-CAM, and Random-CAM, with the ROAD method employed for consistent evaluation. Our results indicate that for a pruned model, we can achieve a cross-validation accuracy of 88% with 79% sparsity, demonstrating the effectiveness of
pruning in improving efficiency from 84% without compromising performance. We also highlight the challenges of evaluating cleansing quality of CCE images, emphasize the importance of explainability in clinical applications, and discuss the challenges associated with using the ROAD method for our task. Finally, we employ a variant of adaptive temperature scaling to calibrate the pruned models for an external
dataset.
Read the full abstract ▾
All authors Puneet Sharma, Kristian Dalsbø Hindberg, Benedicte Schelde-Olesen, Ulrik Deding, Esmaeil S. Nadimi, Jan-Matthias Braun
Computer Vision
Local-to-Global Logical Explanations for Deep Vision Models
Bhavan Vasu, Giuseppe Raffa and Prasad Tadepalli
While deep neural networks are extremely effective at classifying images, they remain opaque and hard to interpret. We introduce local and global explanation methods for black-box models that generate explanations in terms of human-recognizable primitive concepts. Both the local explanations for a single image and the global explanations for a set of images are cast as logical formulas in monotone disjunctive-normal-form (MDNF), whose satisfaction guarantees that the model yields a high score on a given class. We also present an
algorithm for explaining the classification of examples into multiple classes in the form of a monotone explanation list over primitive concepts. Despite their simplicity and interpretability we show that the explanations maintain high fidelity and coverage with respect to the blackbox models they seek to explain in challenging
vision datasets.
Read the full abstract ▾
Computer Vision
SQuID: A Benchmark for Quantitative Spatial Reasoning on Satellite Imagery
Peter A. Massih, Eric Cosatto and Martin Renqiang Min
Vision-
Language Models (VLMs) describe satellite images fluently but struggle with the quantitative spatial
reasoning that remote sensing (RS) analysts perform routinely, such as counting buildings, measuring forest area in hectares and computing distances in meters. We introduce SQuID (Satellite Quantitative Intelligence
Dataset), a
benchmark of 2,000 questions over 587 images from four sources at 0.3 to 1.0 m ground sampling distance, covering 24 question types in three difficulty tiers. Answers are computed from curated
segmentation masks or, for 50 questions, from human consensus, and numeric answers are scored against tolerance ranges informed by human annotations. SQuID also releases the reference masks behind its answers, so that modular systems can be evaluated using reference rather than predicted masks, and 75,687 training question-answer pairs on disjoint images (SQuID-Train). Among 15 evaluated systems, the best zero-shot end-to-end VLM reaches 28.1%, and RS-specialized VLMs score lower. As a baseline, we propose QASR (Quantitative Agentic Spatial
Reasoning), in which an LLM writes a program that segments the image and measures the masks with
geometric operators. QASR reaches 42.0%. On the 1,950 mask-derived questions, running its programs on the reference masks instead of predicted masks raises accuracy from 42.2% to 82.6%, so
segmentation is its main source of error. Fine-tuning Qwen2.5-VL-7B on SQuID-Train raises its accuracy from 16.1% to 40.7%, close to QASR. Multi-condition questions (Tier 3) remain the hardest in every setting, at 62.0% for QASR with reference masks and 13.2% after fine-tuning. These results leave clear room for end-to-end models that learn metric spatial
reasoning, and SQuID provides the data to train and evaluate them.
Read the full abstract ▾
Computer Vision
Deep Image Prior with L0 Gradient Regularizer for Image Smoothing
Nhat Thanh Tran, Kevin Bui and Jack Xin
Image smoothing is a fundamental image processing operation that preserves the underlying structure, such as strong edges and contours, and removes minor details and textures in an image. Many image smoothing
algorithms rely on computing local window statistics or solving an optimization problem. Recent state-of-the-art methods leverage deep learning, but they require a carefully curated training
dataset. Because constructing a proper training
dataset for image smoothing is challenging, we propose DIP-$\ell_0$, a deep image prior framework that incorporates the $\ell_0$ gradient regularizer. This framework can perform high-quality image smoothing without any training data. To properly minimize the associated loss function that has the nonconvex, nonsmooth $\ell_0$ “norm", we develop an alternating direction method of multipliers
algorithm that utilizes an off-the-shelf $\ell_0$ gradient minimization solver. Numerical experiments demonstrate that the proposed DIP-$\ell_0$ outperforms many image smoothing
algorithms in edge-preserving image smoothing and JPEG artifact removal.
Read the full abstract ▾
Computer Vision
Leveraging Transformer Decoder for Automotive Radar Object Detection
Changxu Zhang, Zhaoze Wang, Tai Fei et al. (+5 more)
In this paper, we present a
Transformer-based architecture for 3D radar object detection that uses a novel
Transformer Decoder as the prediction head to directly regress 3D bounding boxes and class scores from radar feature representations. To bridge multi-scale radar features and the decoder, we propose Pyramid
Token Fusion (PTF), a lightweight module that converts a feature pyramid into a unified, scale-aware
token sequence. By formulating detection as a set prediction problem with learnable object queries and positional encodings, our design models long-range spatial-temporal correlations and cross-feature interactions. This approach eliminates dense proposal generation and heuristic post-processing such as extensive non-maximum suppression (NMS) tuning. We evaluate the proposed framework on the RADDet, where it achieves significant improvements over state-of-the-art radar-only baselines.
Read the full abstract ▾
All authors Changxu Zhang, Zhaoze Wang, Tai Fei, Christopher Grimm, Yi Jin, Claas Tebruegge, Ernst Warsitz, Markus Gardill
Computer Vision
Organ-Aware Attention Improves CT Triage and Classification
Lavsen Dahal, Yubraj Bhandari, Geoffrey D. Rubin et al. (+1 more)
There is an urgent need for triage and classification of high-volume medical imaging modalities such as computed
tomography (CT), which can improve patient care and mitigate radiologist burnout. Study-level CT triage requires calibrated predictions with localized evidence; however, off-the-shelf
Vision Language Models (VLM) struggle with 3D anatomy, protocol shifts, and noisy report supervision. This study used the two largest publicly available chest CT
datasets: CT-RATE and RADCHEST-CT (held-out external test set). Our carefully tuned supervised baseline (instantiated as a simple Global Average Pooling head) establishes a new supervised state of the art, surpassing all reported linear-probe VLMs. Building on this baseline, we present ORACLE-CT, an encoder-agnostic, organ-aware head that pairs Organ-Masked
Attention (mask-restricted, per-organ pooling that yields spatial evidence) with Organ-Scalar Fusion (lightweight fusion of normalized volume and mean-HU cues). In the chest setting, ORACLE-CT masked
attention model achieves AUROC 0.86 on CT-RATE; in the abdomen setting, on MERLIN (30 findings), our supervised baseline exceeds a reproduced zero-shot VLM baseline obtained by running publicly released weights through our pipeline, and adding masked
attention plus scalar fusion further improves performance to AUROC 0.85. Together, these results deliver state-of-the-art supervised classification performance across both chest and abdomen CT under a unified evaluation protocol. The source code is available at https://github.com/lavsendahal/oracle-ct.
Read the full abstract ▾
All authors Lavsen Dahal, Yubraj Bhandari, Geoffrey D. Rubin, Joseph Y. Lo
Computer Vision
Practical Insights into Semi-Supervised Object Detection Approaches
Chaoxin Wang, Bharaneeshwar Balasubramaniyam, Anurag Sangem et al. (+2 more)
Learning in data-scarce settings has recently gained significant
attention in the research community. Semi-supervised object detection(SSOD) aims to improve detection performance by leveraging a large number of unlabeled images alongside a limited number of labeled images(a.k.a.,few-shot learning). In this paper, we present a comprehensive comparison of three state-of-the-art SSOD approaches, including MixPL, Semi-DETR and Consistent-Teacher, with the goal of understanding how performance varies with the number of labeled images. We conduct experiments using the MS-COCO and Pascal VOC
datasets, two popular object detection
benchmarks which allow for standardized evaluation. In addition, we evaluate the SSOD approaches on a custom Beetle
dataset which enables us to gain insights into their performance on specialized
datasets with a smaller number of object categories. Our findings highlight the trade-offs between accuracy, model size, and latency, providing insights into which methods are best suited for low-data regimes.
Read the full abstract ▾
All authors Chaoxin Wang, Bharaneeshwar Balasubramaniyam, Anurag Sangem, Nicolais Guevara, Doina Caragea
Computer Vision
A Lightweight Model-Driven 4D Radar Framework for Pervasive Human Detection in Harsh Conditions
Zhenan Liu, Amir Khajepour and George Shaker
Pervasive sensing in industrial and underground environments is severely constrained by airborne
dust, smoke, confined geometry, and metallic structures, which rapidly degrade optical and
LiDAR based perception. Elevation resolved 4D mmWave radar offers strong resilience to such conditions, yet there remains a limited understanding of how to process its sparse and anisotropic point clouds for reliable human detection in enclosed, visibility degraded spaces. This paper presents a fully model-driven 4D radar perception framework designed for real-time execution on embedded edge hardware. The system uses radar as its sole perception modality and integrates domain aware multi threshold filtering, ego motion compensated temporal accumulation, KD tree Euclidean clustering with Doppler aware refinement, and a rule based 3D classifier. The framework is evaluated in a
dust filled enclosed trailer and in real underground mining tunnels, and in the tested scenarios the radar based detector maintains stable pedestrian identification as camera and
LiDAR modalities fail under severe visibility degradation. These results suggest that the proposed model-driven approach provides robust, interpretable, and computationally efficient perception for safety-critical applications in harsh industrial and subterranean environments.
Read the full abstract ▾
Computer Vision
Spherical Geometry Diffusion: Generating High-quality 3D Face Geometry via Sphere-anchored Representations
Junyi Zhang, Yiming Wang, Yunhong Lu et al. (+5 more)
A fundamental challenge in text-to-3D face generation is achieving high-quality geometry. The core difficulty lies in the arbitrary and intricate distribution of vertices in 3D space, making it challenging for existing models to establish clean connectivity and resulting in suboptimal geometry. To address this, our core insight is to simplify the underlying
geometric structure by constraining the distribution onto a simple and regular
manifold, a topological sphere. Building on this, we first propose the Spherical Geometry Representation, a novel face representation that anchors
geometric signals to uniform spherical coordinates. This guarantees a regular point distribution, from which the mesh connectivity can be robustly reconstructed. Critically, this canonical sphere can be seamlessly unwrapped into a 2D map, creating a perfect synergy with powerful 2D generative models. We then introduce Spherical Geometry
Diffusion, a conditional
diffusion framework built upon this 2D map. It enables diverse and controllable generation by jointly modeling geometry and texture, where the geometry explicitly conditions the texture synthesis process. Our method's effectiveness is demonstrated through its success in a wide range of tasks: text-to-3D generation, face reconstruction, and text-based 3D editing. Extensive experiments show that our approach substantially outperforms existing methods in
geometric quality, textual fidelity, and inference efficiency.
Read the full abstract ▾
All authors Junyi Zhang, Yiming Wang, Yunhong Lu, Qichao Wang, Wenzhe Qian, Xiaoyin Xu, David Gu, Min Zhang
Computer Vision
Real-Time 4D Radar Perception for Robust Human Detection in Harsh Enclosed Environments
Zhenan Liu, Yaodong Cui, Amir Khajepour et al. (+1 more)
This paper introduces a novel methodology for generating controlled, multi-level
dust concentrations in a highly cluttered environment representative of harsh, enclosed environments, such as underground mines, road tunnels, or collapsed buildings, enabling repeatable mm-wave propagation studies under severe electromagnetic constraints. We also present a new 4D mmWave radar
dataset, augmented by camera and
LiDAR, illustrating how
dust particles and reflective surfaces jointly impact the sensing functionality. To address these challenges, we develop a threshold-based noise filtering framework leveraging key radar parameters (RCS, velocity, azimuth, elevation) to suppress ghost targets and mitigate strong multipath reflections at the raw data level. Building on the filtered point clouds, a cluster-level, rule-based classification pipeline exploits radar semantics-velocity, RCS, and volumetric spread-to achieve reliable, real-time pedestrian detection without extensive domainspecific training. Experimental results confirm that this integrated approach significantly enhances clutter mitigation, detection robustness, and overall system resilience in
dust-laden mining environments.
Read the full abstract ▾
All authors Zhenan Liu, Yaodong Cui, Amir Khajepour, George Shaker
Computer Vision
MultiST: A Cross-Attention-Based Multimodal Model for Spatial Transcriptomic
Wei Wang, Quoc-Toan Ly, Chong Yu et al. (+1 more)
Spatial transcriptomics (ST) enables transcriptome-wide profiling while preserving the spatial context of tissues, offering unprecedented opportunities to study tissue organization and
cell-
cell interactions in situ. Despite recent advances, existing methods often lack effective integration of histological morphology with molecular profiles, relying on shallow fusion strategies or omitting tissue images altogether, which limits their ability to resolve ambiguous spatial domain boundaries. To address this challenge, we propose MultiST, a unified
multimodal framework that jointly models spatial
topology, gene expression, and tissue morphology through cross-
attention-based fusion. MultiST employs graph-based gene encoders with
adversarial alignment to learn robust spatial representations, while integrating color-normalized histological features to capture molecular-morphological dependencies and refine domain boundaries. We evaluated the proposed method on 13 diverse ST
datasets spanning two organs, including human
brain cortex and breast cancer tissue. MultiST yields spatial domains with clearer and more coherent boundaries than existing methods, leading to more stable pseudotime trajectories and more biologically interpretable
cell-
cell interaction patterns. The MultiST framework and source code are available at https://github.com/LabJunBMI/MultiST.git.
Read the full abstract ▾
All authors Wei Wang, Quoc-Toan Ly, Chong Yu, Jun Bai
Computer Vision
CausalSpatial: A Benchmark for Object-Centric Causal Spatial Reasoning
Wenxin Ma, Chenlong Wang, Ruisheng Yuan et al. (+6 more)
Humans can look at a static scene and instantly predict what happens next – will moving this object cause a collision? We call this ability Causal Spatial
Reasoning. However, current
multimodal large
language models (MLLMs) cannot do this, as they remain largely restricted to static spatial perception, struggling to answer "what-if" questions in a 3D scene. We introduce CausalSpatial, a diagnostic
benchmark evaluating whether models can anticipate consequences of object motions across four tasks: Collision, Compatibility, Occlusion, and Trajectory. Results expose a severe gap: humans score 84% while GPT-5 achieves only 54%. Why do MLLMs fail? Our analysis uncovers a fundamental deficiency: models over-rely on textual chain-of-thought
reasoning that drifts from visual evidence, producing fluent but spatially ungrounded
hallucinations. To address this, we propose the Causal Object World model (COW), a framework that externalizes the simulation process by generating
videos of hypothetical dynamics. With explicit visual cues of causality, COW enables models to ground their
reasoning in physical reality rather than linguistic priors. We make the
dataset and code publicly available here: https://github.com/CausalSpatial/CausalSpatial
Read the full abstract ▾
All authors Wenxin Ma, Chenlong Wang, Ruisheng Yuan, Hao Chen, Nanru Dai, S. Kevin Zhou, Yijun Yang, Alan Yuille, Jieneng Chen
Computer Vision
Enginuity: Building an Open Multi-Domain Dataset of Complex Engineering Diagrams
Ethan Seefried, Prahitha Movva, Naga Harshita Marupaka et al. (+2 more)
We propose Enginuity - the first open, large-scale, multi-domain engineering diagram
dataset with comprehensive structural annotations designed for automated diagram parsing. By capturing hierarchical component relationships, connections, and semantic elements across diverse engineering domains, our proposed
dataset would enable
multimodal large
language models to address critical downstream tasks including structured diagram parsing, cross-modal information
retrieval, and AI-assisted engineering simulation. Enginuity would be transformative for AI for Scientific Discovery by enabling artificial intelligence systems to comprehend and manipulate the visual-structural knowledge embedded in engineering diagrams, breaking down a fundamental barrier that currently prevents AI from fully participating in scientific workflows where diagram interpretation, technical drawing analysis, and visual
reasoning are essential for hypothesis generation, experimental design, and discovery.
Read the full abstract ▾
All authors Ethan Seefried, Prahitha Movva, Naga Harshita Marupaka, Tilak Kasturi, Tirthankar Ghosal
Computer Vision
Deep Learning for Semantic Segmentation of 3D Ultrasound Data
Chenyu Liu, Marco Cecotti, Harikrishnan Vijayakumar et al. (+3 more)
Developing cost-efficient and reliable perception systems remains a central challenge for automated vehicles.
LiDAR and camera-based systems dominate, yet they present trade-offs in cost, robustness and performance under adverse conditions. This work introduces a novel framework for learning-based 3D semantic
segmentation using Calyo Pulse, a modular, solid-state 3D ultrasound sensor system for use in harsh and cluttered environments. A 3D U-Net architecture is introduced and trained on the spatial ultrasound data for volumetric
segmentation. Results demonstrate robust
segmentation performance from Calyo Pulse sensors, with potential for further improvement through larger
datasets, refined ground truth, and weighted loss functions. Importantly, this study highlights 3D ultrasound sensing as a promising complementary modality for reliable autonomy.
Read the full abstract ▾
All authors Chenyu Liu, Marco Cecotti, Harikrishnan Vijayakumar, Patrick Robinson, James Barson, Mihai Caleap
Computer Vision · Cross-listed from cs.CL
Aligning Agentic World Models via Knowledgeable Experience Learning
Baochang Ren, Yunzhi Yao, Rui Sun et al. (+3 more)
Current Large
Language Models (LLMs) exhibit a critical modal disconnect: they possess vast semantic knowledge but lack the procedural grounding to respect the immutable laws of the physical world. Consequently, while these
agents implicitly function as world models, their simulations often suffer from physical
hallucinations-generating plans that are logically sound but physically unexecutable. Existing
alignment strategies predominantly rely on resource-intensive training or fine-tuning, which attempt to compress dynamic environmental rules into static model parameters. However, such parametric encapsulation is inherently rigid, struggling to adapt to the open-ended variability of physical dynamics without continuous, costly retraining. To bridge this gap, we introduce WorldMind, a framework that autonomously constructs a symbolic World Knowledge Repository by synthesizing environmental feedback. Specifically, it unifies Process Experience to enforce physical feasibility via prediction errors and Goal Experience to guide task optimality through successful trajectories. Experiments on EB-ALFRED and EB-Habitat demonstrate that WorldMind achieves superior performance compared to baselines with remarkable cross-model and cross-environment transferability.
Read the full abstract ▾
All authors Baochang Ren, Yunzhi Yao, Rui Sun, Shuofei Qiao, Ningyu Zhang, Huajun Chen
Computer Vision
A Semantic Decoupling-Based Two-Stage Rainy-Day Attack for Revealing Weather Robustness Deficiencies in Vision-Language Models
Chengyin Hu, Xiang Chen, Zhe Jia et al. (+4 more)
Vision-
Language Models (VLMs) are trained on image-text pairs collected under canonical visual conditions and achieve strong performance on
multimodal tasks. However, their robustness to real-world weather conditions, and the stability of cross-modal semantic
alignment under such structured perturbations, remain insufficiently studied. In this paper, we focus on rainy scenarios and introduce the first
adversarial framework that exploits realistic weather to attack VLMs, using a two-stage, parameterized perturbation model based on semantic decoupling to analyze rain-induced shifts in decision-making. In Stage 1, we model the global effects of rainfall by applying a low-dimensional global modulation to condition the
embedding space and gradually weaken the original semantic decision boundaries. In Stage 2, we introduce structured rain variations by explicitly modeling multi-scale raindrop appearance and rainfall-induced illumination changes, and optimize the resulting non-differentiable weather space to induce stable semantic shifts. Operating in a non-pixel parameter space, our framework generates perturbations that are both physically grounded and interpretable. Experiments across multiple tasks show that even physically plausible, highly constrained weather perturbations can induce substantial semantic misalignment in mainstream VLMs, posing potential safety and reliability risks in real-world deployment. Ablations further confirm that illumination modeling and multi-scale raindrop structures are key drivers of these semantic shifts.
Read the full abstract ▾
All authors Chengyin Hu, Xiang Chen, Zhe Jia, Weiwen Shi, Fengyu Zhang, Jiujiang Guo, Yiwei Wei
Computer Vision
ConvMambaNet: A Hybrid CNN-Mamba State Space Architecture for Accurate and Real-Time EEG Seizure Detection
Md. Nishan Khan, Kazi Shahriar Sanjid, Md. Tanzim Hossain et al. (+3 more)
Epilepsy is a chronic neurological disorder marked by recurrent seizures that can severely impact quality of life. Electroencephalography (
EEG) remains the primary tool for monitoring neural activity and detecting seizures, yet automated analysis remains challenging due to the temporal complexity of
EEG signals. This study introduces ConvMambaNet, a hybrid deep learning model that integrates Convolutional Neural Networks (CNNs) with the Mamba Structured State Space Model (SSM) to enhance temporal feature extraction. By
embedding the Mamba-SSM block within a CNN framework, the model effectively captures both spatial and long-range temporal dynamics. Evaluated on the CHB-MIT Scalp
EEG dataset, ConvMambaNet achieved a 99% accuracy and demonstrated robust performance under severe class imbalance. These results underscore the model's potential for precise and efficient seizure detection, offering a viable path toward real-time, automated epilepsy monitoring in clinical environments.
Read the full abstract ▾
All authors Md. Nishan Khan, Kazi Shahriar Sanjid, Md. Tanzim Hossain, Asib Mostakim Fony, Istiak Ahmed, M. Monir Uddin