Computer Vision
SceneBench: A Hierarchical Benchmark for Vision-Language Understanding of 3D Scenes
Anubhav Khanal, Prabigya Acharya, Roshni Poudel et al. (+5 more)
Vision-
language models excel at 2D image understanding but remain limited in 3D spatial
reasoning. Progress is hindered by limitations in current
benchmarks. First, 3D
datasets often rely on point clouds that capture geometry but discard rich visual features like texture, text, and materials. Second, annotations treat objects in isolation while ignoring real-world hierarchical organization (scenes, rooms, functional areas, object groups). Third, evaluation tasks focus narrowly on basic recognition rather than multi-step spatial
reasoning. In this context, we introduce SceneBench, a
benchmark of 966 photorealistic 3D scenes reconstructed with Gaussian Splatting and densely annotated with hierarchical semantics spanning scenes, rooms, functional areas, object groups, and individual objects. These annotations are produced through a human-in-the-loop pipeline combining
vision-
language models with roughly 1,500 human-hours of iterative refinement and verification, producing over 183K annotated nodes with textual descriptions and 3D bounding boxes. Building on this representation, we define three evaluation tasks: Existence-Based Questions probing object attributes, Spatial Intelligence Questions covering counting, size comparison, distance, and directional relations, and Grounded Question-
Reasoning-Answer (QRA) triplets requiring multi-step
reasoning across semantic levels. Experiments with state-of-the-art
vision-
language models show that while models perform well on basic recognition tasks (e.g., up to 85% accuracy for detection), performance drops substantially on hierarchical and compositional
reasoning (e.g., down to 60% for counting), revealing limitations not captured by existing
benchmarks. SceneBench provides a realistic testbed for developing and evaluating models capable of fine-grained spatial
reasoning in photorealistic 3D environments.
Read the full abstract ▾
All authors Anubhav Khanal, Prabigya Acharya, Roshni Poudel, Sujan Kapali, Bigyan Bhatta, Pramish Paudel, Francois Rameau, Danda Pani Paudel
Computer Vision
Hyperbolic Contrastive Learning with Entailment for Spatial Transcriptomics
Daniela Vega, Paula Cárdenas, Hannah Ceballos et al. (+2 more)
Spatial Transcriptomics (ST) has transformed biomedical research by enabling the spatial mapping of gene expression across tissue sections. However, high operational costs, specialized equipment requirements, and sensitivity to experimental noise limit the accessibility and scalability of ST. Recent computer
vision approaches aim to overcome these limitations by predicting spatial gene expression directly from histopathology images. While effective, current approaches often suffer from gene expression over-smoothing and overly uniform predictions across tissue regions, suggesting that further progress depends on learning representations that reflect the hierarchical and asymmetric structure of gene regulation and tissue morphology. To address these issues, we propose
Hyperbolic Contrastive Learning with Entailment for Spatial Transcriptomics (HyCLoST), a
hyperbolic contrastive learning model that captures the intrinsic hierarchical relationships within ST data. By leveraging
hyperbolic geometry and a gene-to-image entailment loss, HyCLoST learns structured, biologically grounded representations that improve gene expression prediction accuracy, achieving a 6% reduction in MSE and an 8% increase in PCC across 26 ST
datasets, over previous methods. Our source code is publicly available at https://github.com/BCV-Uniandes/HyCLoST
Read the full abstract ▾
All authors Daniela Vega, Paula Cárdenas, Hannah Ceballos, Leonardo Manrique, Pablo Arbelaéz
Computer Vision · Cross-listed from eess.IV
A Sentinel-2 benchmark dataset for deep-learning active-fire segmentation across 25 California wildfires
Shreyan Mitra, Mohammadreza Narimani and Parastoo Farajpoor
This article describes an open image
dataset for developing and evaluating active-fire
segmentation methods in satellite imagery. The
dataset contains 2,148 image-mask pairs from 25 California wildfires, with acquisitions spanning July 2020 to August 2026. Each image is a 512x512-pixel, three-channel composite derived from Sentinel-2 Level-2A bands B12, B11 and B8A at 20 m spatial sampling. A fixed linear rendering is applied throughout the
dataset. Corresponding masks distinguish background, SWIR-rule active fire and invalid observations. The masks were generated from shortwave-infrared brightness and near-infrared contrast, followed by constrained neighborhood growth. The release includes chip-level metadata and an incident-disjoint partition containing 18 training, three validation and four test fires. Among the image pairs, 841 contain active-fire labels; these labels occupy 0.0766% of all grid
cells. A mask-blind analyst review covers 233 test chips and provides a separate assessment of the rule-generated labels at chip and connected-component levels. Reference training and evaluation code accompanies the data, including a ResNet-34 U-Net implementation with validation-based checkpoint and threshold selection. The archived images, masks, metadata and review annotations support research on rare-class
segmentation, learning from algorithmic labels and transfer across fire incidents. The versioned
dataset is deposited on Zenodo, with preparation and reuse software maintained in a public GitHub repository.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Occupancy Network-Guided Autonomous Robotic Partial Nephrectomy
Ethan Kilmer, Pit Henrich, Jiawei Ge et al. (+13 more)
Autonomous soft-tissue cancer surgery has been limited to interventions on organ surfaces, because current systems cannot perceive and adapt to anatomy once it deforms or is cut. We introduce the first
vision-guided autonomous system capable of performing complete tumor resections for partial nephrectomy. Our system integrates conditional occupancy networks, trained entirely in a physics-based simulation, that infer full 3-D anatomy (tumor, margin tissue, and kidney) from single-view partial point clouds. These occupancy networks maintain intraoperative tracking even as tissue is cut and deformed, enabling adaptive planning and execution. The
surgical platform combines a depth camera for capturing surface point clouds, dual robotic arms for electrosurgical cutting and
vacuum-based tissue manipulation, and an autonomous control strategy for tumor resection. In patient-derived hydrogel phantoms under an open partial nephrectomy setting, the
robot performed eight consecutive autonomous tumor resections comprising 77 electrosurgical cuts, with all cuts achieving negative
surgical margins and 1.61 $\pm$ 0.48 mm mean absolute margin error. This work demonstrates, for the first time, a foundation for supervised autonomous closed-loop, imaging-driven, margin-negative tumor removal in phantoms.
Read the full abstract ▾
All authors Ethan Kilmer, Pit Henrich, Jiawei Ge, Paul M. Scheikl, Laura Connolly, Soum D. Lokeshwar, Joseph Chen, Justin D. Opfermann, Kaitlyn Kumar, Lauren Shepard, Ahmed Ghazi, Nirmish Singla, Richard J. Cha, Kevin Cleary, Franziska Mathis-Ullrich, Axel Krieger
Computer Vision · Cross-listed from eess.IV
Multisource Remote Sensing and Geospatial Analysis of Vineyard Wildfire Impacts and Resilience: The 2019 Kincade Fire
Parastoo Farajpoor, Mahla Ardebili Pour, Mohammad Bagher Ghiasi et al. (+1 more)
Working agricultural landscapes are often treated as background to wildfire disasters, even though they are managed fuel mosaics, productive assets, and parts of regional infrastructure systems. We examine vineyard wildfire resilience during the electrically initiated 2019 Kincade Fire in Sonoma County, California, using an open, event-anchored geospatial framework spanning 4,581 vineyard fields (8,813.2 ha), wildland vegetation, surveyed structures, roads, overhead smoke, and post-fire greenness. Sentinel-2, OpenET, gridMET, soils, terrain, NOAA smoke polygons, an ignition-date OpenStreetMap network, and three-dimensional data inventories were analyzed at native decision scales. Vineyard pixels showed substantially lower descriptive dNBR than wildland pixels inside the perimeter (means 0.130 and 0.337). This contrast did not identify a universal vineyard firebreak effect: a segment-clustered boundary model gave a small negative contrast at 100 m (tau = -0.0166) but changed across bandwidths, failed slope continuity, disappeared in a 100 m donut, and produced a wrong-signed placebo. A 250 m spatial GAM reversed the unconditional pattern: after conditioning on location, terrain, and water use, vineyard fraction was positively associated with dNBR, while residual Moran's I remained 0.519. Beyond spectral impact, all mapped vineyards intersected overhead smoke on at least one day (mean 7.78 potential smoke-days per field), 34.2% of road-network nodes were dead ends, and inside-perimeter vineyards showed a larger greenness deficit through 2021 (recovery ratios 0.815 inside and 0.854 outside). Lower immediate spectral impact therefore did not imply complete resilience. The study provides a reproducible urban-rural informatics template separating descriptive contrasts, conditional associations, exposure indicators, and recovery evidence for decisions in working landscapes.
Read the full abstract ▾
All authors Parastoo Farajpoor, Mahla Ardebili Pour, Mohammad Bagher Ghiasi, Mohammadreza Narimani
Computer Vision
DenseFace: Bias Mitigation in Face Recognition via Density-Aware Probabilistic Matching
Mansur Bultygov, Vadim Seliutin, Dmitry Nekhaev et al. (+1 more)
Despite steady progress in face recognition, current face recognition models still suffer from significant demographic biases. While approaches for bias mitigation have been proposed, existing methods often impose constraints on the training procedure and result in the degradation of recognition accuracy. To address this issue, we here introduce a method that reduces racial bias in pre-trained face recognition models without compromising their accuracy. To this end, we model face
embeddings of each person by von Mises-Fisher (MF) distribution. We next observe the dependency between demographic attributes and the density of MF distributions, and propose DenseFace, a probabilistic face matching procedure that accounts for differences in MF distributions. Our extensive experiments demonstrate DenseFace to consistently reduce racial bias in strong face recognition models varying in network architectures, training
datasets and loss functions. Notably, DenseFace preserves recognition accuracy and requires no retraining of the underlying face recognition model. Our work also investigates previously adopted bias measures and makes suggestions.
Read the full abstract ▾
All authors Mansur Bultygov, Vadim Seliutin, Dmitry Nekhaev, Ivan Laptev
Computer Vision · Cross-listed from cs.LG
A Chosen Future Can Still Be Rewritten: Causal Writability in Video Models
Xingyun Wang, Haomin Zheng, Man Yuan et al. (+2 more)
When a
video model generates physically incorrect motion, did it fail to learn the correct motion, or did it learn it but fail to use it? We show the latter: the correct motion remains available inside the model and can still be made to control the generated
video. We train on
videos where red masses oscillate slowly and blue masses oscillate quickly, then test a red mass with fast observed motion. Even when the model generates slow motion in this conflicting case, a low-dimensional edit predicted from simple physical variables restores the correct fast motion. We call this ability causal writability. At fixed strength, we find a sharp depth boundary: the same edit changes the
video before the boundary but not after it. This closure marks commitment for that write. The motion signal nevertheless remains, and a stronger downstream write can restore physical motion, while excessive gain overshoots. Early causal writability predicts which errors training later corrects: those errors are writable at more network depths than errors that persist. We reproduce both causal writability and its sharp closure in a pretrained 1.3B
video model, supporting generality across model scale and training regime.
Read the full abstract ▾
All authors Xingyun Wang, Haomin Zheng, Man Yuan, Leqian Yang, Ziming Liu
Computer Vision
Anatomical Grounding and Leakage-Aware Multimodal Contrastive Learning for Alzheimer's Disease Classification from Structural MRI
Paul-Gabriel Nicolae and Irina Georgiana Mocanu
Deep networks trained on structural MRI for Alzheimer's disease (AD) staging often reach reasonable accuracy while attending to anatomically irrelevant regions, and
multimodal models that add clinical tables frequently rely on variables that were used to assign the diagnostic label in the first place. We study both issues with a deliberately lightweight slice-based encoder (ResNet18 with a one-layer
Transformer over slices) on 1,075 baseline T1-weighted scans from ADNI-1. First, we use FastSurfer
segmentations as an anatomical reference: YOLOv8 models trained on
segmentation-derived labels localize Alzheimer-relevant structures with mAP_50 above 0.96, and a Grad-CAM comparison shows that the image-only classifier frequently attends to the skull, orbits and background. Second, we adapt a CLIP-style image - tabular contrastive framework and organize ADNIMERGE variables along a label-leakage spectrum. Fusion with cognitive scores yields 87.3% three-way accuracy, which we treat as a leakage-driven upper bound rather than an imaging result; fusion with regional volumes yields 73.0%. We observe that the choice of contrastive target changes what the image encoder learns: on MCI vs. CN, the image-only head reaches 52.4% when the encoder is aligned to cognitive scores and 73.8% when aligned to volumes, although no tabular input is used at inference. Third, restricting the input to a per-subject crop of the medial temporal lobe raises image-only three-way accuracy from 58.7% to 65.1%. All results come from single runs on a small balanced test set, and we report confidence intervals and the protocol differences that prevent direct comparison with published numbers.
Read the full abstract ▾
Computer Vision
LynnReal-Omni: Native multi-modal Video Generation for Agentic Visual Workflows
Xiaofeng Mao, Peijia Lin, Shaohao Rui et al. (+3 more)
Video diffusion models are
stochastic and hard to control: precise content often requires repeated sampling without guaranteed success, and long-horizon scenes drift in appearance, interactions, and temporal coherence. Agentic visual creation provides explicit references, editable 3D scenes, or executable game states for stable control, but does not by itself guarantee high object or character fidelity. Combining the two can enable stable, high-quality generation. To realize this combination, we present LynnReal-Omni, a native
multimodal video generation framework built on a 32B shared
multimodal diffusion transformer that unifies text-to-
video, image-conditioned generation, reference-guided generation, structural control, editing, degraded
video restoration, and long-
video generation. It accepts heterogeneous visual inputs, including appearance references, editable 3D renders, and game recordings, allowing
agents to compose visual conditions within a unified model. We also train a dedicated 27B Flash shared
multimodal diffusion transformer for real-time rendering. We build a systematic data pipeline for
video cleaning, subject association,
multimodal annotation, and aligned control construction, yielding a curated corpus of multi-shot audiovisual segments, and introduce MSAVP, a 100-
prompt, 20-metric evaluation design that separates instruction following, generating plausibility, visual quality, temporal behavior, and audio coordination. LynnReal-Omni-Flash further reduces inference cost through model and decoding acceleration, including a lightweight VAE decoder; on one H100, warm generation and decoding of a 22-frame 540p
video take 843 ms with LynnReal-Omni and 377 ms with Flash. These results provide a foundation for real-time streaming
video generation, making LynnReal-Omni a unified, controllable, and efficient basis for agentic visual creation.
Read the full abstract ▾
All authors Xiaofeng Mao, Peijia Lin, Shaohao Rui, Yibo Zhang, Haibin Wan, Weijie Ma
Computer Vision · Cross-listed from cs.NI
Proportional-Fair Resource Allocation and Dual-Threshold Early-Exit Inference for Secure Cooperative Multi-Layer Edge Intelligence
Thai T. Vu, John Le, Tu N. Nguyen et al. (+3 more)
This paper proposes FREDI (Fair Resource Allocation for Edge Dual-Threshold Inference), a secure wireless edge-intelligence framework for event-triggered inference in a cooperative user equipment (UE)–edge server (ES)–cloud system. Each UE performs early-exit convolutional neural network (CNN) screening using dual confidence thresholds, while critical events are securely offloaded to an edge server for detailed classification. We formulate a proportionally-fair utility maximization problem that jointly optimizes UE–ES association, wireless and processing resources, and confidence thresholds. FREDI decomposes the problem into proportional-fair resource allocation and dual-threshold inference optimization. We prove that the detected-critical event set is set-monotone non-increasing in both thresholds, and exploit the finite empirical confidence domain for exact threshold optimization. An empirical resource–utility response envelope yields a computable global suboptimality bound and a sufficient condition for global optimality. By pre-eliminating infeasible UE–ES pairs and exactly projecting out bandwidth and transmit-power variables, the resource-allocation subproblem is reduced to a mixed-integer exponential-cone program solvable to the certified global optimality within a prescribed gap. Numerical results with early-exit MobileNetV2 and ShuffleNetV2 demonstrate near-perfect UE
fairness with aggregate utility close to a Sum-Utility
benchmark, reveal security-induced resource fragmentation, and demonstrate the Stage-A scalability from 6 to 144 UEs with median solving time below 0.1~s in the tested configurations.
Read the full abstract ▾
All authors Thai T. Vu, John Le, Tu N. Nguyen, Jun Shen, Quang Vinh Duong, Ha Nguyen
Computer Vision
TRACE: Two-Stage Detector-Response Estimation With Angular Cosine Expansion for Ring Artifact Correction in Photon-Counting CT
Jigang Duan, Heran Wang, Ligen Shi et al. (+3 more)
Detector response nonuniformity introduces systematic projection errors and ring artifacts in
photon-counting detector computed
tomography (PCD-CT). In measured PCD-CT data, residual stripe amplitudes vary slowly with projection angle, which fixed-bias models cannot adequately capture. We propose TRACE, a two-stage unsupervised sinogram decomposition method for estimating and correcting these response-related errors. TRACE represents stripes as a fixed bias plus low-order discrete cosine transform (DCT) components, using a small number of coefficients to describe angular variations at each detector element. A learnable analysis–synthesis architecture represents the ideal projections, while two-stage optimization separates them from fixed and then dynamic stripes. An angular-gradient soft orthogonality constraint suppresses correlated variations within the shared DCT gradient subspace, reducing the leakage of object structures into the artifact estimate. All parameters are optimized directly on the measured sinogram without paired training data. Experiments on measured QRM mouse phantom and porcine trotter data show that TRACE suppresses ring artifacts and improves image uniformity while preserving edge sharpness, soft-tissue texture, and trabecular detail.
Read the full abstract ▾
All authors Jigang Duan, Heran Wang, Ligen Shi, Zheng Sun, Ping Yang, Xing Zhao
Computer Vision
Integrating Multi-view Multi-light Surface Reconstruction into Cultural Heritage Workflows
Baptiste Brument, Robin Bruneau, Benjamin Coupry et al. (+6 more)
Cultural heritage documentation increasingly relies on image-based 3D surface reconstruction, with photogrammetry software making such workflows accessible to archaeologists, conservators, and heritage technicians. These tools have been successful for conventional multi-view acquisition, but they do not routinely exploit richer multi-view, multi-light data, despite its potential for improving fine-scale surface reconstruction. This limitation is particularly relevant in heritage contexts, where controlled-light acquisition devices such as RTI domes are already used to capture illumination-varying image sets. The challenge is therefore to connect these existing acquisition practices with recent computer
vision methods in a form that can be used within operational heritage workflows. In this work, we address this need by integrating state-of-the-art components from computer
vision for multi-view, multi-light surface reconstruction into Meshroom, an open-source photogrammetry framework. Rather than proposing a new reconstruction
algorithm, our contribution is to assemble and expose existing advanced methods, namely a complete photometric stereo ecosystem (calibrated, self-calibrated and universal), automatic object masking, and multi-view normal-and-reflectance integration, within a usable heritage-oriented workflow. The proposed system thus provides an intermediate software layer between computer
vision research code and practical cultural heritage applications, making recent techniques easier to use and evaluate.
Read the full abstract ▾
All authors Baptiste Brument, Robin Bruneau, Benjamin Coupry, Vincent Demoulin, Jean Mélou, Antoine Laurent, Fabien Castan, Jean-Denis Durou, Lilian Calvet
Computer Vision
VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention
Xingyang Li, Dongyun Zou, Shining Zhang et al. (+8 more)
Diffusion Transformers deliver state-of-the-art
video generation, but their long spatiotemporal sequences make
attention the dominant deployment cost, and a deployable low-bit kernel must be accurate and fast. Accuracy is limited by outliers: a block's quantization scale is set by its largest entries, leaving typical entries confined to a narrow range of representable values. Prior work smooths queries and keys, but value outliers follow no fixed channel or spatiotemporal structure and remain the dominant source of output error. Speed is limited by softmax: low-bit Tensor Cores accelerate only the two
matrix multiplications, so the high-precision exponential between them becomes the longest pipeline stage on datacenter GPUs. We propose VC-
Attention, a training-free low-bit
attention framework that addresses both by pairing Value smoothing with a fused probability Cast. V-Smooth reorders value
tokens by lightweight online clustering, so the
tokens in a hardware block quantize well together. It quantizes only the residual after subtracting the block mean, and restores that mean from the row sum the online softmax already maintains. ExpCast-FP8 maps log-domain scores directly to E4M3 probability codes with one fused multiply-add, eliminating the FP32 exponential and the format conversion. We implement VC-
Attention for B200, B300, H200, RTX PRO 6000, and RTX 5090. Across Wan2.2, LongCat-
Video, HunyuanVideo-1.5, and MiniMax-H3, VC-
Attention improves fidelity over low-bit baselines, speeds up the
attention kernel over BF16 FlashAttention-4 by 1.46-1.59x on datacenter Blackwell and Hopper and by 2.3-3.6x on workstation cards, and generates a clip 1.13-1.19x and 1.36-1.70x faster end to end.
Read the full abstract ▾
All authors Xingyang Li, Dongyun Zou, Shining Zhang, Jiacheng Chen, Haocheng Xi, Lvmin Zhang, Jun-Yan Zhu, Song Han, Zhekai Zhang, Yujun Lin, Muyang Li
Computer Vision
SURE-Map: Self-Correcting Streaming Geometric Foundation Models
Mingkai Liu, Hao Zhao and Xingxing Zuo
Streaming
geometric foundation models are emerging as a compelling alternative to SLAM systems. Yet this streaming nature introduces a fundamental issue: each prediction is made from limited context, which is vulnerable to dynamic objects and weak textures. Small local errors accumulate into severe
geometric distortion and long-horizon scale drift. We argue that reliable streaming reconstruction requires
geometric foundation models to be not only predictive, but also self-correcting. We introduce SURE-Map, a self-correcting framework built upon two complementary principles. First, we explicitly model cross-view
geometric uncertainty. Unlike conventional depth or point confidence, which primarily reflects the reliability of individual-view prediction, our uncertainty directly measures whether the jointly predicted pose and depth induce geometrically consistent cross-view pixel correspondences. Second, because local correction alone cannot eliminate slowly accumulating scale errors, we introduce multi-timescale self-correction: fast consecutive-frame inference preserves streaming efficiency, while sparse keyframe-window inference provides longer-range
geometric evidence to periodically recalibrate the scale of recent trajectories. SURE-Map establishes new state-of-the-art performance for online feed-forward reconstruction across long-horizon
benchmarks, reducing ATE-RMSE from 24.00 to 17.24 m on KITTI, 5.11 to 4.74 m on Oxford Spires, and 31.37 to 28.58 m on VBR, with further improvements to 15.17, 4.63, and 22.12 m when incorporating loop-closure refinement. Project page: https://mingkai-liu.github.io/projects/sure-map/.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
TopoRig: Topology-Agnostic Facial Rigging via Multi-Source Supervision
Andrew Fleet, Soroush Mehraban, Vida Adeli et al. (+2 more)
Automatic facial rigging across heterogeneous mesh
topologies remains challenging because high-quality expression supervision is often tied to canonical templates, while deformation transfer to arbitrary meshes can introduce
geometric artifacts and correspondence errors. We present TopoRig, a
topology-agnostic facial rigging framework that predicts FACS-conditioned deformations directly on input mesh vertices while preserving the original
topology. Starting from the ICT FaceKit expression model, we construct complementary supervision from accurate but template-biased common-
topology rigs,
topology-diverse but noisier transferred rigs, and targeted image-based cues for controls poorly captured by
geometric transfer. TopoRig combines local surface geometry, landmark-relative semantic features, global shape context, and FACS controls to predict per-vertex displacements. We train on 3,496 generated identities using 45 non-gaze expression controls from the 53-control ICT FaceKit vocabulary. On held-out identities and unseen mesh
topologies, TopoRig more faithfully reproduces the reference expression space than prior neural facial-rigging methods, while qualitative results show consistent localized deformations across diverse character geometries. Ablations demonstrate that semantic landmark features and complementary supervision improve cross-identity and cross-
topology generalization. Overall, TopoRig amortizes heterogeneous and imperfect expression supervision into a single
topology-preserving deformation model.
Read the full abstract ▾
All authors Andrew Fleet, Soroush Mehraban, Vida Adeli, Cole Clifford, Babak Taati
Computer Vision · Cross-listed from eess.SP
Transforming harmonic coefficients for 3D splat compression
Tam Thuc Do, Philip A. Chou and Gene Cheung
We address the problem of color attribute compression for 3D splats. We show that all images generated by 3D splats are linear in the coefficients for each color channel, each spherical harmonic, and each splat, and we identify a basis for the space of all such images. We identify an inner product for the coefficient space that induces the squared error loss on images. We show that orthonormalizing the coefficients with respect to this innner product before coding can yield over 2 dB gain.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Bench2Dex: Benchmarking Visuo-Tactile Bimanual Dexterous Manipulation Across Dexterous Hands
Zhenjie Yang, Yideng Zhang, Dongjie Zhang et al. (+21 more)
Tactile sensing provides contact information that can be difficult to infer from
vision alone, but tactile hardware for dexterous hands has not converged to a common design. Dexterous hands differ in finger structure, contact surfaces, and sensor layouts, while simulated tactile signals still differ from measurements produced by physical sensors. These factors make it difficult to study visuo-tactile manipulation across diverse dexterous hands within a consistent experimental setting. We present Bench2Dex, a simulation
benchmark for visuo-tactile bimanual manipulation across 12 dexterous hands. We adapt existing
robot models with a shared simulated tactile interface that converts local contact geometry into image-like tactile observations. The interface provides a consistent observation format across different hand morphologies without attempting to reproduce the output of a specific physical tactile sensor. Bench2Dex includes 26 bimanual manipulation tasks that involve tool use, articulated-object interaction, and multi-stage manipulation, together with about 1.3K human-teleoperated demonstrations. The
benchmark provides synchronized visual, tactile, proprioceptive, action, and object-state observations, together with executable task metrics. For robustness, we group seven perturbation types into invariance axis, where the correct action does not change, and equivariance axis, where the correct action changes together with the perturbation. We evaluate ACT,
Diffusion Policy, pi0.5, and GR00T N1.5 on Bench2Dex and report their performance and failure modes. Bench2Dex is meant as a platform for studying visuo-tactile learning across dexterous hands. It does not assume that simulated tactile observations can replace real tactile sensing; it offers a shared setting for
algorithm development while tactile hardware and simulation models are still evolving.
Read the full abstract ▾
All authors Zhenjie Yang, Yideng Zhang, Dongjie Zhang, Chenyu Jiang, Xianshuai Liu, Yufeng Li, Zuhao Ge, Xingyu Jiao, Zheng Zhang, Kaiyu He, He Wang, Yuwen Zhong, Yi Deng, Muyun Jiang, Xianliang Huang, Haisheng Su, Donghang Zhang, Jian Zhang, Xue Yang, Hongyang Li, Zuxuan Wu, Yu-Gang Jiang, Xiaosong Jia, Junchi Yan
Computer Vision · Cross-listed from cs.AI
Predicting build orientation for SLM dental parts: a comparison of rotation representations and direct vector regression
Felix Schmalzel, Reimar Waitz, Moritz Kronberger et al. (+1 more)
Build orientation for selective
laser melting (SLM) manufacturing of dental parts is usually chosen manually by technicians. We treat orientation prediction as supervised machine learning of the part's up-axis from technician-labeled production data, and test which rotation representations produce the best results. Using $n\approx2400$ patient-specific dental parts, we trained a ResNet-50 multi-view image backbone and a PointNeXt-S point-cloud backbone, both pretrained and fine-tuned end-to-end, on 13 up-axis representations spanning six classical $SO(3)$ parameterizations and seven representations defined directly on the unit sphere $S^2$. We report the geodesic angular error between predicted and ground-truth up-axis on a test set, with and without test-time augmentation (TTA) over $K=21$ known rotations. With TTA, the octahedral map achieves the lowest mean angular error ($10.6^\circ$, ResNet-50). The three lowest-error results overall are direct $S^2$ representations, though this may reflect label noise in the unsupervised in-plane component of the $SO(3)$ targets rather than a topological advantage. von Mises-Fisher collapses to a near-constant prediction when trained with PointNeXt-S but not with ResNet-50. TTA reduces mean angular error by 31-73 % across almost every representation and backbone. Overall, test-time augmentation over a small set of known rotations is the most consistent driver of accuracy, whereas the best-performing representation is strongly backbone-dependent.
Read the full abstract ▾
All authors Felix Schmalzel, Reimar Waitz, Moritz Kronberger, Thorsten Schöler
Computer Vision · Cross-listed from cs.HC
Can a Neural Encoding Model Replicate an fMRI Visualization Study?
Erfan Nasirzadeh Orang and Zack While
Most knowledge of graphical perception comes from behavioral studies. Understanding from a neural perspective is much more limited due in part to neuroimaging studies' expensiveness and difficulty to conduct. In this paper, we evaluate whether Meta's Tribe V2 neural encoding model can recover neural contrasts from a visualization fMRI study. Specifically, we evaluate Tribe V2 through a conceptual replication of the visualization-viewing component of a prior comparison of Bubble charts and three-dimensional Surface charts in color and grayscale. We generate TRIBE-predicted cortical responses for the original stimuli and compare the resulting contrasts with those reported in the human study. The model reproduced the direction of 11 of 14 reported cortical effects, with agreement concentrated in visual-processing regions. This agreement characterizes the model's
alignment with the prior human-generated fMRI results rather than independently confirming them. We discuss the limitations encountered when working with this model for in-silico replication and hope to encourage future work exploring this new avenue for neuroimaging studies in visualization. Supplemental materials are available at https://osf.io/8a96x/.
Read the full abstract ▾
Computer Vision
V-ICAL Bench: Evaluating Video In-Context Learning for Multimodal Agents in Interactive Environments
Ziqian Fan, Shibo Xu, Junjie Li et al. (+8 more)
While In-Context Learning (ICL) enables models to adapt from exemplars without parameter updates,
multimodal ICL remains largely underexplored, particularly regarding
video demonstrations in interactive environments. For
multimodal agents, learning from
videos presents unique challenges: they must translate in-context demonstrations into executable policies, ground these policies in novel visual states, and iteratively refine actions based on environmental feedback. We introduce V-ICAL, a novel
benchmark designed to evaluate
video-based ICL for
multimodal agents. Comprising 342 interactive tasks across 37 environments, V-ICAL utilizes human-curated demonstration
videos as task-specific behavioral exemplars, evaluating
agents through sustained interaction from a target initialization. The
benchmark seamlessly connects in-context knowledge induction with core agentic capabilities, including state grounding, temporal
memory, planning, and adaptation in dynamic environments. Extensive evaluations across 19 state-of-the-art
multimodal agents reveal significant limitations: the best-performing model, Seed-2.1-Pro, achieves a score of only 54.4/100, while other leading models (e.g., Gemini-3.1-Pro, GPT-5.6) fail to surpass 50, far below the human baseline of 83.6. Controlled comparisons further demonstrate that current
agents struggle to reliably translate
video exemplars into effective policies, failing to yield consistent performance gains. Ultimately, V-ICAL exposes a critical gap in the ICL capabilities of
multimodal agents, underscoring an urgent need for future research.
Read the full abstract ▾
All authors Ziqian Fan, Shibo Xu, Junjie Li, Xiangyu Zhao, Shengyuan Ding, Yifan Yang, Zhenjie Yang, Haodong Duan, Yue Zhou, Zhihang Zhong, Xue Yang
Computer Vision
MambaMPD: A Mamba-Driven Segmentation Framework for Marine Pollution Detection from Remote Sensing Imagery
Shuaiyu Chen, Wei Han, Peng Ren et al. (+2 more)
Accurate marine pollution detection (MPD) is essential for protecting coastal ecosystems and marine biodiversity.
Vision Mamba models have shown promise in remote-sensing semantic
segmentation by efficiently capturing long-range dependencies and global context, yet their potential for MPD remains underexplored. MPD is particularly challenging because of low signal-to-noise ratios, fragmented pollution patterns, and indistinct boundaries caused by the visual similarity between pollutants and the surrounding sea. To address these issues, we propose MambaMPD, an enhanced Mamba-based framework incorporating two complementary structural priors: Frequency-Aware Augmentation (FAA) and multi-scale Edge-Guided
Attention (EGA). FAA integrates wavelet transforms into the encoder to decompose features into multi-scale frequency subbands, enabling the model to capture low-frequency contextual semantics and high-frequency structural details needed to identify small, low-contrast, and irregular pollution patterns. EGA adaptively fuses hierarchical,
Laplacian-derived boundary cues with deep semantic representations, refining encoder features before decoding to sharpen boundaries and reduce ambiguity in visually confusing, spatially fragmented scenes. Together, these modules improve sensitivity to subtle pollution signals while preserving fine boundary structures. A U-Net-style decoder with squeeze-and-excitation
attention and deep supervision progressively restores and refines semantic and spatial information across scales. Extensive experiments on two
benchmark MPD
datasets show that MambaMPD achieves higher mIoU than competing methods while requiring substantially less computation than foundation-model-based approaches. On MADOS, it improves F1 by 3.6% over OSDMamba; on M4D, it raises Oil Spill IoU by 6.82% over TransOilSeg.
Read the full abstract ▾
All authors Shuaiyu Chen, Wei Han, Peng Ren, Chunbo Luo, Zeyu Fu
Computer Vision
Don't Send What You Don't Need: Question-Guided Token Pruning as a Privacy Defense for Vision-Language Models
Md Khalid Syfullah and Alvi Ataur Khalil
Visual Question Answering (VQA) with
Vision-
Language Models (VLMs) is increasingly used in
privacy-sensitive and bandwidth-constrained settings.
Federated Learning (FL), Split Learning (SL), and U-Shaped Split Learning (USL) keep raw data local, but transmitting all visual
tokens across a model partition remains costly and can expose private information. We propose QPriv-VL, a question-guided,
privacy-aware
token-
pruning framework for FL, SL, and USL that prunes visual
tokens before transmission based on task utility and
privacy sensitivity. Its core component is a lightweight Dynamic Threshold Predictor (DTP) that jointly estimates a sample-specific
pruning ratio and a
token-level retention mask in one forward pass. DTP combines question relevance, computed from cross-modal similarity between visual patches and the pooled question
embedding, with a sensitivity signal derived from frozen DINOv2 features. This allows the model to suppress potentially sensitive regions while preserving patches useful for answering the question, without requiring sensitivity labels. We evaluate QPriv-VL on GQA, OK-VQA, VQAv2, SLAKE, VQA-RAD, and PathVQA against four
privacy attack families: FSHA, FORA, iDLG, and attribute-inference membership inference attacks. DTP matches or outperforms fixed-ratio
pruning while using substantially fewer transmitted
tokens. On VQA-RAD, it reduces membership-inference attack success from 0.99 to 0.76-0.79, lowers FSHA and FORA reconstruction PSNR relative to fixed-ratio
pruning, and preserves competitive VQA accuracy using about 40% of the original visual-
token budget. A sensitivity exclusion ratio of 1.20 +/- 0.18 indicates preferential removal of
privacy-sensitive patches, while explainability analysis shows that retention adapts to question semantics rather than generic visual saliency.
Read the full abstract ▾
Computer Vision
Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration
Matteo Barbieri, Giammarco La Barbera, Juan Pablo De La Plata et al. (+3 more)
Multimodal image registration is a key component of many clinical workflows, yet it remains challenging because corresponding anatomical structures often exhibit substantially different image intensities across modalities. In this work, we present a comprehensive
benchmark of intra-patient 3D
multimodal deformable registration methods across three
datasets covering different anatomical regions and difficulty levels, including both synthetic deformation recovery and real clinical scenarios. We evaluate classical optimization-based approaches and modern learning-based methods, including recent deep learning and foundation models, using complementary metrics: Average Dice similarity coefficient (DSC), average 95th-percentile Hausdorff distance (HD95), and a modality-independent structural similarity measure based on the MIND self-similarity context (MIND-SSC). Results show high variability across
datasets, with learning-based methods demonstrating superior performance on large synthetic
benchmarks, while only limited improvements are observed in real pelvic registration. A key finding of this study is the consistent disagreement between
geometric metrics (DSC, HD95) and image-based similarity metrics (MIND-SSC), highlighting that improved overlap does not necessarily imply better global
multimodal correspondence. Furthermore, anatomy-guided approaches achieve the highest overlap scores but exhibit degraded performance outside of segmented regions, revealing a trade-off between label-driven
alignment and global structural coherence. Overall, our results indicate that no current method achieves robust performance across anatomies and modalities. We demonstrate that intra-patient 3D
multimodal registration requires multi-criteria evaluation, including deformation-based metrics, and remains an open problem.
Read the full abstract ▾
All authors Matteo Barbieri, Giammarco La Barbera, Juan Pablo De La Plata, Sabine Sarnacki, Isabelle Bloch, Pietro Gori
Computer Vision
Circuit-MLLM: Topological Logic-Guided Latent-Space Visual Reasoning for Circuit Schematic Understanding
Jinyuan Deng, Yuqi Jiang, Wenjing Huang et al. (+3 more)
Through pre-training on extensive text and image
datasets, current multi-modal large
language models (MLLMs) achieve strong performance on general tasks. However,
circuit schematics present a unique challenge for MLLMs due to their dense component layouts and distinct topological logic, demanding fine-grained structural parsing to extract the electrical semantics. To address this, we propose
Circuit-MLLM, a
multimodal reasoning framework that reformulates
circuit topology analysis as a process of device localization, path tracing, and sequential
reasoning within the latent space. We introduce a
circuit knowledge mining mechanism that deeply aligns the model's latent representations with structurally rich features derived from multi-granularity
circuit vision experts, enabling the model to effectively internalize topological semantics. Building upon these internalized semantics, we devise a
topology-guided sequencing strategy that decouples
reasoning from the rigid raster-scan order, enforcing stepwise inference along the
circuit's topological logic in latent space. Across diverse
circuit analysis tasks,
Circuit-MLLM consistently outperforms strong baselines, notably achieving a 25% higher average score than GPT-5.1, which demonstrates the effectiveness of our framework in
circuit schematic
topology analysis. Code is publicly available at https://github.com/IC-Yuan/
Circuit-MLLM.
Read the full abstract ▾
All authors Jinyuan Deng, Yuqi Jiang, Wenjing Huang, Xin Li, Qi Sun, Cheng Zhuo
Computer Vision · Cross-listed from cs.GR
KaiNinja: Extending Native 3D Generators to the Part Level
Ruihan Yu, Lian Fu, Muyao Niu et al. (+9 more)
Native 3D generators turn one image into a single mesh. TRELLIS.2 and its peers deliver high-fidelity non-watertight geometry with materials, but the output is one fused object, while downstream work such as editing, rigging and simulation operates on part-level assets. A naive idea is to run a 3D
segmentation network on the fused mesh that TRELLIS.2 generates, but such pipelines are slow and bounded by the accuracy of the
segmentation. We want a simple way to extend an existing native 3D generator to the part level. But we face a critical problem: the O-Voxel grid stores one sheet of surface per voxel, so a single volume cannot represent the interface where two parts touch, at any resolution. We introduce a dual-volume representation to solve this problem and put forward KaiNinja, a part-level extension of TRELLIS.2 built on a dual-volume form of its O-Voxel representation. KaiNinja keeps the generation speed and quality of TRELLIS.2 while extending it to the part level, with no mask or segmenter in the pipeline. Its training data come from sources of many kinds, including CAD models and assets authored by an LLM-driven
agent; to our knowledge it is the first 3D generative model trained on
agent-authored part data. Surprisingly, we also find that whole-object fidelity improves over the same backbone fine-tuned on the same
dataset. Against part generation pipelines of different paradigms, it lowers whole-object Chamfer distance by 40% and raises strict part F-score by 16%.
Read the full abstract ▾
All authors Ruihan Yu, Lian Fu, Muyao Niu, Zheng-hui Huang, Yu-Ju Tsai, Sho Kuno, Fengbo Lan, Yonghao Yu, Erwin Wu, Ming-Hsuan Yang, Kaipeng Zhang, Zhixiang Wang
Computer Vision
From Model Patterns to Abstract Semantics in Compositional Zero-Shot Learning
Weize Li, Zhicheng Zhao and Fei Su
Compositional Zero Shot Learning aims to recognize unseen compositions by recombining learned primitives. Recent methods rely on
vision language models and attempt to explicitly model contextual variations of primitives through multiple representations. However, such approaches are limited by fixed variant capacity and competition between abstract and concrete semantics. In this work, we present a new perspective that views primitive variations as the context-driven activation of concrete visual cues rather than independent entities. Based on it, we propose CLEAR, a CLoze-style
rEAsoning-based Re-ranking framework inspired by human perceptual processes. CLEAR extracts conditional variants from the primitive candidate set in a coarse-to-fine manner, performs cloze-style
reasoning to infer high-level semantics, and re-ranks predictions to correct biases toward salient concrete primitives. Extensive experiments demonstrate that CLEAR consistently improves the Base Model and outperforms state-of-the-art methods on the challenging C-GQA and MIT-States
datasets. Code is available at https://github.com/buptLwz/CLEAR.
Read the full abstract ▾
Computer Vision
MedDiME: Efficient Latent Diffusion with Adaptive Masking for Medical Counterfactual Generation
Yan Zeng, Changlu Guo, Anders Nymark Christensen et al. (+2 more)
Medical counterfactual generation modifies images to change model predictions for interpretability. However, existing
diffusion-based approaches are often prohibitively slow and
memory-intensive, making them difficult to apply in high-resolution settings. Moreover, existing masking strategies are tightly coupled with pixel-space representations, making them incompatible with latent-space
diffusion editing. To address these challenges, we propose MedDiME, a latent-space classifier-guided
diffusion framework that reduces computational and
memory overhead while introducing a latent-compatible, gradient-driven adaptive masking mechanism for spatially precise medical counterfactual generation. Extensive experiments demonstrate that MedDiME achieves high-quality counterfactual generation with significant efficiency gains compared to prior classifier-guided
diffusion baselines, achieving up to 40 times faster inference and 13 times lower peak GPU
memory usage.
Read the full abstract ▾
All authors Yan Zeng, Changlu Guo, Anders Nymark Christensen, Morten Rieger Hannemose, Anders Bjorholm Dahl
Computer Vision
Human-Grounded Calibration for Long-Text Image-Text Congruence in Vision-Language Models
Alessandro Gambetti and Qiwei Han
Long-text image–text congruence scoring is increasingly important for
vision-
language systems that must evaluate whether detailed textual descriptions match visual content. However, raw similarity scores from dual-encoder models are difficult to interpret as calibrated congruence measures, especially under the modality gap between image and text
embeddings. This paper proposes Congruency Score (CS), a lightweight calibration layer that maps image–text similarity evidence into a bounded score. Using DOCCI and Urban1k, we evaluate four frozen
vision-
language backbones and show that observed reductions in post-projection centroid distance do not uniformly improve image–text
retrieval performance. Human-grounded evaluations on DOCCI further reveal a trade-off: direct post-hoc calibration preserves high association with human judgments, whereas selected projection-based configurations can reduce threshold-relevant slope and intercept distortions at the cost of
retrieval performance and association strength. These results establish long-text image–text congruence scoring as a calibrated score-estimation problem, where
retrieval performance, human association, and threshold calibration must be evaluated as distinct objectives. CS provides a lightweight way to expose and operationalize this separation.
Read the full abstract ▾
Computer Vision
SAM3D-Part: Interactive Part Selection and Generation from 3D Objects
Jiahao Chang, Dong Du, Wanhu Sun et al. (+6 more)
Part-level control is essential for modern 3D asset creation, where objects are frequently edited, reused, animated, or fabricated through their individual components. In many such workflows, users need only several specific components rather than a complete object decomposition. However, existing 3D generation methods produce all parts regardless of user intent, while promptable 3D
segmentation methods typically output partial surfaces instead of reusable complete meshes. In addition, image-conditioned part generators further struggle to preserve hidden geometry and accurate placement without directly conditioning on the source mesh. To address these problems, we present SAM3D-Part, a
prompt-driven framework for selective part generation from input 3D object meshes. Given a source mesh and a part
prompt, SAM3D-Part first encodes the source geometry into compact mesh features and aligns them with the rendered image, selective mask, and point-map observations via pixel-wise channel fusion. The fused representation conditions a feed-forward generative model to produce only the queried component as a completed mesh. To place the generated part back into the source coordinate frame, SAM3D-Part predicts dense per-voxel correspondences and estimates the part transformation from distributed spatial evidence rather than a single global pose code. For sequential multi-part queries, previously generated parts are stored in a part cache and reused as contextual constraints, reducing conflicts among independently requested components. Extensive experiments and ablations demonstrate that SAM3D-Part can significantly improve source
alignment, reduce conditioning cost, and enable consistent selective part generation, achieving state-of-the-art. Code and weights will be available at https://github.com/Jiahao620/sam3d-part.
Read the full abstract ▾
All authors Jiahao Chang, Dong Du, Wanhu Sun, Yujian Zheng, Chuanyu Pan, Bowen Zhao, Chongjie Ye, Yuanming Hu, Xiaoguang Han
Computer Vision
ModaLens: Measuring Image Sensitivity in Report-Conditioned Medical VLMs
Sebastián Andrés Cajas Ordóñez, Maximin Lange, Quang Bui et al. (+9 more)
A radiology report can already answer a clinical question, so it is hard to tell whether a
vision-
language model also uses the image. ModaLens, a paired image-swap audit, measures how report availability changes image sensitivity: MedGemma-27B on 3,199 paired MIMIC-CXR cases from 293 patients, all 14 questions per case (13 finding-specific and one composite), each image replaced by one from another study, usually of the same patient, with question and report fixed. Under an explicit answer instruction, the model's generated answer changes on 4.26 percent of trials with the report and 20.94 percent without it, a paired increase of 16.7 points (patient-clustered 95 percent CI 15.6 to 17.7), so report availability reduces image-swap sensitivity under this protocol; the original
prompt with a lowercase first-
token readout gives 4.70 percent against 17.07 percent, and substitutions also move continuous answer scores where the binary prediction does not change. The labels are derived from reports, which limits conclusions about visual correctness; the direction replicates in two further model lineages. Code, the exact
prompts and a run record for every number are at https://github.com/criticaldata/MODALENS.
Read the full abstract ▾
All authors Sebastián Andrés Cajas Ordóñez, Maximin Lange, Quang Bui, Anqi Peter Li, Felipe Ocampo Osorio, Rafi Al Attrach, Kushul Reddy Palakala, Sahil Kapadia, Zakaria Laouabdia Sellami, Xinyue Zhang, Ashley Zhang, Leo Anthony Celi