Computer Vision · Cross-listed from q-bio.BM
Reconstruction-Aware Cryo-EM Particle Picking
Riku Itsuji, Yuanhao Wang, Xingjian Li et al. (+3 more)
Cryo-electron microscopy (cryo-EM) determines the structures of
proteins and macromolecular assemblies at near-atomic resolution, and the final 3D reconstruction depends on extracting a clean particle stack from noisy micrographs. This extraction decomposes into three sub-tasks, namely particle picking, contamination removal, and 2D class selection. Each of them, however, is trained and evaluated in isolation, and none is optimized for the reconstruction. We instead integrate the three sub-tasks into a single pipeline posed against downstream reconstruction quality. We instantiate the pipeline with a state-of-the-art component for each sub-task, CryoTransformer picking permissively, MicrographCleaner masking contamination, and CryoSift selecting 2D classes by a continuous quality score, and close the loop with a fine-tuning step that returns the surviving particles to the picker. The pipeline achieves a better 3D resolution than every picker we compare. We also show that the best 2D F1 is not the best resolution, so particle selection is better treated as one reconstruction-aware pipeline judged by the map it delivers.
Read the full abstract ▾
All authors Riku Itsuji, Yuanhao Wang, Xingjian Li, Seonghui Min, Hideo Saito, Min Xu
Computer Vision
BlobBoards: Robust Markers for Accurate Pose
James Pritts, Till Sittart, Hendrik Sauer et al. (+4 more)
We propose BlobBoards, a fiducial marker system comprising a dense, multi-scale field of Gaussian blobs and a feature-based pipeline for joint detection, identification, and pose estimation. Each board is registered from hundreds of blob features whose dense spatial coverage constrains pose, while multiple scales preserve detectability across large changes in focal length, distance, and obliquity. Learned local descriptors are matched to the reference pattern and spatially verified, so the correspondences determine pose and certify identity. Against motion-capture ground truth, BlobBoards achieve median translation errors of 3.6-5.0 mm, reducing AprilTag's median translation error by 89% on small boards and 70% on large ones. They also produce far fewer large-rotation failures than state-of-the-art tag systems. BlobBoards achieve the highest detection rate, 80% versus 74% for AprilTag and 58% for ArUco, with the largest margin on the smallest markers. Under 50% occlusion, they still detect 69% of boards with essentially unchanged median translation error, while AprilTag and ArUco detect none. In experiments BlobBoards give state-of-the-art detection rate, pose accuracy and occlusion robustness.
Read the full abstract ▾
All authors James Pritts, Till Sittart, Hendrik Sauer, Silja Janßen, Felix Seegräber, David Nakath, Kevin Köser
Computer Vision
Text-Driven Artistic Staging: 3D Posing, Lighting, and Camera References from Paintings
Yunge Wen
Artists coordinate human pose, illumination, and camera placement to convey narrative and
emotion, but existing generative methods typically model these elements independently. We introduce text-to-editable 3D staging, a task that jointly generates human poses, a dominant light, and a camera configuration from an affective description. We construct 11,911 text–staging pairs from 2,328 figurative paintings by reconstructing SMPL bodies, estimating low-frequency illumination, recovering camera parameters, and pairing each scene with ArtEmis descriptions. We train a flow-matching
transformer that supports variable numbers of figures and produces multiple staging alternatives for each
prompt. On held-out descriptions, the model achieves 32.2%
retrieval R@1, compared with 16.6% for CLIP-based nearest-neighbor
retrieval, while approximately preserving corpus-level diversity. These results demonstrate the feasibility of generating editable, emotionally conditioned 3D staging references from text.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Explainable Artificial Intelligence (XAI) in Computational Pathology: Definitions, Taxonomy, and Recommendations
Shubham Innani, Suhang You, Adam Shephard et al. (+13 more)
Computational pathology (CompPath) is transforming medicine by leveraging artificial intelligence (AI)
algorithms to support diagnosis, prognosis, and treatment prediction from gigapixel whole-slide images. Clinical adoption is progressing, but is constrained by concerns about safety, accountability, and regulatory oversight in high-stakes clinical environments. Explainable AI (XAI) systems hold promise for building trust and enabling verification, yet the literature remains fragmented due to inconsistent terminology, overlapping methodological families, ad hoc validation, and current reviews. This review aims to formalize XAI methods in CompPath through the: i) introduction of a pathology-centric vocabulary comprising seven core terms; ii) development of a taxonomy across methodological families and three orthogonal axes (stage, type, scope); and iii) establishment of a task-driven framework that maps five clinical questions to recommended methods, method evaluation, and deployment context. Five key gaps between current XAI capabilities and clinical deployment are identified, and actionable steps are proposed to advance XAI for CompPath.
Read the full abstract ▾
All authors Shubham Innani, Suhang You, Adam Shephard, Bhakti Baheti, Francesco Ciompi, Joe Yeong, Nasir Rajpoot, Michael Feldman, Solene Florence Kammerer-Jacquet, Dimitrios Makris, Geert Litjens, Anne L. Martel, Jana Lipkova, April Khademi, Spyridon Bakas, for the MICCAI SIG-CompPath
Computer Vision
FigMirror: Ground It, Code It, Plot It
Xiaohan Zhao, Jiacheng Liu, Yaxin Luo et al. (+1 more)
Converting scientific figures into executable code has gained increasing
attention, yet existing methods primarily focus on reproducing the reference figure itself. A more practical setting is to plot new data while preserving the visual style of a reference figure (e.g., color scheme and typography). Prior approaches mimic the reference through pixel-level optimization and struggle to carry its style to new data. We show that the key to this task lies in the coordinate grounding and coding capabilities present in modern computer-use models. We propose FigMirror, an agentic framework that unlocks these capabilities through Grounded Measurement, which locates visual elements by coordinates and measures their properties through executable code. We further introduce PlotTwin-Bench, an expert-curated
benchmark with fine-grained code and image-level style metrics. Experiments show that FigMirror consistently outperforms existing methods on reference-conditioned style transfer. All plots in this paper are generated by FigMirror, except those produced by other methods for comparison. Our code and data are available at: https://github.com/VILA-Lab/FigMirror.
Read the full abstract ▾
All authors Xiaohan Zhao, Jiacheng Liu, Yaxin Luo, Zhiqiang Shen
Computer Vision
A Large-scale Evaluation of Text-guided Models for Facial Editing
Rahul Nair, Saurav Pandit and Hannah Kerner
Facial appearance editing powers popular applications like FaceApp and Photoshop. Generative
Adversarial Networks (GANs) and 3D Morphable Models (3DMMs) have been widely used for facial editing. GANs can perform varied facial edits (e.g., changing hair color, hairstyle), but often produce unstable edits. 3DMMs produce stable edits, but can only alter pose and facial expression. Recently, text-guided
diffusion models like Nano Banana have become popular for image editing. Text-guided models are a compelling alternative to GANs and 3DMMs since they can produce both stable and varied image edits. While text-guided models have been widely tested for whole-scene edits (e.g., “make the woman play a guitar”), they have not been comprehensively tested for facial editing. We conducted the first large-scale evaluation ($\sim1$M images evaluated) of six popular text-guided models on a sequential facial editing task. We present Face-Edit-Attributes, the largest collection of $169$ facial editing attributes focused on hair, accessories, and pose edits. We compared model performance using two popular celebrity face
datasets: CelebA and CelebSET. Our results show that most models performed hair and accessory edits well, but struggled with editing pose. All models over-edit (e.g., changing hair color when asked only to change the hairstyle). We also evaluated demographic biases in each model. Our results show surprising biases in overediting: almost all models created more overedits for dark-skinned male faces and old faces. The code and data for our results (including our repository of $\sim 1$M images) can be accessed \href{https://github.com/rahul1801/Face-Edit-Bench}{\textcolor{blue}{here}}.
Read the full abstract ▾
Computer Vision
Blind Stereoscopic Omnidirectional Image Quality Assessment Using Predictive Coding Hierarchy
Wei Zhou and André Kaup
Stereoscopic omnidirectional images (SOIs) have provided users with newly immersive quality of experience in virtual reality environments. However, developing efficient and accurate perceptual quality assessment metrics for SOIs remains challenging due to many factors such as freely changeable field of views and binocular
vision. In this paper, based on the characteristics of the human visual system (HVS), we propose a Predictive Coding Hierarchy-inspired metric (PCH) for blind/no-reference stereoscopic omnidirectional image quality assessment. Motivated by the viewing process of SOIs, the proposed PCH includes a local cyclopean perception module, a global predictive perception module, and a visual quality regressor. First, observers browse different spherical sceneries from viewports, and aggregate the local visual information to infer the perceptual quality of SOIs. Therefore, we extract various viewports, followed by cyclopean conversion and saliency detection to approach the perception and
attention of the human
brain. After the local aggregation, viewers then infer the global scene in their minds. Based on the binocular mechanism, we fuse left and right views to perform predictive coding hierarchy modelling. Finally, the visual quality regressor is exploited to obtain the ultimate quality score related to both local and global perceptual cues. Extensive experiments demonstrate that the proposed PCH achieves competitive and consistently improved performance compared with state-of-the-art quality assessment methods.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CR
Breaking Darknet CAPTCHAs with general purpose LLM
Benjamin Fehrensen and Jens Hubler
Our work evaluates the effectiveness of automated methods for solving CAPTCHA challenges commonly encountered in darknet environments. These CAPTCHAs are typically designed to operate without JavaScript, resulting in distinct characteristics compared to mainstream CAPTCHA systems. Our study considers three representative challenge types: open-circle localization, rotation-based
alignment, and object-selection CAPTCHAs. The experiments reveal a systematic limitation of contemporary MLLMs: while they are generally capable of identifying relevant visual structures, they frequently struggle with precise spatial localization and
geometric transformations. These deficiencies can be mitigated either through task reformulation or by augmenting the models with specialized image processing tools. These deficiencies can be mitigated by task reformulation or by equipping the model with specialized image-processing tools. We therefore propose a hybrid framework in which an MLLM serves as a high-level
reasoning and orchestration layer while delegating
geometric computations to deterministic
algorithms via the Model Context Protocol (MCP). The resulting system achieves success rates above 90% across all evaluated CAPTCHA types and demonstrates that combining the complementary strengths of MLLMs and classical computer
vision yields a more accurate and efficient solver than either approach alone.
Read the full abstract ▾
Computer Vision
Beyond Representation Learning: A Systematic Study of Joint-Embedding Predictive Generation for 3D Brain MRI
Meng Zhou, Wenhao You, Yuxing Chen et al. (+1 more)
Joint-
embedding predictive architectures (JEPAs) have primarily been developed for self-supervised representation learning. Denoising JEPA (D-JEPA) recently demonstrated strong generative capabilities on natural images, yet the applicability to 3D medical imaging remains unexplored. Building on the D-JEPA framework, we present Med-D-JEPA, a systematic adaptation and evaluation of joint-
embedding predictive generation for 3D
brain MRI. Med-D-JEPA operates on continuous latent
tokens produced by a 3D KL-regularized
adversarial variational autoencoder, and combines masked context prediction, representation-level
alignment, per-
token diffusion, and iterative next-set-of-
token sampling. We evaluate unconditional and class-conditional generation quality on BraTS2019 and OASIS-1
datasets; downstream classification utility; and preliminary whole-tumor
segmentation on BraTS2020. Across different generation settings, Med-D-JEPA achieves superior or competitive performance compared to several strong baselines on fidelity and diversity metrics. Compared to training with real samples, Med-D-JEPA-based synthetic pretraining improves classification AUC from 0.63 to 0.85 on BraTS2019 and from 0.78 to 0.87 on OASIS-1. In the
segmentation study, pretraining on Med-D-JEPA samples improves Dice from 0.74 to 0.80 and reduces HD95 from 13.40 to 9.56 mm. These findings establish joint-
embedding predictive generation as a promising direction for 3D medical image synthesis and encourage further research in this direction.
Read the full abstract ▾
All authors Meng Zhou, Wenhao You, Yuxing Chen, Yueying Tian
Computer Vision
ClearText-Video: A Large-Scale Text-Centric Video Dataset Bridging Video Restoration and Scene-Text Enhancement
Jinlong Li, Jiaming Ding, Dingfu Lu et al. (+8 more)
Multimodal Large
Language Models (MLLMs) have recently made strong progress in visual–linguistic understanding. However, their performance on text-centric
video reasoning remains highly sensitive to input quality. Real-world user-provided
videos often contain motion blur, compression artifacts, noise, and low-resolution text, which impair reliable text reading and downstream
reasoning. Whether MLLMs can robustly read and reason about real-world scene text under diverse quality conditions remains a fundamental open question. We introduce ClearText-
Video (CTVid), a large-scale, scene-text-aware
benchmark for studying text-centric
video understanding under controlled quality variation. CTVid contains 4,639 real-world text-rich
egocentric videos, 550K+ frames, 1.6M human-verified scene-text annotations, and 220K+ spatial/temporal question–answer pairs in Chinese and English. For each high-quality
video, CTVid provides content-matched Degraded-Quality and Restored-Quality variants, supporting two task families: Text-Centric
Video Restoration and Multi-Quality VideoQA. We evaluate 18 representative restoration methods and 16 state-of-the-art MLLMs on CTVid. The results show that visual enhancement does not guarantee textual fidelity or downstream
reasoning gains: blur is more damaging than low resolution, restored
videos can alter the textual evidence used by MLLMs, and OCR-only pipelines remain far below direct
multimodal reasoning. CTVid exposes the gap between
video restoration and text-grounded understanding, providing a rigorous foundation for restoration-aware, quality-robust text-centric
video systems.
Read the full abstract ▾
All authors Jinlong Li, Jiaming Ding, Dingfu Lu, Malcolm Hsiu, Chuang Ke, Kangning Yang, Bochen Guan, Lan Fu, Jie Cai, Huiming Sun, Zibo Meng
Computer Vision · Cross-listed from cs.RO
Adversarial Calibration Attack on Autonomous Vehicles
Liangkai Liu, Qingzhao Zhang and Kang G. Shin
Autonomous vehicles (AVs) rely on accurate camera-
LiDAR calibration for
multimodal sensor fusion. In practice, calibration can drift due to vibration, temperature variation, or minor sensor displacement, motivating online calibration
algorithms that detect and correct misalignment at runtime while allowing the vehicle to continue operating without a factory visit. Existing AV attacks largely assume correct calibration. We instead identify online sensor calibration as a new attack plane. A corrupted calibration update can persist across subsequent fusion operations, causing system-wide errors that propagate from perception to planning and control. We present
Adversarial Calibration Attack (ACA), the first physical attack against camera-
LiDAR online calibration. Using a single
adversarial poster, ACA first spoofs the miscalibration detector to trigger the calibration process and then steers the calibration
estimator toward an incorrect transformation. A unified optimization jointly designs the poster's geometry and texture for both objectives. We evaluate ACA across
benchmark datasets, simulation, and physical experiments. On
benchmark datasets such as KITTI and nuScenes, ACA induces up to 33.9 degrees mean rotational calibration error, thereby severely degrading object detection. In the CARLA simulator, the attack causes a collision when the corrupted calibration is accepted in vulnerable scenarios crafted by the attacker. On a real Husky
robot, a printed
adversarial poster successfully reproduces the calibration error. These results demonstrate that online calibration is a practical and safety-critical attack surface for AVs.
Read the full abstract ▾
Computer Vision
FairReL: Deepfake Detection using Fairness-Aware Representation Learning
Xiaoman Lu, Jiaqi Li, Shuntian Zheng et al. (+2 more)
Although recent deepfake detectors achieve high overall accuracy, their errors remain unevenly distributed across demographic subgroups, with real faces from certain groups more often misclassified as fake. Existing
fairness-aware detectors typically regularise the entire feature representation, without identifying or controlling the specific components that drive unfair predictions. Such coarse intervention can over-suppress useful forgery cues while leaving demographic structure in component-specific subspaces. To address this, we identify two subgroup-sensitive components: multi-scale spatial features, which encode local facial and forgery patterns, and fine-tuning-induced residual features, which adapt the backbone to the unfair training distribution. We propose FairReL, a
fairness-aware representation-learning framework that targets both components with dedicated demographic supervision. FairReL uses an SVD-decomposed foundation-model backbone to isolate the fine-tuning-induced residual representation, and introduces two complementary losses. Group-Conditional Wavelet Decorrelation (GCWD) suppresses subgroup-imbalanced structure across spatial wavelet sub-bands, while Subspace-Localised Mean
Alignment (SLMA) aligns subgroup means within each real/fake class in the residual representation. Experiments on FF++, Celeb-DF, DFD and DFDC show that, against the state-of-the-art
fairness-aware detector, FairReL improves unseen-
dataset AUC by 3.9% while reducing subgroup FPR disparity by 10.2%. Code is available at https://github.com/xiaoman89/FairReL .
Read the full abstract ▾
All authors Xiaoman Lu, Jiaqi Li, Shuntian Zheng, Huiping Chen, Yu Guan
Computer Vision · Cross-listed from eess.IV
Medical Foundation Model Features as Perceptual Loss for Brain MRI Contrast Dose Simulation
Changsheng Fang, Dayang Wang, T. Campbell Arnold et al. (+2 more)
Perceptual losses are widely used in medical image synthesis because they encourage agreement in high-level structure beyond voxel-wise intensity similarity. In practice, most perceptual losses are still computed with natural-image backbones such as VGG16 or ResNet50, even when the target domain is
magnetic resonance imaging (MRI). This mismatch may weaken supervision for anatomy, contrast enhancement, and acquisition variability. We test whether medical foundation model features provide a more suitable perceptual loss for
brain MRI contrast dose simulation. The study has two stages. First, we compare RadImageNet, SegVol, and BrainIAC with ImageNet-pretrained VGG16 and ResNet50 as frozen feature extractors on four public medical imaging
benchmarks: thyroid ultrasound, breast ultrasound, anterior cruciate ligament knee MRI, and meniscus knee MRI. RadImageNet achieves the lowest mean rank across the Stage I representation suite and is selected as $\phi^\star$. Second, we replace only the VGG16 feature extractor in an existing iterative
brain MRI dose simulation framework with $\phi^\star$. The generator, reconstruction loss,
adversarial loss, auxiliary losses, optimization schedule, and loss weights are kept unchanged. Standard metrics change modestly, with PSNR increasing from 41.63 to 41.74, SSIM from 0.9739 to 0.9754, RMSE decreasing from 0.1384 to 0.1369, and residual-uptake CNR from 0.0085 to 0.0082. The visual results show the main effect: RadImageNet reduces residual enhancement in marked structures, follows a more faithful dose-reduction trajectory, and remains close to the acquired 10% low-dose target. These results support domain-aligned radiology features as a practical perceptual feature space for MRI dose simulation, while leaving clinical equivalence and larger-cohort validation as future work.
Read the full abstract ▾
All authors Changsheng Fang, Dayang Wang, T. Campbell Arnold, Enhao Gong, Srivathsa Pasumarthi
Computer Vision
Inter-3D VQA: A Roadside Multimodal Benchmark for 3D Spatiotemporally Grounded Visual Question Answering
Shaozu Ding, Linan Song and Dajiang Suo
Recent advances in visual question answering (VQA) and
multimodal large
language models (MLLMs) have enabled natural-
language reasoning over traffic scenes. However, existing
benchmarks are largely built from ego-vehicle views or 2D roadside
videos, limiting their ability to evaluate 3D-grounded
reasoning over real-world distances, trajectories, infrastructure
topology, and safety-critical interactions. We introduce Inter-3D VQA, a large-scale roadside
multimodal benchmark for 3D spatiotemporally grounded VQA at intersections. Built from synchronized point clouds and multi-view images, Inter-3D VQA contains 407K QA pairs covering lane-level positions, object relationships, motion patterns, and near-miss-oriented interaction
reasoning. We further propose Inter-Geo, an MLLM baseline that integrates object- and scene-level aligned
LiDAR representations, and Inter-Metrics, a unified evaluation framework for textual consistency, numerical accuracy, and semantic correctness. Experiments show that Inter-Geo outperforms image-based VLMs, especially on grounded spatial and temporal
reasoning tasks. Our
benchmark and codes are available at https://github.com/ASU-Suo-Lab/Inter-3D-VQA .
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Efficient JPEG Restoration in the Wavelet Domain via Mean Flows
Stefan-Alexandru Asandei and Mihai-Alexandru Radu
Latest JPEG restoration systems achieve strong quality with large models, yet often remain too slow and expensive for efficient on-device deployment. We present a 65M-parameter generative restorer that attains the lowest LPIPS at QF 10 and 20 on LIVE-1, Urban100, and DIV2K-val while sustaining 8.05 images/s at $1024\times1024$ on a single RTX 3090, roughly $4.9\times$ the reported throughput of one-step SODiff at one-twentieth of its parameters. Trained from scratch, the model replaces the learned VAE encoder-decoder with an exactly invertible two-level Haar transform, predicts a clean wavelet-domain residual through a rank-enhanced linear-
attention DiT that estimates compression severity internally, and is optimized with an improved MeanFlow objective that enables inference in one or two network evaluations without
distillation. Large pretrained priors remain stronger under severe compression (QF 5), whereas our model prioritizes throughput for deployment-constrained restoration.
Read the full abstract ▾
Computer Vision
MANTLE: A Framework for Adaptive In-Situ Planetary Perception Using a Modular Uplink Principle
Pranav Durai and Gary Doran
Planetary surface exploration missions rely increasingly on autonomous robotic platforms capable of interpreting complex terrain to ensure safe
navigation, enable targeted science, and improve operational efficiency, as demonstrated across past Mars missions from Viking through Perseverance. Among the key perception capabilities, landform classification provides contextual information for landing site selection and scientific analysis, while boulder
segmentation supports hazard assessment and path planning. This paper presents MANTLE, a multi-task adaptive network for terrain and landform extraction. The model uses a shared DINOv2 backbone for high-level feature extraction with task-specific heads: a classification head for large-scale landform classification, and a
segmentation head for pixel-wise boulder localization, each trained on curated
datasets built respectively from HiRISE orbital imagery and MSL surface-level imagery. The classification head achieved a test accuracy of 92.56% across seven Martian terrain classes, while the
segmentation head achieved a validation IoU of 0.753 and showed strong cross-sol generalization on a held-out test set from previously unseen rover traverses. A key advantage of MANTLE is its modular, extensible design, formalized here as the Modular Uplink Principle: only a shared, frozen backbone needs to remain onboard, while subsequent perception capabilities are trained on Earth as lightweight task-specific heads and uplinked without retraining the full model. This work demonstrates two such high-impact capabilities, terrain classification and boulder
segmentation, as an initial realization of a framework built to support many more over a mission's lifetime. With this foundation, future explorers need not arrive on Mars fully formed, but can continue to learn, adapt, and grow more capable with every uplink.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction
Tianyi Wang, Jiazhou Chen, Yiming Xu et al. (+7 more)
Video world models increasingly serve as data engines, action planners, and simulators for
embodied AI, but conventional
embodied world model (EWM)
benchmarks lack a unified 3D-grounded protocol for establishing whether generated rollouts preserve the underlying 3D scene state or translate into executable actions. We introduce RoboPhys-3D, a 3D-grounded EWM
benchmark built on RoboTwin 2.0, covering 50 manipulation tasks across four regimes, with 5,000 episodes and 25,000 multi-view ground-truth
videos. A defining feature of RoboPhys-3D is that generated and ground-truth
videos are processed through the same 3D reconstruction pipeline, enabling reconstruction-induced error to be distinguished from generation-induced error. The RoboPhys-3D
benchmark organizes 50 complementary metrics into 18 sub-dimensions across four levels: pixel-level fidelity, 3D geometry consistency, state-level understanding, and task-level completeness. We further introduce Average Full Score, a hierarchical score averaging all 50 metrics for comprehensive evaluation, and RoboPhyscore, a compact task-aligned score averaging the metrics most strongly correlated with task success. Among the four representative
video world models, Cosmos 3 achieves the highest RoboPhyscore (0.6330, 92.7% of ground truth), while state- and execution-grounded metrics reveal substantial failures that perceptual and
vision-
language model-based judgments fail to capture. RoboPhyscore further exhibits strong agreement with human evaluation (Pearson r = 0.9761 and Spearman h̊o} = 0.8962), demonstrating the importance of grounded, execution-aware evaluation for EWM capability.
Read the full abstract ▾
All authors Tianyi Wang, Jiazhou Chen, Yiming Xu, Xiangyu Li, Tianyi Zeng, Chih-Hsien Chou, Ning Lu, Liang Peng, Junfeng Jiao, Christian Claudel
Computer Vision · Cross-listed from eess.IV
Evaluating the Effects of Inter-Observer and Model Variability on Radiological Peritoneal Cancer Index Assessment
Savvas Saragiotis, Pieter C. Gort, Lotte J.S. Fleurkens-Ewals et al. (+5 more)
Deep learning
segmentation models are often evaluated using
geometric metrics such as Dice, HD95, and ASD, yet it remains unclear to what extent improvements in these metrics translate into clinically meaningful changes in downstream decision-making. The metric-to-decision gap is examined using radiological Peritoneal Cancer Index (rPCI) region
segmentation on contrast-enhanced CT, where a consensus definition provides anatomically grounded 3D regions and the clinically used PCI 20 threshold enables decision-level evaluation. Inter-observer variability is quantified across four experts on ten abdominal CT scans, and a published nnU-Net based rPCI
segmentation model is benchmarked against this human reference using Dice, HD95, and ASD across all 13 regions. To relate
geometric differences to clinical impact, a probabilistic peritoneal metastasis simulation is implemented on majority-vote rPCI maps, propagating region-boundary variability into variability of derived (r)PCI scores and classification at the PCI 20 cutoff. Observers showed high agreement (mean Dice $0.87$), while the model matched human performance in most regions but deviated more in regions 4, 8, and the small-bowel regions (9-12). Across simulations, score differences were typically small (mean $\Delta$rPCI $\approx 0.3$-$0.6$) for both observers and the model, and decision flips occurred predominantly when the reference score was near 20. These results suggest that rPCI-derived scoring is generally robust to typical
segmentation variability, while highlighting borderline cases as the main setting where expert review remains essential.
Read the full abstract ▾
All authors Savvas Saragiotis, Pieter C. Gort, Lotte J.S. Fleurkens-Ewals, Anna F. van Herwijnen, Marion Tops-Welten, L.D. Kampmeijer, Joost Nederend, Fons van der Sommen
Computer Vision · Cross-listed from eess.IV
SCoPE-Reg: Efficient Rigid Ultrasound Slice-to-Volume Registration via State-Space Correlation and Closed-Form Pose Estimation
Niklas Schwarz, Jens Kleesiek and Moritz Rempe
Ultrasound-guided interventions can require localization of an untracked 2D frame within a 3D anatomical reference. Rigid slice-to-volume registration (SVR) estimates this six-degree-of-freedom pose but remains challenging because of limited anatomical context, acoustic artifacts, and view-dependent appearance. Existing methods often use dense cross-
attention, whose cost scales with the product of slice and volume
token counts, or direct pose regression without explicit correspondence constraints. We introduce SCoPE-Reg, combining state-space slice–volume interaction, dense 3D coordinate prediction, and parameter-free weighted Kabsch estimation. On SVR tasks from CAMUS and $\mu$-RegPro, SCoPE-Reg yields mean target registration errors of $0.73$ mm and $2.27$ mm against $1.24$ mm and $2.63$ mm for the state of the art (SOTA), reduces peak error on CAMUS by 56% below SOTA ($12.5\!\to\!5.5$ mm), and registers $100\%$ and $80\%$ of frames within $3$ mm. On CAMUS at $128^2$ it retains the lowest error at increasing pose-perturbation magnitude. It holds $6.49$ M parameters independent of resolution, sustaining $51$ FPS at $512^2$. SCoPE-Reg establishes a SOTA in rigid ultrasound SVR: by coupling correspondence-based accuracy with bounded worst-case error and resolution-independent cost, it becomes viable at native acquisition resolution during intervention, where prior methods trade accuracy, reliability, or frame rate against one another. Supplementary code provided and will be open-sourced upon acceptance.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Evaluating the Safety of Deep Learning-Based Brain MRI Reconstruction
Dat Tat Mai, Thai Viet Pham, Thu Nguyen Thi Dang et al. (+1 more)
Objective: Deep learning accelerates
brain MRI four- to tenfold, but models can erase lesions or synthesize false tissue - failures pixel-averaged metrics like PSNR and SSIM miss. We review whether current evaluation practices detect this blind spot. Methods: Following PRISMA 2020, we searched seven databases without date limits, including 263 studies (1995-2026), appraised them using QUADAS-2 and matched instruments, and synthesized narratively. Categories were derived from titles, abstracts, and controlled vocabulary; reported prevalence figures represent floors. Duplicate screening achieved high agreement (Fleiss kappa = 0.877), as did appraisal (0.788; 0.390 where observable). Extraction is unaudited. Results: Only 18 of 263 studies (6.8%) recorded both a fidelity metric and reader assessment on identical data, leaving the central surrogate unmeasured. Reader studies mostly measured inter-reader agreement, which was weak: fastMRI 2020 concordance reached 0.457 and 0.386 (Kendall W), improving only where SSIM diverged. Erasing a 100 mm3 lacunar infarct shifts global PSNR by 0.03 dB under the stated error model. As the corpus grew fivefold, reader assessments dropped from 32% to 18%, recovering to 21%. Generative models - most associated with
hallucination (39%) - were among the least reader-evaluated (11.3%), while self-supervised models reached 47% with zero reader evaluation. Only 5% released code and ran reader studies; none evaluated a model observer; no named
dataset covered acute stroke or hemorrhage. Conclusions: On these floors, current evaluation practices cannot certify diagnostic safety. We derive five requirements safety-oriented evaluations must meet.
Read the full abstract ▾
All authors Dat Tat Mai, Thai Viet Pham, Thu Nguyen Thi Dang, James Jin Kang
Computer Vision · Cross-listed from eess.IV
Coronary Mask Guided Registration for Continuous Time 4D Cardiac CT Dataset Construction
Yuang Wang, Shuo Wang, Changyu Chen et al. (+12 more)
Objective: Clinical cardiac CT multiphase reconstructions generally provide acceptable image quality in end-diastole (ED) or end-systole (ES) phases, but in other phases may exhibit motion artifacts, especially in the right coronary artery (RCA). This limits ground-truth availability in 4D cardiac CT imaging research. We aim to construct a 4D cardiac CT
dataset that is generally suitable to serve as pseudo ground truth. Methods: We propose Coronary Mask Guided Registration (CMGR) to produce a motion-preserved, artifact-reduced, and continuous-time 4D cardiac CT sequence from the clinical multiphase reconstruction of each patient. For artifact reduction, CMGR uses the ED or ES phase as the reference phase and warps the reference volume with deformation fields to produce the sequence. For motion preservation, CMGR registers the reference phase to each non-reference phase of the multiphase reconstruction. To capture the motion of both the RCA and other cardiac structures in each registration, CMGR regularizes RCA masks and incorporates them into image-domain registration. Time-continuity is achieved by interpolating the deformation fields for non-reference phases to arbitrary times. Results: CMGR outperformed representative image-domain registration methods in capturing RCA motion and providing reasonable RCA shape, and showed competitive performance in capturing whole-heart motion. Additionally, CMGR reduced motion artifacts from clinical multiphase reconstructions, and intermediate CMGR frames generally provided plausible transitions between discrete cardiac phases. Conclusion: CMGR provides an effective approach for constructing continuous-time 4D cardiac CT
datasets. Significance: The
dataset can be used in system design simulations and in reconstruction
algorithm development, thereby facilitating advances in cardiac CT imaging.
Read the full abstract ▾
All authors Yuang Wang, Shuo Wang, Changyu Chen, Dufan Wu, Pengfei Jin, Yunqiang An, Yang Gao, Bin Lu, Dongrui Dai, Muge Du, Yan Yan, Dong Li, Liang Li, Li Zhang, Zhiqiang Chen
Computer Vision
SignRR: Retrieve and Refine Real Motion for Sign Language Production
Fidel Omar Tito Cruz, Angie Sanchez Marquina, Summy Farfan et al. (+1 more)
Sign
language production (SLP) aims to generate continuous signing motion from spoken
language, often through gloss-to-pose generation. Prior work mainly follows two paradigms. Generative models synthesize motion from a learned prior or from noise, without reference to an observed signing instance, making rare hand configurations and signer-specific articulation difficult to preserve.
Retrieval-based methods reuse real, well-articulated motion segments, but concatenating segments from different signers and co-articulation contexts can introduce rhythm and style inconsistencies across the full sequence, not only at segment boundaries. These limitations suggest a complementary solution: use
retrieval to provide realistic articulation, and use learned refinement to impose the global coherence that
retrieval alone lacks. We therefore propose retrieve-and-refine, a paradigm that starts from real retrieved motion and refines it into a globally coherent signing sequence rather than generating motion from scratch. Our framework, SignRR, initializes motion from a dictionary of real sign segments and refines the full sequence with a part-aware Residual VQ-VAE, where residual quantization preserves fine hand articulation and temporal length differences are handled in the latent space. Experiments on PHOENIX14T and CSL-Daily show that SignRR achieves state-of-the-art back-translation performance while maintaining competitive pose quality.
Read the full abstract ▾
All authors Fidel Omar Tito Cruz, Angie Sanchez Marquina, Summy Farfan, Gissella Bejarano
Computer Vision
GeBDA: Building Damage Assessment as Text-Based Sequence Prediction
Olivier Dietrich, Krishna Sapkota, Konrad Schindler et al. (+1 more)
Conventionally, Building Damage Assessment (BDA) is tackled either with dedicated network architectures or by fine-tuning geospatial image foundation models. In this work, we ask whether a general-purpose
Vision-
Language Model (VLM) can localize buildings and grade their damage through autoregressive sequence generation alone. We cast BDA as predicting a variable-length set of bounding boxes, each specified by its coordinates and a damage label. Our preliminary implementation, based on the open Gemma model, achieves promising damage mapping results from only bi-temporal satellite images and a suitable text
prompt.
Read the full abstract ▾
All authors Olivier Dietrich, Krishna Sapkota, Konrad Schindler, Genady Beryozkin
Computer Vision
Video Generative Models as Geometry Learner
Haosen Yang, Jifei Song, Zhensong Zhang et al. (+2 more)
Recent generative approaches to geometry estimation adapt pretrained image
diffusion models and treat the task as image-conditioned generation. Leveraging off-the-shelf image
diffusion models, they either (i) train task-specific geometry models (for depth and surface normal estimation) independently, losing the opportunity of exploring the intrinsic correlation of these
geometric targets, or (ii) jointly fine-tune modified image
diffusion backbones (e.g., altered self-
attention), which typically demands substantial labeled data. To overcome these limitations in a principled fashion, we repurpose pretrained
video generative models as a unified and data-efficient framework for geometry estimation, formulated innovatively as a next-frames prediction task. Our method, GeoNeXt, inherits naturally structured knowledge and richer priors from the
video model, while further adapting them for joint modeling of images and geometry targets (image <-> geometry), enabling more data efficient and effective learning of geometry. Extensive experiments validate our method for zero-shot
monocular depth and surface normal estimation across diverse
datasets, outperforming both previous task-specific and unified generative competitors while using substantially less training data. Notably, our method rivals discriminative state-of-the-art approaches trained on over 100x more data and even standouts on several
benchmarks.
Read the full abstract ▾
All authors Haosen Yang, Jifei Song, Zhensong Zhang, Xiatian Zhu, Jiankang Deng
Computer Vision
Texture Image Classification Using DWT AlexNet Feature Fusion and Deep Neural Networks
Arun D. Kulkarni
Texture image classification plays a significant role in computer
vision applications, including industrial inspection, medical image analysis, remote sensing, and object recognition. Handcrafted features can capture local texture characteristics but may have limited capability to represent complex visual patterns. In contrast, deep learning models automatically learn discriminative representations but may not fully exploit the multiscale spatial-frequency information inherent in texture images. This paper proposes a hybrid feature fusion framework, termed DWT_AlexNet_DNN, which combines Discrete Wavelet Transform (DWT) features with deep features extracted using AlexNet for texture image classification.
Read the full abstract ▾
Computer Vision
Learning the Target Priors Before Image Translation: A Decoupled Training Paradigm for Cross-Modal Image Translation in Remote Sensing
Keyan Hu, Mingtao Wang, Ziyu Zhou et al. (+4 more)
Cross-modal image translation in remote sensing must preserve source-observed content while matching the target-domain distribution. Existing methods jointly learn the target prior and cross-modal dependence from scarce paired data, overlooking a key asymmetry: only the latter intrinsically requires cross-modal correspondence. We formalize this distinction through conditional-score and denoising-risk analyses and propose Learning the Target Priors Before Image Translation (LTP-BIT), a prior-first paradigm that decouples the two learning tasks. LTP-BIT first learns a target-domain generative prior from large-scale unpaired imagery, then retains the pretrained backbone weights and learns source-conditioned control through P-DART, a parameter-efficient dual-stream architecture. Controlled experiments show that prior matching and scaling primarily improve target-domain realism, whereas instance fidelity relies more strongly on conditional adaptation. LTP-BIT achieves state-of-the-art performance across SAR-to-RGB and NIR-to-RGB
benchmarks using only 9.81% task-specific parameters. On QXS-SAROPT, it retains near-full-data instance fidelity with only 25% of the paired samples.
Read the full abstract ▾
All authors Keyan Hu, Mingtao Wang, Ziyu Zhou, Tiandong Shi, Haifeng Li, Ji Qi, Chao Tao
Computer Vision
Anatomy-Aware Promptable Segmentation with Online Interactive Training for AUTOPET V
Pablo Lozano-Jimenez, Sergio Romero-Tapiador and Ruben Tolosana
We present an anatomy-aware, promptable model for whole-body lesion
segmentation in FDG and PSMA PET/CT, developed for the AUTOPET V challenge. The proposed method is built as family of nnU-Net-based models and trained in two stages: i) a pre-training stage that produces a strong initial
segmentation, and ii) an online interactive stage that learns to exploit scribble
prompts, refining the prediction over successive interactions. Anatomical context is incorporated through organ supervision using a single shared head that predicts lesions and organs from the same features, which reduces false positives arising from physiological uptake. Also as the tracer (i.e., FDG/PSMA) is not provided at inference, we add a tracer classifier based on image processing and a random forest over coronal MIP features, routing each study to a combined FDG+PSMA model or to a PSMA-specific model. Across four-fold cross-validation the organ-supervised model achieves the best and most stable performance, the interactive stage improves the Dice score monotonically with each
prompt, and PSMA-specific training yields the strongest tracer-wise results.
Read the full abstract ▾
Computer Vision
LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation
Yixuan Ding, Jiahao Kong, Wei Huang et al. (+2 more)
Autoregressive
video diffusion enables scalable long-
video generation by producing chunks from a bounded recent context. While recency-based caching preserves local continuity, it evicts historical cues needed when subjects, objects, scenes, or attributes reappear. Existing
memory mechanisms expose models to nonlocal history, but access alone does not ensure effective use. Our analysis reveals that
video DiT layers exhibit distinct preferences for current, recent, and distant context, suggesting that long-range
memory requires deciding both what to retrieve and where to use it. We introduce LayerRecall, a current-conditioned, layer-selective
memory router that retrieves relevant historical K/V states and injects them only into backbone-specific
memory-sensitive layers while preserving local
attention elsewhere. To reduce reliance on scarce high-quality long-horizon
videos and explicit
memory-allocation labels, we further propose Cross-Horizon Prediction Matching (CHPM), which uses a privileged long-context reference to supervise the bounded-
memory router in prediction space. Across 100 multi-shot evaluation
prompts, LayerRecall achieves the best overall results on MemoBench and MovieBench while matching its backbone on VBench-Long, demonstrating stronger long-range recovery without sacrificing local continuity. Qualitative analyses further reveal
memory-guided self-correction, whereby initially mismatched local attributes return to their historical appearance without resetting ongoing motion or scene structure. Additional analyses show cross-backbone portability and negligible inference overhead.
Read the full abstract ▾
All authors Yixuan Ding, Jiahao Kong, Wei Huang, Ruijie Quan, Yi Yang
Computer Vision
ARC-CT: Anatomy-Routed Contrastive Vision-Language Learning for 3D Chest CT
Huseyin Umut Isik, Mehmet Alp Ozaydin, Sila Kurugol et al. (+1 more)
Contrastive
vision-
language learning uses paired chest CT volumes and radiology reports to learn abnormality classifiers without manually annotated labels. However, two characteristics of chest CT challenge conventional global contrastive learning. First, many critical abnormalities are small or anatomically localized, and pooling an en- tire volume into a single
embedding may dilute their visual evidence. Second, the standard contrastive objective treats every other scan in a batch as a negative. Because many chest CTs share abnormalities, this objective incorrectly pushes co-positive pairs apart. We propose Anatomy-Routed Contrastive Learning for 3D Chest CT (ARC-CT), a region-aware framework that addresses these limitations using only la- bels extracted from reports by an LLM, with no manual annotations or bounding boxes. ARC-CT combines three components: (1) an Anato- myQFormer localizing evidence via queries constrained by automatically generated organ masks; (2) a label-Jaccard soft InfoNCE objective in- tegrating the standard one-hot target with the label-set overlap of each pair, which reduces false-negative penalties between studies that share clinical findings; and (3) an organ-level
alignment loss connecting mask- pooled visual features to organ-specific report text extracted offline with a large
language model. ARC-CT achieves a 0.86 mask-free macro AUC across 18 abnormalities using a compact 3D ResNet-18 backbone. Over- all, ARC-CT outperforms both comparable efficient baselines and sev- eral larger
transformer models. Our code and weights are available at https://github.com/arc-ct/arc-ct.
Read the full abstract ▾
All authors Huseyin Umut Isik, Mehmet Alp Ozaydin, Sila Kurugol, Şeyda Ertekin
Computer Vision
Prompt-Guided Interactive Segmentation of Interstitial Lung Disease in Thoracic CT
Vasilis Dedousis, Lubnaa Abdur Rahman, Lorenzo Brigat{\omicron} et al. (+8 more)
Accurate
segmentation of interstitial lung disease (ILD) patterns is essential for quantitative disease assessment and longitudinal monitoring. However, existing approaches remain limited by relying on dense annotations and producing static predictions that cannot be refined, motivating interactive approaches. While promptable models show promise in interactive
segmentation, their adaptation to ILDs remains largely unexplored. To address this gap, we investigate
prompt-guided foundation models for ILD refinement and present, to the best of our knowledge, the first adaptation of MedSAM2 for interactive 3D ILD
segmentation on thoracic CT. We investigate three fine-tuning strategies and multiple clinically motivated
prompts: bounding-boxes (BBox), point, lasso, and scribble. On a
dataset spanning seven ILD patterns and healthy lung tissue, full model fine-tuning performed best, improving the average Dice score by 4.7 percentage points over MedSAM2.While BBox
prompts achieve the strongest performance, non-native MedSAM2 interactions such as lasso and scribble
prompts also prove effective. Finally, we present and evaluate a proof-of-concept end-to-end workflow in which MedSAM2 is initialized from an automatic
segmentation prior and subsequently refined using radiologist
prompts. Model weights and plug-ins made available at: https://github.com/AIHNlab/ILD-SemiSegTool.
Read the full abstract ▾
All authors Vasilis Dedousis, Lubnaa Abdur Rahman, Lorenzo Brigat{\omicron}, Ethan Dack, Andreas Christe, Christoph Frank, Manuela Funke-Chambour, Justus Roos, Adrian Huber, Lukas Ebner, Stavroula Mougiakakou