Computer Vision
Parameter-Efficient pretrained-CT-to-MRI Transfer for Rectal Cancer Segmentation: Performance-Calibration Trade-offs
Aneesh Rangnekar, Jorge Tapias Gomez, Joseph O Deasy et al. (+1 more)
Accurate rectal cancer
segmentation from
magnetic resonance imaging (MRI) is essential for adaptive radiotherapy and tumor response assessment, but deployment also requires computational efficiency and informative, calibrated uncertainty estimates. We therefore introduce SWIFT, a SWin pretrained model wIth parameter-eFficient and Tumor-aware fine-tuning for rectal cancer
segmentation. A Swin V2 encoder pretrained on 10,444 public 3D CT volumes using a DINOv2-style objective was adapted to T2-weighted MRI through four cumulative configurations: full fine-tuning (SWIFT), decoder compression (SWIFTe), low-rank adaptation (SWIFTe-LoRA), and a four-member LoRA-decoder ensemble (SWIFTe-LDE4).
Geometric accuracy, tumor detection, radiomic agreement, and probability calibration were evaluated on a held-out 247-case test set from a single-institution cohort acquired using 1.5 or 3 Tesla GE scanners. Compared with SWIFT, SWIFTe reduced total parameters by 70.1% (from 72.8M to 21.8M) and increased tumor detection rate from 89.9% to 93.9%, while achieving a slightly lower median surface DSC (0.61 versus 0.62) and improved radiomic agreement. In a separate SWIFTe ablation, removing tumor-aware augmentation reduced detection from 93.9% to 89.9% but increased surface DSC from 0.61 to 0.64, demonstrating a detection-boundary-agreement trade-off. SWIFTe-LoRA used 14.6% of SWIFTe's trainable parameters while retaining similar
segmentation performance. SWIFTe-LDE4 achieved the lowest calibration errors among the four configurations after temperature scaling (expected calibration error, 0.217; Brier score, 0.222), although the absolute expected calibration error indicates residual miscalibration. Similar efficiency-calibration patterns were observed using the public VoCo checkpoint, supporting robustness across pretrained initializations rather than external clinical generalizability.
Read the full abstract ▾
All authors Aneesh Rangnekar, Jorge Tapias Gomez, Joseph O Deasy, Harini Veeraraghavan
Computer Vision
Temporal Sensitivity Analysis of Tessera Embeddings
Julia Guerrero-Viu, Alex López-Cifuentes, Ignacio Pérez-Villar et al. (+1 more)
Many Earth Observation applications need land-use/land-cover maps that are both precise and frequently updated, yet the strongest Earth Observation foundation models build their
embeddings from a full year of observations. We present a controlled study of the temporal sensitivity of Tessera, one of these leading foundation models, for land-use/land-cover mapping. Keeping the encoder frozen, we recompute its
embeddings over varying observation windows, from a full year down to a single day. We use them as inputs to a linear probe and a UNet
segmentation head, benchmarking both of them against from-scratch networks on LUCAS, DynamicEarthNet, and PASTIS-R
datasets. We show that the value of the
embeddings is task-dependent. Where classes are separated by phenology, as for the crop types of PASTIS-R, they reach a mean Intersection-over-Union of $58.3$, about $46\%$ above the best from-scratch model. Where classes are temporally stable (e.g., forests in DynamicEarthNet and LUCAS),
embedding-based and from-scratch models match only under full supervision. On both
datasets, Tessera
embeddings remain markedly more label-efficient. Degradation under shorter temporal windows is gradual and class-dependent. Contracting the window from one year to one month costs $39\%$ of the
segmentation accuracy on PASTIS-R but only $5\%$ on DynamicEarthNet. Single-day
embeddings still classify land cover in LUCAS at $3.4$ times the chance level. Our study shows that temporal coverage is therefore a tunable cost rather than a fixed prerequisite, opening regimes such as near-real-time mapping and faster land-use/land-cover refresh cycles.
Read the full abstract ▾
All authors Julia Guerrero-Viu, Alex López-Cifuentes, Ignacio Pérez-Villar, Fabio Pacifici
Computer Vision
Ancient-Bench: A Comprehensive Multi-millennial, Multi-medium, and Multi-script Benchmark for Ancient Chinese Artifact Text Recognition
Hiuyi Cheng, Nuo Xu, Yuyi Zhang et al. (+9 more)
Ancient Chinese artifact text recognition is fundamental to heritage digitization, and
benchmarks for ancient texts are essential for evaluating current model capabilities. However, existing
benchmarks suffer from ”fragmentation”, manifested in limited temporal coverage, limited medium diversity, and incomplete script types. Therefore, we present Ancient-Bench, a comprehensive
benchmark of 2,700 images for ancient Chinese artifact text recognition, featuring three dimensions: Multi-millennial (spanning 3,000 years of character
evolution), Multi-medium (covering nine artifact categories), and Multi-script (encompassing seven historical script forms). To enable consistent and fair evaluation across heterogeneous media, we further define three annotation standards tailored to the medium-specific characteristics of ancient texts: symbol standardization, character standardization, and parsing standardization. Extensive experiments on Ancient-Bench covering general
Vision-
Language Models (VLMs) and OCR-specialist models reveal that ancient Chinese artifact text recognition remains fundamentally unsolved, with persistent challenges in variant characters, specialized symbols, and
hallucination. The
dataset is available at https://github.com/SCUT-DLVCLab/Ancient_Bench.
Read the full abstract ▾
All authors Hiuyi Cheng, Nuo Xu, Yuyi Zhang, Xuhan Zheng, Wei Pan, Jing Zhang, Dezhi Peng, Minghui Liao, Yihua Teng, Jihao Wu, Haoyu Ren, Lianwen Jin
Computer Vision
Magpie: Real-Time World Renderer for Interactive Games
Xiaoyu Zhan, Xinyu Wang, Xiaohong Zhang et al. (+6 more)
Modern game development relies heavily on conventional graphics pipelines. High-quality visual content requires modeling, material authoring, animation, lighting, effects, and runtime optimization, making asset production expensive and extending the development cycle of game prototypes. Recently,
video foundation models are beginning to change film and
video production, but games differ from linear media, they require not only continuous and realistic imagery, but also stable and reproducible gameplay rules, object states, and interaction outcomes. We present Magpie, a real-time generative world-rendering system for interactive games. Magpie separates gameplay execution from visual generation. Designers define scenes and rules in a game engine. At runtime, the Game Engine resolves player actions and maintains world state, while an independent Render Server generates visual output from white-box frames produced by the engine. Magpie provides a system-level implementation path for applying generative models to real-time game rendering. It preserves gameplay designability and reproducibility, and reduces the dependence of early game prototypes on complete visual assets.
Read the full abstract ▾
All authors Xiaoyu Zhan, Xinyu Wang, Xiaohong Zhang, Huanjie Zhu, Tengjiao Sun, Pengcheng Fang, Jiaxing Yu, Yanwen Guo, Dongjie Fu
Computer Vision · Cross-listed from cond-mat.mtrl-sci
Data-efficient crack quantification in lithium-ion cathodes using foundation model transfer
Thorsten Tegetmeyer-Kleine, Thomas Schmitt, Phillip Aquino et al. (+3 more)
Battery lifetime is central to sustainable electrification, yet the particle cracking that drives lithium-ion cathode aging is hard to measure: quantitative microscopy of this degradation is bottlenecked by annotation, because each destructive electron-microscopy cross-section spans hundreds of megapixels and pixel-level expert labelling requires hours per image. We show that a frozen self-supervised
vision-
transformer encoder, combined with a lightweight trainable decoder and iterative model-assisted annotation, turns this sparse labelling budget into population-scale degradation measurements. Applied to three 120-megapixel NMC cathode cross-sections representing initial, cycled-aged and calendar-aged states, the framework distinguishes intragranular cracks from early- and late-stage intergranular cracks and yields per-particle distributions of crack width, tortuosity and area fraction. Late intergranular crack coverage reaches 4.6% in the cycled sample versus 0.5% in the initial and calendar-aged samples, forming more tortuous, higher-coverage networks, consistent with degradation from repeated electrochemical cycling rather than elevated-temperature storage alone. A single destructive image yields the population-level statistics needed for lifetime-extending design, aging assessment and second-life decisions.
Read the full abstract ▾
All authors Thorsten Tegetmeyer-Kleine, Thomas Schmitt, Phillip Aquino, Christiane Rahe, Dirk Uwe Sauer, Weihan Li
Computer Vision
ReViCo: Unveiling the Limitations of VLMs in Visual Text Understanding via Error Correction
Bojun Zhang, Junhong Liang, Feifei Zhai et al. (+2 more)
Vision Language Models (VLMs) have shown great success in general visual tasks, yet they still struggle to deeply understand text within images. In this paper, we introduce ReViCo (Real Visual Correction), a
benchmark designed to evaluate VLM text understanding through a novel task of visual text error correction. ReViCo challenges models to identify and fix text errors in real-world images, which requires a profound understanding of the interplay between visual text and its surrounding visual context. We
benchmark various VLMs using two distinct paradigms:
prompt-based strategy and targeted model training, both aimed at pushing the limits of current models. Our experiments reveal a striking performance gap between even the best VLMs and human, and further analysis also shows that most models struggle to accurately perceive the visual text, resulting in frequent correction errors. By highlighting these gaps, ReViCo provides a new
benchmark foundation for developing more robust and text-aware VLMs.
Read the full abstract ▾
All authors Bojun Zhang, Junhong Liang, Feifei Zhai, Fengxian Ji, Yu Zhou
Computer Vision · Cross-listed from cs.NE
ANTShapes Benchmarking Datasets for Event-Based Neuromorphic Object Classification
M. Middleton, H. Kayan, B. Sen Bhattacharya et al. (+7 more)
Object classification in event-based computer
vision is a task that is attracting considerable research
attention. Event-based object classification is a fundamental task in the fields of security and applied computer
vision, which typically use synchronous frame-based cameras and computing pipelines for operation. This approach has several practical flaws. The size, weight and power consumption of the device could prohibit deployment at the extreme edge or in covert sensing environments. Besides this, there are security concerns inherent in cloud-based or other off-device computation approaches due to the requirement of sending and receiving potentially sensitive data. Furthermore, this transmission of data introduces latency and requires consistent connectivity to the cloud infrastructure to function. The use of Spiking Neural Networks (SNNs) hosted on neuromorphic devices attempts to solve several issues present in this conventional approach. Research into event-based object classification methods are hindered by the lack of high-quality
vision datasets to use. To this end, the ANTShapes simulation tool has been previously proposed to create and label event-based
vision datasets. In this paper, four novel
datasets of varying difficulties are created using the tool and are benchmarked against existing spiking
datasets commonly used for event-based
vision research (N-MNIST, CIFAR10-DVS, DVSGesture and POKER-DVS). Classification is performed using a convolutional SNN. This work simultaneously provides four
datasets with rich details for future experiments to use and validates the output of the ANTShapes
dataset simulation tool as being suitable for its purpose.
Read the full abstract ▾
All authors M. Middleton, H. Kayan, B. Sen Bhattacharya, T. Ali, E. Baikas, M. Vousden, C. Perera, O. Rhodes, E. Gheorghiu, M. A. Trefzer
Computer Vision
EditaLive! Unified Character Video Editing for Live Streaming
Zhiyuan Li, Chi-Man Pun, Peng-Tao Jiang et al. (+2 more)
Conventional
video editing primarily focuses on scene-level content, whereas live streaming places greater emphasis on the human subject. However, directly applying existing
video-editing methods to human-centric live streaming remains challenging, as they may introduce facial-expression inconsistencies and typically depend on multiple offline inference steps, making them unsuitable for real-time interaction. We propose EditaLive, a novel framework for real-time streaming character
video editing. In detail, we start from a pretrained image animation model (Wan-Animate), which naturally decouples appearance from motion, and repurpose it as the base model for instruction-based human-centric
video editing by reference frame editing and
video reconstruction via the collected CharEdit-50K
dataset. Besides, we adapt the model from offline bidirectional to causal streaming generation, and design an aligned self-rollout
distillation strategy that compresses the model into a two-step sampler, where fixed RoPE and align forcing reduce training–inference discrepancies, and first-frame preserved sparse
attention filters redundant historical information to mitigate appearance drift. Extensive experiments demonstrate that EditaLive delivers state-of-the-art editing performance with faithful preservation of facial expressions and low-latency real-time streaming inference.
Read the full abstract ▾
All authors Zhiyuan Li, Chi-Man Pun, Peng-Tao Jiang, Bo Li, Xiaodong Cun
Computer Vision · Cross-listed from cs.MM
How AI Experiences Art: Emergent Aesthetic Structure in a Self-Supervised Multimodal Embedding Space
Corey D.C. Heath
Aesthetics are an important part of the symbolism of artistic works. Although subjective, humans categorize art based on the
emotion evoked regardless of modality. What remains under-explored is how AI models form their own aesthetic categorization of human-produced media without explicit labels or cross-modal supervision. We present a self-supervised framework that projects four modalities (text, audio, image and
video) into a shared 256-dimensional
embedding space and applies iterative clustering to discover aesthetic structure. We discuss the divergence between AI-generated cluster assignments and human affective register labels on a weakly supervised
multimodal dataset. This work has applications in understanding how AI structures cross-modal similarity, organizing heterogeneous media collections for
Retrieval-Augmented Generation (RAG), and automated data labeling.
Read the full abstract ▾
Computer Vision
Automated 2D and 3D Segmentation of AMD and DME Lesions in OCT
Lucia Sundberg, Zhihao Zhao and M. Ali Nasseri
Age-related macular degeneration (AMD) and diabetic macular edema (DME) are leading causes of
vision loss, and optical coherence
tomography (OCT) is the standard modality for detecting and monitoring the subtle lesions that drive treatment decisions. Most deep-learning
segmentation work for OCT is validated only in-domain, leaving generalization to clinical data collected under different acquisition protocols largely untested. This work develops and systematically ablates four lesion-
segmentation pipelines – 2D and 3D variants for AMD and DME – reaching Dice scores of 0.76 to 0.82 with strong volumetric and surface calibration (r vol, r surf greater than or equal to 0.97 across all four pipelines) on an in-domain validation set. The ablation process establishes a full-volume, calibration-aware adoption standard that catches mechanisms an ordinary slice-level evaluation would keep, and identifies ensemble composition as the most consistent driver of improvement. To test generalization, the models are evaluated on OLIVES, an external clinical cohort with no lesion-level ground truth, using a proxy-metric framework built around biomarker AUROC, central subfield thickness (CST) correlation, and longitudinal concordance. Predictions track clinical biomarkers outside the training distribution, though less strongly than in-domain – evidence for, not validation of, automated lesion-burden tracking as a clinical tool.
Read the full abstract ▾
Computer Vision
SpatialCrafter: Single Image World Modeling with Generative 3D Proxies
Chuan Fang, Lingteng Qiu, Yixun Liang et al. (+8 more)
Explorable image-to-scene generation is essential for applications in gaming, robotics, and virtual reality. Existing methods based on
video diffusion model (VDM) commonly rely on incomplete conditioning signals such as sparse point clouds or 2D panoramas, leading to
stochastic hallucinations, long-term drifts and suboptimal 3D consistency. We present SpatialCrafter, a novel two-stage framework that addresses these issues by introducing a global 3D proxy for high-fidelity image-to-scene generation. Specifically, we decompose the generation process into global proxy generation and appearance refinement. For proxy generation, we propose a Point-anchored Sparse Structure~(PaSS) Flow module that predicts a spatially aligned and geometrically consistent 3D proxy. For appearance refinement, we re-frame the VDM as a Generative Deferred Refiner which synthesizes high-frequency photorealistic details upon proxy-defined scene geometry. To better integrate the proxy with the pre-trained VDM, we introduce Parallel Geometry Injection and Proxy-Aware Corruption training strategies, which improve robustness to proxy artifacts without disrupting the pretrained generative
manifold. Furthermore, as no suitable
dataset exists for this explorable scene generation task, we construct a new large-scale
dataset of 115K scenes. To the best of our knowledge, it is the first hybrid
dataset for image-to-scene generation. Extensive experiments on both synthetic and real-world
datasets show that SpatialCrafter outperforms state-of-the-art methods, mitigates long-term drift, and remains robust and consistent under rapid camera motion and extreme viewpoint changes. Our project page: \href{https://fangchuan.github.io/SpatialCrafter/}{fangchuan.github.io/SpatialCrafter/}
Read the full abstract ▾
All authors Chuan Fang, Lingteng Qiu, Yixun Liang, Rui Chen, Kunming Luo, Zhaohua Zheng, Tongyuan Bai, Feipeng Tian, Zilong Dong, Zihan Zhou, Ping Tan
Computer Vision
Beyond Classification: Task-Dependent Learnability under Privacy-Motivated Image Transformations
Leon Ranke, Wolfgang Hübner, Ronny Hug et al. (+2 more)
Privacy-Enhancing Technologies (PETs) in computer
vision often rely on noise or image perturbations to protect visual data while securely processing it, creating a trade-off between task performance and protection. This trade-off is commonly evaluated using image classification, which primarily captures semantic separability and remains robust despite significant
geometric, spatial layout or local boundary alterations. As a result, it is too simplistic as a proxy for generic
vision tasks. Exhaustive downstream-task evaluation, however, is computationally expensive because models must often be trained for each PET transformation and parameter setting. We therefore propose a compute-aware multi-task protocol for evaluating PETs in model training. It combines lightweight proxy tasks that target complementary aspects of visual structure while remaining simple and fast to compute. Across irreversible
privacy transformations, key-based block primitives, and learnable image encryption schemes, we demonstrate that PETs with similar classification accuracy can differ substantially on other tasks. The outcomes highlight the need for PET evaluation protocols that move beyond classification-only reporting.
Read the full abstract ▾
All authors Leon Ranke, Wolfgang Hübner, Ronny Hug, Michael Arens, Jürgen Beyerer
Computer Vision
Video-OPSD: Exploiting Privileged Visual Evidence for On-Policy Self-Distillation in Video Large Language Models
Ziyue Wang, Shiqi Huang, Weiwen Xu et al. (+2 more)
On-policy self-
distillation (OPSD) has recently emerged as an effective post-training paradigm that improves policy optimization through dense
token-level supervision from a privileged self-teacher. Despite its promise, OPSD remains largely underexplored for
Video Large
Language Models (
Video-LLMs). Existing methods typically construct privileged teachers by augmenting their context with additional information while keeping the primary input unchanged for both teacher and student.
Video reasoning, however, offers a distinct source of privileged supervision within the primary input itself: long
videos contain substantial temporal redundancy, and only a small subset of frames provides the evidence necessary to answer a question. Building on this observation, we present $\textbf{Video-OPSD}$, an OPSD framework that exploits privileged visual evidence for both self-teacher construction and knowledge transfer. First, our Evidence-Grounded Self-Teacher conditions the teacher exclusively on annotated evidence frames while the student continues to reason over the complete
video. This focused visual input enables the teacher to provide more informative supervision. Second, our Evidence-Guided
Token Optimization adaptively weights
token-level
distillation according to each
reasoning token's reliance on privileged visual evidence, thereby emphasizing perceptually grounded
reasoning. Experiments across
video understanding and
reasoning benchmarks show that $\textbf{Video-OPSD}$ consistently improves upon Standard OPSD across multiple backbones and achieves performance comparable to GRPO while requiring substantially less training time, establishing an effective and efficient post-training approach for
Video-LLMs.
Read the full abstract ▾
All authors Ziyue Wang, Shiqi Huang, Weiwen Xu, Bihan Wen, Xudong Jiang
Computer Vision
Anatomy-Guided Foundation Model Adaptation with Within-Case Prototype Supervision for Standard Plane Detection in Fetal Ultrasound Blind Sweeps
Yuzhe Zhao
Detecting the fetal abdominal circumference standard plane in low-cost obstetric blind sweeps is a highly imbalanced frame-classification problem: positive frames account for under 3% of a sequence, form short contiguous segments, and are poorly handled by off-the-shelf ultrasound and
vision foundation models. We propose AnatoProto, a lightweight sequence-level framework that adapts a frozen BiomedCLIP encoder to fetal blind sweeps through four components: (i) anatomy-weighted spatial pooling that uses nnU-Net abdominal-region probabilities as a spatial prior to reweight BiomedCLIP patch
tokens, so frozen semantic features are aggregated onto anatomically meaningful regions; (ii) a within-case prototype loss that pulls each frame
embedding toward the mean of positive frames of the same sweep, exploiting case-level structure unavailable at the frame level; (iii) a three-stage cascade refinement (frame->segment->case-level rejecter) that lifts the prediction unit from noisy frames to structurally-constrained segments; and (iv) a hybrid prediction head that jointly models per-frame stability and inter-frame boundary transitions to suppress boundary false positives. On the ACOUSLIC-AI
benchmark, AnatoProto reaches a test F1 of 67.72, outperforming the strongest foundation-model baseline (FetalCLIP + PRS, F1 = 54.52) by +13.20 F1 and the strongest
video temporal-action-detection baseline (TriDet + PRS) by +15.76 F1. A synergy study, backed by
embedding geometry and paired-bootstrap confidence intervals, shows that the prototype loss and anatomy-weighted pooling are not additive: applied alone the prototype loss reduces recall by 12 points, but combined with anatomy-weighted pooling it increases recall by 6.5 points – a sign-flip we trace to the accuracy of the within-case prototype.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Omni-Interactive Universal Embedder
Wei-Yao Wang, Kazuya Tateishi, Shuyang Cui et al. (+4 more)
Multimodal representation learning has been shifting from traditional two-tower architectures to large
language model (LLM)-based embedders due to their strong instruction-following capabilities. Despite this progress, existing approaches primarily focus on
language and image modalities, which also remain the dominant modalities for user-conditioned interactions in current embedders. In this paper, we propose the first Omni-Interactive Universal Embedder (OmniUE), which not only learns a unified
embedding space across text,
video, and audio by leveraging intermediate-layer representations from dedicated learnable
tokens, but also supports omni-interactive querying, enabling users to provide inputs in the form of text, visual regions of interest, and audio spans. Within OmniUE, visual and audio segmenters process diverse user interactions and integrate them with an omni-LLM to produce user-conditioned any-to-any
embeddings via context aggregation. To evaluate OmniUE's omni-interactive capabilities, we introduce OmniCHOIR, benchmarking models for omni-interactive compositional audio
retrieval based on the given text,
video, and audio as well as unimodal or
multimodal interaction
prompts. OmniUE consistently surpasses state-of-the-art baselines across diverse modalities, with average improvements of 10.4% on textual-interactive
video benchmarks (MMEB-v2-
video), 1.0% on audio tasks (MAEB), 83.1% on visual-interactive
benchmarks (SCaR), and 21.9% on our omni-interactive OmniCHOIR
benchmark. We believe that jointly advancing omni-modal representation learning and omni-interactive querying paves the way toward universal embedders. Project page: https://wywywang.github.io/omniue-demo/.
Read the full abstract ▾
All authors Wei-Yao Wang, Kazuya Tateishi, Shuyang Cui, Christian Simon, Takashi Shibuya, Shusuke Takahashi, Yuki Mitsufuji
Computer Vision
Multi-Person Human Motion Forecasting in Complex Scenes
Serdar Ozsoy, Lars Doorenbos and Juergen Gall
Accurately
forecasting the movement of people in complex scenes requires
reasoning over the past and present state of the entire environment. In this context, effectively incorporating object information and social interactions into a unified framework remains particularly challenging. To address this, we propose Object-Conditioned Social
Diffusion (OCSD), a conditional
diffusion model that integrates motion history, multi-person interactions, and object cues into a single framework. OCSD uses an object-conditioning mechanism that modulates denoising at every timestep, enabling fine-grained human-object
reasoning, and a social encoder that models the interactions between all humans in the scene. As a result, our model naturally handles varying group sizes, complex social interactions, and supports sampling multiple plausible futures. Extensive experiments show that OCSD achieves state-of-the-art results on the Humans in Kitchens (HiK) and HOI-M3
benchmarks. It reduces the two-second path error by 121.5 mm (31.3%) on HiK and 130.5 mm (33.2%) on HOI-M3 compared to prior work, and produces more realistic long-term forecasts.
Read the full abstract ▾
Computer Vision
Differentiable Jitter Correction using Deep Learning-based Image Quality Metric for Phase-Contrast Micro-CT
Junan Chen, Yiting Jia, Joscha Maier et al. (+6 more)
This paper proposes a fully differentiable jitter correction method for X-ray phase-contrast micro computed
tomography using a deep learning-based image quality metric that estimates and compensates per-projection rigid jitter directly from the acquired projection data, without a pre-scan motion-free reference. The approach builds on a gradient-based auto-focus strategy adapted to parallel-beam geometry. A set of candidate objective functions is benchmarked in a controlled study, and the sensitivity of the visual information fidelity (VIF) metric to the jitter artifact is verified with the target phase-contrast data. To operate without a clean reference, a compact 3D convolutional neural network is trained to predict the VIF score from a single corrupted volume. A spatially selective total variation penalty applied exclusively to the image background is introduced to penalize spurious high-frequency structures that otherwise emerge during optimization. Experiments on biological specimens acquired at different synchrotron beamlines are conducted. Evaluation uses jitter motion applied to simulated and experimentally acquired projection data. The result confirms that the integrated pipeline reliably recovers fine structural detail lost due to jitter, with generalization demonstrated across morphologically distinct samples.
Read the full abstract ▾
All authors Junan Chen, Yiting Jia, Joscha Maier, Dominik John, Sami Wirtensohn, Imke Greving, Silja Flenner, Matthias Wieczorek, Julia Herzen
Computer Vision
MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models
Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan et al. (+6 more)
Reliable evaluation of
vision-
language models (VLMs) and medical
vision-
language models (Medical-VLMs) requires calibrated confidence, particularly under realistic clinical conditions. However, existing efforts mainly focused on improving accuracy, leaving calibration in the medical domain underexplored. To this end, we propose MVC-Bench, a calibration-centric
benchmark for medical image classification with VLMs and Medical-VLMs. MVC-Bench assesses the calibration across three axes: (i) robustness to modality, backbone, and domain shift (ii) effectiveness of calibration strategies and
prompt-tuning methods (iii) stability under
prompt-template and random-seed variations. The
benchmark covers eight different backbones, three medical modalities, including fundus imaging, histopathology, and chest X-ray under in-domain and domain shift settings. It compares post-hoc calibration, train-time calibration, and zero-shot inference methods, together with six
prompt-tuning methods. Across more than 1638 controlled experiments, we report accuracy and Expected Calibration Error (ECE) as primary metrics, and further report results with complementary calibration measures, including Maximum Calibration Error (MCE) and Adaptive Calibration Error (ACE). We further investigate the underlying causes of miscalibration in VLMs and Medical-VLMs and propose a simple train-time calibration method, Multi-Class Margin (MCM) regularization, which achieves lowest ECE on 10 out of 12 settings in in-domain and remains competitive under domain shifts. Collectively, MVC-Bench provides a structured evaluation framework and actionable guidance for improving calibration in safety-critical medical workflows.
Read the full abstract ▾
All authors Ashshak Sharifdeen, Shihab Aaqil Ahamed, Ufaq Khan, Muhammad Akhtar Munir, Sujair Ibrahim, Mohamed Rafeek Mareer Ahamed, Yutong Xie, Imran Razzak, Muhammad Haris Khan
Computer Vision
Virtual iEEG from Scalp EEG: Charting the Landscape of Source Imaging, Intracranial Inference and Reconstruction
Dongyi He, Xiangkai Wang, Hongjie Yan et al. (+3 more)
Intracranial electroencephalography (iEEG) provides temporally precise and spatially specific access to neural activity from focal and deep
brain regions, but its invasiveness and restricted anatomical coverage limit routine use. These constraints have motivated scalp-to-intracranial inference, termed virtual iEEG when model outputs carry iEEG-defined event, feature, representation, or contact-level waveform semantics. This review presents a target-centred framework distinguishing event inference, feature translation, and waveform reconstruction, while separating predictability from observability, identifiability, fidelity, and utility. Evidence is evaluated according to cohort independence, anatomical and spectral coverage, train–test separation, and target-patient adaptation. Current studies support inference of selected intracranial events, low-frequency components, and task-related representations, but not unique recovery of arbitrary contact-level activity. Stronger validation requires appropriate controls, source-imaging baselines, uncertainty assessment, and incremental-utility testing. Future progress depends on independent paired
datasets and prospective evidence that virtual iEEG adds value beyond scalp
EEG and
EEG source imaging.
Read the full abstract ▾
All authors Dongyi He, Xiangkai Wang, Hongjie Yan, Luping Song, Wai Ting Siok, Nizhuan Wang
Computer Vision
Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning
Hengyuan Xu, Wei Cheng, Yumeng Ji et al. (+4 more)
Explicit visual intermediates can help
multimodal large
language models (MLLMs) externalize spatial evidence and updated visual states, but their utility depends on whether an image editor can faithfully realize the required transformation. We introduce Aphanta, an automated task-discovery and closed-loop diagnostic framework for the MLLM -> image editor -> MLLM pipeline. Aphanta evaluates three conditions—direct
reasoning,
reasoning with an editor-generated intermediate, and
reasoning with an idealized reference intermediate—to separate potential visual headroom from the practical utility of current editors. Across 20 candidate tasks and multiple editor–MLLM combinations, we find that utility is strongly task-conditioned. Gains concentrate in visual cue injection, grounding, and counterfactual state realization, whereas intermediates requiring symbol-sensitive construction or structural extrapolation are substantially less reliable. On the selected positive-task subset, our consolidated Qwen pipeline improves the mean task score from 0.343 to 0.445 ($+10.2$ points; $+29.7\%$ relative), while the full study also retains filtered and unsuccessful tasks to expose the boundary. These results position image editing as a specialized visual workspace rather than a universal
reasoning mechanism, and establish Aphanta as a reusable protocol for measuring task–representation
alignment, editor realization, and downstream pipeline utility.
Read the full abstract ▾
All authors Hengyuan Xu, Wei Cheng, Yumeng Ji, Xuanyang Zhang, Xianfang Zeng, Gang Yu, Xingjun Ma
Computer Vision
TempJail: Temporal Jailbreak Attacks against Image-to-Video Generation Models
Qi Lu, Zehui Guo, David Yuanda Gan et al. (+4 more)
In recent years, image-to-
video (I2V) generation models have made remarkable progress in subject consistency and temporal coherence, enabling high quality
video synthesis. However, these advances also introduce new safety risks. Existing studies mainly focus on jailbreak attacks involving single frame violations, while largely overlooking the temporal dimension unique to
video generation models. In this paper, we investigate three attack scenarios and uncover a temporal vulnerability in I2V systems: unsafe semantics may emerge not from a single frame, but from semantic composition over time. We further identify two key challenges in such attacks: temporal abstraction and semantic camouflage. To address these issues, we propose TempJail, a novel temporal jailbreak framework for I2V systems. For temporal abstraction, we decompose a target malicious caption into an initial frame visual condition and a temporal text instruction. For semantic camouflage, on the image side we model semantic injection as controlled latent perturbation in
diffusion sampling and introduce gradient guidance from pretrained encoders. On the text side, we rewrite the caption into an innocuous “subject-action-scene” template that bypasses safety filters while preserving temporal guidance. In the black-box inference phase, these two modalities jointly enable malicious semantics to be gradually triggered over time. Experiments on closed-source commercial models, including Kling, Seedance, Veo and PixVerse, show that TempJail improves attack success rate over prior state-of-the-art methods by 23.3% under GPT-5.2 evaluation and 22.0% under human evaluation. Our codes are available at \href{https://github.com/luqi-glory/TempJail}{GitHub}.
Read the full abstract ▾
All authors Qi Lu, Zehui Guo, David Yuanda Gan, Zijing Li, Hengda Zhang, Weijun Xu, Qiankun Zhang
Computer Vision · Cross-listed from cs.LG
ClusterAttention: A training-free speedup of bidirectional attention
Kasper Nordenram and Amelie Dittmann
We introduce ClusterAttention, a general training-free speedup of bidirectional
attention at large
token counts. We point out two common assumptions in contemporary training-free methods;
attention sparsity, and context that can be leveraged, such as structure in the input or multiple similar forward passes, and show when they fail. Our proposed method utilizes a fast
attention-aware recursive clustering method, and compensation of excluded clusters through their mean. The clustering method gives power-of-two cluster sizes, allowing block-sparse
attention to match dense
attention in GPU throughput. On TabPFN-3 arXiv:2605.13986, a model where none of the assumptions hold, ClusterAttention is to our knowledge the first method to provide a substantial speedup over the default
attention, while consistently keeping over 99% of its accuracy. On the largest
dataset from the TALENT
benchmark suite, it makes processing of the training
dataset close to 8x faster at nearly 11x
attention speedup. ClusterAttention is also competitive with domain-specific methods, while avoiding any of the domain-specific engineering. On
video-generation with Wan 2.1-T2V-14B arXiv:2503.20314 it produces output closer to dense
attention at a larger speedup (1.8x vs 1.4x) than SVOO arXiv:2603.18636, a leading method in this domain, with both evaluated without offline calibration.
Read the full abstract ▾
Computer Vision
Geo-LoRA: Geometry-Aware Subspace Evolution for Low-Rank Adaptation in Continual Learning
Yibo Feng
Rehearsal-free class-incremental learning (CIL) with LoRA adapters remains challenging because the low-rank subspaces updated across tasks evolve without
geometric control, causing unstable shared representations and repetitive collapse of task-specific updates into previously occupied directions. We introduce Geo-LoRA, a geometry-aware framework that explicitly regulates how low-rank subspaces, both shared and task-specific, evolve during continual learning. For the shared branch, Subspace Projection Preservation (SPP) constrains consecutive updates to follow smooth trajectories on the Grassmann
manifold, and Adaptive Core-Slack
Alignment (ACSA) decomposes transitions into principal and residual components, aligning the former while modulating the latter to balance stability and plasticity. For the task-specific branch, Median-Calibrated Block Overlap (MCBO) imposes a statistical constraint via normalized projection overlap, penalizing excessive reuse to mitigate subspace crowding. These constraints jointly regulate the
evolution of all LoRA subspaces across layers and tasks without introducing additional adapter types beyond standard LoRA. Geo-LoRA provides a principled
geometric formulation for continual low-rank adaptation and consistently achieves state-of-the-art performance across multiple
benchmark datasets and different task lengths.
Read the full abstract ▾
Computer Vision
RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing
Zijian Kan, Wei Wang, Long Luo et al. (+6 more)
Reward models play an essential role in aligning visual generative models, yet most existing visual reward models use a single scalar score or rely on fixed criteria that cannot adapt to different instructions. This limits both interpretability and task sensitivity, especially for text-to-image generation and instruction-based image editing, where different inputs require different evaluation dimensions. We propose RubricRM, a pairwise generative reward modeling framework that first produces an input-specific rubric with evaluation dimensions, weights, and scoring criteria, and then applies the rubric to score candidate images. We train dedicated RubricRM models for text-to-image generation and image editing using a two-stage training pipeline: supervised fine-tuning teaches the model the rubric-based scoring paradigm, while GRPO further improves scoring through fine-grained dimension-level rewards. Experiments on multiple generation and editing
benchmarks show that RubricRM outperforms existing specialized reward models and remains competitive with strong proprietary MLLM judges despite using smaller backbones. Our models, data, and code are available at https://github.com/zijiankan/RubricRM.
Read the full abstract ▾
All authors Zijian Kan, Wei Wang, Long Luo, Bing Zhao, Xuan Ren, Weixu Qiao, Wenbo Li, Hu Wei, Lin Qu
Computer Vision
Per-View Gaussian Predictions Enable Training-Free Distractor Filtering in Feed-Forward 3DGS
Kangmin Seo and Jae-Pil Heo
Feed-forward 3D Gaussian Splatting reconstructs an explicit Gaussian representation from multiple input images in one network execution, making 3D reconstruction increasingly accessible for casual captures. However, such captures frequently contain transient objects that appear in only a subset of the views. Such content can be encoded into the per-view Gaussians associated with the inputs that observe it and remain in the combined representation despite being observed by no other input. As a result, it may produce blurred, duplicated, or floating artifacts in novel views. We introduce a training-free filtering procedure that exploits this per-view prediction structure. For each input, we exclude its associated Gaussians and render the same camera using the remaining representation, revealing content that is inconsistent with the other inputs. Feature similarity forms candidate regions, and rendering-based verification retains only candidates whose removal reduces reconstruction error in the other input views. The procedure operates on a single frozen prediction without retraining or scene-specific optimization. Across three reconstruction models and two distractor
benchmarks, it consistently improves novel-view quality with varying numbers of input views. On clean scenes, evaluations across four models show that the original reconstructions are largely preserved.
Read the full abstract ▾
Computer Vision
KISS-GS: 3D Gaussian Splatting Compression Kept Simple
Wieland Morgenstern, Friedrich Elias Branschke, Florian Fleischmann et al. (+5 more)
Scene reconstruction with 3D Gaussian Splatting (3DGS) has become common, however deployment remains painful as the uncompressed file sizes can be massive. Current 3DGS compression systems combine multiple strategies for file size reduction, which can obscure where gains come from and limit component reuse across training pipelines. To make the gains more transparent, we propose KISS-GS, a modular compression pipeline named after the principle of keeping things simple, designed to decouple compression entirely from training. Given a 3DGS scene reconstructed with vanilla 3DGS, we are able to reduce it through compaction by 15.7x using a combination of state-of-the-art
pruning schemes. Then we encode it into an image-based format designed for simple, ubiquitous decoding. With the SOG-XT format, we propose a novel extension to Self-Organizing Gaussians with two main contributions: (i) Self-organizing 2D Codebooks and (ii) Parallel Representative Assignment Smoothing (PRAS), which leverages the
symmetry of quaternion and scale parameterizations to produce 2D attribute grids more amenable to encoding. This encoding reduces scene size by 6.6x. We show that optional encoding-aware fine-tuning yields a further 2.2x. Across standard 3DGS
benchmarks, our simple and modular approach thus achieves a total of 85x to 319x reductions in the size of the scene over uncompressed vanilla 3DGS, setting new
benchmarks for real-world scenes and surpassing tightly integrated methods in rate-distortion. Decoding relies solely on web-native image formats, and the modular design makes each stage easy to combine with future advances in reconstruction and compaction. Code and project page: https://fraunhoferhhi.github.io/KISS-GS/
Read the full abstract ▾
All authors Wieland Morgenstern, Friedrich Elias Branschke, Florian Fleischmann, Adrian Szatmari, Paul Schlack, Florian Barthel, Peter Eisert, Anna Hilsmann
Computer Vision · Cross-listed from cs.RO
4DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulation
Zehao Qi, Haochen Luo, Jia-Wang Bian et al. (+2 more)
Embodied agents need environments that are visually diverse, physically interactive, and changing over time. Procedural simulators can generate large interactive scene collections, and recent 4D generators produce compelling visual dynamics. Combining these properties in one environment, however, still demands extensive manual effort, and the result is rarely editable or controllable enough to reuse at scale. We present 4DSynth, a controllable procedural system that turns a natural-
language description, a blueprint mask, or a single photograph into an editable 4D environment with explicit geometry, animated actors, collision-free trajectories, and physics-ready simulation state. Multiple scene routes share one geometry-grounded representation, so the same pipeline handles animation, camera planning, rendering, and task generation. To validate the full pipeline, we construct 4DSynth-Nav, an interactive
navigation benchmark generated entirely from 4DSynth's procedural scenes. Two
vision-
language models evaluated across three difficulty tiers both fail the majority of tasks and stall after early subtasks. The same procedural controllability that produces these environments also makes each failure reproducible and each difficulty axis independently tunable. This paper presents both a controllable generation pipeline and the scalable
benchmark it enables, offering a practical foundation for developing and evaluating
embodied agents.
Read the full abstract ▾
All authors Zehao Qi, Haochen Luo, Jia-Wang Bian, Zeyu Ma, Shuyang Sun
Computer Vision
AraMS-28k: The Largest Publicly Released Line-Level Dataset of Historical Arabic Manuscripts with Margin and Insertion-Anchor Annotations
Mohamed Guechaoui, Mohamed Diaa Zellagui, Souleyman Chaib et al. (+1 more)
We introduce AraMS-28k, the largest publicly released line-level
dataset of genuine historical Arabic manuscripts, comprising 14 books, 3,043 pages, and 28,600 annotated text lines (27,971 main-text, 629 margin). Thirteen books are hand-copied manuscripts spanning three script traditions – Naskh, Ruq'ah, and Maghrebi – and one is a lithographed printed edition included to broaden format diversity. Each line is labelled as main-text or margin, and margin lines that have an unambiguous attachment point in the main text are further annotated with an insertion anchor, recovering the manuscript's true non-linear reading order at line-level granularity – to our knowledge the first such annotation released for a historical Arabic manuscript corpus. Because reference transcriptions are fully vocalised while manuscript hands are typically undiacritised, we release both the raw diacritised transcription and a diacritic-normalised counterpart for every line. The
dataset was constructed with RefLAM, a reference-grounded annotation pipeline that aligns
multimodal-LLM OCR against independently sourced clean transcriptions and routes every line through human review, combining automatic verification with expert oversight. We describe the construction and quality-control process, present the annotation schema, report
dataset statistics at both the corpus and per-book level, and provide baseline HTR results using Kraken and HATFormer, including a cross-script generalisation gradient from in-distribution pages to fully unseen books. AraMS-28k is released with page images, line-level annotations, and fixed train/val/test splits under CC BY-NC-SA 4.0 to support reproducible research on Arabic manuscript recognition, layout analysis, and reading-order recovery.
Read the full abstract ▾
All authors Mohamed Guechaoui, Mohamed Diaa Zellagui, Souleyman Chaib, Sahraoui Dhelim
Computer Vision · Cross-listed from stat.AP
Climate Physics Dynamic Matching
Gurjeet Sangra Singh, Frantzeska Lavda and Alexandros Kalousis
Deep generative models such as flow matching and
diffusion models have shown potential for learning complex dynamical systems, but typically act as black boxes that neglect underlying physical structure, while physics-based models governed by partial differential equations are often incomplete due to missing source terms, or uncertain parametrisations. We present
Climate Physics Dynamic Matching (ClimPhyDM), a variational simulation-free dynamics informed framework for weather
forecasting that combines an advection-type physics prior with data-driven components in a variational framework. % to capture the stochasticity and multi-modality of unresolved atmospheric dynamics. On the ERA5
benchmark at hourly (42-hour) and monthly (5-month) resolutions, ClimPhyDM outperforms ClimODE, and GB-DM, keeping the lower error at extended horizon, indicating improved temporal stability and resistance to error accumulation, while its simulation-free paradigm also enables training on a single modest 12 GB consumer GPU.
Read the full abstract ▾
Computer Vision
Tether the Subject, Release the Scene: Query-Aware Memory Routing for Long-Horizon Autoregressive Video Generation
Chen Li, Peng Zhang, Hanyu Zhou et al. (+5 more)
Streaming autoregressive
video models generate long
videos chunk by chunk, using historical
memory to maintain consistency. Existing methods typically expose subject and scene queries to history through similar policies. This stabilizes the subject, but can also lock backgrounds, viewpoints, and scene structure to previously generated states even when local motion continues. We call this failure
memory-anchored scene under-progression; consistency and motion metrics alone can miss it. We introduce TetherMem, a training-free, query-aware spatiotemporal
memory router for frozen
video generators. TetherMem separates subject and scene queries and modulates historical access with region- and age-conditioned priors: subject queries retain identity-bearing history, while scene queries reduce reliance on subject history and stale backgrounds. Across 2,400 blinded pairwise judgments from 10 annotators, TetherMem achieves the highest estimated expected preference among eight streaming long-
video baselines for overall quality (0.780) and scene progression (0.769). On complete 30-second
videos, it sustains changes in background, viewpoint, and scene state while preserving subject recognizability and temporal continuity.
Read the full abstract ▾
All authors Chen Li, Peng Zhang, Hanyu Zhou, Jialong Zuo, Fei Wang, Daiguo Zhou, Nong Sang, Changxin Gao