Computer Vision
CRISP: Compositional Relations as Invariant Structural Priors for Domain Generalization
Dat Nguyen and Duc-Duy Nguyen
Domain generalization requires identifying stable representations that support reliable classification across domains. Domains may differ in low-level attributes, such as color, texture, or visual style, while preserving the same structural relationships among their underlying components. Existing methods primarily address these differences by improving the training process or aligning features across domains. However, since they leave this shared compositional structure implicit, they may overlook a more reliable source of invariance and consequently generalize less effectively to unseen domains. We propose Compositional Relational Invariance from Spatial Primitives (CRISP), an image classification framework that factors visual recognition into visual primitives and their relational composition. We represent these compositions using soft unary, binary, and ternary predicates over primitive locations and appearance, yielding differentiable measures of spatial and visual
alignment that can be learned end-to-end. To learn primitives and relational structure jointly, we design an end-to-end architecture with three components: (1) a visual backbone that extracts generalized features, (2) a concept bottleneck layer that maps these features to primitive heatmaps with differentiable spatial coordinates, and (3) a structural scoring layer that evaluates candidate spatial relations among the detected primitives. Finally, we compute class probability from the joint evidence of its class-specific relational compositions and localized primitive appearance. We evaluate \method{} on five real-world image-classification
datasets from the widely used DomainBed suite, covering shifts in depiction style,
dataset provenance, and camera-trap location and achieving the new state-of-the-art on both
benchmarks.
Read the full abstract ▾
Computer Vision
PlotPick: AI-powered batch extraction of numerical data from scientific figures
Tommy Carstensen
Systematic reviews and meta-analyses often need numerical data reported only in figures, and extracting them with interactive digitisers usually requires a person to select and calibrate each figure. We present PlotPick, an open-source tool that uses
vision-
language models (VLMs) to extract tabular data from batches of scientific figures, and we
benchmark the kind of model it calls: nine VLMs from four providers on ChartX and six of them on PlotQA, against DePlot, a dedicated chart-to-table model, with every system scored on the same items by numeric F1 (F1 over unlabelled numbers at 5% relative tolerance). On six ChartX chart types (n=300) all nine VLMs outperform DePlot in aggregate, at 79.1-96.0% against 74.3%. The lead comes mainly from box plots, where DePlot scores 24.8% against 64.2-97.3%; pooled over the other five types, seven VLMs keep a lead of 4.7 to 11.6 points and the two weakest do not. On a subset of the PlotQA test split (n=529; 427 horizontal bar charts), scored by a lenient best-series variant of the metric, DePlot reaches 87.0%; the two strongest VLMs are level with it or slightly above it, and four fall 3.8 to 30.3 points below. DePlot was trained on PlotQA's training split. Both
benchmarks use synthetic charts, and the metric ignores which series a value belongs to. The application itself was not evaluated: its figure detection, structured output,
prompt and default model were not tested, and the one benchmarked model it offers, Claude Haiku 4.5, is one of the four below DePlot on PlotQA. Accuracy on biomedical figures has not been established, and every extracted value must be checked against its source figure. This version corrects version 1, which scored most PlotQA replies against category labels instead of plotted values; its claim that every VLM outperformed DePlot on both
benchmarks is withdrawn. PlotPick is available at https://plotpick.streamlit.app/.
Read the full abstract ▾
Computer Vision
T2I-VeRW: Part-level Fine-grained Perception for Text-to-Image Vehicle Retrieval
Xiao Wang, Ziwen Wang, Weizhe Kong et al. (+5 more)
Vehicle Re-identification (Re-ID) aims to retrieve the most similar image to a given query from images captured by non-overlapping cameras. Extending vehicle Re-ID from image-only queries to text-based queries enables
retrieval in real-world scenarios where only a witness description of the target vehicle is available. In this paper, we propose PFCVR, a Part-level Fine-grained Cross-modal Vehicle
Retrieval model for text-to-image vehicle re-identification. PFCVR constructs locally paired images and texts at the part level and introduces learnable part-query
tokens that aggregate both part-specific and full-sentence context before aligning with visual part features. On top of this explicit local
alignment, a bi-directional mask recovery module lets each modality reconstruct its masked content under the guidance of the other, implicitly bridging local correspondences into global feature
alignment. Furthermore, we construct a new large-scale
dataset called T2I-VeRW, which contains 14,668 images covering 1,796 vehicle identities with fine-grained part-level annotations. Experimental results on the T2I-VeRI
dataset show that PFCVR achieves 29.2% Rank-1 accuracy, improving over the best competing method by +3.7% percentage points. On the newly proposed T2I-VeRW
benchmark, PFCVR achieves 55.2% Rank-1 accuracy, outperforming a comprehensive set of recent state-of-the-art methods. Source code will be released on https://github.com/Event-AHU/Neuromorphic_ReID
Read the full abstract ▾
All authors Xiao Wang, Ziwen Wang, Weizhe Kong, Wentao Wu, Yuehang Li, Aihua Zheng, Chenglong Li, Jin Tang
Computer Vision
Adding Thermal Awareness to Visual Systems in Real-Time via Distilled Diffusion Models
Yuchen Guo, Junli Gong, Wenjun Dong et al. (+2 more)
Purely RGB-based
vision models often fail to provide reliable cues in challenging scenarios such as nighttime and fog, leading to degraded performance and safety risks. Infrared imaging captures heat-emitting sources and provides critical complementary information, but existing high-fidelity fusion methods suffer from prohibitive latency, rendering them impractical for real-time edge deployment. To address this, we propose FusionProxy, a real-time image fusion module designed as a fully independent, plug-and-play component with
diffusion level quality. FusionProxy exploits two complementary statistics of a teacher sample ensemble: per-pixel variance in raw image space, used to weight pixel-level supervision, and per-pixel variance inside frozen foundation backbones, used to route feature-level
alignment spatially. Once trained, FusionProxy can be directly integrated into any visual perception system without joint optimization. Extensive experiments demonstrate that our method achieves superior performance on static recognition tasks and significantly enhances robustness in dynamic tasks, including closed-loop autonomous driving. Crucially, FusionProxy achieves real-time inference speeds on diverse platforms, from high-end GPUs to commodity hardware, providing a flexible and generalizable solution for all-day perception.
Read the full abstract ▾
All authors Yuchen Guo, Junli Gong, Wenjun Dong, Yiuming Cheung, Weifeng Su
Computer Vision
Neuromorphic visual attention for Sign-language recognition on SpiNNaker
Sarka Liskova, Olha Vedmedenko, Mazdak Fatahi et al. (+3 more)
Sign-
language recognition has achieved substantial gains in classification accuracy in recent years; however, the latency and power requirements of most existing methods limit their suitability for real-time deployment. Neuromorphic sensing and processing offer an alternative paradigm based on sparse, event-driven computation that supports low-latency and energy-efficient perception. In this work, we introduce an end-to-end neuromorphic architecture for American Sign
Language (ASL) fingerspelling recognition that integrates a spiking visual
attention mechanism for online region-of-interest extraction with a compact spiking neural network deployed on the SpiNNaker neuromorphic platform. We
benchmark the proposed system against two
datasets: a synthetically generated event-based version of the Sign
Language MNIST
dataset and a natively recorded ASL-DVS
dataset, whilst providing a comprehensive overview of Sign-
language recognition and related work. This work yields competitive performance in simulation (92.27%) and comparable performance on neuromorphic hardware deployment (83.1%), while achieving the most energy-efficient architecture (0.565 mW) and low latency (3 ms) across all benchmarked approaches. Despite its compact design, the system demonstrates the suitability of task-dependent visual
attention applications for edge deployment.
Read the full abstract ▾
All authors Sarka Liskova, Olha Vedmedenko, Mazdak Fatahi, Matej Hoffmann, P. Michael Furlong, Giulia D Angelo
Computer Vision
4DThinker: Thinking with 4D Imagery for Dynamic Spatial Understanding
Zhangquan Chen, Manyuan Zhang, Xinlei Yu et al. (+9 more)
Dynamic spatial
reasoning from
monocular video is essential for bridging visual intelligence and the physical world, yet remains challenging for
vision-
language models (VLMs). Prior approaches either verbalize spatial-temporal
reasoning entirely as text, which is inherently verbose and imprecise for complex dynamics, or rely on external
geometric modules that increase inference complexity without fostering intrinsic model capability. In this paper, we present 4DThinker, the first framework that enables VLMs to "think with 4D" through dynamic latent mental imagery, i.e., internally simulating how scenes evolve within the continuous hidden space. Specifically, we first introduce a scalable, annotation-free data generation pipeline that synthesizes 4D
reasoning data from raw
videos. We then propose Dynamic-Imagery Fine-Tuning (DIFT), which jointly supervises textual
tokens and 4D latents to ground the model in dynamic visual semantics. Building on this, 4D Reinforcement Learning (4DRL) further tackles complex
reasoning tasks via outcome-based rewards, restricting policy gradients to text
tokens to ensure stable optimization. Extensive experiments across multiple dynamic spatial
reasoning benchmarks demonstrate that 4DThinker consistently outperforms strong baselines and offers a new perspective toward 4D
reasoning in VLMs. Our code is available at https://github.com/zhangquanchen/4DThinker.
Read the full abstract ▾
All authors Zhangquan Chen, Manyuan Zhang, Xinlei Yu, Xiang An, Bo Li, Xin Xie, ZiDong Wang, Mingze Sun, Shuang Chen, Hongyu Li, Xiaobin Hu, Ruqi Huang
Computer Vision
iPhoneBlur: A Difficulty-Stratified Benchmark for Consumer Device Motion Deblurring
Abdullah Al Shafi and Kazi Saeed Alam
Motion blur restoration on consumer mobile devices is typically evaluated using aggregate metrics that obscure performance variation across blur difficulty, masking model behavior under real deployment conditions. This work introduces iPhoneBlur, a difficulty-stratified
benchmark of 7,400 image pairs synthesized from high-framerate iPhone 17 Pro
videos captured in diverse real-world scenarios. Samples are partitioned into Easy, Medium, and Hard categories through PSNR-guided adaptive temporal windowing, with stratification validated by monotonic 2.2x increase in optical flow magnitude across tiers. Each sample includes comprehensive metadata enabling investigation of ISP-aware and difficulty-adaptive restoration strategies. Spectral analysis confirms synthesized blur exhibits high-frequency suppression patterns consistent with authentic motion degradation. Evaluation of six architectures reveals consistent 7-9 dB performance degradation from Easy to Hard subsets, a substantial gap entirely hidden by aggregate reporting. The
benchmark further exposes a domain gap between professional and consumer cameras which targeted fine-tuning substantially recovers. By coupling difficulty stratification with deployment-critical metadata, iPhoneBlur enables systematic assessment of model reliability and failure modes for resource-constrained edge systems.
Read the full abstract ▾
Computer Vision
Prompt-Free and Efficient SAM2 Adaptation for Biomedical Semantic Segmentation via Dual Adapters
Hinako Mitsuoka and Kazuhiro Hotta
Segment Anything Model 2 (SAM2) demonstrated impressive zero-shot capabilities on natural images but faces challenges in biomedical
segmentation due to significant domain shifts and
prompt dependency. To address these limitations, we propose a
prompt-free, parameter-efficient fine-tuning framework designed for multi-class
segmentation on variable-sized inputs. We introduce a convolutional Positional Encoding Generator to adapt effectively to arbitrary aspect ratios and present a dual-adapter strategy: High-Performance Adapter utilizing deformable convolutions for precise boundary modeling and Lightweight Adapter employing structural re-parameterization to minimize inference latency. Experiments on ISBI 2012, Kvasir-SEG, Synapse, and ACDC
datasets demonstrate that our approach significantly outperforms strong adaptation baselines. Specifically, our method improved
segmentation accuracy by up to 19.66% over the vanilla SAM2, while reducing computational costs by approximately 87% compared to heavyweight medical SAM adaptations, establishing a superior trade-off between accuracy and efficiency.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
TableVista: Benchmarking Multimodal Table Reasoning under Visual and Structural Complexity
Zheyuan Yang, Liqiang Shang, Junjie Chen et al. (+6 more)
We introduce TableVista, a comprehensive
benchmark for evaluating foundation models in
multimodal table
reasoning under visual and structural complexity. TableVista consists of 3,000 high-quality table
reasoning problems, where each instance is expanded into 10 distinct visual variants through our multi-style rendering and transformation pipeline. This process encompasses diverse scenario styles, robustness perturbations, and
vision-only configurations, culminating in 30,000
multimodal samples for a multi-dimensional evaluation. We conduct an extensive evaluation of 29 state-of-the-art open-source and proprietary foundation models on TableVista. Through comprehensive quantitative and qualitative analysis, we find that while evaluated models remain largely stable across diverse rendering styles, they exhibit pronounced performance degradation on complex structural layouts and
vision-only settings, revealing that current models struggle to maintain
reasoning consistency when structural complexity combines with visually integrated presentations. These findings highlight critical gaps in current
multimodal capabilities, providing insights for advancing more robust and reliable table understanding models.
Read the full abstract ▾
All authors Zheyuan Yang, Liqiang Shang, Junjie Chen, Xun Yang, Chenglong Xu, Bo Yuan, Chenyuan Jiao, Yaoru Sun, Yilun Zhao
Computer Vision
MobileEgo Anywhere: Open Infrastructure for long horizon egocentric data on commodity hardware
Senthil Palanisamy, Abhishek Anand, Satpal Singh Rathore et al. (+3 more)
Vision-
language-action (VLA) models have driven demand for large-scale
egocentric datasets, yet the hardware and infrastructure to collect long-horizon data remain inaccessible.
Datasets today typically have episodes only a few minutes long, which fails to capture the long-horizon temporal dependencies that complex robotic task execution requires. We present MobileEgo Anywhere, a framework for collecting hour-plus
egocentric trajectories on commodity mobile hardware that uses modern smartphone sensors for long-term pose tracking without the hardware barriers of traditional robotics data collection. We release three components: (1) STERA, an open-source
video-processing pipeline that converts raw mobile captures into standardized, training-ready formats for VLA and foundation-model research; (2) a free mobile app that lets any user record
egocentric activity; and (3) a 200-hour
dataset of diverse, long-form
egocentric data with persistent state tracking across 584 sessions. We further show this data is a usable training signal:mid-training a VLA on it lowers held-out action-prediction error.
Read the full abstract ▾
All authors Senthil Palanisamy, Abhishek Anand, Satpal Singh Rathore, Pratyush Patnaik, Shubhanshu Khatana, Ekaksh Janweja
Computer Vision
RAWild: Sensor-Agnostic RAW Object Detection via Physics-Guided Curve and Grid Modeling
Shuhong Liu, Gengjia Chang, Jun Liu et al. (+4 more)
Camera sensor RAW data offers intrinsic advantages for object detection, including deeper bit depth, preserved physical information, and freedom from image signal processor (ISP) distortions. However, varying exposure conditions, spectral sensitivities, and bit depths across devices introduce substantially larger domain gaps than sRGB, making sensor-agnostic generalization a fundamental challenge. In this study, we present RAWild, a physics-guided global-local tone mapping framework for sensor-agnostic RAW object detection. By factoring sensor-induced variations into a global tonal correction and a spatially adaptive local color adjustment, both driven by RAW distribution priors, our framework enables a single network to train jointly across heterogeneous sensors. To further support cross-sensor generalization, we construct a physics-based RAW simulation pipeline that synthesizes realistic sensor outputs spanning diverse spectral sensitivities, illuminants, and sensor non-idealities. Extensive experiments across multiple RAW
benchmarks covering bit depths from 10 to 24 demonstrate state-of-the-art (SOTA) performance under single-
dataset, mixed-
dataset, and challenging robustness settings.
Read the full abstract ▾
All authors Shuhong Liu, Gengjia Chang, Jun Liu, Xuangeng Chu, Yinqiang Zheng, Tatsuya Harada, Ziteng Cui
Computer Vision
Whole-body CT attenuation and volume charts from routine clinical scans via evidence-grounded LLM report filtering
Christian Wachinger, Bernhard Renger, Christopher Späth et al. (+2 more)
Interpreting quantitative CT biomarkers, such as organ volume and tissue attenuation, requires large-scale healthy reference distributions. However, creating these is challenging because clinical
datasets are often heavily enriched with pathology. Here, we develop an evidence-grounded, cross-verified large
language model (LLM) ensemble to filter pathological findings from radiology reports, enabling the construction of pathology-reduced cohorts from over 350,000 CT examinations. Five LLMs, first, flag structure-level abnormality candidates grounded in verbatim report evidence and, second, resolve disagreements via cross-verification. Using distribution-aware generalized additive models for location, scale, and shape, we establish comprehensive whole-body reference charts for 106 anatomical structures (volumes and attenuation) across adulthood, accounting for age, sex, contrast enhancement, and acquisition parameters. Longitudinal analyses reveal structure- and contrast-dependent changes distinct from cross-sectional trends. These resources facilitate
covariate-adjusted centile scoring from routine CT, supporting standardized quantitative phenotyping, multi-site imaging studies, and scalable opportunistic screening research.
Read the full abstract ▾
All authors Christian Wachinger, Bernhard Renger, Christopher Späth, Jan Kirschke, Marcus Makowski
Computer Vision
Backdoor Mitigation in Object Detection via Adversarial Fine-Tuning
Kealan Dunnett, Reza Arablouei, Dimity Miller et al. (+2 more)
Backdoor attacks can implant malicious behaviours into deep models while preserving performance on clean data, posing a serious threat to safety-critical
vision systems. Although
backdoor mitigation has been studied extensively for image classification, defenses for object detection remain comparatively underdeveloped.
Adversarial fine-tuning is a common
backdoor mitigation approach in classification, but adapting it to detection is nontrivial as classification-oriented
adversarial generation does not match the detection attack space, where attacks may cause object misclassification or disappearance, and standard detection losses can dilute the repair signal across many predictions. We address these challenges through a detection-aware
adversarial fine-tuning framework for mitigating object-detection
backdoors when the defender has access only to a compromised detector and a small clean
dataset, without knowing the attack objective. For
adversarial generation that does not require knowledge of the attack objective, we introduce soft-branch minimisation, which uses a soft gate to combine objectives aligned with misclassification and disappearance attacks, together with a detection-aware classification-loss maximisation. For targeted repair, we introduce a dual-objective fine-tuning loss applied to target-matched predictions, concentrating the defensive update on predictions most relevant to the
backdoor behaviour. Experiments across CNN- and
Transformer-based detectors show that our approach more effectively reduces attack success while preserving true detections, compared with classification-oriented baselines, and maintains competitive clean detection performance.
Read the full abstract ▾
All authors Kealan Dunnett, Reza Arablouei, Dimity Miller, Volkan Dedeoglu, Raja Jurdak
Computer Vision
Think, then Score: Decoupled Reasoning and Scoring for Video Reward Modeling
Yuan Wang, Ouxiang Li, Yulong Xu et al. (+8 more)
Recent advances in generative
video models are increasingly driven by post-training and test-time scaling, both of which critically depend on the quality of
video reward models (RMs). An ideal reward model should predict accurate rewards that align with human preferences across diverse scenarios. However, existing paradigms face a fundamental dilemma: Discriminative RMs regress rewards directly on features extracted by
multimodal large
language models (MLLMs) without explicit
reasoning, making them prone to shortcut learning and heavily reliant on massive data scaling for generalization. In contrast, Generative RMs with Chain-of-Thought (CoT)
reasoning exhibit superior interpretability and generalization potential, as they leverage fine-grained semantic supervision to internalize the rationales behind human preferences. However, they suffer from inherent optimization bottlenecks due to the coupling of
reasoning and scoring within a single autoregressive inference chain. To harness the generalization benefits of CoT
reasoning while mitigating the training instability of coupled
reasoning and scoring, we introduce DeScore, a training-efficient and generalizable
video reward model. DeScore employs a decoupled “think-then-score” paradigm: an MLLM first generates an explicit CoT, followed by a dedicated discriminative scoring module consisting of a learnable query
token and a regression head that predicts the final reward. DeScore is optimized via a two-stage framework: (1) a discriminative cold start incorporating a random mask mechanism to ensure robust scoring capabilities, and (2) a dual-objective reinforcement learning stage that independently refines CoT
reasoning quality and calibrates the final reward, ensuring that higher-quality
reasoning directly translates to superior model performance.
Read the full abstract ▾
All authors Yuan Wang, Ouxiang Li, Yulong Xu, Borui Liao, Jiajun Liang, Jinghan Li, Meng Wang, Xintao Wang, Pengfei Wan, Kuien Liu, Xiang Wang
Computer Vision · Cross-listed from cs.LG
From Drops to Grid: Noise-Aware Spatio-Temporal Neural Process for Rainfall Estimation
Rafael Pablos Sarabia, Joachim Nyborg, Morten Birk et al. (+1 more)
High-resolution rainfall observations are crucial for weather
forecasting, water management, and hazard mitigation. Traditional operational measurements are often biased and low-resolution, limiting their ability to capture local rainfall. Accurate high-resolution rainfall maps require integrating sparse surface observations, yet existing deep learning densification methods are hindered by rainfall's skewed, localized nature, noise, and limited spatio-temporal fusion. We present DropsToGrid, a Neural Process-based method that generates dense rainfall fields by fusing temporal sequences from noisy, irregularly distributed private weather stations with spatial context from radar. Leveraging multi-scale feature extraction, temporal
attention, and multi-modal fusion, the model produces
stochastic, continuous rainfall estimates and explicitly quantifies uncertainty. Evaluations on real-world
datasets demonstrate that DropsToGrid outperforms both operational and deep learning baselines, generating accurate high-resolution rainfall maps with well-calibrated uncertainty, even when only few stations are available and in cross-regional scenarios.
Read the full abstract ▾
All authors Rafael Pablos Sarabia, Joachim Nyborg, Morten Birk, Ira Assent
Computer Vision
Plug-and-play Class-aware Knowledge Injection for Prompt Learning with Visual-Language Model
Junhui Yin, Nan Pu, Xinyu Zhang et al. (+4 more)
Prompt learning has become an effective and widely used technique in enhancing
vision-
language models (VLMs) such as CLIP for various downstream tasks, particularly in zero-shot classification within specific domains. Existing methods typically focus on either learning class-shared
prompts for a given domain or generating instance-specific
prompts through conditional
prompt learning. While these methods have achieved promising performance, they often overlook class-specific knowledge in
prompt design, leading to suboptimal outcomes. The underlying reasons are: 1) class-specific
prompts offer more fine-grained supervision compared to coarse class-shared
prompts, which helps prevent misclassification of data from different classes into a single class; 2) compared to class-specific
prompts, instance-specific
prompts neglect the richer class-level information across multiple instances, potentially causing data from the same class to be divided into multiple classes. To effectively supplement the class-specific knowledge into existing methods, we propose a plug-and-play Class-Aware Knowledge Injection (CAKI) framework. CAKI comprises two key components, i.e., class-specific
prompt generation and query-key
prompt matching. The former encodes class-specific knowledge into
prompts from few-shot samples that belong to the same class and stores the learned
prompts in a class-level knowledge bank. The latter provides a plug-and-play mechanism for each test instance to retrieve relevant class-level knowledge from the knowledge bank and inject such knowledge to refine model predictions. Extensive experiments demonstrate that our CAKI effectively improves the performance of existing methods on base and novel classes. Code is publicly available at \href{https://github.com/yjh576/CAKI}{this https URL}.
Read the full abstract ▾
All authors Junhui Yin, Nan Pu, Xinyu Zhang, Lingfeng Yang, Lin Wu, Xiaojie Wang, Zhun Zhong
Computer Vision
Architecture-agnostic Lipschitz-constant Bayesian header and its application to resolve semantically proximal classification errors with vision transformers
Frederik Schäfer, Luis Mandl, Lars Kälber et al. (+1 more)
Label noise remains a critical bottleneck for the generalization of supervised deep learning models, particularly when errors are structured rather than random. Standard robust training methods often fail in the presence of such semantically proximal classification errors. This work presents an architecture-agnostic Lipschitz-constant
Bayesian header that can be integrated into feature extractors such as
vision transformers, yielding the bi-Lipschitz-constrained
Bayesian Vision Transformer (LipB-ViT). In contrast to conventional
Bayesian layers, our approach enforces spectral normalization on both the mean and log-variance of the variational weights, which promotes calibrated predictive uncertainty and mitigates noise amplification. We further propose a novel metric to jointly capture uncertainty and confidence across misclassification rates, as well as an adaptive arithmetic-mean fusion scheme that combines feature-space proximity with predictive uncertainty to detect corrupted labels outperforming the state of the art k-nearest neighbor based identification methods by more than 7% reaching a recall of more than 0.93 at 15% semantically misclassified labels. Although computational costs increase due to Monte Carlo sampling, the method offers plug-and-play compatibility with pre-trained backbones and consistent hyperparameters across domains, suggesting strong utility for high-stakes applications with variable annotation reliability. The stabilized confidence estimates serve as the foundation for an analysis pipeline that jointly assesses
dataset quality and label noise, yielding a second novel metric for their combined quantification. Lastly, we systematically evaluate LipB-ViT under both structured (
adversarial) and unstructured noise at inference time, demonstrating its robustness in realistic high-noise and attack scenarios. We compare its performance against baseline methods.
Read the full abstract ▾
All authors Frederik Schäfer, Luis Mandl, Lars Kälber, Tim Ricken
Computer Vision
Understanding Cross-Language Transfer Improvements in Low-Resource HTR: The Role of Sequence Modeling
Sana Al-azzawi, Chang Liu, Nudrat Habib et al. (+2 more)
Handwritten Text Recognition (HTR) for Arabic-script
languages benefits from cross-
language joint training under low-resource conditions, particularly when using CRNN-based models that combine convolutional encoders with sequence modeling. However, it remains unclear whether these improvements are better explained by shared visual representations or sequence-level dependencies. In this work, we conduct a controlled architectural study of line-level Arabic-script HTR, comparing CNN-only models with CTC decoding and CRNN models under identical single-script and multi-script training regimes. Experiments are performed on Arabic (KHATT), Urdu (NUST-UHWR), and Persian (PHTD)
datasets under low-resource settings (K in {100, 500, 1000}). Our results show a clear divergence in transfer behavior: while CNN-only models exhibit limited or unstable improvements, CRNN models achieve better performance under multi-script training, particularly in the most data-constrained regimes. Focusing on transfer improvements (delta CER) rather than absolute performance, we find that cross-
language improvements are associated with sequence-level modeling, while sharing visual representations learned by the CNN encoder, corresponding to similarities in character shapes across scripts, alone appears to be insufficient. This finding suggests that contextual modeling plays an important role in enabling effective transfer in low-resource scenarios, and that similar behavior may extend to other low-resource
language settings.
Read the full abstract ▾
All authors Sana Al-azzawi, Chang Liu, Nudrat Habib, Elisa Barney, Marcus Liwicki
Computer Vision
Detecting AI-Generated Videos with Spiking Neural Networks
Minsuk Jang, Yujin Yang, Hee-Seon Kim et al. (+3 more)
Modern AI-generated
videos are photorealistic at the single-frame level, leaving inter-frame dynamics as the main remaining axis for detection. Existing detectors typically handle this temporal evidence in three ways: feeding the full frame sequence to a generic temporal backbone, reducing one dominant temporal cue to fixed
video-level descriptors, or comparing temporal features to real-
video statistics through a detection metric. These strategies degrade sharply under cross-generator evaluation, where artifact type and timescale vary across generators. On caption-paired
benchmark, GenVidBench, we identify two signatures that prior detectors do not jointly exploit: AI-generated
videos exhibit smoother frame-to-frame temporal residuals at the pixel level, and more compact trajectories in the semantic feature space, indicating a temporal smoothness gap at both levels. We further observe that, when raw
video is fed into a Spiking Neural Networks (SNNs), fake clips elicit firing predominantly at object and motion boundaries, unlike real clips, suggesting that the SNN responds to temporal artifacts localized at edges. These cues are sparse, asynchronous, and concentrated at moments of change, which makes SNNs a natural choice for this task: their event-driven, sparsely-activated dynamics align with the structure of the residual signal in a way that dense ANN backbones do not. Building on this observation, we propose MAST, a detector that processes multi-channel temporal residuals with a spike-driven temporal branch alongside a frozen semantic encoder for cross-generator generalization. On the GenVideo
benchmark, MAST achieves 93.14% mean accuracy across 10 unseen generators under strict cross-generator evaluation, matching or surpassing the strongest ANN-based detectors and demonstrating the practical applicability of SNNs to AI-generated
video detection.
Read the full abstract ▾
All authors Minsuk Jang, Yujin Yang, Hee-Seon Kim, Minseok Son, Younghun Kim, Changick Kim
Computer Vision
MTL-MAD: Multi-Task Learners are Effective Medical Anomaly Detectors
Bogdan Alexandru Bercean, Florinel Alin Croitoru, Vlad Hondru et al. (+3 more)
Anomaly detection in medical images is a challenging task, since anomalies are not typically available during training. Recent methods leverage a single pretext task coupled with a large-scale pre-trained model to reach state-of-the-art performance. Instead, we propose to learn multiple self-supervised and pseudo-labeling tasks from scratch, using a joint model based on Mixture-of-Experts (MoE). By carefully integrating multiple proxy tasks, the joint model effectively learns a robust representation of normal anatomical structures, so that anomaly scores can be derived based on how well the multi-task learner (MTL) solves each task during inference. We perform comprehensive experiments on BMAD, a recent
benchmark that comprises a broad range of medical image modalities. The empirical results indicate that our multi-task learner is an effective anomaly detector, outperforming all state-of-the-art competitors on BMAD. Moreover, our model produces interpretable anomaly maps, potentially helping physicians in providing more accurate diagnoses.
Read the full abstract ▾
All authors Bogdan Alexandru Bercean, Florinel Alin Croitoru, Vlad Hondru, Ciprian Mihai Ceausescu, Andreea Iuliana Ionescu, Radu Tudor Ionescu
Computer Vision
DBMSolver: A Training-free Diffusion Bridge Sampler for High-Quality Image-to-Image Translation
Sankarshana Venugopal, Mohammad Mostafavi and Jonghyun Choi
Diffusion-based image-to-image (I2I) translation excels in high-fidelity generation but suffers from slow sampling in state-of-the-art
Diffusion Bridge Models (DBMs), often requiring dozens of function evaluations (NFEs). We introduce DBMSolver, a training-free sampler that exploits the semi-linear structure of DBM's underlying SDE and ODE via exponential integrators, yielding highly-efficient 1st- and 2nd-order solutions. This reduces NFEs by up to 5x while boosting quality (e.g., FID drops 53% on DIODE at 20 NFEs vs. 2nd-order baseline). Experiments on inpainting, stylization, and semantics-to-image tasks across resolutions up to 256x256 show DBMSolver sets new SOTA efficiency-quality tradeoffs, enabling real-world applicability. Our code is publicly available at https://github.com/snumprlab/dbmsolver.
Read the full abstract ▾
Computer Vision
Training-Free Dense Hand Contact Estimation with Multi-Modal Large Language Models
Daniel Sungho Jung and Kyoung Mu Lee
Dense hand contact estimation requires both high-level semantic understanding and fine-grained
geometric reasoning of human interaction to accurately localize contact regions. Recently, multi-modal large
language models (MLLMs) have demonstrated strong capabilities in understanding visual semantics, enabled by
vision-
language priors learned from large-scale data. However, leveraging MLLMs for dense hand contact estimation remains underexplored. There are two major challenges in applying MLLMs to dense hand contact estimation. First, encoding explicit 3D hand geometry is difficult, as MLLMs primarily operate on
vision and
language modalities. Second, capturing fine-grained vertex-level contact remains challenging, as MLLMs tend to focus on high-level semantics rather than detailed
geometric reasoning. To address these challenges, we propose ContactPrompt, a training-free and zero-shot approach for dense hand contact estimation using MLLMs. To effectively encode 3D hand geometry, we introduce a detailed hand-part
segmentation and a part-wise vertex-grid representation that provide structured, localized
geometric information. To enable accurate and efficient dense contact prediction, we develop a multi-stage contact
reasoning with part conditioning, progressively bridging global semantics and fine-grained geometry. Therefore, our method effectively leverages the
reasoning capabilities of MLLMs to enable precise dense hand contact estimation. Surprisingly, the proposed approach achieves performance comparable to the previous state-of-the-art method trained on large-scale dense contact
datasets, without requiring any task-specific training.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
3DSS: 3D Surface Splatting for Inverse Rendering
Mae Younes and Adnane Boukhayma
We present 3D Surface Splatting (3DSS), the first differentiable surface splatting renderer for physically-based inverse rendering from multi-view images. Our central insight is that the surface separation problem at the heart of surface splatting admits a direct formulation in terms of the reconstruction kernels themselves. From this foundation we derive a coverage-based compositing model whose per-layer opacity arises directly from the accumulated Elliptical Weighted Average reconstruction weight, yielding anti-aliased silhouettes and informative visibility gradients at sparsely covered edges. Combined with forward microfacet shading under co-optimized HDR environment lighting and density-aware adaptive refinement, 3DSS jointly recovers shape, spatially-varying BRDF materials, and illumination. Because the optimized representation is a set of oriented surface samples, it bridges natively to mesh-based workflows via surface reconstruction from oriented point cloud methods. We evaluate 3DSS against mesh-based, implicit, and Gaussian-splatting baselines across geometry reconstruction, novel-view synthesis, and novel-illumination relighting.
Read the full abstract ▾
Computer Vision
InkDiffuser: High-Fidelity One-shot Chinese Calligraphy via Differentiable Morphological Optimization
Kunchong Shi and Jing Zhang
Current Chinese calligraphy generation methods suffer from poor stroke rendering and unrealistic ink morphology, resulting in outputs with limited visual fidelity and artistic fluidity. To address this problem, we propose InkDiffuser, a
diffusion-based generative framework for one-shot Chinese calligraphy synthesis. To guarantee high-fidelity rendering, we introduce two core contributions: a high-frequency enhancement mechanism and a Differentiable Ink Structure (DIS) loss that explicitly regularizes ink morphology. Inspired by the observation that high-frequency information in individual samples typically carries contour details, we enhance content extraction by explicitly fusing high-frequency representations for more accurate font structure. Furthermore, we propose a differentiable ink structure loss that integrates differentiable morphological operations into the
diffusion process. By allowing the model to learn an explicit decomposition of ink-trace structures, DIS facilitates fine-grained refinement of stroke contours and delivers significantly improved visual realism in the generated calligraphy. Extensive experiments on various calligraphic styles and complex characters demonstrate that InkDiffuser can generate superior calligraphy fonts with realistic ink rendering effects from only a single reference glyph and outperform existing few-shot font generation approaches in structural consistency, detail fidelity, and visual authenticity. The code is available at the following address: https://github.com/JingVIPLab/InkDiffuser.
Read the full abstract ▾
Computer Vision
Align3D-AD: Cross-Modal Feature Alignment and Dual-Prompt Learning for Zero-shot 3D Anomaly Detection
Letian Bai, Xuanming Cao, Juan Du et al. (+1 more)
Zero-shot 3D anomaly detection aims to identify anomalies without access to training data from target categories. However, existing methods mainly rely on projecting 3D observations into multi-view representations that primarily capture
geometric cues rather than realistic visual semantics and process them with
vision encoders pretrained on RGB data, leading to a significant domain gap between the encoder and the projected representations. To address this issue, we propose Align3D-AD, a unified two-stage framework that leverages the RGB modality from auxiliary categories as cross-modal guidance for zero-shot 3D anomaly detection. First, we introduce a cross-modal feature
alignment paradigm that maps rendering features into the RGB semantic space. Unlike prior works that implicitly rely on pretrained encoders, our method enables direct semantic transfer from RGB observations. A semantic consistency reweighting strategy is further introduced to refine feature
alignment by reweighting local regions according to holistic semantic consistency. Second, we propose a modality-aware
prompt learning framework with dual-
prompt contrastive
alignment. By assigning independent
prompts to RGB-aligned and rendering features, our method captures complementary semantics across modalities, while the contrastive
alignment further enhances
prompt representations to improve discriminability. Extensive experiments on MVTec3D-AD, Eyecandies, and Real3D-AD demonstrate that Align3D-AD consistently outperforms existing zero-shot methods under both one-vs-rest and cross-
dataset settings, highlighting its generalization capability and robustness. Code and the
dataset will be made available once our paper is accepted.
Read the full abstract ▾
All authors Letian Bai, Xuanming Cao, Juan Du, Chengyu Tao
Computer Vision
VideoRouter: Query-Adaptive Dual Routing for Efficient Long-Video Understanding
Kuanwei Lin, Wenhao Zhang and Ge Li
Video large
multimodal models increasingly face a scalability bottleneck: long
videos produce excessively long visual-
token sequences, which sharply increase
memory and latency during inference. While existing compression methods are effective in specific settings, most are either weakly query-aware or apply a fixed compression policy across frames, proving suboptimal when visual evidence is unevenly distributed over time. To address this, we present VideoRouter, a query-adaptive dual-router framework built on InternVL for budgeted evidence allocation. The Semantic Router predicts the dominant allocation policy, choosing between broad temporal coverage and adaptive high-resolution preservation, while the Image Router uses early LLM layers to score frame relevance. This enables aggressive compression on less relevant frames while preserving detail on critical evidence frames. To train both routers, we build
Video-QTR-10K for allocation-policy supervision and
Video-FLR-200K for frame-relevance supervision. Experiments on VideoMME, MLVU, and LongVideoBench show that VideoRouter consistently improves over the InternVL baseline under comparable or lower budgets, achieving up to a 67.9%
token reduction.
Read the full abstract ▾
Computer Vision
Unifying Scientific Communication: Fine-Grained Correspondence Across Scientific Media
Megha Mariam K.M, Vineeth N. Balasubramanian and C.V. Jawahar
The communication of scientific knowledge has become increasingly
multimodal, spanning text, visuals, and
speech through materials such as research papers, slides, and recorded presentations. These different representations collectively convey a study's
reasoning, results, and insights, offering complementary perspectives that enrich understanding. However, despite their shared purpose, such materials are rarely connected in a structured way. The absence of explicit links across formats makes it difficult to trace how concepts, visuals, and explanations correspond, limiting unified exploration and analysis of research content. To address this gap, we introduce the
Multimodal Conference
Dataset (MCD), the first
benchmark that integrates research papers, presentation
videos, explanatory
videos, and slides from the same works. We evaluate a range of
embedding-based and
vision-
language models to assess their ability to discover fine-grained cross-format correspondences, establishing the first systematic
benchmark for this task. Our results show that
vision-
language models are robust but struggle with fine-grained
alignment, while
embedding-based models capture text-visual correspondences well but equations and symbolic content form distinct clusters in the
embedding space. These findings highlight both the strengths and limitations of current approaches and point to key directions for future research in
multimodal scientific understanding. To ensure reproducibility, we release the resources for MCD at https://github.com/meghamariamkm2002/MCD
Read the full abstract ▾
Computer Vision
ChartZero: Synthetic Priors Enable Zero Shot Chart Data Extraction
Md Touhidul Islam, Yasir Mahmud, Sujan Kumar Saha et al. (+2 more)
Automated data extraction from line charts remains fundamentally bottlenecked by extreme stylistic diversity and a severe scarcity of comprehensively annotated, real-world
datasets. Current end-to-end pipelines depend heavily on costly manual annotations, crippling their ability to generalize across arbitrary aesthetics and grid layouts. Furthermore, existing models suffer from two critical failure modes during reconstruction. First, extracting thin, intersecting curves frequently causes structural fragmentation and the erasure of fine visual details, as standard architectures struggle against complex backgrounds. Second, semantic association is notoriously error-prone; current pipelines rely on rigid spatial heuristics that easily break down against the unpredictable legend placements of in-the-wild charts. Finally, measuring true progress is hindered by evaluation protocols that assess isolated sub-tasks rather than holistic, end-to-end data reconstruction. To address these foundational issues, we introduce ChartZero, a parsing framework that leverages synthetic priors to enable robust zero-shot chart data extraction. By training exclusively on a purely synthetic
dataset of simple mathematical functions, our model completely bypasses the real-world annotation bottleneck. We overcome curve fragmentation via a novel Global Orthogonal Instance (GOI) loss, and replace brittle spatial rules with an open-vocabulary,
Vision-
Language Model (VLM)-guided legend matching strategy. Accompanied by a new metric and
benchmark specifically designed for full end-to-end reconstruction, our evaluations demonstrate that ChartZero significantly advances generalized plot digitization without requiring real-world supervision. Code and
dataset will be released upon acceptance.
Read the full abstract ▾
All authors Md Touhidul Islam, Yasir Mahmud, Sujan Kumar Saha, Mark Tehranipoor, Farimah Farahmandi
Computer Vision
CXR-ContraBench: Benchmarking Negated-Option Attraction in Medical VLMs
Zhengru Fang, Yanan Ma, Yu Guo et al. (+5 more)
When a chest X-ray shows consolidation but the question asks which finding is present, a medical
vision-
language model may answer "No consolidation." This is more than an incorrect choice: it is a polarity reversal that emits a clinical statement contradicting the image. We study this failure as negated-option attraction, where a model is drawn to a negated answer option even when it conflicts with both the visual evidence and the question. We introduce CXR-ContraBench (Chest X-Ray Contradiction
Benchmark), a diagnostic
benchmark spanning internal ReXVQA slices and external OpenI and CheXpert protocols. The
benchmark centers on present-finding questions, where selecting "No X" despite visible X creates the main clinical risk, and uses absent-finding questions as secondary tests of whether models copy negated wording. Across CheXpert protocols, the failure is substantial and persistent. On a strict direct presence probe, MedGemma and Qwen2.5-VL reach only 31.49% and 30.21% accuracy, respectively; on a matched 135,754-record CheXpert training-split protocol, both models select negated options on over 62% of presence questions. Chain-of-thought prompting reduces some presence-side reversals but does not eliminate them and can amplify absence-side contradictions. Finally, QCCV-Neg (Question-Conditioned Consistency Verifier for Negation) deterministically repairs the measured polarity-confused subset without retraining, raising MedGemma and Qwen2.5-VL to 96.60% and 95.32% accuracy on the direct presence probe. These results show that standard accuracy can hide a clinically meaningful inference-time polarity failure. Source code and
benchmark construction scripts are available at https://github.com/fangzr/cxr-contrabench-code.
Read the full abstract ▾
All authors Zhengru Fang, Yanan Ma, Yu Guo, Senkang Hu, Yixian Zhang, Hangcheng Cao, Wenbo Ding, Yuguang Fang
Computer Vision
Na-IRSTD: Enhancing Infrared Small Target Detection via Native-Resolution Feature Selection and Fusion
Qian Xu, Chi Zhang, Qiming Zhang et al. (+3 more)
Infrared small target detection (IRSTD) faces the inherent challenge of precisely localizing dim targets amid complex background clutter. While progress has been made, existing methods usually follow conventional strategies to downsample features and discard small targets' details, resulting in suboptimal performance. In this paper, we present Na-IRSTD, a native-resolution feature extraction and fusion framework for IRSTD. This framework elegantly incorporates native-resolution features to preserve subtle target cues, overcoming the resolution limitations of existing infrared approaches and significantly improving the model's ability to localize small targets. We also introduce an effective
token reduction and selection strategy, which selects target patches with high accuracy and confidence, boosting the low-level details of the feature while effectively reducing native-resolution patch
tokens compared to dense processing, thereby avoiding imposing an unbearable computational burden. Extensive experiments demonstrate the robustness and effectiveness of our
token reduction and selection strategy across multiple public
datasets. Ultimately, our Na-IRSTD model achieves state-of-the-art performance on four
benchmarks.
Read the full abstract ▾
All authors Qian Xu, Chi Zhang, Qiming Zhang, Xi Li, Haojuan Yuan, Mingjin Zhang