Computer Vision · Cross-listed from cs.LG
Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation
Alexi Gladstone, Heng Ji and Yilun Du
The deep learning revolution, kicked off by AlexNet, taught us that end-to-end training beats decomposing a problem into hand-designed stages. Generative modeling, however, has remained the exception-despite generative models being remarkably capable, they are still not trained end-to-end. This is because, at its core, generative modeling is about handling distributions with many modes, and existing scalable approaches handle this the same way, by factoring the generation procedure, which prevents end-to-end generation. In this work, we introduce Explorative Modeling, a new paradigm that instead factors the training loop, exploring K candidate matches between model generations and data, and training on the best, so predictions commit to modes rather than blurring them. We find Explorative Models (XMs) useful in two settings. First, increasing exploration adds a third pretraining axis beyond parameters and data for existing generative models-where scaling exploration monotonically improves performance across both continuous and discrete domains (images,
video, and
language). Notably, gains from exploration increase with scale, climbing from 7% to 36% as data scales and from 13% to 23% as models grow, with efficiency gains more than doubling at 3x the compute. Concretely, exploration improves FLOP efficiency by 4.1x, sample efficiency by 6.2x, parameter efficiency by 47%, lifts the strongest of image-generation recipes to a near-state-of-the-art 1.43 FID on ImageNet without guidance, enables scaling how end-to-end existing models are, and unlocks scaling generalization. Second, XMs enable end-to-end reconstructive generative modeling, matching
diffusion on control tasks with 16-256x fewer inference steps. Together, these results establish XMs as both a new pretraining axis for existing generative models and a standalone end-to-end generative modeling paradigm.
Read the full abstract ▾
Computer Vision
Shared Semantic Codebook Distillation for Unpaired Cross-Modal Medical Classification
Dillan Imans, Phuoc-Nguyen Bui, Duc-Tai Le et al. (+1 more)
Cross-modal knowledge
distillation can transfer diagnostic knowledge from a strong but costly teacher modality to a cheaper and more deployable student modality. In medical image analysis, however, the two modalities are often unpaired: they are collected from different patient cohorts and occupy geometrically incompatible feature spaces. This makes instance-level
distillation invalid and direct feature matching unreliable. To address these challenges, we propose Shared Semantic Codebook
Distillation (SSCD), which compares teacher and student representations through a shared discrete codebook. Each image is represented as a distribution over a common, modality-agnostic vocabulary, and knowledge is transferred by aligning these distributions across modalities, both globally and class-conditionally, without requiring paired samples or directly comparable raw features. The codebook is evolved online by exponential moving average and kept diverse through
entropy regularization and dead-code restart. At inference, all teacher-side and codebook modules are discarded, leaving only the student encoder and classifier. On two heterogeneous unpaired settings, OCT-to-fundus retinal disease classification and CT-to-chest-X-ray pneumonia classification, SSCD improves the student from 64.5 to 70.2 macro-F1 and from 73.8 to 76.3 macro-F1, respectively, outperforming all evaluated
distillation baselines on both settings. Code and pretrained models are available at https://github.com/DillanImans/SSCD-unpaired-
distillation
Read the full abstract ▾
All authors Dillan Imans, Phuoc-Nguyen Bui, Duc-Tai Le, Hyunseung Choo
Computer Vision
Bunraku: Turning a Single Illustration into an Editable Live2D Character
Junhao Chen, Jingjia Mao, Dayong Li et al. (+6 more)
Live2D is the dominant 2D character-animation format for anime characters and virtual avatars, representing each character as a stack of RGBA layers driven by per-layer mesh deformation. Despite its wide use in virtual streaming, mobile games, and interactive characters, authoring a Live2D model still demands weeks of manual layer separation, occlusion completion, mesh placement, and keyframing, and no prior generative method produces such a structured asset end-to-end. We present the first system that, from a single illustration, generates all the structured information a Live2D runtime consumes: ordered RGBA layers, a deformation mesh per layer, and the parameter-driven keypose vertex offsets that make the character move. Stage 1 casts layered decomposition as a layered
diffusion process under a Live2D-aware organ-level taxonomy, producing an ordered RGBA stack with hidden-region completion. Stage 2 builds a content-conforming triangle mesh for each layer from its alpha channel alone, then predicts the keypose displacement field of all layers jointly: every vertex of every layer is one
token, self-
attention spans layer boundaries, and each displacement is factorised into a bounded direction and a log-magnitude. Joint rather than independent prediction is what makes the result a coherent character instead of separately plausible parts, and is our largest gain; scaling the network 112x yields none. On 50 held-out characters, under true generation with no teacher forcing, Stage 2 attains a per-vertex direction cosine of 0.768 (median 0.828). Because a layer's mesh derives from its alpha channel, a clothing layer can be re-textured from a natural-
language instruction while the mesh and predicted animation are reused byte-for-byte. We further contribute Live2D-Bench, the first standardized
benchmark for the task, and an 8,884-model Live2D corpus with layer and animation supervision.
Read the full abstract ▾
All authors Junhao Chen, Jingjia Mao, Dayong Li, Chenghai Li, Saining Zhang, Zhihao Li, Hao Zhao, Yufei Wang, Ruqi Huang
Computer Vision · Cross-listed from cs.LG
Position, Not Provenance: Separating Reasoning Mediation from Sycophancy in Medical Vision-Language Models
Supratik Bhowal, Subhrajyoti Basu, Aritra Gir Mahanta et al. (+1 more)
Medical
vision-
language models (VLMs) generate chain-of-thought (CoT)
reasoning before answering clinical questions, but whether this
reasoning causally influences predictions remains unclear. We present CoT-Mediate, a behavioral framework that perturbs a single clinically meaningful attribute within a model's own generated
reasoning and measures whether the resulting prediction follows the edited
reasoning. Our framework combines a dual-arm protocol comparing re-prompted evidence with prefix-forced continuation, together with a provenance-controlled intervention that varies only the attributed source of identical
reasoning to disentangle
reasoning mediation from sycophancy. We evaluate LLaVA-Med and MedGemma on 1,000 VQA-RAD samples each. Prefix-forced continuation consistently yields higher mediation faithfulness than re-prompting, while the provenance analysis reveals distinct model-specific deference behaviors. Across both models, removing visual evidence increases reliance on injected
reasoning, whereas laterality is the least faithfully tracked clinical attribute. These results show that the mechanism used to inject
reasoning substantially affects measured faithfulness and that contextual position, rather than stated provenance, is the primary determinant of whether medical VLMs use their generated
reasoning.
Read the full abstract ▾
All authors Supratik Bhowal, Subhrajyoti Basu, Aritra Gir Mahanta, Anik Pal Chowdhury
Computer Vision
VETO: Towards Protecting Images From Frontier AI Editing
Jonas Grebe, Hossein Shakibania, Tobias Braun et al. (+2 more)
The rise of powerful, accessible image-editing models such as FLUX.2 has brought high-fidelity editing within broad reach. Their capabilities now extend beyond localized modifications to extracting and recontextualizing objects and identities in entirely new scenes. By allowing
prompt and generation
tokens to attend directly to reference-image
tokens, modern models blur the boundary between conventional editing and text-to-image synthesis. This expanded generative freedom also broadens the space of potential misuse, as harmful transformations are no longer confined to a predictable set of localized edits. Existing anti-edit defenses are designed to disrupt the semantic bottleneck of the reference-image encoding in legacy
diffusion pipelines. However, newer editors distill reference information through joint-
attention blocks, thereby often circumventing these protections. We therefore introduce VETO, a subtle anti-edit cloak that disrupts this inner mechanism through which modern models read the source image. Additionally, as existing editing
benchmarks leave comprehensive recontextualizations largely untested, we introduce VetoBench, which evaluates defenses not only on conventional localized edits but also on broader contextual shifts. Across two contemporary editing models and three
benchmarks, VETO consistently outperforms existing defenses while providing a stronger protection-fidelity trade-off.
Read the full abstract ▾
All authors Jonas Grebe, Hossein Shakibania, Tobias Braun, Marcus Rohrbach, Anna Rohrbach
Computer Vision · Cross-listed from eess.IV
Toward Multi-Modal Deep Learning for Pulmonary Disease Classification: A Texture-Based Machine Learning Pilot Study on Public Chest X-Ray Data
Yogisri Pujitha Chinthoti
Automated classification of pulmonary disease from chest radiographs is a widely studied application of machine learning in medical imaging. This paper presents a pilot study evaluating classical texture- and gradient-based feature representations for distinguishing COVID-19 from other forms of pneumonia using the publicly available COVID-19 Image Data Collection (668 posteroanterior/anteroposterior radiographs from 408 patients). Using histogram of oriented gradients (HOG) and gray-level co-occurrence
matrix (GLCM) texture descriptors with classical classifiers (logistic regression, random forest, and support vector machine), evaluated under patient-level 5-fold stratified cross-validation to prevent data leakage, we obtain a best mean accuracy of 75.4% and AUC of 0.755, modestly exceeding the 71.6% majority-class baseline. We report these results transparently, including their limitations, and use them to motivate and scope a proposed multi-modal deep learning architecture – combining convolutional and
transformer-based encoders across imaging modalities – as a direction for future work requiring access to larger, multi-institutional, ethically sourced
datasets.
Read the full abstract ▾
Computer Vision
OVEarth-Bench: Evaluating Category Breadth and Query Diversity for Open-Vocabulary Earth Observation
Kaiyu Li, Zepeng Xin, Zixuan Jiang et al. (+4 more)
Open-vocabulary Earth observation (EO) aims to localize geospatial concepts specified in natural
language rather than a fixed label set. Existing
benchmarks, however, usually cover narrow category vocabularies or limited query forms. To fill this gap, we introduce OVEarth-Bench, which extends existing evaluation in two directions: category breadth, through broad hierarchical category coverage with positive and negative expressions, and query diversity, through vocabulary, referring, and
reasoning queries. The
benchmark supports mask and box localization under a unified zero-shot protocol. We evaluate a broad set of general and EO-specific methods. The evaluation reveals that: (1) the performance of current methods remains limited, while broader category coverage yields more stable model rankings; (2) MLLM-based methods achieve the strongest overall performance; and (3) EO-specific methods generally underperform general models and rarely match the strongest methods. These findings provide guidance for future open-vocabulary EO method design and highlight the importance of developing more realistic, diverse, high-quality, and large-scale
benchmarks for reliable evaluation. Our data and evaluation package are released at https://earth-insights.github.io/OVEarth-bench.
Read the full abstract ▾
All authors Kaiyu Li, Zepeng Xin, Zixuan Jiang, Jing Fu, Lanxuan Xue, Lingyu Zhang, Xiangyong Cao
Computer Vision
Theatre Chapbooks At Scale: A Statistical Comparative Analysis of Typography
Diego Belzarena, Seginus Mowlavi, Paula Casariego Castiñeira et al. (+3 more)
We propose a statistical methodology that quantifies the similarity of typefaces between printed historical books. This provides a tool that accelerates philological analysis. Using character prototypes derived from clustering and aligning automatically extracted character images, the method defines a typeface distance between any two books. To produce actionable outputs, we develop an a contrario statistical framework to interpret the significance of the computed typeface distances. We apply the method to the philological study of 17 th -century Spanish printed theatre chapbooks in a quantity that exceeds the capabilities of systematic visual inspection by human experts. Our method enables the automatic comparison of Roman and Italic types extracted from different books. After validation by human experts, our method has led to new printer attributions being discovered, and former printer attributions being revised. This success strongly suggests that our method has the potential to enable digital bibliography on a larger scale than was previously possible.
Read the full abstract ▾
All authors Diego Belzarena, Seginus Mowlavi, Paula Casariego Castiñeira, Alejandra Ulla Lorenzo, Gregory Randall, Jean-Michel Morel
Computer Vision
TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM
Hengyi Xie, Chenfei Yao, Xianjin Wu et al. (+4 more)
Vision-
language-action (VLA) models commonly adopt an LLM-centric $V \to L \to A$ pathway, processing visual observations and
language instructions through a large
language model before predicting
robot actions. Although effective, this design incurs substantial computation and
memory overhead. In this work, we introduce TurboVLA, a compact VLA architecture built on a direct $V + L \to A$ mapping. Instead of using a large
language model as the central interface between perception and action, TurboVLA independently encodes visual observations and
language instructions, directly exchanges information between them through lightweight bidirectional
vision-
language interaction, and predicts continuous action chunks with a compact decoder. This simple design directly constructs task-conditioned representations while avoiding the overhead of an LLM-centered execution pathway. On LIBERO, TurboVLA achieves 97.6% average success with only 0.2B parameters, 31.2 ms inference latency, and 0.9 GiB inference VRAM on a consumer-grade RTX 4090. Notably, a 0.4B TurboVLA achieves 88.06% success on RoboTwin 2.0, even matching or outperforming substantially larger VLA policies. These results demonstrate that the simple $V + L \to A$ design of TurboVLA can achieve high performance without requiring an LLM-centric execution pathway, offering a new perspective on how
vision,
language, and action can be connected for efficient robotic manipulation. Code is available at https://github.com/H-EmbodVis/TurboVLA.
Read the full abstract ▾
All authors Hengyi Xie, Chenfei Yao, Xianjin Wu, Yingying Zhu, Dingkang Liang, Xiang Bai, Han Ding
Computer Vision
VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion
Zador Pataki, Paul-Edouard Sarlin and Marc Pollefeys
Accurately recovering the camera's calibration and metric poses for any unconstrained
video would unlock large-scale training data for
navigation and scene understanding. The dominant approaches to this problem are severely limited: Simultaneous Localization and Mapping (SLAM) is sensitive to initialization and transient failures due to its causal, incremental nature; it is often over-optimized for real-time operation and generally requires known camera calibration; while Structure-from-Motion (SfM) typically forgoes any image ordering, enabling optimal initialization and global optimization, but lacks robustness to visual
symmetries and extreme motions. To bridge this gap, we introduce a system that combines the strong sequential constraints of SLAM with the flexibility and global optimization of offline SfM, enabling the metric reconstruction of arbitrary, long, uncalibrated
videos. This system leverages recent advances in wide-baseline dense image matching, treats temporal ordering as a first-class citizen for reliable loop closure, and augments global optimization with metric
monocular depth priors. As a result, thorough evaluations on diverse, challenging
datasets that exhibit extreme motion and visual
symmetries reveal that our approach is significantly more robust and accurate than both state-of-the-art SLAM and SfM, classical or learned, with given or unknown camera calibration. The code is publicly available at https://github.com/cvg/vidmap.
Read the full abstract ▾
Computer Vision
HumanCLAW: Can Vision-Language Models Act Through a Body?
Li Siyao, Jiawei Gu, Shuai Liu et al. (+15 more)
Evaluating whether a
vision-
language model (VLM) can act through a physical body is challenging. The outcome of an action couples the VLM's decision with motor control. When a task fails, it is hard to tell whether the VLM made a bad choice or the motor controller simply failed to execute it, e.g., losing balance and falling. In this work, we introduce HumanCLAW, an evaluation framework that decouples action decision-making from low-level execution. At every step, a harnessed, off-the-shelf VLM issues an atomic skill command, and the command is translated into a sub-second chunk of continuous full-body motion with real physical consequences, including
gravity and collisions. The body can therefore act freely in the physical world, while execution-side disturbances, balance and motor errors, are factored out. What remains measurable is the model's action intelligence: its moment-to-moment choice of what the body should execute next. Based on this framework, we build HumanCLAW-Bench: 1,218 long-horizon,
egocentric find-navigate-interact episodes across 41 indoor scenes. We test nine state-of-the-art VLMs and find that none solves the
benchmark; the best model reaches only a 16.8% success rate. Recognizing the target is not the bottleneck. What current VLMs lack is
embodied self-awareness: they lose track of their own body, failing to tell where it is, whether it has reached the goal, or whether it has hit an obstacle.
Read the full abstract ▾
All authors Li Siyao, Jiawei Gu, Shuai Liu, Kairui Hu, Zekun Li, Linjie Li, Chengcheng Tang, Po-Chen Wu, Ivan Shugurov, Lingni Ma, Michael Zollhoefer, Sizhe An, Abhay Mittal, Amy Zhao, Ranjay Krishna, Manling Li, Ziwei Liu, Chuan Guo
Computer Vision
Anatomy Contextualized Adaptation of CT Foundation Models
Roshan Kenia, Stephanie L McNamara and William Lotter
CT
vision-
language foundation models have demonstrated promising performance across downstream tasks, but are typically trained with whole-volume representations that dilute fine-grained anatomical signals. Fine-grained
vision-
language pre-training addresses this by aligning anatomy-level visual features with anatomy-specific text, but in doing so discards the global context that whole-volume models provide. Furthermore, existing fine-grained approaches train from scratch, making them computationally expensive. We introduce Anatomy Contextualized Adaptation (ACA), a lightweight framework that adapts frozen CT foundation model representations for anatomy-level
vision-
language alignment while enhancing global contextualization. ACA uses TotalSegmentator to decompose CT volumes into anatomy-level
embeddings, which are refined via a
transformer that captures cross-anatomy relationships, and aligned to both per-anatomy and scan-level text extracted from radiology reports. Evaluated on Merlin and CT-RATE, ACA consistently outperforms both the frozen foundation model baselines and existing fine-grained methods in zero-shot finding classification, while requiring less than one hour of training once
embeddings are cached. The
attention weights learned by ACA's inter-anatomy
transformer additionally indicate plausible cross-anatomy context routing. Altogether, these results support ACA as a lightweight approach for adapting CT foundation models to anatomically grounded
vision-
language alignment while preserving and enhancing global anatomical context.
Read the full abstract ▾
Computer Vision
Explainable and Resource-Efficient Spatial Reasoning in Multimodal LLMs for Decision-Critical Applications
Piyush Jain, Kousik Dasgupta, Rajarshi Roy et al. (+1 more)
As
Multimodal Large
Language Models (MLLMs) are increasingly deployed in decision-critical pipelines such as robotics,
embodied AI, and safety monitoring, the opacity of their spatial judgments limits operator trust and auditability. MLLMs demonstrate strong
reasoning but often struggle with fine-grained spatial understanding and object
hallucination. Prior work, ByDeWay, introduced Layered-Depth-Based Prompting (LDP), a training-free framework that mitigates
hallucinations by structuring
prompts using
monocular depth estimation. However, coarse depth layering falls short in resolving object-to-object spatial relationships within the same
geometric plane, such as projective ("left of", "above") and topological ("inside", "touching") relations. We propose ByDeWay-V2, which integrates explicit spatial relational context alongside depth cues, expressed as human-readable predicates that serve as auditable evidence for downstream decision support. Using an open-vocabulary object detector (YOLO-World-L), our framework computes pairwise
geometric relations between detected objects and injects them as structured spatial predicates into the MLLM
prompt, bridging 3D scene depth and 2D spatial semantics without any training. We evaluate ByDeWay-V2 on the Visual Spatial
Reasoning (VSR) and BLINK
benchmarks across multiple MLLMs, with
hallucination grounding assessed via POPE. On the BLINK spatial subset, ByDeWay-V2 achieves a 46 percent relative F1 improvement over LDP for Qwen2.5-VL, and recovers BLIP-Base's spatial
reasoning on VSR from near-random performance to a competitive F1 of 0.53. Our lightest configuration operates under a strict 40-
token context budget on CPU, showing the framework's suitability for resource-constrained, real-time decision-support settings.
Read the full abstract ▾
All authors Piyush Jain, Kousik Dasgupta, Rajarshi Roy, Subarna Tripathi
Computer Vision
SeasonStereo: Robust Dense Stereo Matching for Multi-Date Satellite Imagery via Generative AI
Álvaro Díaz-Laureano, Roger Marí, Elías Masquil et al. (+2 more)
Accurate 3D reconstruction from satellite imagery typically relies on near-simultaneous stereo pairs, limiting its applicability to diachronic settings where multi-date images exhibit varying seasonal and illumination conditions. Training dense stereo matching models robust to appearance changes is a long-standing challenge, as aligned multi-date imagery and ground-truth geometry are costly to obtain at scale. We propose SeasonStereo, a scalable framework that addresses disparity estimation from diachronic satellite images by training on synthetic image pairs with controlled seasonal appearance variation, while leveraging zero-shot
geometric priors from foundation models. SeasonStereo matches the accuracy of state-of-the-art
LiDAR-supervised models, while producing sharper
geometric details without requiring aligned real multi-date training products or
LiDAR-derived labels. As a result, SeasonStereo offers a practical path toward large-scale 3D reconstruction from heterogeneous satellite images with reduced supervision cost.
Read the full abstract ▾
All authors Álvaro Díaz-Laureano, Roger Marí, Elías Masquil, Pablo Arias, Gabriele Facciolo
Computer Vision · Cross-listed from cs.RO
DLAM: Distributional Latent Actions with Temporal Constraints
Zuojin Tang, Feifan Luo, Haoyun Liu et al. (+10 more)
Vision-
language-action (VLA) models remain constrained by scarce action-labeled
robot data, whereas action-free
videos offer abundant observations of physical change. Latent action models can extract such priors, but reconstruction-trained codes may predict future observations without the structure required for joint generation with
robot actions. Existing structured methods add temporal constraints but retain deterministic transition points, so residual errors in locally inferred transitions may propagate and compound under recursive composition. We introduce DLAM, a distributional latent-action model that represents each transition as a diagonal Gaussian. Reconstruction conditioned on the reference frame grounds the mean in observed visual change, while normalized composition and reversal over equal-gap triplets constrain both the mean and dimension-wise variance. Variance composition uses a lightweight shared-correlation coefficient to account for dependence between adjacent transitions that share an intermediate frame, whereas reversal negates the mean and preserves the variance. For downstream policy learning, we freeze the encoder and train a flow-matching policy to jointly generate mean transition sequences and
robot actions. On held-out transitions, DLAM learns more temporally consistent latent dynamics than existing latent-action baselines and achieves stronger direct and cumulative reconstruction on held-out
videos. Under the same controlled $\pi_0$ transfer protocol, it also improves policy performance on MetaWorld MT50, LIBERO, and real-world manipulation tasks. Controlled ablations show that normalized mean constraints account for most of the reconstruction gain, while learned variance and correlation-aware composition provide complementary improvements in downstream control.
Read the full abstract ▾
All authors Zuojin Tang, Feifan Luo, Haoyun Liu, Botai Yuan, Dekang Qi, Ronghan Chen, Yandan Yang, Tong Lin, Xinyuan Chang, Mu Xu, Bin Liu, De Ma, Zhiheng Ma
Computer Vision
Towards Grounded GI Endoscopy VQA via Multi-Task Learning on Small VLMs
Itbaan Safwan, Ramail Khan, Muhammad Annas Shaikh et al. (+1 more)
Gastrointestinal (GI) endoscopic image analysis has shifted from single-label classification toward visual question answering (VQA), where a model must answer free-form clinical questions about an image. While recent
vision-
language models (VLMs) achieve promising answer accuracy on this task, clinical adoption also requires the model's internal representations to reflect the visual evidence behind its answers. We propose a simple multi-task fine-tuning recipe that constructs auxiliary grounding and description tasks from an existing VQA
dataset with minimal additional annotation: expert-annotated polyp masks are reused directly, while a GI-domain pretrained classifier with Grad-CAM localization provides weak supervision for finding categories that lack ground-truth masks. Three small VLM backbones are fine-tuned with low-rank adaptation under matched VQA-only and multi-task recipes on Kvasir-VQA-x1, and we show consistent accuracy gains together with improved implicit
alignment between answer
tokens and the relevant image region, evaluated on both in-distribution and out-of-distribution data.
Read the full abstract ▾
All authors Itbaan Safwan, Ramail Khan, Muhammad Annas Shaikh, Muhammad Atif Tahir
Computer Vision
Veritas++: Value-aware On-Policy Distillation for Perception-Enhanced AIGI Detection
Hao Tan, Jun Lan, Zichang Tan et al. (+7 more)
The growing capability of image generation models has made synthetic images a routine presence in open media, making robust and generalizable AI-Generated Image (AIGI) detection increasingly essential. While multi-modal large
language models (MLLMs) offer a transparent alternative to black-box binary scoring, we observe that current MLLM-based detectors still exhibit notable perception bottlenecks in capturing fine-grained anomalies. They primarily focus on how visual evidence is organized and synthesized, leaving the intrinsic perception less optimized. To mitigate this gap, we present Veritas++, a perception-enhanced
reasoning framework that establishes reliable perception as the foundation of authenticity
reasoning. Rather than directly optimizing the model's explanatory ability, we ground AIGI detection on three basic perception abilities, i.e., capturing fine-grained visual details, semantic anomalies and pixel-level differences. Building on this insight, we introduce Perception-oriented Learning (PoRL), which replaces open-ended description supervision with verifiable rewards to explicitly strengthen these capacities. To further integrate enhanced perception with
reasoning, we introduce Value-aware On-Policy
Distillation (VaOPD), an adaptive
distillation mechanism that prioritizes high-value
distillation signals over uniform supervision, internalizing perception-aware
reasoning through a privileged self-teacher. Extensive experiments across standard, in-the-wild and emerging
benchmarks demonstrate that Veritas++ achieves promising generalization. The perception learning effectively bridges the perception gap and yields seamless gains on detection, while VaOPD further enables efficient capability evolvement without sacrificing existing performance. Code and checkpoints are available at https://github.com/EricTan7/VeritasPP.
Read the full abstract ▾
All authors Hao Tan, Jun Lan, Zichang Tan, Ajian Liu, Zijian Yu, Chuanbiao Song, Huijia Zhu, Weiqiang Wang, Jun Wan, Zhen Lei
Computer Vision
FreqForcing: Autoregressive Long Video Generation via Spectral Self-Anchoring
Jiatong Li, Leo Liang, Linghe Kong et al. (+1 more)
Autoregressive
video diffusion models enable real-time streaming
video generation. However, errors introduced during self-rollout accumulate over long horizons, manifesting as color drift, motion stagnation, and eventual visual collapse. In this paper, we characterize this phenomenon from a frequency-domain perspective: error accumulation appears as a pronounced energy drift in the low-frequency bands. We further investigate the effectiveness of
attention sink in the frequency domain, and find that it improves the
video quality by alleviating the spectral energy drift to some extent, but cannot fully resolve it. Motivated by the above analysis, we propose FreqForcing, a training-free framework that addresses error accumulation in long-
video generation via Spectral Self-Anchoring (SSA). The proposed SSA leverages the low-frequency components of anchor
attention to maintain long-horizon visual stability, while preserving dynamic motion through the high-frequency components of local
attention. Our FreqForcing extends Self-Forcing pretrained on 5s clips to two-minute generation, achieving 24x extrapolation. Extensive experiments show that FreqForcing outperforms existing training-free methods quantitatively and qualitatively while remaining competitive with representative training-based approaches.
Read the full abstract ▾
All authors Jiatong Li, Leo Liang, Linghe Kong, Yulun Zhang
Computer Vision
Step-Attention Refinement of DINOv3 Features for Efficient Anterior Eye Segmentation
Philippe Baumstimler, Jean-Mathieu Gagnon, Sébastien Gagné et al. (+3 more)
Anterior eye segment (AES)
segmentation is a key component of both ocular biometrics and emerging clinical image analysis applications. However, heterogeneous acquisition conditions and limited annotations in medical settings hinder the robustness and generalization of existing methods. Foundation models (FMs) such as DINOv3 offer strong transfer capabilities, but efficiently adapting their representations to dense prediction tasks remains challenging. In this study, we investigate robust AES
segmentation in clinical settings, and propose a lightweight architecture built upon a distilled DINOv3 ViT-Small backbone. We introduce a step-
attention feature refinement module that progressively adapts multi-level
transformer representations before convolutional decoding, enabling efficient exploitation of pretrained features with few parameters. We evaluate the proposed approach on a private
dataset of 333 clinically acquired AES images spanning eight ophthalmic acquisition protocols and annotated for seven anatomical classes. Compared with convolutional and
transformer-based baselines, including DINOv3-based methods, our approach achieves the best overall performance, reaching 85.55% mIoU when fully fine-tuned. It also demonstrates the strongest robustness to domain shift across four unseen public AES
segmentation datasets. These results establish a strong baseline for robust AES
segmentation in clinical settings and highlight the importance of decoder design for effectively adapting FMs representations to medical
segmentation tasks.
Read the full abstract ▾
All authors Philippe Baumstimler, Jean-Mathieu Gagnon, Sébastien Gagné, Mathieu Duchesneau, Clément Playout, Lama Séoud
Computer Vision
SciFigQual-Bench: A Benchmark for Scientific Figure Quality Assessment with Full-Manuscript Context
Zihan Deng, Chuanzhi Xu, Huiqi Liang et al. (+4 more)
Scientific figure quality is bound to the manuscript: a crop can be visually clear and still contradict its caption or the paragraph that cites it. Most image quality assessment and chart-understanding methods target a detached visual surface or a question-answering score, rather than
alignment among the figure, its caption, and the citing text. A single end-to-end
prompt that sees the image and the text together still mixes what is visible with what the caption and the citing paragraph claim, so fluent wording can raise a visual score and missing text can be treated as low quality. To address this gap, we introduce SciFigQual-Bench, a manuscript-linked
benchmark for published CS-conference figures that binds each figure to its caption and to index-resolved citing paragraphs, and scores visual clarity, layout, caption consistency, context consistency, and misleading risk, leaving a dimension unevaluated when its evidence is absent. SFQ-
Agent reads the image and the text in separate calls and records modality-specific evidence. A cross-modal judge scores caption and citing-text
alignment from those reports, and a deterministic runner keeps the visual scores, sets misleading risk by a fixed rule, and applies the written caps, so each dimension follows the rubric instead of a single end-to-end
prompt. Experiments comparing this staged judge with single-pass and OCR-sidecar protocols find the closest point-estimate fit to mean human ratings under staged judging, while the gap between protocols remains small. Caption consistency remains the main gap, and agreement with the rater mean answers a different question from agreement among human raters.
Read the full abstract ▾
All authors Zihan Deng, Chuanzhi Xu, Huiqi Liang, Haoyang Li, Xiaozhen Zhong, Weidong Cai, Lequan Yu
Computer Vision
Feixiang Liu, Qiang Qiu, Lanbo Sun et al. (+3 more)
Closed yes/no spatial
benchmarks can reward a correct answer even when the image adds little support beyond no-image contexts. Under a fixed forced-choice interface, Visual Credit Audit (VCA) separates two estimands: whether the
benchmark image gives the model's declared decision more support than text-only and blank controls, and whether the model responds to relation-specific visual evidence. The first audit is training- and label-free and does not require an answer flip. Applying labels yields dependence-credited correctness (D-CC); on correct items, it equals same-control gold-aligned positive gain, while prediction
alignment extends the audit to errors. Across four open MLLMs and two spatial
benchmarks, 12.73-26.25% of decisions are correct yet uncredited. Matched same-split image permutation reduces D-CC by 21.25-47.80 points, with every paired 95% interval above zero. Fixed-pixel relation contrasts and a 3x3 evidence-source factorial show why null controls cannot identify relation response. Among controlled correct-but-uncredited agreement decisions, response to relation reversal spans 81.57-100.00%, while 32.11% pooled change answer. Independently audited outcomes on 108 geometry-compatible edits provide a bounded natural-image correspondence check. VCA thereby decomposes
benchmark success into correctness, additional image support, and relation-consistent response.
Read the full abstract ▾
All authors Feixiang Liu, Qiang Qiu, Lanbo Sun, Nan Wei, Huawei Shen, Xueqi Cheng
Computer Vision
SciFigAlign: Scoring Scientific Figures by Fine-tuned Alignment of Visuals with Manuscript Evidence
Chuanzhi Xu, Zihan Deng, Huiqi Liang et al. (+4 more)
Scientific figure assessment in peer review differs fundamentally from general image quality evaluation: a figure must be visually legible, faithfully support the manuscript's claims, and communicate evidence with a clear visual hierarchy. However, if we apply traditional image assessment methods to scientific figure quality assessment, limitations emerge: classic IQA models capture perceptual quality or aesthetics but cannot judge whether a figure serves the paper's scientific argument; CLIP-based methods assess generic image-text correspondence, yet lack understanding of manuscript context; and zero-shot LLM/VLM judges, when repurposed for figure scoring, often yield overly concentrated scores with limited fusion of visual and textual evidence. We introduce an annotated
dataset of 3,857 scientific figures from peer-reviewed conference papers, each rated along four peer-review-oriented dimensions: Clarity, Relevance, Informativeness, and Structure. We propose SciFigAlign, a fine-tuned
multimodal scorer that grounds figure quality assessment in manuscript evidence. Given a figure crop, caption, citing paragraphs, and light paper context, SciFigAlign fine-tunes CLIP and SciBERT end-to-end with per-modality cross-
attention and CubeMLP fusion, jointly optimizing SmoothL1 regression with a within-paper ranking hinge loss. Under paper-level splits, SciFigAlign achieves a macro MAE of 0.3524 and a within-paper pairwise accuracy of 81.64% on the test set with n = 396, a 59% relative error reduction over the best LLM-as-judge baseline with MAE 0.864. Ablations confirm that manuscript-grounded inputs, citing-context denoising, and ranking supervision are all critical, showing that scientific figure assessment requires learned
alignment between visual content and manuscript evidence rather than prompting alone, even with state-of-the-art VLMs.
Read the full abstract ▾
All authors Chuanzhi Xu, Zihan Deng, Huiqi Liang, Chengkun Yue, Zhanlin Cui, Pengfei Ye, Weidong Cai
Computer Vision
ScratchSim: A Procedural Synthetic Data Pipeline for Surface Scratch Detection
Paul Julius Kühn, Saptarshi Neil Sinha, Tiago Kleist et al. (+3 more)
While automated defect detection such as the detection of surface scratched is an important aspect in industrial quality control, the scarcity of annotated defect data make this task challenging. This paper presents a procedural rendering pipeline that generates large-scale annotated synthetic training data using BlenderProc, with configurable material appearance, camera modes, and domain randomization, producing automatic COCO-format annotations. To show the potential of our approach, we evaluate four training strategies, namely synthetic-only, real-only, mixed, and fine-tuning from synthetic weights, across two objects with different material properties and three lightweight edge-deployable detectors, YOLOX, YOLO26, and LW-DETR. Our evaluation show that fine-tuning from synthetic weights consistently outperforms real-only training, and that mixed training effectively recovers performance under scarce real-data conditions, with findings validated across both convolutional and
transformer-based architectures. The proposed approach enables scalable defect detection without the burden of large real annotated
datasets, making it practical for on-device industrial inspection. The pipeline scripts for generating synthetic scratches, 3D model, and both the synthetic and real annotated scratch
datasets for a glossy toy Ferrari car are publicly available at https://github.com/saptarshineil/ScratchSim.
Read the full abstract ▾
All authors Paul Julius Kühn, Saptarshi Neil Sinha, Tiago Kleist, Richard Hoffmann, Arjan kuijper, Michael Weinmann
Computer Vision
Object Detection for Autonomous Driving in Chinese Rural Scenes: An Experimental Study on Real-Synthetic Data Mixing and Model Evaluation
Danning Zhu, Ziyan Lin and Jing Wu
Currently, autonomous driving object detection models face significant data scarcity and generalization challenges when navigating complex Chinese rural traffic scenarios. To address these limitations, we propose a novel real-synthetic mixed object detection
dataset tailored specifically for Chinese rural roads and systematically evaluate the performance of 13 mainstream detectors under different real-to-synthetic data ratios, thereby providing empirical evidence for model selection and data strategy design in rural autonomous driving scenarios. Our
dataset combines real-world images captured in Weishi County, Henan Province, with parameterized virtual scenes generated via Unreal Engine. To accurately reflect the unique realities of rural traffic, we define a comprehensive 14-category object system encompassing region-specific elements such as electric tricycles, low-speed vehicles (LSVs), and roadside stalls. Under a unified training protocol, we systematically evaluate 13 mainstream detectors – spanning the YOLOv5, YOLOv8, YOLO11, and YOLO26 series, as well as RT-DETR-L – across three data configurations: an all-real baseline, a 1:0.5 real-to-virtual mix, and a 1:1 mix. Experimental results demonstrate that a moderate injection of synthetic data (1:0.5 ratio) effectively enhances detection performance, with YOLO11m achieving the highest mAP@0.5 of 0.758. However, a higher proportion of synthetic data (1:1) introduces domain shifts that offset the benefits of data scaling. While most models reliably identify distinct local vehicles, significant perceptual bottlenecks remain for long-tail, non-standard objects like stalls and railings. This research provides crucial empirical evidence and novel insights for model selection and synthetic data strategies, facilitating the practical deployment of autonomous driving perception systems in rural areas.
Read the full abstract ▾
Computer Vision
Mitigating Compounding Error via Video Representation Regularization
Taiye Chen, Qi Zhang and Yisen Wang
Video diffusion-based world models enable long autoregressive
video generation for robotics, autonomous driving and simulation tasks, yet sliding-window autoregressive inference suffers from severe error accumulation that degrades frame quality over time. Although this phenomenon has been widely observed, the underlying mechanism of compounding error and how to achieve stable long-horizon generation remain largely unresolved. In this paper, we investigate the internal representation dynamics of
video world models and discover that compounding error is tightly coupled with dimensional collapse of hidden representations. Specifically, the effective rank of model representations sharply decreases at the onset of generation drift, revealing a strong connection between representational degradation and long-term rollout instability. Furthermore, we find that pure training data scaling fails to boost model resistance to error drift, contradicting mainstream scaling paradigms. To address this problem, we propose
video representation regularization, a lightweight training constraint that stabilizes latent representations and suppresses iterative error accumulation. Compared with
Diffusion Forcing, our method achieves improvements from 38.65 to 55.56 and from 44.37 to 72.08 on the Aesthetic Quality and Imaging Quality metrics of VBench. Our work establishes the first connection between autoregressive
video drifting and model internal representations, adopts erank as a quantitative metric for error accumulation, reveals counterintuitive scaling limitations for
video world models, and presents a simple yet effective regularization strategy to improve long
video generation robustness.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Lottery Tickets Are Not Deployment Tickets
Bum Jun Kim
Reports on how sparsification, compression, and lottery tickets change model behavior have been mixed in the prior literature, with beneficial effects observed in some studies and adverse effects in others. Moreover, prior work has not considered actual deployment conditions, where decision logic is already fixed for the incumbent. To assess these mixed findings from a practical standpoint, we study the production-replacement question at the deployment level, namely whether an accuracy-matched lottery ticket or another sparse challenger can replace an incumbent dense model without reconfiguring downstream decision logic. We therefore audit a broad, protocol-specific panel of deployment-relevant behaviors spanning calibration, OOD response, class-level reliability, representations, and downstream policy decisions, and summarize clean-accuracy-excluded deviations with a behavioral-compatibility distance. Across extensive experiments, sparse candidates repeatedly recover dense-reference accuracy yet remain behaviorally different; in several study-band-matched settings, LTs also show lower corruption accuracy. In small-gap settings with fixed-threshold policy diagnostics, lottery-ticket replacement changes 7% to 10% of accept–review decisions. This churn creates precisely the burden that drop-in replacement is meant to avoid: reconfiguring and revalidating downstream decision logic. These findings establish the limits of clean-accuracy certification: Establishing compatibility with a fixed incumbent is distinct from attributing churn uniquely to sparsity or treating every measured deviation as harmful. Our theory explains the routing result: Even exact pointwise top-1 agreement cannot bound fixed-threshold decision changes, and small confidence shifts near the operating boundary can generate first-order routing churn.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CR
InkShield: Writing Style Protection Against Unauthorized Handwriting Mimicry
Jian Xiong, Wenbo Jiang, Zihan Wang et al. (+4 more)
Recent handwritten text generators can reproduce a writer's style from publicly available references, posing risks of document forgery and identity misuse. An attacker may use a publicly available handwritten note or signature sample to generate forged recommendation letters or authorization forms, leading to document fraud, identity misuse, and misleading decisions. However, existing protections against unauthorized image editing or synthesis transfer poorly to handwriting style mimicry. Designed for natural images with complex backgrounds, they often optimize perturbations over the whole image. For sparse handwriting images, such global perturbations become conspicuous in blank background regions and largely degrade the visual quality. In this work, we propose InkShield, a proactive writing-style defense that protects reference images before release. InkShield selects a decoy writer to define a style-displacement direction, optimizes perturbations with a frozen handwriting-generation surrogate, and confines them to ink-stroke edges to avoid conspicuous background artifacts. On IAM, the average Top-1/Top-5 rates at which generated samples are retrieved as the target writer by two independent writer evaluators decrease from 11.94%/36.52% to 2.03%/8.79%. Meanwhile, the protected references remain visually close to the originals (LPIPS 0.0078), and the generated text remains readable. InkShield also exhibits transferability to other handwriting generators. Overall, InkShield provides practical protection against unauthorized handwriting style mimicry.
Read the full abstract ▾
All authors Jian Xiong, Wenbo Jiang, Zihan Wang, Rui Zhang, Wenshu Fan, Hongwei Li, Guowen Xu
Computer Vision
Robust RPC Bundle Adjustment for Multi-Date Satellite Imagery with Season-Invariant Correspondences
Roger Marí, Elías Masquil, Xavier Bou et al. (+2 more)
Accurate refinement of Rational
Polynomial Camera (RPC) models is essential for high-quality satellite image geolocation. In ground control point (GCP)-free multi-view pipelines, this refinement is commonly performed through bundle adjustment from automatically extracted image correspondences. However, conventional RPC bundle adjustment pipelines rely on handcrafted feature matching, which becomes unreliable in multi-date collections affected by seasonal, illumination, and land-cover changes. We propose an appearance-aware RPC refinement pipeline that combines learned local feature matching for season-invariant correspondences with global image descriptors for selecting visually compatible image pairs. This reduces redundant and error-prone matching while preserving the connectivity of the matching graph. Experiments on seasonally diverse WorldView-3 images show that our pipeline improves GCP-free relative RPC refinement over open-source baselines, achieving lower
geometric consistency errors while substantially reducing matching time on collections with 39-42 views. By making RPC refinement more robust to diachronic appearance variation, our approach enables more effective use of multi-date satellite imagery.
Read the full abstract ▾
All authors Roger Marí, Elías Masquil, Xavier Bou, Thibaud Ehret, Gabriele Facciolo
Computer Vision
Progressive Multimodal Alignment for Continual Instruction Tuning
Duzhen Zhang, Yahan Yu, Qiaoyi Su et al. (+2 more)
Multimodal Large
Language Models (MLLMs) rely on a projector to align visual representations with the
language embedding space, making it central to cross-modal understanding. In
Multimodal Continual Instruction Tuning (MCIT), however, shifting visual distributions and evolving instruction semantics cause this shared projector to drift, leading to projector-level forgetting, an issue largely overlooked by methods that focus primarily on the LLM backbone. We introduce Progressive
Multimodal Alignment (PMA), a framework that enables the projector to adapt continually while preserving previously learned
alignment. PMA detects
multimodal distribution shifts via a lightweight representation descriptor and progressively expands projector experts only when needed. An expandable router integrates expert outputs based on
multimodal features, while the original pretrained projector is retained as a stable
alignment anchor. This progressive mechanism balances stability and plasticity with sub-linear parameter growth and serves as a method-agnostic add-on to existing MCIT approaches. Extensive experiments on two recent MCIT
benchmarks demonstrate that mitigating projector-level forgetting yields consistent gains over prior state-of-the-art methods when combined with PMA. Moreover, PMA scales across diverse MLLM backbones, demonstrating robust and broadly applicable MCIT performance.
Read the full abstract ▾
All authors Duzhen Zhang, Yahan Yu, Qiaoyi Su, Jiahua Dong, Tielin Zhang
Computer Vision
From Keypoints to Predictive Distributions: Post-Hoc Uncertainty for YOLO-Pose Models
Alexej Klushyn, Juan Rivero Sesma, Florian Seligmann et al. (+3 more)
YOLO-Pose models provide efficient keypoint localization, but do not quantify the associated spatial uncertainty. We introduce a lightweight post-hoc probabilistic extension that augments a trained YOLO-Pose model with calibrated bivariate predictive distributions over keypoint locations, centered at the model's original predictions. Concretely, we train additional probabilistic heads with an importance-weighted negative log-likelihood to predict an input-dependent $2\times2$ dispersion
matrix for each keypoint, followed by Gaussian calibration for broad downstream compatibility or Student-$t$ calibration for distributional fidelity. Complementing this, we propose an evaluation protocol that combines a suite of distributional calibration diagnostics with average keypoint precision (AKP), a keypoint-level extension of the COCO AP protocol for assessing reliability rankings. Experiments on COCO show that the learned uncertainty estimates enable effective keypoint-level reliability ranking, Student-$t$ calibration best captures the empirical residual distribution, and uncertainty-based
pruning removes unreliable keypoints. A central application-level demonstration is
vision-based aircraft landing, where calibrated covariances for runway keypoints support uncertainty-aware aircraft position estimation and downstream sensor fusion.
Read the full abstract ▾
All authors Alexej Klushyn, Juan Rivero Sesma, Florian Seligmann, Richard Kurle, Kinh Tieu, Jayant Sen Gupta