Computer Vision
DobicVLM: Aligning Chest X-Ray Report Generation with Clinically-Grounded Programmatic Rewards via Group Relative Policy Optimization
Thanni Adewuyi, Angelica Obayi, Andem Aniekan et al. (+10 more)
Medical imaging is a cornerstone of diagnostics, yet automated chest X-ray report generation struggles with structural adherence, anatomical completeness, and semantic faithfulness. We introduce DobicVLM, a
vision-
language model combining supervised fine-tuning on MedGemma-4B with Group Relative Policy Optimization (GRPO) and clinically-grounded programmatic rewards. Our approach uses interpretable, rule-based reward components; structural verification, anatomical checklist, semantic similarity, and length constraints to enforce clinical standards without neural reward models. Trained on 1,000 de-identified image-report pairs from a private clinical
dataset (with ethics approval and compliance to local regulations), DobicVLM is evaluated via blinded expert review on 69 held-out cases. DobicVLM outperforms Gemini 2.5 Flash across the majority of criteria, achieving the highest impression accuracy (27.2%) and medical terminology (86.5%) compared to both Gemini 2.5 Flash and MedGemma 4B baselines, with minor trade-offs in completeness and referrals. This demonstrates GRPO's value for transparent
alignment in resource-limited settings. Keywords:
Vision-
Language Models, Radiology Report Generation, Reinforcement Learning, Medical AI, GRPO
Read the full abstract ▾
All authors Thanni Adewuyi, Angelica Obayi, Andem Aniekan, Samuel Okoko, Angel Ezendu, Ephraim Usani, Ademide Animasaun, Philip Chibundu, Christian Maurice, Mary Donald Essien, Oluwaseun Odunsi, Oluwasegun Oguntuase, Abiodun Adereni
Computer Vision
Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model
Sibo Wang, Jie Zhang, Shiguang Shan et al. (+2 more)
While Large
Vision-
Language Models (LVLMs), represented by LLaVA and GPT-4V, have demonstrated remarkable capabilities, their visual inputs remain vulnerable to
adversarial attacks, posing significant security risks. Existing defense methods predominantly target single-task scenarios (e.g., zero-shot classification) and consequently lack generalizability across various
multimodal tasks. To address this limitation, we propose a dual
adversarial fine-tuning framework that jointly optimizes visual and semantic supervision signals from two modalities, enhancing model robustness while generalizing across multiple downstream tasks. The proposed framework comprises two core components, i.e., $\textbf{Visual}$ supervision branch and $\textbf{Semantic}$ supervision branch. The former branch leverages features from clean images, extracted via a frozen original
vision encoder, to guide
adversarial robustness while the latter incorporates caption-image
alignment as a contextual signal to preserve semantic coherence under attack. Moreover, our method achieves cross-task robustness by simply replacing the CLIP
vision encoder in the original model, with no need of separate task-specific retraining or architecture modifications.Extensive experiments demonstrate that our approach outperforms the state-of-the-art method in
adversarial robustness evaluation across zero-shot classification, image captioning, and visual question answering (VQA) tasks.
Read the full abstract ▾
All authors Sibo Wang, Jie Zhang, Shiguang Shan, Xilin Chen, Wen Gao
Computer Vision · Cross-listed from cs.RO
NGPS: GPS-Denied Aerial Geo-Localization and 2.5D Reconstruction via Deep Satellite Image Matching and Multi-Rate Sensor Fusion
Sanket Sharma
We present NGPS (Next-Generation Positioning System), a visual geo-localization framework for high-altitude
UAVs that provides GPS-free absolute positioning by matching down-facing images to georeferenced satellite imagery with deep features. The system combines (1) adaptive confidence-weighted UKF fusion, where NGPS covariance is modulated by RANSAC inlier ratio, reprojection error, and match confidence; (2) velocity-predictive kernel extraction, using VIO velocity to predict the satellite search region; and (3) an asynchronous multi-rate temporal priority queue that interleaves absolute position (1-2 Hz), VIO (10-20 Hz), and IMU (100-200 Hz) updates in chronological order. Globally optimized poses from VINS pose-graph optimization, anchored by NGPS corrections, further enable real-time 2.5D georeferenced orthomosaic reconstruction. On five flight sequences (60-150 m AGL), NGPS achieves 2.94 m position RMSE, with worst-case ATE 6.04 m at 150 m AGL and 2 m/s, yielding a 3.5x improvement over standalone
monocular VIO. The system runs in real time on an NVIDIA Jetson Orin NX. Part of the implementation is open-sourced at https://github.com/snktshrma/ngps_flight.
Read the full abstract ▾
Computer Vision
Learning Explicit Physical Parameter Control and Benchmarking for Video Generation
Yanxun Li, Hao Wen, Bingze Song et al. (+7 more)
Recent advances in image-to-
video generation have improved visual realism, making physically grounded and controllable dynamics an important step toward future world simulation. Current models often generate plausible motion, but it is not reliably governed by explicit physical causes, and instance-level constraints can leak or become entangled in multi-object interactions. We attribute this gap to two missing pieces: large-scale, fine-grained physical parameterization, and model designs that correctly bind physical attributes to instances and emphasize dynamics over appearance. To bridge this gap, we introduce PhyParam-
Dataset, an interaction-centric collection of 130K physically simulated
videos with dense physical parameterization, including force vectors, object material properties, and environmental constants across five representative rigid-body motion types. Built on this data, we present PhyParam, a physics-guided image-to-
video diffusion model that conditions on object-level forces, masses, friction, restitution, and scene-level
gravity via a lightweight physical-
attention routing mechanism, and further improves motion learning with semantic-structural feature-space supervision. We also establish PhyParam-Bench, a
benchmark for physical-law consistency in image-to-
video generation, with a multi-level protocol evaluating temporal dynamics, spatial stability, and semantic–physical
alignment. Experiments show that PhyParam improves physical consistency while maintaining high visual fidelity, advancing explicit rigid-body physical-parameter control for image-to-
video generation. We will publicly release the
dataset,
benchmark, and code to support future research.
Read the full abstract ▾
All authors Yanxun Li, Hao Wen, Bingze Song, Jiashu Zhu, Aiming Hao, Chubin Chen, Jintao Chen, Jiahong Wu, Xiangxiang Chu, Miao Wang
Computer Vision
TAP-RAG: Task-Aware Policy Control for Long-Document Multimodal Question Answering
Zhong Ji, Keqi Jin, Yan Zhang et al. (+1 more)
Long-document
multimodal question answering requires more than retrieving relevant chunks from a large document. Different queries require different evidence behavior. Existing
multimodal RAG systems improve evidence access through text chunks, page images, graph links, or heterogeneous document elements, but they often apply a largely query-agnostic evidence-use strategy. We present TAP-RAG, a task-aware policy-controlled RAG framework for long-document
multimodal QA. TAP-RAG contains a main controller, the Task-Aware Policy Controller (TAPC), and two policy-guided evidence executors: Task-Aware Query-Guided Flow
Diffusion (TA-QFD) and Task-Aware Visual Enhancement (TAVE). For each query, TAPC predicts the task prior, estimates visual/local/global evidence signals, and produces an executable policy. TA-QFD then expands textual and structural evidence over the
multimodal document graph, while TAVE selectively inspects page images when visual or layout evidence is needed. A guarded synthesis stage fuses text, visual, and structural evidence and abstains when support is insufficient. On DocBench and MMLongBench-Doc, TAP-RAG achieves the best overall accuracy among the compared systems, improving over a matched
multimodal-RAG baseline by +9.1 points (61.1 to 70.2) and +4.5 points (42.2 to 46.7), respectively.
Read the full abstract ▾
All authors Zhong Ji, Keqi Jin, Yan Zhang, Jiasheng Li
Computer Vision
SynGallery: A Synthetic Gallery of Real Paintings for Instance-Level Artwork Recognition
Patryk Bartkowiak, Jakub Markil, Bartosz Kotrys et al. (+3 more)
Instance-level artwork recognition requires matching a handheld visitor photograph to a specific work in a large museum collection. This is challenging because painting
datasets typically provide clean catalog images for training, while test queries are captured under oblique viewpoints, gallery lighting, reflections, frames, and other scene-level variations. We present SynGallery, a synthetic gallery
dataset for artwork
retrieval that addresses this gap without collecting additional real photographs. Starting from catalog images of real paintings, we place each artwork into a procedurally generated 3D gallery scene and render it from multiple viewpoints under varied
geometric and appearance conditions, while preserving the exact identity of the original work. The resulting
dataset contains 24,490 rendered views of 4,898 paintings from the Met
benchmark. We show that these synthetic views provide a stronger training signal than the corresponding studio photographs. At the same number of training data points, training only on SynGallery improves art painting recognition from 67.18 to 73.47 GAP$^-$. When added to the full Met training set, SynGallery improves the published
benchmark protocol from 35.97 to 38.48 GAP. Ablation experiments show that the gain comes from scene-level view variation rather than photographic realism: reducing the five rendered viewpoints to a single frontal view removes most of the improvement, while simulating capture artifacts such as blur, sensor noise, and image compression consistently reduces performance.
Read the full abstract ▾
All authors Patryk Bartkowiak, Jakub Markil, Bartosz Kotrys, Dominik Michels, Sören Pirk, Wojtek Palubicki
Computer Vision
Local Label-Informed Feature Transfer for Generating Ground-Truth Medical Images: A Comparison of GAN- and Diffusion-Based Approaches
Rick Wilming, Irem Ozseker, Luca Matteo Cornils et al. (+4 more)
Validating Explainable Artificial Intelligence (XAI) methods in medical imaging requires ground-truth data with known locations of informative features. However, current approaches rely on expert annotations, which are prone to labeling errors, or on hand-crafted artificial perturbations superimposed onto healthy images to mimic lesions or malignant features, which lack clinical realism. We present Local Label-Informed Feature Transfer (LLIFT), a framework for generating semi-synthetic
brain magnetic resonance images with realistic lesions placed in user-controlled regions, which does not require pixel-level lesion annotations during training. We implement LLIFT with two generative paradigms: LLIFT-GAN, a custom GAN that learns pathological features from binary class labels alone, and LLIFT-DM, a
diffusion-based inpainting pipeline conditioned on bounding-box masks via ControlNet. Both approaches are evaluated on
brain magnetic resonance imaging data derived from the Human Connectome Project. In evaluations, both achieve Fréchet Inception Distance scores, with respect to the real pathological distribution, that are comparable to the inter-class reference between healthy and pathological images in the given
dataset. Furthermore, qualitative inspection confirms the realism of lesion structures. The resulting
benchmark datasets provide spatially controlled ground truth data for evaluating XAI methods in medical imaging.
Read the full abstract ▾
All authors Rick Wilming, Irem Ozseker, Luca Matteo Cornils, Ahcène Boubekki, Benedict Clark, Danny Panknin, Stefan Haufe
Computer Vision
Wave2Body: Rethinking mmWave Human Pose Estimation as Radar-to-Body Token Translation
Bo Liang, Chen Gong, Wei Gao et al. (+1 more)
Millimeter-wave (mmWave) radar enables
privacy-friendly human sensing, but its sparse point clouds are physical measurements of view-dependent electromagnetic reflections and only indirectly characterize body articulation. Recovering a complete 3D pose from such partial, geometry-dependent observations is therefore under-constrained. Existing methods directly regress joint coordinates from paired radar-pose data, relying on the same limited paired supervision to learn radar perception, human-body structure, and their
alignment. This coupling can encourage
dataset-specific shortcuts under ambiguous radar observations. We propose Wave2Body, a radar-to-body
token translation framework that decouples these learning targets using a self-supervised mmWave tokenizer, a pretrained compositional body tokenizer that defines the output space, and a lightweight translator between them. Experiments on M4Human and mmBody show that Wave2Body achieves stronger cross-domain generalization than previous methods while incurring much lower computational costs for training and inference. All the code and experiment results are publicly available at https://github.com/Galaxywalk/Wave2Body.
Read the full abstract ▾
All authors Bo Liang, Chen Gong, Wei Gao, Chenren Xu
Computer Vision
Reliability-Aware 3D Geometric Injection for Universal Person Re-identification
Bohan Su, Jiashuo Wang, Fangyi Liu et al. (+1 more)
Universal person re-identification (ReID) aims to retrieve pedestrian identities across diverse real-world scenarios, including severe occlusions, clothing changes, and cross-modality shifts, within a unified model. However, existing 2D representations fundamentally struggle with spatial ambiguities due to a lack of depth and topological awareness, while naively introducing
monocular 3D priors often causes severe negative transfer due to
geometric estimation noise under extreme visual degradation. To safely harness the clothing-invariant and canonical structural properties of 3D geometry, we propose UniGeo, a Universal
Monocular 3D-Enhanced ReID framework driven by a Consistency-Aware Reliability Gate and Dual-Stream Residual Fusion. Specifically, the processing of 3D information is strategically decoupled into
geometric extraction and dynamic utilization. To provide pure structural compensation, we project
monocular 3D parameters into kinematic joint representations, explicitly capturing instance-level
geometric topology to resolve appearance-based ambiguities. To robustly incorporate these cues without perturbing the reliable 2D feature space, we isolate the 3D prior as a late-stage structural residual; modulated by the consistency-aware gate, this mechanism adaptively filters
geometric noise and enables controlled fallback to the pure 2D baseline. Extensive experiments show that our method improves challenging, structure-sensitive scenarios while preserving competitive performance on clean domains. Code is available at https://github.com/BohanSu/UniGeo.
Read the full abstract ▾
All authors Bohan Su, Jiashuo Wang, Fangyi Liu, Mang Ye
Computer Vision
Think Sparse, Predict Dense: Continuous Thought Machines for Image Super-Resolution
Zekai Shi
Continuous Thought Machines introduce an internal temporal dimension in which
neuron-level histories and synchronization-derived representations evolve over a sequence of thought ticks. Extending this mechanism to dense visual prediction is non-trivial, because tasks such as image super-resolution require spatial evidence to remain available at every output location rather than being compressed into a single global representation. In the proposed window-level use of CTM, the thought dynamics produce a compact summary representation for each local window. DQ-CTM transforms this compact thought representation into window-aligned dense queries through a structured low-rank, parameter-efficient compact-to-dense query mechanism. Each position within a window receives its own query, while shared thought dynamics progressively refine the dense representation across ticks. In its super-resolution instantiation, termed ThinkSR, encoded feature maps are partitioned into local visual windows without
token pooling, restored to the original feature field after shared refinement, and decoded into a high-resolution image. Preliminary experiments under a fixed four-tick training horizon reveal a progressive reconstruction trajectory. PSNR-Y increases from 28.1045 dB at $T=0$ to 30.2817 dB at $T=4$, while PSNR-RGB increases from 26.6271 dB to 28.7781 dB and the mean $\ell_1$ error decreases from 0.034602 to 0.023545. All 100 evaluated images improve from $T=1$ to $T=4$. These initial results establish the feasibility of sparse latent thought for dense spatial reconstruction and motivate broader continuous-thought architectures for dense
vision.
Read the full abstract ▾
Computer Vision
MOPDA: Mixed-Trajectory On-Policy Distillation for Language-Guided Industrial Anomaly Detection
Shuimu Chen, Jing Jin, Nan Su et al. (+5 more)
Large
vision-
language models (LVLMs) have shown strong potential for industrial anomaly detection (IAD) by providing image-level anomaly judgments and interpretable
reasoning. However, reliably translating generated judgments into precise pixel-level localization remains challenging. We propose Mixed-Trajectory On-Policy
Distillation for
Language-Guided Industrial Anomaly Detection (MOPDA), the first framework to introduce on-policy self-
distillation into LVLM-based IAD. For judgment learning, \method introduces Mixed-Trajectory Supervision, combining student-generated on-policy trajectories with evidence-conditioned teacher trajectories under a shared
token-level
distillation objective. Student trajectories preserve supervision on deployment-relevant response paths, while teacher trajectories provide complementary evidence-conditioned supervision. For dense localization,
Language-guided Visual Anchoring uses the final judgment as a compact semantic condition to construct image-specific normal and abnormal anchors, which are contrasted with dense visual features to produce anomaly maps. This keeps
language as semantic guidance while grounding pixel-level responses in visual evidence. Under a strict cross-
dataset zero-shot protocol on five IAD
benchmarks, \method outperforms the evaluated LVLM-based baselines on most detection, localization, and judgment metrics while remaining competitive with CLIP-based methods. Ablations further validate both mixed-trajectory supervision and final-judgment conditioning.
Read the full abstract ▾
All authors Shuimu Chen, Jing Jin, Nan Su, Hongbo Xu, Zebang Cheng, Wenming Yang, Fei Ma, Guijin Wang
Computer Vision
Open-Vocabulary Gaze Object Prediction: Benchmark and Method
Binglu Wang, Sensen Niu, Ying Chen et al. (+1 more)
Gaze Object Prediction (GOP) aims to localize and recognize the objects humans attend to, a task crucial for understanding human-centric interactions. However, existing methods are typically trained under a closed-vocabulary paradigm with a fixed label space and evaluated on scene-specific
datasets, limiting their applicability to real-world scenarios where gaze targets often follow a long-tail distribution or belong to unseen categories. To address this gap, we introduce Diverse Scenes for Gaze object prediction (DiSG), a
benchmark containing 86 in-the-wild categories that facilitates the evaluation of Open-Vocabulary GOP (OVGOP). Building on DiSG, we propose a framework that leverages text-driven object discovery to localize potential gaze candidates, with a gaze-guided selection module to pinpoint the intended target from the candidate objects. Furthermore, to better capture semantic knowledge across diverse in-the-wild categories, we introduce Gradient-Informed Selection Tuning (GIST) to selectively update parameters most relevant to a given class vocabulary. Extensive experiments demonstrate that our proposed model performs effectively in open-vocabulary settings and also outperforms existing methods in the conventional closed-vocabulary setting. The
benchmark and code is available at https://github.com/sensniu/ovgop.
Read the full abstract ▾
All authors Binglu Wang, Sensen Niu, Ying Chen, Guangyu Guo
Computer Vision
In-Context Learning for Wound Classification with Small Multimodal Language Models
George Martvel, Oskar Gustafsson, John Pavia et al. (+1 more)
Wound image classification is often treated as a task-specific supervised learning problem, requiring substantial amounts of manually labelled data and retraining when the label space or deployment setting changes. This study evaluated whether small
multimodal language models (SMLMs) can provide a training-free alternative for wound classification through
retrieval-based in-context learning (ICL). Experiments used two public wound-image
datasets: the Kaggle wound
dataset (1469 images, 10 classes) and the Medetec
dataset (560 images, 9 classes). Eleven SMLMs from the Qwen 3.5, Ministral 3, and Gemma 4 families were evaluated under zero-shot prompting and few-shot prompting with random support examples,
embedding-based k-nearest-neighbour (kNN)
retrieval, and kNN
retrieval followed by maximal marginal relevance reranking (MMR).
Retrieval-only weighted-kNN controls, support-set reduction experiments, and support-context size sweeps were used to assess the effects of
retrieval, model scale, and
prompt length. Query-conditioned ICL consistently outperformed zero-shot and random few-shot prompting. On the Kaggle
dataset, the best result was achieved by Qwen 3.5 27B with kNN+MMR, reaching 0.872 accuracy and 0.871 F1 score. On Medetec, Qwen 3.5 27B with kNN+MMR reached 0.678 accuracy and 0.670 F1. Larger models exceeded matched weighted-kNN controls, indicating use of retrieved examples beyond nearest-neighbour voting.
Retrieval-based ICL degraded modestly under support-set reduction, and most gains saturated with 8-10 support images.
Retrieval-based ICL allows SMLMs to perform adaptable wound image classification without task-specific retraining. Compact retrieved contexts may support practical and
privacy-conscious deployment, although performance remains dependent on model scale,
retrieval strategy, and
dataset difficulty.
Read the full abstract ▾
All authors George Martvel, Oskar Gustafsson, John Pavia, Ernst Ahlberg
Computer Vision
ZeroSplat: Generalized Referring Segmentation in 3D Gaussian Splatting
Jiayu Ding, Meilu Song, Xiaoyi Zhang et al. (+3 more)
Recent advancements in 3D Gaussian Splatting (3DGS) have enabled
language-guided scene understanding. However, existing Referring 3D Gaussian Splatting (R3DGS) methods are fundamentally restricted to single-target queries. To reflect the ambiguity of real-world instructions, we introduce the Generalized Referring 3D Gaussian Splatting
Segmentation (GR3DGS) task, which requires dynamically segmenting an arbitrary number of targets (0, 1, or $N$). To facilitate comprehensive evaluation of this new task, we construct two new
benchmarks: GR-LERF and GR-ScanNet. Crucially, existing R3DGS paradigms exhibit fundamental technical bottlenecks that severely limit their performance on the GR3DGS task: they lack intrinsic 3D point-level understanding by operating merely on 2D rendered pixels, and they incur prohibitive computational overhead by requiring per-scene optimization to embed heavy semantic features. To dismantle these bottlenecks, we propose ZeroSplat, a novel training-free and zero-feature framework. ZeroSplat lifts 2D
Vision-
Language Model (VLM) priors into 3D space through robust multi-view
geometric constraints. This strategy enables intrinsic point-level understanding without incurring any additional feature storage. Extensive experiments demonstrate that ZeroSplat significantly outperforms state-of-the-art methods across generalized and single-target scenarios while maintaining exceptional efficiency. Project Page: https://inkmind-ai.github.io/ZeroSplat
Read the full abstract ▾
All authors Jiayu Ding, Meilu Song, Xiaoyi Zhang, Hongbo Jin, Yichen Jin, Xiangtian Si
Computer Vision
UVFaceFusion: Fast Multi-view Topologically Consistent Face Reconstruction in the Wild via UV-space Neural Fusion
Xin Ming, Yuxuan Han, Junhai Yong et al. (+1 more)
Reconstructing high-fidelity facial geometry with an assigned
topology is essential for digital avatar creation and animation, yet existing automated methods often trade off
geometric fidelity and in-the-wild generalization. We present UVFaceFusion, a feed-forward framework for multi-view, fixed-
topology face reconstruction from daily images. Our key idea is to replace heuristic topological optimization with learnable neural fusion in a canonical UV space. Given multi-view images, we first obtain dense point maps and facial UV correspondences of each view using VGGT and Pixel3DMM, respectively. Then, the view-specific point maps are lifted into the canonical UV domain and fused with a novel mask-aware neural fusion network. The network predicts a complete UV-space point map, from which a fixed-
topology mesh is directly sampled. Although trained only on Ava-256, UVFaceFusion generalizes well to multiple public
benchmarks and in-the-wild captures, benefiting from its canonical UV-space geometry-to-geometry fusion that reduces dependence on
dataset-specific appearance and capture conditions. Experiments on various
benchmarks show that UVFaceFusion achieves state-of-the-art reconstruction accuracy while reconstructing a mesh from 16 input views in less than 3 seconds on a single RTX 4090. Code is available at https://github.com/grignarder/UVFaceFusion.
Read the full abstract ▾
All authors Xin Ming, Yuxuan Han, Junhai Yong, Feng Xu
Computer Vision
STS-NET: Spatio-Temporal Stress Network for Self-Supervised Crop Stress Detection using Satellite Image Time Series
Pradeep Dalal, Rajiv Ranjan, Sushil Ghildiyal et al. (+2 more)
Early and accurate detection of crop stress is essential to improve agricultural productivity and ensure global food security. However, collecting a large labeled crop stress
dataset is a challenging task. To address this challenge, we introduce a novel spatial-temporal stress network (STS-NET), built on a self-supervised 3D-convolutional autoencoder (3D-CAE), designed to utilize Satellite Image Time Series (SITS) data for crop stress detection. STS-NET exploits four vegetation indices: Normalized Difference Vegetation Index (NDVI), Normalized Difference Vegetation Index (GNDVI), Red-Edge Chlorophyll Index (RECI) and Normalized Difference Red-Edge Index (NDRE) obtained from high resolution Planetscope imagery to capture spatiotemporal stress patterns. The model is trained on our BSPT (Barnala Spatial-Temporal)
dataset and evaluated on a real-world sugarcane
dataset collected over a year from a 2.5-acre test plot located in Lakhimpur-Kheri (LK) district in Uttar Pradesh in India. STS-NET achieved a precision of 97. 98% for water stress, 85.08% for nitrogen stress, and 83.47% for combined stress. The results demonstrate the potential of STS-NET in effectively detecting stress in sugarcane crops with minimal reliance on labeled data. Furthermore, STS-NET can serve as a robust feature extractor for simpler models.
Read the full abstract ▾
All authors Pradeep Dalal, Rajiv Ranjan, Sushil Ghildiyal, Shashank Tamaskar, Neeraj Goel
Computer Vision
Moving Alphabet: A Controlled Study of Training Data for Text-to-Video Generation
Amber Yijia Zheng, Lu Liu, Raymond A. Yeh et al. (+1 more)
Text-to-
video generation has advanced significantly over the past five years through scaling of model size, data, and compute. Unlike model architecture, training data is often underexplored. Real-world data curation is complex and non-trivial, involving clip selection from raw
videos and captioning to create
video-text pairs for learning text-to-
video mappings. We study how data distribution and caption quality impact text-to-
video models. To enable controlled experiments, we introduce Moving Alphabet, a procedural testbed that renders letters with varying fonts, colors, sizes, and positions, moving in different directions and speeds against a black background. This design allows precise control over data distribution and caption quality by corrupting ground-truth metadata. Our experiments yield three findings: a) a diverse and balanced distribution of
video content and duration is critical for generalization; b) caption quality significantly affects both model performance and training efficiency, suggesting that text-to-
video models are bounded by
video understanding capabilities; and c) classifier-free guidance and fine-tuning on high-quality data provide partial recovery from models trained on corrupted captions, but cannot fully compensate for poor pre-training data. We believe these insights can inform the development of large-scale text-to-
video models, and we advocate for greater
attention to the science of pre-training data.
Read the full abstract ▾
All authors Amber Yijia Zheng, Lu Liu, Raymond A. Yeh, Xi Yin
Computer Vision
Image Editing Models are Numerical Solvers
Ulysse Mizrahi
We investigate whether a pretrained generative image-editing model can provide a common interface for numerical simulation. Physical inputs and solutions are rendered as images, while scalar quantities such as material properties, diffusivity, and loading parameters enter through lightweight adapters. Using established numerical and analytic solvers for supervision, we apply the same architecture and training protocol to heterogeneous
elliptic equations, forced heat and Burgers
evolution, complex Ginzburg-Landau dynamics, two-dimensional Navier-Stokes prediction, potential flow, elasticity, eikonal travel time, phase-field fracture, and entropic optimal transport. The results show that a pretrained image model can represent diverse static and time-dependent physical mappings, including unstable and shock-like behavior, when each task is expressed through a suitable visual encoding. This work is a capability study rather than an attempt to surpass specialized solvers. It also identifies fundamental constraints: image and latent representations complicate numerical range selection and direct enforcement of governing equations or invariants, while a failed Kuramoto-Sivashinsky experiment indicates that representation errors prevent meaningful long-horizon simulation of chaotic systems.
Read the full abstract ▾
Computer Vision
CGMap: A Geospatially Aware Deep Learning Framework for Crop Gap Mapping Using UAV
Karan Sharma, Rajiv Ranjan, Dinesh Kumar et al. (+1 more)
In India, crop germination is primarily monitored by visual inspection and manual counting, which are prone to errors, despite their crucial role in determining eventual yield potential. This paper highlights a deep learning based pipeline which uses object detection methods and drone imagery to assess and provide a precise count of sugarcane germination in fields. The approch uses a pre-trained AI model to find germinated plant sampling and identify gaps, also known as “bald spots”, which restricts field productivity. The techniques used here relies on the YOLOV8 architecture, which was trained on a carefully selected
dataset of
UAV photos taken in various agroclimatic zones of India. Here, we bring upon a novel orientation-normalization technique that uses minimum Spanning Trees (MST) to account for variations in planting geometry, allowing for dependable row and column extraction across a variety of field layouts. By converting detected seedlings into spatial point clouds, emergence gaps can be inferred from the anticipated spacing between plants. A geospatial germination map exported in Well-Known Text (WKT) format is the end result, and it can be easily incorporated into GIS platforms used by sugar mills and agronomists to direct transplant initiatives. Timely interventions based on the insights provided by the
algorithm can significantly increase yield, resulting in higher profits. Hence, support proper allocation of resources, avoid wastage, and enhance long-term sustainability.
Read the full abstract ▾
All authors Karan Sharma, Rajiv Ranjan, Dinesh Kumar, Shashank Tamaskar
Computer Vision
Privileged Lesion-Context Relational Distillation for Mask-Free Skin Lesion Classification
Abu Mukaddim Rahi, Md Mithun Hossain, Md Zulficar Hasan Joy et al. (+2 more)
Accurate skin lesion classification can benefit from lesion
segmentation masks, but requiring masks or an auxiliary
segmentation model during inference reduces clinical practicality and increases computational complexity. This work introduces Privileged Lesion-Context Relational
Distillation (PLCRD), a teacher-student framework that exploits lesion masks exclusively during training while preserving image-only inference. The privileged teacher jointly analyzes the original dermoscopic image and its mask-guided lesion region to learn lesion-specific and contextual diagnostic representations. An image-only student is then trained through complementary knowledge-transfer mechanisms that convey the teacher's diagnostic distribution, lesion-focused
attention, inter-lesion relational geometry, and lesion-context structure. PLCRD decomposes deep representations into lesion and contextual
embeddings and transfers their relational organization through inter-lesion similarity
alignment, lesion-context affinity matching, separation regularization, and class-aware relational learning. This formulation avoids direct feature matching between heterogeneous teacher and student architectures and enables the student to internalize mask-informed diagnostic structure without accessing masks at deployment. The framework was evaluated on HAM10000 using lesion-disjoint data partitioning and externally validated on ISIC 2018 without retraining. PLCRD achieved a lesion-level macro-F1 of 0.773 +/- 0.018, balanced accuracy of 0.764 +/- 0.023, and macro-AUROC of 0.976 +/- 0.002 on HAM10000, together with a macro-F1 of 0.732 +/- 0.008 on ISIC 2018. The results indicate that privileged lesion annotations can be transformed into transferable relational knowledge, yielding a practical and interpretable approach to mask-free skin lesion classification.
Read the full abstract ▾
All authors Abu Mukaddim Rahi, Md Mithun Hossain, Md Zulficar Hasan Joy, M. F. Mridha, Md. Jakir Hossen
Computer Vision · Cross-listed from cs.CR
GLID: Gated Local Intrinsic Dimension Repairs the Blind Spots of Face-Forgery Detectors
Guang Yang and Fengchen Liu
Fine-tuned foundation-model detectors dominate face-forgery
benchmarks, yet they stay blind to generator families absent from training. We present GLID, a detector that repairs this blind spot with geometry instead of data. GLID treats the patch
tokens of a single image as a sample from a
manifold and estimates their local intrinsic dimension (LID) at several depths of a frozen
vision transformer. This 12-dimensional, training-free signal enters a fine-tuned detector through a confidence gate whose strength is calibrated purely in-distribution. On a 16-axis cross-generator
benchmark, GLID reaches 0.805 mean AUC, first among retrained state-of-the-art baselines and never significantly behind the strongest of them on any axis. It lifts the generation axes by +0.084 AUC while moving reenactment by only -0.005. Two empirical laws explain the design. First, forged faces bend the
token manifold at family-specific depths: GAN artifacts peak at the last layer,
diffusion artifacts peak mid-network, and the pattern survives four backbones, three dimension
estimators, and non-face imagery. Second, fine-tuning absorbs auxiliary gains exactly where training data covers: injecting 1% target-family images erases a +0.100 gain, so
geometric signals matter precisely where data is unavailable. The deterministic signal also cuts the cross-seed spread of accuracy 5.5x. Code, preregistered analysis gates, and per-image scores accompany the paper.
Read the full abstract ▾
Computer Vision
Cross-Modal UAV Object Tracking: State-Aware Representation Learning and A Unified Benchmark
Yun Xiao, Zhihong Hong, Jiandong Jin et al. (+3 more)
Unmanned Aerial Vehicle (
UAV) object tracking has emerged as a popular research field with broad practical applications. Modern
UAVs are increasingly equipped with both visible light and thermal infrared sensors. However, due to constraints in communication bandwidth, computational resources and power consumption, current systems often activate one modality and switch between modalities to maintain robust tracking in complex scenarios. Such modality switch inevitably leads to significant appearance change and sudden spatial shift, posing great challenges for existing tracking
algorithms. To handle this problem, we propose a novel State-Aware Representation Learning Approach called SARLA, which perceives the inconsistent modality states of current frame with template and last frame in the target representations to adapt to the sudden changes in both appearance and position, for robust cross-modal object tracking. In particular, we propose the Modality State Aware Representation Module (MSARM) and Spatial State Aware Representation Module (SSARM). MSARM guides the model to learn appearance correlation, bridging the modality gap, while SSARM models cross-frame spatial correlation to mitigate sudden spatial shift impacts. In addition, we design a spatial shift prediction loss to further handle the effects of spatial variation caused by modality switch. To promote the development of this research field, we establish a large-scale
video benchmark called CM-UOT, which consists of 1079 cross-modal sequences with an average
video length greater than 621 frames and encompasses over 671K frames in total. Extensive experiments on CM-UOT
dataset demonstrate the superior performance of the proposed SARLA against 20 excellent tracking methods. The source code,
datasets, and evaluation protocols associated with this work are publicly available at: https://github.com/hongsmile365/sarla-.
Read the full abstract ▾
All authors Yun Xiao, Zhihong Hong, Jiandong Jin, Chenglong Li, Jin Tang, Amir Hussain
Computer Vision
Bounding Boxes to Improve Small Language Model Performance on Vision-Based Grading Tasks
Lachlan McGinness
The deployment of Small
Language Models (SLMs) in educational settings offers significant advantages in terms of
privacy, cost, and scalability. However, SLMs often struggle with complex
vision-based tasks, such as grading handwritten student exams, due to the high computational cost of processing large images and the visual distractions present on a full page. In this paper, we investigate whether cropping student responses using bounding boxes can improve the accuracy and computational efficiency of SLMs on a short-answer grading task. Using a
dataset of scanned handwritten responses from the 2025 Australian Physics Olympiad, we evaluate the performance of several models ranging from 4B to 72B parameters under varying conditions of Chain of Thought (CoT) prompting and image cropping. Our results demonstrate that using bounding boxes significantly improves grading accuracy and reduces computational cost (FLOPs) across models. We conclude that bounding boxes are a crucial pre-processing step for deploying SLMs in large-scale,
vision-based educational assessments.
Read the full abstract ▾
Computer Vision
Posterior Samplings are Missing Modalities Generators for Medical Image Translation
Jonghun Kim
Magnetic resonance imaging comes in various modality contrasts that provide complementary anatomical and pathological information. Complete
multimodal acquisitions are often unavailable due to time and protocol constraints. This leads to real-world
datasets with missing modalities, where conventional medical image translation methods are typically limited to fixed source-target settings or require retraining for each observed source-target pair. We propose a unified framework that formulates missing-modality generation as a linear inverse problem under a joint distribution and solves it via posterior sampling with a flow matching model. By learning a joint prior over the complete modality set, our method can reconstruct arbitrary missing modalities at inference time by guiding the sampling trajectory to enforce measurement consistency with observed modalities. We further mitigate inter-modality error propagation in multi-target generation by adopting a many-to-one sampling strategy. Experiments on BraTS and IXI
datasets show that our method achieves the best performance over baselines across most missing-modality scenarios. In downstream tumor
segmentation, synthesized images from our method result in higher
segmentation performance, indicating better preservation of clinically relevant structures.
Read the full abstract ▾
Computer Vision
Weakly Supervised Pathology-Informed Representation Learning for PET-Based Content Retrieval of Intra-Tumour Heterogeneity
Rajat Vashistha, Sandra Brosda, Lauren G. Aoude et al. (+8 more)
We propose a weakly supervised 18FFDG PET representation-learning framework for content based medical image
retrieval, using H&E derived information during training while preserving PET-only inference. The proposed method was designed to use H&E derived information during training while maintaining PET only inference. A teacher student training strategy was used to learn the PET tumour derived voxel representations, from which global and hotspot conditioned
embeddings were generated along with maps of intra tumour heterogeneity in our oesophegeal cancer test case. A progressive ablation strategy was used to evaluate the contribution of different supervision mechanisms.
Retrieval performance was assessed across cross-validation folds using metrics including mean average precision, normalised discounted cumulative gain and mean reciprocal rank. Additional analyses evaluated ablation performance, hotspot faithfulness through perturbation/deletion experiments, prototype-specific PET uptake behaviour and indirect patient level concordance between learned PET prototype classes and selected histomic features. Progressive introduction of pathology informed supervision and hotspot modelling improved PET
retrieval performance compared with global PET representations and conventional PET baselines. Across the ablation ladder, PET hotspot conditioned representations consistently provided stronger
retrieval than global
embeddings, indicating that focusing on informative tumour subregions improved sensitivity to intra tumour heterogeneity. Histopathology concordance further showed that the learned classes were not simply high uptake PET regions; instead, they demonstrated distinct heterogeneity in 18F FDG uptake.
Read the full abstract ▾
All authors Rajat Vashistha, Sandra Brosda, Lauren G. Aoude, Christine Jestin Hannan, James M. Lonie, Jessica Ng, Andrew Nathanson, Ellie Vloedmans, Caroline Cooper, Andrew P. Barbour, Viktor Vegh
Computer Vision
SkyEV: RGB-Event UAV detection and tracking dataset and baseline
Jakub Mandula, Sebastian Heusinger, Julian Moosmann et al. (+2 more)
Detecting
UAVs in air spaces has become increasingly important due to
UAVs widespread availability and easy usage. However, due to their small size, they are typically difficult to detect at a sufficient range. For the training of optimized detection
algorithms,
datasets have been published, covering optical sensing methods ranging from infrared to regular RGB to event-sensor-based. However, these
datasets often fail to reflect realistic counter-
UAV scenarios, lacking critical factors such as camera ego-motion, extremely small target scales, and diverse lens configurations, and introduce compression artefacts on the frame images. To address this gap, we introduce SkyEV, an open-source
dataset featuring highly synchronized uncompressed RGB and event-based data. SkyEV distinguishes itself by capturing complex real-world conditions, including significant camera motion and varied optical setups, which are essential for testing the fundamental trade-off between Field of View and detection range. Furthermore, we provide a unified data loader and establish an experimental baseline using a multi-modal architecture, demonstrating the
dataset's efficacy in detecting challenging, small-scale targets.
Read the full abstract ▾
All authors Jakub Mandula, Sebastian Heusinger, Julian Moosmann, Christian Vogt, Michele Magno
Computer Vision
Continual Video-MLLM Adaptation over Evolving Domains
Rui Cheng, Meixing Shi, Yuxiang Cai et al. (+3 more)
Video multimodal large
language models have shown strong capability in
video understanding, yet their adaptation to sequentially evolving domains remains underexplored. In real-world deployments,
video data often arrives continuously from heterogeneous domains, requiring the model to acquire new domain-specific knowledge without overwriting previously learned capabilities. Existing continual learning methods typically rely on shared adaptation spaces, which can induce severe cross-domain interference and catastrophic forgetting. We propose Distribution-Aware Expert Routing, a parameter-efficient framework for continual
Video-MLLM adaptation over evolving domains. DAER maintains domain-isolated lightweight experts while keeping the pretrained
Video-MLLM backbone frozen, thereby decoupling domain-specific adaptation from the general
multimodal knowledge of the pretrained model. To enable fine-grained specialization, we introduce an intra-domain distribution-aware routing mechanism that matches each input to expert-level prototype reservoirs using MMD. To address the absence of task identities at inference time, we further propose an inter-domain routing mechanism that performs prototype matching in a discriminative subspace for robust domain identification. In addition, we introduce adaptive domain merging to improve parameter scalability and adopt a two-stage optimization strategy to stabilize expert specialization during continual learning. We evaluate DAER by curating a domain-incremental
benchmark built from ten VidQA
datasets covering diverse visual environments and
reasoning demands. Experiments on two strong
Video-MLLM backbones show that DAER consistently outperforms prior methods.
Read the full abstract ▾
All authors Rui Cheng, Meixing Shi, Yuxiang Cai, Jingcai Guo, Jianwei Yin, Zhi Chen
Computer Vision · Cross-listed from cs.RO
Confidence-Gated Vision-Only Heading Alignment for UAV-UGV Cooperative Systems
Reza Ahmari, Vahid Hemmati, Parham Kebria et al. (+3 more)
Vision-based heading prediction is useful for
UAV–UGV cooperation, but accurate prediction alone does not guarantee that every predicted heading should be issued directly as a control command. This paper investigates the decision problem of when and how a fixed
vision-based heading predictor should be trusted for command issuance. A lightweight confidence-gated framework is proposed in which execution decisions are made using two interpretable reliability proxies derived from the perception stream: bounding-box area as a visibility-related proxy and short-window variation in predicted heading as a stability-related proxy. During low-confidence intervals, the framework compares the baseline freeze-HOLD policy with a bounded-blend fallback that updates the issued command conservatively. The method is evaluated on a real
UAV–UGV
dataset under clean and perturbed conditions. The results show that confidence gating creates a clear trade-off among execution rate, executed-frame accuracy, issued-command accuracy, and smoothness. The results further show that sparse execution can cause severe stale-command error under the baseline freeze-HOLD policy, whereas the bounded-blend fallback substantially improves command-level behavior under the same gate decisions. These findings highlight that reliable perception-driven autonomy depends not only on prediction accuracy, but also on decision-aware command issuance during low-confidence
Read the full abstract ▾
All authors Reza Ahmari, Vahid Hemmati, Parham Kebria, Olusola Odeyomi, Kaushik Roy, Abdollah Homaifar
Computer Vision
Generative World Renderer at the Speed of Play
Guixu Lin, Zheng-Hui Huang, Siqi Yang et al. (+3 more)
Generative world renderer AlayaRenderer receives structured world states exported from physics engines and synthesizes RGB frames. Unlike models that generate frames from text/control-hints
prompts, AlayaRenderer preserves scene structure without altering the underlying world dynamics. This demonstrates an alternative path toward interactive world modeling and user-controllable play. However, the original AlayaRenderer is too computationally expensive for real-time deployment. This technical report introduces AlayaRenderer-Flash, a real-time-oriented generative forward world renderer that pushes AlayaRenderer from 0.56 FPS to 31.54 FPS, reaching the speed of play. AlayaRenderer-Flash reformulates the original renderer as a few-step autoregressive streaming model and introduces lightweight distilled codecs for efficient latent encoding and frame reconstruction. It retains the teacher model's G-buffer and text-
prompt interfaces while enabling continuous rendering over input streams of unbounded length. We evaluate AlayaRenderer-Flash on G-buffer streams across content preservation, temporal consistency, cross-window stability,
prompt controllability, and runtime efficiency. Our results show that AlayaRenderer-Flash substantially reduces inference cost while preserving the core rendering capabilities of the teacher model. By integrating AlayaRenderer-Flash with a physics engine, we build a fully playable generative world running at 30 FPS.
Read the full abstract ▾
All authors Guixu Lin, Zheng-Hui Huang, Siqi Yang, Ming-Hsuan Yang, Kaipeng Zhang, Zhixiang Wang
Computer Vision
Decoupled Pipeline with Proposal Reranking and Score Fusion for Positive-Unlabeled Marine Species Detection
Robert James Brock, Sebastian Maximilian Krupa and Jason Kahei Tam
The FathomNetCLEF 2026 competition combines
underwater object detection and fine-grained marine species classification under a positive-unlabeled evaluation setting. The provided training labels are sparse, while the hidden test set is out-of-distribution relative to the training imagery, creating both annotation incompleteness and source-shift challenges. We describe DS@GT ARC's multi-stage system developed for this setting while keeping model training restricted to the data provided by the competition. The final private-leaderboard model uses a frozen Megalodon YOLOv8x detector as a class-agnostic proposal generator, combines global and tiled inference with tile-edge filtering, classifies expanded proposal crops with a LoRA-finetuned DINOv3 ViT-H classifier, and ranks predictions using weighted
geometric fusion of detector and classifier confidence. This system placed 12th out of 102 teams. A closely related variant added a locally trained TTN-inspired validity head as a light reranking signal, improving public-leaderboard and proxy-evaluation performance but slightly reducing private-leaderboard performance. Across experiments, the strongest lesson was that train-derived validation and detector-only metrics were not reliable enough for model selection. Instead, we used proxy
datasets only for validation and comparison, and combined those signals with leaderboard feedback and targeted ablations. These experiments showed that reserving proposal recall, avoiding over-aggressive filtering, and improving downstream ranking were more effective than fine-tuning the detector or directly training on noisy pseudo-labels. Code: https://github.com/dsgt-arc/fathomnetclef-2026.
Read the full abstract ▾