Computer Vision · Cross-listed from cs.LG
Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View
Yixian Xu, Yuanrui Zhang, Shengjie Luo et al. (+2 more)
Reinforcement learning (RL) post-training provides a direct way to align
diffusion models with human preferences and task-specific rewards. However, current RL
algorithms for
diffusion models remain fragmented: reverse-trajectory methods rely on discretized likelihood ratios, whereas forward-matching methods train on reward-labeled noising versions of the rollout samples. This paper shows that these seemingly different losses arise from a single path-space principle. Starting from the regularized
diffusion-RL objective, we use importance sampling between sampling SDEs to obtain an explicit policy-gradient
estimator on trajectory space. The
estimator contains the
stochastic Itô integral underlying Flow-GRPO-type updates; we derive an equivalent variance-reduced value-gradient form that recovers the forward-matching structure of AWM and DiffusionNFT. This identifies the empirical gap between these method families as a variance-reduction effect rather than a difference in RL principle. The derivation yields a unified design space organized by value-gradient estimation, weight functions, and sampling choices. Within this space, we propose a multi-sample KDE value-gradient
estimator that reuses rollout groups, together with scale-bounded weight families that retain stable existing recipes while excluding singular ones. Experiments on SD3.5-M and Qwen-Image models validate the variance-reduction explanation and show that the resulting recipe improves over prior
diffusion-RL baselines.
Read the full abstract ▾
All authors Yixian Xu, Yuanrui Zhang, Shengjie Luo, Liwei Wang, Di He
Computer Vision
GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure
Mohamed Abdelsamad, Bin Yang, Michael Ulrich et al. (+4 more)
3D object detection from
LiDAR point clouds is a core problem in autonomous driving. Recent advances in self-supervised learning (SSL) enable scalable pretraining and transfers well to per-point tasks such as semantic and panoptic
segmentation, but transfer to 3D detection remains weaker. We analyze recent SSL methods and find that most objectives are defined only on measured
LiDAR returns from visible surfaces, leaving occluded and unobserved regions unconstrained. This visible-surface bias can be sufficient for point-wise prediction, but 3D detection requires robustness to missing structure. To address this gap, we propose GhostPoint, an SSL framework that hallucinates latent features in local neighborhoods around discovered instances, generated via a novel instance voxel dilation. In GhostPoint, an encoder processes observed returns, and an additional predictor infers neighborhood representations from observed context. In addition to standard encoder-level supervision, we introduce a predictor-level supervision scheme on sampled voxels from generated neighborhoods. Specifically, observed (visible/masked) voxels match teacher-encoder targets, while unobserved voxels match teacher-predictor
hallucinations. This design encourages the learned representation to explicitly model structure beyond observed returns. Extensive evaluations on nuScenes and Waymo demonstrate that our method achieves state-of-the-art performance, consistently improving downstream 3D detection, especially under sparse scans and limited labels.
Read the full abstract ▾
All authors Mohamed Abdelsamad, Bin Yang, Michael Ulrich, Miao Zhang, Yakov Miron, Alexandru Paul Condurache, Abhinav Valada
Computer Vision · Cross-listed from eess.IV
UMPIRE-Net: Unrolled Magnitude-Phase Regularization Network for Accelerated MRI
Mahdi Saberi, Toygan Kiliç and Mehmet Akçakaya
MRI reconstruction from undersampled k-space measurements is an ill-posed inverse problem. Physics-driven deep learning (PD-DL) methods have shown strong performance for this task by combining the MRI forward model with learned image regularization within
algorithm-unrolling frameworks. However, most existing PD-DL methods reconstruct complex-valued images directly, thereby implicitly coupling magnitude and phase within a single learned representation. This coupled regularization may be suboptimal in reconstruction settings where accurate phase modeling plays an important role, such as partial
Fourier (PF) imaging, where recovery of the omitted asymmetric k-space measurements depends on the underlying image phase. In such scenarios, explicit modeling of magnitude and phase as separate components may reduce the reliance on externally estimated or predefined phase information. To this end, we propose UMPIRE-Net (Unrolled Magnitude-Phase In REgularization Network), a PD-DL method that introduces separate learned regularizers for magnitude and phase components, together with a novel data-fidelity formulation that enforces measurements consistency. We evaluate UMPIRE-Net for accelerated MRI with PF across different
datasets and acceleration factors. Experimental results demonstrate that our proposed method improves reconstruction quality compared with a conventional complex-valued PD-DL baseline, yielding sharper images and reduced artifacts. Code available at: https://github.com/MahdiSaberii/UMPIRE-Net
Read the full abstract ▾
Computer Vision
CRAFT: Constrained Reward via Attention Fine-Tuning for Subject Personalization without Composed Targets
Jihun Park, Kyoungmin Lee, Jongmin Gim et al. (+6 more)
Subject-driven image personalization—generating new images that preserve the identity of one or several reference subjects in novel scenes—is a foundational capability for modern visual content creation. It is currently dominated by generalized methods that fine-tune a pretrained
multimodal diffusion transformer (MMDiT) on hundreds of thousands to millions of paired (reference, composed-target) examples, where each composed target is a synthesized image of the subject in a novel scene. Producing such targets demands a costly multi-stage curation pipeline—LLM-based
prompt generation, T2I-based composed-target synthesis, reference-subject extraction, VLM-based quality filtering, and correspondence labeling—and tightly couples each method to a particular target synthesizer and curation choice. We introduce CRAFT (Constrained Reward via
Attention Fine-Tuning), a single-step ReFL framework that fine-tunes a pre-trained reference-aware MMDiT via LoRA adapters using a compact reference-only data construction—$10$K reference images and subject masks, with no composed-target supervision. CRAFT realizes a Where to look principle:
attention-level rewards align noise- and phrase-
token attention with the correct reference subject, and the resulting per-subject
attention masks gate a pixel-level identity reward to keep image-space supervision consistent with the learned
attention routing. Applied to FLUX.2-klein-9B, CRAFT achieves state-of-the-art performance on XVerseBench \rev{while using no composed-target supervision—only $10$K reference-only samples, whereas prior generalized methods require $150$K to over $2$M composed-target pairs}. The same recipe transfers to other reference-aware backbones, consistently improving performance. Project page: https://jihun999.github.io/projects/CRAFT/.
Read the full abstract ▾
All authors Jihun Park, Kyoungmin Lee, Jongmin Gim, Hyeonseo Jo, Jaeyeul Kim, Han Zou, Zhenpeng Zhan, Yan Zhang, Sunghoon Im
Computer Vision
IRGNN: Efficient Invariant Radar Graph Neural Network for Radar Point Cloud Object Detection
Xiao Guo, Wanke Xia, Lili Yang et al. (+1 more)
Perception is a fundamental component of autonomous driving systems. While
LiDAR-based methods have achieved remarkable progress in object detection, their reliability can degrade under adverse weather conditions. Radar point clouds provide a robust alternative due to their resilience to bad weather and low-illumination scenarios. However, radar point clouds are typically sparse, unordered, and less informative than
LiDAR data, making it challenging to directly apply existing
LiDAR-based perception methods. To address these challenges, we propose IRGNN, an Invariant Radar Graph Neural Network for radar point cloud object detection. IRGNN first reconstructs radar point clouds into graph representations using translation- and rotation-invariant feature designs, enabling robust modeling of sparse radar measurements. It then employs an improved message passing neural network (MPNN) with residual connections and a virtual node layer to enhance local feature propagation and global context modeling. Finally, task-specific heads are applied to the learned graph representations for object classification and bounding box prediction. Experimental results on the RadarScenes
dataset show that IRGNN outperforms existing radar-based object detection methods and achieves competitive performance. In addition, IRGNN significantly reduces computational cost and
memory usage during inference, demonstrating its effectiveness and practical potential for efficient radar-based perception in autonomous driving.
Read the full abstract ▾
All authors Xiao Guo, Wanke Xia, Lili Yang, Caicong Wu
Computer Vision
Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Dissemination
Shuo Liang, Yixing Ma, Pengfei Zhou et al. (+33 more)
Recent
video generators can fabricate realistic depictions of wars, disasters, public emergencies, and other real-world crises, creating substantial risks of misinformation. Existing
benchmarks, however, provide limited evidence on detector and generator behavior in such settings, including how detectability varies with generation conditions, how people perceive generated
videos, and whether detectors remain reliable during social dissemination. To address this gap, we introduce RA-Bench, a
benchmark for AI-generated
video detection that uses Real
videos as Anchors. RA-Bench contains 17,886
videos, comprising 1,830 real-
video anchors across 10 social-risk categories and 16,056 generated clips from four open-source and five closed-source generators. Based on RA-Bench, we organize our evaluation along three dimensions. We first assess detector generalization across seven traditional detectors, ten zero-shot
multimodal models under three review settings, and two MLLMs specifically fine-tuned on AI-generated
video detection. Across these methods, none of the three detector families generalizes consistently across RA-Bench instances. We then examine how detectability varies with generation quality, conditioning information, and sampling seeds. These analyses show that generation properties affect detector families differently, while source-level detection patterns remain stable across seeds. Finally, we study human authenticity judgments and detector reliability during social dissemination. We find that
videos that mislead people are also difficult for current detectors, and that social dissemination makes detection harder. Together, these findings show that current methods struggle to detect realistic AI-generated
videos, highlighting the need for detectors robust to evolving
video generators.
Read the full abstract ▾
All authors Shuo Liang, Yixing Ma, Pengfei Zhou, Zhenglin Wan, Xingyan Chen, Zihan Mei, Manting Li, Feihan Chen, Zhiwen Wang, Bin Xu, Haotian Zhang, Jiajun Song, Shiya Su, Run Liu, Zhenghang Ni, Yifa Yu, Jintao Hong, Bolong Feng, Yifei Liu, Zirui Zhang, Jingxuan Zhang, Songlin Zhao, Yifan Bai, Kang Tan, Yizhe Liu, Junhao Du, Yongtao Ge, Zhaopan Xv, Xinyuan Zhang, Mengru Ma, Chunhua Shen, Wei Wang, Yang You, Zheng Zhu, Kaipeng Zhang, Wangbo Zhao
Computer Vision
Yingjie Ma, Zitong Yu, Wei Jia et al. (+2 more)
Existing palm presentation attack detection (PAD)
datasets are often limited by static imagery, restricted acquisition conditions, or insufficient
multimodal video data, hindering systematic evaluation across environments, modalities, and attack types. We present GBU-Palm, a large-scale
multimodal video dataset and
benchmark containing 21,326
videos from 105 subjects and 210 palms across six acquisition environments, including bona fide, Print, and Replay presentations, with 6,310 synchronized RGB-NIR samples. We construct leakage-controlled protocols that separate palm identity and attack lineage and
benchmark four representative
video architectures under environment-matched and held-out-environment settings. Results reveal substantial architecture-dependent degradation under environmental shift and show that RGB-NIR fusion does not consistently outperform RGB-only input. We further analyze model behavior through true accept (TA), true reject (TR), false accept (FA), and false reject (FR) decomposition, spectral masking, temporal-order intervention, and frozen-backbone NIR probing, revealing distinct failure patterns and evidence utilization across architectures. GBU-Palm provides a unified and challenging
benchmark for developing and evaluating robust
multimodal palm PAD methods under cross-environment conditions.
Read the full abstract ▾
All authors Yingjie Ma, Zitong Yu, Wei Jia, Ajay Kumar, Linlin Shen
Computer Vision · Cross-listed from cs.CL
A Survey of Large Models in Sports
Yichen Xu, Jianzhe Ma, Chuhan Wang et al. (+4 more)
Sports have witnessed growing global enthusiasm in recent years, serving as a vital force for physical health, cultural exchange, social connection, and economic growth. The rapid advancement of large models, particularly (
multimodal) large
language models (M)LLMs, has demonstrated transformative potential to reshape sports understanding, analysis, and interaction across diverse domains. This paper presents a comprehensive survey of large models in sports, including (i) an overview of tasks and applications across different participant groups; (ii) a detailed analysis of sports-related
datasets and
benchmarks; and (iii) a critical discussion of current challenges and future directions. Our goal is to establish a foundation for advancing research and practical development of large-model-driven sports intelligence. An open-source GitHub repository is maintained at: https://github.com/Road2Redemption/Awesome_Large_Models_In_Sports1.
Read the full abstract ▾
All authors Yichen Xu, Jianzhe Ma, Chuhan Wang, Zhonghao Cao, Liangyu Chen, Wenxuan Wang, Qin Jin
Computer Vision
Weakly Supervised Polar Low Segmentation in Sentinel-1 SAR Imagery
Andrea Federici, Jakob Grahn, Giacomo Boracchi et al. (+1 more)
Polar lows are intense maritime cyclones that form rapidly at high latitudes. Deep learning can detect them in Synthetic Aperture Radar (SAR) imagery, but pixel-level
segmentation remains an open challenge. No pixel-level masks are available for training, and a polar low's extent is inherently subjective, with diffuse boundaries that even experts delineate inconsistently. We propose Constrained Region Erasing with Soft Targets (CREST), a Weakly Supervised Semantic
Segmentation (WSSS) framework that generates masks solely from image-level labels. Our approach builds on
Adversarial Erasing (AER), which iteratively mines discriminative regions, erases them, and retrains a classifier to reveal complementary cues that become pseudo-labels for
segmentation. However, standard AER also collects irrelevant background features, degrading pseudo-label quality. CREST addresses this with (i) a Constrained Ordinal Region Expansion (CORE) module that encodes the spatial-connectedness prior of polar lows, constraining region expansion from a high-confidence seed, and (ii) a Dynamic Bootstrapping (DB) loss that treats the mining order as a proxy for label reliability, attenuating supervision from noisier, later-mined regions. On Sentinel-1 SAR data, CREST follows the cyclone structure more closely than standard AER, and returns a multi-class rather than binary mask whose classes indicate the reliability assigned to each region. We further evaluate on BUS-UCLM breast ultrasound and PASCAL VOC person data, whose targets satisfy the same connectedness prior but come with the dense masks the SAR data lacks. On both
datasets, CREST performs better than the equivalent AER pipeline under identical settings.
Read the full abstract ▾
All authors Andrea Federici, Jakob Grahn, Giacomo Boracchi, Filippo Maria Bianchi
Computer Vision
TRIAGE: Risk-Controlled Pseudo-Label Admission for Annotation-Efficient Semi-Supervised Retinal OCT Classification
Md Ashraful Hossen Akash, Shyla Afroge, Abdullah Al Mamun et al. (+2 more)
The advanced retinal disease diagnosing imaging modality, optical coherence
tomography (OCT), encounters a lack of automation because of the high expenses for annotations performed by specialists. The use of SSL solves the problem of insufficient annotations using unlabeled B-scans; however, most of the current techniques for generating pseudo-labels are based on prediction confidence without considering the asymmetry between different types of errors. This paper proposes TRIAGE, a risk-controlled semi-supervised framework for OCT scans classification, which uses the concept of a patient-level conformal risk controller with an asymmetric cost
matrix. TRIAGE unites three crucial modules: a hierarchical classifier that is capable of working with partially abnormal supervision of the disease subtypes, a patient-grouped conformal risk controller with primal-dual coverage control, and a context-aware
Transformer teacher for cross-slice verification. On the
dataset from Noor Eye Hospital (16,822 B-scans, 161 patients, and 554 volumes) with a test set of unseen patients, TRIAGE demonstrates 89.66% scan-level accuracy, 0.8805 macro-F1, 0.9641 macro-AUC, and an 8.34% under-grading rate when using only 20% of the labeled data. With only 5% of the labeled data, TRIAGE keeps 76.88% accuracy and a 0.1656 under-grading rate. Compared with the other six state-of-the-art semi-supervised methods, TRIAGE significantly outperforms them with ablation study demonstrating the contribution of each module in the overall framework performance (by 42.7% in terms of under-grading rate comparing to fixed threshold methods). TRIAGE demonstrates 98.00% accuracy for 3-class classification with 1% labeled data and 95.94% accuracy for 8-class classification with 10% labeled data on the OCT-C8
dataset.
Read the full abstract ▾
All authors Md Ashraful Hossen Akash, Shyla Afroge, Abdullah Al Mamun, Md. Kishor Morol, Tze Hui Liew
Computer Vision
Intelligent Detection of Mechanical, Electrical, and Plumbing (MEP) Metrics Based on 2D Floor Plans
Tarandeep Singh Mandhiratta, ANK Zaman and Abdul-Rahman Mawlood-Yunis
This research developed a neural network-based model to extract various information from 2D floor plans. We detect lighting symbols, identify the appropriate type of light, and extract the associated texts with lights. The study aims to enable efficient floor designing and determining the number and type of lights needed per floor, i.e., allow efficient design and estimate the power requirement of the floor plan. The model was developed using Mask RCNN as the base. The images were annotated and converted into a Coco data format for training the model. The model achieved bbox\_mAP and segm\_mAP values of 0.7596 and 0.7111, respectively. It also performed well at different IoU thresholds, i.e., with bbox\_mAP 50 and segm\_mAP 75 values of 0.9850 and 0.9219, respectively. The developed model will help various industries, such as architecture and construction, to improve design time and create efficient workflows by automatically detecting Mechanical, Electrical, and Plumbing (MEP) objects from floor plans, and it is the first step towards building tools that will help energy-efficient building design.
Read the full abstract ▾
Computer Vision
Spatial Message Passing in Language Space for Pathology Image Interpretation
Jing-Cheng Yang, Hao-Jung Wang, Jinhao Du et al. (+4 more)
Multimodal Large
Language Models (MLLMs) can generate pathological descriptions from histological images, but gigapixel Whole Slide Images (WSIs) exceed their visual context limits. The standard tiling workaround makes WSIs tractable yet severs the tissue neighborhoods that define tumor-stroma interfaces and morphology. We introduce Spatial
Language Message Passing (SLMP), a framework that performs spatial
reasoning entirely in
language space, human-readable by construction. SLMP represents a WSI region as a spatial text graph: tiles are nodes initialized with MLLM descriptions, and edges encode spatial adjacency. For each tile, an LLM refines its description by integrating
language messages from adjacent tiles under a shared aggregation policy that, on the tile grid, acts as an adaptive local kernel operating on text rather than learned
embeddings. This policy is an inspectable
prompt that can be refined from model-observed tissue phenotypes via textual gradients, enabling automatic semantic optimization from local cellular context to broader tissue morphology without fine-tuning MLLM weights. On representative HER2 and CAMELYON16 regions, SLMP improves tile-level tumor description accuracy in settings spanning general-purpose and pathology-specialized backbones, with gains of +3.3 to +19.6 percentage points. Random-neighbor ablations confirm that these gains stem from spatial context rather than additional text alone, and inspecting the optimized policies reveals interpretable, tissue-specific decision rules. Besides, without any weight updates or fine-tuning the backbone MLLM, SLMP substantially improves general-purpose MLLMs and narrows its gap to pathology-specialized counterparts, offering a transparent and flexible mechanism for incorporating spatial
reasoning into MLLM-based pathology analysis.
Read the full abstract ▾
All authors Jing-Cheng Yang, Hao-Jung Wang, Jinhao Du, Yang Hu, Ming-shan Tsai, Jens Rittscher, Bin Li
Computer Vision
Conditional Neural Optimal Transport for Predicting Cellular Phenotypes from Molecular Structure
Gauthier Avité, Maxime Sanchez-Renauld, Nicolas Bourriez et al. (+1 more)
High-content microscopy enables systematic profiling of cellular responses to chemical perturbations, but the scale of the chemical space makes exhaustive phenotypic characterization experimentally infeasible. This motivates computational models that can predict image-derived phenotypes without acquiring the corresponding treated
cells. We formulate molecule-induced phenotype prediction as an inductive conditional transport problem in image representation space. Given a negative-control phenotype and the structure of a molecule, we aim to predict the phenotype induced by the corresponding molecule. We first evaluate classical optimal transport baselines and show that static couplings do not yield useful predictions on large-scale phenotypic image
datasets. We then introduce a molecule-conditioned Neural Optimal Transport (NOT) model with a Monge-Gap regularization training objective that learns to transport negative-control unperturbed phenotypes toward perturbed phenotypes using molecular structure as conditioning information. NOT recovers molecule-specific phenotypic effects while reducing microscopy-associated technical variation, thereby facilitating comparisons across experimental batches. On unseen active molecules, the model outperforms baseline approaches, demonstrating that chemically conditioned transport can generalize beyond the molecules observed during training. We identified the molecular encoder as the main limitation to this generalization, while transport in a compressed representation space improves performance and scalability. These results establish NOT as a promising framework for predicting cellular phenotypes from molecular structure and negative-control phenotypes, while highlighting the development of more informative molecular representations as a key direction for improving out-of-distribution performance.
Read the full abstract ▾
All authors Gauthier Avité, Maxime Sanchez-Renauld, Nicolas Bourriez, Auguste Genovesio
Computer Vision · Cross-listed from cs.SD
Acoustic UAV Detection in Battlefield Scenarios: Handling Noise, Domain Shift, and Weak Labels
Vadym Vilhurin, Volodymyr Sydorskyi and Andrii Shevtsov
Passive acoustic sensing offers a critical, cost-efficient, and, crucially, passive alternative for detecting small unmanned aerial vehicles. However, the practical deployment of acoustic systems is discouraged by extreme environmental noise and sensor-induced domain shift caused by heterogeneous hardware. This paper addresses these challenges by introducing a robust framework optimized for real-world battlefield conditions. We propose the integration of Per-Channel Energy Normalization (PCEN) and
attention-based pooling to enhance feature extraction under low signal-to-noise ratio scenarios. We further propose a domain-aware training strategy that leverages auxiliary classes and multi-microphone data to mitigate cross-domain performance degradation. Evaluated on a unique
dataset of combat-zone recordings from the Ukrainian frontlines, our approach significantly outperforms existing baselines, increasing the F1 score from 55.4% to 78.6%. This paper was originally presented at the International Conference on Military Communication and Information Systems (ICMCIS), organized by the Information Systems Technology (IST) Scientific and Technical Committee, IST-224-RSY - the ICMCIS, held in Bath, United Kingdom, 12-13 May 2026.
Read the full abstract ▾
Computer Vision
Seeing Red, Thinking Bad: Color Bias in Vision Language Models
Kohsuke Ide, Ryousuke Yamada, Yoshihiro Fukuhara et al. (+2 more)
Vision language models (VLMs) are increasingly used in industrial decision-making systems, such as recruitment support and recommendation. This motivates careful analysis of how VLMs process visual and textual information. In this work, we study how VLMs interpret text rendered as an image, and investigate the influence of visual styling biases. To this end, we introduce Stealth Visual
Prompts, which subtly change visual styling of text, such as color and contrast, while preserving semantic content. Using these
prompts, we systematically control the visual styling of words in text and measure their impact on the analysis performed by VLMs. We further analyze how such visual perturbations affect the latent representations of the
vision encoder. From our experiments, we observed that coloring positive words in green consistently shifts sentiment predictions toward a positive direction. As a result, VLMs often fail to properly account for negative words present in the text. Our analysis suggests that this behavior is correlated with changes in the latent representations of the
vision encoder induced by color variations. In addition, we show that reducing text–background contrast increases reliance on visually salient cues and leads to more incorrect Visual Question Answering (VQA) outputs. These results suggest that the visual styling of rendered text can guide VLMs' interpretation in ways that diverge from human semantic understanding. Project page: https://github.com/KohsukeIde/color-bias-vlm
Read the full abstract ▾
All authors Kohsuke Ide, Ryousuke Yamada, Yoshihiro Fukuhara, Hirokatsu Kataoka, Yutaka Satoh
Computer Vision · Cross-listed from cs.RO
PRM-as-a-Judge 1.5: A Toolkit for Robot Process Assessment
Yuyang Liu, Yanqing Shen, Ruike Chen et al. (+22 more)
Fine-grained robotic evaluation matters for understanding
embodied models, going beyond binary success rates and rule-based process scores. We present PRM-as-a-Judge 1.5, a toolkit for
robot process assessment that turns rollout
videos into dense progress curves and derives multiple fine metrics. PRM-as-a-Judge 1.5 introduces three metrics, building on version 1.0, that characterize failure-side progress, post-drawdown recovery, and success-side execution quality, helping users understand
embodied model capability. Based on the rollout
videos from
benchmarks, we perform a comprehensive assessment of the
embodied models, providing some fine-grained metric results and key findings. We further introduce RoboPulse++ to evaluate the reliability of process reward models (PRM), providing evaluators with a more accurate testing platform. Moreover, we release a user-friendly assessment suite, including the
benchmark, metric implementation, and visualization tools, to support reproducible manipulation process evaluation. We call on the community to rethink how
robots are evaluated and establish transparent, procedural, and reproducible assessment as a foundation for the next generation of
embodied intelligence.
Read the full abstract ▾
All authors Yuyang Liu, Yanqing Shen, Ruike Chen, Jifan Zhao, Yuxuan Tian, Yichi Zhang, Tianfeng Long, Zixuan Yin, Yipu Wang, Ziheng Qin, Wenxing Tan, Yang Shi, Mingyu Cao, Runze Xiao, Ziqi Wang, Zhixin Yin, Shiwei Chu, Yi-Fan Zhang, Yao Mu, Yuheng Ji, Yihao Wang, Jun Yan, Zhongyuan Wang, Pengwei Wang, Xiaolong Zheng
Computer Vision
MAGneT-3D: Monocular and Domain-Generalizable Temporal 3D Detection
Mohamed Kotb, Johannes Meier, Christoph Reich et al. (+3 more)
Monocular temporal 3D detection aims to detect objects in 3D, given a
monocular video. Query-based 3D detectors unify detection and cross-view association, but their learnable queries fit the spatial distribution of the training data (e.g., field-of-view). We show that this issue is especially severe when these models are applied to
monocular video, hindering generalization to unseen
datasets and environments. To address this limitation, we introduce MAGneT-3D, the first method for domain-generalized
monocular temporal 3D object detection. Instead of relying on static learnable queries, we propose a Domain-Robust Anchor Generator (DRAG) approach that adaptively derives 3D proposals during inference. To further enable domain generalization, we propose a Temporal Refinement and Identity Merging (TRIM) strategy, reducing dependence on specific 3D proposals. To enable comprehensive domain-generalization evaluation, we establish a cross-
dataset benchmark spanning nuScenes, Waymo, Lyft, and ONCE. Under zero-shot domain shifts, MAGneT-3D outperforms all baselines, improving NDS from 12.1% to 18.6% while also increasing in-domain accuracy.
Read the full abstract ▾
All authors Mohamed Kotb, Johannes Meier, Christoph Reich, Oussema Dhaouadi, Luis Denninger, Daniel Cremers
Computer Vision
Learning to Forecast Crop Growth from Earth Observation Data
Dominik Senti, Mehmet Ozgur Turkoglu, Michele Volpi et al. (+1 more)
Forecasting crop growth across agricultural landscapes is important for improving the productivity, resilience, and operational management of farming systems. In this work, we investigate whether Earth observation time series and meteorological drivers can be used to predict future canopy development at country scale. We focus on winter wheat and formulate crop growth prediction as
forecasting future leaf area index (LAI) trajectories beyond the last available Sentinel-2 observation. We evaluate this task on a multi-year
dataset which spans the entire country of Switzerland, containing over 20 million pixel-level Sentinel-2-derived LAI time series paired with meteorological variables. Because cloud cover and revisit gaps leave LAI supervision sparse, models fit the few valid (cloud-free) LAI observations yet oscillate implausibly between them, producing trajectories no real canopy could follow. We introduce a lightweight unimodal shape regulariser which improves trajectory plausibility with negligible loss in accuracy. We compare deep learning sequence-to-sequence (Seq2Seq) models with classic machine learning baselines and show that Seq2Seq models generalise well across years, achieving $\mathrm{R}^2$ above 0.8 and consistently outperforming conventional approaches. Together, these results demonstrate that remote sensing and weather-driven sequence modelling can learn crop growth dynamics at landscape scale. S
Read the full abstract ▾
All authors Dominik Senti, Mehmet Ozgur Turkoglu, Michele Volpi, Helge Aasen
Computer Vision · Cross-listed from cs.RO
Accelerating Large-scale Bundle Adjustment for LiDAR Mapping via Parallel Computing
Yixi Cai, Rundong Li, Yuhan Xie et al. (+3 more)
LiDAR bundle adjustment is widely utilized in mapping to construct globally consistent point cloud maps. In this paper, we propose the first fully parallel computing framework to accelerate
LiDAR bundle adjustment for large-scale mapping, incorporating three key techniques. First, we design an adaptive, asynchronous data loading strategy to efficiently process large-scale point cloud
datasets on
memory-constrained GPUs. Secondly, we present a novel bottom-up voxelization method for extracting planar features, enabling fully parallelized pre-processing. Thirdly, we build upon a majorization-minimization formulation to accelerate compute-intensive tasks in the optimization via parallel computation, including the computation of residuals, Jacobian and Hessian
matrices, and a parallel increment solver. To support our design, we provide both theoretical and experimental analysis of the time complexity of our approach. Extensive benchmarking on large-scale public
datasets across various computational platforms validates the robustness and adaptability of our approach, achieving up to a tenfold improvement in computational efficiency while preserving mapping accuracy comparable to state-of-the-art methods. To benefit future research, the implementation code is available on GitHub.
Read the full abstract ▾
All authors Yixi Cai, Rundong Li, Yuhan Xie, Qingwen Zhang, Patric Jensfelt, Fu Zhang
Computer Vision
Darakshan Rashid, Raza Imam, Ufaq Khan et al. (+9 more)
Temporal
vision-
language models (TVLMs) offer a reusable,
prompt-based interface for
surgical video understanding, yet, their robustness under clinically realistic acquisition artifacts in endoscopy remains insufficiently characterized. In practice, degradations such as defocus, haze, motion blur, noise, cautery smoke, and packet loss introduce structured distribution shifts which may compromise
video-text
alignment. We study the robustness of temporal VLMs under such shifts caused by corruptions in clip frames. We introduce Endo-C6, a compact corruption
benchmark of six endoscopy-realistic perturbations evaluated at a fixed high severity, and apply it to public Gastrointestinal (GI) endoscopy and laparoscopic cholecystectomy
videos. Under a standardized
prompt protocol, we
benchmark 3 recent
surgical TVLM baselines and analyze robustness in both mean and worst-case settings, spanning 294
dataset-level evaluations. Finally, we present RobustEndoCLIP, obtained by few-shot parameter-efficient tuning with VeRA, outperforming existing TVLM baselines. Our findings show that off-the-shelf TVLMs can exhibit severe worst-case collapse under endoscopy-specific corruptions, whereas lightweight few-shot adaptation can substantially improve corrupted performance and robustness without changing the
prompt-based interface. We expect Endo-C6 to support standardized robustness reporting and promote more reliable clinical
vision-
language systems.
Read the full abstract ▾
All authors Darakshan Rashid, Raza Imam, Ufaq Khan, Muhammad Bilal, Shazad Ashraf, Dwarikanath Mahapatra, Mohammad Yaqub, Muhammad Haris Khan, Imran Razzak, Brejesh Lall, Lena Maier-Hein, Yutong Xie
Computer Vision
Zero-Shot Skeleton-Based Action Anticipation
Hongsong Wang, Pengbo Yan, Yang Zhang et al. (+1 more)
Action anticipation (AA) aims to recognize ongoing human or
humanoids actions from partial observations, enabling
robots to predict intentions before the actions are completed. Although skeleton-based AA offers efficiency advantages, existing approaches assume that all action classes are seen during training, which limits their deployment in real-world scenarios where novel actions inevitably arise. To address this gap, we study the new task of Zero-Shot Skeleton-Based Action Anticipation (ZS-SkAA). This task requires recognizing unseen action classes using only limited early-stage skeleton sequences, combining the challenges of partial observations, temporal dynamics, and zero-shot generalization. To establish foundational research for ZS-SkAA, we introduce:(1) A baseline model comprising a spatio-temporal feature extractor and a mutual information estimation and maximization module. This baseline model explicitly aligns partial visual features with semantic class
embeddings across modalities by estimating and maximizing their mutual information, enhancing generalization to unseen classes.(2) A
benchmark protocol using the NTU RGB+D
dataset, which is adapted for rigorous ZS-SkAA evaluation. Experiments demonstrate the effectiveness of our model as a strong baseline for ZS-SkAA, achieving high zero-shot accuracy on NTU RGB+D. This work establishes ZS-SkAA as a vital research direction for real-world systems requiring generalization to novel actions.
Read the full abstract ▾
All authors Hongsong Wang, Pengbo Yan, Yang Zhang, Qiuxia Lai
Computer Vision
AppleScab-LT: A Longitudinal Real-Field Apple Scab Dataset for Temporal Disease Progression Analysis
Aamir Hilal, Shabir Ahmad Sofi and Neeraj Goel
The development of reliable plant disease monitoring systems is constrained by limited longitudinal
datasets capturing disease progression under natural field conditions. Although existing plant disease
datasets have advanced image-based recognition, most consist of static images acquired at a single time point, limiting analysis of temporal disease
evolution and severity progression. To address this gap, this study presents AppleScab-LT, a longitudinal real-field
dataset developed to monitor apple scab progression through repeated observations of individually tracked infected leaves. Guided by a research-question-driven framework, the
dataset was systematically developed, validated, and characterized for reliable longitudinal disease analysis. AppleScab-LT was constructed through systematic orchard monitoring under natural environmental conditions, incorporating longitudinal leaf tracking, expert-guided disease verification, polygon-based annotation, leaf isolation, disease severity quantification, and temporal sequence construction. A comprehensive quality assurance framework, including standardized annotation protocols, expert validation, automated integrity checks, sequence-level verification, and temporal consistency analysis, was applied throughout curation. The
dataset contains 21 longitudinal leaf sequences, 2,101 high-resolution images, and 264 progressive temporal samples from repeated monitoring of same infected leaves. It captures variability in severity accumulation, progression rates, monitoring duration, and inter-leaf progression. Quantitative disease descriptors based on pixel severity, color-intensity severity, and normalized relative severity provide standardized measurements for temporal disease analysis. AppleScab-LT provides a reliable resource for temporal disease intelligence, disease progression modelling, precision agriculture, and future crop health monitoring
Read the full abstract ▾
Computer Vision
RankT2I: A Submodular Framework for Discovering Interpretable and Diverse Semantics in Text-to-Image Models
Ritika Allada and Pinar Yanardag
Recent advances in text-to-image (T2I) models have revolutionized the field of image generation and editing. However, identifying semantics that a T2I model can successfully edit in an image continues to be a challenging task. Most existing approaches require users to manually specify semantics to modify a particular image, a time-consuming process that often involves extensive trial and error. In this paper, we present RankT2I, a novel, training-free, and model-agnostic framework that automates the discovery of editable semantics in
diffusion and FLUX-based models. Given a visual domain, we first utilize a
multimodal vision-
language model to gather a broad set of candidate semantics. We then frame semantic discovery as a set selection problem and use a submodular objective to identify semantics that are relevant, editable, and diverse. Our method helps users efficiently identify a wide range of semantics for text-to-image editing models across several domains while outperforming existing methods.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
MMUSV-Sim: A Perception-Oriented Simulation and Data-Generation Platform for Multi-USV Cooperative Perception
Ziao Li, Jianxiong Ye, Biao Tang et al. (+4 more)
Cooperative perception among multiple unmanned surface vehicles (USVs) combines complementary observations to extend maritime target sensing beyond the view range and field of a single platform. Developing such systems at scale calls for a unified workflow for configurable multi-USV scenarios,
multimodal acquisition, and shared annotations. We present MMUSV-Sim, a perception-oriented maritime simulation and data-generation platform built on Unreal Engine 5 and Project AirSim. It provides island, open-sea, and port environments; configurable weather, time of day, and wave conditions; a diverse vessel asset library; and spline-based multi-vessel motion. MMUSV-Sim acquires RGB, depth, semantic,
LiDAR, and radar observations across multiple USVs and captures a common world state for per-
agent annotation export. Experiments verify that the configured wave settings produce the intended changes in vessel heave, roll, and pitch, and evaluate the
geometric consistency between projected annotations and semantic renderings. In
LiDAR-based cooperative BEV vessel detection experiments on the generated multi-USV
dataset, Early Fusion achieves an AP@0.5 of 72.74, compared with 45.54 using a single USV.
Read the full abstract ▾
All authors Ziao Li, Jianxiong Ye, Biao Tang, Leping Zhang, Kun Zuo, Siyu Huang, Chenqiang Gao
Computer Vision
LightTeaNet: A Weakly Supervised Lightweight CNN for Multi-Label Tea Leaf Disease Detection and Localization
Naif Haider Chowdhury, Md Rahim, Syed Farhan Hasan et al. (+2 more)
Tea is known as an important crop in many parts of South and Southeast Asia, yet the production of tea is still hampered by the multiple diseases that decrease the quantity and quality. Traditional methods of inspection, which are manual, are not consistent, labor-intensive, and depend on extensive monitoring. This paper introduces a lightweight convolutional neural network (CNN) designed for weakly supervised multi-label classification and disease localization in tea leaves called LightTeaNet. LightTeaNet learns directly from image-level labels and employs Class Activation Mapping (CAM) to localize disease-affected regions automatically, unlike conventional object detection models such as YOLO, which require extensive bounding box annotations. For Parameter efficiency, the network integrates Depthwise Separable Convolutions, and for enhanced feature discrimination, it integrates Channel
Attention. LightTeaNet has achieved a Precision of 0.9615, a Recall of 0.8772, and an F1-score of 0.9179, while it shows mAP@0.50=0.1810 without any manual annotations, which delivers a competitive localization performance in the experimental results. These results validate the model as an interpretable as well as a resource-efficient framework for intelligent disease monitoring in agriculture.
Read the full abstract ▾
All authors Naif Haider Chowdhury, Md Rahim, Syed Farhan Hasan, Murad Hasan, Prithwiraj Bhattacharjee
Computer Vision
Concept Guidance: Precise, Training-Free Latent Control for Text-to-Image Generation
Nikolai Röhrich, Isabell Hans, Felix Krause et al. (+1 more)
Text-to-image
diffusion models have two major drawbacks that severely limit their practical utility: (1) standard models lack an intrinsic mechanism for continuous, concept-specific guidance (e.g., for precisely controlling how aesthetically pleasing an image looks), and (2) they lack reliability for tasks requiring high local coherence (e.g., generating text or human hands). To tackle these issues, we introduce a novel notion of concept-wise mutual information and find large, concept-dependent differences between individual layers, demonstrating that the generation of specific structures is localized in distinct parts of the network. We exploit this insight by reinforcing the impact of concept-relevant layers in Concept Guidance (CoG), a precise, target-specific guidance method that works for models out-of-the-box without additional training, external models, gradients, or
prompt engineering. CoG first quantifies each layer's concept-specific impact and then guides denoising using a weighted combination of predictions generated with concept-relevant layers skipped. We demonstrate performance increases across various targets and popular models like PixArt-alpha, SD3, SD3.5, and FLUX.1-dev. Code is available at https://github.com/CompVis/concept_guidance
Read the full abstract ▾
All authors Nikolai Röhrich, Isabell Hans, Felix Krause, Björn Ommer
Computer Vision
SCVIB: Editable State-Conditioned Visual Instance Binding for Multi-Turn Personalized Localization
Xiongtai Yang, Ziyan He and Tao Wang
Stored visual evidence can remain unchanged while its relevance to a current request changes with interaction state. To study this state-dependent source relevance, we introduce SCVIB, a controlled diagnostic
benchmark for editable state-conditioned visual instance binding. SCVIB contains 1,050 manually verified support-query pairs and 1,500 multi-turn episodes across five visual domains and four target-state dependencies. Executable state semantics separate target resolution from evidence selection and query-side localization. Direct Qwen3-VL-8B inference achieves 99.53% target-resolution accuracy but 62.00% Joint@0.5, revealing a 37.53-point resolution-localization gap; 563 of its 570 joint failures follow correct target identification. Routing comparisons and fixed-checkpoint support interventions diagnose a post-resolution evidence-utilization bottleneck: successful resolution must be coupled with relevant visual evidence and effective query-side grounding. We introduce TT-VG (Transition-Trace Visual Grounding), a state-conditioned visual grounding framework with an explicit and auditable state-to-evidence-to-grounding path. Target-State Transition Trace (TSTT) executes editable state and selects an auditable support record; Visual Evidence Grounding Adaptation (VEGA) learns support-conditioned grounding from target appearance, boxed support context, and the query. The canonical TT-VG configuration reaches 69.80% Joint@0.5, improving over direct inference by 7.80 points at the combined evidence-utilization and grounding interface. Task adaptation improves the DINOv2 and MASA
retrieval baselines to 51.47% and 55.67%, respectively, but both remain substantially below VEGA. These findings highlight state-dependent source selection and evidence utilization as consequential interfaces for interactive
multimodal information fusion.
Read the full abstract ▾
Computer Vision
CSG-Mamba: A Convolutional Scoring Gating Vision State Space Network for Endoscopic Polyp Segmentation
Yuliang Wang, Jiaqi Wu, Jiaye Song et al. (+1 more)
Accurate polyp
segmentation is critical for computer-aided colonoscopy, yet endoscopic images often contain low-contrast boundaries, mucosal texture interference, specular highlights, and device-dependent appearance shifts.
Vision State Space Models (SSMs) provide efficient long-range modeling with linear complexity, but existing
Vision Mamba
segmentation models typically convert 2D features into 1D scanning sequences, which may weaken local
geometric continuity and over-smooth irregular contours. We propose CSG-Mamba, a convolutional scoring gating
Vision State Space network for endoscopic polyp
segmentation. Built on a VM-UNet-style asymmetric U-shaped encoder-decoder, CSG-Mamba inserts a Convolutional Scoring Gating (CSG) module at the semantically rich bottleneck. CSG generates a local spatial score map through pointwise and large-kernel depthwise convolutions and recalibrates state-space features by multiplicative gating. Experiments with three random seeds show that CSG-Mamba achieves 0.9220 Dice and 15.87 HD95 on Kvasir-SEG, and 0.7418 Dice and 0.6570 mIoU on CVC-ColonDB, outperforming the baselines on most overlap and recall metrics while maintaining competitive boundary accuracy.
Read the full abstract ▾
All authors Yuliang Wang, Jiaqi Wu, Jiaye Song, Shuxia Ren
Computer Vision
Self-Supervised Visual On-Policy Distillation
Yijiang Li, Yijun Liang, Yunjie Tian et al. (+6 more)
Visual on-policy
distillation relies heavily on an informative teacher-student asymmetry, through either a larger, stronger teacher or privileged supervision, such as reference answers or ground-truth regions of interest. This raises a fundamental question: where can informative asymmetry come from when nothing privileged is available? We answer this by inverting where the asymmetry comes from. Rather than adding privileged information to the teacher, we subtract information from the student. This asymmetry creates the same effective learning signal for free as a teacher with access to information unavailable to the student, without ground-truth annotations, rewards, or a separate stronger teacher model. Building on this principle, we introduce Self-Supervised Visual On-Policy
Distillation (S$^2$VOPD), a simple yet effective method that constructs on-policy learning signals from asymmetric augmented views. S$^2$VOPD distills the teacher's distribution conditioned on the original image on-policy into the student distribution conditioned on a strongly augmented view of the same image. We systematically explore a broad design space of visual augmentations and uncover that (1) asymmetry matters: all four augmentation families improve performance, while symmetric self-
distillation degrades it; (2) strength matters: performance peaks at a moderate strength; and (3) the gap must remain task-consistent: augmentations that completely remove the question-relevant evidence can induce large but uninformative discrepancies. Across six fine-grained perception
benchmarks, S$^2$VOPD improves Qwen3.5-4B from 70.7% to 77.4%, above all open-source models compared, up to Qwen3-VL at 235B, and surpasses GPT-5.4. While holding training data the same, it recovers 96% of the improvement achieved by methods with privileged information. Website is at https://williamium3000.github.io/s2vopd
Read the full abstract ▾
All authors Yijiang Li, Yijun Liang, Yunjie Tian, Bingyang Wang, Ke Zhang, Zhenfei Yin, Di Fu, Philip Torr, Nuno Vasconcelos
Computer Vision
PISA: A Pseudo-Individual Source-Domain Feature Adaptation Framework for Test-Time Open-Vocabulary Object Detection
Ziyan He, Xiongtai Yang and Tao Wang
Open-vocabulary object detection test-time adaptation (OVOD-TTA) aims to address the performance degradation that pre-trained base models suffer when encountering image-domain shifts. Existing source-free OVOD-TTA methods rely either on refined test-time information for re-scoring or on pseudo-labels for self-training, leading to significant accuracy degradation when initial predictions are poor. Meanwhile, most conventional source-domain estimation methods recover abstract, sparse representations suitable for the classification task, but fail to capture the dense, concrete features required for detection. To address these issues, we propose PISA, a novel source-free OVOD-TTA method that can be seamlessly integrated into open-vocabulary visual backbones. The core components of our method are the Corruption-Invariant Feature Extractor (CIFE), the Feature
Alignment Module (FAM), and a multi-scale
alignment framework (BAA). To capture detection-suitable features, we develop CIFE to exploit the invariance of CLIP's visual features across corrupted images, ensuring robustness against various corruptions. We further develop FAM and BAA for the pre-training and adaptation to transform the corruption-invariant features into pseudo-individual source-domain features that are close to the original source-domain features. In this way, dense and concrete pseudo-individual source-domain features are used for supervision instead of unreliable pseudo-label signals. Experiments on the corrupted VOC-C, COCO-C, and LVIS-C
benchmarks across three base models demonstrate that PISA substantially improves both the localization precision and the category recognition accuracy of the original models. Notably, PISA achieves state-of-the-art performance without requiring access to source-domain data, surpassing existing methods by 3.92% in AP@50% on COCO-C.
Read the full abstract ▾