Computer Vision
SSD: Spatially Speculative Decoding Accelerates Autoregressive Image Generation
Shilong Xiang, Zirui Zhang, Lijun Yu et al. (+1 more)
Autoregressive image models treat images as 1D
token sequences, inheriting the next-
token factorization of
language models. This flattening discards a useful property of images: visual
tokens are organized and correlated in two-dimensional space. We introduce Spatially Speculative Decoding (SSD), an inference-time decoding framework that exploits this spatial structure for accelerated generation. Rather than speculating only along the flattened sequence, SSD predicts both the adjacent horizontal
token and the
token directly below it, allowing multiple spatial directions to advance in parallel. This reduces the number of backbone forward evaluations and alleviates the
memory bottleneck of autoregressive decoding. SSD accelerates image generation by up to 11.03x in wall-clock time while maintaining generation quality on DPG-Bench and GenEval. These results show that spatial structure provides a simple and effective source of parallelism for faster autoregressive image generation.
Read the full abstract ▾
All authors Shilong Xiang, Zirui Zhang, Lijun Yu, Chengzhi Mao
Computer Vision
CalTennis: Large Multi-View Tennis Video Dataset and Benchmark of Monocular-to-3D Pose Estimation
Ilona Demler, Xinran Xie, Blake Werner et al. (+2 more)
The Caltech Tennis
Dataset (CalTennis) is a large-scale
video benchmark for evaluating
monocular-to-3D pose estimation in the wild. CalTennis comprises over 11 million frames (51 hours) of tennis practice and match play from 40 players, captured with 2-6 synchronized cameras at 60 Hz. It is 10 times larger than existing in-the-wild human motion
video datasets and 3 times larger than existing MOCAP-ground-truthed
datasets, and it is the first large-scale
benchmark to provide synchronized multi-view recordings of expert athletic motion. The multi-view setup enables inexpensive, label-free evaluation of
monocular-to-3D pose estimation
algorithms. We describe a simple, standardized protocol that enables data collection without specialized equipment or expertise, along with fully automated
video calibration and synchronization. Benchmarking state-of-the-art
monocular-to-3D pose methods on CalTennis, we find that while 3D joint angle recovery is now quite accurate, all models struggle to estimate depth and foot contact consistently. We further propose two novel performance metrics, footwork and stability, as well as qualitatively study body shape inconsistency. These metrics expose previously underexplored failure modes and point to concrete opportunities for improvement in pose estimation and action analysis.
Read the full abstract ▾
All authors Ilona Demler, Xinran Xie, Blake Werner, Anna Szczuka, Pietro Perona
Computer Vision
The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation
Nicolas Dufour, Alexei A. Efros and Patrick Pérez
The Frechet Inception Distance (FID) is the de facto arbiter of image generation, yet most papers report just a single number from a single trained model using a single sampling seed. How reproducible is that number if we retrain the model, or merely resample from it? In this paper, we treat FID as a random variable on a two-axis panel of training and generation seeds, and measure its variance directly on several hundred SiT networks trained on class-conditional ImageNet 256x256. We report surprising findings: (a) Retraining the model using the same recipe with a different seed moves FID 3.2x more (in Inception feature space) than redrawing samples from a fixed network. (b) That gap is driven by three factors: random initialisation, data ordering, and the per-step Gaussian noise of the flow-matching loss. (c) Increasing compute or model size barely tightens the spread, holding the FID coefficient of variation (CoV) inside a 1-2% band. (d) Per-
cell classifier-free-guidance tuning halves the spread but reshuffles which seeds work best, and a lucky training seed reaches the same FID with up to 2x less compute than an unlucky one. Based on these findings, we recommend a new FID evaluation protocol: evaluate under per-
cell optimal guidance, treat any FID gap below the empirically measured ~1.3% CoV as inconclusive, and report an error bar over several training seeds rather than a single FID number.
Read the full abstract ▾
Computer Vision
VisDom: Sparse Novel View Synthesis with Visible Domain Constraint
Mariia Gladkova, Tarun Yenamandra, Edmond Boyer et al. (+3 more)
Sparse novel view synthesis (NVS) remains challenging due to the ambiguity of recovering 3D geometry from few input views. While NeRF- and Gaussian Splatting (GS)-based methods perform well with dense supervision, they often overfit in sparse settings, producing floating artifacts and inconsistent geometry. Silhouette consistency is commonly used as a regularizer, but it remains insufficient, as silhouette-consistent regions can extend beyond the true object geometry. We introduce VisDom, a learning-free
geometric constraint that augments classical carving-based visual hull reconstruction by enforcing a minimum multi-view visibility requirement. Specifically, we define a visible domain as the subset of 3D space observed by at least $K$ views and use it as an additional filtering criterion on top of standard silhouette-based reconstruction. This provides a stronger spatial prior in sparse-view settings. We integrate VisDom into both implicit (NeRF) and explicit (GS) pipelines by restricting volumetric sampling and guiding Gaussian placement during optimization. Experiments on three challenging
datasets show consistent improvements in sparse-view NVS, enabling high-quality object-centric reconstruction from as few as four input images. Our method is domain-agnostic, requires only silhouettes, and introduces no learned parameters, making it a simple complement to existing approaches. Applying VisDom on top of GaussianObject further improves performance on Omni3D and MipNeRF360, while matching or surpassing it at 22 $\times$ lower training cost.
Read the full abstract ▾
All authors Mariia Gladkova, Tarun Yenamandra, Edmond Boyer, Robert Maier, Tony Tung, Daniel Cremers
Computer Vision · Cross-listed from cs.CL
StylisticBias: A Few Human Visual Cues Drive Most Social Biases in MLLMs
Shaghayegh Kolli, Timo Cavelius, Nafiseh Nikeghbal et al. (+2 more)
Multimodal large
language models (MLLMs) are increasingly deployed in personally and societally consequential settings, yet the visual cues that shape how these models judge people remain poorly understood. Prior work often compares different (groups of) individuals, making it difficult to separate appearance effects from identity differences. We introduce StylisticBias, a controlled
benchmark for evaluating attribute-level social bias in MLLMs. We generate 500 photorealistic base faces and create about 50 single-attribute variations per face, producing about 25K images. This design keeps identity fixed and changes one visual attribute at a time. It lets us measure how specific cues shift model judgments. We evaluate six MLLMs across 25 binary social judgment scenarios. We find that age and body type dominate identity-level effects, while fashion style and other visual cues drive the largest attribute-level shifts. We further find that about 15 attributes account for nearly 80% of the total variation, showing that bias is concentrated in a small set of visual cues. Sensitivity is strongest in judgments that are semantically aligned with appearance, especially socioeconomic and style-related judgments. We release StylisticBias as a
benchmark for fine-grained bias evaluation in
multimodal models. Code and
dataset: https://github.com/timo-cavelius/StylisticBias and https://hf.co/
datasets/shaghayegh/stylistic-bias-
dataset.
Read the full abstract ▾
All authors Shaghayegh Kolli, Timo Cavelius, Nafiseh Nikeghbal, Samantha Dalal, Jana Diesner
Computer Vision
SARLO-80: Worldwide Slant SAR Language Optic Dataset 80cm
Solène Debuysère, Nicolas Trouvé, Nathan Letheule et al. (+2 more)
Multimodal foundation models have advanced rapidly thanks to large optical
benchmarks, but comparable resources for synthetic aperture radar (SAR) remain limited. Existing SAR–optical
datasets largely rely on low-resolution, intensity-only Ground Range Detected~(GRD) products and do not preserve complex-valued SAR measurements or native acquisition geometry, which restricts physically grounded
multimodal learning. In particular, large-scale public
datasets combining very-high-resolution (VHR) SAR SLC, aligned optical imagery, and natural-
language descriptions are still lacking. We present a VHR SAR–optical–text
dataset built from open-access Umbra spotlight acquisitions distributed as Sensor Independent Complex Data (SICD). From around 2,500 worldwide scenes (VV/HH, 20cm–2m native resolution), we standardize all SAR data to an 80cm slant-range grid via band-limited FFT resampling and tile the imagery into 1024 by 1024 patches. For each SAR patch, we retrieve a high-resolution optical tile and warp it into the SAR grid using local coordinate correspondences for local pixel-level
alignment. We further generate three caption variants (SHORT/MID/LONG) per sample to support
vision–
language training and evaluation. Our
dataset contains 119,566 triplets (complex and amplitude slant-range SAR patch, aligned optical patch, natural-
language description) covering 257 locations across 72 countries and a broad range of land types and infrastructures. We release fixed train/validation/test splits and the full preprocessing and baseline code to enable reproducible
benchmarks for
multimodal alignment on cross-modal
retrieval and conditional generation in native SAR geometry. The
dataset is publicly available on the Hugging Face Hub at https://huggingface.co/
datasets/ONERA/SARLO-80.
Read the full abstract ▾
All authors Solène Debuysère, Nicolas Trouvé, Nathan Letheule, Elise Colin, Georgia Channing
Computer Vision
HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining
Juncheng Ma, Jianxin Bi, Yufan Deng et al. (+19 more)
Embodied foundation models are expected to benefit from data scaling like large
language models, but face a much tighter data bottleneck. Teleoperated real-
robot trajectories remain the dominant pretraining source due to their precise action supervision and embodiment
alignment, yet their scalability is limited by high collection cost, acquisition difficulty, and low behavioral and environmental diversity. These limitations have sparked interest in
egocentric human
video as a scalable, substantially lower-cost, and more diverse alternative for
embodied model pretraining. However, its effectiveness compared to teleoperated real-
robot data remains underexplored. To address this question, we conduct a systematic study comparing
egocentric human
video and teleoperated real-
robot trajectories as pretraining data sources for
embodied foundation models, under fixed post-training and validation protocols. Surprisingly, we find that
egocentric data, when processed through a carefully designed filtering and labeling pipeline, is not merely a viable substitute for model pretraining but can lead to superior performance. With the same amount of pretraining data, models pretrained on
egocentric data achieve a 24% lower validation loss on real-
robot action prediction, as well as 52.5% and 90% higher success rates on in-distribution and out-of-distribution real-
robot task execution, respectively. This finding verifies a scalable paradigm for
embodied foundation models: pretrain on
egocentric human
video to learn diverse world representations, then adapt with a small amount of labeled real-
robot data for action-space
alignment. We hope this study encourages broader exploration of
egocentric data and offers guidance for data quality assessment before costly
robot data collection.
Read the full abstract ▾
All authors Juncheng Ma, Jianxin Bi, Yufan Deng, Xuanran Zhai, Kewei Zhang, Ye Huang, Bo Liang, Shukai Gong, Jiankai Tu, Xiaotian Tang, Jiaxin Li, Kaiqi Chen, Duomin Wang, Yuqi Wang, Bingyi Kang, Eric Huang, Zhiyang Dou, Zhen Dong, Enze Xie, Wojciech Matusik, Tat-Seng Chua, Daquan Zhou
Computer Vision
S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligence
Yalun Dai, Hao Li, Shulin Tian et al. (+8 more)
Real-world spatial intelligence requires
reasoning over a continuous and evolving 3D world, yet existing VLMs and tool-augmented
agents largely remain tied to static, stateless inference from isolated visual observations. We introduce \textbf{S-
Agent}, a spatial tool-use agentic paradigm for understanding and
reasoning over continuous multi-view images and
videos. By formulating spatial
reasoning as spatio-temporal evidence accumulation rather than isolated frame-level prediction, S-
Agent reshapes spatial perception into scene-centric understanding beyond frame-centric recognition. Specifically, S-
Agent casts the VLM as a semantic planner that decides what evidence is needed, while a hierarchy of spatial tools and experts grounds objects in 2D, lifts them into 3D
geometric evidence, and aggregates this evidence into high-level spatial knowledge (e.g., counting, measurement, orientation, and relative position). Additionally, a temporal
memory mechanism, including Scene
Memory for maintaining the evolving scene state and
Agent Memory for accumulating
reasoning context, enables evidence integration across frames and
reasoning steps. Comprehensive experiments on multi-view and
video spatial
reasoning benchmarks show that S-
Agent consistently improves both open-source and closed-source VLMs in a training-free manner. Beyond inference-time augmentation, supervised fine-tuning (SFT) on S-
Agent-generated spatial trajectories S-300K yields S-
Agent-8B, a compact spatial
agent that significantly surpasses similar-scale baselines (e.g., Qwen3-VL-8B) and performs comparably to advanced closed-source models (e.g., GPT-5.4 and Gemini 3).
Read the full abstract ▾
All authors Yalun Dai, Hao Li, Shulin Tian, Runmao Yao, Yuhao Dong, Fangzhou Hong, Zhaoxi Chen, Fangfu Liu, Tao Wang, Kim-Hui Yap, Ziwei Liu
Computer Vision
FreeStyle: Free Control of Style-Content Dual-Reference Generation from Community LoRA Mining
Jinghong Lan, Wei Cheng, Yunuo Chen et al. (+10 more)
Style-content dual-reference generation aims to synthesize an image that preserves the structure and semantics of a content reference while adopting the style of a separate style reference.Despite recent progress, this setting remains challenging because models must balance content fidelity, style
alignment, and instruction following avoiding semantic leakage from the style reference.A key bottleneck is the lack of large-scale triplet data with clean content-style separation and broad long-tail style coverage.In this work, we propose FreeStyle, a scalable dual-reference generation framework based on community LoRA mining.We treat community LoRAs as compositional anchors for style and content, and design a rigorous generation and filtering pipeline to construct large-scale Style-Reference and Content-Reference triplets across multiple base models.To address content leakage, we adopt a two-stage curriculum with stage-specific disentanglement mechanisms: an
attention-level enrichment constraint that suppresses style-reference leakage in the style-transfer stage, and a frequency-aware RoPE modulation strategy that targets positional-correspondence-based leakage in the harder dual-reference stage.We also introduce a
benchmark covering both style-reference and dual-reference generation, with evaluations on style similarity, content preservation, aesthetics, instruction following, and leakage rejection. The
benchmark incorporates a style-invariant Content
Alignment Score (CAS) and introduces a calibrated VLM-based Rejection Score for evaluating generation reliability and leakage suppression.Extensive experiments show that our model achieves a strong balance among style
alignment, content preservation, and leakage suppression.
Read the full abstract ▾
All authors Jinghong Lan, Wei Cheng, Yunuo Chen, Ziqi Ye, Peng Xing, Yixiao Fang, Rui Wang, Yufeng Yang, Xuanyang Zhang, Xianfang Zeng, Difan Zou, Gang Yu, Chi Zhang
Computer Vision · Cross-listed from cs.RO
Fast Human Attention Prediction for Fixation-guided Active Perception in Autonomous Navigation
Fatma Youssef Mohammed, Grzegorz Malczyk and Kostas Alexis
Human visual
attention relies on structured scanpaths to efficiently process scenes, yet instilling this behavior into
robot autonomy is in its infancy and hindered by the high,computational costs of existing predictive models. To address this, we introduce GazeLNN, a computationally lightweight,scanpath prediction model that leverages
Liquid Neural Networks as its recurrent engine and employs MobileNetV3 for feature extraction. Operating auto-regressively, the architecture predicts sequential fixation heatmaps conditioned on the current visual stimulus and fixation history. Despite requiring only 0.61 GFLOPs, GazeLNN achieves state-of-the-art performance on the MIT Low Resolution
dataset achieving 0.47 ScanMatch score. It outperforms existing recurrent baselines across diverse evaluation metrics, while reducing computational costs by 99.40% and accelerating inference by up to six times. To investigate the role of human
attention modeling in
robot autonomy and demonstrate the practical utility of this highly efficient architecture, we integrate GazeLNN into an active camera-
robot control policy trained via Reinforcement Learning. This integration enables human-fixation-guided perception during autonomous
navigation, validated through successful real-world deployments on an aerial
robot.
Read the full abstract ▾
Computer Vision
How Fragile Are Training-Free AI-Generated Image Detectors? A Controlled Audit of Score Direction, Preprocessing, and Compression
Jingwen Zhou and Mingzhe Wang
Training-free detectors of AI-generated images promise generator-agnostic deployment without classifier training, yet their reported numbers are rarely compared under a single controlled protocol. We audit two representative training-free scores – an autoencoder-reconstruction score (AEROBLADE-style) and a noise-perturbation feature-similarity score (RIGID-style) – plus a naive feature-kNN control, on a common 1,500-image GenImage-derived
benchmark spanning seven generators and JPEG compression at quality 70 and 50. The audit yields three cautionary findings. (i) Implementation details masquerade as method differences: replacing the LPIPS backbone (AlexNet -> VGG-16) changes overall AUROC by +0.085, and switching between resize-to-512 and native-resolution preprocessing flips per-generator conclusions by up to 0.38 AUROC. (ii) Score direction is not a property of the method but of its hyperparameters: the RIGID-style score is inverted (AUROC < 0.5) on SD1.5 and Wukong at noise level sigma=0.05, recovers to >0.5 for every generator at sigma=0.01, and collapses to 0.15 at sigma=0.3. (iii)
Dataset format bias inflates robustness claims: without unified re-encoding, AUROC under JPEG-50 exceeds the clean condition for the AlexNet-backbone reconstruction score; after bias correction the residual anomaly localizes to a single generator (BigGAN). The audited scores have complementary per-generator failure sets, but naive z-score fusion does not beat the best single score, indicating that exploiting complementarity requires direction-aware combination.
Read the full abstract ▾
Computer Vision
Scalable Training of Spatially Grounded 2D Vision-Language Models for Radiology
Yusuf Salcan, Simon Ging, Robin Tibor Schirrmeister et al. (+4 more)
We study how to train visually grounded
vision-
language models (VLMs) for radiology without manual spatial annotations. We introduce RefRad2D, a large-scale bilingual (German/English)
dataset of 1.2M CT and MR image-text pairs derived from clinical practice, with task-specific VQA and spatial grounding subsets generated automatically via LLM-based curation and automated
segmentation. Trained on this data, our model RadGrounder jointly performs report generation, visual question answering, and spatial grounding via bounding-box detection or
segmentation. On external VQA
benchmarks (Slake, VQA-RAD), RadGrounder achieves competitive results with specialized medical VLMs. Adding our clinical data to the training mixture improves open-ended VQA over fine-tuning on the downstream
datasets alone, showing the transferability of our
dataset. Crucially, adding grounding supervision does not degrade
language quality, enabling spatially verifiable outputs at no cost to VQA performance.
Read the full abstract ▾
All authors Yusuf Salcan, Simon Ging, Robin Tibor Schirrmeister, Philipp Arnold, Elmar Kotter, Behzad Bozorgtabar, Thomas Brox
Computer Vision
PCFootprint: A Large-Scale Dataset and Benchmark for Vectorized Building Footprint Extraction from Aerial LiDAR Point Clouds
Haoyuan Shen, Kuihao Wang, Ruisheng Wang et al. (+1 more)
Building footprint extraction is a fundamental task in photogrammetry, remote sensing, and computer
vision. Recent image-based methods have achieved remarkable progress in extracting vectorized footprints from high-resolution optical imagery. However, optical imagery inherently susceptible to occlusions, perspective distortions, and residual relief displacement, yielding incomplete or misaligned footprint extraction. Furthermore, the lack of explicit elevation information limits its direct applicability to Level of Detail building modeling. In this paper, we present PCFootprint, the first large-scale public
dataset for footprint extraction from airborne
laser scanning point clouds. PCFootprint comprises \num{33000} tiles derived from the Estonian Land and Spatial Development Board, covering diverse urban and rural landscapes. Each tile spans \qtyproduct{128 x 128}{\m} with systematically aligned vectorized footprints aligned to point clouds. The
dataset includes a \num{3000} tiles cross-domain test set for evaluating generalization across geographic regions. We establish comprehensive
benchmarks by evaluating mainstream methods. Experimental results reveal significant challenges including high intra-class variance, data imbalance, and noise across complex geospatial environments. We believe PCFootprint will advance future research in building modeling, urban scene understanding, and geospatial analysis. The PCFootprint
dataset is publicly available at \url{https://huggingface.co/
datasets/Haoyuan-Shen/PCFootprint}.
Read the full abstract ▾
All authors Haoyuan Shen, Kuihao Wang, Ruisheng Wang, Yujun Liu
Computer Vision
InfantFace: Detecting infant faces in neonatal clinical environments
Abdullah Bin-Obaid, Maria M. Cobo, Rebeccah Slater et al. (+2 more)
Reliable localisation of the neonatal face is the first step for several
video-camera based non-contact assessments such as pain and distress related facial expression analysis, pain scoring, cardiorespiratory signal extraction and cessation of breathing alerts. However, major challenges persist in neonatal clinical environments. Cluttered backgrounds, illumination changes and poor lighting conditions can reduce the accuracy of face detection models. Clinical interventions, monitoring equipment and, in some cases, medical devices can obstruct the face, making visual assessment difficult. We propose a one-stage YOLOv11m-based model tailored for face detection of infants in neonatal clinical environments. We combined multiple publicly available
datasets (VGGFace2, CelebA, FDDB, WIDER FACE) to train and evaluate our proposed model. We then fine-tuned our model on a neonatal research
dataset involving 228
videos from 114 recording sessions of 113 independent infants. Before fine-tuning, our model achieved an AP50 of 0.87, surpassing the performance of three state-of-the-art general face detectors. Performance improved further to an AP50 of 0.96 after clinical-domain adaptation. Evaluating face detection performance across different
datasets remains a challenge due to the lack of publicly available neonatal
datasets. Prioritising the creation of such
datasets, while upholding appropriate
privacy safeguards and ethical standards in their creation and use, would greatly support further progress in this field.
Read the full abstract ▾
All authors Abdullah Bin-Obaid, Maria M. Cobo, Rebeccah Slater, Lionel Tarassenko, Mauricio Villarroel
Computer Vision
Spectral Query-Key Product Weight Steering for Training-Free VLM Hallucination Mitigation
Karn Tiwari, Varnith Chordia and Prathosh A P
Vision-
language models (VLMs) often generate fluent but visually unsupported descriptions, especially by mentioning objects absent from the image. We propose QK Product Steering, a data-free, training-free, and zero-inference-cost weight edit for reducing object
hallucination. The method directly edits the per-head query-key product, the operator that produces pre-softmax
attention logits, by suppressing a small number of dominant singular modes in selected middle layers. The edited product is then mapped back to the query weights through a closed-form query-only update while keeping shared key weights fixed, making the edit compatible with grouped-query
attention. We further decompose the QK product into symmetric and antisymmetric components to distinguish mutual content-similarity patterns from directional
attention patterns. Across three GQA-based VLMs, QK Product Steering achieves an average relative CHAIR$_s$ reduction of $4.0\%$, while matched random-mode controls show negligible change. Interpretability ablations show that the
hallucination signal is specific to dominant QK modes and is primarily localized to the symmetric mutual-
attention channel. Overall, QK Product Steering offers a simple alternative to decoding-time mitigation, requiring no additional data, fine-tuning, or inference-time overhead while largely preserving general
multimodal capability.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
On the Redundancy of Timestep Embeddings in Diffusion Models
José A. Chávez
Diffusion models rely heavily on explicit timestep
embeddings to modulate the denoising process across various noise scales. In this work, we challenge the necessity of these temporal signals by analyzing their impact on U-Net and
Diffusion Transformer architectures. Beyond empirical evidence, we provide a theoretical framework demonstrating that, under certain conditions, the global minimizer of the
diffusion training objective can be achieved without explicit timestep conditioning. Our findings reveal a surprising robustness when timestep
embeddings are completely removed. Extensive ablation studies on the CelebA and CIFAR-10
datasets show that these time-agnostic models can maintain high structural fidelity and even surpass their conditioned counterparts in competitive metrics, including FID, precision, and recall. Our analysis suggests these architectures can implicitly infer noise scales from the corrupted input under specific assumptions, rendering explicit temporal conditioning redundant. This study challenges long-standing temporal conditioning paradigms and paves the way for more efficient and structurally focused generative architectures.
Read the full abstract ▾
Computer Vision
FlowBender: Feedback-Aware Training for Self-Correcting Conditional Flows
Daniel Gilo, Sven Elflein, Ido Sobol et al. (+1 more)
Conditional
diffusion and flow models routinely fail to satisfy the very constraints that define their task. For instance, a depth-conditioned model often produces images whose re-extracted depth disagrees with the input, even though the forward operator–the depth predictor defining the constraint–is available during both training and inference. Existing approaches generally fall into two categories: supervised models that treat the conditioning signal as a static cue and ignore
alignment information at inference, and guidance-based methods that consult it through hand-tuned linear updates, typically trading fidelity to the condition against the plausibility of the generated sample. We argue that the fundamental gap in both paradigms is that the model is never trained to utilize its own
alignment error. We introduce FlowBender, a closed-loop framework that treats this error as a first-class input, training the network to learn a correction policy conditioned on inference-time feedback. At each step, an unguided look-ahead pass estimates the clean signal, a task-specific deviation is computed via the forward operator, and a refinement pass consumes this signal to produce a corrected velocity. We propose several variants of FlowBender, including a gradient-based formulation for differentiable operators and a zero-order variant for non-differentiable settings such as JPEG compression. For efficient sampling, we introduce a prior-step shortcut that enables closed-loop correction at a minimal additional computational cost. Across image-to-image translation, restoration, and 3D mesh texturing, FlowBender consistently outperforms standard supervised baselines,
alignment-loss-augmented training, and state-of-the-art inference-time guidance, improving fidelity and plausibility simultaneously rather than trading them against each other. Project page: https://flow-bender.github.io/
Read the full abstract ▾
All authors Daniel Gilo, Sven Elflein, Ido Sobol, Or Litany
Computer Vision
Geometry-Aware Superpixel Graph Transformer with Metadata for Skin Lesion Classification
Muhammad Azeem, Tanveer Hussain, Amr Ahmed et al. (+1 more)
Automated skin cancer classification from dermoscopic images remains challenging due to heterogeneous lesion structure, strong intra-class variability, and subtle visual differences between benign and malignant cases. Existing CNN/ViT pipelines typically rely on global or patch-level features and often combine patient metadata via late fusion, which limits spatially grounded
multimodal reasoning. We present a novel region-based graph learning framework that explicitly models lesions as graphs of spatially coherent superpixel regions represented as frozen CNN features. To capture fine-grained lesion arrangements, we encode inter-regional geometry as edge attributes and introduce a dedicated metadata context node connected to all regions, providing structured integration of demographic/clinical variables within the same relational space. Node representations are updated using our edge-aware graph
transformer followed by
attention-driven propagation, and a final graph-level
embedding for benign-malignant classification. Experiments on four public
benchmarks demonstrate that explicit region-level relational modeling and graph-native
multimodal fusion yield consistent gains over the state-of-the-art. Consequently, we establish a new graph-centric perspective in which CNN features are modeled as relational nodes and improved through contextual integration, yielding more expressive and robust classifications.
Read the full abstract ▾
All authors Muhammad Azeem, Tanveer Hussain, Amr Ahmed, Ardhendu Behera
Computer Vision
Reliability-Aware Prototype Calibration for Frozen Pose-Flow Video Anomaly Detection
Ning Dong, Yingna Su, Xin Dong et al. (+3 more)
Pose-flow
video anomaly detectors are attractive for one-class surveillance because they provide likelihood-based rankings for tracked skeleton windows. However, a single likelihood score may hide
multimodal normal behavior and be sensitive to pose-observation noise. We study a frozen-detector setting in which the pose-flow backbone, cached skeleton tracks, and evaluation pipeline are fixed. Reliability-Aware Prototype Calibration (RPC) is a post-hoc score calibration method for this setting. It adds a standardized nearest-prototype deviation in the frozen latent space to the standardized flow score, and uses keypoint confidence only to gate this added
geometric evidence. Thus, RPC preserves the original density signal while correcting the ranking with empirical normal-mode structure under pose reliability. Across two frozen pose-flow backbones and four
datasets, RPC improves frame-level AUROC in all eight backbone-
dataset pairs, with gains ranging from 0.34 to 4.49 percentage points and averaging 2.03 points. Ablation and reliability analyses show that prototype deviation is the main corrective signal, while reliability gating is most useful when pose observations are less trustworthy. These results suggest that lightweight post-hoc calibration can strengthen cached pose-flow systems when retraining or reproducing the full pose pipeline is impractical.
Read the full abstract ▾
All authors Ning Dong, Yingna Su, Xin Dong, Ziyun Jiao, Xinnian Guo, Zhuangzhuang Pan
Computer Vision
Through the PRISM: Preference Representation in Intermediate States of Video Diffusion Models
Haoxuan Wu, Lai Man Po, Mengyang Liu et al. (+3 more)
Video reward models are central to aligning
video diffusion models with human preferences, yet existing ones are external
vision-
language models that score clean, decoded pixels. Scoring an intermediate denoising state therefore requires either finishing the remaining denoising steps or decoding a noisy latent that the reward model cannot reliably interpret. We ask whether a
video generator can itself act as the evaluator, discriminating human preferences directly from its noisy latents. We propose PRISM (Preference Representation in Intermediate States of
Diffusion Models), a latent
video reward model that keeps a pre-trained
video diffusion transformer frozen, reads features from its early blocks, and maps them to a timestep-conditioned reward through a lightweight query-based aggregation head with only 21.8M trainable parameters. PRISM outperforms pixel-space reward models at high noise levels on two
benchmarks while remaining competitive with 8B-parameter
vision-
language reward models on clean
videos. This noise robustness enables early-stage Best-of-N sampling: selecting among candidates after only a few denoising steps reduces sampling cost by up to 10x and surpasses the strongest pixel-space baseline on each generator at 3-4x lower cost.
Read the full abstract ▾
All authors Haoxuan Wu, Lai Man Po, Mengyang Liu, Kun Li, Hongzheng Yang, Wei Liu
Computer Vision
GEN-Guard: Correcting Generalization Failures for Deployable Federated Surgical AI
Julia Alekseenko, Pietro Mascagni, AI4SafeChole Consortium et al. (+1 more)
Federated Learning (FL) in
surgical video AI enables collaborative model training without sharing sensitive data. However, standard evaluation practices - selecting the "best" global model based only on validation data from participating hospitals - can lead to suboptimal deployment choices. We identify this critical failure mode as performance leakage, where the selected model overfits internal federation data and fails to generalize to unseen institutions. We propose GEN-Guard, a practical post-hoc framework to detect and correct generalization failures in
federated surgical AI. It integrates Generalization Detection via Client-Blocked Evaluation (CBE), which validates performance on isolated client distributions to prevent performance leakage, and Generalization Correction through Disagreement-Aware
Distillation (DAD), which learns adaptive feature-level corrections for cross-institutional robustness. Both components operate after standard FL convergence while providing robust support for zero-shot adaptation to unseen environments. We first quantify the severity of performance leakage, observing Model Selection Failures (MSFs) exceeding 80% under standard evaluation. GEN-Guard is evaluated on two multi-center clinical challenges:
surgical phase recognition in laparoscopic cholecystectomy and polyp
segmentation in colonoscopy. Across both
datasets, GEN-Guard consistently corrects these failures, improving in-federation F1 scores by up to 2 points, unseen-institution performance by up to 3 points, and worst-case institutional performance by 3-9 points. Performance leakage represents a systematic and previously under-recognized risk in
federated surgical AI. GEN-Guard provides a practical solution for detecting and correcting such failures. By improving cross-institutional robustness and zero-shot generalization, it strengthens the reliability of FL for real-world
surgical deployment.
Read the full abstract ▾
All authors Julia Alekseenko, Pietro Mascagni, AI4SafeChole Consortium, Nicolas Padoy
Computer Vision
CUPID: Reconstructing UV Texture Maps for Interpretable Person-of-Interest Deepfake Detection
Giovanni Affatato, Sara Mandelli, Edoardo Daniele Cannas et al. (+2 more)
Deepfakes targeting a high-profile individual, known as Person-of-Interest (POI), are a threat to modern democracies and societies. Current POI deepfake detection methods still struggle to combine robustness to post-processing, efficiency and interpretability, key aspects of modern deepfake detectors. In this paper we propose CUPID, a POI
video deepfake detector that combines UV texture maps, a facial appearance representation derived from 3D face reconstructions, with the representation learning capabilities of the Masked Autoencoder (MAE). Our method does not require any deepfake
videos in its training phase. Moreover, it does not even require including a specific POI in the training set: the combination of UV texture maps extracted from real
video frames and the MAE context-guided reconstruction yields a latent space that captures rich and discriminative facial features even for identities unseen during training. In the testing phase, the
embeddings extracted from a query
video depicting the POI can be matched against pristine reference
videos to assess the
video authenticity. Furthermore, operating in the UV space naturally provides an additional layer of interpretability. Specifically, we can extract decoded residual maps that highlight which facial regions of a test
video deviate most from the identity representation of the corresponding POI. Experiments on four deepfake
datasets show that CUPID outperforms the current state of the art on most
datasets and achieves the best overall robustness against strong downscaling and compression, while also providing substantially faster inference. Our experimental code will be released at https://github.com/polimi-ispl/CUPID.
Read the full abstract ▾
All authors Giovanni Affatato, Sara Mandelli, Edoardo Daniele Cannas, Paolo Bestagini, Stefano Tubaro
Computer Vision
CMDS-AD: Cross-Modal Dual-Stream Decoupling for Few-Shot Anomaly Detection
Junhao Cai, Junyu Chen, Deyu Zeng et al. (+4 more)
Few-shot anomaly detection remains challenging due to limited training data. Multi-modal anomaly detection (MAD) offers a viable solution, leveraging 3D
geometric cues to enrich 2D RGB representations and compensate for this scarcity. However, existing MAD methods apply spatially uniform feature processing, conflating stable macroscopic structures with high-frequency localized defect signals, exacerbating cross-modal misalignment and inflating false-positive rates. To overcome this, we present CMDS-AD, a Cross-Modal Dual-Stream Anomaly Detection framework. A LoRA-guided
diffusion model generates diverse RGB samples to mitigate extreme data scarcity. For 3D normal augmentation, we employ a pre-trained
diffusion model as a normal
estimator. Crucially, this
estimator inherently acts as a non-linear low-pass filter, directly extracting low-frequency normal representations from RGB inputs. This establishes an auxiliary estimated stream of purely low-frequency information, anchoring robust structural templates and assisting the uncompressed real stream, containing coupled high- and low-frequency components, to precisely isolate micro-defects. A Coordinate-Aware Hierarchical Feature Mapper adaptively aligns cross-modal semantics, while a multiplicative scoring mechanism filters modality-specific noise. Under the extreme 1-shot setting, CMDS-AD achieves absolute performance gains of 5.7% (I-AUROC) and 2.0% (AUPRO) on MVTec 3D-AD, alongside 7.7% and 5.6% improvements on EyeCandies, establishing a new state-of-the-art. Code is available at https://github.com/Junhaocai27/CMDS-AD
Read the full abstract ▾
All authors Junhao Cai, Junyu Chen, Deyu Zeng, Junhao Pang, Qiwei Liang, Xiaopin Zhong, Zongze Wu
Computer Vision · Cross-listed from cs.LG
Integrating national forest inventory, airborne lidar, and satellite imagery for wall-to-wall mapping of forest structure with computer vision
Luke J. Zachmann, David D. Diaz, Vincent A. Landau et al. (+8 more)
Remote sensing is increasingly relied upon to deliver actionable science for forest and wildfire risk management across large landscapes. Wall-to-wall, annually updated maps are a persistent need for effective forest management. Many planning systems and data collections combine disparate data sources with different purposes, vintages, and prediction quality, which leads to confounding behavior in operational planning systems. We introduce the VibrantForests framework, developed and applied to map forest attributes and provide a coherent foundation for effective forest and wildfire planning. VibrantForests includes a satellite-based forest structure model trained on
lidar-derived samples and applied across the contiguous United States to concurrently generate estimates of canopy cover, canopy height, aboveground live tree biomass, basal area, and quadratic mean diameter at 10-meter resolution. We demonstrate predictive capability spanning the full spectrum of forest conditions ranging from sparse-canopy/low-biomass to dense-canopy/high-biomass. Results show that our model extends the range at which saturation is commonly encountered in comparable passive-sensor models, and reduces regression-to-mean behavior that commonly produces overestimation of forest attributes in small/sparse conditions and underestimation in large/dense conditions. The VibrantForests framework addresses a key limitation in large-area forest and wildfire planning by delivering coherent wall-to-wall estimates of management-relevant attributes at annual cadence and 10m resolution.
Read the full abstract ▾
All authors Luke J. Zachmann, David D. Diaz, Vincent A. Landau, Chelsey Walden-Schreiner, Tony Chang, Nathan E. Rutenbeck, Katharyn A. Duffy, Kiarie Ndegwa, Andreas Gros, Scott Conway, Guy Bayes
Computer Vision
U$^2$Mamba: A Two-level Nested U-structure Mamba for Salient Object Detection
Junhui Li, Jialu Li and Youshan Zhang
Mamba-based models have emerged as a promising alternative for salient object detection (SOD), offering significant advantages in modeling long sequences. However, existing models often fail to explore contextual information and the depth of the entire architecture. This paper introduces U$^2$Mamba, a powerful and innovative U-structured network for salient object detection. We propose multiscale Mamba U-blocks (MMUBs) that enhance the model depth to improve local feature extraction capabilities. Our newly developed nested U-structure, incorporating MMUBs, enables the network to integrate various receptive fields from shallow and deep layers, thereby collecting richer contextual information and longer-range data without being constrained by resolution. Instead of using the traditional deep supervision scheme and top-level supervised training, we propose a hierarchical training supervision method where the loss is computed at each level during the training process. Extensive experiments demonstrate that U$^2$Mamba achieves highly competitive performance against state-of-the-art methods. The source code is available at \url{https://github.com/JL021/U2Mamba}.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Efficiently Linking Real Scenes with Synthetic Data Generation for AI-based Cognitive Robotics and Computer Vision Applications
Paul Koch, Vivek Chavan, André Sers et al. (+4 more)
AI
vision models are a driving factor for the potential use case scenarios of cognitive robotics within in the industry and household applications. A large array of methods from semantic environment analysis towards 6D and grasping pose estimation have been proposed based on the latest AI achievements. However, such advancements require further strong and efficient methods w.r.t. training data and AI-architectures, which are capable in synergy to tackle current challenges, precision limits, and scalability beyond domain gaps. In this paper, we discuss these current limits and trends in the related state-of-the-art which are challenging those. Further we discuss our current work in progress on bridging the domain gap between simulations and real world applications by linking those in the training data generation.
Read the full abstract ▾
All authors Paul Koch, Vivek Chavan, André Sers, Adem Karakurt, Paul Hofmann, Mohamad Zaher Ziadeh, Jörg Krüger
Computer Vision
Single-Stage Hierarchical Rectification for Weakly Supervised Histopathology Segmentation
Duc T. Nguyen, Hoang-Long Nguyen, Thanh-Ha DO et al. (+1 more)
Existing weakly supervised semantic
segmentation (WSSS) methods in computational pathology rely on a multi-stage paradigm: class activation map (CAM) generation, offline pseudo-mask refinement, and fully supervised retraining. While established, this decoupled approach presents fundamental limitations. The multi-stage process not only incurs high computational training costs but also suffers from error propagation: local texture biases in shallow CNN layers generate false-positive artifacts that subsequent refinement steps often fail to correct. To address these persistent challenges through a simple yet highly effective approach, we propose the Single-Stage Hierarchical Rectification (SSHR) framework. Rather than passively refining CAMs post-hoc, our method proactively purifies intermediate feature representations during the forward pass. We introduce a Hierarchical Feature Rectification Module (HFRM) that utilizes deep global semantic context to filter out local anomalies in shallow layers. This mechanism generates high-fidelity activation maps directly within a single training loop. Experiments on the LUAD-HistoSeg and BCSS
datasets demonstrate that SSHR outperforms state-of-the-art multi-stage methods. Furthermore, SSHR reduces training duration by 2 to 5 times. This efficiency minimizes computational overhead and accelerates clinical translation for large-scale histopathology workflows. The code is available at: https://github.com/trongduc-nguyen/SSHR
Read the full abstract ▾
All authors Duc T. Nguyen, Hoang-Long Nguyen, Thanh-Ha DO, Huy-Hieu Pham
Computer Vision
SPOT-E: Test-Time Entropy Shaping with Visual Spotlights for Frozen VLMs
Bo Yin, Xiaobin Hu, Chengming Xu et al. (+6 more)
Vision-
language models (VLMs) often underperform on evidence intensive tasks because decisive visual evidence are small, localized, and easy to overlook, leading to failures in evidence readout even when high-level
reasoning is intact. Prior inference-time visual interventions can improve grounding without retraining, but they are largely open-loop and lack a mechanism to verify whether highlighted evidence is actually used. We study answer-span prediction
entropy as a model-internal feedback signal and show that naive
entropy minimization is ambiguous, since low
entropy may arise from evidence-grounded confidence or shortcut collapse. To resolve this ambiguity, we introduce low-
entropy anchors and an
entropy-shaping objective that reduces answer uncertainty while preserving baseline high-confidence
tokens. We instantiate this principle in SPOT-E, a plug-and-play test-time method that produces question-conditioned spotlights, optimized per instance via light-weight tuning based on Group Relative Policy Optimization (GRPO). Across all
benchmarks and different VLM families, SPOT-E yields consistent gains and improved robustness under visual corruptions. Code is publicly available at: https://github.com/YinBo0927/SPOT-E
Read the full abstract ▾
All authors Bo Yin, Xiaobin Hu, Chengming Xu, Ruolin Shen, Mo Yang, Jiangning Zhang, Peng-Tao Jiang, Cheng Tan, Shuicheng Yan
Computer Vision
BAFIS: Dataset + Framework to assess occupational Bias and Human Preference in modern Text-to-image Models
Thomas Klassert, Adrian Ulges and Biying Fu
Generative artificial intelligence has the potential to improve productivity and transform the production of creative content. However, existing research indicates that image generation models are significantly influenced by biases. This work investigates the inherent biases and
language-induced biases present in text-to-image models within the context of occupation-related image generation, complementing established metrics with human preference feedback. We present a comprehensive evaluation of five current text-to-image models: Midjourney v6.1, Stable
Diffusion 3 Medium, DALL-E 3, Playground v2.5, and FLUX.1-dev , focusing on gender and ethnicity bias, image quality, and
prompt alignment. To facilitate this evaluation, we developed the "Battle-Arena for Fair Image Synthesis" (BAFIS), a platform designed to collect human feedback on bias in generated images. Furthermore, we created a
dataset comprising 21,140 synthetic images generated using
multilingual prompts, which serves as a basis for our analysis. We further place our results within a broader social context by comparing them to official statistics from the German Federal Employment Agency. Our findings reveal systematic biases in text-to-image models, with established evaluation metrics in partial correlation with subjective user ratings. Thus, our research emphasizes the need for including human preferences to develop fairer and more inclusive text-to-image models.
Read the full abstract ▾
Computer Vision
DeepForestVisionV2: Ecology-Driven Taxonomy Expansion for Camera-Trap Monitoring in African Tropical Forests
Hugo Magaldi, Theau d'Audiffret, Etienne Francois Akomo-Okoue et al. (+24 more)
Camera-trap monitoring in African tropical forests increasingly extends beyond closed-canopy interiors to riverbanks, clearings, and park edges. Among available open tools for African forest camera-trap classification, DeepForestVision is the only one providing a matched offline workflow for both photographs and
videos, and previous work showed that it outperformed other available baselines on a comparable
benchmark. However, it was designed for closed-canopy, ground-level forest interiors and uses a 35-class prediction space that becomes too coarse when deployments encounter arboreal primates, birds, semi-aquatic taxa, or human-associated confounders such as livestock. We present DeepForestVisionV2, an ecology-driven expansion from 35 to 64 prediction classes (61 animal classes plus human, vehicle, and blank) designed to address three recurrent deployment gradients: vertical stratification, scene openness, and anthropogenic interfaces. DeepForestVisionV2 retains the same offline workflow and is trained on 1,535,010 photographs and 243,354
videos from multi-country African tropical-forest projects. Evaluation combines a cross-country cropped-photo validation set, used to assess robustness across sites and camera-trap settings, with three held-out Uganda
video benchmarks spanning the targeted gradients. On the validation set, DeepForestVisionV2 reaches 0.86 accuracy, 0.82 macro-F1, and 0.81 balanced accuracy. On the deployment
benchmarks, it preserves or improves baseline accuracy despite its harder classification task, while increasing the number of identified taxa from 22 to 29 in forest-interior
videos and from 4 to 9 at riverbanks. In the park-edge use case, it raises accuracy from 0.62 to 0.86 and reduces false alarms from 11 to 0. These results show that DeepForestVisionV2 materially improves field utility while preserving robustness across sites, habitats, and camera-trap settings.
Read the full abstract ▾
All authors Hugo Magaldi, Theau d'Audiffret, Etienne Francois Akomo-Okoue, Bala Amarasekaran, Naomi Anderson, Claire Auger, Noemie Cappelle, Daniel Cornelis, Raphael Cornette, Tobias Deschner, Gabriel Dubus, Davy Fonteyn, Rosa M. Garriga, Jennifer Hatlauf, Innocent Kasekendi, Raymond Katumba, Aram Kazandjian, Alfred Ngomanda, Stephan Ntie, Simone Pika, Xavier Rufray, Harold Rugonge, John Justice Tibesigwa, Peter van Lunteren, Hadrien Vanthomme, Joeri A. Zwerts, Sabrina Krief