Computer Vision
Vera: Identity-Faithful Human Subject-to-Video Generation
Yulong Xu, Xinyue Liu, Shujuan Li et al. (+6 more)
Subject-to-
video (S2V) generation has made substantial progress in preserving reference subjects across diverse categories, yet generic subject consistency remains insufficient for human-centric generation. A
video may appear globally consistent while identity-critical human details still drift across frames, poses, and interactions. This issue becomes more severe in multi-person scenarios, where incorrect identity-role binding leads to subject confusion, attribute swapping, and excessive copying of reference-specific appearance cues. We propose Vera, a unified human-centric S2V framework for single- and multi-person generation. We first construct a million-pair identity-aligned human image-
video dataset through person-level cross-clip
retrieval, providing explicit identity correspondence and diverse references. Built on this
dataset, Vera introduces two complementary designs. Identity-Focal Masked Supervision (IFMS) strengthens identity-aware learning with spatially focused supervision while reducing interference from irrelevant artifacts. Reference-Aware Layer-wise
Attention (RALA) regulates how
video tokens interact with reference identity cues in the DiT backbone, preserving stable identity anchors and enhancing layer-aware identity readout. Extensive experiments demonstrate that Vera improves human identity consistency, multi-person subject binding, and motion naturalness, while reducing identity confusion and excessive reference-image copying.
Read the full abstract ▾
All authors Yulong Xu, Xinyue Liu, Shujuan Li, huafeng shi, Yan Zhou, Jiwen Liu, Xintao Wang, Yu Shen Liu, Huaibo Huang
Computer Vision
Not All Patches are Equal: Sampling Matters for Visible-Infrared Pre-Training
Qiwei Ma, Bin Deng, Junjie Zhu et al. (+5 more)
Visible-infrared (VIS-IR)
alignment is a key pre-training task for robust multi-sensor perception. Most existing methods use uniform patch-wise contrastive learning, but this can be unreliable in VIS-IR data because imaging-physics differences make some spatially paired regions inherently less comparable, and aligning them with equal strength hinders representation learning and downstream transfer. In this paper, we revisit VIS-IR pre-training from a sampling perspective and propose Importance-Aware Sampling (IAS), which adjusts training emphasis based on patch reliability. Specifically, IAS (i) derives patch weights from infrared structural cues and uses them to reweight the contrastive objective; (ii) learns a soft importance mask with a lightweight sampler, optionally warm-started from the hand-crafted prior; and (iii) employs a patch curriculum learning strategy that gradually expands from high-reliability regions to harder patches. It is worth noting that IAS is plug-and-play and works with both patch-/correlation-level
alignment (e.g., UNIV-style) and image-level contrastive baselines (e.g., ImageBind-style). Extensive experiments on multiple VIS-IR
benchmarks demonstrate consistent improvements over strong baselines, including for IR semantic
segmentation, IR object detection and VIS semantic
segmentation and cross-modal
retrieval task. Code will be released on https://github.com/KlayMa527/IAS.
Read the full abstract ▾
All authors Qiwei Ma, Bin Deng, Junjie Zhu, Qiangjuan Huang, Puhong Duan, Ke Yang, Xudong Kang, Shutao Li
Computer Vision
RS-RIE-Bench: Benchmarking Reasoning-Guided Remote Sensing Image Editing
Zihan Qin, Boao Xu, Zhao Dong et al. (+4 more)
Remote sensing image editing aims to modify remote sensing images according to natural
language instructions while preserving geographic rules and sensor observation characteristics. Existing
benchmarks mainly target natural images or general visual scenes, and thus may not fully capture the
reasoning, regional control, and sensor-consistency abilities required in remote sensing editing. To fill this gap, we introduce RS-RIE-Bench, the first
benchmark for
reasoning-guided remote sensing image editing. RS-RIE-Bench organizes tasks into three categories: temporal
reasoning, causal
reasoning, and spatial
reasoning. These categories capture temporal
evolution, causal consequence, and spatial imaging consistency in remote sensing scenes. The evaluation protocol covers three dimensions: target region plausibility, non-target region preservation, and image quality consistency. We further demonstrate the feasibility of MLLM-based evaluation through cross-judge consistency analysis and stratified expert review. Systematic evaluation on eight open-source and closed-source image editing models shows that current models still have clear limitations in
reasoning-guided remote sensing editing. Even the strongest model achieves only 24.28% overall accuracy under the strict joint-satisfaction criterion, while the mean relaxed joint-4 success rate across all eight models is 32.23%. Causal
reasoning and spatial
reasoning remain especially challenging, and several open-source models are close to zero in some categories. These results show that RS-RIE-Bench can effectively reveal the limitations of current models in geographic
reasoning, regional control, and sensor-consistent generation. It also provides a standardized
benchmark and a clear research direction for future remote sensing intelligent editing models.
Read the full abstract ▾
All authors Zihan Qin, Boao Xu, Zhao Dong, Yingping Sun, Ziheng Jiao, Junying Wang, Hongwei Wang
Computer Vision
PerceptDrive: Perception Prior World-Action Modeling with Adaptive Expert Routing for End-to-End Autonomous Driving
Yushan Liu, Tianxiong Lv, Bohua Wang et al. (+11 more)
Frozen perception foundation models encode rich
geometric, semantic, and dynamic knowledge. Yet narrow conditioning interfaces may attenuate task-relevant cues, while static fusion cannot adjust expert contributions to each scene. We cast this challenge as the prior-to-plan transfer problem and introduce PerceptDrive, a perception prior world-action modeling framework with adaptive expert routing. PerceptDrive feeds teacher-distilled priors from a frozen, driving-adapted provider and dense observation latents from a frozen self-supervised
video encoder into a trainable expert-routed world-action model. Expert-specific query branches process these signals, while a prior-retention objective anchors each branch to its prior. A router predicts soft gates from a shared scene representation and combines the expert conditions before trajectory generation. During training, privileged rule-based sub-metric estimates for branch-specific trajectory drafts provide soft-gate
distillation targets. The predicted action-free future latent conditions a flow-matching actor. At inference, privileged components are absent; with one front-facing camera, PerceptDrive generates one trajectory per planning step without test-time scoring, reranking, or search. Experiments show that PerceptDrive achieves state-of-the-art performance with 90.4 PDMS on NAVSIM v1 and 90.2 EPDMS on NAVSIM v2, outperforming existing methods. Ablations confirm complementary gains from prior retention and scene-conditioned routing, alongside differential reliance on the three priors. These results demonstrate that preserving and adaptively routing perception priors improves direct planning without test-time candidate selection.
Read the full abstract ▾
All authors Yushan Liu, Tianxiong Lv, Bohua Wang, Hangqi Fan, Chenxu Zhao, He Zheng, Xuchang Zhong, Yifan Xie, Congyang Zhao, Zhihao Liao, Leigang Luo, Yang Cai, Xiao-Ping Zhang, Wenbo Ding
Computer Vision
StreamHOI: Interaction-aware Temporal Memory Adaptation for Streaming HOI Video Generation
Zejing Rao, Haoxian Zhang, Xiaoqiang Liu et al. (+5 more)
Existing human–object interaction (HOI)
video generation methods are largely limited to offline short-
video generation with complex driving conditions, making them unsuitable for real-time interactive applications. We present StreamHOI, a low-latency streaming framework for long-duration HOI
video generation. Instead of converting heavily conditioned HOI pipelines into streaming systems, we study how an image-to-
video streaming generator should organize historical
memory to preserve interactions under bounded latency. We find that the standard sink-local
memory design faces a trade-off in streaming HOI generation, and different
transformer blocks show different historical-
memory preferences for HOI regions and surrounding regions. To match
memory composition with block behavior, StreamHOI performs offline HOI-aware block profiling and applies bias-guided
memory-specialized training to adapt the generator to block-specific
memory layouts. We further introduce a
memory distance scaling module to strengthen long-range access to early interaction states. Extensive comparisons with both long-
video baselines and recent HOI generation methods demonstrate that StreamHOI achieves strong interaction plausibility, object fidelity, human quality and efficiency, reaching 17.6 FPS with 0.75s first-chunk latency.
Read the full abstract ▾
All authors Zejing Rao, Haoxian Zhang, Xiaoqiang Liu, Yiping Meng, Guoxin Zhang, Pengfei Wan, Fan Tang, Tong-Yee Lee
Computer Vision
SHFormer: Dynamic Spectral Filtering Convolutional Neural Network and High-pass Kernel Generation Transformer for Adaptive MRI Reconstruction
Sriprabha Ramanarayanan, Rahul G. S., Mohammad Al Fahim et al. (+3 more)
Attention Mechanism (AM) selectively focuses on essential information for imaging tasks and captures relationships between distant pixel neighborhoods to compute feature representations. Accelerated MRI reconstruction benefits from AM, as the imaging process involves
Fourier domain measurements that influence image representation non-locally. However, AM-based models are more adept at capturing low-frequency information with limited capacity for high-frequency representations, restricting models to smooth reconstruction. Additionally, AM-based models need mode-specific retraining for
multimodal MRI data, as their knowledge is restricted to local contextual variations that may be inadequate to capture transferable features across heterogeneous domains. To address these challenges, we propose a neuromodulation-based discriminative multi-spectral AM for scalable MRI reconstruction that can (i) propagate context-aware high-frequency details for high-quality reconstruction, and (ii) capture features reusable across deviated unseen domains in
multimodal MRI. The proposed network consists of a spectral filtering CNN to capture mode-specific transferable features and a dynamic high-pass kernel generation
transformer focusing on high-frequency details. We evaluate our model on comparative studies in supervised and self-supervised learning,
diffusion model-based training, closed-set and open-set generalization under heterogeneous MRI data, and interpretation-based analysis. Our method offers scalable, high-quality reconstruction with best improvement margins of ~1 dB in PSNR and ~0.01 in SSIM under unseen scenarios. Code: https://github.com/sriprabhar/SHFormer
Read the full abstract ▾
All authors Sriprabha Ramanarayanan, Rahul G. S., Mohammad Al Fahim, Keerthi Ram, Ramesh Venkatesan, Mohanasankar Sivaprakasam
Computer Vision
Real-Time EEG Cap Electrode Detection for Guided Point-of-Care Placement
William Lehn-Schiøler, Mads Sverker Nilsson and Nicki Skafte Detlefsen
We present a two-stage
vision system that detects
EEG cap electrodes in a live webcam stream and validates their anatomical placement in real time. A single-class YOLO detector localises electrodes; a
geometric stage assigns each detection to a named 10-20 role from facial landmarks. Evaluating under subject-disjoint leave-one-subject-out (LOSO) cross-validation across five subjects wearing the clinically-validated Small/Medium/Large caps, the detector attains mAP@.5 = 0.94 +/- 0.07 across five held-out folds (0.96 pooled). A dedicated leave-one-cap-out axis, holding out every frame of a cap regardless of subject, leaves Medium and Large mAP@.5 within 0.01 of LOSO (0.97, 0.97) while Small drops to 0.72 +/- 0.28, a gap confounded with subject familiarity rather than cap style.
Geometric augmentation (rotation, perspective, mixup) improves in-plane-roll robustness and temporal-electrode recall at no inference cost, and a landmark-driven head crop extends the usable distance range, lifting mAP@.5 from 0.23 to 0.45 at 0.6 x apparent scale. A compact mobile-candidate backbone (YOLOv10n) keeps the detector at real-time throughput (19 FPS) on a commodity CPU at 640 px.
Read the full abstract ▾
Computer Vision
HeadCast: Casting Attention Heads for Efficient Autoregressive Video Generation
Jinliang Shen, Lianghao Su, Zheming Li et al. (+4 more)
Autoregressive (AR)
video diffusion models have become a promising paradigm for long and streaming
video synthesis, but the continuously growing Key-Value (KV) cache makes
attention the dominant inference cost, especially at high resolution where each frame contributes many
tokens. Existing remedies either evict the cache with coarse heuristics that cause inter-frame flickering, or require model re-training. We propose HeadCast, a training-free, plug-and-play acceleration framework built on the observation that a pre-trained AR model's
attention heads exhibit stable, heterogeneous behaviors. After a short warm-up, HeadCast performs a one-time classification at the maximum-noise step that sorts every head into one of four archetypes: Sink, Dummy, Spatial, and Global, and restructures the monolithic KV cache into head-specific pathways. Crucially, it retains the Global heads that preserve the long-range temporal consistency aggressive eviction destroys. Because the Spatial pathway operates on a fixed-size grid, its savings grow with resolution: across state-of-the-art AR models, HeadCast accelerates inference by up to 1.62x at 720P and 1.95x at 1080P, while keeping VBench quality on par with full
attention and largely flicker-free. Code is available at https://github.com/sjlgaga/HeadCast .
Read the full abstract ▾
All authors Jinliang Shen, Lianghao Su, Zheming Li, Kang He, ZiLiang Lai, Yanbing Jiang, Chengru Song
Computer Vision
RIM: A Retrieval-In-Matching Framework for Cross-Domain Global Visual Localization of UAVs
Xin Li, Siyuan Duan, Shang Wang et al. (+3 more)
Global visual localization of unmanned aerial vehicles (
UAVs) using remote-sensing reference maps has attracted increasing
attention. However, differences in acquisition time and imaging platform between
UAV and reference imagery introduce substantial cross-domain appearance and viewpoint shifts, challenging robust six-degree-of-freedom (6-DoF) pose estimation. To mitigate these shifts, we render
UAV-viewpoint references from Google 3D Tiles across locations, altitudes, and orientations. A two-stage strategy adapts SALAD with pose-near positives and geographically distant hard negatives; local
geometric consistency then re-ranks the Top-K candidates. We further propose
Retrieval-In-Matching (RIM), which freezes the adapted DINOv2-B retriever and distills a local-descriptor decoder from its
token field and a shallow VGG19 detail stream. One query-side DINOv2-B backbone forward therefore supports both SALAD
retrieval and local description, eliminating a second foundation-model backbone while preserving the
retrieval descriptors by construction. We evaluate RIM zero-shot on the reconstructed EPFL Urbanscape and self-collected Chang'an Park
datasets, both geographically disjoint from the training data. RIM outperforms ten
retrieval baselines. Under the full 3D distance metric at 25/50 m, it improves Recall@1 over SALAD by 8.55/13.77 percentage points on EPFL and 4.45/8.94 points on Park. At Top-K=5, the measured online query path through
retrieval, candidate matching, and robust
geometric verification takes 90.8 ms: 1.2 times faster than the strongest separate sparse-matching baseline and over 30 times faster than RoMa, while maintaining comparable re-ranking accuracy. These results demonstrate an efficient
UAV global visual localization pipeline under unreliable satellite
navigation. The source code is available at https://github.com/curious-energy/RIM.
Read the full abstract ▾
All authors Xin Li, Siyuan Duan, Shang Wang, Zhimin Mao, Bingliang Hu, Geng Zhang
Computer Vision
ENTRAP-VL: A Taxonomic Probe for Dual Contextual Entrainment in Vision-Language Models
Karan Goyal, Afreen Hossain, Debojyoti Das et al. (+1 more)
Contextual entrainment is the tendency of a model to let auxiliary context in its input pull its output, independently of whether that context is relevant, true, or even meaningful. Recently, it has been identified and given a mechanistic account in unimodal
language models. Whether and how it manifests in
vision-
language models (VLMs) is, by contrast, largely unexamined, and the field lacks a purpose-built instrument with which to investigate it. We take the position that studying contextual entrainment in VLMs requires more than porting an existing text-only
benchmark to the
multimodal setting: it requires a taxonomically structured, dual-modality instrument whose conditions are constructed around the item at hand (the depicted image in the textual stream, the textual query in the visual stream). We argue that the move to VLMs is substantive rather than incremental. It makes entrainment a dual phenomenon, drivable independently by textual and by visual context, and it opens a veracity distinction (context that is false of the depicted scene yet possible in the world) that has no counterpart in the unimodal, world-knowledge-only formulation of prior work. To make this position concrete and actionable, we introduce ENTRAP-VL (ENTRainment Assessment Probe for
Vision and
Language), a manually curated
dataset of 1,500 items across eight categories, organized by a taxonomy that spans two axes, i.e., the association of context with the item and its relationship to truth, and split into a textual-entrainment stream (eight context conditions) and a visual-entrainment stream (three context conditions). We do not claim to measure entrainment in any particular model; we provide the instrument, the taxonomy that motivates it, and the evaluation protocols it enables, so that the community can investigate the phenomenon rigorously. We will release the
dataset and its documentation publicly.
Read the full abstract ▾
All authors Karan Goyal, Afreen Hossain, Debojyoti Das, Vishal Bhutani
Computer Vision
Development of an automated, reliable, and clinically meaningful artificial intelligence (AI) tool for diagnosing cardiac disease from conventional cardiovascular magnetic resonance (CMR) images
Sina Amirrajab, Volker Vehof, Michael Bietenbeck et al. (+6 more)
Aims: Cardiovascular
magnetic resonance (CMR) imaging enables non-invasive assessment of myocardial structure, function, and pathology, but requires substantial experience in interpretation of CMR images that could be supported by artificial intelligence (AI)-based models. However, use of AI models for enhanced CMR reading is limited by labor-intensive data curation, suboptimal model performance, and unclear implementation pathways. Methods and results: We developed an automated data curation pipeline for CMR-based cardiovascular disease (CVD) diagnosis, integrating open-source locally-run large
language models (LLMs) to extract diagnostic labels from narrative CMR reports and preprocessing
multimodal imaging data, including cine and late-gadolinium-enhancement (LGE) CMR sequences. Three
vision foundation models (DINO, VST, UMedPT) were fine-tuned across these modalities in a two-stage approach. The
dataset comprised hypertrophic cardiomyopathy (HCM), dilated cardiomyopathy (DCM), ischemic cardiomyopathy (ICM), cardiac amyloidosis (CA), and normal controls (NOR). A total of 988 curated cases were randomly divided into 742 for training and 246 for validation. Fine-tuned AI-models achieved high discriminative diagnostic performance on an independent test set comprising 1067 patients , with individual AUC-ROC values of up to 0.937 for the correct diagnosis of HCM and 0.945 for cardiac amyloidosis. Ensemble strategies combining multiple models and modalities further improved AI-based diagnostic accuracy and robustness, achieving the highest overall diagnostic performance for HCM (AUC=0.959, CI [0.936-0.978]), CA (AUC=0.966, CI [0.939-0.986]), NOR (AUC=0.872, CI [0.852-0.894]), DCM (AUC=0.848, CI [0.808-0.885]) and ICM (AUC=0.840, CI [0.809-0.868]). All training and inference code, along with the trained model weights, are publicly available on https://github.com/sinaamirrajab/CMR_CVD.
Read the full abstract ▾
All authors Sina Amirrajab, Volker Vehof, Michael Bietenbeck, Nuriye Akyol, Redouane Bouras, Khuraman Isgandarova, Alexandru Zlibut, Philipp Stalling, Ali Yilmaz
Computer Vision
Factor-Informed Uncertainty Distillation for Gaze Estimation
Mohammadreza Jamalifard, Yaxiong Lei, Javier Fumanal Idocin et al. (+3 more)
Deep gaze estimation works well in controlled capture but degrades in unconstrained settings, where systems must reject unreliable predictions. Single-pass uncertainty (e.g., heteroscedastic regression) infers uncertainty from pixels without explicit input-validity cues, while sampling based methods are often too costly for real time use. We propose Factor-Informed Uncertainty
Distillation (FIUD), a teacher-student framework that aligns uncertainty with interpretable image-quality failure modes. A gradient-boosting teacher predicts expected gaze error from factors such as illumination, sharpness, eye visibility and
symmetry; a neural student distills these signals via curriculum learning and ranking supervision into a lightweight single-pass uncertainty head. Across ETH-XGaze, Gaze360, and MPIIFaceGaze (>300k samples), FIUD improves uncertainty, error rank correlation and selective prediction versus deterministic and sampling-based baselines, with the largest gains in unconstrained settings.
Read the full abstract ▾
All authors Mohammadreza Jamalifard, Yaxiong Lei, Javier Fumanal Idocin, Parastoo Azizinezhad, Tom Foulsham, Javier Andreu-Perez
Computer Vision
GaussianSeed: Hierarchical Gaussian Seeding for High-Resolution 3D Occupancy Prediction
Xinzhuo Li, Xianghui Pan, Jiayuan Du et al. (+4 more)
Vision-centric 3D occupancy prediction provides dense scene representations essential for autonomous driving and robotic
navigation, yet existing methods struggle to scale to high voxel resolutions due to prohibitive computational costs. To address this, we introduce GaussianSeed, a progressive multi-scale Gaussian occupancy prediction framework that organizes primitives into a coarse-to-fine hierarchy. Benefiting from this hierarchical design, GaussianSeed effectively circumvents the
memory bottlenecks inherent in dense representations, successfully scaling to a $0.1\text{m}$ spatial resolution while maintaining real-time inference capabilities. To comprehensively evaluate high-resolution
geometric perception, we further construct TJScenes, a panoramic six-camera occupancy
dataset with highly detailed $0.1\text{m}$ annotations. Extensive experiments on Occ3D-nuScenes and TJScenes demonstrate that GaussianSeed delivers the lowest latency among all evaluated methods while maintaining highly competitive accuracy, advancing the efficiency-quality frontier of high-resolution 3D occupancy prediction. Codes are available at https://github.com/Athameral/GUSD
Read the full abstract ▾
All authors Xinzhuo Li, Xianghui Pan, Jiayuan Du, Wei Wei, Liuyi Wang, Chengju Liu, Qijun Chen
Computer Vision
Importance-Aware OBS Pruning for Diffusion Models
Ba-Thinh Lam, Srijan Das and Hieu Le
We propose importance-aware
pruning for
diffusion models, a training-free framework that prioritizes preserving parameters critical to semantically salient image regions. To do so, we incorporate spatial importance maps – derived from conditioning signals or model
attention – into the
pruning objective. This produces parameter rankings aligned with perceptual relevance rather than uniform reconstruction error. On MS-COCO
dataset, our proposed approach consistently retains subject fidelity and structural correctness at high compression ratios where conventional
pruning causes visible degradation. These results demonstrate that content-aware objectives are key to perceptually faithful compression of generative models.
Read the full abstract ▾
Computer Vision
A Systematic Benchmark of Intensity Normalisation Methods for 3D Knee MRI Segmentation and Cross-Domain Generalisability
Oliver Mills, Philip Conaghan and Samuel Relton
Robust out-of-the-box performance is essential for the clinical deployment of deep learning models in medical imaging. An important but underexplored factor affecting model generalisability is intensity normalisation, particularly for
magnetic resonance imaging (MRI), where image intensities vary across scanners and protocols. In this study, we systematically compared seven normalisation methods and their impact on the performance of a 3D U-Net model for meniscus
segmentation from knee MRI. The methods included standard scaling approaches, histogram-based techniques, and a Gaussian Mixture Model (GMM)-based method. Models were trained on the IWOAI 2019
dataset and evaluated on both internal and external test sets (SKM-TEA) to assess generalisability. Performance was similar internally but differences were significant on external data, with Z-score, Nyúl histogram matching, and CLAHE showing greater robustness than other methods. However, these differences were small compared to the significant performance drop observed between
datasets. Overall, while intensity normalisation had a measurable effect on model generalisability, its impact was limited relative to the effects of domain shift, highlighting the need for complementary strategies for robust deployment.
Read the full abstract ▾
Computer Vision
Toward Seasonal Guidelines for Robust Deep-Learning Sentinel-2 Building Detection in Different Area Types
Michał Romaszewski, Kamil Drejer, Katarzyna Kołodziej et al. (+7 more)
Sentinel-2 imagery offers open access, global coverage, and frequent revisit times, making it attractive for practical building mapping at scale; however, its native 10m resolution makes building vs non-building classification challenging, particularly for small or sub-pixel buildings, and performance can vary with both seasonality and the heterogeneity of built-up environments. This paper introduces a Sentinel-2 building-detection framework designed to systematically quantify these effects and to support more formalised, practice-oriented model selection. We construct a dedicated multi-temporal Sentinel-2
dataset over the Warsaw region and derive binary ground-truth masks by rasterising official Polish topographic database (BDOT10k) building footprints onto the Sentinel-2 pixel grid. Using two established convolutional
segmentation backbones (U-Net and DeepLabV3+), we first perform scene-specific fine-tuning to select a robust architecture and identify the best monthly models for L1C and L2A products separately. We then conduct cross-temporal inference by applying each best monthly model to all scenes, enabling an assessment of (i) which months provide favourable training and inference conditions, (ii) how performance transfers between seasons, (iii) the impact of processing level, and (iv) how these effects differ across built-up typologies. Based on these results, we provide practical guidance for routine Sentinel-2 building classification under varying acquisition periods and settlement characteristics.
Read the full abstract ▾
All authors Michał Romaszewski, Kamil Drejer, Katarzyna Kołodziej, Anna Zawadzka, Stanisław Lewiński, Przemysław Głomb, Marek Ruciński, Michal Krupiński, Krzysztof Gryguc Przemysław Sekuła, Szymon Sala
Computer Vision
STEREOFLOW: Progressive Stereo Matching with StereoDiT and Transition Flow Matching
Hao Wang, Haoran Geng, Xiaotong Yang et al. (+7 more)
Stereo matching is a fundamental task in 3D reconstruction. Despite remarkable advances, the prevailing paradigms formulate stereo matching as a deterministic regression problem, collapsing the
multimodal distribution modeling into a single-point estimation. This formulation suffers from a regression-to-mean bias, frequently struggling with ambiguous regions. In contrast, we introduce a prior-guided generative framework that integrates deterministic matching regression and generative distribution modeling within a complementary formulation. Built upon this formulation, we introduce StereoFlow through three key components: (i) a two-stage progressive cascade matching network that progressively produces multi-resolution stereo conditions with complementary matching cues; (ii) a pixel
diffusion transformer (termed StereoDiT) with a frequency-decoupled architecture for modeling correspondence ambiguity; (iii) a few-step flow matching objective (termed Transition Flow Matching) for efficient optimization. In summary, \textsc{StereoFlow} achieves strong
geometric consistency and rich fine-grained details in ill-posed, discontinuous regions and under zero-shot generalization. Extensive experiments demonstrate that the proposed StereoFlow establishes multiple state-of-the-art results across
benchmarks, including Scene Flow, KITTI, ETH3D, and Middlebury.
Read the full abstract ▾
All authors Hao Wang, Haoran Geng, Xiaotong Yang, Jing Tang, Songlin Wei, Linlong Lang, Yeying Jin, Zheng Zhu, Zhaoxin Fan, Biao Leng
Computer Vision
Forecasting the Number of Harvest-ready Fruits of Sweet Peppers Using Multimodal Time-Series Data
Enrico Pallotta, Mohamed Farag, Esra Guclu et al. (+3 more)
Accurate yield
forecasting at the individual-plant level is critical for precision agriculture and supply-chain planning, yet public
datasets capturing both visual growth dynamics and per-plant measurement labels are scarce. In this paper, we introduce a novel, annotated image time-series
dataset of 691 sweet pepper plants monitored over two growing seasons, comprising 4837 images with per-plant fruit counts categorized by maturity. We propose a
multimodal deep learning framework that fuses high-dimensional image features, extracted using the DinoV3 encoder, with numerical count measurements. Our architecture utilizes a Long Short-Term
Memory (LSTM) network to model temporal dependencies and handles irregular sampling intervals common in greenhouse monitoring. Through quantitative experiments, we demonstrate that this
multimodal approach reduces RMSE over a persistence baseline by 33% and 38% in the 2022 and 2023 seasons, respectively, with a further 1.2% average gain over a measurement-only model. Furthermore, we employ Deep Ensembles and Gaussian Negative Log-Likelihood (NLL) to provide calibrated uncertainty estimates, with an Uncertainty Calibration Error (UCE) ranging from 0.39 to 0.89 depending on the cross-season evaluation direction, offering a principled confidence signal for real-world agricultural decision-making. We release the
dataset and code to support reproducible research and to accelerate development of data-driven yield
forecasting methods for horticultural crops.
Read the full abstract ▾
All authors Enrico Pallotta, Mohamed Farag, Esra Guclu, Chris McCool, Ribana Roscher, Juergen Gall
Computer Vision · Cross-listed from cs.RO
Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Training
Taewon Seo, Seonae Jeon, Giwon Lee et al. (+2 more)
Accurate motion prediction of surrounding
agents and safe motion planning are two closely coupled key tasks for social
robot navigation in crowded environments. Deploying these systems on resource-constrained edge devices necessitates compact, unified models that can perform both tasks simultaneously. However, within these compact shared encoders, recent unified models often overlook severe representational conflicts that arise from the distinct objectives of predicting neighbor behaviors versus ego-centric safety planning. To address this issue, we first identify the Skill Conflict$\unicode{x2014}$a phenomenon where overlapping parameter assignments cause distinct tasks to compete for the same weights, preventing the model from fully specializing in individual skills. To resolve this, we propose a novel model-merging-based framework, Disjoint Parameter Training (DPT). DPT mitigates performance degradation caused by Skill Conflict through distributed parameter learning, which separates the key parameter regions of each task while preserving their core capabilities prior to merging. In addition, we observe that sparse merging, which selectively integrates only the most influential parameters for each task rather than combining all task-specific parameters, yields optimal performance by preventing interference among adjacent features and concentrating representational capacity. DPT can be applied in parallel with a variety of merging methods. Evaluated on standard crowd
navigation benchmarks (JRDB and JTA), our framework demonstrates superior performance, validating its versatility and effectiveness for safe, resource-efficient
robot navigation.
Read the full abstract ▾
All authors Taewon Seo, Seonae Jeon, Giwon Lee, Kuk-Jin Yoon, Daehee Park
Computer Vision
OffNadirLoc: Benchmark and Framework for Challenging UAV-to-Satellite Geo-Localization under Large Off-Nadir Views
Qian Qiao, Wenye Liu, Ting Liu et al. (+2 more)
Cross-view geo-localization between
UAV and satellite imagery remains a fundamental yet highly challenging task, especially under large off-nadir views where drastic perspective distortions, occlusions, and appearance gaps occur. Existing
benchmarks and methods primarily focus on near-nadir scenarios and often overlook the importance of structural scene understanding and intra-domain relational constraints, limiting their performance in real-world deployments. In this work, we introduce OffNadirLoc, a new
benchmark for large off-nadir
UAV-to-satellite geo-localization. To tackle the unique challenges posed by off-nadir perspectives, we further propose ONLoc, a framework that incorporates a structure-aware contextual weighting mechanism to dynamically emphasize reliable local features while suppressing ambiguous or repetitive regions. Additionally, we design a view-coherent learning strategy, which treats one satellite image and the corresponding
UAV images from multiple views as a cohesive semantic group. This set-level supervision enables the model to learn viewpoint-invariant and discriminative features, making it more effective at capturing multi-view consistency than conventional pairwise contrastive learning. Extensive experiments on the OffNadirLoc
benchmark and four near-nadir
datasets demonstrate that our method consistently outperforms state-of-the-art approaches while exhibiting strong zero-shot generalization to unseen
datasets without additional training. The code will be released at https://montalario.github.io/offnadirloc/.
Read the full abstract ▾
All authors Qian Qiao, Wenye Liu, Ting Liu, Jiuhe Shu, Peng Wang
Computer Vision
ETPDesigner: Multi-Agent Orchestration for Interactive Multimodal Electronic Theater Program
Mengtian Li, Xinru Guo, Xiaoru Lin et al. (+3 more)
Electronic Theater Programs (ETPs) serve as critical promotional media in the performing arts, comprising a multi-page collection of heterogeneous visual assets such as theatrical posters, performance details, and character portraits. However, existing text-to-image paradigms struggle with such complex design tasks due to their inability to comprehend long-context narratives and maintain visual consistency across multiple distinct pages. To address this, we introduce ETPDesigner, a collaborative Multi-
Agent framework that directly synthesizes high-quality ETPs from raw dramatic scripts. Emulating a professional design pipeline, our framework orchestrates specialized
agents for semantic script analysis, core poster synthesis, functional background generation, and the stratified composition of character assets. Central to ETPDesigner is a global style anchor mechanism that extracts visual priors from the core poster to enforce strict aesthetic uniformity across all generated components. Furthermore, we elevate the ETP from a static publication to an immersive interactive companion. By integrating portrait animation, customized
speech synthesis, and persona-grounded Large
Language Models (LLMs), our system enables users to engage in real-time, voice-enabled conversations with the generated virtual characters. To rigorously
benchmark this task, we construct ETP-Pro, a domain-specific
benchmark of professional theater posters and high-quality character portraits. Extensive evaluations demonstrate our method's superiority in producing semantically faithful, aesthetically consistent, and highly interactive program sets.
Read the full abstract ▾
All authors Mengtian Li, Xinru Guo, Xiaoru Lin, Xiao Rong, Zhifeng Xie, Chaofeng Chen
Computer Vision
SIINR: Structurally Informed Implicit Neural Representations for super-resolution with uncertainty quantification of clinical quality diffusion MRI datasets
Tom Hendriks, William Consagra, Anna Vilanova et al. (+2 more)
Diffusion Magnetic Resonance Imaging (dMRI) is a powerful tool for probing
brain microstructure, but clinical acquisitions are often limited by low out-of-plane resolution, resulting in degraded structural information and reduced utility for advanced analysis. We introduce SIINR (Structurally Informed Implicit Neural Representations), a general framework for super-resoltion of clinical dMRI
datasets while quantifying uncertainty in the reconstructed outputs. SIINR utilizes a supervised 3D U-net as a prior and combines it with a self-supervised implicit neural representation (INR) that fuses the high-resolution prior and the original low-resolution data. The INR enables joint modeling across spatial and angular domains, enforces data consistency, and provides analytic approximate posterior distributions for downstream uncertainty quantification. We validate the framework on a diverse set of open-access dMRI
datasets, demonstrating that SIINR outperforms standard interpolation methods in both quantitative error metrics and qualitative anatomical fidelity. Experiments on clinical cases, including subjects with multiple sclerosis and
brain lesions, illustrate the framework its ability to propagate intensity changes and flag uncertain regions in challenging scenarios. SIINR is flexible, modular, and can be adapted to different upsampling ratios and downstream tasks, providing a principled approach for enhancing clinical dMRI and supporting robust interpretation of derived neuroimaging metrics.
Read the full abstract ▾
All authors Tom Hendriks, William Consagra, Anna Vilanova, Yogesh Rathi, Maxime Chamberland
Computer Vision
G-MAD: A Game-Based Data Generation Framework for Multi-View RGB-T Aerial Object Detection
Yechan Kim, JongHyun Park, Dongho Yoon et al. (+2 more)
This work introduces G-MAD, an open-source framework that uses Arma3 to generate synchronized multi-view RGB-T data for aerial object detection. G-MAD addresses key limitations of real-world aerial
dataset construction, including limited viewpoint control, imperfect RGB-T
alignment and high annotation cost. The framework supports structured scenario specification, controllable multi-view camera placement, simultaneous visible/thermal capture, and automatic bounding box annotation using engine-level
geometric metadata. These capabilities enable controlled studies of viewpoint variation, multi-modal fusion, and synthetic-to-real transfer in aerial object detection. Besides, using G-MAD, we construct and release AMOD, a new large-scale multi-view aerial RGB-T object detection
benchmark. The source code and the
dataset are available at https://unique-chan.github.io/G-MAD-Project.
Read the full abstract ▾
All authors Yechan Kim, JongHyun Park, Dongho Yoon, Namhoon Jung, Moongu Jeon
Computer Vision
WearWow: Native 2K Multi-Garment Virtual Try-On via Adaptive Token Packing and Preference Alignment
Xujie Zhang, Runyan Du, Song Chang et al. (+7 more)
Synthesizing native 2K multi-garment virtual try-on is a formidable frontier in digital fashion, critically bottlenecked by two fundamental limitations: the O(N^2)
memory explosion induced by 2k conditions, and the spectral bias of
diffusion models that over-smooths high-frequency fabric details. We present WearWow, an end-to-end, mask-free generative framework that pioneers ultra-high-resolution multi-garment synthesis. To mitigate the
memory explosion , we propose Adaptive 2D
Token Packing (ATP). ATP leverages inherent garment sparsity to algorithmically pack heterogeneous items onto a unified 2D canvas and prune uninformative background
tokens, minimizing the effective sequence length and subsequent
memory overhead while rigorously preserving 2D spatial priors. To rectify texture degradation, we introduce the Multi-dimensional Try-on Reward (MTR) system. MTR synergizes a Semantic Guidance Reward to explicitly drive tactile restoration with a Cloth Distribution Reward to implicitly anchor the physical distribution, a joint formulation that effectively mitigates the severe reward hacking. Furthermore, we curate WearWow-2K, an extreme-quality
dataset comprising native 2K triplets, providing physically correct spatial interactions that naturally empower the model's mask-free generation. Extensive experiments demonstrate that WearWow establishes a new state-of-the-art, exceeding existing commercial baselines in native 2K multi-garment synthesis.
Read the full abstract ▾
All authors Xujie Zhang, Runyan Du, Song Chang, Jiang Li, Dongliang Shao, Liping Wu, Wei Luo, Xiaochao Qu, Luoqi Liu, Xiaodan Liang
Computer Vision
LoRFT: Benchmarking Long-Range Vehicle Trajectory Reconstruction from Fixed Highway Cameras
Yufan Zhu, Kefu Yi, Xueju Zhang et al. (+3 more)
Long-range vehicle trajectories provide important spatio-temporal evidence for traffic safety analysis, autonomous driving evaluation, and data-driven traffic management, yet continuously recovering them from fixed highway cameras remains difficult. As vehicles recede into distant road regions, perspective compression and scale decay often fragment or prematurely terminate automatic tracklets, even when their continuation remains identifiable from motion consistency across neighboring frames. We formulate this problem as recovering the far-range continuation of a vehicle trajectory from a reliable near-field tracklet. We introduce LoRFT, to our knowledge the first open
benchmark dedicated to long-range vehicle trajectory reconstruction from fixed highway cameras. LoRFT comprises 22 expressway surveillance scenes, 366,109
video frames, 6,601 manually verified trajectories, 2,694,889 bounding boxes, road-geometry annotations, scene-level splits, and evaluation scripts. We further propose Map-RSTNet, a map-aware residual sequence-to-sequence model that reconstructs distant trajectories in a road-geometry-aligned state space and dynamically refreshes local road geometry during decoding. On LoRFT, Map-RSTNet reduces ADE, FDE, and 5-second RMSE by 11.0%, 15.4%, and 10.5%, respectively, relative to the strongest baseline. These results demonstrate that road-geometry-aware reconstruction can extend usable trajectory records from existing fixed-camera infrastructure. LoRFT provides a reproducible testbed for long-range vehicle trajectory reconstruction.
Read the full abstract ▾
All authors Yufan Zhu, Kefu Yi, Xueju Zhang, Yunyang Tian, Long Chen, Zixuan Xiao
Computer Vision
MV-Bench: Benchmarking Multimodal Large Language Models for Coordinated Multi-View Interface Construction
Yue Zhao, Hongxu Liu, Feiyu Wang et al. (+5 more)
Multimodal large
language models (MLLMs) are increasingly expected to automate visualization development by generating code directly from visual designs. However, existing evaluations mainly focus on single-chart generation and overlook coordinated multi-view interface construction, which requires joint
reasoning about data semantics, view coordination, and interaction logic. Consequently, MLLM capabilities in this setting remain underexplored, and the field lacks a dedicated
benchmark for systematic assessment. We introduce MV-Bench, a
benchmark for evaluating MLLMs on coordinated multi-view interface construction. Instead of relying on incomplete or inconsistent open-source implementations, we use Tableau workbook files as ground truth because they explicitly encode data bindings, visual mappings, and interactions. We develop a multi-stage pipeline that converts these specifications into executable web interfaces through structured intermediate representations. The
benchmark contains 92 base interfaces and 1,048 verified instances created by recombining chart types,
datasets, and interaction patterns. Each instance includes executable code, a rendered interface, a
dataset, and interaction annotations. We evaluate five state-of-the-art MLLMs in a single-pass setting using metrics for visual fidelity, data binding correctness, and interaction completeness. The strongest model achieves 75.45 percent accuracy in visual layout reproduction, but only 21.71 percent in data binding and 11.68 percent in interaction completeness. These results show that current MLLMs can reproduce visual appearance but remain limited in generating the data semantics and interactive logic required by coordinated multi-view interfaces. Iterative refinement improves code executability but does not substantially reduce the gap in data binding and interaction generation.
Read the full abstract ▾
All authors Yue Zhao, Hongxu Liu, Feiyu Wang, Xiaoyu Yang, Tong Ge, Zhen Yang, Chao Wang, Qiong Zeng
Computer Vision
StrokeSeg2: Stroke Lesion Segmentation in Clinical Research Workflows
Youwan Mahé, Axel Plessis, Stéphanie Leplaideur et al. (+3 more)
Deep learning frameworks like nnU-Net achieve state-of-theart
brain lesion
segmentation performance but remain difficult to deploy in clinical research environments due to, among other reasons, software dependencies and computational requirements. We introduce StrokeSeg2, a lightweight, modular, cross-platform C++/Qt framework designed to adapt resource-intensive 3D stroke
segmentation pipelines into portable and reproducible applications. To improve compatibility with standard clinical workstations, we investigate the combined effect of architectural compression through knowledge
distillation and inference optimisation using ONNX Runtime with Float16 quantisation. Across heterogeneous hardware configurations (CPU, integrated GPU, and dedicated GPU) architectural
distillation emerged as the primary contributor to efficiency gains, contributing to over 90% reduction in energy consumption and an average 84% reduction in inference time. Specifically, we identify a 0.84M-parameter student model as the most favourable trade-off, reducing the original 102.3M-parameter teacher architecture to a 2.1 MB disk footprint while preserving robust lesion localisation and competitive
segmentation performance. This small footprint supports the development of a self-contained installer for clinical workstation targets. Finally, StrokeSeg2 packages these optimisations into standalone installers for Windows, macOS, and Linux. By providing both graphical and commandline interfaces without Docker or external environment dependencies, StrokeSeg2 facilitates deployment of high-performance
segmentation workflows for routine clinical research pipelines.
Read the full abstract ▾
All authors Youwan Mahé, Axel Plessis, Stéphanie Leplaideur, Elise Bannier, Florent Leray, Francesca Galassi
Computer Vision
OSVE: One Step Video Editing with One Step Diffusion Models
Habin Lim and Gyeong-Moon Park
Text-guided
video editing with
diffusion models is impractically slow, hindered by costly multi-step sampling and inversion. We present OSVE, the first framework to successfully adapt one-step Text-to-Image (T2I) models for high-quality
video editing, addressing the core challenges of inversion, editability, and temporal consistency. To bypass slow iterative inversion, we train a learnable encoder that predicts the initial noise for each frame in a single forward pass. This encoder is trained with a novel Structure-Aware Editing (SAE) loss on a curated
dataset of structurally-aligned image pairs, teaching it to preserve the source
video's geometry during edits. For temporal coherence, we introduce Unified-Frame Editing (UFE), a technique that concatenates frame latents to facilitate cross-frame
attention in a single generation step. Furthermore, for long
videos, a sliding-window strategy with an anchor frame maintains global consistency. Our extensive experiments demonstrate that OSVE achieves editing quality comparable or superior to state-of-the-art multi-step methods, while operating approximately 155–171 times faster. This breakthrough paves the way for practical, real-time
video editing applications. Code is available at https://github.com/KU-VGI/OSVE.
Read the full abstract ▾
Computer Vision
Latent-Action-Guided Video-Language Feature Learning for Surgical Instrument-Tissue Interaction Recognition
Jiajun Cheng, Sainan Liu, Subarna Tripathi et al. (+2 more)
Recognizing instrument–tissue interactions is essential for context-aware
surgical AI.
Vision-
language models offer a natural way to inject semantic structure into
surgical representations by aligning
video features with textual action descriptions. However, pretrained encoders may lack spatial coherence, while global semantic
alignment does not ensure precise spatial and temporal representations. By analyzing frame-to-frame feature changes, we find that semantic
alignment increases their dimensionality, but larger increases do not necessarily improve recognition; encoders also differ in how strongly dominant changes localize to interaction regions. Motivated by these findings, we introduce \ours{}, which compresses frame-to-frame changes into latent actions and predicts next-frame features during end-to-end
video–
language alignment. Without additional spatial or motion annotations, \ours{} improves the interaction grounding of leading feature changes and temporal-direction sensitivity in our evaluated settings. We further characterize how action capacity and prediction strength affect recognition across encoders and triplet components. Using image encoders without large-scale
video pretraining, \ours{} achieves competitive recognition with faster inference and smaller INT4 accuracy drops than V-JEPA2/2.1.
Read the full abstract ▾
All authors Jiajun Cheng, Sainan Liu, Subarna Tripathi, Xiaofan Yu, Shan Lin
Computer Vision
MTVDiff: Multimodal Conditional Latent Diffusion for Enhanced Thermal-to-Visible Face Translation
Zhiyuan Xia, Haojie Li, Jingyu Lin et al. (+2 more)
Thermal-to-visible face translation presents fundamental challenges including
geometric discontinuities, semantic attribute mismatches, and identity degradation. We propose MTVDiff, a novel
multimodal latent
diffusion framework that synergistically integrates depth and textual information to address these limitations while preserving identity characteristics. The MTVDiff framework presents three core technical contributions: (1) a Dual-Branch Cross-
Attention Fusion (DBCAF) module for multi-scale thermal-depth feature extraction and fusion; (2) a Gated Text-to-Visual Feature
Alignment mechanism for semantically-guided generation; and (3) Spatial Feature Transformations (SFT) for adaptive
multimodal prior integration. Extensive experiments on the MCXFace and SpeakingFaces
datasets demonstrate that our
multimodal approach significantly outperforms existing GAN-based and
diffusion-based approaches across multiple metrics, achieving substantial improvements in both image quality and face verification performance, with FID reductions of up to 48.3% and Rank-1 accuracy improvements of up to 8.9%. Our work provides a robust solution for face recognition systems operating under varying illumination conditions and advances the state-of-the-art in cross-spectral facial image translation through effective
multimodal integration.
Read the full abstract ▾
All authors Zhiyuan Xia, Haojie Li, Jingyu Lin, Yiguo Qiao, Cunjian Chen