Computer Vision · Cross-listed from cs.CL
When Modality Gap Reduction Fails: Prediction-Level Hubness in CLIP
Shota Sato, Hajime Kiyama, Tosho Hirasawa et al. (+1 more)
Reducing the modality gap between image and text representations in CLIP is widely expected to improve cross-modal
alignment and downstream performance. However, a smaller average image-text gap does not necessarily lead to consistent accuracy gains. We analyze this mismatch from the perspective of the decision structure in zero-shot classification, i.e. selecting the most similar class-text prototype for an input image. Zero-shot accuracy depends not only on average image–text
alignment, but also on class-wise decision margins. Using Linear correction as an analytically tractable case, we show that modality gap correction can alter the relative decision structure among classes and cause predictions to concentrate on a small subset of classes. We refer to this output-space failure mode as prediction-level hubness. Furthermore, experiments across multiple
datasets show that accuracy degradation under gap correction is consistently associated with increased prediction concentration, both for Linear correction and for learning-based correction methods. This provides a systematic explanation of why modality gap reduction does not consistently improve CLIP zero-shot accuracy from the perspective of downstream decision structure. Our results suggest that gap correction should be evaluated not only by average
alignment, but also by its impact on downstream prediction structure.
Read the full abstract ▾
All authors Shota Sato, Hajime Kiyama, Tosho Hirasawa, Mamoru Komachi
Computer Vision
IT-TextFusion: Iterative Text-Image Interaction with Text-Guided Residual Refinement for Degradation-Aware Image Fusion
Siyang Liu, Peiyi Zhou, Tianle Jin et al. (+3 more)
Text-guided image fusion has recently emerged as an effective paradigm for integrating multi-modal information while enabling flexible and task-oriented fusion control. However, existing text-guided fusion methods often rely on shallow semantic-visual interaction and limited
attention mechanisms, which restrict their ability to robustly handle complex degradations and fully exploit textual guidance. In this paper, we propose an iterative text-guided image fusion framework that incorporates text-conditioned feature interaction across multiple fusion and refinement stages. The proposed method integrates deepest-level Cross-
Attention, multi-scale Cross-Gate Fusion, and stage-specific text-conditioned modulation, allowing the global text
embedding to condition hierarchical feature fusion and residual refinement. By repeatedly injecting the pooled text
embedding across hierarchical decoder and refinement stages, the proposed framework provides degradation-aware global semantic conditioning while preserving complementary information from the visible and infrared modalities. Experiments on several
benchmark datasets show that the proposed method improves several information-preservation and perceptual-quality metrics, while exhibiting metric-dependent trade-offs on some
datasets.
Read the full abstract ▾
All authors Siyang Liu, Peiyi Zhou, Tianle Jin, Rongrong Bian, Zheke Jin, Mengze Gao
Computer Vision · Cross-listed from cs.LG
Let Confidence Change, Not the Prediction: Prediction-Preserving Repair for Post-hoc Calibration
Daehwan Kim, Haejun Chung and Ikbeom Jang
Post-hoc calibration corrects reported confidence, yet a multiclass calibrator can also change the associated top-1 prediction. Accuracy captures only the net effect of these changes on correctness, not how often predictions change; the Top-1 Prediction Change Rate (TPCR) instead measures this frequency. We propose Calibrator-Output Repair for Top-1 Decision Preservation (CORD), the first post-fit adapter to impose exact prediction preservation by repairing the full calibrated probability vector. From the original and calibrated outputs alone, CORD determines the mass assigned to the original top-1. The calibrated conditional distribution allocates the remaining mass over the other classes, yielding a repaired vector whose own argmax recovers the original prediction. On the calibration split, CORD coordinates the repaired masses to retain the calibrated outputs' mean mass on original predictions whenever attainable. The adapter alters neither the fitted calibrator nor its direct output, fits no additional supervised map, and requires no user- or validation-tuned hyperparameter. Across CIFAR-10/100 and ImageNet-1K, CORD attains zero TPCR by construction and lowers mean ECE, NLL, and Brier relative to the corresponding direct outputs in every
dataset; paired gains persist under distribution shift and across calibration-set sizes. CORD thus removes the preservation constraint from calibrator fitting and assigns exact recovery of the original decision to subsequent output repair. Our code is available at https://github.com/labhai/CORD.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Lightweight Interpretable RGB-Guided Hyperspectral Super-Resolution under Real Cross-resolution Misalignment
Mohamad Jouni, Aurélien Godet and Mauro Dalla Mura
Compact snapshot hyperspectral cameras provide rich instantaneous spectral measurements for ground-level machine
vision, but at lower spatial resolution than standard RGB cameras. RGB-guided hyperspectral super-resolution (HSR) addresses this limitation by transferring spatial detail from a high-resolution RGB guide to a low-resolution hyperspectral image (HSI). These dual-camera systems are typically in a horizontal rig geometry, requiring cross-camera image
alignment due to different fields of view. However, residual misregistration can inject spurious high-frequency details. Existing learned unaligned-fusion methods are usually trained for a fixed spectral support and spatial scale factors and can be computationally demanding, limiting their flexibility across sensors. We propose a lightweight and interpretable RGB-guided HSR framework combining cross-modal flow
alignment with model-based Gram-Schmidt orthogonalization fusion. The method first warps the RGB guide onto the HSI grid, then estimates an energy-based confidence weight map by measuring local
alignment reliability. This map is then used both in a weighted least-squares spectral regression and in a gated fusion between the super-resolved estimate and an HSI-preserving estimate. Unlike existing learned methods, the proposed framework has a low computational footprint and supports VIS-NIR spectral supports and scale factors without retraining. Experiments on the Real
benchmark show that the proposed method improves reconstruction accuracy over learned fusion baselines while remaining substantially faster. On a 34-frame sequence acquired with our real RGB-HSI dual-camera setup, a reduced-resolution quantitative evaluation validates the method under genuine cross-sensor radiometric, noise, and
geometric differences, while native-resolution qualitative results demonstrate deployment on the full 51-band VIS-NIR acquisition.
Read the full abstract ▾
Computer Vision
Dyn-3D: Unveiling and Resolving Ego-Motion Ambiguity in Vision-Language Models
Jiayu Ding, Zhuodong Liu, Lei Zhang et al. (+6 more)
As
Vision-
Language Models (VLMs) tackle dynamic 3D spatial
reasoning, ego-motion perception becomes essential to resolve
monocular scale ambiguity. However, current models often overfit to smooth trajectory priors rather than genuinely understanding physical motion. Consequently, their spatial
reasoning degrades severely under large displacements, a phenomenon we term Kinematic Collapse. This failure stems from spurious visual-motion correlations in natural
videos and a lack of explicit physical supervision. To evaluate this, we introduce Dyn-3D, a
benchmark using counterfactual 3D rendering to rigorously decouple visual changes from true kinematic properties. Furthermore, we propose the TempoVista framework, featuring the Kinematic-GSPO
algorithm. By
embedding metric physical ground truth into policy optimization, TempoVista explicitly grounds visual representations in 3D space. Experiments demonstrate that our approach significantly improves both motion estimation and robust spatial
reasoning by utilizing camera dynamics as an effective
geometric calibration signal.
Read the full abstract ▾
All authors Jiayu Ding, Zhuodong Liu, Lei Zhang, Manyu Xiong, Hongbo Jin, Haoran Tang, Hongbo Zhang, Changen Zhu, Wenbo Xing
Computer Vision · Cross-listed from cs.LG
SAGE: Subpopulation-Aware Generative Enhancement for Mitigating Spurious Correlations
Yiming Luo, Rongqiang Zhao and Jie Liu
Spurious correlations pose a significant challenge to the robustness of modern machine learning. The inherent imbalance in
dataset distributions often leads traditional Empirical Risk Minimization (ERM) models to rely on majority spurious attributes for classification, resulting in poor performance on minority groups. This problem becomes particularly challenging when the spurious attributes are unavailable. Existing group-label-free methods often upsample minority groups or misclassified real training examples; repeating the same instances can reduce effective diversity and encourage overfitting. To mitigate these spurious correlations from a data-centric perspective in the absence of prior knowledge, we introduce Subpopulation-Aware Generative Enhancement (SAGE), a two-stage generative augmentation framework. Using cluster-derived sub-labels and class labels, we fine-tune a conditional generative model and text encoder, generating targeted synthetic data to fill underrepresented regions in the training set and construct a balanced validation set for last-layer reweighting. We experimentally show that SAGE achieves 89.5%, 85.7%, and 79.1% worst-group accuracy on Waterbirds, CelebA, and MetaShift, respectively, outperforming the best group-label-free baselines by up to 7.7 percentage points.
Read the full abstract ▾
Computer Vision
ViTAMINS: An Empirical Study of Training Self-Supervised Vision Transformers with Synthetic Hard Negatives
Nikos Giakoumoglou, Andreas Floros, Kleanthis-Marios Papadopoulos et al. (+1 more)
We introduce ViTAMINS, a method that integrates synthetic hard negatives into unsupervised
vision transformer pretraining to improve representation quality. Our approach is thoroughly benchmarked on ImageNet and transfer learning, image
retrieval, copy detection, and image,
video segmentation tasks. Notably, our proposed negatives give rise to emergent properties, where learned representations contain explicit information about the semantic content of an image and serve as excellent classifiers (up to +11.3% over baselines). ViTAMINS achieves these benefits through simple modifications to existing contrastive frameworks and outperforms competing methods while being more resource efficient, e.g., our ViT-B surpasses V-JEPA with ViT-L. Our findings motivate reconsidering contrastive learning as a simpler yet powerful alternative to dominant generative and self-
distillation approaches.
Read the full abstract ▾
All authors Nikos Giakoumoglou, Andreas Floros, Kleanthis-Marios Papadopoulos, Tania Stathaki
Computer Vision · Cross-listed from cs.CR
MultiGait: A Multi-Sensor Multi-Perspective Multi-Session Biometric Inference Benchmark and its Dataset
Julian Todt, Felix Morsbach, Philip Dissert et al. (+1 more)
A lack of suitable
datasets has limited the research into the
privacy risks of novel smart city sensors, such as thermal cameras, depth cameras, and
lidar. Given the number of unsubstantiated
privacy claims and their potential widespread deployment into many people's everyday life, understanding the
privacy risks of these sensors – in isolation and in like-for-like comparisons – is crucial. With MultiGait, we collected the first multi-sensor, multi-perspective, multi-session gait-focused
dataset, for the corresponding, and additional more far-reaching investigations. The
dataset, validated with multiple state-of-the-art recognition systems, comprises various walking modes and annotated personal attributes for 199 individuals, to ensure the benefit for advanced studies including cross-sensor recognition and anonymization at the edge. MultiGait represents a foundation for rigorous
privacy investigations, demonstrated through an extensive identity inference
benchmark across eight sensors, four perspectives, and three recording sessions. Our
benchmark incidentally reveals that sensors often assumed to be
privacy-friendly do still entail considerable identity inference risks, while the poor cross-session generalization of existing methods underscores an important research gap.
Read the full abstract ▾
All authors Julian Todt, Felix Morsbach, Philip Dissert, Thorsten Strufe
Computer Vision
Fi-ImageNet-1k: An OOD Benchmark From the Inside of the ImageNet-1k Validation Set
Ruslan Rozumnyi, Matěj Suchánek, Tomáš Vojíř et al. (+2 more)
Out-of-distribution (OOD) detection predicts whether a test image belongs to none of the predefined classes. To evaluate this task,
benchmarks need images from outside the in-distribution (ID) data; typically, these are defined or collected in an ad hoc fashion. Since no ground truth is perfect, ID-labeled
datasets themselves contain a natural source of OOD images. We exploit such annotation errors and present Fi-ImageNet-1k, an OOD
dataset built from ImageNet-1k validation images that the recent ReImageNet reannotation effort assigned to no ImageNet-1k class. Each image was examined by expert human annotators supported by evidence from MLLMs, VLMs, and reverse image search, comparing it against all visually similar ID classes. We keep only images that could be assigned a specific class outside the ImageNet-1k label space. The resulting Fi-ImageNet-1k, with 655 images from 522 classes, is substantially more challenging than any commonly used OOD
dataset. No evaluated combination of classifier and OOD detector achieves a false positive rate below 51% at 95% true positive rate (FPR@95). Compared to the recent NINCO, our
dataset is 3.8x more challenging in the FPR@95 metric for state-of-the-art supervised OOD detection methods.
Read the full abstract ▾
All authors Ruslan Rozumnyi, Matěj Suchánek, Tomáš Vojíř, Klára Janoušková, Jiří Matas
Computer Vision · Cross-listed from physics.med-ph
PyDoseRT Proton: A GPU Pencil-Beam Engine with a Convolutional Residual-Correction Network for Fast Proton Dose Calculation
Lukas Zimmermann, Hermann Fuchs, Attila Simkó et al. (+1 more)
Architecture category. Hybrid method: a physics-based analytical pencil-beam (PB) dose engine followed by a 3-D convolutional residual-correction network (RepVGG-U-Net). We addressed the DoseRAD2026
proton dose-prediction task with PyDoseRT
Proton, a GPU-accelerated engine implemented in PyTorch and augmented by a learned residual toward Monte Carlo (MC) accuracy. A double-Gaussian PB kernel was calibrated to GATE/Geant4 integrated depth doses in water in two stages: a classical per-energy curve fit, then a gradient-based fit of the full 3-D dose through the PyTorch physics engine as it retains a differentiable execution path for gradient-based optimization of dose-dependent objectives. The engine computes each beamlet on a beam's-eye-view (BEV)
lattice with variance-preserving Gaussian splitting, an analytic
nuclear halo, and a
Fermi-Eyges heterogeneity term, then rotates the result into the patient frame. Additionally, a compact residual U-Net predicts an additive correction in BEV space. It is conditioned on voxelwise material-label
embeddings, a discrete energy
embedding and spot size. The same model was used for all anatomical sites (thoracic and abdominal). It was trained with a patient-space L1 objective emphasizing the scored high-dose region and multi-scale BEV deep supervision. The submitted CT configuration obtained preliminary-test beamlet MAE 0.0066, image-z IDD distance 0.0025, plan MAE 0.0049, 98.30% gamma pass rate (1%/1 mm), and DVH error 0.460.
Read the full abstract ▾
All authors Lukas Zimmermann, Hermann Fuchs, Attila Simkó, Gerd Heilemann
Computer Vision
Low-Quality Face Recognition using Center Aligned Representations and Local Margin Constraints
Vedat Can Dilaver and Benjamin S. Riggan
Low-quality face recognition (LQFR) remains challenging due to the difficulty of matching degraded query (probe) images against low-quality (LQ) enrollment (gallery) imagery and the scarcity of training data for large-scale models. While recent face recognition (FR) models perform well on high-quality (HQ) imagery, their accuracy drops significantly on LQ images with extremely low signal-to-noise ratio (SNR). Moreover, fine-tuning HQ-pretrained models on LQ data often improves LQ recognition at the expense of HQ generalization. This trade-off becomes more pronounced in modern evaluation settings spanning multiple
datasets with varying image quality levels. To address these limitations, we propose a unified framework that combines three main components: (1) Local Probability Margin (LPM), which estimates per-sample difficulty directly from the model's discriminative landscape; (2) Nested
Attention Module (NAM), a new low-rank adapter module that embeds a self-
attention mechanism within selected
transformer layers; and (3) Quality Gating Protocol (QGP), where an off-the-shelf image quality
estimator modulates the adapter contribution at test time, enabling a single model to handle the full quality spectrum without sacrificing HQ performance. Experiments on surveillance (TinyFace, SurvFace) and standard (IJB-B, IJB-C) face recognition
benchmarks demonstrate consistent gains in both identification and verification. Code and models will be released at github.com/candllq/nam.
Read the full abstract ▾
Computer Vision
Shiyu Li, Zi-Yuan Hu, Shijia Huang et al. (+3 more)
Despite their strong
multimodal understanding ability,
multimodal large
language models (MLLMs) incur substantial computational overhead when processing long visual
token sequences. To reduce inference costs, recent studies have explored visual
token pruning through
vision-centric or text-guided strategies. However, these methods often overlook high-norm outlier
tokens, i.e.,
tokens with abnormally large feature norms, leading to suboptimal
pruning decisions. In this work, we show that such high-norm outlier
tokens are highly redundant in both feature and spatial dimensions, yet are often mistakenly preserved as informative cues by existing methods. Motivated by this observation, we propose SinkPruner, a training-free visual
token pruning framework for efficient MLLM inference. SinkPruner follows a coarse-to-fine design with two key modules: a visual sanitizer that filters high-norm redundancies and alleviates
attention sink and
attention dispersion, and a text-guided pruner that further retains
tokens semantically aligned with the text query. Extensive experiments on twelve image-
language and four
video-
language benchmarks demonstrate the effectiveness, efficiency, and generalizability of our framework. Notably, SinkPruner preserves 96.5% (91.8%) of the original performance of LLaVA-1.5 (Qwen2.5-VL) under an 89%
token reduction. Experiments further indicate that our visual sanitizer exhibits promising transferability in enhancing the performance of existing
pruning methods. Our code is available at https://github.com/LaVi-Lab/SinkPruner.
Read the full abstract ▾
All authors Shiyu Li, Zi-Yuan Hu, Shijia Huang, Yanyang Li, Yiwu Zhong, Liwei Wang
Computer Vision
Does This Moment Justify the Recommendation? Counterfactual Behavior-Grounded Evidence Retrieval for Personalized Video Recommendation
Xin Liu
Personalized
video recommendation predicts user preference at the
video level, while temporal
video grounding localizes query-relevant moments. However, strong localization does not establish whether the retrieved moment constitutes valid evidence for recommending the
video to a particular user. We study counterfactual behavior-grounded evidence
retrieval, which separates where personalized evidence occurs from whether such evidence exists and evaluates whether model predictions respond consistently when that evidence is replaced. We introduce CBGER-10K, containing 5,000 controlled factual–counterfactual pairs for 3,026 users, where each pair replaces only the focal behavior-supported segment while preserving the user, temporal position, and hard distractors. We further propose CBGER, a compact framework that decouples segment-level localization from
video-level evidence estimation and learns both through structured counterfactual supervision. CBGER achieves $0.4432$ MRR, $0.6977$ Pair Accuracy, and $0.6987$ Intervention Consistency across five adapted personalized-highlight and temporal-grounding baselines. Notably, compared with QD-DETR, its MRR improvement is not statistically significant, while Pair Accuracy improves by $11.03$ points. These results show that accurate temporal localization does not necessarily imply reliable personalized evidence existence, motivating explicit evaluation of Whether alongside Where.
Read the full abstract ▾
Computer Vision
CQF-HMR: Continuous Quaternion Flows for Probabilistic 3D Human Mesh Recovery from a Single Image
Cuong Le, Bao-Long Tran, Pavlo Melnyk et al. (+3 more)
Recovering 3D digital humans from a single 2D image is an ill-posed computer
vision problem due to the loss of depth information. Probabilistic 3D human pose estimation compensates for this by estimating a set of 3D hypotheses from a prior distribution via generative models. However, most prior work focuses only on 3D keypoints, which often leads to implausible poses that are difficult to apply to downstream tasks, e.g. animation or digital humans. SMPL-based methods are more scalable thanks to the explicit body priors, but it requires more complex modeling of the generation process due to the non-additive nature of the joint rotations. In this work, we propose a novel approach for probabilistic 3D humans using quaternion-constrained continuous normalizing flows conditioned on 2D pose estimations. Our proposed quaternion flows show significant advantages over approaches using other rotation representations. Experiments demonstrate state-of-the-art results of our method on Human3.6M, particularly in ambiguous settings, and comparable pose estimation accuracy on challenging 3DPW and EMDB
benchmarks.
Read the full abstract ▾
All authors Cuong Le, Bao-Long Tran, Pavlo Melnyk, Tahereh Dehdarirad, Bastian Wandt, Mårten Wadenbäck
Computer Vision
EvoGS: Modeling Deformation Evolution for Dynamic Gaussian Splatting
Wei Dong, Shahram Shirani, Jun Chen et al. (+1 more)
Recent extensions of 3D Gaussian Splatting (3DGS) enable real-time novel view synthesis in dynamic scenes by learning time-conditioned Gaussian deformations. However, existing MLP-based methods typically estimate deformations independently at each timestamp, making them less robust to large or abrupt motions. To address this issue, we propose EvoGS, a 3DGS-based dynamic reconstruction framework that models Gaussian deformation as a temporal
evolution process. EvoGS maintains persistent deformation states for each Gaussian, extrapolates future states from historical deformation states, and corrects the predictions with MLP-derived observations. The correction is adaptively weighted using a temporal residual
memory and
evolution statistics such as deformation velocity and trajectory deviation. To further improve reconstruction quality, EvoGS introduces deformation-aware densification. Clone and split operations are performed along corrected deformation directions, while an uncertainty-aware strategy suppresses densification for Gaussians with unstable deformation histories. Experiments show that EvoGS improves dynamic novel view synthesis quality and achieves competitive performance across
benchmarks.
Read the full abstract ▾
All authors Wei Dong, Shahram Shirani, Jun Chen, Han Zhou
Computer Vision
Semi-Supervised Virtual Staining via Morphology Preservation and Histopathological Realism Constraints
Baoshun Wang, Weiping Lin, Linwu Wang et al. (+3 more)
Virtual staining aims to computationally generate target-stained histopathological images while reducing the cost and time associated with conventional staining procedures. However, existing methods rely predominantly on strictly paired and accurately registered training data, which are difficult and expensive to obtain in routine practice. To reduce this dependence, we propose a stable semi-supervised virtual staining framework that jointly exploits both limited paired data and abundant unpaired source images. Directly incorporating unpaired images is challenging because their generated results lack corresponding targets for supervision, potentially leading to unrealistic staining, morphological degradation, or even training collapse. To obtain reliable supervision from these images, Hessian-derived morphology preservation extracts structural cues from each source image and constrains the generated output to retain tissue morphology. Histopathological realism constraints further guide the output toward plausible target-stain characteristics, preventing the source-derived structural supervision from degenerating into contour enhancement or simple color transformation. Together, the two components suppress structural and appearance drift, stabilize semi-supervised stain translation, and promote the preservation of diagnostically relevant information. Extensive experiments on H&E-to-IHC translation for Ki67 and HER2, as well as FFPE-to-H&E translation, demonstrate consistent improvements in image quality, morphology preservation, robustness, and downstream diagnostic performance. Code will be available.
Read the full abstract ▾
All authors Baoshun Wang, Weiping Lin, Linwu Wang, Yihuang Hu, Baptiste Magnier, Liansheng Wang
Computer Vision · Cross-listed from eess.IV
Prior-Guided Implicit Neural Representations for Single-Subject Diffusion MRI Super-Resolution
Abdulkader Ghandoura, Marsil Zakour, William Consagra et al. (+1 more)
Resolving complex fiber geometries in
brain white matter requires high-resolution
diffusion MRI at the cost of long acquisition times. This leads many clinical protocols to opt for low-resolution scans, making downstream microstructure estimation and tractography challenging. Implicit neural representations (INRs) can model the
diffusion signal continuously, enabling native single-subject super-resolution by querying the network at arbitrary spatial coordinates, yet existing methods often suffer from long training times and lack a mechanism to incorporate anatomical priors to regularize super-resolution by constraining the space of plausible reconstructions. To address these limitations, we propose a novel transfer-learning framework that pre-trains an INR on a high-resolution template and then adapts it to subject-specific scans via registration and fine-tuning. For $4\times$ through-plane super-resolution from 5 mm to 1.25 mm on Human Connectome Project (HCP) data, our method reduces NRMSE by 36-49% and increases FSIM by 24-43% over a recent baseline with $6\times$ faster training, outperforming competing INR-based methods across both image quality and domain-specific metrics. Code is available on the project page at https://abdulkaderghandoura.github.io/research/msc-thesis/ .
Read the full abstract ▾
All authors Abdulkader Ghandoura, Marsil Zakour, William Consagra, Yogesh Rathi
Computer Vision
ReFlowSET: Representation-Aligned Latent Flow Matching for SAR-to-EO Image Translation
Jeonghyeok Do, Seungchul Lee and Munchurl Kim
SAR-to-EO image translation aims to generate electro-optical (EO) imagery from synthetic aperture radar (SAR) observations. Existing latent
diffusion approaches typically inherit a predetermined autoencoder, although reconstruction fidelity can vary substantially across codecs and modalities. Because the latent codec affects the round-trip preservation of both SAR conditions and EO targets, codec selection constitutes a fundamental design choice; nevertheless, existing methods largely rely on codecs pretrained on natural images. To remedy this, we introduce ReFlowSET, a conditional latent flow-matching framework that selects its codec through a joint SAR–EO reconstruction audit. Rather than inheriting a heavyweight pretrained generator, ReFlowSET trains a substantially smaller conditional DiT from scratch in the selected latent space, using dual-stream SAR conditioning followed by joint feature refinement. To provide semantic guidance for this from-scratch training, intermediate noisy-EO features are aligned with clean target-EO representations extracted by a frozen
vision foundation model. This
alignment is used only during training and introduces no additional inference cost. Experiments on QXS-SAROPT and SAR2Opt demonstrate state-of-the-art performance across diverse perceptual fidelity and distributional metrics. Code and pretrained weights are publicly available at https://github.com/KAIST-VICLab/ReFlowSET.
Read the full abstract ▾
Computer Vision
Conditional Flow Matching for Cross-Field MRI Harmonisation
Baris Imre, Aram Salehi, Levente Baljer et al. (+3 more)
Magnetic resonance images of the same subject look markedly different across field strengths, which complicates the comparison and pooling of data across sites. We address cross-field
brain-MRI translation for the MRIxFields2026 challenge, and in particular its Task~3: a single model that translates between any directed pair of the five field strengths and across three contrasts. We phrase the problem as a conditional flow matching path: because the source and target volumes are spatially registered, we learn a velocity field that carries the source slice directly to the target slice, rather than starting from noise. To learn this mapping from only three paired subjects, the unified model is trained in three stages: a degradation-bridge pretraining that distills a restoration prior from the abundant unpaired retrospective cohort, a cross-field finetuning over all directed pairs on the paired cohort, and an
adversarial refinement that sharpens the output. At inference, we integrate the learned velocity with a second-order Heun solver in a handful of steps. A restoration prior learned without any paired data already reaches a mean SSIM of 0.837, and each subsequent training stage improves on it. A single 6.3M-parameter model thereby covers all 60 field-pair and contrast combinations, with inference in five solver steps per slice. On the challenge evaluation set the model reaches a mean SSIM of 0.909, averaged over the three contrasts, outperforming regression and
diffusion baselines built on the identical network on all three challenge metrics.
Read the full abstract ▾
All authors Baris Imre, Aram Salehi, Levente Baljer, Andrew Webb, Marius Staring, Efe Ilicak
Computer Vision
Candidate-Expanding Routing with Permutation-Stabilized Experts for Mixed-Format Medical VQA
Hai-Dang Nguyen and Huy-Hieu Pham
Mixed-format medical visual question answering (VQA) requires stable option selection and machine-readable free-text output. The two formats fail differently: multiple-choice predictions can change with option symbols or positions, while clinically plausible open answers can fail automated evaluation when serialization is malformed. We address both challenges with an answer-text
memory, a permutation-stabilized
vision–
language expert, and a sparse candidate- expanding router. The cyclic schedule follows prior work; our contribution is to make expert top-2 a routable candidate alongside
memory and expert top-1. On a 1,403-case retrospective internal analysis, this expansion improves a matched binary router from 88.95% to 91.73% (+2.78 percentage points; 95% CI 1.57–3.99), with 56 rescued errors and 17 regressions. Oracle coverage rises from 90.31% to 96.15%, and the final submitted configuration reaches 92.23% on the same retrospective split. For open questions, strict generation and deterministic guards produce 475/475 schema- valid participant-facing outputs without repair, retry, or hard-gate failure. Visual ablations reveal substantial textual dependence. Candidate expansion supplies the principal controlled routing gain; open-path evidence establishes output-contract validity rather than clinical correctness in medical use or deployment.
Read the full abstract ▾
Computer Vision
PredErase: Training-Free Object-and-Effect Removal with Predictive Latent Guidance
Waikit Xiu, Qiang Lu, Junbiao Chen et al. (+1 more)
Removing an object is not the same as filling its mask. Cast shadows and contact shading usually lie outside the user-provided instance mask M_obj, so a frozen Fill model that edits only that mask leaves the object's photometric footprint on nearby surfaces. Supervised removers learn this joint erasure from paired clean plates. Training-free editors freeze pretrained weights, yet most still treat M_obj as the entire editable support and steer sampling with CLIP or DINO energies that do not predict the occluded scene. We present PredErase, a training-free inference procedure on frozen FLUX.2 and I-JEPA. The method separates where Fill may rewrite pixels from what structure should occupy the hole. A contact-band expansion M_flux of M_obj exposes local residuals on the supporting plane. I-JEPA, pretrained for masked
token prediction, supplies a context-conditioned hole target in representation space; sparse projected gradients align decoded Fill completions with that target inside the instance, while coordinates outside the packed support stay locked. Under instance-only masks on RemovalBench, RORD-Val, and DEFACTO-Val, PredErase improves the native FLUX.2 backbone. Supervised removers remain stronger on several full-image appearance metrics; the supported claim is training-free object-and-effect editing of frozen Fill, not replacement of paired-data erasers.
Read the full abstract ▾
All authors Waikit Xiu, Qiang Lu, Junbiao Chen, Xiying Li
Computer Vision
ASSERT: Adaptive Stochastic Sampling for Robust Diffusion Models on Analog Compute-in-Memory Hardware
Yuannuo Feng, Yizhe Chen, Wenshuai Yao et al. (+4 more)
Diffusion models achieve strong image generation quality but incur high iterative denoising costs. Analog compute-in-
memory (CIM) can accelerate
matrix-vector multiplications, yet spatial
memory variations perturb weights and accumulate during sampling. Unlike conventional neural networks,
diffusion models' temporal sensitivity to hardware noise remains underexplored. We investigate
diffusion inference using a noise model calibrated and validated against measurements collected from multiple physical CIM chips. Our results show that the early, high-noise denoising stage is substantially more vulnerable than the final refinement stage. A first-order trajectory analysis attributes this behavior to the repeated propagation of correlated prediction errors induced by a fixed hardware mapping. Based on this observation, we propose ASSERT, a training-free sampler that uses higher stochasticity early and smoothly transitions to deterministic denoising. The injected stochasticity changes subsequent activation trajectories and thereby reduces their
alignment with persistent spatial errors. Across the evaluated settings, ASSERT achieves up to 2.58$\times$ lower FID than deterministic DDIM on high-resolution
datasets and 7.68$\times$ lower FID in the CIFAR-10 step-count study, without changing model parameters or the number of network evaluations.
Read the full abstract ▾
All authors Yuannuo Feng, Yizhe Chen, Wenshuai Yao, Yuxin Xie, Ngai Wong, Wenyong Zhou, Wang Kang
Computer Vision
CERF: Communication-Efficient and Retraining-Free Collaborative Perception
Jiuwu Hao, Ziyi Ni, Liguo Sun et al. (+5 more)
Collaborative perception shares information among multiple
agents to obtain a comprehensive scene representation, enhancing the perceptual capability of individual
agents. However, most existing methods rely on transmitting and fusing dense feature maps for collaboration, which incurs inevitable communication overhead and heterogeneity challenges, limiting their practicality for real-world deployment. To address these challenges, we propose CERF, a novel Communication-Efficient and Retraining-Free framework for open heterogeneous collaborative perception. In CERF, we introduce a new virtual modality (termed Poture), which is generated from the perception outputs of other
agents, to augment the extracted Bird's Eye View (BEV) features of the ego
agent. To mitigate transmission delays, we employ a Kalman-filter based tracker and a motion
forecasting model to derive the current predictions from historical perception results. Extensive experiments demonstrate that CERF achieves performance comparable to mainstream intermediate-collaboration methods while reducing communication overhead by 95% across various downstream tasks. Furthermore, CERF enables seamless integration of unknown heterogeneous
agents into the existing collaborative framework without additional retraining costs. Code is available at https://github.com/uestchjw/CERF.
Read the full abstract ▾
All authors Jiuwu Hao, Ziyi Ni, Liguo Sun, Yuting Wan, Yueyang Wu, Ti Xiang, Haolin Song, Pin Lv
Computer Vision
From Terminology to Diagrams: Visual-Instruction Generation for Scientific Diagram Understanding
Raul Ortega and José Manuel Gómez-Pérez
Vision-
language models (VLMs) have demonstrated strong performance in visual question answering with natural images. However, they continue to struggle with scientific diagrams, which are designed to convey functional or relational meaning rather than literal scenes. We therefore introduce a framework for generating large-scale diagram-grounded instruction data by leveraging terminology derived from scientific curricula. Our approach systematically extracts domain concepts, synthesizes atomic facts, retrieves relevant diagrams from the web, and generates
multimodal supervision in the form of diagram captions and multiple-choice questions. Using this pipeline, we construct SciGram, a
dataset of over 194K diagrams and 1.4M visual instructions across life, earth, and physical sciences. Despite relying on noisy web data and synthetic annotations, models fine-tuned on SciGram achieve substantial improvements on diagram-centric
benchmarks, including TQA, ScienceQA, and AI2D, outperforming or matching state-of-the-art VLMs while using fewer training instances. Furthermore, augmenting existing models such as LLaVA OneVision with SciGram establishes new state-of-the-art performance on diagram question answering. Our results highlight the effectiveness of terminology-grounded instruction generation as a general strategy for improving
vision-
language reasoning in scientific domains. To support future research in scientific diagram understanding, we release both the SciGram
dataset and models.
Read the full abstract ▾
Computer Vision
Beyond the Image Plane: World-Grounded Queries for Multi-Object Tracking
Orcun Cetintas, Guillem Brasó, Tim Meinhardt et al. (+1 more)
Monocular videos record 3D scenes as sequences of 2D image-plane projections, obscuring depth and spatial relationships. Multi-object trackers localize and associate objects primarily using appearance and geometry observed only in the image plane, inheriting these ambiguities. To address this limitation, we introduce
PLANET, an end-to-end multi-object tracker designed to move beyond the image plane. As an enabling step, we lift existing 2D tracking
datasets into 3D. We then form world-grounded queries by
embedding reconstructed 3D scene geometry into the features and positional encodings used during query formation. An auxiliary 3D location prediction task further encourages the queries to encode object positions during training. A complementary dual-resolution temporal
memory preserves this evidence across longer temporal gaps. As a result,
PLANET achieves state-of-the-art performance across three diverse
benchmarks.
Read the full abstract ▾
All authors Orcun Cetintas, Guillem Brasó, Tim Meinhardt, Laura Leal-Taixé
Computer Vision
On-the-Fly3R: Towards Robust Online 3D Reconstruction with Feed-Forward 3R Models for Large-Scale UAV Scenarios
Zhe Shen, Liyuan Lou, Yifei Yu et al. (+4 more)
While feed-forward 3D reconstruction (3R) offers efficient end-to-end modeling, its application in large-scale
UAV mapping is hindered by the prohibitive
memory cost of
Transformer attention. Current scalable streaming 3R methods assume temporally and spatially continuous inputs, rendering them ineffective for the weakly ordered or unordered image streams common in cross-strip
UAV operations. To address this, we propose On-the-Fly3R, a training-free, progressive online 3D reconstruction framework for large-scale
UAV images that upgrades various 3R backbones for large-scale
UAV scenarios. Our method enables reconstruction from unordered inputs via
retrieval-guided dynamic subset construction, which adaptively selects spatially relevant images. To further improve the robustness, a validation-rejection-retry mechanism is designed to guarantee global consistency, performing a pre-integration consistency check and automatically rejecting misaligned images and retrying with alternative subset. Finally, inspired by VSLAM, pose graph optimization based on the
retrieval loop closure is employed to mitigate camera drift. Evaluations on several
UAV benchmarks show that our On-the-Fly3R successfully scales various 3R models to over 5,000 images across square-kilometer
UAV scenes, delivering substantially superior accuracy compared to several SOTA streaming 3R methods. Code is available at https://github.com/Sh1nZzz/On_the_Fly3R
Read the full abstract ▾
All authors Zhe Shen, Liyuan Lou, Yifei Yu, Guanbo Wang, Quanjian Ji, Xin Wang, Zongqian Zhan
Computer Vision · Cross-listed from cs.RO
Zhixin Wang, Chengzheyi Yao, Leyuan Liu et al. (+2 more)
Vision-and-
Language Navigation (VLN) requires an
agent to navigate through unseen 3D environments according to natural-
language instructions. Explicit
reasoning can improve instruction understanding and semantic grounding, but autoregressive generation at every step accumulates large decision-stage latency over multi-step
navigation. We propose VerNav, a verifier-first framework for low-latency LLM-based VLN. The verifier reduces decision-stage latency by replacing per-step autoregressive generation with batched action verification, while an
entropy-based adaptive generator is invoked only for uncertain decisions to produce compact state evidence. To further improve
navigation performance with the verifier, we introduce a two-stage
alignment scheme: (i) VPO improves local action-preference
alignment in static verifier training, and (ii) step-level reinforcement fine-tuning provides dense progress rewards over multi-step
navigation rollouts during dynamic task execution. Experiments on the Room-to-Room (R2R)
benchmark show that the verifier-only decision path of VerNav achieves competitive
navigation performance among representative LLM-based VLN
agents while reducing average decision-stage LLM latency per step by more than $10\times$ compared with autoregressive methods.
Read the full abstract ▾
All authors Zhixin Wang, Chengzheyi Yao, Leyuan Liu, Xiaosong Zhang, Yongzhao Zhang
Computer Vision
A multicenter benchmark and clinically structured metric for coronary CTA report generation
Zhiyu Ye, Yue Sun, Limiao Zou et al. (+7 more)
Reliable evaluation of automated coronary computed
tomography angiography (CCTA) report generation requires standardized multicentre
benchmarks and clinically structured metrics. We established a four-centre
benchmark comprising 3,021 CCTA series from 818 patient-report pairs to evaluate seven open-source three-dimensional
vision-
language models. We developed CSM$_{\text{CCTA}}$, a clinically structured metric for CCTA report evaluation, with patient-, vessel-, and segment-level variables defined according to clinical guidelines. Report pairs are compared at the finest shared anatomical level, and the contributions of different clinical components are weighted based on expert assessments. We estimated these weights using 70 expert-scored cases and evaluated clinical
alignment in a non-overlapping set of 30 cases. CSM$_{\text{CCTA}}$ showed a strong correlation with radiologist scores (Pearson's $r=0.97$, $p<0.001$), exceeding the next-best metric, FORTE ($r=0.70$), by 0.27, and agreed with expert preferences in 115 of 160 pairwise comparisons (71.9%). Under controlled perturbations, CSM$_{\text{CCTA}}$ remained stable to clinically equivalent wording and decreased monotonically with progressive information omission. In the multicenter
benchmark, the CCTA-trained C2RG model achieved the highest CSM$_{\text{CCTA}}$ scores across all four hospitals, although its performance remained far from optimal. In contrast, CCTA-irrelevant reports accounted for up to 98.7% of the outputs from generalist models. Together, the
benchmark provides a standardized setting for model comparison, while CSM$_{\text{CCTA}}$ enables clinically structured evaluation of finding agreement and anatomical specificity. These results support a more clinically aligned and anatomically resolved approach to evaluating CCTA report generation. Code is available at https://openi.pcl.ac.cn/OpenMedIA/CSM_CCTA.
Read the full abstract ▾
All authors Zhiyu Ye, Yue Sun, Limiao Zou, Cheng Xu, Keting Xu, Tong Hu, Yue Yu, Hairong Zheng, Yining Wang, Tong Zhang
Computer Vision
HELIOS: From midnight to noon, continuous outdoor urban scene relighting
Hala Djeghim, Nathan Piasco, Luis Roldão et al. (+4 more)
Modifying the illumination of driving images is a fundamental challenge, as most
datasets are captured at specific times of day. Existing methods rely on synthetic data or paired multi-illumination supervision, which limits their generalization to the diverse and challenging conditions of real-world scenarios. To address this, we propose HELIOS, a novel image relighting approach that relies on unlabeled real-world
datasets without requiring any paired images for training. Our approach integrates albedo-based conditioning into a cycle-consistent
diffusion pipeline to prevent identity collapse and ensure accurate domain translation. To handle low-visibility nighttime conditions, we introduce a robust albedo
distillation strategy that transfers structural stability from the daytime domain. Additionally, we replace traditional text
prompts with a fine-grained control mechanism based on GPS-derived
solar angles, enabling smooth and continuous lighting manipulation across the day-night cycle. Through extensive evaluation and a user study, we demonstrate that HELIOS produces structurally consistent and realistic results in both night-to-day and day-to-night tasks, outperforming state-of-the-art methods.
Read the full abstract ▾
All authors Hala Djeghim, Nathan Piasco, Luis Roldão, Moussab Bennehar, Dzmitry Tsishkou, Céline Loscos, Désiré Sidibé
Computer Vision
Vision-Language-Guided Pseudo-Labels for Unsupervised Domain Adaptation in Semantic Segmentation for Waste Sorting
Udo Schlegel, Shubhangi, Gabriel Dax et al. (+3 more)
Obtaining labeled data for semantic
segmentation in applied settings (e.g., autonomous driving, industrial waste sorting) is expensive and often infeasible at scale. We present a cross-modal pseudo-labeling pipeline that enables unsupervised domain adaptation without any target-domain annotations. The pipeline is built on two core foundation models: SAM generates class-agnostic region proposals, and EVA-CLIP assigns semantic labels based on region-text similarity, with confidence filtering ensuring that only reliable pseudo-labels are used for self-training a
segmentation model. As an optional extension, BLIP provides
language-grounded verification for ambiguous regions, thereby improving pseudo-label quality without altering the overall pipeline. Evaluated on two domain shifts, synthetic-to-real autonomous driving and, with a primary focus, lab-to-factory industrial waste sorting, the pipeline consistently improves over source-only baselines. Our results demonstrate that pseudo-label quality, not quantity, is a decisive factor in self-training under domain shift, and that cross-modal
language grounding offers a practical path to reliable automatic annotation in deployment-critical applications.
Read the full abstract ▾
All authors Udo Schlegel, Shubhangi, Gabriel Dax, Sai Rahul Kaminwar, Florian Karl, Thomas Seidl