Computer Vision
Camera-Agnostic Pruning of 3D Gaussian Splats via Descriptor-Based Beta Evidence
Peter Fasogbon, Ugurcan Budak, Patrice Rondao Alface et al. (+1 more)
The
pruning of 3D Gaussian splats is essential for reducing their complexity to enable efficient storage, transmission, and downstream processing. However, most of the existing
pruning strategies depend on camera parameters, rendered images, or view-dependent measures. This dependency becomes a hindrance in emerging camera-agnostic exchange settings, where splats are shared directly as point-based representations (e.g., .ply). In this paper, we propose a camera-agnostic, one-shot, post-training
pruning method for 3D Gaussian splats that relies solely on attribute-derived neighbourhood descriptors. As our primary contribution, we introduce a hybrid descriptor framework that captures structural and appearance consistency directly from the splat representation. Building on these descriptors, we formulate
pruning as a statistical evidence estimation problem and introduce a Beta evidence model that quantifies per-splat reliability through a probabilistic confidence score. Experiments conducted on standardized test sequences defined by the ISO/IEC MPEG Common Test Conditions (CTC) demonstrate that our approach achieves substantial
pruning while preserving reconstruction quality, establishing a practical and generalizable alternative to existing camera-dependent
pruning strategies.
Read the full abstract ▾
All authors Peter Fasogbon, Ugurcan Budak, Patrice Rondao Alface, Hamed Rezazadegan Tavakoli
Computer Vision
SatGeo-NeRF: Geometrically Regularized NeRF for Satellite Imagery
Valentin Wagner, Sebastian Bullinger, Michael Arens et al. (+1 more)
We present SatGeo-NeRF, a geometrically regularized NeRF for satellite imagery that mitigates overfitting-induced
geometric artifacts observed in current state-of-the-art models using three model-agnostic regularizers.
Gravity-Aligned Planarity Regularization aligns depth-inferred, approximated surface normals with the
gravity axis to promote local planarity, coupling adjacent rays via a corresponding surface approximation to facilitate cross-ray gradient flow. Granularity Regularization enforces a coarse-to-fine geometry-learning scheme, and Depth-Supervised Regularization stabilizes early training for improved
geometric accuracy. On the DFC2019 satellite reconstruction
benchmark, SatGeo-NeRF improves the Mean Altitude Error by 13.9% and 11.7% relative to state-of-the-art baselines such as EO-NeRF and EO-GS.
Read the full abstract ▾
All authors Valentin Wagner, Sebastian Bullinger, Michael Arens, Rainer Stiefelhagen
Computer Vision
The Golden Subspace: Where Efficiency Meets Generalization in Continual Test-Time Adaptation
Guannan Lai, Da-Wei Zhou, Zhenguo Li et al. (+1 more)
Continual Test-Time Adaptation (CTTA) aims to enable models to adapt online to unlabeled data streams under distribution shift without accessing source data. Existing CTTA methods face an efficiency-generalization trade-off: updating more parameters improves adaptation but severely reduces online inference efficiency. An ideal solution is to achieve comparable adaptation with minimal feature updates; we call this minimal subspace the golden subspace. We prove its existence in a single-step adaptation setting and show that it coincides with the row space of the pretrained classifier. To enable online maintenance of this subspace, we introduce the sample-wise Average Gradient Outer Product (AGOP) as an efficient proxy for estimating the classifier weights without retraining. Building on these insights, we propose Guided Online Low-rank Directional adaptation (GOLD), which uses a lightweight adapter to project features onto the golden subspace and learns a compact scaling vector while the subspace is dynamically updated via AGOP. Extensive experiments on classification and
segmentation benchmarks, including autonomous-driving scenarios, demonstrate that GOLD attains superior efficiency, stability, and overall performance. Our code is available at https://github.com/AIGNLAI/GOLD.
Read the full abstract ▾
All authors Guannan Lai, Da-Wei Zhou, Zhenguo Li, Han-Jia Ye
Computer Vision
A Variational Latent-Space Framework for Uncertainty-Aware Spectral Image Emulation
Chedly Ben Azizi, Claire Guilloteau, Gilles Roussel et al. (+1 more)
Synthetic spectral image generation is essential for remote sensing simulation and mission design, yet physically based radiative transfer models (RTMs) remain computationally expensive. Existing learning-based emulators reduce this cost, but are mostly deterministic parameter-to-spectrum regressors with limited spatial modeling and uncertainty information. We formulate spectral image emulation as a parameter-conditioned latent-variable problem and propose a variational autoencoder (VAE)-based framework combining nonlinear spectral-image representations, fast inference, and per-pixel uncertainty estimates. The framework is instantiated at spectrum and spatial–spectral levels through two-step VAE pretraining and latent mapping. We evaluate it on PROSAIL-simulated hyperspectral vegetation cubes (211 bands) and real Sentinel-3 OLCI multispectral ocean-colour imagery (21 bands) against classical regression emulators and a deep CNN baseline. Results show that no single architecture is optimal: pixel-to-pixel models perform best on controlled hyperspectral simulations, whereas a fully convolutional VAE is more robust on noisy real observations with missing or contaminated pixels. VAE-based emulators also achieve high throughput for large-scale generation. On Sentinel-3, the spatial–spectral VAE provides predictive intervals closer to empirical errors than pixel-wise neural and classical emulators, although absolute calibration remains incomplete. A look-up-table-based
retrieval experiment further shows that reconstruction fidelity alone does not ensure reliable leaf area index and chlorophyll
retrieval. Emulators should therefore be evaluated in representative remote-sensing end-use scenarios, not by reconstruction metrics alone.
Read the full abstract ▾
All authors Chedly Ben Azizi, Claire Guilloteau, Gilles Roussel, Matthieu Puigt
Computer Vision
SHAPE: Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation for Medical Image Segmentation
Linkuan Zhou, Yinghao Xia, Yufei Shen et al. (+6 more)
Unsupervised Domain Adaptation (UDA) is essential for deploying medical
segmentation models across diverse clinical environments. Existing methods are fundamentally limited, suffering from semantically unaware feature
alignment that results in poor distributional fidelity and from pseudo-label validation that disregards global anatomical constraints, thus failing to prevent the formation of globally implausible structures. To address these issues, we propose SHAPE (Structure-aware Hierarchical Unsupervised Domain Adaptation with Plausibility Evaluation), a framework that reframes adaptation towards global anatomical plausibility. Built on a DINOv3 foundation, its Hierarchical Feature Modulation (HFM) module first generates features with both high fidelity and class-awareness. This shifts the core challenge to robustly validating pseudo-labels. To augment conventional pixel-level validation, we introduce Hypergraph Plausibility Estimation (HPE), which leverages hypergraphs to assess the global anatomical plausibility that standard graphs cannot capture. This is complemented by Structural Anomaly
Pruning (SAP) to purge remaining artifacts via cross-view stability. SHAPE significantly outperforms prior methods on cardiac and abdominal cross-modality
benchmarks, achieving state-of-the-art average Dice scores of 90.08% (MRI->CT) and 78.51% (CT->MRI) on cardiac data, and 87.48% (MRI->CT) and 86.89% (CT->MRI) on abdominal data. The code is available at https://github.com/BioMedIA-repo/SHAPE.
Read the full abstract ▾
All authors Linkuan Zhou, Yinghao Xia, Yufei Shen, Xiangyu Li, Wenjie Du, Cong Cong, Leyi Wei, Ran Su, Qiangguo Jin
Computer Vision
CLEAR: Context-Aware Learning with End-to-End Mask-Free Inference for Adaptive Video Subtitle Removal
Qingdong He, Chaoyi Wang, Peng Tang et al. (+2 more)
Video subtitle removal aims to distinguish text overlays from background content while preserving temporal coherence. Existing
diffusion-based methods necessitate explicit mask sequences during both training and inference phases, which restricts their practical deployment. In this paper, we present CLEAR (Context-aware Learning for End-to-end Adaptive
Video Subtitle Removal), a mask-free framework that achieves truly end-to-end inference through context-aware adaptive learning. Our two-stage design decouples prior extraction from generative refinement: Stage I learns disentangled subtitle representations via self-supervised orthogonality constraints on dual encoders, while Stage II employs LoRA-based adaptation with generation feedback for dynamic context adjustment. Notably, our method only requires 0.77% of the parameters of the base
diffusion model for training. On Chinese subtitle
benchmarks, CLEAR outperforms mask-dependent baselines by + 6.77dB PSNR and -74.7% VFID, while demonstrating superior zero-shot generalization across six
languages (English, Korean, French, Japanese, Russian, German), a performance enabled by our generation-driven feedback mechanism that ensures robust subtitle removal without ground-truth masks during inference.
Read the full abstract ▾
All authors Qingdong He, Chaoyi Wang, Peng Tang, Yifan Yang, Xiaobin Hu
Computer Vision · Cross-listed from eess.IV
HMS-VesselNet: Hierarchical Multi-Scale Attention Network with Topology-Preserving Loss for Retinal Vessel Segmentation
Amarnath R
Retinal vessel
segmentation methods based on standard overlap losses tend to miss thin peripheral vessels because these structures occupy very few pixels and have low contrast against the background. We propose HMS-VesselNet, a hierarchical multi-scale network that processes fundus images across four parallel branches at different resolutions and combines their outputs using learned fusion weights. The training loss combines Dice, binary cross-
entropy, and centerline Dice to jointly optimize area overlap and vessel continuity. Hard example mining is applied from epoch 20 onward to concentrate gradient updates on the most difficult training images. Tested on 68 images from DRIVE, STARE, and CHASE_DB1 using 5-fold cross-validation, the model achieves a mean Dice of 88.72 +/- 0.67%, Sensitivity of 90.78 +/- 1.42%, and AUC of 98.25 +/- 0.21%. In leave-one-
dataset-out experiments, AUC remains above 95% on each unseen
dataset. The largest improvement is in the recall of thin peripheral vessels, which are the structures most frequently missed by standard methods and most critical for early detection of diabetic retinopathy.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.IR
ADaFuSE: Adaptive Diffusion-generated Image and Text Fusion for Interactive Text-to-Image Retrieval
Zhuocheng Zhang, Xingwu Zhang, Kangheng Liang et al. (+3 more)
Recent advances in interactive text-to-image
retrieval (I-TIR) use
diffusion models to bridge the modality gap between the textual information need and the images to be searched, resulting in increased effectiveness. However, existing frameworks fuse multi-modal views of user feedback by simple
embedding addition. In this work, we show that this static and undifferentiated fusion indiscriminately incorporates generative noise produced by the
diffusion model, leading to performance degradation for up to 55.62% samples. We further propose ADaFuSE (Adaptive
Diffusion-Text Fusion with Semantic-aware Experts), a lightweight fusion model designed to align and calibrate multi-modal views for
diffusion-augmented I-TIR, which can be plugged into existing frameworks without modifying the backbone encoder. Specifically, we introduce a dual-branch fusion mechanism that employs an adaptive gating branch to dynamically balance modality reliability, alongside a semantic-aware mixture-of-experts branch to capture fine-grained cross-modal nuances. Via thorough evaluation over four standard I-TIR
benchmarks, ADaFuSE achieves state-of-the-art performance, surpassing DAR by up to 3.49% in Hits@10 with only a 5.29% parameter increase, while exhibiting stronger robustness to noisy and longer interactive queries. These results show that generative augmentation coupled with principled fusion provides a simple, generalizable alternative to fine-tuning for interactive
retrieval.
Read the full abstract ▾
All authors Zhuocheng Zhang, Xingwu Zhang, Kangheng Liang, Guanxuan Li, Richard Mccreadie, Zijun Long
Computer Vision
Not All Layers Are Created Equal: Adaptive LoRA Ranks for Personalized Image Generation
Donald Shenaj, Federico Errica and Antonio Carta
Low Rank Adaptation (LoRA) is the de facto fine-tuning strategy to generate personalized images from pre-trained
diffusion models. Choosing a good rank is extremely critical, since it trades off performance and
memory consumption, but today the decision is often left to the community's consensus, regardless of the personalized subject's complexity. The reason is evident: the cost of selecting a good rank for each LoRA component is combinatorial, so we opt for practical shortcuts such as fixing the same rank for all components. In this paper, we take a first step to overcome this challenge. Inspired by variational methods that learn an adaptive width of neural networks, we let the ranks of each layer freely adapt during fine-tuning on a subject. We achieve it by imposing an ordering of importance on the rank's positions, effectively encouraging the creation of higher ranks when strictly needed. Qualitatively and quantitatively, our approach, LoRA$^2$, achieves a competitive trade-off between DINO, CLIP-I, and CLIP-T across 29 subjects while requiring much less
memory and lower rank than high rank LoRA versions. Code: https://github.com/donaldssh/NotAllLayersAreCreatedEqual.
Read the full abstract ▾
Computer Vision
Deep S2P: Integrating Learning Based Stereo Matching Into the Satellite Stereo Pipeline
Elías Masquil, Thibaud Ehret, Pablo Musé et al. (+1 more)
Digital Surface Model generation from satellite imagery is a core task in Earth observation and is commonly addressed using classical stereoscopic matching
algorithms in satellite pipelines as in the Satellite Stereo Pipeline (S2P). While recent learning-based stereo matchers achieve state-of-the-art performance on standard
benchmarks, their integration into operational satellite pipelines remains challenging due to differences in viewing geometry and disparity assumptions. In this work, we integrate several modern learning-based stereo matchers, including StereoAnywhere, MonSter, Foundation Stereo, and a satellite fine-tuned variant of MonSter, into the Satellite Stereo Pipeline, adapting the rectification stage to enforce compatible disparity polarity and range. We release the corresponding code to enable reproducible use of these methods in large-scale Earth observation workflows. Experiments on satellite imagery show consistent improvements over classical cost-volume-based approaches in terms of Digital Surface Model accuracy, although commonly used metrics such as mean absolute error exhibit saturation effects. Qualitative results reveal substantially improved
geometric detail and sharper structures, highlighting the need for evaluation strategies that better reflect perceptual and structural fidelity. At the same time, performance over challenging surface types such as vegetation remains limited across all evaluated models, indicating open challenges for learning-based stereo in natural environments.
Read the full abstract ▾
All authors Elías Masquil, Thibaud Ehret, Pablo Musé, Gabriele Facciolo
Computer Vision
Thermal Topology Collapse: Universal Physical Patch Attacks on Infrared Vision Systems
Chengyin Hu, Yikun Guo, Yuxian Dong et al. (+7 more)
Infrared pedestrian detectors are increasingly deployed in all-weather perception systems, but their robustness against physical
adversarial attacks remains insufficiently understood. Existing infrared physical attacks are mostly instance-specific, requiring perturbations to be optimized for particular samples, poses, or scenes, which limits scalability under changing deployment conditions. This paper proposes Universal Physical Patch Attack (UPPA), a universal cold-patch framework for infrared pedestrian detection. UPPA is built on the observation that infrared physical perturbations should exploit smooth, low-frequency thermal structures rather than visible-light texture patterns. It represents the attack carrier as
topology-constrained Bézier Curved-Blocks, providing a compact and manufacturable
geometric parameterization, and optimizes one shared perturbation with Particle Swarm Optimization (PSO) under Thin Plate Spline (TPS) deformation and Expectation over Transformation (EOT) imaging transformations. The optimized pattern is then deployed as wearable cold patches without sample-specific re-optimization during deployment. Experiments on five infrared
datasets and nine pedestrian detectors show consistent digital attack performance, strong cross-
dataset and cross-model transferability, and a 92.59% attack success rate in real-world physical experiments. Ablation, visualization, and defense analyses show that Curved-Blocks disrupt pedestrian thermal feature aggregation and remain difficult for image-restoration-style defenses to remove. These results reveal a practical universal physical vulnerability in current infrared pedestrian detection systems and provide a
benchmark for robustness evaluation.
Read the full abstract ▾
All authors Chengyin Hu, Yikun Guo, Yuxian Dong, Qike Zhang, Kalibinuer Tiliwalidi, Yiwei Wei, Haitao Shi, Jiujiang Guo, Jiahuan Long, Xiang Chen
Computer Vision
Manifold-Aware Exploration for Reinforcement Learning in Video Generation
Mingzhe Zheng, Weijie Kong, Yue Wu et al. (+9 more)
Group Relative Policy Optimization (GRPO) methods for
video generation like FlowGRPO remain far less reliable than their counterparts for
language models and images. This gap arises because
video generation has a complex solution space, and the ODE-to-SDE conversion used for exploration can inject excess noise, lowering rollout quality and making reward estimates less reliable, which destabilizes post-training
alignment. To address this problem, we view the pre-trained model as defining a valid
video data
manifold and formulate the core problem as constraining exploration within the vicinity of this
manifold, ensuring that rollout quality is preserved and reward estimates remain reliable. We propose SAGE-GRPO (Stable
Alignment via Exploration), which applies constraints at both micro and macro levels. At the micro level, we derive a precise
manifold-aware SDE with a logarithmic
curvature correction and introduce a gradient norm equalizer to stabilize sampling and updates across timesteps. At the macro level, we use a dual trust region with a periodic moving anchor and stepwise constraints so that the trust region tracks checkpoints that are closer to the
manifold and limits long-horizon drift. We evaluate SAGE-GRPO on HunyuanVideo1.5 using the original VideoAlign as the reward model and observe consistent gains over previous methods in VQ, MQ, TA, and visual metrics (CLIPScore, PickScore), demonstrating superior performance in both reward maximization and overall
video quality. The code and visual gallery are available at https://dungeonmassster.github.io/SAGE-GRPO-Page/.
Read the full abstract ▾
All authors Mingzhe Zheng, Weijie Kong, Yue Wu, Dengyang Jiang, Yue Ma, Xuanhua He, Bin Lin, Kaixiong Gong, Zhao Zhong, Liefeng Bo, Qifeng Chen, Harry Yang
Computer Vision
Paweł Borsukiewicz, Daniele Lunghi, Melissa Tessa et al. (+2 more)
While the rapid development of facial recognition
algorithms has enabled numerous beneficial applications, their widespread deployment has raised significant concerns about the risks of mass surveillance and threats to individual
privacy. In this paper, we introduce
Adversarial Camouflage as a novel solution for protecting users'
privacy. This approach is designed to be efficient and simple to reproduce for users in the physical world. The
algorithm starts by defining a low-dimensional pattern space parameterized by color, shape, and angle. Optimized patterns, once found, are projected onto semantically valid facial regions for evaluation. Our method maximizes recognition error across multiple architectures, ensuring high cross-model transferability even against black-box systems. It significantly degrades the performance of all tested state-of-the-art face recognition models during simulations and demonstrates promising results in real-world human experiments, while revealing differences in model robustness and evidence of attack transferability across architectures.
Read the full abstract ▾
All authors Paweł Borsukiewicz, Daniele Lunghi, Melissa Tessa, Jacques Klein, Tegawendé F. Bissyandé
Computer Vision
Adaptive Video Distillation: Mitigating Oversaturation and Temporal Collapse in Few-Step Generation
Yuyang You, Yongzhi Li, Jiahui Li et al. (+3 more)
Video generation has recently emerged as a central task in the field of generative AI. However, the substantial computational cost inherent in
video synthesis makes model
distillation a critical technique for efficient deployment. Despite its significance, there is a scarcity of methods specifically designed for
video diffusion models. Prevailing approaches often directly adapt image
distillation techniques, which frequently lead to artifacts such as oversaturation, temporal inconsistency, and mode collapse. To address these challenges, we propose a novel
distillation framework tailored specifically for
video diffusion models. Its core innovations include: (1) an adaptive regression loss that dynamically adjusts spatial supervision weights to prevent artifacts arising from excessive distribution shifts; (2) a temporal regularization loss to counteract temporal collapse, promoting smooth and physically plausible sampling trajectories; and (3) an inference-time frame interpolation strategy that reduces sampling overhead while preserving perceptual quality. Extensive experiments and ablation studies on the VBench and VBench2
benchmarks demonstrate that our method achieves stable few-step
video synthesis, significantly enhancing perceptual fidelity and motion realism. It consistently outperforms existing
distillation baselines across multiple metrics.
Read the full abstract ▾
All authors Yuyang You, Yongzhi Li, Jiahui Li, Yadong Mu, Quan Chen, Peng Jiang
Computer Vision
Climate Prompting: Generating the Madden-Julian Oscillation using Video Diffusion and Low-Dimensional Conditioning
Sulian Thual, Feiyang Cai, Jingjing Wang et al. (+1 more)
Generative Deep Learning is a powerful tool for modeling of the Madden-Julian oscillation (MJO) in the tropics, yet its relationship to traditional theoretical frameworks remains poorly understood. Here we propose a
video diffusion model, trained on atmospheric reanalysis, to synthetize long MJO sequences conditioned on key low-dimensional metrics. The generated MJOs capture key features including composites, power spectra and multiscale structures including convectively coupled waves, despite some bias. We then
prompt the model to generate more tractable MJOs based on intentionally idealized low-dimensional conditionings, for example a perpetual MJO, an isolated modulation by seasons and/or the El Nino-Southern Oscillation, and so on. This enables deconstructing the underlying processes and identifying physical drivers. The present approach provides a practical framework for bridging the gap between low-dimensional MJO theory and high-resolution atmospheric complexity and will help tropical atmosphere prediction.
Read the full abstract ▾
All authors Sulian Thual, Feiyang Cai, Jingjing Wang, Feng Luo
Computer Vision
Multi-View Deformable Convolution Meets Visual Mamba for Coronary Artery Segmentation
Xiaochan Yuan and Pai Zeng
Accurate
segmentation of coronary arteries from computed
tomography angiography (CTA) images is of paramount clinical importance for the diagnosis and treatment planning of cardiovascular diseases. However, coronary artery
segmentation remains challenging due to the inherent multi-branching and slender tubular morphology of the vasculature, compounded by severe class imbalance between foreground vessels and background tissue. Conventional convolutional neural network (CNN)-based approaches struggle to capture long-range dependencies among spatially distant vascular structures, while
Vision Transformer (ViT)-based methods incur prohibitive computational overhead that hinders deployment in resource-constrained clinical settings. Motivated by the recent success of state space models (SSMs) in efficiently modeling long-range sequential dependencies with linear complexity, we propose MDSVM-UNet, a novel two-stage coronary artery
segmentation framework that synergistically integrates multidirectional snake convolution (MDSConv) with residual visual Mamba (RVM). In the encoding stage, we introduce MDSConv, a deformable convolution module that learns adaptive offsets along three orthogonal anatomical planes – sagittal, coronal, and axial – thereby enabling comprehensive multi-view feature fusion that faithfully captures the elongated and tortuous geometry of coronary vessels. In the decoding stage, we design an RVM-based upsampling decoder block that leverages selective state space mechanisms to model inter-slice long-range dependencies while preserving linear computational complexity. Furthermore, we propose a progressive two-stage
segmentation strategy: the first stage performs coarse whole-image
segmentation to guide intelligent block extraction, while the second stage conducts fine-grained block-level
segmentation to recover vascular details and suppress false positives..
Read the full abstract ▾
Computer Vision
SteelDefectX: A Multi-Form Vision-Language Dataset and Benchmark for Steel Surface Defect Analysis
Shuxian Zhao, Jie Gui, Baosheng Yu et al. (+1 more)
Steel surface defect analysis is critical for industrial quality control, yet existing
benchmarks rely primarily on label-only annotations, limiting fine-grained semantic understanding and systematic evaluation of
vision-
language models. To address this gap, we introduce SteelDefectX, a
vision-
language dataset with multi-form textual annotations for steel surface defect analysis, comprising 7,778 images across 25 defect categories. At the class level, the
dataset provides defect names, representative visual attributes, and industrial causes. At the sample level, each image is annotated with three forms of textual representations: (1) free-form natural
language descriptions, (2) structured attribute annotations, and (3) template-based sentences. These annotations provide flexible textual supervision with varying levels of expressiveness and controllability. We further establish a comprehensive
benchmark covering
vision-
language classification,
segmentation, and cross-
dataset transfer, along with additional evaluations such as
retrieval and text-guided localization. Experimental results reveal a trade-off between structure and flexibility in textual representations. Structured attributes provide more stable semantic
alignment, while natural
language descriptions improve transferability and fine-grained spatial grounding. These findings highlight the critical role of textual design in industrial
vision-
language learning. SteelDefectX provides a new
benchmark for studying semantic
alignment and generalization in industrial
vision-
language learning. The code and
dataset are available at https://github.com/Zhaosxian/SteelDefectX.
Read the full abstract ▾
All authors Shuxian Zhao, Jie Gui, Baosheng Yu, Dacheng Tao
Computer Vision
Beyond Strict Pairing: Arbitrarily Paired Training for High-Performance Infrared and Visible Image Fusion
Yanglin Deng, Tianyang Xu, Chunyang Cheng et al. (+3 more)
Infrared and visible image fusion(IVIF) combines complementary modalities while preserving natural textures and salient thermal signatures. Existing solutions predominantly rely on extensive sets of rigidly aligned image pairs for training. However, acquiring such data is often impractical due to the costly and labour-intensive
alignment process. Besides, maintaining a rigid pairing setting during training restricts the volume of cross-modal relationships, thereby limiting generalisation performance. To this end, this work challenges the necessity of Strictly Paired Training Paradigm (SPTP) by systematically investigating UnPaired and Arbitrarily Paired Training Paradigms (UPTP and APTP) for high-performance IVIF. We establish a theoretical objective of APTP, reflecting the complementary nature between UPTP and SPTP. More importantly, we develop a practical framework capable of significantly enriching cross-modal relationships even with severely limited and unaligned training data. To validate our propositions, three end-to-end lightweight baselines, alongside a set of innovative loss functions, are designed to cover three classic frameworks (CNN,
Transformer, GAN). Comprehensive experiments demonstrate that the proposed APTP and UPTP are feasible and capable of training models on a severely limited and content-inconsistent infrared and visible
dataset, achieving performance comparable to that of a
dataset 100$\times$ larger in SPTP. This finding fundamentally alleviates the cost and difficulty of data collection while enhancing model robustness from the data perspective, delivering a feasible solution for IVIF studies. The code is available at \href{https://github.com/yanglinDeng/IVIF_unpair}{\textcolor{blue}{https://github.com/yanglinDeng/IVIF\_unpair}}.
Read the full abstract ▾
All authors Yanglin Deng, Tianyang Xu, Chunyang Cheng, Hui Li, Xiao-jun Wu, Josef Kittler
Computer Vision
Ctrl-A: Control-Driven Machine Learning using Adaptive Online Data Augmentation
Jesper B. Christensen, Ciaran Bench, Spencer A. Thomas et al. (+4 more)
We introduce ControlAugment (Ctrl-A), an automated data augmentation
algorithm for image-
vision tasks, which incorporates principles from control theory for online adjustment of augmentation strength distributions during model training. Ctrl-A eliminates the need for initialization of individual augmentation strengths. Instead, augmentation strength distributions are dynamically, and individually, adapted during training based on a control-loop architecture and what we define as relative operation response curves. Using an operation-dependent update procedure provides Ctrl-A with the potential to reduce the use of augmentation operations that negatively impact model performance, alleviating the need to manually engineer augmentation policies for new image classification tasks. Experiments on the CIFAR-10, CIFAR-100, SVHN-core, and Tiny ImageNet
benchmark datasets using WideResNet-28-10 and ResNet architectures demonstrate that Ctrl-A provides competitive performance across a range of image classification
benchmarks.
Read the full abstract ▾
All authors Jesper B. Christensen, Ciaran Bench, Spencer A. Thomas, Hüsnü Aslan, David Balslev-Harder, Nadia A. S. Smith, Alessandra Manzin
Computer Vision
Clinical Graph-Mediated Distillation for Unpaired MRI-to-CFI Hypertension Prediction
Dillan Imans, Phuoc-Nguyen Bui, Duc-Tai Le et al. (+1 more)
Retinal fundus imaging enables low-cost and scalable hypertension (HTN) screening, but HTN-related retinal cues are subtle, yielding high-variance predictions.
Brain MRI provides stronger vascular and small-vessel-disease markers of HTN, yet it is expensive and rarely acquired alongside fundus images, resulting in modality-siloed
datasets with disjoint MRI and fundus cohorts. We study this unpaired MRI-fundus regime and introduce Clinical Graph-Mediated
Distillation (CGMD), a framework that transfers MRI-derived HTN knowledge to a fundus model without paired
multimodal data. CGMD leverages shared structured biomarkers as a bridge by constructing a clinical similarity kNN graph spanning both cohorts. We train an MRI teacher, propagate its representations over the graph, and impute
brain-informed representation targets for fundus patients. A fundus student is then trained with a joint objective combining HTN supervision, target
distillation, and relational
distillation. Experiments on our newly collected unpaired MRI-fundus-biomarker
dataset show that CGMD consistently improves fundus-based HTN prediction over standard
distillation and non-graph imputation baselines, with ablations confirming the importance of clinically grounded graph connectivity. Code is available at https://github.com/DillanImans/CGMD-unpaired-
distillation.
Read the full abstract ▾
All authors Dillan Imans, Phuoc-Nguyen Bui, Duc-Tai Le, Hyunseung Choo
Computer Vision
Cascade-Free Mandarin Visual Speech Recognition via Semantic-Guided Cross-Representation Alignment
Lei Yang, Yi He, Fei Wu et al. (+1 more)
Chinese mandarin visual
speech recognition (VSR) is a task that has advanced in recent years, yet still lags behind the performance on non-tonal
languages such as English. One primary challenge arises from the tonal nature of Mandarin, which limits the effectiveness of conventional sequence-to-sequence modeling approaches. To alleviate this issue, existing Chinese VSR systems commonly incorporate intermediate representations, most notably pinyin, within cascade architectures to enhance recognition accuracy. While beneficial, in these cascaded designs, the subsequent stage during inference depends on the output of the preceding stage, leading to error accumulation and increased inference latency. To address these limitations, we propose a cascade-free architecture based on multitask learning that jointly integrates multiple intermediate representations, including phoneme and viseme, to better exploit contextual information. The proposed semantic-guided local contrastive loss temporally aligns the features, enabling on-demand activation during inference, thereby providing a trade-off between inference efficiency and performance while mitigating error accumulation caused by projection and re-
embedding. Experiments conducted on publicly available
datasets demonstrate that our method achieves superior recognition performance.
Read the full abstract ▾
All authors Lei Yang, Yi He, Fei Wu, Shilin Wang
Computer Vision
Anatomical Token Uncertainty for Transformer-Guided Active MRI Acquisition
Lev Ayzenberg, Shady Abu-Hussein, Raja Giryes et al. (+1 more)
Full data acquisition in MRI is inherently slow, which limits clinical throughput and increases patient discomfort. Compressed Sensing MRI (CS-MRI) seeks to accelerate acquisition by reconstructing images from under-sampled k-space data, requiring both an optimal sampling trajectory and a high-fidelity reconstruction model. In this work, we propose a novel active sampling framework that leverages the inherent discrete structure of a pretrained medical image tokenizer and a latent
transformer. By representing anatomy through a dictionary of quantized visual
tokens, the model provides a well-defined probability distribution over the latent space. We utilize this distribution to derive a principled uncertainty measure via
token entropy, which guides the active sampling process. We introduce two strategies to exploit this latent uncertainty: (1) Latent
Entropy Selection (LES), projecting patch-wise
token entropy into the $k$-space domain to identify informative sampling lines, and (2) Gradient-based
Entropy Optimization (GEO), which identifies regions of maximum uncertainty reduction via the $k$-space gradient of a total latent
entropy loss. We evaluate our framework on the fastMRI singlecoil Knee and
Brain datasets at $\times 8$ and $\times 16$ acceleration. Our results demonstrate that our active policies outperform state-of-the-art baselines in perceptual metrics, and feature-based distances. Our code is available at https://github.com/levayz/TRUST-MRI.
Read the full abstract ▾
All authors Lev Ayzenberg, Shady Abu-Hussein, Raja Giryes, Hayit Greenspan
Computer Vision
Timing In stand-up Comedy: Text, Audio, Laughter, Kinesics (TIC-TALK): Pipeline and Database for the Multimodal Study of Comedic Timing
Yaelle Zribi, Florian Cafiero, Vincent Lépinay et al. (+1 more)
Stand-up comedy, and humor in general, are often studied through their verbal content. Yet live performance relies just as much on
embodied presence and audience feedback. We introduce TIC-TALK, a
multimodal resource with 5,400+ temporally aligned topic segments capturing
language, gesture, and audience response across 90 professionally filmed stand-up comedy specials (2015-2024). The pipeline combines BERTopic for 60 s thematic
segmentation with dense sentence
embeddings, Whisper-AT for 0.8 s laughter detection, a fine-tuned YOLOv8-cls shot classifier, and YOLOv8s-pose for raw keypoint extraction at 1 fps. Raw 17-joint skeletal coordinates are retained without prior clustering, enabling the computation of continuous kinematic signals-arm spread, kinetic energy, and trunk lean-that serve as proxies for performance dynamics. All streams are aligned by hierarchical temporal containment without resampling, and each topic segment stores its sentence-BERT
embedding for downstream similarity and clustering tasks. As a concrete use case, we study laughter dynamics across 24 thematic topics: kinetic energy negatively predicts audience laughter rate (r = -0.75, N = 24), consistent with a stillness-before-punchline pattern; personal and bodily content elicits more laughter than geopolitical themes; and shot close-up proportion correlates positively with laughter (r = +0.28), consistent with reactive montage.
Read the full abstract ▾
All authors Yaelle Zribi, Florian Cafiero, Vincent Lépinay, Chahan Vidal-Gorène
Computer Vision
Benchmarking Recurrent Event-Based Object Detection for Industrial Multi-Class Recognition on MTevent
Lokeshwaran Manohar and Moritz Roidl
Event cameras are attractive for industrial robotics because they provide high temporal resolution, high dynamic range, and reduced motion blur. However, most event-based object detection studies focus on outdoor driving scenarios or limited class settings. In this work, we
benchmark recurrent ReYOLOv8s on MTevent for industrial multi-class recognition and use a non-recurrent YOLOv8s variant as a baseline to analyze the effect of temporal
memory. On the MTevent validation split, the best scratch recurrent model (C21) reaches 0.285 mAP50, corresponding to a 9.6% relative improvement over the non-recurrent YOLOv8s baseline (0.260). Event-domain pretraining has a stronger effect: GEN1-initialized fine-tuning yields the best overall result of 0.329 mAP50 at clip length 21, and unlike scratch training, GEN1-pretrained models improve consistently with clip length. PEDRo initialization drops to 0.251, indicating that mismatched source-domain pretraining can be less effective than training from scratch. Persistent failure modes are dominated by class imbalance and human-object interaction. Overall, we position this work as a focused benchmarking and analysis study of recurrent event-based detection in industrial environments.
Read the full abstract ▾
Computer Vision
The Universal Normal Embedding
Chen Tasker, Roy Betser, Eyal Gofer et al. (+2 more)
Generative models and
vision encoders have largely advanced on separate tracks, optimized for different goals and grounded in different mathematical principles. Yet, they share a fundamental property: latent space Gaussianity. Generative models map Gaussian noise to images, while encoders map images to semantic
embeddings whose coordinates empirically behave as Gaussian. We hypothesize that both are views of a shared latent source, the Universal Normal
Embedding (UNE): an approximately Gaussian latent space from which encoder
embeddings and DDIM-inverted noise arise as noisy linear projections. To test our hypothesis, we introduce NoiseZoo, a
dataset of per-image latents comprising DDIM-inverted
diffusion noise and matching encoder representations (CLIP, DINO). On CelebA, linear probes in both spaces yield strong, aligned attribute predictions, indicating that generative noise encodes meaningful semantics along linear directions. These directions further enable faithful, controllable edits (e.g., smile, gender, age) without architectural changes, where simple orthogonalization mitigates spurious entanglements. Taken together, our results provide empirical support for the UNE hypothesis and reveal a shared Gaussian-like latent geometry that concretely links encoding and generation. Code and data are available https://rbetser.github.io/UNE/
Read the full abstract ▾
All authors Chen Tasker, Roy Betser, Eyal Gofer, Meir Yossef Levi, Guy Gilboa
Computer Vision
Online Adaptation of Visual Odometry Frontends with Image-Conditioned Reinforcement Learning
Simone Nascivera, Leonard Bauersfeld, Jeff Delaune et al. (+1 more)
Visual odometry (VO) frontends are typically tuned offline by domain experts on pre-recorded
datasets and then deployed with fixed hyperparameters. Yet a configuration that performs best on a
benchmark is not guaranteed to remain best when texture, illumination, motion blur, sensor noise, or computational conditions change at deployment. We propose a frontend that instead adapts its parameters automatically and continuously. We formulate frontend tuning as a sequential decision-making problem and introduce an image-conditioned reinforcement-learning policy that combines a lightweight
embedding of the current image with a compact set of frontend statistics. At each decision step, the policy selects the FAST detection threshold, KLT patch size, and RANSAC rejection threshold; a privileged critic provides additional context only during training. Trained on synthetic TartanAirV2 data, the policy transfers zero-shot to a
monocular-inertial OpenVINS pipeline on EuRoC, TUM-VI, and UZH-FPV. On synthetic test sequences, the learned policy improves the tracking-computation trade-off over an optimized static parameter configuration. On the three real-world
benchmarks, it reduces mean ATE by up to 8% and runtime by up to 57% relative to static parameters baseline. These results show that image-conditioned online adaptation can improve the accuracy-computation trade-off beyond a single configuration selected offline.
Read the full abstract ▾
All authors Simone Nascivera, Leonard Bauersfeld, Jeff Delaune, Davide Scaramuzza
Computer Vision
Dynamic Exposure Burst Image Restoration
Woohyeok Kim, Jaesung Rim, Daeyeon Kim et al. (+1 more)
Burst image restoration aims to reconstruct a high-quality image from burst images, which are typically captured using manually designed exposure settings. Although these exposure settings significantly influence the final restoration performance, the problem of finding optimal exposure settings has been overlooked. In this paper, we present Dynamic Exposure Burst Image Restoration (DEBIR), a novel burst image restoration pipeline that enhances restoration quality by dynamically predicting exposure times tailored to the shooting environment. In our pipeline, Burst Auto-Exposure Network (BAENet) estimates the optimal exposure time for each burst image based on a preview image, as well as motion magnitude and gain. Subsequently, a burst image restoration network reconstructs a high-quality image from burst images captured using these optimal exposure times. For training, we introduce a differentiable burst simulator and a three-stage training strategy. Our experiments demonstrate that our pipeline achieves state-of-the-art restoration quality. Furthermore, we validate the effectiveness of our approach on a real-world camera system, demonstrating its practicality.
Read the full abstract ▾
All authors Woohyeok Kim, Jaesung Rim, Daeyeon Kim, Sunghyun Cho
Computer Vision
SHARP: Spectrum-aware Highly-dynamic Adaptation for Resolution Promotion in Remote Sensing Synthesis
Bingxuan Zhao, Qing Zhou, Chuang Yang et al. (+2 more)
Text-to-image synthesis for remote sensing (RS) lacks an accessible, high-performance generative foundation, as directly training
diffusion models at large, high resolutions is computationally prohibitive. Training-free resolution promotion via Rotary Position
Embedding (RoPE) extrapolation offers an efficient alternative, but existing
algorithms apply static scaling rules tailored to natural scenes, whereas RS imagery is dominated by dense, minute instances that hinge on high-frequency structural integrity. We present a comprehensive framework for large-scale RS synthesis. First, we curate over 100{,}000 image-text pairs to train RS-FLUX, a domain-specialized generative prior. Second, we propose SHARP, a training-free extrapolation
algorithm that introduces a Rational Decay Scheduler to modulate RoPE frequencies throughout denoising: strong positional extrapolation early on enforces coherent global layouts, and progressive relaxation later recovers dense high-frequency details. Extensive experiments show that SHARP consistently achieves state-of-the-art performance across multiple promoted resolutions with negligible ($<$4%) overhead, offering an efficient and structurally faithful solution for large-scale RS generation.
Read the full abstract ▾
All authors Bingxuan Zhao, Qing Zhou, Chuang Yang, Junyu Gao, Qi Wang
Computer Vision · Cross-listed from eess.IV
Cycle Inverse-Consistent TransMorph: A Balanced Deep Learning Framework for Brain MRI Registration
Jiaqi Shang, Haojin Wu, Yinyi Lai et al. (+3 more)
Deformable image registration plays a fundamental role in medical image analysis by enabling spatial
alignment of anatomical structures across subjects. While recent deep learning-based approaches have significantly improved computational efficiency, many existing methods remain limited in capturing long-range anatomical correspondence and maintaining deformation consistency. In this work, we present a cycle inverse-consistent
transformer-based framework for deformable
brain MRI registration. The model integrates a Swin-UNet architecture with bidirectional consistency constraints, enabling the joint estimation of forward and backward deformation fields. This design allows the framework to capture both local anatomical details and global spatial relationships while improving deformation stability. We conduct a comprehensive evaluation of the proposed framework on a large multi-center
dataset consisting of 2851 T1-weighted
brain MRI scans aggregated from 13 public
datasets. Experimental results demonstrate that the proposed framework achieves strong and balanced performance across multiple quantitative evaluation metrics while maintaining stable and physically plausible deformation fields. Detailed quantitative comparisons with baseline methods, including ANTs, ICNet, and VoxelMorph, are provided in the appendix. Experimental results demonstrate that CICTM achieves consistently strong performance across multiple evaluation criteria while maintaining stable and physically plausible deformation fields. These properties make the proposed framework suitable for large-scale neuroimaging
datasets where both accuracy and deformation stability are critical.
Read the full abstract ▾
All authors Jiaqi Shang, Haojin Wu, Yinyi Lai, Zongyu Li, Chenghao Zhang, Jia Guo
Computer Vision
Let's Think with Images Efficiently! An Interleaved-Modal Chain-of-Thought Reasoning Framework with Dynamic and Precise Visual Thoughts
Xu Liu, Yongheng Zhang, Qiguang Chen et al. (+3 more)
Recently, Interleaved-modal Chain-of-Thought (ICoT)
reasoning has achieved remarkable success by leveraging both
multimodal inputs and outputs, attracting increasing
attention. While achieving promising performance, current ICoT methods still suffer from two major limitations: (1) Static Visual Thought Positioning, which statically inserts visual information at fixed steps, resulting in inefficient and inflexible
reasoning; and (2) Broken Visual Thought Representation, which involves discontinuous and semantically incoherent visual
tokens. To address these limitations, we introduce Interleaved-modal Chain-of-Thought
reasoning with Dynamic and Precise Visual Thoughts (DaP-ICoT), which incorporates two key components: (1) Dynamic Visual Thought Integration adaptively introduces visual inputs based on
reasoning needs, reducing redundancy and improving efficiency. (2) Precise Visual Thought Guidance ensures visual semantically coherent and contextually aligned representations. Experiments across multiple
benchmarks and models demonstrate that DaP-ICoT achieves state-of-the-art performance. In addition, DaP-ICoT significantly reduces the number of inserted images, leading to a 72.6% decrease in
token consumption, enabling more efficient ICoT
reasoning.
Read the full abstract ▾
All authors Xu Liu, Yongheng Zhang, Qiguang Chen, Yao Li, Sheng Wang, Libo Qin