Computer Vision
A Hybrid Approach for Closing the Sim2real Appearance Gap in Game Engine Synthetic Datasets
Stefanos Pasios
Video game engines have been an important source for generating large volumes of visual synthetic
datasets for training and evaluating computer
vision algorithms that are to be deployed in the real world. While the visual fidelity of modern game engines has been significantly improved with technologies such as ray-tracing, a notable sim2real appearance gap between the synthetic and the real-world images still remains, which limits the utilization of synthetic
datasets in real-world applications. In this letter, we investigate the ability of a state-of-the-art image generation and editing
diffusion model (FLUX.2-4B Klein) to enhance the photorealism of synthetic
datasets and compare its performance against a traditional image-to-image translation model (REGEN). Furthermore, we propose a hybrid approach that combines the strong geometry and material transformations of
diffusion-based methods with the distribution-matching capabilities of image-to-image translation techniques. Through experiments, it is demonstrated that REGEN outperforms FLUX.2-4B Klein and that by combining both FLUX.2-4B Klein and REGEN models, better visual realism can be achieved compared to using each model individually, while maintaining semantic consistency. The code is available at: https://github.com/stefanos50/Hybrid-Sim2Real
Read the full abstract ▾
Computer Vision
LabBuilder: Protocol-Grounded 3D Layout Generation for Interactable and Safe Laboratory
Jianbao Cao, Zhangrui Zhao, Bohan Feng et al. (+15 more)
Automated laboratories hold the promise of accelerating scientific discovery, yet their deployment is bottlenecked by the difficulty of designing safe and executable environments. While simulator-based design offers scalability, existing 3D scene generation methods are primarily tailored for household settings, optimizing for visual plausibility while neglecting the protocol grounding and layout-level safety constraints essential for scientific experimentation. We present LabBuilder, an end-to-end system that generates and verifies 3D laboratory layouts from concise textual specifications. It operates through three tightly coupled components: LabForge first curates a meta-
dataset of annotated assets and chemical knowledge, translating natural
language specifications into structured protocols; building on these protocols, LabGen synthesizes laboratory layouts via an iterative, constraint-aware optimization strategy; finally, LabTouchstone evaluates the resulting layouts as a unified
benchmark. Extensive experiments demonstrate that LabBuilder significantly outperforms existing state-of-the-art methods, producing laboratory environments that are realistic and valid under modeled
geometric, chemical-safety, and
navigation constraints.
Read the full abstract ▾
All authors Jianbao Cao, Zhangrui Zhao, Bohan Feng, Zixuan Hu, Rui Li, Haiyuan Wan, Chenxi Li, Jingyuan Li, Wenzhe Cai, Lei Bai, Wanli Ouyang, Lingyu Duan, Di Huang, Minting Pan, Sha Zhang, Xinzhu Ma, Shixiang Tang, Dongzhan Zhou
Computer Vision
Beyond Known Objects: A Novel Framework for Open-Set Object Detection using Negative-Aware Norm
Yuchen Zhang, Yao Lu and Johannes Betz
Open-Set Object Detection (OSOD) is crucial for autonomous driving, where perception systems must recognize and localize both known and previously unseen objects in complex, dynamic environments. While recent approaches deliver promising results, they often require retraining the detector extensively to learn objectness, which describes the likelihood that a bounding box tightly encloses a valid object, regardless of whether its category was learned during training. Deviating from existing work, we hypothesize that standard off-the-shelf detectors may already contain helpful cues for objectness, owing to their training on numerous and diverse known categories. Building on this idea, we propose NAN-SPOT, a training-light framework that does not require to retrain the base object detector and estimates objectness by leveraging a hidden layer metric called Negative-Aware Norm (NAN), requiring only minutes of training on just hundreds of images. To support comprehensive evaluation, we introduce COCO-Open, an expanded version of the existing COCO-Mixed
dataset, increasing unknown object annotations from 433 to 1853, making it the most exhaustively labeled
dataset for OSOD to the best of our knowledge. Experimental results demonstrate that NAN-SPOT achieves even better performance on unknown object detection than methods requiring heavy training, without compromising performance on known objects. This efficiency and robustness make NAN-SPOT a promising step towards open-world perception in autonomous driving.
Read the full abstract ▾
Computer Vision
Rethinking Electro-Optical Vision Foundation Models for Remote Sensing Retrieval: A Controlled Comparison with Generalist VFM
Hyobin Park, Minseok Seo and Dong-Geol Choi
Vision foundation models have attracted significant
attention for their ability to leverage large-scale unlabeled visual data. This advantage is particularly important in remote sensing, where data acquisition is costly and annotation often requires expert knowledge. Recent electro-optical
vision foundation models aim to learn domain-specific representations from remote sensing imagery, but it remains unclear whether they are more effective than strong generalist
vision foundation models under
retrieval-based evaluation. In this study, we conduct a controlled comparison between representative EO-specific and generalist
vision foundation models for remote sensing image
retrieval. Using the same
datasets,
retrieval protocol, and evaluation metric, we evaluate both in-domain performance and cross-scene generalization. Our results show that strong generalist
vision foundation models are competitive with, and in some cases outperform, existing EO-specific models. Moreover, EO-specific models often suffer from substantial degradation under cross-scene evaluation, while generalist models show more stable transfer. These findings suggest that EO pretraining alone does not guarantee stronger
retrieval-oriented remote sensing representations. We discuss the limitations of current EO-specific pretraining strategies and highlight the need for future EO
vision foundation models to better exploit the physical, spatial, spectral, and geographic characteristics of remote sensing imagery.
Read the full abstract ▾
Computer Vision
EdgeLPR: On the Deep Neural Network trade-off between Precision and Performance in LiDAR Place Recognition
Pierpaolo Serio, Hetian Wang, Zixiang Wei et al. (+4 more)
Place recognition is essential for long-term autonomous
navigation, enabling loop closure and consistent mapping. Although deep learning has improved performance, deploying such models on resource-constrained platforms remains challenging. This work explores efficient
LiDAR-based place recognition for EdgeAI by leveraging Bird's Eye View representations to enable lightweight image-based networks. We
benchmark representative architectures without aggregation heads using a unified descriptor scheme based on global pooling and linear projection, and evaluate performance under FP32, FP16, and INT8 quantization. Experiments reveal trade-offs between accuracy, robustness, and efficiency: FP16 matches FP32 with lower cost, while INT8 introduces architecture-dependent degradation. Overall, the presented results are a strong basis for future research on 'use-case'-aware quantisation of Neural Networks for Edge deployment.
Read the full abstract ▾
All authors Pierpaolo Serio, Hetian Wang, Zixiang Wei, Vincenzo Infantino, Lorenzo Gentilini, Lorenzo Pollini, Valentina Donzella
Computer Vision
WindowQuant: Mixed-Precision KV Cache Quantization based on Window-Level Similarity for VLMs Inference Optimization
Wei Tao, Xiaoyang Qu, Peiqiang Wang et al. (+4 more)
Recently,
video language models (VLMs) have been applied in various fields. However, the visual
token sequence of the VLM is too long, which may cause intolerant inference latency and GPU
memory usage. Existing methods propose mixed-precision quantization to the key-value (KV) cache in VLMs based on
token granularity, which is time-consuming in the search process and hardware inefficient during computation. This paper introduces a novel approach called WindowQuant, which employs window-adaptive mixed-precision quantization to optimize the KV cache. WindowQuant consists of two modules: window-level quantization search and window-level KV cache computation. Window-level quantization search quickly determines the optimal bit-width configuration of the KV cache windows based on the similarity scores between the corresponding visual
token windows and the text
prompt, maintaining the model accuracy. Furthermore, window-level KV cache computation reorders the KV cache windows before quantization, avoiding the hardware inefficiency caused by mixed-precision quantization in inference computation. Extensive experiments demonstrate that WindowQuant outperforms state-of-the-art VLM models and KV cache quantization methods on various
datasets.
Read the full abstract ▾
All authors Wei Tao, Xiaoyang Qu, Peiqiang Wang, Guokuan Li, Jiguang Wan, Kai Lu, Jianzong Wang
Computer Vision
SpectraDINO: Modality-Conditioned Adaptation of RGB Vision Foundation Models Across Infrared Bands
Yagiz Nalcakan, Hyeongjin Ju, Incheol Park et al. (+3 more)
Vision foundation models (VFMs) pretrained on large-scale RGB data provide strong general-purpose representations, yet infrared perception, which is essential for robotics and driving in low light and adverse weather, still relies on backbones built per sensor. Existing transfer methods bridge the visible-infrared gap for only a single band or a single RGB-IR pair, leaving no unified backbone that spans the infrared spectrum. We present SpectraDINO, a modality-conditioned adaptation of an RGB
vision foundation model that supports NIR, SWIR, and LWIR with a single shared ViT. All modality-specific behavior is conditioned on the active sensor via lightweight input stems, learned modality
embeddings, and per-modality bottleneck adapters, while the
transformer weights remain shared. We propose a staged protocol to distill the frozen DINOv2 teacher into a spectral student via cosine
distillation, symmetric contrastive
alignment, patch-level correspondence, and a queue-based neighborhood-preservation loss, and then selectively fine-tune the deeper blocks to adapt without forgetting RGB priors. Across seven detection and
segmentation benchmarks spanning all three infrared bands, this single conditioned backbone matches or surpasses modality-specific methods, improving mIoU over the best published results by 2.6 points on FMB and 1.4 on SemanticRT, and exceeding identically fine-tuned foundation-model baselines by 2.7 mAP on SWIR detection.
Read the full abstract ▾
All authors Yagiz Nalcakan, Hyeongjin Ju, Incheol Park, Sanghyeop Yeo, Youngwan Jin, Shiho Kim
Computer Vision
Fine-Tuning Impairs the Balancedness of Foundation Models in Long-tailed Personalized Federated Learning
Shihao Hou, Chikai Shang, Zhiheng Yang et al. (+5 more)
Personalized
federated learning (PFL) with foundation models has emerged as a promising paradigm enabling clients to adapt to heterogeneous data distributions. However, real-world scenarios often face the co-occurrence of non-IID data and long-tailed class distributions, presenting unique challenges that remain underexplored in PFL. In this paper, we investigate this long-tailed personalized
federated learning and observe that current methods suffer from two limitations: (i) fine-tuning degrades performance below zero-shot baselines due to the erosion of inherent class balance in foundation models; (ii) conventional personalization techniques further transfer this bias to local models through parameter or feature-level fusion. To address these challenges, we propose
Federated Learning via Gradient Purification and Residual Learning (FedPuReL), which preserves balanced knowledge in the global model while enabling unbiased personalization. Specifically, we purify local gradients using zero-shot predictions to maintain a class-balanced global model, and model personalization as residual correction atop the frozen global model. Extensive experiments demonstrate that FedPuReL consistently outperforms state-of-the-art methods, achieving superior performance on both global and personalized models across diverse long-tailed scenarios. The code is available at https://github.com/shihaohou/FedPuReL.
Read the full abstract ▾
All authors Shihao Hou, Chikai Shang, Zhiheng Yang, Jiacheng Yang, Xinyi Shang, Junlong Gao, Yiqun Zhang, Yang Lu
Computer Vision
InfiltrNet: Dual-Branch CNN-Transformer Architecture for Brain Tumor Infiltration Risk Prediction
S M Asif Hossain and Shruti Kshirsagar
Gliomas are aggressive
brain tumors that infiltrate surrounding tissue beyond the visible tumor margins observed on
Magnetic Resonance Imaging (MRI). Predicting the spatial extent of this infiltration is essential for
surgical planning and radiation therapy, yet existing deep learning approaches focus on segmenting the visible tumor rather than estimating infiltration risk in the surrounding tissue. This paper presents InfiltrNet, a novel dual-branch architecture that combines a convolutional neural network (CNN) encoder with a Swin
Transformer encoder through cross-
attention fusion modules to predict three-zone infiltration risk maps from
multimodal MRI. A label generation strategy based on distance transforms is proposed to derive reproducible infiltration risk zones from standard
Brain Tumor
Segmentation (BraTS) annotations. InfiltrNet is trained with a combined Dice-CrossEntropy and boundary-aware loss augmented by auxiliary supervision heads at intermediate decoder levels. Extensive experiments on BraTS 2020 and BraTS 2025 demonstrate that InfiltrNet outperforms five established baselines. Explainability analysis using GradCAM++ and Occlusion sensitivity confirms that the model attends to clinically relevant peritumoral regions.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.SD
Toward Fine-Grained Speech Inpainting Forensics:A Dataset, Method, and Metric for Multi-Region Tampering Localization
Tung Vu, Yen Nguyen, Hai Nguyen et al. (+2 more)
Recent advances in voice cloning and text-to-
speech synthesis have made partial
speech manipulation - where an adversary replaces a few words within an utterance to alter its meaning while preserving the speaker's identity - an increasingly realistic threat. Existing audio deepfake detection
benchmarks focus on utterance-level binary classification or single-region tampering, leaving a critical gap in detecting and localizing multiple inpainted segments whose count is unknown a priori. We address this gap with three contributions. First, we introduce MIST (Multiregion Inpainting
Speech Tampering), a large-scale
multilingual dataset spanning 6
languages with 1-3 independently inpainted word-level segments per utterance, generated via LLM-guided semantic replacement and neural voice cloning, with fake content constituting only 2-7% of each utterance. Second, we propose ISA (Iterative Segment Analysis), a backbone-agnostic framework that performs coarse-to-fine sliding-window classification with gap-tolerant region proposal and boundary refinement to recover all tampered regions without prior knowledge of their count. Third, we define SF1@tau, a segment-level F1 metric based on temporal IoU matching that jointly evaluates region count accuracy and localization precision. Zero-shot evaluation reveals that partial inpainting at word granularity remains unsolved by existing deepfake detectors: utterance-level classifiers trained on fully synthesized
speech assign near zero fake probability to MIST utterances where only 2-7% of content is manipulated. ISA consistently outperforms non-iterative baselines in this challenging setting, and the
dataset, code, and evaluation toolkit are publicly released.
Read the full abstract ▾
All authors Tung Vu, Yen Nguyen, Hai Nguyen, Cuong Pham, Cong Tran
Computer Vision · Cross-listed from cs.GR
Generative Modeling with Orbit-Space Particle Flow Matching
Sinan Wang, Jinjin He, Shenyifan Lu et al. (+3 more)
We present Orbit-Space
Geometric Probability Paths (OGPP), a particle-native flow-matching framework for generative modeling of particle systems. OGPP is motivated by two insights: (i) particles are defined up to permutation
symmetries, so anonymous indexing inflates per-index target variance and yields curved, hard-to-learn flows; and (ii) particles live in physical space, so the flow terminal velocity has physical meaning and can encode
geometric attributes, e.g., surface normals. OGPP instantiates three key components: (1) orbit-space canonicalization of the probability-path terminal endpoint, (2) particle index
embeddings for role specialization, and (3)
geometric probability paths with arc-length-aware terminal velocities that generate normals as a byproduct of the flow. We evaluate OGPP on minimal-surface
benchmarks, where it reduces metric error by up to two orders of magnitude in a single inference step; on ShapeNet, where it matches the state of the art with 5x fewer steps and reaches airplane EMD comparable to DiT-3D with 26x fewer parameters and 5x fewer steps; and on single-shape encoding, where it produces normals and reconstructions competitive with 6D generators while operating entirely in 3D.
Read the full abstract ▾
All authors Sinan Wang, Jinjin He, Shenyifan Lu, Ruicheng Wang, Greg Turk, Bo Zhu
Computer Vision
NTIRE 2026 Challenge on Efficient Low Light Image Enhancement: Methods and Results
Jiebin Yan, Chenyu Tu, Weixia Zhang et al. (+8 more)
This paper presents a comprehensive review of the NITRE 2026 Efficient Low Light Image Enhancement (E-LLIE) Challenge, highlighting the proposed solutions and final outcomes. This challenge focuses on mobile image enhancement under low-light conditions, aiming to design lightweight networks that improve enhancement quality while ensuring practical deployability under limited computational resources. A total of 207 participants registered, 27 teams submitted valid entries, and 17 teams ultimately provided valid factsheet. Based on these submissions, this paper provides a systematic evaluation of recent methods for E-LLIE, offering a comprehensive overview of state-of-the-art progress and demonstrating significant improvements in both performance and efficiency.
Read the full abstract ▾
All authors Jiebin Yan, Chenyu Tu, Weixia Zhang, Zhihua Wang, Peibei Cao, Qinghua Lin, Yuming Fang, Xiaoning Liu, Zongwei Wu, Zhuyun Zhou, Radu Timofte
Computer Vision
MultiSense-Pneumo: A Multimodal Learning Framework for Pneumonia Screening in Resource-Constrained Settings
Dineth Jayakody, Pasindu Thenahandi and Chameli Dommanige
Pneumonia remains a leading global cause of morbidity and mortality, particularly in low-resource settings where access to imaging, laboratory testing, and specialist care is limited. Clinical assessment relies on heterogeneous evidence, including symptoms, respiratory patterns, spoken descriptions, and chest imaging, making frontline screening inherently
multimodal. However, many existing computational approaches remain unimodal and focus primarily on radiographs. In this work, we present MultiSense-Pneumo, a
multimodal research prototype for pneumonia-oriented screening and triage support that integrates structured symptom descriptors, cough audio, spoken
language, and chest radiographs. The system combines deterministic symptom triage, LightGBM-based acoustic classification, domain-
adversarial radiograph analysis using ResNet-18,
transformer-based
speech recognition, and an interpretable late-fusion operator. Each modality is transformed into a normalized concern signal and aggregated into a unified screening estimate. The fusion weights are hand-specified and are treated as heuristic, interpretable parameters rather than learned or clinically optimized values. MultiSense-Pneumo is implemented with offline execution in mind on standard laptop-class hardware, but it is not presented as a deployment-validated or clinically validated diagnostic system. Experimental results demonstrate strong component-level performance of the radiograph pathway under synthetic domain shifts, while also highlighting important limitations, especially reduced abnormal-class recall for cough acoustics and the absence of paired end-to-end
multimodal patient evaluation. MultiSense-Pneumo is therefore intended as a framework and component-level prototype for screening and triage research.
Read the full abstract ▾
Computer Vision
Validity-Gated Evaluation of Multimodal Machine Unlearning
Abdullah Ahmad Khan, Hamid Laga and Ferdous Sohel
Machine unlearning aims to remove the influence of training data without retraining a model from scratch. Evaluation of
vision-
language models relies on forget accuracy (FA), retain accuracy (RA), membership inference, and similarity to a retain-only reference, but evaluation pipelines are rarely tested. We propose a validity-gated unlearning audit (VGUA) with three checks: the forget set must be memorized without duplicate leakage into the retain set, the scorer must measure correctness, and the training code must implement the intended procedure. Applying these checks reveals that random MMBench splits leak duplicate questions, a lenient scorer hides memorization, and two coding errors remove the signal while training completes. After correcting these problems, we evaluate 234 unlearned models using six methods, three learning rates, and three backbones. Each model is compared with a retain-only reference trained using low-rank adaptation (LoRA). The results depend on update strength. Among models with RA at or above 0.70, lower FA generally indicates closer agreement with the retrained FA-RA operating point. Aggressive updates overshoot this point, collapse RA, and reverse method rankings. Relative to the original model, 78% of healthy models move closer to the retrained model's scores, whereas 94% move farther from it in activation space. Membership-inference advantage does not reliably track retraining
alignment. Under option-shuffled training, unlearned models answer 25% of reordered forgotten ScienceQA questions, compared with 6% for the retain-only reference. These results show that
multimodal unlearning conclusions depend on pipeline validity, update strength, and the comparison criterion.
Read the full abstract ▾
Computer Vision
Super-Resolution of Airborne Laser Scanning Point Clouds for Forest Inventory
Jinyuan Shao, Sangyoong Park, Chunxi Zhao et al. (+2 more)
Airborne
Laser Scanning (ALS) can collect point clouds across large areas, enabling large-scale forest inventory. However, ALS point clouds are sparse and noisy, resulting in inaccurate individual-tree-level forest inventory, such as stem localization and tree size estimation. To overcome this problem, we propose a deep learning model, 3D Forest Super Resolution (3DFSR), to simultaneously improve point density and reduce noise for ALS forest point cloud. 3DFSR is a voxel-based CNN with a U-Net architecture. The proposed 3DFSR is evaluated on ALS point clouds collected in both temperate forests in the U.S. and boreal forests in Germany. Experimental results demonstrate that 3DFSR can generate finer point clouds of tree structure than other state-of-the-art point cloud super-resolution
algorithms, achieving 0.249 m Chamfer Distance and 2.711 m Hausdorff Distance. Furthermore, to verify the effectiveness of 3DFSR point clouds in forest inventory, we conduct stem detection, DBH measurements, and stem reconstruction on both original ALS point clouds and 3DFSR enhanced point clouds. We find that stem detection and reconstruction
algorithms developed for TLS/MLS point clouds can directly work on our 3DFSR point clouds, and DBH can be derived with circle-fitting method. F1 score of stem detection is improved from 0.71 on original ALS point clouds to 0.97 on 3DFSR point clouds; DBH estimation improves from 13.45 cm RMSE using allometric equations to 6.43 cm using circle fitting; comparing to stems reconstruction from MLS point clouds, stem reconstructed from 3DFSR point clouds has 0.170 m of Chamfer Distance and 0.377 m of Hausdorff Distance, and 0.95 R2 volume estimation. Finally, we find that the proposed 3DFSR is applicable to process point densities from 10 to 1700 points/m2; it also can be generalized across data collected from different
LiDAR platforms without transfer learning.
Read the full abstract ▾
All authors Jinyuan Shao, Sangyoong Park, Chunxi Zhao, Ayman Habib, Songlin Fei
Computer Vision
SlimDiffSR: Toward Lightweight and Efficient Remote Sensing Image Super-Resolution via Diffusion Model Distillation
Ce Wang, Zhenyu Hu and Wanjie Sun
Diffusion models have recently achieved remarkable performance in image super-resolution (SR), but their high computational cost limits practical deployment in remote sensing applications. To address this issue, we propose SlimDiffSR, a lightweight and efficient
diffusion-based framework for real-world remote sensing image super-resolution. Unlike existing single-step
diffusion methods that rely on fixed timesteps, we first introduce an uncertainty-guided timestep assignment strategy to construct a stronger single-step teacher model, where reconstruction difficulty is explicitly linked to
diffusion timesteps, enabling adaptive generative strength. Building upon this teacher, we further present a structured
pruning strategy tailored to remote sensing imagery, which systematically removes redundant semantic modules and replaces standard operations with lightweight designs, including frequency-separable convolution, direction-separable convolution, and a query-driven global aggregation module. These components explicitly exploit the unique characteristics of remote sensing data, such as sparse high-frequency details, strong directional patterns, and long-range spatial dependencies. To enhance knowledge transfer, we incorporate Maximum Mean Discrepancy (MMD) into the
distillation process to align feature distributions between the teacher and student models. Extensive experiments on multiple remote sensing
benchmarks demonstrate that SlimDiffSR achieves a favorable balance between efficiency and reconstruction quality. In particular, it attains up to $200\times$ inference acceleration and a $20\times$ reduction in model parameters compared with multi-step
diffusion models, while achieving competitive perceptual quality and clearly outperforming existing lightweight
diffusion baselines in efficiency. The code is available at: https://github.com/wwangcece/SlimDiffSR.
Read the full abstract ▾
Computer Vision
RAFNet: Region-Aware Fusion Network for Pansharpening
Jianing Zhang, Zijian Zhou and Kai Sun
Pansharpening aims to generate high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) and high-resolution panchromatic (PAN) images. Although deep learning has advanced this field, mainstream frequency-based methods relying on standard scaled dot-product
attention suffer from quadratic computational complexity and fail to exploit the inherent regional sparsity of remote sensing imagery. Furthermore, existing spatial enhancement strategies typically employ static convolution kernels, which struggle to adapt to the complex frequency and regional variations of PAN and MS images. To address these bottlenecks, we propose a Region-Aware Fusion (RAFNet) Network that synergistically models spatial and frequency information. Specifically, we design a Spatial Adaptive Refinement (SAR) module that leverages the discrete wavelet transform (DWT) for directional frequency separation and K-means clustering for regional partitioning, which enables the dynamic construction of region-specific adaptive convolution kernels, achieving spatially and frequency-adaptive feature enhancement. Moreover, we introduce a Clustered Frequency Aggregation (CFA) module based on a sparse
attention mechanism guided by the semantic clusters, which executes a region-aware sparse
attention strategy that drastically reduces computational redundancy while ensuring high-quality frequency feature extraction. In addition we integrated these modules into a progressive, multi-level spatial-frequency network architecture to facilitate robust interaction and accurate image reconstruction. Extensive experiments on multiple
benchmark datasets demonstrate that the proposed RAFNet significantly outperforms state-of-the-art pansharpening methods in both reduced- and full-resolution assessments. The code is available at https://github.com/PatrickNod/RAFNet.
Read the full abstract ▾
Computer Vision
Heterogeneous Model Fusion for Privacy-Aware Multi-Camera Surveillance via Synthetic Domain Adaptation
Peggy Joy Lu, Wei-Yu Chen, Yao-Tsung Huang et al. (+1 more)
We propose HeroCrystal, a novel
privacy-preserving framework for multi-camera domain-adaptive object detection, addressing challenges such as data
privacy, class imbalance, and heterogeneous architectures. Our framework consists of three key stages. In the Generated Stage, we introduce a one-shot, target-aware
diffusion-based generation module that learns visual style from a single target-domain image while leveraging
prompt-based control to synthesize specific object instances. Unlike conventional style transfer-based methods that require large target
datasets and ignore semantic-level discrepancies, our approach enables
privacy-preserving augmentation to reduce ethical concerns, and introduces controllable rare object generation to mitigate long-tailed category degradation. In the
Federated Stage, we employ probabilistic Faster R-CNN on the client side to improve localization accuracy, and a dynamic model contrastive strategy to suppress domain-specific bias. The server side performs model fusion across heterogeneous architectures without accessing raw data. Finally, in the Distilled Stage, we propose an inconsistent categories integration
algorithm to resolve label inconsistency and architecture heterogeneity across clients. Extensive experiments on multiple cross-domain detection
benchmarks demonstrate that our method outperforms existing multi-source domain adaptation and
federated learning baselines under multi-class,
privacy-preserving settings. Our method improves mAP by +2.1% over prior
privacy-preserving approaches and achieves a new state-of-the-art mAP of 33.4%, highlighting the effectiveness of HeroCrystal in enabling practical multi-camera AI surveillance systems. The source code is publicly available at https://github.com/ccuvislab/HeroCrystal.
Read the full abstract ▾
All authors Peggy Joy Lu, Wei-Yu Chen, Yao-Tsung Huang, Vincent Shin-Mu Tseng
Computer Vision · Cross-listed from cs.LG
Manifold-Aligned Guided Integrated Gradients for Reliable Feature Attribution
Soyeon Kim, Seongwoo Lim, Kyowoon Lee et al. (+1 more)
Feature attribution is central to diagnosing and trusting deep neural networks, and Integrated Gradients (IG) is widely used due to its axiomatic properties. However, IG can yield unreliable explanations when the integration path between a baseline and the input passes through regions with noisy gradients. While Guided Integrated Gradients reduces this sensitivity by adaptively updating low-gradient-magnitude features, input-space guidance still produces intermediate inputs that deviate from the data
manifold. To address this limitation, we propose
Manifold-Aligned Guided Integrated Gradients (MA-GIG), which constructs attribution paths in the latent space of a pre-trained variational autoencoder. By decoding intermediate latent states, MA-GIG biases the path toward the learned generative
manifold and reduces exposure to implausible input-space regions. Through qualitative and quantitative evaluations, we demonstrate that MA-GIG produces faithful explanations by aggregating gradients on path features proximal to the input. Consequently, our method reduces off-
manifold noise and outperforms prior path-based attribution methods across multiple
datasets and classifiers. Our code is available at https://github.com/leekwoon/ma-gig/.
Read the full abstract ▾
All authors Soyeon Kim, Seongwoo Lim, Kyowoon Lee, Jaesik Choi
Computer Vision
Cross-Polarization Fusion of VV AND VH SAR Observations for Improved Flood Mapping
Jagrati Talreja, Tewodros Syum Gebre and Leila Hashemi Beni
Synthetic Aperture Radar (SAR) imagery is widely used for flood monitoring due to its all-weather and day-night imaging capability. However, flood mapping using single-polarization SAR data remains challenging in complex environments where surface and volume scattering coexist. In this paper, we investigate the effectiveness of cross-polarization fusion of VV and VH SAR observations for improved flood mapping. A deep learning-based
segmentation framework is employed to jointly exploit complementary information from VV and VH polarizations. To ensure a fair evaluation, three configurations are compared under identical training conditions: VV only, VH only, and fused VV-VH input. Performance is assessed using standard flood mapping metrics, including Intersection over Union (IoU) and F1-score, along with qualitative visual analysis. Experimental results demonstrate that VV-VH fusion consistently outperforms single-polarization models, particularly in vegetated and heterogeneous flood regions, leading to more accurate flood boundary delineation. The findings highlight the importance of cross-polarization SAR fusion for enhancing the reliability of SAR-based flood mapping in disaster monitoring applications.
Read the full abstract ▾
Computer Vision
SpecEdit: Training-Free Acceleration for Diffusion based Image Editing via Semantic Locking
Zhengan Yan, Shikang Zheng, Haoran Qin et al. (+9 more)
Diffusion-based image editing offers strong semantic controllability, but remains computationally expensive due to iterative high-resolution denoising over all spatial
tokens. Dynamic-resolution sampling reduces this cost by performing early steps at reduced resolution. However, existing approaches prioritize upsampling using low-level heuristics such as edge detection or channel variance, which are weakly aligned with editing semantics and may lead to structural inconsistency. Moreover, spatial regions are often upsampled without verifying whether semantic modification is actually required, resulting in redundant high-resolution computation and accumulated errors. Therefore, we propose SpecEdit, a training-free dynamic-resolution framework tailored for
diffusion-based image editing. SpecEdit follows a draft-and-verify scheme: a low-resolution draft first estimates the semantic outcome, after which
token-level discrepancies are used to identify edit-relevant
tokens for high-resolution denoising, while the remaining
tokens stay at a coarse resolution. Experiments on Qwen-Image-Edit and FLUX.1-Kontext-dev demonstrate up to 10x and 7x acceleration, while maintaining strong quality. SpecEdit is complementary to step
distillation and other acceleration techniques, achieving up to 13x speedup when combined with existing methods. Our code is in supplementary material and will be released on GitHub.
Read the full abstract ▾
All authors Zhengan Yan, Shikang Zheng, Haoran Qin, Xiaobing Tu, Yinggui Wang, Jiacheng Liu, Jiaxuan Ren, Yuqi Lin, Peiliang Cai, Jinkui Ren, Xiantao Zhang, Linfeng Zhang
Computer Vision
FLoRA: Fusion-Latent for Optical Reconstruction and Flood Area Segmentation via Cross-Modal Multi-Task Distillation Network
Jagrati Talreja, Tewodros Syum Gebre and Leila Hashemi-Beni
Accurate flood water mapping is critical for disaster management, yet current methods struggle to fully exploit the potential of spaceborne imagery. Optical data offers high interpretability but is limited by environmental conditions, whereas SAR provides reliable all-weather coverage with reduced visual interpretability. FLoRA (Fusion Latent for Optical Reconstruction and Area
Segmentation) is a cross-modal multi-task framework that jointly reconstructs high-fidelity optical imagery and segments flood water regions from Sentinel 1 SAR by fusing the complementary strengths of optical and SAR data. During training, a lightweight optical teacher (driven by RGB and NDVI priors) provides pyramidal features that guide SAR representations into a fusion latent space via multiscale windowed cross
attention and FiLM conditioning, with gated residuals preventing overcorrection. This design enables multi-task learning across two complementary objectives: (a) SAR-to-optical translation for fine-grained RGB reconstruction and (b) flood water region
segmentation for hydrologic interpretation. The dual decoders are optimized using Charbonnier SSIM for structural fidelity, edge FFT magnitude losses for spectral realism, and Dice BCE hydrology-aware edge
alignment for precise flood water delineation. A feature
distillation constraint further aligns fused SAR features with the optical teacher's
manifold. Evaluations on SEN1FLOODS11, DEEPFLOOD, and SEN12MS demonstrate that FLoRA surpasses fusion baselines in PSNR, SSIM, and LPIPS, demonstrating that multi-modal fusion within a teacher-guided latent space yields semantically faithful and physically consistent flood-water intelligence from spaceborne observations.
Read the full abstract ▾
Computer Vision
Video Generation with Predictive Latents
Yian Zhao, Feng Wang, Qiushan Guo et al. (+4 more)
Video Variational Autoencoder (VAE) enables latent
video generative modeling by mapping the visual world into compact spatiotemporal latent spaces, improving training efficiency and stability. While existing
video VAEs achieve commendable reconstruction quality, continued optimization of reconstruction does not necessarily translate into improved generative performance. How to enhance the diffusability of
video latents remains a critical and unresolved challenge. In this work, inspired by principles of predictive world modeling, we investigate the potential of predictive learning to improve the
video generative modeling. To this end, we introduce a simple and effective predictive reconstruction objective that unifies predictive learning with
video reconstruction. Specifically, we randomly discard future frames and encode only partial past observations, while training the decoder to reconstruct the observed frames and predict future ones simultaneously. This design encourages the latent space to encode temporally predictive structures and build a more coherent understanding of
video dynamics, thereby improving generation quality. Our model, termed Predictive
Video VAE (PV-VAE), achieves superior performance on
video generation, with 52% faster convergence and a 34.42 FVD improvement over the Wan2.2 VAE on UCF101. Furthermore, comprehensive analyses demonstrate that PV-VAE not only exhibits favorable scalability, with generative performance improving alongside VAE training, but also yields consistent gains in downstream
video understanding, underscoring a latent space that effectively captures temporal coherence and motion priors.
Read the full abstract ▾
All authors Yian Zhao, Feng Wang, Qiushan Guo, Chang Liu, Xiangyang Ji, Jian Zhang, Jie Chen
Computer Vision
From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
Le Zhang, Jihan Yang, Soundarya Krishnan et al. (+11 more)
Human-level agentic intelligence extends beyond low-level
geometric perception, evolving from recognizing where things are to understanding what they are for. While existing
benchmarks effectively evaluate the
geometric perception capabilities of
multimodal large
language models (MLLMs), they fall short of probing the higher-order cognitive abilities required for grounded intelligence. To address this gap, we introduce the Spatial-Functional Intelligence
Benchmark (SFI-Bench), a
video-based
benchmark with over 1,500 expert-annotated questions derived from diverse
egocentric indoor
video scans. SFI-Bench systematically evaluates two complementary dimensions of advanced
reasoning: (1) Structured Spatial
Reasoning, which requires understanding complex layouts and forming coherent spatial representations, and (2) Functional
Reasoning, which involves inferring object affordances and their context-dependent utility. The
benchmark includes tasks such as conditional counting, multi-hop relational
reasoning, functional pairing, and knowledge-grounded troubleshooting, directly challenging models to integrate perception,
memory, and inference. Our experiments reveal that current MLLMs consistently struggle to combine spatial
memory with functional
reasoning and external knowledge, highlighting a critical bottleneck in achieving grounded intelligence. SFI-Bench therefore provides a diagnostic tool for measuring progress toward more cognitively capable and truly grounded
multimodal agents.
Read the full abstract ▾
All authors Le Zhang, Jihan Yang, Soundarya Krishnan, Jimit Majmudar, Xiou Ge, Prasoon Puri, Prathamesh Nandkishor Saraf, Shruti Bhargava, Dhivya Piraviperumal, Yinan Ling, Cindy Pan, Hong Yu, Aishwarya Agrawal, Bo-Hsiang Tseng
Computer Vision
Zhuoyang Lyu, Yiyang Zhang, Tongxin Wang et al. (+1 more)
Ultrasound foundation models have achieved strong performance on structured prediction tasks but remain exclusively
vision-based, limiting zero-shot and few-shot transfer to novel tasks where task-specific annotation is scarce. We address this gap with EchoCare-CLIP, a CLIP-style dual-encoder contrastive framework that aligns ultrasound images with clinical text in a shared
embedding space. We curate a multi-organ corpus of over 16K image-text pairs spanning breast, liver, lung, and thyroid, with over 78% of captions derived from expert-annotated reports, and complement the remainder with a three-tier template-based and LLM-based caption generation pipeline. We evaluate model configurations spanning two text encoder families (CLIP, BioClinicalBERT) and two caption strategies (template-based, LLM-generated) against OpenAI CLIP and BiomedCLIP baselines. Our trained models consistently improve cross-modal
alignment over baselines, with the best configuration achieving a paired
alignment score of 0.682. However, stronger
alignment does not guarantee better downstream performance: CLIP-based variants with partial fine-tuning achieve the strongest zero-shot classification on external held-out
datasets (0.709 on BUSI; 0.626 on AULI), while full end-to-end fine-tuning degrades transfer due to overfitting. On linear probing and few-shot adaptation, model rankings are
dataset-dependent, reflecting a trade-off between domain adaptation and representational generalizability. We further show that template-based captions match or outperform LLM-generated captions, suggesting lexical diversity is not a proxy for caption quality. Taken together, our results demonstrate that ultrasound
vision-
language alignment is achievable from public data alone, but robust clinical transfer requires careful balancing of domain adaptation, encoder capacity, and caption supervision quality.
Read the full abstract ▾
All authors Zhuoyang Lyu, Yiyang Zhang, Tongxin Wang, Ruirui Lan