Computer Vision
Lean Learning Beyond Clouds: Efficient Discrepancy-Conditioned Optical-SAR Fusion for Semantic Segmentation
Chenxing Meng, Wuzhou Quan, Yingjie Cai et al. (+3 more)
Cloud occlusion severely degrades the semantic integrity of optical remote sensing imagery. While incorporating Synthetic Aperture Radar (SAR) provides complementary observations, achieving efficient global modeling and reliable cross-modal fusion under cloud interference remains challenging. Existing methods rely on dense global
attention to capture long-range dependencies, yet such aggregation indiscriminately propagates cloud-induced noise. Improving robustness typically entails enlarging model capacity, which further increases computational overhead. Given the large-scale and high-resolution nature of remote sensing applications, such computational demands hinder practical deployment, leading to an efficiency-reliability trade-off. To address this dilemma, we propose EDC, an efficiency-oriented and discrepancy-conditioned optical-SAR semantic
segmentation framework. A tri-stream encoder with Carrier
Tokens enables compact global context modeling with reduced complexity. To prevent noise contamination, we introduce a Discrepancy-Conditioned Hybrid Fusion (DCHF) mechanism that selectively suppresses unreliable regions during global aggregation. In addition, an auxiliary cloud removal branch with teacher-guided
distillation enhances semantic consistency under occlusion. Extensive experiments demonstrate that EDC achieves superior accuracy and efficiency, improving mIoU by 0.56% and 0.88% on M3M-CR and WHU-OPT-SAR, respectively, while reducing the number of parameters by 46.7% and accelerating inference by 1.98$\times$. Our implementation is available at https://github.com/mengcx0209/EDC.
Read the full abstract ▾
All authors Chenxing Meng, Wuzhou Quan, Yingjie Cai, Liqun Cao, Liyan Zhang, Mingqiang Wei
Computer Vision
Predictive Regularization Against Visual Representation Degradation in Multimodal Large Language Models
Enguang Wang, Qiang Wang, Yuanchen Wu et al. (+5 more)
While
Multimodal Large
Language Models (MLLMs) excel at
vision-
language tasks, the cost of their
language-driven training on internal visual foundational competence remains unclear. In this paper, we conduct a detailed diagnostic analysis to unveil a pervasive issue: visual representation degradation in MLLMs. Specifically, we find that compared to the initial visual features, the visual representation in the middle layers of LLM exhibits both a degradation in global function and patch structure. We attribute this phenomenon to a visual sacrifice driven by the singular text-generation objective, where the model compromises its visual fidelity to optimize for answer generation. We argue that a robust MLLM requires both strong cross-modal
reasoning and core visual competence, and propose Predictive Regularization (PRe) to force degraded intermediate features to predict initial visual features, thereby maintaining the inherent visual attributes of the MLLM's internal representations. Extensive experiments confirm that mitigating this visual degradation effectively boosts
vision-
language performance, underscoring the critical importance of fostering robust internal visual representations within MLLMs for comprehensive
multimodal understanding.
Read the full abstract ▾
All authors Enguang Wang, Qiang Wang, Yuanchen Wu, Ke Yan, Xinbin Yuan, Shouhong Ding, Xialei Liu, Ming-Ming Cheng
Computer Vision
Less is More in Semantic Space: Intrinsic Decoupling via Clifford-M for Fundus Image Classification
Yifeng Zheng
Multi-label fundus diagnosis requires features that capture both fine-grained lesions and large-scale retinal structure. Many multi-scale medical
vision models address this challenge through explicit frequency decomposition, but our ablation studies show that such heuristics provide limited benefit in this setting: replacing the proposed simple dual-resolution stem with Octave Convolution increased parameters by 35% and computation by a 2.23-fold increase in computation; without improving mean accuracy, while a fixed wavelet-based variant performed substantially worse. Motivated by these findings, we propose Clifford-M, a lightweight backbone that replaces both feed-forward expansion and frequency-splitting modules with sparse
geometric interaction. The model is built on a Clifford-style rolling product that jointly captures
alignment and structural variation with linear complexity, enabling efficient cross-scale fusion and self-refinement in a compact dual-resolution architecture. Without pre-training, Clifford-M achieves a mean AUC-ROC of 0.8142 and a mean macro-F1 (optimal threshold) of 0.5481 on ODIR-5K using only 0.85M parameters, outperforming substantially larger mid-scale CNN baselines under the same training protocol. When evaluated on RFMiD without fine-tuning, it attains 0.7425 +/- 0.0198 macro AUC and 0.7610 +/- 0.0344 micro AUC, indicating reasonable robustness to cross-
dataset shift. These results suggest that competitive and efficient fundus diagnosis can be achieved without explicit frequency engineering, provided that the core feature interaction is designed to capture multi-scale structure directly.
Read the full abstract ▾
Computer Vision
Qunchao Jin, Yiliao Song and Qi Wu
Vision-
Language Navigation (VLN) systems are fundamentally constrained by partial observability, as an
agent can only accumulate knowledge from locations it has personally visited. As multiple
robots increasingly coexist in shared environments, a natural question arises: can
agents navigating the same space benefit from each other's observations? In this work, we introduce Co-VLN, a minimalist, model-agnostic framework for systematically investigating whether and how peer observations from concurrently navigating
agents can benefit VLN. When independently navigating
agents identify common traversed locations, they exchange structured perceptual
memory, effectively expanding each
agent's receptive field at no additional exploration cost. We validate our framework on the R2R
benchmark under two representative paradigms (the learning-based DUET and the zero-shot MapGPT), and conduct extensive analytical experiments to systematically reveal the underlying dynamics of peer observation sharing in VLN. Results demonstrate that
vision-sharing enabled model yields substantial performance improvements across both paradigms, establishing a strong foundation for future research in collaborative
embodied navigation.
Read the full abstract ▾
Computer Vision
ME-IQA: Memory-Enhanced Image Quality Assessment via Re-Ranking
Kanglong Fan, Tianhe Wu, Wen Wen et al. (+6 more)
Reasoning-induced
vision-
language models (VLMs) advance image quality assessment (IQA) with textual
reasoning, yet their scalar scores often lack sensitivity and collapse to a few values, so-called discrete collapse. We introduce ME-IQA, a plug-and-play, test-time
memory-enhanced re-ranking framework. It (i) builds a
memory bank and retrieves semantically and perceptually aligned neighbors using
reasoning summaries, (ii) reframes the VLM as a probabilistic comparator to obtain pairwise preference probabilities and fuse this ordinal evidence with the initial score under Thurstone's Case V model, and (iii) performs gated reflection and consolidates
memory to improve future decisions. This yields denser, distortion-sensitive predictions and mitigates discrete collapse. Experiments across multiple IQA
benchmarks show consistent gains over strong
reasoning-induced VLM baselines, existing non-
reasoning IQA methods, and test-time scaling alternatives.
Read the full abstract ▾
All authors Kanglong Fan, Tianhe Wu, Wen Wen, Jianzhao Liu, Le Yang, Yabin Zhang, Yiting Liao, Junlin Li, Li Zhang
Computer Vision
MEMO: Human-like Crisp Edge Detection Using Masked Edge Prediction
Jiaxin Cheng, Yue Wu and Yicong Zhou
Learning-based edge detection models trained with cross-
entropy loss often suffer from thick edge predictions, which deviate from the crisp, single-pixel annotations typically provided by humans. While previous approaches to achieving crisp edges have focused on designing specialized loss functions or modifying network architectures, we show that a carefully designed training and inference strategy alone is sufficient to achieve human-like edge quality. In this work, we introduce the Masked Edge Prediction MOdel (MEMO), which produces both accurate and crisp edges using only cross-
entropy loss. We first construct a large-scale synthetic edge
dataset to pre-train MEMO, enhancing its generalization ability. Subsequent fine-tuning on downstream
datasets requires only a lightweight module comprising 1.2% additional parameters. During training, MEMO learns to predict edges under varying ratios of input masking. A key insight guiding our inference is that thick edge predictions typically exhibit a confidence gradient: high in the center and lower toward the boundaries. Leveraging this, we propose a novel progressive prediction strategy that sequentially finalizes edge predictions in order of prediction confidence, resulting in thinner and more precise contours. Our method achieves visually appealing, post-processing-free, human-like edge maps and outperforms prior methods on crispness-aware evaluations.
Read the full abstract ▾
Computer Vision
PiLoT: Neural Pixel-to-3D Registration for UAV-based Ego and Target Geo-localization
Xiaoya Cheng, Long Wang, Yan Liu et al. (+5 more)
We present PiLoT, a unified framework that tackles
UAV-based ego and target geo-localization. Conventional approaches rely on decoupled pipelines that fuse GNSS and Visual-Inertial Odometry (VIO) for ego-pose estimation, and active sensors like
laser rangefinders for target localization. However, these methods are susceptible to failure in GNSS-denied environments and incur substantial hardware costs and complexity. PiLoT breaks this paradigm by directly registering live
video stream against a geo-referenced 3D map. To achieve robust, accurate, and real-time performance, we introduce three key contributions: 1) a Dual-Thread Engine that decouples map rendering from core localization thread, ensuring both low latency while maintaining drift-free accuracy; 2) a large-scale synthetic
dataset with precise
geometric annotations (camera pose, depth maps). This
dataset enables the training of a lightweight network that generalizes in a zero-shot manner from simulation to real data; and 3) a Joint Neural-Guided
Stochastic-Gradient Optimizer (JNGO) that achieves robust convergence even under aggressive motion. Evaluations on a comprehensive set of public and newly collected
benchmarks show that PiLoT outperforms state-of-the-art methods while running over 25 FPS on NVIDIA Jetson Orin platform. Our code and
dataset is available at: https://github.com/Choyaa/PiLoT.
Read the full abstract ▾
All authors Xiaoya Cheng, Long Wang, Yan Liu, Xinyi Liu, Hanlin Tan, Yu Liu, Maojun Zhang, Shen Yan
Computer Vision · Cross-listed from cs.LG
OmniPatch: A Universal Adversarial Patch for ViT-CNN Cross-Architecture Transfer in Semantic Segmentation
Aarush Aggarwal, Akshat Tomar, Amritanshu Tiwari et al. (+1 more)
Robust semantic
segmentation is crucial for safe autonomous driving, yet deployed models remain vulnerable to black-box
adversarial attacks when target weights are unknown. Most existing approaches either craft image-wide perturbations or optimize patches for a single architecture, which limits their practicality and transferability. We introduce OmniPatch, a training framework for learning a universal
adversarial patch that generalizes across images and both ViT and CNN architectures without requiring access to target model parameters.
Read the full abstract ▾
All authors Aarush Aggarwal, Akshat Tomar, Amritanshu Tiwari, Sargam Goyal
Computer Vision
Memory-Efficient Fine-Tuning Diffusion Transformers via Dynamic Patch Sampling and Block Skipping
Sunghyun Park, Jeongho Kim, Hyoungwoo Park et al. (+6 more)
Diffusion Transformers (DiTs) have significantly enhanced text-to-image (T2I) generation quality, enabling high-quality personalized content creation. However, fine-tuning these models requires substantial computational complexity and
memory, limiting practical deployment under resource constraints. To tackle these challenges, we propose a
memory-efficient fine-tuning framework called DiT-BlockSkip, integrating timestep-aware dynamic patch sampling and block skipping by precomputing residual features. Our dynamic patch sampling strategy adjusts patch sizes based on the
diffusion timestep, then resizes the cropped patches to a fixed lower resolution. This approach reduces forward & backward
memory usage while allowing the model to capture global structures at higher timesteps and fine-grained details at lower timesteps. The block skipping mechanism selectively fine-tunes essential
transformer blocks and precomputes residual features for the skipped blocks, significantly reducing training
memory. To identify vital blocks for personalization, we introduce a block selection strategy based on cross-
attention masking. Evaluations demonstrate that our approach achieves competitive personalization performance qualitatively and quantitatively, while reducing
memory usage substantially, moving toward on-device feasibility (e.g., smartphones, IoT devices) for large-scale
diffusion transformers.
Read the full abstract ▾
All authors Sunghyun Park, Jeongho Kim, Hyoungwoo Park, Debasmit Das, Sungrack Yun, Munawar Hayat, Jaegul Choo, Fatih Porikli, Seokeon Choi
Computer Vision
Smart Operation Theatre: An AI-based System for Surgical Gauze Counting
Saraf Krish, Cai Yiyu and Huang Li Hui
During surgeries, there is a risk of medical gauzes being left inside patients' bodies, leading to "Gossypiboma" in patients and can cause serious complications in patients and also lead to legal problems for hospitals from malpractice lawsuits and regulatory penalties. Diagnosis depends on imaging methods such as X-rays or CT scans, and the usual treatment involves
surgical excision. Prevention methods, such as manual counts and RFID-integrated gauzes, aim to minimize gossypiboma risks. However, manual tallying of 100s of gauzes by nurses is time-consuming and diverts resources from patient care. In partnership with Singapore General Hospital (SGH) we have developed a new prevention method, an AI-based system for gauze counting in
surgical settings. Utilizing real-time
video surveillance and object recognition technology powered by YOLOv5, a Deep Learning model was designed to monitor gauzes on two designated trays labelled "In" and "Out". Gauzes are tracked from the "In" tray, prior to their use in the patient's body & in the "Out" tray post-use, ensuring accurate counting and verifying that no gauze remains inside the patient at the end of the surgery. We have trained it using numerous images from Operation Theatres & augmented it to satisfy all possible scenarios. This study has also addressed the shortcomings of previous project iterations. Previously, the project employed two models: one for human detection and another for gauze detection, trained on a total of 2800 images. Now we have an integrated model capable of identifying both humans and gauzes, using a training set of 11,000 images. This has led to improvements in accuracy and increased the frame rate from 8 FPS to 15 FPS now. Incorporating doctor's feedback, the system now also supports manual count adjustments, enhancing its reliability in actual surgeries.
Read the full abstract ▾
Computer Vision
CTCal: Rethinking Text-to-Image Diffusion Models via Cross-Timestep Self-Calibration
Xiefan Guo, Xinzhu Ma, Haiyu Zhang et al. (+1 more)
Recent advancements in text-to-image synthesis have been largely propelled by
diffusion-based models, yet achieving precise
alignment between text
prompts and generated images remains a persistent challenge. We find that this difficulty arises primarily from the limitations of conventional
diffusion loss, which provides only implicit supervision for modeling fine-grained text-image correspondence. In this paper, we introduce Cross-Timestep Self-Calibration (CTCal), founded on the supporting observation that establishing accurate text-image
alignment within
diffusion models becomes progressively more difficult as the timestep increases. CTCal leverages the reliable text-image
alignment (i.e., cross-
attention maps) formed at smaller timesteps with less noise to calibrate the representation learning at larger timesteps with more noise, thereby providing explicit supervision during training. We further propose a timestep-aware adaptive weighting to achieve a harmonious integration of CTCal and
diffusion loss. CTCal is model-agnostic and can be seamlessly integrated into existing text-to-image
diffusion models, encompassing both
diffusion-based (e.g., SD 2.1) and flow-based approaches (e.g., SD 3). Extensive experiments on T2I-Compbench++ and GenEval
benchmarks demonstrate the effectiveness and generalizability of the proposed CTCal. Our code is available at https://github.com/xiefan-guo/ctcal.
Read the full abstract ▾
All authors Xiefan Guo, Xinzhu Ma, Haiyu Zhang, Di Huang
Computer Vision
Mamba Learns in Context: Structure-Aware Domain Generalization for Multi-Task Point Cloud Understanding
Jincen Jiang, Qianyu Zhou, Yuhang Li et al. (+5 more)
While recent
Transformer and Mamba architectures have advanced point cloud representation learning, they are typically developed for single-task or single-domain settings. Directly applying them to multi-task domain generalization (DG) leads to degraded performance.
Transformers effectively model global dependencies but suffer from quadratic
attention cost and lack explicit structural ordering, whereas Mamba offers linear-time recurrence yet often depends on coordinate-driven serialization, which is sensitive to viewpoint changes and missing regions, causing structural drift and unstable sequential modeling. In this paper, we propose Structure-Aware Domain Generalization (SADG), a Mamba-based In-Context Learning framework that preserves structural hierarchy across domains and tasks. We design structure-aware serialization (SAS) that generates transformation-invariant sequences using centroid-based
topology and geodesic
curvature continuity. We further devise hierarchical domain-aware modeling (HDM) that stabilizes cross-domain
reasoning by consolidating intra-domain structure and fusing inter-domain relations. At test time, we introduce a lightweight spectral graph
alignment (SGA) that shifts target features toward source prototypes in the spectral domain without updating model parameters, ensuring structure-preserving test-time feature shifting. In addition, we introduce MP3DObject, a real-scan object
dataset for multi-task DG evaluation. Comprehensive experiments demonstrate that the proposed approach improves structural fidelity and consistently outperforms state-of-the-art methods across multiple tasks including reconstruction, denoising, and registration.
Read the full abstract ▾
All authors Jincen Jiang, Qianyu Zhou, Yuhang Li, Kui Su, Meili Wang, Jian Chang, Jian Jun Zhang, Xuequan Lu
Computer Vision
SATTC: Structure-Aware Label-Free Test-Time Calibration for Cross-Subject EEG-to-Image Retrieval
Qunjie Huang and Weina Zhu
Cross-subject
EEG-to-image
retrieval for visual decoding is challenged by subject shift and hubness in the
embedding space, which distort similarity geometry and destabilize top-k rankings, making small-k shortlists unreliable. We introduce SATTC (Structure-Aware Test-Time Calibration), a label-free calibration head that operates directly on the similarity
matrix of frozen
EEG and image encoders. SATTC combines a
geometric expert, subject-adaptive whitening of
EEG embeddings with an adaptive variant of Cross-domain Similarity Local Scaling (CSLS), and a structural expert built from mutual nearest neighbors, bidirectional top-k ranks, and class popularity, fused via a simple Product-of-Experts rule. On THINGS-
EEG2 under a strict leave-one-subject-out protocol, standardized inference with cosine similarities, L2-normalized
embeddings, and candidate whitening already yields a strong cross-subject baseline over the original ATM
retrieval setup. Building on this baseline, SATTC further improves Top-1 and Top-5 accuracy, reduces hubness and per-class imbalance, and produces more reliable small-k shortlists. These gains transfer across multiple
EEG encoders, supporting SATTC as an encoder-agnostic, label-free test-time calibration layer for cross-subject neural decoding.
Read the full abstract ▾
Computer Vision
VSD-MOT: End-to-End Multi-Object Tracking in Low-Quality Video Scenes Guided by Visual Semantic Distillation
Jun Du
Existing multi-object tracking
algorithms typically fail to adequately address the issues in low-quality
videos, resulting in a significant decline in tracking performance when image quality deteriorates in real-world scenarios. This performance degradation is primarily due to the
algorithms' inability to effectively tackle the problems caused by information loss in low-quality images. To address the challenges of low-quality
video scenarios, inspired by
vision-
language models, we propose a multi-object tracking framework guided by visual semantic
distillation (VSD-MOT). Specifically, we introduce the CLIP Image Encoder to extract global visual semantic information from images to compensate for the loss of information in low-quality images. However, direct integration can substantially impact the efficiency of the multi-object tracking
algorithm. Therefore, this paper proposes to extract visual semantic information from images through knowledge
distillation. This method adopts a teacher-student learning framework, with the CLIP Image Encoder serving as the teacher model. To enable the student model to acquire the capability of extracting visual semantic information suitable for multi-object tracking tasks from the teacher model, we have designed the Dual-Constraint Semantic
Distillation method (DCSD). Furthermore, to address the dynamic variation of frame quality in low-quality
videos, we propose the Dynamic Semantic Weight Regulation (DSWR) module, which adaptively allocates fusion weights based on real-time frame quality assessment. Extensive experiments demonstrate the effectiveness and superiority of the proposed method in low-quality
video scenarios in the real world. Meanwhile, our method can maintain good performance in conventional scenarios.
Read the full abstract ▾
Computer Vision
Weakly supervised multimodal segmentation of acoustic borehole images with depth-aware cross-attention
Jose Luis Lima de Jesus Silva
Acoustic borehole images provide high-resolution borehole-wall structure, but large-scale interpretation remains difficult because dense expert annotations are rarely available and subsurface information is intrinsically
multimodal. The challenge is developing weakly supervised methods combining two-dimensional image texture with depth-aligned one-dimensional well-logs. Here, we introduce a weakly supervised
multimodal segmentation framework that refines threshold-guided pseudo-labels through learned models. This preserves the annotation-free character of classical thresholding and clustering workflows while extending them with denoising, confidence-aware pseudo-supervision, and physically structured fusion. We establish that threshold-guided learned refinement provides the most robust improvement over raw thresholding, denoised thresholding, and latent clustering baselines.
Multimodal performance depends strongly on fusion strategy: direct concatenation provides limited gains, whereas depth-aware cross-
attention, gated fusion, and confidence-aware modulation substantially improve agreement with the weak supervisory reference. The strongest model, confidence-gated depth-aware cross-
attention (CG-DCA), consistently outperforms threshold-based, image-only, and earlier
multimodal baselines. Targeted ablations show its advantage depends specifically on confidence-aware fusion and structured local depth interaction rather than model complexity alone. Cross-well analyses confirm this performance is broadly stable. These results establish a practical, scalable framework for annotation-free
segmentation, showing
multimodal improvement is maximized when auxiliary logs are incorporated selectively and depth-aware.
Read the full abstract ▾
Computer Vision
Premier: Personalized Preference Modulation with Learnable User Embedding in Text-to-Image Generation
Zihao Wang, Yuxiang Wei, Xinpeng Zhou et al. (+5 more)
Text-to-image generation has advanced rapidly, yet it still struggles to capture the nuanced user preferences. Existing approaches typically rely on
multimodal large
language models to infer user preferences, but the derived
prompts or latent codes rarely reflect them faithfully, leading to suboptimal personalization. We present Premier, a novel preference modulation framework for personalized image generation. Premier represents each user's preference as a learnable
embedding and introduces a preference adapter that fuses the user
embedding with the text
prompt. To enable accurate and fine-grained preference control, the fused preference
embedding is further used to modulate the generative process. To enhance the distinctness of individual preference and improve
alignment between outputs and user-specific styles, we incorporate a dispersion loss that enforces separation among user
embeddings. When user data are scarce, new users are represented as linear combinations of existing preference
embeddings learned during training, enabling effective generalization. Experiments show that Premier outperforms prior methods under the same history length, achieving stronger preference
alignment and superior performance on text consistency, ViPer proxy metrics, and expert evaluations.
Read the full abstract ▾
All authors Zihao Wang, Yuxiang Wei, Xinpeng Zhou, Tianyu Zhang, Tao Liang, Yalong Bai, Hongzhi Zhang, Wangmeng Zuo
Computer Vision
Cross-modal Fuzzy Alignment Network for Text-Aerial Person Retrieval and A Large-scale Benchmark
Yifei Deng, Chenglong Li, Yuyang Zhang et al. (+2 more)
Text-aerial person
retrieval aims to identify targets in
UAV-captured images from eyewitness descriptions, supporting intelligent transportation and public security applications. Compared to ground-view text–image person
retrieval,
UAV-captured images often suffer from degraded visual information due to drastic variations in viewing angles and flight altitudes, making semantic
alignment with textual descriptions very challenging. To address this issue, we propose a novel Cross-modal Fuzzy
Alignment Network, which quantifies the
token-level reliability by fuzzy logic to achieve accurate fine-grained
alignment and incorporates ground-view images as a bridge
agent to further mitigate the gap between aerial images and text descriptions, for text–aerial person
retrieval. In particular, we design the Fuzzy
Token Alignment module that employs the fuzzy membership function to dynamically model
token-level association strength and suppress the influence of unobservable or noisy
tokens. It can alleviate the semantic inconsistencies caused by missing visual cues and significantly enhance the robustness of
token-level semantic
alignment. Moreover, to further mitigate the gap between aerial images and text descriptions, we design a Context-Aware Dynamic
Alignment module to incorporate the ground-view
agent as a bridge in text–aerial
alignment and adaptively combine direct
alignment and
agent-assisted
alignment to improve the robustness. In addition, we construct a large-scale
benchmark dataset called AERI-PEDES by using a chain-of-thought to decompose text generation into attribute parsing, initial captioning, and refinement, thus boosting textual accuracy and semantic consistency. Experiments on AERI-PEDES and TBAPR demonstrate the superiority of our method.
Read the full abstract ▾
All authors Yifei Deng, Chenglong Li, Yuyang Zhang, Guyue Hu, Jin Tang
Computer Vision
The Role of Initialization in 3D Gaussian Splatting
Ivan Desiatov and Torsten Sattler
3D Gaussian Splatting (3DGS) has become the method of choice for photo-realistic novel view synthesis (NVS), due to its efficiency and compelling visual quality. 3DGS represents the scene as a set of 3D Gaussians, parameterized by their position, spatial extent, and view-dependent color. Starting from an initial point cloud, 3DGS refines the Gaussians' parameters to reconstruct a set of training images as accurately as possible. Typically, a sparse Structure-from-Motion point cloud is used as initialization. Thus, in order to obtain a full scene representation, 3DGS methods rely on a densification stage. In this paper, we systematically study how initialization affects 3DGS NVS performance and
geometric quality, using several densification strategies. We show that dense initialization does not lead to consistent visual improvements when paired with strong densification. Despite that, it helps in generalization to off-trajectory views and significantly improves
geometric accuracy of the scenes. Our code is available at https://github.com/deivse/ivd_splat.
Read the full abstract ▾
Computer Vision
High-Quality and Efficient Turbulence Mitigation with Events
Xiaoran Zhang, Jian Ding, Yuxing Duan et al. (+4 more)
Turbulence mitigation (TM) is highly ill-posed due to the
stochastic nature of atmospheric
turbulence. Most methods rely on multiple frames recorded by conventional cameras to capture stable patterns in natural scenarios. However, they inevitably suffer from a trade-off between accuracy and efficiency: more frames enhance restoration at the cost of higher system latency and larger data overhead. Event cameras, equipped with microsecond temporal resolution and efficient sensing of dynamic changes, offer an opportunity to break the bottleneck. In this work, we present EHETM, a high-quality and efficient TM method inspired by the superiority of events to model motions in continuous sequences. We discover two key phenomena: (1)
turbulence-induced events exhibit distinct polarity alternation correlated with sharp image gradients, providing structural cues for restoring scenes; and (2) dynamic objects form spatiotemporally coherent “event tubes” in contrast to irregular patterns within turbulent events, providing motion priors for disentangling objects from
turbulence. Based on these insights, we design two complementary modules that respectively leverage polarity-weighted gradients for scene refinement and event-tube constraints for motion decoupling, achieving high-quality restoration with few frames. Furthermore, we construct two real-world event-frame
turbulence datasets covering atmospheric and thermal cases. Experiments show that EHETM outperforms SOTA methods, especially under scenes with dynamic objects, while reducing data overhead and system latency by approximately 77.3% and 89.5%, respectively. Our code is available at: https://github.com/Xavier667/EHETM.
Read the full abstract ▾
All authors Xiaoran Zhang, Jian Ding, Yuxing Duan, Haoyue Liu, Gang Chen, Yi Chang, Luxin Yan
Computer Vision
Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs
Huan Zheng, Yucheng Zhou, Tianyi Yan et al. (+6 more)
Multimodal Large
Language Models (MLLMs) have demonstrated remarkable potential in medical image analysis. However, their application in gastrointestinal endoscopy is currently hindered by two critical limitations: the misalignment between general model
reasoning and standardized clinical cognitive pathways, and the lack of causal association between visual features and diagnostic outcomes. In this paper, we propose a novel Clinical-Cognitive-Aligned (CogAlign) framework to address these challenges. First, we endow the model with rigorous clinical analytical capabilities by constructing the hierarchical clinical cognition
dataset and employing Supervised Fine-Tuning (SFT). Unlike conventional approaches, this strategy internalizes the hierarchical diagnostic logic of experts, ranging from anatomical localization and morphological evaluation to microvascular analysis, directly into the model. Second, to eliminate visual bias, we provide a theoretical analysis demonstrating that standard supervised tuning inevitably converges to spurious background correlations. Guided by this insight, we propose a counterfactual-driven reinforcement learning strategy to enforce causal rectification. By generating counterfactual normal samples via lesion masking and optimizing through clinical-cognition-centric rewards, we constrain the model to strictly ground its diagnosis in causal lesion features. Extensive experiments demonstrate that our approach achieves State-of-the-Art (SoTA) performance across multiple
benchmarks, significantly enhancing diagnostic accuracy in complex clinical scenarios.
Read the full abstract ▾
All authors Huan Zheng, Yucheng Zhou, Tianyi Yan, Dubing Chen, Hongbo Lu, Wenlong Liao, Tao He, Pai Peng, Jianbing Shen
Computer Vision
Satellite-to-Street: Synthesizing Post-Disaster Views from Satellite Imagery via Generative Vision Models
Yifan Yang, Lei Zou and Wendy Jepson
In the immediate aftermath of natural disasters, rapid situational awareness is critical. Traditionally, satellite observations are widely used to estimate damage extent. However, they lack the ground-level perspective essential for characterizing specific structural failures and impacts. Meanwhile, ground-level data (e.g., street-view imagery) remains largely inaccessible during time-sensitive events. This study investigates Satellite-to-Street View Synthesis to bridge this data gap. We introduce two generative strategies to synthesize post-disaster street views from satellite imagery: a
Vision-
Language Model (VLM)-guided approach and a damage-sensitive Mixture-of-Experts (MoE) method. We
benchmark these against general-purpose baselines (Pix2Pix, ControlNet) using a proposed Structure-Aware Evaluation Framework. This multi-tier protocol integrates (1) pixel-level quality assessment, (2) ResNet-based semantic consistency verification, and (3) a novel VLM-as-a-Judge for perceptual
alignment. Experiments on 300 disaster scenarios reveal a critical realism–fidelity trade-off: while
diffusion-based approaches (e.g., ControlNet) achieve high perceptual realism, they often hallucinate structural details. Quantitative results show that standard ControlNet achieves the highest semantic accuracy, 0.71, whereas VLM-enhanced and MoE models excel in textural plausibility but struggle with semantic clarity. This work establishes a baseline for trustworthy cross-view synthesis, emphasizing that visually realistic generations may still fail to preserve critical structural information required for reliable disaster assessment.
Read the full abstract ▾
Computer Vision
MFSR: MeanFlow Distillation for One Step Real-World Image Super Resolution
Ruiqing Wang, Kai Zhang, Yuanzhi Zhu et al. (+3 more)
Diffusion- and flow-based models have advanced Real-world Image Super-Resolution (Real-ISR), but their multi-step sampling makes inference slow and hard to deploy. One-step
distillation alleviates the cost, yet often degrades restoration quality and removes the option to refine with more steps. We present Mean Flows for Super-Resolution (MFSR), a new
distillation framework that produces photorealistic results in a single step while still allowing an optional few-step path for further improvement. Our approach uses MeanFlow as the learning target, enabling the student to approximate the average velocity between arbitrary states of the Probability Flow ODE (PF-ODE) and effectively capture the teacher's dynamics without explicit rollouts. To better leverage pretrained generative priors, we additionally improve original MeanFlow's Classifier-Free Guidance (CFG) formulation with teacher CFG
distillation strategy, which enhances restoration capability and preserves fine details. Experiments on both synthetic and real-world
benchmarks demonstrate that MFSR achieves efficient, flexible, and high-quality super-resolution, delivering results on par with or even better than multi-step teachers while requiring much lower computational cost.
Read the full abstract ▾
All authors Ruiqing Wang, Kai Zhang, Yuanzhi Zhu, Hanshu Yan, Shilin Lu, Jian Yang
Computer Vision
IBCapsNet: Information Bottleneck Capsule Network for Noise-Robust Representation Learning
Canqun Xiang, Chen Yang and Jiaoyan Zhao
Capsule networks (CapsNets) are superior at modeling hierarchical spatial relationships but suffer from two critical limitations: high computational cost due to iterative dynamic routing and poor robustness under input corruptions. To address these issues, we propose IBCapsNet, a novel capsule architecture grounded in the Information Bottleneck (IB) principle. Instead of iterative routing, IBCapsNet employs a one-pass variational aggregation mechanism, where primary capsules are first compressed into a global context representation and then processed by class-specific variational autoencoders (VAEs) to infer latent capsules regularized by the KL divergence. This design enables efficient inference while inherently filtering out noise. Experiments on MNIST, Fashion-MNIST, SVHN and CIFAR-10 show that IBCapsNet matches CapsNet in clean-data accuracy (achieving 99.41% on MNIST and 92.01% on SVHN), yet significantly outperforms it under four types of synthetic noise - demonstrating average improvements of +17.10% and +14.54% for clamped additive and multiplicative noise, respectively. Moreover, IBCapsNet achieves 2.54x faster training and 3.64x higher inference throughput compared to CapsNet, while reducing model parameters by 4.66%. Our work bridges information-theoretic representation learning with capsule networks, offering a principled path toward robust, efficient, and interpretable deep models. Code is available at https://github.com/cxiang26/IBCapsnet
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
ToFormer: Towards Large-scale Scenario Depth Completion for Lightweight ToF Camera
Juncheng Chen, Tiancheng Lai, Xingpeng Wang et al. (+4 more)
Time-of-Flight (ToF) cameras possess compact design and high measurement precision to be applied to various
robot tasks. However, their limited sensing range restricts deployment in large-scale scenarios. Depth completion has emerged as a potential solution to expand the sensing range of ToF cameras, but existing research lacks dedicated
datasets and struggles to generalize to ToF measurements. In this paper, we propose a full-stack framework that enables depth completion in large-scale scenarios for short-range ToF cameras. First, we construct a multi-sensor platform with a reconstruction-based pipeline to collect real-world ToF samples with dense large-scale ground truth, yielding the first LArge-ScalE scenaRio ToF depth completion
dataset (
LASER-ToF). Second, we propose a sensor-aware depth completion network that incorporates a novel 3D branch with a 3D-2D Joint Propagation Pooling (JPP) module and
Multimodal Cross-Covariance
Attention (MXCA), enabling effective modeling of long-range relationships and efficient 3D-2D fusion under non-uniform ToF depth sparsity. Moreover, our network can utilize the sparse point cloud from visual SLAM as a supplement to ToF depth to further improve prediction accuracy. Experiments show that our method achieves an 8.6% lower mean absolute error than the second-best method, while maintaining lightweight design to support onboard deployment. Finally, to verify the system's applicability on real
robots, we deploy proposed method on a quadrotor at a 10Hz runtime, enabling reliable large-scale mapping and long-range planning in challenging environments for short-range ToF cameras.
Read the full abstract ▾
All authors Juncheng Chen, Tiancheng Lai, Xingpeng Wang, Bingxin Liao, Baozhe Zhang, Chao Xu, Yanjun Cao
Computer Vision · Cross-listed from cs.AI
Attention in Space: Functional Roles of VLM Heads for Spatial Reasoning
Xueqi Ma, Shuo Yang, Yanbei Jiang et al. (+6 more)
Despite remarkable advances in large
Vision-
Language Models (VLMs), spatial
reasoning remains a persistent challenge. In this work, we investigate how
attention heads within VLMs contribute to spatial
reasoning by analyzing their functional roles through a mechanistic interpretability lens. We introduce CogVSR, a
dataset that decomposes complex spatial
reasoning questions into step-by-step subquestions designed to simulate human-like
reasoning via a chain-of-thought paradigm, with each subquestion linked to specific cognitive functions such as spatial perception or relational
reasoning. Building on CogVSR, we develop a probing framework to identify and characterize
attention heads specialized for these functions. Our analysis across diverse VLM families reveals that these functional heads are universally sparse, vary in number and distribution across functions. Notably, spatially specialized heads are fewer than those for other cognitive functions, highlighting their scarcity. We propose methods to activate latent spatial heads, improving spatial understanding. Intervention experiments further demonstrate their critical role in spatial
reasoning: removing functional heads leads to performance degradation, while emphasizing them enhances accuracy. This study provides new interpretability driven insights into how VLMs attend to space and paves the way for enhancing complex spatial
reasoning in
multimodal models.
Read the full abstract ▾
All authors Xueqi Ma, Shuo Yang, Yanbei Jiang, Shu Liu, Zhenzhen Liu, Jiayang Ao, Xingjun Ma, Sarah Monazam Erfani, James Bailey
Computer Vision
A Multihead Continual Learning Framework for Fine-Grained Fashion Image Retrieval with Contrastive Learning and Exponential Moving Average Distillation
Ling Xiao and Toshihiko Yamasaki
Most fine-grained fashion image
retrieval (FIR) methods assume a static setting, requiring full retraining when new attributes appear, which is costly and impractical for dynamic scenarios. Although pretrained models support zero-shot inference, their accuracy drops without supervision, and no prior work explores class-incremental learning (CIL) for fine-grained FIR. We propose a multihead continual learning framework for fine-grained fashion image
retrieval with contrastive learning and exponential moving average (EMA)
distillation (MCL-FIR). MCL-FIR adopts a multi-head design to accommodate evolving classes across increments, reformulates triplet inputs into doublets with InfoNCE for simpler and more effective training, and employs EMA
distillation for efficient knowledge transfer. Experiments across four
datasets demonstrate that, beyond its scalability, MCL-FIR achieves a strong balance between efficiency and accuracy. It significantly outperforms CIL baselines under similar training cost, and compared with static methods, it delivers comparable performance while using only about 30% of the training cost. The source code is publicly available in https://github.com/Dr-LingXiao/MCL-FIR.
Read the full abstract ▾
Computer Vision
ScaleEdit-12M: Scaling Open-Source Image Editing Data Generation via Multi-Agent Framework
Guanzhou Chen, Erfei Cui, Changyao Tian et al. (+6 more)
Instruction-based image editing has emerged as a key capability for unified
multimodal models (UMMs), yet constructing large-scale, diverse, and high-quality editing
datasets without costly proprietary APIs remains challenging. Previous image editing
datasets either rely on closed-source models for annotation, which prevents cost-effective scaling, or employ fixed synthetic editing pipelines, which suffer from limited quality and generalizability. To address these challenges, we propose ScaleEditor, a fully open-source hierarchical multi-
agent framework for end-to-end construction of large-scale, high-quality image editing
datasets. Our pipeline consists of three key components: source image expansion with world-knowledge infusion, adaptive multi-
agent editing instruction-image synthesis, and a task-aware data quality verification mechanism. Using ScaleEditor, we curate ScaleEdit-12M, the largest open-source image editing
dataset to date, spanning 23 task families across diverse real and synthetic domains. Fine-tuning UniWorld-V1 and Bagel on ScaleEdit yields consistent gains, improving performance by up to 10.4% on ImgEdit and 35.1% on GEdit for general editing
benchmarks and by up to 150.0% on RISE and 26.5% on KRIS-Bench for knowledge-infused
benchmarks. These results demonstrate that open-source, agentic pipelines can approach commercial-grade data quality while retaining cost-effectiveness and scalability. Both the framework and
dataset will be open-sourced.
Read the full abstract ▾
All authors Guanzhou Chen, Erfei Cui, Changyao Tian, Danni Yang, Ganlin Yang, Yu Qiao, Hongsheng Li, Gen Luo, Hongjie Zhang
Computer Vision
GaussianPile: A Unified Sparse Gaussian Splatting Framework for Slice-based Volumetric Reconstruction
Di Kong, Yikai Wang, Wenjie Guo et al. (+8 more)
Slice-based volumetric imaging is widely applied and it demands representations that compress aggressively while preserving internal structure for analysis. We introduce GaussianPile, unifying 3D Gaussian splatting with an imaging system-aware focus model to address this challenge. Our proposed method introduces three key innovations: (i) a slice-aware piling strategy that positions anisotropic 3D Gaussians to model through-slice contributions, (ii) a differentiable projection operator that encodes the finite-thickness point spread function of the imaging acquisition system, and (iii) a compact encoding and joint optimization pipeline that simultaneously reconstructs and compresses the Gaussian sets. Our CUDA-based design retains the compression and real-time rendering efficiency of Gaussian primitives while preserving high-frequency internal volumetric detail. Experiments on microscopy and ultrasound
datasets demonstrate that our method reduces storage and reconstruction cost, sustains diagnostic fidelity, and enables fast 2D visualization, along with 3D voxelization. In practice, it delivers high-quality results in as few as 3 minutes, up to 11x faster than NeRF-based approaches, and achieves consistent 16x compression over voxel grids, offering a practical path to deployable compression and exploration of slice-based volumetric
datasets.
Read the full abstract ▾
All authors Di Kong, Yikai Wang, Wenjie Guo, Yifan Bu, Boya Zhang, Yuexin Duan, Xiawei Yue, Wenbiao Du, Yiman Zhong, Yuwen Chen, Cheng Ma
Computer Vision
RayMap3R: Inference-Time RayMap for Dynamic 3D Reconstruction
Feiran Wang, Zezhou Shang, Gaowen Liu et al. (+1 more)
Streaming feed-forward 3D reconstruction enables real-time joint estimation of scene geometry and camera poses from RGB images. However, without explicit dynamic
reasoning, streaming models can be affected by moving objects, causing artifacts and drift. In this work, we propose RayMap3R, a training-free streaming framework for dynamic scene reconstruction. We observe that RayMap-based predictions exhibit a static-scene bias, providing an internal cue for dynamic identification. Based on this observation, we construct a dual-branch inference scheme that identifies dynamic regions by contrasting RayMap and image predictions, suppressing their interference during
memory updates. We further introduce reset metric
alignment and state-aware smoothing to preserve metric consistency and stabilize predicted trajectories. Our method achieves state-of-the-art performance among streaming approaches on dynamic scene reconstruction across multiple
benchmarks.
Read the full abstract ▾
All authors Feiran Wang, Zezhou Shang, Gaowen Liu, Yan Yan
Computer Vision
Improving Diffusion Generalization with Weak-to-Strong Segmented Guidance
Liangyu Yuan, Yufei Huang, Mingkun Lei et al. (+5 more)
Diffusion models generate synthetic images through an iterative refinement process. However, the misalignment between the simulation-free objective and the iterative process often causes accumulated gradient error along the sampling trajectory, which leads to unsatisfactory results and a failure to generalize. Guidance techniques like Classifier Free Guidance (CFG) and AutoGuidance (AG) alleviate this by extrapolating between the main and inferior signal for stronger generalization. Despite empirical success, the effective operational regimes of prevalent guidance methods are still under-explored, leading to ambiguity when selecting the appropriate guidance method given a precondition. In this work, we first conduct synthetic comparisons to isolate and demonstrate the effective regime of guidance methods represented by CFG and AG from the perspective of weak-to-strong principle. Based on this, we propose a hybrid instantiation called SGG under the principle, taking the benefits of both. Furthermore, we demonstrate that the W2S principle along with SGG can be migrated into the training objective, improving the generalization ability of unguided
diffusion models. We validate our approach with comprehensive experiments. At inference time, evaluations on SD3 and SD3.5 confirm that SGG outperforms existing training-free guidance variants. Training-time experiments on
transformer architectures demonstrate the effective migration and performance gains in both conditional and unconditional settings. Code is available at https://github.com/851695e35/SGG.
Read the full abstract ▾
All authors Liangyu Yuan, Yufei Huang, Mingkun Lei, Tong Zhao, Ruoyu Wang, Changxi Chi, Yiwei Wang, Chi Zhang