Computer Vision
Local-GS: Accelerating 3D Gaussian Splatting via Tile-Local Warp Coherence
Yang Luo, Yan Gong, Yongsheng Gao et al. (+3 more)
3D Gaussian Splatting (3DGS) has significantly advanced real-time novel view synthesis by representing scenes as dense collections of anisotropic 3D Gaussian primitives. However, the irregular spatial distribution of Gaussians often leads to poor GPU utilization, as warp divergence and redundant computation degrade rendering performance. To address this, we present Local-GS, a warp-coherent rendering paradigm that, organizes Gaussian primitives with respect to SIMT (Single Instruction, Multiple Threads) execution boundaries rather than scene geometry. Specifically, we propose three warp-coherent stages: a hoisting stage that precomputes shared parameters at tile level, a culling stage that discards warps with no contribution, and a blending stage that replaces per-pixel branching with a uniform instruction stream. Across extensive
benchmarks on multiple
datasets, Local-GS improves efficiency without compromising quality. As a plug-and-play optimization, it provides additional performance gains to all tested baselines, culminating in a $7.76\times$ speedup on Deep Blending scenes.
Read the full abstract ▾
All authors Yang Luo, Yan Gong, Yongsheng Gao, Jie Zhao, Xinyu Zhang, Huaping Liu
Computer Vision · Cross-listed from cs.LG
Assessing Reliability of Symbol Detection in Concept Bottleneck Models
Javier Fumanal-Idocin and Javier Andreu-Perez
Concept Bottleneck Models (CBMs) are a relevant tool for explainable Artificial Intelligence because they make their predictions through human-interpretable symbols. However, high task accuracy does not guarantee that these symbols are detected faithfully: jointly trained CBMs may encode task-specific shortcuts in the bottleneck, making their explanations unreliable. In this paper, we study concept-detection reliability by swapping independently trained concept detectors and classification heads that share the same symbolic vocabulary. We use the resulting performance degradation, concept-level metrics, and symbol-wise uncertainty estimates to identify concepts that are especially prone to spurious firing. Finally, we propose a reliability-aware training strategy in which a shared concept detector is optimized with multiple classification heads and penalized for relying on globally or instance-wise unreliable symbols. On CUB-200-2011 with full concept supervision, detectors and heads are almost freely interchangeable (swap drop below one accuracy point, relative retention above $99\%$, and no concept detected below chance), whereas on a controlled synthetic task we show that, as the concept-supervision weight is reduced, models keep near-perfect task accuracy while swapped accuracy and agreement with the ground-truth concepts collapse to chance. Our reliability-aware training substantially mitigates this leakage, roughly doubling swap accuracy in the leaky regime.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI
Kairos Team: Fei Wang, Shan You, Qiming Zhang et al. (+20 more)
We introduce Kairos, a regret-aware native world-action model stack for Physical AI. Kairos is motivated by the view that a physical world model should not aim to fully simulate all future pixels, but should learn and maintain the information most relevant to embodiment control: object state, spatial relations, contact conditions, task progress, action consequences, failure boundaries, and deployment uncertainty. Kairos establishes three model-side prerequisites toward this goal. First, it learns control-relevant information through a Cross-Embodiment Data Curriculum, which organizes open-world
videos, human behavioral data, and
robot interactions into an intervention-strength progression from passive physical observation to intentional behavior and
embodied action grounding. Second, it maintains control-sufficient states through a unified understanding, generation, and prediction architecture equipped with Hybrid Linear Temporal
Attention, where local, mid-range, and global temporal pathways support multi-timescale state maintenance under efficient inference. Third, it deploys these states through a Deployment-Aware System Co-Design, treating latency,
memory footprint, and hardware compatibility as first-order constraints for future observation, action, and feedback loops. Experiments on
embodied world-model
benchmarks, world-action
benchmarks, long-horizon generation, and inference-efficiency evaluation show that Kairos achieves superior performance while offering a favorable efficiency to capability trade-off.
Read the full abstract ▾
All authors Kairos Team: Fei Wang, Shan You, Qiming Zhang, Tao Huang, Zuoyi Fu, Zhisheng Zheng, Yunlong Xi, Feng Lv, Xiaoming Wu, Zeyu Liu, Cong Wan, Pu Li, Ruiqing Yang, Xiaoou Li, Wei Wang, Kangkang Zhu, Yuwei Zhang, Shi Fu, Zheng Zhang, Xiaoning Wu, Xuzeng Fan, Dacheng Tao, Xiaogang Wang
Computer Vision
BadWorld: Adversarial Attacks on World Models
Linghui Shen, Mingyue Cui and Xingyi Yang
Visual world models (VWMs) synthesize interactive, action-conditioned rollouts from a single context image. However, it remains an open question how robust these models are to
adversarial perturbations. Standard
adversarial attacks fail to assess this vulnerability because attackers lack ground-truth future
videos and cannot predict subsequent user controls. We introduce BadWorld, a label-free
adversarial framework tailored for autoregressive VWMs that systematically overcomes both constraints. First, to bypass the need for future supervision, we propose a self-supervised velocity attack that directly disrupts the early denoising dynamics of the model. Second, to ensure the attack generalizes across unpredictable user actions, we formulate a trajectory-adaptive bi-level optimization that actively mines hard control sequences to forge control-agnostic perturbations. Evaluated on representative VWMs with continuous and discrete controls, BadWorld exposes severe structural fragility. Visually indistinguishable
adversarial images reliably trigger catastrophic degradation in future rollouts, leading to incomplete denoising, structural collapse, and control inconsistency. These findings reveal critical risks for deploying VWMs in safety-critical systems while highlighting a practical mechanism for
privacy protection.
Read the full abstract ▾
Computer Vision
Active Reference Acquisition in Few-Shot Font Generation
Shinnosuke Matsuo
Few-shot font generation aims to synthesize the remaining glyphs of a font given one or a few reference glyphs while preserving stylistic consistency, thereby supporting font designers in efficiently completing a typeface. Existing methods primarily focus on improving generation quality given a fixed reference set. However, when the current reference glyphs are insufficient to represent the target style, few-shot font generation may fail to produce satisfactory results. In practical scenarios, additional reference glyphs can often be obtained from the designer when necessary. Accordingly, we propose a new framework, Active Reference Acquisition in Few-Shot Font Generation, in which the model sequentially decides which character to acquire next as an additional reference. Furthermore, we propose a reference part-coverage-based acquisition function to efficiently query the designer. Motivated by the observation that font styles are well characterized by local structural parts, we represent each glyph using a histogram of local features and select query characters that maximize the expected part coverage of the reference set. By prioritizing characters that contain parts not yet covered by the current references, the proposed method progressively expands the diversity of visual parts in the reference set. As a result, generation quality is improved with fewer queries. Experiments on the Google Fonts
dataset demonstrate that the proposed method achieves higher generation quality than random querying and reference-agnostic baselines. The code is available at https://github.com/matsuo-shinnosuke/ActiveRef-FontGen.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Lost at the End: Primacy Bias in Multimodal Retrieval-Augmented Question Answering
Jieyuan Liu, Jianyang Gu, Shijie Chen et al. (+2 more)
Knowledge-based visual question answering (KB-VQA) lets
vision-
language systems answer questions that exceed their parametric knowledge by conditioning a reader on passages retrieved from a Wikipedia-derived knowledge base. In pure-text long-context LLMs, retrieved-context use follows the U-shaped lost-in-the-middle effect of Liu et al. (2024): information at the start and end of context is used, the middle is lost. Whether this transfers to deployed
multimodal KB-VQA is open. To close this gap, we design the first controlled probe of reader-side position dependence in
multimodal KB-VQA: a gold-position protocol in which only the gold passage's
prompt slot varies within question. We run it on three open-source 7B/8B VLM readers and two KB-VQA
benchmarks with up to 20 retrieved passages. The shape flips from U to primacy: gold-at-first beats gold-at-last by 16 to 26 points on all six combinations of reader and
benchmark, an effect we call Lost at the End; the gap holds at every scale we test, 3B to 32B, attenuating at 32B. Three targeted ablations narrow the cause. A text-only control that removes the image and changes nothing else shows the primacy is already present in text mode and does not depend on the image. Image-position and distractor-shuffle ablations trace the effect to
prompt slot 0 of the instruction-tuned reader, where a second answer-bearing passage placed later is largely wasted. On a frozen reader, three
retrieval-side fixes (MMR, oracle reranking, rank-based reordering) all fail to improve on the deployment default. Our findings indicate that recall@k is the wrong metric for deployed KB-VQA and that the remaining headroom sits on the reader side; we release our protocol as a controlled instrument for evaluating reader-side interventions.
Read the full abstract ▾
All authors Jieyuan Liu, Jianyang Gu, Shijie Chen, Jefferson Chen, Zhen Wang
Computer Vision
Unified Multimodal Model for Brain MRI Imputation and Understanding
Zhiyun Song, Che Liu, Tian Xia et al. (+2 more)
Multimodal large
language models (MLLMs) hold great potential for medicine, as they inherit knowledge from LLM and allow multiple data modalities to be integrated, analysed and interpreted in natural
language. However, the field of medical MLLMs is constrained by non-trivial challenges, notably the scarcity of high-quality training data and the frequent occurrence of missing data in the real-world clinical setting. Here, we propose a novel unified
multimodal model, UniBrain, for
brain magnetic resonance image (MRI) analysis. To address potential missing
brain MRI modalities, we employ a unified training strategy to perform joint imaging modality imputation and
brain image understanding. During training, an interleaved and description-enriched data flow is constructed to train the model in an autoregressive manner, enabling medical
reasoning with generated
multimodal data. A self-
alignment strategy is introduced to leverage dense image
embeddings to learn fine-grained anatomical features without requiring detailed image captions. Furthermore, we propose a dynamic hidden state mechanism to alleviate the exposure bias during long-context
multimodal inference. Extensive experiments on multi-disease
brain MRI
dataset demonstrate that UniBrain achieves high performance for
brain image imputation, understanding, and disease diagnosis under various extents of modality incompleteness.
Read the full abstract ▾
All authors Zhiyun Song, Che Liu, Tian Xia, Avinash Kori, Wenjia Bai
Computer Vision
Uncertainty Quality of VGGT: An Analysis on the DTU Benchmark Dataset
Markus Hillemann, Robert Langendörfer, Steven Landgraf et al. (+1 more)
Visual Geometry Grounded
Transformer (VGGT) has already attracted a great deal of
attention in a short period of time, not least due to the Best Paper Award at CVPR-2025. Similar to
DUSt3R and MASt3R, VGGT aims to bring about a paradigm shift by replacing established methods like bundle adjustment and feature matching with a simple, unified, feed-forward neural network that predicts camera poses, depth maps, and dense 3D structure directly from multiple images of a scene in a few seconds. A key aspect is its ability to process an arbitrary number of views consistently in a single forward pass without any post-processing or iterative optimization. For photogrammetry, this opens new possibilities for real-time, scalable, and accessible 3D reconstruction. In this context, not only high reconstruction accuracy but also high-quality uncertainty estimates are crucial, as they foster trust and enable robust quality assurance. This paper therefore investigates the quality of VGGT's uncertainty predictions. The analysis identifies an effective confidence threshold for filtering VGGT's raw output and demonstrates that enhancing uncertainty quality holds strong potential for improving the accuracy of its 3D reconstructions.
Read the full abstract ▾
All authors Markus Hillemann, Robert Langendörfer, Steven Landgraf, Markus Ulrich
Computer Vision
AURA: Active-Response Attribution under Treatment Ambiguity in Bacterial Cytological Profiling
Kartik Jhawar, Mrunmayee Deshpande, Wilfried Moreira et al. (+2 more)
When a bacterial sample is exposed to several antibiotics, not every applied drug necessarily acts: if the organism is resistant to one of them, that drug leaves no morphological trace. The clinically meaningful quantity is therefore not which antibiotics were applied, but which ones were active. We show that these two are sharply decoupled in real E. coli microscopy - naively assuming the applied combination equals the active one is correct only about 37% of the time - yet existing computational tools are ill-suited to recovering the active set. Forward perturbation models such as scGen, CPA, and IMPA are designed to predict appearance from treatment, not the reverse, and inverting them degrades sharply; discriminative image classifiers tend to memorise strain- and batch-specific texture and fail to transfer across experimental replicates. We introduce AURA, which reframes the task as constrained, energy-based inverse attribution. Its central inductive bias is that the active set must be a subset of the applied set; this collapses the candidate space and lets AURA infer the active subset of applied antibiotics by decomposing residual morphology into antibiotic response atoms and selecting the subset with the lowest reconstruction energy, using no strain label at test time. AURA-E adds evidence-aware abstention, withholding a prediction when candidate explanations remain near-equally plausible. On cross-replicate transfer in an E. coli cytological profiling
dataset, AURA recovers the active antibiotic combination with 95.47% exact-match accuracy.
Read the full abstract ▾
All authors Kartik Jhawar, Mrunmayee Deshpande, Wilfried Moreira, Guillermo C. Bazan, Lipo Wang
Computer Vision
MVOFormer: Flow-Semantic Transformer for Robust Monocular Visual Odometry
Jituo Li, Shunwang Sun, Jialu Zhang et al. (+5 more)
Monocular visual odometry (MVO) is foundational to autonomous
navigation and robotic localization. However, existing learning-based MVO approaches often struggle with either a lack of interpretable, complementary features or overly complex multi-stage architectures. These limitations inherently restrict their robustness and cross-domain generalization. In this work, we propose MVOFormer, a novel
transformer framework for robust
monocular visual odometry. Our architecture features a Flow-Semantic Dual Branch Encoder that synergizes dense
geometric motion cues with object-centric semantic priors, explicitly distinguishing static structures from dynamic distractors. These representations are then fused by an Iterative
Multimodal Decoder, enabling coarse-to-fine pose refinement while dynamically suppressing
attention on unreliable regions. Extensive evaluations demonstrate that, without any target-domain fine-tuning, MVOFormer achieves superior zero-shot generalization and robustness, significantly outperforming prior learning-based frame-to-frame methods across diverse
benchmarks including TartanAir, KITTI, TUM-RGBD, and ETH3D-SLAM.
Read the full abstract ▾
All authors Jituo Li, Shunwang Sun, Jialu Zhang, Xinqi Liu, Jinyao Hu, Zhicheng Lu, Sajad Saeedi, Guodong Lu
Computer Vision
Decoupled Object-Centric Video Understanding for Generating Robotic Manipulation Commands
Thanh Nguyen Canh, Thanh-Tuan Tran, Haolan Zhang et al. (+3 more)
Translating
video demonstrations into executable
robot commands remains challenging because existing methods often fail to identify which objects are functionally involved in the demonstrated action. As a result, they may generate commands that are linguistically plausible but operationally ambiguous. We propose an object-centric
video understanding framework that decouples action recognition from object identification to generate precise, grammar-free manipulation commands. Our approach integrates Temporal Shift Modules (TSM) for efficient spatio-temporal action classification with a novel Object Selection
algorithm that identifies task-relevant objects through trajectory-based role classification, blur detection, and overlap minimization. The selected objects are then processed by
Vision-
Language Models (VLMs) for robust category recognition and zero-shot generalization. Evaluated on a modified Something-Something V2
dataset, our method achieves 86.79% action classification accuracy and BLEU-4 scores of 0.337 on standard objects and 0.261 on novel objects. These results improve over the strongest task-specific baseline by 80.2% and 143.9%, respectively. Larger gains are observed in METEOR and CIDEr, reaching 157.9% and 171.7% on novel objects. Across all semantic metrics, our approach consistently outperforms task-specific methods and remains competitive with, or surpasses, large general-purpose VLMs while retaining a modular, object-centric design.
Read the full abstract ▾
All authors Thanh Nguyen Canh, Thanh-Tuan Tran, Haolan Zhang, Ziyan Gao, Xiem HoangVan, Nak Young Chong
Computer Vision
ResEdit: Residual embeddings for precise generative image editing
Ahmet Canberk Baykal, Valentin Deschaintre, Yannick Hold-Geoffroy et al. (+4 more)
Conditional
diffusion image generators can be repurposed for editing through inversion, without the need for large-scale paired fine-tuning data. However, producing high-quality, targeted edits while maintaining image identity and global consistency remains challenging, as weakly conditioned inversion often embeds conflicting image features into the noise. We demonstrate that incorporating a residual image encoding as additional conditioning enables both improved identity preservation and better editability. We optimize this residual encoding to provide a strong conditioning signal for reconstruction, thereby reducing the reliance on inversion and susceptibility to its aforementioned pitfalls. To ensure this residual does not interfere with desired edits, we incorporate a gradient reversal-based optimization strategy that disentangles the residual from the edited condition. We illustrate our method's ability to produce high-fidelity results across precise intrinsic-based editing and relighting, and show proof-of-concept text-guided manipulation.
Read the full abstract ▾
All authors Ahmet Canberk Baykal, Valentin Deschaintre, Yannick Hold-Geoffroy, Michael Fischer, Anna Frühstück, Cengiz Öztireli, Iliyan Georgiev
Computer Vision
PermaVid: Consistent Video Generation Across Edits via Disentangled Context Memory
Shuai Yang, Bingjie Gao, Ziwei Liu et al. (+3 more)
Consistent
video generation under editing operations requires persistence: when edits modify scene appearance or layout, subsequent generations should remain coherent across time and viewpoints. However, existing
memory designs struggle to maintain long-term consistency after such modifications, as stored contexts may become outdated or invalid. To address this, we propose PermaVid, a novel framework built upon a multi-modal context
memory that disentangles spatial context into semantic appearance and
geometric structure, together with an edit-aware
memory update and
retrieval strategy that keeps
memory evolution aligned with subsequent observations. Specifically, we develop two complementary
memory banks: an RGB context
memory that captures appearance-aware observations while implicitly encoding geometry, and a depth context
memory that preserves geometry-only structure disentangled from semantics. Building on this design, we introduce a
memory-guided
video generation model that performs multi-modal feature fusion under reference conditions drawn from mixed-modality
memory contexts. Experiments demonstrate that our method maintains strong long-term semantic and structural consistency after edits, significantly outperforming state-of-the-art methods.
Read the full abstract ▾
All authors Shuai Yang, Bingjie Gao, Ziwei Liu, Jiaqi Wang, Dahua Lin, Tong Wu
Computer Vision
Hierarchical Fine-Grained Aerial Object Detection
Yan Zhang, Fang Xu, Wen Yang et al. (+1 more)
Fine-grained aerial object detection, driven by the intrinsic granularity of real-world object categories, is crucial for advanced scene understanding in remote sensing. Existing methods largely inherit the paradigm of coarse-grained object detection, relying solely on single-label supervision and thus struggling to distinguish model-level categories with subtle structural differences. However, for each specific model (e.g., Boeing 787), structured prior knowledge such as attributes and hierarchies offers discriminative semantics across multiple granularities. Motivated by this, we present ExpertDet, a scheme that incorporates expert-informed cues to enhance fine-grained aerial object detection. Specifically, we design
Vision-aware Masked Attribute Modeling (VMAM), which aligns attribute semantics with visual structures by reconstructing randomly masked attributes from visual cues, enabling the detector to capture subtle structural distinctions. We further propose Hierarchical Visual Instance Promotion (HierVIP), which builds a visual prototype tree based on hierarchical relations and imposes taxonomy-aware constraints to preserve cross-level semantic continuity while enhancing category discrimination. Moreover, we curate a new fine-grained object detection
benchmark for Precise recognition of model-specific Ships and Planes from aerial imagery, PSP, covering 106 ship classes and 30 airplane models, respectively, featuring the most extensive collection of model-specific categories among existing aerial object detection
datasets to date. We
benchmark state-of-the-art object detection
algorithms on the PSP
benchmark. Extensive evaluation demonstrates that ExpertDet consistently outperforms other fine-grained competitors across hierarchy levels. The
dataset,
benchmark, and code are available at https://nnnnerd.github.io/PSP-
Benchmark/.
Read the full abstract ▾
All authors Yan Zhang, Fang Xu, Wen Yang, Gui-Song Xia
Computer Vision · Cross-listed from cs.RO
V2P-Manip: Learning Dexterous Manipulation from Monocular Human Videos
Kaihan Chen, Yanming Shao, Haifeng Ji et al. (+2 more)
Achieving autonomous robotic dexterous manipulation requires precise, human-like action sequences at scale. As a scalable supplement to costly teleoperation data, extracting trajectories with both visual fidelity and physical plausibility from
monocular videos represents a promising frontier in
embodied AI. To this end, we introduce V2P-Manip, an efficient framework designed to learn dexterous manipulation policies directly from human demonstration
videos. We establish an efficient, integrated pipeline encompassing 3D asset acquisition, trajectory estimation, and dexterous policy learning. To bridge the gap between visual perception and physical constraints, we introduce a two-stage refinement process to enforce spatial
alignment and physical consistency. Evaluations on the TACO and OakInk
benchmarks demonstrate that our approach significantly outperforms previous methods in pose accuracy, adaptability to unstructured environments, and training efficiency. Ultimately, experimental results confirm an average success rate of over 75% across multiple synthetic manipulation tasks and validate the adaptability of the extracted manipulation priors across diverse dexterous hand embodiments.
Read the full abstract ▾
All authors Kaihan Chen, Yanming Shao, Haifeng Ji, Xiaokang Yang, Yao Mu
Computer Vision
Beer-Lambert Guided Representation Learning for Unsupervised Anomaly Detection in Sub-THz Food Inspection Images
Gyutae Hwang and Sang Jun Lee
Food manufacturing requires reliable inspection systems to detect foreign material contamination and maintain product safety. Sub-THz transmission imaging provides material-dependent attenuation characteristics that are useful for detecting low-density contaminants in food products. However, existing unsupervised anomaly detection methods mainly rely on RGB-pretrained visual representations, which may not adequately capture the transmission behavior of Sub-THz images. This paper proposes a Beer-Lambert guided representation learning framework for unsupervised anomaly detection in Sub-THz food inspection images. The proposed method introduces an attenuation decomposition module as an auxiliary regularization module that constrains student representations through attenuation reconstruction during training. In addition to the conventional one-class setting, we introduce a Leave-One-Food-Out protocol to evaluate generalization capability under unseen food categories. Experimental results on the Inline-Food-Inspection-THz
dataset show that the proposed method improves overall anomaly detection performance over the baseline method.
Read the full abstract ▾
Computer Vision
Instance-Aware Knowledge Distillation for Semi-Supervised Learning of an On-Board Multi-Task Dense Prediction Model for Collision Avoidance System
Gyutae Hwang and Sang Jun Lee
Collision avoidance systems have evolved toward camera-based deep learning approaches for driving scene understanding. However, deployment in edge environments such as country clubs is constrained by limited computational resources and unreliable communication infrastructure. Moreover, constructing large-scale
datasets for the target domain involves substantial annotation cost. To address these limitations, we propose an instance-aware knowledge
distillation framework for semi-supervised learning. Specifically, we generate pseudo labels that mitigate teacher bias by leveraging domain priors from the teacher and instance-centric knowledge from foundation models. The trained lightweight student is deployed in the proposed collision avoidance system and performs multiple dense prediction tasks in real-time. The system detects frontal obstacles and encodes their spatial information into controller area network messages for automated guided vehicle operation. To achieve this, we construct a large-scale country club
dataset and perform field validation of the proposed system. Experimental results demonstrate that the student outperforms the large teacher in instance
segmentation while mitigating performance degradation in
monocular depth estimation. Compared with the teacher, the student reduces FLOPs by 22.68$\times$ and parameters by 14.33$\times$, achieving 6.46 FPS on a low-cost edge device.
Read the full abstract ▾
Computer Vision
RGFVR: Reference-Guided Face Video Restoration with Flow Matching
Cem Eteke, Batuhan Tosun and Eckehard Steinbach
Face
video restoration from degraded observations is challenging, as it requires simultaneously recovering visual fidelity, temporal consistency, and subject identity. Existing approaches are often either reference-free, which can lead to identity loss when person-specific facial details are lost, or subject-specific, which limits generalization to unseen identities. We propose a subject-agnostic, reference-guided framework for identity-preserving face
video restoration. Our method introduces bimodal perceptual-descriptive identity conditioning into a pretrained flow-based text-to-
video generator and employs a two-stage training strategy to strengthen identity guidance during restoration. Experiments show that our approach improves restoration fidelity, temporal consistency, and identity preservation, achieving superior performance under challenging
video degradations, including downsampling, blur, noise, and compression artifacts. The code is available under: https://github.com/batuhanntosun/RG-FVR.
Read the full abstract ▾
Computer Vision
SP$^3$: Spherical Priors for Plug-and-Play Restoration
Sean Man, Ron Raphaeli, Matan Kleiner et al. (+1 more)
In this paper, we introduce SP$^3$, a novel Plug-and-Play
algorithm that accelerates maximum a posteriori image restoration by replacing denoisers with Spherical Encoders (SE) as generative priors. SP$^3$ approximates the intractable proximal prior step by utilizing the SE tightly structured latent space as a robust projection onto the natural image
manifold. Alternating this projection with a closed-form data-consistency step, via Half-Quadratic Splitting, achieves stable convergence without requiring gradient computation during inference. This unique formulation unlocks "anytime" restoration capabilities, producing sharp, plausible images from the first iteration. Evaluations across a variety of image restoration tasks demonstrate that SP$^3$ achieves perceptual quality comparable to state-of-the-art zero-shot
diffusion and flow methods while being $3$-$630\times$ faster.
Read the full abstract ▾
All authors Sean Man, Ron Raphaeli, Matan Kleiner, Or Ronai
Computer Vision
Towards UAV Image Dehazing: A UAV Atmospheric Scattering Model, Benchmark, and Geometry-Aware Deep Unfolding Network
Wenxuan Fang, Jiangwei Weng, Yu Zheng et al. (+5 more)
In
UAV applications, haze significantly obscures distant details and weaken structural information, hindering the recovery of details. Current
UAV scenarios still face two key challenges: (i) paired hazy/clean images from the real world are unobtainable, while the classical atmospheric scattering model is inadequate for modeling the spatially non-uniform haze in
UAV imagery; (ii) existing dehazing methods struggle to remove the heavy haze accumulated in the upper regions of
UAV images. To address these issues, we first propose a
UAV Atmospheric Scattering Model (UASM), which explicitly incorporates flight altitude, viewing pitch, and extinction to characterize the non-uniform haze distribution in
UAV imaging. Based on UASM, we develop a physics-driven dehazing framework, termed Geometry-aware Proximal Deep Unfolding Network (GP-DUN). Specifically, GP-DUN consists of three key modules: a Latent Geometry
Estimator (LGE) that infers transmittance consistent with
UAV imaging geometry, a Geometry-aware Gradient Descent Module (GeoGDM) that embeds UASM into the data-fidelity term and performs physics-consistent closed-form updates, and an Pooling-Expert Proximal Mapping Module (PE-PMM) that learns an implicit prior to restore textures and structures beyond the capability of explicit physical modeling. In addition, we further construct UASM-HazeSet, which provides controllable paired synthetic data together with 2,285 real
UAV haze images for testing. Extensive experiments show that GP-DUN consistently outperforms existing methods on both UASM-HazeSet and real
UAV haze
benchmarks.
Read the full abstract ▾
All authors Wenxuan Fang, Jiangwei Weng, Yu Zheng, Junkai Fan, Guangfa Wang, Xiang Chen, Jian Yang, Jun Li
Computer Vision
GraphBEV++: Multi-Modal Feature Alignment for Autonomous Driving
Ziying Song, Caiyan Jia, Lin Liu et al. (+3 more)
Feature misalignment in BEV perception is a critical yet often overlooked challenge in autonomous driving, especially under calibration uncertainties between
LiDAR and camera sensors. To address this issue, we propose a robust multi-modal fusion framework, GraphBEV++, which systematically mitigates projection-induced misalignment. The framework consists of two key modules: LocalAlign-v2 and GlobalAlign-v2. LocalAlign-v2 introduces neighborhood-aware depth features via graph matching to correct local misalignment. It supports both LSS-based and query-based BEV representations, making it compatible with BEVFusion and BEVFormer architectures for consistent cross-paradigm
alignment. GlobalAlign-v2 encompasses two variants: Deformable and
Diffusion. The Deformable variant addresses global misalignment in LSS-based multi-modal BEV by explicitly learning cross-modal feature offsets. In contrast, the
Diffusion variant targets implicit misalignment in query-based BEV by injecting noise to simulate misalignment and employing a denoising process to recover aligned features. Experimental results show that GraphBEV++ achieves state-of-the-art performance under misalignment noise on nuScenes and Waymo subset, improves long-range detection on Argoverse2, and generalizes effectively to the 3D occupancy prediction task, consistently improving occupancy estimation accuracy and robustness under both clean and noisy settings. Furthermore, GraphBEV++ effectively alleviates misalignment issues in end-to-end autonomous driving. Compared with five baselines (UniAD, VAD, FusionAD, MomAD, and WoTE), it demonstrates superior performance in both open-loop (nuScenes) and closed-loop (Bench2Drive and NAVSIM) evaluations across perception, prediction, and planning tasks.
Read the full abstract ▾
All authors Ziying Song, Caiyan Jia, Lin Liu, Shaoqing Xu, Lei Yang, Yadan Luo
Computer Vision
What Should a Streaming Video Model Remember?
Haonan Ge, Yiwei Wang, Hang Wu et al. (+1 more)
Streaming
video understanding models must answer queries at any moment during an ongoing stream, using only what they have observed so far and under fixed
memory and computation budgets. Existing methods address this by adding
memory banks,
retrieval modules, or visual
token compression to preserve long-range history. However, strong recent-window baselines show that indiscriminate history injection can dilute current-scene perception, suggesting that the key challenge is not whether to use
memory, but how to allocate it selectively. We formulate this as budgeted online latent evidence allocation and propose SelectStream, a selective latent-
memory framework that keeps the current observation directly visible to a frozen VLM while exposing historical information only through a compact, query-conditioned evidence budget. Three coordinated mechanisms govern when to write, what to preserve, and how to retrieve: surprise-driven adaptive windowing, priority-preserving consolidation, and query-conditioned graph
reasoning over a fixed-capacity latent
memory graph. Retrieved evidence is calibrated and injected as latent
tokens for answer generation, without replaying frames or growing the context with stream length. Experimental results show that SelectStream achieves strong online streaming performance and preserves general
video understanding, reaching 82.67% on StreamingBench, 67.03% on OVO-Bench, and 74.4% average accuracy on offline
video benchmarks, while outperforming strong recent-window baselines and prior streaming
memory methods.
Read the full abstract ▾
All authors Haonan Ge, Yiwei Wang, Hang Wu, Yujun Cai
Computer Vision
When the Past Matters: FlashBack Memory for Precipitation Nowcasting
Yuhao Du, Boxiao Huang, Chengrong Wu et al. (+1 more)
Accurate precipitation nowcasting is crucial for disaster mitigation and socio-economic planning, yet existing methods often struggle with false alarms, missed events, and long range dependency modeling at high spatiotemporal resolution. To address these challenges, we propose FlashBack
Memory (FB), a module that dynamically retrieves key historical states and integrates them via an adaptive fusion gate, enhancing the spatiotemporal representation capability of recurrent-based models. We incorporate FB into PredRNN, PredRNNpp, MIM, MotionRNN, and PredRNN-V2, and evaluate on CIKM2017, Shanghai2020, and SEVIR
datasets. Experimental results demonstrate that FB significantly improves MSE, MAE, SSIM, and CSI metrics, particularly for high-intensity rainfall and long-sequence predictions, while reducing false alarms and missed events and enhancing temporal consistency and spatial localization. The proposed method provides a general and efficient
memory enhancement mechanism, improving the overall performance of recurrent-based precipitation nowcasting models.
Read the full abstract ▾
All authors Yuhao Du, Boxiao Huang, Chengrong Wu, Jiankai Zhang
Computer Vision
Chronological Blindness: Benchmarking Temporal Reasoning in Vision-Language Models with CHRONOSIGHT
Parthaw Goswami and Jaynto Goswami Deep
Human perception of visual scenes is inherently temporal. We instinctively recognise whether a fruit is ripening or rotting, whether construction is progressing or being demolished, and approximately how much time separates two photographs of the same subject. Whether large
vision-
language models (VLMs) share this competence remains an open and practically important question. We introduce CHRONOSIGHT, a rigorously controlled
benchmark evaluating five dimensions of visual temporal
reasoning: CHRONORANK (chronological ordering of image sequences), CHRONOLOCATE (ordinal stage localisation from a single image), CHRONODELTA (estimation of time elapsed between two images on a logarithmic scale), CHRONOREVERSE (detection of temporally reversed sequences), and CHRONOODD (identification of a temporal outlier within a set). The
benchmark comprises 1{,}000 items across eight process families (biological growth, food transformation, physical weathering, construction, environmental change, human ageing, astronomical phenomena, and urban dynamics) spanning timescales from minutes to millennia. We evaluate eight open-source VLMs (500 M to 19 B parameters) under two prompting regimes and collect human performance baselines. Human performance averages 0.89 across tasks; the best open model (Qwen2.5-VL-7B) reaches 0.40 under direct prompting, a gap we term chronological blindness. Lightweight LoRA fine-tuning on 151 examples raises CHRONODELTA accuracy from near-zero to 0.43, transferring zero-shot to related tasks (CHRONOODD: 0.37; CHRONOREVERSE: 0.64)suggesting the bottleneck is partly instruction following rather than visual perception.
Benchmark, code, and predictions will be released upon acceptance.
Read the full abstract ▾
Computer Vision
Differentiable Packing of Irregular 3D Objects with Adaptive Container Estimation
Palak Gupta and Shanmuganathan Raman
Most existing approaches either fix the container in advance or optimize only a single container dimension through an outer search loop, leaving the remaining dimensions as a manual tuning problem. We present a differentiable packing framework that jointly optimizes all 6N object pose parameters and all three container side lengths inside a single gradient-based loop. The formulation combines six physics-inspired, differentiable loss terms computed directly on triangle meshes through axis-aligned bounding-box proxies. An adaptive squeezing mechanism periodically tightens the container whenever the overlap loss falls below a pair-count-scaled threshold, producing a large initial drop in container volume, followed by small refinements. All pairwise computations are written in tensor-broadcasting form, giving a 3.4 to 54 times speedup over a reference loop-based implementation. The pipeline is implemented in Python and PyTorch, with no physics engine, FFT library, or convex decomposition. On multiple object categories, the method produces containers that are 11 to 32 percent smaller than time-matched DBLF and simulated-annealing baselines at N =100, while running in under 4 minutes per instance on a single consumer GPU.
Read the full abstract ▾
Computer Vision
Attention-Based Prototype Calibration for Multi-Rater Few-Shot Medical Image Segmentation
Truong Vu, Minh Khoi Ho and Yutong Xie
Few-shot medical image
segmentation methods typically assume a single ground-truth annotation, overlooking systematic variability across expert raters commonly observed in clinical
datasets. We propose an
attention-based prototype calibration framework for few-shot multi-rater
segmentation that models rater-specific deviations from a consensus representation in prototype space. A lightweight yet principled
attention operator directly refines rater prototypes without modifying the backbone feature extractor, making the approach fully compatible with existing prototype-based few-shot
segmentation methods. This design preserves semantic consistency while enabling personalized
segmentation outputs with minimal computational overhead. Experiments on multi-rater medical imaging
datasets demonstrate consistent improvements over baseline prototype approaches, highlighting the effectiveness of structured prototype calibration for modeling annotation variability.
Read the full abstract ▾
Computer Vision
HAFMat: Hybrid Priors Guided Adaptive Fusion for Single-Image Human Material Estimation
Yu Jiang, Jiahao Xia, Jiongming Qin et al. (+2 more)
Physically based rendering (PBR) material estimation is a fundamental appearance decomposition task with broad applications in virtual content creation, relighting, and digital human rendering. However, estimating PBR materials from a single human image remains highly ill-posed, since illumination, geometry, and reflectance are heavily entangled in the observed appearance. To mitigate this ambiguity, we propose HAFMat, a hybrid-prior-guided framework for single-image human material estimation. Our method introduces guidance maps that encode complementary cues, including appearance, body geometry, structure, and prior material predictions from pre-trained models. A key observation is that these guidance cues are heterogeneous: some cues mainly provide texture-level constraints, while others convey higher-level semantic information. To exploit this property, we design a Multi-layer Adaptive Feature Fusion Mechanism, which adaptively fuses guidance features with decoder features at different stages. This design enables texture-dominant and semantic-dominant cues to guide material decoding at appropriate levels, leading to more accurate and physically plausible material estimation. Extensive experiments on both synthetic and real data demonstrate that our method achieves state-of-the-art performance in material estimation and downstream relighting.
Read the full abstract ▾
All authors Yu Jiang, Jiahao Xia, Jiongming Qin, Jianchi Sun, Chunxia Xiao
Computer Vision
Training-free sparse attention based on cumulative energy filtering
Chunlu Li, Yixuan Pan, Bai Du et al. (+5 more)
Sparse
attention accelerates
Diffusion Transformers (DiTs) for
video generation by computing only the important
tokens while skipping the rest. The
token selection strategy is key to balancing sparsity and accuracy. We formulate the
token filtering process as a dual-goal optimization problem: maximizing sparsity and minimizing accuracy degradation. Existing
algorithms cannot fulfill both objectives simultaneously. For example, Top-p only considers the accuracy constraint, while Top-k maintains a fixed computational budget but loosens the accuracy constraint. This paper demonstrates that maintaining a fixed recall rate is sufficient for ensuring accuracy, whereas a fixed threshold is suboptimal for reducing computational cost. Therefore, we propose a dynamic thresholding scheme to improve sparsity while maintaining the same level of accuracy. Furthermore, our
algorithm is deeply integrated with Flash
Attention (FA), eliminating the need for any additional masking computation overhead. Experimental results on Wan 2.2 validate that, compared to the BLASST
algorithm which is also integrated with FA, our dynamic thresholding strategy enhances sparsity from 61.42% to 82% with a VBench metric drop of less than 5%. This results in an approximate 15% in
attention computation and a $1.61\times$ increase in computational efficiency, which is 1.18x higher than that of BLASST.
Read the full abstract ▾
All authors Chunlu Li, Yixuan Pan, Bai Du, Zhenyuan Chen, Yanzhao Li, Hui Dong, Hui Wang, Zhiqiang Zou
Computer Vision
Explainable Flood Segmentation on Sentinel-1 SAR1 Imagery Using CNN and Transformer Architectures
Arundhuti Banerjee and David Daou
Rapid and accurate flood prediction is essential for disaster response and mitigation planning. Synthetic Aperture Radar (SAR) sensors in satellites are well-suited for this purpose because they operate independently of weather and daylight conditions. Although SAR-based data enable all-weather flood monitoring, distinguishing flooded land from permanent water remains a significant challenge, particularly when flooding is defined strictly as inundated land. This study provides a comprehensive comparison of convolutional neural network (CNN) and
vision transformer architectures for multi-class flood
segmentation using Sentinel-1 SAR imagery, specifically trained to separate flooded land from permanent water bodies and land. Three state-of-the-art (SOTA)CNN-based models, U-Net, U-Net++, and DeepLabV3 with ResNet-34 backbone, and three SegFormer variants (b0,b1,b2) were evaluated in two
benchmark datasets, the ETCI NASA
dataset and SenFloods11, using scene-based data splits to ensure a realistic assessment of spatial generalization. The results demonstrate that SegFormer-b2 significantly outperforms the U-Net baseline on the ETCI
dataset (higher flood IoU across all 7 test scenes in the Wilcoxon signed-rank test), while after fine-tuning on Sen1Floods11, the advantage narrows to within the range of scene variability and is concentrated in spatially fragmented flood events. The study includes both qualitative and quantitative explainability techniques to visually comprehend model decisions and systematically assess prediction reliability. Qualitative analysis reveals that SegFormer-b2 produces more spatially coherent Grad-CAM activations focused on flood-relevant features, while U-Net generates more informative uncertainty estimates along flood boundaries.
Read the full abstract ▾
Computer Vision
DDTNet: Degradation Disentanglement and Transfer Network for Test-Time All-in-One De-weathering Adaptation
Kuan-Hung Lin, Fu-Jen Tsai, Yan-Tsung Peng et al. (+3 more)
All-in-one adverse weather image restoration aims to remove multiple degradations, such as rain, haze, and snow, using a single unified model. Despite their broad applicability, existing methods typically compromise performance, delivering balanced but suboptimal results for individual degradation types. This issue becomes more pronounced when a domain gap exists between training and testing data. Motivated by the observation that modeling degradation patterns is more feasible than recovering clean content, we propose the Degradation Disentanglement and Transfer Network (DDTNet), which focuses specifically on degradation transfer. By disentangling degradation patterns from target-domain degraded images and transferring them to source domain clean images, DDTNet generates domain-adaptive paired training data. These pairs are then used to fine-tune restoration models, significantly enhancing their adaptability across diverse weather conditions and domains. The core of DDTNet is the Degradation Disentanglement Module (DDM), which comprises Degradation Coupled
Attention (DCA) to capture both general and weather-specific features, thereby enabling effective disentanglement and transfer of degradation patterns. Experimental results demonstrate that DDTNet significantly and consistently improves existing all-in-one models across real-world deraining, desnowing, and dehazing
datasets.
Read the full abstract ▾
All authors Kuan-Hung Lin, Fu-Jen Tsai, Yan-Tsung Peng, Min-Hung Chen, Chia-Wen Lin, Yen-Yu Lin