Computer Vision · Cross-listed from cs.MM
Estimating SSIM from MSE for DCT-Based Compressed Images via Modeling Local Error Statistics
Luc Trudeau and Maria G. Martini
Efficient and perceptually meaningful quality assessment is a fundamental requirement for image and
video processing, compression, and streaming systems. This article shows that, in the context of Discrete Cosine Transform ( DCT)-based compressed images, Structural Similarity Index ( SSIM ) can be approximated from global Peak Signal to Noise Ratio (PSNR) or Mean Square Error ( MSE) using local statistics derived only from the reference image. While prior work assumes access to local MSE, we propose two approaches to approximate local MSE by redistributing the global MSE using variance or standard-deviation-based weighting. Experiments on the Kodak and Xiph Subset1
datasets across a range of JPEG quality levels demonstrate that both approaches provide accurate and robust SSIM approximations, substantially outperforming the global MSE baseline. The proposed framework is designed to extend naturally to
video, where reference-derived statistics can be amortized across multiple encodes of the same content.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Abduction Without a Body? Representational Grounding and the Abduction Loop for Scientific Hypothesis Generation
Michael Farmer
Can scientific abduction occur without continuous sensorimotor embodiment? Recent arguments in AI and philosophy of science hold that genuine hypothesis generation requires an
agent continuously coupled to the physical world. We defend a narrower claim: online embodiment is not necessary for every abductive scientific act. Our focus is identity abduction: the inference that two independently developed structures are one object under an explicit correspondence, reached through representational grounding rather than bodily interaction. An
agent may acquire new inferential affordances not through physical interaction but through transformations into representations that expose latent invariants. Scientific diagrams are a practical substrate because they embody independently evolved conventions that partially canonicalize
symmetry,
topology, and operator structure across disciplines - a property we develop as convention space, which answers a hard
retrieval problem: finding mathematically related work when two fields share no discriminating vocabulary. We operationalize the mechanism as an architecture, the Abduction Loop: representation generation, motif extraction, convention-space canonicalization, cross-domain
retrieval, identity-hypothesis generation, and
adversarial verification, with abstention as the designed default. A documented episode, in which a
multimodal model given a figure of a gravitational-
memory transport model generated and then verified the hypothesis that its central differential complex is equivalent to the spherical Kaiser-Squires mass-mapping complex of weak-
lensing cosmology, serves as a motivating possibility witness from which the architecture is abstracted, not as evidence of general capability. We close with a falsifiable evaluation program, the DAB-30
benchmark. The contribution is a mechanistic proposal, an architecture, and a test program.
Read the full abstract ▾
Computer Vision
Jiayu Chen, Zhikun Jiang, Maoliang Li et al. (+6 more)
Video Diffusion Transformers (VDiTs) enable high-fidelity generation but incur quadratic cost from dense 3D self-
attention. Existing head- and block-level sparse methods share computation budgets across queries, overlooking
token-specific
attention demand. We observe that retained density varies across queries yet correlates log-linearly with
attention entropy, while dominant interactions form query-centered neighborhoods with
token-dependent radii. Based on these findings, we propose
Token Radius
Attention (TRA), a training-free framework that maps query
entropy to an analytic
token budget and converts it into a temporally decayed radius without explicit key ranking. Fused
entropy extraction, warm-up reuse, and block-sparse mask construction further reduce overhead. Across seven Wan2.1, Wan2.2, and HunyuanVideo T2V/I2V configurations, TRA retains only 9-19% of
attention interactions and achieves 1.56x-2.05x speedup with competitive generation quality. Code is available at https://github.com/IF-LAB-PKU/
Token-Radius-
Attention.
Read the full abstract ▾
All authors Jiayu Chen, Zhikun Jiang, Maoliang Li, Jiayi Luo, Jiawei Yang, Zihao Zheng, Hengyi Zhang, Guojie Luo, Xiang Chen
Computer Vision
DyFrDet: Towards Accurate Small Object Detection via Dynamic Frequency Suppression with Label Disambiguation
Zihan Yang, Yang Guo, Hongxing Zhang et al. (+2 more)
Despite the remarkable progress over the past decades, accurately identifying small objects remains challenging because of their insufficient visual cues. Previous works typically attempt to construct discriminative representation of the small objects. However, the wide range frequency domain noises and label ambiguities have been greatly overlooked, which significantly hinders the accurate localization. To address these issues, we propose a novel small object detection (SOD) detector termed DyFrDet, which is able to precisely localize the small object by dynamically suppressing the background distractions in frequency domain. Specifically, we propose a Dynamic Frequency-aware Feature Pyramid Network (DyFrFPN) to adaptively suppress low-frequency redundancy and excessive high-frequency noises. The DyFrFPN transforms the hierarchical features into frequency domain representation, and introduces a Dynamic Band Predictor (DBP) to preserve the discriminative components for small object identification. Afterwards, we present a novel Label Disambiguation Module (LDM), which leverages probabilistic distributions to explicitly model and alleviate the inherent ambiguity of target labels, yielding efficient improvement in localization precision of the small objects with low-resolution. Extensive experiments demonstrate that DyFrDet achieves state-of-the-art performance across multiple
benchmarks, indicating its effectiveness and robustness in various challenging scenarios. Our code is available at https://github.com/ManOfStory/DyFrDet.
Read the full abstract ▾
All authors Zihan Yang, Yang Guo, Hongxing Zhang, Dan Lu, Siyuan Yao
Computer Vision
Fermat Active Laplace Learning for Semi-Supervised Hyperspectral Image Classification
Vutichart Buranasiri and James M. Murphy
Two active learning
algorithms for hyperspectral image (HSI) classification are proposed that combine density-aware Fermat distances with Poisson-reweighted harmonic label propagation. Our methods actively query points using an uncertainty-based acquisition function, extending Poisson ReWeighted Laplace Learning (PWLL). Our first
algorithm, Fermat Active Laplace Learning (FALL), builds an affinity
matrix using Fermat distances between all data points. Then, PWLL is run with a diagonal perturbation using the minimum-norm acquisition function. In contrast, Approximate FALL (A-FALL) computes Fermat distances between each data point and landmark pixels selected via farthest-point sampling and constructs the affinity
matrix using landmark multidimensional scaling. After several query rounds, A-FALL selects the Fermat exponent $p$ using a leave-one-out cross-validation variant. FALL and A-FALL leverage Fermat distances and subsequent harmonic label propagation to provide a density-aware estimation of the data
manifold, improving labeling accuracy. Experiments on Salinas A and Pavia show the effectiveness of FALL and the scalability of A-FALL to large HSI scenes.
Read the full abstract ▾
Computer Vision
EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation
Jiayu Chen, Xiaoyu Wu, Rongshan Gao et al. (+6 more)
Audio-driven
video generation (A2V) has achieved promising progress in synthesizing temporally coherent and audio-visually aligned
videos, yet its inference remains expensive due to the iterative denoising process of
diffusion models. Existing caching methods mainly exploit temporal redundancy in visual features while overlooking the cross-modal
alignment of A2V, where audio drives visual generation with highly non-uniform temporal importance. In this paper, we identify two levels of misalignment in existing A2V caching methods: temporal-semantic and computation-storage misalignment. To address them, we propose EchoCache, an energy-guided cross-modal caching framework for efficient A2V generation. EchoCache leverages audio time-frequency energy as a saliency anchor to guide latent-level cache updates and further introduces a dynamic timestep-latent caching mechanism with quantized cache management for joint efficiency and
memory optimization. Extensive experiments on mainstream A2V models show that EchoCache consistently improves the latency-quality trade-off while preserving generation quality and audio-visual consistency. In particular, on Wan2.2-S2V over the EMTD
benchmark, EchoCache achieves a 2.46x speedup with the best overall performance. Code is available at https://github.com/IF-LAB-PKU/EchoCache.
Read the full abstract ▾
All authors Jiayu Chen, Xiaoyu Wu, Rongshan Gao, Maoliang Li, Zihao Zheng, Xinhao Sun, Hailong Zou, Guojie Luo, Xiang Chen
Computer Vision
Action-grounded tissue affordance enables anticipatory auto-framing that lowers surgeon cognitive workload during laparoscopic surgery
Jiayu Gu, Yiwei Wang, Jie Zhang et al. (+17 more)
In laparoscopy, surgeon gaze tracks where the instruments will act; easing this demand through visual
attention modeling requires dense labels of those interaction loci. These encode tacit knowledge: experts converge on consensus loci yet struggle to state the rules. Here we show that such labels can be recovered from completed actions in
surgical videos, in which recorded instrument trajectories are converted into dense, continuous supervision. DiffeoAfford grounds tissue affordance by attaching instrument tips to the tissue and transporting them through deformation using diffeomorphism-constrained tracking, matching context-informed annotators' accuracy. Trained on these labels and never on gaze, a real-time model aligns with surgeon gaze more closely in space and time than does camera-assistant gaze. The framework also transfers across procedures: on hysterectomy
videos, a separately trained predictor reaches 95.16% directional consistency with subsequent camera motion. In 12 paired cholecystectomies (24 procedures), the auto-framing application AffordView, which proactively centers predicted targets in view, lowered surgeon cognitive workload on converging subjective, physiological, and behavioral measures, including a reduced number of verbal instructions to the camera assistant. Deriving supervision from action rather than manual annotation offers a scalable route to anticipatory assistance.
Read the full abstract ▾
All authors Jiayu Gu, Yiwei Wang, Jie Zhang, Guojun Cao, Keshen Lyu, Song Zhou, Yimeng Chen, Haorui Wang, Qingmin Feng, Shenchao Shi, Hongkuan Shi, Qiuyu Yu, Qiang Xie, Huan Zhao, Wenbin Chen, Caihua Xiong, Chidan Wan, Jing Samantha Pan, Xiong Cai, Han Ding
Computer Vision
Grounding Agentic VLMs with Dedicated Segmentation for Fine-Grained Vehicle Damage Assessment
Vishwajeet Shivaji Hogale, Anjali Pai and Nitya Ravi
Vision-
language models (VLMs) are increasingly deployed as
reasoning agents in real-world visual assessment pipelines, yet their spatial grounding remains unreliable for fine-grained, visually ambiguous targets. We study this gap in the context of automated vehicle damage assessment, where fine-grained defects such as scratches and hairline cracks occupy few pixels, produce weak gradient signal, and are easily confused with reflections and surface texture. We show that a state-of-the-art VLM (Qwen-VL) achieves strong semantic classification accuracy (87.3%) on this task but is systematically ungrounded at the spatial level: it hallucinates damage in reflective regions, misses elongated scratches entirely, and produces spatially inconsistent outputs when prompted for localization. We propose TinyDamage, a hybrid architecture that delegates spatial grounding to a dedicated multi-task
segmentation model while reserving the VLM for semantic
reasoning and report generation. On the
segmentation side, we find that the choice of loss function has an outsized and underexplored effect on tiny-object grounding: focal loss, widely used for class imbalance, collapses tiny-damage detection to zero, while a supervised contrastive objective measurably improves damage/background separability. We integrate the
segmentation model into a 7-node LangGraph
agent pipeline that grounds every VLM generation step in the
segmentation output, and show that this grounding reduces the report
hallucination rate from 92% (text-only) and 78% (image-only) to 31% in a controlled evaluation on 100 human-verified reports. We introduce DET_l, a permissive per-category detection metric for evaluating tiny-object grounding under class imbalance, and report latency and reliability characteristics of the deployed pipeline.
Read the full abstract ▾
Computer Vision
Calibrated Similarity and Graph Clustering for Open-Set Animal Re-Identification
Mohamed ElBassat, Seifeldin Elkerdany, Mohamed ElBialy et al. (+5 more)
AnimalCLEF26 addresses discovery-oriented animal re-identification, where systems must both attach query images to known individuals and discover unseen individuals by clustering them correctly. We present a similarity-to-clustering pipeline for this setting across Eurasian lynx, fire salamander, loggerhead sea turtle, and Texas horned lizard images. The method first isolates the target specimen using
segmentation and then applies lightweight species-specific preprocessing for lynx, sea turtle, and salamander images to enhance identity-relevant visual cues, while Texas horned lizard images are used after
segmentation only. Pairwise similarities are then estimated with WildFusion by calibrating and combining a MiewID global descriptor with two local matching branches, ALIKED + LightGlue and DISK + LightGlue. The resulting query-query similarities are refined and converted into identity clusters using graph-based clustering, while query-database similarities are used to attach confident samples to known identities. We evaluate training-free and fine-tuned MiewID variants, including Dynamic ArcFace and SphereFace2-Focal adaptations, and combine them in the final ensemble. Our selected ensemble substantially improves on the WildFusion baseline, achieving the best public ARI of 0.72124 and a private ARI of 0.70393, while a simpler preprocessing-before-calibration variant achieves the best private ARI of 0.71087. These results indicate that calibrated global-local fusion with species-aware preprocessing choices is effective for open-set wildlife re-identification under challenging field conditions and visual variation. The implementation code is available on GitHub.
Read the full abstract ▾
All authors Mohamed ElBassat, Seifeldin Elkerdany, Mohamed ElBialy, Gamal Abouelhamd, Jana Ghoneim, Assem Elkady, Mohamed Elboraay, Nelly Semenova
Computer Vision
ISRS-DETR: Detection-Guided Click Propagation for Remote Sensing Interactive Segmentation
Thanh Duc Pham, Anh Nguyen, Duong Duc Hieu et al. (+1 more)
Interactive
segmentation reduces the prohibitive cost of pixel-level annotation by allowing users to delineate objects with a few clicks. However, applying this paradigm directly to remote sensing imagery is non-trivial: ultra-high resolutions, small object sizes, and sparse spatial distributions all degrade
segmentation quality. Recent work has addressed the resolution barrier and achieved competitive results in interactive
segmentation for remote sensing (ISRS). However, they treat all instances of a class within an image as a single objective target. Consequently, interactions spent on one object contribute nothing to its same-class neighbours, and satisfactory masks may demand up to 40 clicks per image, hindering the practicality of these frameworks. We observe that remote sensing scenes exhibit markedly strong inter-object correlation, meaning a single clicked object is highly informative about the rest of its category. Building on this, we propose ISRS-DETR, a detection-guided interactive
segmentation framework that injects object-level evidence into both training and inference. Our ISRS-DETR employs an RF-DETR decoder with the interactive
segmentation backbone to localise co-occurring same-class objects, and introduces a Dynamic Top-K Click Selection strategy that retains only reliable proposals and converts each into a simulated click, so one user interaction propagates across an entire class. Experiments on three standard remote sensing
benchmarks show that ISRS-DETR achieves state-of-the-art accuracy while substantially reducing Number of Clicks per Image (NoC-I). All codes and data splits will be released for reproducibility upon acceptance.
Read the full abstract ▾
All authors Thanh Duc Pham, Anh Nguyen, Duong Duc Hieu, Minh-Tan Pham
Computer Vision
MoRAL: Sensor-Grounded BEV Reasoning for Compact VLMs toward Edge-Oriented Autonomous Driving
Ambarish Govindarajulu Kaliamurthi and Kaikai Liu
Deploying
vision-
language models (VLMs) for safety-critical spatial
reasoning on resource-constrained autonomous driving platforms requires both compact model size and reliable metric grounding. We present MoRAL (
Multimodal Reasoning for Autonomous
Language Models), a two-stage fine-tuning pipeline that teaches Cosmos-Reason2-2B to first read a physics-encoded Bird's Eye View (BEV) representation and then reason over it for driving decisions. The BEV image encodes
LiDAR metric distance as color bands, object class as cluster morphology, and radar Doppler velocity as directional wedge overlays, externalizing spatial perception into the input image so that no learned 3D backbone is required at inference. Stage 1 fine-tunes the
vision encoder on 60,000 grounding records; zero-shot baselines produce no parseable BEV outputs, confirming the vocabulary requires explicit training. Stage 2 fine-tunes the full model (52M parameters, 2.4% of total) on 57,696 chain-of-thought records generated by Cosmos-Reason2-8B as teacher, spanning eight driving question types. On 2,304 held-out nuScenes frames evaluated by Gemma 4 (31B) calibrated against human review, MoRAL wins seven of eight question types over a zero-shot 8B baseline despite using four times fewer parameters, with the largest margins on question types requiring structured multi-step physics
reasoning. Emergency braking recall improves from 10.8% to 47.8%, output degeneration falls from 94.1% to 20.8%, and the full pipeline fits a consumer 8 GB GPU at 42 tok/s without quantization. These results establish a reproducible foundation for compact, physics-grounded VLM
reasoning on mobile edge platforms.
Read the full abstract ▾
Computer Vision
UAV-Based Environmental Monitoring of Rip-Current Indicators Using Wavelet-Derived Texture Features
Yonatan Ben Avraham, Baruch Binyaminov and Yehudit Aperstein
Rip currents are recurrent coastal natural hazards that threaten beachgoers and create operational challenges for lifeguards and coastal managers. Reliable monitoring from standard RGB (red-green-blue) imagery acquired by unmanned aerial vehicles (
UAVs) remains difficult because hazardous channels often appear as subtle gaps in breaking waves, foam texture, or sediment patterns, and these signatures are affected by illumination, sea state, and environmental noise. This study presents a physically informed coastal environmental monitoring workflow for detecting visually expressed rip-current indicators that integrates wavelet-derived spatial-frequency texture features with deep learning. We evaluate multiple strategies for incorporating Discrete Wavelet Transform features into convolutional architectures, from computationally efficient channel replacement to dual-stream fusion with
attention mechanisms. Performance is assessed against a standard RGB baseline using a task specific convolutional neural network for image-level presence classification and a YOLOv8 model for object-level localization. Under the evaluated
dataset conditions, integrating wavelet derived texture features improves performance over RGB-only models. The dual-stream architecture achieves the strongest classification performance, exceeding 95% accuracy with high recall, while channel replacement is most effective for YOLOv8 object detection, reaching 94% mAP@50 for localization. Explainable artificial intelligence analyses provide qualitative evidence that the models attend to visually plausible wave-gap regions associated with rip currents. These results suggest that under the conditions of the evaluated
dataset, physically informed wavelet integration may support
UAV-based decision-support tools for interpretable beach-safety risk mitigation.
Read the full abstract ▾
Computer Vision
InfiniSplat: Implicit Gaussian Decoding for Large-Baseline Monocular View Synthesis
Jiawei Wang, Hao Yu, Yongzhen Hu et al. (+7 more)
Single-image feed-forward 3D Gaussian Splatting (3DGS) aims to directly generate a renderable 3D scene representation from one input image, avoiding the cost of multi-view capture and per-scene optimization. However, existing methods are often constrained by a pixel-aligned representation, where Gaussians are predicted from fixed image-grid locations. Such pixel-aligned primitives can produce promising nearby-view renderings, but they remain weakly coupled to underlying scene surfaces and struggle to preserve coherent structures under large viewpoint shifts. We present InfiniSplat, a feed-forward single-image 3DGS framework that moves from a pixel-aligned representation toward a surface-aligned representation. InfiniSplat constructs this representation by first using geometry-guided sampling to place 2D supports according to depth-induced local surface structure, and then applying a query-conditioned implicit decoder to predict Gaussian attributes from the image features queried at these supports. By grounding support locations in geometry while decoupling Gaussian prediction from fixed pixel centers, InfiniSplat produces Gaussian layouts that better follow scene surfaces and reduce scattered primitives caused by grid discretization. Across multiple cross-
dataset NVS evaluations, InfiniSplat achieves state-of-the-art performance compared with single-image feed-forward baselines, and demonstrates zero-shot generalization from Hypersim indoor synthetic training to complex open-world scenes. Project page: https://zju3dv.github.io/InfiniSplat.
Read the full abstract ▾
All authors Jiawei Wang, Hao Yu, Yongzhen Hu, Xinyi Yang, Tao Ni, Xin Zhan, Junbo Chen, Xiaowei Zhou, Ruizhen Hu, Sida Peng
Computer Vision
Learning to Tessellate: Point Cloud Generation via Recursive Spectral Partitioning
Monan Sun, Bangzhen Liu, Huaidong Zhang et al. (+1 more)
Autoregressive models have emerged as an effective paradigm for point cloud generation. However, most existing approaches rely on heuristic tokenization strategies, such as spatial sorting or
stochastic downsampling, which often disrupt intrinsic point cloud
topology and weaken the structural coherence of the generated shapes. In this paper, we present PointRSP, an autoregressive framework that reformulates point cloud generation as a
topology-preserving tessellation process via recursive spectral partitioning. Instead of constructing
token sequences heuristically, we introduce a
topology-aware partitioning autoencoder that decomposes an unstructured point cloud into a non-balanced binary tree through a hybrid recursive spectral partitioning strategy. This hierarchical representation provides a deterministic
geometric blueprint that preserves topological relationships while capturing multiscale structural dependencies within a quantized latent space. To synthesize shapes in this space, we propose a dual-stream cascaded generator that jointly models structural
evolution and feature synthesis. In addition, we design a geometry-calibrated positional encoding mechanism that anchors latent
embeddings using multi-scale structural centers, which stabilizes cascaded generation during the early stages of structural formation. Extensive experiments show that PointRSP achieves state-of-the-art performance in generation quality and diversity, demonstrating strong generalization across complex 3D
topologies.
Read the full abstract ▾
All authors Monan Sun, Bangzhen Liu, Huaidong Zhang, Shengfeng He
Computer Vision
DF$^3$: World Modeling via Decoder-Free Feature Forecasting in Autonomous Navigation
Jiaming Chen, Guoan Xu, Aoshen Huang et al. (+3 more)
Forecasting future states from
video sequences is a critical challenge for autonomous robotic systems and a fundamental objective of world modeling. Prior generative methods operating at the pixel level inevitably overemphasize task-irrelevant details, leading to prohibitive computational overhead. While latent-based approaches attempt to mitigate this by predicting features directly, the persistent reliance on heavy decoders for state-to-task mapping remains a computational bottleneck. In this work, we propose Decoder-Free Feature
Forecasting (DF$^3$), a novel framework that models world
evolution entirely within the latent space and directly derives task outputs, completely eliminating the need for a decoder. Specifically, DF$^3$ injects learnable spatial queries into the terminal blocks of a frozen
vision foundation model to extract future state representations directly. By employing a lightweight, unified Motion-Aware Context Fusion (MACF) mechanism that seamlessly integrates coarse flow warping with fine-grained latent cross-correlation, these queries interact with historical
token representations to explicitly align and forecast the feature of the next frame. Subsequently, a specialized set of task queries probes these forecasted features for the downstream task. Extensive experiments on public
benchmarks and zero-shot deployment in a robotic simulator demonstrate that DF$^3$ achieves performance comparable to state-of-the-art methods while offering superior efficiency and flexibility for integrated perception and control.
Read the full abstract ▾
All authors Jiaming Chen, Guoan Xu, Aoshen Huang, Haozhuo Zhang, Yang Li, Wei Pan
Computer Vision
Loggia dei Lanzi: AI Thermography Enhancement Comparisons through 3D Photogrammetry
Scott McAvoy, Jonathan Klingspon, George Bent et al. (+4 more)
The Loggia dei Lanzi in the Piazza della Signoria is one of Florence's most prominent structures visited by millions every year. Its construction history spans multiple centuries of modification. This paper presents the results of a thermal imaging campaign conducted in December 2025, using a FLIR T1020 HD camera, revealing hidden architectural features including walled-up openings and material transitions beneath the plaster surface. The favorable winter ambient conditions provided a feature-rich
benchmark upon which to compare the results of enhancement
algorithms and artificial intelligence models. We evaluate the application of AI-based image enhancement to thermal heritage documentation through a comparison of three tiers of image resolution in a photogrammetric Structure-from-Motion (SfM) pipeline: native resolution, FLIR's hardware-based pixel-shifted super-resolution (UltraMax), and state of the art AI-upscaled imagery models. We quantify the effect of each resolution tier on feature detection and tie-point generation, assessing whether the additional detail produced by super-resolution, whether hardware or AI-derived, translates into meaningfully denser and more accurate 3D thermal models. Our results contribute to the emerging intersection of artificial intelligence and heritage thermography by providing a direct comparison of hardware microscanning and AI super-resolution within a thermal photogrammetric workflow for cultural heritage. All
datasets are made publicly available and accessible within an interactive 3D archival framework, and integrated into a custom citywide extended reality overlay application.
Read the full abstract ▾
All authors Scott McAvoy, Jonathan Klingspon, George Bent, Dave Pfaff, Aviral Agarwal, Maurizio Seracini, Falko Kuester
Computer Vision
USP-Mamba: Unmixing-Derived Spectral and Structural Prompting for Hyperspectral Image Super-Resolution
Shi Chen, Jie Zhang and Yicong Zhou
Hyperspectral image super-resolution aims to reconstruct high-resolution imagery while preserving dense spectral information. Recently, Mamba-based models have shown promising potential for this task by capturing long-range dependencies with linear computational complexity. Nevertheless, their causal sequence modeling requires two-dimensional hyperspectral features to be unfolded along predefined scanning orders, which disrupts spatial adjacency and restricts the effective propagation of contextual information. Moreover, state-space parameterization of existing models is predominantly derived from generic learned representations, without explicit
alignment with the intrinsic characteristics of the hyperspectral image. To address this issue, we propose an Unmixing-derived Spectral and Structural Prompting Mamba framework, termed USP-Mamba, which adapts Mamba state
evolution through composition-aware spectral priors and image-dependent structural
prompts. Specifically, an unmixing-informed spectral
prompt captures the global material composition of the input image and provides persistent conditioning throughout reconstruction. Injected into the Mamba sequence and progressively adapted across layers, it steers state
evolution toward composition-consistent reconstruction. We introduce feature-level structural
prompts comprising spatial and frequency components to provide image-dependent local guidance. The spatial
prompt promotes structure-sensitive state encoding for local detail preservation, while the frequency
prompt enables region-adaptive transitions between homogeneous regions and high-frequency details. Finally, complementary
Hilbert and Semantic-Guided Neighboring scans preserve spatial continuity and strengthen non-local semantic dependency modeling. Extensive experiments on different
datasets demonstrate that the proposed method consistently outperforms representative approaches.
Read the full abstract ▾
Computer Vision
Does Explainability Transfer? A Controlled Benchmark of Attribution Methods on Vision Transformers and CNNs
Sathiyamohan Nishankar, Nethmi Pathirana, Pubudu Sanjeewani et al. (+2 more)
Most evidence on the effectiveness of explainable artificial intelligence (XAI) attribution methods has been established on convolutional neural networks (CNNs), with limited investigation into whether these conclusions generalize to the diverse
Vision Transformer (ViT) architectures that now dominate computer
vision. This paper presents a controlled
benchmark that evaluates attribution quality across five dimensions: faithfulness, localization, robustness, complexity, and computational cost. A standardized framework assesses 13 attribution methods from four algorithmic families on eight representative backbones spanning CNNs, isotropic ViTs, hierarchical
transformers, hybrid architectures, and linear-
attention transformers. The results show that attribution performance is strongly architecture-dependent and that rankings established on CNNs do not reliably transfer to
transformer-based models. CAM-based methods achieve the highest scores under the conventional bounding-box localization metric on CNNs and most ViTs but perform poorly on linear-
attention architectures. Pixel-level dense-mask evaluation further reveals that these gains largely reflect metric saturation rather than accurate localization. CAM-based methods also exhibit limited robustness on global-
attention transformers, whereas
attention rollout provides consistently stable explanations with poor localization. Furthermore, faithfulness correlation offers limited discrimination between attribution methods, highlighting the limitations of single-metric evaluation. These findings challenge prevailing conclusions on attribution performance and demonstrate the need for architecture-aware, multi-dimensional evaluation. The open-source code for the evaluation framework and
benchmark results is available at https://github.com/Nishan-Charlie/VIT_XAI_Bench.
Read the full abstract ▾
All authors Sathiyamohan Nishankar, Nethmi Pathirana, Pubudu Sanjeewani, Asanka Perera, Selvarajah Thuseethan
Computer Vision
GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experience
Sitong Gong, Caixin Kang, Tianyu Yan et al. (+7 more)
A wearable assistant should both answer questions about its visual history and recognize when that history is useful to the present situation. Existing
video-
memory systems primarily support question-conditioned recall, whereas proactive assistants typically use separate
memory and control mechanisms. We introduce GROVE, a training-free framework that supports both behaviors with one
memory grown causally from a continuous
video stream. GROVE retains fine-grained perceptual evidence and incrementally consolidates it into time-stamped moments, coherent episodes, and recurring cross-day patterns. Each stratum is paired with a scale-native
retrieval skill for locating an observation, replaying an activity, or traversing long-range regularities. Reactive QA and proactive assistance share this
memory and access interface, differing in whether
retrieval is initiated by a user query or the current situation. Across multiple
benchmarks including the challenging MM-lifelong and EgoServe, GROVE achieves the best results among the compared methods. Controlled ablations show that the temporal strata and their access skills are complementary, with patterns providing the largest benefit when evidence spans multiple days. Code will be available at https://github.com/SitongGong/GROVE.
Read the full abstract ▾
All authors Sitong Gong, Caixin Kang, Tianyu Yan, Guo Chen, Bo Zheng, Kaipeng Zhang, Yunzhi Zhuge, Xiang Ruan, Huchuan Lu, Yifei Huang
Computer Vision
Loop-Mamba: A Loop Mamba with Degradation-Aware and Shared Memory for Old Photo Restoration
Runci Bai, Yucheng Xin, Pu Wang et al. (+8 more)
Old photographs often suffer from multiple coupled degradations, including scratches, cracks, fading, blur, noise, and missing regions, severely degrading both visual quality and semantic content. We propose Loop-Mamba, a lightweight loop-based state-space framework that formulates old photo restoration as progressive state
evolution, where a persis- tent restoration state is continuously propagated and refined through iterative computation. Specifically, we introduce a Semantic-Guided Degradation
Estimator (SGDE) to explicitly model heterogeneous degradations by jointly predicting local degradation maps and global degradation scores, providing degradation-aware guidance for state
evolution. We further develop a Shared Structural
Memory Mamba (S$^2$M- Mamba), which maintains a persistent restoration state across iterations, enabling persistent state
evolution through shared structural
memory for robust long-range structural reconstruction. Benefiting from first-order state recursion, Loop-Mamba propagates latent restoration states through recurrent tran- sitions instead of repeatedly stacking deep feature transformations, thereby alleviating gradient dilution while avoiding the computational overhead inherent in iterative CNN- and
Transformer-based restoration frameworks. A lightweight multi-directional scanning strategy further enhances direc- tional information aggregation and preserves structural continuity. To better evaluate restoration quality, we introduce the task-oriented Old Photo Damage Recovery Score (ODRS), which jointly measures degradation recovery and structural reconstruction fidelity. Experimental results on the public SynOld
benchmark demonstrate that Loop-Mamba consistently outperforms previous state-of-the-art methods across both conventional restoration metrics and the proposed ODRS.
Read the full abstract ▾
All authors Runci Bai, Yucheng Xin, Pu Wang, Yongcong Wang, Chen Wu, Dianjie Lu, Guijuan Zhang, Pengwen Dai, Guangwei Gao, Siyuan Yao, Zhuoran Zheng
Computer Vision
Context-Aware Mixture of Domain Experts for Bodily Expression of Emotion in the Wild
Mohammad Mahdi Dehshibi and David Masip
The same body posture can convey entirely different
emotions depending on its surrounding context, yet most methods for recognising bodily
emotions treat scene and object cues as auxiliary feature augmentations rather than as structured priors over the plausibility of
emotions. We introduce the Context-Aware Mixture of Domain Experts (CA-MoDE) for bodily
emotion recognition. CA-MoDE incorporates dedicated scene and object experts to generate soft distributions over
emotion categories conditioned on their respective domains. These domain-conditioned soft predictions serve as structured contextual priors that modulate the body expert's predictions at the distributional level rather than at the feature level. To fuse these multi-domain signals, we propose a task-tailored max-endorsement gating strategy that selects the strongest contextual signal across experts for each
emotion dimension. Our gating strategy mitigates the signal dilution that typically occurs when conflicting or uninformative context distributions are averaged. CA-MoDE achieves an
Emotion Recognition Score of 0.3269 on the Body
Language Database. By outperforming existing temporal models using only single still images, our framework demonstrates that explicitly modelling structured spatial context can serve as a complementary discriminative proxy for the behavioural dynamics typically captured by
video.
Read the full abstract ▾
Computer Vision
Implicit Neural Representations for Multimodal Longitudinal Image Imputation and Interpolation
Sina Wendrich, Lukas Förner, Zoe Reinke et al. (+5 more)
Longitudinal multiparametric MRI is central to follow-up imaging in oncology, yet real-world clinical data are characterised by missing sequences, heterogeneous acquisition protocols, and varying spatial resolutions across time points. We propose a patient-specific conditional implicit neural representation (INR) that models
multimodal longitudinal MRI as a continuous function of world coordinates, time, and modality conditioning. The model is trained with
stochastic modality dropout to handle incomplete data, and its continuous coordinate-space formulation enables both spatial and temporal interpolation without resampling to a fixed voxel grid. A self-consistency-based confidence
estimator is derived from cross-modal reconstruction performance at inference time. We evaluate the framework on longitudinal MRI from paediatric
brain tumour patients, demonstrating statistically significant improvements over linear interpolation for T1CE and FLAIR (p < 0.05), with mean MS-SSIM of 0.95 $\pm$ 0.02 for T1CE. Predicted confidence correlates strongly with true reconstruction quality (Pearson r up to 0.996), suggesting reliable deployment potential in heterogeneous clinical settings.
Read the full abstract ▾
All authors Sina Wendrich, Lukas Förner, Zoe Reinke, Kartikay Tehlan, Ansgar Berlis, Michael Frühwald, Matthias Wagner, Thomas Wendler
Computer Vision
Global-Scale Self-Supervised Spatiotemporal Learning for NDVI Time-Series Reconstruction
Ang Li, Menghui Jiang, Xiaobin Guan et al. (+2 more)
Accurate and efficient reconstruction of cloud-contaminated and noise-corrupted NDVI time series remains a challenge in remote sensing. Deep learning provides a promising solution for modeling complex spatiotemporal dependencies; however, its application is often limited by the difficulty of obtaining paired clear-sky and degraded NDVI data for identical spatiotemporal locations. To address this issue, we propose GloSSR, a Global-scale Self-supervised Spatiotemporal framework for NDVI Reconstruction. The framework constructs supervisory signals by artificially degrading relatively clean NDVI observations with realistic cloud contamination patterns, producing self-supervised training pairs that closely mimic real-world degradation. It further introduces an end-to-end spatiotemporal learning network that jointly captures long-range temporal dependencies and short-term spatiotemporal correlation through a bidirectional
Transformer with a ConvLSTM architecture. A temporal-channel
attention-based reconstruction module is incorporated to enhance informative features, while a spatiotemporal prior constraint is designed to preserve both fine-scale structures and long-term phenological trends during optimization. Extensive evaluations on MODIS NDVI data demonstrate the effectiveness of the proposed framework across both artificial and real-world scenarios. In artificial degraded-pixel reconstruction experiments, GloSSR consistently outperforms the comparison methods. Time-series analyses based on real observations further demonstrate that the proposed framework can accurately characterize vegetation dynamics and capture the key phenological states. Long-term vegetation trend analysis and the transferability analysis to AVHRR data validate the scalability of the framework and illustrate its broad applicability for large-scale environmental monitoring.
Read the full abstract ▾
All authors Ang Li, Menghui Jiang, Xiaobin Guan, Dong Chu, Huanfeng Shen
Computer Vision · Cross-listed from cs.RO
TravKAN: Fast and Interpretable Nonlinear Traversability Analysis with Kolmogorov-Arnold Networks
Daniel Fusaro, Simone Mosco, Wanmeng Li et al. (+1 more)
Traversability analysis is a fundamental capability for autonomous mobile
robots operating in unstructured environments. While modern machine learning approaches such as deep neural networks and gradient-boosted trees achieve strong predictive performance, they lack interpretability and provide limited insight into the underlying terrain-
robot interaction dynamics. In this paper, we propose TravKAN, a Kolmogorov-Arnold Network-based framework for fast, scalable, and interpretable traversability estimation. TravKAN represents multivariate decision functions through compositions of learnable univariate functions, enabling compact architectures and symbolic extraction of analytic expressions after training. In addition, we introduce a novel set of handcrafted features derived from the reflectivity channel of
LiDAR sensors. To the best of our knowledge, reflectivity has not been systematically exploited for handcrafted traversability descriptors, despite its potential to capture material and surface properties complementary to
geometric cues. We evaluate TravKAN on public, real-world urban and off-road
datasets and compare it against strong baselines. TravKAN achieves strong performance across all metrics, outperforming conventional deep models and approaching the performance of XGBoost. TravKAN-Lite, i.e., TravKAN's symbolic representation, reveals meaningful nonlinear feature interactions and provides a compact, deployment-friendly, and fast analytic model. Ablation studies further show the robustness of our method to architectural variations and quantify the contribution of the proposed reflectivity-based features. These properties make TravKAN attractive for robotic systems requiring transparency, real-time computational efficiency, and interpretability in safety-critical decision-making.
Read the full abstract ▾
All authors Daniel Fusaro, Simone Mosco, Wanmeng Li, Alberto Pretto
Computer Vision
Gaetano Chiriaco, Luca Barco, Andrea Bragagnolo et al. (+2 more)
Geospatial foundation models aim to learn representations that transfer across regions and sensors, yet evaluating them on specific tasks requires large, high-quality, multi-modal
benchmarks that measure how well such models extract value from data. Concerning flood mapping, existing
datasets rarely combine bi-temporal SAR and co-registered optical imagery at scale, leaving the value of foundation models for this downstream task largely untested. We introduce GEOID-Flood, a large-scale multi-modal flood
segmentation benchmark, derived from Copernicus Emergency Management Service activations, spanning 219 events across 65 countries over ten years. The
dataset provides more than 14,000 tiles with co-registered pre- and post-event Sentinel-1, in GRD and RTC format, pre-event Sentinel-2 composite, and DEM, including manually validated labels that separate background from permanent water and flooded water. Using this
benchmark, we evaluate foundation models against conventional encoders across single-image, multi-temporal, and multi-modal protocols. We report three main findings: foundation models offer a consistent but modest advantage; optical-SAR fusion with finetuning best resolves transient flooding; and models trained on GEOID-Flood transfer to unseen events better than those trained on existing
datasets.
Dataset and code available at https://github.com/links-ads/geoid-flood.
Read the full abstract ▾
All authors Gaetano Chiriaco, Luca Barco, Andrea Bragagnolo, Claudio Rossi, Edoardo Arnaudo
Computer Vision
CalibBEV: LiDAR-Camera Calibration via BEV Alignment
Filippo D'Addeo, Lorenzo Cipelli, Adriano Cardace et al. (+3 more)
We present CalibBEV, a novel Bird's Eye View (BEV)
alignment approach for
LiDAR-camera calibration. Our method unifies
LiDAR and camera data into a shared 3D spatial representation, enabling accurate and robust cross-modal calibration. CalibBEV extracts sensor-wise BEV features from each modality using domain-specific architectures and estimates the calibration
matrix through a two-step
alignment process. First, we perform an implicit
alignment by regressing a coarse calibration
matrix directly from the BEV features. To ease this
alignment, we enforce semantic consistency between BEV representations across modalities using a contrastive loss inspired by CLIP, guiding both networks toward a unified feature space. In the second step, we leverage our BEV formulation to explicitly align the features of one modality with the other, refining the initial coarse estimate into a final, more accurate calibration
matrix. CalibBEV significantly outperforms prior point-to-pixel matching methods, achieving state-of-the-art calibration accuracy. On the KITTI and nuScenes
benchmarks, our method reduces the Relative Rotation Error (RRE) by 51% and 68%, and the Relative Translation Error (RTE) by 80% and 91%, respectively, compared to previous methods.
Read the full abstract ▾
All authors Filippo D'Addeo, Lorenzo Cipelli, Adriano Cardace, Emanuele Ghelfi, Andrea Zinelli, Massimo Bertozzi
Computer Vision
The Push-Forward Transform for Continuous and Robust Comparison of Dynamic Shapes
Roua Rouatbi, Juan-Esteban Suarez Cardona and Ivo F. Sbalzarini
We introduce a mathematical framework for shape comparison based on mapping functions from the shape domain to a common reference domain. This Push-Forward Transform enables invariant and robust comparison of shapes, preserving intrinsic
geometric information. Quantitatively comparing shapes and their temporal
evolution is a fundamental challenge in image analysis. Meaningful shape comparison requires representations that are invariant to transformations that do not alter shape itself, such as translation, rotation, reflection, re-parametrization, and uniform scaling, while remaining sensitive to intrinsic
geometric variation. Existing approaches often rely on sensitive parameterizations, landmark correspondence, or learned representations that are difficult to interpret and reproduce. We show that the Push-Forward Transform (PF-T) applied to Signed Distance Functions (SDFs) yields a continuous representation that captures both boundary and interior geometry. We derive an interpretable morphometric that quantifies shape similarity and reveals features such as skeletal
topology and rotational
symmetries. The push-forward transform applies consistently to two- and three-dimensional shapes, extends to time-evolving geometries, and supports the joint analysis of shape and additional scalar fields defined over shapes, such as intensity or molecular signals. We present the mathematical formulation, describe an efficient
algorithm, and
benchmark the approach on 2D, 3D, and temporal data sets.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
TRACE: Ergodic Trajectory Optimization for Active Scene Reconstruction
Ziyue Zheng, Linli Shi, Bingkun He et al. (+2 more)
Existing active reconstruction systems with Gaussian-splatting maps select observations greedily, optimizing a single next-best-view (NBV) at each step and connecting the chosen views by short-horizon path planning. This greedy decoupling disregards the global structure of scene information, producing inefficient trajectories that waste sensing capacity in transit between selected views. In this work, we study active reconstruction as an ergodic coverage problem: the time-averaged spatial statistics of the sensor trajectory should match a target information distribution induced by the current map. Our approach derives this target distribution online from uncertainty and visibility, and calculates ergodic trajectories via a kernel-ergodic horizon planner with gradient flow and footprint depletion, closing the loop between mapping and trajectory optimization. We thoroughly evaluate TRACE on the Replica
dataset against the Next-Best-View (NBV) baselines, improving PSNR by 1.5 dB. Code: https://github.com/spikelab-jhu/trace-active-reconstruction.
Read the full abstract ▾
All authors Ziyue Zheng, Linli Shi, Bingkun He, Wen Jiang, Ziyun Wang
Computer Vision
A General-Purpose VLM Can Teach an Astronomy Foundation Model to Better Recognize Galaxy Morphology
Dichang Zhang, Jiaqi Deng, Yixuan Shao et al. (+7 more)
Existing astronomy foundation models provide strong
galaxy representations, but adapting them to new survey conditions and survey-specific morphology recognition tasks still requires substantial human supervision. We show that VLM-based VQA systems contain meaningful visual-semantic priors that can serve as weak supervision for downstream morphology classifiers and improve morphology classification under limited human-label budgets. We first introduce a survey-oriented VQA
benchmark spanning two representative imaging regimes and evaluate state-of-the-art VLMs on
galaxy morphology questions. The results show that these models capture useful morphology signals and informative uncertainty, but are not sufficiently reliable to replace human annotators. Motivated by this finding, we use a general-purpose VLM as a morphology teacher for Zoobot, an astronomy foundation model pretrained on large-scale
Galaxy Zoo annotations. Across two survey domains and multiple annotation budgets, the VLM teacher consistently improves Zoobot's downstream morphology classification. These results demonstrate that a general-purpose VLM provides knowledge complementary to an astronomy foundation model and can teach it to better recognize
galaxy morphology under limited human supervision. The resulting pipeline is designed for label-efficient adaptation to forthcoming large-scale surveys, including the Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST) and the Nancy Grace Roman Space
Telescope. The
benchmark and code are publicly available at https://github.com/fw-ic/VLM-morphology-teacher.
Read the full abstract ▾
All authors Dichang Zhang, Jiaqi Deng, Yixuan Shao, Yuanpeng Liu, Jiali Cui, Zhiqiang Lao, Heather Yu, Liang Peng, Simon Birrer, Dimitris Samaras
Computer Vision
SpikeRestormer: Towards Energy-Efficient All-in-One Image Restoration via Unified Event Reasoning
Shengkai Hu, Jie Shao, Jiaqi Ma et al. (+5 more)
ANN-based All-in-One image restoration (AiOIR) unifies diverse degradation handling but incurs high computational costs, limiting its real-time deployment. While Spiking Neural Networks (SNNs) offer a low-power alternative, applying them to static images remains challenging. This difficulty arises because explicit event signals are absent, and degradation cues are heavily entangled with scene structures, hindering the learning of reliable restoration-oriented spike events. To address these issues, we propose SpikeRestormer, an energy-efficient SNN for AiOIR that performs event
reasoning over internally generated spike cues. Specifically, we propose a degradation-event perception process to extract spike-based degradation events through Subtractive Degradation Event
Attention (SDEA). Moreover, we introduce Hierarchical
Bayesian Skip Masking (HBSM) and Additive Restoration Event
Attention (AREA) processes for event-reliability inference and restoration-event construction, respectively. By integrating these complementary processes, SpikeRestormer formulates restoration as a unified process of degradation-event perception, degradation-event reliability inference, and restoration-event construction, liberating the potential of SNNs for energy-efficient AiOIR. Extensive experiments show that SpikeRestormer delivers competitive performance against ANN-based methods and establishes new state-of-the-art results among SNN-based methods with significantly lower energy consumption.
Read the full abstract ▾
All authors Shengkai Hu, Jie Shao, Jiaqi Ma, Xu Zhang, Keying Wu, Qilu Zhu, Beihang Song, Jun Wan