Computer Vision
Energy-based Tissue Manifolds for Longitudinal Multiparametric MRI Analysis
Kartikay Tehlan, Lukas Förner, Sina Wendrich et al. (+5 more)
We propose a
geometric framework for longitudinal multi-parametric MRI analysis based on patient-specific energy modelling in sequence space. Rather than operating on images with spatial networks, each voxel is represented by its multi-sequence intensity vector ($T1$, $T1c$, $T2$, FLAIR, ADC), and a compact implicit neural representation is trained via denoising score matching to learn an energy function $E_{\theta}(\mathbf{u})$ over $\mathbb{R}^d$ from a single baseline scan. The learned energy landscape provides a differential-
geometric description of tissue regimes without
segmentation labels. Local minima define tissue basins, gradient magnitude reflects proximity to regime boundaries, and
Laplacian curvature characterises local constraint structure. Importantly, this baseline energy
manifold is treated as a fixed
geometric reference: it encodes the set of contrast combinations observed at diagnosis and is not retrained at follow-up. Longitudinal assessment is therefore formulated as evaluation of subsequent scans relative to this baseline geometry. Rather than comparing anatomical
segmentations, we analyse how the distribution of MRI sequence vectors evolves under the baseline energy function. In a paediatric case with later recurrence, follow-up scans show progressive deviation in energy and directional displacement in sequence space toward the baseline tumour-associated regime before clear radiological reappearance. In a case with stable disease, voxel distributions remain confined to established low-energy basins without systematic drift. The presented cases serve as proof-of-concept that patient-specific energy
manifolds can function as
geometric reference systems for longitudinal mpMRI analysis without explicit
segmentation or supervised classification, providing a foundation for further investigation of
manifold-based tissue-at-risk tracking in neuro-oncology.
Read the full abstract ▾
All authors Kartikay Tehlan, Lukas Förner, Sina Wendrich, Nico Schmutzenhofer, Michael Frühwald, Matthias Wagner, Nassir Navab, Thomas Wendler
Computer Vision
Multiple Domain Generalization Using Category Information Independent of Domain Differences
Reiji Saito and Kazuhiro Hotta
Domain generalization is a technique aimed at enabling models to maintain high accuracy when applied to new environments or
datasets (unseen domains) that differ from the
datasets used in training. Generally, the accuracy of models trained on a specific
dataset (source domain) often decreases significantly when evaluated on different
datasets (target domain). This issue arises due to differences in domains caused by varying environmental conditions such as imaging equipment and staining methods. Therefore, we undertook two initiatives to perform
segmentation that does not depend on domain differences. We propose a method that separates category information independent of domain differences from the information specific to the source domain. By using information independent of domain differences, our method enables learning the
segmentation targets (e.g., blood vessels and
cell nuclei). Although we extract independent information of domain differences, this cannot completely bridge the domain gap between training and test data. Therefore, we absorb the domain gap using the
quantum vectors in Stochastically Quantized Variational AutoEncoder (SQ-VAE). In experiments, we evaluated our method on
datasets for vascular
segmentation and
cell nucleus segmentation. Our methods improved the accuracy compared to conventional methods.
Read the full abstract ▾
Computer Vision
DINO-QPM: Adapting Visual Foundation Models for Globally Interpretable Image Classification
Robert Zimmermann, Thomas Norrenbrock and Bodo Rosenhahn
Although visual foundation models like DINOv2 provide state-of-the-art performance as feature extractors, their complex, high-dimensional representations create substantial hurdles for interpretability. This work proposes DINO-QPM, which converts these powerful but entangled features into contrastive, class-independent representations that are interpretable by humans. DINO-QPM is a lightweight interpretability adapter that pursues globally interpretable image classification, adapting the Quadratic Programming Enhanced Model (QPM) to operate on strictly frozen DINO backbones. While classification with visual foundation models typically relies on the CLS
token, we deliberately diverge from this standard. By leveraging average-pooling, we directly connect the patch
embeddings to the model's features and therefore enable spatial localisation of DINO-QPM's globally interpretable features within the input space. Furthermore, we apply a sparsity loss to minimise spatial scatter and background noise, ensuring that explanations are grounded in relevant object parts. With DINO-QPM we make the level of interpretability of QPM available as an adapter while exceeding the accuracy of DINOv2 linear probe. Evaluated through an introduced Plausibility metric and other interpretability metrics, extensive experiments demonstrate that DINO-QPM is superior to other applicable methods for frozen visual foundation models in both classification accuracy and explanation quality.
Read the full abstract ▾
Computer Vision
Bridging MRI and PET physiology: Untangling complementarity through orthogonal representations
Sonja Adomeit, Kartikay Tehlan, Lukas Förner et al. (+7 more)
Multimodal imaging analysis often relies on joint latent representations, yet these approaches rarely define what information is shared versus modality-specific. Clarifying this distinction is clinically relevant, as it delineates the irreducible contribution of each modality and informs rational acquisition strategies. We propose a subspace decomposition framework that reframes
multimodal fusion as a problem of orthogonal subspace separation rather than translation. We decompose Prostate-Specific Membrane Antigen (PSMA) PET uptake into an MRI-explainable physiological envelope and an orthogonal residual reflecting signal components not expressible within the MRI feature
manifold. Using multiparametric MRI, we train an intensity-based, non-spatial implicit neural representation (INR) to map MRI feature vectors to PET uptake. We introduce a projection-based regularization using singular value decomposition to penalize residual components lying within the span of the MRI feature
manifold. This enforces mathematical orthogonality between tissue-level physiological properties (structure,
diffusion, perfusion) and intracellular PSMA expression. Tested on 13 prostate cancer patients, the model demonstrates that residual components spanned by MRI features are absorbed into the learned envelope, while the orthogonal residual is largest in tumour regions. This indicates that PSMA PET contains signal components not recoverable from MRI-derived physiological descriptors. The resulting decomposition provides a structured characterization of modality complementarity grounded in representation geometry rather than image translation.
Read the full abstract ▾
All authors Sonja Adomeit, Kartikay Tehlan, Lukas Förner, Katharina Weisser, Helen Scholtiseek, David Kaufmann, Julie Steinestel, Constantin Lapa, Thomas Kröncke, Thomas Wendler
Computer Vision · Cross-listed from cs.RO
An RTK-SLAM Dataset for Absolute Accuracy Evaluation in GNSS-Degraded Environments
Wei Zhang, Vincent Ress, David Skuddis et al. (+2 more)
RTK-SLAM systems integrate simultaneous localization and mapping (SLAM) with real-time kinematic (RTK) GNSS positioning, promising both relative consistency and globally referenced coordinates for efficient georeferenced surveying. A critical and underappreciated issue is that the standard evaluation metric, Absolute Trajectory Error (ATE), first fits an optimal rigid-body transformation between the estimated trajectory and reference before computing errors. This so-called SE(3)
alignment absorbs global drift and systematic errors, making trajectories appear more accurate than they are in practice, and is unsuitable for evaluating the global accuracy of RTK-SLAM. We present a geodetically referenced
dataset and evaluation methodology that expose this gap. A key design principle is that the RTK receiver is used solely as a system input, while ground truth is established independently via a geodetic total station. This separation is absent from all existing
datasets, where GNSS typically serves as (part of) the ground truth. The
dataset is collected with a handheld RTK-SLAM device, comprising two scenes. We evaluate
LiDAR-inertial, visual-inertial, and
LiDAR-visual-inertial RTK-SLAM systems alongside standalone RTK, reporting direct global accuracy and SE(3)-aligned relative accuracy to make the gap explicit. Results show that SE(3)
alignment can underestimate absolute positioning error by up to 76%. RTK-SLAM achieves centimeter-level absolute accuracy in open-sky conditions and maintains decimeter-level global accuracy indoors, where standalone RTK degrades to tens of meters. The
dataset, calibration files, and evaluation scripts are publicly available at https://rtk-slam-
dataset.github.io/.
Read the full abstract ▾
All authors Wei Zhang, Vincent Ress, David Skuddis, Uwe Soergel, Norbert Haala
Computer Vision
Learning to Search: A Decision-Based Agent for Knowledge-Based Visual Question Answering
Zhuohong Chen, Zhenxian Wu, Yunyao Yu et al. (+6 more)
Knowledge-based visual question answering (KB-VQA) requires
vision-
language models to understand images and use external knowledge, especially for rare entities and long-tail facts. Most existing
retrieval-augmented generation (RAG) methods adopt a fixed pipeline that sequentially retrieves information, filters it, and then produces an answer. Such a design makes it difficult to adapt to diverse question types. Moreover, it separates
retrieval from
reasoning, making it hard for the model to decide when to search, how to refine queries, or when to stop. As a result, the retrieved evidence is often poorly aligned with the question. To address these limitations, we reformulate KB-VQA as a search-
agent problem and model the solving process as a multi-step decision-making procedure. At each step, the
agent selects one of four actions-Answer, Image
Retrieval, Text
Retrieval, and Caption-based on its current information state. We further design an automated pipeline to collect multi-step trajectories that record the
agent's
reasoning process, tool usage, and intermediate decisions. These trajectories are then used as supervision for fine-tuning. Experiments on InfoSeek and E-VQA demonstrate that our method achieves state-of-the-art performance, consistently outperforming prior baselines and confirming the effectiveness of our framework.
Read the full abstract ▾
All authors Zhuohong Chen, Zhenxian Wu, Yunyao Yu, Hangrui Xu, Zirui Liao, Zhifang Liu, Xiangwen Deng, Pen Jiao, Haoqian Wang
Computer Vision
USCNet: Transformer-Based Multimodal Fusion with Segmentation Guidance for Urolithiasis Classification
Changmiao Wang, Songqi Zhang, Yongquan Zhang et al. (+9 more)
Kidney stone disease ranks among the most prevalent conditions in urology, and understanding the composition of these stones is essential for creating personalized treatment plans and preventing recurrence. Current methods for analyzing kidney stones depend on postoperative specimens, which prevents rapid classification before surgery. To overcome this limitation, we introduce a new approach called the Urinary Stone
Segmentation and Classification Network (USCNet). This innovative method allows for precise preoperative classification of kidney stones by integrating Computed
Tomography (CT) images with clinical data from Electronic Health Records (EHR). USCNet employs a
Transformer-based
multimodal fusion framework with CT-EHR
attention and
segmentation-guided
attention modules for accurate classification. Moreover, a dynamic loss function is introduced to effectively balance the dual objectives of
segmentation and classification. Experiments on an in-house kidney stone
dataset show that USCNet demonstrates outstanding performance across all evaluation metrics, with its classification efficacy significantly surpassing existing mainstream methods. This study presents a promising solution for the precise preoperative classification of kidney stones, offering substantial clinical benefits. The source code has been made publicly available: https://github.com/ZhangSongqi0506/KidneyStone.
Read the full abstract ▾
All authors Changmiao Wang, Songqi Zhang, Yongquan Zhang, Yifei Wang, Liya Liu, Nannan Li, Xingzhi Li, Jiexin Pan, Yi Jiang, Xiang Wan, Hai Wang, Ahmed Elazab
Computer Vision
CSA-Graphs: A Privacy-Preserving Structural Dataset for Child Sexual Abuse Research
Carlos Caetano, Camila Laranjeira, Clara Ernesto et al. (+5 more)
Child Sexual Abuse Imagery (CSAI) classification is an important yet challenging problem for computer
vision research due to the strict legal and ethical restrictions that prevent the public sharing of CSAI
datasets. This limitation hinders reproducibility and slows progress in developing automated methods. In this work, we introduce CSA-Graphs, a
privacy-preserving structural
dataset. Instead of releasing the original images, we provide structural representations that remove explicit visual content while preserving contextual information. CSA-Graphs includes two complementary graph-based modalities: scene graphs describing object relationships and skeleton graphs encoding human pose. Experiments show that both representations retain useful information for classifying CSAI, and that combining them further improves performance. This
dataset enables broader research on computer
vision methods for child safety while respecting legal and ethical constraints.
Read the full abstract ▾
All authors Carlos Caetano, Camila Laranjeira, Clara Ernesto, Artur Barros, João Macedo, Leo S. F. Ribeiro, Jefersson A. dos Santos, Sandra Avila
Computer Vision
A Utility-preserving De-identification Pipeline for Cross-hospital Radiology Data Sharing
Chenhao Liu, Zelin Wen, Yan Tong et al. (+7 more)
Large-scale radiology data are critical for developing robust medical AI systems. However, sharing such data across hospitals remains heavily constrained by
privacy concerns. Existing de-identification research in radiology mainly focus on removing identifiable information to enable compliant data release. Yet whether de-identified radiology data can still preserve sufficient utility for large-scale
vision-
language model training and cross-hospital transfer remains underexplored. In this paper, we introduce a utility-preserving de-identification pipeline (UPDP) for cross-hospital radiology data sharing. Specifically, we compile a blacklist of
privacy-sensitive terms and a whitelist of pathology-related terms. For radiology images, we use a generative filtering mechanism that synthesis a
privacy-filtered and pathology-reserved counterparts of the original images. These synthetic image counterparts, together with ID-filtered reports, can then be securely shared across hospitals for downstream model development and evaluation. Experiments on public chest X-ray
benchmarks demonstrate that our method effectively removes
privacy-sensitive information while preserving diagnostically relevant pathology cues. Models trained on the de-identified data maintain competitive diagnostic accuracy compared with those trained on the original data, while exhibiting a marked decline in identity-related accuracy, confirming effective
privacy protection. In the cross-hospital setting, we further show that de-identified data can be combined with local data to yield better performance.
Read the full abstract ▾
All authors Chenhao Liu, Zelin Wen, Yan Tong, Junjie Zhu, Xinyu Tian, Yuchi Liu, Ashu Gupta, Syed M. S. Islam, Tom Gedeon, Yue Yao
Computer Vision
Accuracy Improvement of Semi-Supervised Segmentation Using Supervised ClassMix and Sup-Unsup Feature Discriminator
Takahiro Mano, Reiji Saito and Kazuhiro Hotta
In semantic
segmentation, the creation of pixel-level labels for training data incurs significant costs. To address this problem, semi-supervised learning, which utilizes a small number of labeled images alongside unlabeled images to enhance the performance, has gained
attention. A conventional semi-supervised learning method, ClassMix, pastes class labels predicted from unlabeled images onto other images. However, since ClassMix performs operations using pseudo-labels obtained from unlabeled images, there is a risk of handling inaccurate labels. Additionally, there is a gap in data quality between labeled and unlabeled images, which can impact the feature maps. This study addresses these two issues. First, we propose a method where class labels from labeled images, along with the corresponding image regions, are pasted onto unlabeled images and their pseudo-labeled images. Second, we introduce a method that trains the model to make predictions on unlabeled images more similar to those on labeled images. Experiments on the Chase and COVID-19
datasets demonstrated an average improvement of 2.07% in mIoU compared to conventional semi-supervised learning methods.
Read the full abstract ▾
Computer Vision
Assessing the Added Value of Onboard Earth Observation Processing with the IRIDE HEO Service Segment
Parampuneet Kaur Thind, Charles Mwangi, Giovanni Varetto et al. (+3 more)
Current operational Earth Observation (EO) services, including the Copernicus Emergency Management Service (CEMS), the European Forest Fire Information System (EFFIS), and the Copernicus Land Monitoring Service (CLMS), rely primarily on ground-based processing pipelines. While these systems provide mature large-scale information products, they remain constrained by downlink latency, bandwidth limitations, and limited capability for autonomous observation prioritisation. The International Report for an Innovative Defence of Earth (IRIDE) programme is a national Earth observation initiative led by the Italian government to support public authorities through timely, objective information derived from spaceborne data. Rather than a single constellation, IRIDE is designed as a constellation of constellations, integrating heterogeneous sensing technologies within a unified service-oriented architecture. Within this framework, Hawk for Earth Observation (HEO) enables onboard generation of data products, allowing information extraction earlier in the processing chain. This paper examines the limitations of ground-only architectures and evaluates the added value of onboard processing at the operational service level. The IRIDE burnt-area mapping service is used as a representative case study to demonstrate how onboard intelligence can support higher spatial detail (sub-three-metre ground sampling distance), smaller detectable events (minimum mapping unit of three hectares), and improved system responsiveness. Rather than replacing existing Copernicus services, the IRIDE HEO capability is positioned as a complementary layer providing image-driven pre-classification to support downstream emergency and land-management workflows. This work highlights the operational value of onboard intelligence for emerging low-latency EO service architectures.
Read the full abstract ▾
All authors Parampuneet Kaur Thind, Charles Mwangi, Giovanni Varetto, Lorenzo Sarti, Andrea Papa, Andrea Taramelli
Computer Vision
SurFITR: A Dataset for Surveillance Image Forgery Detection and Localisation
Qizhou Wang, Guansong Pang and Christopher Leckie
We present the Surveillance Forgery Image Test Range (SurFITR), a
dataset for surveillance-style image forgery detection and localisation, in response to recent advances in open-access image generation models that raise concerns about falsifying visual evidence. Existing forgery models, trained on
datasets with full-image synthesis or large manipulated regions in object-centric images, struggle to generalise to surveillance scenarios. This is because tampering in surveillance imagery is typically localised and subtle, occurring in scenes with varied viewpoints, small or occluded subjects, and lower visual quality. To address this gap, SurFITR provides a large collection of forensically valuable imagery generated via a
multimodal LLM-powered pipeline, enabling semantically aware, fine-grained editing across diverse surveillance scenes. It contains over 137k tampered images with varying resolutions and edit types, generated using multiple image editing models. Extensive experiments show that existing detectors degrade significantly on SurFITR, while training on SurFITR yields substantial improvements in both in-domain and cross-domain performance. SurFITR is publicly available on GitHub.
Read the full abstract ▾
Computer Vision
Novel Anomaly Detection Scenarios and Evaluation Metrics to Address the Ambiguity in the Definition of Normal Samples
Reiji Saito, Satoshi Kamiya and Kazuhiro Hotta
In conventional anomaly detection, training data consist of only normal samples. However, in real-world scenarios, the definition of a normal sample is often ambiguous. For example, there are cases where a sample has small scratches or stains but is still acceptable for practical usage. On the other hand, higher precision is required when manufacturing equipment is upgraded. In such cases, normal samples may include small scratches, tiny
dust particles, or a foreign object that we would prefer to classify as an anomaly. Such cases frequently occur in industrial settings, yet they have not been discussed until now. Thus, we propose novel scenarios and an evaluation metric to accommodate specification changes in real-world applications. Furthermore, to address the ambiguity of normal samples, we propose the RePaste, which enhances learning by re-pasting regions with high anomaly scores from the previous step into the input for the next step. On our scenarios using the MVTec AD
benchmark, RePaste achieved the state-of-the-art performance with respect to the proposed evaluation metric, while maintaining high AUROC and PRO scores. Code: https://github.com/ReijiSoftmaxSaito/Scenario
Read the full abstract ▾
Computer Vision
Location Is All You Need: Continuous Spatiotemporal Neural Representations of Earth Observation Data
Mojgan Madadikhaljan, Jonathan Prexl, Isabelle Wittmann et al. (+2 more)
In this work, we present LIANet (Location Is All You Need Network), a coordinate-based neural representation that models multi-temporal spaceborne Earth observation (EO) data for a given region of interest as a continuous spatiotemporal neural field. Given only spatial and temporal coordinates, LIANet reconstructs the corresponding satellite imagery. Once pretrained, this neural representation can be adapted to various EO downstream tasks, such as semantic
segmentation or pixel-wise regression, importantly, without requiring access to the original satellite data. LIANet intends to serve as a user-friendly alternative to Geospatial Foundation Models (GFMs) by eliminating the overhead of data access and preprocessing for end-users and enabling fine-tuning solely based on labels. We demonstrate the pretraining of LIANet across target areas of varying sizes and show that fine-tuning it for downstream tasks achieves competitive performance compared to training from scratch or using established GFMs. The source code and
datasets are publicly available at https://github.com/mojganmadadi/LIANet/tree/v1.0.1.
Read the full abstract ▾
All authors Mojgan Madadikhaljan, Jonathan Prexl, Isabelle Wittmann, Conrad M Albrecht, Michael Schmitt
Computer Vision
AnchorSplat: Feed-Forward 3D Gaussian Splatting with 3D Geometric Priors
Xiaoxue Zhang, Xiaoxu Zheng, Yixuan Yin et al. (+5 more)
Recent feed-forward Gaussian reconstruction models adopt a pixel-aligned formulation that maps each 2D pixel to a 3D Gaussian, entangling Gaussian representations tightly with the input images. In this paper, we propose AnchorSplat, a novel feed-forward 3DGS framework for scene-level reconstruction that represents the scene directly in 3D space. AnchorSplat introduces an anchor-aligned Gaussian representation guided by 3D
geometric priors (e.g., sparse point clouds, voxels, or RGB-D point clouds), enabling a more geometry-aware renderable 3D Gaussians that is independent of image resolution and number of views. This design substantially reduces the number of required Gaussians, improving computational efficiency while enhancing reconstruction fidelity. Beyond the anchor-aligned design, we utilize a Gaussian Refiner to adjust the intermediate Gaussiansy via merely a few forward passes. Experiments on the ScanNet++ v2 NVS
benchmark demonstrate the SOTA performance, outperforming previous methods with more view-consistent and substantially fewer Gaussian primitives.
Read the full abstract ▾
All authors Xiaoxue Zhang, Xiaoxu Zheng, Yixuan Yin, Tiao Zhao, Kaihua Tang, Michael Bi Mi, Zhan Xu, Dave Zhenyu Chen
Computer Vision
PRISM: Rethinking Atmospheric Scattering Reconstruction as a Unified Understanding and Restoration Model for Real-world Dehazing
Chengyu Fang, Chunming He, Yuelin Zhang et al. (+6 more)
Real-world image dehazing (RID) aims to remove haze-induced degradation from real scenes. This task remains challenging due to non-uniform haze distribution, spatially varying color shifts, and the scarcity of paired real hazy-clean data. In PRISM, we propose Proximal Scattering Atmosphere Reconstruction (PSAR), a physically structured framework that jointly reconstructs the clear scene and scattering variables under the atmospheric scattering model, making the restoration process more interpretable in complex real-world conditions. To bridge the synthetic-to-real gap, we design an online non-uniform haze synthesis pipeline and a Selective Self-
Distillation Adaptation (SSDA) scheme for unpaired real-world scenarios, which enables the model to selectively learn from high-quality perceptual targets while leveraging its intrinsic scattering understanding to audit residual haze and guide self-refinement. Experiments on real-world
benchmarks demonstrate that PRISM achieves competitive performance on RID tasks.
Read the full abstract ▾
All authors Chengyu Fang, Chunming He, Yuelin Zhang, Chubin Chen, Chenyang Zhu, Hongqiu Wang, Longxiang Tang, Xiu Li, Sina Farsiu
Computer Vision · Cross-listed from hep-ex
Towards foundation-style models for energy-frontier heterogeneous neutrino detectors via self-supervised pre-training
Saúl Alonso-Monsalve, Fabio Cufino, Umut Kose et al. (+2 more)
Accelerator-based
neutrino physics is entering an energy-frontier regime in which interactions reach the TeV scale and produce exceptionally dense, overlapping detector signatures. In this regime, event interpretation becomes impractical for conventional reconstruction approaches, particularly when labelled data are scarce and the analysis spans diverse downstream objectives. We present a sparse ViT framework for learning reusable representations from heterogeneous detector data. Self-supervised pre-training combines masked autoencoder reconstruction with relational voxel-level objectives for hierarchy, ghost and particle identification, and the resulting shared encoder is then jointly fine-tuned across classification and regression tasks. Evaluated on simulated events from the proposed FASERCal concept at the LHC, we find that pre-training consistently improves
neutrino flavour and charm-
quark identification, momentum regression, and vertex reconstruction over training from scratch, with the addition of relational objectives yielding further gains in the most topologically complex channels. Interpretability analyses further show that pre-training yields a more structured latent space, while detector-subsystem ablations recover physically plausible channel-dependent roles for the heterogeneous inputs. A data-efficiency study shows that, with roughly $10^3$ labelled events, the pre-trained encoder already matches the flavour-classification performance of a randomly initialised model trained on an order of magnitude more data. The learned representations also transfer effectively to publicly available
benchmarks spanning different detector technologies and energy scales, matching or exceeding published baselines. These results support self-supervised pre-training on
multimodal detector data as a scalable route towards reusable representations for
neutrino and particle-detector analysis.
Read the full abstract ▾
All authors Saúl Alonso-Monsalve, Fabio Cufino, Umut Kose, Anna Mascellani, André Rubbia
Computer Vision · Cross-listed from cs.RO
KITE: Keyframe-Indexed Tokenized Evidence for VLM-Based Robot Failure Analysis
Mehdi Hosseinzadeh, King Hang Wong and Feras Dayoub
We present KITE, a training-free, keyframe-anchored, layout-grounded front-end that converts long
robot-execution
videos into compact, interpretable tokenized evidence for
vision-
language models (VLMs). KITE distills each trajectory into a small set of motion-salient keyframes with open-vocabulary detections and pairs each keyframe with a schematic bird's-eye-view (BEV) representation that encodes relative object layout, axes, timestamps, and detection confidence. These visual cues are serialized with
robot-profile and scene-context
tokens into a unified
prompt, allowing the same front-end to support failure detection, identification, localization, explanation, and correction with an off-the-shelf VLM. On the RoboFAC
benchmark, KITE with Qwen2.5-VL substantially improves over vanilla Qwen2.5-VL in the training-free setting, with especially large gains on simulation failure detection, identification, and localization, while remaining competitive with a RoboFAC-tuned baseline. A small QLoRA fine-tune further improves explanation and correction quality. We also report qualitative results on real dual-arm
robots, demonstrating the practical applicability of KITE as a structured and interpretable front-end for
robot failure analysis. Code and models are released on our project page: https://m80hz.github.io/kite/
Read the full abstract ▾
Computer Vision
Not all tokens contribute equally to diffusion learning
Guoqing Zhang, Lu Shi, Wanru Xu et al. (+4 more)
With the rapid development of conditional
diffusion models, significant progress has been made in text-to-
video generation. However, we observe that these models often neglect semantically important
tokens during inference, leading to biased or incomplete generations under classifier-free guidance. We attribute this issue to two key factors: distributional bias caused by the long-tailed
token frequency in training data, and spatial misalignment in cross-
attention where semantically important
tokens are overshadowed by less informative ones. To address these issues, we propose Distribution-Aware Rectification and Spatial Ensemble (DARE), a unified framework that improves semantic guidance in
diffusion models from the perspectives of distributional debiasing and spatial consistency. First, we introduce Distribution-Rectified Classifier-Free Guidance (DR-CFG), which regularizes the training process by dynamically suppressing dominant
tokens with low semantic density, encouraging the model to better capture underrepresented semantic cues and learn a more balanced conditional distribution. This design mitigates the risk of the model distribution overfitting to
tokens with low semantic density. Second, we propose Spatial Representation
Alignment (SRA), which adaptively reweights cross-
attention maps according to
token importance and enforces representation consistency, enabling semantically important
tokens to exert stronger spatial guidance during generation. This mechanism effectively prevents low semantic-density
tokens from dominating the
attention allocation, thereby avoiding the dilution of the spatial and distributional guidance provided by high semantic-density
tokens. Extensive experiments on multiple
benchmark datasets demonstrate that DARE consistently improves generation fidelity and semantic
alignment, achieving significant gains over existing approaches.
Read the full abstract ▾
All authors Guoqing Zhang, Lu Shi, Wanru Xu, Linna Zhang, Sen Wang, Fangfang Wang, Yigang Cen
Computer Vision
ModuSeg: Decoupling Object Discovery and Semantic Retrieval for Training-Free Weakly Supervised Segmentation
Qingze He, Fagui Liu, Dengke Zhang et al. (+2 more)
Weakly supervised semantic
segmentation aims to achieve pixel-level predictions using image-level labels. Existing methods typically entangle semantic recognition and object localization, which often leads models to focus exclusively on sparse discriminative regions. Although foundation models show immense potential, many approaches still follow the tightly coupled optimization paradigm, struggling to effectively alleviate pseudo-label noise and often relying on time-consuming multi-stage retraining or unstable end-to-end joint optimization. To address the above challenges, we present ModuSeg, a training-free weakly supervised semantic
segmentation framework centered on explicitly decoupling object discovery and semantic assignment. Specifically, we integrate a general mask proposer to extract
geometric proposals with reliable boundaries, while leveraging semantic foundation models to construct an offline feature bank, transforming
segmentation into a non-parametric feature
retrieval process. Furthermore, we propose semantic boundary purification and soft-masked feature aggregation strategies to effectively mitigate boundary ambiguity and quantization errors, thereby extracting high-quality category prototypes. Extensive experiments demonstrate that the proposed decoupled architecture better preserves fine boundaries without parameter fine-tuning and achieves highly competitive performance on standard
benchmark datasets. Code is available at https://github.com/Autumnair007/ModuSeg.
Read the full abstract ▾
All authors Qingze He, Fagui Liu, Dengke Zhang, Qingmao Wei, Quan Tang
Computer Vision
Synthetic Dataset Generation for Partially Observed Indoor Objects
Jelle Vermandere, Maarten Bassier and Maarten Vergauwen
Learning-based methods for 3D scene reconstruction and object completion require large
datasets containing partial scans paired with complete ground-truth geometry. However, acquiring such
datasets using real-world scanning systems is costly and time-consuming, particularly when accurate ground truth for occluded regions is required. In this work, we present a virtual scanning framework implemented in Unity for generating realistic synthetic 3D scan
datasets. The proposed system simulates the behaviour of real-world scanners using configurable parameters such as scan resolution, measurement range, and distance-dependent noise. Instead of directly sampling mesh surfaces, the framework performs ray-based scanning from virtual viewpoints, enabling realistic modelling of sensor visibility and occlusion effects. In addition, panoramic images captured at the scanner location are used to assign colours to the resulting point clouds. To support scalable
dataset creation, the scanner is integrated with a procedural indoor scene generation pipeline that automatically produces diverse room layouts and furniture arrangements. Using this system, we introduce the V-Scan
dataset, which contains synthetic indoor scans together with object-level partial point clouds, voxel-based occlusion grids, and complete ground-truth geometry. The resulting
dataset provides valuable supervision for training and evaluating learning-based methods for scene reconstruction and object completion.
Read the full abstract ▾
Computer Vision
IQ-LUT: interpolated and quantized LUT for efficient image super-resolution
Yuxuan Zhang, Zhikai Dong, Xinning Chai et al. (+4 more)
Lookup table (LUT) methods demonstrate considerable potential in accelerating image super-resolution inference. However, pursuing higher image quality through larger receptive fields and bit-depth triggers exponential growth in the LUT's index space, creating a storage bottleneck that limits deployment on resource-constrained devices. We introduce IQ-LUT, which achieves a reduction in LUT size while simultaneously enhancing super-resolution quality. First, we integrate interpolation and quantization into the single-input, multiple-output ECNN, which dramatically reduces the index space and thereby the overall LUT size. Second, the integration of residual learning mitigates the dependence on LUT bit-depth, which facilitates training stability and prioritizes the reconstruction of fine-grained details for superior visual quality. Finally, guided by knowledge
distillation, our non-uniform quantization process optimizes the quantization levels, thereby reducing storage while also compensating for quantization loss. Extensive benchmarking demonstrates our approach substantially reduces storage costs (by up to 50x compared to ECNN) while achieving superior super-resolution quality.
Read the full abstract ▾
All authors Yuxuan Zhang, Zhikai Dong, Xinning Chai, Xiangyun Zhou, Yi Xu, Zhengxue Cheng, Li Song
Computer Vision
Generative Phomosaic with Structure-Aligned and Personalized Diffusion
Jaeyoung Chung, Hyunjin Son and Kyoung Mu Lee
We present the first generative approach to photomosaic creation. Traditional photomosaic methods rely on a large number of tile images and color-based matching, which limits both diversity and structural consistency. Our generative photomosaic framework synthesizes tile images using
diffusion-based generation conditioned on reference images. A low-frequency conditioned
diffusion mechanism aligns global structure while preserving
prompt-driven details. This generative formulation enables photomosaic composition that is both semantically expressive and structurally coherent, effectively overcoming the fundamental limitations of matching-based approaches. By leveraging few-shot personalized
diffusion, our model is able to produce user-specific or stylistically consistent tiles without requiring an extensive collection of images.
Read the full abstract ▾
Computer Vision
Canopy Tree Height Estimation Using Quantile Regression: Modeling and Evaluating Uncertainty in Remote Sensing
Karsten Schrödter, Jan Pauls and Fabian Gieseke
Accurate tree height estimation is vital for ecological monitoring and biomass assessment. We apply
quantile regression to existing tree height estimation models based on satellite data to incorporate uncertainty quantification. Most current approaches for tree height estimation rely on point predictions, which limits their applicability in risk-sensitive scenarios. In this work, we show that, with minor modifications of a given prediction head, existing models can be adapted to provide statistically calibrated uncertainty estimates via
quantile regression. Furthermore, we demonstrate how our results correlate with known challenges in remote sensing (e.g., terrain complexity, vegetation heterogeneity), indicating that the model is less confident in more challenging conditions.
Read the full abstract ▾
Computer Vision
CAAP: Capture-Aware Adversarial Patch Attacks on Palmprint Recognition Models
Renyang Liu, Jiale Li, Jie Zhang et al. (+6 more)
Palmprint recognition is deployed in security-critical applications, including access control and palm-based payment, due to its contactless acquisition and highly discriminative ridge-and-crease textures. However, the robustness of deep palmprint recognition systems against physically realizable attacks remains insufficiently understood. Existing studies are largely confined to the digital setting and do not adequately account for the texture-dominant nature of palmprint recognition or the distortions introduced during physical acquisition. To address this gap, we propose CAAP, a capture-aware
adversarial patch framework for palmprint recognition. CAAP learns a universal patch that can be reused across inputs while remaining effective under realistic acquisition variation. To match the structural characteristics of palmprints, the framework adopts a cross-shaped patch
topology, which enlarges spatial coverage under a fixed pixel budget and more effectively disrupts long-range texture continuity. CAAP further integrates three modules: ASIT for input-conditioned patch rendering, RaS for
stochastic capture-aware simulation, and MS-DIFE for feature-level identity-disruptive guidance. We evaluate CAAP on the Tongji, IITD, and AISEC
datasets against generic CNN backbones and palmprint-specific recognition models. Experiments show that CAAP achieves strong untargeted and targeted attack performance with favorable cross-model and cross-
dataset transferability. The results further show that, although
adversarial training can partially reduce the attack success rate, substantial residual vulnerability remains. These findings indicate that deep palmprint recognition systems remain vulnerable to physically realizable, capture-aware
adversarial patch attacks, underscoring the need for more effective defenses in practice. Code available at https://github.com/ryliu68/CAAP.
Read the full abstract ▾
All authors Renyang Liu, Jiale Li, Jie Zhang, Cong Wu, Xiaojun Jia, Shuxin Li, Wei Zhou, Kwok-Yan Lam, See-kiong Ng
Computer Vision
MAR-GRPO: Stabilized GRPO for AR-diffusion Hybrid Image Generation
Xiaoxiao Ma, Jiachen Lei, Tianfei Ren et al. (+6 more)
Reinforcement learning (RL) has been successfully applied to autoregressive (AR) and
diffusion models. However, extending RL to hybrid AR-
diffusion frameworks remains challenging due to interleaved inference and noisy log-probability estimation. In this work, we study masked autoregressive models (MAR) and show that the
diffusion head plays a critical role in training dynamics, often introducing noisy gradients that lead to instability and early performance saturation. To address this issue, we propose a stabilized RL framework for MAR. We introduce multi-trajectory expectation (MTE), which estimates the optimization direction by averaging over multiple
diffusion trajectories, thereby reducing
diffusion-induced gradient noise. To avoid over-smoothing, we further estimate
token-wise uncertainty from multiple trajectories and apply multi-trajectory optimization only to the top-k% uncertain
tokens. In addition, we introduce a consistency-aware
token selection strategy that filters out AR
tokens that are less aligned with the final generated content. Extensive experiments across multiple
benchmarks demonstrate that our method consistently improves visual quality, training stability, and spatial structure understanding over baseline GRPO and pre-RL models. Code is available at: https://github.com/AMAP-ML/mar-grpo.
Read the full abstract ▾
All authors Xiaoxiao Ma, Jiachen Lei, Tianfei Ren, Jie Huang, Siming Fu, Aiming Hao, Jiahong Wu, Xiangxiang Chu, Feng Zhao
Computer Vision
Auditing Demographic Bias in Facial Landmark Detection for Fair Human-Robot Interaction
Pablo Parte, Roberto Valle, José M. Buenaposada et al. (+1 more)
Fairness in human-
robot interaction critically depends on the reliability of the perceptual models that enable
robots to interpret human behavior. While demographic biases have been widely studied in high-level facial analysis tasks, their presence in facial landmark detection remains unexplored. In this paper, we conduct a systematic audit of demographic bias in this task, analyzing the age, gender, and race biases. To this end, we introduce a controlled statistical methodology to disentangle demographic effects from confounding visual factors. Our analysis demonstrates that visual confounders, particularly head pose and face resolution, heavily outweigh the impact of demographic attributes. Notably, after accounting for these confounders, performance disparities across gender and race vanish. However, we identify a statistically significant age-related bias, with higher localization errors for older individuals. This shows that
fairness issues can emerge even in low-level
vision components and can propagate through the HRI pipeline. We argue that
auditing and correcting such biases is a necessary step toward trustworthy and equitable
robot perception systems.
Read the full abstract ▾
All authors Pablo Parte, Roberto Valle, José M. Buenaposada, Luis Baumela
Computer Vision
Compression as an Adversarial Amplifier Through Decision Space Reduction
Lewis Evans, Harkrishan Jandu, Zihan Ye et al. (+2 more)
Image compression is a ubiquitous component of modern visual pipelines, routinely applied by social media platforms and resource-constrained systems prior to inference. Despite its prevalence, the impact of compression on
adversarial robustness remains poorly understood. We study a previously unexplored
adversarial setting in which attacks are applied directly in compressed representations, and show that compression can act as an
adversarial amplifier for deep image classifiers. Under identical nominal perturbation budgets, compression-aware attacks are substantially more effective than their pixel-space counterparts. We attribute this effect to decision space reduction, whereby compression induces a non-invertible, information-losing transformation that contracts classification margins and increases sensitivity to perturbations. Extensive experiments across standard
benchmarks and architectures support our analysis and reveal a critical vulnerability in compression-in-the-loop deployment settings. Code will be released.
Read the full abstract ▾
All authors Lewis Evans, Harkrishan Jandu, Zihan Ye, Yang Lu, Shreyank N Gowda
Computer Vision
Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderation
Zhiheng Li, Zongyang Ma, Yuntong Pan et al. (+8 more)
Multimodal Large
Language Models (MLLMs) are increasingly being deployed as automated content moderators. Within this landscape, we uncover a critical threat:
Adversarial Smuggling Attacks. Unlike
adversarial perturbations (for misclassification) and
adversarial jailbreaks (for harmful output generation),
adversarial smuggling exploits the Human-AI capability gap. It encodes harmful content into human-readable visual formats that remain AI-unreadable, thereby evading automated detection and enabling the dissemination of harmful content. We classify smuggling attacks into two pathways: (1) Perceptual Blindness, disrupting text recognition; and (2)
Reasoning Blockade, inhibiting semantic understanding despite successful text recognition. To evaluate this threat, we constructed SmuggleBench, the first comprehensive
benchmark comprising 1,700
adversarial smuggling attack instances. Evaluations on SmuggleBench reveal that both proprietary (e.g., GPT-5) and open-source (e.g., Qwen3-VL) state-of-the-art models are vulnerable to this threat, producing Attack Success Rates (ASR) exceeding 90%. By analyzing the vulnerability through the lenses of perception and
reasoning, we identify three root causes: the limited capabilities of
vision encoders, the robustness gap in OCR, and the scarcity of domain-specific
adversarial examples. We conduct a preliminary exploration of mitigation strategies, investigating the potential of test-time scaling (via CoT) and
adversarial training (via SFT) to mitigate this threat. Our code is publicly available at https://github.com/zhihengli-casia/smugglebench.
Read the full abstract ▾
All authors Zhiheng Li, Zongyang Ma, Yuntong Pan, Ziqi Zhang, Xiaolei Lv, Bo Li, Jun Gao, Jianing Zhang, Chunfeng Yuan, Bing Li, Weiming Hu
Computer Vision
NTIRE 2026 Challenge on Bitstream-Corrupted Video Restoration: Methods and Results
Wenbin Zou, Tianyi Liu, Kejun Wu et al. (+37 more)
This paper reports on the NTIRE 2026 Challenge on Bitstream-Corrupted
Video Restoration (BSCVR). The challenge aims to advance research on recovering visually coherent
videos from corrupted bitstreams, whose decoding often produces severe spatial-temporal artifacts and content distortion. Built upon recent progress in bitstream-corrupted
video recovery, the challenge provides a common
benchmark for evaluating restoration methods under realistic corruption settings. We describe the
dataset, evaluation protocol, and participating methods, and summarize the final results and main technical trends. The challenge highlights the difficulty of this emerging task and provides useful insights for future research on robust
video restoration under practical bitstream corruption.
Read the full abstract ▾
All authors Wenbin Zou, Tianyi Liu, Kejun Wu, Huiping Zhuang, Zongwei Wu, Zhuyun Zhou, Radu Timofte, Kim-Hui Yap, Lap-Pui Chau, Yi Wang, Shiqi Zhou, Xiaodi Shi, Yuxiang Chen, Yilian Zhong, Shibo Yin, Yushun Fang, Xilei Zhu, Yahui Wang, Chen Lu, Zhitao Wang, Lifa Ha, Hengyu Man, Xiaopeng Fan, Priyansh Singh, Sidharth, Krrish Dev, Soham Kakkar, Vinit Jakhetiya, Ovais Iqbal Shah, Wei Zhou, Linfeng Li, Qi Xu, Zhenyang Liu, Kepeng Xu, Tong Qiao, Jiachen Tu, Guoyi Xu, Yaoxin Jiang, Jiajia Liu, Yaokun Shi