Computer Vision
IDDM: Identity-Decoupled Personalized Diffusion Models with a Tunable Privacy-Utility Trade-off
Linyan Dai, Xinwei Zhang, Haoyang Li et al. (+2 more)
Personalized text-to-image
diffusion models (e.g., DreamBooth, LoRA) enable users to synthesize high-fidelity avatars from a few reference photos for social expression. However, once these generations are shared on social media platforms (e.g., Instagram, Facebook), they can be linked to the real user via face recognition systems, enabling identity tracking and profiling. Existing defenses mainly follow an anti-personalization strategy that protects publicly released reference photos by disrupting model fine-tuning. While effective against unauthorized personalization, they do not address another practical setting in which personalization is authorized, but the resulting public outputs still leak identity information. To address this problem, we introduce a new defense setting, termed model-side output immunization, whose goal is to produce a personalized model that supports authorized personalization while reducing the identity linkability of public generations, with tunable control over the
privacy-utility trade-off to accommodate diverse
privacy needs. To this end, we propose Identity-Decoupled personalized
Diffusion Models (IDDM), a model-side defense that integrates identity decoupling into the personalization pipeline. Concretely, IDDM follows an alternating procedure that interleaves short personalization updates with identity-decoupled data optimization, using a two-stage schedule to balance identity linkability suppression and generation utility. Extensive experiments across multiple
datasets, diverse
prompts, and state-of-the-art face recognition systems show that IDDM consistently reduces identity linkability while preserving high-quality personalized generation.
Read the full abstract ▾
All authors Linyan Dai, Xinwei Zhang, Haoyang Li, Qingqing Ye, Haibo Hu
Computer Vision · Cross-listed from cs.LG
Super-Resolving Coarse-Resolution Weather Forecasts With Flow Matching
Aymeric Delefosse, Anastase Charantonis and Dominique Béréziat
Machine learning-based weather
forecasting models now surpass state-of-the-art numerical weather prediction systems, but training and operating these models at high spatial resolution remains computationally expensive. We present a modular framework that decouples
forecasting from spatial resolution by applying learned generative super-resolution as a post-processing step to coarse-resolution forecast trajectories. We formulate super-resolution as a
stochastic inverse problem, using a residual formulation to preserve large-scale structure while reconstructing unresolved variability. The model is trained with flow matching exclusively on reanalysis data and is applied to global medium-range forecasts. We evaluate (i) design consistency by re-coarsening super-resolved forecasts and comparing them to the original coarse trajectories, and (ii) high-resolution forecast quality using standard ensemble verification metrics and spectral diagnostics. Results show that super-resolution preserves large-scale structure and variance after re-coarsening, introduces physically consistent small-scale variability, and achieves competitive probabilistic forecast skill at 0.25{\deg} resolution relative to an operational ensemble baseline, while requiring only a modest additional training cost compared with end-to-end high-resolution
forecasting.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Beyond Symbolic Solving: Multi Chain-of-Thought Voting for Geometric Reasoning in Large Language Models
Md. Abu Bakor Siddique, Shahrin Hossain, Sadman Ahmed Siam et al. (+3 more)
Geometric Problem Solving (GPS) remains at the heart of enhancing mathematical
reasoning in large
language models because it requires the combination of diagrammatic understanding, symbolic manipulation and logical inference. In existing literature, researchers have chiefly focused on synchronising the diagram descriptions with text literals and solving the problem. In this vein, they have either taken a neural, symbolic or neuro-symbolic approach. But this solves only the first two of the requirements, namely diagrammatic understanding and symbolic manipulation, while leaving logical inference underdeveloped. The logical inference is often limited to one chain-of-thought (CoT). To address this weakness in hitherto existing models, this paper proposes MARS-GPS, that generates multiple parallel
reasoning rollouts augmented with Python code execution for numerical verification, ranks them using
token-level
entropy as a confidence signal, and aggregates answers through a multi-stage voting and self-verification pipeline. Empirical results show that MARS-GPS with 8 parallel rollouts achieves 88.8% on Geometry3K, a nearly +11% improvement over the prior state-of-the-art, with accuracy scaling consistently as the number of rollouts increases from 1 to 16 (+6.0% on ablation subset). We provide our code and data in an anonymous repository: https://anonymous.4open.science/r/MARS-GPS-DE55.
Read the full abstract ▾
All authors Md. Abu Bakor Siddique, Shahrin Hossain, Sadman Ahmed Siam, Syed Rifat Raiyan, Hasan Mahmud, Md Kamrul Hasan
Computer Vision
Towards Physically Realizable Adversarial Attenuation Patch against SAR Object Detection
Yiming Zhang, Weibo Qin and Feng Wang
Deep neural networks have demonstrated excellent performance in SAR target detection tasks but remain susceptible to
adversarial attacks. Existing SAR-specific attack methods can effectively deceive detectors; however, they often introduce noticeable perturbations and are largely confined to digital domain, neglecting physical implementation constrains for attacking SAR systems. In this paper, a novel
Adversarial Attenuation Patch (AAP) method is proposed that employs energy-constrained optimization strategy coupled with an attenuation-based deployment framework to achieve a seamless balance between attack effectiveness and stealthiness. More importantly, AAP exhibits strong potential for physical realization by aligning with signal-level electronic jamming mechanisms. Experimental results show that AAP effectively degrades detection performance while preserving high imperceptibility, and shows favorable transferability across different models. This study provides a physical grounded perspective for
adversarial attacks on SAR target detection systems and facilitates the design of more covert and practically deployable attack strategies. The source code is made available at https://github.com/boremycin/SAAP.
Read the full abstract ▾
Computer Vision
PixelPrune: Pixel-Level Adaptive Visual Token Reduction via Predictive Coding
Nan Wang, Zhiwei Jin, Chen Chen et al. (+1 more)
Document understanding and GUI interaction are among the highest-value applications of
Vision-
Language Models (VLMs), yet they impose exceptionally heavy computational burden: fine-grained text and small UI elements demand high-resolution inputs that produce tens of thousands of visual
tokens. We observe that this cost is largely wasteful – across document and GUI
benchmarks, only 22–71% of image patches are pixel-unique, the rest being exact duplicates of another patch in the same image. We propose PixelPrune, which exploits this pixel-level redundancy through predictive-coding-based compression,
pruning redundant patches before the
Vision Transformer (ViT) encoder. Because it operates in pixel space prior to any neural computation, PixelPrune accelerates both the ViT encoder and the downstream LLM, covering the full inference pipeline. The method is training-free, requires no learnable parameters, and supports pixel-lossless compression ($\tau{=}0$) as well as controlled lossy compression ($\tau{>}0$). Experiments across three model scales and document and GUI
benchmarks show that PixelPrune maintains competitive task accuracy while delivering up to 4.2$\times$ inference speedup and 1.9$\times$ training acceleration. Code is available at https://github.com/OPPO-Mente-Lab/PixelPrune.
Read the full abstract ▾
All authors Nan Wang, Zhiwei Jin, Chen Chen, Haonan Lu
Computer Vision
A 4D Representation for Training-Free Agentic Reasoning from Monocular Laparoscopic Video
Maximilian Fehrentz, Nicolas Stellwag, Robert Wiebe et al. (+7 more)
Spatiotemporal
reasoning is a fundamental capability for artificial intelligence (AI) in soft tissue surgery, paving the way for intelligent assistive systems and autonomous robotics. While 2D
vision-
language models show increasing promise at understanding
surgical video, the spatial complexity of
surgical scenes suggests that
reasoning systems may benefit from explicit 4D representations. Here, we propose a framework for equipping
surgical agents with spatiotemporal tools based on an explicit 4D representation, enabling AI systems to ground their natural
language reasoning in both time and 3D space. Leveraging models for point tracking, depth, and
segmentation, we develop a coherent 4D model with spatiotemporally consistent tool and tissue semantics. A
Multimodal Large
Language Model (MLLM) then acts as an
agent on tools derived from the explicit 4D representation (e.g., trajectories) without any fine-tuning. We evaluate our method on a new
dataset of 134 clinically relevant questions and find that the combination of a general purpose
reasoning backbone and our 4D representation significantly improves spatiotemporal understanding and allows for 4D grounding. We demonstrate that spatiotemporal intelligence can be "assembled" from 2D MLLMs and 3D computer
vision models without additional training. Code, data, and examples are available at https://tum-ai.github.io/surg4d/
Read the full abstract ▾
All authors Maximilian Fehrentz, Nicolas Stellwag, Robert Wiebe, Nicole Thorisch, Fabian Grob, Patrick Remerscheid, Ken-Joel Simmoteit, Benjamin D. Killeen, Christian Heiliger, Nassir Navab
Computer Vision
Shape Representation using Gaussian Process mixture models
Panagiotis Sapoutzoglou, George Terzakis, Georgios Floros et al. (+1 more)
Traditional explicit 3D representations, such as point clouds and meshes, demand significant storage to capture fine
geometric details and require complex indexing systems for surface lookups, making functional representations an efficient, compact, and continuous alternative. In this work, we propose a novel, object-specific functional shape representation that models surface geometry with Gaussian Process (GP) mixture models. Rather than relying on computationally heavy neural architectures, our method is lightweight, leveraging GPs to learn continuous directional distance fields from sparsely sampled point clouds. We capture complex
topologies by anchoring local GP priors at strategic reference points, which can be flexibly extracted using any structural decomposition method (e.g. skeletonization, distance-based clustering). Extensive evaluations on the ShapeNetCore and IndustryShapes
datasets demonstrate that our method can efficiently and accurately represent complex geometries.
Read the full abstract ▾
All authors Panagiotis Sapoutzoglou, George Terzakis, Georgios Floros, Maria Pateraki
Computer Vision
Sparkle: A Robust and Versatile Representation for Point Cloud based Human Motion Capture
Yiming Ren, Yujing Sun, Aoru Xue et al. (+2 more)
Point cloud-based motion capture leverages rich spatial geometry and
privacy-preserving sensing, but learning robust representations from noisy, unstructured point clouds remains challenging. Existing approaches face a struggle trade-off between point-based methods (geometrically detailed but noisy) and skeleton-based ones (robust but oversimplified). We address the fundamental challenge: how to construct an effective representation for human motion capture that can balance expressiveness and robustness. In this paper, we propose Sparkle, a structured representation unifying skeletal joints and surface anchors with explicit kinematic-
geometric factorization. Our framework, SparkleMotion, learns this representation through hierarchical modules
embedding geometric continuity and kinematic constraints. By explicitly disentangling internal kinematic structure from external surface geometry, SparkleMotion achieves state-of-the-art performance not only in accuracy but crucially in robustness and generalization under severe domain shifts, noise, and occlusion. Extensive experiments demonstrate our superiority across diverse sensor types and challenging real-world scenarios.
Read the full abstract ▾
All authors Yiming Ren, Yujing Sun, Aoru Xue, Kwok-Yan Lam, Yuexin Ma
Computer Vision
Perturb-and-Restore: Simulation-driven Structural Augmentation Framework for Imbalance Chromosomal Anomaly Detection
Yilan Zhang, Hanbiao Chen, Changchun Yang et al. (+10 more)
Detecting structural chromosomal abnormalities is crucial for accurate diagnosis and management of genetic disorders. However, collecting sufficient structural abnormality data is extremely challenging and costly in clinical practice, and not all abnormal types can be readily collected. As a result, deep learning approaches face significant performance degradation due to the severe imbalance and scarcity of abnormal chromosome data. To address this challenge, we propose a Perturb-and-Restore (P&R), a simulation-driven structural augmentation framework that effectively alleviates data imbalance in chromosome anomaly detection. The P&R framework comprises two key components: (1) Structure Perturbation and Restoration Simulation, which generates synthetic abnormal chromosomes by perturbing chromosomal banding patterns of normal chromosomes followed by a restoration
diffusion network that reconstructs continuous chromosome content and edges, thus eliminating reliance on rare abnormal samples; and (2) Energy-guided Adaptive Sampling, an energy score-based online selection strategy that dynamically prioritizes high-quality synthetic samples by referencing the energy distribution of real samples. To evaluate our method, we construct a comprehensive structural anomaly
dataset consisting of over 260,000 chromosome images, including 4,242 abnormal samples spanning 24 categories. Experimental results demonstrate that the P&R framework achieves state-of-the-art (SOTA) performance, surpassing existing methods with an average improvement of 8.92% in sensitivity, 8.89% in precision, and 13.79% in F1-score across all categories.
Read the full abstract ▾
All authors Yilan Zhang, Hanbiao Chen, Changchun Yang, Yuetan Chu, Siyuan Chen, Jing Wu, Jingdong Hu, Na Li, Junkai Su, Yuxuan Chen, Ao Xu, Xin Gao, Aihua Yin
Computer Vision
MotionGrounder: Grounded Multi-Object Motion Transfer via Diffusion Transformer
Samuel Teodoro, Yun Chen, Agus Gunawan et al. (+3 more)
Motion transfer enables controllable
video generation by transferring temporal dynamics from a reference
video to synthesize a new
video conditioned on a target caption. However, existing
Diffusion Transformer (DiT)-based methods are limited to single-object
videos, restricting fine-grained control in real-world scenes with multiple objects. In this work, we introduce MotionGrounder, a DiT-based framework that firstly handles motion transfer with multi-object controllability. Our Flow-based Motion Signal (FMS) in MotionGrounder provides a stable motion prior for target
video generation, while our Object-Caption
Alignment Loss (OCAL) grounds object captions to their corresponding spatial regions. We further propose a new Object Grounding Score (OGS), which jointly evaluates (i) spatial
alignment between source
video objects and their generated counterparts and (ii) semantic consistency between each generated object and its target caption. Our experiments show that MotionGrounder consistently outperforms recent baselines across quantitative, qualitative, and human evaluations.
Read the full abstract ▾
All authors Samuel Teodoro, Yun Chen, Agus Gunawan, Soo Ye Kim, Jihyong Oh, Munchurl Kim
Computer Vision
Disentangling to Re-couple: Resolving the Similarity-Controllability Paradox in Subject-Driven Text-to-Image Generation
Shuang Li, Chao Deng, Hang Chen et al. (+8 more)
Subject-Driven Text-to-Image (T2I) Generation aims to preserve a subject's identity while editing its context based on a text
prompt. A core challenge in this task is the "similarity-controllability paradox", where enhancing textual control often degrades the subject's fidelity, and vice-versa. We argue this paradox stems from the ambiguous role of text
prompts, which are often tasked with describing both the subject and the desired modifications, leading to conflicting signals for the model. To resolve this, we propose DisCo, a novel framework that first Disntangles and then re-Couples visual and textual information. First, our textual-visual decoupling module isolates the sources of information: subject identity is extracted exclusively from the reference image with the entity word of the subject, while the text
prompt is simplified to contain only the modification command, where the subject refers to general pronouns, eliminating descriptive ambiguity. However, this strict separation can lead to unnatural compositions between the subject and its contexts. We address this by designing a dedicated reward signal and using reinforcement learning to seamlessly recouple the visually-defined subject and the textually-generated context. Our approach effectively resolves the paradox, enabling simultaneous high-fidelity subject preservation and precise textual control. Extensive experiments demonstrate that our method achieves state-of-the-art performance, producing highly realistic and coherent images.
Read the full abstract ▾
All authors Shuang Li, Chao Deng, Hang Chen, Liqun Liu, Zhenyu Hu, Te Cao, Mengge Xue, Yuan Chen, Peng Shu, Huan Yu, Jie Jiang
Computer Vision
LinguDistill: Recovering Linguistic Ability in Vision-Language Models via Selective Cross-Modal Distillation
Patrick Amadeus Irawan, Erland Hilman Fuadi, Shanu Kumar et al. (+2 more)
Turning a pretrained
language model (LM) into a
vision-
language model (VLM) through
multimodal fine-tuning often erodes its native
language ability, a form of catastrophic forgetting that shows up even on text-only tasks. This loss is hard to undo with further fine-tuning, and existing remedies add adapters or
alignment modules that increase architectural complexity and inference cost. We propose LinguDistill, an adapter-free knowledge
distillation method that uses the original frozen LM as the teacher during
multimodal post-training. To let a text-only teacher supervise
vision-conditioned outputs, we introduce layer-wise KV-cache sharing, which exposes the teacher to the student's
multimodal representations without changing either architecture. We then apply
distillation selectively, on
language-heavy data only, so the teacher restores linguistic ability while the student keeps its visual grounding on document and OCR tasks. LinguDistill recovers the
language and knowledge performance lost during
multimodal fine-tuning, matching the original VLM on average over text-only
benchmarks (ARC, HellaSwag) and exceeding it on ScienceQA, while keeping
vision-heavy performance close to standard fine-tuning. Since the teacher is dropped after training, the final model adds no parameters and no inference cost. More broadly, our results show that a model's own pre-adaptation backbone is a practical teacher for undoing forgetting, suggesting a simple recipe for keeping
language ability intact as models are extended to new modalities.
Read the full abstract ▾
All authors Patrick Amadeus Irawan, Erland Hilman Fuadi, Shanu Kumar, Alham Fikri Aji, Yova Kementchedjhieva
Computer Vision
Patrick Glandorf, Thomas Norrenbrock and Bodo Rosenhahn
Vision Transformers (ViTs) have demonstrated state-ofthe-art performance in several
benchmarks, yet their high computational costs hinders their practical deployment. Patch
Pruning offers significant savings, but existing approaches restrict
token reduction to deeper layers, leaving early-stage compression unexplored. This limits their potential for holistic efficiency. In this work, we present a novel
Video Patch
Pruning framework (VPP) that integrates temporal prior knowledge to enable efficient sparsity within early ViT layers. Our approach is motivated by the observation that prior features extracted from deeper layers exhibit strong foreground selectivity. Therefore we propose a fully differentiable module for temporal mapping to accurately select the most relevant patches in early network stages. Notably, the proposed method enables a patch reduction of up to 60% in dense prediction tasks, exceeding the capabilities of conventional image-based patch
pruning, which typically operate around a 30% patch sparsity. VPP excels the high-sparsity regime, sustaining remarkable performance even when patch usage is reduced below 55%. Specifically, it preserves stable results with a maximal performance drop of 0.6% on the Youtube-VIS 2021
dataset.
Read the full abstract ▾
Computer Vision
Continual Vision-Language Learning for Remote Sensing: Benchmarking and Analysis
Xingxing Weng, Ruifeng Ni, Chao Pang et al. (+5 more)
Current remote sensing
vision-
language models (RS VLMs) demonstrate impressive performance in image interpretation but rely on static training data, limiting their ability to accommodate continuously emerging sensing modalities and downstream tasks. This exposes a fundamental challenge: enabling RS VLMs to continually adapt without catastrophic forgetting. Despite its practical importance, the continual learning capability of RS VLMs remains underexplored, and no dedicated
benchmark currently exists. In this work, we present CLeaRS, a comprehensive
benchmark for continual
vision-
language learning in remote sensing. CLeaRS comprises 10 curated subsets with over 207k image-text pairs, spanning diverse interpretation tasks, sensing modalities, and application scenarios. We further define three evaluation protocols: long-horizon, modality-incremental, and task-incremental settings, to systematically assess continual adaptation. Extensive benchmarking of diverse
vision-
language models reveals catastrophic forgetting across all settings. Moreover, representative continual learning methods, when adapted to RS VLMs, exhibit limited effectiveness in handling task, instruction, and modality transitions. Our findings underscore the need for developing continual learning methods tailored to RS VLMs.
Read the full abstract ▾
All authors Xingxing Weng, Ruifeng Ni, Chao Pang, XiangYu Hao, Yishan Wang, Xiaokang Zhang, Wei Xu, Gui-Song Xia
Computer Vision
Multicentric thrombus segmentation using an attention-based recurrent network with gradual modality dropout
Sofia Vargas-Ibarra, Vincent Vigneron, Hichem Maaref et al. (+1 more)
Detecting and delineating tiny targets in 3D
brain scans is a central yet under-addressed challenge in medical imaging.In ischemic stroke, for instance, the culprit thrombus is small, low-contrast, and variably expressed across modalities(e.g., susceptibility-weighted T2 blooming,
diffusion restriction on DWI/ADC), while real-world multi-center dataintroduce domain shifts, anisotropy, and frequent missing sequences. We introduce a methodology that couples an
attention-based recurrent
segmentation network (UpAttLLSTM), a training schedule that progressively increases the difficulty of hetero-modal learning, with gradual modality dropout, UpAttLLSTM aggregates context across slices via recurrent units (2.5D) and uses
attention gates to fuse complementary cues across available sequences, making it robust to anisotropy and class imbalance. Gradual modality dropout systematically simulates site heterogeneity,noise, and missing modalities during training, acting as both augmentation and regularization to improve multi-center generalization. On a monocentric cohort, our approach detects thrombi in >90% of cases with a Dice score of 0.65. In a multi-center setting with missing modalities, it achieves-80% detection with a Dice score around 0.35. Beyond stroke, the proposed methodology directly transfers to other small-lesion tasks in 3D medical imaging where targets are scarce, subtle, and modality-dependent
Read the full abstract ▾
All authors Sofia Vargas-Ibarra, Vincent Vigneron, Hichem Maaref, Sonia Garcia-Salicetti
Computer Vision
DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale
Sicheng Zuo, Zixun Xie, Wenzhao Zheng et al. (+6 more)
End-to-end autonomous driving has evolved from the conventional paradigm based on sparse perception into
vision-
language-action (VLA) models, which focus on learning
language descriptions as an auxiliary task to facilitate planning. In this paper, we propose an alternative
Vision-Geometry-Action (VGA) paradigm that advocates dense 3D geometry as the critical cue for autonomous driving. As vehicles operate in a 3D world, we think dense 3D geometry provides the most comprehensive information for decision-making. However, most existing geometry reconstruction methods (e.g., DVGT) rely on computationally expensive batch processing of multi-frame inputs and cannot be applied to online planning. To address this, we introduce a streaming Driving Visual Geometry
Transformer (DVGT-2), which processes inputs in an online manner and jointly outputs dense geometry and trajectory planning for the current frame. We employ temporal causal
attention and cache historical features to support on-the-fly inference. To further enhance efficiency, we propose a sliding-window streaming strategy and use historical caches within a certain interval to avoid repetitive computations. Despite the faster speed, DVGT-2 achieves superior geometry reconstruction performance on various
datasets. The same trained DVGT-2 can be directly applied to planning across diverse camera configurations without fine-tuning, including closed-loop NAVSIM and open-loop nuScenes
benchmarks.
Read the full abstract ▾
All authors Sicheng Zuo, Zixun Xie, Wenzhao Zheng, Shaoqing Xu, Fang Li, Hanbing Li, Long Chen, Zhi-Xin Yang, Jiwen Lu
Computer Vision
Revisiting Human-in-the-Loop Object Retrieval with Pre-Trained Vision Transformers
Kawtar Zaher, Olivier Buisson and Alexis Joly
Building on existing approaches, we revisit Human-in-the-Loop Object
Retrieval, a task that consists of iteratively retrieving images containing objects of a class-of-interest, specified by a user-provided query. Starting from a large unlabeled image collection, the aim is to rapidly identify diverse instances of an object category relying solely on the initial query and the user's Relevance Feedback, with no prior labels. The
retrieval process is formulated as a binary classification task, where the system continuously learns to distinguish between relevant and non-relevant images to the query, through iterative user interaction. This interaction is guided by an Active Learning loop: at each iteration, the system selects informative samples for user annotation, thereby refining the
retrieval performance. This task is particularly challenging in multi-object
datasets, where the object of interest may occupy only a small region of the image within a complex, cluttered scene. Unlike object-centered settings where global descriptors often suffice, multi-object images require more adapted, localized descriptors. In this work, we formulate and revisit the Human-in-the-Loop Object
Retrieval task by leveraging pre-trained ViT representations, and addressing key design questions, including which object instances to consider in an image, what form the annotations should take, how Active Selection should be applied, and which representation strategies best capture the object's features. We compare several representation strategies across multi-object
datasets highlighting trade-offs between capturing the global context and focusing on fine-grained local object details. Our results offer practical insights for the design of effective interactive
retrieval pipelines based on Active Learning for object class
retrieval.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Compact Keyframe-Optimized Multi-Agent Gaussian Splatting SLAM
Monica M.Q. Li, Pierre-Yves Lajoie, Jialiang Liu et al. (+1 more)
Efficient multi-
agent 3D mapping is essential for robotic teams operating in unknown environments, but dense representations hinder real-time exchange over constrained communication links. In multi-
agent Simultaneous Localization and Mapping (SLAM), systems typically rely on a centralized server to merge and optimize the local maps produced by individual
agents. However, sharing these large map representations, particularly those generated by recent methods such as Gaussian Splatting, becomes a bottleneck in real-world scenarios with limited bandwidth. We present an improved multi-
agent RGB-D Gaussian Splatting SLAM framework that reduces communication load while preserving map fidelity. First, we incorporate a compaction step into our SLAM system to remove redundant 3D Gaussians, without degrading the rendering quality. Second, our approach performs centralized loop closure computation without initial guess, operating in two modes: a pure rendered-depth mode that requires no data beyond the 3D Gaussians, and a camera-depth mode that includes lightweight depth images for improved registration accuracy and additional Gaussian
pruning. Evaluation on both synthetic and real-world
datasets shows up to 85-95% reduction in transmitted data compared to state-of-the-art approaches in both modes, bringing 3D Gaussian multi-
agent SLAM closer to practical deployment in real-world scenarios. Code: https://github.com/lemonci/coko-slam
Read the full abstract ▾
All authors Monica M.Q. Li, Pierre-Yves Lajoie, Jialiang Liu, Giovanni Beltrame
Computer Vision
Multimodal Language Models Cannot Spot Spatial Inconsistencies
Om Khangaonkar, Hadi J. Rad and Hamed Pirsiavash
Spatial consistency is a fundamental property of the visual world and a key requirement for models that aim to understand physical reality. Despite recent advances,
multimodal large
language models (MLLMs) often struggle to reason about 3D geometry across multiple views. Rather than asking models to describe scene attributes, we introduce a more challenging task: given two views of the same scene, identify the object that violates 3D motion consistency. We propose a simple and scalable method for generating realistic, spatially inconsistent image pairs from multi-view scenes, enabling systematic evaluation of this capability. Our results show that state-of-the-art MLLMs significantly underperform human observers and exhibit substantial variability across different scene attributes, revealing a fragile and incomplete understanding of 3D structure. We hope our findings underscore the need for approaches that develop a more deeply grounded understanding of the physical world.
Read the full abstract ▾
Computer Vision
HICT: High-precision 3D CBCT reconstruction from a single X-ray
Wen Ma, Jiaxiang Liu, Zikai Xiao et al. (+3 more)
Accurate 3D dental imaging is vital for diagnosis and treatment planning, yet CBCT's high radiation dose and cost limit its accessibility. Reconstructing 3D volumes from a single low-dose panoramic X-ray is a promising alternative but remains challenging due to
geometric inconsistencies and limited accuracy. We propose HiCT, a two-stage framework that first generates geometrically consistent multi-view projections from a single panoramic image using a
video diffusion model, and then reconstructs high-fidelity CBCT from the projections using a ray-based dynamic
attention network and an X-ray sampling strategy. To support this, we built XCT, a large-scale
dataset combining public CBCT data with 500 paired PX-CBCT cases. Extensive experiments show that HiCT achieves state-of-the-art performance, delivering accurate and geometrically consistent reconstructions for clinical use.
Read the full abstract ▾
All authors Wen Ma, Jiaxiang Liu, Zikai Xiao, Ziyang Wang, Feng Yang, Zuozhu Liu
Computer Vision
An Approach to Enriching Surgical Video Datasets for Fine-Grained Spatial-Temporal Understanding of Vision-Language Models
Lennart Maack and Alexander Schlaefer
Surgical video understanding is a crucial prerequisite for advancing Computer-Assisted Surgery. While
vision-
language models (VLMs) have recently been applied to the
surgical domain, existing
surgical vision-
language datasets lack in capturing and evaluating complex, interleaved spatial-temporal dynamics. Creating large scale
datasets that accurately represent fine-grained spatial-temporal relationships in
surgical videos is challenging due to costly manual annotations or error-prone generation using large
language models. To address this gap, we introduce the SurgSTU-Pipeline, a deterministic generation pipeline featuring temporal and spatial continuity filtering to reliably create
surgical datasets for fine-grained spatial-temporal
multimodal understanding. Applying this pipeline to publicly available
surgical datasets, we create the SurgSTU
dataset, comprising 6711
video clips densely extended with 150k fine-grained spatial-temporal question-answer samples. Our comprehensive evaluation shows that while state-of-the-art generalist VLMs struggle in zero-shot settings, their spatial-temporal capabilities can be improved through in-context learning. A fine-tuned VLM on the SurgSTU training
dataset achieves highest performance among all spatial-temporal tasks, validating the
dataset's efficacy to improve spatial-temporal understanding of VLMs in
surgical videos. The project is available here: https://lennart-maack.github.io/SurgSTU-project
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Using predefined vector systems to speed up neural network multimillion class classification
Nikita Gabdullin and Ilya Androsov
Label prediction in neural networks (NNs) has O(n) complexity proportional to the number of classes. This holds true for classification using fully connected layers and cosine similarity with some set of class prototypes. In this paper we show that if NN latent space (LS) geometry is known and possesses specific properties, label prediction complexity can be significantly reduced. This is achieved by associating label prediction with the O(1) complexity closest cluster center search in a vector system used as target for latent space configuration (LSC). The proposed method only requires finding indexes of several largest and lowest values in the
embedding vector making it extremely computationally efficient. We show that the proposed method does not change NN training accuracy computational results. We also measure the time required by different computational stages of NN inference and label prediction on multiple
datasets. The experiments show that the proposed method allows to achieve up to 11.6 times overall acceleration over conventional methods. Furthermore, the proposed method has unique properties which allow to predict the existence of new classes.
Read the full abstract ▾
Computer Vision
PrivHAR-Bench: A Graduated Privacy Benchmark Dataset for Video-Based Action Recognition
Samar Ansari
Existing research on
privacy-preserving Human Activity Recognition (HAR) typically evaluates methods against a binary paradigm: clear
video versus a single
privacy transformation. This limits cross-method comparability and obscures the nuanced relationship between
privacy strength and recognition utility. We introduce PrivHAR-Bench, a multi-tier
benchmark dataset designed to standardize the evaluation of the
Privacy-Utility Trade-off in
video-based action recognition. PrivHAR-Bench applies a graduated spectrum of visual
privacy transformations: from lightweight spatial obfuscation to cryptographic block permutation, to a curated subset of 15 activity classes selected for human articulation diversity. Each of the 1,932 source
videos is distributed across 9 parallel tiers of increasing
privacy strength, with additional background-removed variants to isolate the contribution of human motion features from contextual scene bias. We provide lossless frame sequences, per-frame bounding boxes, estimated pose keypoints with joint-level confidence scores, standardized group-based train/test splits, and an evaluation toolkit computing recognition accuracy and
privacy metrics. Empirical validation using R3D-18 demonstrates a measurable and interpretable degradation curve across tiers, with within-tier accuracy declining from 88.8% (clear) to 53.5% (encrypted, background-removed) and cross-domain accuracy collapsing to 4.8%, establishing PrivHAR-Bench as a controlled
benchmark for comparing
privacy-preserving HAR methods under standardized conditions. The
dataset, generation pipeline, and evaluation code are publicly available.
Read the full abstract ▾
Computer Vision
Dong-Jae Lee, Sunghyun Baek and Junmo Kim
Large
Vision Language Models show impressive performance across image and
video understanding tasks, yet their computational cost grows rapidly with the number of visual
tokens. Existing
token pruning methods mitigate this issue through empirical approaches while overlooking the internal mechanism of
attention. In this paper, we propose a novel training free
token pruning framework grounded in the dual form perspective of
attention. We reformulate
attention as an implicit linear layer whose weight
matrix is the sum of rank 1 outer products, each generated by a single
token's key value pair.
Token pruning thus reduces to selecting an optimal subset of these rank 1 updates that best approximates the original dual weight
matrix. Extending this perspective to standard softmax
attention in LVLMs, we derive a novel metric quantifying both a
token's information magnitude and information duplication. To efficiently select the subset with the proposed metric, we introduce Progressive Chunked Maximal Marginal Relevance. Extensive experiments demonstrate that our method achieves a better trade off between performance and efficiency, while providing another perspective on existing
pruning approaches.
Read the full abstract ▾
Computer Vision
A Benchmark of State-Space Models vs. Transformers and BiLSTM-based Models for Historical Newspaper OCR
Merveilles Agbeti-Messan, Pierrick Tranouez, Stéphane Nicolas et al. (+2 more)
End-to-end OCR for historical newspapers remains challenging, as models must handle long text sequences, degraded print quality, and complex layouts. While
Transformer-based recognizers dominate current research, their quadratic complexity limits efficient paragraph-level transcription and large-scale deployment. We investigate linear-time State-Space Models (SSMs), specifically Mamba, as a scalable alternative to
Transformer-based sequence modeling for OCR. We present to our knowledge, the first OCR architecture based on SSMs, combining a CNN visual encoder with bi-directional and autoregressive Mamba sequence modeling, and conduct a large-scale
benchmark comparing SSMs with
Transformer- and BiLSTM-based recognizers. Multiple decoding strategies (CTC, autoregressive, and non-autoregressive) are evaluated under identical training conditions alongside strong neural baselines (VAN, DAN, DANIEL) and widely used off-the-shelf OCR engines (PERO-OCR, Tesseract OCR, TrOCR, Gemini). Experiments on historical newspapers from the Bibliotheque nationale du Luxembourg, with newly released >99% verified gold-standard annotations, and cross-
dataset tests on Fraktur and Antiqua lines, show that all neural models achieve low error rates (~2% CER), making computational efficiency the main differentiator. Mamba-based models maintain competitive accuracy while halving inference time and exhibiting superior
memory scaling (1.26x vs 2.30x growth at 1000 chars), reaching 6.07% CER at the severely degraded paragraph level compared to 5.24% for DAN, while remaining 2.05x faster. We release code, trained models, and standardized evaluation protocols to enable reproducible research and guide practitioners in large-scale cultural heritage OCR available at https://github.com/MarcoPerson/ssm-ocr-
benchmark.
Read the full abstract ▾
All authors Merveilles Agbeti-Messan, Pierrick Tranouez, Stéphane Nicolas, Clément Chatelain, Thierry Paquet
Computer Vision
TTA-Vid: Generalized Test-Time Adaptation for Video Reasoning
Soumya Shamarao Jahagirdar, Edson Araujo, Anna Kukleva et al. (+7 more)
Recent
video reasoning models have shown strong results on temporal and
multimodal understanding, yet they depend on large-scale supervised data and multi-stage training pipelines, making them costly to train and difficult to adapt to new domains. In this work, we leverage the paradigm of Test-Time Reinforcement Learning on
video-
language data to allow for adapting a pretrained model to incoming
video samples at test-time without explicit labels. The proposed test-time adaptation for
video approach (TTA-Vid) combines two components that work simultaneously: (1) a test-time adaptation that performs step-by-step
reasoning at inference time on multiple frame subsets. We then use a batch-aware frequency-based reward computed across different frame subsets as pseudo ground truth to update the model. It shows that the resulting model trained on a single batch or even a single sample from a
dataset, is able to generalize at test-time to the whole
dataset and even across
datasets. Because the adaptation occurs entirely at test time, our method requires no ground-truth annotations or dedicated training splits. Additionally, we propose a multi-armed bandit strategy for adaptive frame selection that learns to prioritize informative frames, guided by the same reward formulation. Our evaluation shows that TTA-Vid yields consistent improvements across various
video reasoning tasks and is able to outperform current state-of-the-art methods trained on large-scale data. This highlights the potential of test-time reinforcement learning for temporal
multimodal understanding.
Read the full abstract ▾
All authors Soumya Shamarao Jahagirdar, Edson Araujo, Anna Kukleva, M. Jehanzeb Mirza, Saurabhchand Bhati, Samuel Thomas, Brian Kingsbury, Rogerio Feris, James R. Glass, Hilde Kuehne
Computer Vision
TP-Seg: Task-Prototype Framework for Unified Medical Lesion Segmentation
Jiawei Xu, Qiangqiang Zhou, Dandan Zhu et al. (+3 more)
Building a unified model with a single set of parameters to efficiently handle diverse types of medical lesion
segmentation has become a crucial objective for AI-assisted diagnosis. Existing unified
segmentation approaches typically rely on shared encoders across heterogeneous tasks and modalities, which often leads to feature
entanglement, gradient interference, and suboptimal lesion discrimination. In this work, we propose TP-Seg, a task-prototype framework for unified medical lesion
segmentation. On one hand, the task-conditioned adapter effectively balances shared and task-specific representations through a dual-path expert structure, enabling adaptive feature extraction across diverse medical imaging modalities and lesion types. On the other hand, the prototype-guided task decoder introduces learnable task prototypes as semantic anchors and employs a cross-
attention mechanism to achieve fine-grained modeling of task-specific foreground and background semantics. Without bells and whistles, TP-Seg consistently outperforms specialized, general and unified
segmentation methods across 8 different medical lesion
segmentation tasks covering multiple imaging modalities, demonstrating strong generalization, scalability and clinical applicability.
Read the full abstract ▾
All authors Jiawei Xu, Qiangqiang Zhou, Dandan Zhu, Yong Chen, Yugen Yi, Xiaoqi Zhao
Computer Vision
MoonAnything: A Vision Benchmark with Large-Scale Lunar Supervised Data
Clémentine Grethen, Yuang Shi, Simone Gasparini et al. (+1 more)
Accurate perception of lunar surfaces is critical for modern lunar exploration missions. However, developing robust learning-based perception systems is hindered by the lack of
datasets that provide both
geometric and photometric supervision. Existing lunar
datasets typically lack either
geometric ground truth, photometric realism, illumination diversity, or large-scale coverage. In this paper, we introduce MoonAnything, a unified
benchmark built on real lunar topography with physically-based rendering, providing the first comprehensive
geometric and photometric supervision under diverse illumination with large scale. The
benchmark comprises two complementary sub-
datasets : i) LunarGeo provides stereo images with corresponding dense depth maps and camera calibration enabling 3D reconstruction and pose estimation; ii) LunarPhoto provides photorealistic images using a spatially-varying BRDF model, along with multi-illumination renderings under real
solar configurations, enabling reflectance estimation and illumination-robust perception. Together, these
datasets offer over 130K samples with comprehensive supervision. Beyond lunar applications, MoonAnything offers a unique setting and challenging testbed for
algorithms under low-textured, high-contrast conditions and applies to other airless celestial bodies and could generalize beyond. We establish baselines using state-of-the-art methods and release the complete
dataset along with generation tools to support community extension: https://github.com/clementinegrethen/MoonAnything.
Read the full abstract ▾
All authors Clémentine Grethen, Yuang Shi, Simone Gasparini, Géraldine Morin
Computer Vision
CL-VISTA: Benchmarking Continual Learning in Video Large Language Models
Haiyang Guo, Yichen Shi, Fei Zhu et al. (+6 more)
Video Large
Language Models (
Video-LLMs) require continual learning to adapt to non-stationary real-world data. However, existing
benchmarks fall short of evaluating modern foundation models: many still rely on models without large-scale pre-training, and prevailing
benchmarks typically partition a single
dataset into sub-tasks, resulting in high task redundancy and negligible forgetting on pre-trained
Video-LLMs. To address these limitations, we propose CL-VISTA, a
benchmark tailored for continual
video understanding of
Video-LLMs. By curating 8 diverse tasks spanning perception, understanding, and
reasoning, CL-VISTA induces substantial distribution shifts that effectively expose catastrophic forgetting. To systematically assess CL methods, we establish a comprehensive evaluation framework comprising 6 distinct protocols across 3 critical dimensions: performance, computational efficiency, and
memory footprint. Notably, the performance dimension incorporates a general
video understanding assessment to assess whether CL methods genuinely enhance foundational intelligence or merely induce task-specific overfitting. Extensive benchmarking of 10 mainstream CL methods reveals a fundamental trade-off: no single approach achieves universal superiority across all dimensions. Methods that successfully mitigate catastrophic forgetting tend to compromise generalization or incur prohibitive computational and
memory overheads. We hope CL-VISTA provides critical insights for advancing continual learning in
multimodal foundation models.
Read the full abstract ▾
All authors Haiyang Guo, Yichen Shi, Fei Zhu, Wenzhuo Liu, Hongbo Zhao, Fanhu Zeng, Shijie Ma, Da-Han Wang, Xu-Yao Zhang
Computer Vision
When AI and Experts Agree on Error: Intrinsic Ambiguity in Dermatoscopic Images
Loris Cino, Pier Luigi Mazzeo, Alessandro Martella et al. (+3 more)
The integration of artificial intelligence (AI), particularly Convolutional Neural Networks (CNNs), into dermatological diagnosis demonstrates substantial clinical potential. While existing literature predominantly
benchmarks algorithmic performance against human experts, our study adopts a novel perspective by investigating the intrinsic complexity of dermatoscopic images. Through rigorous experimentation with multiple CNN architectures, we isolated a subset of images systematically misclassified across all models-a phenomenon statistically proven to exceed random chance. To determine if these failures stem from algorithmic biases or inherent visual ambiguity, expert dermatologists independently evaluated these challenging cases alongside a control group. The results revealed a collapse in human diagnostic performance on the AI-misclassified images. First, agreement with ground-truth labels plummeted, with Cohen's kappa dropping to a mere 0.08 for the difficult images, compared to a 0.61 for the control group. Second, we observed a severe deterioration in expert consensus; inter-rater reliability among physicians fell from moderate concordance (Fleiss kappa = 0.456) on control images to only modest agreement (Fleiss kappa = 0.275) on difficult cases. We identified image quality as a primary driver of these dual systematic failures. To promote transparency and reproducibility, all data, code, and trained models have been made publicly available
Read the full abstract ▾
All authors Loris Cino, Pier Luigi Mazzeo, Alessandro Martella, Giulia Radi, Renato Rossi, Cosimo Distante