Computer Vision
CAIRN: Cross-Room 3D Scene Understanding with Topology-Aware Large Multimodal Models
He Liang, Chenyang Ma, Yiming Zhang et al. (+4 more)
Existing 3D scene-grounded Large
Language Models (3D-LLMs) focus on answering questions grounded in simplified single-room 3D scenes, lacking the ability to reason over real-world household environments containing multiple interconnected rooms and diverse object categories. We introduce CAIRN, a
topology-aware 3D-LLM for multi-room 3D scene understanding. CAIRN aligns
transformer attention with scene hierarchy, giving the model explicit awareness of object-level relations and room-level connectivity. It enriches object
tokens with room-local relational context via a graph neural network, introduces learned room
tokens for room-level abstraction, and applies a hierarchical
attention mask with
geometric bias to route information according to scene
topology. CAIRN is developed on CAIRN-MR, a
benchmark we introduce on HM3D for multi-room 3D scene understanding, covering grounding, captioning, and four question-answering tasks that progressively evaluate from intra-room perception to cross-room
reasoning. Experiments show that CAIRN outperforms prior 3D-LLMs by a large margin across all CAIRN-MR tasks while remaining competitive on five single-room
benchmarks.
Read the full abstract ▾
All authors He Liang, Chenyang Ma, Yiming Zhang, Sangyun Shin, Andrew Markham, Niki Trigoni, Yuhang He
Computer Vision · Cross-listed from cs.AI
Bridging Physical Reasoning and Task Generalization via Visual Action Outcome Reasoning Alignment
Han-Jun Ko, Jr-Jen Chen, Haobo Yuan et al. (+4 more)
Vision-
language models (VLMs) struggle to generalize in interactive physical
reasoning, particularly under unseen tasks and environments. Two key failure modes are prominent: hallucinated chain-of-thought (CoT)
reasoning that contradicts physical reality, and misalignment between the model's
reasoning and actions. We present VAORA (Visual Action Outcome
Reasoning Alignment), a novel reward design that directly addresses both issues. VAORA introduces two complementary rewards: Visual
Alignment Reward, which anchors VLM
reasoning to the visual context independent of the
agent action itself, and Visual-Action
Alignment Reward, which grounds
reasoning in the visual outcome induced by the model's action. Together, these rewards suppress hallucinated CoT and reduce the gap between
reasoning and behavior. To improve training stability, we further employ smooth, dense rewards by estimating success probabilities using a pre-trained in-domain expert
agent. Experiments on PHYRE and Virtual Tool support our performances across novel-task and unseen-environment settings, confirming that grounded and generalizable physical intelligence can be induced through VAORA.
Read the full abstract ▾
All authors Han-Jun Ko, Jr-Jen Chen, Haobo Yuan, Hsin-Ying Lee, Tiancheng Shen, Ming-Hsuan Yang, Yu-Chiang Frank Wang
Computer Vision
Point as Skeleton: Accumulated Point Cloud Enhanced Autoregressive Generation for Closed-Loop Autonomous Driving Simulation
Songbur Wong, Xiaosong Jia, Junqi You et al. (+12 more)
Evaluating end-to-end autonomous driving (E2E-AD) remains challenging, as existing driving simulation methods often trade off closed-loop interactivity (e.g., CARLA) and real-world visual fidelity (e.g., nuScenes). We present \textbf{Point as Skeleton}, a generative sensor simulation framework for state-updated autoregressive driving
video generation, in which an autoregressive generator synthesizes visual observations from step-wise updated ego states, actor states, scene maps, and point-cloud skeleton conditions. To support closed-loop rollout, we introduce Reset-and-Roll, which adapts rolling
diffusion inference to simulation by preventing future-conditioned latent states from being committed across simulation steps. To stabilize error accumulation during step-wise autoregressive rollout, we introduce point-cloud skeletons that decouple foreground and background assets and project them into camera-view painted-point and template-depth conditions, providing appearance and
geometric cues. We further implement a nuPlan-based renderer-level closed-loop generative interface for evaluating generation under ego deviations from the original log. Experiments on nuScenes and nuPlan show that Point as Skeleton improves autoregressive generation quality during closed-loop rollout, demonstrating its potential for visually faithful closed-loop driving simulation. The code is available at https://github.com/krauwu/point-as-skeleton.
Read the full abstract ▾
All authors Songbur Wong, Xiaosong Jia, Junqi You, Bo Zhang, Pei Xu, Renqiu Xia, Yuping Qiu, Shaofeng Zhang, Zelin Zhao, Xuechao Yan, Yuchen Zhou, Yurui Chen, Wen Guo, Hang Xu, Junchi Yan
Computer Vision
AirflowAttack: Thermal-Airflow Adversarial Perturbations against Infrared Remote-Sensing Vision-Language Models
Cong Su, Jiaju Han, Xuemeng Sun et al. (+5 more)
Vision-
language models (VLMs) are increasingly deployed on infrared (IR) remote sensing imagery in security-critical settings, yet their
adversarial robustness remains unexamined. We present AirflowAttack, to our knowledge the first
adversarial attack for IR remote-sensing VLMs and the first to weaponize thermal-airflow
turbulence as the perturbation prior. A lightweight generator synthesizes a single input-agnostic perturbation regularized toward physically plausible airflow patterns. Optimized on one surrogate CLIP model, it attains a mean zero-shot scene-classification attack success rate (ASR, the fraction of samples whose top-1 class changes) of 48.5% across five diverse CLIP backbones, far exceeding four IR-specific physical baselines (27.7–37.0%). Applied to six state-of-the-art VLMs, it cuts scene-classification accuracy by up to 38.2% relative, yet paradoxically makes some models more confident in their IR analysis, confabulating the perturbation as genuine thermal evidence such as temperature gradients and convection. Ablations show the airflow prior raises physical plausibility at no measurable cost to attack success. Together with a
benchmark spanning eleven models and four tasks, these findings expose critical vulnerabilities in the rapidly expanding IR VLM ecosystem.
Read the full abstract ▾
All authors Cong Su, Jiaju Han, Xuemeng Sun, Chengyin Hu, Qike Zhang, Jiujiang Guo, Yiwei Wei, Jiahuan Long
Computer Vision · Cross-listed from cs.CR
Assessing the Operational Impact of Poisoning Attacks over Augmented 3D Point Cloud Public Datasets for Connected and Autonomous Vehicles
Marwan Lazrag, Badis Hammi, Lorena Gonzalez-Manzano et al. (+1 more)
Poisoning attacks against public
datasets lead to major concerns, such as (i) misclassification of perceived objects when the poisoned data is used for training and (ii)
embedding of
backdoors that may eventually be triggered later on, when specific conditions in the system apply over the learned models. Its impact over data augmentation models is unclear. While data augmentation reduces the likelihood of poisoning attack success, some valid questions remain. Is data augmentation affecting the impact of poisoning attacks? can it increase the number of poisoned samples or injected
backdoors? We explore in this paper some of these questions. We assess the effects of augmenting poisoned 3D point cloud
datasets and validate that poisoning is able to evade the sanitizing nature of augmentation techniques when using the concrete case of Generative
Adversarial Network (GAN) techniques to exemplify the case of data augmentation processing. We also validate that poisoning propagates over the augmented
datasets and perturbs the decision made by general-purpose classifiers, in the end. All the experimental material (including tools,
datasets, and classifiers) is publicly available, to facilitate reproducibility and to foster further research in the topic.
Read the full abstract ▾
All authors Marwan Lazrag, Badis Hammi, Lorena Gonzalez-Manzano, Joaquin Garcia-Alfaro
Computer Vision
Mitigating Domain Shift in Conditioned Floor Plan Generation: Synthetic Pre-training for Data-Efficient Adaptation
Matthieu Ospici, Arnaud Gueze, Luc Bourrat et al. (+1 more)
Robustness to domain shift is a key requirement for floor plan generative models to be applicable beyond the single
dataset they were trained on, as floor plans vary widely across regions due to distinct architectural cultures, spatial constraints, and construction practices, while acquiring new annotated
datasets remains costly and domain-specific. Yet, no prior work has studied this robustness in the context of conditioned floor plan generation. In this paper, we evaluate state-of-the-art models from two fundamentally different generative paradigms across three public
datasets (RPLAN, MagicPlan and Swiss Dwellings) and show that they are highly sensitive to domain shift, with up to an order of magnitude performance degradation when transferred across domains. To mitigate this with minimal target-domain supervision, we introduce a procedural method to generate a large-scale synthetic training
dataset that enforces strict physical constraints (non-overlapping rooms, valid door placement, graph consistency) while intentionally sacrificing architectural realism through highly irregular spatial arrangements and aggressive
geometric perturbation of room shapes. We show that pre-training on this synthetic data considerably improves zero-shot cross-domain performance, outperforming in-domain training on MagicPlan. Furthermore, it provides a highly effective initialization for fine-tuning, accelerating target domain adaptation and outperforming real-world initialization baselines by up to 40% in a low-data regime.
Read the full abstract ▾
All authors Matthieu Ospici, Arnaud Gueze, Luc Bourrat, Adrien Bernhardt
Computer Vision
Prompt-Adapter Context Routing for Parameter-Efficient Multi-Shot Long Video Extrapolation
Anna Córdoba, Adam Puente Tercero, Nerea Angulo Hijo et al. (+4 more)
We present PACR-
Video, a parameter-efficient framework for multi-shot long
video extrapolation that preserves recurring entities, scene structure, visual style, and causal progression without full generator fine-tuning. PACR-
Video keeps a text-to-
video diffusion transformer frozen and augments it with low-rank temporal adapters conditioned by learned shot-role
prompt tokens. To maintain long-horizon coherence, it builds a recursive
prompt bank that stores compact entity, location, action, and style
prompts from previous shots, then routes them through adapter gates according to predicted narrative dependencies. A Shot-Local/Story-Global tuning objective combines next-shot reconstruction, cross-shot identity contrast, and
prompt sparsity regularization, while an adapter composition schedule balances early-shot visual consistency with later-shot event progression and viewpoint change. Across six multi-shot and long-
video benchmarks, PACR-
Video outperforms text-to-
video, tuning-based,
memory-augmented, streaming, and recursive-context baselines on distributional quality, semantic
alignment, identity consistency, temporal smoothness, motion stability, transition coherence, and human preference. These results show that compact
prompt routing and lightweight temporal adaptation provide sufficient controllable capacity for stable long
video extrapolation.
Read the full abstract ▾
All authors Anna Córdoba, Adam Puente Tercero, Nerea Angulo Hijo, Mar Linares Tercero, Julia Barrientos, Ainhoa Miranda, Jesús Olivera
Computer Vision
A VLM-Enhanced Framework for Comprehensive Traffic Sign Condition Assessment Integrating Daytime Visual Performance and Nighttime Retroreflectivity Evaluation
Linlin Zhang, Neema Jakisa Owor, Xiang Yu et al. (+2 more)
Traffic signs are crucial components of road safety, serving as visual tools under all lighting conditions. The Manual on Uniform Traffic Control Devices (MUTCD) specifies daytime visual factors such as legibility and color contrast, and nighttime retroreflectivity requirements. Traditional assessment methods rely on manual inspections, which the Federal Highway Administration (FHWA) notes are subjective, labor-intensive and pose safety concerns, while retroreflectometers are expensive and unaffordable for smaller agencies. Most existing studies focus on either daytime factors or nighttime retroreflectivity but rarely integrate both aspects comprehensively. This study develops a novel framework that systematically evaluates traffic signs through integrated daytime-nighttime assessment. The methodology employs three fine-tuned
Vision Language Models (VLMs) for daytime visual performance assessment across four key factors: legibility, color, surface and shape integrity, and surrounding environment conditions. VLM predictions are converted to numerical scores through sentiment analysis and Contrastive
Language-Image Pre-Training (CLIP) scoring, while nighttime performance is assessed using
LiDAR-derived retroreflectivity following established calibration procedures. The framework integrates these components into a comprehensive Sign Condition Index (SCI) for maintenance guidance. Evaluation results demonstrated that LLaVA and Qwen outperformed InternVL, achieving bidirectional cosine similarity scores of 0.67-0.76 across all factors. Among 462 validated traffic signs, 68 were flagged by the proposed framework as requiring immediate replacement due to inadequate retroreflectivity performance. This research provides a cost-effective alternative to traditional manual inspections for comprehensive traffic sign condition assessment.
Read the full abstract ▾
All authors Linlin Zhang, Neema Jakisa Owor, Xiang Yu, Abby Watts, Yaw Adu-Gyamfi
Computer Vision
EgoPolice: A Benchmark for Egocentric Video Understanding in High-Stakes Police Body-Worn Camera Footage
Max Gonzalez Saez-Diez, Jihoon Chung, Adam D. Wolsky et al. (+5 more)
We introduce EgoPolice, a carefully curated
dataset of real,
egocentric police-civilian interactions, sourced from publicly available body-worn camera
videos. We select police-civilian action labels that are critical for police behavioral research and annotate them at a second-by-second granularity. The
videos feature rapid and irregular camera motion, dense human interactions, and rare high-stakes events, making the
dataset a challenging
benchmark for motion-robust and context-aware
egocentric perception. We provide two different tasks, classification and multiple-choice question-answering, and
benchmark both open-source and closed-source models. We find that even the best
video models like Gemini 2.5 Pro still struggle to accurately predict high-risk actions such as "Weapon Out". Beyond serving as a
benchmark, EgoPolice provides a foundation for developing models capable of identifying events of interest in large-scale body-worn camera
video repositories, enabling more efficient downstream human review.
Read the full abstract ▾
All authors Max Gonzalez Saez-Diez, Jihoon Chung, Adam D. Wolsky, Gregory Lanzalotto, Dean Knox, Jonathan Mummolo, Brandon M. Stewart, Olga Russakovsky
Computer Vision
Verification of Dynamic Holographic Behavior in Identity Documents
Glen Pouliquen, Joseph Chazalon, Guillaume Chiron et al. (+2 more)
This paper addresses the remote verification of the authenticity of Optically Variable Devices (commonly known as holograms) on identity documents. Typically placed over the cardholder's photo, these devices provide strong and easily verifiable security for human inspection but pose challenges for automated verification. Existing approaches easily cover static frauds (e.g. paper photocopy) and can be evaluated for such, but their capacity to detect real, dynamic fraud cases (e.g. handcrafted hologram) has not been evaluated to date because of the lack of public
datasets. Furthermore, they are usually trained to detect known attack types, and few of them can generalize to new, unseen attacks. This work features three contributions to address these limitations: 1) a new public
dataset, MIDV-DynAttack, which extends the existing MIDV-Holo
dataset with realistic, static and dynamic attacks against identity document specimens, tripling the number of attack samples compared to the original
dataset, 2) a novel verification method which can assess the authenticity of a specific hologram thanks to the analysis of its dynamic behavior and appearance, can be trained without dynamic attack samples, and exhibits new state-of-the-art performance, 3) a
benchmark of existing approaches which follows a clear evaluation protocol and emphasizes the inability of other approaches to deal with dynamic attacks, as well as new challenging attacks to deal with. Code and
dataset are publicly available at https://github.com/EPITAResearchLab/pouliquen.25.icdar.
Read the full abstract ▾
All authors Glen Pouliquen, Joseph Chazalon, Guillaume Chiron, Thierry Géraud, Ahmad Montaser Awal
Computer Vision
Andha-Dhun: A First Look at Audio Descriptions in Hindi
Ritabrata Chakraborty, Divy Kala, Nisheeth Bhooshan Gupta et al. (+3 more)
Audio Descriptions (ADs) narrate visual content for Blind and Low
Vision (BLV) audiences during gaps in audiovisual media. There is growing momentum around ADs in movies and TV shows, and with mandates from India's Central Board of Film Certification (CBFC), there is a need to expand ADs beyond English. Yet, there is no work that generates ADs for any Indian
language. To address this gap, we present the first systematic study of ADs in Hindi, contributing to aspects such as data, generation, and evaluation. We introduce Andha-Dhun, the first
dataset of human-authored Hindi ADs collected from 8 full-length movies. We explore two approaches for generating ADs in Hindi: (i) directly from English dense
video descriptions, and (ii) translating English ADs into Hindi. We evaluate these approaches using perplexity and LLM-as-a-judge metrics to assess fluency and quality respectively. We also analyze movies that have both English and Hindi human-authored ADs and find that naive translation introduces artifacts and narrows diversity compared to original Hindi ADs. Direct machine translation fails to adapt cultural references, while human-translated ADs do better but still fall short. Our findings emphasize that the purpose of Hindi ADs is accessibility for Indian BLV audiences, and that this requires adapting content for the audience more than strict fidelity to the source.
Read the full abstract ▾
All authors Ritabrata Chakraborty, Divy Kala, Nisheeth Bhooshan Gupta, Ganji Sreeram, Pailla Balakrishna Reddy, Makarand Tapaswi
Computer Vision
Analysis-by-Proxy: Localization Signals in VLMs Operating as Condition Encoders
Yoav Baron, Sara Dorfman, Roni Paiss et al. (+2 more)
Vision-
Language Models (VLMs) are increasingly utilized as the conditioning backbone for
diffusion-based image editing due to their remarkable
multimodal reasoning capabilities. While standalone VLMs demonstrate strong localization capabilities, editing pipelines frequently struggle to maintain this accuracy, particularly in complex, multi-entity scenes. In this work, we investigate this performance gap, hypothesizing that it stems from treating the VLM as a condition encoder. In this role, the model is restricted to a single forward pass, preventing the autoregressive generation process for which it was optimized, thereby failing to fully expose its capabilities. To investigate whether this spatial understanding persists when the VLM is used as a condition encoder, we introduce Analysis-by-Proxy. In this framework, we train a lightweight, interpretable proxy model on the VLM's intermediate representations using an auxiliary localization task. By analyzing the VLM through this proxy, we uncover the specific VLM representations that encode localization information. Our findings expose a fundamental mismatch between how spatial knowledge is represented within a VLM condition encoder and how it is extracted by current editing pipelines. We reveal that under single-pass constraints, the localization signal does not reliably propagate to the predefined layer configurations commonly used for conditioning. Instead, this crucial signal remains hidden within intermediate representations, at locations that vary depending on the input
prompt. Using our introduced Analysis-by-Proxy framework, we reveal the fundamental failures of existing condition extraction strategies in editing pipelines, opening the door to more principled design of conditioning architectures.
Read the full abstract ▾
All authors Yoav Baron, Sara Dorfman, Roni Paiss, Daniel Cohen-Or, Or Patashnik
Computer Vision
PIPBench: A Profile-Inclusive Framework for Personalized Image Generation Evaluation
Yuhang Wu, Shuxiang Zhang, Wee Hian Ching et al. (+2 more)
Recent text-to-image models such as DALLE-3 excel at following diverse
prompts yet remain blind to individual aesthetic preferences. We study personalized image generation, where models must align outputs with a user's implicit visual preferences based on a few historically preferred images and a short
prompt. To this end, we introduce PIPBench, the first profile-inclusive
benchmark for evaluating personalized image generation. We further propose a novel data construction pipeline that leverages psychological and demographic profiling dimensions for both real-user data collection and scalable
agent-based data generation. Using PIPBench, we conduct a thorough evaluation of representative line of methods. Our experiments reveal key limitations in existing methods, suggesting new challenges and opportunities for personalized text-to-image synthesis. Project page: https://wuyuhang05.github.io/PIPBench/
Read the full abstract ▾
All authors Yuhang Wu, Shuxiang Zhang, Wee Hian Ching, Chi Zhang, Miao Liu
Computer Vision · Cross-listed from cs.RO
WristMimic: Full-Body Humanoid Control with Wrist-Guided Manipulation
Wongyun Yu, Youngwoon Kim and Minsu Cho
Retargeting human object interaction demonstrations to physics based simulation requires reproducing not only body motion but also the object motion and contacts that make manipulation succeed. However, position only hand trajectories do not specify the contact forces needed to manipulate objects, and directly tracking them can overconstrain contact rich finger behavior. We introduce WristMimic, a wrist guided whole body control framework that explicitly separates contact free body motion from contact rich hand manipulation. The contact free body and wrist are guided by kinematic pose targets, whereas the fingers are not directly supervised by human hand pose. Instead, they learn grasping and manipulation behaviors from object tracking and contact outcomes. Our key insight is that the wrist is the natural gate between these two regimes. It is largely free from contact and can be tracked kinematically, yet it determines the global hand configuration and places the fingers within reachable grasp affordances. To ensure reliable wrist placement during interaction, we introduce wrist specific reset constraints and reward prioritization. Experiments show that WristMimic matches or surpasses methods using full finger pose supervision while enabling finger agnostic retargeting across diverse hand embodiments.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
TILDE: TILt-based Distributional Erasure for Concept Unlearning
Naveen George, Naoki Murata, Yuhta Takida et al. (+2 more)
Concept unlearning in text-to-image
diffusion models is critical for safe and practical deployment: with rising
privacy concerns, copyright disputes, trademark constraints, and safety regulations, deployed systems must be able to suppress unwanted concepts after training. Existing methods often remove the target concept effectively, but practical unlearning also requires an equally fundamental property: the unlearned model should retain quality, diversity, and semantic coverage on benign generation. The gold standard is a retain-only model trained from scratch without the unwanted data. However, common erasure objectives do not specify which post-unlearning distribution should approximate this reference, leaving retention as an implicit consequence of the update rule. We propose TILDE, TILt-based Distributional Erasure, which formulates concept unlearning as a distributional
alignment problem: the desired target is the minimum-deviation conditional distribution from the pretrained model under a forgetting constraint. This energy-tilted, anchor-free target suppresses concept-expressing images while preserving benign relative mass for each
prompt. We instantiate this principle with residual $\nabla$-GFlowNet training, which learns the score correction induced by the forget energy relative to the pretrained
diffusion model. Across objects, artistic styles, and characters, TILDE achieves strong forgetting while improving retention and distributional fidelity over prior baselines.
Read the full abstract ▾
All authors Naveen George, Naoki Murata, Yuhta Takida, Konda Reddy Mopuri, Yuki Mitsufuji
Computer Vision
XRFormer: Multiscale Tokenization for XRF Representation Learning
Sofiane Daimellah, Sylvie Le Hégarat-Mascle and Clotilde Boust
X-ray fluorescence (XRF) spectroscopy is a key modality for material analysis in cultural heritage. However, automated learning from XRF spectra remains challenging: XRF spectra are complex one-dimensional signals composed of sharp elemental peaks, broader structures, and background variations that are not taken into account by existing learning-based models. This paper introduces XRFormer, a
transformer architecture tailored to XRF spectra through a multiscale convolutional tokenizer that injects locality and multi-resolution inductive biases before global self-
attention. The tokenizer progressively reduces spectral resolution while increasing
embedding dimensionality, and the resulting
token sequence is processed by a standard
transformer encoder. We further investigate self-supervised pretraining for XRF representation learning using Masked Spectral modeling (MSM) and a physics-informed Peak Presence Prediction (PPP) objective. Experiments on the Pigments Checker STANDARD v.5
dataset for pigment identification and unmixing show that XRFormer consistently outperforms ViT, SpectralFormer (with and without CAF), and a 1D-CNN baseline for pigment identification. For pigment unmixing, XRFormer achieves robust abundance estimation while maintaining significantly higher parameter efficiency than SpectralFormer, operating at a lower
token resolution (128 vs. 512
tokens) and with less than half the number of parameters (1.5M vs. 3.37M). MSM yields consistent gains across both tasks, while PPP further enhances performance for both identification and unmixing when tuned with an appropriate peak prominence. These results highlight multiscale, modality-aware tokenization as an effective and parameter efficient foundation for
transformer-based XRF modeling under data-limited conditions. A GitHub repository is provided at https://github.com/sofiane1010/XRFormer.
Read the full abstract ▾
Computer Vision
HoloCount: A Holistic Visual Counting Benchmark for MLLMs
Jinhong Deng, Limeng Qiao and Guanglu Wan
Visual counting is a fundamental pillar of
multimodal intelligence, requiring a seamless integration of fine-grained grounding and spatial
reasoning. While
Multimodal Large
Language Models (MLLMs) have achieved remarkable success in qualitative scene understanding, their quantitative precision remains a significant bottleneck, often characterized by persistent numerical
hallucinations. Existing counting
benchmarks primarily focus on basic perception in simplified contexts, failing to capture the complex failure modes that emerge under logical constraints or
adversarial conditions. To address these limitations, we introduce HoloCount, a holistic and diagnostically rich
benchmark structured around a three-level hierarchical taxonomy. HoloCount evaluates MLLMs across: (1) Semantic Counting, focusing on atomic and property-based enumeration; (2) Analytical Counting, assessing logical composition through spatial and set-based
reasoning; and (3) Robustness Testing, probing model integrity against adverse scenarios and grounded counter-priors, such as high-density scenes and linguistic biases. Through an exhaustive evaluation of over 20 state-of-the-art MLLMs, we reveal a critical performance gap: even top-tier models degrade significantly as tasks transition from perception to complex analytical
reasoning and adverse scenarios. Our findings provide a systematic landscape of current MLLM counting capabilities and offer a roadmap for developing more grounded and reliable
multimodal systems. The
dataset is available at https://mm-mvr.github.io/HoloCount/.
Read the full abstract ▾
Computer Vision
Temporal Modeling of Optically Variable Devices in Identity Documents
Glen Pouliquen, Joseph Chazalon, Guillaume Chiron et al. (+3 more)
Robust remote verification of identity documents relies on analyzing faint, transparent security features like Optically Variable Devices (OVDs), or "holograms", within user-captured
videos under uncontrolled conditions. Current systems, however, face critical limitations: existing methods often treat
video frames in isolation, neglecting the intrinsic dynamic nature of OVDs and leaving systems vulnerable to swapping attacks, or focus on general
holographic presence and lack the ability to verify specific OVD types. Moreover, the economic infeasibility of frame-by-frame
video annotation makes supervised training impractical. In this work, we introduce two novel approaches for verifying the dynamic behavior of transparent OVDs protecting the holder's portrait, specifically designed for open-set scenarios where attack types are unknown during training. We demonstrate that these approaches can be trained without any attack samples in a self-supervised setting, surpassing previous state-of-the-art methods on public
datasets while adhering strictly to industrial constraints. Our results confirm that modeling temporal dynamics is essential for defeating sophisticated attacks under realistic conditions, and underscores the promise of sequence modeling and anomaly detection for OVD verification. Code is available at https://github.com/EPITAResearchLab/pouliquen.26.icdar.
Read the full abstract ▾
All authors Glen Pouliquen, Joseph Chazalon, Guillaume Chiron, Oriol Ramos Terrades, Thierry Géraud, Ahmad Montaser Awal
Computer Vision
What Images Cannot Say: Language-Guided Olfactory Representation Learning
Eleftherios Tsonis, Xi Wang and Vicky Kalogeiton
Images tell us what a scene looks like, but rarely what it would feel like to be there. While recent
datasets pair visual scenes with electronic-nose measurements, aligning smell signals with images remains challenging because many olfactory cues arise from contextual environmental factors that are not directly visible in pixels. We introduce SCENT, a
multimodal framework that uses
language guidance as a semantic bridge between
vision and olfaction. Our approach leverages
Vision-
Language Models (VLMs) to generate scene descriptors capturing objects, environmental context, and plausible ambient smell cues suggested by the visual scene. These descriptors provide semantic guidance for learning olfactory representations. We train a smell encoder that maps electronic-nose signals into a shared
embedding space aligned with both visual and textual representations, and introduce a languageguided latent decomposition that separates object-specific odors from contextual environmental contributions. Experiments on the New York Smells
dataset demonstrate that SCENT significantly improves crossmodal
retrieval compared to
vision-only baselines, achieving state-of-theart performance on smell-to-image and smell-to-text
retrieval tasks. In addition, our framework produces interpretable olfactory representations that enable the disentanglement of complex smell mixtures. Our results reveal the importance of contextual semantic information for grounding olfactory perception in
multimodal learning and pave the way for future research in this area.
Read the full abstract ▾
Computer Vision
FADRA: Frequency-Aware Diffusion with Residual Adaptation for Video Face Restoration
Jin Jiang, Jia Wang, Panwen Hu et al. (+2 more)
Video face restoration (VFR) aims to recover high-quality and temporally consistent facial details from severely degraded
video sequences; however, existing methods still struggle to balance spatial fidelity and temporal coherence under complex degradations. To address this, we propose FADRA, a frequency-aware
diffusion framework with iterative residual adaptation specifically tailored for robust VFR. We first leverage the strong temporal consistency of a pre-trained text-to-
video diffusion model and introduce lightweight LoRA adapters together with a Low-Quality (LQ) Pixel-
Alignment Feature Fusion module to efficiently adapt the frozen generative prior to the VFR task. To further adapt the frozen
diffusion backbone to the downstream VFR task beyond LoRA-based adaptation, we introduce a Repeated Residual Adaptation Head (RRAH) for step-wise residual refinement after the
diffusion backbone. To make this refinement explicitly guided by the degraded observation, RRAH further takes the LQ latent together with the current velocity prediction as input, allowing the model to repeatedly revisit LQ cues and predict residual updates at each flow-matching step. This LQ-guided repeated residual adaptation helps recover fine facial details while preserving the inherent temporal priors of the pre-trained model. Furthermore, to ensure the structural integrity of perceptually important details, we introduce a Frequency-Aware Loss that provides explicit supervision across multiple spectral bands, emphasizing visually sensitive frequency components that are crucial for perceptual quality and prone to temporal jittering. Extensive experiments demonstrate that FADRA recovers better facial structures and produces more temporally consistent
videos than state-of-the-art methods, leading to clear gains in both quantitative metrics and visual perception. Our project page is available.
Read the full abstract ▾
All authors Jin Jiang, Jia Wang, Panwen Hu, Weiran Zhao, Shengcai Liao
Computer Vision · Cross-listed from cs.RO
Learning to Throw Objects Safely in Multi-Obstacle Environments
Mohammadreza Kasaei, Klemen Voncina and Hamidreza Kasaei
Robotic throwing enables fast and efficient object placement beyond the
robot's immediate workspace, but reliable throwing in cluttered environments remains underexplored. Existing approaches, such as TossingBot, learn throwing strategies from visual input but assume obstacle-free settings. In this paper, we address the problem of throwing objects into a target basket while avoiding obstacles placed randomly in the scene. We introduce a potential field state representation that compactly encodes both basket attraction and obstacle repulsion on a fixed-size grid, enabling reinforcement learning (RL) policies to generalize across arbitrary numbers and configurations of obstacles. The policy is initialized from kinesthetic demonstrations and optimized in simulation using three state-of-the-art RL
algorithms (SAC, DDPG, TD3). Among these, SAC achieves the most consistent performance across scenarios. We compare the potential field representation against explicit state encodings and demonstrate that it achieves higher success rates and better scalability to unseen obstacle configurations. Real-
robot experiments with unseen throwable objects confirm robust sim-to-real transfer, achieving up to $90\%$ success in cluttered scenes. These results demonstrate that PFR provides a practical and robust representation for safe and efficient robotic throwing in unstructured environments. A
video showcasing our experiments is available at: https://youtu.be/ZZnJf8ua2dE
Read the full abstract ▾
Computer Vision
VaseMuseum: Digital Intelligent Museum for Ancient Greek Pottery
Jiazi Wang, Nonghai Zhang, Qiushi Xie et al. (+5 more)
Vision-
language models (VLMs) have made interactive digital museums increasingly feasible by connecting 3D digitization with natural-
language artifact exploration. However, in cultural heritage domains such as ancient Greek pottery, reliable VLM assistance is limited by two challenges. First, open-ended interpretation requires grounding fine-grained 2D/3D visual evidence in specialized curatorial knowledge, yet the
retrieval process may introduce weak sources and unverifiable references. Second, when the available evidence is incomplete, noisy, or ambiguous, VLMs often produce confident but unsupported answers instead of calibrated uncertainty. To address these challenges, we propose VaseMuseum, a lightweight and modular
multimodal agent framework for intelligent digital museums of ancient Greek pottery. VaseMuseum combines an interactive virtual museum with VaseAgent, which supports both 2D images and 3D artifacts through
multimodal perception, 3D-aware
reasoning, external knowledge
retrieval, and inference-time reliability control. Specifically, VaseAgent retrieves evidence from authoritative web and museum knowledge sources, and source-level control selects diverse and verifiable evidence before generation. Meanwhile, response-level control checks generated claims against the evidence pool and encourages neutral, evidence-bounded answers when support is insufficient or conflicting. Moreover, a training-free GRPO-style selection mechanism favors responses with valid references and calibrated confidence without updating the VLM backbone. Experiments in a realistic digital museum simulation show that VaseMuseum improves citation validity, reduces
hallucinations on knowledge-intensive queries, and produces more neutral answers under ambiguity compared with search-enabled VLM baselines.
Read the full abstract ▾
All authors Jiazi Wang, Nonghai Zhang, Qiushi Xie, Zeyu Zhang, Yufeng Chen, Yang Zhao, Ling Shao, Hao Tang
Computer Vision · Cross-listed from cs.RO
ActionCache: Training-Free Acceleration for Vision-Language-Action Models with Action Caching and Refinement
Ryuji Oi, Hikari Otsuka, Kosuke Matsushima et al. (+4 more)
Vision-
Language-Action (VLA) models have emerged as a promising approach for generalizable robotic manipulations. In particular, flow-matching-based VLA models have shown remarkable success due to their capability to generate precise and smooth action sequences and capture
multimodal distributions. However, the iterative denoising process in the action head acts as a major computational bottleneck, posing a critical challenge for real-time deployment. To address this challenge, we propose ActionCache, a plug-and-play external cache that opportunistically reuses past intermediate actions to warm-start generations from the vicinity of target actions, drastically reducing the inference latency. Specifically, ActionCache stores the intermediate actions with compact
multimodal keys, which enables
retrieval from similar past contexts across different episodes or even different tasks. Experimental results in simulation and real-world environments demonstrate that ActionCache maintains high task success rates in a low-latency regime, achieving action head inference acceleration of up to $10.44\times$ and $40.17\times$ for representative flow-based VLA, $\pi_{0.5}$ and GR00T-N1.6, respectively.
Read the full abstract ▾
All authors Ryuji Oi, Hikari Otsuka, Kosuke Matsushima, Yuki Ichikawa, Masato Motomura, Tatsuya Kaneko, Daichi Fujiki
Computer Vision · Cross-listed from eess.IV
TMF-RSE: Tri-Modal Fusion with Regional Semantics and Evidential Uncertainty for Lung Severity Scoring
Fadi Abdeladhim Zidi, Salah Eddine Bekhouche, Abdellah Zakaria Sellam et al. (+3 more)
Accurate quantification of lung disease severity from chest imaging is critical for clinical decision-making and resource allocation. We propose a tri-modal deep learning framework, TMF-RSE (Tri-Modal Fusion with Regional Semantics and Evidential Uncertainty), that combines appearance features from two-dimensional chest inputs, structural features from lung
segmentation masks, and semantic features from
vision-
language models (VLMs) for severity quantification. Our approach employs complementary fusion mechanisms that integrate semantic guidance, structural priors, and hierarchical interactions across modalities. The model employs evidential regression to provide both severity predictions and uncertainty estimates. Experiments on the Per-COVID-19 CT and RALO
datasets show that TMF-RSE outperforms recent
transformer-based baselines, achieving MAE of 4.02 and Pearson correlation of 0.9629 on Per-COVID-19 validation, and 0.339 MAE / 0.973 PC on RALO geographic extent.
Read the full abstract ▾
All authors Fadi Abdeladhim Zidi, Salah Eddine Bekhouche, Abdellah Zakaria Sellam, Gaby Maroun, Fadi Dornaika, Cosimo Distante
Computer Vision
Generalized Synthetic Image Detection with Enhanced RGB-Noise Representation Learning
Zhen Li, Gang Cao, Tian Zhang et al. (+2 more)
The rapid advancement of large-scale generative models has accelerated the spread of highly deceptive AI-generated images, making generalized synthetic image detection a critical imperative. Existing forensic networks often struggle with cross-model generalization and realworld degradations due to their reliance on single-domain representations and conventional binary classification optimization. To overcome these limitations, we propose RNSIDNet, a novel forensic framework that achieves robust detection through enhanced RGB-Noise representation learning. Specifically, our method employs a dual-branch architecture where global RGB semantics, extracted by an
attention-refined CLIP backbone, dynamically modulate highfrequency noise artifacts captured by Bayar convolutions via a Feature-wise Linear Modulation (FiLM) module. To further enhance the learned representations, we design a Hard Sample-aware Contrastive Learning (HSCL) strategy. By explicitly penalizing challenging training samples, HSCL reshapes the latent feature space to maximize the discriminative margin between pristine and synthetic domains. Extensive experiments across eight public
benchmark datasets verify that our model achieves state-of-the-art performance, delivering superior generalization ability, robustness, and computational efficiency. Code and
dataset will be publicly available on https://github.com/multimediaFor/RNSIDNet.
Read the full abstract ▾
All authors Zhen Li, Gang Cao, Tian Zhang, Lifang Yu, Shaowei Weng
Computer Vision · Cross-listed from cs.RO
OrchardBench: A Physically-Grounded, GPU-Parallel Apple-Orchard Simulation Benchmark for Agricultural Robotics
Humphrey Munn
Robotic tree-fruit harvesting is a flagship problem for agricultural automation, but progress is bottlenecked by the cost and irreproducibility of field experiments: an orchard is available only weeks a year, every tree is different, and a control error can permanently damage the crop or the plant. The tree models used in graphics and agronomy are geometrically detailed but physically inert, while the GPU-parallel simulators used in
robot learning contain no plausible trees. We present OrchardBench, a physically-grounded, GPU-parallel simulation of apple-orchard trees on the Newton engine. Each tree is grown by a
stochastic L-system and instantiated as a fully articulated body: branches are compliant torsional spring-dampers whose stiffness follows Euler-Bernoulli beam theory, they break at a wood modulus of rupture and fall as free hinges, and apples are independent bodies on stem tethers that detach at literature-grounded pull forces and load the branch when pulled. A moving, density-controllable foliage layer occludes the canopy as real leaves do. Every physical parameter is tied to a published source. Per-environment domain randomization makes each batched world a distinct tree, and a mobile manipulator with a wrist depth camera closes the loop with
geometric fruit perception and an autonomous harvesting baseline. Careful engineering of the solver and the model lets OrchardBench run many parallel environments at interactive rates on a laptop GPU. We define the tasks and a metric suite spanning harvest completeness, throughput, and plant damage (with a per-canopy-zone breakdown), and report baseline results across foliage, fruit load, terrain, canopy zone, and parallelism. The analytic baseline succeeds on about 40% of the fruit it detects and harvests only about an eighth of the reachable fruit on a tree, leaving clear headroom for novel autonomy approaches.
Read the full abstract ▾
Computer Vision
Jincheng Ying, Li Wenlin, Minghui Xu et al. (+1 more)
Diffusion models generate high-quality images, but their inference cost comes from two sources: large denoising networks and repeated denoising steps. Existing compression pipelines usually attack these costs separately.
Pruning reduces the network, but most
pruning methods still rely on a long post-
pruning retraining stage to recover a many-step sampler. Step
distillation reduces the number of denoising steps, but it usually assumes a student that can already follow the teacher well enough to receive useful
distillation gradients. This paper asks whether post-
pruning retraining can be replaced by step
distillation. We find that the direct replacement fails: after
pruning an EDM2-XS teacher, starting SiDA from the pruned checkpoint produces unusable samples. We introduce a short teacher-
alignment repair stage as a bridge between
pruning and step
distillation. The bridge matches the pruned generator to the teacher on noisy real-image latents, then hands the repaired checkpoint to one-step
distillation. On ImageNet-512, the original EDM2-XS baseline uses 124.713M parameters and 63 network evaluations, reaching an FID of 3.53. With a suitable
distillation objective, our 20% pruned one-step generator uses 98.826M parameters and one network evaluation, reaching an FID of 3.12. With 30%
pruning, the model uses 88.029M parameters and one network evaluation, with an FID of 4.26.
Read the full abstract ▾
All authors Jincheng Ying, Li Wenlin, Minghui Xu, Yinhao Xiao
Computer Vision · Cross-listed from cs.AI
Driving the Wrong Way: Leveraging Interpretability in End2End Autonomous Driving Models
Franz Motzkus and Sebastian Bernhard
The increasing adoption of end-to-end learning for autonomous driving introduces increased model complexity and opacity, raising the risk of learning undesired or erroneous behavior. In this work, we integrate unsupervised dictionary learning as a post hoc interpretability module within state-of-the-art driving models to decompose driving behavior into semantically meaningful concepts while demonstrating their causal influence on the model's driving decisions. We propose a stepwise framework for extracting and interpreting meaningful concepts from the end-to-end model and connecting them to the multifaceted model outputs, thereby revealing the underlying decision-making logic for the prediction of future trajectories. Furthermore, targeted interventions at the concept level allow us to manipulate and correct driving decisions, resulting in measurable improvements in overall driving performance. We thus demonstrate how interpretability can effectively be used to reduce model opacity, uncover erroneous behavior, and enable targeted mitigation, ultimately boosting model performance.
Read the full abstract ▾
Computer Vision
Synthetic-to-Real Translation for Class-Agnostic Motion Prediction
Yizheng Wu, Hongwei Fan, Kewei Wang et al. (+8 more)
Motion understanding is critical for ensuring safety and robustness in autonomous driving systems, driving increasing interest in motion prediction. A key challenge in this domain is the high cost associated with acquiring real-world motion labels. It is therefore ideal if we could transfer motion knowledge from synthetic data to real data. In this context, we explore the potential of synthetic-to-real translation for motion prediction (SRMP). However, the most used naive motion regression methods are notably sensitive to the synthetic-to-real domain shift, resulting in unreliable knowledge translation. To address this, we propose a novel approach integrating a motion knowledge translation framework with two key components: (1) objectness-aware motion prediction, which explicitly models the joint distribution of motion patterns and objectness priors to improve domain-invariant feature learning, and (2) objectness-aided motion enhancement, a motion label refinement mechanism that leverages learned objectness priors to filter motion noise. Furthermore, we present a physically-based pipeline for generating Motion4D, the first synthetic 4D
LiDAR dataset tailored for SRMP research, addressing the lack of synthetic motion
datasets. Experimental results demonstrate that our approach effectively bridges the domain gaps and yields superior performance on real scenes.
Read the full abstract ▾
All authors Yizheng Wu, Hongwei Fan, Kewei Wang, Ruibo Li, Xingyi Li, Xiao Song, Zhe Wang, Chenjing Ding, Dongliang Wang, Zhiguo Cao, Guosheng Lin
Computer Vision
Token-Based Dual-view Fusion and Adaptation of Large Vision Models for Breast Cancer Classification
Aysan Ghayouri Pirsoltan, Shima Babakordi and Mohammad Reza Mohammadi
Accurate breast cancer classification from mammography requires effective integration of complementary information from craniocaudal (CC) and mediolateral oblique (MLO) views, which provide a more complete characterization of breast abnormalities. However, existing multi-view learning approaches typically rely on feature-level aggregation or single-stage cross-
attention, which can entangle view-specific and shared representations and restrict interaction to limited network depths. To address these limitations, we propose a
token-centric dual-view learning framework that unifies
prompt-based adaptation and cross-view fusion within a frozen
vision transformer backbone. The framework reformulates inter-view interaction as structured
token-level communication, where dedicated fusion
tokens explicitly encode bidirectional information exchange between CC and MLO views via cross-
attention, serving as intermediate carriers of cross-view dependencies rather than relying on direct feature fusion. Unlike conventional methods that apply fusion at a single layer, fusion modules are inserted at multiple
transformer depths, enabling progressive and repeated interaction across the encoder hierarchy. Fusion
tokens are reintegrated into the
token sequence and refined by subsequent
transformer layers, facilitating hierarchical propagation of complementary information while preserving view-specific structure. Experiments on VinDr-Mammo and CMMD
datasets demonstrate consistent improvements over linear probing,
prompt-only adaptation, and conventional fusion baselines. On the VinDr-Mammo BI-RADS classification task, the framework achieves 50.40% F1-score and 0.8090 AUC, including a 0.10 AUC improvement over a dual-view fusion baseline in the binary setting. Ablation studies further validate the effectiveness of
token-based fusion and multi-depth interaction design.
Read the full abstract ▾