Computer Vision
Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Models
Lexiang Xiong, Qi Li, Jingwen Ye et al. (+1 more)
Vision-
Language Models (VLMs) frequently "hallucinate" - generate plausible yet factually incorrect statements - posing a critical barrier to their trustworthy deployment. In this work, we propose a new paradigm for diagnosing
hallucinations, recasting them from static output errors into dynamic pathologies of a model's computational cognition. Our framework is grounded in a normative principle of computational rationality, allowing us to model a VLM's generation as a dynamic cognitive trajectory. We design a suite of information-theoretic probes that project this trajectory onto an interpretable, low-dimensional Cognitive State Space. Our central discovery is a governing principle we term the
geometric-information duality: a cognitive trajectory's
geometric abnormality within this space is fundamentally equivalent to its high information-theoretic surprisal.
Hallucination detection is counts as a
geometric anomaly detection problem. Evaluated across diverse settings - from rigorous binary QA (POPE) and comprehensive
reasoning (MME) to unconstrained open-ended captioning (MS-COCO) - our framework achieves state-of-the-art performance. Crucially, it operates with high efficiency under weak supervision and remains highly robust even when calibration data is heavily contaminated. This approach enables a causal attribution of failures, mapping observable errors to distinct pathological states: perceptual instability (measured by Perceptual
Entropy), logical-causal failure (measured by Inferential Conflict), and decisional ambiguity (measured by Decision
Entropy). Ultimately, this opens a path toward building AI systems whose
reasoning is transparent, auditable, and diagnosable by design.
Read the full abstract ▾
All authors Lexiang Xiong, Qi Li, Jingwen Ye, Xinchao Wang
Computer Vision
Learning Latent Proxies for Controllable Single-Image Relighting
Haoze Zheng, Zihao Wang, Xianfeng Wu et al. (+5 more)
Single-image relighting is highly under-constrained: small illumination changes can produce large, nonlinear variations in shading, shadows, and specularities, while geometry and materials remain unobserved. Existing
diffusion-based approaches either rely on intrinsic or G-buffer pipelines that require dense and fragile supervision, or operate purely in latent space without physical grounding, making fine-grained control of direction, intensity, and color unreliable. We observe that a full intrinsic decomposition is unnecessary and redundant for accurate relighting. Instead, sparse but physically meaningful cues, indicating where illumination should change and how materials should respond, are sufficient to guide a
diffusion model. Based on this insight, we introduce LightCtrl that integrates physical priors at two levels: a few-shot latent proxy encoder that extracts compact material-geometry cues from limited PBR supervision, and a lighting-aware mask that identifies sensitive illumination regions and steers the denoiser toward shading relevant pixels. To compensate for scarce PBR data, we refine the proxy branch using a DPO-based objective that enforces physical consistency in the predicted cues. We also present ScaLight, a large-scale object-level
dataset with systematically varied illumination and complete camera-light metadata, enabling physically consistent and controllable training. Across object and scene level
benchmarks, our method achieves photometrically faithful relighting with accurate continuous control, surpassing prior
diffusion and intrinsic-based baselines, including gains of up to +2.4 dB PSNR and 35% lower RMSE under controlled lighting shifts.
Read the full abstract ▾
All authors Haoze Zheng, Zihao Wang, Xianfeng Wu, Yajing Bai, Yexin Liu, Yun Li, Xiaogang Xu, Harry Yang
Computer Vision
Self-Distillation of Hidden Layers for Self-Supervised Representation Learning
Scott C. Lowe, Anthony Fuller, Sageev Oore et al. (+2 more)
The landscape of self-supervised learning (SSL) is currently dominated by generative approaches (e.g. MAE) that reconstruct raw low-level data, and predictive approaches (e.g. I-JEPA) that predict high-level abstract
embeddings. While generative methods are stable due to their reliable training targets based on ground-truth data, they are computationally inefficient for high-redundancy modalities like imagery, and their training objective does not prioritize learning high-level, conceptual features. Conversely, predictive methods often suffer from training instability due to their reliance on the non-stationary targets of final-layer self-
distillation. We introduce Bootleg, a method that bridges this divide by tasking the model with predicting latent representations from multiple hidden layers of a teacher. This hierarchical objective forces the model to capture features at varying levels of abstraction simultaneously. We demonstrate Bootleg significantly outperforms comparable baselines (+10% vs. I-JEPA) on frozen probe classification of ImageNet-1K, iNaturalist-21, and VTAB, and semantic
segmentation of ADE20K, Cityscapes, and COCO-Stuff.
Read the full abstract ▾
All authors Scott C. Lowe, Anthony Fuller, Sageev Oore, Evan Shelhamer, Graham W. Taylor
Computer Vision
Kimodo: Scaling Controllable Human Motion Generation
Davis Rempe, Mathis Petrovich, Ye Yuan et al. (+21 more)
High-quality human motion data is becoming increasingly important for applications in robotics, simulation, and entertainment. Recent generative models offer a potential data source, enabling human motion synthesis through intuitive inputs like text
prompts or kinematic constraints on poses. However, the small scale of public mocap
datasets has limited the motion quality, control accuracy, and generalization of these models. In this work, we introduce Kimodo, an expressive and controllable kinematic motion
diffusion model trained on 700 hours of optical motion capture data. Our model generates high-quality motions while being easily controlled through text and a comprehensive suite of kinematic constraints including full-body keyframes, sparse joint positions/rotations, 2D waypoints, and dense 2D paths. This is enabled through a carefully designed motion representation and two-stage denoiser architecture that decomposes root and body prediction to minimize motion artifacts while allowing for flexible constraint conditioning. Experiments on the large-scale mocap
dataset justify key design decisions and analyze how the scaling of
dataset size and model size affect performance.
Read the full abstract ▾
All authors Davis Rempe, Mathis Petrovich, Ye Yuan, Haotian Zhang, Xue Bin Peng, Yifeng Jiang, Tingwu Wang, Umar Iqbal, David Minor, Michael de Ruyter, Jiefeng Li, Chen Tessler, Edy Lim, Eugene Jeong, Sam Wu, Ehsan Hassani, Michael Huang, Jin-Bey Yu, Chaeyeon Chung, Lina Song, Olivier Dionne, Jan Kautz, Simon Yuen, Sanja Fidler
Computer Vision
Clinically Aware Synthetic Image Generation for Concept Coverage in Chest X-ray Models
Amy Rafferty, Rishi Ramaesh and Ajitha Rajan
Deep learning models for chest X-ray diagnosis are constrained by limited coverage of clinically meaningful concept combinations in publicly available training
datasets. While synthetic image generation has been explored to increase data diversity, existing methods rarely enforce clinical or anatomical constraints, limiting utility for improving model reliability. We propose CARPA, a clinically aware and anatomically grounded framework for synthetic chest X-ray generation that applies targeted perturbations to clinical concept vectors while preserving anatomical structure. By producing anatomically faithful synthetic images with controlled concept insertions and deletions, CARPA expands clinically relevant concept coverage. We evaluate CARPA across seven backbone architectures by fine-tuning models on synthetic subsets and testing on a held-out MIMIC-CXR
benchmark. Compared to prior concept perturbation approaches, fine-tuning on CARPA-generated images consistently improves precision-recall performance, reduces predictive uncertainty, and improves model calibration. Structural and semantic analyses demonstrate high anatomical fidelity, strong concept
alignment, and low semantic uncertainty. Evaluation by two expert radiologists further confirms realism and clinical agreement. Together, these results show that anatomically grounded concept perturbations enable more effective use of synthetic data, improving both performance and reliability of chest X-ray classification models and supporting safer clinical deployment.
Read the full abstract ▾
Computer Vision
FreeTalk: Emotional Topology-Free 3D Talking Heads
Federico Nocentini, Thomas Besnier, Claudio Ferrari et al. (+2 more)
Speech-driven 3D facial animation has advanced rapidly, yet most approaches remain tied to registered template meshes, preventing effective deployment on raw 3D scans with arbitrary
topology. At the same time, modeling controllable emotional dynamics beyond lip articulation remains challenging, and is often tied to template-based parameterizations. We address these challenges by proposing FreeTalk, a two-stage framework for
emotion-conditioned 3D talking-head animation that generalizes to unregistered face meshes with arbitrary vertex count and connectivity. First, Audio-To-Sparse (ATS) predicts a temporally coherent sequence of 3D landmark displacements from
speech audio, conditioned on an
emotion category and intensity. This sparse representation captures both articulatory and affective motion while remaining independent of mesh
topology. Second, Sparse-To-Mesh (STM) transfers the predicted landmark motion to a target mesh by combining intrinsic surface features with landmark-to-vertex conditioning, producing dense per-vertex deformations without template fitting or correspondence supervision at test time. Extensive experiments show that FreeTalk matches specialized baselines when trained in-domain, while providing substantially improved robustness to unseen identities and mesh
topologies. Code and pre-trained models will be made publicly available.
Read the full abstract ▾
All authors Federico Nocentini, Thomas Besnier, Claudio Ferrari, Stefano Berretti, Mohamed Daoudi
Computer Vision · Cross-listed from cs.LG
Federated Learning of Binary Neural Networks: Enabling Low-Cost Inference
Nitin Priyadarshini Shankar, Soham Lahiri, Sheetal Kalyani et al. (+1 more)
Federated Learning (FL) enables distributed clients to jointly train a global model without sharing their raw data. However, deploying the resulting deep neural networks on resource-constrained edge devices remains challenging due to their inference-time computational and
memory requirements. Binary Neural Networks (BNNs) offer substantial efficiency gains by representing weights and activations using a single bit, but achieving competitive accuracy under such aggressive quantization remains difficult. Rotation-aware binarization reduces quantization errors by aligning real-valued weights with their binary representations. Despite its effectiveness in centralized learning, applying rotation-aware binarization in FL is non-trivial due to discrepancies between local and global models, particularly under heterogeneous client data distributions. To address this challenge, we propose FedBNN, a
federated rotation-aware binarization framework that leverages global model information to guide local binarization while incorporating locally learned rotation information into server-side construction of the global binary model. Experiments across multiple
benchmark datasets, model architectures, and degrees of data heterogeneity demonstrate that the proposed integrated methodology achieves accuracy competitive with full-precision
federated models while generally outperforming existing
federated binary learning approaches. Overall, FedBNN enables accurate binary inference in heterogeneous
federated settings, with savings of up to $58\times$ in FLOPs and $32\times$ in
memory over full-precision models, offering a practical balance between predictive performance and deployment efficiency on resource-constrained edge devices.
Read the full abstract ▾
All authors Nitin Priyadarshini Shankar, Soham Lahiri, Sheetal Kalyani, Saurav Prakash
Computer Vision
Real-Time Oriented Object Detection Transformer in Remote Sensing Images
Zeyu Ding, Yong Zhou, Jiaqi Zhao et al. (+4 more)
Recent real-time detection
transformers have gained popularity due to their simplicity and efficiency. However, these detectors do not explicitly model object rotation, especially in remote sensing imagery where objects appear at arbitrary angles, leading to challenges in angle representation, matching cost, and training stability. In this paper, we propose a real-time oriented object detection
transformer, the first real-time end-to-end oriented object detector to the best of our knowledge, that addresses the above issues. Specifically, angle distribution refinement is proposed to reformulate angle regression as an iterative refinement of probability distributions, thereby capturing the uncertainty of object rotation and providing a more fine-grained angle representation. Then, we incorporate a Chamfer distance cost into bipartite matching, measuring box distance via vertex sets, enabling more accurate
geometric alignment and eliminating ambiguous matches. Moreover, we propose oriented contrastive denoising to stabilize training and analyze four noise modes. We observe that a ground truth can be assigned to different index queries across different decoder layers, and analyze this issue using the proposed instability metric. We design a series of model variants and experiments to validate the proposed method. Notably, our O2-DFINE-L, O2-RTDETR-R50 and O2-DEIM-R50 achieve 77.73%/78.45%/80.15% AP50 on DOTA1.0 and 132/119/119 FPS on the 2080ti GPU. Code is available at https://github.com/wokaikaixinxin/ai4rs.
Read the full abstract ▾
All authors Zeyu Ding, Yong Zhou, Jiaqi Zhao, Wen-Liang Du, Xixi Li, Rui Yao, Abdulmotaleb El Saddik
Computer Vision
RSGen: Enhancing Layout-Driven Remote Sensing Image Generation with Diverse Edge Guidance
Xianbao Hou, Yonghao He, Zeyd Boukhers et al. (+4 more)
Diffusion models have significantly mitigated the impact of annotated data scarcity in remote sensing (RS). Although recent approaches have successfully harnessed these models to enable diverse and controllable Layout-to-Image (L2I) synthesis, they still suffer from limited fine-grained control and fail to strictly adhere to bounding box constraints. To address these limitations, we propose RSGen, a plug-and-play framework that leverages diverse edge guidance to enhance layout-driven RS image generation. Specifically, RSGen employs a progressive enhancement strategy: 1) it first enriches the diversity of edge maps composited from retrieved training instances via Image-to-Image generation; and 2) subsequently utilizes these diverse edge maps as conditioning for existing L2I models to enforce pixel-level control within bounding boxes, ensuring the generated instances strictly adhere to the layout. Extensive experiments across three baseline models demonstrate that RSGen significantly boosts the capabilities of existing L2I models. For instance, with CC-Diff on the DOTA
dataset for oriented object detection, we achieve remarkable gains of +9.8/+12.0 in YOLOScore mAP50/mAP50-95 and +1.6 in mAP on the downstream detection task. Our code will be publicly available: https://github.com/D-Robotics-AI-Lab/RSGen
Read the full abstract ▾
All authors Xianbao Hou, Yonghao He, Zeyd Boukhers, John See, Hu Su, Wei Sui, Cong Yang
Computer Vision
Ruonan Yu, Zhenxiong Tan, Zigeng Chen et al. (+2 more)
Diffusion Transformers (DiTs) have demonstrated remarkable scalability and quality in image and
video generation, prompting growing interest in extending them to controllable generation and editing tasks. However, compared to the image counterparts, progress in
video control and editing remains limited, mainly due to the scarcity of paired
video data and the high computational cost of training
video diffusion models. To address this issue, in this paper, we propose a
video-free tuning framework termed ViFeEdit for
video diffusion transformers. Without requiring any forms of
video training data, ViFeEdit achieves versatile
video generation and editing, adapted solely with 2D images. At the core of our approach is an architectural reparameterization that decouples spatial independence from the full 3D
attention in modern
video diffusion transformers, which enables visually faithful editing while maintaining temporal consistency with only minimal additional parameters. Moreover, this design operates in a dual-path pipeline with separate timestep
embeddings for noise scheduling, exhibiting strong adaptability to diverse conditioning signals. Extensive experiments demonstrate that our method delivers promising results of controllable
video generation and editing with only minimal training on 2D image data. Codes are available https://github.com/Lexie-YU/ViFeEdit.
Read the full abstract ▾
All authors Ruonan Yu, Zhenxiong Tan, Zigeng Chen, Songhua Liu, Xinchao Wang
Computer Vision
Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation
Yuanfan Zheng, Kunyu Peng, Xu Zheng et al. (+1 more)
Cross-domain panoramic semantic
segmentation has attracted growing interest as it enables comprehensive 360{\deg} scene understanding for real-world applications. However, it remains particularly challenging due to severe
geometric Field of View (FoV) distortions and inconsistent open-set semantics across domains. In this work, we formulate an open-set domain adaptation setting, and propose Extrapolative Domain Adaptive Panoramic
Segmentation (EDA-PSeg) framework that trains on local perspective views and tests on full 360{\deg} panoramic images, explicitly tackling both
geometric FoV shifts across domains and semantic uncertainty arising from previously unseen classes. To this end, we propose the Euler-Margin
Attention (EMA), which introduces an angular margin to enhance viewpoint-invariant semantic representation, while performing amplitude and phase modulation to improve generalization toward unseen classes. Additionally, we design the Graph Matching Adapter (GMA), which builds high-order graph relations to align shared semantics across FoV shifts while effectively separating novel categories through structural adaptation. Extensive experiments on four
benchmark datasets under camera-shift, weather-condition, and open-set scenarios demonstrate that EDA-PSeg achieves state-of-the-art performance, robust generalization to diverse viewing geometries, and resilience under varying environmental conditions. The code is available at https://github.com/zyfone/EDA-PSeg.
Read the full abstract ▾
All authors Yuanfan Zheng, Kunyu Peng, Xu Zheng, Kailun Yang
Computer Vision
Anchor then Polish for Low-light Enhancement
Tianle Du, Mingjia Li, Hainuo Wang et al. (+1 more)
Low-light image enhancement is challenging due to entangled degradations, mainly including poor illumination, color shifts, and texture interference. Existing methods often rely on complex architectures to address these issues jointly but may overfit simple physical constraints, leading to global distortions. This work proposes a novel anchor-then-polish (ATP) framework to fundamentally decouple global energy
alignment from local detail refinement. First, macro anchoring is customized to (greatly) stabilize luminance distribution and correct color by learning a scene-adaptive projection
matrix with merely 12 degrees of freedom, revealing that a simple linear operator can effectively align global energy. The macro anchoring then reduces the task to micro polishing, which further refines details in the wavelet domain and chrominance space under
matrix guidance. A constrained luminance update strategy is designed to ensure global consistency while directing the network to concentrate on fine-grained polishing. Extensive experiments on multiple
benchmarks show that our method achieves state-of-the-art performance, producing visually natural and quantitatively superior low-light enhancements.
Read the full abstract ▾
All authors Tianle Du, Mingjia Li, Hainuo Wang, Xiaojie Guo
Computer Vision
Automated Counting of Stacked Objects in Industrial Inspection
Corentin Dumery, Noa Etté, Aoxiang Fan et al. (+4 more)
Visual object counting is a fundamental computer
vision task in industrial inspection, where accurate, high-throughput inventory tracking and quality assurance are critical. Moreover, manufactured parts are often too light to reliably deduce their count from their weight, or too heavy to move the stack on a scale safely and practically, making automated visual counting the more robust solution in many scenarios. However, existing methods struggle with stacked 3D items in containers, pallets, or bins, where most objects are heavily occluded and only a few are directly visible. To address this important yet underexplored challenge, we propose a novel 3D counting approach that decomposes the task into two complementary subproblems: estimating the 3D geometry of the stack and its occupancy ratio from multi-view images. By combining
geometric reconstruction with deep learning-based depth analysis, our method can accurately count identical manufactured parts inside containers, even when they are irregularly stacked and partially hidden. We validate our 3D counting pipeline on large-scale synthetic and diverse real-world data with manually verified total counts, demonstrating robust performance under realistic inspection conditions.
Read the full abstract ▾
All authors Corentin Dumery, Noa Etté, Aoxiang Fan, Ren Li, Jingyi Xu, Hieu Le, Pascal Fua
Computer Vision
Evaluating Time Awareness and Cross-modal Active Perception of Large Models via 4D Escape Room Task
Yurui Dong, Ziyue Wang, Shuyun Lu et al. (+5 more)
Multimodal Large
Language Models (MLLMs) have recently made rapid progress toward unified Omni models that integrate
vision,
language, and audio. However, existing environments largely focus on 2D or 3D visual context and
vision-
language tasks, offering limited support for temporally dependent auditory signals and selective cross-modal integration, where different modalities may provide complementary or interfering information, which are essential capabilities for realistic
multimodal reasoning. As a result, whether models can actively coordinate modalities and reason under time-varying, irreversible conditions remains underexplored. To this end, we introduce EscapeCraft-4D, a customizable 4D environment for assessing selective cross-modal perception and time awareness in Omni models. It incorporates trigger-based auditory sources, temporally transient evidence, and location-dependent cues, requiring
agents to perform spatio-temporal
reasoning and proactive
multimodal integration under time constraints. Building on this environment, we curate a
benchmark to evaluate corresponding abilities across powerful models. Evaluation results suggest that models struggle with modality bias, and reveal significant gaps in current model's ability to integrate multiple modalities under time constraints. Further in-depth analysis uncovers how multiple modalities interact and jointly influence model decisions in complex
multimodal reasoning environments.
Read the full abstract ▾
All authors Yurui Dong, Ziyue Wang, Shuyun Lu, Dairu Liu, Xuechen Liu, Fuwen Luo, Peng Li, Yang Liu
Computer Vision
MV2UV: Generating High-quality UV Texture Maps with Multiview Prompts
Zheng Zhang, Qinchuan Zhang, Yuteng Ye et al. (+5 more)
Generating high-quality textures for 3D assets is a challenging task. Existing multiview texture generation methods suffer from the multiview inconsistency and missing textures on unseen parts, while UV inpainting texture methods do not generalize well due to insufficient UV data and cannot well utilize 2D image
diffusion priors. In this paper, we propose a new method called MV2UV that combines 2D generative priors from multiview generation and the inpainting ability of UV refinement to get high-quality texture maps. Our key idea is to adopt a UV space generative model that simultaneously inpaints unseen parts of multiview images while resolving the inconsistency of multiview images. Experiments show that our method enables a better texture generation quality than existing methods, especially in unseen occluded and multiview-inconsistent parts.
Read the full abstract ▾
All authors Zheng Zhang, Qinchuan Zhang, Yuteng Ye, Zhi Chen, Penglei Ji, Mengfei Li, Wenxiao Zhang, Yuan Liu
Computer Vision
Real-Time Human Frontal View Synthesis from a Single Image
Fangyu Lin, Yingdong Hu, Lunjie Zhu et al. (+4 more)
Photorealistic human novel view synthesis from a single image is crucial for democratizing immersive 3D telepresence, eliminating the need for complex multi-camera setups. However, current rendering-centric methods prioritize visual fidelity over explicit
geometric understanding and struggle with intricate regions like faces and hands, leading to temporal instability. Meanwhile, human-centric frameworks suffer from
memory bottlenecks since they typically rely on an auxiliary model to provide informative structural priors for
geometric modeling, which limits real-time performance. To address these challenges, we propose PrismMirror, a geometry-guided framework for instant frontal view synthesis from a single image. By avoiding external
geometric modeling and focusing on frontal view synthesis, our model optimizes visual integrity for telepresence. Specifically, PrismMirror introduces a novel cascade learning strategy that enables coarse-to-fine
geometric feature learning. It first directly learns coarse
geometric features, such as SMPL-X meshes and point clouds, and then refines textures through rendering supervision. To achieve real-time efficiency, we distill this unified framework into a lightweight linear
attention model. Notably, PrismMirror is the first
monocular human frontal view synthesis model that achieves real-time inference at 24 FPS, significantly outperforming previous methods in both visual authenticity and structural accuracy.
Read the full abstract ▾
All authors Fangyu Lin, Yingdong Hu, Lunjie Zhu, Zhening Liu, Yushi Huang, Zehong Lin, Jun Zhang
Computer Vision
Gym-V: A Unified Vision Environment System for Agentic Vision Research
Fanqing Meng, Lingxiao Du, Jiawei Gu et al. (+9 more)
As agentic systems increasingly rely on reinforcement learning from verifiable rewards, standardized “gym” infrastructure has become essential for rapid iteration, reproducibility, and fair comparison.
Vision agents lack such infrastructure, limiting systematic study of what drives their learning and where current models fall short. We introduce Gym-V, a unified platform of 179 procedurally generated visual environments across 10 domains with controllable difficulty, enabling controlled experiments that were previously infeasible across fragmented toolkits. Using it, we find that observation scaffolding is more decisive for training success than the choice of RL
algorithm, with captions and game rules determining whether learning succeeds at all. Cross-domain transfer experiments further show that training on diverse task categories generalizes broadly while narrow training can cause negative transfer, with multi-turn interaction amplifying all of these effects. Gym-V is released as a convenient foundation for training environments and evaluation toolkits, aiming to accelerate future research on agentic VLMs.
Read the full abstract ▾
All authors Fanqing Meng, Lingxiao Du, Jiawei Gu, Jiaqi Liao, Linjie Li, Zijian Wu, Xiangyan Liu, Ziqi Zhao, Mengkang Hu, Zichen Liu, Jiaheng Zhang, Michael Qizhe Shieh
Computer Vision
AnyCrowd: Instance-Isolated Identity-Pose Binding for Arbitrary Multi-Character Animation
Zhenyu Xie, Ji Xia, Michael Kampffmeyer et al. (+9 more)
Controllable character animation has advanced rapidly in recent years, yet multi-character animation remains underexplored. As the number of characters grows, multi-character reference encoding becomes more susceptible to latent identity
entanglement, resulting in identity bleeding and reduced controllability. Moreover, learning precise and spatio-temporally consistent correspondences between reference identities and driving pose sequences becomes increasingly challenging, often leading to identity-pose mis-binding and inconsistency in generated
videos. To address these challenges, we propose AnyCrowd, a
Diffusion Transformer (DiT)-based
video generation framework capable of scaling to an arbitrary number of characters. Specifically, we first introduce an Instance-Isolated Latent Representation (IILR), which encodes character instances independently prior to DiT processing to prevent latent identity
entanglement. Building on this disentangled representation, we further propose Tri-Stage Decoupled
Attention (TSDA) to bind identities to driving poses by decomposing self-
attention into: (i) instance-aware foreground
attention, (ii) background-centric interaction, and (iii) global foreground-background coordination. Furthermore, to mitigate
token ambiguity in overlapping regions, an Adaptive Gated Fusion (AGF) module is integrated within TSDA to predict identity-aware weights, effectively fusing competing
token groups into identity-consistent representations...
Read the full abstract ▾
All authors Zhenyu Xie, Ji Xia, Michael Kampffmeyer, Panwen Hu, Zehua Ma, Yujian Zheng, Jing Wang, Zheng Chong, Xujie Zhang, Xianhang Cheng, Xiaodan Liang, Hao Li
Computer Vision
Detection of Autonomous Shuttles in Urban Traffic Images Using Adaptive Residual Context
Mohamed Aziz Younes, Nicolas Saunier and Guillaume-Alexandre Bilodeau
The progressive automation of transport promises to enhance safety and sustainability through shared mobility. Like other vehicles and road users, and even more so for such a new technology, it requires monitoring to understand how it interacts in traffic and to evaluate its safety. This can be done with fixed cameras and
video object detection. However, the addition of new detection targets generally requires a fine-tuning approach for regular detection methods. Unfortunately, this implementation strategy will lead to a phenomenon known as catastrophic forgetting, which causes a degradation in scene understanding. In road safety applications, preserving contextual scene knowledge is of the utmost importance for protecting road users. We introduce the Adaptive Residual Context (ARC) architecture to address this. ARC links a frozen context branch and trainable task-specific branches through a Context-Guided Bridge, utilizing
attention to transfer spatial features while preserving pre-trained representations. Experiments on a custom
dataset show that ARC matches fine-tuned baselines while significantly improving knowledge retention, offering a data-efficient solution to add new vehicle categories for complex urban environments.
Read the full abstract ▾
Computer Vision
Pointing-Based Object Recognition
Lukáš Hajdúch and Viktor Kocur
This paper presents a comprehensive pipeline for recognizing objects targeted by human pointing gestures using RGB images. As human-
robot interaction moves toward more intuitive interfaces, the ability to identify targets of non-verbal communication becomes crucial. Our proposed system integrates several existing state-of-the-art methods, including object detection, body pose estimation,
monocular depth estimation, and
vision-
language models. We evaluate the impact of 3D spatial information reconstructed from a single image and the utility of image captioning models in correcting classification errors. Experimental results on a custom
dataset show that incorporating depth information significantly improves target identification, especially in complex scenes with overlapping objects. The modularity of the approach allows for deployment in environments where specialized depth sensors are unavailable.
Read the full abstract ▾
Computer Vision
AI Evasion and Impersonation Attacks on Facial Re-Identification with Activation Map Explanations
Noe Claudel, Weisi Guo and Yang Xing
Facial identification systems are increasingly deployed in surveillance and yet their vulnerability to
adversarial evasion and impersonation attacks pose a critical risk. This paper introduces a novel framework for generating
adversarial patches capable of both evasion and impersonation attacks against deep re-identification models across non-overlapping cameras. Unlike prior approaches that require iterative patch optimisation for each target, our method employs a conditional encoder-decoder network to synthesize
adversarial patches in a single forward pass, guided by multi-scale features from source and target images. The patches are optimised with a dual
adversarial objective comprising of pull and push terms. To enhance imperceptibility and aid physical deployment, we further integrate naturalistic patch generation using pre-trained latent
diffusion models. Experiments on standard pedestrian (
Market-1501, DukeMTMCreID) and facial recognition
benchmarks (CelebA-HQ, PubFig)
datasets demonstrate the effectiveness of the proposed method. Our
adversarial evasion attacks reduce mean Average Precision from 90% to 0.4% in white-box settings and from 72% to 0.4% in black-box settings, showing strong cross-model generalization. In targeted impersonation attacks, our framework achieves a success rate of 27% on CelebA-HQ, competing with other patch-based methods. We go further to use clustering of activation maps to interpret which features are most used by
adversarial attacks and propose a pathway for future countermeasures. The results highlight the practicality of
adversarial patch attacks on
retrieval-based systems and underline the urgent need for robust defense strategies.
Read the full abstract ▾
Computer Vision
RieMind: Geometry-Grounded Spatial Agent for Scene Understanding
Fernando Ropero, Erkin Turkoz, Daniel Matos et al. (+6 more)
Visual
Language Models (VLMs) have increasingly become the main paradigm for understanding indoor scenes, but they still struggle with metric and spatial
reasoning. Current approaches rely on end-to-end
video understanding or large-scale spatial question answering fine-tuning, inherently coupling perception and
reasoning. In this paper, we investigate whether decoupling perception and
reasoning leads to improved spatial
reasoning. We propose an agentic framework for static 3D indoor scene
reasoning that grounds an LLM in an explicit 3D scene graph (3DSG). Rather than ingesting
videos directly, each scene is represented as a persistent 3DSG constructed by a dedicated perception module. To isolate
reasoning performance, we instantiate the 3DSG from ground-truth annotations. The
agent interacts with the scene exclusively through structured
geometric tools that expose fundamental properties such as object dimensions, distances, poses, and spatial relationships. The results we obtain on the static split of VSI-Bench provide an upper bound under ideal perceptual conditions on the spatial
reasoning performance, and we find that it is significantly higher than previous works, by up to 16%, without task specific fine-tuning. Compared to base VLMs, our agentic variant achieves significantly better performance, with average improvements between 33% to 50%. These findings indicate that explicit
geometric grounding substantially improves spatial
reasoning performance, and suggest that structured representations offer a compelling alternative to purely end-to-end visual
reasoning.
Read the full abstract ▾
All authors Fernando Ropero, Erkin Turkoz, Daniel Matos, Junqing Du, Antonio Ruiz, Yanfeng Zhang, Lu Liu, Mingwei Sun, Yongliang Wang
Computer Vision
Spectral Rectification for Parameter-Efficient Adaptation of Foundation Models in Colonoscopy Depth Estimation
Xiaoxian Zhang, Minghai Shi and Lei Li
Accurate
monocular depth estimation is critical in colonoscopy for lesion localization and
navigation. Foundation models trained on natural images fail to generalize directly to colonoscopy. We identify the core issue not as a semantic gap, but as a statistical shift in the frequency domain: colonoscopy images lack the strong high-frequency edge and texture gradients that these models rely on for
geometric reasoning. To address this, we propose SpecDepth, a parameter-efficient adaptation framework that preserves the robust
geometric representations of the pre-trained models while adapting to the colonoscopy domain. Its key innovation is an adaptive spectral rectification module, which uses a learnable wavelet decomposition to explicitly model and amplify the attenuated high-frequency components in feature maps. Different from conventional fine-tuning that risks distorting high-level semantic features, this targeted, low-level adjustment realigns the input signal with the original inductive bias of the foundational model. On the public C3VD and SimCol3D
datasets, SpecDepth achieved state-of-the-art performance with an absolute relative error of 0.022 and 0.027, respectively. Our work demonstrates that directly addressing spectral mismatches is a highly effective strategy for adapting
vision foundation models to specialized medical imaging tasks. The code will be released publicly after the manuscript is accepted for publication.
Read the full abstract ▾
Computer Vision
Jiangyang Li, Cong Wan, SongLin Dong et al. (+4 more)
Vision-and-
Language Navigation (VLN) requires
agents to navigate photo-realistic environments following natural
language instructions. Current methods predominantly rely on imitation learning, which suffers from limited generalization and poor robustness to execution perturbations. We present NavGRPO, a reinforcement learning framework that learns goal-directed
navigation policies through Group Relative Policy Optimization. By exploring diverse trajectories and optimizing via within-group performance comparisons, our method enables
agents to distinguish effective strategies beyond expert paths without requiring additional value networks. Built on ScaleVLN, NavGRPO achieves superior robustness on R2R and REVERIE
benchmarks with +3.0% and +1.71% SPL improvements in unseen environments. Under extreme early-stage perturbations, we demonstrate +14.89% SPL gain over the baseline, confirming that goal-directed RL training builds substantially more robust
navigation policies. Code and models will be released.
Read the full abstract ▾
All authors Jiangyang Li, Cong Wan, SongLin Dong, Chenhao Ding, Qiang Wang, Zhiheng Ma, Yihong Gong
Computer Vision
IRIS: Intersection-aware Ray-based Implicit Editable Scenes
Grzegorz Wilczyński, Mikołaj Zieliński, Krzysztof Byrski et al. (+3 more)
Neural Radiance Fields achieve high-fidelity scene representation but suffer from costly training and rendering, while 3D Gaussian splatting offers real-time performance with strong empirical results. Recently, solutions that harness the best of both worlds by using Gaussians as proxies to guide neural field evaluations, still suffer from significant computational inefficiencies. They typically rely on
stochastic volumetric sampling to aggregate features, which severely limits rendering performance. To address this issue, a novel framework named IRIS (Intersection-aware Ray-based Implicit Editable Scenes) is introduced as a method designed for efficient and interactive scene editing. To overcome the limitations of standard ray marching, an analytical sampling strategy is employed that precisely identifies interaction points between rays and scene primitives, effectively eliminating empty space processing. Furthermore, to address the computational bottleneck of spatial neighbor lookups, a continuous feature aggregation mechanism is introduced that operates directly along the ray. By interpolating latent attributes from sorted intersections, costly 3D searches are bypassed, ensuring
geometric consistency, enabling high-fidelity, real-time rendering, and flexible shape editing. Code can be found at https://github.com/gwilczynski95/iris.
Read the full abstract ▾
All authors Grzegorz Wilczyński, Mikołaj Zieliński, Krzysztof Byrski, Joanna Waczyńska, Dominik Belter, Przemysław Spurek
Computer Vision
A PPO-Based Bitrate Allocation Conditional Diffusion Model for Remote Sensing Image Compression
Yuming Han, Jooho Kim and Anish Shakya
Existing remote sensing image compression methods still explore to balance high compression efficiency with the preservation of fine details and task-relevant information. Meanwhile, high-resolution drone imagery offers valuable structural details for urban monitoring and disaster assessment, but large-area
datasets can easily reach hundreds of gigabytes, creating significant challenges for storage and long-term management. In this paper, we propose a PPO-based bitrate allocation Conditional
Diffusion Compression (PCDC) framework. PCDC integrates a conditional
diffusion decoder with a PPO-based block-wise bitrate allocation strategy to achieve high compression ratios while maintaining strong perceptual performance. We also release a high-resolution drone image
dataset with richer structural details at a consistent low altitude over residential neighborhoods in coastal urban areas. Experimental results show compression ratios of 19.3x on DIV2K and 21.2x on the drone image
dataset. Moreover, downstream object detection experiments demonstrate that the reconstructed images preserve task-relevant information with negligible performance loss.
Read the full abstract ▾
Computer Vision
Oscillating Dispersion for Maximal Light-throughput Spectral Imaging
Jiuyun Zhang, Zhan Shi, Linsen Chen et al. (+1 more)
Existing computational spectral imaging systems typically rely on coded aperture and beam splitters that block a substantial fraction of incident light, degrading reconstruction quality under light-starved conditions. To address this limitation, we develop the Oscillating Dispersion Imaging Spectrometer (ODIS), which for the first time achieves near-full light throughput by axially translating a disperser between the conjugate image plane and a defocused position, sequentially capturing a panchromatic (PAN) image and a dispersed measurement along a single optical path. We further propose a PAN-guided Dispersion-Aware Deep Unfolding Network (PDAUN) that recovers high-fidelity spectral information from maskless dispersion under PAN structural guidance. Its data-fidelity step derives an FFT-Woodbury preconditioned solver by exploiting the cyclic-convolution property of the ODIS forward model, while a Dispersion-Aware Deformable Convolution module (DADC) corrects sub-pixel spectral misalignment using PAN features. Experiments show state-of-the-art performance on standard
benchmarks, and cross-system comparisons confirm that ODIS yields decisive gains under low illumination. High-fidelity reconstruction is validated on a physical prototype.
Read the full abstract ▾
All authors Jiuyun Zhang, Zhan Shi, Linsen Chen, Xun Cao
Computer Vision
MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction
Jiacheng Dong, Huan Li, Sicheng Zhou et al. (+3 more)
Reconstruction is a fundamental task in 3D
vision and a fundamental capability for spatial intelligence. Particularly, streaming 3D reconstruction is central to real-time spatial perception, yet existing recurrent online models often suffer from progressive degradation on long sequences due to state drift and forgetting, motivating inference-time remedies. We present MeMix, a training-free, plug-and-play module that improves streaming reconstruction by recasting the recurrent state into a
Memory Mixture. MeMix partitions the state into multiple independent
memory patches and updates only the least-aligned
memory patches while exactly preserving others. This selective update mitigates catastrophic forgetting while retaining $O(1)$ inference
memory, and requires no fine-tuning or additional learnable parameters, making it directly applicable to existing recurrent reconstruction models. Across standard
benchmarks (ScanNet, 7-Scenes, KITTI, etc.), under identical backbones and inference settings, MeMix reduces reconstruction completeness error by 15.3% on average (up to 40.0%) across 300–500 frame streams on 7-Scenes. The code is available at https://dongjiacheng06.github.io/MeMix/
Read the full abstract ▾
All authors Jiacheng Dong, Huan Li, Sicheng Zhou, Wenhao Hu, Weili Xu, Yan Wang
Computer Vision
Generative Video Compression with One-Dimensional Latent Representation
Zihan Zheng, Zhaoyang Jia, Naifu Xue et al. (+7 more)
Recent advancements in generative
video codec (GVC) typically encode
video into a 2D latent grid and employ high-capacity generative decoders for reconstruction. However, this paradigm still leaves two key challenges in fully exploiting spatial-temporal redundancy: Spatially, the 2D latent grid inevitably preserves intra-frame redundancy due to its rigid structure, where adjacent patches remain highly similar, thereby necessitating a higher bitrate. Temporally, the 2D latent grid is less effective for modeling long-term correlations in a compact and semantically coherent manner, as it hinders the aggregation of common contents across frames. To address these limitations, we introduce Generative
Video Compression with One-Dimensional (1D) Latent Representation (GVC1D). GVC1D encodes the
video data into extreme compact 1D latent
tokens conditioned on both short- and long-term contexts. Without the rigid 2D spatial correspondence, these 1D latent
tokens can adaptively attend to semantic regions and naturally facilitate
token reduction, thereby reducing spatial redundancy. Furthermore, the proposed 1D
memory provides semantically rich long-term context while maintaining low computational cost, thereby further reducing temporal redundancy. Experimental results indicate that GVC1D attains superior compression efficiency, where it achieves bitrate reductions of 60.4% under LPIPS and 68.8% under DISTS on the HEVC Class B
dataset, surpassing the previous
video compression methods.Project: https://gvc1d.github.io/
Read the full abstract ▾
All authors Zihan Zheng, Zhaoyang Jia, Naifu Xue, Jiahao Li, Bin Li, Zongyu Guo, Xiaoyi Zhang, Zhenghao Chen, Houqiang Li, Yan Lu
Computer Vision
GATE-AD: Graph Attention Network Encoding For Few-Shot Industrial Visual Anomaly Detection
Aggelos Psiris, Yannis Panagakis, Maria Vakalopoulou et al. (+1 more)
Few-Shot Industrial Visual Anomaly Detection (FS-IVAD) is a critical task in modern manufacturing, where automated product inspection systems must identify rare defects using only a handful of normal, defect-free training samples. Existing approaches typically model normality by assessing each patch largely through its individual appearance, e.g., through similarity to stored normal features, reconstruction from support samples, or distance from a learned normal representation. This paper introduces GATE-AD, whose key contribution is to instead model few-shot normality through local contextual relations: a patch is considered normal when it can be consistently recovered from its learned dependencies with its spatial neighbors. To realize this formulation, frozen self-supervised ViT patch
tokens are organized on a spatial neighbor graph and processed by a Graph
Attention Network (GAT), while representation
alignment and a Scaled Cosine Error (SCE) objective enable reconstruction-based anomaly scoring. Node masking and
attention dropout further regularize learning in the low-shot regime. On MVTec AD, VisA, and MPDD, GATE-AD achieves strong detection and localization performance across $1$- to $8$-shot settings, attaining the highest image-level AUROC in $10$ of $12$ settings and the highest image-level AP in $8$ of $12$ settings under a unified evaluation protocol, while also achieving the lowest inference latency among the compared methods. In order to facilitate reproducibility and further research, the source code of GATE-AD is available at https://github.com/gthpapadopoulos/GATE-AD.
Read the full abstract ▾
All authors Aggelos Psiris, Yannis Panagakis, Maria Vakalopoulou, Georgios Th. Papadopoulos