Computer Vision
A survey of facial recognition techniques
Aya Kaysan Bahjat
As multimedia content is quickly growing, the field of facial recognition has become one of the major research fields, particularly in the recent years. The most problematic area to researchers in image processing and computer
vision is the human face which is a complex object with myriads of distinctive features that can be used to identify the face. The survey of this survey is particularly focused on most challenging facial characteristics, including differences in the light, ageing, variation in poses, partial occlusion, and facial expression and presents methodological solutions. The factors, therefore, are inevitable in the creation of effective facial recognition mechanisms used on facial images. This paper reviews the most sophisticated methods of facial detection which are Hidden Markov Models, Principal Component Analysis (PCA), Elastic Cluster Plot Matching, Support Vector Machine (SVM), Gabor Waves, Artificial Neural Networks (ANN), Eigenfaces, Independent Component Analysis (ICA), and 3D Morphable Model. Alongside the works mentioned above, we have also analyzed the images of a number of facial databases, namely JAFEE, FEI, Yale, LFW, AT&T (then called ORL), and AR (created by Martinez and Benavente), to analyze the results. However, this survey is aimed at giving a thorough literature review of face recognition, and its applications, and some experimental results are provided at the end after a detailed discussion.
Read the full abstract ▾
Computer Vision
Synthetic FMCW Radar Range Azimuth Maps Augmentation with Generative Diffusion Model
Zhaoze Wang, Changxu Zhang, Tai Fei et al. (+5 more)
The scarcity and low diversity of well-annotated automotive radar
datasets often limit the performance of deep-learning-based environmental perception. To overcome these challenges, we propose a conditional generative framework for synthesizing realistic Frequency-Modulated Continuous-Wave radar Range-Azimuth Maps. Our approach leverages a generative
diffusion model to generate radar data for multiple object categories, including pedestrians, cars, and cyclists. Specifically, conditioning is achieved via Confidence Maps, where each channel represents a semantic class and encodes Gaussian-distributed annotations at target locations. To address radar-specific characteristics, we incorporate Geometry Aware Conditioning and Temporal Consistency Regularization into the generative process. Experiments on the ROD2021
dataset demonstrate that signal reconstruction quality improves by \SI{3.6}{dB} in Peak Signal-to-Noise Ratio over baseline methods, while training with a combination of real and synthetic
datasets improves overall mean Average Precision by 4.15% compared with conventional image-processing-based augmentation. These results indicate that our generative framework not only produces physically plausible and diverse radar spectrum but also substantially improves model generalization in downstream tasks.
Read the full abstract ▾
All authors Zhaoze Wang, Changxu Zhang, Tai Fei, Christopher Grimm, Yi Jin, Claas Tebruegge, Ernst Warsitz, Markus Gardill
Computer Vision
Ground What You See: Hallucination-Resistant MLLMs via Caption Feedback, Diversity-Aware Sampling, and Conflict Regularization
Miao Pan, Wangjie Gan, Jintao Chen et al. (+4 more)
While
Multimodal Large
Language Models (MLLMs) have achieved remarkable success across diverse tasks, their practical deployment is severely hindered by
hallucination issues, which become particularly acute during Reinforcement Learning (RL) optimization. This paper systematically analyzes the root causes of
hallucinations in MLLMs under RL training, identifying three critical factors: (1) an over-reliance on chained visual
reasoning, where inaccurate initial descriptions or redundant information anchor subsequent inferences to incorrect premises; (2) insufficient exploration diversity during policy optimization, leading the model to generate overly confident but erroneous outputs; and (3) destructive conflicts between training samples, where Neural Tangent Kernel (NTK) similarity causes false associations and unstable parameter updates. To address these challenges, we propose a comprehensive framework comprising three core modules. First, we enhance visual localization by introducing dedicated planning and captioning stages before the
reasoning phase, employing a quality-based caption reward to ensure accurate initial anchoring. Second, to improve exploration, we categorize samples based on the mean and variance of their reward distributions, prioritizing samples with high variance to focus the model on diverse and informative data. Finally, to mitigate sample interference, we regulate NTK similarity by grouping sample pairs and applying an InfoNCE loss to push overly similar pairs apart and pull dissimilar ones closer, thereby guiding gradient interactions toward a balanced range. Experimental results demonstrate that our proposed method significantly reduces
hallucination rates and effectively enhances the inference accuracy of MLLMs.
Read the full abstract ▾
All authors Miao Pan, Wangjie Gan, Jintao Chen, Wenqi Zhang, Bing Sun, Jianwei Yin, Xuhong Zhang
Computer Vision
SAPL: Semantic-Agnostic Prompt Learning in CLIP for Weakly Supervised Image Manipulation Localization
Xinghao Wang, Changtao Miao, Dianmo Sheng et al. (+6 more)
Malicious image manipulation threatens public safety and requires efficient localization methods. Existing approaches depend on costly pixel-level annotations which make training expensive. Existing weakly supervised methods rely only on image-level binary labels and focus on global classification, often overlooking local edge cues that are critical for precise localization. We observe that feature variations at manipulated boundaries are substantially larger than in interior regions. To address this gap, we propose Semantic-Agnostic
Prompt Learning (SAPL) in CLIP, which learns text
prompts that intentionally encode non-semantic, boundary-centric cues so that CLIPs
multimodal similarity highlights manipulation edges rather than high-level object semantics. SAPL combines two complementary modules Edge-aware Contextual
Prompt Learning (ECPL) and Hierarchical Edge Contrastive Learning (HECL) to exploit edge information in both textual and visual spaces. The proposed ECPL leverages edge-enhanced image features to generate learnable textual
prompts via an
attention mechanism,
embedding semantic-irrelevant information into text features, to guide CLIP focusing on manipulation edges. The proposed HECL extract genuine and manipulated edge patches, and utilize contrastive learning to boost the discrimination between genuine edge patches and manipulated edge patches. Finally, we predict the manipulated regions from the similarity map after processing. Extensive experiments on multiple public
benchmarks demonstrate that SAPL significantly outperforms existing approaches, achieving state-of-the-art localization performance.
Read the full abstract ▾
All authors Xinghao Wang, Changtao Miao, Dianmo Sheng, Tao Gong, Qi Chu, Nenghai Yu, Quanchen Zou, Deyue Zhang, Xiangzheng Zhang
Computer Vision
Two-step Authentication: Multi-biometric System Using Voice and Facial Recognition
Kuan Wei Chen, Ting Yi Lin, Wen Ren Yang et al. (+2 more)
We present a cost-effective two-step authentication system that integrates face identification and speaker verification using only a camera and microphone available on common devices. The pipeline first performs face recognition to identify a candidate user from a small enrolled group, then performs voice recognition only against the matched identity to reduce computation and improve robustness. For face recognition, a pruned VGG-16 based classifier is trained on an augmented
dataset of 924 images from five subjects, with faces localized by MTCNN; it achieves 95.1% accuracy. For voice recognition, a CNN speaker-verification model trained on LibriSpeech (train-other-360) attains 98.9% accuracy and 3.456% EER on test-clean. Source code and trained models are available at https://github.com/NCUE-EE-AIAL/Two-step-Authentication-Multi-biometric-System.
Read the full abstract ▾
All authors Kuan Wei Chen, Ting Yi Lin, Wen Ren Yang, Aryan Kesarwani, Riya Singh
Computer Vision
Deepfake detectors are DUMB: A benchmark to assess adversarial training robustness under transferability constraints
Adrian Serrano, Erwan Umlil and Ronan Thomas
Deepfake detection systems deployed in real-world environments are subject to adversaries capable of crafting imperceptible perturbations that degrade model performance. While
adversarial training is a widely adopted defense, its effectiveness under realistic conditions – where attackers operate with limited knowledge and mismatched data distributions - remains underexplored. In this work, we extend the DUMB –
Dataset soUrces, Model architecture and Balance - and DUMBer methodology to deepfake detection. We evaluate detectors robustness against
adversarial attacks under transferability constraints and cross-
dataset configuration to extract real-world insights. Our study spans five state-of-the-art detectors (RECCE, SRM, XCeption, UCF, SPSL), three attacks (PGD, FGSM, FPBA), and two
datasets (FaceForensics++ and Celeb-DF-V2). We analyze both attacker and defender perspectives mapping results to mismatch scenarios. Experiments show that
adversarial training strategies reinforce robustness in the in-distribution cases but can also degrade it under cross-
dataset configuration depending on the strategy adopted. These findings highlight the need for case-aware defense strategies in real-world applications exposed to
adversarial attacks.
Read the full abstract ▾
Computer Vision
Adaptive Conditional Contrast-Agnostic Deformable Image Registration with Uncertainty Estimation
Yinsong Wang, Xinzhe Luo, Siyi Du et al. (+1 more)
Deformable multi-contrast image registration is a challenging yet crucial task due to the complex, non-linear intensity relationships across different imaging contrasts. Conventional registration methods typically rely on iterative optimization of the deformation field, which is time-consuming. Although recent learning-based approaches enable fast and accurate registration during inference, their generalizability remains limited to the specific contrasts observed during training. In this work, we propose an adaptive conditional contrast-agnostic deformable image registration framework (AC-CAR) based on a random convolution-based contrast augmentation scheme. AC-CAR can generalize to arbitrary imaging contrasts without observing them during training. To encourage contrast-invariant feature learning, we propose an adaptive conditional feature modulator (ACFM) that adaptively modulates the features and the contrast-invariant latent regularization to enforce the consistency of the learned feature across different imaging contrasts. Additionally, we enable our framework to provide contrast-agnostic registration uncertainty by integrating a variance network that leverages the contrast-agnostic registration encoder to improve the trustworthiness and reliability of AC-CAR. Experimental results demonstrate that AC-CAR outperforms baseline methods in registration accuracy and exhibits superior generalization to unseen imaging contrasts. Code is available at https://github.com/Yinsong0510/AC-CAR.
Read the full abstract ▾
All authors Yinsong Wang, Xinzhe Luo, Siyi Du, Chen Qin
Computer Vision
VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction
Longbin Ji, Xiaoxiong Liu, Junyuan Shang et al. (+4 more)
Recent advances in
video generation have been dominated by
diffusion and flow-matching models, which produce high-quality results but remain computationally intensive and difficult to scale. In this work, we introduce VideoAR, the first large-scale Visual Autoregressive (VAR) framework for
video generation that combines multi-scale next-frame prediction with autoregressive modeling. VideoAR disentangles spatial and temporal dependencies by integrating intra-frame VAR modeling with causal next-frame prediction, supported by a 3D multi-scale tokenizer that efficiently encodes spatio-temporal dynamics. To improve long-term consistency, we propose Multi-scale Temporal RoPE, Cross-Frame Error Correction, and Random Frame Mask, which collectively mitigate error propagation and stabilize temporal coherence. Our multi-stage pretraining pipeline progressively aligns spatial and temporal learning across increasing resolutions and durations. Empirically, VideoAR achieves new state-of-the-art results among autoregressive models, improving FVD on UCF-101 from 99.5 to 88.6 while reducing inference steps by over 10x, and reaching a VBench score of 81.74-competitive with
diffusion-based models an order of magnitude larger. These results demonstrate that VideoAR narrows the performance gap between autoregressive and
diffusion paradigms, offering a scalable, efficient, and temporally consistent foundation for future
video generation research.
Read the full abstract ▾
All authors Longbin Ji, Xiaoxiong Liu, Junyuan Shang, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang
Computer Vision
WaveRNet: Wavelet-Guided Frequency Learning for Multi-Source Domain-Generalized Retinal Vessel Segmentation
Chanchan Wang, Yuanfang Wang, Qing Xu et al. (+1 more)
Domain-generalized retinal vessel
segmentation is critical for automated ophthalmic diagnosis, yet faces significant challenges from domain shift induced by non-uniform illumination and varying contrast, compounded by the difficulty of preserving fine vessel structures. While the Segment Anything Model (SAM) exhibits remarkable zero-shot capabilities, existing SAM-based methods rely on simple adapter fine-tuning while overlooking frequency-domain information that encodes domain-invariant features, resulting in degraded generalization under illumination and contrast variations. Furthermore, SAM's direct upsampling inevitably loses fine vessel details. To address these limitations, we propose WaveRNet, a wavelet-guided frequency learning framework for robust multi-source domain-generalized retinal vessel
segmentation. Specifically, we devise a Spectral-guided Domain Modulator (SDM) that integrates wavelet decomposition with learnable domain
tokens, enabling the separation of illumination-robust low-frequency structures from high-frequency vessel boundaries while facilitating domain-specific feature generation. Furthermore, we introduce a Frequency-Adaptive Domain Fusion (FADF) module that performs intelligent test-time domain selection through wavelet-based frequency similarity and soft-weighted fusion. Finally, we present a Hierarchical Mask-
Prompt Refiner (HMPR) that overcomes SAM's upsampling limitation through coarse-to-fine refinement with long-range dependency modeling. Extensive experiments under the Leave-One-Domain-Out protocol on four public retinal
datasets demonstrate that WaveRNet achieves state-of-the-art generalization performance. The source code is available at https://github.com/Chanchan-Wang/WaveRNet.
Read the full abstract ▾
All authors Chanchan Wang, Yuanfang Wang, Qing Xu, Guanxin Chen
Computer Vision
Context-Aware Decoding for Faithful Vision-Language Generation
Mehrdad Fazli, Bowen Wei and Ziwei Zhu
Hallucinations, generating responses inconsistent with the visual input, remain a critical limitation of large
vision-
language models (LVLMs), especially in open-ended tasks such as image captioning and visual
reasoning. In this work, we probe the layer-wise generation dynamics that drive
hallucinations and propose a training-free mitigation strategy. Employing the Logit Lens, we examine how LVLMs construct next-
token distributions across decoder layers, uncovering a pronounced commitment-depth gap: truthful
tokens accumulate probability mass on their final candidates earlier than hallucinatory ones. Drawing on this discovery, we introduce Context
Embedding Injection (CEI), a lightweight method that harnesses the hidden state of the last input
token-the context
embedding-as a grounding signal to maintain visual fidelity throughout decoding and curb
hallucinations. Evaluated on the CHAIR, AMBER, and MMHal-Bench
benchmarks (with a maximum
token length of 512), CEI outperforms state-of-the-art baselines across three LVLMs, with its dynamic variant yielding the lowest overall
hallucination rates. By integrating novel mechanistic insights with a scalable intervention, this work advances the mitigation of
hallucinations in LVLMs.
Read the full abstract ▾
Computer Vision
Performance of a Deep Learning-Based Segmentation Model for Pancreatic Tumors on Public Endoscopic Ultrasound Datasets
Pankaj Gupta, Priya Mudgil, Niharika Dutta et al. (+11 more)
Background: Pancreatic cancer is one of the most aggressive cancers, with poor survival rates. Endoscopic ultrasound (EUS) is a key diagnostic modality, but its effectiveness is constrained by operator subjectivity. This study evaluates a
Vision Transformer-based deep learning
segmentation model for pancreatic tumors. Methods: A
segmentation model using the USFM framework with a
Vision Transformer backbone was trained and validated with 17,367 EUS images (from two public
datasets) in 5-fold cross-validation. The model was tested on an independent
dataset of 350 EUS images from another public
dataset, manually segmented by radiologists. Preprocessing included grayscale conversion, cropping, and resizing to 512x512 pixels. Metrics included Dice similarity coefficient (DSC), intersection over union (IoU), sensitivity, specificity, and accuracy. Results: In 5-fold cross-validation, the model achieved a mean DSC of 0.651 +/- 0.738, IoU of 0.579 +/- 0.658, sensitivity of 69.8%, specificity of 98.8%, and accuracy of 97.5%. For the external validation set, the model achieved a DSC of 0.657 (95% CI: 0.634-0.769), IoU of 0.614 (95% CI: 0.590-0.689), sensitivity of 71.8%, and specificity of 97.7%. Results were consistent, but 9.7% of cases exhibited erroneous multiple predictions. Conclusions: The
Vision Transformer-based model demonstrated strong performance for pancreatic tumor
segmentation in EUS images. However,
dataset heterogeneity and limited external validation highlight the need for further refinement, standardization, and prospective studies.
Read the full abstract ▾
All authors Pankaj Gupta, Priya Mudgil, Niharika Dutta, Kartik Bose, Nitish Kumar, Anupam Kumar, Jimil Shah, Vaneet Jearth, Jayanta Samanta, Vishal Sharma, Harshal Mandavdhare, Surinder Rana, Saroj K Sinha, Usha Dutta
Computer Vision
Yohann Perron, Vladyslav Sydorov, Christophe Pottier et al. (+1 more)
Current approaches for segmenting ultra high resolution images either slide a window, thereby discarding global context, or downsample and lose fine detail. We propose a simple yet effective method that brings explicit multi scale
reasoning to
vision transformers, simultaneously preserving local details and global awareness. Concretely, we process each image in parallel at a local scale (high resolution, small crops) and a global scale (low resolution, large crops), and aggregate and propagate features between the two branches with a small set of learnable relay
tokens. The design plugs directly into standard
transformer backbones (eg ViT and Swin) and adds fewer than 2 % parameters. Extensive experiments on three ultra high resolution
segmentation benchmarks, Archaeoscape, URUR, and Gleason, and on the conventional Cityscapes
dataset show consistent gains, with up to 15 % relative mIoU improvement. Code and pretrained models are available at https://archaeoscape.ai/work/relay-
tokens/ .
Read the full abstract ▾
All authors Yohann Perron, Vladyslav Sydorov, Christophe Pottier, Loic Landrieu
Computer Vision
Phase4DFD: Multi-Domain Phase-Aware Attention for Deepfake Detection
Zhen-Xin Lin and Shang-Kuan Chen
Recent deepfake detection methods have increasingly explored frequency domain representations to reveal manipulation artifacts that are difficult to detect in the spatial domain. However, most existing approaches rely primarily on spectral magnitude, implicitly under exploring the role of phase information. In this work, we propose Phase4DFD, a phase aware frequency domain deepfake detection framework that explicitly models phase magnitude interactions via a learnable
attention mechanism. Our approach augments standard RGB input with Fast
Fourier Transform (FFT) magnitude and local binary pattern (LBP) representations to expose subtle synthesis artifacts that remain indistinguishable under spatial analysis alone. Crucially, we introduce an input level phase aware
attention module that uses phase discontinuities commonly introduced by synthetic generation to guide the model toward frequency patterns that are most indicative of manipulation before backbone feature extraction. The attended multi domain representation is processed by an efficient BNext M backbone, with optional channel spatial
attention applied for semantic feature refinement. Extensive experiments on the CIFAKE and DFFD
datasets demonstrate that our proposed model Phase4DFD outperforms state of the art spatial and frequency-based detectors while maintaining low computational overhead. Comprehensive ablation studies further confirm that explicit phase modeling provides complementary and non-redundant information beyond magnitude-only frequency representations.
Read the full abstract ▾
Computer Vision
Bidirectional Channel-selective Semantic Interaction for Semi-Supervised Medical Segmentation
Kaiwen Huang, Yizhe Zhang, Yi Zhou et al. (+2 more)
Semi-supervised medical image
segmentation is an effective method for addressing scenarios with limited labeled data. Existing methods mainly rely on frameworks such as mean teacher and dual-stream consistency learning. These approaches often face issues like error accumulation and model structural complexity, while also neglecting the interaction between labeled and unlabeled data streams. To overcome these challenges, we propose a Bidirectional Channel-selective Semantic Interaction~(BCSI) framework for semi-supervised medical image
segmentation. First, we propose a Semantic-Spatial Perturbation~(SSP) mechanism, which disturbs the data using two strong augmentation operations and leverages unsupervised learning with pseudo-labels from weak augmentations. Additionally, we employ consistency on the predictions from the two strong augmentations to further improve model stability and robustness. Second, to reduce noise during the interaction between labeled and unlabeled data, we propose a Channel-selective Router~(CR) component, which dynamically selects the most relevant channels for information exchange. This mechanism ensures that only highly relevant features are activated, minimizing unnecessary interference. Finally, the Bidirectional Channel-wise Interaction~(BCI) strategy is employed to supplement additional semantic information and enhance the representation of important channels. Experimental results on multiple benchmarking 3D medical
datasets demonstrate that the proposed method outperforms existing semi-supervised approaches.
Read the full abstract ▾
All authors Kaiwen Huang, Yizhe Zhang, Yi Zhou, Tianyang Xu, Tao Zhou
Computer Vision
LayerGS: Decomposition and Inpainting of Layered 3D Human Avatars via 2D Gaussian Splatting
Yinghan Xu and John Dingliana
We propose a novel framework for decomposing arbitrarily posed humans into animatable multi-layered 3D human avatars, separating the body and garments. Conventional single-layer reconstruction methods lock clothing to one identity, while prior multi-layer approaches struggle with occluded regions. We overcome both limitations by encoding each layer as a set of 2D Gaussians for accurate geometry and photorealistic rendering, and inpainting hidden regions with a pretrained 2D
diffusion model via score-
distillation sampling (SDS). Our three-stage training strategy first reconstructs the coarse canonical garment via single-layer reconstruction, followed by multi-layer training to jointly recover the inner-layer body and outer-layer garment details. Experiments on two 3D human
benchmark datasets (4D-Dress, Thuman2.0) show that our approach achieves better rendering quality and layer decomposition and recomposition than the previous state-of-the-art, enabling realistic virtual try-on under novel viewpoints and poses, and advancing practical creation of high-fidelity 3D human assets for immersive applications. Our code is available at https://github.com/RockyXu66/LayerGS
Read the full abstract ▾
Computer Vision
Kidney Cancer Detection Using 3D-Based Latent Diffusion Models
Jen Dusseljee, Sarah de Boer and Alessa Hering
In this work, we present a novel latent
diffusion-based pipeline for 3D kidney anomaly detection on contrast-enhanced abdominal CT. The method combines Denoising
Diffusion Probabilistic Models (DDPMs), Denoising
Diffusion Implicit Models (DDIMs), and Vector-Quantized Generative
Adversarial Networks (VQ-GANs). Unlike prior slice-wise approaches, our method operates directly on an image volume and leverages weak supervision with only case-level pseudo-labels. We
benchmark our approach against state-of-the-art supervised
segmentation and detection models. This study demonstrates the feasibility and promise of 3D latent
diffusion for weakly supervised anomaly detection. While the current results do not yet match supervised baselines, they reveal key directions for improving reconstruction fidelity and lesion localization. Our findings provide an important step toward annotation-efficient, generative modeling of complex abdominal anatomy.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Router-Suggest: Dynamic Routing for Multimodal Auto-Completion in Visually-Grounded Dialogs
Sandeep Mishra, Devichand Budagam, Anubhab Mandal et al. (+3 more)
Real-time
multimodal auto-completion is essential for digital assistants, chatbots, design tools, and healthcare consultations, where user inputs rely on shared visual context. We introduce
Multimodal Auto-Completion (MAC), a task that predicts upcoming characters in live chats using partially typed text and visual cues. Unlike traditional text-only auto-completion (TAC), MAC grounds predictions in
multimodal context to better capture user intent. To enable this task, we adapt MMDialog and ImageChat to create
benchmark datasets. We evaluate leading
vision-
language models (VLMs) against strong textual baselines, highlighting trade-offs in accuracy and efficiency. We present Router-Suggest, a router framework that dynamically selects between textual models and VLMs based on dialog context, along with a lightweight variant for resource-constrained environments. Router-Suggest achieves a 2.3x to 10x speedup over the best-performing VLM. A user study shows that VLMs significantly excel over textual models on user satisfaction, notably saving user typing effort and improving the quality of completions in multi-turn conversations. These findings underscore the need for
multimodal context in auto-completions, leading to smarter, user-aware assistants.
Read the full abstract ▾
All authors Sandeep Mishra, Devichand Budagam, Anubhab Mandal, Bishal Santra, Pawan Goyal, Manish Gupta
Computer Vision
Goal Force: Teaching Video Models To Accomplish Physics-Conditioned Goals
Nate Gillman, Yinghua Zhou, Zitian Tang et al. (+6 more)
Recent advancements in
video generation have enabled the development of “world models” capable of simulating potential futures for robotics and planning. However, specifying precise goals for these models remains a challenge; text instructions are often too abstract to capture physical nuances, while target images are frequently infeasible to specify for dynamic tasks. To address this, we introduce Goal Force, a novel framework that allows users to define goals via explicit force vectors and intermediate dynamics, mirroring how humans conceptualize physical tasks. We train a
video generation model on a curated
dataset of synthetic causal primitives-such as elastic collisions and falling dominos-teaching it to propagate forces through time and space. Despite being trained on simple physics data, our model exhibits remarkable zero-shot generalization to complex, real-world scenarios, including tool manipulation and multi-object causal chains. Our results suggest that by grounding
video generation in fundamental physical interactions, models can emerge as implicit neural physics simulators, enabling precise, physics-aware planning without reliance on external engines. We release all
datasets, code, model weights, and interactive
video demos at our project page.
Read the full abstract ▾
All authors Nate Gillman, Yinghua Zhou, Zitian Tang, Evan Luo, Arjan Chakravarthy, Daksh Aggarwal, Michael Freeman, Charles Herrmann, Chen Sun
Computer Vision
GeoSurDepth: Harnessing Foundation Model for Spatial Geometry Consistency-Oriented Self-Supervised Surround-View Depth Estimation
Weimin Liu, Wenjun Wang and Joshua H. Meng
Accurate surround-view depth estimation provides a competitive alternative to
laser-based sensors and is essential for 3D scene understanding in autonomous driving. While empirical studies have proposed various approaches that primarily focus on enforcing cross-view constraints at photometric level, few explicitly exploit the rich
geometric structure inherent in both
monocular and surround-view setting. In this work, we propose GeoSurDepth, a framework that leverages geometry consistency as the primary cue for surround-view depth estimation. Concretely, we utilize
vision foundation models as pseudo geometry priors and feature representation enhancement tool to guide the network to maintain surface normal consistency in spatial 3D space and regularize object- and texture-consistent depth estimation in 2D. In addition, we introduce a novel view synthesis pipeline where 2D-3D lifting is achieved with dense depth reconstructed via spatial warping, encouraging additional photometric supervision across temporal and spatial contexts, and compensating for the limitations of target-view image reconstruction. Finally, a newly-proposed adaptive joint motion learning strategy enables the network to adaptively emphasize informative spatial geometry cues for improved motion
reasoning. Extensive experiments on KITTI, DDAD and nuScenes demonstrate that GeoSurDepth achieves SoTA performance, validating the effectiveness of our approach. Our framework highlights the importance of exploiting geometry coherence and consistency for robust self-supervised depth estimation.
Read the full abstract ▾
Computer Vision
Boosting Latent Diffusion Models via Disentangled Representation Alignment
John Page, Xuesong Niu, Kai Wu et al. (+1 more)
Latent
Diffusion Models (LDMs) rely heavily on the compressed latent space provided by Variational Autoencoders (VAEs) for high-quality image generation. Recent studies have attempted to obtain generation-friendly VAEs by directly adopting
alignment strategies from LDM training, leveraging
Vision Foundation Models (VFMs) as representation
alignment targets. However, such
alignment paradigms overlook the fundamental differences in representational requirements between LDMs and VAEs. Simple feature mapping from local patches to high-dimensional semantics can induce semantic collapse, leading to the loss of fine-grained attributes. In this paper, we reveal a key insight: unlike LDMs that benefit from high-level global semantics, a generation-friendly VAE must possess strong semantic disentanglement capabilities to preserve fine-grained, attribute-level information in a structured manner. To address this discrepancy, we propose the Semantic-Disentangled VAE (Send-VAE). Deviating from previous shallow
alignment approaches, Send-VAE introduces a non-linear mapping architecture to effectively bridge the local structures of VAEs and the dense semantics of VFMs, thereby encouraging emergent disentangled properties in the latent space without explicit regularization. Extensive experiments establish a new paradigm for evaluating VAE latent spaces via low-level attribute separability and demonstrate that Send-VAE achieves state-of-the-art generation quality (FID of 1.21) on ImageNet 256x256.
Read the full abstract ▾
All authors John Page, Xuesong Niu, Kai Wu, Kun Gai
Computer Vision
SceneFoundry: Generating Interactive Infinite 3D Worlds
ChunTeng Chen, YiChen Hsu, YiWen Liu et al. (+5 more)
The ability to automatically generate large-scale, interactive, and physically realistic 3D environments is crucial for advancing robotic learning and
embodied intelligence. However, existing generative approaches often fail to capture the functional complexity of real-world interiors, particularly those containing articulated objects with movable parts essential for manipulation and
navigation. This paper presents SceneFoundry, a
language-guided
diffusion framework that generates apartment-scale 3D worlds with functionally articulated furniture and semantically diverse layouts for robotic training. From natural
language prompts, an LLM module controls floor layout generation, while
diffusion-based posterior sampling efficiently populates the scene with articulated assets from large-scale 3D repositories. To ensure physical usability, SceneFoundry employs differentiable guidance functions to regulate object quantity, prevent articulation collisions, and maintain sufficient walkable space for robotic
navigation. Extensive experiments demonstrate that our framework generates structurally valid, semantically coherent, and functionally interactive environments across diverse scene types and conditions, enabling scalable
embodied AI research. project page: https://anc891203.github.io/SceneFoundry-Demo/
Read the full abstract ▾
All authors ChunTeng Chen, YiChen Hsu, YiWen Liu, WeiFang Sun, TsaiChing Ni, ChunYi Lee, Min Sun, YuanFu Yang
Computer Vision
Adaptive Disentangled Representation Learning for Incomplete Multi-View Multi-Label Classification
Quanjiang Li, Zhiming Liu, Tianxiang Xu et al. (+2 more)
Multi-view multi-label learning frequently suffers from simultaneous feature absence and incomplete annotations, due to challenges in data acquisition and cost-intensive supervision. To tackle the complex yet highly practical problem while overcoming the existing limitations of feature recovery, representation disentanglement, and label semantics modeling, we propose an Adaptive Disentangled Representation Learning method (ADRL). ADRL achieves robust view completion by propagating feature-level affinity across modalities with neighborhood awareness, and reinforces reconstruction effectiveness by leveraging a
stochastic masking strategy. Through disseminating category-level association across label distributions, ADRL refines distribution parameters for capturing interdependent label prototypes. Besides, we formulate a mutual-information-based objective to promote consistency among shared representations and suppress information overlap between view-specific representation and other modalities. Theoretically, we derive the tractable bounds to train the dual-channel network. Moreover, ADRL performs prototype-specific feature selection by enabling independent interactions between label
embeddings and view representations, accompanied by the generation of pseudo-labels for each category. The structural characteristics of the pseudo-label space are then exploited to guide a discriminative trade-off during view fusion. Finally, extensive experiments on public
datasets and real-world applications demonstrate the superior performance of ADRL.
Read the full abstract ▾
All authors Quanjiang Li, Zhiming Liu, Tianxiang Xu, Tingjin Luo, Chenping Hou
Computer Vision
FlyPose: Towards Robust Human Pose Estimation From Aerial Views
Hassaan Farooq, Marvin Brenner and Peter Stütz
Unmanned Aerial Vehicles (
UAVs) are increasingly deployed in close proximity to humans for applications such as parcel delivery, traffic monitoring, disaster response and infrastructure inspections. Ensuring safe and reliable operation in these human-populated environments demands accurate perception of human poses and actions from an aerial viewpoint. This perspective challenges existing methods with low resolution, steep viewing angles and (self-)occlusion, especially if the application demands realtime feasibile models. We train and deploy FlyPose, a lightweight top-down human pose estimation pipeline for aerial imagery. Through multi-
dataset training, we achieve an average improvement of 6.8 mAP in person detection across the test-sets of Manipal-
UAV, VisDrone, HIT-
UAV as well as our custom
dataset. For 2D human pose estimation we report an improvement of 16.3 mAP on the challenging
UAV-Human
dataset. FlyPose runs with an inference latency of ~20 milliseconds including preprocessing on a Jetson Orin AGX Developer Kit and is deployed onboard a quadrotor
UAV during flight experiments. We also publish FlyPose-104, a small but challenging aerial human pose estimation
dataset, that includes manual annotations from difficult aerial perspectives: https://github.com/farooqhassaan/FlyPose.
Read the full abstract ▾
Computer Vision
ViTNT-FIQA: Training-Free Face Image Quality Assessment with Vision Transformers
Guray Ozgur, Eduarda Caldeira, Tahar Chettaoui et al. (+4 more)
Face Image Quality Assessment (FIQA) is essential for reliable face recognition systems. Current approaches primarily exploit only final-layer representations, while training-free methods require multiple forward passes or backpropagation. We propose ViTNT-FIQA, a training-free approach that measures the stability of patch
embedding evolution across intermediate
Vision Transformer (ViT) blocks. We demonstrate that high-quality face images exhibit stable feature refinement trajectories across blocks, while degraded images show erratic transformations. Our method computes Euclidean distances between L2-normalized patch
embeddings from consecutive
transformer blocks and aggregates them into image-level quality scores. We empirically validate this correlation on a quality-labeled synthetic
dataset with controlled degradation levels. Unlike existing training-free approaches, ViTNT-FIQA requires only a single forward pass without backpropagation or architectural modifications. Through extensive evaluation on eight
benchmarks (LFW, AgeDB-30, CFP-FP, CALFW, Adience, CPLFW, XQLFW, IJB-C), we show that ViTNT-FIQA achieves competitive performance with state-of-the-art methods while maintaining computational efficiency and immediate applicability to any pre-trained ViT-based face recognition model.
Read the full abstract ▾
All authors Guray Ozgur, Eduarda Caldeira, Tahar Chettaoui, Jan Niklas Kolf, Marco Huber, Naser Damer, Fadi Boutros
Computer Vision · Cross-listed from cs.AI
PII-VisBench: Evaluating Personally Identifiable Information Safety in Vision Language Models Along a Continuum of Visibility
G M Shahariar, Zabir Al Nazi, Md Olid Hasan Bhuiyan et al. (+1 more)
Vision Language Models (VLMs) are increasingly integrated into
privacy-critical domains, yet existing evaluations of personally identifiable information (PII) leakage largely treat
privacy as a static extraction task and ignore how a subject's online presence–the volume of their data available online–influences
privacy alignment. We introduce PII-VisBench, a novel
benchmark containing 4000 unique probes designed to evaluate VLM safety through the continuum of online presence. The
benchmark stratifies 200 subjects into four visibility categories: high, medium, low, and zero–based on the extent and nature of their information available online. We evaluate 18 open-source VLMs (0.3B-32B) based on two key metrics: percentage of PII probing queries refused (Refusal Rate) and the fraction of non-refusal responses flagged for containing PII (Conditional PII Disclosure Rate). Across models, we observe a consistent pattern: refusals increase and PII disclosures decrease (9.10% high to 5.34% low) as subject visibility drops. We identify that models are more likely to disclose PII for high-visibility subjects, alongside substantial model-family heterogeneity and PII-type disparities. Finally, paraphrasing and jailbreak-style
prompts expose attack and model-dependent failures, motivating visibility-aware safety evaluation and training interventions.
Read the full abstract ▾
All authors G M Shahariar, Zabir Al Nazi, Md Olid Hasan Bhuiyan, Zhouxing Shi
Computer Vision
FeatureSLAM: Feature-enriched 3D gaussian splatting SLAM in real time
Christopher Thirgood, Oscar Mendez, Erin Ling et al. (+2 more)
We present a real-time tracking SLAM system that unifies efficient camera tracking with photorealistic feature-enriched mapping using 3D Gaussian Splatting (3DGS). Our main contribution is integrating dense feature rasterization into the novel-view synthesis, aligned with a visual foundation model. This yields strong semantics, going beyond basic RGB-D input, aiding both tracking and mapping accuracy. Unlike previous semantic SLAM approaches (which embed pre-defined class labels) FeatureSLAM enables entirely new downstream tasks via free-viewpoint, open-set
segmentation. Across standard
benchmarks, our method achieves real-time tracking, on par with state-of-the-art systems while improving tracking stability and map fidelity without prohibitive compute. Quantitatively, we obtain 9% lower pose error and 8% higher mapping accuracy compared to recent fixed-set SLAM baselines. Our results confirm that real-time feature-embedded SLAM, is not only valuable for enabling new downstream applications. It also improves the performance of the underlying tracking and mapping subsystems, providing semantic and
language masking results that are on-par with offline 3DGS models, alongside state-of-the-art tracking, depth and RGB rendering.
Read the full abstract ▾
All authors Christopher Thirgood, Oscar Mendez, Erin Ling, Jon Storey, Simon Hadfield
Computer Vision
TAGRPO: Boosting GRPO on Image-to-Video Generation with Direct Trajectory Alignment
Jin Wang, Jianxiang Lu, Guangzheng Xu et al. (+10 more)
Recent studies have demonstrated the efficacy of integrating Group Relative Policy Optimization (GRPO) into flow matching models, particularly for text-to-image and text-to-
video generation. However, we find that directly applying these techniques to image-to-
video (I2V) models often fails to yield consistent reward improvements. To address this limitation, we present TAGRPO, a robust post-training framework for I2V models inspired by contrastive learning. Our approach is grounded in the observation that rollout
videos generated from identical initial noise provide superior guidance for optimization. Leveraging this insight, we propose a novel GRPO loss applied to intermediate latents, encouraging direct
alignment with high-reward trajectories while maximizing distance from low-reward counterparts. Furthermore, we introduce a
memory bank for rollout
videos to enhance diversity and reduce computational overhead. Despite its simplicity, TAGRPO achieves significant improvements over DanceGRPO in I2V generation. The deliverables will be updated at https://tagrpo.github.io/ .
Read the full abstract ▾
All authors Jin Wang, Jianxiang Lu, Guangzheng Xu, Comi Chen, Haoyu Yang, Linqing Wang, Peng Chen, Mingtao Chen, Zhichao Hu, Longhuang Wu, Shuai Shao, Qinglin Lu, Ping Luo
Computer Vision
Rotate Your Character: Revisiting Video Diffusion Models for High-Quality 3D Character Generation
Jin Wang, Jianxiang Lu, Comi Chen et al. (+9 more)
Generating high-quality 3D characters from single images remains a significant challenge in digital content creation, particularly due to complex body poses and self-occlusion. In this paper, we present RCM (Rotate your Character Model), an advanced image-to-
video diffusion framework tailored for high-quality novel view synthesis (NVS) and 3D character generation. Compared to existing
diffusion-based approaches, RCM offers several key advantages: (1) transferring characters with any complex poses into a canonical pose, enabling consistent novel view synthesis across the entire viewing orbit, (2) high-resolution orbital
video generation at 1024x1024 resolution, (3) controllable observation positions given different initial camera poses, and (4) multi-view conditioning supporting up to 4 input images, accommodating diverse user scenarios. Extensive experiments demonstrate that RCM outperforms state-of-the-art methods in both novel view synthesis and 3D generation quality.
Read the full abstract ▾
All authors Jin Wang, Jianxiang Lu, Comi Chen, Guangzheng Xu, Haoyu Yang, Peng Chen, Na Zhang, Yifan Xu, Longhuang Wu, Shuai Shao, Qinglin Lu, Ping Luo
Computer Vision
SketchVL: Policy Optimization via Fine-Grained Credit Assignment for Chart Understanding and More
Muye Huang, Lingling Zhang, Yifei Li et al. (+2 more)
Charts are high-density visual carriers of complex data and medium for information extraction and analysis. Due to the need for precise and complex visual
reasoning, automated chart understanding poses a significant challenge to existing
Multimodal Large
Language Models (MLLMs). Many MLLMs trained with reinforcement learning (RL) face the challenge of credit assignment. Their advantage estimation, typically performed at the trajectory level, cannot distinguish between correct and incorrect
reasoning steps within a single generated response. To address this limitation, we introduce SketchVL, a novel MLLM that optimized with FinePO, a new RL
algorithm designed for fine-grained credit assignment within each trajectory. SketchVL's methodology involves drawing its intermediate
reasoning steps as markers on the image and feeding the annotated image back to itself, creating a robust, multi-step
reasoning process. During training, the FinePO
algorithm leverages a Fine-grained Process Reward Model (FinePRM) to score each drawing action within a trajectory, thereby precisely assigning credit for each step. This mechanism allows FinePO to more strongly reward correct
tokens when a trajectory is globally successful, and more heavily penalize incorrect
tokens when the trajectory is globally suboptimal, thus achieving fine-grained reinforcement signals. Experiments show that SketchVL learns to align its step-level behavior with the FinePRM, achieving an average performance gain of 7.23% over its base model across chart
datasets, natural image
datasets, and mathematics, providing a promising new direction for training powerful
reasoning models.
Read the full abstract ▾
All authors Muye Huang, Lingling Zhang, Yifei Li, Yaqiang Wu, Jun Liu
Computer Vision · Cross-listed from cs.LG
Infinite-Precision Autoregressive Modeling for Vector Graphics and Layouts
Yeonsang Shin, Insoo Kim, Bongkeun Kim et al. (+2 more)
While
Transformer-based autoregressive models excel in data generation, their
token discretization strategy inherently limits their precision in continuous domains. We analyze the scalability limitations of existing discretization-based approaches for generating hybrid discrete-continuous sequences, particularly in high-precision domains such as logos, layouts, and
semiconductor circuit designs, where precision loss potentially leads to visual artifacts, aesthetic degradation, and even functional failure. To address the challenge, we propose a novel unified framework that jointly models discrete and continuous values for variable-length sequences. Our approach employs a hybrid approach that combines categorical prediction for discrete values with
diffusion-based modeling for continuous values, incorporating two key technical components: an end-of-sequence (EOS) logit adjustment mechanism that uses an MLP to dynamically adjust EOS
token logits based on sequence context, and a length regularization term integrated into the loss function. Additionally, we present ContLayNet, a large-scale
benchmark comprising 334K high-precision
semiconductor layout samples with specialized evaluation metrics that capture functional correctness, where precision errors significantly impact performance. Experiments on multiple domains show that our approach achieves higher-fidelity hybrid vector representations than discretization-based and fixed-schema baselines, while effectively scaling to high-precision generation.
Read the full abstract ▾
All authors Yeonsang Shin, Insoo Kim, Bongkeun Kim, Keonwoo Bae, Bohyung Han