Computer Vision
ExtraVAR: Stage-Aware RoPE Remapping for Resolution Extrapolation in Visual Autoregressive Models
Feihong Yan, Shaoyu Liu, Haixuan Wang et al. (+4 more)
Visual Autoregressive (VAR) models have emerged as a strong alternative to
diffusion for image synthesis, yet their fixed training resolution prevents direct generation at higher resolutions. Naively transferring training-free extrapolation methods from LLMs or
diffusion models to VAR yields three characteristic failure modes: global repetition, local repetition, and detail degradation. We trace them to a unified band-stage mismatch: VAR generates images in a coarse-to-fine, scale-wise process where each stage is driven by a distinct dominant RoPE frequency band, and each failure mode emerges when the dominant band of a particular stage is disrupted. Building on this insight, we propose Stage-Aware RoPE Remapping, a training-free strategy that assigns each frequency band a stage-specific remapping rule, jointly suppressing all three failure modes. We further observe that
attention becomes systematically dispersed as the image resolution increases. Existing methods typically depend on predefined
attention scaling factors, which are neither adaptive to the target resolution nor capable of faithfully capturing the actual extent of
attention dispersion. We therefore propose
Entropy-Driven Adaptive
Attention Calibration, which quantifies dispersion via a resolution-invariant normalized
entropy and yields a closed-form per-head scaling factor that realigns the extrapolated-resolution
attention entropy with its training-resolution counterpart. Extensive experiments show that our method consistently outperforms prior resolution-extrapolation methods in both structural coherence and fine-detail fidelity. Our code is available at https://github.com/feihongyan1/ExtraVAR.
Read the full abstract ▾
All authors Feihong Yan, Shaoyu Liu, Haixuan Wang, Shuai Lu, Linfeng Zhang, Huiqi Li, Xiangyang Ji
Computer Vision
Only Train Once: Uncertainty-Aware One-Class Learning for Face Authenticity Detection
Qingchao Jiang, Zhenxuan Hou, Zhiying Zhu et al. (+3 more)
The rapid
evolution of generative paradigms has enabled the creation of highly realistic imagery, which escalates the risks of identity fraud and the dissemination of disinformation. Most existing approaches frame face forgery detection as a fully supervised binary classification problem. Consequently, these models typically exhibit significant performance decay when tasked with detecting forgeries from previously unseen generative paradigms. Furthermore, these methods focus exclusively on either DeepFakes or fully synthesized faces, thereby failing to provide a generalized framework for universal face forgery detection. In this paper, we address this challenge by introducing FADNet (Face Authenticity Detector Net), which reformulates face forgery detection as a one-class classification (OCC) task. By training exclusively on authentic facial data to capture their intrinsic representations, FADNet flags any image whose feature
embedding deviates significantly from the learned distribution of real faces as a forgery. In this way, FADNet requires only a single training process without retraining for each specific forgery method. The framework incorporates Evidential Deep Learning (EDL) to quantify predictive uncertainty and utilizes a plug-and-play pseudo-forgery image generator (PFIG) to tighten decision boundaries around authentic data. Extensive experimental evaluations on the DF40 and ASFD
benchmarks demonstrate that FADNet achieves superior performance and generalization capabilities. Specifically, FADNet substantially outperforms existing state-of-the-art (SOTA) methods, yielding a remarkable average accuracy of 96.63% and an average precision of 98.83%. Our code is available at https://github.com/hzx111621/Only-Train-Once.
Read the full abstract ▾
All authors Qingchao Jiang, Zhenxuan Hou, Zhiying Zhu, Zhenxing Qian, Xinpeng Zhang, Zaiwang Gu
Computer Vision
Slum Detection and Density Mapping with AlphaEarth Foundations: A Representation Learning Evaluation Across 12 Global Cities
Shuyang Hou, Ziqi Liu, Haoyue Jiao et al. (+7 more)
Pixel-level slum mapping has long been constrained by limited cross-city generalisation, the absence of continuous density estimation, and weak global comparability. AlphaEarth Foundations (AEF), a globally consistent 64-dimensional annual surface
embedding at 10 m, offers a new analysis-ready basis for lightweight slum monitoring, but its applicability to slum detection - an indirectly coupled task shaped by both built form and socio-economic processes - remains untested. We evaluate AEF on slum classification and sub-pixel density estimation across 12 cities and 69 city-year pairs (2017-2024), using GRAM pseudo-masks as supervisory labels. The evaluation spans four training strategies, two protocols (random split and 3x3 spatial block cross-validation), six auxiliary feature configurations, and five baseline models, complemented by representation-level analyses (PCA, SHAP) and full-AOI mapping. Five findings emerge. (1) Same-city cross-year training is optimal under both protocols (median spatial F1 = 0.616, R^2 = 0.466); temporal expansion outperforms cross-city transfer, indicating city-scale representational drift. (2) Regression R^2 is driven primarily by zero/non-zero boundary discrimination: positive-pixel R^2 is consistently negative across all cities, revealing limited capacity to model intra-pixel density gradients at 10 m. (3) PC36 is consistently top-ranked across tasks; classification saturates at k = 32 while regression remains unsaturated at k = 64. (4) POI features yield the largest density gain (Delta R^2 = +0.064). (5) For six cities meeting dual-task usability thresholds, full-AOI inference across 2017-2024 preserves slum cluster structure (mean SSIM = 0.926). The study delineates the capabilities and complementarity needs of foundation-model
embeddings for slum monitoring.
Read the full abstract ▾
All authors Shuyang Hou, Ziqi Liu, Haoyue Jiao, Zhangyan Xu, Xiaopu Zhang, Lutong Xie, Yaxian Qing, Jianyuan Liang, Xuefeng Guan, Huayi Wua
Computer Vision
MUSDA: Multi-source Multi-modality Unsupervised Domain Adaptive 3D Object Detection for Autonomous Driving
Xiaohu Lu, Hamed Khatounabadi and Hayder Radha
With the advancement of autonomous driving, numerous annotated multi-modality
datasets have become available. This presents an opportunity to develop domain-adaptive 3D object detectors for new environments without relying on labor-intensive manual annotations. However, traditional domain adaptation methods typically focus on a single source domain or a single modality, limiting their effectiveness in multi-source, multi-modality scenarios. In this paper, we propose a novel framework for multi-source, multi-modality unsupervised domain adaptation in 3D object detection for autonomous driving. Given multiple labeled source domains and one unlabeled target domain, our framework first introduces hierarchical spatially-conditioned (HSC) domain classifiers, which jointly align features from both camera and
LiDAR modalities at two distinct levels for each source-target domain pair. To effectively leverage information from multiple source domains, we construct a prototype graph between each pair of domains. Based on this, we develop a prototype graph weighted (PGW) multi-source fusion strategy to aggregate predictions from multiple source detection heads. Experimental results on three widely used 3D object detection
datasets - Waymo, nuScenes, and Lyft - demonstrate that our proposed framework effectively integrates information across both modalities and source domains, consistently outperforming state-of-the-art methods.
Read the full abstract ▾
Computer Vision
Hystar: Hypernetwork-driven Style-adaptive Retrieval via Dynamic SVD Modulation
Yujia Cai, Boxuan Li, Chenghao Xu et al. (+1 more)
Query-based image
retrieval (QBIR) requires retrieving relevant images given diverse and often stylistically heterogeneous queries, such as sketches, artworks, or low-resolution previews. While large-scale
vision–
language representation models (VLRMs) like CLIP offer strong zero-shot
retrieval performance, they struggle with distribution shifts caused by unseen query styles. In this paper, we propose the Hypernetwork-driven Style-adaptive
Retrieval (Hystar), a lightweight framework that dynamically adapts model weights to each query's style. Hystar employs a hypernetwork to generate singular-value perturbations ($\Delta S$) for
attention layers, enabling flexible per-input adaptation, while static singular-value offsets on MLP layers ensure cross-style stability. To better handle semantic confusions across styles, we design StyleNCE as part of Hystar, an optimal-transport-weighted contrastive loss that emphasizes hard cross-style negatives. Extensive experiments on multi-style
retrieval and cross-style classification
benchmarks demonstrate that Hystar consistently outperforms strong baselines, achieving state-of-the-art performance while being parameter-efficient and stable across styles.
Read the full abstract ▾
All authors Yujia Cai, Boxuan Li, Chenghao Xu, Jiexi Yan
Computer Vision · Cross-listed from physics.optics
Measurement-Adapted Eigentask Representations for Photon-Limited Optical Readout
Tianyang Chen, Mandar M. Sohoni, Saeed A. Khan et al. (+5 more)
Optical readout in low-light imaging is fundamentally limited by measurement noise, including
photon shot noise, detector noise, and quantization error. In this regime, downstream inference depends not only on the optical front end, but also on how noisy high-dimensional sensor measurements are represented before classification or decision-making. Here we show that eigentasks provide a measurement-adapted representation for optical sensor outputs by ordering readout features according to their resolvability under noise. Using experimental data from a lens-based optical imaging system and a reanalysis of published data from a single-
photon-detection neural network, we find that eigentask representations frequently outperform standard baselines including principal component analysis and filtering-based compression. The advantage is most pronounced in
photon-limited, few-shot, and higher-difficulty classification regimes. In few-shot MPEG-7 classification, for example, the advantage over other methods reaches about 10 percentage points as the number of classes increases. In these settings, eigentasks yield more informative low-dimensional features and improve sample-efficient downstream learning. These results identify measurement-adapted representation as a promising strategy for optical inference when
photon budget, acquisition time, and task complexity are constrained.
Read the full abstract ▾
All authors Tianyang Chen, Mandar M. Sohoni, Saeed A. Khan, Jérémie Laydevant, Shi-Yuan Ma, Tianyu Wang, Peter L. McMahon, Hakan E. Türeci
Computer Vision
Med-StepBench: A Hierarchical Reasoning Framework for Evaluating Hallucinations in Medical Vision-Language Models
Minh Khoi Nguyen, Dai Lam Le, Amir Reza Jafari et al. (+8 more)
Large
vision-
language models (VLMs) demonstrate strong performance in medical image understanding, but frequently generate clinically plausible yet incorrect statements, raising significant safety concerns. Existing medical
hallucination benchmarks primarily focus on 2D imaging with one-shot diagnostic questions, offering limited insight into whether predictions are grounded in correct localization and abnormality identification, allowing critical
reasoning errors to remain hidden behind seemingly correct diagnoses. We introduce Med-StepBench, the first large-scale
benchmark for step-wise
hallucination detection in 3D oncological PET/CT, comprising over 12,000 images and more than 1,000,000 image-statement pairs across volumetric and multi-view 2D data, which decomposes clinical
reasoning into four expert-designed diagnostic stages. Using clinician-verified annotations, we perform the first step-level evaluation of general-purpose and medical VLMs, revealing systematic failure modes obscured by aggregate accuracy metrics. Furthermore, we show that current VLMs are highly susceptible to
adversarial yet clinically plausible intermediate explanations, which significantly amplify
hallucinations despite contradictory visual evidence. Together, our findings highlight fundamental limitations in grounding multi-step clinical
reasoning and establish Med-StepBench as a rigorous
benchmark for developing safer and more reliable medical VLMs.
Read the full abstract ▾
All authors Minh Khoi Nguyen, Dai Lam Le, Amir Reza Jafari, Tuan Dung Nguyen, Mai Hong Son, Mai Huy Thong, Quang Huy Nguyen, Thanh Trung Nguyen, Reza Farahbakhsh, Noel Crespi, Phi Le Nguyen
Computer Vision
Omni-Persona: Systematic Benchmarking and Improving Omnimodal Personalization
Yeongtak Oh, Dongwook Lee, Sangkwon Park et al. (+2 more)
While
multimodal large
language models have advanced across text, image, and audio, personalization research has remained primarily
vision-
language, with unified omnimodal benchmarking that jointly covers text, image, and audio still limited, and lacking the methodological rigor to account for absent-persona scenarios or systematic grounding studies. We introduce Omni-Persona, the first comprehensive
benchmark for omnimodal personalization. We formalize the task as cross-modal routing over the Persona Modality Graph, encompassing 4 task groups and 18 fine-grained tasks across ~750 items. To rigorously diagnose grounding behavior, we propose Calibrated Accuracy (Cal), which jointly evaluates correct grounding and appropriate abstention, incorporating absent-persona queries within a unified evaluation framework. On our dedicated experiments, three diagnostic findings emerge: (i) recent open-weight models show a consistent audio-vs-visual grounding gap that RLVR partially narrows via dense rule-based supervision; (ii) recall score and parameter scale are incomplete diagnostics, since strong recall can coexist with absent-persona
hallucination and larger models do not always achieve higher Cal, exposing calibration as a separate evaluation axis; and (iii) SFT is limited by the scalability of ground-truth annotation. RLVR improves recall but not calibrated abstention under our reward design, leaving Cal at or below the base model. Omni-Persona thus serves as a diagnostic framework that surfaces the pitfalls of omnimodal personalization, guiding future post-training and reward design.
Read the full abstract ▾
All authors Yeongtak Oh, Dongwook Lee, Sangkwon Park, Heeseung Kim, Sungroh Yoon
Computer Vision · Cross-listed from cs.RO
StereoPolicy: Improving Robotic Manipulation Policies via Stereo Perception
Evans Han, Yunfan Jiang, Yingke Wang et al. (+6 more)
Recent advances in
robot imitation learning have produced powerful visuomotor policies that manipulate diverse objects from visual inputs. However,
monocular observations lack depth information, which is critical for precise manipulation in cluttered or geometrically complex scenes. Explicit depth maps and point clouds are often noisy and fragile in real-world manipulation. We introduce StereoPolicy, a visuomotor policy learning framework that directly leverages synchronized stereo image pairs to improve spatial perception without constructing explicit 3D representations. StereoPolicy processes each image with pretrained 2D
vision encoders and fuses left-right features through a cross-
attention-based Stereo
Transformer, learning task-conditioned binocular representations. The framework integrates with
diffusion-based and pretrained
vision-
language-action (VLA) policies, delivering consistent improvements over RGB, RGB-D, point cloud, and multi-view baselines across three simulation
benchmarks and seven real-
robot tabletop and bimanual mobile manipulation tasks. Our results support task-trained binocular fusion as an effective way to adapt 2D visual representations for manipulation.
Read the full abstract ▾
All authors Evans Han, Yunfan Jiang, Yingke Wang, Haoyue Xiao, Huang Huang, Jianwen Xie, Jiajun Wu, Li Fei-Fei, Ruohan Zhang
Computer Vision
Geometric 4D Stitching for Grounded 4D Generation
Sunwoo Park, Taesung Kwon and Jong Chul Ye
Recent 4D generation methods complete scene-level missing information using generative models and reconstruct the scene into radiance-based representations. However, these pipelines often present
geometric inconsistencies in the generated content, and the radiance-based reconstruction requires expensive optimization. Furthermore, radiance-based representations often absorb these
geometric inconsistencies into their view-dependent nature, failing to enforce the grounded
geometric consistency. To address these issues, we propose
Geometric 4D Stitching, an efficient framework that explicitly identifies missing
geometric regions and complements them with geometrically grounded 4D stitches. As a result, our method constructs 4D scene representations in under 10 minutes on a single NVIDIA RTX 5090 GPU per one-step scene expansion, while improving
geometric consistency. Moreover, we demonstrate that our explicit 4D stitching supports interative expansion of 4D mesh as well as 4D scene editing.
Read the full abstract ▾
Computer Vision
ERASE: Eliminating Redundant Visual Tokens via Adaptive Two-Stage Token Pruning
Yuna Lee, Kyoungho Min and Yulhwa Kim
Recent advances in
Vision-
Language Models (VLMs) enable large
language models (LLMs) to process high-resolution images, significantly improving real-world
multimodal understanding. However, this capability introduces a large number of
vision tokens, incurring substantial computational overhead. To mitigate this issue, various
vision token compression methods have been proposed. Existing methods often estimate different sources of visual redundancy using learned representations or fixed
pruning schedules. We propose ERASE, an adaptive two-stage framework that separates image-level redundancy removal from instruction-dependent
token pruning. Stage 1 derives image-dependent
token retention from lightweight raw-image statistics, while Stage 2 progressively removes instruction-irrelevant
tokens across decoder layers. Experiments demonstrate substantial
token reduction while preserving accuracy: on Qwen2.5-VL-7B, ERASE retains 95.70% of the original model's accuracy at 25%
token retention. Our code is available at https://github.com/Tuna-Luna/ERASE.
Read the full abstract ▾
Computer Vision
INFANiTE: Implicit Neural representation for high-resolution Fetal brain spatio-temporal Atlas learNing from clinical Thick-slicE MRI
Xiaotian Hu, Mingxuan Liu, Hongjia Yang et al. (+10 more)
Spatio-temporal fetal
brain atlases are important for characterizing normative neurodevelopment and identifying congenital anomalies. However, existing atlas construction pipelines necessitate days for slice-to-volume reconstruction (SVR) to generate high-resolution 3D
brain volumes and several additional days for iterative volume registration, thereby rendering atlas construction from large-scale cohorts prohibitively impractical. We address these limitations with INFANiTE, an Implicit Neural Representation (INR) framework for high-resolution Fetal
brain spatio-temporal Atlas learNing from clinical Thick-slicE MRI scans, bypassing both the costly SVR and the iterative non-rigid registration steps entirely, thereby substantially accelerating atlas construction. Extensive experiments demonstrate that INFANiTE outperforms existing baselines in subject consistency, reference fidelity, intrinsic quality and biological plausibility, even under challenging sparse-data settings. Additionally, INFANiTE reduces the end-to-end processing time (i.e., from raw scans to the final atlas) from days to hours compared to the traditional 3D volume-based pipeline (e.g., SyGN), facilitating large-scale population-level fetal
brain analysis. Code: https://github.com/hu2274898/INFANiTE
Read the full abstract ▾
All authors Xiaotian Hu, Mingxuan Liu, Hongjia Yang, Tongxi Song, Yijin Li, Yifei Chen, Haoxiang Li, Zihan Li, Yingqi Hao, Ziyu Li, Yi Liao, Haibo Qu, Qiyuan Tian
Computer Vision
OZ-TAL: Online Zero-Shot Temporal Action Localization
Chaolei Han, Hongsong Wang, Xin Gong et al. (+1 more)
Online Temporal Action Localization (On-TAL) aims to detect the occurrence time and category of actions in untrimmed streaming
videos immediately upon their completion. Recent advancements in this field focus on developing more sophisticated frameworks, shifting from Online Action Detection (OAD)-based aggregation paradigm to instance-level understanding. However, existing approaches are typically trained on specific domains and often exhibit limited generalization capabilities when applied to arbitrary
videos, particularly in the presence of previously unseen actions. In this paper, we introduce a new task called Online Zero-shot Temporal Action Localization (OZ-TAL), which aims to detect previously unseen actions in an online fashion. Furthermore, we propose a training-free framework that leverages off-the-shelf
Vision-
Language Models (VLMs) while introducing additional mechanisms to enhance visual representations and mitigate their inherent biases. We establish new
benchmarks and representative baselines for OZ-TAL on THUMOS14 and ActivityNet-1.3, and extensive experiments demonstrate that our method substantially outperforms existing state-of-the-art approaches under both offline and online zero-shot settings.
Read the full abstract ▾
All authors Chaolei Han, Hongsong Wang, Xin Gong, Jie Gui
Computer Vision · Cross-listed from cs.RO
HiDrive: A Closed-Loop Benchmark for High-Level Autonomous Driving
Zhongyu Xia, Guanyu Zhu, Guo Tang et al. (+2 more)
End-to-end autonomous driving has witnessed rapid progress, yet existing
benchmarks are increasingly saturated, with state-of-the-art models achieving near-perfect scores on widely used open-loop and closed-loop
benchmarks. This saturation does not mean that the problem has been solved; instead, it reveals that current
benchmarks remain limited in scenario diversity, object variety, and the breadth of driving capabilities they evaluate. In particular, they lack sufficient long-tail scenarios involving rare but safety-critical objects and fail to assess advanced decision-making such as legal compliance, ethical
reasoning, and emergency response. To address these gaps, we propose HiDrive, a new closed-loop
benchmark for end-to-end autonomous driving that emphasizes long-tail scenarios and a richer evaluation of driving capabilities. HiDrive introduces a diverse set of rare objects and uncommon traffic situations, and expands evaluation from basic driving skills to more advanced capabilities, including rule compliance, moral
reasoning, and context-dependent emergency maneuvers. Correspondingly, we extend previous collision-avoidance-centered metrics into a comprehensive evaluation system that encompasses collision and braking, traffic-rule compliance, and moral-
reasoning indicators. Built on a more advanced physics engine, HiDrive provides physically realistic lighting and high-fidelity visual rendering, offering a more challenging and realistic testbed for assessing whether autonomous driving systems can handle the complexity of real-world deployment. The HiDrive software, source code, digital assets, and documentation are available at https://github.com/VDIGPKU/HiDrive.
Read the full abstract ▾
All authors Zhongyu Xia, Guanyu Zhu, Guo Tang, Wenhao Chen, Yongtao Wang
Computer Vision
Towards Generalist Game Agents: A Survey of Foundation Models in Virtual Worlds
Kuan Zhang, Dongchen Liu, Qiyue Zhao et al. (+12 more)
The real world unfolds along a single set of physics laws, yet human intelligence demonstrates a remarkable capacity to generalize experiences from this singular physical existence into a multiverse of games, each governed by entirely different rules, aesthetics, physics, and objectives. This omni-reality adaptability is a hallmark of general intelligence. As Artificial Intelligence progresses towards Artificial General Intelligence, the multiverse of games has evolved from mere entertainment into the ultimate ground for training and evaluating AGI. The pursuit of this generality has unfolded across four eras: from environment-specific symbolic and reinforcement learning
agents, to current large foundation models acting as generalist players, and toward a future creator stage where
agent both creates new game worlds and continually evolves within them. We trace the full lifecycle of a generalist game player along four interdependent pillars:
Dataset, Model, Harness, and
Benchmark. Every advance across these pillars can be read as an attempt to break one of five fundamental trade-offs that currently bound the whole system. Building on this end-to-end view, we chart a five-step roadmap, progressing from single-game mastery to the ultimate creator stage in which the
agent simultaneously creates and evolves within theoretical game multiverse. Taken together, our work offers a unified lens onto a rapidly shifting field, and a principled path toward the omnipotent generalist
agent capable of seamlessly mastering any challenge within the multiverse of games, thereby paving the way for AGI.
Read the full abstract ▾
All authors Kuan Zhang, Dongchen Liu, Qiyue Zhao, Tianyu Xin, Yue Su, Haisheng Wang, Han Yin, Hongbo Ma, Peize Li, Tianjun Gu, Xiangnan Wu, Xinran Zhang, Yongxuan Li, Zirong Chen, Yiming Li
Computer Vision
Learning to Perceive "Where": Spatial Pretext Tasks for Robust Self-Supervised Learning
Yang Shen, Yusen Cai, Weronika Hryniewska-Guzik et al. (+2 more)
Existing self-supervised learning (SSL) methods primarily learn object-invariant representations but often neglect the spatial structure and relationships among object parts. To address this limitation, we introduce Spatial Prediction (SP), a spatially aware pretext regression task that predicts the relative position and scale between a pair of disentangled local views from the same image. By modeling part-to-part relationships in a continuous
geometric space, SP encourages representations to capture fine-grained spatial dependencies beyond invariant categorical semantics, thereby learning the compositional structure of visual scenes. SP is implemented as a decoupled plug-in and can be seamlessly integrated into diverse SSL frameworks. Extensive experiments show consistent improvements across image recognition, fine-grained classification, semantic
segmentation, and depth estimation, as well as substantial gains in out-of-distribution robustness for object recognition. To evaluate spatial
reasoning, we introduce (1) a position and scale prediction task on image patch pairs and (2) a jigsaw understanding task requiring patch reordering and recognition after reconstruction. Strong performance on these tasks indicates improved spatial structure and
geometric awareness. Overall, explicitly modeling spatial information provides an effective inductive bias for SSL, leading to more structured representations and better generalization. Code and models will be released.
Read the full abstract ▾
All authors Yang Shen, Yusen Cai, Weronika Hryniewska-Guzik, Qing Lin, Mengmi Zhang
Computer Vision
SDTalk: Structured Facial Priors and Dual-Branch Motion Fields for Generalizable Gaussian Talking Head Synthesis
Peng Jia, Zhen Xiao, Jia Li et al. (+3 more)
High-quality, real-time talking head synthesis remains a fundamental challenge in computer
vision. Existing reconstruction- and rendering-based methods typically rely on identity-specific models, limiting cross-identity generalization. To address this issue, we propose SDTalk, a one-shot 3D Gaussian Splatting (3DGS)-based framework that generalizes to unseen identities without personalized training or fine-tuning. Our framework comprises two modules with a two-stage training strategy. In the first stage, we incorporate structured facial priors into the reconstruction module and separately predict 3DGS parameters for visible and occluded regions, enabling complete head reconstruction from a single image. In the second stage, we introduce a dual-branch motion field to model coarse and fine facial dynamics, improving detail fidelity and lip synchronization. Experiments demonstrate that SDTalk surpasses existing methods in both visual quality and inference efficiency.
Read the full abstract ▾
All authors Peng Jia, Zhen Xiao, Jia Li, Xueliang Liu, Zhenzhen Hu, Lingyun Yu
Computer Vision · Cross-listed from cs.RO
JODA: Composable Joint Dynamics for Articulated Objects
Tianhong Gao, Cheng Yu, Yinghao Xu et al. (+1 more)
Articulated objects used in simulation and
embodied AI are typically specified by geometry and kinematic structure, but lack the fine-grained dynamical effects that govern realistic mechanical behavior, such as frictional holding, detents, soft closing, and snap latching. Existing approaches either ignore the detailed structure of dynamics entirely, or use simple models with limited expressiveness. We introduce JODA, a framework for generating joint-level dynamics as a structured three-channel field over the joint degree of freedom, capturing conservative forces, dry friction, and damping. Instantiated using shape-constrained piecewise cubic interpolation (PCHIP), this formulation defines a compact and expressive function space that is both interpretable and compatible with differentiable simulation. Building on this representation, we develop methods for inferring and refining joint dynamics from
multimodal inputs. Given visual observations and joint context, a
vision-
language model proposes structured dynamical primitives, which are composed into a unified dynamics field. The resulting representation supports both direct manipulation and gradient-based refinement. We demonstrate that JODA enables plausible and controllable modeling of diverse joint behaviors, providing a unified interface for inference, editing, and optimization. Code and example assets with their generated profiles will be released upon publication.
Read the full abstract ▾
All authors Tianhong Gao, Cheng Yu, Yinghao Xu, Mengyu Chu
Computer Vision · Cross-listed from cs.AI
LoopVLA: Learning Sufficiency in Recurrent Refinement for Vision-Language-Action Models
Boyang Shen, Kaixiang Yang, Hao Wang et al. (+4 more)
Current
Vision-
Language-Action (VLA) models typically treat the deepest representation of a
vision-
language backbone as universally optimal for action prediction. However, robotic manipulation is composed of many frequent closed-loop spatial adjustments, for which excessive abstraction may waste computation and weaken low-level
geometric cues essential for precise control. Existing early-exit strategies attempt to reduce computation by stopping at predefined layers or applying heuristic rules such as action consistency, but they do not directly answer when a representation is actually sufficient for action. In this paper, we present LoopVLA, a recurrent VLA architecture that jointly learns representation refinement, action prediction, and sufficiency estimation. LoopVLA iteratively applies a shared
Transformer block to refine
multimodal tokens, and at each iteration produces both a candidate action and a sufficiency score that estimates whether further refinement is necessary. By sharing parameters across iterations, LoopVLA decouples refinement from absolute layer indices and grounds sufficiency estimation in the evolving representation itself. Since sufficiency has no direct supervision, we introduce a self-supervised distribution
alignment objective, where intermediate confidence scores are trained to match the relative action quality across refinement steps, thereby linking sufficiency learning to policy optimization signals. Experiments on LIBERO, LIBERO-Plus, and VLA-Arena show that LoopVLA pushes the efficiency-performance frontier of VLA policies, reducing parameters by 45% and improving inference throughput by up to 1.7 times while matching or outperforming strong baselines in task success.
Read the full abstract ▾
All authors Boyang Shen, Kaixiang Yang, Hao Wang, Qiuyu Yu, Qiang Xie, Qiang Li, Zhiwei Wang
Computer Vision
Urban-ImageNet: A Large-Scale Multi-Modal Dataset and Evaluation Framework for Urban Space Perception
Yiwei Ou, Chung Ching Cheung, Jun Yang Ang et al. (+5 more)
We present Urban-ImageNet, a large-scale multi-modal
dataset and evaluation
benchmark for urban space perception from user-generated social media imagery. The corpus contains over 2 Million public social media images and paired textual posts collected from Weibo across 61 urban sites in 24 Chinese cities across 2019-2025, with controlled
benchmark subsets at 1K, 10K, and 100K scale and a full 2M corpus for large-scale training and evaluation. Urban-ImageNet is organized by HUSIC, a Hierarchical Urban Space Image Classification framework that defines a 10-class taxonomy grounded in urban theory. The taxonomy is designed to distinguish activated and non-activated public spaces, exterior and interior urban environments, accommodation spaces, consumption content, portraits, and non-spatial social-media content. Rather than treating urban imagery as generic scene data, Urban-ImageNet evaluates whether machine perception models can capture spatial, social, and functional distinctions that are central to urban studies. The
benchmark supports three tasks within one standardized library: (T1) urban scene semantic classification, (T2) cross-modal image-text
retrieval, and (T3) instance
segmentation. Our experiments evaluate representative
vision,
vision-
language, and
segmentation models, revealing strong performance on supervised scene classification but more challenging behavior in cross-modal
retrieval and instance-level urban object
segmentation. A multi-scale study further examines how model performance changes as balanced training data increases from 1K, 10K to 100K images. Urban-ImageNet provides a unified, theory-grounded, multi-city
benchmark for evaluating how AI systems perceive and interpret contemporary urban spaces across modalities, scales, and task formulations.
Dataset and
benchmark are available at: huggingface.co/
datasets/Yiwei-Ou/Urban-ImageNet and github.com/yiasun/
dataset-2.
Read the full abstract ▾
All authors Yiwei Ou, Chung Ching Cheung, Jun Yang Ang, Xiaobin Ren, Ronggui Sun, Guansong Gao, Kaiqi Zhao, Manfredo Manfredini
Computer Vision
Evidence-based Decision Modeling for Synthetic Face Detection with Uncertainty-driven Active Learning
Qingchao Jiang, Zhenxuan Hou, Zhiying Zhu et al. (+3 more)
With the rapid development of deep generative models, forged facial images are massively exploited for illegal activities. Although existing synthetic face detection methods have achieved significant progress, they suffer from the inherent limitation of overconfidence due to their reliance on the Softmax activation function. Thus, these methods often lead to unreliable predictions when encountering unknown Out-of-Distribution (OOD) images, and cannot ascertain the model's uncertainty in its prediction. Meanwhile, most existing methods require massive high-quality annotated data, which greatly limits their practicability across diverse scenarios. To address these limitations, we propose EMSFD (Evidence-based decision Modeling for Synthetic Face Detection with uncertainty-driven active learning), an approach designed to enhance detection reliability and generalizability. Specifically, EMSFD models class evidence using the Dirichlet distribution and explicitly incorporates model uncertainty into the prediction process. Furthermore, during training, the estimated uncertainty is exploited to prioritize more informative samples from the unlabeled pool for annotation, thereby reducing labeling cost and improving model generalization. Extensive experimental evaluations demonstrate that our method enhances the interpretability of synthetic face detection. Meanwhile, our method yields a 15% increase in accuracy compared to existing state-of-the-art (SOTA) baselines, which demonstrates the superior detection performance and generalizability of our approach. Our code is available at: https://github.com/hzx111621/EMSFD.
Read the full abstract ▾
All authors Qingchao Jiang, Zhenxuan Hou, Zhiying Zhu, Zhenxing Qian, Xinpeng Zhang, Zaiwang Gu
Computer Vision
Frequency Adapter with SAM for Generalized Medical Image Segmentation
Phuoc-Nguyen Bui, Van-Nguyen Pham, Duc-Tai Le et al. (+2 more)
Medical image
segmentation is a critical task in computer-aided diagnosis and treatment planning. However, deep learning models often struggle to generalize across
datasets due to domain shifts arising from variations in imaging protocols, scanner types, and patient populations. Traditional domain generalization (DG) methods utilize causal feature learning,
adversarial consistency, and style augmentation to improve
segmentation robustness. While effective, these approaches rely on explicit feature
alignment,
adversarial objectives, or handcrafted augmentations, which may not fully exploit the capabilities of foundation models. Recently, the Segment Anything Model (SAM) has demonstrated strong generalization capabilities in
segmentation tasks. SAM-based DG methods attempt to improve medical image
segmentation. However, these approaches primarily operate in the spatial domain and overlook frequency-based discrepancies that significantly affect model robustness. In this work, we propose Frequency-based Domain Generalization with SAM (FSAM), a novel framework that integrates Low-Rank Adaptation (LoRA) for efficient fine-tuning and a frequency adapter to incorporate frequency-domain representations for single-source domain generalization. FSAM enhances SAM's
segmentation robustness by extracting domain-invariant high-frequency features, mitigating frequency-related domain shifts. Experimental results on fundus and prostate
datasets demonstrate that FSAM outperforms existing traditional DG and SAM-based DG approaches in domain generalization. Codes and pre-trained models will be made available on GitHub.
Read the full abstract ▾
All authors Phuoc-Nguyen Bui, Van-Nguyen Pham, Duc-Tai Le, Junghyun Bum, Hyunseung Choo
Computer Vision
TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models
Junzhe Chen, Siyuan Meng, Yuxi Chen et al. (+3 more)
Video large
language models (
Video-LLMs) have made strong progress in general
video understanding, but their ability to maintain temporal object consistency remains underexplored. Existing
benchmarks often emphasize event recognition, action understanding, or coarse temporal
reasoning, while rarely testing whether models can preserve the identity, state, and continuity of the same object across occlusion, disappearance, reappearance, state transitions, and cross-object interactions. We introduce TOC-Bench, a diagnostic
benchmark for evaluating temporal object consistency in
Video-LLMs. TOC-Bench is object-track grounded: each queried subject is linked to a per-frame trajectory and a structured temporal event timeline. To ensure that questions require temporally ordered visual evidence rather than
language priors, single-frame shortcuts, or unordered frame cues, we design a three-layer temporal-necessity filtering protocol, which removes 60.7% of candidate QA pairs and retains 17,900 temporally dependent items across 10 diagnostic dimensions. From this pool, we construct a human-verified
benchmark with 2,323 high-quality QA pairs over 1,951
videos. Experiments on representative
Video-LLMs show that temporal object consistency remains a major unsolved challenge, with notable weaknesses in event counting, event ordering, identity-sensitive
reasoning, and
hallucination-aware verification, even when models perform well on general
video understanding
benchmarks. These results suggest that object-centric temporal coherence is a key bottleneck for current
Video-LLMs, and that TOC-Bench provides a focused platform for diagnosing and improving object-aware temporal
reasoning. The resource is available at https://github.com/cjzcjz666/toc_bench.git.
Read the full abstract ▾
All authors Junzhe Chen, Siyuan Meng, Yuxi Chen, Man Zhao, Wenyao Gui, Xiaojie Guo
Computer Vision
Adversarial Attacks Against MLLMs via Progressive Resolution Processing and Adaptive Feature Alignment
Haobo Wang, Xiaorong Ma, Weiqi Luo et al. (+2 more)
Adversarial perturbations can mislead
Multimodal Large
Language Models (MLLMs) recognize a benign image as a specific target object, posing serious risks in safety-critical scenarios such as autonomous driving and medical diagnosis. This makes transfer-based targeted attacks crucial for understanding and improving black-box MLLM robustness. Existing transfer-based targeted attack methods typically rely on the final global features of the surrogate encoder and anchor optimization to original-resolution target crops, leading to their limited transferability and robustness. To address these challenges, we propose Progressive Resolution Processing and Adaptive Feature
Alignment (PRAF-Attack), a targeted transfer-based attack framework that integrates multi-scale global semantic guidance with robust intermediate-layer local
alignment. Unlike prior methods that align only the surrogate encoder's final layer, we design an adaptive feature
alignment strategy that leverages intermediate representations to enhance transferability. Specifically, we introduce an adaptive intermediate layer selection mechanism to identify transferable hierarchical features across surrogate ensembles via gradient consistency, along with an adaptive patch-level optimization strategy that preserves highly correlated local regions through efficient patch filtering. To overcome the reliance on fixed original-resolution target crops, we propose a progressive resolution processing strategy that gradually refines optimization from coarse to fine, enabling the attack to better exploit target information at multiple scales and achieve stronger transferability. We evaluate PRAF-Attack on a diverse suite of black-box MLLMs, including six open-source models and six closed-source commercial APIs. Compared with seven state-of-the-art targeted attack baselines, the proposed PRAF-Attack consistently achieves superior transferability.
Read the full abstract ▾
All authors Haobo Wang, Xiaorong Ma, Weiqi Luo, Xiaojun Jia, Jiwu Huang
Computer Vision · Cross-listed from cs.AI
Hao Liu and Jicheng Liu
A
vision-
language model can look at a
knot diagram and report what it sees, yet fail to act on that structure. KnotBench pairs an 858,318-image corpus from 1,951
prime-
knot prototypes (crossing numbers 3 to 19) with a protocol whose answers are checked against Regina's canonical
knot signature. Its 14 tasks span four families, equivalence judgment, move prediction, identification, and cross-modal grounding; an image-versus-symbol split locates failures along the perception-operation gap. We score Claude Opus 4.7 and GPT-5, each with and without thinking, under a 64K output-
token budget matched on both vendors. Across 56 (task, model) cases, 15 sit at or below a random baseline and 8 of 14 tasks have a best score under 1.5x random. On diagram-to-symbol transcription, no model produces a strictly correct string, and permissive Regina decoding recovers the
knot in 0 to 4 of 100 items. Thinking-mode
reasoning lifts overall accuracy by 1.65 points for Claude and 9.25 points for GPT-5, narrowing the gap only modestly. Read together, the four families suggest current
vision-
language models hold features of a diagram but lack apparatus to simulate moves on those features.
Read the full abstract ▾
Computer Vision
Hyperbolic Distillation: Geometry-Guided Cross-Modal Transfer for Robust 3D Object Detection
Kanglin Ning, Wenrui Li, Houde Quan et al. (+3 more)
Cross-modal knowledge
distillation has emerged as an effective strategy for integrating point cloud and image features in 3D perception tasks. However, the modality heterogeneity, spatial misalignment, and the representation crisis of multiple modalities often limit the efficient of these cross-modal
distillation methods. To address these limitations in existing approaches, we propose a
hyperbolic constrained cross-modal
distillation method for
multimodal 3D object detection (HGC-Det). The proposed HGC-Det framework includes an image branch and a point cloud branch to extract semantic features from two different modalities. The point cloud branch comprises three core components: a 2D semantic-guided voxel optimization component (SGVO), a
hyperbolic geometry constrained cross-modal feature transfer component (HFT), and a feature aggregation-based geometry optimization component (FAGO). Specifically, the SGVO component adaptively refines the spatial representation of the 3D branch by leveraging semantic cues from the image branch, thereby mitigating the issue of inadequate representation fusion. The HFT component exploits the intrinsic
geometric properties of
hyperbolic space to alleviate semantic loss during the fusion of high-dimensional image features and low-dimensional point cloud features. Finally, the FAGO compensates for potential spatial feature degradation introduced by the 2D semantic-guided voxel optimization component. Extensive experiments on indoor
datasets (SUN RGB-D, ARKitScenes) and outdoor
datasets (KITTI, nuScenes) demonstrate that our method achieves a better trade-off between detection accuracy and computational cost.
Read the full abstract ▾
All authors Kanglin Ning, Wenrui Li, Houde Quan, Qifan Li, Xingtao Wang, Xiaopeng Fan
Computer Vision
The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space
Xia Hu, Zhenrui Yue, Brian Potetz et al. (+4 more)
As current
Multimodal Large
Language Models rapidly saturate canonical visual
reasoning benchmarks, a key question emerges: do these strong scores genuinely reflect robust visual understanding? We identify a pervasive vulnerability, the Cartesian Shortcut: models frequently discretize the orthogonal grid-based layouts prevalent in visual
reasoning benchmarks into explicit textual coordinates, offloading
reasoning from visual perception to text-based deduction. To re-evaluate visual
reasoning when this shortcut is unavailable, we introduce Polaris-Bench, which re-formulates 53 visual
reasoning tasks in Polar coordinate space with paired Cartesian counterparts that preserve task semantics, disrupting the orthogonal structure that models exploit. Comprehensive evaluation across $14$ state-of-the-art MLLMs reveals that frontier models achieving $69$–$83\%$ on Cartesian layouts collapse to $31$–$39\%$ on Polar equivalents. Moreover, thinking gains largely vanish on Polar layouts, prompting interventions fail to close the gap, and comparable drops arise on other non-orthogonal layouts. These findings reveal that current MLLMs' visual
reasoning performance is strongly coupled to orthogonal grid structure, a fragility that is consistent across model families but far smaller in humans, who maintain 88.8% accuracy on Polar layouts. The
benchmark, evaluation suite, and leaderboard are publicly available at https://google-deepmind.github.io/polaris-bench.
Read the full abstract ▾
All authors Xia Hu, Zhenrui Yue, Brian Potetz, Howard Zhou, Leonidas Guibas, Chun-Ta Lu, Zhicheng Wang
Computer Vision
Ziyang Wang, Yue Zhang, Shoubin Yu et al. (+6 more)
Next-generation visual assistants, such as smart glasses,
embodied agents, and always-on life-logging systems, must reason over an entire day or more of continuous visual experience. In ultra-long
videos, relevant information is sparsely distributed across hours or days, making
memory a fundamental challenge: models must accumulate information over time, recall prior states, track temporal order, and abstract recurring patterns. However, existing week-long
video benchmarks are primarily designed for perception and recognition, such as moment localization or global summarization, rather than
reasoning that requires integrating evidence across multiple days. To address this gap, we introduce EgoMemReason, a comprehensive
benchmark for week-long
egocentric video understanding through
memory-driven
reasoning. EgoMemReason evaluates three complementary
memory types: entity
memory, tracking how object states evolve and change across days; event
memory, recalling and ordering activities separated by hours or days; and behavior
memory, abstracting recurring patterns from sparse, repeated observations over the whole week period. EgoMemReason comprises 500 questions across three
memory types and six core challenges, with an average of 5.1
video segments of evidence per question and 25.9 hours of
memory backtracking. We evaluate EgoMemReason on 17 methods across MLLMs and agentic frameworks, revealing that even the best model achieves only 39.6% overall accuracy. Further analysis shows that the three
memory types fail for distinct reasons and that performance degrades as evidence spans longer temporal horizons, revealing that long-horizon
memory remains far from solved. We believe EgoMemReason establishes a strong foundation for evaluating and advancing long-context,
memory-aware
multimodal systems.
Read the full abstract ▾
All authors Ziyang Wang, Yue Zhang, Shoubin Yu, Ce Zhang, Zengqi Zhao, Jaehong Yoon, Hyunji Lee, Gedas Bertasius, Mohit Bansal
Computer Vision · Cross-listed from cs.RO
ConsistNav: Closing the Action Consistency Gap in Zero-Shot Object Navigation with Semantic Executive Control
Haosen Wang, Zhenyang Li, Yinqiang Zhang et al. (+9 more)
Zero-shot object
navigation has advanced rapidly with open-vocabulary detectors, image–text models, and
language-guided exploration. However, even after current methods detect a plausible target hypothesis, the
agent may still oscillate between exploration and pursuit, or abandon the object near success. We identify this failure mode as an action consistency gap: semantic evidence is repeatedly reinterpreted at each step without persistent commitment across the episode. We introduce ConsistNav, a training-free zero-shot ObjectNav framework built around a semantic executive composed of three coordinated modules: Finite-State Executive Controller stages target pursuit through guarded semantic phases; Persistent Candidate
Memory accumulates cross-frame target evidence into stable object hypotheses; and Stability-Aware Action Control suppresses rotational stagnation, ineffective pursuit, and unverified stopping. This design changes neither the detector nor the low-level planner; instead, it controls when semantic evidence should influence
navigation and when it should be suppressed or revisited. We conduct extensive experiments on HM3D and MP3D, where ConsistNav achieves state-of-the-art results among compared zero-shot ObjectNav methods and improves SR by 11.4% and SPL by 7.9% over the controlled baseline on MP3D. Ablation studies and real-world deployment experiments further demonstrate the effectiveness and robustness of the proposed executive mechanism.
Read the full abstract ▾
All authors Haosen Wang, Zhenyang Li, Yinqiang Zhang, Zongqi He, Lutao Jiang, Kai Li, Yizhou Zhao, Liaoyuan Fan, Wenjian Hou, Tingbang Liang, Yibin Wen, Defeng Gu
Computer Vision
DA-SegFormer: Damage-Aware Semantic Segmentation for Fine-Grained Disaster Assessment
Kevin Zhu, William Tang, Raphael Hay Tene et al. (+3 more)
Rapid and accurate damage assessment following natural disasters is critical for effective emergency response. However, identifying fine-grained damage levels (e.g., distinguishing minor from major roof damage) in
UAV imagery remains challenging due to the degradation of texture cues during resizing and extreme class imbalance. We propose DA-SegFormer, a damage-aware adaptation of the SegFormer architecture optimized for high-resolution disaster imagery. Our method introduces a Class-Aware Sampling strategy to guarantee exposure to rare damage features, and it integrates Online Hard Example Mining (OHEM) with Dice Loss to dynamically focus on underrepresented classes. In addition, we employ a resolution-preserving inference protocol that maintains native texture details. Evaluated on the RescueNet
dataset, DA-SegFormer achieves 74.61% mIoU, outperforming the baseline by 2.55%. Notably, our improvements yield double-digit gains in critical damage classes: Minor Damage (+11.7%) and Major Damage (+21.3%).
Read the full abstract ▾
All authors Kevin Zhu, William Tang, Raphael Hay Tene, Zesheng Liu, Nhut Le, Maryam Rahnemoonfar