Computer Vision · Cross-listed from cs.LG
RADAR: Relative Angular Divergence Across Representations
Xavier Cadet, Mateusz Nowak and Peter Chin
Machine learning methods rely on data. However, gathering suitable data can be challenging due to availability constraints, cost, or the need for domain expertise. Expanding
datasets with additional sources is a common response to limited data, yet this practice does not always improve downstream performance and can sometimes lead to a loss of performance, known as negative transfer. We propose RADAR, a simple, geometrically grounded metric for estimating cross-domain transferability in foundation models. RADAR analyzes the layer-wise
evolution of representations by measuring angular
alignments and relative changes in distance along layer-to-layer displacement trajectories, and by comparing empirical distributions of within-domain and cross-domain dynamics. We hypothesize that domain transferability is related to the divergence between these trajectory distributions. We evaluate the metric across multiple modalities, including cross-lingual sentiment classification with text
embedding models and cross-domain image classification with foundation
vision models. Across several settings, RADAR provides competitive predictive performance relative to existing transferability metrics on several
vision and text
benchmarks, with particularly strong results when domain transitions are smooth or cleanly separated. Our ablations further suggest that the effectiveness of transferability estimation depends on the geometry of the model's internal representation space, with different modalities favoring different topological formulations.
Read the full abstract ▾
Computer Vision
Scene Reconstruction as Mapping Priors for 3D Detection
Yang Fu, Yuliang Zou, Hao Xiang et al. (+8 more)
In autonomous driving, mapping is critical for motion planning but remains an under-utilized resource for perception tasks such as 3D object detection. Maps can provide robust structural priors of the static environment, helping resolve ambiguities and correct for sensor data sparsity or noise, especially for distant objects or under adverse weather conditions. However, conventional High-Definition (HD) maps are resource-intensive to obtain and maintain, which presents a challenge for efficient, large-scale deployment. In this paper, we propose a scalable solution to systematically leverage mapping to improve 3D detection by overcoming two primary challenges. First, we introduce a pipeline to automatically build dense mapping priors from aggregated sensor data, eliminating the need for human labeling. Second, we design a novel Mapping Priors Augmented 3D Detection (MPA3D) framework to effectively integrate mapping priors with different sensor modalities. Extensive experiments on the Waymo Open
Dataset demonstrate that our approach achieves new state-of-the-art results, proving the effectiveness of scalable reconstructed scene priors for enhancing 3D detection.
Read the full abstract ▾
All authors Yang Fu, Yuliang Zou, Hao Xiang, Xin Huang, Yijing Bai, Chen Song, Weijing Shi, Govind Thattai, Dragomir Anguelov, Mingxing Tan, Yingwei Li
Computer Vision
CoMoGen: COntrollable MOtion Dynamics and Interactions with Mask-Guided Video GENeration
Adil Meric, Lin Geng Foo, Mert Kiray et al. (+3 more)
We present CoMoGen, a controllable
video generation framework that generates realistic interactive dynamics from a single binary mask sequence conditioned on an input image. CoMoGen introduces a lightweight MaskAdapter that encodes binary mask sequences into a latent residual signal, injected into the Multi Modal
Diffusion Transformer (MMDiT) model through a cosine-weighted schedule. Unlike the hierarchical coarse-to-fine design of UNet architectures, MMDiT operates as a sequence of uniform
transformer blocks, making it difficult to identify which layers are responsible for the motion generation. Therefore, we propose a novel way to determine "Motion Layers" operating in the
attention space of MMDiT. We fine-tune the model by using Low-Rank Adaptation (LoRA) to the Motion Layers, without requiring any architecture change in the MMDiT. This selective adaptation enables our method to focus on motion-critical components, yielding reduced computational cost. Despite its simplicity, CoMoGen enables precise subject motion and plausible interactions with surrounding humans, objects, and scenes. Comprehensive experiments on different
datasets show that CoMoGen consistently outperforms prior controllable
video generation methods and achieves state-of-the-art performance in motion fidelity and perceptual realism. Project page: mericadil.github.io/CoMoGen.
Read the full abstract ▾
All authors Adil Meric, Lin Geng Foo, Mert Kiray, Benjamin Busam, Rishabh Dabral, Christian Theobalt
Computer Vision
GazeBehavior Annotation Toolkit (GBAT): AI-powered toolkit for automatic annotation of egocentric eye-tracking and video data of child-caregiver interaction
Iba Baig, Kevin Li, Yanbin Xu et al. (+5 more)
Video recordings of child-caregiver interactions enable investigation of attentional dynamics during naturalistic behavior. Such
multimodal recording also allows researchers to examine how
attention interacts with action and
language use in real time. However, manual annotation of such data is time-consuming. Here, we introduce GazeBehavior Annotation Toolkit, a deep-learning-based toolkit designed to facilitate three key processes in data preprocessing and feature extraction: post-hoc synchronization across multiple
videos, semi-automatic annotation of gaze target categories, and categorization of participants' poses and hand actions. This toolkit improves the efficiency and scalability of feature extraction from human
egocentric eye-tracking and
video data. Such improvement is critical in supporting large-scale and longitudinal investigations of attentional dynamics and naturalistic behavior in human early development.
Read the full abstract ▾
All authors Iba Baig, Kevin Li, Yanbin Xu, Seiji Cattelain, Marie Hallo, Hayato Ono, Sho Tsuji, Ming Bo Cai
Computer Vision
Improved Vision-to-Chart Buoy Association with Learned World-to-Image Projection
Borja Carrillo-Perez
This report presents a lightweight modification to the DETR-based fusion
transformer baseline for the MaCVi 2026
Vision-to-Chart data association challenge. The challenge baseline decoder receives per-buoy queries encoding world-space distance and bearing, forcing the
transformer to implicitly learn the complex
geometric projection from world coordinates to image pixels. Instead, this work trains an additional dedicated MLP, QueryMLP, to explicitly predict the buoy's waterline contact point in the image from chart measurements and IMU orientation data. The predicted pixel coordinates are appended to the baseline decoder query vector, providing a direct spatial prior per buoy and reducing the
geometric reasoning burden on the
transformer decoder. On the challenge leaderboard, the presented approach achieves an Overall score of 0.7386, with F1 = 0.8055 and mIoU = 0.6718, on the held-out test set, placing second among all submissions.
Read the full abstract ▾
Computer Vision
VideoOdyssey: A Benchmark for Ultra-Long-Context and Omni-Modal Video Understanding
Haichen He, Jiayi Zhou, Sifeng Shang et al. (+3 more)
Real-world long
video understanding requires models to perform continuous tracking, information integration and
memory retention over massive temporal spans within extreme
video durations. Mastering this intense cognitive load constitutes the fundamental bottleneck in long
video understanding. While existing
benchmarks have driven progress by scaling up
video duration, their evaluation tasks often require comprehending only short and isolated
video segments, falling short of capturing the challenge of ultra-long-context
reasoning. To measure this cognitive load, we emphasize continuous certificate length, defined as the
video length a human must continuously watch to definitively answer a given question. Driven by this metric, we introduce VideoOdyssey, a
benchmark specifically designed for ultra-long-context and omni-modal
video understanding. VideoOdyssey is characterized by three key features: 1) Extreme
video duration and diversity: spanning 11 domains and 54 subcategories with an average
video duration of 109 minutes; 2) Comprehensive evaluation scenarios: offering two subsets to address different research focuses, i.e., VideoOdyssey-V for probing the limits of visual understanding in MLLMs, and VideoOdyssey-AV for evaluating synchronized audio-visual understanding for omni-modal models; 3) Ultra-long and multi-level continuous certificates: extending the average continuous certificate to 16 minutes for VideoOdyssey-V and 12.8 minutes for VideoOdyssey-AV. Crucially, we design 5 granular levels from seconds to hours, providing a comprehensive diagnostic tool to evaluate models across varying context lengths and cognitive loads. Extensive evaluations show that bottlenecks of current MLLMs extend beyond simple
retrieval to include struggles with continuous
reasoning across varying context lengths, fine-grained perception, and non-verbal omni-modal understanding.
Read the full abstract ▾
All authors Haichen He, Jiayi Zhou, Sifeng Shang, Yihan Hu, Yuanhan Zhang, Kaiyang Zhou
Computer Vision
Suicide Risk Assessment from AI-powered Video Surveillance: An Interpretable Framework for Prevention in Metro Stations
Safwen Naimi, Wassim Bouachir, Guillaume-Alexandre Bilodeau et al. (+1 more)
Understanding and monitoring human behavior in metro stations play an important role in supporting suicide prevention efforts, where early identification of high-risk situations can enable timely intervention. This requires assessing suicide risk from a surveillance
video by jointly
reasoning about the behavior of each passenger, his/her spatial context, and temporal dynamics. However, this assessment using
videos captured by surveillance cameras is challenging, as it demands accurate perception of human motion, understanding of platform geometry, and aggregation of heterogeneous behavioral cues over time. In this work, we formalize the task of Suicide Risk Assessment (SRA) in metro stations and introduce the first interpretable framework that addresses this challenge. Unlike approaches that focus on isolated subtasks or attempt to infer intent directly, our formulation assesses suicide risk from accumulated evidence by incorporating person tracking, activity recognition, semantic
segmentation of the platform, and trajectory-driven risk heatmap modeling. By formalizing SRA as a distinct task and benchmarking a complete operational pipeline achieving 83.2% ROC-AUC on real surveillance data, this work highlights the complexity of suicide risk assessment and opens new directions for research on interpretable AI systems for social good.
Read the full abstract ▾
All authors Safwen Naimi, Wassim Bouachir, Guillaume-Alexandre Bilodeau, Brian Mishara
Computer Vision
Zixuan Lan, Luzhe Sun, Matthew R. Walter et al. (+1 more)
Benchmark accuracy is often implicitly assumed to reflect grounded visual understanding in
vision-
language models (VLMs), yet it remains unclear to what extent such scores truly reflect reliance on visual evidence. Motivated by a surprising observation that removing a substantial fraction of image
tokens only degrades model performance very slightly on a widely used
hallucination benchmark, we systematically investigate this mismatch in a set of open-source VLMs. Our analysis spans multiple levels of granularity, spanning global visual degradation, localized occlusion, question reformulation, answer-space expansion, and decision-level analyses beyond standard accuracy. We further complement these behavioral results with a layer-wise analysis of
vision-
token geometry. Throughout the experiments, we find that although VLMs do incorporate visual input, their predictions are less sensitive to the loss of fine-grained visual evidence that standard accuracy should have suggested. Even when the final prediction remains unchanged, the model's internal support for the correct answer may already be weakened. We further complement a representation-level analysis, which shows increasing similarity among visual
tokens in deeper layers, providing a possible explanation for our findings. Together, these results suggest that current
benchmarks are not sufficient to reliably evaluate fine-grained visual grounding in VLMs.
Read the full abstract ▾
All authors Zixuan Lan, Luzhe Sun, Matthew R. Walter, Jiawei Zhou
Computer Vision · Cross-listed from cs.RO
Extending Deep Event Visual Odometry with Sparse Point-Cloud Export
Alireza Safdari and Sajad Ashraf
Event cameras are well suited for visual odometry under high-speed motion and challenging lighting conditions due to their low latency, high temporal resolution, and high dynamic range. Deep Event Visual Odometry (DEVO) demonstrated that
monocular event-only odometry can achieve strong performance by combining sparse patch tracking, learned patch selection, recurrent correspondence refinement, and differentiable bundle adjustment. In this project, we extend DEVO with a sparse point-cloud export pipeline. Rather than modifying the core odometry formulation, our approach exposes the internal 3D structure already estimated by DEVO and converts it into an explicit point-cloud representation for visualization and further processing. In addition, we implement a practical workflow for data export, format conversion, and point-cloud cleanup. The resulting system preserves the original visual odometry pipeline while enabling sparse
geometric scene output. Experiments on the BOARD SLOW sequence show that the exported sparse cloud is locally consistent with EMVS reconstructions, achieving high precision at a 5 cm threshold, while also highlighting the expected limitations in density, completeness, and sensitivity to accumulated odometry noise.
Read the full abstract ▾
Computer Vision
Which Way Did It Move? Diagnosing and Overcoming Directional Motion Blindness in Video-LLMs
Jongseo Lee, Hyuntak Lee, Sunghun Kim et al. (+3 more)
Video Large
Language Models (
Video-LLMs) have made rapid progress on temporal
video understanding, yet many fail at a basic perceptual primitive: signed image-plane motion direction. On simple
videos of a single object moving left, right, up, or down, most
Video-LLMs perform near chance, with above-chance cases largely attributable to prediction biases rather than genuine direction understanding. We call this failure directional motion blindness. We localize the failure by tracing motion direction information through the
Video-LLM pipeline. Motion direction remains linearly accessible from the
vision encoder, projector, and LLM hidden states, but the readout fails to bind this signal to the correct verbal answer option, revealing a direction binding gap. Although synthetic motion direction instruction tuning reduces this gap on the source domain, motion direction concept vector analysis shows that visual complexity weakens the signal magnitude and limits out-of-domain generalization. We introduce MoDirect, a
dataset family for motion direction instruction tuning and evaluation, and DeltaDirect, a diagnosis-driven, projector-level objective that predicts normalized 2-D motion vectors from adjacent-frame feature deltas. On MoDirect-SynBench, instruction tuning with DeltaDirect improves motion direction accuracy from 25.9% to 85.9%. On MODIRECT-REALBENCH, DeltaDirect improves realworld motion direction accuracy by 21.4 points over the vanilla baseline without real-world tuning data, while preserving standard
video-understanding performance. Our project page is available at https://jong980812.github.io/which-way-did-it-move/
Read the full abstract ▾
All authors Jongseo Lee, Hyuntak Lee, Sunghun Kim, Sooa Kim, Jihoon Chung, Jinwoo Choi
Computer Vision
Cambrian-P: Pose-Grounded Video Understanding
Jihan Yang, Zifan Zhao, Xichen Pan et al. (+6 more)
Camera pose matters. The position and orientation of each viewpoint define a shared spatial coordinate frame that relates observations across
video frames. Yet this signal is largely absent from
multimodal LLMs (MLLMs) for
video understanding, which process frames as isolated 2D snapshots, instead of the persistent scene humans perceive. We revisit pose as a lightweight supervisory signal and introduce Cambrian-P, a
video MLLM augmented with per-frame learnable camera
tokens and a pose regression head. With a carefully designed sampling scheme, the model achieves substantial gains of 4.5-6.5% on spatial
reasoning benchmarks such as VSI-Bench, generalizes across eight additional spatial and general
video QA
benchmarks, and, as a byproduct, achieves state of the art streaming pose estimation on ScanNet. Surprisingly, training on pseudo-annotated poses from in-the-wild
video further improves general
video QA
benchmarks, showing pose helps beyond spatial
reasoning. Together, these results position camera pose as a fundamental signal for
video models that reason about the physical world.
Read the full abstract ▾
All authors Jihan Yang, Zifan Zhao, Xichen Pan, Shusheng Yang, Junyi Zhang, Bingyi Kang, Hu Xu, Shang-Wen Li, Saining Xie
Computer Vision
MotiMotion: Motion-Controlled Video Generation with Visual Reasoning
Lee Hsin-Ying, Hanwen Jiang, Yiqun Mei et al. (+3 more)
Current motion-controlled image-to-
video generation models rigidly follow user-provided trajectories that are often sparse, imprecise, and causally incomplete. Such reliance often yields unnatural or implausible outcomes, especially by missing secondary causal consequences. To address this, we introduce MotiMotion, a novel framework that reformulates motion control as a
reasoning-then-generation problem. To encourage causally grounded and commonsense-consistent interactions, we leverage a training-free
vision-
language reasoner to refine image-space coordinates of primary trajectories and to hallucinate plausible secondary motions. To further improve motion naturalness, we propose a confidence-aware control scheme that modulates guidance strength, enabling the model to closely follow high-confidence plans while correcting artifacts under low-confidence inputs with its internal generative priors. To support systematic evaluation, we curate a new image-to-
video benchmark, MotiBench, consisting of interaction-centric scenes where new events are triggered by motion. Both VLM-based evaluation and a human study on MotiBench demonstrate that MotiMotion produces
videos with more plausible object behaviors and interaction, and is preferred over existing approaches.
Read the full abstract ▾
All authors Lee Hsin-Ying, Hanwen Jiang, Yiqun Mei, Jing Shi, Ming-Hsuan Yang, Zhixin Shu
Computer Vision · Cross-listed from cs.RO
Wenxuan Guo, Xiuwei Xu, Yichen Liu et al. (+7 more)
Vision-and-
Language Navigation (VLN) requires an
agent to ground
language instructions to its own movement within a visual environment. While state-of-the-art methods leverage the
reasoning capabilities of
Vision-
Language Models (VLMs) for end-to-end action prediction, they often lack an explicit and explainable understanding of the relationships between the
agent, the instruction, and the scene. Conversely, explicitly building a scene map for heuristic planning is intuitively appealing but relies on additional 3D sensors and hinders large-scale
vision-
language pre-training. To bridge this gap, we propose AwareVLN, a novel framework that equips the
navigation model with a self-aware
reasoning mechanism, enabling it to understand the
agent's state and task progress in a fully end-to-end and data-driven manner. Our approach features two key innovations: (1) a structural
reasoning module that fosters spatial and task-oriented self-awareness, and (2) an automatic data engine with progress division for effective training. Extensive experiments on various
datasets in Habitat simulator show our AwareVLN significantly outperforms previous state-of-the-art
vision-
language navigation methods. Project page: https://gwxuan.github.io/AwareVLN/.
Read the full abstract ▾
All authors Wenxuan Guo, Xiuwei Xu, Yichen Liu, Xiangyu Li, Hang Yin, Huangxing Chen, Wenzhao Zheng, Jianjiang Feng, Jie Zhou, Jiwen Lu
Computer Vision · Cross-listed from cs.RO
GesVLA: Gesture-Aware Vision-Language-Action Model Embedded Representations
Wenxuan Guo, Ziyuan Li, Meng Zhang et al. (+7 more)
Vision-
Language-Action (VLA) models have shown strong potential for general-purpose
robot manipulation by unifying perception and action. However, existing VLA systems primarily rely on textual instructions and struggle to resolve spatial ambiguity in complex scenes with multiple similar objects. To address this limitation, we introduce gesture as a parallel instruction modality and propose a Gesture-aware
Vision-
Language-Action model (GesVLA). Our approach encodes gesture features directly into the latent space, enabling them to participate in both high-level
reasoning and low-level action generation, and adopts a dual-VLM architecture to achieve tight coupling between gesture representations and action policies. At the data level, we construct a scalable gesture data generation pipeline by rendering hand models onto real-world scene images. This reduces the sim-to-real visual gap while producing rich data with diverse motion patterns and corresponding pointing annotations. In addition, we employ a two-stage training strategy to equip the model with both gesture perception and action prediction capabilities. We evaluate our approach on multiple real-world robotic tasks, including a controlled block manipulation task for validation and more practical scenarios such as product and produce selection. Experimental results show that incorporating gesture consistently improves target grounding accuracy and human-
robot interaction efficiency, especially in complex and cluttered environments. Project page: https://gwxuan.github.io/GesVLA/.
Read the full abstract ▾
All authors Wenxuan Guo, Ziyuan Li, Meng Zhang, Yichen Liu, Yimeng Dong, Chuxi Xu, Yunfei Wei, Ze Chen, Erjin Zhou, Jianjiang Feng
Computer Vision
Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Driving
Jiahao Wang, Bo Sun, Yijing Bai et al. (+12 more)
Robust training and validation of Autonomous Driving Systems (ADS) require massive, diverse
datasets. Proprietary data collected by Autonomous Vehicle (AV) fleets, while high-fidelity, are limited in scale, diversity of sensor configurations, as well as geographic and long-tail-behavioral coverage. In contrast, in-the-wild data from sources like dashcams offers immense scale and diversity, capturing critical long-tail scenarios and novel environments. However, this unstructured, in-the-wild
video data is incompatible with ADS expecting structured, multi-modal sensor inputs for validation and training. To bridge this data gap, we propose Sensor2Sensor, a novel generative modeling paradigm that translates in-the-wild
monocular dashcam
videos into a high-fidelity, multi-modal sensor suite (AV logs) comprising multi-view camera images and
LiDAR point clouds. A core challenge is the lack of paired training data. We address this by converting real AV logs into dashcam-style
videos via 4D Gaussian Splatting (4DGS) reconstruction and novel-view rendering. Sensor2Sensor then utilizes a
diffusion architecture to perform the generative conversion. We perform comprehensive quantitative evaluations on the fidelity and realism of the generated sensor data. We demonstrate Sensor2Sensor's practical utility by converting challenging in-the-wild internet and dashcam footage into realistic, multi-modal data formats, further unlocking vast external data sources for AV development.
Read the full abstract ▾
All authors Jiahao Wang, Bo Sun, Yijing Bai, Vincent Casser, Songyou Peng, Zehao Zhu, Meng-Li Shih, Xander Masotto, Shih-Yang Su, Kanaad V Parvate, Tiancheng Ge, Linn Bieske, Dragomir Anguelov, Mingxing Tan, Chiyu Max Jiang
Computer Vision
DecQ: Detail-Condensing Queries for Enhanced Reconstruction and Generation in Representation Autoencoders
Tianhang Wang, Yitong Chen, Wei Song et al. (+3 more)
Representation Autoencoders (RAEs) leverage frozen
vision foundation models (VFMs) as tokenizer encoders, providing robust high-level representations that facilitate fast convergence and high-quality generation in latent
diffusion models. However, freezing the VFM inherently constrains its spatial reconstruction capacity, limiting fine-grained generation and image editing; in contrast, incorporating reconstruction-oriented signals via fine-tuning disrupts the pretrained semantic space and degrades generative fidelity. To address this trade-off, we propose DecQ, a simple yet effective framework for RAEs. Specifically, DecQ introduces lightweight detail-condensing queries that extract fine-grained information from intermediate VFM features through condenser modules. These queries are incorporated into the decoder to support reconstruction and are jointly generated with patch
tokens during generative modeling. By aggregating information from both shallow and deep layers, DecQ effectively mitigates the reconstruction–generation trade-off, improving both reconstruction quality and generative performance. Our experiments demonstrate that: (1) with only 8 additional queries and 3.9% extra computation, DecQ improves reconstruction over the frozen DINOv2-based RAE, increasing PSNR from 19.13 dB to 22.76 dB; and (2) for generative modeling, DecQ achieves 3.3$\times$ faster convergence than RAE, attaining an FID of 1.41 without guidance and 1.05 with guidance.
Read the full abstract ▾
All authors Tianhang Wang, Yitong Chen, Wei Song, Zuxuan Wu, Min Li, Jiaqi Wang
Computer Vision
Synthetic Data Alone is Enough? Rethinking Data Scarcity in Pediatric Rare Disease Recognition
Ganlin Feng, Yuxi Long, Erin Lou et al. (+4 more)
Children with rare genetic diseases often exhibit distinctive facial phenotypes, yet developing computer
vision systems for early diagnosis remains challenging due to extreme data scarcity,
privacy constraints, and limited data sharing in pediatric settings. These challenges not only hinder automated diagnosis but also restrict the availability of visual resources for clinical genetic counseling. While prior work has shown that synthetic data can augment real
datasets and preserve phenotype-level semantics, it remains unclear whether synthetic data alone is sufficient for learning in ultra-low-resource pediatric settings. In this work, we study the synthetic-only regime for pediatric rare disease recognition. Under a controlled experimental setup, models are trained exclusively on phenotype-aware synthetic facial images at increasing scales. We find that synthetic-only training achieves performance comparable to real-data-only baselines at sufficient scale across multiple backbones, suggesting that high-fidelity synthetic data can approximate clinically meaningful distributions. These findings together further enable the use of synthetic pediatric facial images as
privacy-preserving resources for genetic education and counseling, supporting clinician training and patient communication. Our results highlight the potential of computer
vision to improve data efficiency and expand accessible visual tools in children's healthcare.
Read the full abstract ▾
All authors Ganlin Feng, Yuxi Long, Erin Lou, Lianghong Chen, Zihao Jing, Pingzhao Hu, Wei Xu
Computer Vision
Spectral Tail Auxiliary Learning for AI-Generated Image Detection
Xingyi Li, Jiahui Zhang, Yiheng Li et al. (+2 more)
As generative image models evolve rapidly, the perceptual gap between generated and real images continues to narrow, making AI-generated image detection increasingly challenging. Many existing methods exploit frequency-domain cues for detection, typically described as frequency-domain artifacts or high-frequency discrepancies. However, the specific and recurring spectral regularities remain insufficiently understood and characterized. In this paper, we systematically analyze the one-dimensional radial log-power spectra of real and generated images. We find that generated images do not necessarily exhibit higher or lower energy across the entire spectrum or high-band range. Instead, their spectra deviate from the power-law decay and show an anomalous uplift in the ultra-high-frequency tail. We term this phenomenon spectral tail uplift. We further attribute this phenomenon to nonlinear harmonic accumulation in trained generative models, suggesting that it can serve as a structural cue across generative architectures. Based on this observation, we propose Spectral Tail Auxiliary Learning (STAL), a frequency-domain auxiliary supervision framework for generalizable AI-generated image detection. STAL transfers spectral-tail cues from a tail-aware frequency teacher to a spatial detector during training, while all frequency-domain modules are discarded at inference time. Consequently, STAL introduces no inference overhead. Extensive experiments on 9 public
datasets show that STAL achieves strong generalization and stability across generators, data distributions, and real-world scenarios.
Read the full abstract ▾
All authors Xingyi Li, Jiahui Zhang, Yiheng Li, Yun Cao, Wenhao Wang
Computer Vision
WorldKV: Efficient World Memory with World Retrieval and Compression
Jung Yi, Minjae Kim, Paul Hyunbin Cho et al. (+3 more)
Autoregressive
video diffusion models have enabled real-time, action-conditioned world generation. However, sustaining a persistent world, where revisiting a previously seen viewpoint yields consistent content, remains an open problem. Full KV-cache
attention preserves this consistency but breaks real-time constraints:
memory footprint and
attention cost grow linearly with rollout length. Sliding window inference restores throughput but discards long-term consistency. We propose WorldKV, a training-free framework with two components: World
Retrieval and World Compression. World
Retrieval stores evicted KV-cache chunks in GPU/CPU
memory and selectively retrieves scene-relevant chunks via camera/ action correspondence, inserting them back into the native
attention window without re-encoding. World Compression prunes redundant
tokens within each chunk via key-key similarity to an anchor frame, halving per-chunk storage to fit 2x more history under a fixed budget. On
Matrix-Game-2.0 and LingBot- World-Fast, WorldKV matches or exceeds full-KV
memory fidelity at roughly 2x the throughput, and is competitive with
memory-trained baselines without any fine-tuning. Project Page: https://cvlab-kaist.github.io/WorldKV/
Read the full abstract ▾
All authors Jung Yi, Minjae Kim, Paul Hyunbin Cho, Wooseok Jang, Sangdoo Yun, Seungryong Kim
Computer Vision
AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild
Baiyu Chen, Zechen Li, Wilson Wongso et al. (+5 more)
As wearable and mobile devices become increasingly embedded in daily life, they offer a practical way to continuously sense human motion in the wild. But inertial signals are highly dependent on the sensing setup, including body location, mounting position, sensor orientation, device hardware, and sampling protocol. This setup dependence makes it difficult to learn motion representations that transfer across devices and
datasets, and limits the broader use of wearable IMUs beyond closed-set recognition. We introduce AnyMo, a geometry-aware framework for setup-agnostic human motion modeling. AnyMo uses physics-grounded IMU simulation over dense body-surface placements to generate diverse and plausible synthetic signals, pre-trains a graph encoder from paired synthetic placement views and masked partial observations, tokenizes multi-position IMU into full-body motion
tokens, and aligns these
tokens with an LLM for motion-
language understanding. We evaluate AnyMo on three complementary tasks: zero-shot activity recognition across 14 unseen downstream
datasets, cross-modal
retrieval, and wearable IMU motion captioning, where it improves average Accuracy/F1/R@2 by 11.7%/11.6%/22.6% on HAR, increases zero-shot IMU-to-text and text-to-IMU
retrieval MRR by 15.9% and 28.6%, respectively, and improves zero-shot captioning BERT-F1 by 18.8%. These results support AnyMo as a generalist model for wearable motion understanding in the wild. Project page: https://baiyuchen.com/project/AnyMo.
Read the full abstract ▾
All authors Baiyu Chen, Zechen Li, Wilson Wongso, Lihuan Li, Xiachong Lin, Hao Xue, Benjamin Tag, Flora Salim
Computer Vision
Cross-Domain Human Action Recognition from Multiview Motion and Textual Descriptions
Yannick Porto, Renato Martins, Thomas Chalumeau et al. (+1 more)
Robustness to domain changes is a key capability for effective deployment of human action recognition systems in real-world scenarios, where action categories at inference can present important domain shifts or even unseen actions from training. In this context, improving the recognition capabilities of Zero-Shot Action Recognition models (ZSAR), without requiring strong annotation efforts, remains a central challenge. Most ZSAR approaches assume that actions are observed under
geometric conditions similar to those seen during training. In practice, variations in human body orientation and camera viewpoint add a significant domain gap in ZSAR, substantially limiting generalization to novel action-motion combinations. In this context, this paper presents a novel orientation-aware action recognition approach with improved cross-domain capabilities. Our approach combines motion cues of multiple camera viewpoints and text descriptions of human actions in the training phase. We present a new orientation-aware motion encoding network to learn different motion features, and adapt a specific orientation-aware text
prompt to match the corresponding features at inference. Extensive experiments demonstrate that the proposed method consistently improves ZSAR performance across different recognition
benchmarks, outperforming recent state-of-the-art zero-shot approaches on NTU-RGB+D, BABEL, NW-UCLA, and on two surveillance
datasets. In addition, the learned representations exhibit strong transfer learning capabilities, yielding competitive performance on both cross-domain and same-domain recognition of seen actions. Code and trained models are available at: https://icb-
vision-ai.github.io/OrientationAware-HAR
Read the full abstract ▾
All authors Yannick Porto, Renato Martins, Thomas Chalumeau, Cedric Demonceaux
Computer Vision
Improving Viewpoint-Invariance and Temporal Consistency for Action Detection
Yannick Porto, Renato Martins, Thomas Chalumeau et al. (+1 more)
Viewpoint change invariance and action temporal consistency are critical aspects for the effective deployment of human action detection of untrimmed
videos. Existing appearance-based
video detection methods often struggle with limited viewpoint diversity during training, while motion-based detection approaches frequently fail to model fine-grained temporal relationships across consecutive motion windows. This paper introduces a novel two-stage action detection approach designed to improve both view-invariance and global temporal coherence properties. In the first stage, we extract motion features from augmented virtual viewpoints, solely used at training. Then, the second stage introduces a new view-invariant, multi-scale temporal encoder based on selective state-space sequence modelling to aggregate information across viewpoints and time scales. Experiments on PKU-MMD and BABEL
benchmarks demonstrate that this approach significantly outperforms state-of-the-art methods in all considered splits. Code and trained models are available at: https://icb-
vision-ai.github.io/HydraView-TAD
Read the full abstract ▾
All authors Yannick Porto, Renato Martins, Thomas Chalumeau, Cedric Demonceaux
Computer Vision
Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models
Piotr Kubaty, Patryk Marszałek, Łukasz Struski et al. (+3 more)
Vision-
language models learn powerful
multimodal embeddings, yet their internal semantics remain opaque. While sparse autoencoders (SAEs) can extract interpretable features, they rely on expanding the representation dimension, which compromises the original geometry and introduces redundancy. We introduce CEDAR (Conceptual
Embedding Disentanglement via Adaptive Rotation), a post-hoc method that reveals the compositional structure of pretrained
embeddings without increasing dimensionality. By learning an invertible transformation with a top-$k$ sparsity bottleneck, CEDAR concentrates semantic information into axis-aligned disentangled coordinates. In CLIP-like architecture, individual coordinates can be interpreted with textual concepts, while for generative models such as BLIP, they can be decoded into natural
language descriptions. Experiments demonstrate that CEDAR achieves a competitive reconstruction-sparsity trade-off while producing explanations that are more interpretable and better aligned with human perception. Our results suggest that the apparent
entanglement in
vision-
language representations can be resolved through a suitable change of basis, eliminating the need for overcomplete expansions.
Read the full abstract ▾
All authors Piotr Kubaty, Patryk Marszałek, Łukasz Struski, Adam Wróbel, Jacek Tabor, Marek Śmieja
Computer Vision
Swift Sampling: Selecting Temporal Surprises via Taylor Series
Dahye Kim, Bhuvan Sachdeva, Karan Uppal et al. (+3 more)
While most frames in long-form
video are redundant, the critical information resides in temporal surprises: moments where the actual visual features deviate from their predicted
evolution. Inspired by the human
brain's predictive coding, we introduce Swift Sampling, an elegant, training-free frame selection
algorithm that automatically identifies high-information moments in a
video. Specifically, we model a
video as a differentiable trajectory in the visual latent space and compute the velocity and acceleration of its features. Then, we apply Taylor expansion to project the expected path of subsequent frames. Frames that diverge sharply from this predicted
manifold are identified as temporally surprising frames and selected for sampling. Unlike prior training-free methods that rely on auxiliary networks or
video-specific hyperparameter tuning, Swift Sampling is incredibly lightweight, adding only 0.02x additional computational cost over baseline making it 30x cheaper overhead than leading baselines. Across three long-
video question answering
benchmarks and 10 different downstream tasks, Swift Sampling outperforms uniform sampling and prior query-agnostic baselines. It is especially powerful for long
videos with limited frame budgets improving accuracy by up to +12.5 points.
Read the full abstract ▾
All authors Dahye Kim, Bhuvan Sachdeva, Karan Uppal, Naman Gupta, Vineeth N. Balasubramanian, Deepti Ghadiyaram
Computer Vision
Slimmable ConvNeXt: Width-Adaptive Inference for Efficient Multi-Device Deployment
Janek Haberer, Jon Eike Wilhelm and Olaf Landsiedel
Deploying
vision models across devices with varying resource constraints, or even on a single device where available compute fluctuates due to battery state, thermal throttling, or latency deadlines, typically requires training and maintaining separate models. Width-adaptive inference addresses this by training a single set of shared weights containing multiple nested subnetworks of increasing capacity, but prior CNN-based approaches required switchable batch normalization, while recent scalable methods have focused on
Vision Transformers. We present Slimmable ConvNeXt, which shows that ConvNeXt's modern design, specifically LayerNorm and inverted bottlenecks, makes it particularly suited for channel-width slimming, eliminating the normalization overhead of classical slimmable networks and producing a simpler training pipeline than both prior CNN and ViT approaches. On ImageNet-1k, Slimmable ConvNeXt-T with 3 subnetworks achieves 80.8% top-1 accuracy at 4.5 GMACs and 77.4% at 1.2 GMACs, trained from scratch for 600 epochs. At comparable compute, this exceeds HydraViT's 6-head subnetwork (78.4% at 4.6 GMACs) by 2.4 percentage points and its 3-head configuration (73.0% at 1.3 GMACs) by 4.4 percentage points, while also outperforming MatFormer-S (78.6%) and SortedNet-S (78.2%) at the same GMACs. Scaling to Slimmable ConvNeXt-B further improves maximum accuracy to 82.8% at 15.35 GMACs.
Read the full abstract ▾
Computer Vision
From Abstraction to Instantiation: Learning Behavioral Representation for Vision-Language-Action Model
Bing Hu, Zaijing Li, Rui Shao et al. (+4 more)
Vision-
Language-Action (VLA) models often suffer from performance degradation under distribution shifts, as they struggle to learn generalized behavior representations across varying environments. While existing approaches attempt to construct behavior representations through action-centric latent variables, they are often limited by short-horizon temporal fragmentation and static execution-
alignment, leading to inconsistent behaviors in complex scenarios. To address these limitations, we propose BehaviorVLA, a framework that facilitates robust manipulation through the learning of a temporally coherent behavioral representations. Our approach features two symmetric components: (1) the Visuomotor Behavior Encoder (VBE), which utilizes a causal Mamba-based architecture to aggregate long-horizon trajectory information into a unified behavior representation; and (2) the Phase-conditioned Behavior Decoder (PBD), which decodes this representation into precise actions by dynamically aligning task-level priors with real-time execution progress. Experiments on RoboTwin 2.0, LIBERO, and CALVIN demonstrate state-of-the-art success rates of 58%, 98%, and 4.36 (Avg.Len), respectively. Notably, in real-world sim-to-real transfer, BehaviorVLA matches the performance of OpenVLA-OFT using only 50% of the demonstration data, showcasing its superior data efficiency and generalization.
Read the full abstract ▾
All authors Bing Hu, Zaijing Li, Rui Shao, Junda Chen, April Hua Liu, Wei-Shi Zheng, Liqiang Nie
Computer Vision
SEGA: Spectral-Energy Guided Attention for Resolution Extrapolation in Diffusion Transformers
Javad Rajabi, Kimia Shaban, Koorosh Roohi et al. (+2 more)
Diffusion transformers (DiTs) have emerged as a dominant architecture for text-to-image generation, yet their performance drops when generating at resolutions beyond their training range. Existing training-free approaches mitigate this by modifying inference-time
attention behavior, often through Rotary Position
Embeddings (RoPE) extrapolation combined with
attention scaling. However, these strategies apply a uniform and content-agnostic scaling across RoPE components with distinct frequency characteristics, inducing a trade-off between preserving global structure and recovering fine detail. We introduce SEGA, a training-free method that dynamically scales
attention across RoPE components according to the latent's spatial-frequency structure at each denoising step. This adaptive scaling improves both structural coherence and fine-detail fidelity. Experiments show that SEGA consistently improves high-resolution synthesis across multiple target resolutions, outperforming state-of-the-art training-free baselines.
Read the full abstract ▾
All authors Javad Rajabi, Kimia Shaban, Koorosh Roohi, David B. Lindell, Babak Taati
Computer Vision
SegCompass: Exploring Interpretable Alignment with Sparse Autoencoders for Enhanced Reasoning Segmentation
Zhenyu Lu, Liupeng Li, Jinpeng Wang et al. (+4 more)
While large
language models provide strong compositional
reasoning, existing
reasoning segmentation pipelines fail to transparently connect this
reasoning to visual perception. Current methods, such as latent query
alignment, are end-to-end yet opaque "black boxes". Conversely, textual localization readout is merely readable, not truly interpretable, often functioning as an unconstrained post-hoc step. To bridge this interpretability gap, we propose SegCompass, an end-to-end model that leverages a Sparse Autoencoder (SAE) to forge an explicit, interpretable, and differentiable
alignment pathway. Given an image-instruction pair, SegCompass first generates a chain-of-thought (CoT) trace. The core of our method is an SAE that maps both the CoT and visual
tokens into a shared, high-dimensional sparse concept space. A query codebook selects salient concepts from this space, which are then spatially grounded by a slot mapper into a multi-slot heatmap that guides the final mask decoder. The entire model is trained jointly, unifying reinforcement learning for the
reasoning path with standard
segmentation supervision. This SAE-driven interface provides a "white-box" connection that is significantly more traceable than latent queries and more coherent than textual readouts. Extensive experiments on five challenging
benchmarks demonstrate that SegCompass matches or surpasses state-of-the-art performance. Crucially, our visual and quantitative analyses show a strong correlation between the quality of the learned sparse concepts and final mask accuracy, confirming that SegCompass achieves superior results through its enhanced and inspectable
alignment. Code is available at https://github.com/ZhenyuLU-Heliodore/SegCompass.
Read the full abstract ▾
All authors Zhenyu Lu, Liupeng Li, Jinpeng Wang, Haoqian Kang, Yan Feng, Ke Chen, Yaowei Wang
Computer Vision · Cross-listed from cs.CL
Seeing the Poem: Image-Semantic Detection of AI-Generated Modern Chinese Poetry with MLLMs
Shanshan Wang, Fengying Ye, Hanjia Lyu et al. (+6 more)
Previous detection studies have shown that LLMs cannot be effectively used as detectors, but these studies have not addressed modern Chinese poetry. Moreover, no relevant research has explored the performance of LLMs in detecting modern Chinese poetry. This paper evaluates and enhances the performance of LLMs as detectors for modern Chinese poetry, and proposes an image-semantic guided poetry detection method. Compared with traditional detection approaches, our method innovatively incorporates images that reflect the content of the poetry. Through example-driven approaches, our method effectively integrates information such as meaning, imagery, and feeling from the image, then forms a complementary judgment with the poem text. Experimental results demonstrate that the LLM detectors based on our method outperform baseline detectors based on plain text, and even surpass the best-performing traditional detector, RoBERTa. The Gemini detector using our method achieves a Macro-F1 score of 85.65%, reaching the state-of-the-art level. The performance improvements of different LLM detectors on multiple LLMs-generated data prove the effectiveness of our method.
Read the full abstract ▾
All authors Shanshan Wang, Fengying Ye, Hanjia Lyu, Caiwen Gou, Junchao Wu, Jingming Yao, Chengzhong Xu, Jiebo Luo, Derek F. Wong
Computer Vision
What Does the Caption Really Say? Counterfactual Phrase Intervention for Compositional Data Selection in Vision-Language Pretraining
Hyejin Go, Semi Lee and Hyesong Choi
CLIP-style contrastive pretraining typically curates web-scale image-text pairs using sample-level filtering signals, often based on pair-level
alignment. We show that this signal saturates: once coarse mismatches are removed, stricter global filtering no longer tracks the compositional supervision provided by the retained captions. The reason is structural - a global score conflates whether a pair is broadly plausible with whether the individual object, attribute, and relation phrases inside the caption materially support the image-text match. The latter is what compositional generalization demands, yet pair-level filters are blind to it. We address this with Counterfactual Phrase Intervention (CPI), a phrase-level curation framework that converts controlled nonce-
token substitutions into image-conditioned phrase-sensitivity scores. CPI uses global
alignment only for coarse mismatch removal, then ranks the surviving pool by whether caption phrases measurably affect the image-text score under controlled substitution. We frame CPI as a first-order phrase-sensitivity signal rather than a grounding or identification result, and evaluate it at CC3M scale. Ranking by this signal yields a 50%-data subset that improves VL-CheckList-VG Relation by +1.91 over the full-data baseline and +1.00 over
alignment-only filtering at matched budget, while improving SugarCrepe overall and preserving general transfer. CPI is loss-orthogonal: applied unchanged to NegCLIP, it further improves VL-CheckList-VG Relation by +3.84, with additional CE-CLIP gains in the main text.
Read the full abstract ▾