Computer Vision
VQ-Touch: A Data-Efficient Tactile Generation Framework Across Sensors and Scenarios
Kailin Lyu, Long Xiao, Jianing Zeng et al. (+3 more)
Tactile image generation significantly reduces the dependency on expensive and wear-prone sensors by synthesizing high-fidelity tactile data, offering an efficient solution for tactile information acquisition in robotic perception and human-machine interaction systems. However, existing methods depend on large-scale, diverse
datasets from specific sensors and lack efficient data utilization and robust generalization capabilities, struggling in
vision-limited environments. To address this, we introduce VQ-Touch, a tactile generation framework that supports both cross-sensor and multi-scenario applications. Specifically, to efficiently extract complex deformation and texture features from the data, we propose DM-VQGAN, an effective tactile representation learner. Furthermore, we introduce a discrete
diffusion decoder with a unified conditioning interface, supporting
multimodal generation tasks such as images and labels, and enhances the model's generalization capability through few-shot mixed training, thus achieving compatibility with current mainstream sensors and their variants. Experiments show that VQ-Touch surpasses state-of-the-art methods in multiple tasks.
Read the full abstract ▾
All authors Kailin Lyu, Long Xiao, Jianing Zeng, Di Wu, Lin Shu, Jie Hao
Computer Vision
WorkDrive: Roadwork Chain of Causation for Autonomous Driving
Tianyi Jiang, Wen Zhang, Sihan Yang et al. (+2 more)
Autonomous driving
vision-
language models (VLMs) struggle in roadwork zones, where familiar visual cues such as lane markings and permanent signs are altered or absent, and temporary devices such as cones and barriers redefine the drivable corridor. VLMs can detect these objects, but without explicit guidance they anchor their
reasoning on familiar elements from pre-training and fail to connect work-zone observations to correct planning decisions. We propose WorkDrive, a framework that constructs perception-grounded causal
reasoning for work zones and aligns it with trajectory prediction. An automated multitask perception pipeline extracts structured scene facts and injects them into a Chain-of-Causation (CoC) annotation pipeline, redirecting the annotator's
attention to domain-specific elements. The resulting
reasoning labels are used for supervised fine-tuning, followed by reinforcement learning with a single reward: consistency between lateral meta-actions and the predicted trajectory. On ROADWork, the largest public work-zone
dataset, the proposed roadwork CoC reduces trajectory average displacement error (ADE) by 9.0%, and consistency-based GRPO yields a further 3.0%, achieving progressive improvement over the trajectory-only baseline. Code and data will be publicly released.
Read the full abstract ▾
All authors Tianyi Jiang, Wen Zhang, Sihan Yang, Ming Lu, Wentao Zhang
Computer Vision
AE-UAV: An Air-to-Air Event-Based UAV Tracking Benchmark and a Real-Time Frequency-Domain Tracker
Zixin Jiang, Bing He, Chaoran Xiong et al. (+3 more)
Air-to-air (A2A) unmanned aerial vehicle (
UAV) tracking is fundamental to airborne remote sensing of low-altitude aerial targets. However, the deployment of continuous, real-time tracking systems on
UAVs presents significant challenges. In A2A scenarios, traditional frame-based cameras suffer from severe performance degradation under low illumination, overexposure, and high-speed motion owing to their limited dynamic range and fixed temporal sampling. Although event cameras offer a promising alternative with microsecond temporal resolution and a high dynamic range, current research is bottlenecked by two primary issues: 1) the absence of dedicated A2A event-based
datasets, and 2) the heavy reliance of existing trackers on GPU acceleration and extensive training data, rendering them impractical for resource-constrained
UAVs. To bridge these gaps, we introduce AE-
UAV, an air-to-air event-based
UAV tracking
benchmark. To the best of our knowledge, this is the first airborne-captured event camera
dataset for A2A tracking, comprising 178 flight sequences with continuous-time cubic B-spline annotations. Furthermore, we propose the Fast-Slow Frequency-domain Tracking (FSFT) method. This lightweight, training-free framework seamlessly integrates frequency-domain template matching with search region prediction and detection-based drift correction. Extensive experiments demonstrate that FSFT operates at an ultra-fast 420 frames per second (FPS) on CPU-only hardware. It retains 93.97% of the accuracy of state-of-the-art GPU-dependent methods while delivering a 5.32-fold effective speedup and exhibiting superior temporal resolution generalization, thereby providing a highly efficient and robust solution for airborne remote sensing of aerial targets. The
dataset and source code are available at https://github.com/MSP-xEN/AE-
UAV.
Read the full abstract ▾
All authors Zixin Jiang, Bing He, Chaoran Xiong, Zhenzhen Wang, Xin Zhao, Ling Pei
Computer Vision
Multimodality as Supervision: Self-Supervised Specialization to the Test Environment via Multimodality
Kunal Pratap Singh, Ali Garjani, Rishubh Singh et al. (+6 more)
Cross-modal learning, i.e., learning to predict one modality from another, is a fundamental mechanism for self-supervision via leveraging multimodality. Many practical applications, e.g., deploying a household
robot, involve devices that are equipped with a rich set of sensors that enable
multimodal sensing in their test environment. This presents an opportunity to apply cross-modal learning to the
multimodal data sensed by these devices to learn representations. Findings in developmental psychology also suggest that biological
agents leverage it to build an effective representation of their surroundings. To study this, we propose a controlled setup, where we restrict a user device to just a given test environment. It results in a specialization setup where we attempt to develop a performant model for this specific test environment. Under this setup, we develop Test-Space Training (TST), which performs
multimodal data collection in the test environment and performs self-supervised pre-training on it. We evaluate these models on various downstream tasks in the same environment. Under this setup, we find various interesting insights, such as collecting rich
multimodal data only from the test environment and leveraging cross-modal learning, we can achieve competitive results with generalist models (e.g., DINOv2 and CLIP) pre-trained on large-scale internet
datasets. This enables an alternative scenario where the need for external Internet-scale
datasets for pre-training models is reduced. We also present a set of analyses and ablations that raise intriguing points on substituting data with (multi)modality, and how varying pre-training data enables a tradeoff between a model's abilities to specialise to a test environment, and generalize to held-out spaces.
Read the full abstract ▾
All authors Kunal Pratap Singh, Ali Garjani, Rishubh Singh, Muhammad Uzair Khattak, Efe Tarhan, Jason Toskov, Andrei Atanov, Oğuzhan Fatih Kar, Amir Zamir
Computer Vision
Causal-Adversarial Probing of Clinical Covariates for Prostate MRI Grading
Yipei Wang, Shiqi Huang, Wen Yan et al. (+6 more)
Deep learning models for prostate MRI-based cancer grading may encode clinical
covariates that either reflect useful disease-related signal or non-generalising shortcut information, but their role is usually assumed. We propose a causal-
reasoning framework for probing
covariate dependence in MRI-based International Society of Urological Pathology (ISUP) Grade Group prediction. Rather than treating mpMRI as a direct cause of grade, we model MRI appearance and ISUP grade as observations of latent tumour pathology, and test whether candidate clinical variables act as nuisance correlates, disease-related proxies, or irrelevant
covariates in the learned representation. We implement this using an
adversarial framework that suppresses the decodability of individual clinical
covariate at a time while preserving MRI-based grade prediction. The approach is developed and evaluated on 2,903 prostate MRI examinations, with external validation on 576 patients. We report a set of interesting and previously under-explored imaging-to-clinical-variable interactions in the context of deep learning generalisation. For examples, in binary ISUP Grade Group $\geq2$ classification, suppressing age, BMI, and alcohol use improved AUC by 1.23%, 0.84%, and 1.42%, respectively (all p < 0.05), suggesting reduced non-generalising
covariate information; In contrast, suppressing PSA and prostate volume degraded AUC by 1.91% and 7.61% (all p < 0.001), indicating that these variables carried task-relevant signal. These findings show that
adversarial covariate suppression can provide a practical representation-level analysis for distinguishing potentially harmful dependence from informative signal in prostate MRI grading models.
Read the full abstract ▾
All authors Yipei Wang, Shiqi Huang, Wen Yan, Weixi Yi, Dean C. Barratt, Mark Emberton, Daniel C. Alexander, Veeru Kasivisvanathan, Yipeng Hu
Computer Vision
VideoSEMA: a scalable and efficient Mamba-like attention for video understanding
Nhat Thanh Tran, Fanghui Xue, Shuai Zhang et al. (+4 more)
We present for
video understanding (classification) a split space-time
attention model, VideoSEMA, consisting of a scalable and efficient Mamba-like
attention (SEMA) block in space and a softmax temporal
attention in time. In each frame, SEMA
attention applies a local window
attention in parallel with a global averaging in a Mamba macro-architecture, which is called Mamba-like. Under certain rank conditions, we prove that the computationally cheaper split space-time
attention is equivalent to full space-time
attention. On
benchmark K400 data sets, VideoSEMA out-performs heavier
vision transformer and Mamba models. On
benchmark SSv2 data, VideoSEMA leads in top-1 accuracy among models of similar parameter sizes. As image resolution scales up from standard $224^2$ to $1024^2$ on K400 and without fine-tuning, VideoSEMA degrades much more gracefully than VideoMamba in accuracy. It is promising to extend VideoSEMA to longer
videos with a dilated/sparse temporal
attention.
Read the full abstract ▾
All authors Nhat Thanh Tran, Fanghui Xue, Shuai Zhang, Jiancheng Lyu, Yunling Zheng, Yingyong Qi, Jack Xin
Computer Vision
Variational Inference for Bird's Eye View Segmentation in Autonomous Driving
Jingyue Shi, Huaicheng Li, Junhui Zhao et al. (+1 more)
The bird's eye view (BEV) has emerged as a pivotal approach for environmental perception in autonomous driving, providing a unified spatial representation for vehicles. Nevertheless, despite BEV's significance in addressing the challenges inherent to autonomous driving, effectively fusing data from multiple camera sensors and operating in complex external driving environments remains a considerable challenge. To mitigate this issue, we recast the BEV
segmentation problem within a variational inference framework. In this paper, we propose a novel
transformer-based variational flow transformation network for BEV
segmentation, denoted as TVB. Our architecture implicitly learns the mapping from multiple camera views to a unified canonical BEV map during training by exploiting posterior BEV supervision. TVB employs a conditional variational auto encoder (CVAE) as its backbone and produces multiple BEV map candidates. To augment the realism of the generated BEV maps, we integrate normalizing flows into the map generation process, enabling the construction of more complex and expressive probability distributions. Furthermore, we design a BEV-
attention fusion (BAF) module that harnesses
attention mechanisms to adaptively integrate the multiple candidate BEV maps. Experimental results, evaluated on both the nuScenes and OPV2Vdatasets, demonstrate that our proposed method achieves superior performance in multi-camera view BEV
segmentation and lane environment perception.
Read the full abstract ▾
All authors Jingyue Shi, Huaicheng Li, Junhui Zhao, Yanxiang Jiang
Computer Vision
Pretraining Multiple Instance Learning Networks with Multi-Teacher Distillation from Pathology Slide Foundation Models
Mingxi Fu, Jiawen Li, Renao Yan et al. (+4 more)
Multiple instance learning (MIL) has become the main paradigm for whole-slide image (WSI) analysis in computational pathology. However, existing MIL aggregators are still typically trained from scratch for each downstream task, relying on limited slide-level labels to learn both aggregation mechanisms and downstream discriminative representations simultaneously. As a result, they often suffer from unstable optimization, overfitting, and limited transferability. Similar to pretrained ResNet and
Vision Transformer models in natural image learning, MIL also requires reusable pretrained initialization. However, high-quality slide-level pretraining data remain scarce, and MIL models are usually lightweight and weakly supervised, making large-scale pretraining difficult in practice. To address this challenge, we propose a
distillation-based pretraining framework for MIL, which leverages two slide-level foundation models, TITAN and CARE, as teachers to transfer their representational knowledge into a diverse set of MIL architectures. To effectively balance supervision from different teachers, we further introduce an angular dispersion normalized
distillation loss. The distilled weights are then used as initialization for downstream adaptation. We conduct systematic evaluations on 15
benchmark datasets under both linear probing and full-parameter fine-tuning, and further validate its advantages in few-shot scenarios. Experimental results show that pretraining generally improves MIL aggregators over from scratch training, especially in linear-probing and few-shot settings, while maintaining the computational efficiency of lightweight MIL models. Code is available at https://github.com/fu0201/MIL_Pretrained.
Read the full abstract ▾
All authors Mingxi Fu, Jiawen Li, Renao Yan, Jiali Hu, Qiehe Sun, Tian Guan, Yonghong He
Computer Vision
Team RAS in 11th ABAW Competition: Multimodal Ambivalence Recognition Approach
Elena Ryumina, Maxim Markitantov, Alexandr Axyonov et al. (+4 more)
Automatic recognition of ambivalence and hesitancy is challenging because these states may be expressed through inconsistent linguistic, acoustic, facial, and contextual patterns, while top-performing systems often rely on computationally expensive ensembles. We present a single text-centered
multimodal approach for
video-level ambivalence and hesitancy recognition for the 11th Affective & Behavior Analysis in-the-Wild (ABAW) Challenge. The proposed approach combines linguistic, acoustic, facial, and scene features using text-centered
multimodal fusion model. Text Residual Fusion treats text as the anchor modality and applies gated residual adjustments based on the other modalities. Experiments on the Behavioural Ambivalence/Hesitancy (BAH) corpus confirm that text is the strongest unimodal modality. The Text Residual Fusion model achieves an average Macro F1-score (MF1) of 75.14% across the Development and Public Test subsets. On the Private Test subset, it reaches an MF1 of 78.24%, outperforming the text model by 4.03%. These results demonstrate that complementary
multimodal information can improve recognition performance without requiring a large model ensemble.
Read the full abstract ▾
All authors Elena Ryumina, Maxim Markitantov, Alexandr Axyonov, Fedor Shchetinin, Timur Abdulkadirov, Dmitry Ryumin, Alexey Karpov
Computer Vision
GlobalForge: Towards Robust AI-Generated Image Detection
Manni Cui, Ruiqi Liu, Dianyuan Zou et al. (+8 more)
AI-generated image (AIGI) detectors achieve strong accuracy on clean
benchmarks, but their performance drops sharply after images are propagated through real-world channels. We trace this fragility to what these detectors actually learn: they overfit to local artifacts left by generators in small spatial neighborhoods, which are easily destroyed by common propagation degradations such as JPEG compression and blur. Instead, we shift the discriminative cue from fragile local artifacts to more robust global structure. Building on this, we propose GlobalForge, a framework with two complementary modules. The Local Information Bottleneck (LIB) suppresses local components to block shortcut learning, while the Global Structural
Reasoning (GSR) module forces every
token to gather evidence from distant regions. Both modules are trained jointly under a contrastive structural loss based on degradation that keeps the resulting features stable under degradation. To support fine-grained robustness evaluation, we further introduce RealDeg-Bench, covering 7 common degradation operators and multi-step compound chains. GlobalForge improves average BAcc on 8 in-the-wild
benchmark groups by $\mathbf{5.89\%}$ over the previous state-of-the-art, and is clearly ahead of representative baselines on RealDeg-Bench under both single and compound degradations. Code is available at https://anonymous.4open.science/r/GlobalForge-BE0F/.
Read the full abstract ▾
All authors Manni Cui, Ruiqi Liu, Dianyuan Zou, Ziheng Qin, Jingrui Xu, ZiAn Wang, Jianglan Wei, Han Zhou, Yu Liu, Yan Wang, Shu Wu
Computer Vision
ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationships
Xinyu Liu, Shihao Li, Weihong Lin et al. (+10 more)
Recent
diffusion-based
video generation models have made significant progress in multi-reference image-conditioned
video editing. However, existing methods still struggle to coordinate information from multiple visual sources accurately. We identify a critical deficiency in existing approaches. Existing editing instructions lack explicit reference relationships, and most
multimodal large
language models (MLLMs) cannot generate them reliably. To address this problem, we propose ReBind, a systematic framework that introduces semantic instructions with embedded reference
tokens as the intermediate representation for multi-reference image-conditioned
video editing. Our key insight is
embedding reference
tokens at semantic positions to eliminate ambiguity and establish precise bindings between visual attributes and their sources. We develop ReBind-Instruct, a specialized MLLM that learns to establish explicit bindings between visual attributes and their reference sources through a two-stage progressive scheme for precise reference relationships. We further develop ReBind-Edit, which enables lightweight adaptation of text-to-
video models to coordinate multiple references by binding visual attributes to their designated sources. Extensive experiments demonstrate that ReBind substantially outperforms general-purpose MLLMs in instruction quality and achieves state-of-the-art performance among open-source methods on reference image conditioned
video editing. Our project webpage: https://rebind-mrv2v.github.io/.
Read the full abstract ▾
All authors Xinyu Liu, Shihao Li, Weihong Lin, Xinlong Chen, Yang Shi, Yujin Han, Yiyang Cai, Yanghao Wang, Ruibin Yuan, Yuanxing Zhang, Pengfei Wan, Wenhan Luo, Yike Guo
Computer Vision
VIABench: A Comprehensive Video Benchmark Collected from Blind Individuals for Visual Impairment Assistance
Yunfeng Liu, Yuandong Yang, Jiarui Han et al. (+5 more)
Visually impaired individuals (VIIs) encounter significant daily challenges due to limited access to visual information. Although
Multimodal Large
Language Models (MLLMs) have achieved impressive results on general
vision and
language tasks, their practical utility in real-world blind assistance still remains largely underexplored. To fill this gap, we introduce VIABench, a comprehensive
video benchmark specifically designed to evaluate MLLMs in Visually Impaired Assistance scenarios using first-person
videos recorded or shared by VIIs themselves. VIABench defines three core tasks, each targeting a distinct requirement in visual assistance. Proactive Reminder: Assesses the model's ability to interpret ongoing
video content while proactively anticipating and verbally describing upcoming
navigation-critical events; Visual Question Answering (VQA): Evaluates the model's capacity to answer user-posed questions about the environment or objects within the
video;
Vision-Guided Interaction: Tests context-aware
reasoning to accomplish intentional interactions between user and environment. To ensure a robust and fair evaluation, we propose a rigorous benchmarking pipeline that supports both online (real-time) and offline settings. Our experiments demonstrate that current MLLMs still struggle to deliver comprehensive support for VIIs, especially in the Proactive Reminder task, which demands accurate anticipation and real-time responsiveness. We hope VIABench will drive future research toward developing customized MLLMs for real-world assistance, ultimately improving
navigation and interaction experiences for visually impaired individuals. Code and data will be released at https://github.com/MCG-NJU/VIABench.
Read the full abstract ▾
All authors Yunfeng Liu, Yuandong Yang, Jiarui Han, Zhenpeng Huang, Yuqing Tang, Xiangyu Zeng, Gangshan Wu, Limin Wang
Computer Vision
Autoregressive Modeling of Film with Applications in Video Montage
Marcelo Sandoval-Castañeda, Fabian Caba Heilbron, Shiry Ginosar et al. (+4 more)
This work introduces FilmGPT, an autoregressive
transformer designed to address the challenge of
video montage–turning a collection of raw, "unwatchable" footage into coherent cinematic sequences. Inspired by
language learning in modern LLMs, we train a long-context autoregressive
transformer on a large corpus of movies. The aim is to implicitly capture the "grammar" of film directly from data rather than from hand-coded rules. Unlike other generative models, FilmGPT does not generate any new
video frames. Instead, at inference time, we introduce a footage-constrained decoding
algorithm to select the best next shot from the input raw footage according to the statistical patterns learned from films. We first evaluate these learned statistics directly by using the FilmGPT autoregressive model for next shot prediction on a standard
benchmark of shot sequence ordering, outperforming the previous state of the art. We then evaluate our footage-constrained decoding
algorithm on the full film editing task via a user study, and find that our FilmGPT-based editing significantly outperforms previous approaches. Finally, we demonstrate the applicability of FilmGPT to a wide range of applications in
video montage, from automatic
video segment trimming to human-in-the-loop film editing.
Read the full abstract ▾
All authors Marcelo Sandoval-Castañeda, Fabian Caba Heilbron, Shiry Ginosar, Bryan Russell, Josef Sivic, Alexei A. Efros, Greg Shakhnarovich
Computer Vision · Cross-listed from cs.RO
Image-to-Point Cloud Registration Made Easy with Rectified Flow-based LiDAR Upsampling
Reon Tabata, Kenji Koide, Shuji Oishi et al. (+4 more)
Image-to-Point Cloud Registration (I2P) is essential for integrating camera and
LiDAR in perception and autonomous systems, yet the modality gap between images and point clouds makes it difficult to achieve both high accuracy and strong generalization. In this paper, we propose a simple yet effective I2P method that treats
LiDAR as an imaging sensor: from a single sparse
LiDAR scan, we generate a dense
LiDAR intensity image using Conditional Rectified Flow, match it with a camera image using a pre-trained feature matcher, and estimate the 6-DoF relative pose via PnP-RANSAC. The proposed model is pre-trained through a self-supervised image completion task and fine-tuned on a small amount of
LiDAR data (neither image-point cloud pairs nor ground-truth sensor poses are required), enabling it to scale to diverse
LiDAR and camera configurations. Experiments on the R3LIVE
dataset show that the proposed method achieves a mean error of 4.89{\deg} / 1.63 m, outperforming existing methods, while completing a single registration in approximately 0.68 s.
Read the full abstract ▾
All authors Reon Tabata, Kenji Koide, Shuji Oishi, Masashi Yokozuka, Taku Okawara, Aoki Takanose, Jun Miura
Computer Vision · Cross-listed from cs.AI
Action QFormer: Structured Representation Shaping under Action Supervision in Vision-Language-Action Models
Yufeng Ji, Wenhao Tang, Haoyi Niu et al. (+3 more)
Action supervision in
vision-
language-action (VLA) models is often treated as a downstream objective for learning action prediction. In this paper, we study it instead as a force that shapes inherited
multimodal representations. We show that this shaping has a dual effect: it is necessary for forming action-compatible representations, but when action supervision is applied too directly to the inherited
multimodal pathway, it can also destabilize representations that support
language-side processing and object grounding. To address this tension, we introduce Action QFormer, a query-based action-facing interface that uses instruction-conditioned queries to reorganize inherited
multimodal information into action-facing representations before downstream action generation. In zero-shot sim-to-real
navigation, Action QFormer improves average closed-loop task success from 18.8% to 56.3%, raises fixed-instruction action-generation correctness from 22.5% to 75.5%, and nearly eliminates out-of-distribution instruction generations. Further analyses show that Action QFormer changes how action supervision shapes inherited
multimodal representations, reducing broad upstream rewriting while preserving targeted and sometimes constructive action-supervised adaptation. These results suggest that improving VLA performance requires not only stronger pretrained backbones, but also better ways of selecting and organizing inherited
multimodal information while controlling how it is shaped under action supervision.
Read the full abstract ▾
All authors Yufeng Ji, Wenhao Tang, Haoyi Niu, Koushil Sreenath, Yi Wu, Zhongyu Li
Computer Vision
Knowing You at First Glance: Inferring Apparent Personality from Faces
Shuhuan Chen, Xiangyu Zhu, Weisong Zhao et al. (+3 more)
Inferring apparent personality from facial images is important in social scenarios for
embodied agents in human-
robot interaction. Unlike inferring intrinsic personality traits via conversation, this task models first-impression personality perception based solely on facial appearance before interaction begins. Existing studies mainly focus on the Big Five personality model and often rely on
language or
multimodal inputs. As a result, it remains unclear whether facial cues alone can support meaningful associations with perceived personality traits. This question is particularly relevant for MBTI types, which are widely used in practice and more readily interpretable by large
language models. To this end, we propose GlanceFace, an end-to-end framework for apparent personality inference leveraging
vision-
language models to introduce semantic priors and a semantic-enhanced facial representation module to capture subtle personality-related cues, together with an uncertainty-aware learning strategy to handle noisy and subjective annotations. Extensive experiments demonstrate strong performance on MBTI-based apparent personality
benchmarks and reveal relationships between facial characteristics and perceived personality traits, highlighting its potential to support adaptive initial interaction strategies for
embodied agents. The code and
dataset are available at https://github.com/MrHuan3/GlanceFace.
Read the full abstract ▾
All authors Shuhuan Chen, Xiangyu Zhu, Weisong Zhao, Haichao Shi, Xiao-Yu Zhang, Zhen Lei
Computer Vision · Cross-listed from cs.AI
SportD: How do VLMs physically strategize?
Jasin Cekinmez, Addison J. Wu, Haotian Xia et al. (+6 more)
Vision-
language models (VLMs) can describe a scene, but can they act well within one? We study whether VLMs can make sound strategic decisions, using soccer as an objective testbed with quantifiably-valued actions. We introduce SportD, a
dataset and evaluation consisting of 1421 decision scenarios across professional men's and women's soccer games, where a VLM must decide what action to take next. Models on average select the optimal action around 27% of the time, less often than the professional players, and capture markedly less of the value at stake. Furthermore, they exhibit a clear preference for safer actions, favoring lower-variance, lower-value choices that also make less physical progress toward goal. Frontier VLMs are better at estimating whether an action will succeed, placing the highest-success-probability action among their top choices in 72-85% of cases. Yet VLMs systematically conflate likelihood with value, assigning higher value to actions that are more likely to succeed ($\rho=+0.30$ to $+0.52$), despite no such relationship in the ground truth ($\rho=-0.08$). Modifying the deliberation instructions to encourage risk-taking brings the frontier models closer to the players' skill levels. SportD opens a new direction for rigorously evaluating physical strategic decision-making in VLMs, showing that careful decomposition of their choices can reveal the mechanisms underlying systematic biases such as risk aversion.
Read the full abstract ▾
All authors Jasin Cekinmez, Addison J. Wu, Haotian Xia, Kyumin Andrew Shim, Anay Putty, Jinglin Xiao, Zhuohan Liu, Leo Liu, Weining Shen
Computer Vision
Hough-SIFT: Robust Image Registration for Linear Structures via Hough Space
Masaki Satoh
Image registration is essential in applications such as electronic image stabilization. Scale-Invariant Feature Transform (SIFT), a widely used local keypoint detector and descriptor, typically provides accurate registration; however, it often fails in scenes with strong linear structures (e.g., shutters), where local features become ambiguous. We propose Hough-SIFT, a robust registration method that performs SIFT descriptor matching in Hough space. In this domain, linear structures form distinctive peaks that restore descriptor discriminability. Experiments demonstrate that Hough-SIFT is robust in linear scenes where SIFT frequently fails, while maintaining accuracy comparable to SIFT in normal scenes.
Read the full abstract ▾
Computer Vision
MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generation
Yinhan Zhang, Dingwei Tan, Xianghao Kong et al. (+3 more)
Large-scale
video diffusion models deliver strong generation performance, but full fine-tuning for downstream tasks incurs prohibitive computational costs. Existing parameter-efficient fine-tuning (PEFT) methods have two critical flaws on billion-scale models: they still require substantial trainable parameters, and reward-based training suffers from noise-induced optimization instability in condition-guided tasks. We propose MagicPrompt, a lightweight framework that achieves extreme parameter efficiency and stable reward optimization. It first adopts
Attention-Embedded
Prompt Tuning, which steers generation via lightweight soft
prompts with orders of magnitude fewer parameters while preserving pre-trained knowledge. It further introduces Dual-Space Reward Feedback Optimization, which uses self-supervised latent objectives to improve condition-guided reward training. Experiments show MagicPrompt reaches competitive performance with less than 1% trainable parameters and notably reduces training costs.
Read the full abstract ▾
All authors Yinhan Zhang, Dingwei Tan, Xianghao Kong, Yue Ma, Yeying Jin, Anyi Rao
Computer Vision · Cross-listed from cs.AI
Multi-LLM Collaborative MRI Report Generation for Visual Instruction Tuning in Brain Oncology
Sinyoung Ra, Jonghun Kim and Hyunjin Park
Recent advances in large
language models (LLMs) and their extension to
vision-
language models (VLMs) have made it easier to combine text and images for tasks such as report generation. Existing VLMs in medicine typically focus on 2D images (chest X-rays), and their extension to 3D imaging has been difficult because of the lack of paired 3D imaging-text data. Thus, we introduce a new method for creating a 3D image-text
dataset for
brain oncology using 3D MRI scans of glioma and meningioma cases. We use a cooperative system in which several LLMs work together to generate and check reports, ensuring that they are accurate and clear. By leveraging the new 3D MRI-text
dataset, we further build a VLM that converts MRI scans into
tokens and aligns them with text instructions. Our VLM performed better in report generation and visual question answering tasks than other 2D and 3D methods. Our method not only improves the quality of reports but also helps with better diagnosis and treatment in
brain oncology.
Read the full abstract ▾
Computer Vision
Advanced Image Generation: Negative Prompt Optimization and Latent Classifier Guidance
Vaddi Charan Sai Nandan Reddy, Harini B and Chandana M S
We present a novel system that integrates negative
prompt optimization via a fine-tuned sequence-to-sequence LLM and latent-space classifier guidance to improve the quality of images generated by Stable
Diffusion. Our approach automatically generates optimized negative
prompts, and employs a CNN-RNN hybrid classifier to evaluate and guide
diffusion steps, rolling back low-quality latent updates. Experimental results demonstrate that our dual-guidance framework reduces artifacts and improves semantic fidelity compared to baseline
diffusion.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.HC
Skeleton: Visual Authoring of Non-visual Data Experiences
Frank Elavsky, Chieri Nnadozie, Lucas Nadolskis et al. (+2 more)
When sighted practitioners author accessible data visualizations, they build
navigation structures (the nodes, edges, and input bindings that govern how assistive technologies traverse an interface) entirely in code, with no visual representation. Without a representation to react to, practitioners cannot develop judgment about what makes
navigation good or bad, and the quality ceiling of non-visual experiences is set by the absence of a feedback loop. We address this problem through longitudinal co-design with practitioners across cartography, design systems, and open-source visualization, and make three contributions. First, we introduce an Inspector that renders
navigation graphs as interactive node-link diagrams, and a Dimensions API that expresses
navigation in terms of data dimensions rather than explicit graph construction. Second we present Skeleton, a direct-manipulation authoring environment in which the properties of an accessible
navigation structure are translated into visual representations authors can observe and manipulate. Key techniques include a dual-view editor that simultaneously shows the system's
navigation model and the end user's spatial experience, a scaffolding engine that automates spatial node placement by repurposing a visualization rendering pipeline, a live label-template editor with real-time screen-reader-output preview, and a testing mode that makes traversal sequence visually trackable. Third, we evaluate Skeleton through an in-situ study with 8 practitioners across visualization design, engineering, and research. Making
navigation structure visible changed how practitioners engaged with accessible design: they reconsidered the architecture of their own visualizations, attended to a broader range of input modalities, and shifted from treating accessibility as a compliance task to treating it as a design problem. (abstract shortened for arxiv)
Read the full abstract ▾
All authors Frank Elavsky, Chieri Nnadozie, Lucas Nadolskis, Patrick Carrington, Dominik Moritz
Computer Vision
Breaking the Model Forgetting Cycle in Long-Incremental 3D Object Detection
Peisheng Qian, Jie Xu, Xulei Yang et al. (+1 more)
Incremental 3D object detection requires a detector to learn novel object classes while remembering previously learned ones over sequentially arriving data. Previous methods, primarily based on pseudo-labeling, perform reasonably in short-incremental stages but still suffer from severe model forgetting when dealing with long-incremental sequences. We investigate this failure and reveal a detrimental self-reinforcing cycle: data distribution shift of novel classes causes model forgetting on old classes, which further produces accumulated error in pseudo-labeling that exacerbates model degradation. To address this issue, we draw inspiration from the human learning process and propose the Learning-Dynamics-driven
Memory and Review (LDMR) framework. LDMR monitors per-class detection quality at periodic training checkpoints and uses these learning-dynamics signals to drive two innovative mechanisms, namely (i) human-like intra-stage review that divides each incremental stage into multiple sub-stages' training and concentrates on remembering the most-forgotten objects, and (ii) scene-aware cross-stage
memory evolution that evolves a
memory bank to transfer knowledge between two consecutive stages by jointly considering scene learnability and diversity. Extensive experiments across multiple long-incremental protocols on indoor
benchmarks SUN RGB-D and ScanNetV2 show that LDMR substantially mitigates the model forgetting and outperforms all baselines by a clear margin. Code is available at https://github.com/qianpeisheng/LDMR.
Read the full abstract ▾
All authors Peisheng Qian, Jie Xu, Xulei Yang, Na Zhao
Computer Vision
HyMobileAgent: Data-Environment Co-Scaling for Efficient GUI Agents
Hy Vision Team, Huawen Shen, Zhengyang Tang et al. (+20 more)
As large
multimodal models move from understanding content to operating on digital environments, mobile GUI has emerged as a challenging and consequential testbed for digital
embodied intelligence. Mobile
agents operate under three coupled constraints: precise perception of complex interfaces, scalable acquisition of high-quality interaction data, and robust long-horizon decision making under compounding execution errors. This report presents HyMobileAgent, a mobile GUI
agent built on Hy3.0-VL-A3B, a
vision-native foundation model featuring native any-resolution input, an A3B-scale deployment budget, and a 32K context window to model extended interaction histories. Rather than relying solely on model scaling, we develop a joint data and environment centric scaling framework to address the key bottlenecks of mobile interaction. Our framework integrates a GUI perception flywheel combining mock-interface synthesis, rejection sampling, and icon-specific augmentation; a knowledge pipeline that transforms tutorial
videos into structured interaction data; a million-scale action data pipeline deployed across more than 2000 sandbox and real-device instances with automated failure attribution; the PhoneWorld Mock App Factory, providing a resettable training environment with 34 mock applications and over 34000 tasks; and a structured Planning-and-Reflection mechanism with explicit dead-loop detection for reliable long-horizon execution. We also introduce a progressive training recipe consisting of mid-training, supervised fine-tuning, and reinforcement learning with task-specific reward designs.
Read the full abstract ▾
All authors Hy Vision Team, Huawen Shen, Zhengyang Tang, Shangpin Peng, Liang Wu, Anran Zhang, Weinong Wang, Yiduo Guo, Chenxin Li, Zhengyao Fang, Yang Ding, Junyi Li, Fei Tang, Zheng Ruan, Yi Zhang, Xingran Zhou, Dingchen Yang, Sunqi Fan, Zhiyi Wan, Han Hu, Xin Lai, Pengyuan Lyu, Chengquan Zhang
Computer Vision
AdaTurn: Budget-Aware Test-Time Scaling for Active Visual Perception Agents
Susan Liang, Chao Huang, Filippos Bellos et al. (+3 more)
Active visual
agents solve fine-grained image tasks by interleaving
reasoning with image-grounding actions across multiple turns. However, deployment-time rollout budgets are rarely fixed: some requests permit long rollouts, while others require the
agent to act under a tight turn limit. Existing methods train the policy as if the rollout budget were hidden, so when the available budget is smaller than the trajectory the
agent prefers, the interaction is often truncated before any valid answer is produced; we term this failure catastrophic truncation. To overcome this challenge, we present AdaTurn, a budget-aware framework that conditions the
agent on the allowed number of turns and explicitly trains the boundary behavior induced by the budget. Our key component, Forced-Answer DAPO (FA-DAPO), converts the over-budget event from a masked or penalized failure into a trainable final-decision step, teaching the model to synthesize partial evidence when further tool use is no longer possible. We further randomize rollout budgets during both training and inference and introduce a load-balanced scheduler that makes such operations practical. AdaTurn substantially improves low-budget accuracy, for example raising VisualProbe-Medium from 36.7% to 47.6% at four turns, while preserving strong scaling at larger budgets and transferring effectively to multiple backbones and general
multimodal benchmarks.
Read the full abstract ▾
All authors Susan Liang, Chao Huang, Filippos Bellos, Jing Bi, Jason J Corso, Chenliang Xu
Computer Vision
3D Geometric Tooth Alignment Planning via Deep Reinforcement Learning
Yong Li, Jianwen Lou, Jiayue Ma et al. (+3 more)
3D
geometric tooth
alignment planning, which determines sequential trajectories from initial malocclusion to the final target
alignment, is a cornerstone of modern digital orthodontics. This paper presents a novel deep reinforcement learning (DRL) framework to automate the generation of these
alignment paths. We formulate the planning process as a Markov Decision Process (MDP) to capture its sequential decision-making nature, focusing on optimizing
geometric trajectories while integrating essential spatial constraints, such as inter-dental collision avoidance and path efficiency. The proposed method leverages the Deep Deterministic Policy Gradient (DDPG)
algorithm, enhanced by three key innovations: (1) a
Transformer-based
agent to model complex spatial interactions between teeth and manage high-dimensional state-action spaces; (2) a dynamic masking scheme that restricts movement to a sparse subset of teeth per step, better reflecting the clinical logic of sequential
alignment; and (3) a two-stage curriculum learning strategy that gradually increases task difficulty to ensure training stability and efficient path discovery. We evaluate our approach on a
dataset of 10K expert-designed treatment plans based on clinical data. Experimental results demonstrate that our method outperforms existing baselines in terms of path safety and
geometric efficiency, providing a robust and automated solution for 3D
geometric orthodontic
alignment planning.
Read the full abstract ▾
All authors Yong Li, Jianwen Lou, Jiayue Ma, Yao-Xiang Ding, Youyi Zheng, Haihua Zhu
Computer Vision
SwinAD: Multi-stage feature reconstruction for unsupervised industrial anomaly detection
Huong Ninh, Chien Thai, Mai Xuan Trang et al. (+3 more)
Industrial anomaly detection aims to identify and localize defective regions without relying on exhaustive annotations of all possible defect types. Although recent unsupervised methods have achieved strong performance, most are primarily designed for single-class settings and often struggle in multi-class scenarios, where diverse normal patterns may lead to over-generalization and reduce the discriminative capability between normal and anomalous regions. In this paper, we propose SwinAD, a reconstruction-based framework for multi-class unsupervised anomaly detection that leverages a frozen pretrained Swin
Transformer V2 encoder and a feature diversity-preserving reconstruction decoder. The hierarchical encoder provides semantically rich multi-scale features, while stage-wise bottleneck modules with dropout prevent trivial identity mapping and encourage robust reconstruction of normal patterns. To further improve localization, we introduce a feature diversity-preserving reconstruction framework that maintains complementary reconstruction hypotheses instead of relying on a single decoding branch. The discrepancies between encoder features and the two reconstructed features are then aggregated across multiple scales to produce the final anomaly map. Experiments conducted on three industrial anomaly detection
benchmarks, including MVTec AD, VisA, and Real-IAD, demonstrate that SwinAD achieves competitive image-level performance and strong pixel-level localization accuracy, with particularly notable improvements in pixel-level AP and 1 on MVTec AD. These results indicate that combining hierarchical Swin features with diverse multi-scale reconstruction substantially improve pixel-level localization in multi-class unsupervised anomaly setting.
Read the full abstract ▾
All authors Huong Ninh, Chien Thai, Mai Xuan Trang, Vu-Minh Le, Thanh Ha Le, Long Tran
Computer Vision
Uni-AdaVD: Universal Concept Erasure for Visual Generation via Orthogonal Value Decomposition
Qifan Zhou, Yuan Wang, Yanbin Hao et al. (+4 more)
Visual generative models inevitably absorb undesirable concepts from uncurated pretraining data, making concept erasure essential for safe deployment. Existing erasure methods, however, are often architecture-specific and struggle to remove target concepts while preserving non-target content and generative priors. We present Uni-AdaVD, a universal inference-time concept erasure framework for visual generation. Uni-AdaVD treats the value space of
multimodal attention as a unified intervention space and introduces encoder-aware target representation construction to localize target semantics across heterogeneous text encoders. It further combines orthogonal value decomposition with an adaptive erasing shift to suppress target semantic directions without updating the original model weights. Extensive experiments on U-Net-, DiT-, and autoregressive image generators, as well as text-to-
video models, demonstrate strong performance on single- and multi-concept erasure while preserving non-target priors. These results suggest that Uni-AdaVD provides an efficient and adaptable safety mechanism for modern visual generative models. Our code is available at https://github.com/QifanZhou/Uni-AdaVD.
Read the full abstract ▾
All authors Qifan Zhou, Yuan Wang, Yanbin Hao, Xiang Wang, Kuien Liu, Richang Hong, Meng Wang
Computer Vision
VTM-Nav: Harnessing Cross-Episode Experience for Object-Goal Navigation with Hierarchical Visual-Topological Memory
Xiaoran Xu, Yupeng Wu, Tianyu Xue et al. (+4 more)
Training-free ObjectNav
agents increasingly use
vision-
language models (VLMs), yet typically discard acquired scene knowledge after each request. We study cross-episode ObjectNav, where each request is an independently initialized, single-goal episode and only self-acquired, scene-scoped
memory persists across episodes. We ask whether an
agent with fixed model parameters and
navigation components can reuse such experience without retraining or oracle information. We introduce \method, a training-free framework with a persistent Hierarchical Visual-Topological
Memory (VTM). VTM uses a coarse room
topology to index room-owned visual
memories, distinguishes in-room from remote-visible evidence, and retains successful approach cues. For each request, VTM-Nav re-localizes the
agent in accumulated scene structure, retrieves target-relevant records from plausible rooms, and grounds
memory guidance in candidates derived from the current observation. A conservative execution guard further handles local failures. Under matched 40-step comparisons, VTM-Nav exceeds the
memory-reset WMNav control by 4.6, 2.0, and 0.8 SR points on HM3D v0.1, HM3D v0.2, and MP3D, respectively, with comparable or higher SPL. On HM3D, it also exceeds WMNav harnessed by textual
memory by 3.1 and 5.5 SR points. These results demonstrate effective reuse of cross-episode scene experience through hierarchical visual-topological
memory.
Read the full abstract ▾
All authors Xiaoran Xu, Yupeng Wu, Tianyu Xue, Yifan Xu, Xuanran Dong, Xiaoshan Yang, Changsheng Xu
Computer Vision
Compression of 3D Gaussian Splatting Data Using GPU-friendly Graphics Texture Coding
Amir Said and Randall Rauwendaal
Techniques for modeling 3D scenes from image collections, such as 3D Gaussian Splatting (3DGS), are capable of generating high-quality novel views by leveraging graphics primitives with view-dependent appearance. In 3DGS, spherical harmonic (SH) are employed to model view-dependent color, resulting in a large number of SH coefficients per primitive and large
memory requirements. While compression approaches have been proposed to mitigate this problem, they do not exploit the capabilities of modern Graphics Processing Units (GPUs) for parallel decoding and rendering. In this paper, we propose a method for compressing SH color coefficients using texture compression schemes specifically designed for efficient parallel GPU decoding and supported by dedicated hardware acceleration. It is shown that those methods can compress color coefficients more effectively than 2D textures by exploiting the fact that primitives can be locally grouped and reordered according to color. Furthermore, we introduce a bit-rate control strategy that preserves random access, enabling large-scale parallelization without compromising rendering performance. Experimental results using BC1 and BC7 texture compression formats show that GPU-based decompression can be achieved with negligible or imperceptible degradation in the visual quality of rendered 3DGS scenes.
Read the full abstract ▾