Computer Vision
DeTrack: A Benchmark and Altitude-Aware Dual World Model for Drone-embodied Tracking
Guyue Hu, Haoming Liu, Siyuan Song et al. (+3 more)
Aerial object tracking has broad applications in public safety, emergency rescue, wildlife monitoring, and related fields. However, existing aerial tracking
benchmarks are mainly based on passive 2D
video sequences captured from fixed camera locations or predefined flight paths, where drones are treated as passive cameras rather than
embodied agents that actively perceive, interact, and control their motion in dynamic 3D scenes. In this paper, we define a new drone-
embodied tracking task, termed DeTrack, which requires a drone to track a target in interactive 3D environments using online
egocentric observations and active flight control in a closed loop. We build a large-scale
benchmark containing 11,368 target trajectories across diverse scenes, rendering conditions, semantic regions, and moving distractors, together with evaluation metrics for target visibility, tracking accuracy, and trajectory success. We further propose AaDWorlds, an altitude-aware dual world model framework for drone-
embodied tracking. AaDWorlds consists of an altitude-aware perception module and dual world models that imagine future states under both high- and low-altitude regimes. By combining pseudo altitude-aware observations and imagined future states, AaDWorlds alleviates the intrinsic altitude-mediated contradiction between target visibility and flight safety. Experiments on the DeTrack
benchmark demonstrate that AaDWorlds improves closed-loop tracking performance across all evaluation metrics.
Read the full abstract ▾
All authors Guyue Hu, Haoming Liu, Siyuan Song, Chenglong Li, Feng Chen, Jin Tang
Computer Vision
Spatial Blindness in Whole-Slide Multiple Instance Learning
Xiangyu Li and Ran Su
Whole-slide MIL models are often called context-aware once graphs, Transform ers, or state-space modules are placed above patch
embeddings. We show that this label can be deceptive. On pathology tasks where tissue architecture is part of the diagnostic signal, several strong MIL baselines retain nearly unchanged slide level AUC after patch coordinates are permuted. Their predictions are accurate, but largely compositional. We refer to this failure mode as spatial blindness. Our explanation is optimization-based: dense appearance statistics are learned early under slide-level supervision, leaving weak gradients for sparse spatial relations. ResTopoMIL addresses the issue by first fitting a permutation-invariant prototype histogram and then freezing it while a lightweight graph branch learns the residual under a coordinate-shuffling constraint. The architecture is simple by design; the intervention is in how the spatial branch is trained. Across 9 public WSI bench marks, ResTopoMIL improves classification and survival prediction with 1.15M parameters, restores sensitivity to coordinate perturbation, and gives stronger lo calization evidence on CAMELYON-16.
Read the full abstract ▾
Computer Vision
FastOCR: Dynamic Visual Fixation via KV Cache Pruning for Efficient Document Parsing
Zihan Tang, Leqi Shen, Hui Chen et al. (+7 more)
Vision-
Language Models (VLMs) have shown strong promise on Optical Character Recognition (OCR), yet the sheer number of visual
tokens required to encode dense documents incurs prohibitive inference cost. Existing
pruning methods rely on physical eviction, e.g., permanently discarding visual
tokens during the prefill stage. While effective for natural images, this strategy fundamentally breaks down on OCR, where virtually every visual
token may correspond to a character or structural element, and any irreversible loss leads to catastrophic accuracy degradation. We observe that, although document images appear globally dense and seemingly unprunable, the model's
attention to them is in fact temporally sparse: at each decoding step it concentrates on a small region that shifts gradually across steps, much as a human reader fixates on successive words rather than perceiving an entire page at once. Motivated by this Dynamic Visual Fixation phenomenon, we recast the intractable global
pruning problem as a tractable local, dynamic one and propose FastOCR, a training-free framework with two complementary modules. Specifically, Focal-Guided
Pruning identifies a small set of focal layers and selects the most task-relevant visual
tokens from them at each step, while Cross-Step Fixation Reuse exploits the gradual shift of fixation to warm-start each step from the previous one. By dynamically adjusting which
tokens are attended rather than evicting any from the cache, FastOCR avoids permanent information loss. Extensive experiments show that FastOCR serves as a plug-and-play acceleration module, generalizing consistently across five VLMs of varying sizes and architectures. On Qwen2.5-VL, FastOCR retains 98% of the unpruned model's accuracy while attending to only 5% of the visual
tokens per decoding step, reducing
attention latency by 3.0$\times$.
Read the full abstract ▾
All authors Zihan Tang, Leqi Shen, Hui Chen, Ao Wang, Ben Wan, Yan Feng, Ke Zhang, Sicheng Zhao, Tongxuan Liu, Guiguang Ding
Computer Vision
Medical Context Distorts Decisions in Clinical Vision Language Models
David Restrepo, Ira Ktena, Maria Vakalopoulou et al. (+2 more)
Vision-
language models (VLMs) are increasingly proposed for clinical decision support, yet their reliability in real-world scenarios that require integrating both visual and textual context from medical records remains poorly characterized. This paper identifies three failure modes: (1) modality over-reliance on text over images, (2) spurious reliance on irrelevant clinical history, and (3)
prompt sensitivity across semantically equivalent inputs. We evaluate a diverse set of general-domain and medically-tuned open and closed VLMs on chest x-ray tasks using MIMIC-CXR. By systematically manipulating image-text
alignment, clinical history, and
prompt formulations, we found that VLM decisions are dominated by the text modality, even when visual evidence is available. Moreover, we observed that VLMs are heavily influenced by irrelevant reports, while minor
prompt changes can reverse correct image-based predictions. Our findings underscore the need for explicit safeguards and stress-testing before considering the use of these models in clinical practice.
Read the full abstract ▾
All authors David Restrepo, Ira Ktena, Maria Vakalopoulou, Stergios Christodoulidis, Enzo Ferrante
Computer Vision
VISTA: Variance-Gated Inter-Sequence Test-Time Adaptation for Multi-Sequence MRI Segmentation
Zhipeng Deng, Jiale Zhou, Wenhan Jiang et al. (+4 more)
Deploying multi-sequence
magnetic resonance imaging (MRI)
segmentation models to new clinical environments is challenging due to variations in scanners and acquisition protocols. Although existing TTA methods handle basic per-modality shifts, they often fail under a fundamental dual-shift problem, as their adaptation signals fail to capture modality-interaction shifts that disrupt inter-sequence consistency. To address this, we propose Variance-gated Inter-Sequence Test-time Adaptation (VISTA), a source-free framework that tackles modality-interaction shifts. First, we design an Inter-Sequence Intervention Generator (ISIG) that generates a set of consistency probes by swapping low-frequency spectra and
entropy-localized patches across sequences, preserving anatomical semantics while challenging inter-sequence dependencies. Second, we introduce Cross-View Disagreement-Aware Pseudo Labeling (CDPL), which establishes a voxel-wise reliability metric using cross-view disagreement variance to dynamically gate self-training and enforce interventional consistency, encouraging the network to rely on robust anatomical semantics. Extensive experiments adapting from standard adult MRI (BraTS-GLI-Pre) to African low-field (BraTS-SSA) and pediatric (BraTS-PED) cohorts show improved performance over competing methods under clinical shifts, achieving absolute Dice improvements of +1.89% (SSA) and +2.82% (PED) over the source model. The code is available at https://github.com/dzp2095/VISTA.
Read the full abstract ▾
All authors Zhipeng Deng, Jiale Zhou, Wenhan Jiang, Haolin Wang, Xun Lin, Yafei Ou, Yefeng Zheng
Computer Vision · Cross-listed from cs.LG
Radial-Angular Geometry for Reliable Update Diagnosis in Noisy-Label Learning
Ningkang Peng, Jingyang Mao, Xiaoqian Peng et al. (+2 more)
Noisy-label methods often estimate sample reliability from forward-space signals such as loss, confidence, or
entropy. These signals indicate whether a sample is difficult to predict, but they do not directly test whether its observed label induces a reliable parameter update. This gap matters because hard clean samples and mislabeled samples can have similar loss while inducing different updates. We recast reliability estimation as diagnosis of the observed-label update. The sample-wise empirical Fisher trace gives a backward-space measure of update energy: for the classifier layer, it factorizes into a prediction-residual term and a feature-sensitivity term, so it captures information beyond scalar loss. Trace, however, is still a radial magnitude signal and cannot decide whether a large update is useful or harmful. We therefore propose Relative
Geometric Conflict (RGC), which compares the observed-label gradient with a reference gradient induced by an EMA teacher. The conflict term helps distinguish large but aligned hard-clean updates from large conflicting updates caused by corrupted labels. Across synthetic and real-world noisy-label
benchmarks, RGC improves hard-clean preservation and accuracy under our evaluation protocol.
Read the full abstract ▾
All authors Ningkang Peng, Jingyang Mao, Xiaoqian Peng, Weiguang Qu, Yanhui Gu
Computer Vision
Soap2Soap: Long Cinematic Video Remaking via Multi-Agent Collaboration
Yiren Song, Huilin Zhong, Kevin Qinghong Lin et al. (+2 more)
We study series-level cinematic remaking, a long-horizon
video-to-
video generation problem that localizes full episodes or films via stylization or actor replacement while strictly preserving narrative structure, motion choreography, and character identity across hundreds of shots. Existing
video generation and editing pipelines often break down in this regime due to compounding identity drift, background mutation, and semantic erosion under large camera motions and viewpoint changes. We propose Soap2Soap, a multi-
agent framework that enforces long-term
language-visual consistency through a Dual-Bridge Consistency mechanism: a scene-aware JSON screenplay serving as a persistent semantic backbone, and dynamically allocated visual reference anchors at both scene and shot levels. To suppress drift before
video synthesis, we introduce batch keyframe consistency, jointly generating multiple keyframes in a shared latent context via a grid-based formulation. A closed-loop verification
agent further audits identity, stability, and
alignment to trigger selective regeneration. Experiments on SoapBench demonstrate strong improvements over commercial
video generation APIs in long-term consistency and narrative fidelity.
Read the full abstract ▾
All authors Yiren Song, Huilin Zhong, Kevin Qinghong Lin, Haofan Wang, Mike Zheng Shou
Computer Vision
RadGenome-Anatomy: A Large-Scale Anatomy-Labeled Chest Radiograph Dataset via Physically Grounded Volumetric Projection
Shuchang Ye, Mingyuan Meng, Hao Wang et al. (+2 more)
Anatomical structure labels for chest radiographs are essential for medical image
segmentation and a broad range of downstream diagnostic tasks. However, annotating anatomy directly on 2D chest radiographs is labor-intensive and intrinsically ambiguous, as 3D anatomical structures are projected onto a single 2D plane where boundaries may overlap, be occluded, or appear only partially visible. Consequently, existing anatomy-labeled chest radiograph
datasets remain limited in scale, anatomy coverage, and label reliability. To address these limitations, we introduce RadGenome-Anatomy, the largest anatomy-labeled chest radiograph
dataset, containing over 10 million
segmentation masks across 210 anatomical structures in 25,692 studies. It is constructed by projecting large-scale 3D anatomical masks from CT volumes into 2D radiographic space through canonical radiographic geometry. This shifts annotation from directly tracing uncertain 2D boundaries to defining anatomy in volumetric space, where structures that overlap or become partially invisible in radiographs remain spatially separable. As a result, each 2D mask represents the physically grounded projected footprint of a volumetrically defined structure. The scale and broad anatomical coverage of RadGenome-Anatomy, including structures that are overlapping, partially visible, or difficult to delineate directly, enable research on
geometric measurements as explicit evidence for chest radiograph interpretation. We demonstrate this by training XAnatomy to predict structure-specific masks and derive clinically relevant measurements, achieving diagnostic accuracies of 96.4%, 95.6%, and 89.2% for cardiomegaly, kyphosis, and scoliosis, respectively.
Read the full abstract ▾
All authors Shuchang Ye, Mingyuan Meng, Hao Wang, Usman Naseem, Jinman Kim
Computer Vision
Bridging Data Trials and Task Barriers: A Unified Framework for Sketch Biometric Identification
Decheng Liu, Bin Hu, Xinbo Gao et al. (+4 more)
Different from existing cross-modality identification tasks (e.g., heterogeneous face recognition, sketch re-identification, etc.), we introduce a novel yet practical setting for these related identification tasks, named sketch biometric identification, which aims to continually train a unified model across different data domains, even diverse identification tasks. Sketch biometric identification faces challenges, including scarce real sketch data, high annotation costs,
privacy risks, and insufficient generalization ability of cross-task models. Existing methods usually rely on limited real data or single-task optimization, making it difficult to effectively address the joint challenges of cross-modality and cross-task. This paper proposes a unified framework that integrates efficient synthetic sketch generation and task-sequential continual learning. First, we design an efficient pipeline to generate a large-scale and high-quality synthetic person and face sketch data, which significantly reduces costs and avoids
privacy risks. Meanwhile, we enhance the model's robustness by fusing real data. Second, we construct a universal unified framework for sketch biometric identification, which adopts a task-sequential training strategy: the model first completes sketch person re-identification learning on the person
dataset; subsequently, it maintains the acquired person recognition capability through a trusted sample replay technique and seamlessly performs incremental training on the face
dataset. This enables a single model to simultaneously handle the cross-task capabilities of multiple sketch biometric identification tasks. To support the study of the mentioned sketch biometric identification, we built a new large-scale
benchmark, SketchUnified-BioID, with several practical evaluation protocols.
Read the full abstract ▾
All authors Decheng Liu, Bin Hu, Xinbo Gao, Dawei Zhou, Chunlei Peng, Nannan Wang, Ruimin Hu
Computer Vision
Memory-Augmented Query Intent Understanding for Efficient Chat-based Image Retrieval
Xianke Chen, Daizong Liu, Yushuo Lou et al. (+5 more)
Different from traditional text-to-image
retrieval tasks, chat-based image
retrieval allows the human-interactive system to iteratively clarify and refine user intent through multi-round dialogue, thereby achieving more fine-grained
retrieval results. The key challenge in this task lies in dynamically understanding and updating the user's query intent across dialogue rounds. Although existing works have achieved great performance on this new task, they simply handle history query information either by directly concatenating all previous queries into a long textual sequence or by relying on large
language models to reconstruct the current query from history. Such strategies are computationally redundant and easily lead to inconsistent intent representations as the dialogue progresses. To alleviate these issues, this paper proposes a novel and efficient
memory-based user intent updating framework for the chat-based image
retrieval task, called
Memory-Augmented Query Intent Understanding (MAQIU). It introduces a lightweight memorization module that dynamically aggregates and evolves the semantic representation of query intent across dialogues, while a
memory recall mechanism is further employed to prevent intent forgetting and enhance long-term semantic integrity. In addition, MAQIU also integrates historical image
retrieval results as visual guidance, allowing the model to strengthen cross-round correlations and refine current visual understanding. Extensive experiments demonstrate that MAQIU achieves substantial performance gains while maintaining high computational efficiency, reducing dialogue encoding FLOPs by 86.4% compared with the prior baseline ChatIR. Source code is available at https://github.com/HuiGuanLab/MAQIU.
Read the full abstract ▾
All authors Xianke Chen, Daizong Liu, Yushuo Lou, Xin Tan, Xun Yang, Shuhui Wang, Xun Wang, Jianfeng Dong
Computer Vision
Omni-DuplexEval: Evaluating Real-time Duplex Omni-modal Interaction
Chaoqun He, Mingyang Xiang, Yingjing Xu et al. (+5 more)
Real-time duplex interaction is essential for
multimodal AI systems operating in real-world scenarios, where models must continuously process streaming inputs and respond at appropriate moments. However, most existing
multimodal large
language models (MLLMs) are evaluated in offline settings, where the entire
video input is processed before any response is generated. While recent work has started to explore real-time duplex MLLMs, there is still no comprehensive
benchmark or automatic evaluation method for this setting. To address this gap, we propose Omni-DuplexEval, a
benchmark for systematically evaluating real-time duplex interaction. The
benchmark consists of two complementary scenarios: (1) Real-Time Description, which evaluates the ability to generate continuous, time-aligned responses that track evolving
multimodal inputs, and (2) Proactive Reminder, which evaluates the ability to identify salient events and respond at appropriate moments. Omni-DuplexEval contains 660
videos with fine-grained, human-annotated labels and precise temporal metadata, spanning 9 tasks grounded in real-world scenarios, where all questions are formulated as open-ended queries. We further introduce an automatic evaluation framework based on LLM-as-a-Judge, which enables systematic assessment by jointly evaluating response-content
alignment and response timing through timestamp-aware and sequential
reasoning, achieving strong
alignment with human judgments. Experiments on state-of-the-art duplex MLLMs reveal substantial limitations. The best-performing model achieves only 39.6% overall, while scoring only 20.0% on Proactive Reminder. Our analysis identifies two key challenges: models struggle to balance timely responses with coherent, holistic content generation, and they often fail to determine both when to respond and what to produce. We hope our work facilitates further progress in MLLMs.
Read the full abstract ▾
All authors Chaoqun He, Mingyang Xiang, Yingjing Xu, Bokai Xu, Junbo Cui, Jie Zhou, Yuan Yao, Lijie Wen
Computer Vision
UniPPTBench: A Unified Benchmark for Presentation Generation Across Diverse Input Settings
Bo Zhao, Maosheng Pang, Chen Zhang et al. (+3 more)
Existing works typically focus on presentation generation under isolated input settings, whereas real-world use cases span diverse scenarios, including vague user
prompts, long documents,
multimodal materials, and multiple heterogeneous sources. Moreover, current evaluations are often insufficiently scenario-specific. They mainly rely on generic presentation-quality criteria, such as visual appeal, layout quality, and overall coherence, but fail to assess the core capabilities required by different input settings, including grounded compression, visual-text
alignment, and cross-source synthesis. Consequently, the field lacks a unified
benchmark and a scenario-aware evaluation framework for faithfully diagnosing presentation-generation systems across diverse real-world settings. We present UniPPTBench, a unified
benchmark for presentation generation across four representative input settings: vague-
prompt, long-document,
multimodal-document, and multi-source generation. We further introduce UniPPTEval, a scenario-aware evaluation protocol that combines shared metrics for cross-setting comparison with scenario-specific metrics tailored to the core requirements of each setting. We also provide transparent reference baselines to support reproducible comparison. Experiments on UniPPTBench reveal substantial performance variation across settings and recurring failure modes in content grounding,
multimodal integration, and cross-source synthesis. In particular, strong performance on generic presentation-quality metrics does not necessarily imply strong task fulfillment in grounded scenarios. Together, UniPPTBench and UniPPTEval provide a faithful and diagnostic foundation for evaluating presentation generation across diverse real-world scenarios. Code and data will be publicly available.
Read the full abstract ▾
All authors Bo Zhao, Maosheng Pang, Chen Zhang, Huan Yang, Yixin Cao, Wei Ji
Computer Vision
GeoHand: Unlocking Prior Geometry Knowledge for Monocular 3D Hand Reconstruction
Weiquan Lin, Yaoqing Hu, Liangchen Dai et al. (+2 more)
Monocular 3D hand reconstruction is intrinsically a
geometric problem, yet RGB appearance features alone often struggle to resolve severe ambiguities caused by self-occlusions and hand-object interactions. While introducing depth can explicitly provide spatial cues, raw sensor-captured depth maps are extensively noisy and incomplete, limiting their usefulness for fine-grained hand reconstruction. To bridge this gap, we propose GeoHand, a novel framework that unlocks high-quality
geometric priors from a frozen foundational
monocular geometry
estimator (MoGe2). Recognizing that these priors are oriented toward general scenes, we introduce a map-level GeoAdapter to recalibrate the spatial features, specifically adapting them for detailed hand reconstruction. Furthermore, to systematically integrate these adapted priors without overwhelming intrinsic RGB appearance cues, we employ a gated cross-modal
token fusion strategy. Finally, to secure precise local articulation, we design a Keypoint-Queried Iterative Refiner (KQIR) that uses projected joint locations to query geometry-aware image features for spatial correction. By combining global
geometric disambiguation with local refinement in a unified pipeline, GeoHand achieves state-of-the-art performance on FreiHAND, DexYCB, and HO3Dv3, especially under severe occlusions and hand-object interactions.
Read the full abstract ▾
All authors Weiquan Lin, Yaoqing Hu, Liangchen Dai, Xu Tang, Xingyu Chen
Computer Vision · Cross-listed from cs.CY
Position: Age Estimation Models Do Not Process Biometric Data
Nikita Marshalkin
When a neural network estimates someone's age from a photograph, does it process biometric data? The answer depends on whether identity-discriminative representations arise within the network during inference, a question that may seem trivial to ML researchers but triggers consent requirements under GDPR, statutory damages under BIPA, or high-risk AI classification under the EU AI Act. Yet no regulatory guidance addresses it. This position paper provides empirical evidence: 14 models evaluated across 3 face verification
benchmarks show age
estimators fall orders of magnitude short of identification thresholds. Age estimation models cannot identify individuals. We call on researchers to provide transparency about what systems store and can do, and on regulators to distinguish transient processing from template storage.
Read the full abstract ▾
Computer Vision
VoxShield: Protecting 3D Medical Datasets from Unauthorized Training via Frequency-Aware Inter-Slice Disruption
Xinyao Liu, Zhipeng Deng, Wenhan Jiang et al. (+4 more)
The release of public 3D medical image
segmentation (MIS)
datasets accelerates clinical research but simultaneously heightens risks of unauthorized AI model training. While Unlearnable Examples (UE) offer protection by injecting imperceptible perturbations to prevent effective model learning, existing methods primarily target 2D scenarios. They neglect the volumetric spatial correlations and inter-slice anatomical consistency inherent in 3D medical volumes, which serve as critical learning priors for 3D
segmentation networks. To bridge this gap, we propose VoxShield, a UE framework that explicitly targets the volumetric inductive biases of 3D networks. Our core insight is that by systematically dismantling the cross-slice continuity that 3D architectures rely on, we can fundamentally impair their spatial aggregation process. Specifically, we introduce an Inter-Slice Frequency Consistency Disruption mechanism that maximizes the spectral divergence between adjacent slices, injecting structural incoherence along the $z$-axis. Complementing this structural attack, a Semantic Prediction Disruption module is incorporated. By maximizing the $\ell_1$ divergence between clean and perturbed logits, it forces the injected noise to penetrate the entire network and corrupt the final semantic mapping. Experiments on BraTS19 and
FLARE21 demonstrate that VoxShield successfully degrades 3D
segmentation performance, reducing the DSC from 80.0% to near 0.0% and from 88.6% to 6.8%, respectively. All protections are achieved with minimal perturbation ($\epsilon=4/255$) to preserve high visual fidelity. The code is available at https://github.com/KK266299/VoxShield.
Read the full abstract ▾
All authors Xinyao Liu, Zhipeng Deng, Wenhan Jiang, Haolin Wang, Xun Lin, Yafei Ou, Yefeng Zheng
Computer Vision
GraphMAR: Geometry-Aware Graph Learning Framework for Spatially Adaptive CT Metal Artifact Reduction
Zilong Li, Chenglong Ma, Yiming Lei et al. (+7 more)
Computed
tomography (CT) metal artifact reduction (MAR) aims to reduce the severe streaking artifacts induced by metallic implants and other high-density objects. Effective MAR generally requires both accurate artifact localization and artifact removal. Sinogram-domain methods can exploit explicit
geometric cues, such as metal traces, to identify metal-corrupted measurements, while requiring raw projection data, which is often unavailable in clinical and practical scenarios. Image-domain methods are more flexible and widely applicable, yet they usually lack comparable
geometric guidance, limiting their ability to localize artifacts and leading to suboptimal results. To address this limitation, we propose GraphMAR, a geometry-aware learning framework for explicit artifact identification and spatially adaptive MAR in the image domain. The key idea is to introduce graph-based
geometric modeling as an image-domain analogue of sinogram metal traces. Specifically, we first construct a
geometric graph from the metal mask and derive a
geometric density graph that coarsely localizes artifact-prone regions according to inter-implant geometry. We then design GraphMoE, a graph-routed mixture-of-experts module that builds a polar-coordinate artifact graph in feature space and adaptively routes different experts to different spatial regions for MAR. By aligning the learned routing maps with the
geometric density graph, GraphMAR provides explicit and interpretable artifact localization while enabling region-adaptive artifact reduction. Experiments on both simulated and real-world
datasets demonstrate that GraphMAR achieves superior MAR performance compared with existing methods. To the best of our knowledge, this is the first work to introduce graph-based modeling for CT MAR and to enable explicit artifact identification in the image domain, improving both restoration quality and interpretability.
Read the full abstract ▾
All authors Zilong Li, Chenglong Ma, Yiming Lei, Yuanlin Li, Jing Han, Jiannan Liu, Huidong Xie, Junping Zhang, Yi Zhang, Hongming Shan
Computer Vision
Single-Sample Black-Box Membership Inference Attack against Vision-Language Models via Cross-modal Semantic Alignment
Jiaqing Li, Yajuan Lu, Xiaochuan Shi et al. (+3 more)
Vision-
Language Models (VLMs) have achieved remarkable success, yet their reliance on massive
datasets and unintended memorization of training data raise significant data security risk. Membership Inference Attacks (MIAs) aim to assess these risks by determining whether a data sample was included in a model's training set. However, existing MIA methods against VLMs face critical bottlenecks: gray-box method relies on internal logits that are typically restricted in real-world Application Programming Interfaces (APIs), while black-box method depends on large-scale statistical distributions, which struggle in single-sample scenarios. To this end, we investigate MIAs from the perspective of cross-modal semantic
alignment, and observe that member images exhibit significantly stronger image-caption
alignment due to training memorization, whereas generated captions for non-members may deviate from the original visual content. Leveraging this insight, we propose a novel MIA framework designed for strict black-box and single-sample setting that quantifies such
alignment within a joint
embedding space, thereby bypassing these unrealistic assumptions. We conducted extensive experiments on three open-source and two closed-source VLMs. On the VL-MIA/Flicker
dataset, our method achieves an AUC of 0.821 against LLaVA-1.5, significantly outperforming existing baselines. Furthermore, it remains robust under diverse image perturbations, highlighting its practicality.
Read the full abstract ▾
All authors Jiaqing Li, Yajuan Lu, Xiaochuan Shi, Gang Wu, ZhongYuan Wang, Chao Liang
Computer Vision · Cross-listed from cs.RO
Tactile-based Multimodal Fusion in Embodied Intelligence: A Survey of Vision, Language, and Contact-Driven Paradigms
Zhixiang Cao, Di Tian, Runwei Guan et al. (+11 more)
Tactile sensing is a fundamental modality for
embodied intelligence, offering unique and direct feedback on contact geometry, material properties, and interaction dynamics that remote sensors cannot replace. However, unimodal tactile perception is inherently limited by its sparse spatial coverage and lack of global semantic context. With the recent explosion in deep learning and large
language models, integrating tactile with
vision and
language has become essential to bridge physical interaction with semantic
reasoning, leading to the emergence of
Multimodal Tactile Fusion. Despite rapid progress, the existing researches remain fragmented across disparate
datasets, sensing modalities, and tasks, lacking a unified theoretical framework. To address this gap, this paper provides a comprehensive survey of
multimodal tactile fusion research up to the first quarter of 2026. We propose a hierarchical taxonomy that organizes the field into two primary dimensions:
multimodal datasets and
multimodal methods. On the data side, we categorize resources ranging from Tactile-
Vision datasets, Tactile-
Language datasets, Tactile-
Vision-
Language datasets, and Tactile-
Vision-Other
datasets. On the method side, we structure prior work into three core pillars: (1)
Multimodal Perception and Recognition, which focuses on object understanding and grasp prediction; (2) Cross-Modal Generation, focusing on bidirectional translation between tactile,
vision, and text; and (3)
Multimodal Interaction, emphasizing feedback control and
language-guided manipulation. Furthermore, we summarize representative tactile sensing hardware, review commonly used evaluation metrics and
benchmark settings, and discuss current challenges and promising future directions.
Read the full abstract ▾
All authors Zhixiang Cao, Di Tian, Runwei Guan, Yanzhou Mu, Xiaolou Sun, Shaofeng Liang, Daizong Liu, Tao Huang, Yutao Yue, Henghui Ding, Bin Fang, Alex Zhou, Qing-Long Han, Hui Xiong
Computer Vision · Cross-listed from cs.RO
Efficient Feature-Free Initialization for Monocular Visual-Inertial Systems Using a Feed-Forward 3D Model
Yuantai Zhang, Jiaqi Yang, Huajian Zeng et al. (+5 more)
Fast and reliable initialization is critical for
monocular visual-inertial
navigation systems (VINS), as it establishes the starting conditions for subsequent state estimation. Despite steady progress, most existing methods heavily rely on visual feature correspondences and require 3-4 seconds of sensory data for successful initialization, which limits their applicability and efficiency. With the advent of feed-forward 3D models that can directly predict point clouds from images, we revisit the visual-inertial initialization problem from a concise perspective. In this work, we propose a feature-free initialization framework that leverages up-to-scale point clouds predicted by a feed-forward 3D model, thereby obviating the need for visual feature tracking and estimation. This design substantially reduces system complexity and improves the reliability of initialization. Experiments on public
datasets demonstrate that the proposed feature-free initialization method achieves the highest success rate, exceeding 90%, and significantly reduces the data duration required for successful initialization, typically to under 1.2 s. We further validate our method on a self-collected
dataset covering various indoor and outdoor scenarios, demonstrating robust performance, particularly in visually degraded environments where existing methods often fail. The code and
dataset are available at https://github.com/Yuantai-Z/FF-VIO-Init.
Read the full abstract ▾
All authors Yuantai Zhang, Jiaqi Yang, Huajian Zeng, Changhao Chen, Haoang Li, Liang Li, Dezhen Song, Xingxing Zuo
Computer Vision
Yiren Song, Wangzi Yao, Haofan Wang et al. (+1 more)
Video style transfer aims to render
videos in a target artistic style while preserving content, structure, and motion. While image stylization has advanced rapidly,
video stylization remains challenging due to temporal inconsistency. Most existing methods stylize frames or keyframes and enforce consistency via heuristic temporal propagation, which is brittle under occlusions, disocclusions, and long-term motion, leading to drift and flickering artifacts. We argue that a fundamental bottleneck lies in the lack of large-scale triplet data and a principled training paradigm that jointly models and disentangles style, content, and motion.To address this, we introduce VISTA-1000, a synthetic
dataset with 1,000 styles and motion-aligned triplets of style reference, clean
video, and stylized
video, and propose a
diffusion-
transformer-based in-context
video style transfer framework with a lightweight style adapter for robust style extraction. Extensive experiments demonstrate SOTA performance in style fidelity, temporal consistency, and content preservation.
Read the full abstract ▾
All authors Yiren Song, Wangzi Yao, Haofan Wang, Mike Zheng Shou
Computer Vision
SpecSem-Net: Integrating Spectral and Semantic Features for Robust AI-generated Video Detection
Zixi Wei, Huixuaun Zhang and Xiaojun Wan
The remarkable visual fidelity of recent commercial
video generative models, such as Sora and Veo, renders robust AI-generated
video detection increasingly essential to prevent synthetic content from being indistinguishable from real
videos and exploited for disinformation. However, existing detectors often fail due to an over-reliance on increasingly realistic semantic features, neglecting subtle spectral artifacts. In this paper, we propose SpecSem-Net, the first framework to introduce a semantic-guided spectral denoising mechanism specifically for high-fidelity AI-generated
video detection. Specifically, we design a spectral module to extract high-frequency features via
Fourier-Transform based filtering. Furthermore, to reduce misjudgments arising from spectral noise, we employ a Gated Merging Mechanism to adaptively fuse semantic context, effectively mitigating spectral noise. Additionally, to evaluate detector performance on the latest top-tier generative models, we construct a comprehensive
benchmark comprising 5 SOTA commercial generators. Extensive experiments demonstrate that SpecSem-Net outperforms existing methods, achieving accuracies of 87.25% and 95.59% on our
benchmark and public
datasets, respectively.
Read the full abstract ▾
Computer Vision
Attention Hijacking: Response Manipulation Across Queries in Vision-Language Models
Zhiqiang Wang, Dongrui Liu, Yan Li et al. (+4 more)
Existing
adversarial attacks on
vision-
language models (VLMs) can steer model outputs toward attacker-specified target responses, but their effectiveness often degrades when the same perturbed input is paired with different textual queries. This paper studies cross-query response manipulation, where a single
adversarial example is expected to remain effective across diverse user queries. We first analyze the limitations of existing attacks and find that successful transfer is closely associated with preserving an image-dominant
attention pattern during response generation. Motivated by the observation, we propose
Attention Hijacking, a novel
adversarial attack that explicitly steers internal
attention distributions toward a persistent image-dominant pattern. By amplifying the influence of visual
tokens on target response
tokens while suppressing the competing influence of textual
tokens, our method reduces the dependence of the manipulated output on the specific wording of the query. Extensive experiments on widely used VLMs show that
Attention Hijacking substantially improves cross-query transferability across diverse target responses and unseen queries. The method also extends effectively to multiple attack scenarios, offering new insights into the role of
attention stability in transferable response manipulation for VLMs.
Read the full abstract ▾
All authors Zhiqiang Wang, Dongrui Liu, Yan Li, Zonghao Ying, Wei Xue, Wenhan Luo, Yike Guo
Computer Vision
StyleText: A Large-Scale Dataset and Benchmark for Stylized Scene Text Inpainting
Aleksandr Simonyan and Nipun Jindal
We present StyleText, a large-scale
dataset and
benchmark for localized scene-text inpainting with style preservation. StyleText contains 28,518 image-mask-
prompt triplets grouped into 9,932 scene families, enabling controlled evaluation of text legibility and visual consistency under shared scene context. We construct the
dataset with an automated pipeline that combines LLM
prompt templating, Flux-based source generation with key-value (KV) cache injection, OCR-based semantic filtering, polygon mask extraction, and mask-conditioned FluxFill augmentation. We define a reproducible evaluation protocol using normalized OCR metrics (word accuracy and character error rate) and CLIP image-image similarity with explicit preprocessing. A FluxFill+LoRA baseline trained on StyleText improves OCR accuracy substantially over initialization while maintaining scene style consistency, establishing a strong reference point for future comparisons.
Read the full abstract ▾
Computer Vision
LongDPM: Overlap-Aware 4D Reconstruction from Long Monocular Videos
Chenyi Xu, Yihao Wu, Liqi Yan et al. (+4 more)
Recovering a dynamic 3D scene from a long
monocular video is crucial for dense geometry, camera motion, and temporal correspondence to remain consistent in a shared coordinate system. Existing methods face two key challenges: (1) feed-forward reconstruction models provide accurate local predictions but are limited to short clips, and (2) long-range trackers preserve correspondences without producing dense sequence-level reconstruction. This paper presents LongDPM, a novel overlap-aware framework for scalable long-range
monocular dynamic reconstruction. First, LongDPM processes long
videos in overlapping chunks, keeping inference
memory bounded by the chunk length. Second, it connects chunk-local coordinate systems through confidence-weighted registration with static-aware overlap abstraction. Third, it associates dynamic identities across chunk boundaries and fuses matched trajectories to recover coherent long-range 3D motion. Experimental results demonstrate that LongDPM achieves superior long-range reconstruction and tracking performance, reducing dense tracking EPE over V-DPM on PointOdyssey, Kubric-F, and Kubric-G, while obtaining the best TUM-dynamics ATE for camera pose estimation.
Read the full abstract ▾
All authors Chenyi Xu, Yihao Wu, Liqi Yan, Chao Yang, Jianhui Zhang, Fangli Guan, Pan Li
Computer Vision
HierEdit: Region-Aware Hierarchical Diffusion for Efficient High-Resolution Editing
Yuyao Zhang, Alexander Huang-Menders and Yu-Wing Tai
High-resolution image editing is essential for professional and creative applications, yet existing
multimodal diffusion-based editors remain computationally inefficient and constrained to relatively low resolutions. Current approaches redundantly process the entire image canvas or rely on large-scale high-resolution
datasets, resulting in substantial training and inference costs. We introduce HierEdit, a region-aware hierarchical
diffusion framework designed for efficient and scalable high-resolution image editing. Our method first performs edits on a low-resolution proxy using an off-the-shelf editing model to generate a reference and to localize the modified regions. A hierarchical local-window
diffusion model (Local-Window MMDiT) that refines only edited regions within the original high-res image, while reusing the unaltered regions as conditioning inputs. The low-resolution proxy further provides structural guidance and intermediate denoising supervision (Inference Acceleration) , ensuring consistent global semantics and stable generation without the need for full-resolution
attention computation. This targeted and hierarchical design enables fast, high-fidelity editing of images up to 4K resolution without any specialized high-resolution training data. Extensive experiments demonstrate that HierEdit achieves competitive visual quality on commodity-resolution
datasets while significantly accelerating inference and extending seamlessly to ultra-high-resolution 4K editing. Please check our {\href{https://peteryyzhang.github.io/HierEdit-page/}{Project Page}}.
Read the full abstract ▾
Computer Vision
LISA: Language-guided Interference-aware Spatial-Frequency Attention for Driver Gaze Estimation
Jun Ma, Zhenye Yang, Ruichen Zhou et al. (+3 more)
Driver gaze estimation serves as a fundamental metric for evaluating driver attentiveness in modern monitoring systems. Beyond being vulnerable to sudden lighting changes and sensor noise, spatial-domain models struggle to disentangle authentic gaze cues from irrelevant visual attributes. In this paper, we propose LISA, a
Language-guided Interference-aware Spatial-Frequency
Attention framework that combines frequency-domain priors with
vision-
language knowledge. Observing that the amplitude spectrum remains relatively stable even under spatial perturbations, we design a dual-domain fusion mechanism. It integrates stable low-frequency semantics into high-frequency details, employing spatial
attention to precisely target ocular regions. To reduce semantic ambiguity, we also introduce a training-time disentanglement strategy. Using a frozen CLIP encoder and orthogonal regularization, we explicitly separate gaze features from appearance interference. Experiments on two
benchmarks show that LISA achieves state-of-the-art performance, with significantly improved robustness against occlusions and lighting variations. The code repository is available at https://github.com/Mason-bupt/LISA.
Read the full abstract ▾
All authors Jun Ma, Zhenye Yang, Ruichen Zhou, Pei Zhang, Huan Li, Jinpeng Chen
Computer Vision
HyperVision: A Channel-Adaptive Ground-Based Hyperspectral Vision Pre-trained Backbone
Guanyiman Fu, Jingtao Li, Zihang Cheng et al. (+8 more)
While hyperspectral imaging provides rich spatial-spectral information across hundreds of narrow wavelength bands for precise material identification, ground-based hyperspectral pre-trained backbones remain absent, constrained by varying spectral configurations across sensors, limited annotations and heterogeneous labeling schemes, and the limited scale and scene diversity of existing
datasets. To address these challenges and enable universal perception, we propose HyperVision, the first ground-based hyperspectral pre-trained backbone. First, to handle varying spectral configurations, HyperVision adopts a channel-adaptive dynamic
embedding mechanism to map heterogeneous inputs into a unified
token space. Second, we develop an unsupervised representation learning framework. Specifically, to address limited annotations and heterogeneous labeling schemes, a multi-source pseudo-labeling method is introduced to fuse spatial structures from SAM2 and fine-grained spectral material information from HyperFree. Furthermore, to enrich scene diversity and compensate for limited
dataset scale, a cross-modal knowledge
distillation mechanism is utilized to transfer rich semantic representations from a pre-trained RGB
vision model to our backbone. Pre-trained on a collection of 15k images from 26 diverse ground-based
datasets, HyperVision demonstrates exceptional generalization. Requiring only efficient head-only adaptation without adjusting backbone parameters, it outperforms state-of-the-art task-specific methods across three downstream tasks under varying sensor configurations, yielding up to a 16.3% relative improvement in hyperspectral semantic
segmentation $\mathrm{Acc}_{\mathrm{M}}$, a 2.1% relative gain in object tracking AUC, and a 35.5% reduction in salient object detection MAE. The source code and pre-trained models are available at https://github.com/lronkitty/HyperVision.
Read the full abstract ▾
All authors Guanyiman Fu, Jingtao Li, Zihang Cheng, Zhuanfeng Li, Diqi Chen, Yan Xu, Xiangyu Liu, Fengchao Xiong, Jianfeng Lu, Chengrong Chen, Jun Zhou
Computer Vision
CLAP: Contrastive Latent-space Prompt Optimization for End-to-end Autonomous Driving
Ruiyang Zhu, Yuehan He, Boyuan Zheng et al. (+4 more)
End-to-end autonomous driving systems powered by
Vision-
Language-Action (VLA) models achieve strong performance on common driving scenarios, yet remain brittle in rare but safety-critical long-tail situations such as active construction zones and complex yielding geometries. In this paper, we present a method that addresses the long-tail challenging scenes beyond data scaling and model training. We introduce CLAP (Contrastive Latent-space
Prompt optimization), a location-aware adaptation framework that augments a frozen VLA driving model with per-roadblock soft
prompts, optimized from crowdsourced data and retrieved on demand via Vehicle-to-Everything (V2X) communication. Our approach rests on two observations from VLAs' latent space: (i) at the VLA's hidden-state layer, scenarios from the same roadblock cluster tightly and occupy compact regions of the latent space; and (ii) within a single roadblock, long-tail and normal frames are heavily intermixed in the latent representation, making it difficult to improve one without disturbing the other. CLAP addresses this via a two-stage pipeline: supervised contrastive learning to discover a roadblock-specific hard-scene direction, followed by directionally regularized
prompt optimization that selectively improves challenging frames while preserving normal frame performance. On the NAVSIM
benchmark with various state-of-the-art VLA backbones, CLAP reduces challenging scenario planning error by 24% with no regression on normal frames, significantly improving planning performance.
Read the full abstract ▾
All authors Ruiyang Zhu, Yuehan He, Boyuan Zheng, Zesen Zhao, Ahmad Chalhoub, Qingzhao Zhang, Z. Morley Mao
Computer Vision
Beyond Detection: A Structure-Aware Framework for Scene Text Tracking
Chenmin Yu, Liu Yu, Daiqing Wu et al. (+3 more)
Modern visual object trackers show impressive results on general targets, yet their performance drops substantially when dealing with scene text. Although currently underexplored, tracking text in
videos is essential for dynamic text manipulations such as
segmentation, removal, and editing. To fill this gap, this paper formalizes this specific task as Scene Text Tracking and presents the first systematic work for it. We identify three primary challenges in this task: 1) severe
geometric distortions from perspective shifts, 2) high visual ambiguity across different instances, and 3) high sensitivity to fine-grained structural details. To address these issues, we propose SymTrack, a unified detection-free framework with synergistic dual-branch design. It integrates a Cross-Expert Calibration mechanism to reduce semantic bias, along with a Predictive
Token Rectification mechanism to correct structural imbalances, complemented by an Adaptive Inference Engine that stabilizes predictions under motion constraints. Considering the lack of dedicated
benchmarks for this task, we utilize three
datasets from
video text spotting to construct a
benchmark with high-quality annotations. Extensive experiments demonstrate that SymTrack sets the new state-of-the-art on all three
benchmarks, outperforming previous best trackers by up to 11.97% AUC on $ \text{BOVText}_{\text{SOT}} $. Overall, our work promotes efficient and thorough text tracking, paving the way toward more generalized
video text manipulation.
Read the full abstract ▾
All authors Chenmin Yu, Liu Yu, Daiqing Wu, Gengluo Li, Zeyu Chen, Yu Zhou
Computer Vision · Cross-listed from cs.AI
Is VLA Reasoning Faithful? Probing Safety of Chain-of-Causation in Autonomous Driving Models
Nicanor Mayumu, Xiaoheng Deng and Patrick Mukala
We present the first systematic study of faithfulness in
Vision-
Language-Action (VLA) driving models, analyzing 300 Alpamayo-R1-10B inferences across 100 diverse PhysicalAI-AV scenarios. Our main finding is that output natural-
language rationales with trajectories may be significantly unfaithful: (i) overall
reasoning fidelity is only 42.5%, with Chain-of-Causation matching scene reality less than half the time; (ii) 94 missed pedestrians in one-third of pedestrian-relevant scenes; (iii) 97.7% trajectory fragility under mild visual perturbations; and (iv) only 48.3% mean
reasoning-action consistency, with 53.3% of inferences exhibiting low consistency, including 37.9% of stop-claimed cases where the model continues instead. We formalize faithfulness information-theoretically, define entity and action fidelity with verification criteria, and outline a four-component safety architecture aligned with these results.
Read the full abstract ▾