Computer Vision
COBICount: Separating Object and Background Responses for Remote Sensing Object Counting Without Training on Target Data
Junjing Zheng, Zhiyi Zhou, Ningrui Yang et al. (+1 more)
Remote sensing object counting estimates how many buildings, vehicles, or ships appear in overhead images. Most supervised counters predict a density map, whose sum gives the object count, and assume similar categories, sizes, and backgrounds. Applying them across regions, sensors, or categories often requires target data or further training, which may be costly or unavailable. We study source-only counting. Training for the counting task and model selection use one group of images that shares an object category and similar imaging conditions, with one point marking each object. Target images and information remain unavailable until the model is fixed. This reduces data preparation but makes transfer harder. A model trained on one source may place high density values, called responses, on real objects and repeated background structures. Road edges, parking grids, roof boundaries, and water boundaries may then be counted as objects, creating candidate origin ambiguity. COBICount separates response generation, acceptance, and background suppression. Candidate Evidence (CE) generates possible responses. Candidate Acceptance (CA) keeps compact responses centered on objects. Bias Isolation (BI) reduces responses associated with repeated background structures. Their outputs form the final density map. Trained on RSOC Building and evaluated directly on DOTA Large Vehicle, Small Vehicle, and Ship, COBICount achieves the lowest mean absolute error (MAE) averaged over the target domains among the compared methods, 174.132. It uses 5.07 million parameters and 17.41 billion floating point operations for a 512x512 input. COBICount improves transfer without target data or training for each target. The code will be available at: https://github.com/yixuxi22/COBICount.
Read the full abstract ▾
All authors Junjing Zheng, Zhiyi Zhou, Ningrui Yang, Hongying Meng
Computer Vision
Rethinking Multi-Image Re-Representation in Multi-Image Understanding
Gengyuan Zhang, Xiao Han, Xinyu Xie et al. (+2 more)
Multi-image understanding requires MLLMs not only to recognise the content of individual images, but also to organise visual evidence distributed across them. We study this problem through multi-image re-representation, viewing prompted Chain-of-Thought
reasoning and agentic visual tool use as different ways of re-organising visual evidence during
reasoning. We introduce Mosaic, a general-purpose multi-image visual harness that enables an MLLM to actively construct visual intermediates with ten composable image operations. We compare five re-representation settings on existing multi-image
benchmarks and on MosaicBench, a new grounding-focused
benchmark for fine-grained multi-image understanding. Our experiments show that the relative benefits of textual and visual re-representation are strongly task-dependent. Visual re-representation is particularly effective for tasks requiring precise visual evidence, including hypothesis testing, precision comparison, and orientation-sensitive
reasoning, while tasks dominated by higher-level semantic content show smaller or less consistent gains. Building on this finding, we train MosaicAgent-8B to use Mosaic with reinforcement learning using only accuracy and format rewards. Without demonstration trajectories or rewards for specific tool-use, the
agent learns to compose visual operations over multiple steps and exhibits diverse problem-solving patterns unpromptedly. Code and data will be released at https://github.com/gengyuanmax/Mosaic.
Read the full abstract ▾
All authors Gengyuan Zhang, Xiao Han, Xinyu Xie, Tong Liu, Volker Tresp
Computer Vision
TexTailor: Texture-Preserving Video Virtual Try-On via Adaptive Garment Conditioning
Zijing Qin, Jun Zhou, Ruicheng Zhang et al. (+5 more)
Video virtual try-on has attracted increasing
attention due to its broad potential in digital fashion and intelligent e-commerce. However, existing methods primarily focus on low-resolution settings and still face substantial challenges when extended to high-resolution scenarios. These limitations can be attributed to two main factors: (1) the insufficient utilization of rich garment reference information, and (2) the lack of explicit positional modeling between garment and
video representations during cross-modal interaction, which weakens fine-grained local correspondence. To address these issues, we propose TexTailor, a high-fidelity
video virtual try-on framework built upon a pretrained
video Diffusion Transformer. Specifically, we introduce a timestep-adaptive modulation mechanism to dynamically adjust garment visual representations throughout denoising. We further develop a frame-aligned positional encoding strategy to strengthen garment-to-
video correspondence, together with a multi-source injection design that reduces interference among heterogeneous conditions. Extensive experiments on multiple
video virtual try-on
benchmarks, including the high-resolution Eevee
dataset, demonstrate that TexTailor achieves competitive performance in garment detail preservation, temporal consistency, and overall
video quality.
Read the full abstract ▾
All authors Zijing Qin, Jun Zhou, Ruicheng Zhang, Jiaqi Hou, Zunnan Xu, Ronghui Li, Zhenyu Xie, Xiu Li
Computer Vision · Cross-listed from cs.RO
MotionWeave: Learning Motion-Centered Future Dynamics for Vision-Language-Action Policies
Jingqiu Wang and Yan Wang
Vision-
Language-Action (VLA) models have recently incorporated world models to provide richer dynamic supervision beyond sparse action labels. However, explicitly predicting future images or
videos may include control-irrelevant appearance, while guidance derived from holistic future visual representations and shared global action features may fail to establish timestep-specific correspondence between actions and local visual changes. To address this issue, we propose MotionWeave, a motion-centric future-dynamics framework for action-chunk prediction with two modules: the Action-Induced Motion Grounder (AIMG) and the Horizon Residual Composer (HRC). Specifically, AIMG conditions on action and proprioceptive representations to construct horizon-specific queries that localize interaction regions associated with each future action timestep from current visual
tokens. HRC extracts differences between interaction representations at adjacent horizons, encodes them as temporal motion cues, and injects them into action
tokens through a gated residual. During training,
robot-arm masks rendered from future frames are used to construct KL-based motion-grounding supervision, while inference uses only the current observation. On six MetaWorld tasks, MotionWeave achieves a 75.3% average success rate, an absolute gain of 8.6% over {\pi}0 (66.7%), especially on sustained-interaction tasks. Our code is available at https://github.com/autu-mn/MotionWeave.
Read the full abstract ▾
Computer Vision
Rethinking Generative Image Compression at Extremely Low Bitrates
Tianyu Zhang, Zhaoyang Jia, Houqiang Li et al. (+1 more)
Generative image compression produces visually plausible reconstructions at low bitrates, yet their behavior as the rate approaches zero remains largely unexplored. When pushed below normal operating rates, representative codecs undergo semantic collapse: rather than gracefully losing source-specific detail, they produce malformed or unrecognizable content. Our analysis identifies two factors. As the bitrate decreases, reconstruction losses increasingly conflict with semantic objectives on gradients and visual results, while pixel-space and reconstruction-oriented VAE
diffusion models become less efficient on semantic preservation. Guided by these findings, we introduce RAE-CoD, a compression-oriented
diffusion (CoD) built in a representation autoencoder (RAE) space with direct
alignment between compressed and source representations, preserving recognizable, naturally structured content for a $256\times256$ image with as few as 16 bits. We evaluate this framework using five
vision foundation models (VFM) and a blinded
vision-
language model protocol. On MSCOCO-30K, RAE-CoD stands out from all evaluation. At 0.001-0.008 bpp, it reduces relative VFM feature MSE and Fréchet Distance ratio by at least 25.7% and 69.1% over the best competitors. Meanwhile, semantic recognizability and quality of the reconstructions remain nearly constant while source consistency falls smoothly, replacing abrupt semantic collapse with a graceful transition toward unconditional generation. Code will be released at https://github.com/LuizScarlet/RAE-CoD.
Read the full abstract ▾
All authors Tianyu Zhang, Zhaoyang Jia, Houqiang Li, Dong Liu
Computer Vision
BMASH: Ball-Motion-Aware Soccer Header Spotting
Ahmed Endris Hasen, Muhammad Shahzad Khan, Nikolaos Passalis et al. (+1 more)
Recent advances in computer
vision have made broadcast sports
videos increasingly useful for event analysis, performance assessment, and player-safety applications. In soccer, however, header spotting remains a challenging problem due to the subtle and short-lived nature of header events. This paper focuses on soccer header spotting: identifying moments in broadcast
videos where the ball contacts a player's head. We first adapt and evaluate
Video Swin as a strong action-recognition baseline for this task, and then introduce BMASH, a ball-motion-aware fusion framework that integrates detector-derived ball features. BMASH combines
Video Swin action representations with ball-presence and motion features from frame-level soccer-ball detection, integrating player-action context with ball dynamics to distinguish headers from visually similar events. We evaluate BMASH using game-level splits with separate test matches and rotating validation folds, considering both centered-window classification and continuous full-
video spotting. Results show that
Video Swin provides a strong baseline for header spotting, while BMASH improves clip-level AP and ROC-AUC over the corresponding
Video Swin baseline. In continuous full-
video spotting, BMASH achieves a comparable event-level F1-performance with a different precision–recall trade-off.
Read the full abstract ▾
All authors Ahmed Endris Hasen, Muhammad Shahzad Khan, Nikolaos Passalis, Jenni Raitoharju
Computer Vision
MegaAvatar: Controllable Talking Avatar Generation
Junyao Gao, Sibo Liu, Weidong Zhang et al. (+2 more)
This report presents MegaAvatar, a controllable talking avatar generation framework built on top of the Wan2.2-TI2V-5B model. Compared with previous talking-avatar methods that mainly rely on audio or reference-image conditioning, we introduce additional SMPL-X-derived 3D guidance, enabling global control over body pose and head motion. Specifically, we render the driving SMPL-X sequence into dense mesh frames and encode them with a lightweight 3D convolutional encoder, whose outputs are injected into the latent
tokens to provide overall motion control. Furthermore, we extend Wan2.2-TI2V-5B with additional audio and face cross-
attention modules to enable fine-grained expression control and preserve the input identity, respectively. In addition, we implement an audio-to-SMPL-X model to predict an SMPL-X sequence conditioned on the reference image and input audio, allowing MegaAvatar to support audio-driven inference without user-provided SMPL-X frames. Experiments show that MegaAvatar achieves high-quality talking avatar generation with controllable body and head motion,
speech-synchronized facial expressions, and consistent identity preservation. MegaAvatar also supports inference with flexible resolutions and
video lengths. Codes,
dataset, models will be avaliable in https://github.com/Jeoyal/MegaAvatar
Read the full abstract ▾
All authors Junyao Gao, Sibo Liu, Weidong Zhang, Cairong Zhao, Jun Zhang
Computer Vision
PLRS-IC: A Dual-Calibration Framework for Chest X-Ray Vision-Language Alignment
Qixing Zhao and Jinpeng Li
Fine-grained
vision-
language alignment in chest radiography enables zero-shot classification, grounding, and segmentation without task-specific annotations. However, this
alignment is fundamentally hindered by two intertwined sources of ambiguity: projection-induced visual mismatch and patient-agnostic semantic overlap. First, at the local feature level, frontal and lateral radiographs exhibit distinct appearances for the same clinical finding, rendering a shared patch-text similarity geometry inherently suboptimal. Compounding this visual ambiguity is a semantic mismatch during global contrastive optimization, where instance-level objectives penalize cross-patient pairs as strict negatives even when they share identical positive clinical concepts. To address this dual ambiguity, we propose PLRS-IC, a unified dual-calibration framework for chest X-ray representation learning. At the local
alignment stage, Projection-Conditioned Low-Rank Residual Similarity (PLRS) dynamically adapts patch-text matching to projection-specific
manifolds using a bounded, parameter-efficient low-rank residual. At the global optimization stage, Information-Content-Calibrated Soft False-Negative Suppression (IC-SFNS) leverages a corpus-derived information-theoretic prior to soften the penalty of semantically overlapping negatives without altering original contrastive assignments. Extensive experiments across nine public zero-shot
benchmark settings demonstrate that our framework yields consistent improvements in classification, grounding, and segmentation, validating the necessity of dual-calibration in medical
vision-
language pre-training.
Read the full abstract ▾
Computer Vision
Universal Cross-Prompt Adversarial Attacks on Promptable Concept Segmentation
Ziqi Zhou, Yifan Hu, Yufei Song et al. (+5 more)
The Segment Anything Model (SAM) achieves remarkable performance in visual segmentation. The latest SAM3 extends promptable segmentation to concept-level prediction, broadening the scope of segmentation foundation models. While recent works reveal that SAM and SAM2 are vulnerable to
adversarial examples, the robustness of SAM3 under the concept segmentation paradigm remains unexplored. In addition, existing
adversarial attacks on SAM-series models exhibit limited cross-
prompt transferability. To this end, we propose AdvPCS, a universal cross-
prompt adversarial attack for Promptable Concept Segmentation (PCS), including a min-max
prompt optimization strategy, a global-local perception deception attack, and a temporal transition deviation attack. Specifically, we first identify the hardest-to-attack
prompts via min-max bilevel optimization. In the inner maximization, we enhance diversity over candidate point, box, and text
prompts. In the outer minimization, we select
prompts with the highest responses based on the confidence scores output by the detector. Given the selected
prompts, we apply the perception deception attack to minimize both global and local existence probabilities under joint prompting and employ the temporal
memory misalignment attack to maximize inter-frame semantic inconsistency and corrupt
memory pointers. Extensive experiments on four
benchmark datasets show that a single universal
adversarial perturbation (UAP) generated by our method generalizes across frames from different
videos and achieves strong attack performance under point, box, and text
prompts. In particular, it reduces the average mIoU of various PCS models on the SA-CO
dataset to below 5% under text
prompts, demonstrating strong attack ability.
Read the full abstract ▾
All authors Ziqi Zhou, Yifan Hu, Yufei Song, Haowen Jiang, Xianlong Wang, Shengshan Hu, Dezhong Yao, Leo Yu Zhang
Computer Vision · Cross-listed from cs.RO
The Planning Limits of Latent World Models
Ali Alrasheed, Basim Azam and Naveed Akhtar
World models offer a promising way to help
robots understand how the physical world evolves and plan complex behaviours through imagination. Yet existing studies mainly demonstrate what these models can accomplish, leaving unclear when their predictions remain useful for planning and where they fail. We study this question using action-conditioned predictors built on five frozen self-supervised visual backbones: V-JEPA 2, V-JEPA 2.1, VideoMAEv2, VideoPrism, and DINOv2. We use frozen backbones to test representations intended to transfer across environments. We evaluate these models on diverse Meta-World manipulation tasks and real-
robot interactions from BridgeData V2. We find that a world model guides action selection reliably only when the goal lies within, or slightly beyond, the trajectory it imagines during planning. With five-step rollouts, the length the predictor was trained on, the world model ranks actions reliably only for targets five to ten control steps ahead, whereas task goals lie 16 to 53 steps away. Neither an 81-fold larger predictor nor longer-rollout training extends this range; the encoder affects both range and closed-loop success, with V-JEPA 2.1 performing most consistently. More fundamentally, the limit persists under perfect prediction: using the real simulator, success falls from 92% to 41% as the target moves from five to twenty steps ahead of a five-step rollout. Planning therefore requires either longer imagined trajectories or closer subgoals. For distant goals, pure imagination succeeds in 23% of episodes, planning with feedback (MPC) raises success to 30%, imagining as far as the goal to 47%, and nearby expert subgoals to 76%. Used within its plannable range, a world model can also improve a
vision-
language-action (VLA) policy: choosing among eight actions the VLA proposes raises its success from 65% to 77% across 16 different tasks.
Read the full abstract ▾
Computer Vision
Emergent Multi-View Geometry Through Self-Distillation
David Nordström, Thibaut Loiseau, Vincent Lepetit et al. (+3 more)
Over a century ago, Henri Poincaré argued that a motionless observer cannot acquire the notion of space. Yet, most visual representation learning methods operate on individual images, while those that leverage multiple views rely on RGB reconstruction, entangling geometry with appearance. We propose Poincar3, a self-supervised method that learns representations from multiple views through self-
distillation instead of RGB reconstruction. We combine masked patch and image-level
distillation with a teacher that observes additional views, enabling training from scratch without explicit 3D supervision. Poincar3 outperforms both previous single and multi-view self-supervised approaches such as DINOv3, MuM, and Muskie on correspondence estimation, camera pose estimation, and 3D reconstruction. Using a lightweight Poincaré adapter, we also find that our learned features encode camera motion more accurately than existing self-supervised representations.
Read the full abstract ▾
All authors David Nordström, Thibaut Loiseau, Vincent Lepetit, Michael Felsberg, Guillaume Bourmaud, Fredrik Kahl
Computer Vision
DC-SAE: Deep Compression Semantic Autoencoder for Faster Diffusion Convergence
Xu Huang, Ye Huang, Zijun Liao et al. (+6 more)
High-compression tokenizers are essential for scaling latent image generative models. However, aggressive compression creates a fundamental tradeoff between reconstruction fidelity and generation efficiency: high compression image encoder always increases the learning difficulty of
diffusion training, resulting in slow model convergence. Recent representation autoencoders speed up the
diffusion training by improving the latent feature's expressive capability by replacing VAE encoders with pretrained semantic encoders, yet they are typically limited to moderate compression and lose pixel-level details necessary for faithful reconstruction. To achieve both high compression and fast
diffusion training, we propose DC-SAE, a Decoupled Compact Semantic Autoencoder designed for high-compression image generation with accelerated
diffusion model convergence. DC-SAE consists of two key components: (1) a macro-level architecture design that leverages semantic encoders to enable higher compression ratios, and (2) a pixel-level encoder that preserves low-level details, ensuring high-fidelity image reconstruction. We empirically demonstrate that DC-SAE performs strongly on image generation tasks, achieving both compact latent representations and efficient training dynamics. Specifically, on the ImageNet
dataset with $512 \times 512$ resolution, DC-SAE achieves $32\times$ spatial compression, with 29.79 PSNR and 3.37 gFID, substantially outperforming the previous state-of-the-art high-compression tokenizer baselines DC-AE by 13.5% and 54.9% on PSNR and gFID, respectively, maintaining comparable throughput and faster
diffusion model training convergence. Beyond class-conditional generation, a $1.6$B-parameter DiT using DC-SAE achieves 0.84 on GenEval and 86.007 on DPG-Bench for text-to-image generation at $1024\times1024$ resolution.
Read the full abstract ▾
All authors Xu Huang, Ye Huang, Zijun Liao, Yuwei Niu, Xiaojie Li, Menghan Zhou, De Wen Soh, Xiaotong Li, Daquan Zhou
Computer Vision
Uruqi: Learning Spatial Cognition from Visual Experience
Shichao Li, Meiqi Wang, Fei Su et al. (+1 more)
Spatial intelligence requires maintaining a coherent understanding of the world as the embodied
agent moves. Like humans, the
agent must use its own motion to interpret changes across observations and update object locations and spatial relations accordingly. Despite spatial post-training having substantially broadened the spatial intelligence of
vision-
language models (VLMs), they still struggle with two atomic spatial capabilities: tracking self-motion and mapping the surrounding world during motion. To address this gap, we provide dense multi-turn supervision over interleaved atomic capabilities within each training episode, mimicking the visual experience of a continuously moving
agent that reasons as it observes. To scale this up, we synthesize 11,738 motif-driven camera trajectories over a broad range of 3D scenes, supporting self-motion tracking, persistent object mapping, and rich spatial operations within each visual experience. By training models to reason over these atomic questions, our URUQI$_{\mathrm{Syn}}$-8B improves accuracy from 15.84% to 47.73% on our Uruqi
benchmark comprising 52k questions across 2.7k episodes. URUQI-SI-Mix-8B further reaches 50.41%, comparable to the 50.08% achieved by GPT-6 Astra. Trained solely on our synthesized data, URUQI$_{\mathrm{Syn}}$-8B achieves an average relative accuracy improvement of 17.13% over its InternVL3-8B backbone across three external spatial
benchmarks. These results highlight continuous visual experience as a scalable source of supervision for developing spatial cognition in VLMs.
Read the full abstract ▾
All authors Shichao Li, Meiqi Wang, Fei Su, Zhicheng Zhao
Computer Vision
Fiber-Resolved Microstructure Quantification from Multi-Shell Diffusion MRI using Detection Transformers
Sebastian Endt, Marcus Wirth, Johannes Reinhold Schlund et al. (+1 more)
Fiber orientation and compartmental microstructure are central to the characterization of white matter tissue in
diffusion MRI, yet existing methods either resolve fiber orientations without quantifying microstructure, or quantify microstructure while assuming a fixed number of compartments and a single fiber direction. Nonparametric approaches that recover both require tensor-valued
diffusion encoding and computationally expensive Monte-Carlo inversion of an ill-posed inverse Laplace transform. We propose to reframe this problem as an object detection-like task, adopting the Detection
Transformer (DETR) architecture to jointly predict mean diffusivity (MD), fractional anisotropy (FA), main fiber direction, and signal fraction for a variable number of compartments per voxel from standard multi-shell
diffusion MRI with linear encoding. Hungarian matching during training resolves permutation invariance across compartments. We introduce mean Average Precision as a reproducible
benchmark metric. Evaluated on synthetic test data with up to five compartments per voxel, our model achieves $R^2=0.95$ for MD, $R^2=0.88$ for FA, and a median angular error of 4.2{\deg}, with performance scaling naturally with compartmental signal fraction.
Read the full abstract ▾
All authors Sebastian Endt, Marcus Wirth, Johannes Reinhold Schlund, Marion Irene Menzel
Computer Vision
Aligning Thoughts with Answers: Probability Rewards to Tame Thinking Drift
Pengzhan Sun, Shiu-hong Kao, Shijie Li et al. (+4 more)
This paper studies thinking–answer consistency in
vision-
language models. We focus on Visual Intention Grounding, where a model infers a target object based on a human intention query and predicts a bounding box. We reveal that previous IoU-based reinforcement learning (RL) frameworks suffer from “thinking drift”, where the model produces a correct bounding box, despite having an incorrect
reasoning process pointing to a different target object. Thus, we propose Rita (ReInforcing Thinking–Answer consistency) as a novel RL paradigm to tame the drift. Specifically, Rita introduces two
reasoning-label-free RL rewards, constructed from the conditional probability of reference answers: a thinking reward and a consistency reward. It also adopts a difficulty-aware data filtering strategy that selects informative easy-to-medium samples for RL using rollout error rate and reward variance. Extensive experiments on EgoIntention and the new RefEgo-Int
benchmarks show that Rita performs consistently superior to the supervised finetuning approaches and vanilla RL-finetuned frameworks.
Read the full abstract ▾
All authors Pengzhan Sun, Shiu-hong Kao, Shijie Li, Yongyi Su, Junbin Xiao, Arjun Reddy Akula, Angela Yao
Computer Vision
MEND: Label-Free Detection, Localisation, and Correction of Latent Hallucination in World Models
Ali J Alrasheed, Aryan Yazdan Parast, Basim Azam et al. (+2 more)
World Models are appearing as the next major frontier in computer
vision. However, their robustness is currently largely unexplored. We identify the phenomenon of
hallucination in latent World Models: given a state and an action, the predicted next latent can decode to a scene that never occurs. Because the prediction is statistically ordinary and is fed back autoregressively by the model, the error is both silent and compounding. We study whether such latent
hallucination can be detected, localised, and corrected at inference time, on a frozen self-supervised world model in the absence of ground-truth error labels. We introduce Masked Empirical-Bayes Neural Denoising (MEND), a single conditional score network trained by denoising score matching on real transitions, whose score field serves three roles: its magnitude detects
hallucination, its per-
token field localises it to specific image patches, and it defines an inference-time correction direction. On two navigation environments MEND detects
hallucination with an AUROC of up to 0.80 without using actions, exceeding a single-Gaussian density baseline while also localising the error (per-
token AUPRC up to 0.87) and correcting it, all from one score field. Our correction reliably reduces single-step latent error and improves predictions. We identify that a part of the error is tangent to the data
manifold, hence, we focus on detection and localisation while highlighting promises of the correction.
Read the full abstract ▾
All authors Ali J Alrasheed, Aryan Yazdan Parast, Basim Azam, James Bailey, Naveed Akhtar
Computer Vision · Cross-listed from cs.RO
Exploiting Vulnerabilities: Universal Adversarial Attacks on Vision-Language-Action Models in Robotics
Songhua Yang, Ziyu Liu, Yuanwei Liu et al. (+5 more)
Recently,
Vision-
Language-Action (VLA) models have revolutionized robotic manipulation by seamlessly integrating visual perception,
language understanding, and action generation in an end-to-end learning framework. However, since these models are designed to interact directly with the physical world and humans, their security is critical, and even small vulnerabilities can lead to catastrophic failures. In this work, we propose the Universal
Adversarial Object, a sphere with optimized surface texture that significantly degrades task success rates when placed within the
robot's field of view. Specifically, our approach introduces a multi-level attack framework that jointly disrupts trajectory planning, task execution, and action control. We validate our method in both simulated and real-world robotic settings. Experimental results demonstrate that the
adversarial object reduces the average task success rates by 31.2%-39.9% for two representative VLA models (Pi0 and RDT), with success rates dropping to near zero in complex scenarios. Index Terms–
Vision-
Language-Action models,
adversarial attack, robotic security, universal
adversarial object
Read the full abstract ▾
All authors Songhua Yang, Ziyu Liu, Yuanwei Liu, Xuetao Li, Xuanye Fei, He Huang, Zheng Wang, Miao Li
Computer Vision
TripleFlow: Training-Free Video Object Removal by Bridging Residual Editing and Native Generation
Songhe Wang, Lifu Wei, Shuolin Xu et al. (+2 more)
Video object removal presents a uniquely difficult editing challenge. Because a removal
prompt specifies only what to erase rather than what to generate, the model must infer and reconstruct a highly specific occluded background entirely from the surrounding context. Existing training-free methods struggle with this because their editing mechanisms act primarily as localized erasers. They fail to actively synthesize the missing background details and often leave behind ghosting artifacts. To solve this, we propose TripleFlow, a training-free framework that tightly couples erasure and generation. It coordinates a source flow, a residual flow, and a synthesis flow throughout the entire process. By reusing a single target prediction, the residual flow isolates and suppresses the object, while the synthesis flow independently reconstructs the occluded background. Crucially, TripleFlow injects this newly synthesized background back into the editing trajectory at every step. This continuous feedback loop ensures that the generated structures actively guide the removal process, achieving seamless completion that is spatiotemporally consistent with the unedited scene. Extensive evaluations across five challenging
benchmarks demonstrate that TripleFlow establishes a new state-of-the-art, significantly outperforming existing baselines in both reconstruction fidelity and temporal consistency.
Read the full abstract ▾
All authors Songhe Wang, Lifu Wei, Shuolin Xu, Charles A. Kamhoua, David Miller
Computer Vision
OP-CAD: On-Policy Clean-Audio Distillation for Robust Audio-Visual Reasoning
Xingming Shui, Dapeng Chen, Bowei Liu et al. (+5 more)
Omni-modal large
language models deployed in real-world environments encounter external noise that can interfere with their perception and understanding of
multimodal inputs. We study their robustness in audio-visual understanding, focusing on question answering under environmental noise and competing
speech. The challenge is to resist acoustic interference while preserving useful audio evidence. On-policy
distillation provides dense teacher feedback on student-generated responses, but uniform
token weighting does not explicitly prioritize positions affected by acoustic interference. We introduce OP-CAD (On-Policy Clean-Audio
Distillation), a curriculum-based privileged self-
distillation framework for robust audio-visual understanding. Training progresses from mild to severe environmental noise and competing
speech, with selective
token-level supervision at each stage. The student generates responses from corrupted audio-visual input, while a frozen teacher uses clean audio and the verified answer to supervise the same response prefixes. To allocate this supervision, OP-CAD compares teacher predictions under clean, corrupted, and visual-only contexts without revealing the answer. These matched comparisons measure sensitivity to audio removal and corruption; a bounded weighting rule emphasizes positions identified by either signal while retaining supervision throughout the response. OP-CAD outperforms the compared methods across all evaluated noise conditions. Paired analyses further show improved preservation of clean-correct answers under strong interference, with no observed aggregate clean-accuracy penalty. These results demonstrate the value of directing clean-teacher supervision toward acoustically sensitive predictions for robust audio-visual
reasoning.
Read the full abstract ▾
All authors Xingming Shui, Dapeng Chen, Bowei Liu, Jingqi Tian, Minfu Li, Kun Yi, Jiapeng Hong, Yansong Tang
Computer Vision
MindWorldBench: Evaluating Mental-State-to-Behavior Reasoning in Image-to-Video Generation
Ruiqi Li, Xuanyi Liu, Sijia Li et al. (+9 more)
Current image-to-
video models achieve visual realism and physical plausibility, but
reasoning about mental states remains unexplored. Actions are driven by belief, desire, and perception, requiring inference beyond explicit instructions. We introduce MindWorldBench to evaluate mental-state-conditioned
video generation. We formalize this as mental-state-to-behavior
reasoning, where models generate actions from a world state and latent variables without explicit action
prompts. MindWorldBench utilizes Zero-Action Prompting and a counterfactual design with 744
prompts to isolate the causal effects of mental states. An automated pipeline evaluates
video quality, commonsense plausibility, and mental-state consistency. Evaluations of 11 models show that despite visual fidelity and physical
reasoning, models fail to align behaviors with latent mental states. We identify a failure mode, termed Omniscient Bias, where models default to the objective world state rather than human's subjective belief. These results demonstrate a disconnect between visual generation and cognitive
reasoning, suggesting a need for explicit mental-state modeling in
video generation systems. Project website: https://richard2049-lee.github.io/MindWorldBench/
Read the full abstract ▾
All authors Ruiqi Li, Xuanyi Liu, Sijia Li, Haofeng Wang, Yuxin Liu, Feng Xie, Songchao Tan, Shiqi Wang, Hanwei Zhu, Yizong Wang, Chuanmin Jia, Siwei Ma
Computer Vision
When Can Text Replace Vision? Structural Bottlenecks in Diagram Reasoning
Yunbei Zhang, Janet Wang, Jihun Hamm et al. (+1 more)
Can structured text replace
vision for diagram
reasoning? A wrong answer after textualization can arise because the representation omits information the question needs, or because the solver fails to use information that is present. We introduce a diagnostic protocol to distinguish these explanations. Using the same solver model and generation settings, we compare three input conditions: the original image, question-blind structure extracted by a
vision-
language model, or gold structure derived from the diagram source. Validity-triggered recovery tests truncation and schema failure, question-relevant fidelity measures preservation of answer-critical structure, and matched edge interventions test the effect of error location. On a reserved holdout of 240 public FlowGen diagrams, evaluated under a frozen protocol, gold structure reaches 87% accuracy while direct
vision and learned text both remain below 30%. The aggregate comparison includes source-derived relation labels that may not be printed in the image and uses different learned and gold graph encodings, so it does not isolate extraction error alone. Retrying only invalid extractions makes nearly every public representation schema-valid yet leaves accuracy essentially unchanged. The public learned-text deficit relative to gold more than doubles with structural difficulty. Question-relevant
topology predicts correctness better than whole-graph
topology. In an exposed intervention study, a single answer-relevant edge edit reduces the primary solver's original-answer accuracy to near zero, while matched irrelevant edits largely preserve it. Supplied structure requires fewer solving
tokens than
vision, but learned acquisition removes this advantage at single use. These comparisons motivate evaluating acquired text by the answer-relevant evidence it preserves and by the solver's ability to use that representation.
Read the full abstract ▾
All authors Yunbei Zhang, Janet Wang, Jihun Hamm, Chandan K Reddy
Computer Vision
Asking the World: Generalist Physical Reasoning through Agentic World Modeling and Probing
Shenxiang Zeng, Chen Yang, Peiyao Chen et al. (+3 more)
Physical
reasoning from
video requires inferring latent physical properties and dynamics beyond direct observation. Direct VLM inference remains unreliable on complex physical tasks without explicit modeling and validation, while predefined tool pipelines rely on task- and domain-specific priors that limit generalization across materials, dynamics, and
reasoning tasks. We introduce Asking the World (ATW), a generalist
agent that constructs and interrogates task-relevant executable worlds through two adaptive stages: World Modeling calibrates a world from
video, while World Probing queries, simulates, and intervenes on it to obtain question-relevant evidence. Rather than prescribing the operations in either stage, ATW determines how to model and probe according to the scene and question. We develop PolyWorld Engine, a lightweight and highly programmable Warp-based multiphysics simulator for constructing and probing worlds with rigid bodies, soft bodies, cloth, ropes, fluids, and their coupled interactions. CEM-based system identification recovers task-relevant dynamics during World Modeling. The resulting world becomes an active workspace for question-directed physical experiments rather than a predetermined downstream tool. We evaluate ATW on CLEVRER, ContPhy, and three real-world scenarios. Using Gemini-3-Flash as its base VLM, ATW achieves 80.82% overall per-question accuracy on CLEVRER, improving direct Gemini-3-Flash by 46.50 points, GPT-5.5 by 13.58 points, and PhysMind by 8.27 points. On ContPhy, it reaches 70.56% overall accuracy, surpassing Gemini-3-Flash by 28.10 points and GPT-5.5 by 3.53 points. Across the three real-world scenarios, ATW achieves 71.67% accuracy, 28.33 points above GPT-5.5. These results establish agentic world modeling and probing as an effective, execution-grounded approach to generalist physical
reasoning.
Read the full abstract ▾
All authors Shenxiang Zeng, Chen Yang, Peiyao Chen, Guohui Zhang, Jiansheng Fan, Chen Wang
Computer Vision
Feature-Aware Token Attack for Compression-Triggered Stealthy Failures in Large Vision-Language Models
Shilinlu Yan, Bowen Chen, Yuechen Zhang et al. (+3 more)
Visual-
token compression improves the efficiency of large
vision-
language models, but can expose failures that full-
token evaluation misses. We study
adversarial images that preserve full-
token correctness yet induce errors after compression, even when both inference paths succeed on the clean image. Creating such failures is challenging because perturbing
token importance can also damage the visual content needed for full-
token inference. We propose Feature-Aware
Token Attack (FATA), which couples
attention suppression with cosine-based feature preservation on a fixed set of salient clean-image
tokens. In the primary LLaVA-1.5-7B setting, FATA uses only
vision-encoder gradients, without access to the deployed compressor,
token budget, or downstream task. Across four visually dependent task subsets and four compressors under a controlled reconstruction protocol, FATA achieves SR = 96.3% full-
token accuracy retention and CBR = 22.1% conditional blinding, compared with 89.8% and 15.7% for CAA. Ablations support the role of both objectives in balancing compressed-path failure against full-
token preservation. FATA also has the lowest measured detection rate among four attacks across three evaluated detectors at a 5% false-positive rate. These findings motivate assessing
adversarial robustness jointly across full-
token and compressed inference.
Read the full abstract ▾
All authors Shilinlu Yan, Bowen Chen, Yuechen Zhang, Zhenhong Zhou, Li Sun, Sen Su
Computer Vision
Uncertainty-Aware Consistency Distillation for Few-Step Video Generation
Lingyu Liu, Yaxiong Wang, Li Zhu et al. (+1 more)
We study few-step
video generation, i.e., distilling a multi-step
video generator, which typically requires tens of sampling steps, incurring substantial latency and compute, into a few-step student. Consistency
distillation is a common recipe, in which a multi-step teacher provides the consistency targets for a few-step student. However, these teacher-guided targets are not equally trustworthy, and the content is harder to learn where it varies rapidly over time, e.g., moving foliage shadows or flowing water. We observe that supervision reliability follows the local difficulty of the content rather than semantic complexity: regions that change little yield consistent endpoint predictions, whereas regions with large temporal variation produce larger discrepancies that coincide with the largest perceptual errors. Motivated by this observation, we propose Uncertainty-Aware Consistency
Distillation (UACD), which reweights consistency supervision at each spatiotemporal region using a local, parameter-free uncertainty estimate. Specifically, we construct two independently perturbed teacher-guided consistency paths, whose student endpoint predictions provide a consensus target; the discrepancy between the student's direct prediction and this target is the uncertainty proxy. We then relax the consistency penalty on high-uncertainty regions through an exponential weight, while keeping the full penalty elsewhere, since the student cannot be expected to match targets that are hard to learn. To preserve perceptual quality under aggressive step reduction, we integrate feature-space
adversarial training with semantic
alignment. With parameter-efficient LoRA adaptation of the 50-step Wan model, our method achieves state-of-the-art 4-step generation on VBench 2.0 (0.556 mean score) and is preferred over competing methods in a user study.
Read the full abstract ▾
All authors Lingyu Liu, Yaxiong Wang, Li Zhu, Zhedong Zheng
Computer Vision
Perceptual Color Difference Modeling Using Machine Learning and Human Similarity Judgments
Elnara Kadyrgali, Muragul Muratbekova, Adilet Yerkin et al. (+6 more)
Accurate assessment of color differences is essential for applications ranging from digital design to quality control. While existing color difference metrics, such as CIEDE2000, aim to approximate human perception, they may still exhibit inconsistencies with perceptual judgments. In this study, we investigate a data-driven approach to color-difference estimation based directly on human evaluations. We collect similarity judgments for 2,000 systematically generated color pairs, each rated by seven observers using a four-point ordinal scale. These judgments are then used to train regression models using different color representations, including RGB channel differences, HSI differences, and COLIBRI fuzzy linguistic categories. Experiments with five regression
algorithms show that the choice of color model has a greater influence on prediction performance than the choice of regression
algorithm. Using COLIBRI features alone, linear regression achieves an R2 of 0.595, outperforming RGB and HSI representations, which achieve R2 values of 0.479 and 0.493, respectively. The best performance is obtained by LightGBM using the combined representation, reaching an R2 of 0.703. The results indicate that human perceptual color differences are better captured when numerical color coordinates are complemented by graded perceptual categories, highlighting the potential of data-driven models for perceptually aligned color-difference estimation.
Read the full abstract ▾
All authors Elnara Kadyrgali, Muragul Muratbekova, Adilet Yerkin, Nuray Toganas, Ayan Igali, Malika Ziyada, Aruzhan Burambekova, Jamaladdin Hasanov, Pakizar Shamoi
Computer Vision
GeoGAT: Bidirectional Temporal Sampling Meets Hierarchical Graph Attention for Global Video Geo-localization
Junchao Cui, Xuanzi Ma, Wenqi Shi et al. (+4 more)
Global
video geo-localization aims to infer the geographic location of a
video worldwide, evaluating performance across four geographic hierarchies: city, state/province, country, and continent. Existing methods typically employ one-way uniform sampling to process
video frames and train independent classifiers for each hierarchy, which leads to the loss of key geographic cues and prediction conflicts between hierarchies, especially for complex multi-shot edited
videos. To address these limitations, we propose GeoGAT, which integrates bidirectional temporal sampling with graph
attention networks (GATs). Specifically, GeoGAT extracts forward and offset-reversed frame sequences to construct complementary spatiotemporal features. These fused features are then fed into a predefined geographical hierarchy graph, where GATs perform structure-aware message passing, while a dual-constraint mechanism prunes predictions to eliminate cross-hierarchy conflicts. We construct GeoGAT10k, comprising 9,720 multi-shot edited
videos from 166 cities worldwide, specifically to
benchmark generalization ability on complex
video structures. Experimental results on CityGuessr68k and GeoGAT10k demonstrate that GeoGAT eliminates hierarchical conflicts entirely and achieves state-of-the-art performance across all four geographic hierarchies. On CityGuessr68k, GeoGAT outperforms the strongest baseline, evaluated under both classification and
retrieval protocols, by 2.6 percentage points at the city level. On the more challenging GeoGAT10k with multi-shot edited
videos, the accuracy improvement exceeds 24 percentage points, validating strong generalization to complex real-world scenarios.
Read the full abstract ▾
All authors Junchao Cui, Xuanzi Ma, Wenqi Shi, Hangyu Li, Biru Zhu, Chong Fu, Xiangyang Luo
Computer Vision
How to Reduce Localization Ambiguity? Geometry-Semantic Constrained BEV Representation Learning for Satellite-Ground Localization
Junming Feng, Panwang Xia, Qiong Wu et al. (+8 more)
Satellite-ground localization estimates the planar position and yaw orientation of a ground camera within a geo-referenced satellite image. Most recent methods map ground and satellite features into a shared bird's-eye-view (BEV) space and establish spatial correspondences. However, insufficient depth constraints can assign one ground feature to different distances along a viewing direction, creating
geometric ambiguity in BEV feature placement. Similar appearances at different locations can also create descriptor matching ambiguity, while existing descriptor learning lacks explicit semantic supervision to distinguish them. We propose GeoSem-BEV, a geometry-semantic constrained BEV representation learning method. Radial depth supervision constrains distance assignment, and vertical height supervision constrains height aggregation. Shared explicit semantic supervision promotes consistent semantic predictions across views and helps distinguish locations with similar semantics. These constraints improve feature placement and descriptor discriminability, enhancing state-of-the-art BEV localization models. On VIGOR with unknown orientation, GeoSem-BEV reduces mean orientation error by 37.2% and 38.1% in the cross-area and same-area settings, respectively. The corresponding errors are reduced by 10.8% and 15.6% on DReSS-D. On KITTI-CVL, it reduces same-area mean orientation error by 26.8% under 10 degree orientation noise.
Read the full abstract ▾
All authors Junming Feng, Panwang Xia, Qiong Wu, Xudong Lu, Zeyu Jiao, Kun Lv, Zherong Wu, Yi Wan, Peifeng Ma, Li-Ta Hsu, Zhi Zheng
Computer Vision
Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation
Siyuan Liu, Kanghui Tian, Yue Duan et al. (+4 more)
On-policy
distillation (OPD) improves
reasoning by providing
token-level supervision from a teacher on a student's own trajectories. Existing methods primarily focus on enhancing this teacher-side guidance (e.g., by enriching teacher inputs and refining teacher feedback), yet we find that limited student perception is another critical bottleneck in
multimodal OPD. By providing oracle visual facts, the performance of OPD-trained students can still be substantially improved for both weak and strong teachers. To address this bottleneck, we propose S-OPD, a simple
multimodal on-policy
distillation framework that explicitly strengthens student perceptual learning through two objectives. Specifically, Teacher-calibrated Policy Contrast separates student policies under original and masked images with teacher-based
token-level gating, strengthening the student's reliance on visual evidence during
reasoning. Policy Agreement aligns student policies under original and noise-perturbed images, further improving perceptual robustness to visual noise. Notably, our method can be seamlessly plugged into existing OPD frameworks, requiring no additional data annotations, model parameters or inference operations. Extensive experiments on eight
benchmarks across student scales and
distillation paradigms demonstrate consistent performance improvements, with gains of up to 4.25 points on LogicVista. When combined with existing teacher-side supervision methods, our method can yield further gains. Code is available at https://github.com/Sirilaw/S-OPD.
Read the full abstract ▾
All authors Siyuan Liu, Kanghui Tian, Yue Duan, Yutao He, Shangdong Yang, Jian Zhang, Yinghuan Shi
Computer Vision
GRC-Pose: Generation-Reconstruction Correspondence for Prior-Free 6D Object Pose Tracking
Shiyang Liu, Weiquan Lin, Luping Xiao et al. (+4 more)
Prior-free 6D object pose tracking seeks to recover the trajectory of an unseen object from a single RGB
video without object-specific CAD models, posed reference images, or pose annotations.
Geometric foundation models provide complementary object-centric and scene-centric cues, yet SAM3D CAD is indexed by an arbitrary object-local surface parameterization, whereas reconstructed evidence is expressed in a sequence-specific world frame with partial surface coverage. To exploit this complementarity, we formulate tracking as generation-reconstruction correspondence and introduce GRC-Pose, a correspondence-based framework that combines learned correspondence prediction with robust pose estimation. Concretely, GeoCorr-Matcher estimates weighted object-scene correspondences and per-match uncertainty for each pose candidate. FGH-Solver integrates these matches through multiple robust
geometric estimators and sequence-level posterior inference, while a posterior-gated
memory retains only inlier-supported observations through occlusion and viewpoint change. Extensive evaluation shows that with SAM3D CAD, GRC-Pose achieves state-of-the-art Average Recall and motion retention on HOT3D, improving the latter by 58% over prior art. On classical
benchmarks including YCBInEOAT and LINEMOD, it remains highly competitive.
Read the full abstract ▾
All authors Shiyang Liu, Weiquan Lin, Luping Xiao, Jiadong Tang, Yi Yang, Yu Gao, Xingyu Chen
Computer Vision
Beyond Local Linearity: Scale-Resolved Geometry of Learned Image Encoders
Jakub Szymkowiak, Wojtek Pałubicki and Kamil Adamczewski
Understanding how learned representations respond to finite input changes is important for characterizing their sensitivity, invariances, and robustness. Yet existing
geometric analyses are predominantly local and describe only infinitesimal perturbations. We introduce a scale-resolved statistic that compares an encoder's measured feature displacement with its local linear prediction as the perturbation magnitude increases. Across diverse image encoders, we discover a characteristic plateau-rise-peak-decay profile, which we call the bump. The bump is absent at initialization, emerges early during standard training, and does not form under randomized labels or random-noise inputs. Its shape also varies with the training distribution and robustness objective. These results establish departures from local geometry as a signature of how encoder representations are shaped by learning.
Read the full abstract ▾