Computer Vision · Cross-listed from cs.AI
WorldMAP: Bootstrapping Vision-Language Navigation Trajectory Prediction with Generative World Models
Hongjin Chen, Shangyun Jiang, Tonghua Su et al. (+4 more)
Vision-
language models (VLMs) and generative world models are opening new opportunities for
embodied navigation. VLMs are increasingly used as direct planners or trajectory predictors, while world models support look-ahead
reasoning by imagining future views. Yet predicting a reliable trajectory from a single
egocentric observation remains challenging. Current VLMs often generate unstable trajectories, and world models, though able to synthesize plausible futures, do not directly provide the grounded signals needed for
navigation learning. This raises a central question: how can generated futures be turned into supervision for grounded trajectory prediction? We present WorldMAP, a teacher–student framework that converts world-model-generated futures into persistent semantic-spatial structure and planning-derived supervision. Its world-model-driven teacher builds semantic-spatial
memory from generated
videos, grounds task-relevant targets and obstacles, and produces trajectory pseudo-labels through explicit planning. A lightweight student with a multi-hypothesis trajectory head is then trained to predict
navigation trajectories directly from
vision-
language inputs. On Target-Bench, WorldMAP achieves the best ADE and FDE among compared methods, reducing ADE by 18.0% and FDE by 42.1% relative to the best competing baseline, while lifting a small open-source VLM to DTW performance competitive with proprietary models. More broadly, the results suggest that, in
embodied navigation, the value of world models may lie less in supplying action-ready imagined evidence than in synthesizing structured supervision for
navigation learning.
Read the full abstract ▾
All authors Hongjin Chen, Shangyun Jiang, Tonghua Su, Chen Gao, Xinlei Chen, Yong Li, Zhibo Chen
Computer Vision
Shortcut Learning in Glomerular AI: Adversarial Penalties Hurt, Entropy Helps
Mohammad Daouk, Jan Ulrich Becker, Neeraja Kambham et al. (+3 more)
Stain variability is a pervasive source of distribution shift and potential shortcut learning in renal pathology AI. We ask whether lupus nephritis glomerular lesion classifiers exploit stain as a shortcut, and how to mitigate such bias without stain or site labels. We curate a multi-center, multi-stain
dataset of 9,674 glomerular patches (224$\times$224) from 365 WSIs across three centers and four stains (PAS, H&E, Jones, Trichrome), labeled as proliferative vs. non-proliferative. We evaluate
Bayesian CNN and ViT backbones with Monte Carlo dropout in three settings: (1) stain-only classification; (2) a dual-head model jointly predicting lesion and stain with supervised stain loss; and (3) a dual-head model with label-free stain regularization via
entropy maximization on the stain head. In (1), stain identity is trivially learnable, confirming a strong candidate shortcut. In (2), varying the strength and sign of stain supervision strongly modulates stain performance but leaves lesion metrics essentially unchanged, indicating no measurable stain-driven shortcut learning on this multi-stain, multi-center
dataset, while overly
adversarial stain penalties inflate predictive uncertainty. In (3),
entropy-based regularization holds stain predictions near chance without degrading lesion accuracy or calibration. Overall, a carefully curated multi-stain
dataset can be inherently robust to stain shortcuts, and a
Bayesian dual-head architecture with label-free
entropy regularization offers a simple, deployment-friendly safeguard against potential stain-related drift in glomerular AI.
Read the full abstract ▾
All authors Mohammad Daouk, Jan Ulrich Becker, Neeraja Kambham, Anthony Chang, Hien Van Nguyen, Chandra Mohan
Computer Vision
Generative 3D Gaussian Splatting for Arbitrary-ResolutionAtmospheric Downscaling and Forecasting
Tao Han, Zhibin Wen, Zhenghao Chen et al. (+4 more)
While AI-based numerical weather prediction (NWP) enables rapid
forecasting, generating high-resolution outputs remains computationally demanding due to limited multi-scale adaptability and inefficient data representations. We propose the 3D Gaussian splatting-based scale-aware
vision transformer (GSSA-ViT), a novel framework for arbitrary-resolution
forecasting and flexible downscaling of high-dimensional atmospheric fields. Specifically, latitude-longitude grid points are treated as centers of 3D Gaussians. A generative 3D Gaussian prediction scheme is introduced to estimate key parameters, including covariance, attributes, and opacity, for unseen samples, improving generalization and mitigating overfitting. In addition, a scale-aware
attention module is designed to capture cross-scale dependencies, enabling the model to effectively integrate information across varying downscaling ratios and support continuous resolution adaptation. To our knowledge, this is the first NWP approach that combines generative 3D Gaussian modeling with scale-aware
attention for unified multi-scale prediction. Experiments on ERA5 show that the proposed method accurately forecasts 87 atmospheric variables at arbitrary resolutions, while evaluations on ERA5 and CMIP6 demonstrate its superior performance in downscaling tasks. The proposed framework provides an efficient and scalable solution for high-resolution, multi-scale atmospheric prediction and downscaling. Code is available at: https://github.com/binbin2xs/weather-GS.
Read the full abstract ▾
All authors Tao Han, Zhibin Wen, Zhenghao Chen, Fenghua Lin, Junyu Gao, Song Guo, Lei Bai
Computer Vision
Stitch4D: 4D Urban Reconstruction from Sparse Multi-Location Observations via Spatio-Temporal Interpolation
Hina Kogure, Kei Katsumata, Taiki Miyanishi et al. (+1 more)
Dynamic urban environments are often captured by cameras placed at spatially separated locations with little or no view overlap. However, most existing 4D reconstruction methods assume densely overlapping views and struggle under sparse multi-location observations, producing unstable reconstructions in unobserved intermediate regions. To address this practical yet underexplored setting, we propose Stitch4D, a unified 4D reconstruction framework that compensates for missing spatial coverage in sparsely observed urban environments. Stitch4D synthesizes intermediate bridge views between distant camera locations and jointly optimizes real and synthesized observations in a unified coordinate frame with inter-location consistency constraints. By recovering intermediate spatial coverage before optimization, Stitch4D mitigates
geometric collapse and improves reconstruction stability in sparse regions. To evaluate this setting, we introduce Urban Sparse 4D (U-S4D), a controlled CARLA-based
benchmark for free-viewpoint reconstruction under sparse multi-location configurations. Experiments on U-S4D show that Stitch4D consistently outperforms representative 4D reconstruction baselines in image-quality metrics. These results suggest that recovering intermediate spatial coverage is an effective strategy for stabilizing 4D reconstruction in sparse urban environments. The project page is provided in https://stitch4d-project-page.vercel.app/.
Read the full abstract ▾
All authors Hina Kogure, Kei Katsumata, Taiki Miyanishi, Komei Sugiura
Computer Vision
Tarot-SAM3: Training-free SAM3 for Any Referring Expression Segmentation
Weiming Zhang, Dingwen Xiao, Songyue Guo et al. (+5 more)
Referring Expression
Segmentation (RES) aims to segment image regions described by natural-
language expressions, serving as a bridge between
vision and
language understanding. Existing RES methods, however, rely heavily on large annotated
datasets and are limited to either explicit or implicit expressions, hindering their ability to generalize to any referring expression. Recently, the Segment Anything Model 3 (SAM3) has shown impressive robustness in Promptable Concept
Segmentation. Nonetheless, applying it to RES remains challenging: (1) SAM3 struggles with longer or implicit expressions; (2) naive coupling of SAM3 with a
multimodal large
language model (MLLM) makes the final results overly dependent on the MLLM's
reasoning capability, without enabling refinement of SAM3's
segmentation outputs. To this end, we present Tarot-SAM3, a novel training-free framework that can accurately segment from any referring expression. Specifically, Tarot-SAM3 consists of two key phases. First, the Expression
Reasoning Interpreter (ERI) phase introduces
reasoning-assisted
prompt options to support structured expression parsing and evaluation-aware rephrasing. This transforms arbitrary queries into robust heterogeneous
prompts for generating reliable masks with SAM3. Second, the Mask Self-Refining (MSR) phase selects the best mask across
prompt types and performs self-refinement by leveraging rich feature relationships from DINOv3 to compare discriminative regions among ERI outputs. It then infers region affiliation to the target, thereby correcting over- and under-
segmentation. Extensive experiments demonstrate that Tarot-SAM3 achieves strong performance on both explicit and implicit RES
benchmarks, as well as open-world scenarios. Ablation studies further validate the effectiveness of each phase.
Read the full abstract ▾
All authors Weiming Zhang, Dingwen Xiao, Songyue Guo, Guangyu Xiang, Shiqi Wen, Minwei Zhao, Lei Chen, Lin Wang
Computer Vision
Mitigating Entangled Steering in Large Vision-Language Models for Hallucination Reduction
Yuanhong Zhang, Zhaoyang Wang, Xin Zhang et al. (+2 more)
Large
Vision-
Language Models (LVLMs) have achieved remarkable success across cross-modal tasks but remain hindered by
hallucinations, producing textual outputs inconsistent with visual content. Existing methods mitigate
hallucinations but often alter generation behavior, resulting in shorter outputs and shifted
token distributions, especially in latent space steering approaches. We identify that this issue stems from entangled steering signals, where suppressing
hallucinations inadvertently disrupts the model's intrinsic generation behavior. To address this, we propose MESA, an effective plug-and-play framework that performs controlled and selective latent intervention for
hallucination mitigation. Specifically, MESA targets
hallucination-relevant responses while preserving the model's original
token distribution, enabling effective
hallucination reduction without compromising generation behavior. Extensive experiments across diverse generative and discriminative
benchmarks demonstrate that MESA consistently reduces
hallucinations while better preserving generation behavior, outperforming prior methods across multiple LVLM families.
Read the full abstract ▾
All authors Yuanhong Zhang, Zhaoyang Wang, Xin Zhang, Weizhan Zhang, Joey Tianyi Zhou
Computer Vision
ParkSense: Where Should a Delivery Driver Park? Leveraging Idle AV Compute and Vision-Language Models
Die Hu and Henan Li
Finding parking consumes a disproportionate share of food delivery time, yet no system addresses precise parking-spot selection relative to merchant entrances. We propose ParkSense, a framework that repurposes idle compute during low-risk AV states – queuing at red lights, traffic congestion, parking-lot crawl – to run a
Vision-
Language Model (VLM) on pre-cached satellite and street view imagery, identifying entrances and legal parking zones. We formalize the Delivery-Aware Precision Parking (DAPP) problem, show that a quantized 7B VLM completes inference in 4-8 seconds on HW4-class hardware, and estimate annual per-driver income gains of 3,000-8,000 USD in the U.S. Five open research directions are identified at this unexplored intersection of autonomous driving, computer
vision, and last-mile logistics.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Kuramoto Oscillatory Phase Encoding: Neuro-inspired Synchronization for Improved Learning Efficiency
Mingqing Xiao, Yansen Wang, Dongqi Han et al. (+2 more)
Spatiotemporal neural dynamics and oscillatory synchronization are widely implicated in biological information processing and have been hypothesized to support flexible coordination such as feature binding. By contrast, most deep learning architectures represent and propagate information through activation values, neglecting the joint dynamics of rate and phase. In this work, we introduce Kuramoto oscillatory Phase Encoding (KoPE) as an additional, evolving phase state to
Vision Transformers, incorporating a neuro-inspired synchronization mechanism to advance learning efficiency. We show that KoPE can improve training, parameter, and data efficiency of
vision models through synchronization-enhanced structure learning. Moreover, KoPE benefits tasks requiring structured understanding, including semantic and panoptic
segmentation, representation
alignment with
language, and few-shot abstract visual
reasoning (ARC-AGI). Theoretical analysis and empirical verification further suggest that KoPE can accelerate
attention concentration for learning efficiency. These results indicate that synchronization can serve as a scalable, neuro-inspired mechanism for advancing state-of-the-art neural network models. Code is avaliable at https://github.com/microsoft/Neuro-inspired_Phase_Encoding.
Read the full abstract ▾
All authors Mingqing Xiao, Yansen Wang, Dongqi Han, Caihua Shan, Dongsheng Li
Computer Vision
PanoSAM2: Lightweight Distortion- and Memory-aware Adaptions of SAM2 for 360 Video Object Segmentation
Dingwen Xiao, Weiming Zhang, Shiqi Wen et al. (+1 more)
360
video object
segmentation (360VOS) aims to predict temporally-consistent masks in 360
videos, offering full-scene coverage, benefiting applications, such as VR/AR and
embodied AI. Learning 360VOS model is nontrivial due to the lack of high-quality labeled
dataset. Recently, Segment Anything Models (SAMs), especially SAM2 – with its design of
memory module – shows strong, promptable VOS capability. However, directly using SAM2 for 360VOS yields implausible results as 360
videos suffer from the projection distortion, semantic inconsistency of left-right sides, and sparse object mask information in SAM2's
memory. To this end, we propose PanoSAM2, a novel 360VOS framework based on our lightweight distortion- and
memory-aware adaptation strategies of SAM2 to achieve reliable 360VOS while retaining SAM2's user-friendly prompting design. Concretely, to tackle the projection distortion and semantic inconsistency issues, we propose a Pano-Aware Decoder with seam-consistent receptive fields and iterative distortion refinement to maintain continuity across the 0/360 degree boundary. Meanwhile, a Distortion-Guided Mask Loss is introduced to weight pixels by distortion magnitude, stressing stretched regions and boundaries. To address the object sparsity issue, we propose a Long-Short
Memory Module to maintain a compact long-term object pointer to re-instantiate and align short-term
memories, thereby enhancing temporal coherence. Extensive experiments show that PanoSAM2 yields substantial gains over SAM2: +5.6 on 360VOTS and +6.7 on PanoVOS, showing the effectiveness of our method.
Read the full abstract ▾
All authors Dingwen Xiao, Weiming Zhang, Shiqi Wen, Lin Wang
Computer Vision
AnomalyAgent: Agentic Industrial Anomaly Synthesis via Tool-Augmented Reinforcement Learning
Jiaming Su, Tengchao Yang, Ruikang Zhang et al. (+3 more)
Industrial anomaly generation is a crucial method for alleviating the data scarcity problem in anomaly detection tasks. Most existing anomaly synthesis methods rely on single-step generation mechanisms, lacking complex
reasoning and iterative optimization capabilities, making it difficult to generate anomaly samples with high semantic realism. We propose AnomalyAgent, an anomaly synthesis
agent with self-reflection, knowledge
retrieval, and iterative refinement capabilities, aiming to generate realistic and diverse anomalies. Specifically, AnomalyAgent is equipped with five tools:
Prompt Generation (PG), Image Generation (IG), Quality Evaluation (QE), Knowledge
Retrieval (KR), and Mask Generation (MG), enabling closed-loop optimization. To improve decision-making and self-reflection, we construct structured trajectories from real anomaly images and design a two-stage training framework: supervised fine-tuning followed by reinforcement learning. This process is driven by a three-part reward mechanism: (1) task rewards to supervise the quality and location rationality of generated anomalies; (2) reflection rewards to train the model's ability to improve anomaly synthesis
prompt; (3) behavioral rewards to ensure adherence to the trajectory. On the MVTec-AD
dataset, AnomalyAgent achieves IS/IC-L of 2.10/0.33 for anomaly generation, 57.0% classification accuracy using ResNet34, and 99.3%/74.2% AP at the image/pixel level using a simple UNet, surpassing all zero-shot SOTA methods. The code is publicly available at https://github.com/sssjmmm/AnomalyAgent.
Read the full abstract ▾
All authors Jiaming Su, Tengchao Yang, Ruikang Zhang, Zhengan Yan, Haoyu Sun, Linfeng Zhang
Computer Vision
Sampling-Aware 3D Spatial Analysis in Multiplexed Imaging
Ido Harlev, Tamar Oukhanov, Raz Ben-Uri et al. (+2 more)
Highly multiplexed microscopy enables rich spatial characterization of tissues at single-
cell resolution, yet most analyses rely on two-dimensional sections despite inherently three-dimensional tissue organization. Acquiring dense volumetric data in spatial proteomics remains costly and technically challenging, leaving practitioners to choose between 2D sections or 3D serial sections under limited imaging budgets. In this work, we study how sampling geometry impacts the stability of commonly used spatial statistics, and we introduce a geometry-aware reconstruction module that enables sparse yet consistent 3D analysis from serial sections. Using controlled simulations, we show that planar sampling reliably recovers global
cell-type abundance but exhibits high variance for local statistics such as
cell clustering and
cell-
cell interactions, particularly for rare or spatially localized populations. We observe consistent behavior in real multiplexed
datasets, where interaction metrics and neighborhood relationships fluctuate substantially across individual sections. To support sparse 3D analysis in practice, we present a reconstruction approach that links
cell projections across adjacent sections using phenotype and proximity constraints and recovers single-
cell 3D centroids using
cell-type-specific shape priors. We further analyze the trade-off between section spacing, coverage, and redundancy, identifying acquisition regimes that maximize reconstruction utility under fixed imaging budgets. We validate the reconstruction module on a public imaging mass cytometry
dataset with dense axial sampling and demonstrate its downstream utility on an in-house CODEX
dataset by enabling structure-level 3D analyses that are unreliable in 2D. Together, our results provide diagnostic tools and practical guidance for deciding when 2D sampling suffices and when sparse 3D reconstruction is warranted.
Read the full abstract ▾
All authors Ido Harlev, Tamar Oukhanov, Raz Ben-Uri, Leeat Keren, Shai Bagon
Computer Vision
Reinforcement-Guided Synthetic Data Generation for Privacy-Sensitive Identity Recognition
Xuemei Jia, Jiawei Du, Hui Wei et al. (+3 more)
High-fidelity generative models are increasingly needed in
privacy-sensitive scenarios, where access to data is severely restricted due to regulatory and copyright constraints. This scarcity hampers model development–ironically, in settings where generative models are most needed to compensate for the lack of data. This creates a self-reinforcing challenge: limited data leads to poor generative models, which in turn fail to mitigate data scarcity. To break this cycle, we propose a reinforcement-guided synthetic data generation framework that adapts general-domain generative priors to
privacy-sensitive identity recognition tasks. We first perform a cold-start adaptation to align a pretrained generator with the target domain, establishing semantic relevance and initial fidelity. Building on this foundation, we introduce a multi-objective reward that jointly optimizes semantic consistency, coverage diversity, and expression richness, guiding the generator to produce both realistic and task-effective samples. During downstream training, a dynamic sample selection mechanism further prioritizes high-utility synthetic samples, enabling adaptive data scaling and improved domain
alignment. Extensive experiments on
benchmark datasets demonstrate that our framework significantly improves both generation fidelity and classification accuracy, while also exhibiting strong generalization to novel categories in small-data regimes.
Read the full abstract ▾
All authors Xuemei Jia, Jiawei Du, Hui Wei, Jun Chen, Joey Tianyi Zhou, Zheng Wang
Computer Vision
ReconPhys: Reconstruct Appearance and Physical Attributes from Single Video
Boyuan Wang, Xiaofeng Wang, Yongkang Li et al. (+9 more)
Reconstructing non-rigid objects with physical plausibility remains a significant challenge. Existing approaches leverage differentiable rendering for per-scene optimization, recovering geometry and dynamics but requiring expensive tuning or manual annotation, which limits practicality and generalizability. To address this, we propose ReconPhys, the first feedforward framework that jointly learns physical attribute estimation and 3D Gaussian Splatting reconstruction from a single
monocular video. Our method employs a dual-branch architecture trained via a self-supervised strategy, eliminating the need for ground-truth physics labels. Given a
video sequence, ReconPhys simultaneously infers geometry, appearance, and physical attributes. Experiments on a large-scale synthetic
dataset demonstrate superior performance: our method achieves 21.64 PSNR in future prediction compared to 13.27 by state-of-the-art optimization baselines, while reducing Chamfer Distance from 0.349 to 0.004. Crucially, ReconPhys enables fast inference (<1 second) versus hours required by existing methods, facilitating rapid generation of simulation-ready assets for robotics and graphics.
Read the full abstract ▾
All authors Boyuan Wang, Xiaofeng Wang, Yongkang Li, Zheng Zhu, Yifan Chang, Angen Ye, Guosheng Zhao, Chaojun Ni, Guan Huang, Yijie Ren, Yueqi Duan, Xingang Wang
Computer Vision
FlowGuard: Towards Lightweight In-Generation Safety Detection for Diffusion Models via Linear Latent Decoding
Jinghan Yang, Yihe Fan, Xudong Pan et al. (+1 more)
Diffusion-based image generation models have advanced rapidly but pose a safety risk due to their potential to generate Not-Safe-For-Work (NSFW) content. Existing NSFW detection methods mainly operate either before or after image generation. Pre-generation methods rely on text
prompts and struggle with the gap between
prompt safety and image safety. Post-generation methods apply classifiers to final outputs, but they are poorly suited to intermediate noisy images. To address this, we introduce FlowGuard, a cross-model in-generation detection framework that inspects intermediate denoising steps. This is particularly challenging in latent
diffusion, where early-stage noise obscures visual signals. FlowGuard employs a novel linear approximation for latent decoding and leverages a curriculum learning approach to stabilize training. By detecting unsafe content early, FlowGuard reduces unnecessary
diffusion steps to cut computational costs. Our cross-model
benchmark spanning nine
diffusion-based backbones shows the effectiveness of FlowGuard for in-generation NSFW detection in both in-distribution and out-of-distribution settings, outperforming existing methods by over 30% in F1 score while delivering transformative efficiency gains, including slashing peak GPU
memory demand by over 97% and projection time from 8.1 seconds to 0.2 seconds compared to standard VAE decoding.
Read the full abstract ▾
All authors Jinghan Yang, Yihe Fan, Xudong Pan, Min Yang
Computer Vision · Cross-listed from cs.CR
Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection
Wenkui Yang, Chao Jin, Haisu Zhu et al. (+7 more)
Existing red-teaming studies on GUI
agents face two fundamental limitations:
adversarial perturbations require white-box access unavailable in commercial deployments, while
prompt injection is increasingly neutralized by stronger safety
alignment. To study robustness under a more practical threat model, we propose Semantic-level UI Element Injection, a black-box red-teaming paradigm that overlays safety-aligned and harmless UI elements onto screenshots to misdirect the
agent's visual grounding. Our method couples a modular Editor–Overlapper–Victim pipeline with iterative search that samples multiple candidate edits, keeps the best cumulative overlay, and adapts future
prompt strategies based on previous failures. Experiments across 19 victim models spanning 8 model families show that strategic optimization substantially outperforms random injection (3.5-6.9x on the most robust victims) and transfers near-perfectly across architectures, confirming model-agnostic visual-semantic vulnerabilities. After the first successful attack, the victim still clicks the attacker-controlled icon in over 15% of subsequent independent trials versus below 1% for random injection, establishing that strategically placed icons act as persistent attractors that causally redirect grounding rather than introducing incidental clutter.
Read the full abstract ▾
All authors Wenkui Yang, Chao Jin, Haisu Zhu, Weilin Luo, Derek Yuen, Kun Shao, Junxian Duan, Huaibo Huang, Jie Cao, Ran He
Computer Vision
LPM 1.0: Video-based Character Performance Model
Ailing Zeng, Casper Yang, Chauncey Ge et al. (+22 more)
Performance, the externalization of intent,
emotion, and personality through visual, vocal, and temporal behavior, is what makes a character alive. Learning such performance from
video is a promising alternative to traditional 3D pipelines. However, existing
video models struggle to jointly achieve high expressiveness, real-time inference, and long-horizon identity stability, a tension we call the performance trilemma. Conversation is the most comprehensive performance scenario, as characters simultaneously speak, listen, react, and emote while maintaining identity over time. To address this, we present LPM 1.0 (Large Performance Model), focusing on single-person full-duplex audio-visual conversational performance. Concretely, we build a
multimodal human-centric
dataset through strict filtering, speaking-listening audio-
video pairing, performance understanding, and identity-aware multi-reference extraction; train a 17B-parameter
Diffusion Transformer (Base LPM) for highly controllable, identity-consistent performance through
multimodal conditioning; and distill it into a causal streaming generator (Online LPM) for low-latency, infinite-length interaction. At inference, given a character image with identity-aware references, LPM 1.0 generates listening
videos from user audio and speaking
videos from synthesized audio, with text
prompts for motion control, all at real-time speed with identity-stable, infinite-length generation. LPM 1.0 thus serves as a visual engine for conversational
agents, live streaming characters, and game NPCs. To systematically evaluate this setting, we propose LPM-Bench, the first
benchmark for interactive character performance. LPM 1.0 achieves state-of-the-art results across all evaluated dimensions while maintaining real-time inference.
Read the full abstract ▾
All authors Ailing Zeng, Casper Yang, Chauncey Ge, Eddie Zhang, Garvey Xu, Gavin Lin, Gilbert Gu, Jeremy Pi, Leo Li, Mingyi Shi, Shawn Wang, Sheng Bi, Steven Tang, Thorn Hang, Tobey Guo, Vincent Li, Xin Tong, Yikang Li, Yuchen Sun, Yue Zhao, Yuhan Lu, Yuwei Li, Zane Zhang, Zeshi Yang, Zi Ye
Computer Vision
AgriChain Visually Grounded Expert Verified Reasoning for Interpretable Agricultural Vision Language Models
Hazza Mahmood, Yongqiang Yu and Rao Anwer
Accurate and interpretable plant disease diagnosis remains a major challenge for
vision-
language models (VLMs) in real-world agriculture. We introduce AgriChain, a
dataset of approximately 11,000 expert-curated leaf images spanning diverse crops and pathologies, each paired with (i) a disease label, (ii) a calibrated confidence score (High/Medium/Low), and (iii) an expert-verified chain-of-thought (CoT) rationale. Draft explanations were first generated by GPT-4o and then verified by a professional agricultural engineer using standardized descriptors (e.g., lesion color, margin, and distribution). We fine-tune Qwen2.5-VL-3B on AgriChain, resulting in a specialized model termed AgriChain-VL3B, to jointly predict diseases and generate visually grounded
reasoning. On a 1,000-image test set, our CoT-supervised model achieves 73.1% top-1 accuracy (macro F1 = 0.466; weighted F1 = 0.655), outperforming strong baselines including Gemini 1.5 Flash, Gemini 2.5 Pro, and GPT-4o Mini. The generated explanations align closely with expert
reasoning, consistently referencing key visual cues. These findings demonstrate that expert-verified
reasoning supervision significantly enhances both accuracy and interpretability, bridging the gap between generic
multimodal models and human expertise, and advancing trustworthy, globally deployable AI for sustainable agriculture. The
dataset and code are publicly available at: https://github.com/hazzanabeel12-netizen/agrichain
Read the full abstract ▾
Computer Vision
HAWK: Head Importance-Aware Visual Token Pruning in Multimodal Models
Qihui Zhu, Tao Zhang, Yuchen Wang et al. (+9 more)
In
multimodal large
language models (MLLMs), the surge of visual
tokens significantly increases the inference time and computational overhead, making them impractical for real-time or resource-constrained applications. Visual
token pruning is a promising strategy for reducing the cost of MLLM inference by removing redundant visual
tokens. Existing research usually assumes that all
attention heads contribute equally to the visual interpretation. However, our study reveals that different heads may capture distinct visual semantics and inherently play distinct roles in visual processing. In light of this observation, we propose HAWK, a head importance-aware visual
token pruning method that perceives the varying importance of
attention heads in visual tasks to maximize the retention of crucial
tokens. By leveraging head importance weights and text-guided
attention to assess visual
token significance, HAWK effectively retains task-relevant visual
tokens while removing redundant ones. The proposed HAWK is entirely training-free and can be seamlessly applied to various MLLMs. Extensive experiments on multiple mainstream
vision-
language benchmarks demonstrate that HAWK achieves state-of-the-art accuracy. When applied to Qwen2.5-VL, HAWK retains 96.0% of the original accuracy after
pruning 80.2% of the visual
tokens. Additionally, it reduces end-to-end latency to 74.4% of the original and further decreases GPU
memory usage across the tested models. The code is available at https://github.com/peppery77/HAWK.git.
Read the full abstract ▾
All authors Qihui Zhu, Tao Zhang, Yuchen Wang, Zijian Wen, Mengjie Zhang, Shuangwu Chen, Xiaobin Tan, Jian Yang, Yang Liu, Zhenhua Dong, Xianzhi Yu, Yinfei Pan
Computer Vision · Cross-listed from cs.CY
The Weaponization of Computer Vision: Tracing Military-Surveillance Ties through Conference Sponsorship
Noa Garcia and Amelia Katirai
Computer
vision, a core domain of artificial intelligence (AI), is the field that enables the computational analysis, understanding, and generation of visual data. Despite being historically rooted in military funding and increasingly deployed in warfare, the field tends to position itself as a neutral, purely technical endeavor, failing to engage in discussions about its dual-use applications. Yet it has been reported that computer
vision systems are being systematically weaponized to assist in technologies that inflict harm, such as surveillance or warfare. Expanding on these concerns, we study the extent to which computer
vision research is being used in the military and surveillance domains. We do so by collecting a
dataset of tech companies with financial ties to the field's central research exchange platform: conferences. Conference sponsorship, we argue, not only serves as strong evidence of a company's investment in the field but also provides a privileged position for shaping its trajectory. By investigating sponsors' activities, we reveal that 44% of them have a direct connection with military or surveillance applications. We extend our analysis through two case studies in which we discuss the opportunities and limitations of sponsorship as a means for uncovering technological weaponization.
Read the full abstract ▾
Computer Vision
Latent Anomaly Knowledge Excavation: Unveiling Sparse Sensitive Neurons in Vision-Language Models
Shaotian Li, Shangze Li, Chuancheng Shi et al. (+5 more)
Large-scale
vision-
language models (VLMs) exhibit remarkable zero-shot capabilities, yet the internal mechanisms driving their anomaly detection (AD) performance remain poorly understood. Current methods predominantly treat VLMs as black-box feature extractors, assuming that anomaly-specific knowledge must be acquired through external adapters or
memory banks. In this paper, we challenge this assumption by arguing that anomaly knowledge is intrinsically embedded within pre-trained models but remains latent and under-activated. We hypothesize that this knowledge is concentrated within a sparse subset of anomaly-sensitive
neurons. To validate this, we propose latent anomaly knowledge excavation (LAKE), a training-free framework that identifies and elicits these critical neuronal signals using only a minimal set of normal samples. By isolating these sensitive
neurons, LAKE constructs a highly compact normality representation that integrates visual structural deviations with cross-modal semantic activations. Extensive experiments on industrial AD
benchmarks demonstrate that LAKE achieves state-of-the-art performance while providing intrinsic,
neuron-level interpretability. Ultimately, our work advocates for a paradigm shift: redefining anomaly detection as the targeted activation of latent pre-trained knowledge rather than the acquisition of a downstream task.
Read the full abstract ▾
All authors Shaotian Li, Shangze Li, Chuancheng Shi, Wenhua Wu, Yanqiu Wu, Xiaohan Yu, Fei Shen, Tat-Seng Chua
Computer Vision
Image-Guided Geometric Stylization of 3D Meshes
Changwoon Choi, Hyunsoo Lee, Clément Jambon et al. (+2 more)
Recent generative models can create visually plausible 3D representations of objects. However, the generation process often allows for implicit control signals, such as contextual descriptions, and rarely supports bold
geometric distortions beyond existing data distributions. We propose a
geometric stylization framework that deforms a 3D mesh, allowing it to express the style of an image. While style is inherently ambiguous, we utilize pre-trained
diffusion models to extract an abstract representation of the provided image. Our coarse-to-fine stylization pipeline can drastically deform the input 3D model to express a diverse range of
geometric variations while retaining the valid
topology of the original mesh and part-level semantics. We also propose an approximate VAE encoder that provides efficient and reliable gradients from mesh renderings. Extensive experiments demonstrate that our method can create stylized 3D meshes that reflect unique
geometric features of the pictured assets, such as expressive poses and silhouettes, thereby supporting the creation of distinctive artistic 3D creations. Project page: https://changwoonchoi.github.io/GeoStyle
Read the full abstract ▾
All authors Changwoon Choi, Hyunsoo Lee, Clément Jambon, Yael Vinker, Young Min Kim
Computer Vision
Cross-Modal Emotion Transfer for Emotion Editing in Talking Face Video
Chanhyuk Choi, Taesoo Kim, Donggyu Lee et al. (+2 more)
Talking face generation has gained significant
attention as a core application of generative models. To enhance the expressiveness and realism of synthesized
videos,
emotion editing in talking face
video plays a crucial role. However, existing approaches often limit expressive flexibility and struggle to generate extended
emotions. Label-based methods represent
emotions with discrete categories, which fail to capture a wide range of
emotions. Audio-based methods can leverage emotionally rich
speech signals - and even benefit from expressive text-to-
speech (TTS) synthesis - but they fail to express the target
emotions because
emotions and linguistic contents are entangled in emotional
speeches. Images-based methods, on the other hand, rely on target reference images to guide
emotion transfer, yet they require high-quality frontal views and face challenges in acquiring reference data for extended
emotions (e.g., sarcasm). To address these limitations, we propose Cross-Modal
Emotion Transfer (C-MET), a novel approach that generates facial expressions based on
speeches by modeling
emotion semantic vectors between
speech and visual feature spaces. C-MET leverages a large-scale pretrained audio encoder and a disentangled facial expression encoder to learn
emotion semantic vectors that represent the difference between two different emotional
embeddings across modalities. Extensive experiments on the MEAD and CREMA-D
datasets demonstrate that our method improves
emotion accuracy by 14% over state-of-the-art methods, while generating expressive talking face
videos - even for unseen extended
emotions. Code, checkpoint, and demo are available at https://chanhyeok-choi.github.io/C-MET/
Read the full abstract ▾
All authors Chanhyuk Choi, Taesoo Kim, Donggyu Lee, Siyeol Jung, Taehwan Kim
Computer Vision · Cross-listed from eess.IV
MonoUNet: A Robust Tiny Neural Network for Automated Knee Cartilage Segmentation on Point-of-Care Ultrasound Devices
Alvin Kimbowa, Arjun Parmar, Ibrahim Mujtaba et al. (+5 more)
Objective: To develop a robust and compact deep learning model for automated knee cartilage
segmentation on point-of-care ultrasound (POCUS) devices. Methods: We propose MonoUNet, a novel, highly compact
segmentation model consisting of (i) an aggressively reduced U-Net backbone, (ii) a trainable monogenic block that extracts multi-scale local phase features from the input, and (iii) a gating mechanism that injects these features into the encoder stages to reduce sensitivity to variations in ultrasound image appearance. MonoUNet
segmentation performance was evaluated on a multi-site, multi-device knee cartilage ultrasound
dataset using Dice score and mean average surface distance (MASD). Agreement between MonoUNet and manual cartilage outcomes (thickness and echo intensity) was assessed using Bland-Altman analysis with 95% limits of agreement, and reliability was assessed using intraclass correlation coefficient (ICC$_{2,k}$). Results: Overall, MonoUNet outperformed existing lightweight
segmentation models, with average Dice scores ranging from 92.62% to 94.82% and MASD values between 0.133 mm and 0.254 mm. MonoUNet reduces the number of parameters by 10x–700x and computational cost by 14x–2000x relative to existing lightweight models. MonoUNet cartilage outcomes showed excellent reliability and agreement with the manual outcomes: intraclass correlation coefficients (ICC$_{2,k})$=0.96 and bias=2.00% (0.047 mm) for average thickness, and ICC$_{2,k}$=0.99 and bias=0.80% (0.328 a.u.) for echo intensity. Conclusion: Incorporating trainable local phase features improves the robustness of highly compact neural networks for knee cartilage
segmentation across varying acquisition settings and could support scalable ultrasound-based assessment and monitoring of knee osteoarthritis using POCUS devices. The code is publicly available at https://github.com/alvinkimbowa/monounet.
Read the full abstract ▾
All authors Alvin Kimbowa, Arjun Parmar, Ibrahim Mujtaba, Will Wei, Maziar Badii, Matthew Harkey, David Liu, Ilker Hacihaliloglu
Computer Vision
Plug-and-Play Logit Fusion for Heterogeneous Pathology Foundation Models
Gexin Huang, Anqi Li, Yusheng Tan et al. (+4 more)
Pathology foundation models (FMs) have become central to computational histopathology, offering strong transfer performance across a wide range of diagnostic and prognostic tasks. The rapid proliferation of pathology foundation models creates a model-selection bottleneck: no single model is uniformly best, yet exhaustively adapting and validating many candidates for each downstream endpoint is prohibitively expensive. We address this challenge with a lightweight and novel model fusion strategy, LogitProd, which treats independently trained FM-based predictors as fixed experts and learns sample-adaptive fusion weights over their slide-level outputs. The fusion operates purely on logits, requiring no encoder retraining and no feature-space
alignment across heterogeneous backbones. We further provide a theoretical analysis showing that the optimal weighted product fusion is guaranteed to perform at least as well as the best individual expert under the training objective. We systematically evaluate LogitProd on 22
benchmarks spanning WSI-level classification, tile-level classification, gene mutation prediction, and discrete-time survival modeling. LogitProd ranks first on 20/22 tasks and improves the average performance across all tasks by ~3% over the strongest single expert. LogitProd enables practitioners to upgrade heterogeneous FM-based pipelines in a plug-and-play manner, achieving multi-expert gains with $\sim$12$\times$ lower training cost than feature-fusion alternatives.
Read the full abstract ▾
All authors Gexin Huang, Anqi Li, Yusheng Tan, Beidi Zhao, Gang Wang, Zu-Hua Gao, Xiaoxiao Li
Computer Vision · Cross-listed from cs.RO
RoboAgent: Chaining Basic Capabilities for Embodied Task Planning
Peiran Xu, Jiaqi Zheng and Yadong Mu
This paper focuses on
embodied task planning, where an
agent acquires visual observations from the environment and executes atomic actions to accomplish a given task. Although recent
Vision-
Language Models (VLMs) have achieved impressive results in
multimodal understanding and
reasoning, their performance remains limited when applied to
embodied planning that involves multi-turn interaction, long-horizon
reasoning, and extended context analysis. To bridge this gap, we propose RoboAgent, a capability-driven planning pipeline in which the model actively invokes different sub-capabilities. Each capability maintains its own context, and produces intermediate
reasoning results or interacts with the environment according to the query given by a scheduler. This framework decomposes complex planning into a sequence of basic
vision-
language problems that VLMs can better address, enabling a more transparent and controllable
reasoning process. The scheduler and all capabilities are implemented with a single VLM, without relying on external tools. To train this VLM, we adopt a multi-stage paradigm that consists of: (1) behavior cloning with expert plans, (2) DAgger training using trajectories collected by the model, and (3) reinforcement learning guided by an expert policy. Across these stages, we exploit the internal information of the environment simulator to construct high-quality supervision for each capability, and we further introduce augmented and synthetic data to enhance the model's performance in more diverse scenarios. Extensive experiments on widely used
embodied task planning
benchmarks validate the effectiveness of the proposed approach. Our codes will be available at https://github.com/woyut/RoboAgent_CVPR26.
Read the full abstract ▾
Computer Vision
ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitions
Zihao Liu, Xiaoyu Wu, Wenna Li et al. (+2 more)
Open-world
video anomaly detection (OWVAD) aims to detect and explain abnormal events under different anomaly definitions, which is important for applications such as intelligent surveillance and live-streaming content moderation. Recent MLLM-based methods have shown promising open-world generalization, but still suffer from three major limitations: inefficiency for practical deployment, lack of streaming processing adaptation, and limited support for dynamic anomaly definitions in both modeling and evaluation. To address these issues, this paper proposes ESOM, an efficient streaming OWVAD model that operates in a training-free manner. ESOM includes a Definition Normalization module to structure user
prompts for reducing
hallucination, an Inter-frame-matched Intra-frame
Token Merging module to compress redundant visual
tokens, a Hybrid Streaming
Memory module for efficient causal inference, and a Probabilistic Scoring module that converts interval-level textual outputs into frame-level anomaly scores. In addition, this paper introduces OpenDef-Bench, a new
benchmark with clean surveillance
videos and diverse natural anomaly definitions for evaluating performance under varying conditions. Extensive experiments show that ESOM achieves real-time efficiency on a single GPU and state-of-the-art performance in anomaly temporal localization, classification, and description generation. The code and
benchmark will be released at https://github.com/Kamino666/ESOM_OpenDef-Bench.
Read the full abstract ▾
All authors Zihao Liu, Xiaoyu Wu, Wenna Li, Jianqin Wu, Linlin Yang
Computer Vision
RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMs
Liang Yao, Shengxiang Xu, Fan Liu et al. (+7 more)
Earth Observation (EO) systems are essentially designed to support domain experts who often express their requirements through vague natural
language rather than precise, machine-friendly instructions. Depending on the specific application scenario, these vague queries can demand vastly different levels of visual precision. Consequently, a practical EO AI system must bridge the gap between ambiguous human queries and the appropriate multi-granularity visual analysis tasks, ranging from holistic image interpretation to fine-grained pixel-wise predictions. While Multi-modal Large
Language Models (MLLMs) demonstrate strong semantic understanding, their text-based output format is inherently ill-suited for dense, precision-critical spatial predictions. Existing agentic frameworks address this limitation by delegating tasks to external tools, but indiscriminate tool invocation is computationally inefficient and underutilizes the MLLM's native capabilities. To this end, we propose RemoteAgent, an agentic framework that strategically respects the intrinsic capability boundaries of MLLMs. To empower this framework to understand real user intents, we construct VagueEO, a human-centric instruction
dataset pairing EO tasks with simulated vague natural-
language queries. By leveraging VagueEO for reinforcement fine-tuning, we align an MLLM into a robust cognitive core that directly resolves image- and sparse region-level tasks. Consequently, RemoteAgent processes suitable tasks internally while intelligently orchestrating specialized tools via the Model Context Protocol exclusively for dense predictions. Extensive experiments demonstrate that RemoteAgent achieves robust intent recognition capabilities while delivering highly competitive performance across diverse EO tasks.
Read the full abstract ▾
All authors Liang Yao, Shengxiang Xu, Fan Liu, Chuanyi Zhang, Bishun Yao, Rui Min, Yongjun Li, Chaoqian Ouyang, Shimin Di, Min-Ling Zhang
Computer Vision
Beyond Surface Artifacts: Capturing Shared Latent Forgery Knowledge Across Modalities
Jingtong Dou, Chuancheng Shi, Jian Wang et al. (+3 more)
As generative artificial intelligence evolves, deepfake attacks have escalated from single-modality manipulations to complex,
multimodal threats. Existing forensic techniques face a severe generalization bottleneck: by relying excessively on superficial, modality-specific artifacts, they neglect the shared latent forgery knowledge hidden beneath variable physical appearances. Consequently, these models suffer catastrophic performance degradation when confronted with unseen "dark modalities." To break this limitation, this paper introduces a paradigm shift that redefines
multimodal forensics from conventional "feature fusion" to "modality generalization." We propose the first modality-agnostic forgery (MAF) detection framework. By explicitly decoupling modality-specific styles, MAF precisely extracts the essential, cross-modal latent forgery knowledge. Furthermore, we define two progressive dimensions to quantify model generalization: transferability toward semantically correlated modalities (Weak MAF), and robustness against completely isolated signals of "dark modality" (Strong MAF). To rigorously assess these generalization limits, we introduce the DeepModal-Bench
benchmark, which integrates diverse
multimodal forgery detection
algorithms and adapts state-of-the-art generalized learning methods. This study not only empirically proves the existence of universal forgery traces but also achieves significant performance breakthroughs on unknown modalities via the MAF framework, offering a pioneering technical pathway for universal
multimodal defense.
Read the full abstract ▾
All authors Jingtong Dou, Chuancheng Shi, Jian Wang, Fei Shen, Zhiyong Wang, Tat-Seng Chua
Computer Vision
WUTDet: A 100K-Scale Ship Detection Dataset and Benchmarks with Dense Small Objects
Junxiong Liang, Mengwei Bao, Tianxiang Wang et al. (+3 more)
Ship detection for
navigation is a fundamental perception task in intelligent waterway transportation systems. However, existing public ship detection
datasets remain limited in terms of scale, the proportion of small-object instances, and scene diversity, which hinders the systematic evaluation and generalization study of detection
algorithms in complex maritime environments. To this end, we construct WUTDet, a large-scale ship detection
dataset. WUTDet contains 100,576 images and 381,378 annotated ship instances, covering diverse operational scenarios such as ports, anchorages,
navigation, and berthing, as well as various imaging conditions including fog, glare, low-lightness, and rain, thereby exhibiting substantial diversity and challenge. Based on WUTDet, we systematically evaluate 20 baseline models from three mainstream detection architectures, namely CNN,
Transformer, and Mamba. Experimental results show that the
Transformer architecture achieves superior overall detection accuracy (AP) and small-object detection performance (APs), demonstrating stronger adaptability to complex maritime scenes; the CNN architecture maintains an advantage in inference efficiency, making it more suitable for real-time applications; and the Mamba architecture achieves a favorable balance between detection accuracy and computational efficiency. Furthermore, we construct a unified cross-
dataset test set, Ship-GEN, to evaluate model generalization. Results on Ship-GEN show that models trained on WUTDet exhibit stronger generalization under different data distributions. These findings demonstrate that WUTDet provides effective data support for the research, evaluation, and generalization analysis of ship detection
algorithms in complex maritime scenarios. The
dataset is publicly available at: https://github.com/MAPGroup/WUTDet.
Read the full abstract ▾
All authors Junxiong Liang, Mengwei Bao, Tianxiang Wang, Xinggang Wang, An-An Liu, Ryan Wen Liu
Computer Vision
DailyArt: Discovering Articulation from Single Static Images via Latent Dynamics
Hang Zhang, Qijian Tian, Jingyu Gong et al. (+4 more)
Articulated objects are essential for
embodied AI and world models, yet inferring their kinematics from a single closed-state image remains challenging because crucial motion cues are often occluded. Existing methods either require multi-state observations or rely on explicit part priors,
retrieval, or other auxiliary inputs that partially expose the structure to be inferred. In this work, we present DailyArt, which formulates articulated joint estimation from a single static image as a synthesis-mediated
reasoning problem. Instead of directly regressing joints from a heavily occluded observation, DailyArt first synthesizes a maximally articulated opened state under the same camera view to expose articulation cues, and then estimates the full set of joint parameters from the discrepancy between the observed and synthesized states. Using a set-prediction formulation, DailyArt recovers all joints simultaneously without requiring object-specific templates, multi-view inputs, or explicit part annotations at test time. Taking estimated joints as conditions, the framework further supports part-level novel state synthesis as a downstream capability. Extensive experiments show that DailyArt achieves strong performance in articulated joint estimation and supports part-level novel state synthesis conditioned on joints. Project page is available at https://rangooo123.github.io/DaliyArt.github.io/.
Read the full abstract ▾
All authors Hang Zhang, Qijian Tian, Jingyu Gong, Daoguo Dong, Xuhong Wang, Yuan Xie, Xin Tan