Computer Vision
EchoPilot: Training-Free Ultrasound Video Segmentation via Scale-Space Semantic Prompting and Reliability-Gated Memory
Ruiqiang Xiao, Zhaohu Xing, Yijun Yang et al. (+4 more)
Ultrasound
video segmentation is clinically valuable yet difficult due to speckle noise, weak boundaries, and rapid anatomical deformation. Recent promptable foundation models enable point-guided
segmentation, but their direct deployment in ultrasound remains unreliable: a single point provides insufficient spatial context to resolve scale ambiguity, and greedy
memory updates amplify early errors into severe temporal drift. We present EchoPilot, a training-free framework for ultrasound
video segmentation under sparse first-frame interaction, requiring only a single point click and an anatomical category name. EchoPilot orchestrates a frozen medical
vision-
language model (VLM) for semantic localization, a
vision foundation model (VFM) for dense
geometric feature extraction, and a promptable
video segmentor for mask prediction and propagation. To resolve initialization ambiguity, we propose Scale-Space Semantic Prompting, which first selects an optimal contextual view via a parameter-free S.E.E.D. (Semantic Energy-
Entropy Density) criterion, and then synthesizes geometrically precise auxiliary point
prompts from dense foundation features without additional user interaction. To reduce propagation drift, a Reliability-Gated
Memory update is further introduced to selectively freeze the segmentor's
memory bank under uncertain predictions, preventing error accumulation. We also contribute the first dynamic fetal placenta ultrasound
video segmentation dataset with 671 annotated frames. Across three ultrasound
video datasets, EchoPilot achieves state-of-the-art performance under the sparse-interactive setting, consistently outperforming training-free baselines and finetuned specialists.
Read the full abstract ▾
All authors Ruiqiang Xiao, Zhaohu Xing, Yijun Yang, Zhenyan Han, Weiming Wang, Kaishun Wu, Lei Zhu
Computer Vision
LRDDv3: High-Resolution Long-Range Drone Detection Dataset with Range Information and Thermal Data
Knut Peterson, Zaid Mayers, Azmain Yousuf et al. (+5 more)
Unmanned Aerial Vehicles (
UAVs) have quickly become common in various airspaces, representing a wide range of applications from recreation flying to commercial photography and package delivery. With the increasing prevalence of
UAVs, it becomes critical that both manned and unmanned aircraft can detect
UAVs and other flying objects from long range to effectively track movement and ensure safe operation in shared spaces. While several
datasets have been introduced for drone detection, the need for expanded high-quality data persists, especially in the area of high-resolution long-range drone data. To address this, we introduce a high-resolution
dataset of 102,532 long-range RGB images of drones, sampled at 5 FPS from 128 distinct
video clips taken mid flight during 17 different data collection days spread over 8 months to ensure a wide variety of lighting scenarios, flight locations, and background elements. The
dataset boasts comprehensive drone range information across the
dataset, as well as 29,630 IR images, all paired with RGB counterparts from the base
dataset. As one of the first drone detection
datasets to leverage 4K image resolution and paired 640x512 IR images, our work represents a significant advancement to enable the detection of drones at long range. For access to the complete
dataset, please visit https://research.coe.drexel.edu/ece/imaple/lrddv3/
Read the full abstract ▾
All authors Knut Peterson, Zaid Mayers, Azmain Yousuf, Priontu Chowdhury, Asher Zaczepinski, Solmaz Arezoomandan, Reihaneh Maarefdoust, David Han
Computer Vision
Where Concept Erasure Should Occur: Concept-Layer Alignment in Text-to-Video Diffusion Models
Yiwei Xie, Ping Liu and Zheng Zhang
Text-to-
video diffusion transformers encode semantic information unevenly across model depth, which constrains effective concept erasure. We identify a representational bottleneck, termed concept-layer topological
alignment, under which target concepts exhibit higher separability at certain representational depths. Outside these depths, concept and non-target signals remain strongly entangled, limiting the effectiveness of depth-specific erasure. This observation reframes concept erasure as the problem of identifying representational depths where concept-non-target separation naturally emerges. Motivated by this structural constraint, we introduce CLEAR, a separability-driven optimization framework for concept erasure that explicitly enforces concept-layer
alignment. CLEAR operationalizes this principle by formulating layer selection as an optimization problem over concept-non-target separability, rather than relying on layer-agnostic or heuristic choices. To enable this, we introduce a separability-aware objective that favors layers exhibiting stronger concept-non-target separation. Experiments on large-scale text-to-
video diffusion models demonstrate that enforcing concept–layer
alignment leads to more precise concept suppression while preserving overall generative quality.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
How Accurate are Video Quality Models for Diffusion-Based Video Super-Resolution?
Benjamin Herb, Steve Göring, Alexander Raake et al. (+1 more)
Recent
video super-resolution (VSR) approaches use deep neural networks to enhance low-quality input
videos and recover visual detail, with
diffusion-based methods in particular showing promising results. In this paper, we investigate whether existing
video quality models can be used to assess the performance of these
diffusion-based VSR methods, by comparing model predictions with results from a subjective test. The study compares six upscaling methods (Lanczos, Rhea, SCST, DOVE, SeedVR2, Starlight Mini) applied to both compressed (AV1 and DCVC-RT) and uncompressed low-resolution
videos considering the play-out on a UHD-1/4K screen. A range of full- and no-reference quality models are used to assess their applicability to this new type of quality degradation, focusing on within-sequence performance. The results highlight that CNN-based full-reference models, such as LPIPS, DISTS, and CVQA-FR show significantly higher correlation coefficients than both conventional full- as well as the tested no-reference models. Most overestimate the overly sharp results of SCST, with VMAF mainly failing due to spatial inconsistencies introduced by Starlight Mini. None of the tested
video quality models reach sufficient accuracy so as to replace complementary subjective testing. The reference, degraded and upscaled
videos, as well as the user ratings and model scores are made available with the paper at https://github.com/Telecommunication-Telemedia-Assessment/AVT-VQDB-UHD-1-VSR as open data.
Read the full abstract ▾
All authors Benjamin Herb, Steve Göring, Alexander Raake, Rakesh Rao Ramachandra Rao
Computer Vision
Closed-Loop Bidirectional Prompting for Adversarial Robustness of Vision Language Models
Xiao Liu, Jiaxiang Liu, Boci Peng et al. (+6 more)
Vision Language Models adapt well to downstream tasks but are highly vulnerable to
adversarial perturbations that disrupt cross-modal semantic
alignment. Existing defenses are largely unidirectional or structural, failing to exploit bidirectional cross-modal complementarity and instance-wise adaptive protection. To overcome the limitations of unidirectional and static defenses in
adversarial settings, we propose Closed-Loop Bidirectional Prompting, casting robust adaptation as cross-modal agreement recovery via a dynamic feedback loop on frozen encoders. A Semantic Anchor is introduced as a stable prior to constrain cyclic updates and mitigate perturbation-induced feature corruption. Through anchor-based bootstrapping, textual semantics denoise visual representations, while the refined visuals enable instance-adaptive
prompt updating, yielding a rectified and robust consensus. Extensive evaluations across 11
datasets validate state-of-the-art robustness and strong base-to-new generalization, while maintaining a favorable trade-off between computational cost and accuracy.
Read the full abstract ▾
All authors Xiao Liu, Jiaxiang Liu, Boci Peng, Boren Hu, Yusong Wang, Xiwen Chen, Prayag Tiwari, Liming Zhang, Mingkun Xu
Computer Vision · Cross-listed from cs.GR
Curve Skeletonization in Continuous domain for Meshes and Point Clouds
Jai Bardhan, Ramya Hebbalaguppe and Aravind Udupa
Advancements in 3D curve skeletonization are accelerating progress across a wide range of applications. However, developing robust skeletonization
algorithms that capture intricate object details remains challenging. Skeletonization via Local Separators (LS) offers an efficient graph-based approach but suffers from representation inaccuracies due to its discrete nature. To address this, we introduce CSCD, a novel framework for Curve Skeletonization in the Continuous Domain, generalizing LS to
manifolds. Specifically, we present two realizations: CSCD-M for meshes and CSCD-PC for point clouds. CSCD-M leverages the intrinsic triangulation of a mesh for resilience to noise and improved topological preservation, while CSCD-PC employs tufted
Laplacians for enhanced robustness. To our knowledge, CSCD-M is the first intrinsic method for curve skeletonization. Our results show CSCD-M matches LS performance across diverse meshes and outperforms LS (TOG'21) on
benchmarks like Thingi10k
dataset. CSCD-PC qualitatively outperforms CoverageAxis++ (Eurographics'24) and EPCS (CAG'23). Finally, we demonstrate the efficacy of CSCD in a few downstream tasks: object classification, shape
segmentation, identifying handles, tunnels, and constrictions in objects. Project Website: https://cscd-skel.pages.dev
Read the full abstract ▾
Computer Vision
Accelerated Spatio-Temporal Gaussian Rendering via Kinematic and Semantic Priors
Denis Gridusov, Maxim Popov and Sergey Kolyubin
Reconstructing dynamic 3D scenes from multi-view
videos is a foundational task for robotics, AR/VR, and digital twins. While 3D Gaussian Splatting (3DGS) provides state-of-the art scene representation, its dynamic adaptations (4DGS) face significant challenges. Early MLP-based deformation models excel at interpolating within training frames but fail at temporal extrapolation — important feature for
forecasting in predictive systems. To address this, physically grounded approaches have been proposed to enable robust future
forecasting, while, concurrently, semantic 4DGS methods have been developed for scene analysis and
language-driven prompting. However, there is currently no framework that unites both physical consistency and semantic understanding. In this paper, we propose the first, to the best of our knowledge, unified 4DGS approach that is both physically grounded and semantically aware. Furthermore, to optimize computational efficiency, we introduce a novel rigid-body constraint mechanism. Instead of predicting the physical dynamics for every Gaussian particle, our model predicts the motion of a single anchor Gaussian per semantic object, updating the remaining Gaussians via relative transformations. This architectural design significantly accelerates novel view synthesis compared to an unconstrained baseline, while enabling object-accurate extrapolation and intuitive semantic interaction. Our project page: https://be2rlab.github.io/r5dgs/.
Read the full abstract ▾
Computer Vision
AgentGrounder: Zero-Shot 3D Visual Pointcloud Grounding using Multimodal Language Models
Cuong Huynh, Maxim Popov, Denis Gridusov et al. (+1 more)
3D Visual Grounding (3DVG) is an essential capability for
embodied AI, requiring
agents to localize objects in 3D scenes based on natural
language descriptions. Recent zero-shot methods leverage 2D
vision-
language models (LVLMs). However, they often rely on existing sets of multi-view images and struggle with the limited semantic and spatial details provided by standard 3D
segmentation tools. We present $\textbf{AgentGrounder}$, a zero-shot 3D visual grounding framework that operates directly on colored point clouds without task-specific 3D training. Our approach follows a two-stage design: (1) an offline stage that applies 3D model to build an Object Lookup Table (OLT) with instance IDs, semantic labels, 3D bounding boxes; and (2) an online tool-driven
agent that decomposes each query, retrieves only relevant candidates from the OLT, performs
geometric scoring, and triggers image rendering on demand when additional visual evidence (e.g., color, material, or viewpoint-sensitive cues) is required. Compared with fixed anchor-target matching pipelines, this design reduces cascading matching errors and improves context-window efficiency by avoiding
prompts overloaded with irrelevant objects. We evaluate on ScanRefer and Nr3D under a zero-shot setting and observe consistent improvements over SeeGround in our setup, including +2.5% Acc@0.5 on ScanRefer and +6.3% on Nr3D, with a notable +6.3% gain on Nr3D view-independent queries. These results show that combining selective
retrieval,
geometric reasoning, and adaptive visual inspection yields a practical and robust foundation for open-vocabulary 3D grounding. Our code is available at https://github.com/be2rlab/AgentGrounder.
Read the full abstract ▾
All authors Cuong Huynh, Maxim Popov, Denis Gridusov, Sergey Kolyubin
Computer Vision
SP-MoMamba: Superpixel-driven Mixture of State Space Experts for Efficient Image Super-Resolution
Wenbin Zou, Yawen Cui, Yi Wang et al. (+5 more)
State space models (SSMs) have emerged as an efficient paradigm for single-image super-resolution (SR) due to their linear complexity and long-range modeling capabilities. However, existing visual SSMs mainly focus on improving how densely represented image features are traversed, while the construction of the visual sequence itself remains largely tied to predefined spatial layouts. Inspired by Gestalt perceptual grouping, we propose SP-MoMamba, a superpixel-driven mixture of state space experts for efficient SR. Instead of performing state-space modeling over densely serialized pixel features, the proposed Superpixel-driven State Space Model (SP-SSM) organizes spatially coherent features into compact region-level
tokens and performs global sequence modeling over these content-aware representations, reducing redundant computation while facilitating long-range structural interaction. To accommodate image structures with varying representation granularities, we further develop a Multi-Scale Superpixel Mixture of State Space Experts (MSS-MoE), where scale-specific SP-SSM experts model region-level representations at different granularities and a sparse router dynamically selects an appropriate modeling scale. In addition, a Local Spatial Modulation Expert (LSME) complements region-level global modeling by refining local high-frequency details. Extensive experiments demonstrate that SP-MoMamba achieves strong reconstruction performance with a favorable trade-off among model size, computational cost, and inference efficiency.
Read the full abstract ▾
All authors Wenbin Zou, Yawen Cui, Yi Wang, Lap-Pui Chau, Liang Chen, Jinshan Pan, Huiping Zhuang, Guanbin Li
Computer Vision · Cross-listed from eess.IV
A Clinically Validated Foundation Model for Comprehensive Lung Pathology Interpretation
Zhengrui Guo, Zhengyu Zhang, Jiabo Ma et al. (+23 more)
Pathological assessment guides lung cancer diagnosis, treatment selection, and prognostic evaluation, yet current CPath approaches rely on task-specific models for isolated objectives. Although pan-cancer foundation models offer versatility, they lack subspecialty-level depth and have not been evaluated across clinical workflows or prospectively validated in real-world settings. We introduce PulmoFoundation, a multi-center, prospectively validated, randomized controlled trial (RCT)-evaluated foundation model for comprehensive lung pathology assessment across pre-operative, intra-operative, and post-operative care. Built upon Virchow2 via subspecialty-specific pretraining using ~40,000 diagnostic H&E-stained whole-slide images (WSIs), PulmoFoundation was systematically evaluated on ~26,000 WSIs across 32 clinically relevant tasks. In addition to accurately predicting molecular markers and patient survival, our model achieves clinical-grade performance in core diagnostic tasks across biopsy, frozen section, and
surgical resection slides. In a registered prospective study of 1,357 patients across 11 diagnostic tasks, our model achieved an average AUC of 92.3%. Using pre-specified triage thresholds, PulmoFoundation could reduce additional second-review burden for 68.8% of biopsies and 83.0% of frozen sections, and defer 44.5% of IHC stain orders, with PPVs of 1.000, 0.991, and 0.966. Beyond prospective validation, we conducted a crossover RCT with eight pathologists, in which AI assistance improved diagnostic accuracy across 5,264 case-reader pairs (91.7% w/ AI vs. 83.2% w/o AI). AI assistance also reduced median diagnostic time by 18.3%, increased diagnostic confidence by 9.0%, and improved inter-rater agreement from moderate (kappa = 0.55) to substantial (kappa = 0.76). Together, these evaluations support PulmoFoundation as a clinically validated decision-support system for lung pathology.
Read the full abstract ▾
All authors Zhengrui Guo, Zhengyu Zhang, Jiabo Ma, Yihui Wang, Fengtao Zhou, Yingxue Xu, Ling Liang, Chenglong Zhao, Qi Xie, Jinbang Li, Shujing Guo, Fangyi Han, Zhijian Cen, Ziyi Liu, Cheng Jin, Junlin Hou, Zhixuan Chen, Yu Cai, Lijuan Qu, Shifu Chen, Yueping Liu, Zhe Wang, Xiuming Zhang, Muyan Cai, Li Liang, Hao Chen
Computer Vision
DyCoRM: Dynamic Criterion-Aware Reward Modeling for Text-to-Image Generation
Jiaying Qian, Ziheng Jia, Qian Zhang et al. (+5 more)
With the continued advancement of text-to-image (T2I) generation, producing high-quality images is becoming increasingly attainable; consequently, user demands are shifting toward images that better satisfy their specific requirements. As reward models play an increasingly important role in assessing whether generated images align with user preference, this trend introduces an important challenge for reward modeling: rather than relying solely on static and general evaluation dimensions, reward models should account for the task-relevant and fine-grained criteria through which users assess whether generated images meet their specific requirements. To address this challenge, we propose DyCoRM, a dynamic, criterion-aware reward model that grounds task-relevant criteria and performs criterion-aware preference comparison. To support this setting, we construct DyCoDataset-20K, which provides dynamic criteria together with criterion-level annotations, and further derive DyCoBench-1K, a
benchmark for systematically evaluating reward models under dynamic criteria. We further introduce DyCoPick, which applies criterion-aware reward modeling to selecting T2I images. Our contributions establish the first reward modeling framework for dynamic and fine-grained evaluation and practical application in T2I generation.
Read the full abstract ▾
All authors Jiaying Qian, Ziheng Jia, Qian Zhang, Zicheng Zhang, Jiayi Guo, Junqi Zhang, Guangtao Zhai, Xiongkuo Min
Computer Vision
WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation
Kaining Ying, Hengrui Hu, Siyu Ren et al. (+6 more)
Interactive world models are advancing rapidly, yet existing
benchmarks cover only part of the required competencies, leaving no unified standard for systematic evaluation. To fill this gap, we introduce WBench, a comprehensive multi-turn
benchmark for interactive world model evaluation along five dimensions, namely
video quality, setting adherence, interaction adherence, consistency, and physics compliance. WBench contains 289 test cases and 1,058 interaction turns, where each case specifies a world setting and a multi-turn interaction sequence, covering diverse scenes, styles, subjects, and both first- and third-person perspectives, together with four interaction types, including
navigation, subject action, event editing, and perspective switching. For
navigation, WBench unifies text, 6-DoF pose, and discrete-action control, enabling evaluation of models with different native input interfaces. Evaluation uses 22 automatic sub-metrics that combine specialist
vision models with large
multimodal models, and all metrics are validated against human judgments. Across 20 state-of-the-art models, we find that no single model performs strongly across all dimensions. We provide detailed diagnostic insights into the characteristic strengths, weaknesses, and open challenges of each model. Code and data are available at https://github.com/meituan-longcat/WBench.
Read the full abstract ▾
All authors Kaining Ying, Hengrui Hu, Siyu Ren, Jiamu Li, Fengjiao Chen, Ziwen Wang, Xuezhi Cao, Xunliang Cai, Henghui Ding
Computer Vision
MuNet: A Mutualistic Network for Joint 3D Human Mesh Recovery and 3D Clothed Human Reconstruction from Single Images
Yunqi Gao, Leyuan Liu, Yuhan Li et al. (+2 more)
3D human mesh recovery and 3D clothed human reconstruction are inherently related, yet they have long been studied in isolation, thereby overlooking the potential gains of joint optimization. To overcome this limitation, we propose to address these two tasks within a unified framework, which allows their mutual dependencies to be effectively exploited. Building on this idea, we propose MuNet, a mutualistic network for joint 3D human mesh recovery and 3D clothed human reconstruction from single images. First, we adopt 2-
manifold graphs as a unified representation for all 3D models, enabling consistent modeling across 3D human mesh recovery and clothed human reconstruction. Second, we design an end-to-end graph convolutional network that progressively deforms an initial graph into a 3D human mesh and refines it into a detailed 3D clothed human model. Third, we introduce a mutualistic mechanism that allows reciprocal interaction between the two tasks {during training}, where 3D human mesh recovery provides guidance for 3D clothed human reconstruction, and reconstruction feedback refines the 3D human mesh recovery. We extensively evaluate MuNet on six
benchmark datasets for 3D human mesh recovery and 3D clothed human reconstruction, including Human3.6M, 3DPW, MPI-INF-3DHP, THuman2.0, CAPE, and RenderPeople. Experimental results demonstrate that MuNet achieves state-of-the-art performance on both tasks across all
datasets. The code of MuNet is released for research purposes at https://github.com/starVisionTeam/MuNet.
Read the full abstract ▾
All authors Yunqi Gao, Leyuan Liu, Yuhan Li, Changxin Gao, Jingying Chen
Computer Vision
SAM3-Assisted Training of Lightweight YOLO Models for Precision Pig Farming
Marcos Vinicius Mendes Faria, Thiago Borges Pereira, Isabella C.F.S. Condotta et al. (+2 more)
Deep learning-based object detection has revolutionized Precision Livestock Farming (PLF), yet a critical barrier remains: high-performance Foundation Models (such as SAM 3) are too computationally intensive for edge deployment, while lightweight models (like YOLO) require prohibitive manual annotation efforts. This work proposes a fully automated knowledge
distillation pipeline that leverages the Segment Anything Model 3 (SAM 3) to generate zero-shot pseudo-labels for training efficient YOLOv8 detectors. By treating SAM 3 as an offline auto-annotator, we eliminate the manual labeling bottleneck, producing models capable of real-time inference on resource-constrained hardware. We systematically evaluate this approach on the PigLife
dataset, comparing SAM 3-supervised models against human-annotated baselines. Results demonstrate that a SAM 3-trained YOLOv8m achieves a mean Average Precision (mAP) of 79.4% without human intervention, while reducing inference latency by approximately 200$\times$ compared to the teacher model. Furthermore, stratified analysis reveals that in low-occlusion scenarios, the automated pipeline achieves detection rates comparable to human
benchmarks ($AP_{50} > 99\%$). These findings indicate that foundation models can serve as effective, zero-annotation-cost supervisors, enabling scalable edge computing solutions for smart agriculture.
Read the full abstract ▾
All authors Marcos Vinicius Mendes Faria, Thiago Borges Pereira, Isabella C.F.S. Condotta, Thiago Meireles Paixão, Francisco de Assis Boldt
Computer Vision · Cross-listed from cs.RO
When Search Becomes Memory: Accelerating Robot Design Discovery with Self-Evolving Skills
Yunfei Wang, Xiaohao Xu, Yang Li et al. (+1 more)
Large
language models (LLMs) are increasingly used as proposal generators for evolutionary
robot design, yet most loops remain memoryless: simulator results shape the next population but are not preserved as reusable design knowledge. We present Auto-Robotist, a self-evolving LLM
agent that distills morphology-search traces into an explicit natural-
language skill library. Each skill stores a structural archetype, evidence-grounded positive and negative rules, and the evaluated designs that support them, making design
memory inspectable rather than implicit in a population. During search, the
agent retrieves skills to condition LLM edits of elite bodies while retaining a Genetic
Algorithm (GA) mutation path for exploration; after evaluation, it updates the library through Add, Diagnose, and Merge. Across seven EvoGym tasks spanning locomotion, traversal, and object interaction, Auto-Robotist improves cold-start 5x5 search and transfers learned skills to 10x10 design spaces, where reference-conditioned transfer outperforms GA on every task. These results suggest that LLM
agents can convert expensive physical evaluations into reusable, auditable design principles. Our code is publicly available at https://github.com/wangyf9/Auto-Robotist .
Read the full abstract ▾
All authors Yunfei Wang, Xiaohao Xu, Yang Li, Xiaonan Huang
Computer Vision
[CLS] is Not Enough: Multi-Label Recognition via Patch-Level Inference and Adaptive Aggregation
Akang Wang, Xili Deng, Zhanxuan Hu et al. (+3 more)
Vision-
Language Models such as CLIP exhibit strong zero-shot recognition capability by aligning images with textual concepts, yet they often underperform on multi-label recognition where multiple objects co-exist. A key bottleneck is that the [CLS]
token, as a single global visual representation, is insufficient to faithfully encode diverse targets with varying scales, contexts, and co-occurrence patterns. To address this limitation, we present a new multi-label image recognition framework, termed PIAA, which formulates prediction as Patch-level Inference followed by Adaptive Aggregation. Specifically, we first enhance patch-wise predictions from two complementary perspectives: (i) mitigating semantic
entanglement in the visual encoder to obtain more discriminative patch representations, and (ii) learning an unsupervised visual classifier to narrow the
vision-
language modality gap. We then introduce an adaptive aggregation module that consolidates patch-level scores into the final multi-label prediction. Notably, the entire pipeline is fully training-free, requiring no gradient updates or parameter fine-tuning. Experiments show that our method achieves strong improvements with minimal extra computation, exceeding a 6% mAP gain on the challenging NUS-WIDE
benchmark over representative baselines. Code is available at https://github.com/akang-wang/PIAA.
Read the full abstract ▾
All authors Akang Wang, Xili Deng, Zhanxuan Hu, Yi Zhao, Yonghang Tai, Huafeng Li
Computer Vision
Data-driven Head Motion Generation through Natural Gaze-Head Coordination
Xiaohan Liu, Yilin Wen and Yusuke Sugano
We present the first data-driven approach to model temporal gaze-head coordination from large-scale in-the-wild facial
videos. To obtain training data for generalizable learning, we propose an automatic pipeline that extracts natural yet diverse gaze and head motions with off-the-shelf appearance-based gaze
estimators. To capture the probabilistic correlation and temporal dynamics of gaze-head coordination, we build our model on a generative conditional Variational Autoencoder for plausible yet diverse gaze-conditioned head motion generations. We further apply our framework to gaze-controlled facial
video generation, where we enable
video generation with natural and realistic head motion correlated to the input gaze - an aspect that has not been emphasized before. Human evaluation and quantitative comparisons demonstrate our method's effectiveness and validate our design choices, with evaluators showing statistically significant preference for our approach over baseline methods.
Read the full abstract ▾
Computer Vision
An Analysis Focused on Womens Safety: Can VAD Models Be Enhanced by a Multi-modal Dataset?
Sangeeta ., Maddikuntla Sai Prajwal, Debi Prosad Dogra et al. (+4 more)
Women's safety and security are paramount for a modern society. Often, crimes scenes get recorded through low-resolution CCTV cameras limiting the efficiency of
video anomaly detection (VAD) models. Despite substantial progress in VAD research, women-centric anomalies are still underrepresented in
datasets as well as in models. Existing
datasets primarily cover well-lit, high-resolution and close-shot
videos that are inadequate to tackle critical anomalies such as chain snatching, stalking, inappropriate touch, and other subtle forms of crime against women. To address this, we present a new
benchmark, referred to as ExtrAnom. It contains 1001
videos (both anomalies and normal) with four textual annotations; one human-generated and three LLM-generated. The
videos are arranged in 5 different categories of crimes. The
dataset comprises low-light (8%), low-resolution (13%), long-shot (15%), and daytime (64%) anomaly
videos. It includes stalking (3.9%), chain snatching (17.6%), kidnapping (7.3%), assassinations (2.3%), harassment (18.9%), and normal (50%)
videos. It is possible to perform cross-modal and VLM-based validations using ExtrAnom. We have benchmarked it against popular unimodal and multi-modal VAD
datasets (e.g., XD-Violence, UCF-Crime, and UCA) and SOTA methods. Experiments reveal that existing
datasets are insufficient to deal with women-centric anomalies. We believe ExtrAnom can fill this critical gap in VAD research.
Read the full abstract ▾
All authors Sangeeta ., Maddikuntla Sai Prajwal, Debi Prosad Dogra, Kamalakar Vijay Thakare, Hyungjoo Jung, Ig-Jae Kim, Heeseung Choi
Computer Vision
Event-to-Video Reconstruction using Spatio-Temporal and Frequency-Enhanced Deep Neural Networks
Ramna Maqsood, Paulo Nunes, Luís Ducla Soares et al. (+1 more)
Event cameras offer significant advantages over conventional frame-based counterparts, including high temporal resolution, low latency, and energy efficiency. These characteristics make them suitable for high-speed and high-dynamic range scene acquisition scenarios; however, the lack of dense intensity frames limits the direct applicability of conventional computer
vision methods for scene understanding. Event-to-
video (E2V) reconstruction seeks to bridge this gap by converting asynchronous event streams into a sequence of synchronous
video frames. Existing E2V reconstruction methods based on convolutional neural networks and
transformers operate primarily in the spatial domain and often struggle to recover fine structural details while suppressing severe reconstruction artifacts. To address these issues, we propose MSFET-E2V, a novel multiscale frequency-enhanced
transformer model. At its core lies a cross-domain
attention module, which fuses spatio-temporal features with frequency-aware representations derived from the discrete wavelet transform. Unlike prior methods relying solely on spatial
attention, our approach effectively captures both local and global structures by taking into account low- and high-frequency components, enhancing detail preservation and robustness across various motion scenarios. Furthermore, we propose a lightweight wavelet-enhanced skip block that serves as a skip connection, facilitating artifact suppression and structural detail refinement through joint spatial-frequency domain processing. Extensive experiments demonstrate that MSFET-E2V achieves superior performance over state-of-the-art methods on multiple real-world event
datasets, offering significant gains in reconstruction quality. Moreover, compared to the existing
transformer-based method, our proposed model significantly reduces the number of parameters, the GPU
memory usage, and inference time.
Read the full abstract ▾
All authors Ramna Maqsood, Paulo Nunes, Luís Ducla Soares, Caroline Conti
Computer Vision
ATV-Net: Adaptive Triple-View Network with Dynamic Feature Fusion
Sheng-Wei Chan, Hsin-Jui Pan, Chun-Po Shen et al. (+3 more)
Recent advances in semantic
segmentation rely heavily on
attention-based and
transformer-style architectures that, while accurate, introduce considerable architectural complexity and computational cost. This paper asks whether a compact CNN-based
segmentation head can remain competitive by adaptively selecting useful receptive-field evidence. We propose ATV-Net, an Adaptive Triple-View Network that attaches a lightweight head to a conventional backbone. The head organizes three complementary views – point-wise, neighborhood-level, and enlarged context – and fuses them through an Adaptive Decision Gate that generates image-dependent weights from global feature statistics. This allows the model to emphasize different receptive-field responses according to scene content, without dense
attention or multi-scale aggregation. Experiments on Cityscapes and Pascal VOC 2012 show that ATV-Net achieves 80.31% mIoU on Cityscapes with ResNet-101 and 80.90% with ConvNeXt-Tiny, and 86.7% and 88.5% mIoU on Pascal VOC 2012, respectively, while requiring fewer GFLOPs than representative context-aggregation and
attention-based heads. The results indicate that adaptive receptive-field selection remains a practical and effective design choice for CNN-based semantic
segmentation.
Read the full abstract ▾
All authors Sheng-Wei Chan, Hsin-Jui Pan, Chun-Po Shen, Chia-Min Lin, Yung-Che Wang, Jen-Shiun Chiang
Computer Vision
Rethinking VLM Representation for VLA Initialization
Weifeng Lin, Siyuan Huang, Hao Li et al. (+5 more)
Vision-
Language-Action (VLA) models widely adopt pretrained
Vision-
Language Models (VLMs) as policy backbones, yet it remains unclear what kind of pretrained VLM representation is useful as a VLA initialization. In this paper, we study VLA initialization as a controlled representation-design problem along three axes: capability-level
embodied VQA supervision, parameter-update strategy, and
robot-data pretraining. Our experiments show that the original pretrained VLM representation is a key source of action performance. However,
embodied VQA adaptation does not yield uniform gains: its benefit depends on downstream bottlenecks, and gains from different capability domains are not simply additive. For update strategy, LoRA provides a more reliable initialization than Full Finetune, indicating that overly reshaping the pretrained representation can weaken VLA initialization.
Robot-data pretraining further improves VLA initialization, with the strongest variant obtained by staged LoRA-based training. Together, these findings suggest that effective VLM-to-VLA adaptation should inject action-relevant
embodied and
robot-trajectory signals while preserving the pretrained VLM representation that remains useful for action learning.
Read the full abstract ▾
All authors Weifeng Lin, Siyuan Huang, Hao Li, Tingwei Chen, Ruichuan An, Xinyu Wei, Jianbo Liu, Hongsheng Li
Computer Vision
PixelWizard: Towards Efficient High-Fidelity Video Generation at Ultra-Large Spatial Resolution
Wenxue Li, Jingjing Ren, Peng Zhang et al. (+6 more)
High-resolution
video generation faces a coupled bottleneck of optimization instability and prohibitive computational costs. The massive expansion of the
token sequence not only biases optimization toward local textures at the expense of global coherence, leading to structural collapse, but also imposes prohibitive training costs and severe inference latency. To address this, we propose PixelWizard, a framework that hierarchically decouples global structure modeling from fine-grained detail synthesis. PixelWizard first establishes a compact spatiotemporal anchor to concentrate dense structural priors, which then guides fine-grained generation at high resolution. This mitigates the local optimization bias to ensure structural stability without compromising high-frequency details. Leveraging this structural stability, we introduce Noise-Span Aligned Shortcut Training to break the inference bottleneck. By explicitly modeling the step size, this mechanism allows the model to traverse the generation trajectory with large steps. Crucially, we incorporate Exponential Index-Biased Sampling and Adaptive Noise-Span Calibration to align optimization with the shifted noise schedules of high-resolution grids, ensuring robust few-step inference without incurring the heavy overhead of
distillation. Extensive experiments demonstrate that PixelWizard achieves superior visual quality while accelerating the generative sampling of native 2K/4K
videos by over 10x.
Read the full abstract ▾
All authors Wenxue Li, Jingjing Ren, Peng Zhang, Tian Ye, Daiguo Zhou, Jian Luan, Xinhu Zheng, Gang Hua, Lei Zhu
Computer Vision
Addressing Exacerbated Attention Sink for Source-Free Cross-Domain Few-Shot Learning
Shuai Yi, Yixiong Zou, Yuhua Li et al. (+1 more)
Vision-
language models (VLMs) like CLIP have shown impressive generalization capabilities, yet their potential for Cross-Domain Few-Shot Learning (CDFSL) remains underexplored, where the model needs to transfer source-domain information to target domains with scarce training data. While the
attention sink phenomenon has been observed in VLMs for certain tasks, its role in CDFSL scenarios has not been studied. In this paper, we uncover a critical issue overlooked by prior works: standard target-domain few-shot fine-tuning in CDFSL significantly exacerbates the
attention sink problem, leading to poor discriminability across classes. To understand this phenomenon, through extensive experiments, we interpret it as the model's shortcut learning for domain adaptation: to overcome the huge domain gap between the source and target domains, the model shows a high tendency to push
tokens that are initially closer to target-domain classes (i.e., simple
tokens) to be even closer to these classes, exacerbating the
attention sink and wasting the capability of learning other discriminative but initially further
tokens (i.e., hard
tokens). To address this, we propose a novel approach to dynamically re-weight
tokens according to their relevance with target-domain classes during the target-domain finetuning, which explicitly suppresses the model's reliance on these simple
tokens and enhances the learning of hard
tokens, reducing sink
tokens and enhancing discriminability. Extensive experiments on four
benchmark datasets validate the rationale of our method, demonstrating new state-of-the-art performance. Our codes are available at https://github.com/shuaiyi308/TIR.
Read the full abstract ▾
All authors Shuai Yi, Yixiong Zou, Yuhua Li, Ruixuan Li
Computer Vision
Beyond Appearance: Can Multimodal Large Language Models Exploit Vertical Structure for Remote Sensing Natural Scene Understanding?
Jing Huang, Duanchu Wang, Junjie Yang et al. (+5 more)
Multimodal large
language models (MLLMs) have advanced rapidly in remote-sensing analysis, yet existing evaluations remain predominantly 2D-centric. Because spectrally confused regions can appear nearly identical yet differ substantially in vertical structure, appearance alone is often insufficient for reliable semantic interpretation in natural scenes. Vertical structure therefore provides decision-critical physical evidence, yet whether current MLLMs can effectively perceive, ground, and utilize such
geometric evidence remains underexplored. To bridge this gap, we introduce VertiCue-Bench, the first diagnostic
benchmark that uses controlled interventions to probe whether vertical height evidence is actually perceived, grounded, and utilized, and we establish a three-stage evidence-utilization framework of Perception–Grounding–Utilization. By constructing a Representation Intervention Spectrum spanning multiple presentation and interaction modalities, including Raw Visual, Tool-assisted, and Oracle Text conditions, together with controlled counterfactual tests, we conduct an in-depth disentangled diagnosis across 10 state-of-the-art models. Our experiments reveal and formally characterize the Vertical Structure Utilization Gap. Although current models exhibit emerging
geometric perception capabilities, they still struggle to accurately ground vertical evidence to relevant spatial entities and integrate it into high-level semantic decisions. This finding identifies a critical bottleneck in developing physically grounded and geometry-aware remote-sensing MLLMs.
Read the full abstract ▾
All authors Jing Huang, Duanchu Wang, Junjie Yang, Zihang Cheng, Cheng Li, Lin Cui, Zhouyi Wu, Di Wang
Computer Vision
OMGTex: One-stage Multi-style Facial Texture Reconstruction without Geometry Guidance
Zitong Xiao, Yuda Qiu, Zisheng Ye et al. (+1 more)
We propose OMGTex, an end-to-end
diffusion-based framework for reconstructing high-quality and editable facial UV textures from multi-style facial images. Existing texture reconstruction methods face two major limitations: (1) Fragility due to reliance on 3D geometry priors, which are difficult to estimate accurately, especially under facial occlusions or in stylized domains; and (2) A lack of semantic disentanglement, inhibiting region-specific texture editing and style transfer. Our work addresses both challenges simultaneously. Our core innovation is a geometry-free pipeline that directly maps a 2D face image to its corresponding editable UV texture. We introduce two key techniques: First, to address the challenge of UV misalignment common in
diffusion generation, we introduce a gradient-guided refinement strategy at inference time, which explicitly corrects structural consistency. Second, we leverage the inherent semantic distribution capability of
diffusion models and design a novel training paradigm to enhance this tendency, enabling semantic-aware editing of facial texture. Furthermore, to address the data scarcity in multi-style texture reconstruction, we construct CANVAS, the first comprehensive paired texture reconstruction
dataset covering realistic and diverse stylized domains. To the best of our knowledge, OMGTex is the first geometry-free inference framework that achieves robust, style-consistent, and editable facial texture reconstruction across diverse domains. Our method achieves state-of-the-art performance on multiple facial texture
benchmarks.
Read the full abstract ▾
All authors Zitong Xiao, Yuda Qiu, Zisheng Ye, Xiaoguang Han
Computer Vision
DRFusion: Drift-Resilient Temporally Consistent Infrared-Visible Video Fusion
Xingyuan Li, Haoyuan Xu, Shulin Li et al. (+3 more)
Infrared and visible
video fusion is essential for achieving comprehensive perception in dynamic scenes. However, maintaining temporal consistency remains a formidable challenge. Conventional methods relying on optical flow often suffer from
geometric rigidity and ghosting artifacts. Moreover, standard
diffusion-based fusion models typically operate in a frame-by-frame manner; when extended to autoregressive settings, they lack intrinsic temporal constraints and are prone to severe error accumulation and drifting, where minor artifacts amplify over time. To address these limitations, we propose a drift-resilient
video fusion method that reformulates the task as history-conditioned motion generation. We introduce Stabilized History Guidance and Soft Temporal Anchoring to reframe temporal consistency as spectral filtering, implicitly aggregating motion dynamics without rigid
alignment. Furthermore, our Decoupled Structure-Motion Adaptation strategy bridges pre-trained priors and structural constraints via two-stage training and latent refinement. Extensive experiments demonstrate that our method achieves state-of-the-art performance in both fusion quality and temporal stability.
Read the full abstract ▾
All authors Xingyuan Li, Haoyuan Xu, Shulin Li, Xiang Chen, Zhiying Jiang, Jinyuan Liu
Computer Vision
SAFE-Diff: Scale-Aware Attention and Feature-Dispersive Diffusion with Uncertainty Estimation for Contrast-Enhanced Breast MRI Synthesis
Tianyu Zhang, Xinglong Liang, Jarek van Dijk et al. (+13 more)
Synthesizing high fidelity contrast enhanced MRI is clinically valuable for safer and more efficient breast cancer screening, yet remains challenging due to complex lesion textures and heterogeneous enhancement patterns.
Read the full abstract ▾
All authors Tianyu Zhang, Xinglong Liang, Jarek van Dijk, Luyi Han, Chunyao Lu, Antonio Portaluri, Xinghe Xie, Yaofei Duan, Nika Rasoolzadeh, Xin Wang, Yuan Gao, Muzhen He, Yue Sun, Jonas Teuwen, Tao Tan, Ritse Mann
Computer Vision
Concept Unlearning via Cross-Attention Activation Projection for Diffusion Models
Saemi Moon, Suhyeon Jun, Seoyeon Lee et al. (+1 more)
Existing closed-form methods for concept unlearning in text-to-image
diffusion models typically derive editing directions from fixed text
embeddings, which may not fully capture how concepts are expressed across latent states, timesteps, and layers. To capture this variation, we investigate cross-
attention activations collected during denoising. In controlled probing experiments using the same anchor
prompts, activation-derived bases achieve approximately five times the recall of text-derived bases on held-out
prompts expressing the target concepts. Based on this finding, we propose Cross-
Attention Subspace Erasure (CASE), a closed-form method that constructs layer-specific forget and retain subspaces from cross-
attention activations. These subspaces define a retain-constrained linear operator incorporated directly into cross-
attention weights, requiring no gradient-based fine-tuning or additional inference-time computation. Across ten concepts spanning four categories, CASE achieves the highest harmonic-mean score among evaluated baselines in all four categories, balancing suppression, retention,
adversarial robustness, and generation quality. Further experiments demonstrate robustness to recovery attacks and a favorable suppression-retention trade-off when jointly unlearning up to 100 artistic styles. The benefits of activation-derived editing also extend to larger
diffusion models, including SDXL and FLUX.
Read the full abstract ▾
All authors Saemi Moon, Suhyeon Jun, Seoyeon Lee, Dongwoo Kim
Computer Vision
Benchmarking Pathology Foundation Models for Spatial Domain Understanding
Bokai Zhao, Yiyang Zhang, Yuanchi Zhu et al. (+6 more)
Pathology foundation models (PFMs) have emerged as a core approach for learning transferable representations from whole slide images (WSIs), and they are typically benchmarked through downstream clinical endpoints. While such task level evaluations are indispensable, they offer limited insight into what the representations themselves encode, particularly whether PFM
embeddings can distinguish meaningful tissue regions and capture their spatial relationships. We present SpaPath-Bench, a representation level
benchmark designed to diagnose spatial representation capability in PFMs. SpaPath-Bench formulates spatial domain identification (SDI) on paired whole slide image and spatial transcriptomics (ST) data as a diagnostic task. It curates 42 public paired WSI and ST slides, enables large scale evaluation across 19 encoders and seven SDI methods, and measures partition quality using three complementary criteria: unsupervised spatial coherence, transcriptomics referenced agreement, and expert referenced agreement. Across 83K runs, SpaPath-Bench reveals that different pretraining paradigms capture distinct aspects of tissue spatial architecture, and it provides practical guidance for building the next generation of spatially aware computational pathology models. Code and data pipelines are publicly available at https://bokai-zhao.github.io/SpaPath-benchboard/.
Read the full abstract ▾
All authors Bokai Zhao, Yiyang Zhang, Yuanchi Zhu, Hanqing Chao, Long Bai, Tai Ma, Minfeng Xu, Ming Song, Tianzi Jiang
Computer Vision
AI-T2I: Aggregating-and-Isolating Cross-Attention to Diffusion Models for Text-to-Image Synthesis
Shipeng Cao, Biao Qian, Haipeng Liu et al. (+2 more)
Text-to-image synthesis has made significant progress, benefiting from the strong generative capabilities of
diffusion models. However, these models struggle to achieve precise text-to-image
alignment within cross-
attention maps during the denoising process. Existing works primarily focus on inter-subject-
token activations (i.e., cross-
attention scores) overlap for different subjects, overlooking the intra-subject-
token activations scattering issue for identical subjects. In this paper, we propose an Aggregating-and-Isolating cross-
attention approach to
diffusion models for Text-to-Image synthesis, dubbed AI-T2I. Technically, to address the scattering issue, we devise an aggregation loss to identify and consolidate the scattered intra-
token activations, which implicitly helps mitigate the potential overlap issue. Upon that, an isolation loss is further introduced to push the inter-
token activations apart, thus fulfilling precise text-to-image
alignment. Extensive experiments on various
benchmarks demonstrate the superiority of AI-T2I over the state-of-the-art works for text-to-image synthesis. Furthermore, our AI-T2I exhibits excellent generalization across other tasks, e.g., controllable layout generation and personalized generation. Our code is available at https://github.com/Hatter77/AI-T2I.
Read the full abstract ▾
All authors Shipeng Cao, Biao Qian, Haipeng Liu, Yang Wang, Meng Wang