Computer Vision
FlowMotion: Training-Free Flow Guidance for Video Motion Transfer
Zhen Wang, Youcan Xu, Jun Xiao et al. (+1 more)
Video motion transfer aims to generate a target
video that inherits motion patterns from a source
video while rendering new scenes. Existing training-free approaches focus on constructing motion guidance based on the intermediate outputs of pre-trained T2V models, which results in heavy computational overhead and limited flexibility. In this paper, we present FlowMotion, a novel training-free framework that enables efficient and flexible motion transfer by directly leveraging the predicted outputs of flow-based T2V models. Our key insight is that early latent predictions inherently encode rich temporal information. Motivated by this, we propose flow guidance, which extracts motion representations based on latent predictions to align motion patterns between source and generated
videos. We further introduce a velocity regularization strategy to stabilize optimization and ensure smooth motion
evolution. By operating purely on model predictions, FlowMotion achieves superior time and resource efficiency as well as competitive performance compared with state-of-the-art methods.
Read the full abstract ▾
All authors Zhen Wang, Youcan Xu, Jun Xiao, Long Chen
Computer Vision
Learning Adaptive and Visually Aligned Conditions for Generative Zero-Shot Learning
Haojie Pu, Zhuoming Li, Yongbiao Gao et al. (+3 more)
Generative zero-shot learning (ZSL) synthesizes visual features for unseen classes by learning a semantic-conditioned generator from seen classes. Since semantic conditions determine the knowledge transfer from seen to unseen classes, their effectiveness is critical for generating plausible features. However, existing class-level semantic prototypes lack semantic diversity to capture intra-class variations, while exhibiting inconsistent inter-class structures with the visual space due to the semantic-visual gap. To address these issues, we propose Adaptive Attribute Distribution and Visual Structure
Alignment (AAVS), a unified semantic condition learning framework for generative ZSL. Specifically, the Adaptive Attribute Distribution (AAD) learns dimension-adaptive attribute distributions with discriminability-guided variation calibration to capture transferable semantic diversity, while Visual Structure
Alignment (VSA) aligns the diverse attributes with refined visual prototypes to learn visual inter-class structures. By jointly modeling semantic diversity and visual structural consistency, AAVS provides more effective generation conditions for unseen classes to generate visual features. Extensive quantitative and qualitative evaluations on three
benchmarks demonstrate that AAVS outperforms existing generative ZSL methods and validate the effectiveness of the learned generation conditions.
Read the full abstract ▾
All authors Haojie Pu, Zhuoming Li, Yongbiao Gao, Hui Liu, Junhui Hou, Yuheng Jia
Computer Vision
Can we Trust Unreliable Voxels? Exploring 3D Semantic Occupancy Prediction under Label Noise
Wenxin Li, Kunyu Peng, Di Wen et al. (+6 more)
3D semantic occupancy prediction is a cornerstone of robotic perception, yet real-world voxel annotations are inherently corrupted by structural artifacts and dynamic trailing effects. This raises a critical but underexplored question: can autonomous systems safely rely on such unreliable occupancy supervision? To systematically investigate this issue, we establish OccNL, the first
benchmark dedicated to 3D occupancy under occupancy-asymmetric and dynamic trailing noise. Our analysis reveals a fundamental domain gap: state-of-the-art 2D label noise learning strategies collapse catastrophically in sparse 3D voxel spaces, exposing a critical vulnerability in existing paradigms. To address this challenge, we propose DPR-Occ, a principled label-noise-robust framework that constructs reliable supervision through dual-source partial label
reasoning. By synergizing temporal model
memory with representation-level structural affinity, DPR-Occ dynamically expands and prunes candidate label sets to preserve true semantics while suppressing noise propagation. Extensive experiments on SemanticKITTI demonstrate that DPR-Occ prevents
geometric and semantic collapse under extreme corruption. Notably, even at 90% label noise, our method achieves significant performance gains (up to 2.57% mIoU and 13.91% IoU) over existing label noise learning baselines adapted to the 3D occupancy prediction task. By bridging label noise learning and 3D perception, OccNL and DPR-Occ provide a reliable foundation for safety-critical robotic perception in dynamic environments. The
benchmark and source code will be made publicly available at https://github.com/mylwx/OccNL.
Read the full abstract ▾
All authors Wenxin Li, Kunyu Peng, Di Wen, Junwei Zheng, Jiale Wei, Mengfei Duan, Yuheng Zhang, Rui Fan, Kailun Yang
Computer Vision
Spectral and Trajectory Regularization for Diffusion Transformer Super-Resolution
Jingkai Wang, Yixin Tang, Jue Gong et al. (+6 more)
Diffusion transformer (DiT) architectures show great potential for real-world image super-resolution (Real-ISR). However, their computationally expensive iterative sampling necessitates one-step
distillation. Existing one-step
distillation methods struggle with Real-ISR on DiT. They suffer from fundamental trajectory mismatch and generate severe grid-like periodic artifacts. To tackle these challenges, we propose StrSR, a novel one-step
adversarial distillation framework featuring spectral and trajectory regularization. Specifically, we propose an asymmetric discriminative
distillation architecture to bridge the trajectory gap. Additionally, we design a frequency distribution matching strategy to effectively suppress DiT-specific periodic artifacts caused by high-frequency spectral leakage. Extensive experiments demonstrate that StrSR achieves state-of-the-art performance in Real-ISR, across both quantitative metrics and visual perception. The code and models will be released at https://github.com/jkwang28/StrSR .
Read the full abstract ▾
All authors Jingkai Wang, Yixin Tang, Jue Gong, Jiatong Li, Shu Li, Libo Liu, Jianliang Lan, Yutong Liu, Yulun Zhang
Computer Vision
HiPP-Prune: Hierarchical Preference-Conditioned Structured Pruning for Vision-Language Models
Lincen Bai, Hedi Tabia and Raul Santos-Rodriguez
Pruning vision-
language models (VLMs) for efficient deployment is challenging because compression can affect not only task utility but also visual grounding, often amplifying object
hallucinations even at the same sparsity level. We present HiPP-Prune, a hierarchical preference-conditioned structured
pruning framework that treats
pruning as conditional resource allocation under multiple objectives. HiPP-Prune makes plan-level decisions: a single policy invocation outputs a global
pruning blueprint by factorizing decisions into an overall sparsity budget and a layer-wise allocation, enabling queryable trade-offs via a user-specified preference vector. To account for VLM-specific failure modes, our policy state integrates a visual sensitivity signal derived from
attention flow between
vision tokens and
language hidden states, discouraging over-
pruning of
vision-critical layers that facilitate cross-modal fusion. We optimize
pruning plans with plan-level Group Relative Policy Optimization (GRPO) under a multi-objective return that combines task utility,
hallucination robustness (POPE), compression, and a synaptic-flow-inspired stability proxy to reduce unproductive exploration in high-sparsity regimes. Experiments on LLaVA with POPE and ScienceQA demonstrate that HiPP-Prune discovers diverse non-dominated
pruning plans and provides controllable robustness–utility trade-offs under matched sparsity budgets.
Read the full abstract ▾
Computer Vision
ODD-SEC: Onboard Drone Detection with a Spinning Event Camera
Kuan Dai, Hongxin Zhang, Sheng Zhong et al. (+1 more)
The rapid proliferation of drones requires balancing innovation with regulation. To address security and
privacy concerns, techniques for drone detection have attracted significant
attention.Passive solutions, such as frame camera-based systems, offer versatility and energy efficiency under typical conditions but are fundamentally constrained by their operational principles in scenarios involving fast-moving targets or adverse illumination.Inspired by biological
vision, event cameras asynchronously detect per-pixel brightness changes, offering high dynamic range and microsecond-level responsiveness that make them uniquely suited for drone detection in conditions beyond the reach of conventional frame-based cameras.However, the design of most existing event-based solutions assumes a static camera, greatly limiting their applicability to moving carriers–such as quadrupedal
robots or unmanned ground vehicles–during field operations.In this paper, we introduce a real-time drone detection system designed for deployment on moving carriers. The system utilizes a spinning event-based camera, providing a 360{\deg} horizontal field of view and enabling bearing estimation of detected drones. A key contribution is a novel image-like event representation that operates without motion compensation, coupled with a lightweight neural network architecture for efficient spatiotemporal learning. Implemented on an onboard Jetson Orin NX, the system can operate in real time. Outdoor experimental results validate reliable detection with a mean angular error below 2{\deg} under challenging conditions, underscoring its suitability for real-world surveillance applications. We will open-source our complete pipeline to support future research.
Read the full abstract ▾
All authors Kuan Dai, Hongxin Zhang, Sheng Zhong, Yi Zhou
Computer Vision
GazeMoE: Perception of Gaze Target with Mixture-of-Experts
Zhuangzhuang Dai, Zhongxi Lu, Vincent G. Zakka et al. (+3 more)
Estimating human gaze target from visible images is a critical task for
robots to understand human
attention, yet the development of generalizable neural architectures and training paradigms remains challenging. While recent advances in pre-trained
vision foundation models offer promising avenues for locating gaze targets, the integration of multi-modal cues – including eyes, head poses, gestures, and contextual features – demands adaptive and efficient decoding mechanisms. Inspired by Mixture-of-Experts (MoE) for adaptive domain expertise in large
vision-
language models, we propose GazeMoE, a novel end-to-end framework that selectively leverages gaze-target-related cues from a frozen foundation model through MoE modules. To address class imbalance in gaze target classification (in-frame vs. out-of-frame) and enhance robustness, GazeMoE incorporates a class-balancing auxiliary loss alongside strategic data augmentations, including region-specific cropping and photometric transformations. Extensive experiments on
benchmark datasets demonstrate that our GazeMoE achieves state-of-the-art performance, outperforming existing methods on challenging gaze estimation tasks. The code and pre-trained models are released at https://huggingface.co/zdai257/GazeMoE
Read the full abstract ▾
All authors Zhuangzhuang Dai, Zhongxi Lu, Vincent G. Zakka, Luis J. Manso, Jose M Alcaraz Calero, Chen Li
Computer Vision
NOVA: Next-step Open-Vocabulary Autoregression for 3D Multi-Object Tracking in Autonomous Driving
Kai Luo, Xu Wang, Rui Fan et al. (+1 more)
Generalizing across unknown targets is critical for open-world perception, yet existing 3D Multi-Object Tracking (3D MOT) pipelines remain limited by closed-set assumptions and “semantic-blind” heuristics. To address this, we propose Next-step Open-Vocabulary Autoregression (NOVA), an autoregressive association formulation that shifts the data association stage from fragmented distance-based matching toward trajectory-conditioned spatio-semantic modeling. NOVA reformulates 3D trajectories as structured spatio-temporal semantic sequences, enabling the simultaneous encoding of physical motion continuity and deep linguistic priors. By leveraging the autoregressive capabilities of Large
Language Models (LLMs), we transform the tracking task into a principled process of next-step sequence completion. This mechanism allows the model to explicitly utilize the hierarchical structure of
language space to resolve fine-grained semantic ambiguities and maintain identity consistency across complex long-range sequences through high-level commonsense
reasoning. Extensive experiments on nuScenes, V2X-Seq-SPD, and KITTI demonstrate the superior performance of NOVA. Notably, on the nuScenes
dataset, NOVA achieves an AMOTA of 22.41% for Novel categories, yielding a significant 20.21% absolute improvement over the baseline. These gains are realized through a compact 0.5B autoregressive model. Code will be available at https://github.com/xifen523/NOVA.
Read the full abstract ▾
All authors Kai Luo, Xu Wang, Rui Fan, Kailun Yang
Computer Vision
Hierarchical Collaborative Fusion for 3D Instance-aware Referring Expression Segmentation
Keshen Zhou, Runnan Chen, Mingming Gong et al. (+1 more)
Generalised 3D Referring Expression
Segmentation (3D-GRES) localizes objects in 3D scenes based on natural
language, even when descriptions match multiple or zero targets. Existing methods rely solely on sparse point clouds, lacking rich visual semantics for fine-grained descriptions. We propose HCF-RES, a multi-modal framework with two key innovations. First, Hierarchical Visual Semantic Decomposition leverages SAM instance masks to guide CLIP encoding at dual granularities – pixel-level and instance-level features – preserving object boundaries during 2D-to-3D projection. Second, Progressive Multi-level Fusion integrates representations through intra-modal collaboration, cross-modal adaptive weighting between 2D semantic and 3D
geometric features, and
language-guided refinement. HCF-RES achieves state-of-the-art results on both ScanRefer and Multi3DRefer.
Read the full abstract ▾
All authors Keshen Zhou, Runnan Chen, Mingming Gong, Tongliang Liu
Computer Vision · Cross-listed from cs.LG
DC-Merge: Improving Model Merging with Directional Consistency
Han-Chen Zhang, Zi-Hao Zhou, Mao-Lin Luo et al. (+3 more)
Model merging aims to integrate multiple task-adapted models into a unified model that preserves the knowledge of each task. In this paper, we identify that the key to this knowledge retention lies in maintaining the directional consistency of singular spaces between merged multi-task vector and individual task vectors. However, this consistency is frequently compromised by two issues: i) an imbalanced energy distribution within task vectors, where a small fraction of singular values dominate the total energy, leading to the neglect of semantically important but weaker components upon merging, and ii) the
geometric inconsistency of task vectors in parameter space, which causes direct merging to distort their underlying directional geometry. To address these challenges, we propose DC-Merge, a method for directional-consistent model merging. It first balances the energy distribution of each task vector by smoothing its singular values, ensuring all knowledge components are adequately represented. These energy-balanced vectors are then projected onto a shared orthogonal subspace to align their directional geometries with minimal reconstruction error. Finally, the aligned vectors are aggregated in the shared orthogonal subspace and projected back to the original parameter space. Extensive experiments on
vision and
vision-
language benchmarks show that DC-Merge consistently achieves state-of-the-art performance in both full fine-tuning and LoRA settings. The implementation code is available at https://github.com/Tobeginwith/DC-Merge.
Read the full abstract ▾
All authors Han-Chen Zhang, Zi-Hao Zhou, Mao-Lin Luo, Shimin Di, Min-Ling Zhang, Tong Wei
Computer Vision
TaPD: Temporal-adaptive Progressive Distillation for Observation-Adaptive Trajectory Forecasting in Autonomous Driving
Mingyu Fan, Yi Liu, Hao Zhou et al. (+3 more)
Trajectory prediction is essential for autonomous driving, enabling vehicles to anticipate the motion of surrounding
agents to support safe planning. However, most existing predictors assume fixed-length histories and suffer substantial performance degradation when observations are variable or extremely short in real-world settings (e.g., due to occlusion or a limited sensing range). We propose TaPD (Temporal-adaptive Progressive
Distillation), a unified plug-and-play framework for observation-adaptive trajectory
forecasting under variable history lengths. TaPD comprises two cooperative modules: an Observation-Adaptive Forecaster (OAF) for future prediction and a Temporal Backfilling Module (TBM) for explicit reconstruction of the past. OAF is built on progressive knowledge
distillation (PKD), which transfers motion pattern knowledge from long-horizon "teachers" to short-horizon "students" via hierarchical feature regression, enabling short observations to recover richer motion context. We further introduce a cosine-annealed
distillation weighting scheme to balance
forecasting supervision and feature
alignment, improving optimization stability and cross-length consistency. For extremely short histories where implicit
alignment is insufficient, TBM backfills missing historical segments conditioned on scene
evolution, producing context-rich trajectories that strengthen PKD and thereby improve OAF. We employ a decoupled pretrain-reconstruct-finetune protocol to preserve real-motion priors while adapting to backfilled inputs. Extensive experiments on Argoverse 1 and Argoverse 2 show that TaPD consistently outperforms strong baselines across all observation lengths, delivers especially large gains under very short inputs, and improves other predictors (e.g., HiVT) in a plug-and-play manner. Code will be available at https://github.com/zhouhao94/TaPD.
Read the full abstract ▾
All authors Mingyu Fan, Yi Liu, Hao Zhou, Deheng Qian, Mohammad Haziq Khan, Matthias Raetsch
Computer Vision
Low-Latency Event-Based Object Detection with Spatially-Sparse Linear Attention
Haiqing Hao, Zhipeng Sui, Rong Zou et al. (+4 more)
Event cameras provide sequential visual data with spatial sparsity and high temporal resolution, making them attractive for low-latency object detection. Existing asynchronous event-based neural networks exploit this low-latency advantage by updating predictions event by event, but still suffer from two bottlenecks: recurrent architectures are difficult to train efficiently on long sequences, and improving accuracy often increases per-event computation and latency. Linear
attention is appealing because it enables parallel training and recurrent inference. However, its dense state updates make per-event computation scale with the state size, yielding a poor accuracy-efficiency trade-off for object detection, where accurate localization requires fine-grained spatial states. The key challenge is therefore to introduce sparse state activation that exploits the spatial sparsity of events while preserving efficient parallel training. We propose Spatially-Sparse Linear
Attention (SSLA), which introduces a mixture-of-spaces state decomposition and a scatter-compute-gather training procedure, enabling state-level sparsity as well as training parallelism. Building on SSLA, we develop an end-to-end asynchronous linear
attention model, SSLA-Det, for low-latency event-based object detection. On Gen1 and N-Caltech101, SSLA-Det achieves state-of-the-art accuracy among asynchronous methods, reaching 0.375 mAP and 0.515 mAP, respectively, while reducing per-event computation by over 20 times compared with the strongest prior asynchronous baseline, demonstrating the potential of linear
attention for low-latency event-based
vision.
Read the full abstract ▾
All authors Haiqing Hao, Zhipeng Sui, Rong Zou, Zijia Dai, Nikola Zubić, Davide Scaramuzza, Wenhui Wang
Computer Vision
Word-Anchored Temporal Forgery Localization
Tianyi Wang, Xi Shao, Harry Cheng et al. (+2 more)
Current temporal forgery localization (TFL) approaches typically rely on temporal boundary regression or continuous frame-level anomaly detection paradigms to derive candidate forgery proposals. However, they suffer not only from feature granularity misalignment but also from costly computation. To address these issues, we propose word-anchored temporal forgery localization (WAFL), a novel paradigm that shifts the TFL task from temporal regression and continuous localization to discrete word-level binary classification. Specifically, we first analyze the essence of temporal forgeries and identify the minimum meaningful forgery units, word
tokens, and then align data preprocessing with the natural linguistic boundaries of
speech. To adapt powerful pre-trained foundation backbones for feature extraction, we introduce the forensic feature realignment (FFR) module, mapping representations from the pre-trained semantic space to a discriminative forensic
manifold. This allows subsequent lightweight linear classifiers to efficiently perform binary classification and accomplish the TFL task. Furthermore, to overcome the extreme class imbalance inherent to forgery detection, we design the artifact-centric asymmetric (ACA) loss, which breaks the standard precision-recall trade-off by dynamically suppressing overwhelming authentic gradients while asymmetrically prioritizing subtle forensic artifacts. Extensive experiments demonstrate that WAFL significantly outperforms state-of-the-art approaches in localization performance under both in- and cross-
dataset settings, while requiring substantially fewer learnable parameters and operating at high computational efficiency.
Read the full abstract ▾
All authors Tianyi Wang, Xi Shao, Harry Cheng, Yinglong Wang, Mohan Kankanhalli
Computer Vision
EntON: Eigenentropy-Optimized Neighborhood Densification in 3D Gaussian Splatting
Miriam Jäger and Boris Jutzi
We present a novel Eigenentropy-optimized neighboorhood densification strategy EntON in 3D Gaussian Splatting (3DGS) for geometrically accurate and high-quality rendered 3D reconstruction. While standard 3DGS produces Gaussians whose centers and surfaces are poorly aligned with the underlying object geometry, surface-focused reconstruction methods frequently sacrifice photometric accuracy. In contrast to the conventional densification strategy, which relies on the magnitude of the view-space position gradient, our approach introduces a geometry-aware strategy to guide adaptive splitting and
pruning. Specifically, we compute the 3D shape feature Eigenentropy from the
eigenvalues of the covariance
matrix in the k-nearest neighborhood of each Gaussian center, which quantifies the local structural order. These Eigenentropy values are integrated into an alternating optimization framework: During the optimization process, the
algorithm alternates between (i) standard gradient-based densification, which refines regions via view-space gradients, and (ii) Eigenentropy-aware densification, which preferentially densifies Gaussians in low-Eigenentropy (ordered, flat) neighborhoods to better capture fine
geometric details on the object surface, and prunes those in high-Eigenentropy (disordered, spherical) regions. We provide quantitative and qualitative evaluations on two
benchmark datasets: small-scale DTU
dataset and large-scale TUM2TWIN
dataset, covering man-made objects and urban scenes. Experiments demonstrate that our Eigenentropy-aware alternating densification strategy improves
geometric accuracy by up to 33% and rendering quality by up to 7%, while reducing the number of Gaussians by up to 50% and training time by up to 23%. Overall, EnTON achieves a favorable balance between
geometric accuracy, rendering quality and efficiency by avoiding unnecessary scene expansion.
Read the full abstract ▾
Computer Vision
Cut to the Chase: Training-free Multimodal Summarization via Chain-of-Events
Xiaoxing You, Qiang Huang, Lingyu Li et al. (+2 more)
Multimodal Summarization (MMS) aims to generate concise textual summaries by understanding and integrating information across
videos, transcripts, and images. However, existing approaches still suffer from three main challenges: (1) reliance on domain-specific supervision, (2) implicit fusion with weak cross-modal grounding, and (3) flat temporal modeling without event transitions. To address these issues, we introduce **CoE**, a training-free MMS framework that performs structured
reasoning through a **Chain-of-Events** guided by a Hierarchical Event Graph (HEG). The HEG encodes textual semantics into an explicit event hierarchy that scaffolds cross-modal grounding and temporal
reasoning. Guided by this structure, **CoE** localizes key visual cues, models event
evolution and causal transitions, and refines outputs via lightweight style adaptation for domain
alignment. Extensive experiments on eight diverse
datasets demonstrate that **CoE** consistently outperforms state-of-the-art
video CoT baselines, achieving average gains of **+3.04 ROUGE**, **+9.51 CIDEr**, and **+1.88 BERTScore**, highlighting its robustness, interpretability, and cross-domain generalization. Our code is available at https://github.com/youxiaoxing/CoE.
Read the full abstract ▾
All authors Xiaoxing You, Qiang Huang, Lingyu Li, Xiaojun Chang, Jun Yu
Computer Vision
VG3S: Visual Geometry Grounded Gaussian Splatting for Semantic Occupancy Prediction
Xiaoyang Yan, Muleilan Pei and Shaojie Shen
3D semantic occupancy prediction has become a crucial perception task for comprehensive scene understanding in autonomous driving. While recent advances have explored 3D Gaussian splatting for occupancy modeling to substantially reduce computational overhead, the generation of high-quality 3D Gaussians relies heavily on accurate
geometric cues, which are often insufficient in purely
vision-centric paradigms. To bridge this gap, we advocate for injecting the strong
geometric grounding capability from
Vision Foundation Models (VFMs) into occupancy prediction. In this regard, we introduce Visual Geometry Grounded Gaussian Splatting (VG3S), a novel framework that empowers Gaussian-based occupancy prediction with cross-view 3D
geometric grounding. Specifically, to fully exploit the rich 3D
geometric priors from a frozen VFM, we propose a plug-and-play hierarchical
geometric feature adapter, which can effectively transform generic VFM
tokens via feature aggregation, task-specific
alignment, and multi-scale restructuring. Extensive experiments on the nuScenes occupancy
benchmark demonstrate that VG3S achieves remarkable improvements of 12.6% in IoU and 7.5% in mIoU over the baseline. Furthermore, we show that VG3S generalizes seamlessly across diverse VFMs, consistently enhancing occupancy prediction accuracy and firmly underscoring the immense value of integrating priors derived from powerful, pre-trained geometry-grounded VFMs.
Read the full abstract ▾
Computer Vision
Point-Supervised Skeleton-Based Human Action Segmentation
Hongsong Wang, Yiqin Shen, Pengbo Yan et al. (+2 more)
Skeleton-based temporal action
segmentation is a fundamental yet challenging task, playing a crucial role in enabling intelligent systems to perceive and respond to human activities. While fully-supervised methods achieve satisfactory performance, they require costly frame-level annotations and are sensitive to ambiguous action boundaries. To address these issues, we introduce a point-supervised framework for skeleton-based action
segmentation, where only a single frame per action segment is labeled. We leverage
multimodal skeleton data, including joint, bone, and motion information, encoded via a pretrained unified model to extract rich feature representations. To generate reliable pseudo-labels, we propose a novel prototype similarity method and integrate it with two existing methods: energy function and constrained K-Medoids clustering.
Multimodal pseudo-label integration is proposed to enhance the reliability of the pseudo-label and guide the model training. We establish new
benchmarks on PKU-MMD (X-Sub and X-View), MCFS-22, and MCFS-130, and implement baselines for point-supervised skeleton-based human action
segmentation. Extensive experiments show that our method achieves competitive performance, even surpassing some fully-supervised methods while significantly reducing annotation effort.
Read the full abstract ▾
All authors Hongsong Wang, Yiqin Shen, Pengbo Yan, Jie Gui, Yang Zhang
Computer Vision
Adaptive Language-Aware Image Reflection Removal Network
Siyan Fang, Yuntao Wang, Jinpu Zhang et al. (+2 more)
Existing image reflection removal methods struggle to handle complex reflections. Accurate
language descriptions can help the model understand the image content to remove complex reflections. However, due to blurred and distorted interferences in reflected images, machine-generated
language descriptions of the image content are often inaccurate, which harms the performance of
language-guided reflection removal. To address this, we propose the Adaptive
Language-Aware Network (ALANet) to remove reflections even with inaccurate
language inputs. Specifically, ALANet integrates both filtering and optimization strategies. The filtering strategy reduces the negative effects of
language while preserving its benefits, whereas the optimization strategy enhances the
alignment between
language and visual features. ALANet also utilizes
language cues to decouple specific layer content from feature maps, improving its ability to handle complex reflections. To evaluate the model's performance under complex reflections and varying levels of
language accuracy, we introduce the Complex Reflection and
Language Accuracy Variance (CRLAV)
dataset. Experimental results demonstrate that ALANet surpasses state-of-the-art methods for image reflection removal. The code and
dataset are available at https://github.com/fashyon/ALANet.
Read the full abstract ▾
All authors Siyan Fang, Yuntao Wang, Jinpu Zhang, Ziwen Li, Yuehuan Wang
Computer Vision
SpaCRD: Multimodal Deep Fusion of Histology and Spatial Transcriptomics for Cancer Region Detection
Shuailin Xue, Jun Wan, Lihua Zhang et al. (+1 more)
Accurate detection of cancer tissue regions (CTR) enables deeper analysis of the tumor microenvironment and offers crucial insights into treatment response. Traditional CTR detection methods, which typically rely on the rich cellular morphology in histology images, are susceptible to a high rate of false positives due to morphological similarities across different tissue regions. The groundbreaking advances in spatial transcriptomics (ST) provide detailed cellular phenotypes and spatial localization information, offering new opportunities for more accurate cancer region detection. However, current methods are unable to effectively integrate histology images with ST data, especially in the context of cross-sample and cross-platform/batch settings for accomplishing the CTR detection. To address this challenge, we propose SpaCRD, a transfer learning-based method that deeply integrates histology images and ST data to enable reliable CTR detection across diverse samples, platforms, and batches. Once trained on source data, SpaCRD can be readily generalized to accurately detect cancerous regions across samples from different platforms and batches. The core of SpaCRD is a category-regularized variational reconstruction-guided bidirectional cross-
attention fusion network, which enables the model to adaptively capture latent co-expression patterns between histological features and gene expression from multiple perspectives. Extensive
benchmark analysis on 23 matched histology-ST
datasets spanning various disease types, platforms, and batches demonstrates that SpaCRD consistently outperforms existing eight state-of-the-art methods in CTR detection.
Read the full abstract ▾
All authors Shuailin Xue, Jun Wan, Lihua Zhang, Wenwen Min
Computer Vision · Cross-listed from cs.CL
CRIMSON: A Clinically-Grounded LLM-Based Metric for Generative Radiology Report Evaluation
Mohammed Baharoon, Thibault Heintz, Siavash Raissi et al. (+6 more)
We introduce CRIMSON, a clinically grounded evaluation framework for chest X-ray report generation that assesses reports based on diagnostic correctness, contextual relevance, and patient safety. Unlike prior metrics, CRIMSON incorporates full clinical context, including patient age, indication, and guideline-based decision rules, and prevents normal or clinically insignificant findings from exerting disproportionate influence on the overall score. The framework categorizes errors into a comprehensive taxonomy covering false findings, missing findings, and eight attribute-level errors (e.g., location, severity, measurement, and diagnostic overinterpretation). Each finding is assigned a clinical significance level (urgent, actionable non-urgent, non-actionable, or expected/benign), based on a guideline developed in collaboration with attending cardiothoracic radiologists, enabling severity-aware weighting that prioritizes clinically consequential mistakes over benign discrepancies. CRIMSON is validated through strong
alignment with clinically significant error counts annotated by six board-certified radiologists in ReXVal (Kendalls tau = 0.61-0.71; Pearsons r = 0.71-0.84), and through two additional
benchmarks that we introduce. In RadJudge, a targeted suite of clinically challenging pass-fail scenarios, CRIMSON shows consistent agreement with expert judgment. In RadPref, a larger radiologist preference
benchmark of over 100 pairwise cases with structured error categorization, severity modeling, and 1-5 overall quality ratings from three cardiothoracic radiologists, CRIMSON achieves the strongest
alignment with radiologist preferences. We release the metric, the evaluation
benchmarks, RadJudge and RadPref, and a fine-tuned MedGemma model to enable reproducible evaluation of report generation, all available at https://github.com/rajpurkarlab/CRIMSON.
Read the full abstract ▾
All authors Mohammed Baharoon, Thibault Heintz, Siavash Raissi, Mahmoud Alabbad, Mona Alhammad, Hassan AlOmaish, Sung Eun Kim, Oishi Banerjee, Pranav Rajpurkar
Computer Vision
Towards Motion Turing Test: Evaluating Human-Likeness in Humanoid Robots
Mingzhe Li, Mengyin Liu, Zekai Wu et al. (+9 more)
Humanoid robots have achieved significant progress in motion generation and control, exhibiting movements that appear increasingly natural and human-like. Inspired by the Turing Test, we propose the Motion Turing Test, a framework that evaluates whether human observers can discriminate between
humanoid robot and human poses using only kinematic information. To facilitate this evaluation, we present the Human-
Humanoid Motion (HHMotion)
dataset, which consists of 1,000 motion sequences spanning 15 action categories, performed by 11
humanoid models and 10 human subjects. All motion sequences are converted into SMPL-X representations to eliminate the influence of visual appearance. We recruited 30 annotators to rate the human-likeness of each pose on a 0-5 scale, resulting in over 500 hours of annotation. Analysis of the collected data reveals that
humanoid motions still exhibit noticeable deviations from human movements, particularly in dynamic actions such as jumping, boxing, and running. Building on HHMotion, we formulate a human-likeness evaluation task that aims to automatically predict human-likeness scores from motion data. Despite recent progress in
multimodal large
language models, we find that they remain inadequate for assessing motion human-likeness. To address this, we propose a simple baseline model and demonstrate that it outperforms several contemporary LLM-based methods. The
dataset, code, and
benchmark will be publicly released to support future research in the community.
Read the full abstract ▾
All authors Mingzhe Li, Mengyin Liu, Zekai Wu, Xincheng Lin, Junsheng Zhang, Ming Yan, Zengye Xie, Changwang Zhang, Chenglu Wen, Lan Xu, Siqi Shen, Cheng Wang
Computer Vision
Contrastive-to-Self-Supervised: A Two-Stage Framework for Script Similarity Learning
Claire Roman and Philippe Meyer
Learning similarity metrics for glyphs and writing systems faces a fundamental challenge: while individual graphemes within invented alphabets can be reliably labeled, the historical relationships between different scripts remain uncertain and contested. We propose a two-stage framework that addresses this epistemological constraint. First, we train an encoder with contrastive loss on labeled invented alphabets, establishing a teacher model with robust discriminative features. Second, we extend to historically attested scripts through teacher-student
distillation, where the student learns unsupervised representations guided by the teacher's knowledge but free to discover latent cross-script similarities. The asymmetric setup enables the student to learn deformation-invariant
embeddings while inheriting discriminative structure from clean examples. Our approach bridges supervised contrastive learning and unsupervised discovery, enabling both hard boundaries between distinct systems and soft similarities reflecting potential historical influences. Experiments on diverse writing systems demonstrate effective few-shot glyph recognition and meaningful script clustering without requiring ground-truth evolutionary relationships.
Read the full abstract ▾
Computer Vision
Making Training-Free Diffusion Segmentors Scale with the Generative Power
Benyuan Meng, Qianqian Xu, Zitai Wang et al. (+3 more)
As powerful generative models, text-to-image
diffusion models have recently been explored for discriminative tasks. A line of research focuses on adapting a pre-trained
diffusion model to semantic
segmentation without any further training, leading to training-free
diffusion segmentors. These methods typically rely on cross-
attention maps from the model's
attention layers, which are assumed to capture semantic relationships between image pixels and text
tokens. Ideally, such approaches should benefit from more powerful
diffusion models, i.e., stronger generative capability should lead to better
segmentation. However, we observe that existing methods often fail to scale accordingly. To understand this issue, we identify two underlying gaps: (i) cross-
attention is computed across multiple heads and layers, but there exists a discrepancy between these individual
attention maps and a unified global representation. (ii) Even when a global map is available, it does not directly translate to accurate semantic correlation for
segmentation, due to score imbalances among different text
tokens. To bridge these gaps, we propose two techniques: auto aggregation and per-pixel rescaling, which together enable training-free
segmentation to better leverage generative capability. We evaluate our approach on standard semantic
segmentation benchmarks and further integrate it into a generative technique, demonstrating both improved performance broad applicability. Codes are at https://github.com/Darkbblue/goca.
Read the full abstract ▾
All authors Benyuan Meng, Qianqian Xu, Zitai Wang, Xiaochun Cao, Longtao Huang, Qingming Huang
Computer Vision
Optimizing 3D Diffusion Models for Medical Imaging via Multi-Scale Reward Learning
Yueying Tian, Xudong Han, Meng Zhou et al. (+3 more)
Diffusion models have emerged as powerful tools for 3D medical image generation, yet bridging the gap between standard training objectives and clinical relevance remains a challenge. This paper presents a method to enhance 3D
diffusion models using Reinforcement Learning (RL) with multi-scale feedback. We first pretrain a 3D
diffusion model on MRI volumes to establish a robust generative prior. Subsequently, we fine-tune the model using Proximal Policy Optimization (PPO), guided by a novel reward system that integrates both 2D slice-wise assessments and 3D volumetric analysis. This combination allows the model to simultaneously optimize for local texture details and global structural coherence. We validate our framework on the BraTS 2019 and OASIS-1
datasets. Our results indicate that incorporating RL feedback effectively steers the generation process toward higher quality distributions. Quantitative analysis reveals significant improvements in Fréchet Inception Distance (FID) and, crucially, the synthetic data demonstrates enhanced utility in downstream tumor and disease classification tasks compared to non-optimized baselines.
Read the full abstract ▾
All authors Yueying Tian, Xudong Han, Meng Zhou, Rodrigo Aviles-Espinosa, Rupert Young, Philip Birch
Computer Vision
JOPP-3D: Joint Open Vocabulary Semantic Segmentation on Point Clouds and Panoramas
Sandeep Inuganti, Hideaki Kanayama, Kanta Shimizu et al. (+4 more)
Semantic
segmentation across visual modalities such as 3D point clouds and panoramic images remains a challenging task, primarily due to the scarcity of annotated data and the limited adaptability of fixed-label models. In this paper, we present JOPP-3D, an open-vocabulary
segmentation framework that jointly leverages panoramic and point cloud data to enable
language-driven scene understanding. We convert RGB-D panoramic images into their corresponding wide field-of-view tangential perspectives and 3D point clouds, then use these modalities to extract and align foundational
vision-
language features. This allows natural
language querying to generate semantic masks on both input modalities. Experimental evaluation on the Stanford-2D-3D-s and ToF-360
datasets demonstrates the capability of JOPP-3D to produce coherent and semantically meaningful
segmentations across panoramic and 3D domains. Our proposed method achieves a significant improvement compared to the SOTA in open and closed vocabulary 2D and 3D semantic
segmentation. Code: github.com/dfki-av/JOPP-3D
Read the full abstract ▾
All authors Sandeep Inuganti, Hideaki Kanayama, Kanta Shimizu, Mahdi Chamseddine, Soichiro Yokota, Didier Stricker, Jason Rambach
Computer Vision
A Semi-Supervised Framework for Breast Ultrasound Segmentation with Training-Free Pseudo-Label Generation and Label Refinement
Ruili Li, Jiayi Ding, Ruiyu Li et al. (+7 more)
Semi-supervised learning (SSL) has emerged as a promising paradigm for breast ultrasound (BUS) image
segmentation, but it often suffers from unstable pseudo labels under extremely limited annotations, leading to inaccurate supervision and degraded performance. Recent
vision-
language models (VLMs) provide a new opportunity for pseudo-label generation, yet their effectiveness on BUS images remains limited because domain-specific
prompts are difficult to transfer. To address this issue, we propose a semi-supervised framework with training-free pseudo-label generation and label refinement. By leveraging simple appearance-based descriptions (e.g., dark oval), our method enables cross-domain structural transfer between natural and medical images, allowing VLMs to generate structurally consistent pseudo labels. These pseudo labels are used to warm up a static teacher that captures global structural priors of breast lesions. Combined with an exponential moving average teacher, we further introduce uncertainty
entropy weighted fusion and adaptive uncertainty-guided reverse contrastive learning to improve boundary discrimination. Experiments on four BUS
datasets demonstrate that our method achieves performance comparable to fully supervised models even with only 2.5% labeled data, significantly outperforming existing SSL approaches. Moreover, the proposed paradigm is readily extensible: for other imaging modalities or diseases, only a global appearance description is required to obtain reliable pseudo supervision, enabling scalable semi-supervised medical image
segmentation under limited annotations.
Read the full abstract ▾
All authors Ruili Li, Jiayi Ding, Ruiyu Li, Yilun Jin, Shiwen Ge, Yuwen Zeng, Xiaoyong Zhang, Eichi Takaya, Jan Vrba, Noriyasu Homma
Computer Vision
FreeOcc: Training-free Panoptic Occupancy Prediction via Foundation Models
Andrew Caunes, Thierry Chateau and Vincent Fremont
Semantic and panoptic occupancy prediction for road scene analysis provides a dense 3D representation of the ego vehicle's surroundings. Current camera-only approaches typically rely on costly dense 3D supervision or require training models on data from the target domain, limiting deployment in unseen environments. We propose FreeOcc, a training-free pipeline that leverages pretrained foundation models to recover both semantics and geometry from multi-view images. FreeOcc extracts per-view panoptic priors with a promptable foundation
segmentation model and
prompt-to-taxonomy rules, and reconstructs metric 3D points with a reconstruction foundation model. Depth- and confidence- aware filtering lifts reliable labels into 3D, which are fused over time and voxelized with a deterministic refinement stack. For panoptic occupancy, instances are recovered by fitting and merging robust current-view 3D box candidates, enabling instance-aware occupancy without any learned 3D model. On Occ3D-nuScenes, FreeOcc achieves 16.9 mIoU and 16.5 RayIoU train-free, on par with state-of-the-art weakly supervised methods. When employed as a pseudo-label generation pipeline for training downstream models, it achieves 21.1 RayIoU, surpassing the previous state-of-the-art weakly supervised baseline. Furthermore, FreeOcc sets new baselines for both train-free and weakly supervised panoptic occupancy prediction, achieving 3.1 RayPQ and 3.9 RayPQ, respectively. These results highlight foundation-model-driven perception as a practical route to training-free 3D scene understanding.
Read the full abstract ▾
Computer Vision
Reflective Flow Sampling Enhancement
Zikai Zhou, Muyao Wang, Shitong Shao et al. (+4 more)
The growing demand for text-to-image generation has led to rapid advances in generative modeling. Recently, text-to-image
diffusion models trained with flow matching
algorithms, such as FLUX, have achieved remarkable progress and emerged as strong alternatives to conventional
diffusion models. At the same time, inference-time enhancement strategies have been shown to improve the generation quality and text-
prompt alignment of text-to-image
diffusion models. However, these techniques are mainly applicable to conventional
diffusion models and usually fail to perform well on flow models. To bridge this gap, we propose Reflective Flow Sampling (RF-Sampling), a theoretically-grounded and training-free inference enhancement framework explicitly designed for flow models, especially for the CFG-distilled variants (i.e., models distilled from CFG guidance techniques), like FLUX. Departing from heuristic interpretations, we provide a formal derivation proving that RF-Sampling implicitly performs gradient ascent on the text-image
alignment score. By leveraging a linear combination of textual representations and integrating them with flow inversion, RF-Sampling allows the model to explore noise spaces that are more consistent with the input
prompt. Extensive experiments across multiple
benchmarks demonstrate that RF-Sampling consistently improves both generation quality and
prompt alignment. Moreover, RF-Sampling is also the first inference enhancement method that can exhibit test-time scaling ability to some extent on FLUX.
Read the full abstract ▾
All authors Zikai Zhou, Muyao Wang, Shitong Shao, Lichen Bai, Haoyi Xiong, Bo Han, Zeke Xie
Computer Vision
VLM-RobustBench: A Comprehensive Benchmark for Robustness of Vision-Language Models
Rohit Saxena, Alessandro Suglia and Pasquale Minervini
Vision-
language models (VLMs) achieve strong performance on standard, high-quality
datasets, but we still do not fully understand how they perform under real-world image distortions. We present VLM-RobustBench, a
benchmark spanning 49 augmentation types across noise, blur, weather, digital, and
geometric perturbations, evaluated under graded severities (low/mid/high) and binary transforms, yielding 133 corrupted settings. We evaluate VLMs from four families (Qwen, InternVL, Molmo, Gemma) on two complementary
benchmarks: MMBench (visually grounded) and MMMU-Pro (
reasoning-oriented). Our results reveal that visual severity is a weak predictor of difficulty: low-severity spatial perturbations often degrade performance more than visually severe photometric corruptions. In particular, low-severity glass_blur reduces MMBench accuracy by about 8 pp on average across models, while the largest drops arise from resampling and
geometric distortions (e.g., upsample, elastic_transform), reaching up to 34 pp. Overall, our findings suggest current VLMs are semantically strong but spatially fragile, motivating the definition of novel robustness evaluation protocols and training regimes that emphasize resampling and
geometric invariances.
Read the full abstract ▾
Computer Vision
Longitudinal NSCLC Treatment Progression via Multimodal Generative Models
Massimiliano Mantegna, Elena Mulero Ayllón, Alice Natalina Caragliano et al. (+10 more)
Predicting tumor
evolution during radiotherapy is a clinically critical challenge, particularly when longitudinal changes are driven by both anatomy and treatment. In this work, we introduce a Virtual Treatment (VT) framework that formulates non-small
cell lung cancer (NSCLC) progression as a dose-aware
multimodal conditional image-to-image translation problem. Given a CT scan, baseline clinical variables, and a specified radiation dose increment, VT aims to synthesize plausible follow-up CT images reflecting treatment-induced anatomical changes. We evaluate the proposed framework on a longitudinal
dataset of 222 stage III NSCLC patients, comprising 895 CT scans acquired during radiotherapy under irregular clinical schedules. The generative process is conditioned on delivered dose increments together with demographic and tumor-related clinical variables. Representative GAN-based and
diffusion-based models are benchmarked across 2D and 2.5D configurations. Quantitative and qualitative results indicate that
diffusion-based models benefit more consistently from
multimodal, dose-aware conditioning and produce more stable and anatomically plausible tumor
evolution trajectories than GAN-based baselines, supporting the potential of VT as a tool for in-silico treatment monitoring and adaptive radiotherapy research in NSCLC.
Read the full abstract ▾
All authors Massimiliano Mantegna, Elena Mulero Ayllón, Alice Natalina Caragliano, Francesco Di Feola, Claudia Tacconi, Michele Fiore, Edy Ippolito, Carlo Greco, Sara Ramella, Philippe C. Cattin, Paolo Soda, Matteo Tortora, Valerio Guarrasi