Computer Vision
Learning from Limited and Incomplete Data: A Multimodal Framework for Predicting Pathological Response in NSCLC
Alice Natalina Caragliano, Giulia Farina, Fatih Aksu et al. (+10 more)
Major pathological response (pR) following neoadjuvant therapy is a clinically meaningful endpoint in non-small
cell lung cancer, strongly associated with improved survival. However, accurate preoperative prediction of pR remains challenging, particularly in real-world clinical settings characterized by limited data availability and incomplete clinical profiles. In this study, we propose a
multimodal deep learning framework designed to address these constraints by integrating foundation model-based CT feature extraction with a missing-aware architecture for clinical variables. This approach enables robust learning from small cohorts while explicitly modeling missing clinical information, without relying on conventional imputation strategies. A weighted fusion mechanism is employed to leverage the complementary contributions of imaging and clinical modalities, yielding a
multimodal model that consistently outperforms both unimodal imaging and clinical baselines. These findings underscore the added value of integrating heterogeneous data sources and highlight the potential of
multimodal, missing-aware systems to support pR prediction under realistic clinical conditions.
Read the full abstract ▾
All authors Alice Natalina Caragliano, Giulia Farina, Fatih Aksu, Camillo Maria Caruso, Claudia Tacconi, Carlo Greco, Lorenzo Nibid, Edy Ippolito, Michele Fiore, Giuseppe Perrone, Sara Ramella, Paolo Soda, Valerio Guarrasi
Computer Vision
ReactMotion: Generating Reactive Listener Motions from Speaker Utterance
Cheng Luo, Bizhu Wu, Bing Li et al. (+5 more)
In this paper, we introduce a new task, Reactive Listener Motion Generation from Speaker Utterance, which aims to generate naturalistic listener body motions that appropriately respond to a speaker's utterance. However, modeling such nonverbal listener behaviors remains underexplored and challenging due to the inherently non-deterministic nature of human reactions. To facilitate this task, we present ReactMotionNet, a large-scale
dataset that pairs speaker utterances with multiple candidate listener motions annotated with varying degrees of appropriateness. This
dataset design explicitly captures the one-to-many nature of listener behavior and provides supervision beyond a single ground-truth motion. Building on this
dataset design, we develop preference-oriented evaluation protocols tailored to evaluate reactive appropriateness, where conventional motion metrics focusing on input-motion
alignment ignore. We further propose ReactMotion, a unified generative framework that jointly models text, audio,
emotion, and motion, and is trained with preference-based objectives to encourage both appropriate and diverse listener responses. Extensive experiments show that ReactMotion outperforms
retrieval baselines and cascaded LLM-based pipelines, generating more natural, diverse, and appropriate listener motions.
Read the full abstract ▾
All authors Cheng Luo, Bizhu Wu, Bing Li, Jianfeng Ren, Ruibin Bai, Rong Qu, Linlin Shen, Bernard Ghanem
Computer Vision
The Good, the Better, and the Best: Improving the Discriminability of Face Embeddings through Attribute-aware Learning
Ana Dias, João Ribeiro Pinto, Hugo Proença et al. (+1 more)
Despite recent advances in face recognition, robust performance remains challenging under large variations in age, pose, and occlusion. A common strategy to address these issues is to guide representation learning with auxiliary supervision from facial attributes, encouraging the visual encoder to focus on identity-relevant regions. However, existing approaches typically rely on heterogeneous and fixed sets of attributes, implicitly assuming equal relevance across attributes. This assumption is suboptimal, as different attributes exhibit varying discriminative power for identity recognition, and some may even introduce harmful biases. In this paper, we propose an attribute-aware face recognition architecture that supervises the learning of facial
embeddings using identity class labels, identity-relevant facial attributes, and non-identity-related attributes. Facial attributes are organized into interpretable groups, making it possible to decompose and analyze their individual contributions in a human-understandable manner. Experiments on standard face verification
benchmarks demonstrate that joint learning of identity and facial attributes improves the discriminability of face
embeddings with two major conclusions: (i) using identity-relevant subsets of facial attributes consistently outperforms supervision with a broader attribute set, and (ii) explicitly forcing
embeddings to unlearn non-identity-related attributes yields further performance gains compared to leaving such attributes unsupervised. Additionally, our method serves as a diagnostic tool for assessing the trustworthiness of face recognition encoders by allowing for the measurement of accuracy gains with suppression of non-identity-relevant attributes, with such gains suggesting shortcut learning from redundant attributes associated with each identity.
Read the full abstract ▾
All authors Ana Dias, João Ribeiro Pinto, Hugo Proença, João C. Neves
Computer Vision
SRL-MAD: Structured Residual Latents for One-Class Morphing Attack Detection
Diogo J. Paulo, Hugo Proença and João C. Neves
Face morphing attacks represent a significant threat to biometric systems as they allow multiple identities to be combined into a single face. While supervised morphing attack detection (MAD) methods have shown promising performance, their reliance on attack-labeled data limits generalization to unseen morphing attacks. This has motivated increasing interest in one-class MAD, where models are trained exclusively on bona fide samples and are expected to detect unseen attacks as deviations from the normal facial structure. In this context, we introduce SRL-MAD, a one-class single-image MAD that uses structured residual
Fourier representations for open-set morphing attack detection. Starting from a residual frequency map that suppresses image-specific spectral trends, we preserve the two-dimensional organization of the
Fourier domain through a ring-based representation and replace azimuthal averaging with a learnable ring-wise spectral projection. To further encode domain knowledge about where morphing artifacts arise, we impose a frequency-informed inductive bias by organizing spectral evidence into low, mid, and high-frequency bands and learning cross-band interactions. These structured spectral features are mapped into a latent space designed for direct scoring, avoiding the reliance on reconstruction errors. Extensive evaluation on FERET-Morph, FRLL-Morph, and MorDIFF demonstrates that SRL-MAD consistently outperforms recent one-class and supervised MAD models. Overall, our results show that learning frequency-aware projections provides a more discriminative alternative to azimuthal spectral summarization for one-class morphing attack detection.
Read the full abstract ▾
Computer Vision
GUI-CEval: A Hierarchical and Comprehensive Chinese Benchmark for Mobile GUI Agents
Yang Li, Yuchen Liu, Haoyu Lu et al. (+8 more)
Recent progress in
Multimodal Large
Language Models (MLLMs) has enabled mobile GUI
agents capable of visual perception, cross-modal
reasoning, and interactive control. However, existing
benchmarks are largely English-centric and fail to capture the linguistic and interaction characteristics of the Chinese mobile ecosystem. They also focus on isolated skills such as GUI grounding or offline
agent, lacking a unified and fine-grained framework to assess the full capability chain from perception to execution. To address this gap, we introduce GUI-CEval, the first comprehensive
benchmark for Chinese mobile GUI
agents, built entirely on physical device environments. GUI-CEval spans 201 mainstream apps across four device types and adopts a two-level structure that evaluates both atomic abilities and realistic application-level performance along five dimensions: perception, planning, reflection, execution, and evaluation. All data are collected and verified through multi-stage manual processes to ensure authenticity and reproducibility. Extensive experiments on 20 representative MLLMs and multi-
agent systems show that while models such as Qwen2.5-VL and UI-TARS perform competitively, most MLLMs still exhibit clear weaknesses in reflective decision-making and post-action self-evaluation, limiting their reliability in real-world interactions. We hope GUI-CEval provides a comprehensive and interpretable
benchmark to guide capability diagnosis and advance the development of Chinese mobile GUI
agents.
Read the full abstract ▾
All authors Yang Li, Yuchen Liu, Haoyu Lu, Zhiqiang Xia, Hongzhen Wang, Kaiyang Han, Changpeng Yang, Jinyang Wu, Jiaming Xu, Runyu Shi, Ying Huang
Computer Vision
Training-free Detection of Generated Videos via Spatial-Temporal Likelihoods
Omer Ben Hayun, Roy Betser, Meir Yossef Levi et al. (+2 more)
Following major advances in text and image generation, the
video domain has surged, producing highly realistic and controllable sequences. Along with this progress, these models also raise serious concerns about misinformation, making reliable detection of synthetic
videos increasingly crucial. Image-based detectors are fundamentally limited because they operate per frame and ignore temporal dynamics, while supervised
video detectors generalize poorly to unseen generators, a critical drawback given the rapid emergence of new models. These challenges motivate zero-shot approaches, which avoid synthetic data and instead score content against real-data statistics, enabling training-free, model-agnostic detection. We introduce STALL, a simple, training-free, theoretically justified detector that provides likelihood-based scoring for
videos, jointly modeling spatial and temporal evidence within a probabilistic framework. We evaluate STALL on two public
benchmarks and introduce ComGenVid, a new
benchmark with state-of-the-art generative models. STALL consistently outperforms prior image- and
video-based baselines. Code and data are available at https://omerbenhayun.github.io/stall-
video.
Read the full abstract ▾
All authors Omer Ben Hayun, Roy Betser, Meir Yossef Levi, Levi Kassel, Guy Gilboa
Computer Vision
One CT Unified Model Training Framework to Rule All Scanning Protocols
Fengzhi Xu, Ziyuan Yang, Zexin Lu et al. (+4 more)
Non-ideal measurement computed
tomography (NICT), which lowers radiation at the cost of image quality, is expanding the clinical use of CT. Although unified models have shown promise in NICT enhancement, most methods require paired data, which is an impractical demand due to inevitable organ motion. Unsupervised approaches attempt to overcome this limitation, but their assumption of homogeneous noise neglects the variability of scanning protocols, leading to poor generalization and potential model collapse. We further observe that distinct scanning protocols, which correspond to different physical imaging processes, produce discrete sub-
manifolds in the feature space, contradicting these assumptions and limiting their effectiveness. To address this, we propose an Uncertainty-Guided
Manifold Smoothing (UMS) framework to bridge the gaps between sub-
manifolds. A classifier in UMS identifies sub-
manifolds and predicts uncertainty scores, which guide the generation of diverse samples across the entire
manifold. By leveraging the classifier's capability, UMS effectively fills the gaps between discrete sub-
manifolds, and promotes a continuous and dense feature space. Due to the complexity of the global
manifold, it's hard to directly model it. Therefore, we propose to dynamically incorporate the global- and sub-
manifold-specific features. Specifically, we design a global- and sub-
manifold-driven architecture guided by the classifier, which enables dynamic adaptation to subdomain variations. This dynamic mechanism improves the network's capacity to capture both shared and domain-specific features, thereby improving reconstruction performance. Extensive experiments on public
datasets are conducted to validate the effectiveness of our method across different generation paradigms.
Read the full abstract ▾
All authors Fengzhi Xu, Ziyuan Yang, Zexin Lu, Yingyu Chen, Fenglei Fan, Hongming Shan, Yi Zhang
Computer Vision
MER-Bench: Benchmarking Multimodal Emotional Reappraisal for Image-Text Memes
Yiqi Nie, Fei Wang, Junjie Chen et al. (+5 more)
Multimodal Emotional Reappraisal (MER) aims to shift affective interpretation by rewriting and editing a situation without altering its underlying semantic context. Despite advances in controllable editing and sentiment transfer, MER remains challenging under coupled
multimodal constraints, especially in internet memes where images, embedded text, and layout jointly encode affective meaning. To bridge this gap, we present MER-Bench, a systematic
benchmark for MER on image-text memes, containing over 3,000 real-world memes with fine-grained annotations for
emotion transformation and structure-preserving
multimodal editing. MER-Bench spans diverse visual, textual, and structural settings and is paired with a structured MLLM-as-a-Judge evaluation suite, comprising eight fine-grained metrics and three aggregated objectives for emotional
alignment, content fidelity, and transformation strength. Extensive evaluation of 14 advanced
multimodal models reveals that: (1) current models struggle to jointly balance affect, semantics, structure, and text; (2) strong visual editing does not imply effective MER, with poor embedded text rendering and affect-aware rewriting remaining key bottlenecks. Our results identify MER as a critical capability gap in current
multimodal generation systems and highlight affective image-text
reasoning as a key challenge for future work. Project Page: https://github.com/one-seven17/MER-Bench
Read the full abstract ▾
All authors Yiqi Nie, Fei Wang, Junjie Chen, Kun Li, Yudi Cai, Dan Guo, Chenglong Li, Meng Wang
Computer Vision
Reference-Free Omnidirectional Stereo Matching via Multi-View Consistency Maximization
Lehuai Xu, Weiming Zhang, Yang Li et al. (+2 more)
Reliable omnidirectional depth estimation from multi-fisheye stereo matching is pivotal to many applications, such as
embodied robotics. Existing approaches either rely on spherical sweeping with heuristic fusion strategies to build the cost columns or perform reference-centric stereo matching based on rectified views. However, these methods fail to explicitly exploit
geometric relationships between multiple views, rendering them less capable of capturing the global dependencies, visibility, or scale changes. In this paper, we shift to a new perspective and propose a novel reference-free framework, dubbed FreeOmniMVS, via multi-view consistency maximization. The highlight of FreeOmniMVS is that it can aggregate pair-wise correlations into a robust, visibility-aware, and global consensus. As such, it is tolerant to occlusions, partial overlaps, and varying baselines. Specifically, to achieve global coherence, we introduce a novel View-pair Correlation
Transformer (VCT) that explicitly models pairwise correlation volumes across all camera view pairs, allowing us to drop unreliable pairs caused by occlusion or out-of-focus observations. To realize scalable and visibility-aware consensus, we propose a lightweight
attention mechanism that adaptively fuses the correlation vectors, eliminating the need for a designated reference view and allowing all cameras to contribute equally to the stereo matching process. Extensive experiments on diverse
benchmark datasets demonstrate the superiority of our method for globally consistent, visibility-aware, and scale-aware omnidirectional depth estimation.
Read the full abstract ▾
All authors Lehuai Xu, Weiming Zhang, Yang Li, Sidan Du, Lin Wang
Computer Vision
Riemannian Motion Generation: A Unified Framework for Human Motion Representation and Generation via Riemannian Flow Matching
Fangran Miao, Jian Huang and Ting Li
Human motion generation is often learned in Euclidean spaces, although valid motions follow structured non-Euclidean geometry. We present Riemannian Motion Generation (RMG), a unified framework that represents motion on a product
manifold and learns dynamics via Riemannian flow matching. RMG factorizes motion into several
manifold factors, yielding a scale-free representation with intrinsic normalization, and uses geodesic interpolation, tangent-space supervision, and
manifold-preserving ODE integration for training and sampling. On HumanML3D, RMG achieves state-of-the-art FID in the HumanML3D format (0.043) and ranks first on all reported metrics under the MotionStreamer format. On MotionMillion, it also surpasses strong baselines (FID 5.6, R@1 0.86). Ablations show that the compact $\mathscr{T}+\mathscr{R}$ (translation + rotations) representation is the most stable and effective, highlighting geometry-aware modeling as a practical and scalable route to high-fidelity motion generation.
Read the full abstract ▾
Computer Vision
Molecular Identifier Visual Prompt and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsing
Jiahe Song, Chuang Wang, Yinfan Wang et al. (+11 more)
Reaction diagram parsing (RxnDP) is critical for extracting chemical synthesis information from literature. Although recent
Vision-
Language Models (VLMs) have emerged as a promising paradigm to automate this complex visual
reasoning task, their application is fundamentally bottlenecked by the inability to align visual chemical entities with pre-trained knowledge, alongside the inherent discrepancy between
token-level training and reaction-level evaluation. To address these dual challenges, this work enhances VLM-based RxnDP from two complementary perspectives: prompting representation and learning paradigms. First, we propose Identifier as Visual Prompting (IdtVP), which leverages naturally occurring molecule identifiers (e.g., bold numerals like 1a) to activate the chemical knowledge acquired during VLM pre-training. IdtVP enables powerful zero-shot and out-of-distribution capabilities, outperforming existing prompting strategies. Second, to further optimize performance within fine-tuning paradigms, we introduce Re3-DAPO, a reinforcement learning
algorithm that leverages verifiable rewards to directly optimize reaction-level metrics, thereby achieving consistent gains over standard supervised fine-tuning. Additionally, we release the ScannedRxn
benchmark, comprising scanned historical reaction diagrams with real-world artifacts, to rigorously assess model robustness and out-of-distribution ability. Our contributions advance the accuracy and generalization of VLM-based reaction diagram parsing. We will release data, models, and code on GitHub.
Read the full abstract ▾
All authors Jiahe Song, Chuang Wang, Yinfan Wang, Hao Zheng, Rui Nie, Bowen Jiang, Xingjian Wei, Junyuan Gao, Yubin Wang, Bin Wang, Lijun Wu, Jiang Wu, Qian Yu, Conghui He
Computer Vision
Clue Matters: Leveraging Latent Visual Clues to Empower Video Reasoning
Kaixin zhang, Xiaohe Li, Jiahao Li et al. (+4 more)
Multi-modal Large
Language Models (MLLMs) have significantly advanced
video reasoning, yet
Video Question Answering (VideoQA) remains challenging due to its demand for temporal causal
reasoning and evidence-grounded answer generation. Prevailing end-to-end MLLM frameworks lack explicit structured
reasoning between visual perception and answer derivation, causing severe
hallucinations and poor interpretability. Existing methods also fail to address three core gaps: faithful visual clue extraction, utility-aware clue filtering, and end-to-end clue-answer
alignment. Inspired by hierarchical human visual cognition, we propose ClueNet, a clue-aware
video reasoning framework with a two-stage supervised fine-tuning paradigm without extensive base model modifications. Decoupled supervision aligns clue extraction and chain-based
reasoning, while inference supervision with an adaptive clue filter refines high-order
reasoning, alongside lightweight modules for efficient inference. Experiments on NExT-QA,
STAR, and MVBench show that ClueNet outperforms state-of-the-art methods by $\ge$ 1.1%, with superior generalization,
hallucination mitigation, inference efficiency, and cross-backbone compatibility. This work bridges the perception-to-generation gap in MLLM
video understanding, providing an interpretable, faithful
reasoning paradigm for high-stakes VideoQA applications.
Read the full abstract ▾
All authors Kaixin zhang, Xiaohe Li, Jiahao Li, Haohua Wu, Xinyu Zhao, Zide Fan, Lei Wang
Computer Vision
Edit2Interp: Adapting Image Foundation Models from Spatial Editing to Video Frame Interpolation with Few-Shot Learning
Nasrin Rahimi, M{\i}sra Yavuz, Burak Can Biner et al. (+6 more)
Pre-trained image editing models exhibit strong spatial
reasoning and object-aware transformation capabilities acquired from billions of image-text pairs, yet they possess no explicit temporal modeling. This paper demonstrates that these spatial priors can be repurposed to unlock temporal synthesis capabilities through minimal adaptation - without introducing any
video-specific architecture or motion estimation modules. We show that a large image editing model (Qwen-Image-Edit), originally designed solely for static instruction-based edits, can be adapted for
Video Frame Interpolation (VFI) using only 64-256 training samples via Low-Rank Adaptation (LoRA). Our core contribution is revealing that the model's inherent understanding of "how objects transform" in static scenes contains latent temporal
reasoning that can be activated through few-shot fine-tuning. While the baseline model completely fails at producing coherent intermediate frames, our parameter-efficient adaptation successfully unlocks its interpolation capability. Rather than competing with task-specific VFI methods trained from scratch on massive
datasets, our work establishes that foundation image editing models possess untapped potential for temporal tasks, offering a data-efficient pathway for
video synthesis in resource-constrained scenarios. This bridges the gap between image manipulation and
video understanding, suggesting that spatial and temporal
reasoning may be more intertwined in foundation models than previously recognized
Read the full abstract ▾
All authors Nasrin Rahimi, M{\i}sra Yavuz, Burak Can Biner, Yunus Bilge Kurt, Ahmet Rasim Emirdağ{\i}, Süleyman Aslan, Görkay Aydemir, M. Ak{\i}n Y{\i}lmaz, A. Murat Tekalp
Computer Vision
Thermal Image Refinement with Depth Estimation using Recurrent Networks for Monocular ORB-SLAM3
Hürkan Şahin, Huy Xuan Pham, Van Huyen Dang et al. (+2 more)
Autonomous
navigation in GPS-denied and visually degraded environments remains challenging for unmanned aerial vehicles (
UAVs). To this end, we investigate the use of a
monocular thermal camera as a standalone sensor on a
UAV platform for real-time depth estimation and simultaneous localization and mapping (SLAM). To extract depth information from thermal images, we propose a novel pipeline employing a lightweight supervised network with recurrent blocks (RBs) integrated to capture temporal dependencies, enabling more robust predictions. The network combines lightweight convolutional backbones with a thermal refinement network (T-RefNet) to refine raw thermal inputs and enhance feature visibility. The refined thermal images and predicted depth maps are integrated into ORB-SLAM3, enabling thermal-only localization. Unlike previous methods, the network is trained on a custom non-radiometric
dataset, obviating the need for high-cost radiometric thermal cameras. Experimental results on
datasets and
UAV flights demonstrate competitive depth accuracy and robust SLAM performance under low-light conditions. On the radiometric VIVID++ (indoor-dark)
dataset, our method achieves an absolute relative error of approximately 0.06, compared to baselines exceeding 0.11. In our non-radiometric indoor set, baseline errors remain above 0.24, whereas our approach remains below 0.10. Thermal-only ORB-SLAM3 maintains a mean trajectory error under 0.4 m.
Read the full abstract ▾
All authors Hürkan Şahin, Huy Xuan Pham, Van Huyen Dang, Alper Yegenoglu, Erdal Kayacan
Computer Vision
MMSpec: Benchmarking Speculative Decoding for Vision-Language Models
Hui Shen, Xin Wang, Ping Zhang et al. (+11 more)
Vision-
language models (VLMs) achieve strong performance on
multimodal tasks but suffer from high inference latency due to large model sizes and long
multimodal contexts. Speculative decoding has recently emerged as an effective acceleration technique, yet its behavior in VLMs remains insufficiently understood. We introduce MMSpec, the first
benchmark for evaluating speculative decoding in
vision-
language models. MMSpec contains 600
multimodal samples across six task categories and integrates ten representative speculative decoding
algorithms under a unified evaluation framework. Our study reveals three key findings: (1) methods designed for text-only LLMs degrade in
multimodal scenarios, (2)
vision awareness becomes increasingly important at larger batch sizes, and (3) throughput speedup alone does not reliably reflect latency performance. Motivated by these findings, we propose ViSkip, a plug-and-play speculative decoding method that dynamically adapts speculation to
vision tokens and achieves state-of-the-art performance.
Read the full abstract ▾
All authors Hui Shen, Xin Wang, Ping Zhang, Yunta Hsieh, Qi Han, Zhongwei Wan, Ziheng Zhang, Jingxuan Zhang, Jing Xiong, Ziyuan Liu, Yifan Zhang, Hangrui Cao, Chenyang Zhao, Mi Zhang
Computer Vision · Cross-listed from cs.MM
Anchoring Emotions in Text: Robust Multimodal Fusion for Mimicry Intensity Estimation
Lingsi Zhu, Yuefeng Zou, Yunxiang Zhang et al. (+7 more)
Estimating Emotional Mimicry Intensity (EMI) in naturalistic environments is a critical yet challenging task in affective computing. The primary difficulty lies in effectively modeling the complex, nonlinear temporal dynamics across highly heterogeneous modalities, especially when physical signals are corrupted or missing. To tackle this, we propose TAEMI (Text-Anchored Emotional Mimicry Intensity estimation), a novel
multimodal framework designed for the 10th ABAW Competition. Motivated by the observation that continuous visual and acoustic signals are highly susceptible to transient environmental noise, we break the traditional symmetric fusion paradigm. Instead, we leverage textual transcript–which inherently encode a stable, time-independent semantic prior–as central anchors. Specifically, we introduce a Text-Anchored Dual Cross-
Attention mechanism that utilizes these robust textual queries to actively filter out frame-level redundancies and align the noisy physical streams. Furthermore, to prevent catastrophic performance degradation caused by inevitably missing data in unconstrained real-world scenarios, we integrate Learnable Missing-Modality
Tokens and a Modality Dropout strategy during training. Extensive experiments on the Hume-Vidmimic2
dataset demonstrate that TAEMI effectively captures fine-grained emotional variations and maintains robust predictive resilience under imperfect conditions. Our framework achieves a state-of-the-art mean Pearson correlation coefficient across six continuous emotional dimensions, significantly outperforming existing baseline methods.
Read the full abstract ▾
All authors Lingsi Zhu, Yuefeng Zou, Yunxiang Zhang, Naixiang Zheng, Guoyuan Wang, Jun Yu, Jiaen Liang, Wei Huang, Shengping Liu, Ximin Zheng
Computer Vision
Voronoi-based Second-order Descriptor with Whitened Metric in LiDAR Place Recognition
Jaein Kim, Hee Bin Yoo, Dong-Sig Han et al. (+1 more)
The pooling layer plays a vital role in aggregating local descriptors into the metrizable global descriptor in the
LiDAR Place Recognition (LPR). In particular, the second-order pooling is capable of capturing higher-order interactions among local descriptors. However, its existing methods in the LPR adhere to conventional implementations and post-normalization, and incur the descriptor unsuitable for Euclidean distancing. Based on the recent interpretation that associates NetVLAD with the second-order statistics, we propose to integrate second-order pooling with the inductive bias from Voronoi
cells. Our novel pooling method aggregates local descriptors to form the second-order
matrix and whitens the global descriptor to implicitly measure the Mahalanobis distance while conserving the cluster property from Voronoi
cells, addressing its numerical instability during learning with diverse techniques. We demonstrate its performance gains through the experiments conducted on the Oxford Robotcar and Wild-Places
benchmarks and analyze the numerical effect of the proposed whitening
algorithm.
Read the full abstract ▾
All authors Jaein Kim, Hee Bin Yoo, Dong-Sig Han, Byoung-Tak Zhang
Computer Vision
GeoNVS: Geometry Grounded Video Diffusion for Novel View Synthesis
Minjun Kang, Inkyu Shin, Taeyeop Lee et al. (+3 more)
Novel view synthesis requires strong 3D
geometric consistency and the ability to generate visually coherent images across diverse viewpoints. While recent camera-controlled
video diffusion models show promising results, they often suffer from
geometric distortions and limited camera controllability. To overcome these challenges, we introduce GeoNVS, a geometry-grounded novel-view synthesizer that enhances both
geometric fidelity and camera controllability through explicit 3D
geometric guidance. Our key innovation is the Gaussian Splat Feature Adapter (GS-Adapter), which lifts input-view
diffusion features into 3D Gaussian representations, renders geometry-constrained novel-view features, and adaptively fuses them with
diffusion features to correct geometrically inconsistent representations. Unlike prior methods that inject geometry at the input level, GS-Adapter operates in feature space, avoiding view-dependent color noise that degrades structural consistency. Its plug-and-play design enables zero-shot compatibility with diverse feed-forward geometry models without additional training, and can be adapted to other
video diffusion backbones. Experiments across 9 scenes and 18 settings demonstrate state-of-the-art performance, achieving 11.3% and 14.9% improvements over SEVA and CameraCtrl, with up to 2x reduction in translation error and 7x in Chamfer Distance.
Read the full abstract ▾
All authors Minjun Kang, Inkyu Shin, Taeyeop Lee, Myungchul Kim, In So Kweon, Kuk-Jin Yoon
Computer Vision
CyCLeGen: Cycle-Consistent Layout Prediction and Image Generation in Vision Foundation Models
Xiaojun Shan, Haoyu Shen, Yucheng Mao et al. (+5 more)
We present CyCLeGen, a unified
vision-
language foundation model capable of both image understanding and image generation within a single autoregressive framework. Unlike existing
vision models that depend on separate modules for perception and synthesis, CyCLeGen adopts a fully integrated architecture that enforces cycle-consistent learning through image->layout->image and layout->image->layout generation loops. This unified formulation introduces two key advantages: introspection, enabling the model to reason about its own generations, and data efficiency, allowing self-improvement via synthetic supervision under a reinforcement learning objective guided by cycle consistency. Extensive experiments show that CyCLeGen achieves significant gains across diverse image understanding and generation
benchmarks, highlighting the potential of unified
vision-
language foundation models.
Read the full abstract ▾
All authors Xiaojun Shan, Haoyu Shen, Yucheng Mao, Xiang Zhang, Abhay Anand, Bingnan Li, Haiyang Xu, Zhuowen Tu
Computer Vision
Learning Question-Aware Keyframe Selection with Synthetic Supervision for Video Question Answering
Minchan Kwon, Hyounguk Shon and Junmo Kim
Large
multimodal models (LMMs) have recently demonstrated remarkable performance in
video question answering (VideoQA), yet
reasoning over
video remains challenging due to high inference cost and diluted information. Keyframe selection offers efficiency and sharper
reasoning but suffers from sparse supervision and redundant frame choices when relying only on image-text similarity. We present a question-aware keyframe selection framework with two components: pseudo keyframe labels derived from LMMs that provide informative supervision and a coverage regularization that promotes diverse, complementary evidence across time. Experiments on NExT-QA show that our method significantly improves accuracy, especially for temporal and causal question types, establishing keyframe selection as an effective and learnable module for VideoQA.
Read the full abstract ▾
Computer Vision
Pansharpening for Thin-Cloud Contaminated Remote Sensing Images: A Unified Framework and Benchmark Dataset
Songcheng Du, Yang Zou, Jiaxin Li et al. (+4 more)
Pansharpening under thin cloudy conditions is a practically significant yet rarely addressed task, challenged by simultaneous spatial resolution degradation and cloud-induced spectral distortions. Existing methods often address cloud removal and pansharpening sequentially, leading to cumulative errors and suboptimal performance due to the lack of joint degradation modeling. To address these challenges, we propose a Unified Pansharpening Model with Thin Cloud Removal (Pan-TCR), an end-to-end framework that integrates physical priors. Motivated by theoretical analysis in the frequency domain, we design a frequency-decoupled restoration (FDR) block that disentangles the restoration of multispectral image (MSI) features into amplitude and phase components, each guided by complementary degradation-robust
prompts: the near-infrared (NIR) band amplitude for cloud-resilient restoration, and the panchromatic (PAN) phase for high-resolution structural enhancement. To ensure coherence between the two components, we further introduce an interactive inter-frequency consistency (IFC) module, enabling cross-modal refinement that enforces consistency and robustness across frequency cues. Furthermore, we introduce the first real-world thin-cloud contaminated pansharpening
dataset (PanTCR-GF2), comprising paired clean and cloudy PAN-MSI images, to enable robust benchmarking under realistic conditions. Extensive experiments on real-world and synthetic
datasets demonstrate the superiority and robustness of Pan-TCR, establishing a new
benchmark for pansharpening under realistic atmospheric degradations.
Read the full abstract ▾
All authors Songcheng Du, Yang Zou, Jiaxin Li, Mingxuan Liu, Ying Li, Changjing Shang, Qiang Shen
Computer Vision
GT-PCQA: Geometry-Texture Decoupled Point Cloud Quality Assessment with MLLM
Guohua Zhang, Jian Jin, Meiqin Liu et al. (+3 more)
With the rapid advancement of Multi-modal Large
Language Models (MLLMs), MLLM-based Image Quality Assessment (IQA) methods have shown promising generalization. However, directly extending these MLLM-based IQA methods to PCQA remains challenging. On the one hand, existing PCQA
datasets are limited in scale, which hinders stable and effective instruction tuning of MLLMs. On the other hand, due to large-scale image-text pretraining, MLLMs tend to rely on texture-dominant
reasoning and are insufficiently sensitive to
geometric structural degradations that are critical for PCQA. To address these gaps, we propose a novel MLLM-based no-reference PCQA framework, termed GT-PCQA, which is built upon two key strategies. First, to enable stable and effective instruction tuning under scarce PCQA supervision, a 2D-3D joint training strategy is proposed. This strategy formulates PCQA as a relative quality comparison problem to unify large-scale IQA
datasets with limited PCQA
datasets. It incorporates a parameter-efficient Low-Rank Adaptation (LoRA) scheme to support instruction tuning. Second, a geometry-texture decoupling strategy is presented, which integrates a dual-
prompt mechanism with an alternating optimization scheme to mitigate the inherent texture-dominant bias of pre-trained MLLMs, while enhancing sensitivity to
geometric structural degradations. Extensive experiments demonstrate that GT-PCQA achieves competitive performance and exhibits strong generalization.
Read the full abstract ▾
All authors Guohua Zhang, Jian Jin, Meiqin Liu, Chao Yao, Weisi Lin, Yao Zhao
Computer Vision
Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
Xingtai Gui, Meijie Zhang, Tianyi Yan et al. (+5 more)
End-to-end autonomous driving aims to generate safe and plausible planning policies from raw sensor input. Driving world models have shown great potential in learning rich representations by predicting the future
evolution of a driving scene. However, existing driving world models primarily focus on visual scene representation, and motion representation is not explicitly designed to be planner-shared and inheritable, leaving a schism between the optimization of visual scene generation and the requirements of precise motion planning. We present WorldDrive, a holistic framework that couples scene generation and real-time planning via unifying
vision and motion representation. We first introduce a Trajectory-aware Driving World Model, which conditions on a trajectory vocabulary to enforce consistency between visual dynamics and motion intentions, enabling the generation of diverse and plausible future scenes conditioned on a specific trajectory. We transfer the
vision and motion encoders to a downstream Multi-modal Planner, ensuring the driving policy operates on mature representations pre-optimized by scene generation. A simple interaction between motion representation, visual representation, and ego status can generate high-quality, multi-modal trajectories. Furthermore, to exploit the world model's foresight, we propose a Future-aware Rewarder, which distills future latent representation from the frozen world model to evaluate and select optimal trajectories in real-time. Extensive experiments on the NAVSIM, NAVSIM-v2, and nuScenes
benchmarks demonstrate that WorldDrive achieves leading planning performance among
vision-only methods while maintaining high-fidelity action-controlled
video generation capabilities, providing strong evidence for the effectiveness of unifying
vision and motion representation for robust autonomous driving.
Read the full abstract ▾
All authors Xingtai Gui, Meijie Zhang, Tianyi Yan, Wencheng Han, Jiahao Gong, Feiyang Tan, Cheng-zhong Xu, Jianbing Shen
Computer Vision
FAR-Drive: Frame-AutoRegressive Video Generation in Closed-Loop Autonomous Driving
Yaoru Li, Federico Landi, Marco Godi et al. (+6 more)
Despite rapid progress in autonomous driving, reliable training and evaluation of driving systems remain fundamentally constrained by the lack of scalable and interactive simulation environments. Recent generative
video models achieve remarkable visual fidelity, yet most operate in open-loop settings and fail to support fine-grained frame-level interaction between
agent actions and environment
evolution. Building a learning-based closed-loop simulator for autonomous driving poses three major challenges: maintaining long-horizon temporal and cross-view consistency, mitigating autoregressive degradation under iterative self-conditioning, and satisfying low-latency inference constraints. In this work, we propose FAR-Drive, a frame-level autoregressive
video generation framework for autonomous driving. We introduce a multi-view
diffusion transformer with fine-grained structured control, enabling geometrically consistent multi-camera generation. To address long-horizon consistency and iterative degradation, we design a two-stage training strategy consisting of adaptive reference horizon conditioning and blend-forcing autoregressive training, which progressively improves consistency and robustness under self-conditioning. To meet low-latency interaction requirements, we further integrate system-level efficiency optimizations for inference acceleration. Experiments on the nuScenes
dataset demonstrate that our method achieves state-of-the-art performance among existing closed-loop autonomous driving simulation approaches, while maintaining sub-second latency on a single GPU.
Read the full abstract ▾
All authors Yaoru Li, Federico Landi, Marco Godi, Xin Jin, Ruiju Fu, Yufei Ma, Muyang Sun, Heyu Si, Qi Guo
Computer Vision
Bridging the Intention-Expression Gap: Aligning Multi-Dimensional Preferences via Hierarchical Relevance Feedback in Text-to-Image Diffusion
Wenxi Wang, Hongbin Liu, Mingqian Li et al. (+2 more)
Users often possess a clear visual intent but struggle to articulate it precisely in
language. This intention-expression gap makes aligning generated images with latent visual preferences a fundamental challenge in text-to-image
diffusion models. Existing methods either require model training, sacrificing flexibility, or rely on textual feedback, imposing a heavy cognitive burden. Although recent training-free methods use click-based binary preference feedback to reduce user effort, they force Foundation Models (FMs) to infer preferences at the semantic level. When faced with multi-dimensional preferences, FMs suffer from inference overload and fail to identify exact preferred feature values under conflicting user signals. Consequently, a flexible framework for multi-dimensional feature
alignment remains absent. To address this, we propose a Hierarchical Relevance Feedback-Driven (HRFD) framework. Recognizing that multiple features struggle to converge simultaneously, HRFD organizes them into a three-tier hierarchy and adapts relevance feedback to enforce coarse-to-fine convergence, minimizing cognitive load. To bypass FM inference overload, HRFD decouples the process into independent single-feature preference inference tasks. Furthermore, to overcome FMs' failure in identifying preferred values, HRFD employs statistical inference to quantify the distribution divergence of features between "liked" and "disliked" image sets, achieving robust and transparent preference measurement. Crucially, HRFD operates entirely within the external text space, remaining strictly training-free and model-agnostic. Extensive experiments demonstrate that HRFD effectively captures the user's true visual intent, significantly outperforming baseline approaches.
Read the full abstract ▾
All authors Wenxi Wang, Hongbin Liu, Mingqian Li, Junyan Yuan, Junqi Zhang
Computer Vision
Video-CoE: Reinforcing Video Event Prediction via Chain of Events
Qile Su, Jing Tang, Rui Chen et al. (+2 more)
Despite advances in the application of MLLMs for various
video tasks,
video event prediction (VEP) remains relatively underexplored. VEP requires the model to perform fine-grained temporal modeling of
videos and establish logical relationships between
videos and future events, which current MLLMs still struggle with. In this work, we first present a comprehensive evaluation of current leading MLLMs on the VEP task, revealing the reasons behind their inaccurate predictions, including lack of logical
reasoning ability for future events prediction and insufficient utilization of visual information. To address these challenges, we propose Chain of Events (CoE) paradigm, which constructs temporal event chains to implicitly enforce MLLM focusing on the visual content and the logical connections between
videos and future events, incentivizing model's
reasoning capability with multiple training protocols. Experimental results on public
benchmarks demonstrate that our method outperforms both leading open-source and commercial MLLMs, establishing a new state-of-the-art on the VEP task. Codes and models will be released soon.
Read the full abstract ▾
All authors Qile Su, Jing Tang, Rui Chen, Lei Sun, Xiangxiang Chu
Computer Vision
Workflow-Aware Structured Layer Decomposition for Illustration Production
Tianyu Zhang, Dongchi Li, Keiichi Sawada et al. (+1 more)
Recent generative image editing methods adopt layered representations to mitigate the entangled nature of raster images and improve controllability, typically relying on object-based
segmentation. However, such strategies may fail to capture the structural and stylized properties of human-created images, such as anime illustrations. To solve this issue, we propose a workflow-aware structured layer decomposition framework tailored to the illustration production of anime artwork. Inspired by the creation pipeline of anime production, our method decomposes the illustration into semantically meaningful production layers, including line art, flat color, shadow, and highlight. To decouple all these layers, we introduce lightweight layer semantic
embeddings to provide specific task guidance for each layer. Furthermore, a set of layer-wise losses is incorporated to supervise the training process of individual layers. To overcome the lack of ground-truth layered data, we construct a high-quality illustration
dataset that simulated the standard anime production workflow. Experiments demonstrate that the accurate and visually coherent layer decompositions were achieved by using our method. We believe that the resulting layered representation further enables downstream tasks such as recoloring and
embedding texture, supporting content creation, and illustration editing. Code is available at: https://github.com/zty0304/Anime-layer-decomposition
Read the full abstract ▾
All authors Tianyu Zhang, Dongchi Li, Keiichi Sawada, Haoran Xie
Computer Vision
F2HDR: Two-Stage HDR Video Reconstruction via Flow Adapter and Physical Motion Modeling
Huanjing Yue, Dawei Li, Shaoxiong Tu et al. (+1 more)
Reconstructing High Dynamic Range (HDR)
videos from sequences of alternating-exposure Low Dynamic Range (LDR) frames remains highly challenging, especially under dynamic scenes where cross-exposure inconsistencies and complex motion make inter-frame
alignment difficult, leading to ghosting and detail loss. Existing methods often suffer from inaccurate
alignment, suboptimal feature aggregation, and degraded reconstruction quality in motion-dominated regions. To address these challenges, we propose F2HDR, a two-stage HDR
video reconstruction framework that robustly perceives inter-frame motion and restores fine details in complex dynamic scenarios. The proposed framework integrates a flow adapter that adapts generic optical flow for robust cross-exposure
alignment, a physical motion modeling to identify salient motion regions, and a motion-aware refinement network that aggregates complementary information while removing ghosting and noise. Extensive experiments demonstrate that F2HDR achieves state-of-the-art performance on real-world HDR
video benchmarks, producing ghost-free and high-fidelity results under large motion and exposure variations.
Read the full abstract ▾
All authors Huanjing Yue, Dawei Li, Shaoxiong Tu, Jingyu Yang
Computer Vision
EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editing
Zitong Xu, Huiyu Duan, Zhongpeng Ji et al. (+9 more)
Recent text-guided image editing (TIE) models have achieved remarkable progress, while many edited images still suffer from issues such as artifacts, unexpected editings, unaesthetic contents. Although some
benchmarks and methods have been proposed for evaluating edited images, scalable evaluation models are still lacking, which limits the development of human feedback reward models for image editing. To address the challenges, we first introduce EditHF-1M, a million-scale image editing
dataset with over 29M human preference pairs and 148K human mean opinion ratings, both evaluated from three dimensions, i.e., visual quality, instruction
alignment, and attribute preservation. Based on EditHF-1M, we propose EditHF, a
multimodal large
language model (MLLM) based evaluation model, to provide human-aligned feedback from image editing. Finally, we introduce EditHF-Reward, which utilizes EditHF as the reward signal to optimize the text-guided image editing models through reinforcement learning. Extensive experiments show that EditHF achieves superior
alignment with human preferences and demonstrates strong generalization on other
datasets. Furthermore, we fine-tune the Qwen-Image-Edit using EditHF-Reward, achieving significant performance improvements, which demonstrates the ability of EditHF to serve as a reward model to scale-up the image editing. Both the
dataset and code will be released in our GitHub repository: https://github.com/IntMeGroup/EditHF.
Read the full abstract ▾
All authors Zitong Xu, Huiyu Duan, Zhongpeng Ji, Xinyun Zhang, Yutao Liu, Xiongkuo Min, Ke Gu, Jian Zhang, Shusong Xu, Jinwei Chen, Bo Li, Guangtao Zhai
Computer Vision
ILV: Iterative Latent Volumes for Fast and Accurate Sparse-View CT Reconstruction
Seungryong Lee, Woojeong Baek, Joosang Lee et al. (+1 more)
A long-term goal in CT imaging is to achieve fast and accurate 3D reconstruction from sparse-view projections, thereby reducing radiation exposure, lowering system cost, and enabling timely imaging in clinical workflows. Recent feed-forward approaches have shown strong potential toward this overarching goal, yet their results still suffer from artifacts and loss of fine details. In this work, we introduce Iterative Latent Volumes (ILV), a feed-forward framework that integrates data-driven priors with classical iterative reconstruction principles to overcome key limitations of prior feed-forward models in sparse-view CBCT reconstruction. At its core, ILV constructs an explicit 3D latent volume that is repeatedly updated by conditioning on multi-view X-ray features and the learned anatomical prior, enabling the recovery of fine structural details beyond the reach of prior feed-forward models. In addition, we develop and incorporate several key architectural components, including an X-ray feature volume, group cross-
attention, efficient self-
attention, and view-wise feature aggregation, that efficiently realize its core latent volume refinement concept. Extensive experiments on a large-scale
dataset of approximately 14,000 CT volumes demonstrate that ILV significantly outperforms existing feed-forward and optimization-based methods in both reconstruction quality and speed. These results show that ILV enables fast and accurate sparse-view CBCT reconstruction suitable for clinical use. The project page is available at: https://sngryonglee.github.io/ILV/.
Read the full abstract ▾
All authors Seungryong Lee, Woojeong Baek, Joosang Lee, Eunbyung Park