Computer Vision
ISAP-3D: Identity-Slot Aligned Part-Aware 3D Generation
Junlin Hao, Haoshuai Fu, Xibin Song et al. (+4 more)
Part-aware 3D generation aims to synthesize structured objects with semantically meaningful components, yet often suffers from structural ambiguity due to identity-layout
entanglement. Existing methods either infer part identity and spatial layout implicitly, which can lead to unstable part allocation (e.g., slot swapping or part merging), or rely on strong layout conditions that are difficult to obtain in practice. We attribute this ambiguity to identity-slot permutation freedom: without explicit identity-slot
alignment, the correspondence between semantic parts and generation slots is not identifiable during training, allowing multiple slot assignments to fit the same supervision and leading to inconsistent decomposition. Based on this insight, we argue that stable part-aware generation requires identity-aligned one-to-one slot modelling. We therefore propose an identity-slot aligned framework, ISAP-3D, which anchors each part with semantic identity
tokens and performs identity-conditioned one-to-one layout prediction, followed by layout-conditioned geometry synthesis. Structured local-global conditioning maintains identity
alignment across semantic, spatial, and
geometric stages. We also construct a part-level
dataset with a unified semantic protocol to enable learnable and consistent identity-slot
alignment. Extensive experiments demonstrate improved structural stability, controllability, and robustness over state-of-the-art part-aware generation baselines.
Read the full abstract ▾
All authors Junlin Hao, Haoshuai Fu, Xibin Song, Wei Li, Ruigang Yang, Xinggong Zhang, Jinchuan Zhang
Computer Vision
Non-frontal face recognition using GANs and memristor-based classifiers
Semih Vazgecen, Cristian Sestito, Spyros Stathopoulos et al. (+1 more)
Face recognition systems have advanced significantly through deep learning techniques, delivering high performance and robustness in complex scenarios. However, these approaches incur substantial computational overhead, limiting their in situ applicability in resource-constrained platforms such as drones, where they can address challenges including non-frontal facial imagery. Memristor-based neuromorphic systems have emerged as a compelling approach for edge AI applications, combining biologically inspired processing with efficient and scalable computation. In this work, we propose a facial recognition framework that addresses non-frontal pose variations by integrating lightweight generative
adversarial network (GAN)-based pose frontalisation with memristor-based neuromorphic recognition. The experimental results on two
datasets demonstrate the effectiveness of combining
adversarial learning with memristive technology, achieving up to 96% identification accuracy. The proposed approach alleviates the computational bottlenecks of conventional AI and offers a scalable, efficient solution for face recognition in dynamic real-world environments.
Read the full abstract ▾
All authors Semih Vazgecen, Cristian Sestito, Spyros Stathopoulos, Themis Prodromakis
Computer Vision
World Model Self-Distillation: Training World Models to Solve General Tasks
Sebastian Stapf, Pablo Acuaviva Huertos, Aram Davtyan et al. (+1 more)
Pretrained
video generators are promising visual world models that exhibit emergent task-solving abilities; however, their reliance on detailed textual descriptions limits their direct use for planning and decision-making. Existing approaches either outsource this
reasoning to
language or
vision-
language models, or rely on supervised fine-tuning with paired task-execution
videos, which are costly to collect and difficult to scale. We propose a scalable framework that elicits task-solving ability in such models by combining self-
distillation with reinforcement learning. Given an unlabeled scene image, a
vision-
language model generates a candidate task and a detailed step-by-step solution. The solution conditions a pretrained
video diffusion model, the Demonstrator; we distill its behavior into an Executor conditioned only on the image and a short task
prompt. This transfers execution knowledge from caption-guided generation to instruction-conditioned task solving without curated task-
video supervision. We further improve the Executor with reinforcement learning from VLM feedback, exploiting the asymmetry between judging whether a sampled
video satisfies a task and generating the solution. Experiments on our proposed WorldTasks-
Benchmark and the DreamGen robotics
benchmark show that the Executor surpasses the Demonstrator under our VLM-based evaluation protocol and transfers competitively to robotic tasks.
Read the full abstract ▾
All authors Sebastian Stapf, Pablo Acuaviva Huertos, Aram Davtyan, Paolo Favaro
Computer Vision
Tac-DINO: Learning Vision-Tactile Features with Patch Alignment
Hong Li, Yankang Dong, Yue Xu et al. (+8 more)
Touch is the primary medium through which humans interact with the environment. Currently, tactile learning mainly focuses on image-level pretraining or
alignment. However, tactile signals correspond to local object contact, while research into scale
alignment and
holographic matching remains limited and proper
datasets and
benchmarks also lack. To bridge this gap, we first construct a data collection system to acquire a large-scale tactile
dataset, with over 20 K tactile contacts from 505 real-world objects. Building on this
dataset, we design a Vis-Tac
Holographic Matching
Benchmark to evaluate
vision-tactile local-to-global
alignment ability. Then we propose
Vision-Tactile Patch
Alignment (VTPA) methods for
vision-tactile representation learning. Experiments demonstrate that these exceed the performance of methods without
alignment and align with whole-object images.
Read the full abstract ▾
All authors Hong Li, Yankang Dong, Yue Xu, Yihan Tang, Mingzhu Li, Jiamin Qiu, Qihang Yao, Xing Zhu, Yujun Shen, Nan Xue, Yong-Lu Li
Computer Vision
Performance Analysis of YOLOv11 and YOLOv8 for Mixed Traffic Object Detection under Adverse Weather Conditions in Developing Countries
Quoc Thuan Nguyen, Ha Anh Vu, Ngo Dang Thanh Ngan et al. (+1 more)
In modern vehicular systems, robust performance under harsh conditions has become a critical problem of autonomous driving. Our study delivers a comprehensive evaluation of the newest iteration of the YOLO series, which is YOLOv11 Nano architecture benchmarked against the widely adopted YOLOv8 Nano as a baseline on a custom fused
dataset that combines the Indian Driving
Dataset (IDD) [1] and Berkeley Deep Drive
Dataset (BDD100K) [2]. We have analyzed the trade-offs among detection accuracy, inference speed, and computational efficiency in high-
entropy scenarios involving dense mixed traffic, rain, and low-light conditions. Specifically, YOLOv11n achieves a mean Average Precision (mAP@50) of 46.6%, with a notable 3.2% improvement in Precision over the baseline, effectively reducing false positives in cluttered scenes. Furthermore, the proposed model exhibits enhanced energy efficiency, requiring 22% fewer FLOPs (6.3G vs. 8.1G) while maintaining real-time inference speed of 70.9 FPS on a Tesla T4 GPU, offering an optimal trade-off for safety-critical edge deployment.
Read the full abstract ▾
All authors Quoc Thuan Nguyen, Ha Anh Vu, Ngo Dang Thanh Ngan, Minh Phuc Hoang Ngoc
Computer Vision
MFEN:Multi-Frequency Expert Network for Visible-Infrared Person Re-ID
Xulin Li, Yan Lu, Bin Liu et al. (+4 more)
Visible-infrared person re-identification (VI-ReID) is challenging due to the large modality discrepancy between visible and infrared images. We contend that this discrepancy is largely related to differing lighting conditions, including differences in light wavelength and light source type. Recently, frequency-based VI-ReID approaches have achieved notable success because frequency information can better extract identity-relevant contours and details while excluding irrelevant lighting and color. However, existing methods either do not distinguish different frequency bands or focus on only one band, which is insufficient under diverse lighting conditions. To perform comprehensive frequency domain learning, we propose a Multi-Frequency Expert Network (MFEN) that enables multi-frequency modulation and adaptively combines different bands through a mixture-of-experts design. We further introduce Random Frequency Augmentation (RFA) and Frequency Auxiliary Optimization (FAO) to better train MFEN. The three modules are complementary and jointly capture critical frequency-domain details for robust representation learning. Extensive experiments on three VI-ReID
datasets demonstrate the effectiveness of our approach.
Read the full abstract ▾
All authors Xulin Li, Yan Lu, Bin Liu, Qinhong Yang, Qi Chu, Tao Gong, Nenghai Yu
Computer Vision
Metadata-Aware Multi-Prompt Reasoning for Zero-Shot Accident Understanding
Tarandeep Singh, Soumyanetra Pal, Soham Biswas et al. (+1 more)
In this paper, we address the problem of zero-shot understanding of accidents from surveillance
videos by identifying when an impact event occurs, what type of impact it is, and where in the frame it occurs using natural
language. We propose a three-stage pipeline that decomposes the accident understanding into when, what, and where. The first stage extracts a short temporal window around the impact using
vision-
language similarity. In the second stage, we perform metadata-driven multi-
prompt reasoning with five complementary views (baseline, motion, geometry, contrast, and tiebreaker) and resolve disagreement via an
entropy-gated pairwise adjudicator. Finally, we localize the impact of an open-vocabulary detector queried on the predicted accident type and scene layout, and aggregate detections across keyframes using a score-weighted centroid. Our pipeline achieves a substantial improvement in the harmonic-mean score over a centre-of-frame baseline on the zero-shot ACCIDENT @ CVPR
benchmark. We show that decomposing zero-shot
video understanding into temporal localization, semantic classification, and spatial grounding enable more reliable
reasoning with
vision-
language models than direct prompting alone.
Read the full abstract ▾
All authors Tarandeep Singh, Soumyanetra Pal, Soham Biswas, Nishanth Chandran
Computer Vision
Vision Transformers for Face Recognition Need More Registers
Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira et al. (+2 more)
Recent advances in
Vision Transformers (ViTs) for face recognition (FR) have moved beyond the standard CLS-
token paradigm. In this paradigm, a special classification
token (CLS) is prepended to the patch
embeddings and used as a representation of the input for downstream tasks. An alternative approach, Concatenated Patch
Embeddings (CPE), instead leverages all patch
tokens by concatenating them into a single vector, which is then projected into a compact face representation. CPE has been shown to improve recognition performance in comparison to CLS-based ones, but our qualitative analysis of
attention maps showed the presence of artifacts that limit their interpretability. To address this issue, we incorporate register
tokens, learnable
tokens concatenated to the initial patch
embeddings, and processed jointly through the ViT encoder blocks. This mechanism has been shown to produce more structured and interpretable
attention maps compared to baseline ViT. We empirically demonstrate that these artifacts consistently appear across various ViT backbones, including small and large models, and that introducing register
tokens effectively mitigates them. Adding four or eight registers significantly enhances interpretability, with eight registers providing the highest verification accuracies and smoothest
attention structures. Our resulting model, ViT-8R, corresponds to a CPE-based ViT-B architecture augmented with eight register
tokens achieves state-of-the-art performance among ViT-based FR models on large-scale IJB-B and IJB-C
benchmarks. Also, ViT-8R produces substantially clearer
attention maps compared with the baseline model, which offer deeper insight into the model's
attention behavior (https://github.com/TaharChettaoui/ViT-FR-Registers)
Read the full abstract ▾
All authors Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira, Naser Damer, Fadi Boutros
Computer Vision
SpikeTAD: Spiking Neural Networks for End-to-End Temporal Action Detection
Min Yang, Mi Zhou and Limin Wang
Video understanding is a crucial part of computer
vision, with numerous application scenarios. With the increasing popularity of mobile devices, an increasing number of efforts are trying to deploy
video understanding models on them. However, existing
video understanding models are difficult to deploy due to their large size and prohibitive power consumption. Spiking Neural Networks (SNNs) have shown bioplausibility and low power advantages over Artificial Neural Networks (ANNs), especially on neuromorphic chips which are regarded as essential components of future mobile devices. However, excessively long conversion time-steps and severe performance degradation problems limit their application. To solve the problems above, we explore the application of SNNs on temporal action detection (TAD), which is an important task in
video understanding, and propose the first SNN-based end-to-end TAD architecture coined as SpikeTAD. While maintaining extremely low power consumption, SpikeTAD achieves an average mAP of 67.2% in THUMOS14 and 37.42% in ActivityNet-1.3, demonstrating the feasibility of a low-power TAD model. Our code is available at https://github.com/MCG-NJU/SpikeTAD.
Read the full abstract ▾
Computer Vision
ViT-FREE: Efficient Face Recognition via Early Exiting and Synthetic Adaptation
Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira et al. (+2 more)
Vision Transformers (ViTs) have gained significant
attention in computer
vision and shown strong potential for face recognition (FR). However, their high computational cost makes deployment on resource-constrained devices challenging, motivating the need for methods that balance efficiency and accuracy. In this work, we investigate early exiting in pretrained ViTs as a simple yet effective training-free strategy for efficient FR inference. Leveraging the uniform feature dimensionality across
transformer encoder blocks, we introduce ViT-FREE, a multi-exit framework that enables face verification directly from intermediate representations without modifying or retraining the backbone model, and thus, reducing inference cost. Empirically, we show that patch
embeddings and
attention maps evolve progressively across depth, exhibiting high similarity between consecutive ViT blocks and increasing
alignment with the final representation. This indicates gradual feature refinement and
attention convergence, suggesting that intermediate layers already provide stable and discriminative representations suitable for early exiting. Through extensive experiments on multiple FR
benchmarks, we systematically analyze the accuracy-efficiency trade-off across exit depths. Our results demonstrate that later exits achieve a highly favorable balance, with exiting at layer 10 yielding up to a 20% speedup while incurring only a 1.5 drop in verification performance on
benchmarks such as IJB-C. Also, we propose ViT-FREE_FT, a lightweight exit-specific fine-tuning strategy that adapts only the projection layers using a small synthetic
dataset while keeping the
transformer backbone frozen. This approach improves the performance of shallow exits while preserving the efficiency benefits and leaving deeper exits largely unaffected.
Read the full abstract ▾
All authors Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira, Naser Damer, Fadi Boutros
Computer Vision
FitVTON: Fit-aware Virtual Try-On via Body-Garment Size Control
Yiqun Ning, Ao Shen, Chenhang He et al. (+1 more)
While
diffusion-based virtual try-on has achieved impressive visual realism, most methods treat the task as 2D inpainting, prioritizing texture preservation over physical plausibility. Consequently, they often produce plausible-looking images that fail to reflect authentic garment fit across diverse body shapes. We present FitVTON, a Fit-aware virtual try-on model on different bodies in the wild. FitVTON encodes garment-body size through structured text
prompts, and learn from simulated try-on triplets from parameterized garment model. To improve the fitting effects over garment silhouettes, we introduce two auxiliary head to predict the masks for both the garment and the exposed body. We further introduce a texture rectification stage to improve realistic appearance from simulated data. To evaluate the fitting fidelity, we curate a real-world
dataset, FittingEffect3K, combining VLM-based scoring protocol. Both subjective and quantitive experiments show that FitVTON demonstrate authentic fitting fidelity, with significant sizing accuracy and shape preservation over state-of-the-art methods while maintaining competitive image quality. Project Page: https://zenoning.github.io/FitVTON/.
Read the full abstract ▾
All authors Yiqun Ning, Ao Shen, Chenhang He, Lei Zhang
Computer Vision
From Nominal Intensity to Equivalent Rainfall: A Path-Based Credibility Evaluation Framework for Simulated Rainfall in Autonomous-Driving Perception Tests
Tian Xia, Xin Zhao, Shaolingfeng Ye et al. (+1 more)
Credible simulated-rainfall conditions are essential for identifying perception-system boundaries and supporting SOTIF-oriented risk assessment in automated driving. However, closed-field tests are often described only by nominal rainfall intensity or single-point measurements, making it difficult to align simulated rain fields with real rainfall and map test results to real-world scenarios. This paper proposes a path-based credibility evaluation method for simulated rainfall in autonomous-driving perception tests. Using the drop size and velocity joint distribution of real rainfall as the reference, each candidate path is represented by path-equivalent rainfall intensity, an uncertainty band, and a path-averaged Realism of Raindrop Distribution (RRD) score.
Lidar target point-cloud count and mean reflectivity are further used for perception-consistency correction, quantifying the proxy capability of each simulated-rainfall path for real-rainfall perception effects. Experiments are conducted using about 10,000 real-rainfall raindrop-spectrum samples, 728 RainSense perception samples, and 45 spatial sampling points in a 2.4 m x 7.2 m simulated-rainfall area. Results show that spatial non-uniformity remains under the same nominal condition, confirming the need for path-based evaluation. The method identifies Path IV and Path VI as preferable candidates, with results of 11.54 +/- 0.31 mm/h, RRD = 0.43, and 8.28 +/- 0.34 mm/h, RRD = 0.46, respectively. These paths show more balanced performance in rainfall-intensity stability, raindrop-spectrum realism, and perception consistency. The proposed method supports path selection, condition description, and credible interpretation of autonomous-driving perception tests under rainfall.
Read the full abstract ▾
All authors Tian Xia, Xin Zhao, Shaolingfeng Ye, Junyi Chen
Computer Vision
ParseFixer: An Agentic Framework for Document Parsing via Selective Multimodal Correction
LeKai Yu, Hao Liu, Kun Wang et al. (+4 more)
In this report, we present our third-place solution for the DataMFM Challenge Track 1: Document Parsing. This track requires models to recover structured Markdown documents from document page images while preserving textual content and document structure. To address the complementary requirements of accurate content recovery and faithful structure reconstruction, we propose ParseFixer, an agentic framework for backbone parsing and selective correction. ParseFixer consists of two key modules: Full-Page Backbone Parsing (FBP) and Agentic Selective Correction (ASC). FBP produces stable initial Markdown outputs with MinerU2.5 Pro, while ASC detects high-value parsing failures and repairs them through a verify-and-rollback correction process. By placing selective
multimodal correction after open-source backbone parsing, ParseFixer improves the recovery of key document elements without rewriting reliable backbone predictions. On the test set, our final system achieves an overall score of 61.78 and ranks third in Track 1, demonstrating its effectiveness for accurate document parsing. Our code will be released at: https://github.com/iLearn-Lab/CVPRW26-ParseFixer.
Read the full abstract ▾
All authors LeKai Yu, Hao Liu, Kun Wang, Zhiran Li, Ruping Cao, Fan Liu, Yupeng Hu
Computer Vision
SpecLoR: Spectral Lookahead Rectification for Motion-Coherent Text-to-Video Generation
Xu Zhang, Yu Lu, Ruijie Quan et al. (+3 more)
Flow Matching has enabled robust text-to-
video generation via latent ODE sampling. However, velocity approximation and numerical discretization errors inevitably accumulate, causing sampling trajectories to drift. Consequently, generated
videos often suffer from severe spatiotemporal inconsistencies. Nevertheless, directly correcting these drifted, noisy latents is challenging: (i) timestep-dependent noise obscures reliable structural cues; (ii) spatial interventions risk disrupting intricate local geometry while incurring heavy computational costs. To address this, we propose Spectral Lookahead Rectification (SpecLoR), a plug-and-play inference method that bypasses noise via lookahead prediction, and circumvents spatiotemporal
entanglement by shifting corrections to the frequency domain, where universal statistical priors of natural
videos are readily available. First, during early sampling stages, SpecLoR looks ahead to estimate the clean latent $z_{t,0}$ and computes its 3D spatiotemporal spectrum. Next, SpecLoR rectifies the amplitude spectrum to match the prior, leaving the phase intact. Finally, the corrected state is re-noised to resume ODE integration. Experiments on Wan2.2 demonstrate that SpecLoR significantly reduces physical artifacts and enhances motion coherence across multiple
benchmarks with minimal computational overhead (4 additional NFEs).
Read the full abstract ▾
All authors Xu Zhang, Yu Lu, Ruijie Quan, Zhaozheng Chen, Bohan Wang, Yi Yang
Computer Vision
Feature extraction for plant growth estimation
Simbarashe Aldrin Ngorima, Albert Helberg and Marelie H. Davel
Precision agriculture requires the estimation of plant growth stages in real-time. When the plant growth stage is known, the wastage of resources in cultivation, such as nutrients and water, is reduced as only the required resources need to be supplied. Plants at different growth stages, however, have similar morphological features, which can make autonomous growth stage estimation difficult. This paper presents two feature extraction methods for growth stage estimation: one that uses a bank of Gabor filters and morphological operations, and the other that uses pre-trained convolutional neural networks (CNNs) and transfer learning. We test these methods on a publicly available plant growth stage
dataset (“bccr-segset“) for two species, canola and radish, grown and captured under indoor conditions. The two proposed feature extraction methods are compared, using support vector machines and boosted trees as classifiers. We find that both methods are suitable for real-time applications, and that CNN features outperform the hand-crafted features, both with regard to speed and accuracy. The best system (VGG-19 features, classified with a radial basis function support vector machine) obtained an accuracy of 98.4% for both species, processing an image in 0.08 seconds.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.HC
Frozen Multimodal Embeddings for AI-Assisted Interview Assessment of Personality and Cognitive Ability
Kuo-En Hung, Hung-Yue Suen, Shih-Ching Yeh et al. (+1 more)
Predicting psychological traits from asynchronous
video interviews (AVIs) is a challenging problem in AI-assisted interview assessment because labeled
datasets are limited while each response contains high-dimensional visual, acoustic, and verbal signals. This paper presents our solution for the ACM Multimedia AVI Challenge 2026, which evaluates two tasks: Track~1 predicts self-reported HEXACO personality traits from personality-related interview responses, and Track~2 classifies cognitive ability levels from structured AVI responses. We treat the problem as a small-sample representation learning task. Instead of fine-tuning large pretrained models, we use frozen
multimodal encoders, including CLIP for visual features, Whisper for acoustic features and transcripts, and RoBERTa, E5, and DeBERTaV3 for textual representations, followed by low-capacity downstream models. For Track~1, our trait-specific regression and late-fusion system achieves an average validation MSE of 0.2696, improving over the official baseline of 0.3334. Ablation results show a three-step improvement from a global model (0.3189), to per-trait modeling (0.2871), to per-trait late fusion (0.2696), corresponding to a 19.1% relative MSE reduction over the official baseline. For Track~2, a compact subject-attribute baseline reaches 0.5781 accuracy, while our
multimodal ensemble reaches 0.5313, both above the official baseline of 0.4062. We interpret this result as evidence of possible subject-attribute shortcuts in the validation split rather than robust cognitive inference from AVI content. Overall, our findings suggest that AVI-based psychological assessment benefits from trait-specific
multimodal modeling, but cognitive ability prediction requires careful control of
dataset shortcuts.
Read the full abstract ▾
All authors Kuo-En Hung, Hung-Yue Suen, Shih-Ching Yeh, Hsiang-Wen Wang
Computer Vision
Corpus Augmentation for Sign Language Translation via LLM-Guided Video Stitching
Zsolt Robotka, Ádám Rák, Jalal Al-Afandi et al. (+2 more)
Sign
language translation (SLT) converts sign
language video into spoken
language text and holds significant promise for improving accessibility and enabling communication between signing and non-signing communities. While large weakly-aligned
datasets have enabled pre-training at scale and gloss-free methods have reduced reliance on expert annotation, high-quality parallel sign
video-text pairs for fine-tuning remain scarce, limiting generalisation on long-tail vocabulary and unseen constructions. We propose a corpus augmentation approach that requires no additional human annotation, external sign-
language video corpora, or generative
video models, relying only on the existing gloss-annotated training corpus and an LLM for sentence generation: per-gloss clips are extracted from training
videos via CTC forced-
alignment, novel gloss-sentence pairs are generated by a corpus-anchored LLM, and synthetic sequences are assembled through random sentence sampling and clip assignment. The resulting synthetic RGB
video-text pairs are architecture-agnostic at the downstream training stage and can be consumed directly by RGB-based SLT models, or converted into pose or feature representations by pipelines that derive such inputs from
video. Sincan et al. re-evaluated five recent gloss-free methods under strictly identical conditions; the largest verified gain over the GFSLT-VLP baseline was only 0.98 BLEU-4. Our augmentation, applied within the same framework, achieves +2.92 BLEU-4 without any change to architecture or training protocol. We further identify that synthetic data harms
vision-
language pretraining despite improving its objectives, and that optimising clip transitions for visual smoothness is counter-productive under L2-based criteria; we propose that abrupt boundaries may act as a form of implicit regularisation. Code is available at https://github.com/robizso/slt-datagen.
Read the full abstract ▾
All authors Zsolt Robotka, Ádám Rák, Jalal Al-Afandi, András Horváth, György Cserey
Computer Vision
From Content to Knowledge: Lightning Fast Long-Video Understanding with Neural Knowledge Representations
Yuchen Guan, Xiao Li, Zongyu Guo et al. (+4 more)
We propose a new paradigm for long
video understanding by treating a long
video as a Neural Knowledge Representation (NKR). NKR represents
video contents neither as a stream of
tokens nor pre-organized databases, but as an individual small portion of network weights attached to the VLM backbone. The NKR weights are optimized to encapsulate the
video's semantic content via a novel Agentic Knowledge
Distillation (AKD) process, where an
agent automatically synthesizes dense descriptions and question-answer pairs to distill the
video's knowledge into the NKR. While AKD serves as a comprehensive, one-time encoding phase, the resulting NKR transforms the
video into a portable, reusable asset. At inference, the lightweight NKR is mounted onto a frozen
Vision-
Language Model (VLM), enabling direct, query-based understanding without reloading or re-encoding the original
video. This approach decouples
video length from inference cost, offering high amortized efficiency for multi-turn
video understanding. Experiments on the LVBench
benchmark show our method achieves performance comparable to state-of-the-art approaches while reducing end-to-end latency by over two orders of magnitude, opening new possibilities for interactive long-
video understanding.
Read the full abstract ▾
All authors Yuchen Guan, Xiao Li, Zongyu Guo, Xiaoyi Zhang, Xiulian Peng, Chun Yuan, Yan Lu
Computer Vision
Wild3R: Feed-Forward 3D Gaussian Splatting from Unconstrained Sparse Photo Collection
Yuto Furutani, Takashi Otonari, Kaede Shiohara et al. (+1 more)
Feed-forward 3D Gaussian Splatting (3DGS) removes the need for time-consuming per-scene optimization required by traditional 3DGS. However, existing feed-forward approaches struggle with real-world photo collections that include diverse lighting conditions and transient objects. In this paper, we present Wild3R, a feed-forward approach for unconstrained sparse photo collections. The main bottleneck is the lack of training data that provides multiple viewpoints, a variety of illuminations, and transient variations necessary for learning robust scene representations. To address this, we introduce the WildCity
dataset, which comprises 200 scenes, 170 lighting conditions, and transient objects, resulting in 337,500 images in total. By leveraging the
dataset, our model learns appearance consistency across viewpoints conditioned on reference views, while removing transient content. Extensive experiments demonstrate that our method outperforms existing feed-forward approaches and achieves results competitive with prior per-scene optimization-based methods.
Read the full abstract ▾
All authors Yuto Furutani, Takashi Otonari, Kaede Shiohara, Toshihiko Yamasaki
Computer Vision
Task-Aligned Stability Analysis of Vision-Language Models for Autonomous Driving Hazard Detection
Everett Richards
Vision-
language models (VLMs) are increasingly used for scene understanding in autonomous driving, but robustness analysis often relies on task-agnostic
embedding stability alone. We study whether corruption-induced
embedding drift predicts changes in a task-aligned hazard score derived from CLIP image-text similarities. Using controlled corruptions on BDD100K road scenes, we compare
embedding drift against margin drift, defined as the change in hazard score under perturbation. The relationship is highly corruption-dependent: some families exhibit strong coupling between representation drift and decision drift, while others induce hazardous decision instability despite relatively modest
embedding change. Furthermore, corruption families differ in failure direction: most suppress hazard detections via false negatives, while occlusion instead triggers false alarms, suggesting that
benchmark design should account for asymmetric failure modes, not just overall instability rates. These results suggest that robustness
benchmarks should include task-aligned stability measures in addition to
embedding-level perturbation statistics.
Read the full abstract ▾
Computer Vision
Image Quality Assessment of Identity Cards Using Measures from Open Face Image Quality
Gregor Grote, Juan E. Tapia and Christian Rathgeb
This paper addresses the challenge of assessing image quality in ID cards in remote verification systems by applying capture-related quality measures from the Open Face Image Quality (OFIQ) standard to ID card images. Our preprocessing pipeline includes corner detection, perspective normalization, and comprehensive foreground masking to ensure accurate and unbiased quality measure computation. We evaluate the effectiveness of these measures by analyzing their correlation with the performance of three presentation attack detection (PAD)
algorithms across four diverse ID card
datasets, where two
datasets contain bona fide, i.e. pristine, images and two contain printed mock ID cards. Our results suggest that quality assessment based on some OFIQ measures can significantly improve PAD performance.
Read the full abstract ▾
Computer Vision
SG2Loc: Sequential Visual Localization on 3D Scene Graphs
Nicole Damblon, Olga Vysotska, Federico Tombari et al. (+2 more)
Visual localization in complex indoor environments remains a critical challenge for robotics and AR applications. Sequential localization, where pose estimates are refined over time, is important for autonomous
agents. However, traditional methods often require storing extensive image databases or point clouds, leading to significant overhead. This paper introduces a novel, lightweight approach to sequential visual localization using 3D scene graphs. Our method represents the environment with a compact scene graph, where nodes represent objects (with coarse meshes) and edges encode spatial relationships. For each image in the localization phase, we extract per-patch semantic features, predicting object identities. Localization is performed within a particle filter framework. Each particle, representing a camera pose, projects the coarse object meshes from the scene graph into the image, assigning object identities to patches based on visibility. The similarity of the per-patch features, in the input image, and object features from the scene graph determines the weight of a particle. Subsequent images are incorporated sequentially, refining the pose estimate. By leveraging a compact scene graph and efficient semantic matching, our method significantly reduces storage while maintaining performance on real-world
datasets. The code will be available at https://github.com/DmblnNicole/sg2loc.
Read the full abstract ▾
All authors Nicole Damblon, Olga Vysotska, Federico Tombari, Marc Pollefeys, Daniel Barath
Computer Vision
Task-Aware Structured Memory for Dynamic Multi-modal In-Context Learning
Zhirui Chen, Ziwei Chen and Ling Shao
Multi-modal large
language models (MLLMs) depend on in-context learning (ICL) for rapid task adaptation, but their scalability is severely limited by finite context windows and the growing cost of key-value (KV) caches in long multi-modal sequences. Existing
memory compression approaches typically rely on rigid
token removal or sample-dependent importance estimation, which introduces bias, disrupts semantic structure, particularly for visual representations, and yields static
memories that cannot adapt to new queries. We introduce TASM (Task-Aware Structured
Memory), a training-free framework that addresses these limitations through task-aware, structure-preserving, and dynamically accessible
memory construction. TASM employs task-vector guided compression to replace sample-specific signals with a task-level direction that captures shared relevance across demonstrations. To preserve the underlying
manifold, it applies semantics-aware
token merging via bipartite graph matching, aggregating
tokens without destructive
pruning. Finally, TASM structures
memory into a hierarchy comprising a compact Core
Memory and a Latent Bank, facilitating query-adaptive dynamic
retrieval. Evaluations confirm TASM maintains high performance under heavy compression, effectively balancing efficiency with adaptability.
Read the full abstract ▾
Computer Vision
SheafStain: Sheaf-Theoretic Schrödinger Bridge for Spatially and Biologically Coherent Virtual Staining
Hyeongyeol Lim, Hongjun Yoon, Eunjin Jang et al. (+3 more)
Current virtual staining approaches offer the potential for time- and cost-efficient biomarker quantification in cancer diagnostics and prognostics. However, patch-wise inference for gigapixel whole-slide images fails to maintain spatial continuity, yielding artifacts that cause catastrophic mismatches with ground-truth images. Although pathology
Vision Foundation Models (VFMs) offer rich representations, their self-
attention causes varying global contexts to produce inconsistent
embeddings for the same physical region. We formalize and validate this "context contamination" as a sheaf-theoretic problem where these
embeddings form a presheaf whose sections disagree on overlaps, so no global section restricts to them. To address this, we propose SheafStain, a new approach that reinterprets VFM features as sheaf-like sections for spatially and biologically coherent virtual staining. Specifically, SheafStain integrates class and patch
tokens into a Schrödinger Bridge framework as sheaf-like sections. While the class
token anchors biological consistency, patch
tokens form a per-position spatial map. A feature space from a single VFM encoder pretrained on Hematoxylin & Eosin and Immunohistochemistry supervises both input conditioning and output stain
alignment. Departing from prior work that evaluates on isolated $256 \times 256$ patches and either random-crops or resizes the $1024 \times 1024$ ground truth, we translate at $256 \times 256$ and evaluate on the stitched $1024 \times 1024$ outputs across HER2, ER, PR, and Ki-67. SheafStain demonstrates promising results against six prior methods while mitigating patch-boundary stitching artifacts. Code is available at https://github.com/deepnoid-ai/SheafStain.
Read the full abstract ▾
All authors Hyeongyeol Lim, Hongjun Yoon, Eunjin Jang, Daeky Jeong, Won June Cho, Hwamin Lee
Computer Vision
Scene-Adaptive Nonlinear Tone Curves for Pseudo Ground-Truth Generation in Low-Light 3D Gaussian Splatting
Mingzhe Lyu, Jinqiang Cui and Hong Zhang
Low-light novel view synthesis is challenging because dark multi-view images contain noise, weak structural detail, and compressed dynamic range. Recent 3D Gaussian Splatting (3DGS) methods address these challenges by generating pseudo ground-truth (pseudo-GT) images as supervision targets when paired normal-light references are unavailable. Existing pseudo-GT methods apply a uniform linear gain to all pixels, which clips bright regions while providing insufficient enhancement in dark regions, limiting reconstruction quality. We observe that nonlinear tone mappings, long established in 2D low-light enhancement, have not been explored for pseudo-GT generation in 3D reconstruction. Accordingly, we propose a scene-adaptive nonlinear tone-curve framework that replaces linear pseudo-GT with nonlinear alternatives. The framework introduces percentile-based normalisation for scene-agnostic curve application, a scene-adaptive offset for automatic black-level adjustment, and two complementary curves: Adaptive SoftExp (ASE), a bounded exponential curve, and Adaptive Poly3 (AP3), a data-driven cubic
polynomial. The module changes only the pseudo-GT computation and leaves the 3DGS backbone unchanged. Experiments on three
benchmarks covering 21 scenes show that both curves consistently outperform the linear baseline with PSNR improvements up to +4.34 dB on LOM and +3.25 dB on RealX3D. Both curves achieve similar performance despite their different mathematical forms, suggesting the improvement is curve-agnostic. Code is available at https://github.com/lvmingzhe/adaptiveToneCurve
Read the full abstract ▾
Computer Vision
Plan-and-Verify Video Reward Reasoning with Spatio-Temporal Scene Graph Grounding
Hyomin Kim, Junghye Kim, Joanie Hayoun Chung et al. (+4 more)
Reward models for text-to-
video (T2V) generation guide post-training but often fail at fine-grained semantic
alignment. We trace this to two structural weaknesses in existing
reasoning-based reward models: they do not systematically verify every condition described in the
prompt, and the visual evidence supporting each judgment remains implicit in their free-form
reasoning. We propose SG-PVR, a
video reward model that addresses these limitations through plan-and-verify
reasoning grounded in spatio-temporal scene graphs. The verification plan decomposes the
prompt into atomic claims, making the set of requirements to be checked explicit. The spatio-temporal scene graph, encoding entities, attributes, and temporally grounded relations, is extracted from the
video and maintained as a persistent structured visual reference throughout
reasoning. Each claim is verified against both the
video and the scene graph, anchoring judgments in explicit visual evidence. SG-PVR achieves strong performance on semantic
alignment, including fine-grained temporal semantics. As a test-time reranker, it further enhances compositional
alignment in T2V generation.
Read the full abstract ▾
All authors Hyomin Kim, Junghye Kim, Joanie Hayoun Chung, Yoonjin Oh, Kyungjae Lee, Sungbin Lim, Sungwoong Kim
Computer Vision
LASA: A Weak Supervision Method for Open-Vocabulary Scene Sketch Semantic Segmentation
Liwen Yi, Xianlin Zhang, Yue Zhang et al. (+2 more)
Open-vocabulary scene sketch semantic
segmentation aims to assign dense semantic labels to sparse line drawings based on flexible category vocabularies specified at inference time, without relying on pixel-level annotations during training. Unlike natural images, sketches lack texture and color cues, making semantic understanding heavily dependent on stroke layout and spatial configuration, a challenge that renders single-layer
vision-
language features inherently unstable. Our key observation is that
attention maps from different
Vision Transformer layers encode complementary spatial cues: shallow layers capture global structural layouts, while deeper layers focus on local stroke intersections and object parts. This suggests that cross-layer aggregation provides a more robust structural prior than any individual layer alone. Leveraging this insight, we propose a structure-aware framework built upon Layer-wise Accumulated Structural
Attention (LASA), which aggregates multi-layer
attention to guide hierarchical semantic
alignment under weak supervision and refine predictions during inference. Experiments on FS-COCO, SFSD, and FrISS show that LASA improves mIoU by $+3.43$, $+8.01$, and $+15.74$ over the prior weakly supervised baselines, demonstrating consistent gains in both
segmentation accuracy and spatial coherence. Our source code will be made publicly available.
Read the full abstract ▾
All authors Liwen Yi, Xianlin Zhang, Yue Zhang, Yue Ming, Xueming Li
Computer Vision
TextHOI-3D: Text-to-3D Hand-Object Interaction via Discrete Multi-View Generation and Joint Mesh Optimization
Zixiong Hao and Zhencun Jiang
Text-conditioned 3D generation has progressed rapidly for images and isolated objects, but producing a hand-object mesh remains challenging: the output must preserve
language semantics, cross-view consistency, object geometry, articulated hand shape, and physically plausible contact. We present TextHOI-3D, a staged framework that uses generated multi-view observations as an explicit interface between text-conditioned visual generation and geometry-aware hand-object recovery. TextHOI-3D learns a compact VQ
token space for fixed-camera hand-object observations, predicts multi-view visual
tokens from text with a CLIP-conditioned visual autoregressive model, and recovers a unified hand-object mesh through prior initialization, multi-view joint optimization, and anti-penetration refinement. The design separates semantic generation from
geometric recovery while keeping both stages connected by a discrete multi-view representation. On HO3D-derived evaluations, the multi-view setting reduces object CD from 17.26 mm to 4.92 mm and penetration volume from 5.3721 cm^3 to 0.2193 cm^3 compared with a single-view counterpart, while improving hand errors and surface F-scores. These results support multi-view visual
tokens as an effective intermediate representation for text-driven 3D hand-object mesh creation.
Read the full abstract ▾
Computer Vision
Yuansheng Gao, Wenbin Xing, Jiahao Yuan et al. (+4 more)
Video Large
Multimodal Models have achieved remarkable progress in
video understanding, yet they remain prone to
hallucinations, where generated responses are not faithfully supported by the input
video. In this paper, we propose MultiToP, a
multimodal-context-aware visual
token patching framework that mitigates
hallucinations by refining unreliable visual
tokens before
language generation. MultiToP introduces a lightweight Visual
Token Patcher to predict
token-level replacement distributions and selectively substitute unreliable visual
tokens with a dynamic global patch
token. To train the patcher effectively, we further propose information-guided rank calibration, which uses answer-conditioned frame-level information cues derived from the backbone to guide
token replacement. Combined with ground-truth answer supervision and sparsity regularization, MultiToP enables localized visual evidence refinement without modifying the original model. Extensive experiments demonstrate that MultiToP effectively reduces
hallucinations on Vript-HAL with negligible inference overhead, improving the F1 scores of Qwen3-VL-4B-Instruct by 50.60% over the vanilla model. Meanwhile, MultiToP preserves general
video understanding ability, yielding an 18.58% relative accuracy gain on ActivityNet-QA for
Video-LLaVA-7B.
Read the full abstract ▾
All authors Yuansheng Gao, Wenbin Xing, Jiahao Yuan, Kaiwen Zhou, Han Bao, Zonghui Wang, Wenzhi Chen
Computer Vision
A Comprehensive Ecosystem for Open-Domain Customized Video Generation
Jingxu Zhang, Yuqian Hong, Daneul Kim et al. (+6 more)
Recent progress in
video generation has shown impressive visual synthesis capabilities. However, open-domain customized
video generation remains limited by the lack of large-scale, annotated
datasets capturing diverse identity-specific attributes. To address this, we introduce PexelsCustom-1M, the first publicly available million-scale
dataset for identity-preserving
video generation, containing one million curated <identity, text,
video> triplets across 8,000+ categories. Leveraging this, we propose CustoMDiT, a parameter-efficient framework that adapts a pretrained
multimodal Diffusion Transformer into a customized
video generator with only 8% additional learnable parameters. Our method surpasses prior state-of-the-art. However,
benchmarks such as DreamBooth cover only 100 classes, which is insufficient for real-world applications. To overcome this, we construct OpenCustom, a new
benchmark with 1,000+ categories, created via cross-
dataset knowledge fusion from ImageNet and MS-COCO. Extensive experiments confirm the advantages of both our
dataset and model. We will open-source the entire ecosystem–including
dataset, pipeline,
benchmark, and implementations–to support further research.
Read the full abstract ▾
All authors Jingxu Zhang, Yuqian Hong, Daneul Kim, Kai Qiu, Qi Dai, Jianmin Bao, Yifan Yang, Xiaoyan Sun, Chong Luo