Computer Vision
More with Less: a Large Scale Remote Sensing VLM with a Simple Recipe
Stefan Maria Ailuro, Mario Markov, Mohammad Mahdi et al. (+2 more)
Remote sensing
vision-
language models are increasingly expected to support open-ended
reasoning over Earth Observation data and a variety of tasks. Most recent progress in this area has been driven by remote-sensing-specific architectural designs, often introducing new encoders,
alignment modules, or task-specific fusion mechanisms. In this work, we challenge the necessity of such architectural specialization. We show that a generally capable
vision-
language model can achieve competitive or state-of-the-art performance at challenging remote sensing
benchmarks, provided that it is trained at sufficient scale across diverse data and tasks. Our model uses a single
language policy that can either answer directly in text or invoke a localization tool for
segmentation and grounding. To train this heterogeneous behaviour, we employ a multi-task reinforcement learning framework with adaptive task rewards covering multiple-choice VQA, free-form VQA, captioning, detection, and
segmentation across a large variety of input types. Our approach achieves competitive results across a broad set of
benchmarks, including high-resolution, multi-temporal, multi-modal and multi-view tasks. Further, as training data scales, our experiments show consistent improvements across most tasks both in and out of distribution, which correlate with per-task data diversity. These findings suggest that, for remote sensing VLMs, data scale is sufficient even without architectural novelty.
Read the full abstract ▾
All authors Stefan Maria Ailuro, Mario Markov, Mohammad Mahdi, Luc Van Gool, Danda Pani Paudel
Computer Vision
Handwritten and Printed Text Segmentation via Region-Aware Human-Writing Descriptor Engineering
Zhixian Lu, Jianwei Zhang, Lei Zhang et al. (+5 more)
With the increasing demand for reusing paper documents in educational and office settings, accurate
segmentation of handwritten and printed text has become a crucial step in document digitization. Although numerous deep learning models have been developed for this task, their high computational cost limits deployment on resource-constrained edge devices. To address this challenge, we present a lightweight framework optimized for efficient performance on devices with severely limited computational capacity. Our approach begins with the Sentence-level Connected Component
Segmentation algorithm, aimed at extracting coherent sentence-level segments from document images. We then design a novel Region-aware Handwriting Descriptor (RHD) to capture the intrinsic variability of human handwriting at the sentence level. A simple conventional classifier can then be seamlessly integrated with our designed descriptor, demonstrating strong classification performance for distinguishing handwritten and printed sentence-level text images, highlighting that the proposed descriptor is agnostic to the choice of classifier. Extensive experiments are performed on our self-constructed
Multilingual High-Quality Annotated
Dataset for Handwritten and Printed Text
Segmentation (MAD-HPTS) and a public
benchmark PHD-AS, and the experimental results demonstrate that the proposed framework outperforms current state-of-the-art methods in both accuracy and computational efficiency. On MAD-HPTS, our method sacrifices only 1.4% accuracy compared to the leading deep neural network baseline, yet achieves more than 8 times speedup in inference, making it well-suited for lightweight deployment.
Read the full abstract ▾
All authors Zhixian Lu, Jianwei Zhang, Lei Zhang, Fei Yuan, Jin Wang, Chang Liu, Rui Gao, Qiyu Lei
Computer Vision
Distributional Matching for Vector Quantization: A Unified Theoretical and Empirical Framework
Xianghong Fang, Litao Guo, Hengchao Chen et al. (+8 more)
The effectiveness of modern visual representation learning and autoregressive models critically depends on vector quantization (VQ), which discretizes continuous feature representations using a learnable codebook. Despite its widespread use, existing VQ methods often suffer from training instability and codebook collapse, arising from gradient mismatch induced by the straight-through
estimator and the under-utilization of code vectors. In this work, we show that both issues can be traced to a fundamental mismatch between the distributions of feature vectors and code vectors, leading to inefficient representation and information loss. Building on this observation, we propose a distributional matching framework for vector quantization. We introduce principled criteria for desirable VQ behavior and demonstrate through theoretical analysis and empirical evaluation that aligning feature and code vector distributions provides a unifying mechanism for mitigating training instability and codebook collapse. We instantiate this framework using a Wasserstein-based objective with an efficient closed-form under a mild Gaussian approximation, and further show that a nonparametric alternative based on maximum mean discrepancy yields comparable performance. Extensive experiments on visual tokenization
benchmarks support the effectiveness and robustness of the proposed approach.
Read the full abstract ▾
All authors Xianghong Fang, Litao Guo, Hengchao Chen, Yuxuan Zhang, XiaofanXia, Dingjie Song, Yexin Liu, Hao Wang, Harry Yang, Qiang Sun, Yuan Yuan
Computer Vision
HETA++: Global Structure-from-Motion with Hybrid Explicit Translation Averaging
Peilin Tao, Hainan Cui, Mengqi Rong et al. (+1 more)
Global Structure-from-Motion (SfM) offers advantages over incremental methods in terms of efficiency and error distribution. However, the task of translation averaging remains challenging. Many existing methods rely solely on relative translations or feature tracks, which either degrade under collinear camera motion or are susceptible to outliers. In this paper, we propose a novel hybrid explicit translation averaging framework that incorporates both relative translations and feature tracks. Specifically, we first refine the relative translations using global camera rotations and remove globally inconsistent relative translations. Next, we employ convex distance-based objective functions to estimate the initial camera positions and 3D points, followed by refinement using a non-bilinear angle-based objective function. Furthermore, since camera rotations are fixed during translation averaging, inaccurate camera rotations can severely limit the accuracy of camera positions. To address this issue, we then robustly refine both camera rotations and camera positions with selected feature tracks through bounded angle-based refinement and subsequent reprojection-based bundle adjustment. In this step, feature tracks are selected to maintain a balanced spatial distribution and improve optimization efficiency. Finally, we perform a complete bundle adjustment using all reliable feature tracks to refine the camera parameters and 3D points. Extensive experiments on various sequential and unordered real-world
datasets demonstrate the superior accuracy, robustness, and scalability of our approach, outperforming state-of-the-art methods in both accuracy and computational efficiency.
Read the full abstract ▾
All authors Peilin Tao, Hainan Cui, Mengqi Rong, Shuhan Shen
Computer Vision
Orbis 2: A Hierarchical World Model for Driving
Sudhanshu Mittal, Arian Mousakhan, Silvio Galesso et al. (+4 more)
Current world models operate at a single level of abstraction, with most prioritizing perceptual fidelity while lacking the spatial
reasoning and semantic understanding required for real-world downstream tasks. We present a hierarchical driving world model that factorizes future prediction across two levels operating at distinct temporal and abstraction scales: a high-level predictor that forecasts coarse scene structure over extended temporal horizons, and a low-level generator that produces detailed predictions conditioned on the high-level output. This decomposition yields high perceptual fidelity while also capturing strong spatial and semantic representations. We further show that pretraining with a
diffusion forcing objective yields substantially richer internal representations than the standard teacher forcing objective, while teacher forcing – predicting only the next frame from clean context – produces more stable autoregressive rollouts. We therefore introduce a generic two-stage training paradigm that pretrains the model with
diffusion forcing and fine-tunes with teacher forcing, combining the representational benefits of the former with the rollout stability of the latter. Our approach achieves state-of-the-art results across the standard suite of driving world model evaluations on established
benchmarks, including long-horizon generation fidelity, steering responsiveness evaluated on counterfactual scenarios, and internal representation quality. Project page with code, demo, checkpoints and qualitative results: https://lmb-freiburg.github.io/orbis2.github.io/
Read the full abstract ▾
All authors Sudhanshu Mittal, Arian Mousakhan, Silvio Galesso, Karim Farid, Jonannes Dienert, Rajat Sahay, Thomas Brox
Computer Vision
Exo2EgoPose: Leveraging Exocentric Demonstrations for Vision-Language guided Egocentric 3D Hand Pose Forecasting
Zhaofeng Shi, Heqian Qiu, Lanxiao Wang et al. (+2 more)
Perceiving
multimodal cues and
forecasting fine-grained actions from an
egocentric (Ego) perspective is vital for applications like
robot manipulation. However, previous studies either rely mainly on under-informed visual inputs to predict coarse human motions or follow the VRM/VLA paradigm, which suffers from insufficient
robot data and the gap between human and
robot embodiments. We observe that 3D hand pose naturally serves as a unified representation to bridge human-
robot actions. Hence, we investigate an under-explored
Vision-
Language guided
Egocentric 3D Hand Pose
Forecasting (VL-EHPF) task, which aims to predict future Ego 3D hand poses from visual observations, a
language instruction, and pose states. To overcome the limited field-of-view and highly dynamic motions in the Ego view, we propose a framework dubbed Exo2EgoPose, which innovatively leverages holistic and stable exocentric (Exo) demonstrations as guidance to compensate for partial and dynamic Ego-view cues. Specifically, we introduce a Dual-level Exocentric Reconstruction Module (DERM), which incorporates the paired Exo
videos as supervision to reconstruct their
video-level and chunked frame-level representations, thereby modeling spatial contexts and temporal dynamics. Then, the Global-to-Local Modulation Module (GLMM) utilizes the reconstructed hierarchical Exo representations for progressive feature refinement via
attention mechanisms and adaptive modulation, enabling comprehensive Exo guidance for accurate Ego hand pose
forecasting. Extensive experiments on AssemblyHands, Ego-Exo4D, and our newly constructed EgoMe-pose
benchmarks show the superiority of our method, which outperforms state-of-the-art methods by a large margin. Moreover, it demonstrates an effective human-to-
robot transfer capability and yields improvements on the CALVIN
dataset.
Read the full abstract ▾
All authors Zhaofeng Shi, Heqian Qiu, Lanxiao Wang, Xiang Li, Hongliang Li
Computer Vision
Hardware-triggered Time Synchronization of Roadside Multi-lidar, Multi-camera Measurement System for Accurate Data Alignment
Shiva Agrawal, Savankumar Bhanderi, Zhiran Yan et al. (+1 more)
Accurate temporal
alignment of heterogeneous sensors is necessary for reliable environment perception in roadside multi-
lidar, multi-camera systems, particularly in dense urban traffic. For this purpose, an open-source, simple, modular, and configurable hardware-triggered time-synchronization
circuit is presented in this work to perform temporal
alignment or accurate time synchronization between a
lidar and multiple cameras. In the designed
circuit, a
lidar synchronization pulse is used as a reference input, and independently programmable, time-delayed trigger pulses are generated for each camera, allowing flexible adaptation to varying sensor setups and mounting geometries. A series of experiments is conducted on a roadside-mounted perception system comprised of
lidar and three cameras, in which the trigger delay is systematically varied, and its impact on spatial-temporal
alignment is evaluated. For different classes of road users, the overlap between
lidar point cloud measurements and camera measurements is quantified to identify delay configurations that maximize cross-sensor consistency. The proposed
circuit is shown to achieve robust and repeatable synchronization while remaining straightforward to deploy, reconfigure, and extend due to its simple and open-source design. Following validation on a three-camera roadside system, the
circuit is extended to a vehicle platform with seven cameras and a
lidar, providing a low-cost, extensible solution for multi-sensor synchronization across infrastructure and vehicle setups. All hardware
circuit design files and source codes are available at https://github.com/shiva-THI/hardware-trigger-time-sync-
lidar-cameras.
Read the full abstract ▾
All authors Shiva Agrawal, Savankumar Bhanderi, Zhiran Yan, Gordon Elger
Computer Vision
MDND: Unsupervised Learning Guided by Non-Differentiable Refinement for Shape Correspondence
Qinsong Li, Jing Meng, Haibo Wang et al. (+1 more)
Deep functional map frameworks (DFM) for shape correspondence are powerful, yet fundamentally limited by their reliance on end-to-end differentiability. This constraint prevents the integration of highly accurate, non-differentiable refinement techniques, capping their overall performance, especially on challenging non-isometric shapes. To overcome this, we introduce MDND, a novel DFM paradigm built on the principle of merging differentiable and non-differentiable components. Our framework facilitates unsupervised learning guided by an internal, non-differentiable refinement. Specifically, MDND employs a dual-branch architecture: a non-differentiable refinement branch leverages a novel, multiscale iterative solver to produce highly robust correspondences, acting as a refined target. Concurrently, a fully differentiable branch learns to predict correspondences from features. The entire system is trained end-to-end without supervision by enforcing a consistency loss that compels the differentiable branch to learn from the superior, refined results of the non-differentiable branch. Extensive experiments show that MDND sets a new state-of-the-art, demonstrating remarkable robustness on shapes with non-isometric deformations and topological noise.
Read the full abstract ▾
All authors Qinsong Li, Jing Meng, Haibo Wang, Shengjun Liu
Computer Vision
EgoExoMoCap: Distributed Ego-Exo Human Motion Capture
Jiaxi Jiang, Bharat Lal Bhatnagar, Nan Yang et al. (+6 more)
Human motion capture from head-mounted devices (HMDs) offers a scalable way to acquire real-world human motion and interaction data, which is crucial for applications in
embodied AI and VR/AR. Existing approaches focus on either
egocentric body tracking, estimating the motion of the subject wearing the device, or exocentric tracking, capturing the movements of people in the wearer's surroundings. So far, these two paradigms have largely been explored in isolation. In this paper, we propose a novel distributed framework that jointly leverages ego- and exocentric multi-modal signals for human motion estimation from HMDs. Unlike traditional motion capture systems requiring bulky multi-camera setups or obtrusive mocap suits, our approach, EgoExoMoCap, is as simple as two (or more) people, each wearing a pair of smart glasses. The method leverages head (plus potentially wrist) tracking signals for accurate estimation of global motion in the 3D world and combines context-aware image features based on DINOv3 to achieve robustness in the presence of noise and occlusions. Extensive experiments on two in-the-wild
datasets show that our approach can robustly reconstruct motion even in challenging scenarios.
Read the full abstract ▾
All authors Jiaxi Jiang, Bharat Lal Bhatnagar, Nan Yang, Lingni Ma, Sebastian Starke, Robin Kips, Nadine Bertsch, Christian Holz, Federica Bogo
Computer Vision
Von Mises-Fisher Mixture Model with Dynamic Shrinkage for Realistic Test-Time Transduction
Jiazhen Huang, Zhiming Liu, Changhu Wang et al. (+3 more)
A range of methods aim to enhance the performance of
vision-
language models (VLMs) at test time. Among them, transduction has emerged as a promising paradigm due to its strong compatibility and efficiency. However, realistic evaluations often involve highly imbalanced class distributions, which cause performance degradation or even collapse. In this work, we systematically revisit transduction from the perspective of penalized likelihood estimation (PLE), showing that PLE with a KL-divergence anchor term naturally yields an adaptive shrinkage behavior between prior anchors and empirical estimates. From this viewpoint, the brittleness of transductive methods can be attributed to the absence of anchoring mechanism and static modeling of the shrinkage strength. Therefore, we propose Mixture of Von Mises-Fisher Models with Dynamic Shrinkage (MOON). MOON is built upon a mixture of von Mises-Fisher distributions to model feature representations on the unit hypersphere. To handle imbalance, MOON dynamically adjusts the shrinkage strength using zero-shot priors at both instance and class levels. Thus, it suppresses unreliable assignments and prevents harmful updates from outlier classes, thereby mitigating negative transfer. MOON is model-agnostic, training-free, and requires no task-specific hyperparameter tuning. Extensive experiments further validate the advantage of MOON in both performance and efficiency. Our code is available at https://github.com/walawalagoose/MOON
Read the full abstract ▾
All authors Jiazhen Huang, Zhiming Liu, Changhu Wang, Wei Ju, Ziyue Qiao, Xiao Luo
Computer Vision
Test-Time Noise Guided Adaptation for Realistic Autoregressive Video Generation
Dimitrios Karageorgiou, Symeon Papadopoulos, Ioannis Kompatsiaris et al. (+1 more)
Autoregressive
video diffusion models have enabled the generation of arbitrarily long
videos by removing conditioning on future frames, thus greatly improving computational efficiency. Yet, they suffer from error accumulation over time, as the denoised sequence gradually drifts away from the conditioning distribution seen during training. Recent advances attempt to reduce this error by anchoring each generated frame to the learned
manifold of real ones. However, even when all generated individual frames lie close to the real
manifold, there are trajectories which the model lacks sufficient knowledge to continue without exiting it, thus reaching a terminal point. To prevent the model from being trapped in terminal points, we start from the hypothesis that for well-modeled future trajectories the distribution of the predicted noise should match the one of the forward noising process. To enforce such a prior at test time, we introduce Terminal points Avoidance through Noise Guided Optimization (TANGO), which uses the
diffusion model as a critic of its own outputs, by predicting one step forward and requiring an isotropic Gaussian noise prediction. We use the deviation from this expected noise distribution to search for an alternative trajectory that does not lead to a terminal point. Our approach achieves a $3.1\%$ absolute improvement on VBench over state-of-the-art, while reducing Fréchet
Video Distance by $28.3\%$ on average across $15$s
videos. Our code is available on https://mever-team.github.io/tango.
Read the full abstract ▾
All authors Dimitrios Karageorgiou, Symeon Papadopoulos, Ioannis Kompatsiaris, Efstratios Gavves
Computer Vision · Cross-listed from cs.RO
Beyond Frontiers: Scene-Anomaly Guided Autonomous Exploration
Akash Kumbar, Abhinav Raundhal and Madhava Krishna
Autonomous exploration of unknown 3D environments is traditionally driven by coverage-maximizing
geometric heuristics. However, these methods typically determine exploration targets without considering the underlying structural context. This leads to inefficient trajectories often limiting the fidelity of the final 3D reconstruction. To bridge the gap between spatial coverage and reconstruction quality, we introduce a novel paradigm: reframing exploration as a
geometric anomaly minimization problem. We present SCAGE: SCene Anomaly Guided Exploration, a novel autonomous exploration framework that operates directly on unstructured 3D point clouds. Instead of blindly chasing volumetric boundaries, we equip the
robot with a foundational understanding of standard indoor architecture. As the
robot navigates, it continuously evaluates its live 3D observations against these learned expectations. When the incoming geometry contradicts the learned priors of a typical indoor environment, such as a fragmented wall or a partial table, the system flags these regions as scene anomalies. These
geometric inconsistencies act as a guiding signal, naturally drawing the
robot to investigate and resolve these structural anomalies from optimal vantage points. By actively targeting poorly reconstructed regions rather than just empty space, our approach seamlessly couples spatial discovery with high-fidelity mapping. Extensive evaluations demonstrate that SCAGE achieves superior volumetric coverage (~90% in all scenes) and higher 3D reconstruction quality compared to state-of-the-art baselines.
Read the full abstract ▾
Computer Vision
Examining the Associations between Visual and Non-Visual Elements and Cyclists' Route Choices for Various Trip Purposes
Heyang Hua, Koichi Ito and Filip Biljecki
Understanding cyclist preferences for the characteristics of the built environment is important in promoting sustainable urban transportation and active mobility. Despite previous studies on cyclists' route choices, the influence of visual and non-visual factors on these choices for different trip purposes remains unclear; thus, this paper fills this gap through a data-driven case study in Montreal, Canada. Non-visual factors include socioeconomic factors and two-dimensional environments, while visual factors involve visual perception during cycling and are computed using street view images. The study consists of two parts: one part analyzes spatiotemporal information to explore the non-visual factors between the start and end points of cycling trips, and the other part investigates the discrepancies in distributions of these factors between the shortest path and the actual one. The findings reveal the spatiotemporal characteristics that influence active riding choices, such as increased greenery and lower levels of motorization. These insights can inform the planning of street networks and the development of infrastructure to improve the use of active transportation.
Read the full abstract ▾
Computer Vision
HybridSim: A Physics-Learning Hybrid Digital Twin for mmWave Human Sensing
Weitao Xiong, Tianyu Liu, Peng Li et al. (+4 more)
High-fidelity simulation of mmWave radar signals for dynamic human motion is valuable for developing radar-based human sensing models; yet collecting accurately labeled measurements for a specific deployment site remains expensive. We present HybridSim, a physics-learning hybrid simulator that synthesizes mmWave radar signals from dynamic human meshes under a fixed indoor room configuration, explicitly decoupling propagation into two components. To parameterize the human subject, we use a tri-plane representation to extract human features and a Graph Convolutional Network to stabilize optimization and mitigate gradient instability. The direct signal path is modeled via an inverse-rendering formulation with a microfacet BRDF to capture primary surface reflections. In parallel, the indirect path is approximated by combining 3D Gaussian Splatting with a virtual-receiver geometry to fit and reproduce site-specific multipath interference patterns, achieving substantially lower computational cost than explicit full ray tracing. Experiments in a fixed-room setting show improved agreement with a physically based reference and consistent gains on downstream radar-based human sensing tasks when using HybridSim for site-specific data augmentation.
Read the full abstract ▾
All authors Weitao Xiong, Tianyu Liu, Peng Li, Kok Chung Chua, Toa Chean Khim, Pu Wang, Hongfei Xue
Computer Vision
On the Geometry of Learned Representations in Event-Based Multi-Modal Egomotion Estimation
Stefano Silvestrini and Michele Ceresoli
Classical approaches to event-based egomotion estimation, including those adopted by the top-performing teams of the ELOPE challenge, rely on
geometric optimization frameworks such as contrast maximization, homography estimation, or dense optical flow combined with analytic motion inversion. This work investigates the
geometric structure that emerges inside a multi-modal network for egomotion estimation. Event tensors, inertial measurements, and range signals are fused through a cross-modal
attention architecture and trained in a batch setting. We analyze the latent space geometry and
attention dynamics, showing that (i)
embeddings lie on low-dimensional
manifolds aligned with motion variables, (ii)
attention weights adapt with angular excitation and visual reliability, and (iii) the fused representation recovers classical observability cues. These results bridge analytical estimation theory and modern data-driven fusion.
Read the full abstract ▾
Computer Vision
Multimodal Ambivalence and Hesitancy Recognition via Cross-Attention and Gated Fusion
Oussama Berhili, Yassine Ouzar and Larbi Boubchir
We present a
multimodal framework for Ambivalence/Hesitancy (A/H) recognition in
video, developed for the ABAW11 challenge at ECCV 2026. The proposed approach fuses textual, acoustic, and visual modalities extracted from the BAH
dataset using three pretrained encoders: F2LLM-v2-0.6B for transcripts (1024-d), WavLM-Large for audio (1024-d), and VideoMAE V2 for facial
video (768-d). We first establish comprehensive unimodal baselines using classical classifiers (MLP, Random Forest, GBDT), each optimized via Optuna, and obtain a best unimodal Macro F1 of 0.6659 on the test set using text features alone – substantially outperforming the zero-shot
Video-LLaVA baseline (Macro F1: 0.2827). Building on these baselines, we propose a
multimodal fusion architecture that combines bidirectional cross-
attention across all three modalities with a Gated
Multimodal Unit (GMU), with both architectural and optimization hyperparameters selected through a 50-trial Optuna search. This model achieves a Macro F1 of 0.7394 on the validation set, a relative improvement of 11.0% over the best unimodal baseline, confirming that explicit cross-modal interaction captures complementary cues that no single modality provides in isolation. Final predictions on the official, unlabeled private test set are generated using this model and submitted according to the challenge protocol. Code is publicly available at https://github.com/yassineouzar/IUSD_AH/
Read the full abstract ▾
Computer Vision
Modularized Dynamic-Granularity Video LLM for Multi-Event Long Video Understanding
Wei Feng, Xin Wang, Yu-Wei Zhan et al. (+2 more)
Video Large
Language Models (
Video LLMs) have made significant advancements in various
video understanding tasks. However, long-
video scenarios remain challenging due to the tension between limited visual
token budgets and the need to capture multiple key events. Existing approaches typically process long
videos in two stages, i.e., i) select keyframes and ii) perform detailed perception, which exhibit limitations: they lack a modular mechanism for adaptive capacity allocation and self-correction, resulting in unreliable modeling. To tackle these challenges, we propose MoD-VLLM, a novel Modularized Dynamic-Granularity
Video LLM framework for multi-event long
video understanding, which unifies temporal grounding and semantic understanding iteratively and self-reflectively. Specifically, we propose a Positive-Negative
Video Segments Grounding module and a Modularized Dynamic-Granularity Reflection module, which form a closed loop to progressively localize the question-related
video segments. The grounding module instructs a
Video LLM to distinguish relevant from irrelevant
video segments based on the
video question. The reflection module employs a modularized scheduler that dynamically selects fine-grained encoding for relevant positive segments to capture detailed perception and coarse-grained encoding for negative segments to maintain global context. We further propose a dynamic-granularity reinforcement learning strategy, allowing MoD-VLLM to learn optimal grounding policies and dynamic granularity visual representation jointly. Moreover, we propose MEventBench, a challenging Multi-Event Long
Video Benchmark for complex long
video reasoning. Extensive experiments on several long
video understanding
benchmarks and our MEventBench demonstrate that MoD-VLLM significantly outperforms state-of-the-art baselines.
Read the full abstract ▾
All authors Wei Feng, Xin Wang, Yu-Wei Zhan, Yuwei Zhou, Wenwu Zhu
Computer Vision
SlotMem: Character-Addressable Internal Memory for Narrative Long Video Generation
Yilai Liu, Xin Zhang, Shiyuan Zhang et al. (+1 more)
Maintaining recurring character identities across scene transitions and long temporal gaps is a central challenge in narrative long
video generation. Methods targeting global consistency often retrieve
memory using cues that are not aligned with character identity preservation, while recent character-centric variants still rely on coarse frame-level kv
memory that entangles identity with incidental visual factors and lacks a continuous update mechanism under limited
memory capacity. To address these limitations, we propose SlotMem, a character-addressable internal
memory framework for multi-character narrative long
video generation. Specifically, SlotMem uses a Character-Semantic Probe to localize character-relevant visual
tokens from cross-
attention responses, and a
Memory Encoder to compress DiT
tokens into compact role-wise slot
memory. As generation proceeds, a
Memory Writer conservatively updates each character's
memory with new observations, while Character-Wise Cross-
Attention retrieves the role
memory and injects it only into localized
tokens of the same character. Experiments on multiple narrative long
video generation
benchmarks show that SlotMem improves long-range character consistency over existing baselines, while maintaining comparable
video quality. Our code is available at https://github.com/YilaiLiu-HKU/SlotMem.
Read the full abstract ▾
All authors Yilai Liu, Xin Zhang, Shiyuan Zhang, Hongyang Du
Computer Vision
GeoChrono: Benchmarking and Rethinking Long-Term Temporal Understanding in Remote Sensing
Yujie Li, Jiancheng Pan, Zhiwei Wei et al. (+3 more)
Remote sensing offers an unparalleled vantage point for observing the Earth's long-term surface
evolution, yet it demands that a model not only perceive land cover at isolated moments, but also track changes, memorize
evolution histories, and reason across time and space. However, existing studies lack a systematic evaluation that dissects these distinct competencies. To fill this gap, we introduce ChronoBench, a multidimensional
benchmark that decomposes this task into four progressive cognitive levels (i.e., Land Cover Perception, Temporal Recognition, Long-Term
Memory, and Spatio-Temporal
Reasoning). The ChronoBench comprises 12 sub-tasks and 17,689 rigorously validated QA (Question-Answer) pairs. Extensive evaluations reveal that mainstream MLLMs fall drastically behind human experts, with Long-Term
Memory emerging as the most critical bottleneck. Motivated by this finding, we further propose GeoChrono, an MLLM with enhanced capabilities for tracing, memorizing, and
reasoning about long-term geographic
evolution. Leveraging the physical prior that geographic parcels remain spatially fixed while their semantics evolve, we design a Temporal Trajectory Encoder~(TempEnc) that constructs per-location temporal trajectories for dedicated land cover
evolution modeling, and we introduce a Coarse-to-Fine
Token Compressor~(C2FComp) that adaptively preserves dynamic regions while compressing the static background. To support training, we also construct ChronoInstruct, a 104K-sample instruction-tuning
dataset spanning all competency levels for training. GeoChrono achieves state-of-the-art performance on ChronoBench, surpassing the leading commercial MLLMs by over 20%, while C2FComp reduces visual
tokens by over 56% while retaining GeoChrono's 94.6% performance. The code and data will be available at https://github.com/IntelliSensing/GeoChrono
Read the full abstract ▾
All authors Yujie Li, Jiancheng Pan, Zhiwei Wei, Jiuniu Wang, Mugen Peng, Wenjia Xu
Computer Vision
Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging
Zhichao Yang, Tianjiao Gu, Zhixianhe Zhang et al. (+3 more)
Personalized Image Aesthetic Assessment (PIAA) aims to predict aesthetic ratings of images that vary across individuals. The aesthetic preferences manifest to different extents across distinct visual stimuli and exhibit cohort-specific patterns. Motivated by the above fact, this paper presents a
Multimodal Large
Language Model (MLLM)-based approach, which models individual aesthetic preferences by Preference-Rich sample mining and Aesthetically-resonant Cohort merging (PRAC). Specifically, PRAC first identifies preference-rich samples by analyzing both Collective Controversy and Personalized Deviation of images, maximizing the utility of limited user data. Based upon the preference-rich samples, cross-user preference similarities are measured by comparing preference
embeddings. Then, a cohort-based model merging strategy, is proposed by aggregating preference patterns from aesthetically-resonant users, which further enhances the personalization for the target individual. Extensive experiments and comparisons on four
benchmark PIAA databases demonstrate the superiority of the proposed PRAC model over the state-of-the-arts. The code and model will be public at https://github.com/yzc-ippl/PRAC.
Read the full abstract ▾
All authors Zhichao Yang, Tianjiao Gu, Zhixianhe Zhang, Xiangfei Sheng, Pengfei Chen, Leida Li
Computer Vision
Debiasing Text-to-Image Evaluation via Implicit Cultural Alignment Reward Modeling
Bo-An Chang and Yu-Chih Chen
As Text-to-Image (T2I) systems rapidly advance, evaluating the cultural authenticity of synthesized content has become increasingly important for fair and trustworthy generative AI. Existing T2I evaluation metrics and
multimodal judges often rely on visual-semantic representations that underrepresent implicit cultural norms, leading to biased preference judgments and the omission of fine-grained cultural cues. In addition, visual question answering (VQA)-based evaluators typically depend on autoregressive text generation, which limits their scalability for real-time reward modeling. To address these limitations, we introduce an Implicit Cultural
Alignment Reward Model built upon a lightweight 4.2-billion-parameter
Multimodal Large
Language Model (MLLM). Our framework integrates an Implicit Cultural Probe with a Skip-connection Cross-
Attention (SkipCA) mechanism, enabling late-stage semantic features to directly attend to early-stage visual representations and better preserve culturally salient details. Evaluations on 3,323 challenging and carefully curated image pairs from the CulturalFrames
benchmark show that our approach achieves 83.49% pairwise accuracy, with Pearson and Kendall correlation coefficients of 0.5268 and 0.3749, respectively, outperforming representative
vision-
language metrics and MLLM-based evaluators. Moreover, by bypassing autoregressive text generation, our model processes each evaluation in 0.21 seconds under our local inference setup, achieving a $10\times$ speedup over standard VQA-based evaluators. These results suggest that the proposed reward model can provide an efficient and culturally aware scalar signal for preference optimization pipelines such as Reinforcement Learning from Human Feedback and Direct Preference Optimization. Additional resources are available on our project page at https://bensonch1214.github.io/Implicit_Cultural_
Alignment/.
Read the full abstract ▾
Computer Vision
The Third Competition on Document Forgery Detection on ID-Cards and Passports
Juan E. Tapia, Mario Nieto, Juan M. Espin et al. (+4 more)
This paper presents a comprehensive analysis of the results from the Third International Competition on Document Forgery Detection on ID-Cards and Passports, which was held across two distinct tracks. Track 1 evaluates a synthetic-data-based ID-PAD system under controlled but diverse conditions, where the winning team, Incode, achieves an $AV_{Rank}$ of 27.82%, confirming consistent performance across metrics and highlighting the importance of a balanced, generalizable design. In Track 2, the challenge intensifies with heterogeneous attack scenarios across different domains, where Incode again achieved the top position with an $AV_{Rank}$ of 68.71% across thresholds, outperforming some baselines and established methods. These results demonstrate that PAD effectiveness requires not only high accuracy but also consistency across diverse attack types and imaging conditions. The success of this initiative across both tracks underscores the value of collaboration between companies and academic teams. This year, more than 63 teams were registered, and more than 100 submission models were evaluated. This competition has evolved into a leading
benchmark state-of-the-art in PAD on ID documents, setting the standard for performance, reproducibility, and real-world applicability in secure identity verification.
Read the full abstract ▾
All authors Juan E. Tapia, Mario Nieto, Juan M. Espin, Álvaro S. Rocamora, Javier Barrachina, Naser Damer, Christoph Busch
Computer Vision
IoU-PD: IoU-Aware Privileged Distillation for Visual Grounding with Multimodal Large Language Models
Xiuyuan Zhu, Ke Lu, Hao Wu et al. (+4 more)
Visual grounding with
multimodal large
language models is commonly formulated as autoregressive coordinate generation, where a model outputs bounding-box coordinates as text given an image and a referring-expression
prompt. While this interface is simple and compatible with instruction following, it introduces a mismatch between training and evaluation: training optimizes
token-level likelihood over coordinate strings, whereas grounding quality is measured by
geometric overlap. We propose IoU-PD, an IoU-aware privileged
distillation method for coordinate-generating
multimodal large
language models. IoU-PD uses ground-truth boxes not only as coordinate targets, but also as privileged training-time guidance. During training, the student receives the original image and
prompt, while a frozen teacher receives a box-marked image and an augmented
prompt that indicates the marked region. The student is trained with a supervised fine-tuning anchor and a privileged
distillation loss whose
token weights reflect both
geometric importance and teacher reliability. At inference time, IoU-PD requires no box overlay, privileged hint, teacher branch, or additional prediction module. Experiments on standard referring-expression grounding
benchmarks show consistent region-level improvements over strong coordinate-generating baselines, demonstrating that ground-truth boxes can provide useful privileged guidance beyond serving as coordinate labels. Project page: https://xyzzzh.github.io/IoU-PD/
Read the full abstract ▾
All authors Xiuyuan Zhu, Ke Lu, Hao Wu, Siwen Jiao, Zijin Du, Dongming Zhang, Jian Xue
Computer Vision
Event3R: Asynchronous-to-Global 3D Reconstruction from Event Camera via Spatial-Temporal Feature Aggregation
Jian Huang, Haotian Shen, Xinhao Lou et al. (+3 more)
Robust 3D reconstruction is essential for robotics and
embodied perception. Recent feed-forward approaches such as
DUSt3R have demonstrated impressive progress in dense 3D reconstruction from RGB images, achieving global
geometric consistency and strong generalization. However, extending such dense 3D reconstruction to event cameras remains challenging due to their asynchronous, sparse, and highly dynamic nature, as well as the lack of large-scale, well-labeled
datasets. In this work, we introduce Event3R, a feed-forward framework that directly maps asynchronous event streams to globally consistent 3D point clouds. Event3R represents incoming events as spatial-temporal voxels, enabling time-aware feature integration through a temporal
attention module that enhances the module's temporal feature learning. To further strengthen temporal representation learning and reduce reliance on labeled data, we propose a Masked Bin Modeling (MBM) strategy for self-supervised pre-training, enabling robust temporal representation learning with minimal labeled data, and retain it as an auxiliary fine-tuning objective. In addition, contrastive
alignment and consistency regularization losses are incorporated during fine-tuning to reinforce structural correspondence and temporal coherence across views. Extensive experiments on both synthetic and real-world
benchmarks demonstrate that Event3R achieves robust, temporally consistent, and globally aligned 3D reconstructions, significantly outperforming existing event-based methods.
Read the full abstract ▾
All authors Jian Huang, Haotian Shen, Xinhao Lou, Chengrui Dong, Wenpu Li, Peidong Liu
Computer Vision
Per-Stroke Temporal Control for Text-to-Motion via Action Units and Action-Detection Guidance
Euijun Jung and Youngki Lee
Text-to-motion models are competent at the action a
prompt names but unreliable at when each stroke lands: four punches alternating left and right rarely return four separable strokes. We introduce typed temporal events called Action Units (AUs) that make the individual stroke – its body track, action class, time window, and impact timing – an explicit conditioning signal. We ground a frozen text-to-motion backbone on the AU set through a lightweight gated adapter injecting two streams (per-stroke
tokens and a per-frame phase channel), and at inference close residual timing errors with a training-free classifier gradient from a frozen frame-level detector. We measure per-stroke control on StrokeBench, whose
prompts specify count, ordering, track, and core-frame placement, paired with an audited stroke corpus. AU grounding markedly raises the rate of correctly placed single strokes over the strongest prior interface, at the best motion quality among text-, interval-, and frame-level baselines. The prompted core frame emerges as a further steerable axis.
Read the full abstract ▾
Computer Vision
Efficient Difficulty-Aware Dynamic Routing for Diffusion-Based Real-World Image Super-Resolution
Xue Wu, Kang Zhao, Kafeng Wang et al. (+4 more)
Diffusion-based methods have achieved impressive performance in real-world image super-resolution (Real-ISR) by leveraging large pre-trained stable
diffusion (SD) models as powerful generative priors. However, these methods still face two key limitations. First, existing SD-based one-step and multi-step Real-ISR approaches adopt a unified processing paradigm for all input samples, ignoring the varying restoration difficulty across images. Second, the aggressive resolution reduction of the VAE in SD models (e.g., 8x downsampling) leads to irreversible loss of fine-scale details, which cannot be recovered by the subsequent
diffusion process. To address these limitations, we propose a Difficulty-aware Dynamic Routing (DDR) strategy that overcomes the rigid, one-size-fits-all processing paradigm. Specifically, we first design a difficulty
estimator to predict the restoration cost of each input image, enabling automatic assignment to a network of appropriate capacity. Then, we construct a set of Real-ISR networks with varying model capacities by modulating the spatial downsampling ratio of the VAE in the SD backbone, thereby preserving more high-frequency information for challenging cases while maintaining efficiency for simpler inputs. Extensive experiments have demonstrated the superior efficiency and effectiveness of the proposed model compared to recent state-of-the-art methods.
Read the full abstract ▾
All authors Xue Wu, Kang Zhao, Kafeng Wang, Jianfei Chen, Jingwei Xin, Nannan Wang, Xinbo Gao
Computer Vision
GoStop: Reinforcement Learning for Adaptive Temporal Aggregation in Event-Based Feature Tracking
Youngho Kim, Hoonhee Cho, Jae-Young Kang et al. (+1 more)
Feature tracking plays a fundamental role in understanding scene motion and supports various downstream tasks. Event cameras, with their high temporal resolution and asynchronous sensing, enable low-latency and motion-robust perception, making them well-suited for feature tracking under fast and non-linear motion. However, existing event-based feature tracking methods rely on fixed heuristic rules based on hand-tuning for event accumulation. Such strategies fail to adapt to diverse motion dynamics, leading to degraded performance under abrupt motion changes or low-motion scenarios. In this paper, we model event accumulation as a sequential decision-making problem and introduce reinforcement learning (RL) framework to adaptively control the accumulation process for online event-based feature tracking. Our approach trains a RL
agent that decides whether to continue accumulating events or to perform tracking inference based on motion cues. The proposed adaptive temporal
agent enables dynamic adaptation to varying motion patterns without relying on hand-crafted rules. Furthermore, we introduce a Dynamic Event-based Tracking (DEFT)
dataset with dynamic motion distributions to evaluate the robustness of the feature tracking. Extensive experiments demonstrate that integrating our plug-and-play framework to existing feature tracking methods consistently outperforms heuristic-based approaches, improving robustness under dynamic motion while offering a better balance between tracking accuracy and efficiency. Our project codes and
datasets are available at https://github.com/kmax2001/GoSTOP
Read the full abstract ▾
All authors Youngho Kim, Hoonhee Cho, Jae-Young Kang, Kuk-Jin Yoon
Computer Vision
Hierarchical Specialised Ensembles for Classification of Zebrafish Phenotypes Using the Selected Image Recognition Methods
Piotr S. Maciąg, Monika Maciąg and Magdalena Majdan
We propose and evaluate three hierarchical ensemble setups for zebrafish phenotype classification from embryo images. In all setups, stage 1 uses a single four-class classifier to assign images to one of the exclusive phenotypes: Normal, Chorion, Dead, or Other. Images classified as Other are then processed in stage 2, where the ensemble design differs across setups: a single multi-label classifier, two specialized multi-label classifiers, or an ensemble of binary classifiers. We compare these setups using three backbone architectures: ResNet18, ViT, and ConvNeXt. Overall, ConvNeXt achieves the best performance across setups, while the specialized hierarchical ensemble in setup 2 provides the best balance in terms of F1-score. The results show that the proposed specialised hierarchical ensembles are effective for zebrafish phenotype recognition, and suggest that ConvNeXt is particularly useful backbone model.
Read the full abstract ▾
Computer Vision
Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectors
Yilin Wang, Xiangxi Zheng, Dongxing Mao et al. (+6 more)
Understanding long
videos with
multimodal large
language models (MLLMs) requires selecting a compact set of frames from thousands of candidates, yet identifying the right frames seemingly requires understanding the
video first. We resolve this circular dependency with a simple observation: cross-modal
attention at validation-selected extraction layers in MLLMs already provides query-relevant frame evidence without requiring autoregressive generation. We exploit this property to build DAFS (Dynamic
Attention-based Budget-aware Frame Selection), a training-free frame selector. A lightweight MLLM selector, even with only 2B parameters, can extract frame-level evidence by converting selected-layer
attention into relevance scores through query-conditioned aggregation. This enables cross-frame comparison without autoregressive decoding. To handle the selector's own context constraint, we formulate the joint allocation of candidate pool size and per-frame
token budget as a discrete optimization problem solved by dynamic programming. Under a 32-frame budget, our selector improves over uniform sampling by up to 6.4 points on
Video-MME and outperforms prior training-based selectors under matched frame budgets, while generalizing across selector and answerer backbones, and across tasks, without retraining.
Read the full abstract ▾
All authors Yilin Wang, Xiangxi Zheng, Dongxing Mao, Linjie Li, Zhengyuan Yang, Ping Yu, Rui Yan, Yuan Yao, Alex Jinpeng Wang
Computer Vision
PE-Field 4D: Video Generation Models as Canvas
Yunpeng Bai, Haoxiang Li and Qixing Huang
Diffusion Transformers have recently achieved strong performance in
video generation, yet controlling scene geometry under viewpoint changes and camera motion remains challenging. In this work, we revisit the role of positional encoding in
video diffusion transformers and show that it provides a useful spatial bias for geometry-aware control. Specifically, if reference
tokens are encoded according to their projected locations in the target view, the denoising model is encouraged to retrieve content from position aligned regions of the input
video. Building on this observation, we introduce a geometry-aware cross-
attention mechanism that enables target
video latent
tokens to attend to structured context
tokens derived from reference images or frames. To establish correspondence between the reference content and the target camera trajectory, we equip the context
tokens with a projected positional encoding scheme that combines target-view 2D reprojection with depth-aware disambiguation. At the same time, we preserve the original spatiotemporal positional encoding of the generated
video latent, allowing
geometric guidance to be injected while maintaining consistency with the
video model's native latent structure. The resulting framework provides a simple and effective approach for controllable
video generation. It improves spatial controllability in viewpoint-dependent editing tasks, including camera re-trajectory, novel-view
video synthesis, and geometry-aware
video editing, while preserving the generative prior of the underlying
video diffusion model. The code is available at: https://github.com/MTLab/PE-Field.
Read the full abstract ▾