Computer Vision
VisualRouter: Query-Grounded Visual Sampling for Long Video Understanding
Haiyue Zhang, Yi Bin, Xun Jiang et al. (+5 more)
Large
vision-
language models (LVLMs) have achieved significant progress in
video understanding, yet understanding long
videos remains challenging due to the large number of visual
tokens and limited context windows. Visual sampling provides a practical solution by selecting an informative subset of frames. However, existing methods typically either rely on relevance-aware sampling, leading to redundant frame selection and insufficient temporal coverage, or adopt a fixed sampling strategy regardless of query type. In this paper, we propose VisualRouter, a training-free and plug-and-play framework for query-grounded visual sampling. VisualRouter first classifies each query as either global or local and then applies the corresponding sampling strategy. For global queries, it employs a relevance-coverage hybrid strategy that preserves temporal coverage while retaining query-relevant visual evidence. For local queries, it adopts an event-aware frame selection strategy that performs event partitioning, segment-level frame allocation, and intra-event frame selection, jointly balancing relevance, coverage, and diversity with a limited number of input frames. Experiments show that VisualRouter consistently improves multiple LVLMs over uniform sampling, achieving gains of 5.2%, 7.7%, and 11.6% on
Video-MME, LongVideoBench, and MLVU with Qwen2.5-VL-7B, and outperforming existing training-free visual sampling methods under the same setting.
Read the full abstract ▾
All authors Haiyue Zhang, Yi Bin, Xun Jiang, Zeyu Ma, Duo Peng, Guoqing Wang, Yang Yang, Heng Tao Shen
Computer Vision
ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA
Ping-Kun Chiang, Kun-Ru Wu, Po-han Li et al. (+3 more)
Recent advances in large
language models (LLMs) and
vision-
language models (VLMs) have enabled new possibilities for 3D question answering (3D-QA), a key capability for
embodied AI and robotic perception. However, most existing methods rely on 3D-specific training or fine-tuning with costly annotations, limiting their scalability and real-world applicability. We present ViewMind3D, a fully training-free and modular framework for 3D spatial
reasoning over multi-view observations of a scene without requiring complete 3D reconstruction. The framework decomposes the 3D-QA task into four interpretable components: (1) question-driven multi-view selection, (2) guided visual grounding with
language-conditioned object cues, (3) spatial context encoding via a bird's-eye-view (BEV) viewpoint indicator, and (4) structured answer generation through role-based
reasoning. This design enables structured, robust, and interpretable
reasoning without requiring model tuning. Experimental results on ScanQA and SQA3D show that ViewMind3D achieves competitive performance compared to prior training-free and fine-tuned 3D-LLMs. In particular, our method improves performance on spatially grounded question types, such as “What” questions in SQA3D, while maintaining strong overall accuracy (50.8%) and achieving 73.4 CIDEr on ScanQA. These results demonstrate that effective 3D
reasoning can be achieved through modular orchestration of general-purpose LLMs and VLMs for robotic perception in real-world environments.
Read the full abstract ▾
All authors Ping-Kun Chiang, Kun-Ru Wu, Po-han Li, Sandeep Chinchali, Ufuk Topcu, Yu-Chee Tseng
Computer Vision · Cross-listed from cs.LG
Kohn-Sham Spectral Embedding on Sparse Graphs at the Nishimori Temperature for Image Classification
V.S. Usatyuk, D. A. Sapozhnikov and S. I. Egorov
We propose Kohn-Sham Spectral
Embedding (KSSE), an energy-based model replacing the top-layer classifier of convolutional networks with a sparse-graph spectral
embedding at the Nishimori temperature of an associated Random-Bond Ising Model the spectral detectability threshold where class structure becomes marginally distinguishable from disorder. Mapping pre-trained features onto quasi-cyclic low-density parity-check graphs, we construct a regularized
Laplacian (Bethe-Hessian) as an effective Kohn-Sham
Hamiltonian, yielding D independent spectral problems-one per feature channel-solvable in $O(N log N + k_{mode}^{2} N)$ time by FFT on circulant blocks (Pontryagin self-duality), with low-mode Rayleigh-Ritz refinement ($k_{mode}=5$). Physically, this is a k.p effective-mass reduction on a one-dimensional ring
crystal: the circulant support is the perfect
crystal, the data weights a slowly varying
impurity potential, and the Nishimori crossing a
Fermi level at the band edge.
Star-domain surgery optimizes the graph: instead of eliminating all frustrated cycles impossible without destroying the codewords-edge shifts create certified convexity around codewords with bounded residual frustration, with multi-scale fractal certification (basins $D_{2}<1$ vs rough landscapes $D_{2}>3$). The theory includes a generalized Ihara-Bass identity with a sharp spectral threshold, a non-backtracking growth trichotomy with frustration as a gauge-invariant $Z_{2}$ flux, a trapping-set spectral test, exact channel separability with a cup-product obstruction, plus loop-series, convexity, surgery, and quasi-stationarity bounds. On ImageNet-1000 with frozen EfficientNet-B4 features (D=1792) under a transductive protocol, KSSE achieves 88.93% Top-1 accuracy with ~21.24M parameters-beating Swin-L (197M, 86.4-87.3%) and matching the lower end of ViT-H/14 (632M, 88.0-89.5%) with 10x and 30x fewer parameters.
Read the full abstract ▾
Computer Vision
Negative controls reveal volume-driven confounding in radiomics and imaging foundation model features
Katy L. Scott, Sejin Kim, Joshua Siraj et al. (+6 more)
Radiomics and imaging foundation models promise non-invasive biomarkers of tumour biology, yet predictive signatures may reflect tumour volume or acquisition artifacts rather than meaningful image structure. We introduce READII-2-ROQC, an open-source framework that uses volume-preserving negative controls to assess whether radiomic and deep imaging features capture independent spatial signals. READII-2-ROQC generates voxel-perturbed images across tumour, background and whole-image regions using configurable randomization strategies, then compares feature behaviour and model performance between original and control images. Applied to three public cancer imaging cohorts, the framework processed 3,552 tumour volumes and extracted PyRadiomics and foundation-model features from original images and nine matched controls. Reproducing published survival and HPV-status signatures, we show that multiple models retain performance after spatial structure is destroyed, revealing volume-driven or contextual confounding, whereas others show perturbation-sensitive signal. READII-2-ROQC provides a scalable quality-control strategy for developing interpretable, biologically grounded imaging biomarkers and reproducible radiomics workflows.
Read the full abstract ▾
All authors Katy L. Scott, Sejin Kim, Joshua Siraj, Caryn Geady, Matthew Boccalon, Mattea Welch, Mogtaba Alim, Andrew J. Hope, Benjamin Haibe-Kains
Computer Vision · Cross-listed from cs.LG
QQWorld: Quantile-Quantile Matching for World Model Regularization
Zhoushun Yu, Xiaoyu Hu and Xiangyu Xu
Latent world models enable efficient planning by predicting future states in a compact representation space, but their performance depends critically on the quality of the learned latent distribution. LeWorldModel (LeWM) regularizes its latents toward an isotropic Gaussian using the Epps-Pulley (EP) objective. We show that the corrective gradients of EP rapidly vanish for isolated tail samples, leaving heavy-tailed deviations insufficiently controlled. To address this limitation, we propose QQWorld, which replaces EP with a
quantile-
quantile matching objective that directly aligns projected latent samples with rank-matched Gaussian
quantiles, thereby maintaining effective corrective gradients in the tails. We further develop cross-batch QQ, which enlarges the effective ranking pool using detached samples from previous batches, and characterize its bias-variance trade-off. Across four control environments, QQWorld effectively improves the average planning success rate of LeWM, while consistently yielding better Gaussian
alignment and thinner latent tails.
Read the full abstract ▾
Computer Vision
Large scale cross-regional remote sensing flood monitoring framework for operative mapping and impact analysis
Ilya Novikov, Svetlana Illarionova, Ruslan Dzharkinov et al. (+6 more)
Effective flood monitoring is critical for minimizing the impacts of flood disasters on populations and infrastructure. Yet reliable remote sensing across extensive and environmentally diverse regions remains challenging, as most
segmentation algorithms lack the generalisation capacity required for large-scale application, while annotated flood data are scarce and unevenly distributed. This study presents an end-to-end
multimodal framework for Russian Federation territories sustainable flood monitoring and damage assessment based on synthetic aperture radar data, multispectral imagery, and digital elevation models with their derivatives, forming a 21-channel input. Using a self-collected
multimodal dataset covering seven Russian regions, two strategies for water surface detection under limited data conditions were compared: a supervised U-Net++ model and the self-supervised AnySat architecture pre-trained and fine-tuned for the
segmentation task. Under the data conditions of this study, supervised learning proved more effective, while the AnySat-based approach offered greater stability and retains advantages for settings where larger unlabelled data or missing modalities at inference are expected. The best flood area predictions were used to estimate flood impact in urban areas in terms of the area affected, material damage, casualties, and ecological and agricultural impact. The estimations were conducted following the official methodology of the Russian Ministry of Emergency Situations. Applied to the 2019 Tulun flood, the obtained results closely matched official assessments, except for material damage, due to the open-source databases usage. The results demonstrate the potential of deep learning and
multimodal satellite data integration for scalable, reliable flood monitoring across diverse environmental and data-limited conditions.
Read the full abstract ▾
All authors Ilya Novikov, Svetlana Illarionova, Ruslan Dzharkinov, Maria Smirnova, Ayrat Abdullin, Anna Korotkova, Mariia Ulianova, Dmitrii Shadrin, Evgeny Burnaev
Computer Vision
Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfer
Weiquan Lin, Yu Deng, Shiyang Liu et al. (+6 more)
Hand-object interaction (HOI) modeling remains challenging because it requires joint
reasoning about hand articulation, object geometry, contact, semantics, and dynamics under severe visual uncertainty. Foundation models introduce transferable prior knowledge learned from large-scale cross-domain data, offering new ways to address these challenges beyond task-specific data and models. However, the rapidly growing literature remains fragmented, and existing studies typically describe these methods simply as “using large models” without systematically characterizing what knowledge is introduced, where it enters the HOI pipeline, or which HOI uncertainty it helps reduce. This survey presents the first systematic review of foundation-model priors for HOI. We organize the literature into six HOI tasks spanning reconstruction and generation. More importantly, we establish a taxonomy of eight foundation-model sub-priors grouped into
geometric, semantic, and visual families.
Geometric priors encompass shape
retrieval, shape reconstruction, and spatial reconstruction; semantic priors include semantic grounding and
language reasoning; and visual priors cover visual representation, image generation, and
video generation. Based on this taxonomy, we systematically analyze how different priors are represented, injected, and adapted across HOI pipelines and tasks. Beyond how foundation models empower HOI, we further examine how HOI-derived knowledge is used in
robot learning, including human-data pretraining, human-to-
robot skill transfer, and HOI-to-
robot data generation. Finally, we summarize
datasets and evaluation protocols, and discuss limitations and future directions toward more generalizable HOI systems. To support long-term progress, we curate a live repository that continuously aggregates emerging methods and
benchmarks.
Read the full abstract ▾
All authors Weiquan Lin, Yu Deng, Shiyang Liu, Luping Xiao, Xu Tang, Junzhi Yu, Jiaolong Yang, Lei Zhang, Xingyu Chen
Computer Vision
Explaining Image Similarity with Automatically Extracted Concept Activation Vectors
Isaac Roberts, Petra Bevandic, Alexander Schulz et al. (+1 more)
Image similarity underlies many computer
vision applications, yet it is often unclear why two images receive a high or low similarity score. Existing explainability methods often rely on gradient-based attribution maps to provide local justifications for similarity. These approaches struggle to provide global insights into what specifically drives similarity in regions of an
embedding space, such as texture, shape, or color. We introduce a model- and metric-agnostic framework that explains image similarity using Concept Activation Vectors (CAVs) extracted automatically via Sparse Autoencoders (SAEs). Given a pair of images, we perturb their
embeddings along discovered concept directions and measure the resulting change in a chosen similarity function, yielding concept importances. For image pairs, we provide localization with concept attribution maps. We extend this procedure to group-level settings, explaining what drives similarity across a cluster of images rather than a single pair, and further, we introduce Exemplar
Retrieval, aiming to recover samples with similar reasons contributing to similarity. Our experiments show that our latent perturbations are more faithful to the underlying data distribution than pixel-space baselines, and that concept importances linearly recover the true similarity score. Qualitative results further confirm the usefulness of our methods in understanding a model's individual and group similarity judgments.
Read the full abstract ▾
All authors Isaac Roberts, Petra Bevandic, Alexander Schulz, Barbara Hammer
Computer Vision
ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadow
Jin Cao, Zian Meng and Kaipeng Zhang
We present ShadowDancer, a novel approach to any-action, frame-level control of interactive
video world models. The obstacle is representational: existing interfaces either encode an action loosely, leaving how it unfolds for the model to improvise, or encode it exactly through structured signals that serve one family and are hard to acquire, so precise control across diverse dynamics remains impractical. Demonstration
videos are the natural remedy, specifying any dynamics frame by frame; yet a
video shows its dynamics only through one particular appearance, a single shadow of the underlying dynamics, so actions learned from demonstrations transfer poorly to new scenes. ShadowDancer addresses this with two key innovations: (1) shadow pairs,
video pairs that replay the same dynamics under independently resampled appearance, constructed at scale by our Shadow Library, so that a dynamics family becomes controllable exactly when such pairs can be constructed for it; and (2) cross-shadow prediction, which learns actions by predicting one shadow from the other, so that whatever the pairing resamples is discarded by construction and whatever it preserves becomes the action, yielding a unified dynamics representation that drives a block-causal world model. Any demonstrated clip thus becomes a reusable action asset, replayed in new environments without action labels, motion
estimators, or fine-tuning. Experiments demonstrate improved action transfer and long action rollout over strong latent-action and interactive world model baselines across diverse dynamics families, with an average blinded win rate of 86% in rollout comparisons. We show
video results at https://ShadowDancer-1.github.io
Read the full abstract ▾
Computer Vision
Jie Ma, Zhike Qiu, Jie Gao et al. (+4 more)
While visual
token pruning is essential for efficient
Multimodal Large
Language Models (MLLMs), existing training-free methods suffer from a critical limitation: they rely on static, instantaneous heuristics to perform irreversible filtering. This approach ignores the hierarchical nature of MLLMs, where
token importance often evolves dynamically rather than remaining fixed across layers. Consequently,
tokens essential for deep-layer
reasoning are often prematurely discarded by shallow-layer estimates. To address this, we propose Trend-aware
Pruning, a novel framework that elevates
pruning from a local snapshot decision to a temporal trajectory modeling problem. Instead of relying on isolated scores, our method captures the momentum of
attention flow. This enables a dynamic rectification mechanism that selectively reactivates "late-blooming"
tokens, those initially undervalued but exhibiting rising semantic importance, thereby preventing the loss of critical visual cues. Extensive experiments demonstrate that our approach achieves a superior efficiency-performance trade-off across diverse
multimodal tasks. Notably, it reduces visual
tokens by over 77.8%, retaining only approximately 23
tokens in the final layer while maintaining competitive performance, offering a robust and reversible solution for high-efficiency
multimodal inference.
Read the full abstract ▾
All authors Jie Ma, Zhike Qiu, Jie Gao, Jiayi Ji, Qian Chen, Xiaoshuai Sun, Rongrong Ji
Computer Vision
Same Branches, Different Trees: A Bifurcation Connectedness Metric for Coronary Artery Segmentation and FFR-CT Decision Agreement
Maame Owusu-Ansah, Kelvin Lee, Dr Vinod Venugopal et al. (+3 more)
Fractional flow reserve derived from CT angiography (FFR-CT) simulates flow through a patient-specific vessel model, so its accuracy depends on the connectedness of the segmented tree, not only on volumetric overlap: a
segmentation can reach high Dice yet sever a bifurcation, dropping the downstream subtree and reversing the treatment decision.
Topology-aware losses such as clDice and Skeleton Recall act on the global centreline and can miss localised breaks. We study the Bifurcation Connectedness Score (BCS), which scores connectedness at each ground-truth bifurcation, and soft-BCS, its differentiable training surrogate. BCS captures a property of
segmentation quality the standard metrics miss: it responds strongly to breaks in connectedness while staying largely unchanged under connectedness-preserving narrowing. Higher BCS accompanies closer agreement between the FFR-CT decisions a solver makes on predicted versus ground-truth geometry, most clearly in severe disease (OR 2.16, CI [1.23, 4.18]). Both decisions come from the same solver, so this reflects
geometric, not clinical, fidelity. In training, soft-BCS and Skeleton Recall recover the same branches but build different trees. Recovering branches and keeping them connected are separable properties, so we recommend reporting a measure of each.
Read the full abstract ▾
All authors Maame Owusu-Ansah, Kelvin Lee, Dr Vinod Venugopal, Muhammad Moazzam Jawaid, Wenting Duan, James Brown
Computer Vision
AdaAnchor4D: Anchor-Conditioned Spatiotemporal Feature Aggregation for Monocular UAV 4D Reconstruction
Peiyi Xu, Junpeng Zhang, Guanbin Li et al. (+6 more)
Monocular UAV videos provide valuable observations for dynamic reconstruction of complex urban scenes. However, such scenes exhibit pronounced spatiotemporal heterogeneity: different regions follow distinct temporal activity patterns, while the motion states of some dynamic regions may further evolve over time. Although dynamic Gaussian methods based on decomposed shared spatiotemporal feature fields have achieved efficient and accurate reconstruction in object-centric or relatively compact scenes, their commonly adopted fixed plane-wise feature combination mechanisms are less suited to the heterogeneous local dynamics of
UAV scenes, often leading to ghosting artifacts and blurred dynamic details. To address this challenge, we propose AdaAnchor4D, an adaptive anchor deformation framework for
monocular UAV dynamic scene reconstruction. At its core, Anchor-Conditioned Feature Aggregation (ACFA) adaptively aggregates shared spatiotemporal features using anchor-specific aggregation
embeddings and temporal information, allowing different local units to obtain dynamic representations tailored to their local and temporal states. Decoupled Local Geometry Deformation (DLGD) separates anchor-state deformation from local Gaussian geometry deformation, while Density-Adaptive Coordinate Warping (DACW) reparameterizes feature-query coordinates according to the axis-wise anchor distributions, alleviating the mismatch between non-uniform
geometric sampling and uniform grid parameterization. Experiments on
UAV-Arc4D, VisDrone, and UAVDT show that AdaAnchor4D achieves higher rendering quality than representative dynamic Gaussian methods while maintaining real-time rendering performance. The code will be made publicly available.
Read the full abstract ▾
All authors Peiyi Xu, Junpeng Zhang, Guanbin Li, Ronghua Shang, Mingtao Feng, Le Dong, Weisheng Dong, Guangming Shi, Jie Feng
Computer Vision
ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding
Mingkang Dong, Muxin Pu, Jie Li et al. (+8 more)
Streaming
video understanding requires models to continuously retain useful visual evidence before future questions are known. Existing approaches primarily manage the growing visual context according to
token importance, temporal redundancy, or segment-level relevance, but rarely organize evidence around objects that persist and evolve over time. Thus, in this paper, we introduce ObjectStream, a training-free framework that treats latent objects as
memory anchors for streaming
video understanding. ObjectStream induces spatially coherent latent objects directly from frozen
Video-LLM representations, links them across frames into persistent anchors, and maintains their histories under a bounded
memory budget, without requiring external object detectors or
segmentation models. Built on these anchors, ObjectStream preserves three complementary forms of evidence: persistent object histories, transient object changes, and recent visual context. This design enables existing
Video Large
Language Models (
Video-LLMs) to reason over object identities, interactions, and state changes while leaving the underlying model unchanged. Extensive experiments on online streaming and offline long-
video benchmarks demonstrate both effectiveness and efficiency. In online streaming evaluation, ObjectStream improves Qwen2.5-VL-7B by 10.0 points on OVO-Bench Real-Time Visual Perception, while reducing peak GPU mem-ory and TTFT by approximately 50%. On offline long-
video benchmarks, it surpasses the full-
token baseline while discarding 82.5% of visual
tokens. These results highlight latent objects as a practical and effective organizing principle for compact streaming
video memory.
Read the full abstract ▾
All authors Mingkang Dong, Muxin Pu, Jie Li, Bohan Guo, Songruo Chen, Bin Ren, Xu Zheng, Chen Zhao, Tianwen Qian, Mohamed Elhoseiny, Yuqian Fu
Computer Vision
MonoVoc: Decoupling Geometry and Semantics for Lightweight Monocular Open-Vocabulary 3D Gaussians
Pouya Ardekhani, Zahra Dehghanian, Morteza Abolghasemi et al. (+1 more)
Open vocabulary 3D scene understanding is essential for next-generation interactive systems, empowering users to intuitively query and navigate reconstructed environments using natural
language. However, current 3D Gaussian frameworks are often bottlenecked by restrictive multiview capture requirements, costly scene-specific optimization, and the massive
memory overhead of storing dense
language features. We present a novel, training-free pipeline that fundamentally reimagines this paradigm by explicitly decoupling 3D
geometric reconstruction from semantic integration. Given a standard
monocular video sequence as input, our method efficiently outputs a compact, highly interpretable, and fully searchable object-level semantic Gaussian map. Rather than entangling heavy
language embeddings within the mapping loop, we extract geometry independently and ground semantics through a lightweight, modular post-processing framework. Extensive evaluations on the Replica
dataset demonstrate that this decoupled architecture preserves strong rendering fidelity and competitive
segmentation accuracy. Crucially, by replacing dense per-Gaussian storage with modular, object-level semantic
embeddings, our approach delivers an order-of-magnitude reduction in
memory usage compared to SOTA baselines. This provides a highly efficient, scalable, and practical solution for open-vocabulary 3D
retrieval and question answering directly from everyday
monocular video.
Read the full abstract ▾
All authors Pouya Ardekhani, Zahra Dehghanian, Morteza Abolghasemi, Hamid R. Rabiee
Computer Vision
Filling the Pareto-Optimal Front for Affordance Segmentation on Embedded Devices Using RGB-D Cameras
Edoardo Ragusa, Giovanni Paolo Canuti, Simone Lugani et al. (+2 more)
While depth sensors have the potential to complement RGB data for affordance
segmentation in wearable
robots, their usage seems to remain underexplored. The paper proposes two approaches: a reformulated version of hardware-aware neural architecture search, endowed with a newly designed search space to integrate depth (D) information into small-sized deep networks, and a dedicated fine-tuning approach, including a preprocessing layer to merge depth information with RGB data and make it compatible with conventional architectures. In both cases, those methods aim to generate solutions that benefit from modern (portable) hardware accelerators and overcome existing tiny-like approaches, which often fail to tackle critical scenarios due to the severe constraints set by the supporting hardware. Extensive experiments on a pair of real-world
datasets demonstrate the effectiveness of the proposed method as compared with existing solutions. The approach presented in the paper generates, in most cases, solutions that identify the Pareto optimal front to balance generalization performance and hardware requirements. The paper also describes the supporting prototype, including a Jetson Nano board and a RealSense RGB-D camera. When considering the energy profile of the device, the overall system can attain real-time performances within an energy budget that is compatible with standard batteries, such as those used in smartphones.
Read the full abstract ▾
All authors Edoardo Ragusa, Giovanni Paolo Canuti, Simone Lugani, Rodolfo Zunino, Paolo Gastaldo
Computer Vision · Cross-listed from cs.AI
Tycho: Active Abstraction with Programmatic World Models for ARC-AGI-3
Jens Lehmann, Andrei Aioanei and Sahar Vahdati
ARC-AGI-3 turns abstraction into an interactive problem of skill acquisition. A player must infer an unfamiliar game's rules, hidden state, and goal while maintaining action efficiency because every move counts. We formalize these environments as parameterized rendered deterministic Moore machines and introduce Tycho, a coding-
agent system that constructs and uses game-specific models during interaction. Tycho separates actionable observations from intermediate animation, level-completion, and game-over frames. From this structured history, an
agent can model, test, plan with, repair, or bypass a free-form executable hypothesis. In one matched public-set run per policy, we compare four orchestration policies on all 25 public games using Claude Opus 4.8 under matched inference budgets. Actor-requested delegation to a model builder obtains the highest observed mean Relative Human Action Efficiency (RHAE), 88.49. With this selected policy, GPT-5.6 Sol and Opus 5 both reach 100.00 RHAE and complete all 183 levels. Their game-balanced first-run human-replay midranks are 98.5 and 100.0. Opus 5 uses 61% fewer scored actions than the aggregate official human baselines. Automatic repair after verification failures produces models that reproduce observed transitions much more accurately, yet reaches only 83.07 RHAE. Transition match indicates whether a simulator reproduces observed dynamics, not whether it has identified the objective or improves the next action. Strong play also requires deciding when to construct, repair, use, or bypass a model. We call this joint problem active abstraction: generating a testable model from costly interaction and deciding when acquiring or using it is worth its cost.
Read the full abstract ▾
Computer Vision
Beyond Visual Ambiguity: Guiding Robust Monocular Depth Estimation in Challenging Scenarios via Detailed Long Captions
Junrui Zhang, Jiaqi Li, Yiran Wang et al. (+2 more)
Monocular depth estimation (MDE) faces challenges with non-Lambertian surfaces and adverse weather conditions due to the visual ambiguities inherent in single-image limited information. Existing works address them in isolation via image inpainting or augmentation, yielding limited robustness gains.
Language, as a powerful complementary modality to
vision, is demonstrated to enhance the visual perception capabilities of
vision-
language models (VLMs) via detailed long captions. However, prior
language-integrated MDE methods fail to fully harness this potential due to short text input with limited information, coarse global text feature learning, and limited
language guidance during depth decoding. To address these limitations, we propose CapDepth, a novel framework for robust MDE that leverages guidance from detailed long captions to alleviate visual ambiguities in both challenging scenarios. First, we design a detailed long caption input template that explicitly conveys rich spatial relationships among multiple atom sentences. Second, a dynamic caption encoder is introduced to extract fine-grained depth-relevant text features via progressive masked
attention. Finally, we propose a text-adaptive decoder that guides enhanced depth decoding with text features via stable adaptive layer normalization. Extensive experiments validate the efficacy of CapDepth, which outperforms state-of-the-art methods, achieving depth error reductions of 25.0% on non-Lambertian surfaces and 22.0% under adverse weather conditions.
Read the full abstract ▾
All authors Junrui Zhang, Jiaqi Li, Yiran Wang, Liao Shen, Zhiguo Cao
Computer Vision
MSCM-net: A hyperspectral image classiffcation method based on multi-scale convolution and Mamba
Jianjun Chen, Linlin Wang, Lifang Chang et al. (+4 more)
Hyperspectral imaging is widely used in remote sensing and engineering. Therefore, research on its classification methods is crucial. While CNN and
Transformer-based methods have advanced, they still face locality constraints and high computational complexity. To address these issues, we propose an innovative hyperspectral image classification model, MSCM-net. Specifically, first of all, a model architecture combining multi-scale CNN and Mamba is proposed. It consists of a multi-scale feature extraction module (MCSE) and multiple stacked Mamba blocks, which integrates the local feature extraction capability of multi-scale CNN and the long sequence modeling advantage of Mamba. Secondly, the proposed MCSE module consists of multi-scale convolution and SENet. Convolution kernels of different scales extract local information with different receptive fields, enhancing the fusion of spatial and spectral information. Meanwhile, the SENet enables the model to automatically learn the importance of each channel in the multi-scale features. Furthermore, we also propose a dual-branch feature aggregation module, which further effectively extracts and integrates the spectral information contained in the central pixel and the spatial information in the surrounding pixels. Our model has undergone numerous experiments on three widely used
benchmark datasets. The experimental results show that MSCM-net can achieve advanced classification performance while reducing computational complexity.
Read the full abstract ▾
All authors Jianjun Chen, Linlin Wang, Lifang Chang, Limin Huo, Shujiang Song, Yanjia Zhao, Mingwei Shao
Computer Vision
Theia: Large-Scale Multimodal Captioning and Automated Validation of the Incidents1M Dataset for Data-Free Distillation
Simone Giano, Lorenzo Severini, Alessandro Galdelli et al. (+1 more)
The deployment of
Vision-
Language Models (VLMs) in critical domains like disaster management requires high-quality
multimodal datasets, especially for transferring knowledge via Data-Free Knowledge
Distillation (DFKD). However, existing
datasets in this domain either entirely lack descriptive text, such as Incidents1M, or suffer from severe text-image semantic misalignment, such as CrisisMMD. In this work, we present a novel methodology to construct and automatically validate a large-scale
multimodal dataset for disaster response. Starting from the
vision-only Incidents1M, we successfully recovered 100,000 images and generated high-fidelity textual descriptions using two distinct Qwen3.5 architectures: a 4B dense model and a 35B Mixture-of-Experts (MoE) model. To ensure the generated captions provide reliable semantic anchoring for DFKD, we introduce an image-blind LLM-as-a-Judge validation pipeline leveraging Qwen3.5-9B. By intentionally obscuring the original image from the judge, this evaluator accurately simulates the modality gap of the student model during data-free
distillation. Our evaluation across 173,179 label pairs demonstrates a high semantic agreement (78.65/100) between the two architectures. Furthermore, the automated evaluation reveals a conservative captioning behaviour, characterized by a high Precision (77.6%) and low Recall (46.0%). This minimizes the false positive noise, while simultaneously exposing underlying human annotation inconsistencies in the original ground truth. This work provides a scalable, LLM-validated
multimodal dataset and a reproducible framework to advance cross-modal knowledge
distillation.
Read the full abstract ▾
All authors Simone Giano, Lorenzo Severini, Alessandro Galdelli, Adriano Mancini
Computer Vision
TARS: Timestep-Aware Data Scaling for 3D-Free Video Re-Shooting
Jiwen Liu, Shujuan Li, Xiaohan Li et al. (+5 more)
Video re-shooting aims to regenerate
videos with controllable camera motion and viewpoint. Existing methods rely on explicit 3D priors, which are limited by reconstruction quality and often perform poorly when synthesizing previously unseen regions, or on paired
videos with different camera trajectories, whose scarcity hinders generalization. We revisit
video re-shooting through text-driven semantic viewpoint specification, enabling control over shot scale, viewing angle, and first-/third-person perspective. To this end, we propose TARS, a 3D-free
video re-shooting paradigm. Timestep-wise sensitivity analysis reveals that camera motion is primarily established during high-noise stages, where coarse spatiotemporal structures are formed. Based on this insight, we introduce self-supervised training to learn camera dynamics and fundamental visual representations without paired re-shooting data or 3D reconstruction. Through data scaling and joint textual-camera conditioning, TARS supports robust camera and viewpoint control, plausibly synthesizing regions beyond the source view under large camera motions while enabling reverse-angle re-shooting and perspective switching. Extensive experiments show that TARS provides more accurate and temporally consistent camera control than prior methods. Project Page: https://ymlinfeng.github.io/TARS.github.io/
Read the full abstract ▾
All authors Jiwen Liu, Shujuan Li, Xiaohan Li, Zijie Meng, Xinyue Liu, Yulong Xu, Yan Zhou, Guoxin Zhang
Computer Vision
Space2Ground 2.0: A Multi-Source Dataset and Framework for Agricultural Monitoring through Fusion of Street-Level and Satellite Imagery
Iason Tsardanidis, Alkiviadis Koukos, George Choumos et al. (+2 more)
Accurate and scalable parcel-level agricultural monitoring remains challenging because satellite Earth Observation alone provides only an overhead perspective of agricultural parcels, while optical observations are further affected by cloud-induced temporal gaps. This paper presents Space2Ground 2.0, a multi-source framework integrating Sentinel-1 SAR and Sentinel-2 multispectral time series with geo-tagged street-level imagery acquired using vehicle-mounted cameras and shared through the Mapillary platform. A largely automated processing pipeline performs semantic filtering, image quality assessment, viewpoint-based parcel association, and
dataset refinement, transforming large volumes of crowdsourced imagery into parcel-linked, analysis-ready data. Applied over Cyprus during the 2022 growing season, the pipeline produced a curated
dataset of 46,050 annotated street-level images, selected from an initial collection exceeding 900,000 images and linked with satellite information for 8,581 agricultural parcels. The practical value of the
dataset was assessed through parcel-level crop classification experiments using both single- and multi-source observations. The results demonstrate that street-level imagery provides complementary fine-scale visual information that enhances classification when integrated with satellite time series. Overall, Space2Ground 2.0 provides an openly available
benchmark dataset and a reproducible methodology for
multimodal agricultural monitoring, with potential applications in visual verification, reduced reliance on costly field inspections, and data-driven agricultural policy implementation.
Read the full abstract ▾
All authors Iason Tsardanidis, Alkiviadis Koukos, George Choumos, Vasileios Sitokonstantinou, Charalampos Kontoes
Computer Vision
EgoGenesis: Egocentric World-Action Modeling with Online Anchored Projective Memory and Action-3D RoPE
Zexuan Yan, Yuzhou Wu, Yue Ma et al. (+9 more)
Egocentric video offers rich manipulation experience for
embodied AI, yet collecting diverse
egocentric data across scenes, objects, motions, and embodiments remains costly. We present \method, an
egocentric world-action simulator that synthesizes controllable, high-quality manipulation
videos to expand scarce real-world training data. \method{} builds on a pretrained
video generation prior and introduces two geometry-aware conditioning mechanisms. Online Anchored Projective
Memory (OAPM) preserves a first-frame 3D scene anchor while periodically refreshing a recent state during autoregressive generation. Action-3D Rotary Position
Embedding (A3D-RoPE) encodes end-effector motion with camera-aware 3D rotary coordinates, injecting action geometry into skeleton-to-
video cross-
attention for precise control. Together, these components improve visual fidelity,
geometric stability, and action
alignment in long
egocentric rollouts. Moreover, augmenting 400 real trajectories with 400 \method-generated trajectories improves out-of-distribution real-
robot success from 77% to 84% on single-arm tasks and from 53% to 70% on dual-arm tasks, demonstrating that the synthesized data substantially improve downstream WAM generalization.
Read the full abstract ▾
All authors Zexuan Yan, Yuzhou Wu, Yue Ma, Zonghang He, Kaibo Yin, Xiaobing Tu, Yinggui Wang, Jinkui Ren, Xiantao Zhang, Shijian Wang, Jinghong Liu, Linfeng Zhang
Computer Vision · Cross-listed from cs.AI
Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents
Hanzhang Zhou, Panrong Tong, Xu Zhang et al. (+13 more)
GUI
agents have the potential to become a general purpose executor over existing digital devices. To advance them toward real-world use, we envision
agents that operate reliably on real devices, execute workflows across platforms, combine GUI interaction with CLI execution, complete long-horizon tasks, proactively initiate useful services, and autonomously improve their capabilities with minimal human effort. Guided by this
vision, we present Qwen-UI-
Agent, a real-world centric foundation GUI
agent spanning mobile, computer-use, web, and DeepSearch environments. Qwen-UI-
Agent combines diverse sandbox environments with a large-scale real-device mobile runtime. Its unified action space interleaves GUI operations with CLI execution and generates batched actions in a single model turn. An AutoResearch-style data flywheel uses
agents to construct tasks and environments, diagnose failures, and plan subsequent iterations. Online RL supports training on trajectories exceeding 100 turns, with over 10,000 concurrent environments accelerating rollout. A lightweight harness layer supports proactive service initiation and stateful workflows across mobile and computer. Across a broad suite of evaluations, Qwen-UI-
Agent sets state-of-the-art performance on mobile-use
benchmarks while delivering competitive performance on computer- and browser-use tasks against frontier models, including Opus 4.8, Gemini 3.1 Pro, and GPT-5.6 Sol. On mobile use, it achieves 82.1% on MobileWorld, 92.2% on MobileWorld-Real, and 97.5% on AndroidDaily. On computer use, it achieves 79.5% on OSWorld-Verified and a 40.0% partial-progress score on OSWorld-v2. On browser use and GUI grounding, it achieves 73.6% on WebArena and 81.5% on ScreenSpot-Pro, respectively.
Read the full abstract ▾
All authors Hanzhang Zhou, Panrong Tong, Xu Zhang, Quyu Kong, Chenglin Cai, Tianyu Xia, Gongjie Zhang, Jianan Zhang, Long Li, Long Chen, Lei Wang, Gaole Dai, Pengxiang Li, Liangyu Chen, Yue Wang, Steven Hoi
Computer Vision
FaithEyes: Towards Faithful Tool Use via Multi-Agent Process-Image Self-Verification
Haoqing Wang, Xingrun Xing, Ziheng Li et al. (+2 more)
Agentic
vision-
language models (VLMs), which interleave textual
reasoning with explicit tool calls such as cropping and code-based image manipulation, have emerged as a compelling paradigm for reliable and interpretable multi-modal
reasoning. However, recent studies have revealed that such models often use tools unfaithfully. Many process images are irrelevant to the question (e.g., the crops miss the queried target), yet the tool call still receives full credit and the model still answers correctly. Such decorative or misaligned tool calls waste computation and reveal that the model does not faithfully use the evidence it retrieves. This may stem from two limitations of prevailing methods: the tool reward fails to distinguish useful from useless calls, and tool feedback carries no signal of usefulness. To this end, we introduce FaithEyes, a multi-
agent self-judging framework. Concretely, we use a VLM to judge whether each process image helps answer the question. The judgement is injected into the
reasoning context as part of the tool observation to help subsequent
reasoning, and meanwhile is used to scale the tool reward by the helpful-tool ratio to suppress reward hacking. To keep judgement available at evaluation, we further design a multi-
agent framework where the model itself serves as a subagent to judge the tool calls from the main
agent, eliminating any dependence on external models at inference. Training via a two-stage SFT + RL pipeline on adapted open-source data, FaithEyes attains competitive or superior accuracy across visual perception and
reasoning benchmarks, while substantially improving tool faithfulness and reducing inference cost. The homepage is at https://github.com/Mosi-AI/FaithEyes.
Read the full abstract ▾
All authors Haoqing Wang, Xingrun Xing, Ziheng Li, Jianyuan Guo, Yehui Tang
Computer Vision
Scaling Vision-Language Models Is Not Enough to Mitigate Bias
Ioannis Sarridis, Ioannis Kompatsiaris and Symeon Papadopoulos
Vision-
Language Models (VLMs) such as CLIP are now foundational to
multimodal systems, yet their robustness to spurious correlations remains poorly understood at scale. We present the first large-scale empirical study of 194 publicly available VLMs, including 16 model families, covering a wide range of model sizes, 24 training
datasets, and three evaluation
benchmarks, namely ImageNet (overall performance), CelebA (typical single-attribute bias), and UrbanCars (complex multi-attribute biases). Across these settings, the Spearman correlation between model scale and performance weakens as evaluation shifts from ImageNet ($\rho{=}0.68$) to single-attribute ($\rho{=}0.48$) and further to multi-attribute ($\rho{=}0.05$) bias
benchmarks. In contrast, properties of the training data (size and quality) show more consistent relationships with worst-group accuracy across both bias
benchmarks. Notably, curated
datasets yield improvements of up to 25% over uncurated alternatives at a comparable scale. Finally, the effect of architectural choices (e.g., patch size, image resolution) is highly context-dependent, varying with the nature of the
benchmark, including the type of bias and its spatial distribution within images.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
UniCross: Unified Cross-Skill Dexterous Manipulation Synthesis
Hui Zhang, Julian Ferchow, Jie Song et al. (+1 more)
Many dexterous manipulation tasks require the object to remain securely held throughout the interaction. From the perspective of hand-object relational motion, such manipulation comprises four canonical skills: grasping, relocation, in-hand rotation, and in-hand translation. Human hands flexibly compose these skills to accomplish complex tasks. Existing approaches, however, model these skills separately with skill-specific action constraints, objectives, or even dedicated hand morphologies, which breaks the compatibility and continuity required for long-horizon composition. In this work, we present a unified framework that models all four skills in a single formulation that shares the same state and action spaces and a common objective structure. This formulation enables
distillation of a single cross-skill policy conditioned on the relational motion objectives, which achieves strong performance across all four skills, generalizes to unseen objects, remains robust to disturbances, and chains skills into long-horizon manipulation without switching policies. The framework also transfers effectively across different hand morphologies. Overall, our results suggest that different dexterous manipulation skills can be viewed as instantiations of a shared task formulation, revealing the intrinsic consistency. Project page: https://zdchan.github.io/UniCross/
Read the full abstract ▾
All authors Hui Zhang, Julian Ferchow, Jie Song, Mirko Meboldt
Computer Vision
Think with Extra-Image: A Farmland Segmentation Agent Driven by Spatio-Temporal Information Gain
Haiyang Wu, Weiliang Mu, Zhuofei Du et al. (+4 more)
Existing farmland remote sensing image (FRSI)
segmentation follows a "Think with Intra-Image" paradigm, assuming that the current image contains sufficient visual evidence for reliable
segmentation. Yet farmland appearance varies with phenology and spatial context and is often confused with other land-cover, making instantaneous, local observations inadequate. Thus,
segmentation ambiguity stems not only from limited model representation, but more fundamentally from the required spatio-temporal information lying beyond the current image. Based on this insight, we redefine FRSI
segmentation from an information bottleneck perspective as a dynamic decision process driven by task-relevant extra spatio-temporal information gain. We further propose FarmSeeker, a dynamic FRSI
segmentation agent that identifies ambiguous regions, reasons about their causes, and queries extra spatio-temporal information on demand for accurate
segmentation. To evaluate FarmSeeker, we construct GSFS-Bench, the first global-scale, high-resolution FRSI
segmentation benchmark that supports
reasoning-querying. Experiments show that FarmSeeker achieves more stable
segmentation performance than existing methods. The project is publicly available at: https://withoutocean.github.io/FarmSeeker/
Read the full abstract ▾
All authors Haiyang Wu, Weiliang Mu, Zhuofei Du, Dandan Zhong, Kaijie Shi, Haifeng Li, Chao Tao
Computer Vision
S-Avatar: Diffusion-Guided Gaussian Head Avatars from a Single Image
Hail Song, Seokhwan Yang, Jiwon Yang et al. (+2 more)
We propose S-Avatar, a novel method for generating photorealistic 3D head avatars from a single image using a
diffusion-guided 3D model generation module and strategies for animating 3D Gaussian Splatting (3DGS). While single-image head avatar reconstruction is crucial for lifelike Virtual Reality (VR) applications, existing approaches often struggle to preserve 3D consistency under unseen viewpoints. S-Avatar addresses this limitation through a three-stage pipeline. First, a high-resolution 3DGS is synthesized directly from a single image using a
diffusion-based Gaussian splat generation module. Next, the parametric head model FLAME is aligned with the generated 3DGS by optimizing its parameters and spatial transformations. Finally, to adapt the 3DGS to FLAME variations, we construct a binding template that encodes the spatial relationship between the initial splats and FLAME. The dynamic 3D head avatar can then be rendered in real time by deforming the 3DGS with the binding template. By combining
diffusion-guided canonical 3DGS generation with FLAME-based control, our method achieves efficient and accurate reconstruction with enhanced 3D consistency. Evaluations on public
datasets demonstrate that S-Avatar outperforms state-of-the-art methods in novel-view and expression generation, achieving superior realism and consistency. Consequently, our approach represents a significant advance in accessible avatar creation, applicable to a wide range of VR/AR applications. The project page is available at https://github.com/hailsong/savatar.
Read the full abstract ▾
All authors Hail Song, Seokhwan Yang, Jiwon Yang, Woojin Cho, Woontack Woo
Computer Vision
Shoutai Zhu, Tianyang Xu, Bin Sun et al. (+3 more)
Interleaved
multimodal Chain-of-Thought (CoT) improves visual
reasoning by incorporating auxiliary visual evidence into intermediate
reasoning. However, existing approaches remain constrained by externally defined
reasoning traces and visual operations, limiting their ability to develop flexible and abstract visual thinking.
Reasoning with latent has recently offered a promising direction by internalizing intermediate computation into continuous representations. Nevertheless, existing visual-latent methods mainly supervise latent states through
alignment with compressed auxiliary visual features, treating them as proxies for visual observations rather than active
reasoning states. Consequently, they capture the provided evidence but fail to fully internalize the abstract
reasoning process induced by
multimodal CoT. In this paper, we propose OPLD (On-Policy Latent
Distillation), a simple framework that transfers the
reasoning capability induced by privileged
multimodal CoT into latent
reasoning representations. Extensive experiments on diverse
multimodal benchmarks demonstrate that OPLD consistently outperforms existing latent
reasoning methods and achieves state-of-the-art performance on multiple
benchmarks. The results suggest that supervising latent representations at the
reasoning-process level provides a more effective paradigm for
multimodal latent
reasoning than conventional feature-level
alignment.
Read the full abstract ▾
All authors Shoutai Zhu, Tianyang Xu, Bin Sun, Mingyuan Xu, Yu Liu, Qinzhen Guo
Computer Vision
What Makes Deep Learning Work for Traditional Chinese Medicine Tongue Diagnosis? A Comprehensive Ablation Study
Longxia Gao, Linan Wang, Yuhe Han et al. (+3 more)
Deep learning has shown promise for automated tongue diagnosis in traditional Chinese medicine (TCM), yet the design space remains underexplored. We conducted a systematic ablation study spanning 20+ model versions under rigorous 5-fold cross-validation on TongueDx2 (5,109 images, 976 expert-annotated) and a merged
dataset of 11,101 samples. We compared six backbone architectures, four loss functions, five augmentation strategies, and six training strategies. The best 976-sample model achieved weighted-F1 of 0.6625 using ConvNeXt-Tiny with restrained augmentation and weak-group ensemble, while the best 11,101-sample model reached weighted-F1 of 0.7761. Six key design principles emerged: (1) ConvNeXt-Tiny offers optimal parameter efficiency; (2) BCE substantially outperforms Asymmetric Loss (+2.7%); (3) restrained color augmentation is critical; (4) weak-group ensemble replacement (+2.1%) outperforms probability averaging; (5) data scaling yielded +20.6% improvement; (6) expanding from 13 to 45 label dimensions caused catastrophic collapse (0.78 to 0.22). These principles are generalizable to multi-label medical image classification with class imbalance.
Read the full abstract ▾
All authors Longxia Gao, Linan Wang, Yuhe Han, Junze Geng, Meng Zhang, Hanqing Zhao