Computer Vision · Cross-listed from eess.IV
Robustness and Stability Analysis of Differentiable Shift-Variant FBP for Cone-Beam CT under Challenging Acquisition Settings
Chengze Ye, Linda-Sophie Schneider, Yipeng Sun et al. (+5 more)
The differentiable shift-variant filtered backprojection (SV-FBP) framework enables data-driven estimation of redundancy weights for cone-beam CT reconstruction under general source trajectories, removing the need for analytically derived weighting schemes. In this work, we present a systematic study of the robustness and adaptability of differentiable SV-FBP under challenging acquisition settings. We show that the framework remains stable across highly irregular and discontinuous trajectories, indicating that reconstruction performance is largely insensitive to trajectory ordering or continuity. Instead, the spatial distribution of sampling points plays a more dominant role. Under sparse-view conditions, differentiable SV-FBP achieves competitive reconstruction quality while providing an order-of-magnitude reduction in computation time compared to iterative reconstruction methods at moderate sampling densities. However, we identify a clear transition regime under severe undersampling, where the absence of iterative data consistency leads to performance degradation. Furthermore, we demonstrate that the framework remains applicable to non-planar multi-isocenter geometries, such as Lissajous-saddle trajectories, without requiring architectural modifications. These findings provide new insights into the behavior and limitations of the differentiable SV-FBP model and highlight it as a flexible and efficient solution for non-standard and robotic CBCT acquisition scenarios.
Read the full abstract ▾
All authors Chengze Ye, Linda-Sophie Schneider, Yipeng Sun, Mareike Thies, Siyuan Mei, Paula Andrea Pérez-Toro, Siming Bayer, Andreas Maier
Computer Vision
TDSal: Task-Based Top-Down Saliency Prediction Model
Can Mizrakli and Tolga K. Capin
Visual saliency aims to predict the regions of an image most likely to attract human visual
attention. While most saliency models assume free-viewing conditions, human
attention is often shaped by explicit task goals. In this work, we address task-driven saliency prediction by proposing a model that conditions visual
attention on natural-
language task descriptions. The model produces task-dependent saliency maps that reflect how
attention shifts under different viewing intents. Through quantitative and qualitative analysis, we show that incorporating explicit task semantics enables more faithful modeling of goal-directed visual
attention.
Read the full abstract ▾
Computer Vision
Towards Objective Dysgraphia Detection: A Multi-Branch Deep Learning Approach for Online Handwriting Analysis
Lydia Ouhib, Yassine Ouzar, Zoé Pinseel et al. (+2 more)
Dysgraphia is a specific learning disability that is prevalent among school-age children. It affects handwriting coherence, quality, fluency, and legibility, often hindering academic achievement and early learning development. This motor coordination disorder is typically diagnosed through subjective assessments based on clinician observation, which can be timeconsuming and prone to variability. In this paper, we introduce a deep learning-based framework for objective dysgraphia detection using online handwriting data captured via digitizing tablets. The proposed framework relies on two complementary branches: the first pipeline extracts both handcrafted and
embedding-based kinematic features directly from raw temporal signals, while the second leverages image-based representations of the temporal signals generated using continuous wavelet transforms (CWT) and Gramian Angular Fields (GAF). The resulting features are then fused to leverage the complementary strengths of both representations. The four representations were evaluated separately and jointly using the publicly available DiaGraMo
dataset, showing that the fusion of GAF, MOMENT, and hand-crafted kinematic features outperforms each individual representation, as well as other fusion schemes. These findings highlight the potential of the complementarity of image and signal based representations for more objective dysgraphia detection.
Read the full abstract ▾
All authors Lydia Ouhib, Yassine Ouzar, Zoé Pinseel, Stéphane Bouilland, Mehdi Ammi
Computer Vision
Memory-Conditioned Tool Calling for Camera-First Visual Agents
Xiaofan Wu, Xi Zeng, Miaoxia Chen et al. (+5 more)
Recognition tells an
agent what is in an image; personal
memory affects what is worth looking up next. In a camera-first setting the user can send only an image, so the
agent must form the lookups. We study whether personal visual
memory improves
agent-side tool choice and tool arguments, and thereby more user-aligned multi-tool lookups. The design uses a three-layer personal visual
memory (profile, short-term focus, observations) that is loaded on each turn to condition an LLM tool-calling loop under camera-first intake, and includes conflict-aware write-back intended to refresh the user model for later captures. On 800 images paired with synthetic
memory blocks constructed for controlled ablation, removing the full three-layer
memory block reduces tool-query relevance by 0.47 points absolute (4.21 -> 3.74 on a 5-point scale; 11.2% relative) and end-to-end utility by 0.082 absolute (0.842 -> 0.760; 9.7% relative). These results measure
memory conditioning of tool policy under image-only intake with fixed synthetic blocks, not multi-session write-back from live user histories.
Read the full abstract ▾
All authors Xiaofan Wu, Xi Zeng, Miaoxia Chen, Peishan Chen, Shuyan Li, Jiyun Yao, Hanyong Zhong, Jiahao Zhu
Computer Vision · Cross-listed from eess.IV
Performance Benchmarking and Optimisation of Clustering Algorithms for Local and Non-Local Similarity Measure in Medical Image Analysis
Sisipho Hamlomo and Marcellin Atemkeng
Medical imaging generates high-resolution images posing significant storage, transmission, and computational challenges. While low-rank
matrix approximation (LoRMA) techniques offer efficient compression by exploiting structural redundancy, global approaches often fail to preserve local details critical for diagnosis. This paper focuses on clustering techniques that exploit non-local self-similarity to identify structurally similar regions in medical images. These clusters can be used for post-processing tasks such as adaptive image compression. We evaluate five clustering techniques: k-means, mini-batch k-means, agglomerative hierarchical clustering, balanced iterative reducing and clustering using hierarchies (BIRCH), and bisecting k-means across MRI, ultrasound, and chest X-ray modalities. All clustering techniques were optimised using random search, and cluster quality was assessed using the Silhouette score, the Davies-Bouldin (DB) index, and the Calinski-Harabasz (CH) index. Results demonstrate that standard k-means and bisecting k-means generally achieve strong cluster cohesion and separation across modalities. However, they tend to form a small number of clusters with high intra-cluster variability, limiting their effectiveness for post-processing tasks such as adaptive compression. Agglomerative clustering outperformed other techniques for MRI and ultrasound in terms of intra-cluster homogeneity, making it more suitable for preserving fine diagnostic details. For chest X-rays, mini-batch k-means achieved the best balance between clustering quality and intra-cluster compactness. BIRCH consistently underperformed across all modalities.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
TS-Mask VLA: 2D Temporal-Spatial Masking for Vision-Language-Action Model with Effective Bridging
Shengzhuo Yang, Ronghao Yu, Chuanjie Lv et al. (+5 more)
Vision-
language-action (VLA) models aim to understand natural-
language instructions and visual observations, and to generate and execute corresponding actions as
embodied agents. Recently, autoregressive
token-based action generation has driven the development of many representative VLA models. However, this paradigm often reduces action generation to next-
token prediction, thereby lacking explicit modeling of the spatiotemporal structure of action sequences and the disentanglement between
vision-
language representations and actions, which can limit performance in long-horizon and complex scenarios. In this paper, we propose TS-Mask VLA, a
vision-
language-action framework for
robot manipulation. TS-Mask VLA is built upon two key designs: (1) a Discrete
Diffusion Action Expert equipped with a Bridge
Attention conditioning bridge, which enables multi-layer conditioning from the VLM and facilitates more accurate and stable action generation; and (2) a temporal-spatial 2D masking strategy for discrete action
tokens that strengthens the model's understanding of cross-time dependencies and inter-dimensional coupling, leading to more structurally consistent action sequences. We conduct extensive experiments on simulation
benchmarks and real-world tasks. On LIBERO, TS-Mask VLA achieves a 95.7 percent average success rate with only 0.5B parameters, outperforming significantly larger models. On CALVIN, it attains the best average sequence length of 4.19 and strong long-horizon performance. Comprehensive analyses and ablations further validate the effectiveness of our design.
Read the full abstract ▾
All authors Shengzhuo Yang, Ronghao Yu, Chuanjie Lv, Linpeng Peng, Hang Yu, Jie Ren, Jiajun Lv, Yong Liu
Computer Vision · Cross-listed from cs.RO
RASR: Range-Aware Scale Recovery for Metric UAV Navigation
Hongtao Liang, Xinyu Shao, Chenxu Wang et al. (+5 more)
A central challenge in image-goal
UAV navigation under Global
Navigation Satellite System (GNSS) denial is estimating metric distance and heading between current and goal views. Dense pairwise geometry models capture relative scene structure, but without a calibrated metric scale, they cannot directly provide reliable distance estimates for
navigation. Although global scale calibration corrects the dominant scale bias, the remaining errors vary systematically with distance. In this paper, Range-Aware Scale Recovery (RASR) is proposed, which complements global scale calibration with range-aware residual correction. RASR encodes pairwise geometry extracted by a frozen Matching And Stereo 3D Reconstruction (MASt3R) backbone as a compact descriptor and separates the scale-recovery core from task-specific command calibration. On the official online evaluation of the
UAVs in Multimedia 2026 PairUAV challenge, RASR achieved a total error of 0.003189, achieving a lower total error than global scale calibration alone. The results demonstrate that range-aware residual correction improves metric distance estimation beyond global scale calibration. Code and materials are available at https://github.com/lht-research/rasr-pairuav.
Read the full abstract ▾
All authors Hongtao Liang, Xinyu Shao, Chenxu Wang, Yiyao Wan, Jiahuan Ji, Fangwei Ye, Fuhui Zhou, Qihui Wu
Computer Vision · Cross-listed from eess.IV
CHM-Net: Center Heatmap-driven Macro-Micro Modeling Network for MRI-based Microbial Density Stratification
Jiaming Liang, Haolin Chen, Tingting Li et al. (+7 more)
Microbial density is clinically important for tumor assessment and treatment decision-making, and recent advances in deep learning suggest that it can be non-invasively inferred from
multimodal MRI. In this work, MRI-based Microbial Density Stratification (MRI-MDS) is first investigated as a patient-level representation learning task, and Center Heatmap-driven Macro-micro modeling Network (CHM-Net) is introduced for this task. CHM-Net first establishes the link between imaging phenotypes and microbial states through center heatmap-guided small-lesion response localization. Building upon this, it constructs patient-level macro-micro evidence from localized heatmap responses for microbial density prediction. Experiments on the novel GBNPC 2026
dataset constructed for MRI-MDS demonstrate the effectiveness of CHM-Net, achieving superior performance over representative baselines with a 12.06% absolute ACC gain over the strongest competing result. Additionally, auxiliary validation on two 3D medical image
datasets further verifies its robustness across volumetric medical image classification scenarios. The project is available at https://anonymous.4open.science/r/CHM-Net-942E/.
Read the full abstract ▾
All authors Jiaming Liang, Haolin Chen, Tingting Li, Bowen Yu, Qianyan Long, Tinghe Zhang, Xi Zhong, Xiaowei Hu, Xiaoqi Sheng, Hongmin Cai
Computer Vision
Detangled: A Framework for Creating, Editing, and Inferencing Feature Rich Hair Strands
Sarah Jobalia, Yitong Deng, Carolyn Smith et al. (+1 more)
We present a framework for understanding and generating feature rich hair strands. Drawing upon both scientific and cultural expertise, we define strand texture as the various distinctive patterns (curling, switchbacks, twist, etc.) that are formed by forces internal to a hair strand. We begin by proposing a novel five-dimensional parameter space, intended to be a bijection with naturally occurring hair strand textures. This encoding is both qualitatively accessible, allowing users to readily locate their own hair in the parameter space, and quantitatively precise, allowing the generation of individual strands from texture inputs. Importantly, strand texture should be independent from the overall strand direction. In order to disentangle strand texture from the overall strand direction, we identify centerline geometry and use it to map strands into a canonical space (a strand texture space). We construct centerlines using a novel method that cleanly distills complex hair grooms, separating the strand texture from the overall style (parameterized by style guides). We enable the creation of new strands conforming to our parametric description of texture via a generative artificial intelligence approach supervised by a separate neural network trained to label candidate strands according to our five-parameter description. The ability to create new strands conforming to any desired texture enables groom editing using either texture transfer or user-provided inputs. We demonstrate results on a variety of hair types.
Read the full abstract ▾
All authors Sarah Jobalia, Yitong Deng, Carolyn Smith, Ronald Fedkiw
Computer Vision
PanoWorld: Real-World Panoramic Generation
Haoyuan Li, Dizhe Zhang, Yuemei Zhou et al. (+7 more)
In this work, we aim to address the challenge of long-range
memory in panoramic world models by exploiting the rotation-equivariant property of omnidirectional representations, where rotation can be treated as an implicit
geometric transformation.Building on this insight, we propose PanoWorld, which simplifies camera trajectories into translations via fixed headings for both current-action modeling and long-range
memory through Dense Panoramic Ray-Conditioning (DPRC) and Geometry-aware
Memory Augmentation (GMA).Then, a three-stage training pipeline is introduced to progressively optimize each component. To better evaluate physical consistency under large-scale spatial variations and diverse illumination conditions, where existing
datasets are relatively stable, we construct World360, a large-scale
dataset consisting of both real-world
video clips collected via panoramic unmanned aerial vehicles and high-quality simulated clips generated by AirSim360.Extensive experiments on World360 demonstrate the effectiveness of PanoWorld, outperforming alternative methods by a large margin.Our models, training code, and
dataset will be publicly available. More information can be found on our project page: https://lihaoy-ux.github.io/panoworld-page/.
Read the full abstract ▾
All authors Haoyuan Li, Dizhe Zhang, Yuemei Zhou, Xiangkai Zhang, Haoran Feng, Xiaofan Lin, Wenjie Jiang, Bo Du, Ming-Hsuan Yang, Lu Qi
Computer Vision
Scalable Visual Pretraining for Language Intelligence
Yiming Zhang, Zhonghan Zhao, Wenwei Zhang et al. (+14 more)
The rapid progress of large foundation models has been driven predominantly by pretraining on large-scale text corpora. However, many forms of knowledge are conveyed through visual representations, where figures, typeset equations, and page layouts carry rich information that cannot be faithfully or completely captured by text alone. Yet current pretraining approaches discard these visual cues by converting visually rich sources, such as documents and web pages, into plain text for learning
language intelligence. This paper challenges the default assumption that
language models must be trained on text-only representations and shows that Visual Pretraining is a scalable learner for foundation model intelligence. To this end, we conduct a systematic study of unsupervised visual pretraining paradigms that directly leverage visual documents without text extraction. Across multiple backbones and
benchmarks, visual pretraining on the same underlying corpora consistently outperforms text-only pretraining, offering an efficient pathway to scalable
language intelligence.
Read the full abstract ▾
All authors Yiming Zhang, Zhonghan Zhao, Wenwei Zhang, Haiteng Zhao, Tianyang Lin, Huanze Tang, Yunhua Zhou, Demin Song, Kuikun Liu, Haochen Ye, Haian Huang, Yuzhe Gu, Haijun Lv, Qipeng Guo, Bin Liu, Gaoang Wang, Kai Chen
Computer Vision
OpenLongTail: Generative Scaling of Long-Tail Driving Data
Lulin Liu, Nuo Chen, Yan Wang et al. (+15 more)
Scaling robust driving policies is fundamentally bottlenecked by the scarcity of edge cases in curated
datasets. While the real world continuously captures these critical events, such long-tail events remain underutilized when collected from heterogeneous sources. Specifically, diverse but valuable in-the-wild long-tail
videos lack the full view coverage required for training policy models, often missing multi-view poses or originating solely from
monocular dash cameras. This modality gap prevents these ubiquitous observations from being converted into scalable training data for long-tail generalization. We introduce OpenLongTail, an open-source generative data engine for scaling autonomous driving policies under long-tail events. To transform heterogeneous data sources into view-aligned and temporally coherent multi-view assets that are useful for policy learning, we develop a pose-informed extrapolative view synthesis pipeline that generates the missing views. We further enhance cross-view consistency and the temporal
alignment for the newly generated views by injecting Plücker ray geometry into the scalable generation engine. By synthesizing heterogeneous long-tail data, we observe a significant improvement in closed-loop driving robustness in handling long-tail events. By measuring the extrapolative view synthesis and pose metrics, we validate the effectiveness of OpenLongTail in visual fidelity, cross-view consistency, and ego-trajectory recovery.
Read the full abstract ▾
All authors Lulin Liu, Nuo Chen, Yan Wang, Bangya Liu, Wenyan Cong, Hezhen Hu, Boris Ivanovic, Hao Wang, Ziyao Zeng, Xinyu Gong, Yang Zhou, Zixiang Xiong, Dilin Wang, Zhangyang Wang, Weisong Shi, Ruohan Zhang, Marco Pavone, Zhiwen Fan
Computer Vision
Evolution of Accuracy and Visual-Cognitive Errors in a Decade of Vision-Language AI Models
Shravan Murlidaran and Miguel P. Eckstein
Vision language models (VLMs) have made remarkable progress in visual
reasoning during the last decade. Most evaluations have used simple scenes (MS-COCO) that do not showcase complex human interactions or behaviors, only a handful of non-curated human descriptions as a
benchmark, and have not focused on understanding the model's error types. Here, we introduce the Complex Social Behavior (CSB)
dataset, containing 100 images depicting complex social interactions/behaviors. We analyze the progression of scene descriptions over a decade (2017-2025) of VLMs (four pre-
Multimodal Large
Language Models, MLLMs, and five MLLMs). We evaluate the accuracy of the models and 20 human descriptions relative to a gold standard on the CSB
dataset and on a sample from MS-COCO. We analyzed five visual-cognitive error types: object detection, recognition,
hallucination, scene understanding, and spatial dependence. The CSB
dataset showed a more pronounced improvement than MS-COCO in scene description accuracy, with pre-MLLMs achieving much lower accuracy than the bottom-ranked human descriptions and MLLMs attaining accuracies similar to the top-ranked human descriptions. We show that MLLMs have eliminated the gap in scene description accuracy between simpler MS-COCO scenes and scenes depicting complex behaviors (CSB). MLLMs have almost eliminated all error types in our tested
datasets, except for occasionally relying on different image regions for scene descriptions than humans do (spatial dependence error). We also show that detection, recognition, and
hallucination errors have the highest impact on scene description accuracy. Together, our findings provide a more thorough evaluation of how visual
language models have advanced over the last decade.
Read the full abstract ▾
Computer Vision
Revisiting Euler-Angle Regression with Kolmogorov-Arnold Networks
Yangting Sun, Zijun Cui and Yufei Zhang
In many real-world systems, including articulated
robots and biomechanical models, rotations are defined in joint space and naturally parameterized by Euler angles with bounded ranges. Yet regressing Euler angles remains challenging, as their discontinuities and singularities often destabilize training. In this work, we revisit Euler-angle regression and show that its effectiveness depends critically on the interaction between rotation representation, regression architecture, and domain constraints. We introduce a new framework that combines range-aware Euler modeling with Kolmogorov-Arnold Networks (KAN), which replace fixed node-wise activations with learnable univariate functions on edges. We further provide theoretical analysis indicating that bounded Euler ranges motivate a near-additive structure in the regression function, which favors the additive functional form of KAN, and we confirm this trend empirically. Extensive experiments on controlled rotation regression, object pose estimation, and robotic and human inverse kinematics demonstrate consistent improvements in accuracy, convergence, and efficiency. The code will be publicly available.
Read the full abstract ▾
Computer Vision
The Effects of Synthetic Data and Label Distribution on Canola Branch Counting
Amirsalar Darvishpour, Mikolaj Cieslak and Adam Runions
Collecting annotated plant images for automated phenotyping is often slow and expensive. Plant models simulating growth and development can generate unlimited synthetic images with exact labels. However, previous work has established that whether incorporating synthetic data improves performance depends on the ratio of synthetic to real images and the label distribution of the synthetic
dataset. To systematically quantify both factors, we train ResNet-18 models on a canola branch-counting task using a calibrated L-system plant model. We vary each factor independently. Synthetic-to-real ratios of 1:5 to 1:22 broadly improve performance; the best ratio (1:7) reduces mean absolute difference by 7.6% over real-only training. For label distribution, a uniform synthetic distribution is strongly suboptimal (abs. diff. of approximately 1.70); interpolating 90% toward the real distribution yields abs. diff. 0.927, whereas Gaussian smoothing of the real label distribution yields the best overall result (abs. diff. 0.912, a 14.7% improvement over real-only). A minimum of 10 synthetic images per label offers a simpler alternative with modest gains, while 100 per label over-corrects and hurts performance.
Read the full abstract ▾
Computer Vision
4DR360: State Reasoning for Joint 3D Detection and Occupancy Prediction in 4D Radar-Camera Full-Scene Perception
Xiaokai Bai, Lianqing Zheng, Runwei Guan et al. (+3 more)
Reliable autonomous driving requires full-scene perception that couples foreground objects with dense semantic layout. Recently, 4D millimeter-wave radar has emerged as a robust and affordable sensor, yet its sparse returns make radar-camera fusion necessary for comprehensive scene understanding. Existing radar-camera methods mainly optimize detection, while dual-task systems usually decode boxes and occupancy with limited interaction. To address this gap and advance radar-based multi-task learning, we propose \method, a 4D radar-camera framework for 360$^\circ$ full-scene perception, which models semantic occupancy as a persistent scene state rather than a terminal output. \method{} follows a cross-modal state
reasoning paradigm, where the occupancy state is modeled and propagated through stages for coarse-to-fine feature aggregation. Specifically, State-guided BEV Enhancement (SBE) strengthens intra-frame BEV representation, while Doppler-guided Temporal Fusion (DTF) preserves state evidence over longer temporal horizons. Beyond the model, we further extend ManTruckScenes with satellite-map-based generated occupancy labels and pair it with OmniHD-Scenes in a unified cross-
dataset detection-and-occupancy protocol. The resulting experiments cover accuracy, robustness, ablation, and efficiency under one radar-camera multi-task evaluation framework. Code and labels will be released upon acceptance.
Read the full abstract ▾
All authors Xiaokai Bai, Lianqing Zheng, Runwei Guan, Songkai Wang, Siyuan Cao, Hui-liang Shen
Computer Vision
Promptable Concept Segmentation from Above: Evaluating SAM 3's Zero-Shot and One-Shot Capabilities in Remote Sensing
Mohammad Dabaja and Turgay Celik
The deployment of large-scale foundation models, such as the Segment Anything Model 3 (SAM 3), promises a transition toward open-vocabulary, training-free computer
vision. However, their capacity to generalize out-of-distribution to the complex, top-down
geometric structures of Earth Observation imagery remains largely unquantified. Driven by SAM 3's performance disparities in highly specialized domains, we present a comprehensive, multi-task empirical evaluation across remote sensing scene classification, object detection, and instance
segmentation under strict zero-shot and one-shot constraints. To achieve this, we introduce a structural adaptation of SAM 3 by repurposing its decoupled binary presence head into a standalone zero-shot classifier. Furthermore, by systematically isolating textual and visual
prompt modalities across five configurations, we explicitly diagnose the
alignment mechanics within the model's
multimodal decoder. Our findings reveal severe cross-modal interference: while visual
prompts successfully align the decoder to complex remote sensing geometry, textual
prompts inject misaligned, ground-level semantic bias, actively degrading coordinate regression. To
benchmark these capabilities without resource-intensive training, we formulate a novel training-free proxy evaluation protocol for Generalized Zero-Shot tasks (scene classification and instance
segmentation). Ultimately, our results demonstrate that SAM 3 avoids the overfitting commonly seen in legacy domain-adapted models, achieving high Harmonic Mean scores in
segmentation tasks. However, it remains fundamentally constrained by sub-pixel resolution limits and overhead semantic blind spots, charting a definitive mandate for parameter-efficient geospatial fine-tuning of its
multimodal decoder.
Read the full abstract ▾
Computer Vision
Wan-Dancer: A Hierarchical Framework for Minute-scale Coherent Music-to-Dance Generation
Mingyang Huang, Peng Zhang, Li Hu et al. (+5 more)
Generating long-duration, high-definition, and rhythmically synchronized dance
videos directly from music remains a significant challenge, primarily due to the temporal constraints of current
diffusion models, which typically fail beyond 20 seconds. Existing approaches, whether they rely on intermediate 3D skeletons or on end-to-end
video synthesis, suffer from temporal drift, identity inconsistency, and repetitive motion patterns when extended to longer horizons. To address these limitations, we propose a novel hierarchical framework for minute-scale coherent music-to-dance generation. Our method decouples the process into global keyframe planning and local temporal refinement, leveraging full-track musical context to ensure long-range coherence. Key innovations include dynamic frame rate adaptation via time-mapped RoPE
embeddings for precise
alignment, an optical-flow-based loss function to enhance motion continuity, and motion-speed control to preserve high-fidelity details during rapid movements. Extensive experiments demonstrate that our framework surpasses the conventional duration barrier, generating stable, 720p/30fps
videos exceeding one minute with superior temporal stability. Furthermore, the model exhibits robust versatility across five distinct dance genres, conditioned on both audio and textual
prompts, establishing a new state-of-the-art in coherent, long-form dance
video synthesis.
Read the full abstract ▾
All authors Mingyang Huang, Peng Zhang, Li Hu, Guangyuan Wang, Ruoshi Zhang, Yi Lu, Gang Cheng, Bang Zhang
Computer Vision
TCLA: Training-Free Class-wise Logit Adaptation for Medical Vision-Language Models
Tianyou Jiang and Ziyu Zhou
Medical
Vision-
Language Models (VLMs) exhibit strong zero-shot performance, yet their effectiveness still declines on out-of-distribution (OOD) data due to domain shifts and class bias inherited from large-scale pretraining. Existing few-shot adaptation methods typically introduce additional trainable components, which can be unstable in extremely low-data regimes (e.g., 1-shot), and lack robustness on different medical data. We present TCLA, a purely training-free few-shot adaptation method for Medical VLMs, which is fast and model-agnostic. TCLA corrects inference logits based on a small set of support samples, boosting pretrained VLMs performance by improving inter-class deconfusion and reducing domain shift. Extensive experiments on nine
datasets across multiple medical imaging modalities including X-ray, Ultrasound, MRI, CT, Histopathology, demonstrate that TCLA consistently improves OOD performance of Medical VLMs and, in most of cases, outperforms existing training-based adaptation methods.
Read the full abstract ▾
Computer Vision
The Count Is There, but Misaligned: Understanding and Correcting Counting Failures in VLMs
Ahmed Oumar El-Shangiti, Abzal Nurgazy, Hilal AlQuabeh et al. (+2 more)
Despite strong performance on many
multimodal tasks,
vision-
language models (VLMs) still struggle with basic object counting. We investigate whether this reflects missing internal knowledge or a gap between internal representations and verbalized outputs. Training simple probes on activations from four VLMs across five counting
datasets reveals that nonlinear probes can reliably detect counting errors, suggesting that VLMs often encode the correct count even when they output the wrong answer. SVCCA analysis shows that probes trained on ground-truth counts and probes trained on model outputs occupy a partially shared activation subspace but read out along misaligned directions. We further validate our findings using a causal steering intervention, proving that strengthening the direction of count-identified probes does improve model counting performance. Motivated by this result, we propose a detector-guided self-correction method that selectively re-
prompts the model only when an internal error detector predicts failure. This simple inference-time intervention improves counting accuracy by up to 15.6 absolute percentage points, without any parameter updates. Our results establish activation-based error probing as both a practical tool for improving VLM counting and a mechanistic lens on the gap between internal knowledge and model outputs.
Read the full abstract ▾
All authors Ahmed Oumar El-Shangiti, Abzal Nurgazy, Hilal AlQuabeh, Nikolai Rozanov, Kentaro Inui
Computer Vision
ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts
Jiawen Li, Tian Guan, Huijuan Shi et al. (+5 more)
Foundation models are reshaping computational pathology, yet their capabilities remain shaped by pretraining objectives, data sources, and spatial scales, fragmenting complementary expertise across separate backbones. Here we present ALICE, a unified foundation model trained through multi-stage agglomerative
distillation that sequentially distills eight
vision-only,
vision-
language, and slide-level teacher models into dedicated modules of a single backbone. ALICE is pretrained on 24,985,184 tile-level pathology images and 155,604 high-resolution images, and evaluated across 21 task scenarios, 96 downstream tasks, and 48 data sources, spanning region-of-interest tissue analysis,
vision-
language multimodal evaluation, and whole-slide clinical assessment. In all three evaluation settings, ALICE achieved the best average rank among task-matched pathology foundation models. These results demonstrate that agglomerative
distillation can consolidate complementary capabilities from specialized models into a unified backbone for broad computational pathology applications. The model is available at https://github.com/WonderLandxD/ALICE.
Read the full abstract ▾
All authors Jiawen Li, Tian Guan, Huijuan Shi, Xitong Ling, Mingxi Fu, Anjia Han, Chao He, Yonghong He
Computer Vision
Seeing is Free, Speaking is Not: Uncovering the True Energy Bottleneck in Edge VLM Inference
Junfei Zhan, Haoxun Shen, Mingang Guo et al. (+2 more)
Vision-
Language Models (VLMs) are the perceptual backbone of
embodied AI, but their energy footprint on edge hardware remains poorly understood. Existing efficiency efforts focus predominantly on reducing visual
tokens, implicitly treating visual processing as the dominant energy cost. We overturn this implicit assumption through the first systematic energy profiling of on-device VLM inference, spanning five models across three architecture families, four input resolutions, and two hardware platforms (NVIDIA RTX 3070 and Jetson Orin NX). Our analysis yields three findings. First, average inference power is a model-intrinsic constant, invariant to input resolution, image complexity, and
prompt type, with less than 5% variation across all conditions. This means that all energy variation across inputs must arise from variation in inference time, not from variation in power draw. Second, each output
token costs 11 to 39x more wall-clock time than each input
token due to the compute-bound and
memory-bound asymmetry between prefill and decode, making output
token count the dominant driver of both latency and energy. Third, image complexity, measured by the number of objects in an image, induces up to 4.1x energy differences at identical resolution. This variation arises not from increased visual processing cost, but from differences in output length. These findings expose a fundamental limitation of visual
token pruning: even removing all visual
tokens saves at most 10% of total energy for fixed-
token models. Across models spanning 1 billion to 8 billion parameters, controlling output length saves up to 97% of total energy, with the energy dominance of decoding growing stronger at larger model scale. In short, the true energy bottleneck in edge VLM inference is not what the model sees, but how much it says. Code is available at https://github.com/Junfei-Z/seeing-is-free.
Read the full abstract ▾
All authors Junfei Zhan, Haoxun Shen, Mingang Guo, Zixuan Huang, Tengjiao He
Computer Vision
DGSfM: Depth-Guided Scale-Aware Global Structure-from-Motion
Sithu Aung, Viktor Kocur, Yaqing Ding et al. (+2 more)
Global Structure-from-Motion (SfM) is an efficient paradigm for recovering camera poses and sparse 3D structure from unordered images. However, its reliance on scale-ambiguous epipolar geometry makes global positioning sensitive to noisy baseline estimates and weak view-graph constraints, while false edges from visually ambiguous pairs can further degrade reconstruction. We propose DGSfM, a depth-aware global SfM pipeline that uses
monocular depth maps as a scalable prior while preserving explicit multi-view optimization. For each image pair, we use a depth-aware relative pose solver to convert scale-ambiguous epipolar constraints into scale-aware relative pose constraints. We further improve robustness through view-graph filtering and depth-consistency-based correspondence
pruning, which suppress false edges and matches that remain plausible under epipolar geometry alone. Finally, global scale averaging and depth-guided pose-point initialization align
monocular depth maps into a common reconstruction scale and provide stable initialization for global positioning and bundle adjustment. Experiments on ETH3D and IMC2021 show that DGSfM consistently improves over strong global SfM baselines across sparse and dense matching front-ends, achieving substantial gains in pose accuracy. Code and project page: https://dgsfm.github.io/.
Read the full abstract ▾
All authors Sithu Aung, Viktor Kocur, Yaqing Ding, Torsten Sattler, Zuzana Kukelova
Computer Vision
What VGGT Knows About Overlap: Probing Geometric Foundation Models for Co-Visibility
Filippo Ziliotto, Luciano Serafini, Lamberto Ballan et al. (+1 more)
A fundamental challenge in 3D reconstruction and robotic localization is co-visibility: determining which image pairs share overlapping visible surfaces, particularly in scenarios with minimal overlap. We demonstrate that VGGT implicitly encodes co-visibility as an emergent behavior: without any supervision for this task, its internal representations exhibit a clear hierarchical structure mirroring that of large
language models, i.e. early layers build a 3D-aware scene representation, while late layers act as dedicated co-visibility reasoners. In particular, we identify layer L17 as a negative anchor that consistently routes non-co-visible pairs for this backbone, regardless of the evaluation setting, providing task-grounded evidence of layer specialization in a geometry-grounded foundation model. Building on this, we introduce Co-VGGT, which freezes VGGT and trains only a lightweight layer-wise mixture-of-experts head (less than 7.5M parameters) to classify co-visibility from RGB alone, treating each layer as a specialized expert whose
geometric abstraction is adaptively weighted per input pair. On the Co-
VisiON benchmark, Co-VGGT surpasses the human annotation baseline and improves over prior work by more than 25% pairwise and 10% multiview. Pairwise predictions are well-calibrated (ECE=0.030), enabling direct use as edge weights in visibility graphs for downstream SfM and SLAM pipelines without post-hoc correction. Code and data are available.
Read the full abstract ▾
All authors Filippo Ziliotto, Luciano Serafini, Lamberto Ballan, Tommaso Campari
Computer Vision
SigLIP-HD by Fine-to-Coarse Supervision
Lihe Yang, Zhen Zhao and Hengshuang Zhao
High-quality visual representation is a long-standing pursuit in computer
vision. In the context of
multimodal LLMs (MLLMs), feeding higher-resolution images can produce more fine-grained visual
tokens. However, it introduces additional computational and design complexity, due to multiple forward passes and post-processing of increased
tokens. Before simply adopting a higher resolution, have we truly unlocked the model's full perception capability at a standard resolution? Therefore, we study an interesting problem: how to achieve fine visual perception under lower cost without larger images. We present SigLIP-HD in this work. The core is a highly simple fine-to-coarse supervision design. We enforce the coarse feature of a mid-resolution image to mimic the fine-grained feature of its high-resolution version. We build this framework on the advanced SigLIP 2 model. Our final model produces better visual
tokens at exactly the same inference budget. It is validated on extensive MLLM
benchmarks and consistently delivers stronger results than our baseline model, especially on OCR-related tasks.
Read the full abstract ▾
Computer Vision
Decoupling Language Guidance from Backbones for Text-Guided Medical Segmentation
Yungeng Liu, Xuanzi Fang, Haijin Zeng et al. (+2 more)
Text-guided medical image
segmentation leverages clinical semantics to improve lesion delineation, yet many existing models bind cross-modal fusion, supervision, and decoder design into a task-specific architecture. Such tight coupling makes it difficult to reuse
language guidance modules across heterogeneous
vision and text backbones, and often requires redesigning the network when the encoder pair changes. This paper presents BTHA, a backbone-transferable hierarchical adapter framework for text-guided medical image
segmentation. BTHA is built around a stable feature-level interface: given multi-scale visual features and a text representation, it injects semantic guidance through shape-preserving adapters while maintaining the decoder-side tensor contract. To make this interface effective, we introduce a Hierarchical Coarse-to-Fine Supervision Strategy that decomposes learning into global image-text
alignment, multi-scale auxiliary localization, and boundary-aware final mask refinement. We further design a Scale-Adaptive Gated Semantic Guidance (SAGSG) adapter, where resolution-specific gates adaptively control textual injection and channel recalibration suppresses redundant cross-modal responses. Evaluations across diverse
vision and text backbones show that the same adapter and supervision design remains effective across convolutional and
transformer-based visual encoders as well as different
language encoders. Experiments on four public
datasets further demonstrate that BTHA improves strong text-guided baselines with modest computational overhead.
Read the full abstract ▾
All authors Yungeng Liu, Xuanzi Fang, Haijin Zeng, Qi Dai, Yongyong Chen
Computer Vision
Foveation-Guided Dynamic Token Selection for Robust and Efficient Vision Transformers
Ibrahim Batuhan Akkaya, Kishaan Jeeveswaran, Bahram Zonooz et al. (+1 more)
The human visual system (HVS) employs foveated sampling and eye movements to achieve efficient perception, conserving both metabolic energy and computational resources. Drawing inspiration from this robustness and adaptability, we introduce the Foveated Dynamic
Transformer (FDT), a foveation-guided dynamic
token-selection architecture that integrates these mechanisms into a
vision transformer framework. The FDT exhibits strong resilience to various types of noise and
adversarial attacks, despite not being explicitly trained for such challenges. This inherent robustness is achieved through the use of fixation and foveation modules: the fixation module identifies fixation points to filter out irrelevant information, while the foveation module generates foveated
embeddings with multi-scale information. At the 50% fixation-budget setting, FDT achieves higher accuracy than DeiT-S (81.9% vs. 80.9%) while reducing multiply-accumulate operations by 34.57%, highlighting one operating point on its accuracy-efficiency trade-off. These attributes position FDT as an HVS-inspired step toward artificial neural networks that combine adaptive computation with improved resilience.
Read the full abstract ▾
All authors Ibrahim Batuhan Akkaya, Kishaan Jeeveswaran, Bahram Zonooz, Elahe Arani
Computer Vision
Hydra++: Real-Time Hierarchical 3D Scene Graph Construction With Object-Level Shape Estimation
Hyungtae Lim, Nathan Hughes, Xihang Yu et al. (+5 more)
3D scene graphs provide a hierarchical abstraction of environments by encoding spatial entities, such as objects and places, and their relationships. However, existing scene graph systems model object geometry coarsely, relying on partial point clouds or class-level CAD templates, which limits instance-specific shape detail. This paper presents Hydra++, a system-level investigation into how learning-based object shape
estimators can be integrated into a hierarchical 3D scene graph pipeline. Hydra++ incorporates category-agnostic shape estimation and a reprojection-mask consistency check to reject degenerate predictions from partial observations or imprecise
segmentation. In its default CRISP-based configuration, Hydra++ performs online scene graph construction; slower
estimators such as SAM3D are evaluated as modular alternatives to demonstrate generalization-latency trade-offs. Furthermore, to address the challenges of sparse and noisy depth measurements in outdoor environments, Hydra++ supports a hybrid
LiDAR-camera configuration for large-scale operation, improving scene-level reconstruction quality. Experiments in both simulation and real-world outdoor campus scenarios demonstrate that Hydra++ improves object- and scene-level reconstruction quality. Project page is available at https://hydra-plusplus.github.io/.
Read the full abstract ▾
All authors Hyungtae Lim, Nathan Hughes, Xihang Yu, Ruihan Xu, Yun Chang, Jingnan Shi, Rajat Talak, Luca Carlone
Computer Vision
Robustifying Vision-Language Models via Test-Time Prompt Adaptation
Xingyu Zhu, Huanshen Wu, Shuo Wang et al. (+4 more)
Pre-trained
Vision-
Language Models (VLMs) such as CLIP achieve strong zero-shot generalization, but their performance degrades sharply under
adversarial perturbations. Existing test-time adaptation methods typically rely on sample-level confidence heuristics, overlooking the intrinsic distributional structure of the data. This sample-centric approach limits robustness, as it fails to distinguish confident
adversarial mispredictions from true semantic consistency. In this work, we observe that
adversarial distortion is structurally brittle: while holistic representations are corrupted, semantic integrity is often preserved in the distribution of augmented views. Motivated by this insight, we propose RITA, a Robust test-tIme
prompt-TAdaptation framework that shifts from sample-level estimates to distribution-level
alignment. Specifically, RITA employs optimal transport to align the distribution of augmented visual features with textual prototypes, mitigating
adversarial outliers and rectifying cross-modal semantic misalignment. Furthermore, we introduce a dynamic cache to progressively accumulate reliable cues from the test stream for online refinement. Extensive experiments demonstrate that RITA significantly improves
adversarial robustness without compromising clean accuracy.
Read the full abstract ▾
All authors Xingyu Zhu, Huanshen Wu, Shuo Wang, Beier Zhu, Jiannan Ge, Jiaheng Zhang, Long Chen
Computer Vision
Parameter-Efficient Vision-Language Adaptation with Continuous Metadata Conditioning for Animal Re-Identification
Anil Osman Tur, Tonje Knutsen Sordalen, Kim Tallaksen Halvorsen et al. (+1 more)
Long-term animal re-identification (ReID) must remain robust to gradual morphological
evolution and seasonal appearance shifts. Although recent
vision-
language models provide strong pretrained visual representations, adapting them to longitudinal ecological settings remains challenging, particularly under identity and temporal distribution shifts. We present a parameter-efficient CLIP adaptation framework for animal ReID and introduce a continuous metadata-conditioning mechanism that incorporates numerical attributes directly into the
prompt representation during training. While low-rank visual adaptation,
prompt-based supervision, and cross-modal
alignment provide the adaptation framework, the proposed metadata-conditioning strategy constitutes the primary methodological contribution. By preserving the continuous structure of numerical metadata rather than discretizing it into textual categories, the proposed approach enables smooth modulation of the
embedding space during training while maintaining a purely visual inference pipeline. Experiments on a seven-year longitudinal fish
dataset and multiple wildlife
benchmarks demonstrate improved performance under closed-set, open-set, and time-aware evaluation protocols. The results demonstrate that continuous metadata conditioning improves robustness to longitudinal appearance variation and temporal distribution shifts, while parameter-efficient adaptation enables a purely visual inference pipeline without requiring metadata at test time. Code and evaluation splits can be found at: https://github.com/AnilOsmanTur/MetaPrompt-ReID.
Read the full abstract ▾
All authors Anil Osman Tur, Tonje Knutsen Sordalen, Kim Tallaksen Halvorsen, Cigdem Beyan