Computer Vision
SkyVLaM: Multimodal Large Language Model for UAV Video Understanding in Remote Sensing
Kaiwen Jing, Ruixu Jia, Bingyao Li et al. (+3 more)
Recent advances in
Multimodal Large
Language Models (MLLMs) have significantly improved remote sensing (RS)
multimodal understanding.
Language-conditioned
segmentation is crucial for fine-grained target understanding in Unmanned Aerial Vehicle (
UAV)
videos. However, this task remains challenging due to the prevalence of small, visually ambiguous targets and dynamic aerial perspectives. In this paper, we propose SkyVLaM, a
multimodal large
language model for
UAV video understanding. SkyVLaM constructs sparse
tokens directly from patch-level
video representations through a temporal basis perceiver, regularizes the sparse basis to encourage complementary temporal cues, and adaptively selects a temporally coherent dense segment for high-resolution inspection. The resulting sparse and dense
tokens are jointly processed by a large
language model for query-conditioned
segmentation. We further build SkyVid, consisting of SkyVid-VGCG and SkyVid-RVOS for
video grounded conversation generation and referring
video object
segmentation, respectively. SkyVid contains 101
videos, 33.6K frames, and 1.53M pixel-level object instances. Experiments show that SkyVLaM provides a more effective allocation of the visual
token budget and improves
language-conditioned
video segmentation in
UAV scenarios.
Read the full abstract ▾
All authors Kaiwen Jing, Ruixu Jia, Bingyao Li, Ruizhe Ou, Ming Wu, Chuang Zhang
Computer Vision
STAR: Skeletal Token Alignment and Rearrangement for Interaction Recognition
Yuhang Wen, Mengyuan Liu, Zixuan Tang et al. (+3 more)
Understanding physical human-
robot and human-human interactions is a challenging yet emerging topic in 3D
vision. While most existing methods rely on skeleton sequences–effective in low-light and
privacy-sensitive environment–they face two major challenges: 1) learning and effectively exploiting interaction cues from skeletal data, and 2) compensating for the lack of visual information absent in skeletons alone. To address these challenges, we propose skeletal
token alignment and rearrangement (
STAR) for human-
robot and human-human interaction recognition. It learns interaction-specific skeleton features and enriches them using visual cues by aligning skeleton and RGB
video representations in a shared latent space. Specifically,
STAR consists of three key components. First, we design a skeleton encoder that captures fine-grained interdependencies using Entity Rearrangement (ER) and Interactive Spatiotemporal
Tokens (ISTs). Second, we present Visual Interaction Encoding that introduces a Focus on Interactions (FoI) strategy to attend to spatiotemporal regions relevant to interactions in RGB
videos. Finally, these representations are aligned via a contrastive learning objective, with a refinement head further refines predictions. During training,
STAR leverages both skeleton and RGB
video data to learn robust, discriminative interaction representations. At inference time, it operates on skeletons alone, retaining visual-informed benefits while preserving skeleton-only efficiency. Extensive experiments on Chico, HARPER, NTU Mutual 11 and 26
datasets consistently validate our approach by demonstrating superior performance over state-of-the-art methods. Our code is publicly available at https://github.com/Necolizer/
STAR.
Read the full abstract ▾
All authors Yuhang Wen, Mengyuan Liu, Zixuan Tang, Junsong Yuan, Sirui Li, Beichen Ding
Computer Vision
Understanding From Human Perspective: A Multi-agent System for Interactive Egocentric Medical Image Segmentation
Rongjun Ge, Dongyang Wang, Heng Zhu et al. (+3 more)
Interactive
egocentric medical image
segmentation (IEMIS) plays an important role in smart-glasses-assisted medical image review, segmenting the medical targets a clinician refers to from their
egocentric view. Once it succeeds, the object-level visual evidence it provides strengthens the review and underpins fine-grained analysis and clinical decision-making. However, the instruction and the
video both come from the user's
egocentric perspective, which poses two challenges. (1) Semantic ambiguity leaves the model unable to confirm the user-intended target. (2) Visual variability makes the
segmentation jump from frame to frame. In this paper, we propose EgoMed-
Agent, a multi-
agent system that understands the target from the human perspective through two workflows. (1) The Target Confirmation Workflow grounds the instruction against candidate targets with a reliability score, confirming the target when the grounding is reliable and asking the user to clarify when it is not, thereby confirming the
segmentation target. (2) The Localization-Guided Propagation Workflow couples mask propagation with per-frame target localization, using the localized target to correct the propagated mask whenever the two diverge, so the
segmentation stays on the target across the
egocentric video. Extensive experiments show that EgoMed-
Agent reaches 71.34% average Dice, far above the best text-prompted baseline (11.70%). Our code is available at \href{https://github.com/wdyyyyyy/EgoMed-
Agent}{our project page}.
Read the full abstract ▾
All authors Rongjun Ge, Dongyang Wang, Heng Zhu, Zhirui Li, Yang Chen, Yuting He
Computer Vision
Orthogonal Knowledge Refreshing for Domain-Incremental Object Detection
Aoting Zhang, Dongbao Yang, Chang Liu et al. (+3 more)
Domain-incremental object detection (DIOD) requires models to continually adapt to new domains while preserving prior knowledge. Recently, parameter-efficient fine-tuning offers a promising avenue, wherein a pre-trained model is frozen and a small number of learnable parameters are injected for downstream tasks. However, these methods risk overwriting critical past knowledge, triggering inter-domain interference and performance degradation. To address this challenge, we propose Orthogonal Knowledge Refreshing (OKR), a simple yet effective framework for DIOD. OKR incrementally constructs independent domain-specific subspaces via dedicated low-rank branches for each domain, which are seamlessly fused for a holistic decision, enabling conflict-free capacity expansion without domain selection during inference. To minimize knowledge interference during fusion, we present a gradient-based orthogonal refreshing strategy that projects gradient updates of new domains onto the orthogonal complement of the fused historical subspace, supporting continual adaptation without forgetting. Moreover, to mitigate semantic fragmentation across domains, we enforce
topology-aware consistency, aligning the semantic structures of old and new domains. Extensive experiments validate the superiority of OKR, outperforming the best exemplar-free method by significant margins of +5.6% and +6.5% mAP on the Pascal VOC and BDD100K series, respectively.
Read the full abstract ▾
All authors Aoting Zhang, Dongbao Yang, Chang Liu, Xiaopeng Hong, Can Ma, Yu Zhou
Computer Vision
MIS-HCC: Hierarchical Channel Clustering for Efficient Medical Image Segmentation
Bo Zhao, Haoran Yu, Lifei Liu et al. (+5 more)
Medical image
segmentation models require both high accuracy and lightweight design to accommodate real-world medical applications. The deployment of these models on resource-limited medical platforms remains a significant challenge due to their high computational and parameter requirements. Existing
pruning methods for model compression mostly overlook the intrinsic connections and similarity between the internal structures of complex deep neural networks. As a result, compressed models may not effectively retain the basic features of the pretrained network. To solve this problem, we propose a hierarchical clustering compression method for medical image
segmentation models (MIS-HCC). This approach employs hierarchical clustering to partition channels and fuse their parameters efficiently. Specifically, it leverages the Wasserstein distance to represent similarity of channels within layers of pre-trained network, forming a similarity
matrix that guides the clustering process. Channels within each cluster are then fused to produce a compressed network. Experimental results on three medical image
datasets application demonstrate that MIS-HCC outperforms the state-of-the-art methods in both accuracy and compression efficiency, offering an effective solution for deploying medical image
segmentation models on resource-limited medical platforms.
Read the full abstract ▾
All authors Bo Zhao, Haoran Yu, Lifei Liu, Zongcheng Chu, Yining Liu, Chang Liu, Szu-Yu Chen, Zequn Xie
Computer Vision · Cross-listed from cs.RO
From Perception to Assistance: Open-Vocabulary Shared Autonomy for Robotic Manipulation
Murilo Vinicius da Silva, Ricardo V. Godoy, Juliano Negri et al. (+3 more)
Teleoperating a robotic manipulator in industrial environments demands precision that camera-based interfaces alone struggle to deliver. The operator must align the end-effector with a target in clutter, under limited depth perception, and without colliding with the surrounding structures. This paper presents a shared-autonomy framework that assists the operator throughout this process. A single RGB-D camera captures the operator's arm motion and hand gestures without wearables, fiducials, or a calibration stage. The intended target is specified by a free-form text
prompt, grounded by a
vision-
language model in the
robot's gripper camera, and tracked across its onboard cameras by a promptable
video-
segmentation model, resulting in a grasp frame continuously separated from the obstacle map. Every commanded motion is executed by a GPU-accelerated model-predictive controller that enforces self- and environment-collision avoidance against an online volumetric reconstruction, while a potential field corrects the operator's reference toward the grounded target during the final approach. An autonomous mode can be gesture-triggered to complete the grasp on the same target without a separate perception pipeline. The framework is validated on a quadruped mobile manipulator. The interface achieves a positional RMSE of 59 mm relative to motion-capture ground truth, and the controller keeps the arm at least 18 cm from obstacles while the operator deliberately commands the arm into them by 6 cm. In an industrial valve manipulation and a pick-and-place task, the full framework succeeded in all trials, while ablating either the collision or the assistance module produced failures through complementary mechanisms, and autonomous execution succeeded in four of five trials per task.
Read the full abstract ▾
All authors Murilo Vinicius da Silva, Ricardo V. Godoy, Juliano Negri, Gustavo J. G. Lahr, Ranulfo Bezerra, Marcelo Becker
Computer Vision
PACE: Polar Axis-Conditioned Estimation for PairUAV Relative Localization
Ze Rong
PairUAV relative localization maps two
UAV images to a polar
navigation command. Although heading and range share the same pairwise pose context, treating them as homogeneous coordinates forces both outputs to use the same decoder evidence and optimization state. Controlled readout probes reveal a different structure: the two axes favor different decoder-depth combinations, their best checkpoints disagree on 80.8% of a validation trajectory, and range errors exhibit a distinct high-error tail. We introduce method, Polar Axis-Conditioned Estimation, which retains a shared Reloc3r-style pair representation while assigning axis-specific readout interfaces. Heading uses mid/late relational evidence, whereas range remains attached to a direct late metric path. On the official hidden test, the strongest released raw predictor scores 0.002460; the complementary PAAER predictor scores 0.002514 with a slightly lower angle error. Deterministic challenge packaging, reported separately from learned estimation, yields the final score of 0.001874. Code, checkpoints, predictions, and reconstruction tools are available at https://github.com/zerong7777-boop/PairUAV-PACE.
Read the full abstract ▾
Computer Vision
VecFontLLM: Anchor-Guided Direct Synthesis of Chinese Vector Fonts
Hao Yuan, Yuxuan Luo, Xing Chen et al. (+1 more)
Direct generation of Chinese vector fonts is a challenging and ongoing problem. A Chinese vector glyph contains complex component structure, anchor layout, and Bézier curve details, which work at different scales, but a standard vector sequence writes them together in one long sequence, making the task of vector font synthesis challenging. Existing direct vector generators often fail on complex characters, while raster-domain methods must vectorize the synthesized glyph images afterward. To address the above-mentioned problem, this paper proposes VecFontLLM, an anchor-guided
multimodal large
language model for direct few-shot synthesis of Chinese vector fonts. Our key idea is to generate vector glyphs through anchors rather than a standard vector sequence. Specifically, the proposed VecFontLLM first predicts and refines an anchor scaffold that fixes the coarse layout of components and contours, and then completes Bézier control points to recover local
curvature and style. At test time, a confidence-guided generation chain samples multiple component candidates and continues synthesis from the highest-confidence one, improving stability for complex glyphs. This work demonstrates, for the first time, high-quality few-shot synthesis of complex Chinese vector glyphs directly in the vector domain, without raster generation or vectorization. Experiments on several Chinese font
datasets show substantial improvements over existing vector font synthesis methods, competitive glyph rendering quality against raster-domain baselines, and vector command distributions close to real fonts.
Read the full abstract ▾
All authors Hao Yuan, Yuxuan Luo, Xing Chen, Zhouhui Lian
Computer Vision
Semantic Context Matters: Analysis of Color Names Across Domains
Adilet Yerkin, Elnara Kadyrgali, Malika Ziyada et al. (+5 more)
Color naming is influenced not only by physical color values but also by the semantic context in which colors are used. This paper investigates context-dependent color naming by mapping color-name
datasets from Cosmetics, Crayola, and Car-color vocabularies onto the 86 fuzzy color categories of the COLIBRI color model. Contextual variation is analyzed using category coverage, Shannon
entropy, and maximum lift. The results show that the three contexts occupy the COLIBRI color space differently: Cosmetics covers 48 of 86 fuzzy categories, Crayola covers 50, and Car colors cover 40. The results demonstrated that Crayola provides the broadest and most balanced use of the fuzzy color space, Cosmetics is mainly concentrated around warm-tone regions, and Car colors are more specialized around blue and achromatic regions. These findings show that color naming cannot be fully explained by numerical color similarity alone and that semantic context plays an important role in human color interpretation. The proposed framework supports the development of context-aware color models for design analytics, product search, recommendation systems, and human-centered artificial intelligence.
Read the full abstract ▾
All authors Adilet Yerkin, Elnara Kadyrgali, Malika Ziyada, Nuray Toganas, Muragul Muratbekova, Ayan Igali, Aruzhan Sabitkyzy, Pakizar Shamoi
Computer Vision
Induce to Empower: Improving Lightweight Baselines via Foundation Model Induction for Generalized Polyp Segmentation
Shivanshu Agnihotri, Snehashis Majhi, Deepak Ranjan Nayak et al. (+2 more)
Automated polyp
segmentation in colonoscopy continues to pose challenges due to substantial appearance variations and indistinct polyp boundaries. Although emerging foundation models (FMs) such as DINOv2, SAM, and OneFormer, demonstrate remarkable generalization capabilities, their direct transfer to the polyp
segmentation task and deployment in real-time clinical settings are difficult due to lack of large-scale labeled data and high computational demands. In addition, adopting multiple FMs together raises concerns, even though they encode complementary semantic and structural information. While lightweight models, including U-Net, PraNet and U-Net++, are computationally efficient, they often struggle to generalize across
datasets due to limited representational capacity. To address this gap, we propose Lite-Polyp Inductor (Lite-Pi), a novel foundation model induction framework that significantly enhances lightweight polyp
segmentation baselines. Our proposed framework generates FM-specific prototype representations and aligns them semantically with the corresponding foundation model priors through reconstruction-based supervision. Subsequently,
transformer-based fusion is introduced to highlight the polyp relevant representations, including salient boundary information, while preserving complementary semantic cues. Extensive experiments across five polyp
segmentation benchmark datasets demonstrate that Lite-{\pi} significantly improves lightweight baselines, achieving superior generalization performance with minimal computational overhead and thereby, offering a practical solution for generalized polyp
segmentation. Our code is available at GitHub. https://github.com/lostinrepo/Lite-Pi
Read the full abstract ▾
All authors Shivanshu Agnihotri, Snehashis Majhi, Deepak Ranjan Nayak, Dwarikanath Mahapatra, Debesh Jha
Computer Vision
Cross-Coordinate Correspondence Pruning for Image-to-Point Cloud Registration
Xin Liu, Rong Qin, Huipeng Lin et al. (+5 more)
Recent detection-free approaches have shown significant efficacy in image-to-point cloud (I2P) registration by employing a coarse-to-fine matching pipeline. In the coarse stage, down-sampled image features and voxelized point cloud features are typically fused to establish initial coarse correspondences for subsequent refinement. However, existing methods largely overlook the critical role of point cloud density, which fundamentally dictates the quality of coarse correspondences and the final registration results. Specifically, excessively sparse point clouds lead to an insufficient number of inliers, while overly dense ones often introduce a high outlier ratio. Consequently, this creates an inherent density trade-off, thereby significantly limiting the registration accuracy of current approaches. For mitigating this trade-off, we propose a novel Cross-Coordinate Correspondences
Pruning (CCP) strategy to acquire sufficient inliers while ensuring a low outlier ratio. To minimize interference from inter-modal coordinate discrepancies, we first project cross-coordinate coarse correspondences to the 2D image coordinate system for spatial unification. Subsequently, a lightweight
pruning network is responsible for predicting the inlier confidences, which are used to filter coarse outliers, from coordinate
geometric and modal feature dimensions. To maximize inlier recall, we further design a Multi-Density Point Ensemble (MDPE) strategy that consolidates and deduplicates pruned coarse correspondences across varying point cloud densities. Our method achieves a significant performance improvement, surpassing existing state-of-the-art methods by at least 8.6% in Registration Recall across various
benchmarks.
Read the full abstract ▾
All authors Xin Liu, Rong Qin, Huipeng Lin, Leizhi Shu, Jin Wu, Chi-Man Vong, Liang Lin, Jufeng Yang
Computer Vision
Md. Ariful Islam, Md Tanvirul Islam, Md. Maruf Hossain Miru et al. (+1 more)
Clickbait, where
video titles and thumbnails exaggerate or misrepresent content, reduces user trust, wastes
attention, and promotes misinformation on
video-sharing platforms. Detecting Bengali clickbait remains challenging because publicly available
multimodal datasets are limited. To address this gap, we introduce BanClickThumb, a curated
dataset of 7,147 Bengali YouTube thumbnail-title pairs from five content domains, annotated by ten annotators with high agreement (Cohen's Kappa: 0.83-0.93). Using this
dataset, we
benchmark text-only, image-only, and
multimodal approaches. Among unimodal models, BanClickTextFormer (XLM-RoBERTa) achieves 0.82 accuracy, while BanClickImageFormer (SwiftFormer) reaches 0.68. Our proposed
multimodal model, BanClickFusionFormer, combines ViT and XLM-RoBERTa through intermediate fusion and achieves the best accuracy of 0.84. Error analysis shows that dense thumbnail text, figurative
language, and culturally specific slang remain challenging. Our findings demonstrate the effectiveness of
multimodal fusion for Bengali clickbait detection and provide a publicly available
benchmark to support future research on low-resource
multimodal content analysis.
Read the full abstract ▾
All authors Md. Ariful Islam, Md Tanvirul Islam, Md. Maruf Hossain Miru, Md Khalid Syfullah
Computer Vision
VLA-ReID: Video-Level Association for Re-Identification in Multi-Object Tracking with Highly Similar Objects
Yanrong Qin, Xiaoyan Cao and Yao Yao
Multi-object tracking (MOT) aims to localize multiple objects in
videos while preserving their identities over time. Long-term identity preservation remains difficult when objects are small, densely distributed, and highly similar in appearance, as in bee swarm scenes. Existing trackers rely on re-identification (re-ID) models trained through single-instance assignment (instance-level querying). At inference, however, MOT requires global assignment between multiple trajectories and detections, corresponding to
video-level querying. This training-inference mismatch can cause identity switches among visually similar objects. Existing approaches also often require substantial additional annotations to enhance appearance discrimination. We propose
Video-Level Association re-ID (VLA-ReID), which reformulates re-ID as
video-level association modeling. It uses aggregated historical trajectory features as queries and all current-frame detections as candidates, enabling direct optimization of their global association at each frame. In addition, Frame-Common Appearance Estimation (FCAE) estimates a common appearance direction from current-frame detections, while Common-Appearance Suppression (CAS) removes the corresponding component along this direction from trajectory and detection features. This amplifies discriminative differences among highly similar objects without additional annotations. Experiments on BEE24 show that VLA-ReID improves HOTA by 1.1, MOTA by 0.3, AssR by 2.6, AssA by 0.7, and IDF1 by 0.8 over state-of-the-art trackers, while reducing identity switches by 28%. These results demonstrate the effectiveness of
video-level re-ID modeling for appearance-based association in MOT.
Read the full abstract ▾
Computer Vision
Noise-Robust Box-Supervised Infrared Small Target Detection via Physics-Inspired Soft Label Optimization
Xizhe Zhang, Fan Shi, Mianzhao Wang et al. (+3 more)
Infrared small target detection (IRSTD) commonly relies on pixel-level mask supervision. Such annotations, however, are costly and inherently uncertain because infrared targets have blurred boundaries and weak textures. We formulate box-supervised IRSTD as a problem distinct from generic box-to-mask
segmentation and point-supervised IRSTD. Its central challenge is to construct stable pixel-level soft supervision from highly contaminated boxes. To this end, we propose Hotspot-Anchored Label Optimization (
HALO).
HALO localizes a radiometric anchor inside each box under local background-statistics constraints, then synthesizes a Physically Anchored Gaussian (PAG) soft label around the anchor. This turns noisy box supervision into continuous, pixel-level soft labels. The entire process is performed offline before training, remains decoupled from the detector backbone, and requires no online label updates. Experiments on public
datasets show that
HALO is competitive with representative box-supervised methods under standard tight boxes. Under looser or shifted box annotations that better approximate real scenarios,
HALO is substantially more robust while remaining consistent across backbones. We further introduce a contamination-aware operating-regime analysis to characterize the effective boundary of this class of methods and reveal how intrinsic signal-to-clutter ratio relates to performance.
Read the full abstract ▾
All authors Xizhe Zhang, Fan Shi, Mianzhao Wang, Jiangpeng Zheng, Xu Cheng, Shengyong Chen
Computer Vision
STBridge: Shared-Target Alignment for Bridging Understanding and Generation in UMMs
Ye Wang, Hongjun Wang, Hao Fang et al. (+7 more)
Unified
multimodal models (UMMs) aim to integrate visual understanding and generation within a single architecture, but architectural unification alone does not ensure semantic consistency. A model may describe the intended target correctly while generating an inconsistent edit. This exposes an understanding-generation
alignment gap: linguistic and visual outputs live in different spaces, yet should be governed by the same target semantics. We study this gap in image editing, where an instruction defines a target state that can be both described and visually realized. Given a source image and an edit instruction, we compare a UMM's target caption with its edited image to test whether the two outputs converge on the same result. Our analysis shows that existing UMMs remain weakly aligned, especially for fine-grained entities, attributes, spatial relations, and local details, indicating that semantic unification is not achieved by architecture alone. To bridge this gap, we propose STBridge, a shared-target
alignment framework that connects understanding and generation through a common target state. Here the target caption expresses the desired visual result, while the edited image realizes it visually, replacing separate task-specific paths with a shared information flow from target expression to target realization. STBridge follows an align-then-optimize strategy: supervised fine-tuning first establishes the shared-target channel, and sequential reinforcement learning further refines target-centered coordination. Across visual understanding, image generation, and image editing
benchmarks, STBridge consistently improves over the initialization model.
Alignment analysis confirms that STBridge narrows the gap between what the model describes and what it generates, demonstrating shared-target
alignment as an effective post-training strategy for bridging understanding and generation in UMMs.
Read the full abstract ▾
All authors Ye Wang, Hongjun Wang, Hao Fang, Tongyuan Bai, Zuwei Long, Peixian Chen, Wei Liu, Weibo Gu, Xing Sun, Rui Ma
Computer Vision
Denoising Models Develop Human-Like Perceptual Illusion Representations Across Architectures
Gautam Ranka and Paras Chopra
Deep neural networks trained on natural images are shown to produce outputs consistent with human observers for brightness illusions. While this phenomenon has been documented across architectures, all evidence, to date, is measured at the output level: restored pixels, decoded trajectories, or classification decisions. Whether these models actually represent illusions internally, and if so where and how, remains unknown. We show that denoising models develop illusion-sensitive representations at specific internal layers, across varied architectures. Specifically, we identify the layers and channels that discriminate illusory from physically matched control regions. We show that the denoising objective is a more important driver of the effect than the architecture. On domain-appropriate stimuli, these activations track a validated psychophysical model of human brightness perception (FLODOG; Spearman $\rho \geq 0.70$) and scale monotonically with parametric illusion strength. Leveraging these findings, we provide causal evidence via channel ablation showing that illusion-sensitive channels specifically and substantially affect the internal signal. Yet injecting these representations into the generation pipeline produces no measurable pixel shift across all tested architectures; we term such representations perceptual phantoms: active in internal processing yet invisible to any output-based evaluation. While related internal-output dissociations have been characterized in
language models, this is the first such characterization for perceptual representations in denoising
vision models.
Read the full abstract ▾
Computer Vision
Learning Emotion from Motion: Kinetic Multi-Stream Skeleton Modeling with Metadata-Conditioned Weak Label Distributions
Sosuke Suzuki, Yijin Wei, Koichiro Kamide et al. (+3 more)
Skeleton-based
emotion recognition from body motion remains challenging because emotional expressions are often characterized by subtle dynamic and relational motion cues, and hard labels may not fully capture ambiguity among related
emotion categories. For the DIEM-A task in the MMAC ACII 2026 Challenge, we propose a multi-branch skeleton-based
emotion recognition framework that combines a 6D rotation-based branch, a part-aware kinetic multi-stream branch, and a metadata-conditioned weak label distribution learning (LDL) branch. The branches are trained independently and fused by a probability-level ensemble at inference time. In 10-fold leave-performer-out cross-validation, the proposed framework improves Accuracy from 0.271 to 0.366 and Macro-F1 from 0.252 to 0.353 over the rotation-based baseline. Explainability ablations show that velocity and bone streams, as well as arm and leg regions, provide important cues for recognizing emotional body motion.
Read the full abstract ▾
All authors Sosuke Suzuki, Yijin Wei, Koichiro Kamide, Ran Dong, Haoran Xie, Chao Zhang
Computer Vision
The generator is the tracker: Multi-object tracking by painting persistent identity colours
Haiyu Yang and Miel Hostens
Multi-object tracking (MOT) is conventionally decomposed into detection followed by association, with object identity maintained as external state: track buffers, motion models, and appearance
embeddings. We ask whether a
video generator can maintain that state in pixels. We fine-tune a 22B text-to-
video diffusion model (LTX-2.3) with a lightweight in-context LoRA to translate an RGB clip into an ID-map clip, a
video in which every person is painted a flat, distinct color that persists over time: same color, same identity. Long
videos are generated as chained windows, where each window is conditioned on the cleaned tail of the previous one. A brief continuation fine-tune teaches the model to extend a given coloring, after which identity flows through the chain with no tracker, no motion model, and no re-identification module. On the DanceTrack test server, our system, to our knowledge the first generative tracker evaluated there and the only entry with no detector and no tracking stack, reaches 40.3 HOTA. This is well below today's specialist state of the art (>=70 HOTA), but with a unique, inverted error profile: its association score (AssA 44.1) exceeds every tracker of the original
benchmark suite while detection remains the sole deficit. Controlled comparisons show the mechanism matters: the same generated windows linked by classical post-hoc association score 2x worse (18.2 HOTA), and frame-to-frame IoU association fragments tracks that the generator's colors keep whole. On 383 mined occlusion events, the generator re-acquires identities after gaps at a 42% conditional rate where appearance-
embedding baselines score zero, including gaps longer than its temporal context, evidence that the generator's color assignment functions as an emergent re-identification signal. We release code, checkpoints, and the full pre-registered experimental log.
Read the full abstract ▾
Computer Vision
DepthART: Scaling Foundation Monocular Depth to Tiny Models
Feng Xue, Wu Chen, Mingshuai Zhao et al. (+7 more)
Recent
geometric foundation models (e.g., Metric3D, Depth Anything and UniDepth) have substantially improved
monocular depth estimation (MDE) in both cross-scene generalization and metric-scale prediction, yet these gains have not translated to tiny models. We bridge this gap with DepthART (Depth Anything Rethought for Tiny Models), which is a compact MDE model for on-device deployment across diverse scenes. We first identify two capacity-driven bottlenecks in tiny models: (i) overfitting to
dataset-specific distribution bias and (ii) unstable metric adaptation under camera shift, where full fine-tuning easily damages transferable geometry. Accordingly, DepthART combines two simple but effective strategies: a bias-resistant data sampling scheme to reduce distribution bias under the same training budget, and a camera-conditioned fine-tuning protocol that freezes the distilled encoder and adjusts metric scale conditioned on intrinsics while better preserving cross-
dataset generalization. Across
datasets, DepthART consistently surpasses previous tiny baselines in both zero-shot generalization and metric accuracy (e.g., zero-shot $\delta_1$=0.964 for DepthART-S on NYUD v2), and in some cases approaches heavy models. We further provide a scalable model family, with DepthART-S reaching 347/245 FPS (strict FP32) on an RTX A6000 at $224^2/448^2$, 102 FPS (TF32) on a Orin NX 8GB, and over 15 FPS (FP32) on a Jetson Nano 4GB.
Read the full abstract ▾
All authors Feng Xue, Wu Chen, Mingshuai Zhao, Guofeng Zhong, Anlong Ming, Haozhe Wang, Dianqiao Lei, Zhaowen Lin, Haiyang Zhang, Nicu Sebe
Computer Vision
HarmoHOI: Harmonizing Appearance and 3D Motion for Multi-view Hand-Object Interaction Synthesis
Lingwei Dang, Juntong Li, Zonghan Li et al. (+5 more)
Hand-Object Interaction (HOI) synthesis is a cornerstone for animation production and
embodied AI. Despite the strong priors of
video foundation models, multi-view consistent HOI synthesis remains challenging due to complex hand motions and occlusions. We present HarmoHOI, a unified
diffusion framework that jointly and harmoniously generates synchronized multi-view HOI
videos and globally aligned 3D point tracks. Our core insight is that robust multi-view consistency fundamentally requires globally aligned 3D geometry and motion. To this end, we propose a Mixture of Multi-view
Diffusion Transformer that co-models RGB
videos and 3D point tracks. By representing point tracks as pseudo-
videos, we align 3D
geometric signals with the 2D latent space of foundation models, thereby minimizing the domain gap and easing adaptation of priors. To further ensure geometry consistency, we introduce Global Motion Aligning
Diffusion, which refines coarse point tracks into metric-scale, globally aligned 3D trajectories. HarmoHOI enables on-the-fly co-
evolution of 2D appearance and 3D motion during denoising. To overcome the scarcity of multi-view HOI data, we employ a hybrid data curriculum learning strategy that successfully transfers generic priors from single-view data to synchronized multi-view generation. Experimental results show that HarmoHOI achieves state-of-the-art performance in visual quality, motion plausibility, and multi-view
geometric consistency. Project page available at https://droliven.github.io/HarmoHOI_project.
Read the full abstract ▾
All authors Lingwei Dang, Juntong Li, Zonghan Li, Hongwen Zhang, Liang An, Wei Min, Yebin Liu, Qingyao Wu
Computer Vision
Autoregressive B-Rep Shape Generation with Parametric Surfaces
Dafei Qin, Rui Xu, Zeyu Shen et al. (+8 more)
Generative CAD modeling has broad design and application potential. Despite significant advances in Boundary Representation (B-Rep) generation, the dominant representation in CAD, existing methods largely depend on uniformly sampled point- or grid-based geometry representations, sacrificing native surface types and parameters and thereby limiting
geometric fidelity and downstream usability. We present ParaCAD, an autoregressive framework for point-cloud-conditioned B-Rep generation that directly operates on native parametric surfaces. ParaCAD introduces a surface-centric tokenization that explicitly encodes each face by its exact surface type and continuous parameters, preserving the intrinsic semantics of CAD geometry. Our model first generates parametric surfaces with constrained UV domains, and then constructs a valid B-Rep by globally intersecting these surfaces to recover edges and vertices. ParaCAD places point-cloud-conditioned generation at the core of B-Rep synthesis, making it practical for user-guided reconstruction and seamless integration into existing 3D generation pipelines. Extensive experiments demonstrate that ParaCAD produces accurate B-Reps with faithful point-cloud
alignment, outperforming point-based baselines in
geometric precision, robustness, watertightness and downstream usability.
Read the full abstract ▾
All authors Dafei Qin, Rui Xu, Zeyu Shen, Kaichun Qiao, Hongyang Lin, Qixuan Zhang, Huaijin Pi, Lan Xu, Jingyi Yu, Wenping Wang, Taku Komura
Computer Vision
ALLUDE: A Unified Evaluation System for Configurable Attacks in Differentiable Environments
Mansi Phute, Alexander Greenhalgh, Matthew Hull et al. (+8 more)
Adversarial attacks against
vision models like object detectors are often evaluated under limited conditions, leaving their performance under-characterized. Bridging simulation and differentiable rendering enables more robust, end-to-end evaluation of these
adversarial attacks, yet there is no easy-to-use, unified system that offers a rich set of customizable configurations for
adversarial attacks across multiple scenes, objects, environmental and lighting conditions, and camera trajectories. We present ALLUDE, which addresses these gaps, offering first-of-its-kind evaluation capabilities across Linux and Windows. We comprehensively demonstrate ALLUDE's evaluation breadth through a two-pronged strategy: (1) using Latin Hypercube Sampling, we draw a representative subset from 5,400 configurations spanning 10 scene-object pairs, 9 weather conditions, 4 optimizers, 5 camera trajectories, and 3 detection models; (2) we stress-test existing attacks (CAMOU, RAUCA, FCA) under diverse weather conditions and continuous camera trajectories, revealing degradation of attack success across every attack, exposing evaluation gaps in prior work. Through ALLUDE's end-to-end differentiable rendering,
adversarial attacks can be optimized against shifting real-world deployment conditions. Our cross-platform code is open source.
Read the full abstract ▾
All authors Mansi Phute, Alexander Greenhalgh, Matthew Hull, Haoran Wang, Alec Helbling, ShengYun Peng, Elliott Faa, Willian Lunardi, Martin Andreoni, Wenke Lee, Duen Horng Chau
Computer Vision
AdvSerial: Physical Adversarial Attacks on Infrastructure-mounted Pedestrian Detectors via Semantic Feature Suppression
Yuanhao Huang, Yilong Ren, Jinlei Wang et al. (+3 more)
AI-based visual perception systems are increasingly deployed in infrastructure surveillance, including roadside monitoring units, highway cameras, and smart-city pedestrian management systems. The security vulnerability of these systems to physical
adversarial attacks poses a direct threat to the reliable operation of transportation infrastructure. We propose AdvSerial, a dynamic 2D–3D joint optimization framework for generating continuous high-angle physical
adversarial patches against pedestrian detectors in infrastructure-based scenarios. We UV-map a boundary-aware quilted texture onto 3D garments, combine 2D digital attacks with 3D sparse- and continuous-frame rendering, and explicitly suppress person-specific semantic features while enforcing temporal continuity. A Feature Smooth Quilting strategy reduces visible patch boundaries and bounds cross-seam feature discontinuities. A serial-frame loss encourages long uninterrupted sequences of detection failures. In physical world experiments, AdvSerial achieves a 74.8% attack success rate on YOLO-v5 and degrades mean detection confidence from 84.30% to 39.38%. Experiments spanning eight detectors with different architectures demonstrate strong transferability. Notably, it achieves an $89.71%$ attack success rate on YOLO-v2 and resists both patch-detection defenses (NapGuard) and 3D-temporal perception (Sparse4D-v3). The results reveal persistent, temporally consistent failure modes under high-angle surveillance, and motivate the design of motion-aware and 3D-aware defenses for security-critical infrastructure deployments.
Read the full abstract ▾
All authors Yuanhao Huang, Yilong Ren, Jinlei Wang, Xuesong Bai, Jinchuan Zhang, Haiyang Yu
Computer Vision · Cross-listed from cs.RO
DROID-ANCHOR: Odometry-Anchored Recurrent Metric Depth Estimation
Yuxuan Chen and Brook Du
Precise metric depth estimation is fundamental for autonomous
robot navigation, yet
monocular systems inherently suffer from scale ambiguity and scale drift. While recent recurrent flow-based SLAM systems have demonstrated state-of-the-art robustness, they remain scale-ambiguous. In this paper, we propose Metric-DROID, an end-to-end recurrent architecture that anchors visual SLAM to physical reality by integrating proprioceptive odometry. Our framework introduces the following innovations: (1) A LSTM Update Operator that encodes high-frequency odometry sequences into spatial feature maps, providing a persistent metric bias for iterative refinement. (2) An Uncertainty-Aware Metric Backend ($BA_{odom}$) that treats odometry as a
geometric anchor with learned heteroscedastic covariance. By regressing a time-varying metric uncertainty $\Sigma_{o}$, our system intelligently balances visual re-projection and metric translation residuals, effectively mitigating the impact of wheel-slip and sensor noise. (3) We further propose a selective residual fine-tuning strategy to preserve pre-trained
geometric priors while enabling zero-shot metric
alignment.
Read the full abstract ▾
Computer Vision
High-Capacity Robust Watermarking Technology for High-Resolution Images
Shaowu Wu, Wei Lu, Qing Qian et al. (+1 more)
Most existing watermarking techniques are primarily designed for low-resolution images, with few methods tailored for high-resolution images. Moreover, the
embedding capacity is often limited to fixed lengths (e.g., 30, 100, 256 bits, etc.), which struggles to meet practical demands. To address these issues, this paper proposes a high-capacity robust watermarking method for high-resolution images, capable of
embedding a watermark of 4 KB (32,768 bits) into images with a resolution of 1024*1024, achieving an
embedding rate of 0.0313 bpp. Specifically, this paper adopts a block-wise strategy to effectively embed the watermark into local regions, enabling the network to train and learn normally even under low-resource conditions. The encoder and decoder structures respectively employ a reversible symmetric architecture with three convolutional and three deconvolutional layers, ensuring consistency in the coupling and decoupling of the watermark and image features. Additionally, the loss function combines global and local losses with weighted contributions. By incorporating constraints on the visual quality and robustness of local block regions, the overall imperceptibility and robustness of the image are further enhanced. Extensive experimental results verify that the proposed method is effective and feasible in high-resolution image scenarios with high-capacity watermarking, while demonstrating strong robustness against various noise attacks.
Read the full abstract ▾
All authors Shaowu Wu, Wei Lu, Qing Qian, Mingsen Deng
Computer Vision
Searching for Task-Specific Vision Paths: Evolutionary Block Pruning Across Vision-Language Models
Tarun Tomar
Vision-
language models normally execute the same complete
vision encoder for every question, even when OCR, counting, object, attribute, and spatial queries may not require identical computation. We study whether fixed-budget combinations of
vision blocks can be skipped without fine-tuning. A shared K-block route skips one searched set of exactly K blocks for every question, while a capability-specific K-block policy selects one same-size route using a known capability label. We introduce a source-balanced evolutionary search and compare it with independent ranking, contiguous removal, and random routes at matched budgets. Experiments use Qwen2.5-VL-3B-Instruct, SmolVLM2-2.2B-Instruct, and an 876-example image-disjoint selection split. Search transfers across architectures: on SmolVLM2, the searched shared four-block route beats independent construction by 4.91 percentage points. Capability specialization is less stable. On Qwen, the six-block capability policy beats the shared route by 2.17 points, driven by a 7.10-point OCR gain. On sealed IIIT5K, however, the SmolVLM2 OCR-specific route trails its shared route by 13.6 points. Combinatorial search reliably improves route construction, but capability labels do not define universally transferable
vision pathways.
Read the full abstract ▾
Computer Vision
EvoGUI: An Evolution-Aware Benchmark for GUI State-Transition Understanding
Yaohan Yang, Minglei Shi, Borui Zhang et al. (+2 more)
GUI
agents must reason about how actions transform interface states, but end-to-end success rates entangle this ability with perception, grounding, planning, and recovery. We introduce EvoGUI, a diagnostic framework that converts normalized GUI trajectories into three complementary visual question answering probes: temporal ordering, inverse action/value prediction, and contrastive one-step successor discrimination. Their labels are derived from trajectory order and logged actions, requiring no additional task-label annotation after trajectory normalization. We instantiate EvoGUI-Bench from Mind2Web and WebLINX, yielding 3,000 instances across 120 domains, and evaluate 28
vision-
language model configurations zero-shot. The strongest model reaches only 60.4 EvoGain, while model scale and GUI specialization do not reliably predict performance. These results establish EvoGUI-Bench as a scalable diagnostic complement to end-to-end GUI-
agent evaluation while exposing substantial headroom in state-transition understanding. The source code is publicly available at https://github.com/Yyhhh6/EvoGUI.
Read the full abstract ▾
All authors Yaohan Yang, Minglei Shi, Borui Zhang, Jie Zhou, Jiwen Lu
Computer Vision · Cross-listed from cs.RO
Articulated Humanoid Head for a Robot Receptionist Capable of Natural Human Interaction
Tharusha Fonseka, Charuka Bandara, Moshintha Hewavitharana et al. (+4 more)
Humanoid robots have become increasingly popular in applications such as social interaction, education, and service roles, which drives the need for more natural and efficient human-
robot interactions. However, currently available
humanoid heads often face limitations, including high costs, mechanical complexity, and limited adaptability across diverse environments. To address these challenges, we present an articulated
humanoid robot head designed for a receptionist role, integrating a mechanical structure with 21 degrees of freedom (DoF), including mechanisms for the mouth, eyes, eyebrows, and neck, and covered with realistic silicone skin to achieve a human-like appearance and expression. The system integrates a model-based architecture that combines SCRFD, ArcFace, and ByTetrack for face recognition and Llama and Whisper for natural
language processing, with hardware support enabling real-time operations and human re-identification. The conversational ability and re-identification capabilities of the
humanoid robot head were quantitatively measured, while its emotional expressiveness and human likeness were evaluated through a user study, achieving an average human likeness score of 4.13 out of 5.
Read the full abstract ▾
All authors Tharusha Fonseka, Charuka Bandara, Moshintha Hewavitharana, Melisa Arukgoda, Wageesha N. Manamperi, Udaya S. K. Perera Miriya Thanthrige, Peshala Jayasekara