Computer Vision · Cross-listed from cs.RO
Recursive ArUco Markers: A Scalable Fiducial Marker Design for Unmanned Aerial Vehicle Landing Pads
Rafael Munoz-Salinas, Francisco Jose Romero-Ramirez and Sergio Garrido-Jurado
Unmanned Aerial Vehicles (
UAVs) increasingly rely on visual fiducial markers for autonomous
navigation and precision landing. However, standard markers suffer from limited operational ranges, becoming undetectable when the camera is either too far or too close. While recursive and fractal markers have been proposed to address this issue, existing approaches either require the marker's center to remain visible, making them vulnerable to occlusion, or are limited in their recursion depth and placement. We propose a novel Recursive ArUco marker design. Our method allows any standard fiducial marker to be transformed into a recursive marker with an arbitrary depth. By employing a modified bit-sampling strategy during detection, we embed complete markers within both the black and white bits of the parent marker. This approach guarantees unlimited recursion depth and robust detection even with partial occlusion, as it does not rely on the marker's center being visible. Furthermore, by maintaining a single, unique identifier across all recursive scales, our proposal provides an extensive dictionary of multiple unique landing pads. This capability allows fleets of
UAVs to operate simultaneously, with each drone landing at its designated location – a feature not supported by existing Fractal and Harco markers due to their structural and dictionary constraints.
Read the full abstract ▾
Computer Vision
Multimodal Assessment of Pancreatic Cancer Resectability Using Deep Learning
Vincent Ochs, Christoph Kuemmerli, Florentin Bieder et al. (+12 more)
Accurate determination of pancreatic ductal adenocarcinoma (PDAC) resectability relies on evaluating how the tumor interacts with major peripancreatic vessels on CT imaging, yet expert assessment often shows substantial variability. We introduce a fully automated
multimodal deep learning framework that jointly analyzes 3D contrast enhanced CT and structured clinical information to classify patients into the three National Comprehensive Cancer Network (NCCN) resectability categories (upfront resectable, borderline resectable, locally advanced). The approach uses a Swin-UNETR backbone to obtain anatomy aware image representations through auxiliary
segmentation of pancreas, tumor, and vascular structures. These features are fused with a compact clinical
embedding derived from 17 routinely collected variables and processed by a lightweight classification head. Model training is guided by a dynamic multitask objective that adapts the balance between
segmentation and classification based on current tumor Dice performance, promoting feature representations that remain both anatomically informed and discriminative.
Read the full abstract ▾
All authors Vincent Ochs, Christoph Kuemmerli, Florentin Bieder, Julia Wolleb, Joel L. Lavanchy, Julia Ruppel, Jan Liechti, Stephanie Taha-Mehlitz, Christian Andreas Nebiker, Beat Mueller, Giuseppe Kito Fusai, Joerg-Matthias Pollok, Anas Taha, Philippe C. Cattin, Sebastian Staubli
Computer Vision · Cross-listed from eess.IV
TCAM-Diff: Triplane-Aware Cross-Attention Medical Diffusion Model
Zhenkai Zhang, Krista A. Ehinger and Tom Drummond
We introduce TCAM-Diff, a novel 3D medical image generation model that reduces the
memory requirements to encode and generate high-resolution 3D data. This model utilizes a decoder-only autoencoder method to learn triplane representation from dense volume and leverages generalization operations to prevent overfitting. Subsequently, it uses a triplane-aware cross-
attention diffusion model to learn and integrate these features effectively. Furthermore, the features generated by the
diffusion model can be rapidly transformed into 3D volumes using a pre-trained decoder module. Our experiments on three different scales of medical
datasets, BrainTumour 128 x 128 x 128, Pancreas 256 x 256 x 256, and Colon 512 x 512 x 512, demonstrate outstanding results. We utilized MSE and SSIM to assess reconstruction quality and leveraged the Wasserstein Generative
Adversarial Network (W-GAN) critic to assess generative quality. Comparisons with existing approaches show that our method gives better reconstruction and generation results than other encoder-decoder methods with similar-sized latent spaces.
Read the full abstract ▾
Computer Vision
Bake It Till You Make It: Ultrafast Spatial Texture-Atlas Splatting
Neel Kelkar, Simon Niedermayr, Kaloian Petkov et al. (+2 more)
Neural radiance representations in Gaussian Splatting (GS) deliver high-fidelity color detail but impose substantial rendering overhead from network evaluations. We present a learned sparse scene representation based on 2D surfels, which enables baking the neural component into a hardware-accelerated 2D texture atlas and eliminates runtime network inference. Our 2D surfels carry low-frequency geometry and view-dependent appearance, while view-independent, per-primitive high-frequency texture is encoded with a spatial hash grid and converted into the texture atlas. A novel sparsity objective that penalizes per-primitive kernel falloffs achieves a substantially sparser representation than prior methods. With a compute-optimized ray-surfel intersection shader our approach renders roughly $7$ to $9\times$ faster than 3DGS on common
benchmarks, and over an order of magnitude faster on individual scenes, while surpassing the perceptual quality and speed of the fastest sparsification methods (FastGS and Speedy-Splat) at PSNR parity. A quality-optimized variant matches the perceptual quality of the strongest baseline while still rendering several times faster. Because inference is one texture fetch per fragment, frame rates can be pushed to $2{,}700$–$4{,}400$ FPS with no loss in quality when using the GPU hardware rasterizer.
Read the full abstract ▾
All authors Neel Kelkar, Simon Niedermayr, Kaloian Petkov, Klaus Engel, Rüdiger Westermann
Computer Vision
AspectCLIP: Optimizing CLIP Representation Space via Aspect-Guided Consistency Regularization
Yiyang Yao, Shanglin Liu, Jianming Lv et al. (+4 more)
Contrastive
Language-Image Pretraining learns a shared representation space through large-scale contrastive learning. However, existing methods that enforce global consistency regularization overlook a key challenge: the inherent information asymmetry between images and text: captions typically describe only one specific aspect of an image, thus images with similar visual content can be paired with completely divergent textual content and semantic information. Consequently, global regularizers inadvertently impose constraints between visually similar images whose captions describe divergent aspects, introducing semantic distortion into the representation space. We propose AspectCLIP, a framework that reformulates consistency regularization to respect this one-to-many structure. AspectCLIP first partitions training samples into attribute clusters based on textual similarity to identify aspect-coherent groups, then applies full cyclic consistency within each cluster while restricting cross-cluster regularization to prototype-level comparisons. This aspect-guided regularization enforces strict
geometric alignment only when images and texts describe a consistent facet, while allowing flexibility across divergent aspects. Extensive experiments on downstream tasks demonstrate that AspectCLIP consistently outperforms traditional methods and achieves a more structured representation space.
Read the full abstract ▾
All authors Yiyang Yao, Shanglin Liu, Jianming Lv, Chengjun Wang, Jinyi Li, Yuchan Jie, Zhihua Jin
Computer Vision
RainDancer: RGB-Event Video Deraining with Rain-Oriented Spiking Dynamics
Kui Jiang, Runzhe Li, Zhaocheng Yu et al. (+3 more)
Video deraining aims to recover clean visual content from rainy
videos for reliable perception under adverse weather. Existing methods mainly rely on RGB sequences and temporal redundancy, but RGB-only restoration remains ambiguous in dynamic rainy scenes, where rain streaks, textures, boundaries, motion, and occlusions may share similar visual patterns. Event cameras provide complementary motion-sensitive cues with high temporal resolution, but event streams also contain sensor noise and background-triggered responses, so direct RGB-Event fusion may introduce cross-modal interference. To address this issue, we propose RainDancer, a progressive RGB-Event
video deraining framework based on a decompose-before-interact paradigm. The core idea is to separate rain and background components within each modality before cross-modal interaction. In the RGB branch, frame features are progressively decomposed into rain and background representations. In the event branch, a rain-oriented spiking neural network module captures sparse and bursty event dynamics associated with rain motion. Component-level fusion is then performed between semantically aligned representations for structure preservation and rain suppression. We further introduce event-domain supervision to regularize sparse event reconstruction, structural consistency, and gradient orientation. Experiments on synthetic and real RGB-Event
video deraining
datasets demonstrate superior quantitative performance, visual quality, and downstream perception robustness. Code is available at https://github.com/AE86-plus/RainDancer.
Read the full abstract ▾
All authors Kui Jiang, Runzhe Li, Zhaocheng Yu, Guanglu Sun, Junjun Jiang, Xianming Liu
Computer Vision · Cross-listed from eess.IV
Prospective clinical indication, post-hoc report leakage, and fusion design in multi-image chest radiograph classification: a patient-clustered evaluation
Kamran Shahid and Muhammad Munwar Iqbal
Chest radiograph
datasets often combine multiple images with Clinical Indication, Findings, and Impression, although these inputs are produced at different stages of care. We evaluated 15,000 ReXGradient-160K studies with two readable images and five CheXbert-derived report observations. Frozen DenseNet-121 and Bio+ClinicalBERT encoders were used to compare image-only, Indication-only, fixed-order
multimodal, random-swap, DeepSets, and SectionGuard-MI models. Findings and Impression were evaluated only as post-hoc leakage controls. Models were trained with five seeds, and public-test uncertainty was estimated with 2,000 patient-cluster bootstrap replicates. Under U-Ones, macro AUROC was 0.643 for the primary image, 0.694 for two images, 0.749 for Indication, and 0.780 for ordinary two-image-plus-Indication fusion. SectionGuard-MI achieved AUROC 0.783 and AUPRC 0.260. Relative to ordinary fusion, its paired AUROC difference was 0.0031 (95% CI, -0.0042 to 0.0104; adjusted p=0.374), while its AUPRC difference was 0.0289 (95% CI, 0.0095 to 0.0413; adjusted p=0.004). DeepSets had the highest prospective AUROC point estimate (0.787), and random-swap fusion had the highest prospective AUPRC point estimate (0.265) with better calibration than SectionGuard-MI. Full report text alone reached AUROC 0.979 and AUPRC 0.836; AUROC remained above 0.973 after exact or expanded masking. These results show that prospective Indication is strongly associated with report-derived targets, permutation-aware fusion is competitive, and post-hoc report text creates substantial report-label circularity.
Read the full abstract ▾
Computer Vision
EgoProceVQA: A Novel Egocentric Procedural Understanding Task with Self-Skill-Exploration Agent
Junlong Li, Junxi Li, Yuxiang Yang et al. (+3 more)
Most daily activities are inherently procedural. However, existing evaluations for
egocentric video understanding seldom address procedural understanding and largely overlook complex key-step-level
reasoning under the widely used
video question answering (VQA) paradigm for MLLMs. Such capabilities are crucial for building procedural AI assistants deployable on wearable devices. To bridge this gap, we introduce the
Egocentric Procedural Understanding VQA task (EgoProceVQA), which systematically evaluates
egocentric procedural
reasoning abilities of current MLLMs and
agents through six types of key-step-centric questions. Furthermore, we develop EgoProceGen, a data generation platform that efficiently constructs QA data tailored to different question types. Based on this platform, we build a
benchmark with 3,600 questions, four common procedural scenarios, and 31 everyday procedural tasks. Evaluations on EgoProceVQA show that existing MLLMs and
agents still have substantial room for improvement in procedural understanding. Therefore, we further propose EgoProceAgent, a self-skill-exploration agentic framework. We design a generic tool library for procedural understanding and a standardized sub-skill library shared across tools and models, enabling self-exploration without ground-truth supervision. By exploring how to compose and select sub-skills, the
agent discovers effective skill strategies for diverse problems, and attains state-of-the-art performance among open-source models on multiple tasks. Together, our
benchmark, generation platform, and agentic framework establish a unified foundation for EgoProceVQA. Project page: https://z1oong.github.io/EgoProceVQA/.
Read the full abstract ▾
All authors Junlong Li, Junxi Li, Yuxiang Yang, Wenbin Zou, Lap-Pui Chau, Yi Wang
Computer Vision
Ablation-Corrected Evaluation of Attribution Maps in Echocardiographic Ejection-Fraction Models
Hyunkyung Han and Min Jung Kim
Attribution maps for echocardiographic ejection-fraction models are evaluated by their overlap with an expert left-ventricular annotation, compared against a chance level that is computed from an area ratio rather than measured. We measure it. Two architectures trained on the same task attain overlap at 3.55 and 4.20 times measured chance, an eighteen percent difference a reader would take as the size of the gap between them. It is not. Replacing the annotated ventricle with a composition-matched surrogate changes one model's prediction six times more than replacing an equal-area control region and the other's twice, a factor of three; on sixty-two percent of cases for the second, neither the ventricle nor the attributed region moves the prediction at all. The reliability of this measurement, from two independent intervention batches, is 0.90 to 0.98. Overlap does not track the difference: scored against an ablation-derived label of which cases the model depends on the ventricle for, it reaches an area under the curve of 0.52 and 0.67. The design of the ablation also determines its answer, since ablating only the two annotated frames leaves the ventricle indistinguishable from a control region while ablating across all frames does not. A pediatric cohort reproduces all of these. We give the measured chance level, the symmetric-ablation protocol, the reliability estimate, and the ablation-prediction score as things to report alongside overlap.
Read the full abstract ▾
Computer Vision
Groc-PO: Grounded Context Preference Optimization for Truthful Multimodal LLMs
Zhixiao Zheng, Zheren Fu, Zhiyuan Yao et al. (+3 more)
Despite the rapid progress of
Multimodal Large
Language Models (MLLMs), they still suffer from untruthfulness issues, such as visual
hallucinations, content fabrication, and unfaithful
reasoning, which substantially undermine their faithfulness and practical utility.
Alignment methods based on human preference, such as Direct Preference Optimization (DPO), have been widely adopted to address these issues. However,
multimodal reasoning errors often propagate across stages, and final-answer errors can often be traced to mistakes in early grounding stages, yet standard DPO typically applies preference optimization at the final-answer level. This credit-assignment challenge means that supervision for early grounding stages is indirect rather than stage-specific, making it difficult to suppress error propagation arising from grounding drift and context inconsistency. To address this, we propose Grounded Context Preference Optimization (Groc-PO), a grounded preference optimization framework for MLLMs. We further construct the Grounded Context Preference
Dataset (GCPD), organizing multi-stage preference samples around three stages of Object Grounding, Contextual Grounding, and Grounded
Reasoning, to capture the formation, integration, and utilization of grounded context. By introducing more explicit preference supervision over multiple grounded stages, Groc-PO strengthens context-dependent
reasoning and mitigates cross-stage error propagation. Extensive experiments show that, compared with standard DPO and other strong baselines, Groc-PO achieves improved performance in
hallucination mitigation, faithful
reasoning, and overall reliability, supporting the value of more explicit grounded supervision for trustworthy
multimodal reasoning.
Read the full abstract ▾
All authors Zhixiao Zheng, Zheren Fu, Zhiyuan Yao, Chunxiao Liu, Dongming Zhang, Zhendong Mao
Computer Vision · Cross-listed from cs.RO
Towards a Modular Bin-picking Framework for Handling Object Pose Uncertainties
Frederik Hagelskjær
In recent years, there has been growing interest in robust robotic systems for precise bin-picking applications. To achieve reliable performance, such systems must address errors arising from both the object pose estimation and the grasping process. Although various approaches have been proposed, they typically target specific challenges and do not offer general solutions. In this paper, we present a modular framework that jointly handles both error types. The framework incorporates object pose distribution estimation to account for pose uncertainty, which frequently arises in situations with ambiguous observations where a single correct pose cannot be determined. To further reduce uncertainty, we introduce a second-viewpoint module that computes complementary pose distributions, which are subsequently fused. This fusion decreases overall uncertainty and improves system efficiency. Additionally, two independent modules are included to compensate for grasping errors. The modular design allows the components to be combined for optimal performance or used individually, depending on the physical setup. The proposed method is evaluated in a real-world setup with three different objects, with no errors, and all modules are shown to improve efficiency. These results suggest that incorporating pose distributions with grasping pose errors is a promising direction for developing more flexible and reliable robotic production systems. To the best of our knowledge, this is the first framework that jointly addresses both grasping and object pose uncertainties using interchangeable modules. We believe there is ample opportunity to integrate additional modules, resulting in improved performance and flexibility. The current framework is limited to pose uncertainties in SO(2), but it could be extended to SE(3), enabling additional modules to improve the system.
Read the full abstract ▾
Computer Vision
Barnamala: Parameter-Efficient Handwritten Devanagari Recognition at Benchmark Saturation
Ashish Thapa and Samrat Karki
We built a compact convolutional network (1.11 M parameters) for 46-class DHCD Devanagari recognition and reached 99.73%, the highest reported at 15.6x smaller than prior state-of-the-art. We have effectively reached the saturation point: every model tested, large teacher ensembles included, hits the same 11-error intrinsic floor. No configuration achieves a statistically clear win under exact McNemar tests with Wilson confidence intervals. Even without knowledge
distillation, our student matches the nearest large-model baseline (17.32 M parameters; McNemar $p = 0.345$). Outside of DHCD, zero-shot on CMATERdb digits gives 76.6% and fine-tuning reaches 97.8%; corruption robustness is also far better than large baselines (mean corruption accuracy 75.7% vs. 38.7%). All artifacts are at https://github.com/Ampixa/barnamala.
Read the full abstract ▾
Computer Vision
DNA: Dual-stage Native Attribution for Generated Image Source Tracing
Chao Wang, Kejiang Chen, Zijin Yang et al. (+4 more)
The rapid
evolution of image generation has produced numerous within-family variants, making source-model attribution of suspect images increasingly important for digital forensics. Existing proactive methods rely on watermark
embedding or model modification, which may degrade visual quality and limit deployment flexibility. Passive methods often rely on large-scale supervised training or a single reconstruction signal, limiting their ability to handle unknown sources and distinguish highly similar within-family variants. We observe that attribution signals in latent generative models are naturally stratified across architectural levels: VAE-level cues reflect family-shared information, whereas backbone-level cues capture variant-specific behaviors. Motivated by this insight, we propose Dual-stage Native Attribution (DNA), a coarse-to-fine framework that follows this hierarchy without additional neural-network training. The coarse-grained stage uses Autoencoder Double-Reconstruction (AEDR) for efficient open-set family-level screening. The fine-grained stage performs closed-set model-level attribution with Native Prediction Consistency (NPC), which compares native prediction errors of within-family variants across multiple noise levels under semantic conditioning and attributes the source via normalized calibrated scores. To enable systematic evaluation, we construct DNA-30K, a
benchmark for within-family variant attribution under open-set family-level evaluation. It comprises 30,000 images generated by 24 candidate models across six families spanning both denoising
diffusion and flow matching, plus non-candidate generated and natural images as unknown sources. Experiments show that DNA achieves 89.11% end-to-end attribution accuracy on a task where random guessing accuracy is below 1% and outperforms the strongest baseline by 33.81% even when AEDR is used as the coarse-grained stage.
Read the full abstract ▾
All authors Chao Wang, Kejiang Chen, Zijin Yang, Yaofei Wang, Yuang Qi, Weiming Zhang, Nenghai Yu
Computer Vision
When Variance Is Not an Error Map: Calibrated Uncertainty for Radiative Gaussian Splatting in Sparse-View CT
Chulin Zhao, Yiran Xu and Shu Liu
Does an uncertainty map identify where a reconstruction is wrong? In sparse-view computed
tomography (CT), we find a sharp gap between whole-volume evaluation and error localization inside the object. We derive clamp-aware analytic moments for factorized Gaussian-density distributions, with a variance pass through existing rendering interfaces that is $7.9\times$ faster than a 16-sample
estimator. On a 15-scene
benchmark, median variance–error Spearman correlation falls from $0.846$ over the whole volume to $0.108$ in foreground. The pattern recurs across representations and acquisition settings. Two analyses help explain the discrepancy: region contrast dominates global covariance, while $72$–$96\%$ of in-object squared error is shared across independently trained members. Spread is unchanged by a common error, although shared error alone does not determine ranking. Correcting the offset between the deployed reconstruction and predictive mean improves foreground correlation by only $0.0014$. The diagnosis separates two remedies. A log-normal control improves scale transfer without restoring localization; a supervised error predictor raises foreground correlation to $0.427$ on the
benchmark and $0.566$ on eight held-out human subjects under simulated acquisition. A verified retrospective re-execution on eight additional subjects retains a median of $0.537$, with the same frozen predictors. The central lesson is to validate uncertainty in the region, and against the error target, for which it will be used.
Read the full abstract ▾
Computer Vision
Towards Spatial Supersensing in the Wild
Tianjun Gu, Tianyu Xin, Kuan Zhang et al. (+12 more)
Humans can efficiently parse continuous sensory streams, from hours to years, scaffolding an internal world model that grounds spatial
reasoning and prediction. To mimic this capacity, spatial supersensing challenges
multimodal models to move beyond linguistic understanding toward true world modeling. However, their
benchmark relies on synthetic long
videos, formed by concatenating random short clips, and is mostly limited to household scenes, leaving real-world continuity and diversity underexplored. To address the gap, we introduce $\textbf{VSI-Super-Wild}$, a large-scale
benchmark for evaluating spatial supersensing over long temporal horizons in diverse in-the-wild scenes. Notably, inspired by cognitive studies on how humans structure experience, we systematically probe the full triad of world state: the
agent (observer), objects (scene items), and the environment (places and global layout). In total, VSI-Super-Wild contains $\textbf{6,980}$ human-verified question-answer pairs derived from $\textbf{442}$ real-world
videos spanning 8 scene categories, including long-form recordings exceeding 4 hours. Results on VSI-Super-Wild expose a fundamental disconnect: despite advances in static image understanding, models consistently fail at tasks that require coherent world-state tracking over time. We characterize how performance degrades with world-state complexity and temporal horizon, and diagnose four failure modes: spatial collapse, semantic shortcuts, insufficient update, and instance confusion. This taxonomy reveals that models lack mechanisms to bind objects,
agents, and environments into a unified spatial world model, a fundamental gap that defines the path forward for spatial supersensing.
Read the full abstract ▾
All authors Tianjun Gu, Tianyu Xin, Kuan Zhang, Bowen Yang, Kok-Chung Chua, Peize Li, Xinran Zhang, Yupeng Chen, Qiyue Zhao, Qinlei Xie, Jianhang Liu, Yucheng Lu, Yinan Han, Marco Pavone, Yiming Li
Computer Vision
WAVE-Stereo: Warp-Aligned Volume Encoding for Stereo Matching
Zehan Liu, Yage He and Xianwu Gong
Existing iterative stereo matching methods primarily adopt two types of correspondence representation: explicit matching search via correlation volumes and local residual refinement via warped features, yet the two remain separately modeled. We propose WAVE-Stereo, built on a core insight: correlation volumes and feature warping provide complementary matching cues. GeoWarp Correspondence Encoder (GWCE) encodes matching search, residual
alignment, and disparity prior in parallel at the ConvGRU input. To mitigate matching degradation in textureless regions, we propose Periodic Global Context Propagation (PGCP), which propagates global spatial information in a periodic manner. On five real-world
benchmarks – Middlebury, ETH3D, KITTI 2012, KITTI 2015, and Booster – WAVE-Stereo achieves competitive zero-shot generalization accuracy without any external foundation model prior, achieving 3.18% D1-all on KITTI 2015, 4.42% Bad-2.0 on Booster, and 66ms real-time inference, striking a favorable balance between accuracy and efficiency. Our code is available at https://github.com/yamanoko-do/WAVE-Stereo.
Read the full abstract ▾
Computer Vision
Learning Speaker Identity Beyond Language and Modality Constraints: Insights from the POLY-SIM 2026 Challenge
Marta Moscati, Muhammad Saad Saeed, Marina Zanoni et al. (+9 more)
Multimodal speaker identification systems typically assume the availability of complete and homogeneous audio-visual modalities during both training and testing, and assume each speaker only speaks a single
language. However, in real-world applications, such assumptions often do not hold. Visual or audio information may be missing due to occlusions, camera or microphone failures, or
privacy constraints.
Multilingual speakers introduce additional complexity due to linguistic variability across
languages. These situations constitute substantial challenges for the robustness and generalization capabilities of
multimodal speaker identification systems. Aim of the POLY-SIM 2026 challenge is to address these aspects of speaker identification and to provide a standardized setup for the comparison of the proposed solutions.
Read the full abstract ▾
All authors Marta Moscati, Muhammad Saad Saeed, Marina Zanoni, Mubashir Noman, Rohan Kumar Das, Monorama Swain, Yassin Terraf, Yufang Hou, Elisabeth Andre, Khalid Mahmood Malik, Markus Schedl, Shah Nawaz
Computer Vision
Fine-grained CLIP fine-tuning with self-annotated region alignment
Chenyang Zhao, Wei Lin, Janet H. Hsiao et al. (+1 more)
Contrastive
Language-Image Pre-training (CLIP) has been shown to have limitations in its fine-grained dense feature representation, due to its pre-training focusing on matching the whole image to a text description. Considering the large data and computational burden in pre-training a
vision-
language model from scratch, a series of works aim to enhance the fine-grained ability of CLIP through a fine-tuning scheme. However, existing works suffer from a variety of limitations: additional region annotations are usually required, which limits the semantic diversity due to the predefined categories and leads to a large effort to process the training data; and they usually sacrifice CLIP's original ability for global visual representation. To bypass these limitations, we propose SFF-CLIP (Self-annotated Fine-grained Fine-tuning for CLIP), which only uses image-text pairs as input to boost the fine-grained representation ability in the CLIP fine-tuning, while maintaining the global visual-semantic consistency. Concretely, a run-time region-phrase
alignment scheme is designed, which obtains concept phrases from the input sentence, and aligns them with corresponding extracted region-based features using text-specific heat maps. Extensive experiments demonstrate that SFF-CLIP leads to significant performance improvements on fine-grained dense feature representation, as well as maintaining the performance of the original CLIP on image-level tasks. Code will be released later.
Read the full abstract ▾
All authors Chenyang Zhao, Wei Lin, Janet H. Hsiao, Antoni B. Chan
Computer Vision
FreeLit: Paired-Free Indoor Relighting via Physics-Guided Diffusion
Chi-En Yen, Duy-Khanh Ngo, Wen-Wei Tang et al. (+3 more)
Image-based indoor scene relighting remains challenging due to the complex interplay between cluttered geometry and local illumination, requiring precise modeling of light position, color, and intensity. Existing data-driven methods implicitly learn this relationship via paired multi-illumination
datasets. Nevertheless, this data is costly and fails to scale, which is essential for accurate light-source-level control. Conversely, inverse-rendering methods reduce the data dependency by incorporating physical priors; however, they lack the robustness of intrinsic estimation in challenging conditions. In this paper, we present FreeLit, a paired-free framework for controllable indoor relighting that explicitly manipulates light-source location, color, and intensity. Instead of relying on paired supervision, we construct a physics-guided illumination prior from intrinsic scene properties, generating a structured lightmap along with a pseudo-relit image to guide
diffusion-based synthesis. To address instability in intrinsic estimation, especially in low-light scenes, we introduce a relighting-guided intrinsic stabilization strategy that enforces illumination-invariant reflectance through structure-aware
distillation and consistency constraints. Furthermore, we propose controllability-oriented evaluation metrics to quantify
alignment with user-specified illumination color and intensity. Experimental results demonstrate that FreeLit achieves stable, physically consistent, and controllable relighting, with improved robustness in low-light indoor scenes, without requiring paired supervision.
Read the full abstract ▾
All authors Chi-En Yen, Duy-Khanh Ngo, Wen-Wei Tang, Huu-Phu Do, Wen-Hsiao Peng, Ching-Chun Huang
Computer Vision
T3HG-Editor: Text-driven 3D Human Garment Editing with Body Priors Embedded in SMPL-X
Shaoru Sun, Xingtao Wang, Zihan Ma et al. (+4 more)
While 3D Gaussian Editing (3DGE) has seen substantial progress, text-driven 3D human garment editing remains largely underexplored. Existing 3DGE works typically follow a paradigm that applies 2D editing techniques to multi-view rendered images and updates 3D Gaussians based on the modified images. Extending such methods to 3D human garment editing suffers from low-fidelity outcomes, caused by introduced distortions and garment inconsistencies. A promising breakthrough opportunity arises from the SMPL eXpressive (SMPL-X) model that embodies rich prior information for virtual humans. Motivated by this insight, we propose a text-driven 3D human garment editor termed T3HG-Editor, which delivers high-fidelity and garment consistent results by leveraging geometry and joint priors embedded in SMPL-X. Specifically, T3HG-Editor contains three stages, namely obtainment of editable Gaussians, garment consistent editing, and Gaussian updating with overflow
pruning. The obtainment of editable Gaussians begins with seeding Gaussians along SMPL-X normals to generate sufficient near surface Gaussians, followed by a 2D mask constraint that precisely localizes the target Gaussians to be edited. The garment consistent editing aggregates
tokens corresponding to the same SMPL-X vertex across multiple views and propagates them to their original views, enforcing garment consistency without requiring additional training. Gaussian updating with overflow
pruning employs a Signed Distance Function (SDF) defined on SMPL-X to construct a human distance field, which is then integrated with a 2D semantic mask to prune overflowing Gaussians, thus preventing contamination of non-target regions. Experiments on multiple subjects and diverse garment types demonstrate that T3HG-Editor outperforms state-of-the-art methods in both editing quality and garment consistency.
Read the full abstract ▾
All authors Shaoru Sun, Xingtao Wang, Zihan Ma, Wenrui Li, Jiantao Zhou, Debin Zhao, Xiaopeng Fan
Computer Vision
Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation
Boyu Mi, Mengchen Ma, Yifei Yao et al. (+10 more)
Real-world deployment of
embodied agents requires active exploration, visual grounding, and interactive intent disambiguation. However, existing frameworks often rely on privileged simulator states or assume complete instructions, bypassing realistic deployment challenges. To bridge this gap, we present REAL, an agentic framework for open-world mobile manipulation. REAL establishes sim-to-real-consistent environment APIs without oracle perception and integrates a simulated user to enable human-in-the-loop interaction. Within this environment, we design diverse task compositions to drive data collection, supervised fine-tuning, and online reinforcement learning, systematically optimizing
agent performance. To comprehensively evaluate this approach, we introduce REAL-Bench, a
benchmark spanning 241 tasks across active exploration, visual distraction, articulated manipulation, and interactive disambiguation. Experimental results demonstrate that our trained
agent outperforms leading commercial closed-source VLMs on interactive tasks with a 56.9% success rate. Further empirical analysis reveals that our hierarchical training pipeline successfully aligns the model's tool-use capabilities while maintaining robust open-vocabulary
reasoning under extended exploration horizons. Finally, we deploy and evaluate our framework on a physical dual-arm mobile
robot, where it achieves a 78.3% end-to-end success rate over 60 real-world episodes. These physical trials demonstrate robust zero-shot transferability to unseen household scenarios, validating that our sim-to-real-consistent design successfully bridges the reality gap for long-horizon mobile manipulation. Code is available at https://github.com/InternRobotics/REAL.
Read the full abstract ▾
All authors Boyu Mi, Mengchen Ma, Yifei Yao, Xing Gao, Junting Chen, Yangzi Li, Zihou Zhu, Guohao Li, Zhenfei Yin, Tai Wang, Yao Mu, Jiangmiao Pang, Hanqing Wang
Computer Vision
From Surface Forecasting to Observability Forecasting: A Latent World Model for Cloud-Aware EO Monitoring
Mohanad Albughdadi
The bottleneck of Earth Observation processing chains is not the arrival of new imagery but whether the surface is actually visible when the image arrives. We study this as an observability
forecasting problem on EarthNet2021. Given recent multispectral imagery and exogenous weather drivers, the goal is to predict whether the next acquisition will be usable and, if not, when a usable view is likely to return. To do this, we adapt LeWorldModel, a joint-
embedding predictive architecture world model, to cloud-aware Earth Observation sequences. The final pipeline converts raw minicubes into episodic HDF5 sequences with five image channels (blue, green, red, near-infrared, cloud mask) and eight meteorological and calendar
covariates. The resulting model has 18.0M trainable parameters and is trained from scratch on 23,904 training episodes. The trained leWorldModel is evaluated under a locked protocol: linear probes are fit on train only, calibration choices are set on an internal validation split, and the fitted heads are then frozen for valsplit, IID, OOD, and extreme evaluation. On the full frozen-bundle observability
benchmark, LeWorldModel consistently outperforms persistence. For next-step usability, balanced accuracy ranges from 0.769 to 0.887, compared with 0.493 to 0.556 for persistence. For exact first-usable-horizon prediction, accuracy ranges from 0.602 to 0.806, compared with 0.120 to 0.369 for persistence. Against a frozen LightGBM baseline fit on the same training windows, LeWorldModel is better on continuous clear/cloud regression and on exact recovery timing on valsplit, IID, and extreme, while LightGBM is stronger on the simpler binary any-usable-within-six task and is more robust on OOD. In separate sampled diagnostic analyses, LeWM also produces strong ranking-based anomaly signals under synthetic temporal inconsistencies.
Read the full abstract ▾
Computer Vision
Beyond Color Geometry: Evaluating Human-Like Color Representations in Vision Models
Ayan Igali and Pakizar Shamoi
Do
vision models see colors the way humans do? Existing evaluations of color representations usually compare them with
geometric spaces such as CIELAB or with discrete color labels. These references capture perceptual distance or category membership, but not the graded way in which people organize colors. We evaluate color grounding against a fuzzy perceptual model with 86 graded categories fitted to human survey data. The framework can be applied to any image encoder and measures three complementary properties: category boundaries, category compactness, and graded
alignment beyond what color geometry alone can explain. Across eleven
Vision Transformer encoders, the category-level results are broadly similar, whereas graded
alignment differs substantially. Masked Autoencoders achieve the strongest beyond-geometry
alignment, with confidence intervals that do not overlap those of the other encoders. A layer-wise analysis further shows that masked reconstruction preserves this structure toward the output. On natural images, MAE represents surface color globally, while
language-supervised models encode color more strongly in relation to the foreground object. These results show that human-like color grounding has several distinct aspects that should not be reduced to a single score.
Read the full abstract ▾
Computer Vision
Human4K: A Large-Scale 4K Multi-View Mocap Dataset for Whole-Body 3D Human Reconstruction
Tianshun Han, Ziyu Shi, Lijian Liu et al. (+7 more)
Recent advances in 3D human reconstruction have improved overall performance, yet current models still fail in the most challenging real-world scenarios. They often produce unstable geometry, inaccurate limb articulation and unreliable predictions under depth ambiguity or self-occlusion. A key reason is that existing
datasets still lack the combination of high-resolution images, high-precision annotations and diverse whole-body motions required to support robust reconstruction. To address this gap, we present Human4K, a large-scale 4K multi-view whole-body human reconstruction
dataset with mocap-accurate SMPL-X annotations. Human4K contains over six million 4K images captured by an eight-view high-resolution camera system synchronized with a professional Vicon motion capture setup, covering 11 subjects performing complex, highly articulated and strongly self-occluded full-body motions. All sequences are processed by a Motion-Retargeting and Refinement Module (MRRM) to ensure precise
alignment for the full body and extremities. Experimental results show that training with Human4K consistently improves whole-body reconstruction on standard
benchmarks, with particularly large gains for hands, feet and depth-ambiguous limb configurations.
Read the full abstract ▾
All authors Tianshun Han, Ziyu Shi, Lijian Liu, Ajian Liu, Benjia Zhou, Hugo Jair Escalante, Yanyan Liang, Sergio Escalera, Zhen Lei, Jun Wan
Computer Vision
OvisOCR2 Technical Report
Shiyin Lu, Yinglun Li, Yu Xia et al. (+10 more)
We introduce OvisOCR2, a 0.8B document parsing model. OvisOCR2 is designed as an end-to-end parser: given a document page image, it generates a Markdown representation in natural reading order, covering text, formulas, tables, and visual regions. We build a data engine that combines filtered real-document annotations with synthetic pages whose rendered images and Markdown targets are derived from the same HTML source. The training recipe includes supervised fine-tuning, reinforcement learning on a 4B branch with a multi-component reward design, on-policy
distillation into the 0.8B model, and model fusion. On OmniDocBench v1.6, OvisOCR2 achieves a state-of-the-art overall score of 96.58, placing an end-to-end model at the top of this leaderboard previously dominated by pipeline methods and highlighting the potential of end-to-end document parsing. On PureDocBench, OvisOCR2 also achieves the highest Avg3 score of 75.06. Beyond these two public
benchmarks, we evaluate OvisOCR2 on an in-house
benchmark designed to cover a broader set of long-tail and challenging scenarios. OvisOCR2 obtains the best overall performance among the compared methods, providing further evidence of its generalization and robustness. OvisOCR2 is available at https://huggingface.co/ATH-MaaS/OvisOCR2.
Read the full abstract ▾
All authors Shiyin Lu, Yinglun Li, Yu Xia, Yuhui Chen, An-Yang Ji, Jun-Peng Jiang, Qing-Guo Chen, Jianshan Zhao, En Lin, Haijun Li, Cheng Qin, Zhao Xu, Weihua Luo
Computer Vision · Cross-listed from cs.LG
FastCentNN: Accelerating Centroid Neural Network with Entropy Proxy
Le-Anh Tran
Centroid neural network (CentNN) is an unsupervised competitive learning
algorithm in which centroid splitting is triggered only after strict local stabilization, often leading to prolonged low-movement training phases before model expansion. This report proposes FastCentNN, an accelerated variant that addresses this inefficiency by introducing an early splitting strategy based on the total centroid movement per epoch, which serves as a training
entropy proxy. As a result, FastCentNN reduces unnecessary reassignment epochs while preserving the original winner-loser learning dynamics. FastCentNN supports both absolute and stage-relative movement thresholds, allowing the splitting criterion to remain either fixed or adaptive throughout training. Experiments on some
benchmark datasets show that FastCentNN consistently achieves clustering quality comparable to CentNN while reducing runtime by up to 16% on synthetic 2D
datasets and about 5% on high-dimensional
datasets. FastCentNN therefore provides a practical and efficient drop-in replacement for CentNN, retaining its online adaptive learning behavior while offering a simple and interpretable speed-stability trade-off through configurable splitting thresholds.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Video to All-in-focus Image Reconstruction Algorithm for Automated Microscopic Urinalysis
Chinmay Nema, Hari Om Aggrawal, Dipam Goswami et al. (+2 more)
Microscopic urinalysis is a routine diagnostic test at hospitals. Recent studies have demonstrated the effectiveness of deep learning methods to automate microscopic urinalysis. These methods rely on high-quality images of the urine samples in which each
cell is clearly identifiable. However, in practice, the urine sample on a glass slide has a multi-layer structure; hence, all the
cells are not clearly visible within the depth of field of a lens focused at a particular focal plane. It demands acquiring multiple images at different focal planes to correctly identify each
cell in a given urine sample, which is a time-consuming task. In this paper, we propose to simplify the task by recording a
video, in place of acquiring multiple images, while gradually changing the focus of the lens manually by hand. A typical length of the
video is from 2 to 14 seconds. We reconstruct an all-in-focus image from the recorded
video frames and apply a deep learning model to detect and classify urine sediments. As a proof of concept, we conduct experiments on 14
videos acquired by a trained lab technician in a usual diagnostic lab environment and show the effectiveness of the proposed automated urinalysis pipeline with our novel reconstruction
algorithm.
Read the full abstract ▾
All authors Chinmay Nema, Hari Om Aggrawal, Dipam Goswami, Rajiv Gupta, Vinti Agarwal
Computer Vision · Cross-listed from cs.RO
Semantic Anchoring for Robotic Action Representations
Yuan Xu, Youheng Shi, Chengyang Li et al. (+2 more)
Vision-
Language-Action (VLA) models inherit rich semantic representations from pretrained
Vision-
Language Models, yet fine-tuning on limited
robot demonstrations degrades this structure and undermines generalization. A fundamental question therefore arises: what constitutes a good action representation? Inspired by the mirror
neuron theory's insight that observation and execution share an intention-level encoding, we examine whether a
robot's action representations preserve the semantic structure captured by pretrained encoders. Systematic probing confirms that this structure erodes during finetuning, and that its quality synchronizes with both task success and out-of-distribution generalization. We further introduce a plug-and-play method that anchors action representations to a semantic
manifold while decomposing representations into a shared semantic channel and a private channel, all discarded at inference, leaving the deployed model unchanged. Validated on different VLA backbones across simulation and real-world
benchmarks, our method yields up to +18.7% on real-world in-distribution tasks and +21.5% on out-of-distribution generalization.
Read the full abstract ▾
All authors Yuan Xu, Youheng Shi, Chengyang Li, Wentao Zhu, Yizhou Wang
Computer Vision
UniPhysGen: Unified Physical Grounding for Simulation-Ready 3D Assets
Xian Li, Rong Wei, Lujie Yang et al. (+6 more)
Physically grounded 3D assets are increasingly important for
embodied AI and robotic simulation. However, most existing 3D assets lack unified physical semantics, including articulation semantics and intrinsic physical properties, required for realistic interaction. Current approaches either treat these semantics independently or rely on canonicalized object structures, limiting robustness across heterogeneous 3D assets. We present UniPhys, a scalable framework for automatically transforming raw 3D assets into simulation-ready assets with unified physical semantics. Based on UniPhys, we construct UniPhys-40K, a large-scale physically grounded
dataset, together with UniPhys-Bench, a carefully verified
benchmark for unified physical grounding evaluation. We further introduce UniPhysGen, a unified physical grounding model that jointly reasons over articulation semantics and intrinsic physical properties. UniPhysGen incorporates geometry-robust articulation grounding to mitigate
geometric shortcut bias under heterogeneous part decompositions. Extensive experiments demonstrate state-of-the-art performance across articulation grounding and intrinsic physical property estimation tasks, while the resulting assets can be directly deployed in robotic simulation environments for realistic physical interaction. Our code and
dataset will be available at https://github.com/breezexian/UniPhysGen.
Read the full abstract ▾
All authors Xian Li, Rong Wei, Lujie Yang, Haolin Huang, Junyuan Fang, Siliang Tang, Jun Xiao, Rui Tang, Juncheng Li
Computer Vision · Cross-listed from cs.NE
Visual Place Recognition Using Rate-Encoded Spiking Neural Networks with Discrete STDP Learning
Altzi Tsanko, Oikonomou Katerina Maria and Antonios Gasteratos
Spiking Neural Networks (SNNs) trained through unsupervised Spike-Timing-Dependent Plasticity (STDP) have been explored as solutions to visual loop closure problems, driven by the prospect of efficient on-device inference on neuromorphic devices. State-of-the-art STDP-based models deliver high classification accuracy but fail to reach the high Recall at 100% Precision (R@100P) needed for reliable autonomous
navigation. We present a discrete, tensor-native implementation of the STDP-based SNN-VPR pipeline using PyTorch with snnTorch and evaluate it on a 100-place Nordland
dataset using 15 independently-trained networks. The contribution of three decisions in the implementation is investigated. First, we show how to perform
neuron assignment with a closed-form, deterministic tensor pipeline and show that it provides significantly higher R@100P than a standard argmax procedure. However, some of this gain comes from implementation differences compared to prior continuous-time models, which we measure independently. Second, ablation in isolation shows that state reset after each query helps improve R@100P regardless of the way
neurons are assigned. Third, velocity-compensated sliding window aggregation over k consecutive frames reaches R@100P = 100.00% at k = 5 for constant-velocity traversal and an additional 0.20 ms latency. Taken together, these findings show the impact of inference stage design decisions in STDP-based SNN-VPR on recall precision, although the separate contribution of each mechanism and implementation differences is only partially disentangled and needs further examination.
Read the full abstract ▾