Computer Vision
SIC3D: Style Image Conditioned Text-to-3D Gaussian Splatting Generation
Ming He, Zhixiang Chen and Steve Maddock
Recent progress in text-to-3D object generation enables the synthesis of detailed geometry from text input by leveraging 2D
diffusion models and differentiable 3D representations. However, the approaches often suffer from limited controllability and texture ambiguity due to the limitation of the text modality. To address this, we present SIC3D, a controllable image-conditioned text-to-3D generation pipeline with 3D Gaussian Splatting (3DGS). There are two stages in SIC3D. The first stage generates the 3D object content from text with a text-to-3DGS generation model. The second stage transfers style from a reference image to the 3DGS. Within this stylization stage, we introduce a novel Variational Stylized Score
Distillation (VSSD) loss to effectively capture both global and local texture patterns while mitigating conflicts between geometry and appearance. A scaling regularization is further applied to prevent the emergence of artifacts and preserve the pattern from the style image. Extensive experiments demonstrate that SIC3D enhances
geometric fidelity and style adherence, outperforming prior approaches in both qualitative and quantitative evaluations.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
AniGen: Unified $S^3$ Fields for Animatable 3D Asset Generation
Yi-Hua Huang, Zi-Xin Zou, Yuting He et al. (+6 more)
Animatable 3D assets, defined as geometry equipped with an articulated skeleton and skinning weights, are fundamental to interactive graphics,
embodied agents, and animation production. While recent 3D generative models can synthesize visually plausible shapes from images, the results are typically static. Obtaining usable rigs via post-hoc auto-rigging is brittle and often produces skeletons that are topologically inconsistent with the generated geometry. We present AniGen, a unified framework that directly generates animate-ready 3D assets conditioned on a single image. Our key insight is to represent shape, skeleton, and skinning as mutually consistent $S^3$ Fields (Shape, Skeleton, Skin) defined over a shared spatial domain. To enable the robust learning of these fields, we introduce two technical innovations: (i) a confidence-decaying skeleton field that explicitly handles the
geometric ambiguity of bone prediction at Voronoi boundaries, and (ii) a dual skin feature field that decouples skinning weights from specific joint counts, allowing a fixed-architecture network to predict rigs of arbitrary complexity. Built upon a two-stage flow-matching pipeline, AniGen first synthesizes a sparse structural scaffold and then generates dense geometry and articulation in a structured latent space. Extensive experiments demonstrate that AniGen substantially outperforms state-of-the-art sequential baselines in rig validity and animation quality, generalizing effectively to in-the-wild images across diverse categories including animals,
humanoids, and machinery. Homepage: https://yihua7.github.io/AniGen-web/
Read the full abstract ▾
All authors Yi-Hua Huang, Zi-Xin Zou, Yuting He, Chirui Chang, Cheng-Feng Pu, Ziyi Yang, Yuan-Chen Guo, Yan-Pei Cao, Xiaojuan Qi
Computer Vision
LPLCv2: An Expanded Dataset for Fine-Grained License Plate Legibility Classification
Lucas Wojcik, Eduardo A. F. Machoski, Eduil Nascimento Jr. et al. (+2 more)
Modern Automatic License Plate Recognition (ALPR) systems achieve outstanding performance in controlled, well-defined scenarios. However, large-scale real-world usage remains challenging due to low-quality imaging devices, compression artifacts, and suboptimal camera installation. Identifying illegible license plates (LPs) has recently become feasible through a dedicated
benchmark; however, its impact has been limited by its small size and annotation errors. In this work, we expand the original
benchmark to over three times the size with two extra capture days, revise its annotations and introduce novel labels. LP-level annotations include bounding boxes, text, and legibility level, while vehicle-level annotations comprise make, model, type, and color. Image-level annotations feature camera identity, capture conditions (e.g., rain and faulty cameras), acquisition time, and day ID. We present a novel training procedure featuring an Exponential Moving Average-based loss function and a refined learning rate scheduler, addressing common mistakes in testing. These improvements enable a baseline model to achieve an 89.5% F1-score on the test set, considerably surpassing the previous state of the art. We further introduce a novel protocol to explicitly addresses camera contamination between training and evaluation splits, where results show a small impact.
Dataset and code are publicly available at https://github.com/lmlwojcik/LPLCv2-
Dataset.
Read the full abstract ▾
All authors Lucas Wojcik, Eduardo A. F. Machoski, Eduil Nascimento Jr., Rayson Laroca, David Menotti
Computer Vision
AI Driven Soccer Analysis Using Computer Vision
Adrian Manchado, Tanner Cellio, Jonathan Keane et al. (+1 more)
Sport analysis is crucial for team performance since it provides actionable data that can inform coaching decisions, improve player performance, and enhance team strategies. To analyze more complex features from game footage, a computer
vision model can be used to identify and track key entities from the field. We propose the use of an object detection and tracking system to predict player positioning throughout the game. To translate this to positioning in relation to the field dimensions, we use a point prediction model to identify key points on the field and combine these with known field dimensions to extract actual distances. For the player-identification model, object detection models like YOLO and Faster R-CNN are evaluated on the accuracy of our custom
video footage using multiple different evaluation metrics. The goal is to identify the best model for object identification to obtain the most accurate results when paired with SAM2 (Segment Anything Model 2) for
segmentation and tracking. For the key point detection model, we use a CNN model to find consistent locations in the soccer field. Through homography, the positions of points and objects in the camera perspective will be transformed to a real-ground perspective. The segmented player masks from SAM2 are transformed from camera perspective to real-world field coordinates through homography, regardless of camera angle or movement. The transformed real-world coordinates can be used to calculate valuable tactical insights including player speed, distance covered, positioning heatmaps, and more complex team statistics, providing coaches and players with actionable performance data previously unavailable from standard
video analysis.
Read the full abstract ▾
All authors Adrian Manchado, Tanner Cellio, Jonathan Keane, Yiyang Wang
Computer Vision
LMGenDrive: Bridging Multimodal Understanding and Generative World Modeling for End-to-End Driving
Hao Shao, Letian Wang, Yang Zhou et al. (+5 more)
Recent years have seen remarkable progress in autonomous driving, yet generalization to long-tail and open-world scenarios remains a major bottleneck for large-scale deployment. To address this challenge, some works use LLMs and VLMs for
vision-
language understanding and
reasoning, enabling vehicles to interpret rare and safety-critical situations when generating actions. Others study generative world models to capture the spatio-temporal
evolution of driving scenes, allowing
agents to imagine possible futures before acting. Inspired by human intelligence, which unifies understanding and imagination, we explore a unified model for autonomous driving. We present LMGenDrive, the first framework that combines LLM-based
multimodal understanding with generative world models for end-to-end closed-loop driving. Given multi-view camera inputs and natural-
language instructions, LMGenDrive generates both future driving
videos and control signals. This design provides complementary benefits:
video prediction improves spatio-temporal scene modeling, while the LLM contributes strong semantic priors and instruction grounding from large-scale pretraining. We further propose a progressive three-stage training strategy, from
vision pretraining to multi-step long-horizon driving, to improve stability and performance. LMGenDrive supports both low-latency online planning and autoregressive offline
video generation. Experiments show that it significantly outperforms prior methods on challenging closed-loop
benchmarks, with clear gains in instruction following, spatio-temporal understanding, and robustness to rare scenarios. These results suggest that unifying
multimodal understanding and generation is a promising direction for more generalizable and robust
embodied decision-making systems.
Read the full abstract ▾
All authors Hao Shao, Letian Wang, Yang Zhou, Yuxuan Hu, Zhuofan Zong, Steven L. Waslander, Wei Zhan, Hongsheng Li
Computer Vision
Xinmiao Xiong, Bangya Liu, Hao Wang et al. (+7 more)
Geometric Foundation Models (GFMs) have recently advanced
monocular SLAM by providing robust, calibration-free 3D priors. However, deploying these models on dense
video streams introduces significant computational redundancy. Current GFM-based SLAM systems typically rely on post hoc keyframe selection. Because of this, they must perform expensive dense
geometric decoding simply to determine whether a frame contains novel geometry, resulting in late rejection and wasted computation. To mitigate this inefficiency, we propose LeanGate, a lightweight feed-forward frame-gating network. LeanGate predicts a
geometric utility score to assess a frame's mapping value prior to the heavy GFM feature extraction and matching stages. As a predictive plug-and-play module, our approach bypasses over 90% of redundant frames. Evaluations on standard SLAM
benchmarks demonstrate that LeanGate reduces tracking FLOPs by more than 85% and achieves a 5x end-to-end throughput speedup. Furthermore, it maintains the tracking and mapping accuracy of dense baselines. Project page: https://lean-gate.github.io/
Read the full abstract ▾
All authors Xinmiao Xiong, Bangya Liu, Hao Wang, Dayou Li, Nuo Chen, Andrew Feng, Mingyu Ding, Suman Banerjee, Yang Zhou, Zhiwen Fan
Computer Vision
What Matters in Virtual Try-Off? Dual-UNet Diffusion Model For Garment Reconstruction
Loc-Phat Truong, Meysam Madadi and Sergio Escalera
Virtual Try-On (VTON) has seen rapid advancements, providing a strong foundation for generative fashion tasks. However, the inverse problem, Virtual Try-Off (VTOFF)-aimed at reconstructing the canonical garment from a draped-on image-remains a less understood domain, distinct from the heavily researched field of VTON. In this work, we seek to establish a robust architectural foundation for VTOFF by studying and adapting various
diffusion-based strategies from VTON and general Latent
Diffusion Models (LDMs). We focus our investigation on the Dual-UNet
Diffusion Model architecture and analyze three axes of design: (i) Generation Backbone: comparing Stable
Diffusion variants; (ii) Conditioning: ablating different mask designs, masked/unmasked inputs for image conditioning, and the utility of high-level semantic features; and (iii) Losses and Training Strategies: evaluating the impact of the auxiliary
attention-based loss, perceptual objectives and multi-stage curriculum schedules. Extensive experiments reveal trade-offs across various configuration options. Evaluated on VITON-HD and DressCode
datasets, our framework achieves state-of-the-art performance with a drop of 9.5% on the primary metric DISTS and competitive performance on LPIPS, FID, KID, and SSIM, providing both stronger baselines and insights to guide future Virtual Try-Off research.
Read the full abstract ▾
Computer Vision
Deep Learning-Based Tracking and Lineage Reconstruction of Ligament Breakup
Vrushank Ahire, Vivek Kurumanghat, Mudasir Ganaie et al. (+1 more)
The disintegration of
liquid sheets into ligaments and droplets involves highly transient, multi-scale dynamics that are complex to quantify from high-speed shadowgraphy images. Identification of droplets, ligaments, and blobs formed by the breakup of a
liquid sheet, and their tracking from frame to frame, is an essential part of spray analysis. Analysis of such processes poses a fundamental challenge for conventional multi-object tracking frameworks, which enforce strict one-to-one temporal associations and cannot represent one-to-many fragmentation events. We present a two-stage deep learning framework that detects droplets and ligaments and identifies their temporal relationships across successive frames, modeling ligament deformation, fragmentation, and parent-child lineage. In the first stage, a Faster R-CNN detector with a ResNet-50 backbone and Feature Pyramid Network classifies ligaments and droplets in recordings of two impinging Carbopol gel jets. A morphology-preserving synthetic data generation strategy enriches the training set without physically implausible configurations, achieving a held-out test F1 score of up to 0.872 across fourteen original-to-synthetic data configurations, with robustness tested on different atomization conditions and
liquid compositions. In the second stage, a
Transformer-augmented multilayer perceptron classifies inter-frame object associations into continuation, fragmentation (one-to-many), and non-association using physics-informed
geometric features. Despite severe class imbalance, the model achieves 86.1% accuracy, 93.2% precision, and perfect recall (1.00) for fragmentation events on the held-out test set. Together, the two stages preserve parent-child lineage and enable extraction of breakup statistics, including fragment multiplicity and droplet size distributions, offering an automated analysis of the primary mode of atomization.
Read the full abstract ▾
All authors Vrushank Ahire, Vivek Kurumanghat, Mudasir Ganaie, Lipika Kabiraj
Computer Vision
RS-OVC: Open-Vocabulary Counting for Remote-Sensing Data
Tamir Shor, George Leifman and Genady Beryozkin
Object-Counting for remote-sensing (RS) imagery is attracting increasing research interest due to its crucial role in a wide and diverse set of applications. While several promising methods for RS object-counting have been proposed, existing methods focus on a closed, pre-defined set of object classes. This limitation necessitates costly re-annotation and model re-training to adapt current approaches for counting of novel objects that have not been seen during training, and severely inhibits their application in dynamic, real-world monitoring scenarios. To address this gap, in this work we propose RS-OVC - the first Open Vocabulary Counting (OVC) model for Remote-Sensing and aerial imagery. We show that our model is capable of accurate counting of novel object classes, that were unseen during training, based solely on textual and/or visual conditioning.
Read the full abstract ▾
Computer Vision
Unified Multimodal Uncertain Inference
Dengjia Zhang, Alexander Martin, William Jurayj et al. (+3 more)
We introduce Unified
Multimodal Uncertain Inference (UMUI), a
multimodal inference task spanning text, audio, and
video, where models must produce calibrated probability estimates of hypotheses conditioned on a premise in any modality or combination. While uncertain inference has been explored in text, extension to other modalities has been limited to single-modality binary entailment judgments, leaving no framework for fine-grained probabilistic
reasoning in or across other modalities. To address this, we curate a human-annotated evaluation set with scalar probability judgments across audio, visual, and audiovisual settings, and additionally evaluate on existing text and audio
benchmarks. We introduce CLUE (Calibrated Latent Uncertainty Estimation), which combines self-consistent teacher calibration and distribution-based confidence probing to produce calibrated predictions. We demonstrate that our 3B-parameter model achieves equivalent or stronger performance than zero-shot baselines up to 32B parameters across all modalities.
Read the full abstract ▾
All authors Dengjia Zhang, Alexander Martin, William Jurayj, Kenton Murray, Benjamin Van Durme, Reno Kriz
Computer Vision
EfficientSign: An Attention-Enhanced Lightweight Architecture for Indian Sign Language Recognition
Rishabh Gupta and Shravya R. Nalla
How do you build a sign
language recognizer that works on a phone? That question drove this work. We built EfficientSign, a lightweight model which takes EfficientNet-B0 and focuses on two
attention modules (Squeeze-and-Excitation for channel focus, and a spatial
attention layer that focuses on the hand gestures). We tested it against five other approaches on 12,637 images of Indian Sign
Language alphabets, all 26 classes, using 5-fold cross-validation. EfficientSign achieves the accuracy of 99.94% (+/-0.05%), which matches the performance of ResNet18's 99.97% accuracy, but with 62% fewer parameters (4.2M vs 11.2M). We also experimented with feeding deep features (1,280-dimensional vectors pulled from EfficientNet-B0's pooling layer) into classical classifiers. SVM achieved the accuracy of 99.63%, Logistic Regression achieved the accuracy of 99.03% and KNN achieved accuracy of 96.33%. All of these blow past the 92% that SURF-based methods managed on a similar
dataset back in 2015. Our results show that
attention-enhanced learning model provides an efficient and deployable solution for ISL recognition without requiring a massive model or hand-tuned feature pipelines anymore.
Read the full abstract ▾
Computer Vision
InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptation
Zhefan Rao, Bin Zou, Haoxuan Che et al. (+5 more)
Instruction-based
video editing is a natural way to control
video content with text, but adapting a
video generation model into an editor usually appears data-hungry. At the same time, high-quality
video editing data remains scarce. In this paper, we show that a
video generation backbone can become a strong
video editor without large scale
video editing data. We present InsEdit, an instruction-based editing model built on HunyuanVideo-1.5. InsEdit combines a visual editing architecture with a
video data pipeline based on Mutual Context
Attention (MCA), which creates aligned
video pairs where edits can begin in the middle of a clip rather than only from the first frame. With only O(100)K
video editing data, InsEdit achieves state-of-the-art results among open-source methods on our
video instruction editing
benchmarks. In addition, because our training recipe also includes image editing data, the final model supports image editing without any modification.
Read the full abstract ▾
All authors Zhefan Rao, Bin Zou, Haoxuan Che, Xuanhua He, Chong Hou Choi, Yanheng Li, Rui Liu, Qifeng Chen
Computer Vision
3D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decoding
Makanjuola Ogunleye, Eman Abdelrahman and Ismini Lourentzou
Large
multimodal models are increasingly used as the
reasoning core of
embodied agents operating in 3D environments, yet they remain prone to
hallucinations that can produce unsafe and ungrounded decisions. Existing inference-time
hallucination mitigation methods largely target 2D
vision-
language settings and do not transfer to
embodied 3D
reasoning, where failures arise from object presence, spatial layout, and
geometric grounding rather than pixel-level inconsistencies. We introduce 3D-VCD, the first inference-time visual contrastive decoding framework for
hallucination mitigation in 3D
embodied agents. 3D-VCD constructs a distorted 3D scene graph by applying semantic and
geometric perturbations to object-centric representations, such as category substitutions and coordinate or extent corruption. By contrasting predictions under the original and distorted 3D contexts, our method suppresses
tokens that are insensitive to grounded scene evidence and are therefore likely driven by
language priors. We evaluate 3D-VCD on the 3D-POPE and HEAL
benchmarks and show that it consistently improves grounded
reasoning without any retraining, establishing inference-time contrastive decoding over structured 3D representations as an effective and practical route to more reliable
embodied intelligence.
Read the full abstract ▾
Computer Vision
On Semiotic-Grounded Interpretive Evaluation of Generative Art
Ruixiang Jiang and Changwen Chen
Interpretation is essential to deciphering the
language of art: audiences communicate with artists by recovering meaning from visual artifacts. However, current Generative Art (GenArt) evaluators remain fixated on surface-level image quality or literal
prompt adherence, failing to assess the deeper symbolic or abstract meaning intended by the creator. We address this gap by formalizing a Peircean computational semiotic theory that models Human-GenArt Interaction (HGI) as cascaded semiosis. This framework reveals that artistic meaning is conveyed through three modes - iconic, symbolic, and indexical - yet existing evaluators operate heavily within the iconic mode, remaining structurally blind to the latter two. To overcome this structural blindness, we propose SemJudge. This evaluator explicitly assesses symbolic and indexical meaning in HGI via a Hierarchical Semiosis Graph (HSG) that reconstructs the meaning-making process from
prompt to generated artifact. Extensive quantitative experiments show that SemJudge aligns more closely with human judgments than prior evaluators on an interpretation-intensive fine-art
benchmark. User studies further demonstrate that SemJudge produces deeper, more insightful artistic interpretations, thereby paving the way for GenArt to move beyond the generation of "pretty" images toward a medium capable of expressing complex human experience. Project page: https://github.com/songrise/SemJudge.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
VOLTA: The Surprising Ineffectiveness of Auxiliary Losses for Calibrated Deep Learning
Rahul D Ray and Utkarsh Srivastava
Uncertainty quantification (UQ) is essential for deploying deep learning models in safety critical applications, yet no consensus exists on which UQ method performs best across different data modalities and distribution shifts. This paper presents a comprehensive
benchmark of ten widely used UQ baselines including MC Dropout, SWAG, ensemble methods, temperature scaling, energy based OOD, Mahalanobis,
hyperbolic classifiers, ENN, Taylor Sensus, and split conformal prediction against a simplified yet highly effective variant of VOLTA that retains only a deep encoder, learnable prototypes, cross
entropy loss, and post hoc temperature scaling. We evaluate all methods on CIFAR 10 (in distribution), CIFAR 100, SVHN, uniform noise (out of distribution), CIFAR 10 C (corruptions), and Tiny ImageNet features (tabular). VOLTA achieves competitive or superior accuracy (up to 0.864 on CIFAR 10), significantly lower expected calibration error (0.010 vs. 0.044 to 0.102 for baselines), and strong OOD detection (AUROC 0.802). Statistical testing over three random seeds shows that VOLTA matches or outperforms most baselines, with ablation studies confirming the importance of adaptive temperature and deep encoders. Our results establish VOLTA as a lightweight, deterministic, and well calibrated alternative to more complex UQ approaches.
Read the full abstract ▾
Computer Vision
WildDet3D: Scaling Promptable 3D Detection in the Wild
Weikai Huang, Jieyu Zhang, Sijun Li et al. (+15 more)
Understanding objects in 3D from a single image is a cornerstone of spatial intelligence. A key step toward this goal is
monocular 3D object detection–recovering the extent, location, and orientation of objects from an input RGB image. To be practical in the open world, such a detector must generalize beyond closed-set categories, support diverse
prompt modalities, and leverage
geometric cues when available. Progress is hampered by two bottlenecks: existing methods are designed for a single
prompt type and lack a mechanism to incorporate additional
geometric cues, and current 3D
datasets cover only narrow categories in controlled environments, limiting open-world transfer. In this work we address both gaps. First, we introduce WildDet3D, a unified geometry-aware architecture that natively accepts text, point, and box
prompts and can incorporate auxiliary depth signals at inference time. Second, we present WildDet3D-Data, the largest open 3D detection
dataset to date, constructed by generating candidate 3D boxes from existing 2D annotations and retaining only human-verified ones, yielding over 1M images across 13.5K categories in diverse real-world scenes. WildDet3D establishes a new state-of-the-art across multiple
benchmarks and settings. In the open-world setting, it achieves 22.6/24.8 AP3D on our newly introduced WildDet3D-Bench with text and box
prompts. On Omni3D, it reaches 34.2/36.4 AP3D with text and box
prompts, respectively. In zero-shot evaluation, it achieves 40.3/48.9 ODS on Argoverse 2 and ScanNet. Notably, incorporating depth cues at inference time yields substantial additional gains (+20.7 AP on average across settings).
Read the full abstract ▾
All authors Weikai Huang, Jieyu Zhang, Sijun Li, Taoyang Jia, Jiafei Duan, Yunqian Cheng, Jaemin Cho, Matthew Wallingford, Rustin Soraki, Chris Dongjoo Kim, Shuo Liu, Donovan Clay, Taira Anderson, Winson Han, Ali Farhadi, Bharath Hariharan, Zhongzheng Ren, Ranjay Krishna
Computer Vision · Cross-listed from cs.LG
From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneity
Zhuang Qi, Ying-Peng Tang, Lei Meng et al. (+4 more)
Exemplar replay has become an effective strategy for mitigating catastrophic forgetting in
federated continual learning (FCL) by retaining representative samples from past tasks. Existing studies focus on designing sample-importance estimation mechanisms to identify information-rich samples. However, they typically overlook strategies for effectively utilizing the selected exemplars, which limits their performance under continual dynamic heterogeneity across clients and tasks. To address this issue, this paper proposes a
Federated gEometry-Aware correcTion method, termed FEAT, which alleviates imbalance-induced representation collapse that drags rare-class features toward frequent classes across clients. Specifically, it consists of two key modules: 1) the
Geometric Structure
Alignment module performs structural knowledge
distillation by aligning the pairwise angular similarities between feature representations and their corresponding Equiangular Tight Frame prototypes, which are fixed and shared across clients to serve as a class-discriminative reference structure. This encourages
geometric consistency across tasks and helps mitigate representation drift; 2) the Energy-based
Geometric Correction module removes task-irrelevant directional components from feature
embeddings, which reduces prediction bias toward majority classes. This improves sensitivity to minority classes and enhances the model's robustness under class-imbalanced distributions.
Read the full abstract ▾
All authors Zhuang Qi, Ying-Peng Tang, Lei Meng, Guoqing Chao, Lei Wu, Han Yu, Xiangxu Meng
Computer Vision
MARINER: A 3E-Driven Benchmark for Fine-Grained Perception and Complex Reasoning in Open-Water Environments
Xingming Liao, Ning Chen, Muying Shu et al. (+5 more)
Fine-grained visual understanding and high-level
reasoning in real-world open-water environments remain under-explored due to the lack of dedicated
benchmarks. We introduce MARINER, a comprehensive
benchmark built under the novel Entity-Environment-Event (3E) paradigm. MARINER contains 16,629 multi-source maritime images with 63 fine-grained vessel categories, diverse adverse environments, and 5 typical dynamic maritime incidents, covering fine-grained classification, object detection, and visual question answering tasks. We conduct extensive evaluations on mainstream
Multimodal Large
language models (MLLMs) and establish baselines, revealing that even advanced models struggle with fine-grained discrimination and causal
reasoning in complex marine scenes. As a dedicated maritime
benchmark, MARINER fills the gap of realistic and cognitive-level evaluation for maritime
multimodal understanding, and promotes future research on robust
vision-
language models for open-water applications. Appendix and supplementary materials are available at https://lxixim.github.io/MARINER.
Read the full abstract ▾
All authors Xingming Liao, Ning Chen, Muying Shu, Yunpeng Yin, Peijian Zeng, Zhuowei Wang, Nankai Lin, Lianglun Cheng
Computer Vision
ViSAGE @ NTIRE 2026 Challenge on Video Saliency Prediction
Kun Wang, Yupeng Hu, Zhiran Li et al. (+3 more)
In this report, we present our champion solution for the NTIRE 2026 Challenge on
Video Saliency Prediction held in conjunction with CVPR 2026. To exploit complementary inductive biases for
video saliency, we propose
Video Saliency with Adaptive Gated Experts (ViSAGE), a multi-expert ensemble framework. Each specialized decoder performs adaptive gating and modulation to refine spatio-temporal features. The complementary predictions from different experts are then fused at inference. ViSAGE thereby aggregates diverse inductive biases to capture complex spatio-temporal saliency cues in
videos. On the Private Test set, ViSAGE ranked first on two out of four evaluation metrics, and outperformed most competing solutions on the other two metrics, demonstrating its effectiveness and generalization ability. Our code has been released at https://github.com/iLearn-Lab/CVPRW26-ViSAGE.
Read the full abstract ▾
All authors Kun Wang, Yupeng Hu, Zhiran Li, Hao Liu, Qianlong Xiang, Liqiang Nie
Computer Vision
ETCH-X: Robustify Expressive Body Fitting to Clothed Humans with Composable Datasets
Xiaoben Li, Jingyi Wu, Zeyu Cai et al. (+3 more)
Human body fitting, which aligns parametric body models such as SMPL to raw 3D point clouds of clothed humans, serves as a crucial first step for downstream tasks like animation and texturing. An effective fitting method should be both locally expressive-capturing fine details such as hands and facial features-and globally robust to handle real-world challenges, including clothing dynamics, pose variations, and noisy or partial inputs. Existing approaches typically excel in only one aspect, lacking an all-in-one solution. We upgrade ETCH to ETCH-X, which leverages a tightness-aware fitting paradigm to filter out clothing dynamics ("undress"), extends expressiveness with SMPL-X, and replaces explicit sparse markers (which are highly sensitive to partial data) with implicit dense correspondences ("dense fit") for more robust and fine-grained body fitting. Our disentangled "undress" and "dense fit" modular stages enable separate and scalable training on composable data sources, including diverse simulated garments (CLOTH3D), large-scale full-body motions (AMASS), and fine-grained hand gestures (InterHand2.6M), improving outfit generalization and pose robustness of both bodies and hands. Our approach achieves robust and expressive fitting across diverse clothing, poses, and levels of input completeness, delivering a substantial performance improvement over ETCH on both: 1) seen data, such as 4D-Dress (MPJPE-All, 33.0% ) and CAPE (V2V-Hands, 35.8% ), and 2) unseen data, such as BEDLAM2.0 (MPJPE-All, 80.8% ; V2V-All, 80.5% ). Code and models will be released at https://xiaobenli00.github.io/ETCH-X/.
Read the full abstract ▾
All authors Xiaoben Li, Jingyi Wu, Zeyu Cai, Siyuan Yu, Boqian Li, Yuliang Xiu
Computer Vision
GaussiAnimate: Rig Animatable Categories with Level of Dynamics
Jiaxin Wang, Dongxin Lyu, Zeyu Cai et al. (+4 more)
We propose Skelebones, a Scaffold-Skin Rigging System built on three steps: (1) Bones compress temporally consistent Gaussian or mesh sequences into free-form bones with smooth skinning weights, approximating non-rigid deformations via linear blend skinning (LBS); (2) Skeleton extracts the Mean
Curvature Skeleton (MCS) from the canonical shape and temporally refines its
topology and kinematics into a compact skeletal structure; and (3) Binding connects the skeleton and bones through non-parametric Partwise Motion Matching (PartMM), which synthesizes novel bone motions by matching, retrieving, and blending existing ones. Together, these steps compress the dynamics of 4D shapes into compact skelebones that are simultaneously controllable and expressive. The resulting representation is category-agnostic, meaning template-free; motion-adaptive, with dynamic
topology; and
topology-correct, with a skeleton consistent with the surface geometry. PartMM requires no learning. We validate our method on both synthetic and real-world
datasets, achieving substantial reanimation improvements on unseen poses: a 17.3 dB PSNR gain over LBS on DNA-Rendering and a 45.6 dB gain over Bag-of-Bones on ActorsHQ, while preserving high rendering fidelity for characters with complex non-rigid dynamics. PartMM generalizes robustly to both Gaussian and mesh representations, excelling in low-data regimes of approximately 1,000 frames, with a 48.4 RMSE improvement over LBS and improvements of more than 20 over GRU- and MLP-based methods. Code will be publicly released at https://cookmaker.cn/gaussianimate/.
Read the full abstract ▾
All authors Jiaxin Wang, Dongxin Lyu, Zeyu Cai, Zhiyang Dou, Cheng Lin, Anpei Chen, Yuliang Xiu
Computer Vision
When Numbers Speak: Aligning Textual Numerals and Visual Instances in Text-to-Video Diffusion Models
Zhengyang Sun, Yu Chen, Xin Zhou et al. (+4 more)
Text-to-
video diffusion models have enabled open-ended
video synthesis, but often struggle with generating the correct number of objects specified in a
prompt. We introduce NUMINA , a training-free identify-then-guide framework for improved numerical
alignment. NUMINA identifies
prompt-layout inconsistencies by selecting discriminative self- and cross-
attention heads to derive a countable latent layout. It then refines this layout conservatively and modulates cross-
attention to guide regeneration. On the introduced CountBench, NUMINA improves counting accuracy by up to 7.4% on Wan2.1-1.3B, and by 4.9% and 5.5% on 5B and 14B models, respectively. Furthermore, CLIP
alignment is improved while maintaining temporal consistency. These results demonstrate that structural guidance complements seed search and
prompt enhancement, offering a practical path toward count-accurate text-to-
video diffusion. The code is available at https://github.com/H-EmbodVis/NUMINA.
Read the full abstract ▾
All authors Zhengyang Sun, Yu Chen, Xin Zhou, Xiaofan Li, Xiwu Chen, Dingkang Liang, Xiang Bai
Computer Vision
Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models
Shilin Yan, Jintao Tong, Hongwei Xue et al. (+6 more)
The advent of agentic
multimodal models has empowered systems to actively interact with external environments. However, current
agents suffer from a profound meta-cognitive deficit: they struggle to arbitrate between leveraging internal knowledge and querying external utilities. Consequently, they frequently fall prey to blind tool invocation, resorting to reflexive tool execution even when queries are resolvable from the raw visual context. This pathological behavior precipitates severe latency bottlenecks and injects extraneous noise that derails sound
reasoning. Existing reinforcement learning protocols attempt to mitigate this via a scalarized reward that penalizes tool usage. Yet, this coupled formulation creates an irreconcilable optimization dilemma: an aggressive penalty suppresses essential tool use, whereas a mild penalty is entirely subsumed by the variance of the accuracy reward during advantage normalization, rendering it impotent against tool overuse. To transcend this bottleneck, we propose HDPO, a framework that reframes tool efficiency from a competing scalar objective to a strictly conditional one. By eschewing reward scalarization, HDPO maintains two orthogonal optimization channels: an accuracy channel that maximizes task correctness, and an efficiency channel that enforces execution economy exclusively within accurate trajectories via conditional advantage estimation. This decoupled architecture naturally induces a cognitive curriculum-compelling the
agent to first master task resolution before refining its self-reliance. Extensive evaluations demonstrate that our resulting model, Metis, reduces tool invocations by orders of magnitude while simultaneously elevating
reasoning accuracy.
Read the full abstract ▾
All authors Shilin Yan, Jintao Tong, Hongwei Xue, Xiaojun Tang, Yangyang Wang, Kunyu Shi, Guannan Zhang, Ruixuan Li, Yixiong Zou
Computer Vision · Cross-listed from cs.RO
SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worlds
Yunsong Zhou, Hangxu Liu, Xuekun Jiang et al. (+12 more)
Robotic manipulation with deformable objects represents a data-intensive regime in
embodied learning, where shape, contact, and
topology co-evolve in ways that far exceed the variability of rigids. Although simulation promises relief from the cost of real-world data acquisition, prevailing sim-to-real pipelines remain rooted in rigid-body abstractions, producing mismatched geometry, fragile soft dynamics, and motion primitives poorly suited for cloth interaction. We posit that simulation fails not for being synthetic, but for being ungrounded. To address this, we introduce SIM1, a physics-aligned real-to-sim-to-real data engine that grounds simulation in the physical world. Given limited demonstrations, the system digitizes scenes into metric-consistent twins, calibrates deformable dynamics through elastic modeling, and expands behaviors via
diffusion-based trajectory generation with quality filtering. This pipeline transforms sparse observations into scaled synthetic supervision with near-demonstration fidelity. Experiments show that policies trained on purely synthetic data achieve parity with real-data baselines at a 1:15 equivalence ratio, while delivering 90% zero-shot success and 50% generalization gains in real-world deployment. These results validate physics-aligned simulation as scalable supervision for deformable manipulation and a practical pathway for data-efficient policy learning.
Read the full abstract ▾
All authors Yunsong Zhou, Hangxu Liu, Xuekun Jiang, Xing Shen, Yuanzhen Zhou, Hui Wang, Baole Fang, Yang Tian, Mulin Yu, Qiaojun Yu, Li Ma, Hengjie Li, Hanqing Wang, Jia Zeng, Jiangmiao Pang
Computer Vision
E-3DPSM: A State Machine for Event-Based Egocentric 3D Human Pose Estimation
Mayur Deshmukh, Hiroyasu Akada, Helge Rhodin et al. (+2 more)
Event cameras offer multiple advantages in
monocular egocentric 3D human pose estimation from head-mounted devices, such as millisecond temporal resolution, high dynamic range, and negligible motion blur. Existing methods effectively leverage these properties, but suffer from low 3D estimation accuracy, insufficient in many applications (e.g., immersive VR/AR). This is due to the design not being fully tailored towards event streams (e.g., their asynchronous and continuous nature), leading to high sensitivity to self-occlusions and temporal jitter in the estimates. This paper rethinks the setting and introduces E-3DPSM, an event-driven continuous pose state machine for event-based
egocentric 3D human pose estimation. E-3DPSM aligns continuous human motion with fine-grained event dynamics; it evolves latent states and predicts continuous changes in 3D joint positions associated with observed events, which are fused with direct 3D human pose predictions, leading to stable and drift-free final 3D pose reconstructions. E-3DPSM runs in real-time at 80 Hz on a single workstation and sets a new state of the art in experiments on two
benchmarks, improving accuracy by up to 19% (MPJPE) and temporal stability by up to 2.7x. See our project page for the source code and trained models.
Read the full abstract ▾
All authors Mayur Deshmukh, Hiroyasu Akada, Helge Rhodin, Christian Theobalt, Vladislav Golyanik
Computer Vision
Scal3R: Scalable Test-Time Training for Large-Scale 3D Reconstruction
Tao Xie, Peishan Yang, Yudong Jin et al. (+8 more)
This paper addresses the task of large-scale 3D scene reconstruction from long
video sequences. Recent feed-forward reconstruction models have shown promising results by directly regressing 3D geometry from RGB images without explicit 3D priors or
geometric constraints. However, these methods often struggle to maintain reconstruction accuracy and consistency over long sequences due to limited
memory capacity and the inability to effectively capture global contextual cues. In contrast, humans can naturally exploit the global understanding of the scene to inform local perception. Motivated by this, we propose a novel neural global context representation that efficiently compresses and retains long-range scene information, enabling the model to leverage extensive contextual cues for enhanced reconstruction accuracy and consistency. The context representation is realized through a set of lightweight neural sub-networks that are rapidly adapted during test time via self-supervised objectives, which substantially increases
memory capacity without incurring significant computational overhead. The experiments on multiple large-scale
benchmarks, including the KITTI Odometry~\cite{Geiger2012CVPR} and Oxford Spires~\cite{tao2025spires}
datasets, demonstrate the effectiveness of our approach in handling ultra-large scenes, achieving leading pose accuracy and state-of-the-art 3D reconstruction accuracy while maintaining efficiency. Code is available at https://zju3dv.github.io/scal3r.
Read the full abstract ▾
All authors Tao Xie, Peishan Yang, Yudong Jin, Yingfeng Cai, Wei Yin, Weiqiang Ren, Qian Zhang, Wei Hua, Sida Peng, Xiaoyang Guo, Xiaowei Zhou
Computer Vision
Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Experts
Haolei Xu, Haiwen Hong, Hongxing Li et al. (+7 more)
Multimodal Mixture-of-Experts (MoE) models have achieved remarkable performance on
vision-
language tasks. However, we identify a puzzling phenomenon termed Seeing but Not Thinking: models accurately perceive image content yet fail in subsequent
reasoning, while correctly solving identical problems presented as pure text. Through systematic analysis, we first verify that cross-modal semantic sharing exists in MoE architectures, ruling out semantic
alignment failure as the sole explanation. We then reveal that visual experts and domain experts exhibit layer-wise separation, with image inputs inducing significant routing divergence from text inputs in middle layers where domain experts concentrate. Based on these findings, we propose the Routing Distraction hypothesis: when processing visual inputs, the routing mechanism fails to adequately activate task-relevant
reasoning experts. To validate this hypothesis, we design a routing-guided intervention method that enhances domain expert activation. Experiments on three
multimodal MoE models across six
benchmarks demonstrate consistent improvements, with gains of up to 3.17% on complex visual
reasoning tasks. Our analysis further reveals that domain expert identification locates cognitive functions rather than sample-specific solutions, enabling effective transfer across tasks with different information structures.
Read the full abstract ▾
All authors Haolei Xu, Haiwen Hong, Hongxing Li, Rui Zhou, Yang Zhang, Longtao Huang, Hui Xue, Yongliang Shen, Weiming Lu, Yueting Zhuang
Computer Vision
AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation
Ziwei Zhou, Zeyuan Lai, Rui Wang et al. (+6 more)
Text-to-Audio-
Video (T2AV) generation is rapidly becoming a core interface for media creation, yet its evaluation remains fragmented. Existing
benchmarks largely assess audio and
video in isolation or rely on coarse
embedding similarity, failing to capture the fine-grained joint correctness required by realistic
prompts. We introduce AVGen-Bench, a task-driven
benchmark for T2AV generation featuring high-quality
prompts across 11 real-world categories. To support comprehensive assessment, we propose a multi-granular evaluation framework that combines lightweight specialist models with
Multimodal Large
Language Models (MLLMs), enabling evaluation from perceptual quality to fine-grained semantic controllability. Our evaluation reveals a pronounced gap between strong audio-visual aesthetics and weak semantic reliability, including persistent failures in text rendering,
speech coherence, physical
reasoning, and a universal breakdown in musical pitch control. Code and
benchmark resources are available at http://aka.ms/avgenbench.
Read the full abstract ▾
All authors Ziwei Zhou, Zeyuan Lai, Rui Wang, Yifan Yang, Zhen Xing, Yuqing Yang, Qi Dai, Lili Qiu, Chong Luo
Computer Vision
OpenVLThinkerV2: A Generalist Multimodal Reasoning Model for Multi-domain Visual Tasks
Wenbo Hu, Xin Chen, Yan Gao-Tian et al. (+3 more)
Group Relative Policy Optimization (GRPO) has emerged as the de facto Reinforcement Learning (RL) objective driving recent advancements in
Multimodal Large
Language Models. However, extending this success to open-source
multimodal generalist models remains heavily constrained by two primary challenges: the extreme variance in reward
topologies across diverse visual tasks, and the inherent difficulty of balancing fine-grained perception with multi-step
reasoning capabilities. To address these issues, we introduce Gaussian GRPO (G$^2$RPO), a novel RL training objective that replaces standard linear scaling with non-linear distributional matching. By mathematically forcing the advantage distribution of any given task to strictly converge to a standard normal distribution, $\mathcal{N}(0,1)$, G$^2$RPO theoretically ensures inter-task gradient equity, mitigates vulnerabilities to heavy-tail outliers, and offers symmetric update for positive and negative rewards. Leveraging the enhanced training stability provided by G$^2$RPO, we introduce two task-level shaping mechanisms to seamlessly balance perception and
reasoning. First, response length shaping dynamically elicits extended
reasoning chains for complex queries while enforce direct outputs to bolster visual grounding. Second,
entropy shaping tightly bounds the model's exploration zone, effectively preventing both
entropy collapse and
entropy explosion. Integrating these methodologies, we present OpenVLThinkerV2, a highly robust, general-purpose
multimodal model. Extensive evaluations across 18 diverse
benchmarks demonstrate its superior performance over strong open-source and leading proprietary frontier models.
Read the full abstract ▾
All authors Wenbo Hu, Xin Chen, Yan Gao-Tian, Yihe Deng, Nanyun Peng, Kai-Wei Chang
Computer Vision
ParseBench: A Document Parsing Benchmark for AI Agents
Boyang Zhang, Sebastián G. Acosta, Preston Carlson et al. (+4 more)
AI
agents are changing the requirements for document parsing. What matters is semantic correctness: parsed output must preserve the structure and meaning needed for autonomous decisions, including correct table structure, precise chart data, semantically meaningful formatting, and visual grounding. Existing
benchmarks do not fully capture this setting for enterprise automation, relying on narrow document distributions and text-similarity metrics that miss
agent-critical failures. We introduce ParseBench, a
benchmark of ${\sim}2{,}000$ human-verified pages from enterprise documents spanning insurance, finance, and government, organized around five capability dimensions: tables, charts, content faithfulness, semantic formatting, and visual grounding. Across 14 methods spanning
vision-
language models, specialized document parsers, and LlamaParse, the
benchmark reveals a fragmented capability landscape: no method is consistently strong across all five dimensions. LlamaParse Agentic achieves the highest overall score at 84.9%, and the
benchmark highlights the remaining capability gaps across current systems.
Dataset and evaluation code are available on https://huggingface.co/
datasets/llamaindex/ParseBench and https://github.com/run-llama/ParseBench.
Read the full abstract ▾
All authors Boyang Zhang, Sebastián G. Acosta, Preston Carlson, Sacha Bron, Pierre-Loïc Doulcet, Daniel B. Ospina, Simon Suo