Computer Vision
VibeAvatar: Aligning Phonetic Kinematics and Human Aesthetics for High-Fidelity Talking Avatar Synthesis
Qilin Wang, Mingyu Li and Hao Tang
Multi-modal talking avatar synthesis aims to generate realistic talking
videos from a reference portrait and
speech. Despite rapid progress in
diffusion-based methods, existing approaches still struggle to jointly achieve accurate lip articulation, human-preferred motion aesthetics, and efficient inference. We observe that phonetic accuracy and motion aesthetics arise from fundamentally different sources and should be addressed at complementary stages rather than learned implicitly by a single generator. Based on this insight, we propose VibeAvatar, which disentangles these two objectives through a Phonetic Kinematics Adapter (PKA) that converts recognition-oriented
speech features into phonetic-kinematic conditions at the conditioning stage, and an Aesthetic Motion Policy (AMP) that optimizes a flow-consistent
stochastic sampling policy via Group Relative Policy Optimization (GRPO) at the post-training stage. With a lightweight flow-based motion generator operating in a compact 1D warp-based latent motion space, VibeAvatar achieves state-of-the-art results in articulation, aesthetics, and efficiency on both objective metrics and user studies, while generating a 10-second 512px
video in under 10 seconds with only $\sim$3GB VRAM.
Read the full abstract ▾
Computer Vision
FIVE-VLA: Fast and EffectIVE Autonomous Driving with Recurrent Action Memory
Kemal Oksuz, Alexandru Buburuzan, Yuhan Yao et al. (+1 more)
State-of-the-art
vision-
language-action models (VLA) for autonomous driving face critical limitations: excessive parameter counts, inefficient high-resolution image processing, and lack of temporal
memory. We introduce Fast and EffectIVE VLA (FIVE-VLA) to address these through two key contributions. First, we employ an efficient
vision encoder that processes high-resolution ($448 \times 896$) images while generating only 98
tokens, over $5\times$ fewer than existing approaches, and bypass text generation entirely for single-pass trajectory prediction. Second, we propose Recurrent Action
Memory (RAM), a lightweight module that conditions action prediction on previous action
tokens, providing temporal context critical for manoeuvres such as overtaking and emergency braking. With only 641M parameters, FIVE-VLA completes $\sim$10% more routes without traffic rule infractions than the previous state-of-the-art VLA on the challenging Bench2Drive closed-loop driving
benchmark. Non-reactive open-loop simulation on the large-scale real-world NVIDIA Physical AI AV
dataset shows 10.2% and 7.7% lower collision-violation rates than SimLingo in single- and four-view settings, respectively. Additionally, FIVE-VLA runs at $\sim$30 fps on an A100 and $\sim$4 fps on a T4 GPU (proxy to an edge device), representing an 8-30$\times$ speedup over previous methods.
Read the full abstract ▾
All authors Kemal Oksuz, Alexandru Buburuzan, Yuhan Yao, Puneet K. Dokania
Computer Vision
PULSE: Unlocking Practical Image Compression on Single-Thread CPU
Zhaoyang Jia, Tianyu Zhang, Zihan Zheng et al. (+5 more)
Despite recent progress in learned image compression, existing methods remain computationally expensive on resource-constrained hardware, particularly CPUs. We introduce PULSE, a practical codec that enables (1) low-latency decoding on diverse hardware platforms with an ultra-low-complexity 5.2 kMAC/pixel neural receiver, and (2) efficient bit-exact
entropy coding with an integer linear CDF predictor and a meta prior. To recover compression performance under this tight budget, we introduce an agentic
evolution process guided by heuristic probes that iteratively improves the architecture through human-LLM collaboration. PULSE decodes a 1080p image in 126 ms on a single CPU thread while achieving compression performance comparable to HM. After perceptual optimization, PULSE competes with larger perceptual codecs like MS-ILLM. Codes are at https://github.com/microsoft/GenCodec/tree/main/PULSE
Read the full abstract ▾
All authors Zhaoyang Jia, Tianyu Zhang, Zihan Zheng, Wenxuan Xie, Jiahao Li, Bin Li, Houqiang Li, Yan Lu
Computer Vision
On-the-Fly Homographies Calibration for Multi-Camera Tracking
David Voihanski, Mor Sinai and Ben Zion Bobrovsky
Precise multi-camera tracking traditionally relies on rigorous 3D site calibration, yet this requirement is often operationally impossible in large-scale deployments.
Privacy regulations frequently prohibit recording
video for offline calibration; limited bandwidth precludes synchronizing high-resolution streams from hundreds of cameras; and covering immense physical sites with calibration targets is logistically infeasible. We present a multi-camera homography calibration system designed to overcome these barriers through "on-the-fly"
geometric refinement. Starting from coarse manual homographies, we introduce a centroid-based projection optimization (PO) that continuously aligns the ground-plane geometry using live detection streams. Because PO operates asynchronously on already-transmitted, lightweight metadata, it adds zero computational latency to the real-time tracker. This allows the system to adapt automatically to camera movements or environmental changes without human intervention. This optimized geometry feeds a multi-camera bird's-eye-view (BEV) tracker that fuses detections and unifies trajectories across zones. Crucially, by operating strictly on live anonymous metadata, our solution ensures a
privacy-safe, zero-overhead, and resilient tracking pipeline that maintains global consistency in dynamic environments where static, recorded-
video calibration is impossible.
Read the full abstract ▾
Computer Vision
Learning Where to Focus: Self-Supervised Multi-Scale ViTs for Histopathology
Anabel Stammer, Valay Bundele, Mehran Hosseinzadeh et al. (+1 more)
Pathologists diagnose diseases by first locating suspicious tissue and then examining it at higher magnification, whereas self-supervised
vision transformers (ViTs) allocate the same spatial resolution to every image region despite diagnostic evidence being sparse and spanning multiple biological scales. Recent pathology foundation models have substantially improved representation quality by scaling training data and model capacity, but largely retain uniform tokenization. We instead investigate whether pathology representations can be improved by learning where to allocate spatial resolution during self-supervised learning. To this end, we propose CRAFT (Coarse-to-fine Region-Adaptive Feature Tokenization), a DINO-based framework that learns image-dependent mixed-scale representations by using self-supervised
attention to selectively refine informative regions while preserving coarse context, together with a symmetric cross-scale regularization objective that encourages complementary coarse and fine representations. Across CAMELYON16, TCGA-Lung subtype classification, and TCGA-LUAD survival prediction, CRAFT consistently outperforms comparable-scale self-supervised methods while requiring lower inference computation. Despite using only a compact 22M parameter backbone trained on comparatively small pathology
datasets, CRAFT remains competitive with, and often surpasses, substantially larger pathology foundation models.
Read the full abstract ▾
All authors Anabel Stammer, Valay Bundele, Mehran Hosseinzadeh, Hendrik P.A. Lensch
Computer Vision
Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA
Nguyen Hoai Thuong Bui, Thanh Nguyen Vo, Trinh Tra Giang Nguyen et al. (+1 more)
Track 2 of the AI City Challenge 2026 requires both visual question answering (VQA) and traffic event description generation under a challenging synthetic-to real domain shift. Existing
vision-
language approaches often entangle semantic understanding with
language generation, making them susceptible to
hallucination and inconsistent
reasoning across event phases. In this work, we propose a decoupled semantic understanding framework that first resolves predefined traffic questions into structured semantic facts and subsequently uses these facts to guide caption generation. A frozen V-JEPA encoder extracts predictive scene representations, while a lightweight Llama-based predictor produces answers for VQA queries. To improve reliability, we introduce a training-free structured refinement mechanism that exploits statistical priors, inter-question relationships, and temporal event consistency to correct prediction errors. The refined semantic facts are then provided to Qwen3-VL-8B to generate pedestrian and vehicle descriptions for each traffic event. Experimental results on the official 2026 AI City Challenge Track 2
benchmark show that the proposed method achieves 87.09% VQA accuracy and an overall S2 score of 60.0853, ranking first among all participating teams. These results demonstrate that predictive world representations combined with structured semantic refinement enable more accurate and reliable traffic understanding, leading to higher-quality lan guage generation.
Read the full abstract ▾
All authors Nguyen Hoai Thuong Bui, Thanh Nguyen Vo, Trinh Tra Giang Nguyen, Ha Duc Bui
Computer Vision
CARA: Collision-Aware Resolution Adaptation for Multiresolution Hash Encoding Based Image Fitting
Linfeng Ye, Zhixiang Chi, Shayan Mohajer Hamidi et al. (+2 more)
Multiresolution hash encodings have recently enabled fast and high-fidelity implicit neural representations by storing multi-scale features in fixed-size hash tables along a
geometric resolution schedule. However, the standard design is data-agnostic: different resolution levels receive identical hash-table capacity despite large differences in image frequency content. As a result, some levels experience severe hash collisions while others underutilize parameters, leading to inefficient capacity allocation. To address this issue, we propose Collision-Aware Resolution Adaptation (CARA), a method that assigns per-level resolutions by balancing the effective information load across hash levels. This adaptive allocation reduces capacity bottlenecks and improves parameter efficiency. In addition, we introduce an invertible pixel-shuffle transform that reduces hash load factors by redistributing spatial information, thereby mitigating collision-induced information loss without enlarging the hash tables. To support evaluation on extremely high-resolution data, we also curate, to the best of our knowledge, the first uncompressed whole-slide image
dataset for academic research. Experiments on Kodak images, gigapixel natural images, and raw whole-slide images demonstrate that CARA consistently improves the fidelity-parameter trade-off. Our method matches state-of-the-art performance while using only $27.76%$ of the parameters, and achieves up to $6.11$ dB PSNR improvement at comparable parameter counts. Code is provided in the supplementary.
Read the full abstract ▾
All authors Linfeng Ye, Zhixiang Chi, Shayan Mohajer Hamidi, En-hui Yang, Konstantinos N. Plataniotis
Computer Vision
HAP: A Hand-Driven Active Perception Framework for Egocentric Head Motion Prediction
Yunji Feng, Junyi Ma, Guanzhong Sun et al. (+2 more)
Egocentric motion
forecasting has primarily focused on hands and manipulated objects, leaving future human head motion comparatively underexplored. During manipulation, the head both redirects perception toward the target to acquire task-relevant evidence and coordinates with body and hand motion. We therefore formulate future six Degree of Freedom (6-DoF) head-motion prediction conditioned on observed hand motion and inferred target context, and propose HAP, a Hand-Driven Active Perception framework. HAP infers confidence for each target object from observed hand motion and object geometry. Then constructs a dynamic Predictive Target-Centric Amodal Occlusion Graph (P-TAOG) representing current and potential occlusion among candidate objects. Directed graph and causal temporal
reasoning encode the evolving target conditioned perceptual state, which is fused with hand and head motion history. A horizon-wise gate then blends the learned trajectory with a constant velocity prior. We further introduce Bottle, an
egocentric RGB-D
dataset of object manipulation toward specified targets, with coordinated head and hand motion under changing target visibility. Experiments on the public
dataset and Bottle show that HAP achieves lower head motion prediction errors than representative baselines, supporting the value of hand driven intention and dynamic occlusion
reasoning for anticipating human head motion. Code will be released at https://HAP-ego.github.io/HAP.
Read the full abstract ▾
All authors Yunji Feng, Junyi Ma, Guanzhong Sun, Chenyang Xu, Hesheng Wang
Computer Vision
STUNet-Fusion: Spatiotemporal Needle-Tip Localization in Ultrasound Video via Multi-Channel Motion Fusion
Chia-Chi Hsu, Chia-Hsuan Hsu and Che-Chou Shen
Needle-tip localization in ultrasound remains challenging because the needle may appear weak, discontinuous, or partially invisible, while imaging artifacts and anatomical structures can produce similar responses. To address this problem, we propose STUNet-Fusion, a spatiotemporal framework for needle-tip localization in ultrasound
videos. The proposed method formulates the input as a tri-channel spatio-temporal fusion tensor, comprising grayscale appearance, grid-based motion feature, and raw frame difference. A shared ResNet-34 encoder extracts spatial features, ConvLSTM integrates temporal dependencies, and a U-Net decoder reconstructs a dense probability heatmap. The final coordinates are extracted via a soft-argmax operation to achieve sub-pixel localization accuracy. Experimental results demonstrate that this spatiotemporal fusion strategy significantly improves localization robustness compared to conventional baselines.
Read the full abstract ▾
Computer Vision
Accuracy- and Real-Time-Aware 4D Radar Preprocessing for Autonomous Driving Perception Systems
Woo-Jin Jung, Dong-Hee Paek, Jeong-Su Park et al. (+1 more)
4D radar has emerged as a promising next-generation sensor for improving the robustness of autonomous driving perception systems because of its stable sensing capability under adverse weather conditions. However, deploying 4D radar in embedded environments with limited hardware resources requires radar-representation preprocessing that jointly considers perception accuracy, real-time performance, and computational complexity. This paper proposes a preprocessing framework for 4D-radar-based 3D object detection. First, Percentile-based 3D Shape Preservation (P3DP) extracts point clouds from radar tensors while preserving object-shape information and suppressing noise and false alarms. Second, Multi-frame-based Noise Point Discrimination using Kernel Density Estimation (MF-KDE) improves the density and reliability of sparse radar point clouds. Finally, Embedded & NetScore (ENS) evaluates suitability for embedded deployment by jointly considering accuracy, real-time performance, adverse-weather robustness, and model complexity.
Read the full abstract ▾
All authors Woo-Jin Jung, Dong-Hee Paek, Jeong-Su Park, Seung-Hyun Kong
Computer Vision
SVMemAgent: A Streaming Video Memory Agent for Query-Agnostic Online Frame Selection
Dohwan Ko, Ji Soo Lee, Pierce Chuang et al. (+7 more)
Most keyframe selection studies focus on offline settings, assuming access to the full
video and query in advance. In contrast, real-world streaming scenarios require online frame selection under unknown
video duration, without access to either the query or future frames during selection. To address this, we introduce Streaming
Video Memory (SVMem), a compact and representative
memory of previously observed content, updated continuously as the
video stream unfolds. Building on this setting, we propose the Streaming
Video Memory Agent (SVMemAgent), which dynamically maintains a
memory by deciding at each timestep whether to replace an existing
memory frame with the incoming frame or discard it. SVMemAgent is trained using Group Relative Policy Optimization (GRPO) with task-driven rewards derived from diverse question-answer pairs, implicitly exposing the policy to a distribution of queries during training so that SVMem retains generally informative frames at inference, when queries are unavailable. Experiments on both online and offline
video benchmarks show that SVMemAgent consistently outperforms online frame selection baselines and achieves competitive performance with offline methods that assume access to the full
video and query. Through task-driven rewards, SVMemAgent learns an emergent keyframe selection policy that prefers frames containing textual information, which may benefit downstream VideoQA tasks.
Read the full abstract ▾
All authors Dohwan Ko, Ji Soo Lee, Pierce Chuang, Debojeet Chatterjee, Ashish Shenoy, Yichao Lu, Seungwhan Moon, Xin Luna Dong, Vikas Bhardwaj, Hyunwoo J. Kim
Computer Vision
Learning from Distributed Eyes: Leveraging Collaborative Perception for Automated Model Adaptation
Yanan Ma, Yihang Tao, Zhengru Fang et al. (+4 more)
In autonomous driving, perception models often struggle to generalize to new environments due to domain shifts. While unsupervised model adaptation offers a feasible solution without labor-intensive manual labeling, existing methods that rely solely on the ego-vehicle's data often lead to inferior pseudo-labeling performance. To address this critical issue, we propose LDE, Learning from Distributed “Eyes", a novel framework that transforms collaborative perception (CP) into a source of high-quality supervision for model adaptation. This pseudo-labeling approach is hyperparameter-insensitive and relatively reliable, assuming CP often outperforms single-
agent's perception. However, naively implementing this approach encounters (1) the communication bottleneck of sharing rich features under time and bandwidth constraints, (2) the view discrepancy between the CP view and the learner's Field of View (FoV), and (3) the unreliability even in CP-generated labels. To address these issues, we design an adaptation-oriented feature sharing mechanism that selectively transmits the most critical information for adaptation, an FoV filtering method that meticulously eliminates mismatched labels, and a curriculum learning strategy to progressively exploit pseudo labels. Extensive experiments on 3D object detection tasks demonstrate that LDE consistently outperforms both the pre-trained models and state-of-the-art unsupervised adaptation methods.
Read the full abstract ▾
All authors Yanan Ma, Yihang Tao, Zhengru Fang, Zihan Fang, Yiqin Deng, Xianhao Chen, Yuguang Fang
Computer Vision
Antoine Dumoulin, Laurence Boissieux, Joao Regateiro et al. (+2 more)
We present DiT-Garment to model dynamic 3D clothing over human body models in arbitrary motion. Unlike existing methods, DiT-Garment can animate garments with unseen designs and physical materials, while allowing for direct inference of deformations for any target pose. To achieve this, we leverage a 2D
diffusion transformer architecture to learn 3D deformations in a 2D UV-space. As the result is non-deterministic, our generative model learns the distribution of possible outcomes. The template garment is represented as a 3D triangle mesh spatially aligned with a 3D human body model in a standardized pose. To work with different garment designs without the need of a common template or complex graph convolution operations, the
diffusion transformer is conditioned on a 3D position map of the template, represented in UV-space, which allows to implicitly learn a deformation of the 3D space around the body in standard pose. Further conditioning on body motion and physical parameters allows to physically ground the model. We quantitatively and qualitatively evaluate DiT-Garment on both synthetic and real data. While only trained on synthetic simulations of automatically generated cloth designs, our method generalizes to captured and artist-made garment designs. Code and data are available for research purposes at https://dumoulina.github.io/dit-garment/.
Read the full abstract ▾
All authors Antoine Dumoulin, Laurence Boissieux, Joao Regateiro, Pierre Hellier, Stefanie Wuhrer
Computer Vision
Semantic-ITC: A Frame-wise Indoor Mobile Laser Scanning Dataset and Benchmark for Semantic Segmentation
Haiyang Wu, Muhammad Affan, George Vosselman et al. (+1 more)
Semantic labels for indoor mobile
laser scanning (MLS) frames remain largely absent from current point cloud semantic
segmentation benchmarks, which mainly focus on reconstructed indoor scenes or outdoor
LiDAR perception. This paper introduces Semantic-ITC, to the best of our knowledge the first public
dataset and
benchmark for frame-wise indoor MLS semantic
segmentation. The
dataset contains 52 indoor sequences, 79,108 MLS frames, and 1.23 billion labeled points collected in classrooms, corridors, meeting rooms, offices, and study areas. Labels are attached directly to measured
LiDAR points in each frame using 16 semantic classes covering structural elements, furniture, room equipment, vegetation, and other indoor objects. Semantic-ITC preserves the sparse, non-uniform, and frame-wise sampling pattern of indoor MLS, making it distinct from scene-level reconstructed point clouds and mesh-based indoor
datasets. The annotations are produced by a hybrid workflow that combines predictions from a visual foundation model applied to synchronized RGB images, structural information from BIM, and manual refinement, with the final labels assigned to the original
LiDAR frames. A single-frame
benchmark is provided, and the best baseline reaches 79.27% mIoU. Remaining errors are concentrated around object boundaries and ambiguous indoor classes, indicating the challenges of indoor MLS
segmentation under sparse frame geometry and long-tailed class distributions. The
dataset provides a public
benchmark for evaluating semantic
segmentation directly on measured indoor MLS frames and supports future studies on frame-wise indoor MLS semantic
segmentation.
Read the full abstract ▾
All authors Haiyang Wu, Muhammad Affan, George Vosselman, Ville Lehtola
Computer Vision
Learning A Unified Template for Gait Recognition
Panjian Huang, Saihui Hou, Junzhou Huang et al. (+1 more)
"What I cannot create, I do not understand."Human wisdom reveals that creation is one of the highest forms of learning. For example,
Diffusion Models have demonstrated remarkable semantic structure and
memory in image generation, understanding, and restoration, which intuitively benefits representation learning. However, current gait networks rarely embrace this perspective, relying primarily on learning by contrasting gait samples under varying complex conditions, leading to semantic inconsistency and uniformity issues. To address these issues, we propose Origins with generative capabilities whose underlying philosophy is that different entities are generated from a unified template, inherently regularizing gait representations within a consistent and diverse semantic space to capture accurate gait differences. Admittedly, learning this unified template is exceedingly challenging, as it requires the comprehensiveness of the template to encompass gait representations with various conditions. Inspired by
Diffusion Models, Origins diffuses the unified template into timestep templates for gait generative learning, and meanwhile transfers the unified template for gait representation learning. Especially, gait generative and representation learning serve as a unified framework for end-to-end joint training. Extensive experiments on CASIA-B, CCPG,SUSTech1K, Gait3D, GREW and CCGR-MINI demonstrate that Origins performs unified generative and representation learning, achieving superior performance.
Read the full abstract ▾
All authors Panjian Huang, Saihui Hou, Junzhou Huang, Yongzhen Huang
Computer Vision
Beyond Random Couplings: Contrastive Noise Alignment in Generative Flows
Lennart Wittke and Vinicius Azevedo
Diffusion and flow-matching models are typically trained by corrupting data through independently sampled Gaussian noise. While simple and scalable, this forward process induces arbitrary data-noise couplings, forcing the network to learn high-
curvature transports between unrelated endpoints. Existing optimal-transport methods reduce this burden by reassigning fixed noise samples to data, but the source noise distribution itself remains passive. To address this, we introduce Contrastive Noise
Alignment (CNA), a training-time method that creates dynamic, contrastive couplings by optimizing the noise representations directly. By modeling the noise batch as an interacting particle system, CNA employs a cross-modal InfoNCE objective to align noise particles with their paired data targets. To prevent spatial collapse, this
alignment is regularized using an angular
entropy term and a radial norm penalty. We show theoretically that this
equilibrium asymptotically preserves Gaussian structures, maintaining tractability during inference. Empirically, CNA improves the
alignment between noise and data, reduces flow
curvature, and provides better generation quality with fewer required sampling steps. For few-step, pixel-space generation (2-4 NFEs), CNA reduces FID by over 50% compared to standard rectified flow, and by at least 24% against Optimal Transport baselines.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models
Shijie Lian, Bin Yu, Zhaolong Shen et al. (+5 more)
Action tokenizers play a central role in autoregressive
vision-
language-action (VLA) models, determining both the targets for policy training and the executable commands recovered from predicted
tokens. Their fidelity is commonly evaluated using pointwise reconstruction metrics such as mean squared error (MSE), yet small individual errors do not fully characterize how faithfully action adjustments across demonstrations are preserved. After compression, similar actions may still cluster around a representative motion, while the adjustments needed for different contexts are diminished, distorted, or even reversed. We introduce physical rank consistency (PRC) to measure how well tokenization preserves local physical distance rankings after reconstruction. Evaluating decoded actions provides a common reference across
token vocabularies and decoder architectures, complementing pointwise accuracy with a measure of relational fidelity. We further present ActionPiece, which preserves physical action relationships through joint supervision of representation learning and quantization. Physical rank preservation supervises near-far ordering in encoder and quantized feature distances, while quantization regularization applies the same ordering to codeword assignment distributions. Both objectives augment reconstruction, producing discrete action
tokens for standard autoregressive policy learning and execution through a frozen decoder. Under the same Qwen3-VL-4B policy training setup, ActionPiece achieves 94.8% on LIBERO and 68.8% on unseen LIBERO-Plus, with additional evaluations reaching 71.9% on SimplerEnv and 51.5% across VLA-Arena L0-L2. Component ablations show that the two objectives jointly improve PRC and policy success, demonstrating the value of physical relationship supervision for action tokenization.
Read the full abstract ▾
All authors Shijie Lian, Bin Yu, Zhaolong Shen, Xiaopeng Lin, Yichao Du, Zhirui Zhang, Laurence T. Yang, Kai Chen
Computer Vision
CADSplat: Sparse-View 3D Gaussian Splatting Aided by CAD Models for Robust, Photorealistic Digital-Twin Reconstruction
Kristof Overdulve, Lode Jorissen and Nick Michiels
We present CADSplat, a framework that reconstructs photorealistic, geometrically accurate digital twins from sparse ($<15$ views), wide-baseline posed images of an object by regularizing 3D Gaussian Splatting (3DGS) with an explicit CAD shape prior. Using such a prior requires finding a CAD model whose shape resembles the object depicted in the images and determining the pose of each camera relative to the object. We obtain both by matching segmented object silhouettes against silhouettes rendered from a CAD library and keeping the camera-to-object poses of the best-matching model. We then anchor 3D Gaussian primitives to the surface of the retrieved model and jointly optimize the 3DGS parameters, the camera-to-object registration, and a non-rigid deformation field to account for shape differences between the physical object and the CAD model. Across two real-world
datasets, CADSplat outperforms unconstrained, few-shot, and mesh-texturing baselines and degrades gracefully to as few as 3 views. Our experiments show that most of the gain in rendering quality comes from how the splats are constrained—a fixed set of splats tied to a surface and moved by a single smooth deformation field—rather than from the CAD shape itself. The CAD model adds shape knowledge where views are scarcest, in the sparsest captures and on strongly self-occluded objects, and it places every camera in the object's own frame. This enables applications beyond novel-view synthesis, such as markerless augmented reality registration, per-image object pose estimation, physical simulations, and the transfer of part labels from the design to the reconstruction.
Read the full abstract ▾
Computer Vision
GeoCond: A Conditioning-Aware Reliability Adapter for Feed-Forward 3D Reconstruction
David Ahmedt-Aristizabal, Mohammad Ali Armin, Russell Tsuchida et al. (+1 more)
Feed-forward 3D foundation models such as VGGT predict cameras, depth, and point maps in a single pass, but can fail silently under low overlap, low parallax, and extreme relative rotation. Stratified analyses over these factors show that these failures are governed by
geometric conditioning and are poorly captured by native aleatoric confidence. We introduce GeoCond, a lightweight reliability adapter for frozen feed-forward 3D backbones. GeoCond reads the backbone's predicted geometry and outputs pose-level uncertainty and a refinement gate. During training, it can be supervised by frame-permutation orbit variance, ground-truth pose error when labels are available, or cycle residuals from unlabelled independent pose graphs. At inference, the default head requires only one backbone pass and a small MLP. On VGGT, GeoCond improves out-of-distribution (OOD) AUSE (area under the sparsification-error curve; lower is better) from $0.32$ to $0.20$ over native confidence, transfers zero-shot to outdoor extreme-view scenes, and avoids the collapse caused by applying bundle adjustment uniformly. Across multiple backbones, cycle-distilled variants provide a ground-truth-free adaptation route, including cases where permutation variance vanishes on equivariant models. The same reliability signal supports gated refinement, pose-graph weighting, calibration, curation, and capture decisions. Reliable feed-forward 3D reconstruction requires not only predicting geometry, but also knowing when that geometry should be trusted.
Read the full abstract ▾
All authors David Ahmedt-Aristizabal, Mohammad Ali Armin, Russell Tsuchida, Lars Petersson
Computer Vision
CSWAM: Better Causal Semantic Representations for Out-of-Distribution Generalization in World Action Models
Tianbin Liu, Jian Zhu, Taiyi Su et al. (+5 more)
FastWAM-style world action models enable efficient action-only inference, but generalize poorly under visual distribution shifts. Their reconstruction-oriented representations emphasize appearance-specific details, limiting generalization to unseen scenes and objects. Without observation history, the model also lacks temporal evidence for robustly identifying task-relevant state changes and motion in unfamiliar visual conditions. To address these limitations, we present the Causal Semantic World Action Model (CSWAM), which augments FastWAM with a causal semantic expert built on V-JEPA 2.1. V-JEPA provides temporally grounded representations of semantic state changes and motion with less dependence on appearance-specific details. The expert learns their future
evolution from a sparse history of current and past observations and shares the history-derived context with both the
video and action streams through causal
attention. At inference, CSWAM conditions action denoising on the current
video state and observed semantic history, retaining efficient action-only inference. We conduct simulation and real-
robot experiments to evaluate generalization under distribution shifts. With
embodied pretraining, CSWAM raises Randomized success on RoboTwin 2.0 Clean-to-Randomized transfer from 10.16% to 45.18%, a gain of 35.02 percentage points over FastWAM. Across two real-
robot tasks and three OOD difficulty levels, CSWAM improves average success over FastWAM by 42.5 percentage points, from 27.5% to 70.0%.
Read the full abstract ▾
All authors Tianbin Liu, Jian Zhu, Taiyi Su, Jianjun Zhang, Chong Ma, Zitai Huang, Weiyi Lu, Yi Xu
Computer Vision
DR.WILSS: Diffusion-Based Replay for Weakly Supervised Continual Semantic Segmentation
Leon Arthur Marx, Francesco Barbato, Matteo Caligiuri et al. (+1 more)
Weakly supervised class-incremental semantic
segmentation (WILSS) aims to train a
segmentation model over multiple steps, each introducing new concepts to be learned with only image-level supervision. We introduce DR$.$WILSS, an innovative approach to address catastrophic forgetting in continual learning using
diffusion-based generative replay. Our framework leverages
language clues to guide the
diffusion process, employing self-inpainting and regularization techniques to efficiently produce replay data, aiding the learning process. By generating high-quality replay data, the information from previously learned classes can be preserved during continual updates, a critical challenge in incremental learning scenarios. To further align the statistics of replay data with those of training samples, we apply LoRAs to the generative model. Experimental results demonstrate state-of-the-art performance across multiple
benchmarks and generative architectures, while avoiding storage of training data and the use of additional resource-demanding tools during training. The proposed technique enables an optimal tradeoff between training complexity and inference-time accuracy, making DR$.$WILSS a promising solution for real-world applications.
Read the full abstract ▾
All authors Leon Arthur Marx, Francesco Barbato, Matteo Caligiuri, Pietro Zanuttigh
Computer Vision
Occluded Gait Recognition with Mixture of Experts: An Action Detection Perspective
Panjian Huang, Yunjie Peng, Saihui Hou et al. (+4 more)
Extensive occlusions in real-world scenarios pose challenges to gait recognition due to missing and noisy information, as well as body misalignment in position and scale. We argue that rich dynamic contextual information within a gait sequence inherently possesses occlusion-solving traits: 1) Adjacent frames with gait continuity allow holistic body regions to infer occluded body regions; 2) Gait cycles allow information integration between holistic actions and occluded actions. Therefore, we introduce an action detection perspective where a gait sequence is regarded as a composition of actions. To detect accurate actions under complex occlusion scenarios, we propose an Action Detection Based Mixture of Experts (GaitMoE), consisting of Mixture of Temporal Experts (MTE) and Mixture of Action Experts (MAE). MTE adaptively constructs action anchors by temporal experts and MAE adaptively constructs action proposals from action anchors by action experts. Especially, action detection as a proxy task with gait recognition is an end-to-end joint training only with ID labels. In addition, due to the lack of a unified occluded
benchmark, we construct a pioneering Occluded Gait database (OccGait), containing rich occlusion scenarios and annotations of occlusion types. Extensive experiments on OccGait, OccCASIA-B,Gait3D and GREW demonstrate the superior performance of GaitMoE.OccGait is available at https://github.com/BNU-IVC/OccGait.
Read the full abstract ▾
All authors Panjian Huang, Yunjie Peng, Saihui Hou, Chunshui Cao, Xu Liu, Zhiqiang He, Yongzhen Huang
Computer Vision
StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions
Awomo-WM Team: Enhui Ma, Kaiwen Guo, Tingrui Zhang et al. (+5 more)
Modeling physical dynamics, including how objects move, interact, and change state, is central to
video world models for
embodied AI. We present StrucPhysVideo, a family of
video world models that bridges physics-focused data curation with
language- and action-conditioned prediction of scene
evolution. Our data pipeline combines motion-aware
video segmentation, quality and content filtering, and physical relevance verification with structured annotations of objects, materials, and temporally localized interactions. By disentangling camera motion from object behavior and explicitly describing contact, deformation, and state transitions, the pipeline provides supervision grounded in observable physical events. Building on these data, we introduce StrucPhysVideo-TI2V, a sparse Mixture-of-Experts (MoE) text-image-to-
video model trained with a curriculum that progressively emphasizes physical dynamics while retaining general-domain
video data. StrucPhysVideo-TI2V achieves state-of-the-art performance on Physics-IQ Verified, scoring 45.5% and outperforming Cosmos3-Super-Image2
Video by 2.8 percentage points. Caption ablations across backbones further demonstrate the effectiveness of physics-focused supervision. We further extend StrucPhysVideo-TI2V to StrucPhysVideo-IA2V, an interactive image-action-to-
video world model that predicts visual outcomes from
robot end-effector commands. Action conditioning, causal autoregressive generation, and few-step
distillation enable incremental
robot rollouts with only four denoising steps. Together, StrucPhysVideo advances physical dynamics modeling from image- and
language-conditioned
video prediction toward action-driven interaction.
Read the full abstract ▾
All authors Awomo-WM Team: Enhui Ma, Kaiwen Guo, Tingrui Zhang, Wei Song, Yingshui Tan, Jianhua Xu, Tong Zhang, Kaicheng Yu
Computer Vision
Vocabulary-Guided Gait Recognition
Panjian Huang, Saihui Hou, Chunshui Cao et al. (+2 more)
What is a gait? Appearance-based gait networks consider a gait as the human shape and motion information from images. Model-based gait networks treat a gait as the human inherent structure from points. However, the considerations remain vague for humans to comprehend truly. In this work, we introduce a novel paradigm Vocabulary-Guided Gait Recognition, dubbed Gait-World, which attempts to explore gait concepts through human vocabularies with
Vision-
Language Models (VLMs). Although VLMs have achieved the remarkable progress in various
vision tasks, the cognitive capability regarding gait modalities remains limited. The success element in Gait-World is the proper vocabulary
prompt where this paradigm carefully selects gait cycle actions as Vocabulary Base, bridging the gait and vocabulary feature spaces and further promoting human understanding for the gait. How to extract gait features? Although previous gait networks have made significant progress, learning solely from gait modalities on limited gait databases makes it difficult to learn universal gait features for practicality. Therefore, we propose the first Gait-World model, dubbed {\alpha}-Gait, which guides the gait network learning with vocabulary knowledge from VLMs. However, due to the heterogeneity of the modalities, directly integrating vocabulary and gait features is highly challenging as they reside in different
embedding spaces. To address the issues, {\alpha}-Gait designs Vocabulary Relation Mapper and Gait Fine grained Detector to map and establish vocabulary relations in the gait space for detecting corresponding gait features. Extensive experiments on CASIA-B, CCPG, SUSTech1K, Gait3D and GREW reveal the potential value and research directions of vocabulary information from VLMs in the gait field.
Read the full abstract ▾
All authors Panjian Huang, Saihui Hou, Chunshui Cao, Xu Liu, Yongzhen Huang
Computer Vision
Prosthesis-Aware 3D Human Pose Estimation: A Dataset and Benchmark for RSP Users
Yilin Wen, Kechuan Dong, Fumiya Suginaka et al. (+2 more)
Recovering 3D human body motion from
video is important for applications such as rehabilitation assessment and sports performance evaluation. For prosthesis users, this requires capturing both natural body joints and the geometry of the prosthetic device, a challenge that existing methods are not designed to address. Model-based
estimators rely on body models trained on non-amputee individuals and cannot represent prosthesis geometry, while model-free methods lack body kinematic priors and are unreliable under occlusion. This challenge is particularly prominent for users of running-specific prostheses (RSPs), where the RSP has a complex curved geometry and moves dynamically during exercise. To fill this gap, we collect RSP3D, the first 3D
dataset of RSP users, covering essential daily-life and exercise actions from participants with varied amputation conditions, using a multi-camera marker-based motion capture setup. We formally define the task of prosthesis-aware 3D pose estimation, evaluate representative methods in a zero-shot setting, and confirm their individual limitations. We further propose a hybrid baseline combining model-based body joint estimation with model-free RSP shape recovery, establishing a starting point for future research.
Read the full abstract ▾
All authors Yilin Wen, Kechuan Dong, Fumiya Suginaka, Ken Endo, Yusuke Sugano
Computer Vision
A Non-Linear Neuron Based Detection of Isolated Pixels in Binary and Grayscale Images using Contrast Sensitive Receptive Fields
Nassir Mohammad
Identifying isolated points is important in image processing applications such as medical imaging, astronomy and quality control management. Other domains, such as cybersecurity, also present challenges that can be framed as image processing problems. One example of particular interest is the identification of anomalous single nodes in spatially organised networks where groups of nodes in different regions share similar feature values. This task can involve both binary and more complex grayscale images. However, existing methods face limitations: template matching is infeasible for grayscale images, while 2nd order derivative based methods are highly sensitive to noise and require user-specified thresholds. To overcome these issues, a novel method is proposed for detecting meaningful single-pixel deviations in images. This approach modifies and extends a
neuron model, originally designed for anomaly detection, to operate on spatially diameter limited receptive fields that incorporate excitatory and inhibitory regions. The result is a method that is free from user-specified thresholds and parameters, and can be applied to both binary and grayscale images, providing an effective, robust and efficient solution.
Read the full abstract ▾
Computer Vision
MSR: Multiple Subject Reference for Video Generation
Guannan Li, Jiaji Chen, Jingyuan Liao et al. (+2 more)
Conditioning a
video generator on multiple images requires preserving appearance while associating each reference with its intended role. We present MSR (Multiple Subject Reference), a slot-aware conditioning scheme for LTX-based
video generation. Each reference image is independently encoded as a static clip and represented by a separate latent-
token group. A compact
Fourier-feature multilayer perceptron adds a numeric slot
embedding, while slot-dependent temporal offsets modify the group's rotary coordinates. The reference groups are prepended to noisy target
tokens and serve as clean context during target-only flow-matching training. We implement this scheme through low-rank adaptation and release the resulting weights and inference workflows. Qualitative examples demonstrate compositions containing distinct characters and referenced environments in realistic and stylized scenes. Development observations suggest reduced reference confusion relative to an earlier continuous-reference baseline, while similar clothing, complex garments, and viewpoint changes remain challenging. We describe the conditioning mechanism, the retained training configuration, and the observed strengths and limitations of the released system. A supplementary audio-reference experiment adds voice conditioning while keeping the visual parameters frozen.
Read the full abstract ▾
All authors Guannan Li, Jiaji Chen, Jingyuan Liao, Yu Geng, Baolan Qiu
Computer Vision
JigSync: Gauge-Resolved Synchronization for Jigsaw Reassembly under Unknown Piece Orientation
Soham Pahari, Antik Aich Roy and Ujjwal Bhattacharya
Square jigsaw reassembly requires recovering the spatial arrangement of shuffled fragments from their visual content and pairwise relationships. While recent studies have made substantial progress, existing
benchmarks typically assume that all fragments are provided upright, reducing reassembly to a permutation problem. We study the generalized problem in which each fragment may also have gone through an unknown rotation. For this setting we establish a gauge-unobservability
theorem: the minimum of the weighted least-squares objective is exactly invariant under a uniform global rotation of arbitrary magnitude, so no residual-based criterion can recover the global orientation. The
theorem further identifies how the issue of global orientation can be resolved: an orientation anchor estimated from the content of a single fragment, lying outside its scope, suffices. To address the above, we propose JigSync, which attains 63.8% and 31.8% absolute accuracy (AA) on GAP-3 and GAP-5, respectively, the highest reported on both, while additionally recovering a rotation per piece that neither
benchmark requires. We release JigSync, a degradation protocol that sweeps shape, erosion, photometry, grid size, and rotation independently.
Read the full abstract ▾
Computer Vision
Online Multi-Camera 3D Tracking via ID Prediction over Recurrent Sparse Queries
Pragyan Shrestha, Haruto Nakayama and Atom Scott
Online multi camera 3D tracking must maintain scene global identities across synchronized views, yet query-based trackers carry these identities only implicitly in the instance bank, where they fragment upon query interruption. We present an online architecture that recovers association accuracy by predicting IDs explicitly over recurrent sparse queries. An outside-in Sparse4D detector fuses calibrated views into world frame 3D detections while propagating a sparse query bank, and a causal MOTIP ID decoder associates detections against a finite trajectory
memory. We adapt MOTIP's relative-ID prediction and recycled slot runtime to globally fused 3D observations, and introduce metric spatial gating and proximity based newborn recovery. On the official 2026 AI City Challenge Track 1 test set, our method raises HOTA from 29.63 with native instance bank identities to 38.01, primarily through an AssA increase from 20.83 to 31.10, and ranks third on the public leaderboard. Full-sequence validation over all 9,000 frames of each scene shows that decoupled ID training improves HOTA over native identities, whereas continuing detector training alongside the detached ID objective produces scene-dependent gains and losses.
Read the full abstract ▾
Computer Vision
Visual Input and Its Framing Affect Undermine the Stability of Attribute-based Descriptions Produced by Large Vision-Language Models
Xiaomeng Wang, Martha Larson, Ping Mei Vogels et al. (+1 more)
Large
vision-
language models (LVLMs) are commonly prompted with text alone or together with an image. In this paper, we find a new aspect of LVLM vulnerability in its response robustness: adding an image can unexpectedly affect responses even when text
prompts ask about a concept rather than the instance of the concept depicted in the image. For example, adding an image of an animal can change the generated attribute descriptions of its breed, and framing of the image further affects those changes. We study this new problem on seven LVLMs with image sets covering 30 animal breeds and eight food categories, using two types of framings that focus on the subject (subject-focused) or place it in a broader context (subject-in-situation). We draw the following main findings: (1) adding images increases physical-term proportions for animals (16.7% $\to$ 32.8%) and decreases sensory-term proportions for food (83.6% $\to$ 73.2%) among the most frequent terms, averaged over seven models and two framings on correctly recognized images. (2) attribute-term distributions differ significantly between the two framing types for 41.4% of animal and 71.4% of food model–concept pairs, with generally higher physical-term proportions for animals and sensory-term proportions for food under subject-focused than subject-in-situation framing.
Read the full abstract ▾
All authors Xiaomeng Wang, Martha Larson, Ping Mei Vogels, Zhengyu Zhao