Computer Vision · Cross-listed from eess.IV
Fast Trainable Multilinear Bases for Image Compression
Shiwen An, Zhongyi Ni, Huanhai Zhou et al. (+1 more)
The Discrete
Fourier Transform (DFT), the Discrete Cosine Transform (DCT), and their block-wise variants underpin most deployed image and
video codecs. Their effectiveness rests on three properties: their runtime is near-linear (up to a polylogarithmic factor) in the image size, they are exactly invertible, and they carry few to no parameters. We generalize these transforms to
dataset-adaptive isometric multilinear bases that keep all three properties at the cost of a polylogarithmic number of parameters, by parameterizing each basis as an isometric tensor network, inspired by
quantum many-body theory, and training one shared basis per image
dataset with Riemannian optimization. Because the family contains the DFT and the DCT-IV (a variant of the DCT) as exact points, training can start from these transforms, and it improves on its starting point at the compression level it is trained for. We evaluate the bases on natural photographs and line drawings. On photographs, which the cosine basis already fits well, the best trained basis leads the DCT only under aggressive truncation. On Quick Draw line drawings, under a common codec at matched quality, the best trained basis stores each image in about 20% fewer bytes than the 8x8 block cosine transform of the Joint Photographic Experts Group (JPEG) format, not counting the one-off storage of the basis.
Read the full abstract ▾
All authors Shiwen An, Zhongyi Ni, Huanhai Zhou, Jin-Guo Liu
Computer Vision
DisasterTD: Disaster Toponym Disambiguation Using Multimodal LLMs and Cross-View Geolocalization
Wenping Yin, Ziqi Liu, Naixia Mou et al. (+3 more)
Social media imagery (SMI) provides timely and fine-grained ground perspectives that are valuable for situational awareness and emergency response. Unlike satellite or aerial imagery, SMI can capture disaster impacts and ground-level conditions in a timely manner. However, geographic references in SMI are often vague or ambiguous, making accurate geolocalization challenging. To address this issue, we propose DisasterTD, a disaster toponym disambiguation framework that integrates
multimodal large
language model (MLLMs)-based semantic
reasoning with cross-view geolocalization. First, MLLMs extract toponyms and generate candidate geolocations from noisy textual inputs. Then, cross-view matching between SMI, remote sensing imagery (RSI), and optionally street-view imagery (SVI) is used to verify and refine these candidate results. We evaluate DisasterTD on the Hurricane Harvey
dataset, where SMI is augmented with collected RSI and SVI to construct a cross-view
benchmark for disaster geolocalization. The
dataset is divided into four categories based on toponym clarity and ambiguity, allowing a fine-grained performance analysis across scenarios. Results show that DisasterTD consistently outperforms MLLM-only and cross-view-only baselines without disambiguation, achieving geolocalization accuracies of 71.62% within 1000 m, 62.36% within 500 m, 57.99% within 250 m, 52.09% within 100 m, and 47.01% within 50 m, while reducing the mean and median errors to 11.33 km and 0.68 km, respectively. The largest improvements appear in ambiguous toponyms, where semantic
reasoning with cross-view evidence reduces candidate dispersion and errors. These findings demonstrate the effectiveness of integrating MLLM-based candidate generation with cross-view verification for fine-grained disaster geolocalization.
Read the full abstract ▾
All authors Wenping Yin, Ziqi Liu, Naixia Mou, Weijia Li, Danfeng Hong, Hao Li
Computer Vision
Long-Tailed Medical Image Classification
Nathanael Ren and Saagar Arya
In this paper, we examine the difficulties of using standard techniques for medical image classification due to long-tailed distributions (wherein rarer conditions have very few samples) resulting in bias towards diagnosing common diseases and away from rarer diseases. We then discuss and implement deep learning models with techniques such as augmentation to minimize error, especially from rarer diseases. We evaluate various different models with AP, F1 score, AUROC, and loss (all on the validation set). We conclude with the promising results from our best model, and potential applications in the healthcare space.
Read the full abstract ▾
Computer Vision
Head Avatars with Dynamic Explicit Hair
Vanessa Sklyarova, Haonan Chen, Berna Kabadayi et al. (+8 more)
We present DynHair, a novel method for tracking and modeling dynamic hair for human head avatars. From
video input, we reconstruct a dynamic head avatar with an explicit strand-based hair representation using structured 3D Gaussian Splatting. In contrast to the face region of human head avatars, which can be modeled with 3D Gaussians that are attached or generated with respect to some expressive 3D head model, hair is particularly challenging as it exhibits dynamic motion effects. Therefore, we present a novel method that models the dynamic deformations of the hair strands using a temporal network that is conditioned on angular velocity and acceleration of the head, as well as relative
gravity. Specifically, an LSTM encodes the motion history and modulates per-point strand features via FiLM conditioning which further used by MLP to produce physically plausible displacements to canonical hairstyle. We jointly optimize this motion and appearance representation of the hair, with a 3DGS-based representation of the face-region, via differentiable Gaussian splatting with photometric,
geometric, and physics-based supervision. As a result of our method, we retrieve hair tracking of the training
video data and an animatable head avatar with controllable hair dynamics. In our experiments, we demonstrate state-of-the-art performance in terms of hair dynamics, temporal consistency, and generalization across subjects.
Read the full abstract ▾
All authors Vanessa Sklyarova, Haonan Chen, Berna Kabadayi, Tobias Kirschstein, Zicong Fan, Xi Wang, Gerard Pons-Moll, Matthias Nießner, Marc Pollefeys, Michael J. Black, Justus Thies
Computer Vision · Cross-listed from cs.LG
Controllable Diversity in Normalization-Based Implicit Ensembles via Softmax-Temperature Modulation
Mihai Suteu and Ovidiu Serban
Deep ensembles provide the most reliable uncertainty estimates in deep learning, but their cost grows linearly with the number of members. Implicit ensembles lower this cost by sharing a single backbone across members. Member diversity is a primary determinant of ensemble quality, yet no implicit ensemble can shape it during training; existing methods fix it at initialisation or build it into the architecture. We introduce $\sigma$N-Ens, a normalisation-based implicit ensemble that treats each member as a task in a multi-task architecture and modulates the shared backbone through sigmoid-bounded scalers. We also introduce a softmax-temperature regulariser, which shapes the
equilibrium level of sharing between members and traces the accuracy-calibration frontier. Because only normalisation layers are replicated, the mechanism can wrap convolutional and
transformer backbones alike, also allowing pretrained models to be adapted through a short fine-tune. We frame the epistemic uncertainty such an ensemble expresses as modulation uncertainty, and explain why its calibration holds under input corruption, and why its out-of-distribution detection is weaker. Our method is evaluated across ResNets and
transformers on CIFAR-10/100, ImageNet and SST-2. $\sigma$N-Ens matches or outperforms deep ensembles at a fraction of their parameter cost, scales with ensemble size where partitioning methods collapse, and maintains calibration under distribution shift.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.SD
OmniVAE: An Audio-Video VAE with Cross-Modal Alignment for Joint Generation
Jun Zhan, Chen Yang, Yitian Gong et al. (+23 more)
Recent generative models are moving beyond silent
video or standalone audio synthesis toward the joint generation of synchronized audio and
video. Despite this progress, jointly generating audio and
video with fine-grained cross-modal correspondence remains challenging due to their fundamental structural differences. Most existing methods use audio and
video VAEs trained separately. As a result, the two latent spaces lack cross-modal
alignment, leaving the downstream generative model to learn cross-modal synchronization from scratch. We present OmniVAE, a jointly trained audio-
video VAE that learns fine-grained semantic
alignment between audio and
video latent representations. Beyond reconstruction, OmniVAE uses a segment-level audio-
video contrastive objective to capture temporal-semantic correspondence and align the two latent spaces. In parallel, it distills features from pretrained modality-specific semantic encoders into each modality, improving the downstream learnability of both latent spaces. Extensive experiments show that both objectives consistently improve the learnability of the latent spaces, translating into higher generation quality and more accurate cross-modal synchronization in downstream text-to-audio-
video generation. These findings underscore the importance of learning unified representations as a foundation for omnimodal modeling.1
Read the full abstract ▾
All authors Jun Zhan, Chen Yang, Yitian Gong, Donghua Yu, Kuangwei Chen, Wenbo Zhang, Kexin Huang, Qi Luo, Zhe Xu, Ying Zhu, Jin Wang, Tengyue Zhang, Qi Chen, Cheng Chang, Songlin Wang, Junqi Dai, Jiasheng Ye, Xiaogui Yang, Tianyi Liang, Xiangyu Peng, Zhaoye Fei, Shimin Li, Qinyuan Cheng, Xie Chen, Xinchi Chen, Xipeng Qiu
Computer Vision
OmniCache: Multidimensional Hierarchical Feature Caching For Diffusion Models
Zhaoyuan He, Muhammad Muaz and Lili Qiu
High-resolution image and
video diffusion models, including SD3, FLUX, and recent
video diffusion transformers, have substantially improved generative quality but remain expensive at inference time because they repeatedly evaluate
attention-heavy denoisers over many sampling steps. We address this inefficiency by exploiting redundancy in intermediate
diffusion features rather than changing model weights or retraining. We identify four complementary redundancy sources in image and
video generation: intra-frame, inter-frame, motion, and denoising-step redundancy. Based on this analysis, we propose OmniCache, a unified hierarchical caching framework that performs multidimensional feature reuse through
Token Cache, Frame Cache, Block Cache, and Layered Cache. Unlike
token-merging baselines that average matched features, OmniCache uses similarity matching to select cacheable features, skips redundant computation, and restores positionally consistent cached activations, preserving feature order and spatial-temporal structure. The resulting framework reuses spatial features in temporal layers and temporal features in spatial layers, while Layered Cache captures cross-step redundancy at the model-layer level. Across SD3, SVD-XT, and Latte, OmniCache reduces inference latency by up to 35%, 25%, and 28%, respectively, while maintaining visual fidelity and motion coherence in a training-free setting.
Read the full abstract ▾
Computer Vision
STEER: Steerable Dyadic Head Avatars
Kartik Teotia, Helge Rhodin, Hyeongwoo Kim et al. (+2 more)
Facial movement and expression are central to face-to-face communication, conveying turn-taking,
attention, agreement, and engagement alongside
speech. While
speech-driven facial animation has made strong progress in lip synchronization and audio-conditioned motion generation, most methods treat conversational behavior as an emergent byproduct of audio, or expose only coarse sequence-level affect control. As a result, key non-verbal channels such as gaze contact and aversion, rhythmic head motion, and
emotion remain difficult to explicitly control. We present STEER, a controllable 3D dyadic motion prior for reactive conversational head avatars. STEER factorizes conversational behavior into explicit controls for gaze, head rhythm, and
emotion, allowing users to steer how an avatar listens, reacts, and engages with a conversation partner. Since temporally aligned annotations for these behaviors are not available in public dyadic corpora, we introduce a tracking and annotation pipeline that recovers behavioral pseudo-labels from in-the-wild dyadic
video. A causal flow-matching
transformer then learns partner-aware target motion conditioned on audio, partner motion,
emotion and the proposed behavioral controls. We further embed STEER in a photorealistic avatar pipeline by extending a Universal Gaussian Head-Avatar Prior with a learned mapping from tracked parametric motion into its avatar-driving space. This enables controllable animation of high-fidelity Gaussian head avatars without re-training the underlying avatar model. STEER outperforms recent dyadic motion baselines on motion quality, dynamics, and diversity, remains competitive on partner coupling, and enables gaze, head-rhythm, and
emotion edits together with an interactive live deployment. We make our code and
dataset annotations available at our webpage.
Read the full abstract ▾
All authors Kartik Teotia, Helge Rhodin, Hyeongwoo Kim, Marc Habermann, Christian Theobalt
Computer Vision
Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations
Xianghao Jiao, Ruoyu Chen, Wei Wang et al. (+6 more)
Attribution methods are widely used to characterize the evidence underlying model predictions, yet their potential to improve model behavior remains underexplored. Attribution inconsistency under label-preserving
geometric transformations may indicate transformation-sensitive evidence reliance, motivating attribution regularization. However, such supervision is valid only when attribution faithfully reflects the evidence driving predictions. Existing self-supervised methods typically align gradient-based maps such as Grad-CAM, whose limited faithfulness means that attribution consistency need not imply consistency of the underlying decision process, leaving transformation robustness unresolved. We propose an annotation-free attribution regularization framework based on submodular search over image regions. By measuring how candidate subsets affect model outputs, the search extracts compact, class-discriminative evidence as search-derived supervision. We further introduce a submodular ranking loss with path-consistency and termination-
alignment terms that respectively align spatially corresponding candidate rankings along paired search trajectories and encourage the transformed trajectory to satisfy the stopping criterion at the target terminal step. The loss provides a differentiable surrogate for regularizing both final attributions and the otherwise discrete evidence-selection process. Experiments on ImageNet-100 show that our method substantially improves attribution stability, Insertion, and Deletion on ViT-B/16 with only a 0.28-point accuracy drop, with similar gains on ViT-L/16. On ImageNet-1K, it improves transformed-input accuracy on ResNet-50 and ConvNeXt-B while limiting the clean-accuracy drop to 0.30 points, demonstrating more consistent evidence reliance with minimal performance loss. Code will be released soon.
Read the full abstract ▾
All authors Xianghao Jiao, Ruoyu Chen, Wei Wang, Jiazi Hu, Jiawei Liang, Shangquan Sun, Shiming Liu, Qunli Zhang, Xiaochun Cao
Computer Vision
Beyond Appearance: A Multi-cue Framework and Large-scale Benchmark for Pedestrian Association and Tracking on Mobile Aerial-Ground Platforms
Ruiqi Wu, Bingliang Jiao, Ruize Han et al. (+6 more)
Multi-view Multi-object Association and Tracking (MvMoAT) associates objects across camera views and tracks them over time, supporting identity persistence and forensic trajectory reconstruction in multi-platform cooperative perception. Unlike conventional multiple object tracking, MvMoAT faces frequent viewpoint shifts that distort appearance and undermine cross-view association and temporal tracking. We propose FUSION, a viewpoint-robust Feature Unification framework for multi-view aSsociation and IdentificatiON. Its Multi-cue Adaptive Combination (MAC) module adaptively integrates viewpoint-invariant cues with appearance features to improve cross-view association, while Online Multi-view Feature Synchronization (OMFS) aggregates pedestrian features across historical and cross-view frames for temporally consistent tracking. We also introduce RealMvMoAT, a large-scale
benchmark featuring substantial inter- and intra-camera viewpoint variation. It contains 504.9K frames from 7 cameras (5
UAV and 2 ground views) across 10 scenes, with over 7.3M identity-labeled bounding boxes. All cameras exhibit random and substantial motion. To the best of our knowledge, RealMvMoAT is the largest MvMoAT
dataset to date. Its scale, viewpoint diversity, complex platform motion, and realistic trajectories provide a comprehensive resource for future research. Experiments on RealMvMoAT and six public
benchmarks show that FUSION achieves state-of-the-art performance.
Read the full abstract ▾
All authors Ruiqi Wu, Bingliang Jiao, Ruize Han, Hangzheng Yu, Xunkai Jiang, Shining Wang, Yuanqi Hu, Wenxuan Wang, Peng Wang
Computer Vision
PathScale-R1: Cross-scale Reasoning for Pathological Image Analysis
Chi Phan, Tianyi Zhang, Yufeng Wu et al. (+7 more)
Pathological diagnosis is inherently multi-scale, requiring the integration of global tissue architecture at low magnification with cellular morphology at higher magnification. However, existing pathology
benchmarks and
vision-
language models (VLMs) are still largely developed under single-scale settings, limiting their ability to learn clinically meaningful multi-magnification
reasoning. Moreover, naively constructed visual question answering (VQA) tasks may be susceptible to text-only or superficial visual shortcuts, leading to unreliable assessments of visual understanding. To address these limitations, we introduce a
benchmark and training framework for shortcut-resistant cross-scale pathology
reasoning. We design an
Adversarial Text-only Screening strategy for semantic
reasoning questions and a Structure-controlled Distractor Sampling strategy for visual grounding questions, encouraging models to rely on cross-scale visual evidence. Based on this pipeline, we construct PathScale-VQA, a high-quality cross-scale pathology VQA
benchmark with 10,373 multiple-choice questions grounded in 1,368 diagnostic paths across multiple magnification levels. Building on the semantic
reasoning set, PathScale-R1 is optimized through Difficulty-driven
Reasoning Distillation supervised fine-tuning followed by reinforcement learning with a Scale-aware
Reasoning Structure reward, which encourages the use of evidence across magnifications. Extensive experiments demonstrate state-of-the-art performance of PathScale-R1 on cross-scale
reasoning tasks and effective transfer to conventional single-scale pathology VQA. Our code is available at https://github.com/iMVR-PL/PathScale-R1.
Read the full abstract ▾
All authors Chi Phan, Tianyi Zhang, Yufeng Wu, Qiaochu Xue, Jiajie Zhang, Linghan Cai, Zeyu Liu, Sudong Wang, Yueming Jin, Dan Hu
Computer Vision
RoadVGGT: Road-Structure-Aware Feed-Forward Road Surface Reconstruction
Han Jiao, Chen Liu, Jiakai Sun et al. (+6 more)
Large-scale road surface reconstruction supports high-definition mapping, autonomous-driving perception, annotation, and simulation. Existing road-specialized optimization methods can produce high-quality road representations, but they typically require per-scene training and scene-dependent coverage design around the driving trajectory, limiting scalable reconstruction over newly collected roads. To address these limitations, we introduce RoadVGGT, a road-structure-aware feed-forward framework that reconstructs compact Gaussian road surfaces without test-time per-scene optimization. RoadVGGT uses a
geometric foundation model to exploit multi-view images together with provided pose and depth observations, and predicts dense pixel-aligned Gaussian attributes through a learned Gaussian head. To make these dense predictions usable for large road surfaces, we align them into a consistent metric world coordinate system and fuse redundant Gaussians on the road-aligned XY plane through confidence-weighted grid fusion. Category-aware grouping and road–sidewalk junction protection further control fusion around vulnerable road structures. The resulting representation supports RGB and semantic bird's-eye-view maps, elevation estimation, and novel view synthesis. RoadVGGT eliminates the need for per-scene optimization in prior methods, reconstructs complete road surfaces with a compact Gaussian representation, and improves image quality, semantic mapping, and elevation accuracy. Extensive experiments demonstrate the potential of
geometric foundation models for scalable feed-forward road surface reconstruction.
Read the full abstract ▾
All authors Han Jiao, Chen Liu, Jiakai Sun, Zhanjie Zhang, Mengyuan Yang, Yimeng Li, Mofan Zhou, Kun Zhan, Lei Zhao
Computer Vision
DAP-Pose: Deep Temporal Alignment and Physics-aware Cross-modal Sensor Fusion for Robust Pose Estimation
Jianhan Lin, Yuchu Qin, Jiateng Yuan et al. (+2 more)
Robust and accurate pose estimation with multi-modal sensors is fundamental for autonomous vehicles and mobile robotic systems in complex environments. In this paper, we propose DAP-Pose, a unified end-to-end model for robust multi-modal pose estimation. DAP-Pose introduces a Bi-level Cross-modal Fusion (BCF) module that captures complementary semantic and
geometric motion cues from visual, inertial, and GNSS measurements. To handle temporal offsets, we designed a Deep Temporal
Alignment (DTA) module that explicitly aligns asynchronous streams in latent space, enabling coherent motion modeling without strict hardware synchronization. Furthermore, we incorporate physics-aware constraints via
manifold geometry and GNSS-guided absolute metric scale, enforcing motion consistency and mitigating drift. Experiments upon the public KITTI
benchmark dataset were conducted to evaluate the performance of DAP-Pose against existing methods. DAP-Pose achieved the state-of-the-art performance, with the lowest average translation error ($t_{rel}$) of 1.31% and rotation error ($r_{rel}$) of 0.46$^{\circ}$. Furthermore, it accurately estimates poses and maintains robust performance under severe artificially injected temporal misalignment.
Read the full abstract ▾
All authors Jianhan Lin, Yuchu Qin, Jiateng Yuan, Wenbo Zhang, Shuai Gao
Computer Vision · Cross-listed from cs.LG
Source-Free Controlled Adaptation of Teachers for Continual Test-Time Adaptation
Anurag Roy, Riddhiman Moulick, Vinay Kumar Verma et al. (+2 more)
In many real-world scenarios, encountering continual shifts in domain during inference is very common. Consequently, continual test-time adaptation (CTTA) techniques leveraging a teacher-student framework have gained prominence, allowing models to adapt continuously even after deployment. In such a framework, a weight-averaged mean teacher is used to produce pseudo-labels from test data for self-training. The mean teacher gets updated as an exponential moving average of the student parameters using a high value of momentum that is kept fixed even if different distributions of test data are encountered. To combat the resulting drift of the model, we propose a novel controlled teacher adaptation methodology that dynamically sets a proper momentum value depending on the quality of the incoming data. Additionally, we estimate class prototypes from the source pretrained model to help align the target data as they come in. Importantly, our method does not require access to source data or its statistics at any stage of the pipeline, making it truly source-free. We perform extensive experiments on
benchmark datasets to demonstrate that our approach outperforms different state-of-the-art adaptation frameworks, many of which require access to source data.
Read the full abstract ▾
All authors Anurag Roy, Riddhiman Moulick, Vinay Kumar Verma, Saptarshi Ghosh, Abir Das
Computer Vision · Cross-listed from cs.RO
LabRobFail: A Benchmark for Robotic Failure Analysis in Chemical Self-driving Laboratory
Haobo Wang, Baoli Sun, Anqi Zou et al. (+8 more)
The deployment of
embodied agents in self-driving laboratories could accelerate scientific discovery, yet their reliability is constrained by the irreversible and safety-critical nature of chemical experiments. Progress is further hindered by scarce failure data and the lack of fine-grained evaluation protocols. To address these challenges, we introduce LabRobFail, a failure-centric framework for learning and evaluating robotic failure analysis in chemical laboratories. LabRobFail-Sim injects controllable failures at the control, physics, and semantic levels, enabling the construction of LabRobFail-Data, which contains over 20,000 trajectories across 70+ task scenarios, five failure categories, and 11 fine-grained failure types. LabRobFail-Bench evaluates six capabilities spanning task understanding, failure detection, temporal localization, severity assessment, failure classification, and actionable correction. We further develop LabRobFail-VLM, a domain-specialized
vision-
language model that generates structured failure diagnoses and recovery instructions. On seen environments, it achieves 90.83% failure-detection accuracy and 77.21% temporal-localization accuracy, substantially outperforming general-purpose VLMs. When integrated as a real-time supervisor, it improves downstream task success rates by 4-16 percentage points, demonstrating the value of fine-grained failure understanding for closed-loop recovery and reliable laboratory autonomy. Our code and data are available at https://github.com/Su-ISE-2001/SciRobo
Read the full abstract ▾
All authors Haobo Wang, Baoli Sun, Anqi Zou, Dongsheng Huang, Zelin Lv, Ning Wang, Rui Li, Dongzhan Zhou, Weiyu Guo, Zhihui Wang, Wanli Ouyang
Computer Vision
Parameter-Efficient Adaptation of SAM3 for Prompt-Driven Surgical Concept Segmentation
Changjing Liu, Yiming Huang, Beilei Cui et al. (+5 more)
Efficient
surgical segmentation empowers clinical diagnosis, intraoperative monitoring, and downstream robotic pipelines for reconstruction and simulation. Although
prompt-driven foundation models like Segment Anything Model 3 (SAM3) achieve strong
segmentation performance on natural images,
surgical data exhibits domain gaps against its pre-training data, resulting in degraded
segmentation accuracy. Furthermore, existing medical SAM methods require full-parameter fine-tuning, incurring heavy computational consumption and low efficiency. To address these limitations, this work proposes a parameter-efficient Low-Rank Adaptation (LoRA) adaptation of SAM3 for
surgical concept
segmentation. We inject low-rank adapters into the
prompt encoder, detector and tracker while fully freezing the
vision backbone, which only optimizes 0.98% of the total model parameters and supports training on a single consumer GPU. Comprehensive experiments demonstrate that our method consistently outperforms zero-shot SAM3 and other mainstream baselines, and the generated
segmentation results can be directly deployed to support downstream robotic
surgical scene reconstruction and physical simulation pipelines.
Read the full abstract ▾
All authors Changjing Liu, Yiming Huang, Beilei Cui, Liangjing Shao, Long Bai, Yanheng Li, Haoxuan Che, Hongliang Ren
Computer Vision
GNM Head: A Generative aNthropometric Model of the human head
Stylianos Ploumpis, Jan Bednarik, Gaspard Zoss et al. (+29 more)
Parametric models of the human head are essential tools traditionally used in computer
vision and graphics for animation, rendering, and reconstruction. More recently, they serve as crucial conditioning signals within generative large
vision models, allowing for tight spatial control of generated imagery. However, existing publicly available models are typically limited in anatomical scope, modeling only outer geometry while ignoring intra-oral and ocular structures, and frequently suffer from reduced
geometric quality stemming from low-fidelity input
datasets. In this report we introduce a new parametric model dubbed Generative aNthropometric Model (GNM), named as a homophone of the human genome. GNM encompasses the head, face, neck, eyeballs, teeth, and tongue, and it is built on an extensive database of high-resolution 3D scans combined with high-quality anatomy specific artist-made samples. This report details the data provenance, the model architecture including the specialized sub-models for the ocular and intra-oral structures, and shows its SotA performance on fitting target 3D face scans. To foster community innovation, the complete GNM framework is made publicly available.
Read the full abstract ▾
All authors Stylianos Ploumpis, Jan Bednarik, Gaspard Zoss, Ruslan Guseinov, Luca Prasso, Prashanth Chandran, Oliver Boyne, Vasileios Choutas, Timo Bolkart, Daoye Wang, Menglei Chai, Di Qiu, Sebastian Winberg, Gilles Rainer, Lewis Bridgeman, Leonhard Helminger, Edo Collins, Delio Vicini, Jérémy Riviere, Yannick Boetzel, Alexander Koumis, Stylianos Moschoglou, Jay Busch, Cynthia Herrera, Jacob Still, Scott Ysebert, Peter Lincoln, Sergio Orts Escolano, Christoph Rhemann, Erroll Wood, Thabo Beeler, Stefanos Zafeiriou
Computer Vision
Perturbation-Aware Diffusion-Guided Hybrid Segmentation for Robust and Annotation-Efficient Plant Stress Phenotyping
Gurbhit Chaurakoti and Soumyashree Kar
Semantic
segmentation in agricultural imagery is often evaluated under in-domain protocols, yet practical deployment requires robustness to appearance perturbations, limited annotations, and cross domain shift. This paper presents a
diffusion-guided hybrid
segmentation framework in which U-Net, DeepLabV3+, and SegFormer backbones generate coarse masks that are refined by Denoising
Diffusion Probabilistic Models (DDPM), latent
diffusion, or semantic-guided
diffusion. The framework is evaluated through a 3x3 architectural screening study on PlantSegV3, followed by boundary-constrained optimization, perturbation-guided retraining, low-data evaluation, constrained hyperparameter screening, and controlled cross-domain adaptation. On PlantSegV3, the best selected hybrid model achieves 71.83% refined mean Intersection-over-Union (mIoU) and 26.10% refined Boundary-F1, and the selected models remain stable under substantially reduced supervision, demonstrating strong annotation efficiency. Perturbation analysis identifies grayscale conversion, fog, coarse dropout, and shadow as the most disruptive appearance shifts, and the resulting augmentation policy substantially improves robustness during retraining. The adapted models further show effective transfer to external agricultural
datasets under limited target supervision, indicating that
diffusion refinement and boundary-aware optimization provide transferable structural priors. Overall, the results show that carefully matched backbone-refiner pairings, combined with perturbation-aware retraining, can improve structural delineation and robustness under realistic resource and distribution constraints.
Read the full abstract ▾
Computer Vision
Contrastive Parameter Disentanglement for Multi-modal Remote Sensing Image Generation
Yu Zhang, Wenda Zhao, Haojun Tang et al. (+1 more)
Existing remote sensing image generation methods are largely confined to single-modality synthesis and therefore fail to exploit the complementary information inherent in
multimodal imagery. To address this limitation, we propose a contrastive parameter disentanglement framework for
multimodal remote sensing image generation, which generates semantically consistent and structurally aligned images across multiple modalities, including optical, infrared, and synthetic aperture radar (SAR), from a single text
prompt. Specifically, we introduce a contrastive parameter disentanglement module that disentangles shared semantics from modality-specific attributes at the parameter level within an orthogonal core subspace. Based on this module, we develop a disentangled optimization strategy that first constrains the parameter
matrix A of the LoRA adapter to capture modality-invariant semantics through a
multimodal contrastive objective and then guides multiple parameter
matrices B to learn modality-specific attributes under text conditioning. This strategy enables the simultaneous generation of
multimodal images with consistent semantic content and distinct modality characteristics. Furthermore, to ensure structural
alignment across the generated images, we devise a query-key structure transfer mechanism that jointly models
multimodal sampling trajectories during inference by transferring structural correlation priors from an anchor modality to the remaining modalities. Extensive experiments demonstrate that our method outperforms state-of-the-art remote sensing image generation approaches in terms of generation quality, semantic consistency, and structural
alignment, while also achieving superior performance in the downstream object classification task.
Read the full abstract ▾
All authors Yu Zhang, Wenda Zhao, Haojun Tang, Haipeng Wang
Computer Vision
RRTrack: Robust and Recoverable Object 6D Pose Tracking for Dynamic Scenes
Junyue Li, Ye Zheng, Yifan Chen et al. (+2 more)
Robust object 6D pose tracking is critical for robotic systems operating in dynamic and occluded scenes. Per-frame
estimators are accurate but computationally expensive, while current trackers struggle with fast motion and complete occlusion due to their reliance on continuous visibility. To address these challenges, we present RRTrack, an efficient, recoverable object 6D pose tracker that enables robust tracking through fast motion and target disappearance–reappearance. RRTrack introduces a 2D–6D closed-loop tracking strategy that integrates
memory-based
video object
segmentation (VOS) with 6D pose refinement. The 2D branch maintains target localization, and the 6D branch verifies
geometric consistency before
memory updates. In addition, a DINOv2-based dual-bank template matching module is developed to recover lost targets by jointly exploiting offline synthetic templates and online observation anchors while maintaining real-time efficiency. We also introduce a synthetic RGB-D
benchmark comprising three robotic scenarios with fast motion and full occlusion. Experimental results on the synthetic
benchmark demonstrate that RRTrack improves equal-subset mean ADD-S AR by 66.3% and ADD-S AUC by 65.7% over FoundationPose while achieving 55.2 FPS. Real-world experiments further validate the robustness of RRTrack under noisy sensing conditions. Project page: https://github.com/7kevin24/RRTrack
Read the full abstract ▾
All authors Junyue Li, Ye Zheng, Yifan Chen, Zhe Sun, Xuelong Li
Computer Vision
XMatchAD: A Cross-Modal Matching Perspective on Reconstruction-based Anomaly Detection
Mingxiu Cai, Zhe Zhang, Gaochang Wu et al. (+1 more)
The remarkable success of reconstruction-based methods in Unsupervised Anomaly Detection (UAD) lies in their ability to identify and localize anomalies by modeling discrepancies between input images and their reconstructed counterparts. However, these approaches often struggle to capture subtle anomalies and tend to produce blurred anomaly boundaries, which significantly limits their effectiveness, particularly in complex multi-class scenarios. To address these issues, we present XMatchAD, a novel UAD framework that reinterprets the task from a pseudo cross-modal matching perspective. Specifically, the input and reconstructed images are treated as two complementary modalities and their matching relationships are precisely exploited for anomaly detection. First, a pre-trained feature extractor is employed to encode discriminative representations. Second, an
attention-guided cross-modal matching mechanism is introduced to match local inter-modal anomaly-related patterns while mutually refining the features. This enhances the sensitivity to anomalies with diverse shapes and subtle deviations and significantly improves the precision of anomaly detection and localization. Third, we design an adaptive frequency-aware fusion module that further delineates sharp anomaly boundaries through the coupling of high-frequency components from cross-modal multi-scale representations. Comprehensive evaluations on MVTec-AD, VisA, and MPDD
benchmarks demonstrate that our method consistently achieves superior performance, outperforming state-of-the-art methods in multi-class anomaly detection and localization. The code will be released at https://github.com/Mingxiu-Cai/XMatchAD.
Read the full abstract ▾
All authors Mingxiu Cai, Zhe Zhang, Gaochang Wu, Tianyou Chai
Computer Vision
GRAPE: Graduated Routing for Articulated Portrait mesh Estimation
Yunfei Liu, Lijian Lin, Ye Zhu et al. (+1 more)
Articulated portrait mesh estimation is fundamental to 3D understanding, avatar generation, and immersive interaction. Existing approaches primarily rely on 3D Morphable Models (3DMMs). However, face-centric models suffer from the "floating head" assumption, conflating head pose with global rotation due to the lack of neck kinematics. Conversely, body-centric models lack high-fidelity facial expression capabilities. Furthermore, current methods struggle to disentangle jaw articulation from expression blendshapes, often over-relying on expressions for mouth opening. These limitations make
monocular portrait recovery difficult across representation, supervision, and anatomical parameter estimation. To address these limitations, we introduce GRAPE(Graduated Routing for Articulated Portrait mesh Estimation). We build a Portrait Parametric Model (PPM) with an explicit torso-to-head kinematic chain and a canonical injection step to merge FLAME and the SMPL-X torso. We propose a Progressive Anatomical
Alignment (PAA) network, which is composed of a pretrained portrait encoder, a Graduated-Mask Router, and coarse-to-fine experts that follow the portrait anatomical prior. We then train this network with multi-source supervision that combines sparse anatomical keypoints, feature
distillation, foreground mask constraints, and relative geometry constraints. Experiments show that GRAPE improves portrait mesh recovery quality, pose
alignment, and jaw–expression disentanglement over prior methods. We also demonstrate that our method can benefit the downstream tasks of audio-driven talking-head generation and 3D portrait generation.
Read the full abstract ▾
All authors Yunfei Liu, Lijian Lin, Ye Zhu, Yu Li
Computer Vision
WGDnet: Wishart-guided Geometric-aware Deep Network for PolSAR Image Classification
Junfei Shi, Haojia Zhang, Yu Cheng et al. (+1 more)
Polarimetric Synthetic Aperture Radar (PolSAR) classification underpins all-weather Earth observation. Conventional Wishart methods depend on rigid handcrafted operators with limited adaptability, while mainstream deep networks ignore PolSAR native Wishart scattering statistics. Additionally, fixed convolution windows fail to capture multi-scale, multi-directional terrain patterns, harming boundary detection and small-object characterization. To mitigate these drawbacks, we propose WGDNet, a Wishart-guided
geometric-aware deep network. It integrates three core designs: (1) learnable Wishart convolutions with directional kernels for multi-scale statistical edge feature extraction; (2) an orientation-prior aggregation module that estimates dominant local directions and confidences to refine directional Wishart outputs adaptively; (3) GAnet, a scale-direction adaptive
geometric-aware convolution that dynamically reshapes sampling grids to model anisotropic terrain and retain fine details. Our contributions lie in learnable Wishart statistical modeling, orientation-prior feature aggregation, and geometry-adaptive convolution. Evaluations across four real PolSAR
datasets verify WGDNet surpasses existing state-of-the-art approaches in classification accuracy and boundary fidelity.
Read the full abstract ▾
All authors Junfei Shi, Haojia Zhang, Yu Cheng, Yuke Li
Computer Vision · Cross-listed from eess.IV
Oshadha Samarakoon, Dushan Herath, Ishara Ranmandala et al. (+4 more)
Sky-image irradiance studies often compare
forecasting systems in which the image encoder, temporal model, fusion block, target definition, and training recipe all change together. We use a narrower protocol: the
multimodal forecasting pipeline is fixed, and only the visual backbone is varied. The shared setup keeps preprocessing, clear-sky-index normalization, weather-history encoding, fusion, regression head, loss, optimizer schedule, seed, and chronological split policy unchanged. We compare ConvNeXt, Swin
Transformer, VMamba, Spatial Mamba, and MambaVision backbones for 10min-ahead
forecasting on Folsom and a strict matched NREL split. Forecast skill is measured against clear-sky-index smart persistence, and temporal-only rows are reported as weather-history diagnostics rather than as the main ranking criterion. On the Folsom strict split, all evaluated visual-backbone runs improve over smart persistence. In the evaluated single-seed strict runs, VMamba Small and Swin Base reach matched Folsom RMSE values of 65.39 W/m^2 and 65.50 W/m^2; the temporal-only diagnostic reaches 69.51 W/m^2. On the 313-sample NREL strict split, smart persistence remains strongest at 17.48 W/m^2, while the lowest visual RMSE is obtained by Swin Tiny at 23.76 W/m^2. These results provide a reproducible encoder comparison under one fixed
multimodal operating point rather than establishing architecture-level dominance, statistically resolved ranking, or fully optimized
forecasting performance. Code available here: https://github.com/Oshadha345/irradiance_
benchmark
Read the full abstract ▾
All authors Oshadha Samarakoon, Dushan Herath, Ishara Ranmandala, Dilshara Herath, Roshan Godaliyadda, Parakrama Ekanayake, Vijitha Herath
Computer Vision
PathSelect: Sequential Token Selection for Whole Slide Pathology
Jingzhi Chen, Landi He, Zehong Chen et al. (+2 more)
Gigapixel Whole-Slide Images (WSIs) present a fundamental computational bottleneck for
vision-
language models (VLMs) due to extreme sequence lengths. Existing approaches predominantly rely on spatial sampling or training-free
pruning, which risk diluting weak but informative signals, leading to the loss of critical diagnostic evidence due to the spatially diffuse nature of pathological cues. We reformulate WSI
token pruning as a sequential selection process, enabling the model to autonomously learn an optimal routing strategy rather than relying on static heuristics. We herein propose a decoupled routing framework integrated as an active plugin into the fully pre-trained SlideChat base model, leaving both the slide encoder and large
language model frozen. To provide continuous gradients for the non-differentiable
pruning operation during training, we introduce PathSelect. PathSelect employs a variance-preserving noise gate to modulate each patch's information flow via a differentiable Soft Top-K operator, paired with a diagonal-
attention Denoiser that recovers the perturbed representations without semantic leakage. At inference, the PathSelect module is entirely detached. Relying solely on the trained Scorer, a deterministic Hard Top-K operator executes adaptive, data-dependent trajectory termination, significantly accelerating downstream generative processing with exceptionally low sequential
token selection latency. Driven by an empirical average of only 44.86
tokens under a maximum constraint of K = 128, our framework achieves 74.00% overall accuracy on SlideBench (TCGA), representing an approximate 36.6x spatial
token reduction relative to the uncompressed baseline average while consistently outperforming sampling-based counterparts.
Read the full abstract ▾
All authors Jingzhi Chen, Landi He, Zehong Chen, Peihang Wu, Lijian Xu
Computer Vision · Cross-listed from eess.IV
Segmentation Robustness and Predictive Utility in Glioblastoma Radiomics: Evidence for a Trade-off in Survival Modelling
Mariya Miteva and Maria Nisheva-Pavlova
Radiomic biomarkers derived from
magnetic resonance imaging (MRI) have been widely investigated as non-invasive tools for tumor characterization and prognostic modeling in glioblastoma (GBM). However, their clinical translation remains limited, in part due to sensitivity to tumor
segmentation variability. In this study, we systematically investigate the relationship between feature robustness and predictive utility in GBM survival modeling using the University of Pennsylvania Glioblastoma Imaging, Genomics, and Radiomics (UPENN-GBM) cohort. A total of 4,752 radiomic features were obtained from multiparametric MRI across three tumor subregions: enhancing tumor (ET), peritumoral edema (ED), and necrotic core (NC). Feature robustness was quantified using the intraclass correlation coefficient (ICC) based on the automatic and expert-refined
segmentation versions. Among features with valid ICC estimates, 48.1% were classified as robust. Survival prediction was evaluated using cross-validation with Coxnet, Random Survival Forest, and Gradient Boosting Survival Analysis models. In this cohort, radiomic feature inclusion showed no consistent improvement over the clinical baseline, and robustness filtering produced no detectable performance gain. Model-selected features were less robust than the overall feature pool, indicating a lack of enrichment for robustness. These findings suggest that robustness alone is not a reliable criterion for feature selection in radiomics-based survival modelling.
Read the full abstract ▾
Computer Vision
Markerless Motion Capture in Routine Clinical Upper Limb Assessments: Validity and Insights Beyond Ordinal Scoring
Tim Unger, Olivier Lambercy, Roger Gassert et al. (+3 more)
The Action Research Arm Test (ARAT) is a widely-used upper limb outcome measure in neurorehabilitation, but its ordinal scoring is subjective and suffers from limited sensitivity and specificity. We evaluated whether artificial-intelligence (AI)-based markerless motion capture (MMC), embedded into ARAT assessments during clinical routine, accurately reconstructs upper limb movement and yields valid, objective kinematic metrics carrying clinically meaningful information beyond the ordinal score. Across 47 sessions from 20 mixed-neurological patients (1,174 ARAT tasks), biomechanical reconstruction was accurate and robust across impairment levels, and kinematic metrics showed the discrimination pattern expected of a construct-valid measure. In longitudinal case studies, the metrics added the specificity and sensitivity the ordinal score lacks: a domain decomposition exposed patient-specific recovery profiles underlying equal ARAT gains (specificity), and kinematic improvement continued to be detected after the ARAT had saturated (sensitivity). MMC in clinical routine can thus provide valid, objective, sensitive, and specific kinematic measurement complementing ordinal scoring.
Read the full abstract ▾
All authors Tim Unger, Olivier Lambercy, Roger Gassert, Andreas R. Luft, R. James Cotton, Chris Easthope Awai