Computer Vision
Multi-Modal Representation Learning via Semi-Supervised Rate Reduction for Generalized Category Discovery
Wei He, Xianghan Meng, Zhiyuan Huang et al. (+3 more)
Generalized Category Discovery (GCD) aims to identify both known and unknown categories, with only partial labels given for the known categories, posing a challenging open-set recognition problem. State-of-the-art approaches for GCD task are usually built on multi-modality representation learning, which is heavily dependent upon inter-modality
alignment. However, few of them cast a proper intra-modality
alignment to generate a desired underlying structure of representation distributions. In this paper, we propose a novel and effective multi-modal representation learning framework for GCD via Semi-Supervised Rate Reduction, called SSR$^2$-GCD, to learn cross-modality representations with desired structural properties based on emphasizing to properly align intra-modality relationships. Moreover, to boost knowledge transfer, we integrate
prompt candidates by leveraging the inter-modal
alignment offered by
Vision Language Models. We conduct extensive experiments on generic and fine-grained
benchmark datasets demonstrating superior performance of our approach.
Read the full abstract ▾
All authors Wei He, Xianghan Meng, Zhiyuan Huang, Xianbiao Qi, Rong Xiao, Chun-Guang Li
Computer Vision
Gradient based Severity Labeling for Biomarker Classification in OCT
Kiran Kokilepersaud, Mohit Prabhushankar, Ghassan AlRegib et al. (+2 more)
In this paper, we propose a novel selection strategy for contrastive learning for medical images. On natural images, contrastive learning uses augmentations to select positive and negative pairs for the contrastive loss. However, in the medical domain, arbitrary augmentations have the potential to distort small localized regions that contain the biomarkers we are interested in detecting. A more intuitive approach is to select samples with similar disease severity characteristics, since these samples are more likely to have similar structures related to the progression of a disease. To enable this, we introduce a method that generates disease severity labels for unlabeled OCT scans on the basis of gradient responses from an anomaly detection
algorithm. These labels are used to train a supervised contrastive learning setup to improve biomarker classification accuracy by as much as 6% above self-supervised baselines for key indicators of Diabetic Retinopathy.
Read the full abstract ▾
All authors Kiran Kokilepersaud, Mohit Prabhushankar, Ghassan AlRegib, Stephanie Trejo Corona, Charles Wykoff
Computer Vision
ExpPortrait: Expressive Portrait Generation via Personalized Representation
Junyi Wang, Yudong Guo, Boyang Guo et al. (+2 more)
While
diffusion models have shown great potential in portrait generation, generating expressive, coherent, and controllable cinematic portrait
videos remains a significant challenge. Existing intermediate signals for portrait generation, such as 2D landmarks and parametric models, have limited disentanglement capabilities and cannot express personalized details due to their sparse or low-rank representation. Therefore, existing methods based on these models struggle to accurately preserve subject identity and expressions, hindering the generation of highly expressive portrait
videos. To overcome these limitations, we propose a high-fidelity personalized head representation that more effectively disentangles expression and identity. This representation captures both static, subject-specific global geometry and dynamic, expression-related details. Furthermore, we introduce an expression transfer module to achieve personalized transfer of head pose and expression details between different identities. We use this sophisticated and highly expressive head model as a conditional signal to train a
diffusion transformer (DiT)-based generator to synthesize richly detailed portrait
videos. Extensive experiments on self- and cross-reenactment tasks demonstrate that our method outperforms previous models in terms of identity preservation, expression accuracy, and temporal stability, particularly in capturing fine-grained details of complex motion.
Read the full abstract ▾
All authors Junyi Wang, Yudong Guo, Boyang Guo, Shengming Yang, Juyong Zhang
Computer Vision
Monocular Mesh Recovery and Body Measurement of Female Saanen Goats
Bo Jin, Shichao Zhao, Jin Lyu et al. (+5 more)
The lactation performance of Saanen dairy goats, renowned for their high milk yield, is intrinsically linked to their body size, making accurate 3D body measurement essential for assessing milk production potential, yet existing reconstruction methods lack goat-specific authentic 3D data. To address this limitation, we establish the FemaleSaanenGoat
dataset containing synchronized eight-view RGBD
videos of 55 female Saanen goats (6-18 months). Using multi-view DynamicFusion, we fuse noisy, non-rigid point cloud sequences into high-fidelity 3D scans, overcoming challenges from irregular surfaces and rapid movement. Based on these scans, we develop SaanenGoat, a parametric 3D shape model specifically designed for female Saanen goats. This model features a refined template with 41 skeletal joints and enhanced udder representation, registered with our scan data. A comprehensive shape space constructed from 48 goats enables precise representation of diverse individual variations. With the help of SaanenGoat model, we get high-precision 3D reconstruction from single-view RGBD input, and achieve automated measurement of six critical body dimensions: body length, height, chest width, chest girth, hip width, and hip height. Experimental results demonstrate the superior accuracy of our method in both 3D reconstruction and body measurement, presenting a novel paradigm for large-scale 3D
vision applications in precision livestock farming.
Read the full abstract ▾
All authors Bo Jin, Shichao Zhao, Jin Lyu, Bin Zhang, Tao Yu, Liang An, Yebin Liu, Meili Wang
Computer Vision · Cross-listed from eess.IV
Using Unsupervised Domain Adaptation Semantic Segmentation for Pulmonary Embolism Detection in Computed Tomography Pulmonary Angiogram (CTPA) Images
Wen-Liang Lin and Yun-Chien Cheng
While deep learning has demonstrated considerable promise in computer-aided diagnosis for pulmonary embolism (PE), practical deployment in Computed
Tomography Pulmonary Angiography (CTPA) is often hindered by "domain shift" and the prohibitive cost of expert annotations. To address these challenges, an unsupervised domain adaptation (UDA) framework is proposed, utilizing a
Transformer backbone and a Mean-Teacher architecture for cross-center semantic
segmentation. The primary focus is placed on enhancing pseudo-label reliability by learning deep structural information within the feature space. Specifically, three modules are integrated and designed for this task: (1) a Prototype
Alignment (PA) mechanism to reduce category-level distribution discrepancies; (2) Global and Local Contrastive Learning (GLCL) to capture both pixel-level topological relationships and global semantic representations; and (3) an
Attention-based Auxiliary Local Prediction (AALP) module designed to reinforce sensitivity to small PE lesions by automatically extracting high-information slices from
Transformer attention maps. Experimental validation conducted on cross-center
datasets (FUMPE and CAD-PE) demonstrates significant performance gains. In the FUMPE -> CAD-PE task, the IoU increased from 0.1152 to 0.4153, while the CAD-PE -> FUMPE task saw an improvement from 0.1705 to 0.4302. Furthermore, the proposed method achieved a 69.9% Dice score in the CT -> MRI cross-modality task on the MMWHS
dataset without utilizing any target-domain labels for model selection, confirming its robustness and generalizability for diverse clinical environments.
Read the full abstract ▾
Computer Vision
Make Some Noise: Unsupervised Remote Sensing Change Detection Using Latent Space Perturbations
Blaž Rolih, Matic Fučka, Filip Wolf et al. (+1 more)
Unsupervised remote sensing change detection (UCD) aims to localise changes between two images of the same region without relying on labelled training data. Most recent approaches either use a frozen foundation model in a training-free manner or train with synthetic changes generated in pixel space. Both strategies inherently rely on predefined assumptions about change types, typically introduced through handcrafted rules, external
datasets, or auxiliary generative models. Due to these assumptions, such methods fail to generalise beyond a few change types, limiting their real-world usage, especially in rare or complex scenarios. To address this, we propose MaSoN (Make Some Noise), an end-to-end UCD framework that synthesises diverse changes directly in the latent feature space during training. It generates changes dynamically estimated from feature statistics of the target data, enabling diverse yet data-driven variation aligned with the target domain. Since synthesis happens in latent space, it also easily extends to new modalities, such as SAR and multispectral data. MaSoN generalises strongly across diverse change types and improves the average F1 score across five
benchmarks by 14.1 percentage points. Project page: https://blaz-r.github.io/mason_ucd/
Read the full abstract ▾
All authors Blaž Rolih, Matic Fučka, Filip Wolf, Luka Čehovin Zajc
Computer Vision
BigMaQ: A Big Macaque Motion and Animation Dataset Bridging Image and 3D Pose Representations
Lucas Martini, Alexander Lappe, Anna Bognár et al. (+2 more)
The recognition of dynamic and social behavior in animals is fundamental for advancing ethology, ecology, medicine and neuroscience. Recent progress in deep learning has enabled automated behavior recognition from
video, yet an accurate reconstruction of the three-dimensional (3D) pose and shape has not been integrated into this process. Especially for non-human primates, mesh-based tracking efforts lag behind those for other species, leaving pose descriptions restricted to sparse keypoints that are unable to fully capture the richness of action dynamics. To address this gap, we introduce the $\textbf{Big Ma}$ca$\textbf{Q}$ue 3D Motion and Animation
Dataset ($\texttt{BigMaQ}$), a large-scale
dataset comprising more than 750 scenes of interacting rhesus macaques with detailed 3D pose descriptions. Extending previous surface-based animal tracking methods, we construct subject-specific textured avatars by adapting a high-quality macaque template mesh to individual monkeys. This allows us to provide pose descriptions that are more accurate than previous state-of-the-art surface-based animal tracking methods. From the original
dataset, we derive BigMaQ500, an action recognition
benchmark that links surface-based pose vectors to single frames across multiple individual monkeys. By pairing features extracted from established image and
video encoders with and without our pose descriptors, we demonstrate substantial improvements in mean average precision (mAP) when pose information is included. With these contributions, $\texttt{BigMaQ}$ establishes the first
dataset that both integrates dynamic 3D pose-shape representations into the learning task of animal action recognition and provides a rich resource to advance the study of visual appearance, posture, and social interaction in non-human primates. The code and data are publicly available at https://martinivis.github.io/BigMaQ/ .
Read the full abstract ▾
All authors Lucas Martini, Alexander Lappe, Anna Bognár, Rufin Vogels, Martin A. Giese
Computer Vision
GOAL: Geometrically Optimal Alignment for Continual Generalized Category Discovery
Jizhou Han, Chenhao Ding, SongLin Dong et al. (+4 more)
Continual Generalized Category Discovery (C-GCD) requires identifying novel classes from unlabeled data while retaining knowledge of known classes over time. Existing methods typically update classifier weights dynamically, resulting in forgetting and inconsistent feature
alignment. We propose GOAL, a unified framework that introduces a fixed Equiangular Tight Frame (ETF) classifier to impose a consistent
geometric structure throughout learning. GOAL conducts supervised
alignment for labeled samples and confidence-guided
alignment for novel samples, enabling stable integration of new classes without disrupting old ones. Experiments on four
benchmarks show that GOAL outperforms the prior method Happy, reducing forgetting by 16.1% and boosting novel class discovery by 3.2%, establishing a strong solution for long-horizon continual discovery.
Read the full abstract ▾
All authors Jizhou Han, Chenhao Ding, SongLin Dong, Yuhang He, Shaokun Wang, Qiang Wang, Yihong Gong
Computer Vision
ApET: Approximation-Error Guided Token Compression for Efficient VLMs
Qiankun Ma, Ziyao Zhang, Haofei Wang et al. (+3 more)
Recent
Vision-
Language Models (VLMs) have demonstrated remarkable
multimodal understanding capabilities, yet the redundant visual
tokens incur prohibitive computational overhead and degrade inference efficiency. Prior studies typically relies on [CLS]
attention or text-
vision cross-
attention to identify and discard redundant visual
tokens. Despite promising results, such solutions are prone to introduce positional bias and, more critically, are incompatible with efficient
attention kernels such as FlashAttention, limiting their practical deployment for VLM acceleration. In this paper, we step away from
attention dependencies and revisit visual
token compression from an information-theoretic perspective, aiming to maximally preserve visual information without any
attention involvement. We present ApET, an Approximation-Error guided
Token compression framework. ApET first reconstructs the original visual
tokens with a small set of basis
tokens via linear approximation, then leverages the approximation error to identify and drop the least informative
tokens. Extensive experiments across multiple VLMs and
benchmarks demonstrate that ApET retains 95.2% of the original performance on image-understanding tasks and even attains 100.4% on
video-understanding tasks, while compressing the
token budgets by 88.9% and 87.5%, respectively. Thanks to its
attention-free design, ApET seamlessly integrates with FlashAttention, enabling further inference acceleration and making VLM deployment more practical. Code is available at https://github.com/MaQianKun0/ApET.
Read the full abstract ▾
All authors Qiankun Ma, Ziyao Zhang, Haofei Wang, Jie Chen, Zhen Song, Hairong Zheng
Computer Vision
Brewing Stronger Features: Dual-Teacher Distillation for Multispectral Earth Observation
Filip Wolf, Blaž Rolih and Luka Čehovin Zajc
Foundation models are transforming Earth Observation (EO), yet the diversity of EO sensors and modalities makes a single universal model unrealistic. Multiple specialized EO foundation models (EOFMs) will likely coexist, making efficient knowledge transfer across modalities essential. Most existing EO pretraining relies on masked image modeling, which emphasizes local reconstruction but provides limited control over global semantic structure. To address this, we propose a dual-teacher contrastive
distillation framework for multispectral imagery that aligns the student's pretraining objective with the contrastive self-
distillation paradigm of modern optical
vision foundation models (VFMs). Our approach combines a multispectral teacher with an optical VFM teacher, enabling coherent cross-modal representation learning. Experiments across diverse optical and multispectral
benchmarks show that our model adapts to multispectral data without compromising performance on optical-only inputs, achieving state-of-the-art results in both settings, with an average improvement of 3.64 percentage points in semantic
segmentation, 1.2 in change detection, and 1.31 in classification tasks. This demonstrates that contrastive
distillation provides a principled and efficient approach to scalable representation learning across heterogeneous EO data sources. Project page: \textcolor{magenta}{https://wolfilip.github.io/DEO/}.
Read the full abstract ▾
Computer Vision
Contrastive meta-domain adaptation for robust skin lesion classification across clinical and acquisition conditions
Rodrigo Mota, Kelvin Cunha, Emanoel dos Santos et al. (+6 more)
Deep learning models for dermatological image analysis remain sensitive to acquisition variability and domain-specific visual characteristics, leading to performance degradation when deployed in clinical settings. We investigate how visual artifacts and domain shifts affect deep learning-based skin lesion classification. We propose an adaptation strategy, grounded in the idea of visual meta-domains, that transfers visual representations from larger dermoscopic
datasets into clinical image domains, thereby improving generalization robustness. Experiments across multiple dermatology
datasets show consistent gains in classification performance and reduced gaps between dermoscopic and clinical images. These results emphasize the importance of domain-aware training for deployable systems.
Read the full abstract ▾
All authors Rodrigo Mota, Kelvin Cunha, Emanoel dos Santos, Fábio Papais, Francisco Filho, Thales Bezerra, Erico Medeiros, Paulo Borba, Tsang Ing Ren
Computer Vision
DerMAE: Improving skin lesion classification through conditioned latent diffusion and MAE distillation
Francisco Filho, Kelvin Cunha, Fábio Papais et al. (+6 more)
Skin lesion classification
datasets often suffer from severe class imbalance, with malignant cases significantly underrepresented, leading to biased decision boundaries during deep learning training. We address this challenge using class-conditioned
diffusion models to generate synthetic dermatological images, followed by self-supervised MAE pretraining to enable huge ViT models to learn robust, domain-relevant features. To support deployment in practical clinical settings, where lightweight models are required, we apply knowledge
distillation to transfer these representations to a smaller ViT student suitable for mobile devices. Our results show that MAE pretraining on synthetic data, combined with
distillation, improves classification performance while enabling efficient on-device inference for practical clinical use.
Read the full abstract ▾
All authors Francisco Filho, Kelvin Cunha, Fábio Papais, Emanoel dos Santos, Rodrigo Mota, Thales Bezerra, Erico Medeiros, Paulo Borba, Tsang Ing Ren
Computer Vision
M3S-Net: Multimodal Feature Fusion Network Based on Multi-scale Data for Ultra-short-term PV Power Forecasting
Penghui Niu, Taotao Cai, Suqi Zhang et al. (+4 more)
The inherent intermittency and high-frequency variability of
solar irradiance, particularly during rapid cloud advection, present significant stability challenges to high-penetration photovoltaic grids. Although
multimodal forecasting has emerged as a viable mitigation strategy, existing architectures predominantly rely on shallow feature concatenation and binary cloud
segmentation, thereby failing to capture the fine-grained optical features of clouds and the complex spatiotemporal coupling between visual and meteorological modalities. To bridge this gap, this paper proposes M3S-Net, a novel
multimodal feature fusion network based on multi-scale data for ultra-short-term PV power
forecasting. First, a multi-scale partial channel selection network leverages partial convolutions to explicitly isolate the boundary features of optically thin clouds, effectively transcending the precision limitations of coarse-grained binary masking. Second, a multi-scale sequence to image analysis network employs Fast
Fourier Transform (FFT)-based time-frequency representation to disentangle the complex periodicity of meteorological data across varying time horizons. Crucially, the model incorporates a cross-modal Mamba interaction module featuring a novel dynamic C-
matrix swapping mechanism. By exchanging state-space parameters between visual and temporal streams, this design conditions the state
evolution of one modality on the context of the other, enabling deep structural coupling with linear computational complexity, thus overcoming the limitations of shallow concatenation. Experimental validation on the newly constructed fine-grained PV power
dataset demonstrates that M3S-Net achieves a mean absolute error reduction of 6.2% in 10-minute forecasts compared to state-of-the-art baselines. The
dataset and source code will be available at https://github.com/she1110/FGPD.
Read the full abstract ▾
All authors Penghui Niu, Taotao Cai, Suqi Zhang, Junhua Gu, Ping Zhang, Qiqi Liu, Jianxin Li
Computer Vision
TextShield-R1: Reinforced Reasoning for Tampered Text Detection
Chenfan Qu, Yiwu Zhong, Jian Liu et al. (+3 more)
The growing prevalence of tampered images poses serious security threats, highlighting the urgent need for reliable detection methods.
Multimodal large
language models (MLLMs) demonstrate strong potential in analyzing tampered images and generating interpretations. However, they still struggle with identifying micro-level artifacts, exhibit low accuracy in localizing tampered text regions, and heavily rely on expensive annotations for forgery interpretation. To this end, we introduce TextShield-R1, the first reinforcement learning based MLLM solution for tampered text detection and
reasoning. Specifically, our approach introduces Forensic Continual Pre-training, an easy-to-hard curriculum that well prepares the MLLM for tampered text detection by harnessing the large-scale cheap data from natural image forensic and OCR tasks. During fine-tuning, we perform Group Relative Policy Optimization with novel reward functions to reduce annotation dependency and improve
reasoning capabilities. At inference time, we enhance localization accuracy via OCR Rectification, a method that leverages the MLLM's strong text recognition abilities to refine its predictions. Furthermore, to support rigorous evaluation, we introduce the Text Forensics
Reasoning (TFR)
benchmark, comprising over 45k real and tampered images across 16
languages, 10 tampering techniques, and diverse domains. Rich
reasoning-style annotations are included, allowing for comprehensive assessment. Our TFR
benchmark simultaneously addresses seven major limitations of existing
benchmarks and enables robust evaluation under cross-style, cross-method, and cross-
language conditions. Extensive experiments demonstrate that TextShield-R1 significantly advances the state of the art in interpretable tampered text detection.
Read the full abstract ▾
All authors Chenfan Qu, Yiwu Zhong, Jian Liu, Xuekang Zhu, Bohan Yu, Lianwen Jin
Computer Vision
Open-vocabulary 3D scene perception in industrial environments
Keno Moenck, Adrian Philip Florea, Julian Koch et al. (+1 more)
Autonomous
vision applications in production, intralogistics, or manufacturing environments require perception capabilities beyond a small, fixed set of classes. Recent open-vocabulary methods, leveraging 2D
Vision-
Language Foundation Models (VLFMs), target this task but often rely on class-agnostic
segmentation models pre-trained on non-industrial
datasets (e.g., household scenes). In this work, we first demonstrate that such models fail to generalize, performing poorly on common industrial objects. Therefore, we propose a training-free, open-vocabulary 3D perception pipeline that overcomes this limitation. Instead of using a pre-trained model to generate instance proposals, our method simply generates masks by merging pre-computed superpoints based on their semantic features. Following, we evaluate the domain-adapted VLFM "IndustrialCLIP" on a representative 3D industrial workshop scene for open-vocabulary querying. Our qualitative results demonstrate successful
segmentation of industrial objects.
Read the full abstract ▾
All authors Keno Moenck, Adrian Philip Florea, Julian Koch, Thorsten Schüppstuhl
Computer Vision
Efficient endometrial carcinoma screening via cross-modal synthesis and gradient distillation
Dongjing Shan, Yamei Luo, Jiqing Xuan et al. (+7 more)
Early detection of myometrial invasion is critical for the staging and life-saving management of endometrial carcinoma (EC), a prevalent global malignancy. Transvaginal ultrasound serves as the primary, accessible screening modality in resource-constrained primary care settings; however, its diagnostic reliability is severely hindered by low tissue contrast, high operator dependence, and a pronounced scarcity of positive pathological samples. Existing artificial intelligence solutions struggle to overcome this severe class imbalance and the subtle imaging features of invasion, particularly under the strict computational limits of primary care clinics. Here we present an automated, highly efficient two-stage deep learning framework that resolves both data and computational bottlenecks in EC screening. To mitigate pathological data scarcity, we develop a structure-guided cross-modal generation network that synthesizes diverse, high-fidelity ultrasound images from unpaired
magnetic resonance imaging (MRI) data, strictly preserving clinically essential anatomical junctions. Furthermore, we introduce a lightweight screening network utilizing gradient
distillation, which transfers discriminative knowledge from a high-capacity teacher model to dynamically guide sparse
attention towards task-critical regions. Evaluated on a large, multicenter cohort of 7,951 participants, our model achieves a sensitivity of 99.5%, a specificity of 97.2%, and an area under the curve of 0.987 at a minimal computational cost (0.289 GFLOPs), substantially outperforming the average diagnostic accuracy of expert sonographers. Our approach demonstrates that combining cross-modal synthetic augmentation with knowledge-driven efficient modeling can democratize expert-level, real-time cancer screening for resource-constrained primary care settings.
Read the full abstract ▾
All authors Dongjing Shan, Yamei Luo, Jiqing Xuan, Lu Huang, Jin Li, Mengchu Yang, Zeyu Chen, Fajin Lv, Yong Tang, Chunxiang Zhang
Computer Vision
TraceVision: Trajectory-Aware Vision-Language Model for Human-Like Spatial Understanding
Fan Yang, Shurong Zheng, Hongyin Zhao et al. (+5 more)
Recent Large
Vision-
Language Models (LVLMs) demonstrate remarkable capabilities in image understanding and natural
language generation. However, current approaches focus predominantly on global image understanding, struggling to simulate human visual
attention trajectories and explain associations between descriptions and specific regions. We propose TraceVision, a unified
vision-
language model integrating trajectory-aware spatial understanding in an end-to-end framework. TraceVision employs a Trajectory-aware Visual Perception (TVP) module for bidirectional fusion of visual features and trajectory information. We design
geometric simplification to extract semantic keypoints from raw trajectories and propose a three-stage training pipeline where trajectories guide description generation and region localization. We extend TraceVision to trajectory-guided
segmentation and
video scene understanding, enabling cross-frame tracking and temporal
attention analysis. We construct the
Reasoning-based Interactive Localized Narratives (RILN)
dataset to enhance logical
reasoning and interpretability. Extensive experiments on trajectory-guided captioning, text-guided trajectory prediction, understanding, and
segmentation demonstrate that TraceVision achieves state-of-the-art performance, establishing a foundation for intuitive spatial interaction and interpretable visual understanding.
Read the full abstract ▾
All authors Fan Yang, Shurong Zheng, Hongyin Zhao, Yufei Zhan, Xin Li, Yousong Zhu, Chaoyang Zhao Ming Tang, Jinqiao Wang
Computer Vision
One2Scene: Geometric Consistent Explorable 3D Scene Generation from a Single Image
Pengfei Wang, Liyi Chen, Zhiyuan Ma et al. (+3 more)
Generating explorable 3D scenes from a single image is a highly challenging problem in 3D
vision. Existing methods struggle to support free exploration, often producing severe
geometric distortions and noisy artifacts when the viewpoint moves far from the original perspective. We introduce One2Scene, an effective framework that decomposes this ill-posed problem into three tractable sub-tasks to enable immersive explorable scene generation. We first use a panorama generator to produce anchor views from a single input image as initialization. Then, we lift these 2D anchors into an explicit 3D
geometric scaffold via a generalizable, feed-forward Gaussian Splatting network. Instead of treating the panorama as a single image for reconstruction, we project it into multiple sparse anchor views and reformulate the reconstruction task as multi-view stereo matching, which allows us to leverage robust
geometric priors learned from large-scale multi-view
datasets. A bidirectional feature fusion module is used to enforce cross-view consistency, yielding an efficient and geometrically reliable scaffold. Finally, the scaffold serves as a strong prior for a novel view generator to produce photorealistic and geometrically accurate views at arbitrary cameras. By explicitly conditioning on a 3D-consistent scaffold to perform reconstruction, One2Scene works stably under large camera motions, supporting immersive scene exploration. Extensive experiments show that One2Scene substantially outperforms state-of-the-art methods in panorama depth estimation, feed-forward 360{\deg} reconstruction, and explorable 3D scene generation. Project page: https://one2scene5406.github.io/
Read the full abstract ▾
All authors Pengfei Wang, Liyi Chen, Zhiyuan Ma, Yanjun Guo, Guowen Zhang, Lei Zhang
Computer Vision
Training Deep Stereo Matching Networks on Tree Branch Imagery: A Benchmark Study for Real-Time UAV Forestry Applications
Yida Lin, Bing Xue, Mengjie Zhang et al. (+2 more)
Autonomous drone-based tree
pruning needs accurate, real-time depth estimation from stereo cameras. Depth is computed from disparity maps using $Z = f B/d$, so even small disparity errors cause noticeable depth mistakes at working distances. Building on our earlier work that identified DEFOM-Stereo as the best reference disparity generator for vegetation scenes, we present the first study to train and test ten deep stereo matching networks on real tree branch images. We use the Canterbury Tree Branches
dataset – 5,313 stereo pairs from a ZED Mini camera at 1080P and 720P – with DEFOM-generated disparity maps as training targets. The ten methods cover step-by-step refinement, 3D convolution, edge-aware
attention, and lightweight designs. Using perceptual metrics (SSIM, LPIPS, ViTScore) and structural metrics (SIFT/ORB feature matching), we find that BANet-3D produces the best overall quality (SSIM = 0.883, LPIPS = 0.157), while RAFT-Stereo scores highest on scene-level understanding (ViTScore = 0.799). Testing on an NVIDIA Jetson Orin Super (16 GB, independently powered) mounted on our drone shows that AnyNet reaches 6.99 FPS at 1080P – the only near-real-time option – while BANet-2D gives the best quality-speed balance at 1.21 FPS. We also compare 720P and 1080P processing times to guide resolution choices for forestry drone systems.
Read the full abstract ▾
All authors Yida Lin, Bing Xue, Mengjie Zhang, Sam Schofield, Richard Green
Computer Vision
Junhyeok Choi, Sangwoo Mo and Minwoo Chae
Recent advances in
multimodal learning have achieved remarkable success across diverse
vision-
language tasks. However, such progress heavily relies on large-scale image-text
datasets, making training costly and inefficient. Prior efforts in
dataset filtering and
pruning attempt to mitigate this issue, but still require relatively large subsets to maintain performance and fail under very small subsets.
Dataset distillation offers a promising alternative, yet existing
multimodal dataset distillation methods require full-
dataset training and joint optimization of image pixels and text features, making them architecture-dependent and limiting cross-architecture generalization. To overcome this, we propose a learning-free
dataset distillation framework that eliminates the need for large-scale training and optimization while enhancing generalization across architectures. Our method uses CLIP to extract aligned image-text
embeddings, obtains prototypes, and employs an unCLIP decoder to synthesize images, enabling efficient and scalable
multimodal dataset distillation. Extensive experiments demonstrate that our approach consistently outperforms optimization-based
dataset distillation and subset selection methods, achieving state-of-the-art cross-architecture generalization.
Read the full abstract ▾
Computer Vision
RAP: Fast Feedforward Rendering-Free Attribute-Guided Primitive Importance Score Prediction for Efficient 3D Gaussian Splatting Processing
Kaifa Yang, Qi Yang, Yiling Xu et al. (+1 more)
3D Gaussian Splatting (3DGS) has emerged as a leading technology for high-quality 3D scene reconstruction. However, the iterative refinement and densification process leads to the generation of a large number of primitives, each contributing to the reconstruction to a substantially different extent. Estimating primitive importance is thus crucial, both for removing redundancy during reconstruction and for enabling efficient compression and transmission. Existing methods typically rely on rendering-based analyses, where each primitive is evaluated through its contribution across multiple camera viewpoints. However, such methods are sensitive to the number and selection of views, rely on specialized differentiable rasterizers, and have long calculation times that grow linearly with view count, making them difficult to integrate as plug-and-play modules and limiting scalability and generalization. To address these issues, we propose RAP, a fast feedforward rendering-free attribute-guided method for efficient importance score prediction in 3DGS. RAP infers primitive significance directly from intrinsic Gaussian attributes and local neighborhood statistics, avoiding rendering-based or visibility-dependent computations. A compact MLP predicts per-primitive importance scores using rendering loss,
pruning-aware loss, and significance distribution regularization. After training on a small set of scenes, RAP generalizes effectively to unseen data and can be seamlessly integrated into reconstruction, compression, and transmission pipelines. Our code is publicly available at https://github.com/yyyykf/RAP.
Read the full abstract ▾
All authors Kaifa Yang, Qi Yang, Yiling Xu, Zhu Li
Computer Vision
InfScene-SR: Seamless Super-Resolution of Arbitrarily Large Remote-Sensing Scenes via Variance-Preserving Joint Denoising
Shoukun Sun, Zhe Wang, Xiang Que et al. (+2 more)
Diffusion models now give the best perceptual quality in super-resolution (SR), but their architecture and training confine them to small fixed crops. Operational remote sensing needs seamless scenes orders of magnitude larger. Joint denoising fuses overlapping tiles at every reverse step and lets text-to-image
diffusion generate beyond its training crop, but it assumes deterministic ODE samplers. With the
stochastic sampler of SR models such as SR3, the averaging also partly cancels independent per-tile noise. This known variance erosion blurs the detail SR should recover and must be corrected. We carry variance-corrected fusion to conditional SR and derive Spatially-Decoupled Variance Correction (SDVC), an exact reformulation that replaces per-step global normalization with independent per-tile contributions accumulated in one additive pass. SDVC turns the coupled per-step computation into independent tile-local work, so the resulting pipeline, InfScene-SR, runs in parallel across GPUs and makes SR of arbitrarily large scenes feasible. On a 5$\times$ SR task built from NAIP aerial imagery, we evaluate whole scenes with fidelity, perceptual, seam-continuity, and faithfulness metrics. Under one backbone, InfScene-SR is the only fusion strategy that is seamless and sharp at once, the closest to the low-resolution observation among those that synthesize detail, and within 0.003 IoU of native high-resolution imagery on downstream invasive-plant
segmentation. Code is available at https://github.com/TitorX/infscene-sr.
Read the full abstract ▾
All authors Shoukun Sun, Zhe Wang, Xiang Que, Jiyin Zhang, Xiaogang Ma
Computer Vision
VGGT-MPR: VGGT-Enhanced Multimodal Place Recognition in Autonomous Driving Environments
Jingyi Xu, Zhangshuo Qi, Zhongmiao Yan et al. (+5 more)
In autonomous driving, robust place recognition is critical for global localization and loop closure detection. While inter-modality fusion of camera and
LiDAR data in
multimodal place recognition (MPR) has shown promise in overcoming the limitations of unimodal counterparts, existing MPR methods basically attend to hand-crafted fusion strategies and heavily parameterized backbones that require costly retraining. To address this, we propose VGGT-MPR, a
multimodal place recognition framework that adopts the Visual Geometry Grounded
Transformer (VGGT) as a unified
geometric engine for both global
retrieval and re-ranking. In the global
retrieval stage, VGGT extracts geometrically-rich visual
embeddings through prior depth-aware and point map supervision, and densifies sparse
LiDAR point clouds with predicted depth maps to improve structural representation. This enhances the discriminative ability of fused
multimodal features and produces global descriptors for fast
retrieval. Beyond global
retrieval, we design a training-free re-ranking mechanism that exploits VGGT's cross-view keypoint-tracking capability. By combining mask-guided keypoint extraction with confidence-aware correspondence scoring, our proposed re-ranking mechanism effectively refines
retrieval results without additional parameter optimization. Extensive experiments on large-scale autonomous driving
benchmarks and our self-collected data demonstrate that VGGT-MPR achieves state-of-the-art performance, exhibiting strong robustness to severe environmental changes, viewpoint shifts, and occlusions. Our code and data will be made publicly available.
Read the full abstract ▾
All authors Jingyi Xu, Zhangshuo Qi, Zhongmiao Yan, Xuyu Gao, Qianyun Jiao, Songpengcheng Xia, Xieyuanli Chen, Ling Pei
Computer Vision
Towards Personalized Multi-Modal MRI Synthesis across Heterogeneous Datasets
Yue Zhang, Zhizheng Zhuo, Siyao Xu et al. (+6 more)
Synthesizing missing modalities in multi-modal
magnetic resonance imaging (MRI) is vital for ensuring diagnostic completeness, particularly when full acquisitions are infeasible due to time constraints, motion artifacts, and patient tolerance. Recent unified synthesis models have enabled flexible synthesis tasks by accommodating various input-output configurations. However, their training and evaluation are typically restricted to a single
dataset, limiting their generalizability across diverse clinical
datasets and impeding practical deployment. To address this limitation, we propose PMM-Synth, a personalized MRI synthesis framework that not only supports various synthesis tasks but also generalizes effectively across heterogeneous
datasets. PMM-Synth is jointly trained on multiple multi-modal MRI
datasets that differ in modality coverage, disease types, and intensity distributions. It achieves cross-
dataset generalization through three core innovations: a Personalized Feature Modulation module that dynamically adapts feature representations based on
dataset identifier to mitigate the impact of distributional shifts; a Modality-Consistent Batch Scheduler that facilitates stable and efficient batch training under inconsistent modality conditions; and a selective supervision loss to ensure effective learning when ground truth modalities are partially missing. Evaluated on four clinical multi-modal MRI
datasets, PMM-Synth consistently outperforms state-of-the-art methods in both one-to-one and many-to-one synthesis tasks, achieving superior PSNR and SSIM scores. Qualitative results further demonstrate improved preservation of anatomical structures and pathological details. Additionally, downstream tumor
segmentation and radiological reporting studies suggest that PMM-Synth holds potential for supporting reliable diagnosis under real-world modality-missing scenarios.
Read the full abstract ▾
All authors Yue Zhang, Zhizheng Zhuo, Siyao Xu, Shan Lv, Zhaoxi Liu, Jun Qiu, Qiuli Wang, Yaou Liu, S. Kevin Zhou
Computer Vision
Generative 6D Pose Estimation via Conditional Flow Matching
Amir Hamza, Davide Boscaini, Weihang Li et al. (+2 more)
Existing methods for instance-level 6D pose estimation typically rely on neural networks that either directly regress the pose in $\mathrm{SE}(3)$ or estimate it indirectly via local feature matching. The former struggle with object
symmetries, while the latter fail in the absence of distinctive local features. To overcome these limitations, we propose a novel formulation of 6D pose estimation as a conditional flow matching problem in $\mathbb{R}^3$. We introduce Flose, a generative method that infers object poses via a denoising process conditioned on local features. While prior approaches based on conditional flow matching perform denoising solely based on
geometric guidance, Flose integrates appearance-based semantic features to mitigate ambiguities caused by object
symmetries. We further incorporate RANSAC-based registration to handle outliers. We validate Flose on five
datasets from the established BOP
benchmark. Flose outperforms prior methods with an average improvement of +4.5 Average Recall. Project Website : https://tev-fbk.github.io/Flose/
Read the full abstract ▾
All authors Amir Hamza, Davide Boscaini, Weihang Li, Benjamin Busam, Fabio Poiesi
Computer Vision
Pixels Don't Lie (But Your Detector Might): Bootstrapping MLLM-as-a-Judge for Trustworthy Deepfake Detection and Reasoning Supervision
Kartik Kuckreja, Parul Gupta, Muhammad Haris Khan et al. (+1 more)
Deepfake detection models often generate natural-
language explanations, yet their
reasoning is frequently ungrounded in visual evidence, limiting reliability. Existing evaluations measure classification accuracy but overlook
reasoning fidelity. We propose DeepfakeJudge, a framework for scalable
reasoning supervision and evaluation, that integrates an out-of-distribution
benchmark containing recent generative and editing forgeries, a human-annotated subset with visual
reasoning labels, and a suite of evaluation models, that specialize in evaluating
reasoning rationales without the need for explicit ground truth
reasoning rationales. The Judge is optimized through a bootstrapped generator-evaluator process that scales human feedback into structured
reasoning supervision and supports both pointwise and pairwise evaluation. On the proposed meta-evaluation
benchmark, our
reasoning-bootstrapped model achieves an accuracy of 96.2%, outperforming 30x larger baselines. The
reasoning judge attains very high correlation with human ratings and 98.9% percent pairwise agreement on the human-annotated meta-evaluation subset. These results establish
reasoning fidelity as a quantifiable dimension of deepfake detection and demonstrate scalable supervision for interpretable deepfake
reasoning. Our user study shows that participants preferred the reasonings generated by our framework 70% of the time, in terms of faithfulness, groundedness, and usefulness, compared to those produced by other models and
datasets. All of our
datasets, models, and codebase are \href{https://github.com/KjAeRsTuIsK/DeepfakeJudge}{open-sourced}.
Read the full abstract ▾
All authors Kartik Kuckreja, Parul Gupta, Muhammad Haris Khan, Abhinav Dhall
Computer Vision
Universal Pose Pretraining for Generalizable Vision-Language-Action Policies
Haitao Lin, Hanyang Yu, Jingshun Huang et al. (+5 more)
Existing
Vision-
Language-Action (VLA) models often suffer from feature collapse and low training efficiency because they entangle high-level perception with sparse, embodiment-specific action supervision. Since these models typically rely on VLM backbones optimized for Visual Question Answering (VQA), they excel at semantic identification but often overlook subtle 3D state variations that dictate distinct action patterns. To resolve these misalignments, we propose Pose-VLA, a decoupled paradigm that separates VLA training into a pre-training phase for extracting universal 3D spatial priors in a unified camera-centric space, and a post-training phase for efficient embodiment
alignment within
robot-specific action space. By introducing discrete pose
tokens as a universal representation, Pose-VLA seamlessly integrates spatial grounding from diverse 3D
datasets with geometry-level trajectories from robotic demonstrations. Our framework follows a two-stage pre-training pipeline, establishing fundamental spatial grounding via poses followed by motion
alignment through trajectory supervision. Extensive evaluations demonstrate that Pose-VLA achieves state-of-the-art results on RoboTwin 2.0 with a 79.5% average success rate and competitive performance on LIBERO at 96.0%. Real-world experiments further showcase robust generalization across diverse objects using only 100 demonstrations per task, validating the efficiency of our pre-training paradigm.
Read the full abstract ▾
All authors Haitao Lin, Hanyang Yu, Jingshun Huang, He Zhang, Yonggen Ling, Ping Tan, Xiangyang Xue, Yanwei Fu
Computer Vision
ChimeraLoRA: Multi-Head LoRA-Guided Synthetic Datasets
Hoyoung Kim, Minwoo Jang, Jabin Koo et al. (+2 more)
Beyond general recognition tasks, specialized domains and fine-grained settings often encounter data scarcity, especially for tail classes. To obtain less biased and more reliable models under such scarcity, practitioners leverage
diffusion models to supplement underrepresented regions of real data. Specifically, recent studies fine-tune pretrained
diffusion models with LoRA on few-shot real sets to synthesize additional images. While an image-wise LoRA trained on a single image captures fine-grained details yet offers limited diversity, a class-wise LoRA trained over all shots produces diverse images as it encodes class priors yet tends to overlook fine details. To combine both benefits, we separate the adapter into a class-shared LoRA~$A$ for class priors and per-image LoRAs~$\mathcal{B}$ for image-specific characteristics. To expose coherent class semantics in the shared LoRA~$A$, we propose a semantic boosting by preserving class bounding boxes during training. For generation, we compose $A$ with a mixture of $\mathcal{B}$ using coefficients drawn from a Dirichlet distribution. Across diverse
datasets, our synthesized images are both diverse and detail-rich while closely aligning with the few-shot real distribution, yielding robust gains in downstream classification accuracy.
Read the full abstract ▾
All authors Hoyoung Kim, Minwoo Jang, Jabin Koo, Sangdoo Yun, Jungseul Ok
Computer Vision
HDR Reconstruction Boosting with Training-Free and Exposure-Consistent Diffusion
Yo-Tin Lin, Su-Kai Chen, Hou-Ning Hu et al. (+2 more)
Single LDR to HDR reconstruction remains challenging for over-exposed regions where traditional methods often fail due to complete information loss. We present a training-free approach that enhances existing indirect and direct HDR reconstruction methods through
diffusion-based inpainting. Our method combines text-guided
diffusion models with SDEdit refinement to generate plausible content in over-exposed areas while maintaining consistency across multi-exposure LDR images. Unlike previous approaches requiring extensive training, our method seamlessly integrates with existing HDR reconstruction techniques through an iterative compensation mechanism that ensures luminance coherence across multiple exposures. We demonstrate significant improvements in both perceptual quality and quantitative metrics on standard HDR
datasets and in-the-wild captures. Results show that our method effectively recovers natural details in challenging scenarios while preserving the advantages of existing HDR reconstruction pipelines. Project page: https://github.com/EusdenLin/HDR-Reconstruction-Boosting
Read the full abstract ▾
All authors Yo-Tin Lin, Su-Kai Chen, Hou-Ning Hu, Yen-Yu Lin, Yu-Lun Liu
Computer Vision · Cross-listed from cs.DL
Iconographic Classification and Content-Based Recommendation for Digitized Artworks
Krzysztof Kutt and Maciej Baczyński
We present a proof-of-concept system that automates iconographic classification and content-based recommendation of digitized artworks using the Iconclass vocabulary and selected artificial intelligence methods. The prototype implements a four-stage workflow for classification and recommendation, which integrates YOLOv8 object detection with algorithmic mappings to Iconclass codes, rule-based inference for abstract meanings, and three complementary recommenders (hierarchical proximity, IDF-weighted overlap, and Jaccard similarity). Although more engineering is still needed, the evaluation demonstrates the potential of this solution: Iconclass-aware computer
vision and recommendation methods can accelerate cataloging and enhance
navigation in large heritage repositories. The key insight is to let computer
vision propose visible elements and to use symbolic structures (Iconclass hierarchy) to reach meaning.
Read the full abstract ▾