Computer Vision
UFO: Unifying Feed-Forward and Optimization-based Methods for Large Driving Scene Modeling
Kaiyuan Tan, Yingying Shen, Mingfei Tu et al. (+5 more)
Dynamic driving scene reconstruction is critical for autonomous driving simulation and closed-loop learning. While recent feed-forward methods have shown promise for 3D reconstruction, they struggle with long-range driving sequences due to quadratic complexity in sequence length and challenges in modeling dynamic objects over extended durations. We propose UFO, a novel recurrent paradigm that combines the benefits of optimization-based and feed-forward methods for efficient long-range 4D reconstruction. Our approach maintains a 4D scene representation that is iteratively refined as new observations arrive, using a visibility-based filtering mechanism to select informative scene
tokens and enable efficient processing of long sequences. For dynamic objects, we introduce an object pose-guided modeling approach that supports accurate long-range motion capture. Experiments on the Waymo Open
Dataset demonstrate that our method significantly outperforms both per-scene optimization and existing feed-forward methods across various sequence lengths. Notably, our approach can reconstruct 16-second driving logs within 0.5 second while maintaining superior visual quality and
geometric accuracy.
Read the full abstract ▾
All authors Kaiyuan Tan, Yingying Shen, Mingfei Tu, Haohui Zhu, Bing Wang, Guang Chen, Hangjun Ye, Haiyang Sun
Computer Vision
Dropping Anchor and Spherical Harmonics for Sparse-view Gaussian Splatting
Shuangkang Fang, I-Chao Shen, Xuanyang Zhang et al. (+5 more)
Recent 3D Gaussian Splatting (3DGS) Dropout methods address overfitting under sparse-view conditions by randomly nullifying Gaussian opacities. However, we identify a neighbor compensation effect in these approaches: dropped Gaussians are often compensated by their neighbors, weakening the intended regularization. Moreover, these methods overlook the contribution of high-degree spherical harmonic coefficients (SH) to overfitting. To address these issues, we propose DropAnSH-GS, a novel anchor-based Dropout strategy. Rather than dropping Gaussians independently, our method randomly selects certain Gaussians as anchors and simultaneously removes their spatial neighbors. This effectively disrupts local redundancies near anchors and encourages the model to learn more robust, globally informed representations. Furthermore, we extend the Dropout to color attributes by randomly dropping higher-degree SH to concentrate appearance information in lower-degree SH. This strategy further mitigates overfitting and enables flexible post-training model compression via SH truncation. Experimental results demonstrate that DropAnSH-GS substantially outperforms existing Dropout methods with negligible computational overhead, and can be readily integrated into various 3DGS variants to enhance their performances. Project Website: https://sk-fun.fun/DropAnSH-GS
Read the full abstract ▾
All authors Shuangkang Fang, I-Chao Shen, Xuanyang Zhang, Zesheng Wang, Yufeng Wang, Wenrui Ding, Gang Yu, Takeo Igarashi
Computer Vision
Computing a Characteristic Orientation for Rotation-Independent Image Analysis
Cristian Valero-Abundio, Emilio Sansano-Sansano, Raúl Montoliu et al. (+1 more)
Handling
geometric transformations, particularly rotations, remains a challenge in deep learning for computer
vision. Standard neural networks lack inherent rotation invariance and typically rely on data augmentation or architectural modifications to improve robustness. Although effective, these approaches increase computational demands, require specialised implementations, or alter network structures, limiting their applicability. This paper introduces General Intensity Direction (GID), a preprocessing method that improves rotation robustness without modifying the network architecture. The method estimates a global orientation for each image and aligns it to a canonical reference frame, allowing standard models to process inputs more consistently across different rotations. Unlike moment-based approaches that extract invariant descriptors, this method directly transforms the image while preserving spatial structure, making it compatible with convolutional networks. Experimental evaluation on the rotated MNIST
dataset shows that the proposed method achieves higher accuracy than state-of-the-art rotation-invariant architectures. Additional experiments on the CIFAR-10
dataset, confirm that the method remains effective under more complex conditions.
Read the full abstract ▾
All authors Cristian Valero-Abundio, Emilio Sansano-Sansano, Raúl Montoliu, Marina Martínez García
Computer Vision · Cross-listed from cs.RO
LST-SLAM: A Stereo Thermal SLAM System for Kilometer-Scale Dynamic Environments
Zeyu Jiang, Kuan Xu and Changhao Chen
Thermal cameras offer strong potential for
robot perception under challenging illumination and weather conditions. However, thermal Simultaneous Localization and Mapping (SLAM) remains difficult due to unreliable feature extraction, unstable motion tracking, and inconsistent global pose and map construction, particularly in dynamic large-scale outdoor environments. To address these challenges, we propose LST-SLAM, a novel large-scale stereo thermal SLAM system that achieves robust performance in complex, dynamic scenes. Our approach combines self-supervised thermal feature learning, stereo dual-level motion tracking, and
geometric pose optimization. We also introduce a semantic-
geometric hybrid constraint that suppresses potentially dynamic features lacking strong inter-frame
geometric consistency. Furthermore, we develop an online incremental bag-of-words model for loop closure detection, coupled with global pose optimization to mitigate accumulated drift. Extensive experiments on kilometer-scale dynamic thermal
datasets show that LST-SLAM significantly outperforms recent representative SLAM systems, including AirSLAM and DROID-SLAM, in both robustness and accuracy.
Read the full abstract ▾
Computer Vision
LongVideo-R1: Smart Navigation for Low-cost Long Video Understanding
Jihao Qiu, Lingxi Xie, Xinyue Huo et al. (+2 more)
This paper addresses the critical and underexplored challenge of long
video understanding with low computational budgets. We propose LongVideo-R1, an active,
reasoning-equipped
multimodal large
language model (MLLM)
agent designed for efficient
video context
navigation, avoiding the redundancy of exhaustive search. At the core of LongVideo-R1 lies a
reasoning module that leverages high-level visual cues to infer the most informative
video clip for subsequent processing. During inference, the
agent initiates traversal from top-level visual summaries and iteratively refines its focus, immediately halting the exploration process upon acquiring sufficient knowledge to answer the query. To facilitate training, we first extract hierarchical
video captions from CGBench, a
video corpus with grounding annotations, and guide GPT-5 to generate 33K high-quality chain-of-thought-with-tool trajectories. The LongVideo-R1
agent is fine-tuned upon the Qwen-3-8B model through a two-stage paradigm: supervised fine-tuning (SFT) followed by reinforcement learning (RL), where RL employs a specifically designed reward function to maximize selective and efficient clip
navigation. Experiments on multiple long
video benchmarks validate the effectiveness of name, which enjoys superior tradeoff between QA accuracy and efficiency. All curated data and source code are provided in the supplementary material and will be made publicly available. Code and data are available at: https://github.com/qiujihao19/LongVideo-R1
Read the full abstract ▾
All authors Jihao Qiu, Lingxi Xie, Xinyue Huo, Qi Tian, Qixiang Ye
Computer Vision · Cross-listed from cs.LG
From Isolation to Integration: Building an Adaptive Expert Forest for Pre-Trained Model-based Class-Incremental Learning
Ruiqi Liu, Boyu Diao, Hangda Liu et al. (+3 more)
Class-Incremental Learning (CIL) requires models to learn new classes without forgetting old ones. A common method is to freeze a pre-trained model and train a new, lightweight adapter for each task. While this prevents forgetting, it treats the learned knowledge as a simple, unstructured collection and fails to use the relationships between tasks. To this end, we propose the Semantic-guided Adaptive Expert Forest (SAEF), a new method that organizes adapters into a structured hierarchy for better knowledge sharing. SAEF first groups tasks into conceptual clusters based on their semantic relationships. Then, within each cluster, it builds a balanced expert tree by creating new adapters from merging the adapters of similar tasks. At inference time, SAEF finds and activates a set of relevant experts from the forest for any given input. The final prediction is made by combining the outputs of these activated experts, weighted by how confident each expert is. Experiments on several
benchmark datasets show that SAEF achieves SOTA performance.
Read the full abstract ▾
All authors Ruiqi Liu, Boyu Diao, Hangda Liu, Zhulin An, Fei Wang, Yongjun Xu
Computer Vision
TextPecker: Rewarding Structural Anomaly Quantification for Enhancing Visual Text Rendering
Hanshen Zhu, Yuliang Liu, Xuecheng Wu et al. (+7 more)
Visual Text Rendering (VTR) remains a critical challenge in text-to-image generation, where even advanced models frequently produce text with structural anomalies such as distortion, blurriness, and misalignment. However, we find that leading MLLMs and specialist OCR models largely fail to perceive these structural anomalies, creating a critical bottleneck for both VTR evaluation and RL-based optimization. As a result, even state-of-the-art generators (e.g., Seedream4.0, Qwen-Image) still struggle to render structurally faithful text. To address this, we propose TextPecker, a plug-and-play structural anomaly perceptive RL strategy that mitigates noisy reward signals and works with any textto-image generator. To enable this capability, we construct a recognition
dataset with character-level structural-anomaly annotations and develop a stroke-editing synthesis engine to expand structural-error coverage. Experiments show that TextPecker consistently improves diverse text-to-image models; even on the well-optimized Qwen-Image, it significantly yields average gains of 4% in structural fidelity and 8.7% in semantic
alignment for Chinese text rendering, establishing a new state-of-the-art in high-fidelity VTR. Our work fills a gap in VTR optimization, providing a foundational step towards reliable and structural faithful visual text generation.
Read the full abstract ▾
All authors Hanshen Zhu, Yuliang Liu, Xuecheng Wu, An-Lan Wang, Hao Feng, Dingkang Yang, Chao Feng, Can Huang, Jingqun Tang, Xiang Bai
Computer Vision
Yuechen Xie, Xiaoyan Zhang, Yicheng Shan et al. (+6 more)
Vision-
Language Models (VLMs) have been increasingly applied in real-world scenarios due to their outstanding understanding and
reasoning capabilities. Although VLMs have already demonstrated impressive capabilities in common visual question answering and logical
reasoning, they still lack the ability to make reasonable decisions in complex real-world environments. We define this ability as spatial logical
reasoning, which not only requires understanding the spatial relationships among objects in complex scenes, but also the logical dependencies between steps in multi-step tasks. To bridge this gap, we introduce Spatial Logical Question Answering (SpatiaLQA), a
benchmark designed to evaluate the spatial logical
reasoning capabilities of VLMs. SpatiaLQA consists of 9,605 question answer pairs derived from 241 real-world indoor scenes. We conduct extensive experiments on 41 mainstream VLMs, and the results show that even the most advanced models still struggle with spatial logical
reasoning. To address this issue, we propose a method called recursive scene graph assisted
reasoning, which leverages visual foundation models to progressively decompose complex scenes into task-relevant scene graphs, thereby enhancing the spatial logical
reasoning ability of VLMs, outperforming all previous methods. Code and
dataset are available at https://github.com/xieyc99/SpatiaLQA.
Read the full abstract ▾
All authors Yuechen Xie, Xiaoyan Zhang, Yicheng Shan, Hao Zhu, Rui Tang, Rong Wei, Mingli Song, Yuanyu Wan, Jie Song
Computer Vision
When Safety Collides: Resolving Multi-Category Harmful Conflicts in Text-to-Image Diffusion via Adaptive Safety Guidance
Yongli Xiang, Ziming Hong, Zhaoqing Wang et al. (+3 more)
Text-to-Image (T2I)
diffusion models have demonstrated significant advancements in generating high-quality images, while raising potential safety concerns regarding harmful content generation. Safety-guidance-based methods have been proposed to mitigate harmful outputs by steering generation away from harmful zones, where the zones are averaged across multiple harmful categories based on predefined keywords. However, these approaches fail to capture the complex interplay among different harm categories, leading to "harmful conflicts" where mitigating one type of harm may inadvertently amplify another, thus increasing overall harmful rate. To address this issue, we propose Conflict-aware Adaptive Safety Guidance (CASG), a training-free framework that dynamically identifies and applies the category-aligned safety direction during generation. CASG is composed of two components: (i) Conflict-aware Category Identification (CaCI), which identifies the harmful category most aligned with the model's evolving generative state, and (ii) Conflict-resolving Guidance Application (CrGA), which applies safety steering solely along the identified category to avoid multi-category interference. CASG can be applied to both latent-space and text-space safeguards. Experiments on T2I safety
benchmarks demonstrate CASG's state-of-the-art performance, reducing the harmful rate by up to 15.4% compared to existing methods.
Read the full abstract ▾
All authors Yongli Xiang, Ziming Hong, Zhaoqing Wang, Xiangyu Zhao, Bo Han, Tongliang Liu
Computer Vision
MUSE: Harnessing Precise and Diverse Semantics for Few-Shot Whole Slide Image Classification
Jiahao Xu, Sheng Huang, Xin Zhang et al. (+3 more)
In computational pathology, few-shot whole slide image classification is primarily driven by the extreme scarcity of expert-labeled slides. Recent
vision-
language methods incorporate textual semantics generated by large
language models, but treat these descriptions as static class-level priors that are shared across all samples and lack sample-wise refinement. This limits both the diversity and precision of visual-semantic
alignment, hindering generalization under limited supervision. To overcome this, we propose the
stochastic MUlti-view Semantic Enhancement (MUSE), a framework that first refines semantic precision via sample-wise adaptation and then enhances semantic richness through
retrieval-augmented multi-view generation. Specifically, MUSE introduces Sample-wise Fine-grained Semantic Enhancement (SFSE), which yields a fine-grained semantic prior for each sample through MoE-based adaptive visual-semantic interaction. Guided by this prior,
Stochastic Multi-view Model Optimization (SMMO) constructs an LLM-generated knowledge base of diverse pathological descriptions per class, then retrieves and stochastically integrates multiple matched textual views during training. These dynamically selected texts serve as enriched semantic supervisions to stochastically optimize the
vision-
language model, promoting robustness and mitigating overfitting. Experiments on three
benchmark WSI
datasets show that MUSE consistently outperforms existing
vision-
language baselines in few-shot settings, demonstrating that effective few-shot pathology learning requires not only richer semantic sources but also their active and sample-aware semantic optimization. Our code is available at: https://github.com/JiahaoXu-god/CVPR2026_MUSE.
Read the full abstract ▾
All authors Jiahao Xu, Sheng Huang, Xin Zhang, Zhixiong Nan, Jiajun Dong, Nankun Mu
Computer Vision
DA-Cal: Towards Cross-Domain Calibration in Semantic Segmentation
Wangkai Li, Rui Sun, Zhaoyang Li et al. (+2 more)
While existing unsupervised domain adaptation (UDA) methods greatly enhance target domain performance in semantic
segmentation, they often neglect network calibration quality, resulting in misalignment between prediction confidence and actual accuracy—a significant risk in safety-critical applications. Our key insight emerges from observing that performance degrades substantially when soft pseudo-labels replace hard pseudo-labels in cross-domain scenarios due to poor calibration, despite the theoretical equivalence of perfectly calibrated soft pseudo-labels to hard pseudo-labels. Based on this finding, we propose DA-Cal, a dedicated cross-domain calibration framework that transforms target domain calibration into soft pseudo-label optimization. DA-Cal introduces a Meta Temperature Network to generate pixel-level calibration parameters and employs bi-level optimization to establish the relationship between soft pseudo-labels and UDA supervision, while utilizing complementary domain-mixing strategies to prevent overfitting and reduce domain discrepancies. Experiments demonstrate that DA-Cal seamlessly integrates with existing self-training frameworks across multiple UDA
segmentation benchmarks, significantly improving target domain calibration while delivering performance gains without inference overhead.
Read the full abstract ▾
All authors Wangkai Li, Rui Sun, Zhaoyang Li, Yujia Chen, Tianzhu Zhang
Computer Vision
On the Explainability of Vision-Language Models in Art History
Stefanie Schneider
Vision-
Language Models (VLMs) transfer visual and textual data into a shared
embedding space. In so doing, they enable a wide range of
multimodal tasks, while also raising critical questions about the nature of machine 'understanding.' In this paper, we examine how Explainable Artificial Intelligence (XAI) methods can render the visual
reasoning of a VLM - namely, CLIP - legible in art-historical contexts. To this end, we evaluate seven methods, combining zero-shot localization experiments with human interpretability studies. Our results indicate that, while these methods capture some aspects of human interpretation, their effectiveness hinges on the conceptual stability and representational availability of the examined categories.
Read the full abstract ▾
Computer Vision
Hybrid Fusion: One-Minute Efficient Training for Zero-Shot Cross-Domain Image Fusion
Ran Zhang, Xuanhua He and Liu Liu
Image fusion seeks to integrate complementary information from multiple sources into a single, superior image. While traditional methods are fast, they lack adaptability and performance. Conversely, deep learning approaches achieve state-of-the-art (SOTA) results but suffer from critical inefficiencies: their reliance on slow, resource-intensive, patch-based training introduces a significant gap with full-resolution inference. We propose a novel hybrid framework that resolves this trade-off. Our method utilizes a learnable U-Net to generate a dynamic guidance map that directs a classic, fixed
Laplacian pyramid fusion kernel. This decoupling of policy learning from pixel synthesis enables remarkably efficient full-resolution training, eliminating the train-inference gap. Consequently, our model achieves SOTA-comparable performance in about one minute on a RTX 4090 or two minutes on a consumer laptop GPU from scratch without any external model and demonstrates powerful zero-shot generalization across diverse tasks, from infrared-visible to medical imaging. By design, the fused output is linearly constructed solely from source information, ensuring high faithfulness for critical applications. The codes are available at https://github.com/Zirconium233/HybridFusion
Read the full abstract ▾
Computer Vision
FLIM Networks with Bag of Feature Points
João Deltregia Martinelli, Marcelo Luis Rodrigues Filho, Felipe Crispim da Rocha Salvagnini et al. (+3 more)
Convolutional networks require extensive image annotation, which can be costly and time-consuming. Feature Learning from Image Markers (FLIM) tackles this challenge by estimating encoder filters (i.e., kernel weights) from user-drawn markers on discriminative regions of a few representative images without traditional optimization. Such an encoder combined with an adaptive decoder comprises a FLIM network fully trained without backpropagation. Prior research has demonstrated their effectiveness in Salient Object Detection (SOD), being significantly lighter than existing lightweight models. This study revisits FLIM SOD and introduces FLIM-Bag of Feature Points (FLIM-BoFP), a considerably faster filter estimation method. The previous approach, FLIM-Cluster, derives filters through patch clustering at each encoder's block, leading to computational overhead and reduced control over filter locations. FLIM-BoFP streamlines this process by performing a single clustering at the input block, creating a bag of feature points, and defining filters directly from mapped feature points across all blocks. The paper evaluates the benefits in efficiency, effectiveness, and generalization of FLIM-BoFP compared to FLIM-Cluster and other state-of-the-art baselines for parasite detection in optical microscopy images.
Read the full abstract ▾
All authors João Deltregia Martinelli, Marcelo Luis Rodrigues Filho, Felipe Crispim da Rocha Salvagnini, Gilson Junior Soares, Jefersson A. dos Santos, Alexandre X. Falcão
Computer Vision
Training-Free Multi-Concept Image Editing
Niki Foteinopoulou, Ignas Budvytis and Stephan Liwicki
Training-free image editing with
diffusion models is highly desirable yet is complex and remains a significant challenge. While recent optimisation-based methods achieve strong zero-shot edits from text, they still struggle to preserve identity and capture intricate details, such as facial structure, surface texture, or object-specific geometry, that exist below the level of linguistic abstraction. To address this fundamental gap, we propose Concept
Distillation Sampling (CDS). To the best of our knowledge, we are the first to introduce a unified, training-free framework for target-less, multi-concept image editing. CDS overcomes this linguistic bottleneck of previous methods by anchoring the editing process in the certainty of pretrained LoRA adapters. We integrate a highly stable
distillation backbone (featuring ordered timesteps, regularisation, and negative-
prompt guidance) with a novel dynamic weighting mechanism. This approach enables the composition and control of multiple visual concepts directly within the
diffusion process, utilising spatially-aware priors from pretrained LoRA adapters without causing concept clashing. Our method preserves instance concept identity without requiring reference samples of the desired edit. Extensive quantitative and qualitative evaluations demonstrate that CDS establishes a new state-of-the-art over existing training-free editing and multi-LoRA composition methods on the InstructPix2Pix and ComposLoRA
benchmarks. Project Page: https://nickyfot.github.io/cds/.
Read the full abstract ▾
Computer Vision
GatedCLIP: Gated Multimodal Fusion for Hateful Memes Detection
Yingying Guo, Ke Zhang and Zirong Zeng
Detecting hateful content in
multimodal memes presents unique challenges, as harmful messages often emerge from the complex interplay between benign images and text. We propose GatedCLIP, a
Vision-
Language model that enhances CLIP's
multimodal capabilities with specialized architectural improvements for hateful memes detection. Our approach introduces learned projection heads that map CLIP
embeddings to a task-optimized semantic space, a dynamic gated fusion mechanism that adaptively weights visual and textual features, and a contrastive learning objective that maintains cross-modal semantic
alignment. Experiments on the Hateful Memes
dataset demonstrate that GatedCLIP achieves an AUROC of 0.66, substantially outperforming the CLIP baseline (AUROC 0.49) while maintaining computational efficiency with only 350K trainable parameters.
Read the full abstract ▾
Computer Vision
RU4D-SLAM: Reweighting Uncertainty in Gaussian Splatting SLAM for 4D Scene Reconstruction
Yangfan Zhao, Hanwei Zhang, Ke Huang et al. (+3 more)
Combining 3D Gaussian splatting with Simultaneous Localization and Mapping (SLAM) has gained popularity as it enables continuous 3D environment reconstruction during motion. However, existing methods struggle in dynamic environments, particularly moving objects complicate 3D reconstruction and, in turn, hinder reliable tracking. The emergence of 4D reconstruction, especially 4D Gaussian splatting, offers a promising direction for addressing these challenges, yet its potential for 4D-aware SLAM remains largely underexplored. Along this direction, we propose a robust and efficient framework, namely Reweighting Uncertainty in Gaussian Splatting SLAM (RU4D-SLAM) for 4D scene reconstruction, that introduces temporal factors into spatial 3D representation while incorporating uncertainty-aware perception of scene changes, blurred image synthesis, and dynamic scene reconstruction. We enhance dynamic scene representation by integrating motion blur rendering, and improve uncertainty-aware tracking by extending per-pixel uncertainty modeling, which is originally designed for static scenarios, to handle blurred images. Furthermore, we propose a semantic-guided reweighting mechanism for per-pixel uncertainty estimation in dynamic scenes, and introduce a learnable opacity weight to support adaptive 4D mapping. Extensive experiments on standard
benchmarks demonstrate that our method substantially outperforms state-of-the-art approaches in both trajectory accuracy and 4D scene reconstruction, particularly in dynamic environments with moving objects and low-quality inputs. Code available: https://ru4d-slam.github.io
Read the full abstract ▾
All authors Yangfan Zhao, Hanwei Zhang, Ke Huang, Qiufeng Wang, Zhenzhou Shao, Dengyu Wu
Computer Vision
VGGDrive: Empowering Vision-Language Models with Cross-View Geometric Grounding for Autonomous Driving
Jie Wang, Guang Li, Zhijian Huang et al. (+4 more)
The significance of cross-view 3D
geometric modeling capabilities for autonomous driving is self-evident, yet existing
Vision-
Language Models (VLMs) inherently lack this capability, resulting in their mediocre performance. While some promising approaches attempt to mitigate this by constructing Q&A data for auxiliary training, they still fail to fundamentally equip VLMs with the ability to comprehensively handle diverse evaluation protocols. We thus chart a new course, advocating for the infusion of VLMs with the cross-view
geometric grounding of mature 3D foundation models, closing this critical capability gap in autonomous driving. In this spirit, we propose a novel architecture, VGGDrive, which empowers
Vision-
language models with cross-view
Geometric Grounding for autonomous Driving. Concretely, to bridge the cross-view 3D
geometric features from the frozen visual 3D model with the VLM's 2D visual features, we introduce a plug-and-play Cross-View 3D
Geometric Enabler (CVGE). The CVGE decouples the base VLM architecture and effectively empowers the VLM with 3D features through a hierarchical adaptive injection mechanism. Extensive experiments show that VGGDrive enhances base VLM performance across five autonomous driving
benchmarks, including tasks like cross-view risk perception, motion prediction, and trajectory planning. It's our belief that mature 3D foundation models can empower autonomous driving tasks through effective integration, and we hope our initial exploration demonstrates the potential of this paradigm to the autonomous driving community.
Read the full abstract ▾
All authors Jie Wang, Guang Li, Zhijian Huang, Chenxu Dang, Hangjun Ye, Yahong Han, Long Chen
Computer Vision
SIMSPINE: A Biomechanics-Aware Simulation Framework for 3D Spine Motion Annotation and Benchmarking
Muhammad Saif Ullah Khan and Didier Stricker
Modeling spinal motion is fundamental to understanding human biomechanics, yet remains underexplored in computer
vision due to the spine's complex multi-joint kinematics and the lack of large-scale 3D annotations. We present a biomechanics-aware keypoint simulation framework that augments existing human pose
datasets with anatomically consistent 3D spinal keypoints derived from musculoskeletal modeling. Using this framework, we create the first open
dataset, named SIMSPINE, which provides sparse vertebra-level 3D spinal annotations for natural full-body motions in indoor multi-camera capture without external restraints. With 2.14 million frames, this enables data-driven learning of vertebral kinematics from subtle posture variations and bridges the gap between musculoskeletal simulation and computer
vision. In addition, we release pretrained baselines covering fine-tuned 2D detectors,
monocular 3D pose lifting models, and multi-view reconstruction pipelines, establishing a unified
benchmark for biomechanically valid spine motion estimation. Specifically, our 2D spine baselines improve the state-of-the-art from 0.63 to 0.80 AUC in controlled environments, and from 0.91 to 0.93 AP for in-the-wild spine tracking. Together, the simulation framework and SIMSPINE
dataset advance research in
vision-based biomechanics, motion analysis, and digital human modeling by enabling reproducible, anatomically grounded 3D spine estimation under natural conditions.
Read the full abstract ▾
Computer Vision
Real-time Motion Segmentation with Event-based Normal Flow
Sheng Zhong, Zhongyang Ren, Xiya Zhu et al. (+3 more)
Event-based cameras are bio-inspired sensors with pixels that independently and asynchronously respond to brightness changes at microsecond resolution, offering the potential to handle visual tasks in challenging scenarios. However, due to the sparse information content in individual events, directly processing the raw event data to solve
vision tasks is highly inefficient, which severely limits the applicability of state-of-the-art methods in real-time tasks, such as motion
segmentation, a fundamental task for dynamic scene understanding. Incorporating normal flow as an intermediate representation to compress motion information from event clusters within a localized region provides a more effective solution. In this work, we propose a normal flow-based motion
segmentation framework for event-based
vision. Leveraging the dense normal flow directly learned from event neighborhoods as input, we formulate the motion
segmentation task as an energy minimization problem solved via graph cuts, and optimize it iteratively with normal flow clustering and motion model fitting. By using a normal flow-based motion model initialization and fitting method, the proposed system is able to efficiently estimate the motion models of independently moving objects with only a limited number of candidate models, which significantly reduces the computational complexity and ensures real-time performance, achieving nearly a 800x speedup in comparison to the open-source state-of-the-art method. Extensive evaluations on multiple public
datasets fully demonstrate the accuracy and efficiency of our framework.
Read the full abstract ▾
All authors Sheng Zhong, Zhongyang Ren, Xiya Zhu, Dehao Yuan, Cornelia Fermuller, Yi Zhou
Computer Vision
Federated Learning for Cross-Modality Medical Image Segmentation via Augmentation-Driven Generalization
Sachin Dudda Nagaraju, Ashkan Moradi, Bendik Skarre Abrahamsen et al. (+1 more)
Purpose: Developing generalizable medical image
segmentation models is challenging because imaging data are distributed across institutions and differ in modality and acquisition protocol.
Federated learning (FL) enables collaborative training without centralizing raw medical images, but cross-modality domain shifts between computed
tomography (CT) and
magnetic resonance imaging (MRI) can substantially reduce model performance. This study investigates augmentation-driven cross-modality FL for abdominal organ and whole-heart
segmentation. Methods: We evaluate convolution-based spatial augmentation, frequency-domain argumentation, domain-specific normalization, and global intensity nonlinear (GIN) augmentation for
multimodal segmentation. Abdominal organ
segmentation and whole-heart
segmentation are first evaluated using a 2D U-Net framework. For whole-heart
segmentation, we additionally perform native 3D experiments using a self-configuring nnU-Net architecture on the CARE-WHS 2026
dataset, enabling evaluation of whether the observed cross-modality FL behavior persists when moving from slice-based 2D
segmentation to volumetric 3D
segmentation. Results: GIN provides the most consistent cross-modality performance among the evaluated approaches in the original 2D experiments. For pancreas
segmentation, the Dice similarity coefficient (DSC) improved from 0.073 to 0.437 when CT data were incorporated through
federated cross-modality training. In 3D whole-heart
segmentation, FedGIN improved mean DSC over FedAvg from 0.8696 to 0.8901 on the unseen CT center and from 0.7160 to 0.7956 on the unseen MRI center. Relative to centralized GIN training, FedGIN retained 92.4% of performance on unseen CT data and achieved comparable performance on unseen MRI data (0.7956 versus 0.7937).
Read the full abstract ▾
All authors Sachin Dudda Nagaraju, Ashkan Moradi, Bendik Skarre Abrahamsen, Mattijs Elschot
Computer Vision
OrthoDiffusion: A Generalizable Multi-Task Diffusion Foundation Model for Musculoskeletal MRI Interpretation
Tian Lan, Lei Xu, Zimu Yuan et al. (+8 more)
Musculoskeletal disorders represent a significant global health burden and are a leading cause of disability worldwide. While MRI is essential for accurate diagnosis, its interpretation remains exceptionally challenging. Radiologists must identify multiple potential abnormalities within complex anatomical structures across different imaging planes, a process that requires significant expertise and is prone to variability. We developed OrthoDiffusion, a unified
diffusion-based foundation model designed for multi-task musculoskeletal MRI interpretation. The framework utilizes three orientation-specific 3D
diffusion models, pre-trained in a self-supervised manner on 15,948 unlabeled knee MRI scans, to learn robust anatomical features from sagittal, coronal, and axial views. These view-specific representations are integrated to support diverse clinical tasks, including anatomical
segmentation and multi-label diagnosis. Our evaluation demonstrates that OrthoDiffusion achieves excellent performance in the
segmentation of 11 knee structures and the detection of 8 knee abnormalities. The model exhibited remarkable robustness across different clinical centers and MRI field strengths, consistently outperforming traditional supervised models. Notably, in settings where labeled data was scarce, OrthoDiffusion maintained high diagnostic precision using only 10% of training labels. Furthermore, the anatomical representations learned from knee imaging proved highly transferable to other joints, achieving strong diagnostic performance across 11 diseases of the ankle and shoulder. These findings suggest that
diffusion-based foundation models can serve as a unified platform for multi-disease diagnosis and anatomical
segmentation, potentially improving the efficiency and accuracy of musculoskeletal MRI interpretation in real-world clinical workflows.
Read the full abstract ▾
All authors Tian Lan, Lei Xu, Zimu Yuan, Shanggui Liu, Jiajun Liu, Jiaxin Liu, Weilai Xiang, Hongyu Yang, Dong Jiang, Jianxin Yin, Dingyu Wang
Computer Vision · Cross-listed from cs.AI
PyVision-RL: Forging Open Agentic Vision Models via RL
Shitian Zhao, Shaoheng Lin, Ming Li et al. (+4 more)
Reinforcement learning for agentic
multimodal models often suffers from interaction collapse, where models learn to reduce tool usage and multi-turn
reasoning, limiting the benefits of agentic behavior. We introduce PyVision-RL, a reinforcement learning framework for open-weight
multimodal models that stabilizes training and sustains interaction. Our approach combines an oversampling-filtering-ranking rollout strategy with an accumulative tool reward to prevent collapse and encourage multi-turn tool use. Using a unified training pipeline, we develop PyVision-Image and PyVision-
Video for image and
video understanding. For
video reasoning, PyVision-
Video employs on-demand context construction, selectively sampling task-relevant frames during
reasoning to significantly reduce visual
token usage. Experiments show strong performance and improved efficiency, demonstrating that sustained interaction and on-demand visual processing are critical for scalable
multimodal agents.
Read the full abstract ▾
All authors Shitian Zhao, Shaoheng Lin, Ming Li, Haoquan Zhang, Wenshuo Peng, Kaipeng Zhang, Chen Wei
Computer Vision
COMiT: Learning Structured Visual Tokens through Sequential Communication
Aram Davtyan, Yusuf Sahin, Yasaman Haghighi et al. (+4 more)
Discrete image tokenizers provide a sequential interface for
vision and
multimodal models, but are typically optimized for reconstruction or compression and therefore tend to encode local appearance rather than object-level structure. We introduce COMiT, a communication-inspired framework for learning structured discrete visual representations. COMiT constructs a fixed-length latent message through sequential visual observations: at each step, a
transformer processes a localized image crop and updates, refines, and reorganizes the existing
token sequence. After several iterations, the resulting message conditions a flow-matching decoder that reconstructs the complete image. The encoder and decoder are implemented within a single
transformer and trained end-to-end using flow-matching reconstruction and semantic representation-
alignment objectives. COMiT substantially improves compositional generalization and relational
reasoning over prior methods. Our experiments show that, while semantic
alignment helps ground the representation, attentive sequential tokenization is critical for inducing more interpretable, object-centric
token structures.
Read the full abstract ▾
All authors Aram Davtyan, Yusuf Sahin, Yasaman Haghighi, Sebastian Stapf, Pablo Acuaviva, Alexandre Alahi, Paolo Favaro
Computer Vision
Bridging Rendering and Generative Modeling with Monte Carlo Transport Scheduling
Junwei Shu, Wenjie Liu, Hantang Liu et al. (+2 more)
Monte Carlo rendering and modern generative models both transform uncertain states into structured images, yet they are usually studied as separate processes. We introduce Monte Carlo Transport Scheduling, a framework that treats progressive path tracing as a continuous sampling-driven transport process. Our key observation is that the renderer already produces physically valid states along this process: nested Monte Carlo estimates trace a refinement trajectory whose natural time coordinate follows from sampling variance. This view leads to a continuous training framework that learns from real render endpoints rather than synthetic interpolants, preserving the statistical structure of Monte Carlo estimation while enabling arbitrary-step neural refinement. We evaluate the framework on a controlled rendering
benchmark designed to separate transport difficulty from scene context, and show that it yields stable render refinement, supports continuous stopping between rendering states, and transfers as a physical prior for frozen generative samplers. These results suggest a common continuous-time substrate for rendering and generation, where Monte Carlo sampling provides both the physical states and the supervision for learning image transport.
Read the full abstract ▾
All authors Junwei Shu, Wenjie Liu, Hantang Liu, Changbo Wang, Yang Li
Computer Vision
CleanStyle: Plug-and-Play Style Conditioning Purification for Text-to-Image Stylization
Xiaoman Feng, Mingkun Lei, Yang Wang et al. (+2 more)
Style transfer in
diffusion models enables controllable visual generation by injecting the style of a reference image. However, recent encoder-based methods, while efficient and tuning-free, often suffer from content leakage, where semantic elements from the style image undesirably appear in the output, impairing
prompt fidelity and stylistic consistency. In this work, we introduce CleanStyle, a plug-and-play framework that filters out content-related noise from the style
embedding without retraining. Motivated by empirical analysis, we observe that such leakage predominantly stems from the tail components of the style
embedding, which are isolated via Singular Value Decomposition (SVD). To address this, we propose CleanStyleSVD (CS-SVD), which dynamically suppresses tail components using a time-aware exponential schedule, providing clean, style-preserving conditional
embeddings throughout the denoising process. Furthermore, we present Style-Specific Classifier-Free Guidance (SS-CFG), which reuses the suppressed tail components to construct style-aware unconditional inputs. Unlike conventional methods that use generic negative
embeddings (e.g., zero vectors), SS-CFG introduces targeted negative signals that reflect style-specific but
prompt-irrelevant visual elements. This enables the model to effectively suppress these distracting patterns during generation, thereby improving
prompt fidelity and enhancing the overall visual quality of stylized outputs. Our approach is lightweight, interpretable, and can be seamlessly integrated into existing encoder-based
diffusion models without retraining. Extensive experiments demonstrate that CleanStyle substantially reduces content leakage, improves stylization quality and improves
prompt alignment across a wide range of style references and
prompts.
Read the full abstract ▾
All authors Xiaoman Feng, Mingkun Lei, Yang Wang, Dingwen Fu, Chi Zhang
Computer Vision
Monocular Endoscopic Tissue 3D Reconstruction with Multi-Level Geometry Regularization
Yangsen Chen and Hao Wang
Reconstructing deformable endoscopic tissues is crucial for achieving
robot-assisted surgery. However, 3D Gaussian Splatting-based approaches encounter challenges in achieving consistent tissue surface reconstruction, while existing NeRF-based methods lack real-time rendering capabilities. In pursuit of both smooth deformable surfaces and real-time rendering, we introduce a novel approach based on 3D Gaussian Splatting. Specifically, we introduce surface-aware reconstruction, initially employing a Sign Distance Field-based method to construct a mesh, subsequently utilizing this mesh to constrain the Gaussian Splatting reconstruction process. Furthermore, to ensure the generation of physically plausible deformations, we incorporate local
rigidity and global non-
rigidity restrictions to guide Gaussian deformation, tailored for the highly deformable nature of soft endoscopic tissue. Based on 3D Gaussian Splatting, our proposed method delivers a fast rendering process and smooth surface appearances. Quantitative and qualitative analysis against alternative methodologies shows that our approach achieves solid reconstruction quality in both textures and geometries.
Read the full abstract ▾
Computer Vision
Onboard-Targeted Segmentation of Straylight in Space Camera Sensors
Riccardo Gallon, Fabian Schiemenz, Alessandra Menicucci et al. (+1 more)
This study details an artificial intelligence (AI)-based methodology for the semantic
segmentation of space camera faults. Specifically, we address the
segmentation of straylight effects induced by
solar presence around the camera's Field of View (FoV). Anomalous images are sourced from our published
dataset. Our approach emphasizes generalization across diverse
flare textures, leveraging pre-training on a public
dataset (
Flare7k++) including
flares in various non-space contexts to mitigate the scarcity of realistic space-specific data. A DeepLabV3 model with MobileNetV3 backbone performs the
segmentation task. The model design targets deployment in spacecraft resource-constrained hardware. Finally, based on a proposed interface between our model and the onboard
navigation pipeline, we develop custom metrics to assess the model's performance in the system-level context.
Read the full abstract ▾
All authors Riccardo Gallon, Fabian Schiemenz, Alessandra Menicucci, Eberhard Gill
Computer Vision
NGL: Natural Garment Language for Training-Free Sewing Pattern Estimation
Anna Badalyan, Pratheba Selvaraju, Giorgio Becherini et al. (+3 more)
Estimating sewing patterns from images is a practical approach for creating high-quality 3D garments, but it remains challenging due to the scarcity of paired real-world image and sewing-pattern data. Existing methods address this limitation by training
vision-
language models (VLMs) to learn low-level sewing-pattern representations from synthetic garments sampled from parametric garment models. However, they often struggle to generalize to in-the-wild images, fail to capture real-world correlations between garment parts, and are restricted to single-layer outfits. In contrast, we observe that VLMs are effective at describing garments in natural
language, but mapping these descriptions into valid sewing patterns remains difficult. To bridge this gap, we propose NGL (Natural Garment
Language), a novel domain-specific
language that represents garments in terms aligned with VLMs' natural descriptive abilities. Leveraging NGL, we introduce a fully training-free pipeline that queries large VLMs to extract structured garment specifications and deterministically converts them into valid sewing patterns. We evaluate our method on the Dress4D, CloSe and a newly collected
dataset of 253 in-the-wild fashion images. Our approach achieves state-of-the-art performance on standard geometry metrics and is preferred in both human and GPT-based perceptual evaluations compared to existing baselines. Furthermore, NGL recovers multi-layer outfits whereas competing methods focus mostly on single-layer garments, highlighting its strong generalization to real-world images even with occluded parts. These results demonstrate that an efficient garment representation is critical for sewing pattern estimation with VLMs. Our code and data will be released for research use.
Read the full abstract ▾
All authors Anna Badalyan, Pratheba Selvaraju, Giorgio Becherini, Omid Taheri, Victoria Fernandez Abrevaya, Michael Black
Computer Vision
MatchED: Crisp Edge Detection Using End-to-End, Matching-based Supervision
Bedrettin Cetinkaya, Sinan Kalkan and Emre Akbas
Generating crisp, i.e., one-pixel-wide, edge maps remains one of the fundamental challenges in edge detection, affecting both traditional and learning-based methods. To obtain crisp edges, most existing approaches rely on two hand-crafted post-processing
algorithms, Non-Maximum Suppression (NMS) and skeleton-based thinning, which are non-differentiable and hinder end-to-end optimization. Moreover, all existing crisp edge detection methods still depend on such post-processing to achieve satisfactory results. To address this limitation, we propose \MethodLPP, a lightweight, only $\sim$21K additional parameters, and plug-and-play matching-based supervision module that can be appended to any edge detection model for joint end-to-end learning of crisp edges. At each training iteration, \MethodLPP performs one-to-one matching between predicted and ground-truth edges based on spatial distance and confidence, ensuring consistency between training and testing protocols. Extensive experiments on four popular
datasets demonstrate that integrating \MethodLPP substantially improves the performance of existing edge detection models. In particular, \MethodLPP increases the Average Crispness (AC) metric by up to 2–4$\times$ compared to baseline models. Under the crispness-emphasized evaluation (CEval), \MethodLPP further boosts baseline performance by up to 20–35% in ODS and achieves similar gains in OIS and AP, achieving SOTA performance that matches or surpasses standard post-processing for the first time. Code is available at https://cvpr26-matched.github.io.
Read the full abstract ▾