Computer Vision
C2P-VAR: Continual and Compositional Personalization in Visual Autoregressive Models
Junhao Li, Xinhao Zhong, Yi sun et al. (+4 more)
Visual autoregressive (VAR) models have recently emerged as an efficient paradigm for text-to-image generation, yet their personalization capabilities remain largely limited to static, single-concept settings. In practice, users may continuously introduce new concepts and wish to compose multiple personalized concepts within a single image. Such scenarios pose two fundamental challenges: catastrophic forgetting during sequential personalization and feature interference during multi-concept composition. In this work, we study continual and compositional personalization in VAR models and propose C2P-VAR, a unified framework addressing both challenges. For continual personalization, we introduce C2PVAR-S, which identifies concept-relevant parameters from gradient magnitudes and dynamically updates their selection during training. To preserve previously learned concepts, C2PVAR-S applies regularization only to parameters shared by the current and historical concepts, thereby reducing unnecessary interference without introducing additional model components. For multi-concept personalization, we further propose C2PVAR-M, which employs parallel global and concept-specific branches with spatially localized feature fusion and logit aggregation to achieve controllable concept placement and reduce feature
entanglement. Extensive experiments on continual and multi-concept personalization demonstrate that C2P-VAR consistently outperforms existing baselines in subject fidelity, while maintaining competitive text
alignment and introducing negligible storage and inference overhead. Our results establish a unified framework for scalable and controllable personalization of visual autoregressive models.
Read the full abstract ▾
All authors Junhao Li, Xinhao Zhong, Yi sun, Yuxia Qiao, Bin Chen, Shu-Tao Xia, Yaowei Wang
Computer Vision
Real-World On-Vehicle Evaluation of Embedding-Based Anomaly Detection
Albert Schotschneider, Daniel Bogdoll, Svetlana Pavlitska et al. (+2 more)
Detecting anomalies in traffic scenes is crucial for ensuring safety in autonomous driving, yet collecting representative anomalous data remains challenging. Existing anomaly detection methods are highly specialized and rely on normality as defined by the abstract semantic Cityscapes classes, making it difficult to adapt to diverse real-world scenarios. We propose an adaptable real-time anomaly detection method that leverages foundation models in the form of pretrained
vision transformer embeddings to detect deviations via nearest-neighbor similarity in the latent semantic feature space. Based on patch-wise processing, the
algorithm produces dense anomaly masks, allowing for the localization of detected anomalies. The method robustly models normality through a single reference image. This formulation avoids explicit supervision and
dataset-specific training, making it suitable for real-world deployment. We evaluate the method on standard
benchmarks and on an automated vehicle in real-world scenarios. Despite its simplicity, the method achieves good performance on the Road Anomaly
benchmark and demonstrates consistent qualitative behavior in practice, successfully highlighting semantically unusual objects in diverse scenes. These results suggest that simple, reference-based methods can provide useful anomaly signals under realistic operating conditions.
Read the full abstract ▾
All authors Albert Schotschneider, Daniel Bogdoll, Svetlana Pavlitska, Ahmed Abouelazm, Johann Marius Zoellner
Computer Vision
FlowErase-RL: Rethinking Concept Erasure as Reward Optimization in Flow Matching Models
Yi Sun, Zhiqi Zhang, Xinhao Zhong et al. (+5 more)
Recent advances in flow matching models have significantly improved text-to-image generation quality, but also introduce growing safety risks due to the generation of harmful or undesirable content. Existing concept erasure methods are either inference-time interventions with limited effectiveness or rely on supervised fine-tuning (SFT), which requires precisely aligned data and struggles with scalability and multi-concept settings. In this paper, we propose FlowErase-RL, the first GRPO-based framework for concept erasure in flow matching models. We reformulate concept erasure as a reward optimization problem and introduce a dynamic dual-path reward mechanism that jointly optimizes (i) a Concept Erasure (CE) reward to suppress target concepts and (ii) a Non-target Space (NS) reward to preserve generative fidelity. The two reward paths are adaptively balanced during training via a performance-driven switching strategy, enabling stable optimization without explicit supervision. Extensive experiments on nudity, object, and artistic style erasure demonstrate that our method achieves state-of-the-art erasure performance while maintaining strong image quality and semantic
alignment. Moreover, it exhibits robust resistance to
adversarial attacks and scales effectively to multi-concept scenarios. Our results establish a new paradigm for safe and controllable generation in flow matching models.
Read the full abstract ▾
All authors Yi Sun, Zhiqi Zhang, Xinhao Zhong, Yimin Zhou, Shuoyang Sun, Bin Chen, Shu-Tao Xia, Ke Xu
Computer Vision · Cross-listed from cs.GR
Decentralized Direct Volume Rendering: A Browser-Native GPU Architecture for MRI Digital Twins in Resource-Constrained Settings
Oserebameh Augustine Beckley
Digital Twin (DT) technology holds immense potential for
surgical planning and personalized medicine. However, generating interactive, patient-specific anatomical twins currently relies on computationally heavy Server-Side Rendering (SSR) or expensive local workstations, creating significant barriers to deployment, especially in resource-constrained settings (RCS). This paper presents a decentralized, client-side WebGPU architecture that democratizes access to high-fidelity anatomical Digital Twins. By bypassing standard server-side rendering pipelines, the framework executes deterministic single-pass raymarching and morphological gradient calculations directly on low-cost integrated edge GPUs. Eliminating the network latency inherent to cloud-rendered solutions, the system achieves a Time to First Pixel (TTFP) of under 920.0ms and maintains stable interactivity at >= 82.0 FPS. Continuous Interaction Fidelity is maintained via uniform buffers, enabling zero-latency manipulation of tissue parameters for dynamic clinical decision-making. By proving that complex 3D medical simulations of patient-specific MRI scan can be executed natively in the browser without deep learning or external computational dependencies, this architecture provides a scalable, affordable foundation for the widespread clinical adoption of healthcare Digital Twins.
Read the full abstract ▾
Computer Vision
GeoMamba: A Geometry-driven MambaVision Framework and Dataset for Fine-grained Optical-SAR Object Retrieval
Tiantong Fang, Xiuwei Wang, Jing Xiao et al. (+3 more)
Multi-source remote sensing enables complementary observation of ground objects, while cross-modal fine-grained object
retrieval remains challenging, especially under unaligned optical and SAR conditions. Unlike conventional
retrieval settings that rely on paired or spatially aligned samples, practical optical-SAR
retrieval is affected by substantial modality discrepancy, speckle noise, and structural inconsistency, which limit robust cross-modal representation learning. To address this problem, we propose GeoMamba, a geometry-driven framework tailored for optical-SAR fine-grained
retrieval. Specifically, GeoMamba introduces a
Geometric Feature Injection (GFI) module that enhances cross-modal feature interaction and incorporates structural priors, thereby improving the robustness of SAR representations and promoting geometry-consistent feature learning. In addition, a
Geometric Consistency Constraint (GCC) module, together with a Deep Supervision (DS) strategy, imposes hierarchical
geometric constraints using classical operators, which helps preserve informative object structures during representation learning. We further construct a new
dataset, FGOS-as, containing 11 aerospace and maritime categories for evaluating unaligned cross-modal fine-grained object
retrieval in realistic remote sensing scenarios. Extensive experiments on FGOS-as demonstrate that GeoMamba outperforms existing methods, achieving 63.3% mAP and 77.0% Rank-1 accuracy in all-to-all
retrieval setting.
Read the full abstract ▾
All authors Tiantong Fang, Xiuwei Wang, Jing Xiao, Wujie Zhou, Liang Liao, Mi Wang
Computer Vision
LIFT and PLACE: A Simple, Stable, and Effective Knowledge Distillation Framework for Lightweight Diffusion Models
Hyunsoo Han, Sangyeop Yeo and Jaejun Yoo
We demonstrate that in knowledge
distillation for
diffusion models, the teacher network's highly complex denoising process - stemming from its substantially larger capacity - poses a significant challenge for the student model to faithfully mimic. To address this problem, we propose a coarse-to-fine
distillation framework with LInear FiTtingbased
distillation (LIFT) and Piecewise Local Adaptive Coefficient Estimation (PLACE). First, LIFT decomposes the objective into a "coarse"
alignment and a "fine" refinement. The student is then trained on coarse
alignment before proceeding to hard refinement. Second, PLACE extends LIFT to address spatially non-uniform errors by partitioning outputs into error-based groups, providing locally adaptive guidance. Our experiments show that LIFT and PLACE is effective across
diffusion spaces (image/latent), backbones (U-Net/DiT), tasks (unconditional/conditional),
datasets, and even extends to flow-based models such as MMDiT (SD3). Furthermore, under extreme compression with a 1.3M-parameter student (only 1.6% of the teacher), conventional KD fails to provide sufficient guidance for stable training, with FID scores often degrading to 50-200+, but our method remains stably convergent and achieves an FID of 15.73.
Read the full abstract ▾
Computer Vision
Aero-World: Action-Conditioned Aerial Video Generation from Inertial Controls
Abdul Mohaimen Al Radi, Kunyang Li, Yuzhang Shang et al. (+2 more)
Foundation
video models produce visually impressive results, but their use in
embodied AI remains limited because they are primarily trained on natural
language rather than low-level control signals. This limitation is especially pronounced for aerial flight, where motion occurs in unconstrained 6-DoF space and small errors in ego-motion can produce large trajectory drift. Generating aerial
videos that follow fine-grained inertial actions can support scalable training and evaluation of aerial
agents by providing a controllable proxy for real-world or expensive simulation data. To address this problem, we propose Aero-World, a method for converting a pretrained image-to-
video diffusion model into a controllable aerial
video generator. Aero-World injects sequences of translational acceleration and angular velocity into a pretrained latent
diffusion transformer through an action-
token stream. A frozen latent-space Physics Probe, trained independently on real
video–IMU pairs, provides differentiable inertial-consistency supervision during LoRA finetuning while avoiding computationally expensive
video decoding. We further propose AeroBench, a
benchmark for evaluating whether generated drone
videos adhere to low-level action signals. AeroBench uses Action
Alignment Score (AAS) to measure agreement with commanded inertial actions and Physical Consistency Rate (PCR) to measure temporal motion stability. On AeroBench, Aero-World improves mean AAS from 57.7 to 63.6 over action-only finetuning and gives a stronger quality-control trade-off than AirScape, with lower FVD (596.5 vs. 1058.6), higher SSIM (0.595 vs. 0.505), and higher Flow-IMU correlation (0.44 vs. 0.20). These results suggest that frozen Physics Probe supervision is a practical mechanism for adapting pretrained
video generators toward more action-aligned aerial motion.
Read the full abstract ▾
All authors Abdul Mohaimen Al Radi, Kunyang Li, Yuzhang Shang, Mubarak Shah, Yu Tian
Computer Vision
Tango3D: Towards Alignment for Global and Local 2D-3D Correspondence
Zebin He, Mingxin Yang, Shuhui Yang et al. (+4 more)
Existing 3D foundation models typically align point clouds to frozen
vision-
language spaces like CLIP, which achieve strong cross-modal
retrieval by compressing 3D shape into a global vector. However, this global-only
alignment cannot establish fine-grained pixel-to-point correspondence. To solve this, we present Tango3D, a foundation model that unifies dense correspondence and global
retrieval. We use a geometry-aware 2D visual backbone and a pretrained 3D VAE to encode images into 2D patches and point clouds into 3D
tokens. These are mapped into a single shared space to achieve both local pixel-to-point
alignment and global semantic
alignment. To stabilize the joint learning of dense and global objectives, we introduce a three-stage progressive training strategy. Experiments show our model successfully achieves object-level pixel-to-point
alignment while maintaining competitive global
retrieval, a joint capability not offered by existing 3D foundation models. By establishing a fine-grained
alignment feature space, Tango3D injects rich semantics into purely
geometric 3D
tokens, paving the way for a wide range of dense 3D downstream tasks.
Read the full abstract ▾
All authors Zebin He, Mingxin Yang, Shuhui Yang, Hanxiao Sun, Xintong Han, Chunchao Guo, Wenhan Luo
Computer Vision
Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention
Wenhu Zhang, Yiming Wu, Huanyu Wang et al. (+6 more)
Diffusion Language Models (DLMs) enable globally coherent, bidirectional, and controllable text generation, offering advantages over traditional autoregressive LLMs, while scaling to ultra-long sequences remains costly. Many existing block-sparse
attention methods select blocks by fixed sampling patterns over the high-resolution
attention space, such as tail regions or anti-diagonal stripes. Such prior-driven sampling can miss salient
tokens and introduce instability under distribution shifts. In this paper, we propose the Block Approximate Sparse
Attention framework (BA-Att) with block-wise pre-downsampled operation, which identifies informative regions within a compact downsampled space, avoiding reliance on brittle positional priors. To analyze its theoretical behavior, we define an oracle post-downsample
attention map and formalize the approximation error between pre- and post-downsample schemes. Based on this insight, we introduce a lightweight norm-sorting module and a covariance-compensated correction that approximates full covariance using diagonal QK variances, reducing computational complexity. Extensive experiments show that our operator achieves up to 6.95x acceleration over FlashAttention in
attention computation, and maintains near full-
attention performance at 50% sparsity across
language models,
multimodal language models, and
video generation models, demonstrating strong efficiency and generalization.
Read the full abstract ▾
All authors Wenhu Zhang, Yiming Wu, Huanyu Wang, Yaoyang Liu, Huanzhang Dou, Senqiao Yang, Sitong Wu, Hanbin Zhao, Jiaya Jia
Computer Vision
Physics-in-the-Loop: A Hybrid Agentic Architecture for Validated CAD Engineering Design
Elias Berger, Muhammad Usama, Jan Mehlstäubl et al. (+2 more)
Large
Language Models (LLMs) can generate Computer-Aided Design (CAD), yet lack physical comprehension required for reliable engineering design. Instead of attempting to implicitly learn physical laws from data, we propose a Hybrid Agentic-Physical Architecture that embeds validated knowledge-based engineering tools directly into the decision making loop of autonomous AI
agents. In this framework, engineering design is formulated as a closed-loop, sequential decision making process guided by explicit physical verification. Based on a load case, dedicated
agents iteratively plan, generate, evaluate, and revise engineering designs using knowledge-based tools as a feedback signal. We introduce a
benchmark dataset and metrics for assessing functional validity in generative CAD. Our system generates more complex and physically verified designs, with a 4.2 increase in structural complexity and improving compile rate by 3.5% compared to similar agentic methods. The codebase,
prompts and
dataset will be made publicly available to support reproducibility and future research.
Read the full abstract ▾
All authors Elias Berger, Muhammad Usama, Jan Mehlstäubl, Bernhard Saske, Kristin Paetzold-Byhain
Computer Vision
Physics-informed simulation framework for realistic sonar image generation and statistical validation
Kamal Basha S and Athira Nambiar
Synthetic sonar
datasets offer a scalable alternative to costly real-world acquisition, yet their utility remains limited by the absence of rigorous quantitative validation. We present ACOUSIM (ACOustic SIMulation and Validation Platform), a physics-informed framework that evaluates the statistical
alignment between synthetic and real sonar imagery without relying on generative models. A Gazebo-based environment generates sonar-like images by explicitly controlling seabed texture, illumination-driven shadowing, platform altitude, and noise. Realism is quantified against two public sonar
datasets, SeabedObjects-KLSG-II and Sonar Common Target Detection (SCTD), using global intensity and local texture (LBP) distributions assessed via Kullback-Leibler divergence, Jensen-Shannon divergence, and Earth Mover's Distance. Results show strong texture
alignment (KL < 0.07) across all classes, with plane-class intensity
alignment outperforming ship-class due to shadow geometry complexity. ACOUSIM establishes a reproducible, distribution-level baseline for sim-to-real sonar evaluation and directly supports reliable
dataset validation for
underwater image analysis.
Read the full abstract ▾
Computer Vision
WBCAtt+: Fine-Grained Pixel-Level Morphological Annotations for White Blood Cell Images
Satoshi Tsutsui, Winnie Pang, Shuting He et al. (+1 more)
The microscopic examination of white blood
cells (WBCs) plays a fundamental role in pathology and is essential for diagnosing blood disorders such as leukemia and anemia. To support further research on WBC images, multiple
datasets have been proposed. However, they mainly annotate
cell categories, and lack detailed morphological characteristics that pathologists use to explain their interpretations of
cells. To address this gap, we introduce WBCAtt+, a novel
dataset of WBC images densely annotated with 11 morphological attributes and five pixel-level
cell components. With 113k image-level labels and 10k
segmentation maps, WBCAtt+ is the first to provide comprehensive annotations for WBC images. Leveraging this
dataset, we provide baseline models for attribute recognition and semantic
segmentation. We also design an attribute recognition model to incorporate compositional structure of
cells, further improving the recognition performance. Lastly, we showcase various applications enabled by our
dataset, such as explainable AI models, including counterfactual example generation. \revision{The
dataset and code are publicly available\footnote{https://doi.org/10.57967/hf/8143}}.
Read the full abstract ▾
All authors Satoshi Tsutsui, Winnie Pang, Shuting He, Bihan Wen
Computer Vision
DocQT: Improving Document Forgery Localization Robustness via Diverse JPEG Quantization Tables
Kylian Ronfleux-Corail, Guillaume Bernard, Mickaël Coustaty et al. (+1 more)
Document manipulation localization models achieve strong performance on public
benchmarks yet fail to generalize to operational document workflows. We identify a critical and overlooked source of this gap: the mismatch between the narrow distribution of JPEG quantization tables used during training -restricted to standard libjpeg quality factors -and the heterogeneous compression profiles encountered in real-world insurance document pipelines. To isolate this factor, we conduct a controlled factorial study comparing two architectures with contrasting levels of quantization table awareness -FFDN [2] and Mesorch [20] -each trained under either standard quality factor augmentation (Standard-QT ) or operationally calibrated quantization tables sampled from DocQT, a quantization-table bank derived from a MAIF operational image corpus (Real-QT ), and evaluated under three recompression conditions. Training under Real-QT yields substantial localization gains on DocTamper [15] and significantly reduces the pixel-level false positive rate on authentic operational documents, but only for architectures that explicitly ingest the quantization table as input. The released DocQT quantization-table
dataset and compression-reproduction material are directly available at https://github.com/Kyliroco/Improving-Document-Forgery-Localization-Robustness-via-Diverse-JPEG-Quantization-Tables. These results demonstrate that standard quality factor augmentation does not adequately proxy operational compression diversity, and that architectural choices explicitly conditioning on the quantization table provide a meaningful robustness advantage for real-world deployment.
Read the full abstract ▾
All authors Kylian Ronfleux-Corail, Guillaume Bernard, Mickaël Coustaty, Nicolas Sidère
Computer Vision
Cross-View Splatter: Feed-Forward View Synthesis with Georeferenced Images
Matias Turkulainen, Akshay Krishnan, Filippo Aleotti et al. (+6 more)
We present Cross-View Splatter, a feed-forward method that predicts pixel-aligned Gaussian splats for outdoor scenes captured at ground level AND by satellite. Faithful reconstructions require good camera coverage, but ground imagery is time-consuming and hard to capture at scale for large outdoor scenes. Fortunately, satellite imagery can provide a global
geometric prior that is easy to access via public APIs. Cross-View Splatter fuses orthorectified satellite views with GPS-tagged ground photos to predict Gaussian splats in a unified 3D coordinate frame. By aligning ground and bird's-eye feature representations, our model improves scene coverage and novel-view synthesis, compared to ground imagery alone. We train on curated georeferenced
datasets and paired satellite-terrain data, mined from open mapping services. We evaluate our method on a new
benchmark for novel-view synthesis with georeferenced imagery allowing comparison to prior state-of-the-art methods. Our code and data preparation will be available at https://nianticspatial.github.io/cross-view-splatter/.
Read the full abstract ▾
All authors Matias Turkulainen, Akshay Krishnan, Filippo Aleotti, Mohamed Sayed, Guillermo Garcia-Hernando, Juho Kannala, Arno Solin, Gabriel Brostow, Daniyar Turmukhambetov
Computer Vision
CAD-Free Learning of Spacecraft Pose Estimators via NeRF-Based Augmentations
Antoine Legrand, Renaud Detry and Christophe De Vleeschouwer
Spacecraft pose estimation networks require tens of thousands of CAD-rendered images to be trained. This reliance on synthetic CAD data (i) limits applicability to targets with reliable geometry prior, excluding uncooperative or poorly documented spacecraft, and (ii) causes poor generalization to real on-orbit conditions due to unrealistic illumination and material appearance. This paper introduces a NeRF-based image augmentation method that enables the learning of spacecraft pose
estimators from only a few tens to a few hundreds of images. The method learns a Neural Radiance Field of the target and generates a large, diverse
dataset through geometrically-consistent viewpoint and appearance augmentation. This augmented
dataset enables the training of accurate target-specific pose
estimators without requiring a CAD model or large synthetic
datasets. Experiments show that our approach supports the training of accurate pose
estimators from only 25 to 400 realistic images, even under severe illumination variations. When applied on large CAD-based synthetic
datasets, the NeRF-based augmentation also enhances out-of-domain generalization, yielding improved robustness to real on-orbit conditions.
Read the full abstract ▾
Computer Vision
Benchmarking and Evolving Reason-Reflect-Rectify for Reflective Visual Generation
Junjie Wang, Xinghua Lou, Jason Li et al. (+8 more)
Text-to-Image (T2I) models and Unified
Multimodal Models (UMMs) have achieved remarkable progress in visual generation. However, their reliance on a single-pass generation paradigm limits their ability to handle complex
prompts requiring iterative refinement. To enable multi-round Reflective Visual Generation (RVG), we formalize the Reason-Reflect-Rectify (R^3) loop as a core framework and introduce R^3-Bench, a
benchmark of over 600 expert-annotated instances that quantifies iterative
reasoning and rectification capabilities. Evaluation on R^3-Bench reveals a critical gap: while state-of-the-art models can identify generation errors, they fail to generate actionable rectification instructions. To bridge this gap, we propose R^3-Refiner, a dual-stage framework leveraging Group Relative Policy Optimization (GRPO) and a Hierarchical Reward Mechanism (HRM) to better align rectification with reflective
reasoning. Experiments show that R^3-Refiner achieves significant improvements on R^3-Bench (+12.0% in Reflective Verdict Score, +9.0% in Rectification Score), and can be seamlessly integrated with various MLLMs to enhance the generation quality of different T2I models on GenEval++ and T2I-CompBench. Code is available at https://github.com/xiaomoguhz/R3-Bench.
Read the full abstract ▾
All authors Junjie Wang, Xinghua Lou, Jason Li, Ye Tian, Keyu Chen, Yulin Li, Bin Kang, Jacky Mai, Yanwei Li, Zhuotao Tian, Liqiang Nie
Computer Vision
Kai Sheng, Liuyi Wang, Haojie Dai et al. (+5 more)
Vision-and-
language navigation (VLN) requires an
embodied agent to ground natural-
language instructions into executable
navigation actions in unseen environments. Existing zero-shot methods typically rely on additional waypoint prediction modules, which often entangle high-level directional
reasoning with fine-grained local grounding, leading to error-prone and unstable decisions. In this paper, we propose P2DNav, a hierarchical framework for zero-shot
vision-and-
language navigation. P2DNav consists of three core components: Panorama-to-Downview (P2D), Sliding-Window Dialogue
Memory (SDM), and Reflective Reorientation Mechanism (RRM). P2D explicitly decomposes
navigation decision-making into two stages: panoramic direction selection and downview local grounding. It first selects the instruction-relevant direction from a 360{\deg} panorama, and then predicts a pixel-level target point from the downview RGB observation in that direction. In addition, SDM organizes
navigation history as a multi-turn dialogue context and maintains recent visual observations within a sliding window to support long-horizon
navigation. RRM further enables reflective reorientation by assessing the reliability of local grounding based on the downview observation and returning to panoramic direction selection when necessary. Experiments on the R2R-CE
benchmark show that P2DNav achieves strong performance among zero-shot methods. In particular, compared with the state-of-the-art (SOTA) zero-shot waypoint-based and waypoint-free methods, P2DNav achieves SR gains of 146.6% and 58.9%, respectively, demonstrating the effectiveness of P2D, SDM, and RRM for zero-shot VLN. Code will be released for public use.
Read the full abstract ▾
All authors Kai Sheng, Liuyi Wang, Haojie Dai, Jinlong Li, Yongrui Qin, Zongtao He, Chengju Liu, Qijun Chen
Computer Vision · Cross-listed from cs.RO
HEAT: Heterogeneous End-to-End Autonomous Driving via Trajectory-Guided World Models
Hoonhee Cho, Giwon Lee, Jae-Young Kang et al. (+3 more)
End-to-end autonomous driving has emerged as a compelling alternative to traditional modular pipelines by directly mapping raw sensor data to driving actions. While recent approaches achieve strong performance on single-domain
datasets, their performance degrades significantly when trained jointly across multiple heterogeneous domains. In practice, however, autonomous systems must operate across diverse environments with heterogeneous distributions, including different cities, sensor configurations, and traffic patterns, without domain-specific retraining. This gap highlights a key challenge in multi-domain learning: domain-specific variations across heterogeneous domains introduce conflicting learning signals, driving models toward compromised solutions that are suboptimal across domains. To address this, we propose a trajectory-driven learning paradigm that organizes training around planning trajectories, enabling the model to capture domain-invariant representations of driving intent. Furthermore, we incorporate a world model that predicts future latent features conditioned on ego actions, improving feature consistency and mitigating domain-induced biases. We evaluate our approach on three
benchmarks, nuScenes, NAVSIM, and the Waymo end-to-end
dataset, and show substantial improvements over existing methods across all domains. Our results demonstrate that a single unified model can be trained on heterogeneous
datasets while maintaining strong performance within each domain, highlighting a step toward scalable real-world deployment. We will make our code publicly available.
Read the full abstract ▾
All authors Hoonhee Cho, Giwon Lee, Jae-Young Kang, Hyemin Yang, Heejun Park, Kuk-Jin Yoon
Computer Vision
Component-Aware Structure-Preserving Style Transfer for Satellite Visual Sim2Real Data Construction
Zongwu Xie, Yonglong Zhang, Yifan Yang et al. (+3 more)
For camera-based satellite visual sensing, Sim2Real data construction requires images that approach real-domain sensor appearance while retaining the annotations inherited from simulation. Real sensor images of satellite targets with reliable pose labels and component-level masks are difficult to acquire at scale, whereas synthetic rendering provides exact
geometric annotations but suffers from a visible appearance gap. This paper presents a component-aware structure-preserving style transfer framework for satellite visual synthetic-to-real data construction. The method builds weakly paired real–synthetic samples from calibrated real acquisition, ArUco-based camera-pose measurement, CAD rendering, and component masks. It then extracts part-wise real-domain style codes from unlabeled real images and injects them into corresponding synthetic satellite regions through mask-aligned modulation. To keep the generated images usable for downstream sensor-data supervision,
adversarial training is combined with local contrastive consistency, self-regularization, and edge-preserving constraints. Experiments are conducted on 5,000 rendered satellite images and 100 real images captured in a calibrated setup. The real images provide target-domain appearance references and final evaluation images, while the downstream GDRNet pose
estimator is trained only on synthetic or translated synthetic images. Compared with representative image-translation baselines, the proposed method achieves the lowest image distribution discrepancy, with an FID of 54.32 and a KID of 0.048. When the translated data are used to train GDRNet in this target-domain adaptation setting, the ADD pass rate improves to 0.260 and the AUC improves to 0.611. These results indicate that component-level appearance transfer can improve annotation-preserving satellite visual Sim2Real data generation in the considered calibrated setup.
Read the full abstract ▾
All authors Zongwu Xie, Yonglong Zhang, Yifan Yang, Yang Liu, Baoshi Cao, Guanghu Xie
Computer Vision
PrAda: Few-Shot Visual Adaptation for Text-Prompted Segmentation
Gabriele Rosi, Fabio Cermelli, Carlo Masone et al. (+1 more)
Segmenting images is critical for visual understanding but demands extensive pixel-level annotations. Foundational models have enabled new paradigms for predicting new classes guided by textual
prompts, without annotations from the target domain. Yet, on specialized target domains, far from the original pre-training, their performance degrades. We study the errors of existing methods under such domain-shift, finding that misclassification rather than mask generation is the main culprit. To address this, we introduce the novel problem of Few-Shot Visual Adaptation for text-prompted
Segmentation. This kind of adaptation has been largely studied for image classification, but it remains unexplored for
segmentation. We tackle this task with Prototype Adaptation (PrAda), a novel, parameter-efficient method that adapts a frozen text-prompted
segmentation model. Our approach learns class-specific prototypes by combining fine-grained pixel features and high-level
transformer representations, which are then fused with the original text-based predictions through a learned importance factor. This preserves the model's zero-shot potential while enabling strong adaptation to new domains. Experiments across semantic, instance, and panoptic
segmentation on five
benchmarks demonstrate that PrAda yields significant improvements over state-of-the-art and proposed baselines.
Read the full abstract ▾
All authors Gabriele Rosi, Fabio Cermelli, Carlo Masone, Barbara Caputo
Computer Vision
UniRefiner: Teaching Pre-trained ViTs to Self-Dispose Dross via Contrastive Register
Congpei Qiu, Zhaoyu Hu, Wei Ke et al. (+3 more)
Representation learning with
Vision Transformers (ViTs) has advanced rapidly, yet the utility of large-scale models in spatially sensitive tasks is hindered by spurious
tokens. Prior efforts to mitigate this have been limited, often defining these artifacts narrowly, for example, as simple high-norm outliers. We argue that this scope is insufficient. For dense prediction tasks, we posit that any
token failing to encode location-aligned semantics should be treated as a spurious artifact. This broader definition reveals a more complex problem, leading us to systematically categorize and characterize three fundamental types of spurious
tokens that corrupt spatial representations. Based on this comprehensive diagnosis, we propose UniRefiner, a universal refinement framework that teaches pre-trained ViTs to self-dispose of these artifacts. UniRefiner uses contrastive registers to explicitly isolate and redistribute spurious
tokens via a dual objective: (i) it aligns image
tokens with filtered regular
tokens to preserve semantics, and (ii) it aligns register
tokens with detected spurious
tokens to capture the spurious signals. Our method requires only a few epochs of fine-tuning on ~5k images to refine diverse ViTs, including massive models like EVA-CLIP-8B and InternViT-6B. Experiments demonstrate consistent and significant improvements: notably, the refined EVA-CLIP-8B achieves 51.9% mIoU on ADE20K (+9.4%), surpassing specialized
vision models like DINOv2 (49.1%), while zero-shot
segmentation accuracy improves by up to 22%. UniRefiner unlocks the latent spatial potential of existing large-scale foundation models, paving the way for their broader application.
Read the full abstract ▾
All authors Congpei Qiu, Zhaoyu Hu, Wei Ke, Zhuotao Tian, Yanhao Wu, Tong Zhang
Computer Vision
Bézier Degradation Modeling for LiDAR-based Human Motion Capture
Xiaoqi An, Lin Zhao, Jun Li et al. (+2 more)
LiDAR-based 3D human motion capture has broad applications in fields such as autonomous driving and robotics, where accurate motion reconstruction is crucial. However, existing methods often struggle with unstable inputs and severe occlusions, leading to jittery or even failed pose predictions. To address these challenges, we propose BMLiCap, a coarse-to-fine framework that models motion using temporally compressible Bézier curves. By reducing control points through a trajectory-preserving strategy, we obtain a coherent and learning-friendly motion representation. To reconstruct human actions from
LiDAR point-cloud cues, we design a progressive motion-reconstruction module. Specifically, a Time-scale Motion
Transformer (TMT) is introduced to predict motion curves at multiple temporal scales, and a Multi-level Motion Aggregator (MMA) is utilized to adaptively fuse the multi-scale curves to recover detailed, temporally coherent poses, effectively bridging observation gaps caused by occlusions and noise. Across four mainstream
benchmarks LiDARHuman26M, FreeMotion, NoiseMotion, and SLOPER4D, BMLiCap achieves state-of-the-art accuracy and temporal continuity in complex scenes, demonstrating its ability to compensate for severe occlusions and reduce prediction jitter.
Read the full abstract ▾
All authors Xiaoqi An, Lin Zhao, Jun Li, Chen Gong, Jian Yang
Computer Vision
White-Balance First, Adjust Later: Cross-Camera Color Constancy via Vision-Language Evaluation
Shuwei Li, Lei Tan and Robby T. Tan
Color constancy aims to keep object colors consistent under varying illumination. Cross-camera generalization in color constancy remains challenging because learning-based models often overfit to the color response characteristics of the training camera, resulting in degraded performance on images captured by other cameras. We propose VLM-CC, a feedback-guided framework that formulates color constancy as an iterative refinement process. Instead of directly estimating the illuminant from raw input, VLM-CC performs iterative correction driven by
vision-
language model (VLM)-based evaluation. At each iteration, the image is white-balanced using the current estimate and converted to pseudo-sRGB. A lightweight LoRA-tuned VLM then assesses the corrected image, identifying the dominant residual color cast and providing qualitative feedback. This feedback is mapped to a residual illumination direction (red, green, or blue) and used to update the illuminant estimate until convergence. Our key idea is to reframe color constancy as an iterative perceptual feedback problem, leveraging VLM evaluation instead of direct RGB regression. By replacing direct RGB estimation with VLM-guided perceptual feedback, VLM-CC achieves state-of-the-art robustness in cross-camera color constancy across multiple
datasets. Code will be available at https://github.com/NothingIknow/VLM-CC.
Read the full abstract ▾
Computer Vision
Physics Guided Conditional Diffusion Framework for Generative Inverse Design of Manufacturable Metasurface based Absorbers
Vineetha Joy, Jamshed Palai, Satwik Sahu et al. (+3 more)
Inverse design of metasurfaces under continuous electromagnetic constraints requires generation of geometries that simultaneously satisfy stringent spectral specifications and remain manufacturable. Conventional approaches based on iterative full wave simulations are computationally prohibitive for large design spaces, while existing generative models often suffer from poor conditional controllability and limited fabrication awareness. In this regard, we propose a physics guided condition quality enhanced
diffusion framework for the inverse design of metasurface based absorbers. Fabrication-aware constraints are incorporated to ensure practical realizability of the generated designs. The framework introduces a conditioning mechanism for continuous spectral specifications, wherein feature-wise linear modulation propagates the condition across the denoising hierarchy, enabling stable and accurate generation with improved spectral controllability. Further, to embed EM consistency directly into the generative learning process, a pre trained surrogate EM simulator is integrated within the
diffusion training pipeline. The proposed framework generated physically realizable metasurface designs for diverse reflection characteristics in the frequency range of 2 to 18 GHz, achieving a very low average spectral mean squared error of 0.0006 and a high band
alignment accuracy of 0.958. The framework also addresses the fundamentally non-unique nature of inverse EM design by enabling structured
multimodal generation of geometrically distinct yet spectrally consistent metasurface designs for the same target response. The proposed model produces the suitable design in approximately 30 seconds, whereas the conventional approach can take several months under comparable computational resources. The efficiency of the model is also established via experimental measurements.
Read the full abstract ▾
All authors Vineetha Joy, Jamshed Palai, Satwik Sahu, Anshuman Kumar, Amit Sethi, Hema Singh
Computer Vision
Spectral Integrated Gradients for Coarse-to-Fine Feature Attribution
Soyeon Kim, Seongwoo Lim, Kyowoon Lee et al. (+1 more)
Integrated Gradients (IG) is a widely adopted feature attribution method that satisfies desirable axiomatic properties. However, the choice of integration path significantly affects the quality of attributions, and the standard straight-line path introduces all input features simultaneously, often accumulating noisy gradients along the way. To address this limitation, we propose Spectral Integrated Gradients, which constructs integration paths based on singular value decomposition (SVD) of the baseline-to-input difference. By progressively activating singular components from largest to smallest, SIG introduces global structure before fine-grained details, naturally following a coarse-to-fine progression. Through extensive evaluation across diverse image classification
datasets, we demonstrate that SIG produces cleaner attribution maps with reduced noise and achieves improved quantitative performance compared to existing path-based attribution methods. Our code is available at https://github.com/leekwoon/sig/.
Read the full abstract ▾
All authors Soyeon Kim, Seongwoo Lim, Kyowoon Lee, Jaesik Choi
Computer Vision
deadtrees.earth-aerial: A Multi-Resolution Aerial Image Dataset for Tree Cover and Mortality Detection
Ayushi Sharma, Clemens Mosig, Lukas Drees et al. (+9 more)
Forests worldwide are increasingly threatened by
climate change and disturbances such as fire, pests, and pathogens, creating a need for scalable monitoring of tree cover and tree mortality. Aerial imagery from drones and aircraft enables detailed mapping of tree crowns and mortality, but progress is limited by the lack of globally distributed, multi-biome, harmonized
datasets for joint
segmentation of tree cover and mortality. To address this, we introduce two open, machine-learning-ready
datasets for joint
segmentation of tree cover and tree mortality from centimeter-scale aerial imagery across globally distributed sites. Our training
dataset, DTE-aerial-train, comprises 346K image patches of $1024 \times 1024$ pixels at spatial resolutions from 2.5 to 20\,cm, with audited pseudo-labels for tree cover and mortality and expert annotations for rare mortality sites. Our evaluation
dataset, DTE-aerial-bench, is a multi-biome, multi-resolution
benchmark of 25 globally distributed orthoimages (525 patches) with expert annotations for both tree cover and mortality. Both
datasets span tropical, temperate, boreal, and dryland biomes and cover a wide range of forest structures and mortality patterns. Using the
benchmark, we establish reference baselines for mortality and tree cover
segmentation across biomes and spatial resolutions. Improvements over prior models vary by biome: they are largest in boreal and montane forests (F1 from 0.40 to 0.58) and smaller in tropical forests and drylands. For data, model and code, please visit our \href{https://ayushi-3536.github.io/deadtrees_aerial.github.io/}{project page}. An interactive visualization of the
benchmark dataset is also available at \href{https://deadtrees.earth/releases/dte-aerial-bench}{our release page}.
Read the full abstract ▾
All authors Ayushi Sharma, Clemens Mosig, Lukas Drees, Salim Soltani, Janusch Vajna-Jehle, Aaron Sheppard, Belqis Ahmadi, Jonathan Schmid, Paul Neumeier, Nathan Jacobs, Jan Dirk Wegner, Teja Kattenborn
Computer Vision
A novel YOLO26-MoE optimized by an LLM agent for insulator fault detection considering UAV images
João Pedro Matos-Carvalho, Laio Oriel Seman, Stefano Frizzo Stefenon et al. (+2 more)
The inspection of electrical power line
insulators is essential for ensuring grid reliability and preventing failures caused by damaged or degraded insulation components. In recent years, Unmanned Aerial Vehicles (
UAVs) combined with deep learning-based
vision systems have emerged as an effective solution for automating this process. However,
insulator fault detection remains challenging due to small defect regions, heterogeneous fault patterns, complex backgrounds, and varying imaging conditions. To address these challenges, this paper proposes an optimized YOLO26-MoE, a novel object detection architecture that integrates a sparse Mixture-of-Experts (MoE) module into the high-resolution branch of the YOLO26 detector. The proposed modification enables adaptive feature refinement for subtle and diverse fault patterns while preserving the efficiency of a one-stage detection framework. Hyperparameter optimization, final training, and evaluation were coordinated through a tool-augmented Large
Language Model (LLM)
agent. The proposed model achieved 0.9900 mAP@0.5 and 0.9515 mAP@0.5:0.95, outperforming the latest YOLO versions. These results demonstrate that the proposed model provides an effective and reliable solution for
UAV-based
insulator fault detection.
Read the full abstract ▾
All authors João Pedro Matos-Carvalho, Laio Oriel Seman, Stefano Frizzo Stefenon, Mohammad Khalaf Mohammad Khreasat, Gabriel Villarrubia González
Computer Vision
Lens Privacy Sealing: A New Benchmark and Method for Physical Privacy-Preserving Action Recognition
Mengyuan Liu, Ziyi Wang, Peiming Li et al. (+1 more)
RGB camera-based surveillance systems enable human action recognition for public safety and healthcare, yet raise serious
privacy concerns. Existing methods rely on post-capture
algorithms, which fail to protect
privacy during data acquisition. We propose Lens
Privacy Sealing (LPS), a simple hardware solution that physically obscures camera lenses with adjustable laminating film, providing pre-sensor
privacy protection at minimal cost. Unlike software methods or expensive engineered optics, LPS achieves strong
privacy through
stochastic multi-layer scattering that is physically irreversible. We introduce the P$^3$AR
dataset for
privacy-preserving action recognition, featuring both large-scale replay-captured (P$^3$AR-NTU, 114K
videos) and real-world collected (P$^3$AR-PKU) subsets with
privacy attribute annotations. To handle
video degradation from LPS, we propose MSPNet, a single-stage framework incorporating Inter-Frame Noise Suppressor (IFNS) and Cross-Frame Semantic Aggregator (CFSA), enhanced by contrastive
language-image pre-training for robust semantic extraction. Extensive experiments demonstrate that MSPNet with IFNS and CFSA nearly doubles action recognition accuracy compared to baseline methods while suppressing identity recognition to low levels. Comprehensive validation shows LPS achieves a superior
privacy-utility trade-off compared to state-of-the-art hardware methods, resists reconstruction attacks including PSF inversion and data-driven recovery, and generalizes robustly across optical configurations and challenging environments. Code is available at https://github.com/wangzy01/MSPNet.
Read the full abstract ▾
All authors Mengyuan Liu, Ziyi Wang, Peiming Li, Junsong Yuan
Computer Vision
EgoCoT-Bench: Benchmarking Grounded and Verifiable Operation-Centric Chain of Thought Reasoning for MLLMs
Yang Dai, Dian Jiao, Tianwei Lin et al. (+1 more)
The rapid development of
Multimodal Large
Language Models (MLLMs) has led to growing interest in
egocentric video understanding, specifically the ability for MLLMs to recognize fine-grained hand-object interactions, track object state changes over time, and reason about manipulative processes in dynamic environments from a first-person perspective. However, existing
egocentric video benchmarks suffer from limited grounded rationale evaluation, offering limited support for fine-grained operation-centric
reasoning and rarely examining whether model rationales are grounded in explicit spatio-temporal evidence. To address this gap, we introduce EgoCoT-Bench, a fine-grained
egocentric benchmark for grounded and verifiable operation-centric
reasoning with explicit step-by-step rationale annotations. Overall, EgoCoT-Bench comprises 3,172 verifiable QA pairs over 351
egocentric videos separated into four task groups for a total of 12 sub-task groups, encompassing perception and retrospection, anticipation, and high-level
reasoning. The
benchmark is constructed through a spatio-temporal scene graphs (STSG) guided generation framework and is further refined by human annotators to ensure correctness,
egocentric relevance and fine-grained quality. Experimental results show continuing difficulties with
egocentric fine-grained
reasoning and further reveal that many
multimodal models produce explanations that are answer-correct, but have evidence that is inconsistent with the answer. We hope EgoCoT-Bench can serve as a useful testbed for grounded and verifiable
reasoning in
egocentric video understanding. Project page and supplementary materials are available at: https://dstardust.github.io/EgoCoT/.
Read the full abstract ▾
All authors Yang Dai, Dian Jiao, Tianwei Lin, Wenqiao Zhang
Computer Vision
EpiDiffVO: Geometry-Aware Epipolar Diffusion for Robust Visual Odometry
Prateeth Rao
Estimating relative pose from image pairs fundamentally requires only a minimal subset of geometrically consistent correspondences. However, most learning-based approaches rely on dense matching or direct regression, leading to redundancy and reduced
geometric interpretability. In this work, we propose a sparse epipolar matching framework that predicts a compact set of correspondences optimized for
geometric consistency across varying temporal baselines. To address residual noise and misalignment, we introduce an epipolar
diffusion process that models correspondence uncertainty and refines keypoints toward epipolar consistency. The refined correspondences, along with depth cues, are lifted into a graph representation forming a Steiner graph that encodes relational structure between points. A graph neural network learns a compact subset of informative correspondences, which are passed to a differentiable singular value decomposition solver for end-to-end
geometric estimation. Relative pose is recovered from the resulting essential
matrix and evaluated in a visual odometry setting on the TartanAir and KITTI SLAM
datasets. Experimental results demonstrate that combining sparse matching,
diffusion-based refinement, and graph-based subset selection reduces correspondence redundancy while maintaining robust pose estimation across challenging baselines.
Read the full abstract ▾