Computer Vision · Cross-listed from cs.RO
PlanTRansformer: Unified Prediction and Planning with Goal-conditioned Transformer
Constantin Selzer and Fabina B. Flohr
Trajectory prediction and planning are fundamental yet disconnected components in autonomous driving. Prediction models forecast surrounding
agent motion under unknown intentions, producing
multimodal distributions, while planning assumes known ego objectives and generates deterministic trajectories. This mismatch creates a critical bottleneck: prediction lacks supervision for
agent intentions, while planning requires this information. Existing prediction models, despite strong benchmarking performance, often remain disconnected from planning constraints such as collision avoidance and dynamic feasibility. We introduce Plan
TRansformer (PTR), a unified Gaussian Mixture
Transformer framework integrating goal-conditioned prediction, dynamic feasibility, interaction awareness, and lane-level
topology reasoning. A teacher-student training strategy progressively masks surrounding
agent commands during training to align with inference conditions where
agent intentions are unavailable. PTR achieves 4.3%/3.5% improvement in marginal/joint mAP compared to the baseline Motion
Transformer (MTR) and 15.5% planning error reduction at 5s horizon compared to GameFormer. The architecture-agnostic design enables application to diverse
Transformer-based prediction models. Project Website: https://github.com/SelzerConst/PlanTRansformer
Read the full abstract ▾
Computer Vision
Unifying Watermarking via Dimension-Aware Mapping
Jiale Meng, Runyi Hu, Jie Zhang et al. (+3 more)
Deep watermarking methods often share similar encoder-decoder architectures, yet differ substantially in their functional behaviors. We propose DiM, a new multi-dimensional watermarking framework that formulates watermarking as a dimension-aware mapping problem, thereby unifying existing watermarking methods at the functional level. Under DiM, watermark information is modeled as payloads of different dimensionalities, including one-dimensional binary messages, two-dimensional spatial masks, and three-dimensional spatiotemporal structures. We find that the dimensional configuration of
embedding and extraction largely determines the resulting watermarking behavior. Same-dimensional mappings preserve payload structure and support fine-grained control, while cross-dimensional mappings enable spatial or spatiotemporal localization. We instantiate DiM in the
video domain, where spatiotemporal representations enable a broader set of dimension mappings. Experiments demonstrate that varying only the
embedding and extraction dimensions, without architectural changes, leads to different watermarking capabilities, including spatiotemporal tamper localization, local
embedding control, and recovery of temporal order under frame disruptions.
Read the full abstract ▾
All authors Jiale Meng, Runyi Hu, Jie Zhang, Zheming Lu, Ivor Tsang, Tianwei Zhang
Computer Vision
SLIM-Diff: Shared Latent Image-Mask Diffusion with Lp loss for Data-Scarce Epilepsy FLAIR MRI
Mario Pascual-González, Ariadna Jiménez-Partinen, R.M. Luque-Baena et al. (+2 more)
Focal cortical dysplasia (FCD) lesions in epilepsy FLAIR MRI are subtle and scarce, making joint image–mask generative modeling prone to instability and memorization. We propose SLIM-Diff, a compact joint
diffusion model whose main contributions are (i) a single shared-bottleneck U-Net that enforces tight coupling between anatomy and lesion geometry from a 2-channel image+mask representation, and (ii) loss-geometry tuning via a tunable $L_p$ objective. As an internal baseline, we include the canonical DDPM-style objective ($\epsilon$-prediction with $L_2$ loss) and isolate the effect of prediction parameterization and $L_p$ geometry under a matched setup. Experiments show that $x_0$-prediction is consistently the strongest choice for joint synthesis, and that fractional sub-quadratic penalties ($L_{1.5}$) improve image fidelity while $L_2$ better preserves lesion mask morphology. Our code and model weights are available in https://github.com/MarioPasc/slim-diff
Read the full abstract ▾
All authors Mario Pascual-González, Ariadna Jiménez-Partinen, R.M. Luque-Baena, Fátima Nagib-Raya, Ezequiel López-Rubio
Computer Vision
Multi-Resolution Alignment for Voxel Sparsity in Camera-Based 3D Semantic Scene Completion
Zhiwen Yang and Yuxin Peng
Camera-based 3D semantic scene completion (SSC) offers a cost-effective solution for assessing the
geometric occupancy and semantic labels of each voxel in the surrounding 3D scene with image inputs, providing a voxel-level scene perception foundation for the perception-prediction-planning autonomous driving systems. Although significant progress has been made in existing methods, their optimization rely solely on the supervision from voxel labels and face the challenge of voxel sparsity as a large portion of voxels in autonomous driving scenarios are empty, which limits both optimization efficiency and model performance. To address this issue, we propose a Multi-Resolution
Alignment (MRA) approach to mitigate voxel sparsity in camera-based 3D semantic scene completion, which exploits the scene and instance level
alignment across multi-resolution 3D features as auxiliary supervision. Specifically, we first propose the Multi-resolution View
Transformer module, which projects 2D image features into multi-resolution 3D features and aligns them at the scene level through fusing discriminative seed features. Furthermore, we design the Cubic Semantic Anisotropy module to identify the instance-level semantic significance of each voxel, accounting for the semantic differences of a specific voxel against its neighboring voxels within a cubic area. Finally, we devise a Critical Distribution
Alignment module, which selects critical voxels as instance-level anchors with the guidance of cubic semantic anisotropy, and applies a circulated loss for auxiliary supervision on the critical feature distribution consistency across different resolutions. The code is available at https://github.com/PKU-ICST-MIPL/MRA_TIP.
Read the full abstract ▾
Computer Vision
GryphOne: Symbol-Aware Masked Diffusion for Structural Refinement in Offline Handwritten Mathematical Expression Recognition
Takaya Kawakatsu and Ryo Ishiyama
Handwritten mathematical expression recognition (HMER) requires
reasoning over diverse symbols and structures, yet autoregressive models struggle with exposure bias and syntax inconsistency. We present GryphOne, a discrete
diffusion framework which reformulates HMER as iterative symbolic refinement instead of sequential generation. GryphOne progressively refines symbols and relations, removing autoregression and improving consistency. Symbol-aware tokenization and random-masking mutual learning further enhance robustness to handwriting diversity. On the MathWriting
benchmark, GryphOne achieves 5.51% CER and 59.9% EM (ExpRate), outperforming all reimplemented models in the matched setting as well as the commercial HMER system. Held-out evaluation on CROHME 2014-2023 further shows strong cross-
dataset generalization.
Read the full abstract ▾
Computer Vision
Z3D: Zero-Shot 3D Visual Grounding from Images
Nikita Drozdov, Andrey Lemeshko, Nikita Gavrilov et al. (+3 more)
3D visual grounding (3DVG) aims to localize objects in a 3D scene based on natural
language queries. In this work, we explore zero-shot 3DVG from multi-view images alone, without requiring any
geometric supervision or object priors. We introduce Z3D, a universal grounding pipeline that flexibly operates on multi-view images while optionally incorporating camera poses and depth maps. We identify key bottlenecks in prior zero-shot methods causing significant performance degradation and address them with (i) a state-of-the-art zero-shot 3D instance
segmentation method to generate high-quality 3D bounding box proposals and (ii) advanced
reasoning via
prompt-based
segmentation, which utilizes full capabilities of modern VLMs. Extensive experiments on the ScanRefer and Nr3D
benchmarks demonstrate that our approach achieves state-of-the-art performance among zero-shot methods. Code is available at https://github.com/col14m/z3d .
Read the full abstract ▾
All authors Nikita Drozdov, Andrey Lemeshko, Nikita Gavrilov, Anton Konushin, Danila Rukhovich, Maksim Kolodiazhnyi
Computer Vision
Tiled Prompts: Overcoming Prompt Misguidance in Image and Video Super-Resolution
Bryan Sangwoo Kim, Jonghyun Park and Jong Chul Ye
Text-conditioned
diffusion models have advanced image and
video super-resolution by using
prompts as semantic priors, and modern super-resolution pipelines typically rely on latent tiling to scale to high resolutions. In practice, a single global caption is used with the latent tiling, often causing
prompt misguidance. Specifically, a coarse global
prompt often misses localized details (errors of omission) and provides locally irrelevant guidance (errors of commission) which leads to substandard results at the tile level. To solve this, we propose Tiled
Prompts, a unified framework for image and
video super-resolution that generates a tile-specific
prompt for each latent tile and performs super-resolution under locally text-conditioned posteriors to resolve
prompt misguidance with minimal overhead. Our experiments on high resolution real-world images and
videos show that tiled
prompts bring consistent gains in perceptual quality and fidelity, while reducing
hallucinations and tile-level artifacts that can be found in global-
prompt baselines. Project Page: https://bryanswkim.github.io/tiled-
prompts/.
Read the full abstract ▾
Computer Vision
Composable Visual Tokenizers with Generator-Free Diagnostics of Learnability
Bingchen Zhao, Qiushan Guo, Ye Wang et al. (+3 more)
We introduce CompTok, a training framework for learning visual tokenizers whose
tokens are enhanced for compositionality. CompTok uses a
token-conditioned
diffusion decoder. By employing an InfoGAN-style objective, where we train a recognition model to predict the
tokens used to condition the
diffusion decoder using the decoded images, we enforce the decoder to not ignore any of the
tokens. To promote compositional control, besides the original images, CompTok also trains on
tokens formed by swapping
token subsets between images, enabling more compositional control of the
token over the decoder. As the swapped
tokens between images do not have ground truth image targets, we apply a
manifold constraint via an
adversarial flow regularizer to keep unpaired swap generations on the natural-image distribution. The resulting tokenizer not only achieves state-of-the-art performance on image class-conditioned generation, but also demonstrates properties such as swapping
tokens between images to achieve high level semantic editing of an image. Additionally, we propose two metrics that measures the landscape of the
token space that can be useful to describe not only the compositionality of the
tokens, but also how easy to learn the landscape is for a generator to be trained on this space. We show in experiments that CompTok can improve on both of the metrics as well as supporting state-of-the-art generators for class conditioned generation.
Read the full abstract ▾
All authors Bingchen Zhao, Qiushan Guo, Ye Wang, Yixuan Huang, Zhonghua Zhai, Yu Tian
Computer Vision
PWAVEP: Purifying Imperceptible Adversarial Perturbations in 3D Point Clouds via Spectral Graph Wavelets
Haoran Li, Renyang Liu, Hongjia Liu et al. (+3 more)
Recent progress in
adversarial attacks on 3D point clouds, particularly in achieving spatial imperceptibility and high attack performance, presents significant challenges for defenders. Current defensive approaches remain cumbersome, often requiring invasive model modifications, expensive training procedures or auxiliary data access. To address these threats, in this paper, we propose a plug-and-play and non-invasive defense mechanism in the spectral domain, grounded in a theoretical and empirical analysis of the relationship between imperceptible perturbations and high-frequency spectral components. Building upon these insights, we introduce a novel purification framework, termed PWAVEP, which begins by computing a spectral graph wavelet domain saliency score and local sparsity score for each point. Guided by these values, PWAVEP adopts a hierarchical strategy, it eliminates the most salient points, which are identified as hardly recoverable
adversarial outliers. Simultaneously, it applies a spectral filtering process to a broader set of moderately salient points. This process leverages a graph wavelet transform to attenuate high-frequency coefficients associated with the targeted points, thereby effectively suppressing
adversarial noise. Extensive evaluations demonstrate that the proposed PWAVEP achieves superior accuracy and robustness compared to existing approaches, advancing the state-of-the-art in 3D point cloud purification. Code and
datasets are available at https://github.com/a772316182/pwavep
Read the full abstract ▾
All authors Haoran Li, Renyang Liu, Hongjia Liu, Chen Wang, Long Yin, Jian Xu
Computer Vision · Cross-listed from cs.GR
Pi-GS: Sparse-View Gaussian Splatting with Dense {\pi}^3 Initialization
Manuel Hofer, Markus Steinberger and Thomas Köhler
Novel view synthesis has evolved rapidly, advancing from Neural Radiance Fields to 3D Gaussian Splatting (3DGS), which offers real-time rendering and rapid training without compromising visual fidelity. However, 3DGS relies heavily on accurate camera poses and high-quality point cloud initialization, which are difficult to obtain in sparse-view scenarios. While traditional Structure from Motion (SfM) pipelines often fail in these settings, existing learning-based point estimation alternatives typically require reliable reference views and remain sensitive to pose or depth errors. In this work, we propose a robust method utilizing {\pi}^3, a reference-free point cloud estimation network. We integrate dense initialization from {\pi}^3 with a regularization scheme designed to mitigate
geometric inaccuracies. Specifically, we employ uncertainty-guided depth supervision, normal consistency loss, and depth warping. Experimental results demonstrate that our approach achieves state-of-the-art performance on the Tanks and Temples, LLFF, DTU, and MipNeRF360
datasets.
Read the full abstract ▾
Computer Vision
MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learning
Shengyuan Liu, Liuxin Bao, Qi Yang et al. (+6 more)
Medical image
segmentation is evolving from task-specific models toward generalizable frameworks. Recent research leverages Multi-modal Large
Language Models (MLLMs) as autonomous
agents, employing reinforcement learning with verifiable reward (RLVR) to orchestrate specialized tools like the Segment Anything Model (SAM). However, these approaches often rely on single-turn, rigid interaction strategies and lack process-level supervision during training, which hinders their ability to fully exploit the dynamic potential of interactive tools and leads to redundant actions. To bridge this gap, we propose MedSAM-
Agent, a framework that reformulates interactive
segmentation as a multi-step autonomous decision-making process. First, we introduce a hybrid prompting strategy for expert-curated trajectory generation, enabling the model to internalize human-like decision heuristics and adaptive refinement strategies. Furthermore, we develop a two-stage training pipeline that integrates multi-turn, end-to-end outcome verification with a clinical-fidelity process reward design to promote interaction parsimony and decision efficiency. Extensive experiments across 6 medical modalities and 21
datasets demonstrate that MedSAM-
Agent achieves state-of-the-art performance, effectively unifying autonomous medical
reasoning with robust, iterative optimization. Code is available \href{https://github.com/CUHK-AIM-Group/MedSAM-
Agent}{here}.
Read the full abstract ▾
All authors Shengyuan Liu, Liuxin Bao, Qi Yang, Wanting Geng, Boyun Zheng, Chenxin Li, Wenting Chen, Houwen Peng, Yixuan Yuan
Computer Vision
Invisible Clean-Label Backdoor Attacks for Generative Data Augmentation
Ting Xiang, Jinhui Zhao, Changjian Chen et al. (+1 more)
With the rapid advancement of image generative models, generative data augmentation has become an effective way to enrich training images, especially when only small-scale
datasets are available. At the same time, in practical applications, generative data augmentation can be vulnerable to clean-label
backdoor attacks, which aim to bypass human inspection. However, based on theoretical analysis and preliminary experiments, we observe that directly applying existing pixel-level clean-label
backdoor attack methods (e.g., COMBAT) to generated images results in low attack success rates. This motivates us to move beyond pixel-level triggers and focus instead on the latent feature level. To this end, we propose InvLBA, an invisible clean-label
backdoor attack method for generative data augmentation by latent perturbation. We theoretically prove that the generalization of the clean accuracy and attack success rates of InvLBA can be guaranteed. Experiments on multiple
datasets show that our method improves the attack success rate by 46.43% on average, with almost no reduction in clean accuracy and high robustness against SOTA defense methods.
Read the full abstract ▾
All authors Ting Xiang, Jinhui Zhao, Changjian Chen, Zhuo Tang
Computer Vision
PQTNet: Pixel-wise Quantitative Thermography Neural Network for Estimating Defect Depth in Polylactic Acid Parts by Additive Manufacturing
Lei Deng, Wenhao Huang, Chao Yang et al. (+3 more)
Defect depth quantification in additively manufactured (AM) components remains a significant challenge for non-destructive testing (NDT). This study proposes a Pixel-wise Quantitative Thermography Neural Network (PQT-Net) to address this challenge for polylactic acid (PLA) parts. A key innovation is a novel data augmentation strategy that reconstructs thermal sequence data into two-dimensional stripe images, preserving the complete temporal
evolution of heat
diffusion for each pixel. The PQT-Net architecture incorporates a pre-trained EfficientNetV2-S backbone and a custom Residual Regression Head (RRH) with learnable parameters to refine outputs. Comparative experiments demonstrate the superiority of PQT-Net over other deep learning models, achieving a minimum Mean Absolute Error (MAE) of 0.0094 mm and a coefficient of determination (R) exceeding 99%. The high precision of PQT-Net underscores its potential for robust quantitative defect characterization in AM.
Read the full abstract ▾
All authors Lei Deng, Wenhao Huang, Chao Yang, Haoyuan Zheng, Yinbin Tian, Yue Ma
Computer Vision · Cross-listed from cs.RO
RDT2: Exploring the Scaling Limit of UMI Data Towards Zero-Shot Cross-Embodiment Generalization
Songming Liu, Bangguo Li, Kai Ma et al. (+5 more)
Vision-
Language-Action (VLA) models hold promise for generalist robotics but currently struggle with data scarcity, architectural inefficiencies, and the inability to generalize across different hardware platforms. We introduce RDT2, a robotic foundation model built upon a 7B parameter VLM designed to enable zero-shot deployment on novel embodiments for open-vocabulary tasks. To achieve this, we collected one of the largest open-source robotic
datasets–over 10,000 hours of demonstrations in diverse families–using an enhanced, embodiment-agnostic Universal Manipulation Interface (UMI). Our approach employs a novel three-stage training recipe that aligns discrete linguistic knowledge with continuous control via Residual Vector Quantization (RVQ), flow-matching, and
distillation for real-time inference. Consequently, RDT2 becomes one of the first models that simultaneously zero-shot generalizes to unseen objects, scenes, instructions, and even robotic platforms. Besides, it outperforms state-of-the-art baselines in dexterous, long-horizon, and dynamic downstream tasks like playing table tennis. See https://rdt-robotics.github.io/rdt2/ for more information.
Read the full abstract ▾
All authors Songming Liu, Bangguo Li, Kai Ma, Lingxuan Wu, Hengkai Tan, Xiao Ouyang, Hang Su, Jun Zhu
Computer Vision
Full end-to-end diagnostic workflow automation of 3D OCT via foundation model-driven AI for retinal diseases
Jinze Zhang, Jian Zhong, Li Lin et al. (+18 more)
Optical coherence
tomography (OCT) has revolutionized retinal disease diagnosis with its high-resolution and three-dimensional imaging nature, yet its full diagnostic automation in clinical practices remains constrained by multi-stage workflows and conventional single-slice single-task AI models. We present Full-process OCT-based Clinical Utility System (FOCUS), a foundation model-driven framework enabling end-to-end automation of 3D OCT retinal disease diagnosis. FOCUS sequentially performs image quality assessment with EfficientNetV2-S, followed by abnormality detection and multi-disease classification using a fine-tuned
Vision Foundation Model. Crucially, FOCUS leverages a unified adaptive aggregation method to intelligently integrate 2D slices-level predictions into comprehensive 3D patient-level diagnosis. Trained and tested on 3,300 patients (40,672 slices), and externally validated on 1,345 patients (18,498 slices) across four different-tier centers and diverse OCT devices, FOCUS achieved high F1 scores for quality assessment (99.01%), abnormally detection (97.46%), and patient-level diagnosis (94.39%). Real-world validation across centers also showed stable performance (F1: 90.22%-95.24%). In human-machine comparisons, FOCUS matched expert performance in abnormality detection (F1: 95.47% vs 90.91%) and multi-disease diagnosis (F1: 93.49% vs 91.35%), while demonstrating better efficiency. FOCUS automates the image-to-diagnosis pipeline, representing a critical advance towards unmanned ophthalmology with a validated blueprint for autonomous screening to enhance population scale retinal care accessibility and efficiency.
Read the full abstract ▾
All authors Jinze Zhang, Jian Zhong, Li Lin, Jiaxiong Li, Ke Ma, Naiyang Li, Meng Li, Yuan Pan, Zeyu Meng, Mengyun Zhou, Shang Huang, Shilong Yu, Zhengyu Duan, Sutong Li, Honghui Xia, Juping Liu, Dan Liang, Yantao Wei, Xiaoying Tang, Jin Yuan, Peng Xiao
Computer Vision · Cross-listed from cs.LG
R1-SyntheticVL: Is Synthetic Data from Generative Models Ready for Multimodal Large Language Model?
Jingyi Zhang, Tianyi Lin, Huanjin Yao et al. (+3 more)
In this work, we aim to develop effective data synthesis techniques that autonomously synthesize
multimodal training data for enhancing MLLMs in solving complex real-world tasks. To this end, we propose Collective
Adversarial Data Synthesis (CADS), a novel and general approach to synthesize high-quality, diverse and challenging
multimodal data for MLLMs. The core idea of CADS is to leverage collective intelligence to ensure high-quality and diverse generation, while exploring
adversarial learning to synthesize challenging samples for effectively driving model improvement. Specifically, CADS operates with two cyclic phases, i.e., Collective
Adversarial Data Generation (CAD-Generate) and Collective
Adversarial Data Judgment (CAD-Judge). CAD-Generate leverages collective knowledge to jointly generate new and diverse
multimodal data, while CAD-Judge collaboratively assesses the quality of synthesized data. In addition, CADS introduces an
Adversarial Context Optimization mechanism to optimize the generation context to encourage challenging and high-value data generation. With CADS, we construct MMSynthetic-20K and train our model R1-SyntheticVL, which demonstrates superior performance on various
benchmarks.
Read the full abstract ▾
All authors Jingyi Zhang, Tianyi Lin, Huanjin Yao, Xiang Lan, Shunyu Liu, Jiaxing Huang
Computer Vision · Cross-listed from cs.CL
POP: Prefill-Only Pruning for Efficient Large Model Inference
Junhui He, Zhihui Fu, Jun Wang et al. (+1 more)
Large
Language Models (LLMs) and
Vision-
Language Models (VLMs) have demonstrated remarkable capabilities. However, their deployment is hindered by significant computational costs. Existing structured
pruning methods, while hardware-efficient, often suffer from significant accuracy degradation. In this paper, we argue that this failure stems from a stage-agnostic
pruning approach that overlooks the asymmetric roles between the prefill and decode stages. By introducing a virtual gate mechanism, our importance analysis reveals that deep layers are critical for next-
token prediction (decode) but largely redundant for context encoding (prefill). Leveraging this insight, we propose Prefill-Only
Pruning (POP), a stage-aware inference strategy that safely omits deep layers during the computationally intensive prefill stage while retaining the full model for the sensitive decode stage. To enable the transition between stages, we introduce independent Key-Value (KV) projections to maintain cache integrity, and a boundary handling strategy to ensure the accuracy of the first generated
token. Extensive experiments on Llama-3.1, Qwen3-VL, and Gemma-3 across diverse modalities demonstrate that POP achieves up to 1.37$\times$ speedup in prefill latency with minimal performance loss, effectively overcoming the accuracy-efficiency trade-off limitations of existing structured
pruning methods.
Read the full abstract ▾
All authors Junhui He, Zhihui Fu, Jun Wang, Qingan Li
Computer Vision
LEVIO: Lightweight Embedded Visual Inertial Odometry for Resource-Constrained Devices
Jonas Kühne, Christian Vogt, Michele Magno et al. (+1 more)
Accurate, infrastructure-less sensor systems for motion tracking are essential for mobile robotics and augmented reality (AR) applications. The most popular state-of-the-art visual-inertial odometry (VIO) systems, however, are too computationally demanding for resource-constrained hardware, such as micro-drones and smart glasses. This work presents LEVIO, a fully featured VIO pipeline optimized for ultra-low-power compute platforms, allowing six-degrees-of-freedom (DoF) real-time sensing. LEVIO incorporates established VIO components such as Oriented FAST and Rotated BRIEF (ORB) feature tracking and bundle adjustment, while emphasizing a computationally efficient architecture with parallelization and low
memory usage to suit embedded microcontrollers and low-power systems-on-chip (SoCs). The paper proposes and details the algorithmic design choices and the hardware-software co-optimization approach, and presents real-time performance on resource-constrained hardware. LEVIO is validated on a parallel-processing ultra-low-power RISC-V SoC, achieving 20 FPS while consuming less than 100 mW, and benchmarked against public VIO
datasets, offering a compelling balance between efficiency and accuracy. To facilitate reproducibility and adoption, the complete implementation is released as open-source.
Read the full abstract ▾
All authors Jonas Kühne, Christian Vogt, Michele Magno, Luca Benini
Computer Vision
A3-TTA: Adaptive Anchor Alignment Test-Time Adaptation for Image Segmentation
Jianghao Wu, Xiangde Luo, Yubo Zhou et al. (+3 more)
Test-Time Adaptation (TTA) offers a practical solution for deploying image
segmentation models under domain shift without accessing source data or retraining. Among existing TTA strategies, pseudo-label-based methods have shown promising performance. However, they often rely on perturbation-ensemble heuristics (e.g., dropout sampling, test-time augmentation, Gaussian noise), which lack distributional grounding and yield unstable training signals. This can trigger error accumulation and catastrophic forgetting during adaptation. To address this, we propose A3-TTA, a TTA framework that constructs reliable pseudo-labels through anchor-guided supervision. Specifically, we identify well-predicted target domain images using a class compact density metric, under the assumption that confident predictions imply distributional proximity to the source domain. These anchors serve as stable references to guide pseudo-label generation, which is further regularized via semantic consistency and boundary-aware
entropy minimization. Additionally, we introduce a self-adaptive exponential moving average strategy to mitigate label noise and stabilize model update during adaptation. Evaluated on both multi-domain medical images (heart structure and prostate
segmentation) and natural images, A3-TTA significantly improves average Dice scores by 10.40 to 17.68 percentage points compared to the source model, outperforming several state-of-the-art TTA methods under different
segmentation model architectures. A3-TTA also excels in continual TTA, maintaining high performance across sequential target domains with strong anti-forgetting ability. The code will be made publicly available at https://github.com/HiLab-git/A3-TTA.
Read the full abstract ▾
All authors Jianghao Wu, Xiangde Luo, Yubo Zhou, Lianming Wu, Guotai Wang, Shaoting Zhang
Computer Vision · Cross-listed from cs.CR
Time Is All It Takes: Spike-Retiming Attacks on Event-Driven Spiking Neural Networks
Yi Yu, Qixin Zhang, Shuhan Ye et al. (+6 more)
Spiking neural networks (SNNs) compute with discrete spikes and exploit temporal structure, yet most
adversarial attacks change intensities or event counts instead of timing. We study a timing-only adversary that retimes existing spikes while preserving spike counts and amplitudes in event-driven SNNs, thus remaining rate-preserving. We formalize a capacity-1 spike-retiming threat model with a unified trio of budgets: per-spike jitter $\mathcal{B}_{\infty}$, total delay $\mathcal{B}_{1}$, and tamper count $\mathcal{B}_{0}$. Feasible
adversarial examples must satisfy timeline consistency and non-overlap, which makes the search space discrete and constrained. To optimize such retimings at scale, we use projected-in-the-loop (PIL) optimization: shift-probability logits yield a differentiable soft retiming for backpropagation, and a strict projection in the forward pass produces a feasible discrete schedule that satisfies capacity-1, non-overlap, and the chosen budget at every step. The objective maximizes task loss on the projected input and adds a capacity regularizer together with budget-aware penalties, which stabilizes gradients and aligns optimization with evaluation. Across event-driven
benchmarks (CIFAR10-DVS, DVS-Gesture, N-MNIST) and diverse SNN architectures, we evaluate under binary and integer event grids and a range of retiming budgets, and also test models trained with timing-aware
adversarial training designed to counter timing-only attacks. For example, on DVS-Gesture the attack attains high success (over $90\%$) while touching fewer than $2\%$ of spikes under $\mathcal{B}_{0}$. Taken together, our results show that spike retiming is a practical and stealthy attack surface that current defenses struggle to counter, providing a clear reference for temporal robustness in event-driven SNNs. Code is available at https://github.com/yuyi-sd/Spike-Retiming-Attacks.
Read the full abstract ▾
All authors Yi Yu, Qixin Zhang, Shuhan Ye, Xun Lin, Qianshan Wei, Kun Wang, Wenhan Yang, Dacheng Tao, Xudong Jiang
Computer Vision
Global Geometry Is Not Enough for Vision Representations
Jiwan Chung and Seon Joo Kim
A common assumption in representation learning is that globally well-distributed
embeddings support robust and generalizable representations. This focus has shaped both training objectives and evaluation protocols, implicitly treating global geometry as a proxy for representational competence. While global geometry effectively encodes which elements are present, it is often insensitive to how they are composed. We investigate this limitation by testing the ability of
geometric metrics to predict compositional binding across a diverse suite of
vision encoders. We find that standard geometry-based statistics exhibit near-zero correlation with compositional binding. In contrast, functional sensitivity, as measured by the input–output Jacobian, reliably tracks this capability. We further provide an analytic account showing that this disparity arises from objective design, as existing losses explicitly constrain
embedding geometry but leave the local input–output mapping unconstrained. These results suggest that global
embedding geometry captures only a partial view of representational competence and establish functional sensitivity as a critical complementary axis for modeling composite structure.
Read the full abstract ▾
Computer Vision
HypCBC: Domain-Invariant Hyperbolic Cross-Branch Consistency for Generalizable Medical Image Analysis
Francesco Di Salvo, Sebastian Doerrich, Jonas Alle et al. (+1 more)
Robust generalization beyond training distributions remains a critical challenge for deep neural networks. This is especially pronounced in medical image analysis, where data is often scarce and
covariate shifts arise from different hardware devices, imaging protocols, and heterogeneous patient populations. These factors collectively hinder reliable performance and slow down clinical adoption. Despite recent progress, existing learning paradigms primarily rely on the Euclidean
manifold, whose flat geometry fails to capture the complex, hierarchical structures present in clinical data. In this work, we exploit the advantages of
hyperbolic manifolds to model complex data characteristics. We present the first comprehensive validation of
hyperbolic representation learning for medical image analysis and demonstrate statistically significant gains across eleven in-distribution
datasets and three ViT models. We further propose an unsupervised, domain-invariant
hyperbolic cross-branch consistency constraint. Extensive experiments confirm that our proposed method promotes domain-invariant features and outperforms state-of-the-art Euclidean methods by an average of $+2.1\%$ AUC on three domain generalization
benchmarks: Fitzpatrick17k, Camelyon17-WILDS, and a cross-
dataset setup for retinal imaging. These
datasets span different imaging modalities, data sizes, and label granularities, confirming generalization capabilities across substantially different conditions. The code is available at https://github.com/francescodisalvo05/
hyperbolic-cross-branch-consistency .
Read the full abstract ▾
All authors Francesco Di Salvo, Sebastian Doerrich, Jonas Alle, Christian Ledig
Computer Vision
LaVPR: Benchmarking Language and Vision for Place Recognition
Ofer Idan, Dan Badur, Yosi Keller et al. (+1 more)
Visual Place Recognition (VPR) often fails under extreme environmental changes and perceptual aliasing. Beyond these limitations, standard systems cannot perform 'blind' localization from verbal descriptions alone, a capability critical for applications such as emergency response. To address these challenges, we introduce LaVPR, a large-scale
benchmark that extends existing VPR
datasets with over 650,000 rich natural-
language descriptions. Using LaVPR, we investigate two paradigms: Multi-Modal Fusion for enhanced robustness and Cross-Modal
Retrieval for
language-based localization. Our results show that
language descriptions yield consistent gains in visually degraded conditions, with the most significant impact on smaller backbones. Notably, adding
language allows compact models to rival the performance of much larger
vision-only architectures. For cross-modal
retrieval, we establish a baseline using Low-Rank Adaptation (LoRA) and Multi-Similarity loss, which substantially outperforms standard contrastive methods across
vision-
language models. Ultimately, LaVPR enables a new class of localization systems that are both resilient to real-world stochasticity and practical for resource-constrained deployment. Our
dataset and code are available at https://github.com/oferidan1/LaVPR
Read the full abstract ▾
All authors Ofer Idan, Dan Badur, Yosi Keller, Yoli Shavit
Computer Vision
InstaDrive: Instance-Aware Driving World Models for Realistic and Consistent Video Generation
Zhuoran Yang, Xi Guo, Chenjing Ding et al. (+3 more)
Autonomous driving relies on robust models trained on high-quality, large-scale multi-view driving
videos. While world models offer a cost-effective solution for generating realistic driving
videos, they struggle to maintain instance-level temporal consistency and spatial
geometric fidelity. To address these challenges, we propose InstaDrive, a novel framework that enhances driving
video realism through two key advancements: (1) Instance Flow Guider, which extracts and propagates instance features across frames to enforce temporal consistency, preserving instance identity over time. (2) Spatial
Geometric Aligner, which improves spatial
reasoning, ensures precise instance positioning, and explicitly models occlusion hierarchies. By incorporating these instance-aware mechanisms, InstaDrive achieves state-of-the-art
video generation quality and enhances downstream autonomous driving tasks on the nuScenes
dataset. Additionally, we utilize CARLA's autopilot to procedurally and stochastically simulate rare but safety-critical driving scenarios across diverse maps and regions, enabling rigorous safety evaluation for autonomous systems. Our project page is https://shanpoyang654.github.io/InstaDrive/page.html.
Read the full abstract ▾
All authors Zhuoran Yang, Xi Guo, Chenjing Ding, Chiyu Wang, Wei Wu, Yanyong Zhang
Computer Vision
EventFlash: Towards Efficient MLLMs for Event-Based Vision
Shaoyu Liu, Jianing Li, Guanghui Zhao et al. (+4 more)
Event-based
multimodal large
language models (MLLMs) enable robust perception in high-speed and low-light scenarios, addressing key limitations of frame-based MLLMs. However, current event-based MLLMs often rely on dense image-like processing paradigms, overlooking the spatiotemporal sparsity of event streams and resulting in high computational cost. In this paper, we propose EventFlash, a novel and efficient MLLM to explore spatiotemporal
token sparsification for reducing data redundancy and accelerating inference. Technically, we build EventMind, a large-scale and scene-diverse
dataset with over 500k instruction sets, providing both short and long event stream sequences to support our curriculum training strategy. We then present an adaptive temporal window aggregation module for efficient temporal sampling, which adaptively compresses temporal
tokens while retaining key temporal cues. Finally, a sparse density-guided
attention module is designed to improve spatial
token efficiency by selecting informative regions and suppressing empty or sparse areas. Experimental results show that EventFlash achieves a $12.4\times$ throughput improvement over the baseline (EventFlash-Zero) while maintaining comparable performance. It supports long-range event stream processing with up to 1,000 bins, significantly outperforming the 5-bin limit of EventGPT. We believe EventFlash serves as an efficient foundation model for event-based
vision.
Read the full abstract ▾
All authors Shaoyu Liu, Jianing Li, Guanghui Zhao, Yunjian Zhang, Wen Jiang, Ming Li, Xiangyang Ji
Computer Vision
Spiral RoPE: Rotate Your Rotary Positional Embeddings in the 2D Plane
Haoyu Liu, Sucheng Ren, Tingyu Zhu et al. (+5 more)
Rotary Position
Embedding (RoPE) is the de facto positional encoding in large
language models due to its ability to encode relative positions and support length extrapolation. When adapted to
vision transformers, the standard axial formulation decomposes two-dimensional spatial positions into horizontal and vertical components, implicitly restricting positional encoding to axis-aligned directions. We identify this directional constraint as a fundamental limitation of the standard axial 2D RoPE, which hinders the modeling of oblique spatial relationships that naturally exist in natural images. To overcome this limitation, we propose Spiral RoPE, a simple yet effective extension that enables multi-directional positional encoding by partitioning
embedding channels into multiple groups associated with uniformly distributed directions. Each group is rotated according to the projection of the patch position onto its corresponding direction, allowing spatial relationships to be encoded beyond the horizontal and vertical axes. Across a wide range of
vision tasks including classification,
segmentation, and generation, Spiral RoPE consistently improves performance. Qualitative analysis of
attention maps further show that Spiral RoPE exhibits more concentrated activations on semantically relevant objects and better respects local object boundaries, highlighting the importance of multi-directional positional encoding in
vision transformers.
Read the full abstract ▾
All authors Haoyu Liu, Sucheng Ren, Tingyu Zhu, Peng Wang, Cihang Xie, Alan Yuille, Zeyu Zheng, Feng Wang
Computer Vision
PokeFusion Attention: A Lightweight Cross-Attention Mechanism for Style-Conditioned Image Generation
Jingbang Tang
Style-conditioned text-to-image (T2I) generation with
diffusion models requires both stable character structure and consistent, fine-grained style expression across diverse
prompts. Existing approaches either rely on text-only prompting, which is often insufficient to specify visual style, or introduce reference-based adapters that depend on external images at inference time, increasing system complexity and limiting deployment flexibility. We propose PokeFusion
Attention, a lightweight decoder-level cross-
attention mechanism that models style as a learned distributional prior rather than instance-level conditioning. The method integrates textual semantics with learned style
embeddings directly within the
diffusion decoder, enabling effective stylized generation without requiring reference images at inference time. Only the cross-
attention layers and a compact style projection module are trained, while the pretrained
diffusion backbone remains frozen, resulting in a parameter-efficient and plug-and-play design. Experiments on a stylized character generation
benchmark demonstrate that the proposed method improves style fidelity, semantic
alignment, and structural consistency compared with representative adapter-based baselines, while maintaining low parameter overhead and simple inference.
Read the full abstract ▾
Computer Vision
FARTrack: Fast Autoregressive Visual Tracking with High Performance
Guijie Wang, Tong Lin, Yifan Bai et al. (+4 more)
Inference speed and tracking performance are two critical evaluation metrics in the field of visual tracking. However, high-performance trackers often suffer from slow processing speeds, making them impractical for deployment on resource-constrained devices. To alleviate this issue, we propose FARTrack, a Fast Auto-Regressive Tracking framework. Since autoregression emphasizes the temporal nature of the trajectory sequence, it can maintain high performance while achieving efficient execution across various devices. FARTrack introduces Task-Specific Self-
Distillation and Inter-frame Autoregressive Sparsification, designed from the perspectives of shallow-yet-accurate
distillation and redundant-to-essential
token optimization, respectively. Task-Specific Self-
Distillation achieves model compression by distilling task-specific
tokens layer by layer, enhancing the model's inference speed while avoiding suboptimal manual teacher-student layer pairs assignments. Meanwhile, Inter-frame Autoregressive Sparsification sequentially condenses multiple templates, avoiding additional runtime overhead while learning a temporally-global optimal sparsification strategy. FARTrack demonstrates outstanding speed and competitive performance. It delivers an AO of 70.6% on GOT-10k in real-time. Beyond, our fastest model achieves a speed of 343 FPS on the GPU and 121 FPS on the CPU.
Read the full abstract ▾
All authors Guijie Wang, Tong Lin, Yifan Bai, Anjia Cao, Shiyi Liang, Wangbo Zhao, Xing Wei
Computer Vision
ConsisDrive: Identity-Preserving Driving World Models for Video Generation by Instance Mask
Zhuoran Yang and Yanyong Zhang
Autonomous driving relies on robust models trained on large-scale, high-quality multi-view driving
videos. Although world models provide a cost-effective solution for generating realistic driving data, they often suffer from identity drift, where the same object changes its appearance or category across frames due to the absence of instance-level temporal constraints. We introduce ConsisDrive, an identity-preserving driving world model designed to enforce temporal consistency at the instance level. Our framework incorporates two key components: (1) Instance-Masked
Attention, which applies instance identity masks and trajectory masks within
attention blocks to ensure that visual
tokens interact only with their corresponding instance features across spatial and temporal dimensions, thereby preserving object identity consistency; and (2) Instance-Masked Loss, which adaptively emphasizes foreground regions with probabilistic instance masking, reducing background noise while maintaining overall scene fidelity. By integrating these mechanisms, ConsisDrive achieves state-of-the-art driving
video generation quality and demonstrates significant improvements in downstream autonomous driving tasks on the nuScenes
dataset. Our project page is https://shanpoyang654.github.io/ConsisDrive/page.html.
Read the full abstract ▾
Computer Vision
Zhiwen Li, Zhongjie Duan, Jinyan Ye et al. (+4 more)
Replicating In-Context Learning (ICL) in computer
vision remains challenging due to task heterogeneity. We propose VIRAL, a framework that elicits visual
reasoning from a pre-trained image editing model by formulating ICL as conditional generation via visual analogy ($x_s : x_t :: x_q : y_q$). We adapt a frozen
Diffusion Transformer (DiT) using role-aware multi-image conditioning and introduce a Mixture-of-Experts LoRA to mitigate gradient interference across diverse tasks. Additionally, to bridge the gaps in current visual context
datasets, we curate a large-scale
dataset spanning perception, restoration, and editing. Experiments demonstrate that VIRAL outperforms existing methods, validating that a unified V-ICL paradigm can handle the majority of visual tasks, including open-domain editing. Our code is available at https://anonymous.4open.science/r/VIRAL-744A
Read the full abstract ▾
All authors Zhiwen Li, Zhongjie Duan, Jinyan Ye, Cen Chen, Daoyuan Chen, Yaliang Li, Yingda Chen