Computer Vision
RoGS: Adaptive Meshgrid Gaussian for Large-Scale Road Surface Mapping
Tianchen Deng, Zhiheng Feng, Wenhua Wu et al. (+4 more)
Road surface mapping plays a crucial role in autonomous driving, supporting high-definition map generation, lane-level perception, and automatic road annotation. Recent mesh-based road surface reconstruction methods have shown promising results, but they still suffer from limited reconstruction quality and high optimization cost, especially in large-scale driving scenarios. To address these limitations, we propose ROADGS-T, a robust and efficient large-scale road surface mapping framework based on adaptive meshgrid Gaussian representation. Specifically, we model the road surface by placing 2D Gaussian surfels on a meshgrid, where each surfel explicitly stores color, semantic, and
geometric information. Compared with conventional mesh-based representations and 3D Gaussian primitives, the proposed meshgrid Gaussian representation better matches the thin-surface property of roads while significantly reducing redundant primitives and overlap during optimization. To further improve representation efficiency and structural fidelity, we introduce a road-structure-aware adaptive meshgrid strategy, which allocates denser Gaussian surfels to geometrically or semantically complex regions, such as lane markings, road boundaries, and height discontinuities, while maintaining a compact representation in flat road areas. Moreover, instead of relying on a single nearest vehicle pose, we design a trajectory-consistency-guided pose-robust refinement strategy, which estimates local surface priors from multiple neighboring poses and adaptively weights pose-guided height regularization according to their
geometric consistency.
Read the full abstract ▾
All authors Tianchen Deng, Zhiheng Feng, Wenhua Wu, Ziming Li, Chang Nie, Siting Zhu, Hesheng Wang
Computer Vision
Parameter-efficient Prompt Tuning of Vision Foundation Model With Adaptive Focal Loss for Interpretable MCI Screening
Javad Khoramdel, Farhad Hoseyni and Amirhossein Nikoofard
Mild Cognitive Impairment is a critical early stage of cognitive decline that frequently precedes Alzheimer's disease, yet its automated detection from neuropsychological drawing tests remains fundamentally constrained by data scarcity, class imbalance, and diagnostic ambiguity near clinical boundaries. Existing methodologies attempt to bypass these constraints using computationally expensive, fully fine-tuned hybrid architectures that relegate spatial explainability to a post-hoc approximation rather than an intrinsic model property. We propose a parameter-efficient framework utilizing frozen DINOv2-Small model adapted via three modality-specific learnable
prompt tokens while Operating with 1.19 million trainable parameters, each
token serves as a query in a shared cross-
attention layer over the source image patch
tokens. Crucially, spatial explainability is achieved directly through these
attention maps; as a structural consequence of the architecture. Then task-conditioned
embeddings fused via an
attention module to quantify modality-level importance per subject. To handle boundary ambiguity, a MoCA-adapted focal loss introduced that integrates continuous cognitive scores into the training target, loss modulation, and adaptive sample weighting, strictly generalizing standard soft-label approaches. Under stratified five-fold cross-validation, the proposed architecture yields an MCI-class F1 of 0.641 and an AUC of 0.795, outperforming the computationally heavier ResViT baseline by 0.110 in MCI-class F1.
Read the full abstract ▾
Computer Vision
Weakly-Supervised RGB-D Salient Object Detection via SAM-driven Pseudo Annotation and State Space Interaction-based Diffusion
Wenqi Si, Gongyang Li, Shixiang Shi et al. (+1 more)
Weakly-supervised RGB-D Salient Object Detection (SOD) is explored to reduce the heavy burden of pixel-level annotations. But scribble annotations lack the structure and details of objects, resulting in inaccurate saliency maps. In this paper, we propose a novel scribble-supervised RGB-D SOD method, consisting of a Segment Anything Model (SAM)-driven pseudo annotation generation method (SAM-PAG) and a state space interaction-based conditional
diffusion model (\emph{$S^2$Diff}). Specifically, SAM-PAG is tailored to address the issue of sparse supervision information. In SAM-PAG, we adopt the advanced SAM to expand sparse scribbles to dense pixel-level pseudo annotations through the dual-branch structure and the consistency of
segmentation masks. In $S^2$Diff, we adopt the
diffusion model to iteratively refine the noisy saliency maps with the guidance of conditional information, generating accurate saliency maps. Naturally, the core of our $S^2$Diff lies in the acquisition of conditional features and the denoising of saliency maps. For the former, we employ a cross-modal conditional generation module to interweave cross-modal features through frequency integration and implicit-explicit state space interaction, effectively achieving global conditional features. For the latter, we employ a context injection module to mitigate noise interference and to enhance object information with the conditional context. With the close cooperation of SAM-PAG and $S^2$Diff, our method outperforms relevant scribble-supervised methods and achieves competitive performance compared to fully-supervised methods on seven
datasets. The code and results of our method are available at https://github.com/Switch457/WeakS2Diff_SOD.
Read the full abstract ▾
All authors Wenqi Si, Gongyang Li, Shixiang Shi, Weisi Lin
Computer Vision
Video = World + Event Stream
Lianghua Huang, Zhi-Fan Wu, Yupeng Shi et al. (+24 more)
We present Wan-Streamer v0.3, which reframes our native-streaming interaction model under a single organizing view: a
video is a world plus an event stream. The world is the persistent context in which a
video unfolds, including the environment, scene, subjects, ambient acoustic conditions, voice characteristics, and other relatively stable conditions. The event stream is everything that changes over time within that world, including scene or environmental changes, subject behavior,
speech, and other sounds. This yields a general-purpose pretraining task over large amounts of real
video: given a world and incoming input, predict how the world moves, changes, and responds in real time. The resulting competence can be specialized to a broad family of real-time downstream tasks. We instantiate it on real-time full-duplex audio-visual interaction, where the event stream is the
agent's
speech together with free-form behavior. Functionally, the model's
multimodal understanding process is
vision-
language-action-like: it maps
multimodal user input to
language-form
speech and behavior actions. Wan-Streamer v0.3 preserves the v0.2 operating point: 640x368
video at 25 FPS, a 160 ms streaming unit, approximately 200 ms model-side response latency, and approximately 550 ms total interaction latency under a 350 ms bidirectional network budget.
Read the full abstract ▾
All authors Lianghua Huang, Zhi-Fan Wu, Yupeng Shi, Wei Wang, Mengyang Feng, Cheng Yu, Chen Liang, Junjie He, Chen-Wei Xie, Yu Liu, Jingren Zhou, Ang Wang, Bang Zhang, Baole Ai, Chongyang Zhong, Jinwei Qi, Kai Zhu, Pandeng Li, Peng Zhang, Wenyuan Zhang, Xinhua Cheng, Yitong Huang, Yun Zheng, Yuxiang Bao, Yuzheng Wang, Zhiwei Lin, Zoubin Bi
Computer Vision
JADE-GS: Joint Allocation of Deblurring Evidence for Event-Assisted 3D Gaussian Splatting
Haoyu Fu, Jiafeng Huang, Yuchen Wang et al. (+1 more)
Neural radiance fields and 3D Gaussian Splatting assume that each training image is a sharp and geometrically consistent observation of the scene. Motion blur violates this assumption because a single exposure integrates a continuous range of camera poses. Exposure integration also removes the temporal information needed to recover the corresponding sharp observation. Event cameras preserve this information at microsecond resolution and therefore provide a natural complement to conventional images. Existing event-assisted reconstruction methods predominantly obtain image supervision through analytical inversion of the Event Double Integral. Learned restoration from frames and events offers a second prior. Although weaker when used alone, it fails in different regions and provides complementary evidence. We present JADE-GS, which formulates the combination of these priors as spatial evidence allocation. A lightweight Spatial Prior Router predicts a pixelwise allocation using only the blurry frame and event stream, then fuses the two fixed restorations into an additional supervision target. The router is trained without a sharp reference using consistency with the scene under reconstruction and the measured exposure, and is removed after optimization. Experiments show that JADE-GS achieves leading perceptual quality on both
benchmarks, attains the best fidelity on the real
benchmark, and remains competitive on the synthetic one. It requires substantially lower training overhead than
diffusion-based alternatives and preserves native 3DGS rendering with no generative decoding at inference.
Read the full abstract ▾
All authors Haoyu Fu, Jiafeng Huang, Yuchen Wang, Shengjie Zhao
Computer Vision
From Draft to Draft-Free: One-Step Video Object Removal via Privileged Distillation and Fast Planting
Zizhao Chen, Ping Wei, Guang Dai et al. (+2 more)
Video object removal is a fundamental yet challenging task in
video editing. Despite recent progress, existing methods typically fall into two categories. Traditional approaches based on optical flow or
attention mechanisms often introduce noticeable artifacts and yield unnatural results. In contrast,
diffusion-based methods improve visual realism but demand multiple denoising steps, limiting their practicality. To address these issues, we propose From-Draft-to-Draft-Free (D2DF), a framework that distills the ability of transforming coarse drafts into refined
videos into a one-step
video generation model. Within D2DF, a teacher model is trained to refine low-quality removal results ("drafts") into high-fidelity
videos by multiple steps. Then, through Prior-Privileged Consistency
Distillation (PPCD), we distill this capability into a student model that performs one-step removal conditioned on the draft. To eliminate draft dependency, we introduce a Self-Guided Fast Planting (SGFP) module based on our Temporal Masked
Transformer that autonomously generates scene-consistent pseudo-drafts in latent space, enabling a fully draft-free one-step model. Extensive experiments show that both draft-conditioned and draft-free versions achieve state-of-the-art performance on multiple metrics, surpassing traditional and multi-step generative methods in both quality and efficiency. The denoising process for a single
video takes only about 1 second.
Read the full abstract ▾
All authors Zizhao Chen, Ping Wei, Guang Dai, Jingdong Wang, Mengmeng Wang
Computer Vision
On Success and Simplicity: A Second Look at Transferable Vision-Language Attack Pipeline
Yuchen Ren, Zhengyu Zhao, Chenhao Lin et al. (+2 more)
Vision-
Language Pre-training Models (VLPMs) are known to be vulnerable to
adversarial attacks. Recent transferable attacks on VLPMs have followed a common pipeline with complicated loss functions or multi-stage text/image attacks. However, in this paper, we demonstrate that such a sophisticated attack pipeline can be simpler yet more successful. Specifically, we identify three previously overlooked issues caused by inappropriate cross-modal interactions and excessive operations. To address them, we propose the Simple
Vision-
Language Attack (SimVLA) pipeline, which observably improves transferability and efficiency. Experiments on four
datasets and three downstream tasks validate the superiority of our pipeline. For instance, on Flickr30k text-image
retrieval dataset, our SimVLA outperforms the SOTA baseline in R@1 transferability by 8.01%-14.71%, while consuming only about 35.73% of the time and 46.26% of the max VRAM. Overall, the superiority of our SimVLA highlights the importance of leveraging domain knowledge (e.g., our proposed cross-modal word identification), while blindly pursuing intricate operations (e.g, complex loss functions and redundant multi-stage designs) may even be harmful. We hope our SimVLA can serve as a simple yet effective backbone for future extensions. Code is available at https://github.com/RYC-98/SimVLA.
Read the full abstract ▾
All authors Yuchen Ren, Zhengyu Zhao, Chenhao Lin, Bo Yang, Chao Shen
Computer Vision
Stitch-Inferencer: Enhance Endoscopic Video Segmentation and Tracking via Panoramic Reconstruction
Shunsuke Kikuchi, Atsushi Kouno and Hiroki Matsuzaki
Surgical video understanding is fundamental to
navigation systems. Endoscopic perception is often hindered by a limited field-of-view and frequent instrument occlusions, making spatio-temporal context essential for robust inference. These challenges have motivated
video models that aggregate information across frames. However, existing
video models typically store past observations implicitly in learned feature representations, often requiring task-specific
video training, substantial annotated data, and increased computational cost. We propose Stitch-Inferencer, a real-time, model-agnostic inference framework that replaces implicit feature
memory with an explicit image-space panoramic canvas. By stitching valid observations across frames, Stitch-Inferencer preserves previously observed pixels in an online, instrument-free view, expanding the effective field-of-view and providing direct access to regions that are temporarily occluded or absent from the current frame. Downstream
segmentation or tracking models are applied to a compact region of interest on the panorama, and their predictions are reprojected to the current frame, enabling existing models to exploit long-range context without retraining. Experiments on anatomy
segmentation and point/box tracking demonstrate consistent improvements across diverse baselines while preserving real-time throughput. The stitching module alone runs at over 60 FPS, providing a practical inference-time solution to enhance endoscopic perception in computationally constrained intraoperative environments. Source code will be made publicly available.
Read the full abstract ▾
Computer Vision
U-shaped Multi-granularity Learning for Vision-Language Models
Biao Chen, Yunqian Yu, Xiangxu Zhao et al. (+3 more)
The
prompt learning paradigm for
vision-
language models is effective yet faces a granularity dilemma: global
prompts lack fine-grained semantic awareness, while local
prompts ignore contextual associations, limiting cross-task generalization. This dilemma exists in dense prediction tasks. Inspired by U-Net, which unifies multi-level representations across granularities, we propose UPrompt, a U-shaped multi-granularity
prompt learning framework for
vision-
language models. Similar to how U-Net integrates fine and coarse features through symmetric encoder-decoder pathways with cross-level connections, UPrompt constructs parallel multi-granularity representations in both visual and textual modalities, where coarse-to-fine cascaded enhancement propagates global context to refine local details, while fine-to-coarse hierarchical supervision ensures semantic consistency across scales. Extensive experiments on 17
benchmarks validate our effectiveness. UPrompt outperforms MAMET and VPKE by 4.1 and 7.3 rSum on MSCOCO, surpasses CoCoA-Mix by 5.09% in base-to-novel generalization, while maintaining competitive performance with minimal overhead (coarse-grained) and matching PSRC with 1/3 cost (medium-grained).
Read the full abstract ▾
All authors Biao Chen, Yunqian Yu, Xiangxu Zhao, Zhongshu Chen, Mengmeng Jing, Lin Zuo
Computer Vision · Cross-listed from cs.LG
Multi-Axis Max@K Reinforcement Learning for Representative Diversity in Text-to-Image Generation
Ku Onoda, Paavo Parmas, Hiroki Furuta et al. (+4 more)
Text-to-image (T2I) models can synthesize realistic,
prompt-aligned images, yet samples generated for the same
prompt often cover only a small subset of visually distinct modes. This limits diversity and, for person-centric
prompts, can reflect or amplify demographic skew. We formalize this problem as target-mode coverage, the coverage of a predefined set of semantically specified modes, and propose multi-axis max@K, a group-based reinforcement learning objective for improving it in
diffusion-based T2I models. Given a group of samples and one score per target mode, multi-axis max@K first takes the maximum score across samples for each mode and then sums these per-mode maxima. The resulting credit assignment gives a sample positive weight on a mode only when it raises that mode's group maximum, so different samples can contribute to different modes. We validate the credit-assignment mechanism on a synthetic mixture and on SD3.5-M with deterministic pixel-based color rewards, and then apply the same objective to perceived-appearance
fairness. On held-out
prompts, multi-axis max@K improves the
Fairness Score by 0.23-0.36 over the base model under three automatic evaluators, while maintaining image quality and text
alignment. Code is available at https://github.com/KuOnoda/multi-axis-maxk.
Read the full abstract ▾
All authors Ku Onoda, Paavo Parmas, Hiroki Furuta, Soichiro Nishimori, Yuta Oshima, Shohei Taniguchi, Yutaka Matsuo
Computer Vision
DINE: Distance Is Not Enough – Learning Global Deformation Priors for Robust Soft-Tissue Point Cloud Registration
Sara Monji-Azad, Rohit Beer, Marvin Kinz et al. (+2 more)
Non-rigid point cloud registration is central to soft-tissue shape analysis, but large deformations, noise, and outliers make correspondence estimation challenging. Most learning-based methods rely on local objectives such as Chamfer distance, which encourage point-wise proximity but do not constrain the global plausibility of the predicted deformation field. We address this limitation with DINE, a maximum a posteriori framework that augments distance-based registration with a learned statistical prior over displacement vector fields. DINE is applied to two registration backbones, Robust-DefReg and DefTransNet, using a two-stage strategy: a first-stage model is trained with Chamfer distance, its predicted deformation fields are used to estimate a prior, and the model is then refined with a combined distance and negative log-prior objective. We compare a full-field PCA Gaussian prior with a per-vector normalizing-flow prior. Experiments on DeformedTissue and SynBench show lower mean Chamfer distance under deformation and corruption. On DeformedTissue, DINE-PCA reduces Chamfer distance by approximately 27–69% relative to the corresponding Stage-1 backbone across deformation levels, and improves robustness by up to 66% for outliers and 83% for Gaussian noise. On SynBench, improvements are modest at the smallest deformation levels and reach approximately 59–79% from moderate to severe deformation. These results suggest that global deformation plausibility is an important constraint for reliable soft-tissue point cloud registration. (The code will be published soon.)
Read the full abstract ▾
All authors Sara Monji-Azad, Rohit Beer, Marvin Kinz, Claudia Scherl, Jürgen Hesser
Computer Vision
Introspective Attention Modulation for Safe Text-to-Image Generation
Basim Azam, Hossein Rahmani and Naveed Akhtar
State-of-the-art flow based text-to-image (T2I) models exhibit remarkable generative abilities but remain vulnerable to producing unsafe content. Prior safety efforts range from concept erasure and
prompt filtering to classifier-based gating. However, simple techniques like parameter efficient adaptations of the models easily bypass such guardrails. We introduce a unique principled approach that achieves safety by regulating the model's
attention dynamics through inference-time introspection, exhibiting intrinsic robustness. Our method analyzes and rebalances
attention activations throughout image synthesis, steering generations away from unsafe concepts while preserving semantic
alignment. This introspective control ensures safety of deployed models. Across standard and
adversarial safety
benchmarks, our approach achieves remarkable safety scores while maintaining or even improving
alignment and perceptual quality. Our results reveal that
attention-space regulation offers a considerably more promising path to safer
diffusion transformer based image generation than the existing concept erasing mechanism.Our code can be accessed at https://basim-azam.github.io/iam/
Read the full abstract ▾
Computer Vision
Frequency-Structured Field Learning for Light-Field Disparity Estimation
Sara Monji-Azad, Yulin Liu and Jürgen Hesser
Light-field disparity estimation requires global consistency in smooth or textureless regions and local precision near occlusion boundaries, thin structures, and abrupt depth transitions. Existing methods address these requirements through EPI matching, cost-volume or focal-stack construction, view aggregation, or direct convolutional regression, often relying on local windows, discrete disparity hypotheses,
memory-intensive volumes, or
attention-based aggregation. We instead formulate disparity estimation at the field level, predicting disparity from globally and locally updated EPI-derived latent features without explicitly constructing a disparity volume. We introduce FreqLF, an EPI-guided
Fourier-local framework that encodes angular parallax cues from horizontal and vertical EPI stacks together with central-view appearance features. These cues are projected into a latent field and updated through stacked hybrid
Fourier-local layers.
Fourier low-mode updates enable global feature interaction, while local convolutions preserve spatial variations needed for fine disparity detail. A coordinate-conditioned Gaussian-mixture decoder then predicts disparity, using the mixture mean as the final estimate. Experiments on the HCI 4D Light Field
Benchmark show that FreqLF approaches the accuracy of strong supervised baselines while avoiding explicit cost-volume construction in the base model. Ablations confirm the complementary roles of the
Fourier and local branches, and scaling experiments demonstrate practical behavior across spatial resolutions. These results suggest that
Fourier-local latent field learning is a competitive alternative for light-field disparity estimation. The code will be published soon.
Read the full abstract ▾
Computer Vision
VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding
Xinhao Li, Yuhan Zhu, Xiangyu Zeng et al. (+24 more)
Recent advances in
video understanding have spanned motion, long
video, and streaming interaction, driving this field toward real-world applications. Despite this progress, current open-source models remain limited in several ways. They often struggle to generalize across diverse
video types, making them effective only in specific domains. High computational demands further restrict their efficiency and scalability. Moreover, most models are only partially open, with key components such as training code, strategy, or
datasets unavailable, which hinders reproducibility and slows community-driven development. To address these issues, we introduce VideoChat3, a fully open, efficient, and generalist
video-centric MLLM. VideoChat3 advances
video understanding through two complementary designs. For efficiency, we introduce Inflated 3D
Vision Transformer (I3D-ViT) and Adaptive Frame Resolution for Streaming
Video Perception, which enables efficient spatiotemporal representation and reduces the cost of processing
video inputs during training and inference. For effectiveness, we develop a scalable
video data synthesis pipeline that curates three diverse, high-quality training
datasets: VideoChat3-Academic2M, VideoChat3-LV116K, and VideoChat3-OL617K, covering general, long-form, and streaming
video scenarios, improving the model's generalization across domains. By integrating these designs, VideoChat3 achieves a rare balance of broad generalization and computational efficiency. Experiments across general, long-form, and streaming
benchmarks demonstrate that VideoChat3 surpasses prior open-source models with equal or larger parameter counts with only 4B parameters and higher efficiency.
Read the full abstract ▾
All authors Xinhao Li, Yuhan Zhu, Xiangyu Zeng, Yuhao Dong, Haoning Wu, Zhiqiu Zhang, Yuandong Yang, Changlian Ma, Qingyu Zhang, Yansong Shi, Xinyu Chen, Haoran Chen, Zizheng Huang, Jun Zhang, Kun Ouyang, Lin Sui, Ziang Yan, Yicheng Xu, Chenting Wang, Yinan He, Hongjie Zhang, Yi Wang, Yu Qiao, Yali Wang, Ziwei Liu, Kai Chen, Limin Wang
Computer Vision
Still image and spatial-temporal tomato data enabling detection, segmentation, tracking, and video-instance segmentation using strong and weak labels
Michael Halstead, Esra Guclu, Mohamed Farag et al. (+7 more)
In this manuscript we release two
datasets for visual sensing of tomato plants grown in commercial-like settings and acquired using a
robot. The first is BUTom21 which consists of still images and manual annotations. The second is BUTom-ST21 which consists of
video-based data and semi-automated annotations through AI-based methods, referred to as pseudo-labels. In both cases, we provide pixel-level labels for the ripeness of the fruit. The aim is to provide the research community a challenging set of real-world imagery to explore methods to sense and estimate the state of tomato plants and their fruit, which is an important horticultural crop. Importantly, the spatial-temporal
dataset provides individual fruit count and ripeness information enabling researchers to push the boundaries of field-based phenotyping.
Read the full abstract ▾
All authors Michael Halstead, Esra Guclu, Mohamed Farag, Enrico Pallotta, Christian Hund, Ribana Roscher, Maren Bennewitz, Juergen Gall, Cyrill Stachniss, Chris McCool
Computer Vision
Benchmarking Face Recognition without Real Faces
Paweł Borsukiewicz, Daniele Lunghi, Wendkûuni C. Ouédraogo et al. (+2 more)
Synthetic face
datasets have become effective enough to train face recognition models with accuracy rivaling that of models trained on real photographs. This progress sidesteps the ethical and legal burdens of collecting real biometric data, yet evaluation has not kept pace. Even studies that train entirely on synthetic images still rely on real-face
benchmarks to measure performance, leaving the
privacy problem only half solved. We ask whether synthetic
datasets can replace real
benchmarks for face recognition evaluation. We test 12 synthetic
datasets against 7 established real
benchmarks using 24 pre-trained models that span both convolutional and
transformer architectures. Our evaluation covers biometric verification metrics, similarity score distributions, cross-model ranking consistency, and the underlying distributional properties of each
dataset. Benchmarking fidelity varies widely across the synthetic candidates, but the two strongest, MorphFace and Vec2Face, reproduce the relative behavior of real
benchmarks and reach agreement levels that fall within the natural disagreement already observed among the real
benchmarks themselves. These results establish that well-constructed synthetic
datasets can support reliable comparative evaluation for face recognition, moving the field closer to a fully synthetic and
privacy-preserving pipeline for both training and benchmarking.
Read the full abstract ▾
All authors Paweł Borsukiewicz, Daniele Lunghi, Wendkûuni C. Ouédraogo, Jacques Klein, Tegawendé F. Bissyandé
Computer Vision
TanGO: Training-Free 3D Editing via Tangent-Space Guidance and Optimization
Siwoo Lim, Sunjae Yoon, Gwanhyeong Koo et al. (+2 more)
While recent flow-matching 3D generative models (e.g., VecSet) adopt structured representations, their
tokens share global context, causing conventional training-free editing to suffer from semantic artifacts such as collapsed preserved regions or incomplete transformations. To address this, we propose TanGO, a training-free framework that enables adaptive per-
token steering in the tangent space of generative dynamics. To realize this selective control, we formulate a one-step optimal control rule and determine the strength of each
token's control signal using a von Mises-Fisher inspired directional discrepancy derived from the source and target velocity fields. Experiments show that TanGO substantially reduces structural artifacts and achieves state-of-the-art performance, outperforming existing 3D editing baselines. The code is publicly available at https://github.com/siw00-lim/TanGO.
Read the full abstract ▾
All authors Siwoo Lim, Sunjae Yoon, Gwanhyeong Koo, Hyeonseo Yun, Chang D. Yoo
Computer Vision
FlashDecoder: Real-Time Latent-to-Pixel Streaming Decoder with Transformers
Minguk Kang and Suha Kwak
Real-time
video generation demands fast decoding as much as fast denoising, yet current latent
video diffusion models rely on 3D convolutional decoders that are slow and
memory-intensive at high resolutions or for long
video. We introduce FlashDecoder, a fast,
memory-efficient pure-
Transformer video decoder that decodes latents to pixels frame by frame. At each step, the current frame attends only to a fixed-size window of past frames through a rolling KV cache. The fixed temporal window keeps decoding fast and
memory bounded regardless of
video length, enabling constant-latency streaming. Because frames are processed sequentially, temporal causality is enforced without explicit
attention masks, enabling training at resolutions up to 1080p and matching the reconstruction quality of convolutional decoders. On the Wan2.1 and Wan2.2 latent spaces, FlashDecoder matches each convolutional decoder in reconstruction quality (e.g., 41.55dB vs. 41.49dB PSNR at 1080p) while decoding 3.6x-4.7x faster with up to 11x less
memory on a single H100 GPU. With architecture-aware inference optimizations, the speedup widens to 12x.
Read the full abstract ▾
Computer Vision
Selectivity Drives Efficiency: Dataset Pruning for Visual Place Recognition
Tong Jin, Yunpeng Liu, Shuyu Hu et al. (+3 more)
Recent visual place recognition (VPR) studies have increasingly relied on large-scale
datasets to train more robust and discriminative models. Although this trend significantly improves recognition performance, it also introduces substantial storage and training costs, especially when new architectures or training strategies need to be repeatedly developed and evaluated.
Dataset pruning (DP) provides a promising way to improve data efficiency by retaining only informative training data. However, conventional DP methods mainly follow the sample-wise classification paradigm, which overlooks the relation-dependent training nature of VPR, where supervision is typically formed by image pairs rather than independent images. To address this issue, we propose a place-wise
dataset pruning framework tailored for VPR. Instead of
pruning individual images, our method treats each place as the basic
pruning unit and introduces two complementary novel metrics, i.e., intra-place diversity (IPD) and inter-place similarity (IPS), to evaluate the training value of each place. By jointly considering these two metrics, our method ranks all places and constructs a compact yet informative coreset, thereby allowing the pruned
dataset to still support the training of robust and discriminative VPR models. Extensive experiments demonstrate that our method consistently outperforms state-of-the-art DP baselines under different
pruning ratios while reducing selection and training costs. Moreover, by
pruning a merged
dataset roughly 3.5$\times$ the size of GSV-Cities to a comparable scale, our coreset maintains highly competitive performance, achieving 94.5% R@1 on MSLS-val and 97.0% R@1 on Nordland with only NetVLAD. Codes will be made publicly available.
Read the full abstract ▾
All authors Tong Jin, Yunpeng Liu, Shuyu Hu, Chun Yuan, Song Wang, Feng Lu
Computer Vision
Rotational Motion-Induced Error Compensation for Phase-Shifting Profilometry-Based Eye Reconstruction
Seong-Jin An, Sanghoon Jeon, Yatong An et al. (+1 more)
With the proliferation of immersive Head-Mounted Displays (HMDs) for Virtual and Augmented Reality (VR/AR), reliable and high-precision eye tracking has become increasingly important. Conventional 2D image-based methods offer low system complexity but remain limited in stability, accuracy, and robustness. Three-dimensional ocular surface reconstruction can provide richer geomet-ric information, and structured light profilometry is particularly attractive because it enables dense and accurate surface measurement. However, Phase-Shifting Profilometry (PSP), which estimates phase from sequentially acquired fringe images, is highly susceptible to motion-induced errors when the eye rotates between frames. This study proposes a rotational motion compensation framework for PSP-based dynamic 3D eye reconstruction. Relative eye rotation is estimated from image-based motion cues using a user-specific 3D eye model in a spherical-coordinate domain. The estimated motion is then used to compensate for camera-pixel mismatch and phase-shift errors caused by inter-frame rotation. A region-wise optimization strategy is further introduced to reduce residual artifacts by inde-pendently refining the compensation strength in different ocular regions. Experiments with a rotating fake eye under non-uniform motion demonstrate that the proposed method substantially suppresses motion-induced deformation and improves reconstruction accuracy. An additional experiment with a non-spherical rigid object indicates that the compensation principle is not restricted to spherical eye geometry. These results establish a practical basis for stable PSP-based dynamic 3D eye reconstruction toward future high-precision eye tracking in immersive environments.
Read the full abstract ▾
All authors Seong-Jin An, Sanghoon Jeon, Yatong An, Jae-Sang Hyun
Computer Vision
Physics-Informed Diffusion for Biomechanically Plausible 3D Sign Language Generation
Emanuele Colonna, Moises Diaz, Gennaro Vessio et al. (+2 more)
Sign
language production, which generates continuous 3D skeletal motion from spoken
language input, must simultaneously satisfy two constraints: semantic fidelity, so that a deaf viewer can recognize the intended sequence of glosses, and biomechanical plausibility, so that the generated skeleton respects anatomical constraints. Existing approaches optimize semantic reconstruction through coordinate-based objectives that treat the skeleton as an unstructured vector, thus allowing for bone length drift, joint angle violations, and temporarily locked fingers. We introduce PIDiffSign, a physics-informed
diffusion model for gloss-to-pose translation that incorporates anatomical constraints into both the architecture and training objective. The model uses a
Transformer encoder-decoder, where the decoder is conditioned on the
diffusion time step through adaptive zero-initialized layer normalization and cross-attends to gloss representations. A differentiable geometry module enforces bone length consistency and biologically valid joint angles throughout generation. Training combines anthropomorphic, kinematic, angular, and finger-joint constraints with a contrastive gloss-pose
alignment loss and classifier-free guidance for semantically conditioned sampling. Experiments on the PHOENIX14T and CSL-Daily
benchmarks show consistent improvements over a strong
diffusion baseline in pose accuracy, joint-angle correctness, distributional realism, and back-translation quality. These results demonstrate that physics-informed
diffusion improves both motion realism and semantic fidelity for sign
language generation.
Read the full abstract ▾
All authors Emanuele Colonna, Moises Diaz, Gennaro Vessio, Miguel Angel Ferrer, Giovanna Castellano
Computer Vision
Blurring Modal Boundaries: A Unified Survey from Single- to Multi-Modal Person Re-ldentification
Xiao Wang, Bing Wang, Bin Yang et al. (+3 more)
Person re-identification (ReID) serves as a critical component in intelligent surveillance systems, aiming to match identities across disjoint camera networks. While traditional methods primarily rely on single-modal RGB imagery, they are often constrained by environmental challenges such as low illumination and occlusion. To overcome these limitations, the field is rapidly evolving toward cross-modal and multi-modal paradigms. This survey presents a comprehensive overview of this transition, systematically reviewing key cross-modal tasks including visible-infrared (VI-ReID), text-image (TI-ReID), sketch-based (Sketch-ReID), and the emerging Non-Line-of-Sight (NLOS) ReID, which extends perception beyond direct visibility. Furthermore, we examine tri-spectral and multi-modal fusion ReID, discussing how complementary information from diverse sensors enhances robustness. Beyond summarizing
datasets, challenges, and methodologies, we propose a
Transformer-based baseline framework for visible-infrared ReID, designed to effectively capture modality-invariant features. Finally, based on the current landscape, we outline several promising directions for future research.
Read the full abstract ▾
All authors Xiao Wang, Bing Wang, Bin Yang, Cuiqun Chen, Xin Xu, Mang Ye
Computer Vision
An LLM-Based Automatic Sportscast Solution for Robot Soccer Matches
Francesco Petri, Michele Brienza, Daniele Nardi et al. (+3 more)
RoboCup has always been a scenario to develop systems that solve real-world problems. Driven by the main goal of playing against the 2050 FIFA World Cup champions, the RoboCup Soccer leagues need to constantly measure how the research community is progressing. Computing visual statistics from match
videos is a crucial way to track this
evolution. To address this challenge, this paper introduces a fully autonomous, real-time sports commentator for RoboCup matches. By bridging the gap between raw kinematic tracking and natural
language generation, our neuro-symbolic architecture extracts precise statistics from
video streams and turns them into fluent,
hallucination-free narration. The proposed system is capable of generating statistics and commentary both during live match streaming and in post-game analysis, easily adapting to the new dynamism of the league where different
humanoid robots of different sizes share the field. Supplemental materials are available at https://lab-rococo-sapienza.github.io/MARIO/
Read the full abstract ▾
All authors Francesco Petri, Michele Brienza, Daniele Nardi, Domenico Daniele Bloisi, Aldo Gangemi, Vincenzo Suriani
Computer Vision
TAMF-VTON: Token-Wise Sparse Adaptation for Mask-Free Virtual Try-On
Jie Wang, Qian He, Gaofeng He et al. (+2 more)
Diffusion-based virtual try-on (VTON) often relies on
segmentation-derived masks that become unreliable when the reference garment differs substantially from the target person's clothing. Existing methods may also lose fine details and commonly focus on single-garment transfer. We present TAMF-VTON, an inference-time mask-free framework built on Qwen-Image-Edit-2509. Given a person image, one or more garment images, and a text instruction, it performs single- or multi-garment transfer without masks, parsing maps, pose representations, or spatial controls at inference. Its central component is
token-wise sparse Mixture-of-Experts adaptation, which provides input-dependent capacity without predefined expert roles. Timestep-weighted frequency supervision regularizes high-frequency reconstruction, while adaptive offline data construction supplies cross-category and cross-silhouette training tuples; neither is used at inference. TAMF-VTON achieves the best scores on most reported paired and unpaired metrics among evaluated VTON baselines on the standard single-garment VITON-HD and DressCode
benchmarks. Selected qualitative results illustrate multi-garment composition, subject-to-subject transfer, diverse scenes, body-shape variation, and illumination changes. With INT4 on an RTX 4090, TAMF-VTON generates 1K-resolution images under 15 seconds and 2K-resolution images under 30 seconds. Our project can be accessed via https://www.style3d.ai/ai-photoshoot/virtual-clothing-try-on.
Read the full abstract ▾
All authors Jie Wang, Qian He, Gaofeng He, Xiaogang Jin, Huamin Wang
Computer Vision
Rare Concept Generation via Counterfactual Inference in Diffusion Models
Zhengyuan Jiang, Haipeng Liu, Meng Wang et al. (+1 more)
Rare concept generation focuses on synthesizing customized images conditioned on text
prompts that describe objects with unusual attributes. Previous works failed to align the generated images with rare concepts, resulting in incorrect attribute rendering or inconsistent composition of concepts. Such failures, as we observed, stem from the inherent common knowledge bias in the training stage of
diffusion models, where objects are strongly associated with their common attributes, making it difficult to break these associations when generating rare concepts. To address such challenges, in this paper, we propose a novel Counterfactual Inference-based
Diffusion approach, dubbed CI-Diff. CI-Diff blocks the interference of the model's inherent common knowledge bias and utilizes the Natural Direct Effect to capture the independent influence of the text
prompt of rare concepts on image generation so that decoupling the unusual attributes from the rare concepts. To this end, we reformulate the classifier-free guidance mechanism to highlight the atypical attributes. To the best of our knowledge, we are the first to introduce causal inference into the rare concept generation task. Extensive experiments on the RareBench
benchmark validate the superiority of CI-Diff over state-of-the-art
diffusion models. Our code can be accessed from https://github.com/200204jzy/CI-Diff.
Read the full abstract ▾
All authors Zhengyuan Jiang, Haipeng Liu, Meng Wang, Yang Wang
Computer Vision
Clean-Reference Streaming Detection of Lens Occlusion and Photometric Transitions for Camera Tamper Monitoring
Bo Ma, WeiQi Yan and Jinsong Wu
A surveillance camera is an image sensor whose silent physical degradation invalidates every downstream consumer of its data. In-situ integrity alarms for such
vision sensors require low false-alarm rates, bounded computation, and diagnosable behavior under nuisance illumination changes. This paper studies a deliberately narrow streaming integrity monitor for two low-cost sensor-fault signatures: texture-collapsing lens occlusion and abrupt photometric scene transition. The detector compares sampled luminance and local-gradient statistics with a clean-only sliding reference, applies coarse-grid structured-light rejection and mode/rapid-brightness suppression, and emits at most one notification per tamper episode. We formalize the decision predicates and derive a consistency rule for when rapid-brightness suppression makes the scene-transition path unreachable. On 320 in-scope controlled sequences, the default state machine attains 0.800 F1 and 0.822 balanced accuracy (significantly better paired correctness than the strongest baseline, though the F1 margin is not statistically resolved); on a magnitude-swept public audit it attains the highest partial AUC under a 5% false-alarm budget, and a separate extended-stress FPR-constrained sweep reaches 0.925 recall at 0.025 false-positive rate. Public Xiph, Bremen IoT, and UHCTD diagnostics show the fixed predicates preserve low false alarms while recall concentrates inside the declared envelope (UHCTD in-scope covered recall 0.667 versus 0.016 out of scope), and a 9.09-camera-hour verified-negative public audit records zero false alarms. The method is best interpreted as an auditable sensor-health subsystem rather than a universal camera-tamper classifier.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.AS
WanSong v1.0 Technical Report
Binghui Chen, Pandeng Li, Yu Liu et al. (+1 more)
Music generation foundation models have recently attracted significant industry
attention. However, achieving efficient generation and high-fidelity long-form audio while supporting controllability remains challenging. To address these needs, we present WanSong, a simple yet powerful approach for long-form, commercial-grade song generation. Unlike autoregressive (AR) and cascaded multi-stage pipelines (\eg, AR followed by
diffusion), WanSong is a pure
diffusion-based model that directly generates high-fidelity,
multilingual songs up to 5 minutes and outputs dual stems (vocals and background music) in a single run. In addition, our
diffusion framework enables faster inference through step-
distillation, and offers an efficient pathway for fine-tuning and customization to support downstream editing tasks.
Read the full abstract ▾
All authors Binghui Chen, Pandeng Li, Yu Liu, Jingren Zhou
Computer Vision
On the Disagreement in Perturbation-based xAI – Benchmarking Perturbation Choices for Flood Detection from SAR Images
Anastasia Schlegel and Ronny Hänsch
Perturbation-based xAI methods are widely used to analyze the behavior and predictions of deep learning models. By altering input regions and measuring the resulting changes in class probabilities with respect to the original image, they assign relevance scores and generate heatmaps that reflect each region's contribution to the prediction. Despite their apparent simplicity, however, perturbation-based methods are sensitive to parameter choices. In this work, we focus on two key parameters of the perturbation pipeline, namely the patch geometry, including the size and shape of the perturbed regions, and the perturbation type, defined by the replacement scheme. Grounded in the use case of flood detection from Synthetic Aperture Radar imagery, we conduct a comprehensive investigation of how relevance estimation changes under different perturbation settings. Beyond visual inspection of the resulting relevance maps, we evaluate their consistency across perturbation strategies and their faithfulness to the model's
reasoning. We demonstrate how different perturbation choices can steer the resulting relevance maps, yielding ambiguous and even contradictory explanations. Our findings emphasize the importance of methodological settings in perturbation-based xAI. They underscore the need to carefully inspect and evaluate perturbation choices and to treat them as an integral part when interpreting explanations, ensuring a robust understanding of both the explanations and model predictions.
Read the full abstract ▾
Computer Vision
FoMoVLA: Bridging Visual Foresight and Motion Guidance for Vision-Language-Action Models
Wei Li, Peijin Jia, Yuan Ma et al. (+9 more)
Vision-
Language-Action (VLA) models have achieved impressive results in visuomotor policy learning, yet remain fundamentally reactive, mapping current observations and
language to actions without explicit forward prediction of world dynamics. Existing visual foresight methods predict future visual states but lack explicit motion guidance: they show where to go but not how to get there. We argue that future feature prediction and sparse point tracking are naturally complementary: the former provides the goal state, while the latter captures the continuous motion path toward it. We propose FoMoVLA, a framework that augments VLA representations with explicit spatio-temporal supervision by jointly learning future feature foresight and sparse 2D point tracking, enhancing the continuous action policy. FoMoVLA introduces compact foresight
tokens to decode future feature states, decodes sparse temporal 2D point trajectories to model compact
geometric motion, and couples both through a lightweight future-conditioned cross-
attention module that enables consistent
reasoning between anticipated states and point dynamics. Extensive experiments on LIBERO, RoboCasa GR-1 Tabletop, and LIBERO-Plus demonstrate state-of-the-art performance and strong zero-shot generalization. Project page is available at https://liauto-research.github.io/FoMoVLA.
Read the full abstract ▾
All authors Wei Li, Peijin Jia, Yuan Ma, Xuefeng Jiang, Titong Jiang, Sheng Sun, Yujian Li, Xin Wen, Han Hong, Zhikang Liu, Bailin Li, Kun Zhan
Computer Vision
Afsaneh Hasanebrahimi, Hanxun Huang, Christopher Leckie et al. (+2 more)
Vision-
language pre-trained models (VLPs) are widely used in real-world applications. However, they remain vulnerable to
adversarial attacks. Although
adversarial detection methods have demonstrated success in single-modality settings (either
vision or
language), their effectiveness and reliability in
multimodal models such as VLPs remain largely unexplored. In this work, we study the geometry of VLP
embedding spaces and observe structured anisotropy that differs from unimodal
vision models. Our theoretical analysis shows that under this anisotropic structure,
adversarial attacks increase the expected
geometric separation between clean and
adversarial examples (AEs). Specifically, we demonstrate that AEs consistently exhibit greater expected distances to randomly sampled points than their clean counterparts, indicating that AEs tend to push representations out of
manifold regions. Building on these insights, we propose GeoDetect, which leverages these off-
manifold deviations via
geometric scores to identify AEs. Through comprehensive evaluations, we show that our approach reliably detects AEs across diverse VLP architectures and threat settings, covering unimodal and
multimodal attacks as well as adaptive attacks, thereby providing a robust and practical approach to improving the safety and reliability of these models.
Read the full abstract ▾
All authors Afsaneh Hasanebrahimi, Hanxun Huang, Christopher Leckie, James Bailey, Sarah Erfani