Computer Vision · Cross-listed from cs.RO
Robotic Policy Adaptation via Weight-Space Meta-Learning
Christian Bianchi, Siamak Yousefi, Alessio Sampieri et al. (+4 more)
Vision-
Language-Action (VLA) models are emerging as a promising paradigm for robotic manipulation, enabling general-purpose policies trained from large corpora of demonstrations and action labels. However, adapting these models to new tasks still typically requires task-specific demonstrations, action annotations, and additional fine-tuning, making deployment costly and difficult to scale. We propose WIZARD, a weight-space meta-learning framework that sidesteps task-specific fine-tuning by generating task-specific LoRA parameters for a frozen VLA policy. Given only a
language instruction and a short demonstration
video, WIZARD predicts the corresponding adaptation weights in a single forward pass, without target-task action labels or test-time optimization. During meta-training, WIZARD learns to map task evidence directly to expert LoRA updates, capturing relationships between tasks in weight space. Experiments on LIBERO show that WIZARD improves performance by up to ~2x on unseen
dataset collections and up to ~14x on unseen tasks. On a Franka Emika Panda, WIZARD consistently improves over a real-domain adapted baseline, showing that generated adapters provide task-level specialization beyond simulation.
Read the full abstract ▾
All authors Christian Bianchi, Siamak Yousefi, Alessio Sampieri, Andrea Roberti, Luca Rigazio, Fabio Galasso, Luca Franco
Computer Vision
AdaTok: Self-Budgeting Image Tokenization with Quality-Preserving Dynamic Tokens
Xiaocheng Lu, Yuxi Chen, Jie Zhang et al. (+5 more)
Image tokenizers, from 2D grids to recent 1D sequences, typically encode every image with the same fixed number of
tokens. Yet visual complexity is highly heterogeneous, so a uniform budget overspends on simple inputs and underserves complex ones. Existing elastic tokenizers expose variable-length reconstructions, but often leave
token length as a deployment-time operating point, a search target, or an external prediction rather than an output of the tokenizer itself. In this work, we ask whether a discrete visual tokenizer can budget itself in one pass. Our central finding is that actionable elasticity requires a representation–allocation co-design: prefixes must remain decodable across budgets, and the tokenizer must learn which prefix each image needs. We propose AdaTok, a self-budgeting discrete 1D tokenizer. AdaTok combines Prioritized Representation Learning, which orders
tokens with nested tail masking and resolves budget-dependent semantic shift through Multi-Head LoRA decoder heads, with Adaptive
Token Allocation, which trains a lightweight deterministic-group GRPO policy over candidate budgets. Dynamic Pareto Weighting balances fidelity and efficiency during policy training without manual trade-off sweeps. On ImageNet-1K, AdaTok-Full reaches rFID 1.31 at 256
tokens, while AdaTok-Adaptive attains rFID 1.50 using only ~118
tokens on average, outperforming discrete 1D baselines at comparable budgets. In autoregressive image generation, the shorter adaptive representation yields ~2.1x throughput over a fixed 256-
token decode, suggesting that visual
token count can be learned as a content-conditioned output rather than set as a fixed hyperparameter.
Read the full abstract ▾
All authors Xiaocheng Lu, Yuxi Chen, Jie Zhang, Jian Liu, Jingcai Guo, Fangqi Zhu, Tao Han, Song Guo
Computer Vision
OPTIMUS-Prime: Minimal and Sufficient Concept Explanations for Deep Vision Models
Arthur Hoarau, Chenrui Zhu and Vu Linh Nguyen
The growing demand for transparency in automated decision-making has propelled eXplainable Artificial Intelligence (XAI) to the forefront of machine learning research. In computer
vision, however, existing explanation methods often prioritize end-user accessibility at the expense of formal guarantees, leaving a critical gap between practical utility and theoretical rigor. In this paper, we address this gap by introducing OPTIMUS, a novel framework for generating concept-based visual explanations for deep classification models. OPTIMUS explanations take the form of visual heatmaps that not only remain interpretable to end users, but are grounded in the well-established theory of
prime implicants, providing formal guarantees that have been largely absent from existing saliency-based methods. Specifically, OPTIMUS explanations satisfy two desirable properties: sufficiency, ensuring that the highlighted concepts provably guarantee the classifier's prediction, and minimality, ensuring that no strict subset of those concepts retains this guarantee. Together, these properties yield explanations that are both logically tight and visually coherent. We validate our approach on a visual classification
benchmark, demonstrating that OPTIMUS heatmaps naturally and faithfully surface the decision-relevant concepts underlying model predictions.
Read the full abstract ▾
Computer Vision
EvoGS: Constructing Continuous-Layered Gaussian Splatting with Evolution Tree for Scalable 3D Streaming
Yuang Shi, Simone Gasparini, Géraldine Morin et al. (+1 more)
Streaming 3D Gaussian Splatting requires highly scalable, progressive representations. Existing progressive methods rely on discrete layering, accumulating separate splat sets for each level of detail. This structural independence between layers inherently leads to error accumulation, severe splat redundancy, and uncontrolled quality transitions. We propose EvoGS, the first continuous-layering representation. Organized as an
Evolution Tree, EvoGS generates finer details via an explicit, wavelet-inspired parent-child refinement. This empowers child nodes to structurally correct ancestral errors, yield inherently sparse and highly compressible inter-layer signals. Extensive experiments show EvoGS eliminates splat redundancy from over 65% to under 25%. Compared to state-of-the-art baselines, it reduces transmission payload and GPU VRAM footprint by up to 2.4$\times$ and 5.5$\times$, respectively, and achieves smooth quality transitions optimal for real-time adaptive streaming. Project page: https://yuang-ian.github.io/evogs/
Read the full abstract ▾
All authors Yuang Shi, Simone Gasparini, Géraldine Morin, Wei Tsang Ooi
Computer Vision
Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs
Siyuan Xu, Yibing Liu, Peilin Chen et al. (+3 more)
Multimodal large
language models (MLLMs) have raised new
privacy challenges. On the data side, user-provided inputs often include unpredictable sensitive information; while on the downstream task side, model
reasoning depends on rich visual context that may itself be
privacy-sensitive. Existing
privacy protection methods, however, rely on predefined sensitive categories and fixed obfuscation strategies, struggling to tackle such challenges in MLLMs. To address this dilemma, we propose Anchored
Privacy Drifting (APD), a training-free method that drifts
privacy-sensitive elements toward semantically equivalent alternatives while anchoring contextual cues to the source image. To systematically evaluate this dual objective of
privacy protection and contextual preservation, we introduce AdaptShield, a comprehensive
benchmark covering 22
privacy categories, which combines conventional
privacy metrics with MLLM-based assessments of contextual utility. Extensive experiments show that our method achieves balanced improvements in both
privacy sanitization and content retention, with average gains of 10.4% on textual categories and 8.5% under MLLM-based evaluation across four MLLM series, i.e., Qwen2.5, Qwen3, InternVL3, and InternVL3.5.
Read the full abstract ▾
All authors Siyuan Xu, Yibing Liu, Peilin Chen, Yung-Hui Li, Shiqi Wang, Sam Kwong
Computer Vision
Textual Supervision Enhances Geospatial Representations in Vision-Language Models
Marcelo Sartori Locatelli, Fernando Tonucci, Jea Kwon et al. (+5 more)
Geospatial understanding is a critical yet underexplored dimension in the development of machine learning systems for tasks such as image geolocation and spatial
reasoning. In this work, we analyze the geospatial representations acquired by three model families:
vision-only architectures (e.g., ViT),
vision-
language models (e.g., CLIP), and large-scale
multimodal foundation models (e.g., LLaVA, Qwen, and Gemma). By evaluating across image clusters, including people, landmarks, and everyday objects, grouped based on the degree of localizability, we reveal systematic gaps in spatial accuracy and show that textual supervision enhances the learning of geospatial representations. Our findings suggest the role of
language as an effective complementary modality for encoding spatial context and
multimodal learning as a key direction for advancing geospatial AI.
Read the full abstract ▾
All authors Marcelo Sartori Locatelli, Fernando Tonucci, Jea Kwon, Luiz Felipe Vecchietti, Bryan Nathanael Wijaya, Cheng Yaw Low, Virgilio Almeida, Meeyoung Cha
Computer Vision
When Recovery Matters: The Blind Spot of Surrogate Privacy in MLLM Editing
Siyuan Xu, Yibing Liu, Peilin Chen et al. (+3 more)
Multimodal Large
Language Models (MLLMs) enable flexible instruction-driven image editing, but
privacy risks arise when user images expose diverse and user-specific private content. Canonical
privacy protection strategies typically substitute sensitive regions with surrogate content before cloud editing. Yet, the resulting output is often an edited surrogate rather than the desired edited source image, neglecting the local recovery in both design and evaluation scope. To this end, we introduce SPPE (Surrogate-based
Privacy-Preserving Editing), the first recovery-oriented
benchmark covering 36 fine-grained
privacy categories and 65 editing instructions. It defines two complementary tasks: 1) editability assessment, which estimates before cloud interaction whether a surrogate can induce an edit consistent with the original image; and 2) surrogate-to-source edit recovery, which evaluates whether the edited surrogate can be transferred back to the private source with the edit effect preserved. We address each task with a dedicated method: ERMA predicts surrogate editability through instruction-aware
multimodal relation modeling, while \method performs cycle-consistent recovery by using the surrogate editing pair as visual edit evidence and the source image as a source-preserving anchor. Experiments on SPPE and InstructPix2Pix show consistent improvements on both tasks. For editability assessment, ERMA improves over the best-performing baselines by 13.9% in SRCC and 12.3% in PLCC. For surrogate-to-source edit recovery, C2E-S2SER outperforms SOER across all 8 source integrity and edit consistency metrics on SPPE.
Read the full abstract ▾
All authors Siyuan Xu, Yibing Liu, Peilin Chen, Yung-Hui LI, Shiqi Wang, Sam Kwong
Computer Vision
TraRA: Trajectory-level Recognition Aggregation for Video Text Spotting in Urban Surveillance
Duc Tri Tran, Trung Thanh Nguyen, Vijay John et al. (+2 more)
Video Text Spotting (VTS) is essential for urban surveillance and intelligent transportation systems, enabling automated reading of street signs, vehicle markings, and scene text in
video streams. However, reliable recognition remains challenging due to dynamic
video factors common in surveillance scenarios, including motion blur, occlusion, and scale variation, which degrade frame-level recognition. Existing VTS methods typically perform recognition independently on each frame, leading to inconsistent and inaccurate results across sequences. To address these limitations, we propose TraRA (Trajectory-level Recognition Aggregation for VTS), a plug-and-play method that performs trajectory-level text recognition by leveraging temporal and
multimodal consistency. TraRA integrates two key modules: (1) the Temporal Clustering and (2) the
Vision-
Language Aggregation. The former refines noisy trajectories by grouping temporally and visually coherent text instances, while the latter employs a Low-Rank Adaptation-enhanced
Vision-
Language model to fuse visual cues with linguistic context across frames. By aggregating information over entire text trajectories, TraRA achieves robust text recognition even under challenging surveillance conditions. Extensive experiments on four public
benchmarks, including road and urban scene
datasets (RoadText, BOVText, ArTVideo, and ICDAR15), demonstrate that TraRA consistently improves tracking and recognition performance over state-of-the-art VTS methods. The source code is available at https://github.com/trid2912/TraRA.
Read the full abstract ▾
All authors Duc Tri Tran, Trung Thanh Nguyen, Vijay John, Phi Le Nguyen, Yasutomo Kawanishi
Computer Vision
Consistent-Inversion: Reverse Consistency Guidance for Structure-Preserving Visual Editing
Xiaocheng Lu, Jingcai Guo and Song Guo
Text-guided
diffusion models have become effective tools for real-image visual editing, where the edited image must follow a target instruction while preserving editing-irrelevant structure. Most training-free editors rely on inversion: a source image is mapped to a noisy latent trajectory and the terminal latent is reused for target-
prompt denoising. This reuse is useful for preservation, but it also couples source reconstruction and target editing. The resulting trajectory mismatch may either damage background/layout details or over-constrain the intended edit. This paper presents Consistent-Inversion, a training-free reverse consistency guidance framework for structure-preserving visual editing. Instead of treating the inverted source latent as a fixed initialization, Consistent-Inversion checks whether an intermediate target trajectory can be reversed toward the source inversion trajectory under the source
prompt. To make this check well-defined, we construct an auxiliary target-side noise representation, perform source-guided reverse denoising, and use the resulting reverse consistency discrepancy as a correction signal for selected early target denoising steps. The method does not update model parameters, is compatible with inversion-based editors, and introduces only a small inference overhead when applied sparsely. Experiments on PIE-Bench show that Consistent-Inversion improves background and structural fidelity under a unified SD3.5 protocol while maintaining target-
prompt alignment, and compatibility experiments further verify the same correction principle on classical Stable-
Diffusion inversion pipelines.
Read the full abstract ▾
Computer Vision
Native3D: End-to-End 3D Scene Generation via Unified Mesh-Texture Modeling and Semantic Alignment
Yibo Liu, Ziwei Zhang, Haozhou Pang et al. (+3 more)
This paper presents Native3D, the first end-to-end 3D scene generation framework that completely bypasses 2D intermediate representations. Traditional approaches typically require adapting 3D representations to the 2D domain to leverage pre-trained
diffusion models, which inevitably introduces domain adaptation issues including
geometric structural distortion and texture detail degradation. To address these limitations, we design a unified mesh-texture joint representation that simultaneously models both
geometric structures and texture features through a
Transformer-based scene encoder, effectively maintaining spatial relationships and visual consistency among objects within scenes. We further propose the 3D Representation
Alignment Loss (3D REPA Loss), which employs an improved contrastive learning mechanism to align multi-level semantic representations in the latent space, significantly enhancing
geometric and textural fidelity. Experimental results demonstrate that Native3D outperforms existing methods in both generation quality and editing flexibility, providing a novel solution for 3D scene editing.
Read the full abstract ▾
All authors Yibo Liu, Ziwei Zhang, Haozhou Pang, Menghao Li, Lanshan He, Gan Qi
Computer Vision
3DMorph: Single-Image-Guided Local 3D Shape Editing and Morphing
Tobias Preintner, Yunfei Deng, Phillip Müller et al. (+5 more)
Despite recent progress in 3D generation, intuitive editing of existing shapes remains limited. Unlike images, which benefit from well-established inpainting tools, general 3D objects such as meshes still lack simple and effective methods for local shape editing. Existing approaches are often global, domain-specific, require complex user interaction, or focus on appearance (color and texture) rather than geometry. We introduce 3DMorph, a training-free framework for single-image-guided local 3D shape editing and morphing. Given an edited image showing a desired shape modification, our method automatically localizes the relevant 3D region and transfers 2D modifications to 3D while preserving unmodified areas. 3DMorph also enables intermediate shape generation between the original and edited objects, facilitating design exploration. To
benchmark editing quality, we introduce Delta3D, an image-guided local 3D editing
benchmark with paired ground-truth edits. Experimental results show that 3DMorph translates intuitive 2D edits into 3D, outperforming state-of-the-art generative and editing methods.
Read the full abstract ▾
All authors Tobias Preintner, Yunfei Deng, Phillip Müller, Sebastian Illing, Adrian König, Thomas Bäck, Elena Raponi, Niki van Stein
Computer Vision
GP-Adapter: Gaussian Process CLIP-Adapter for Few-Shot Out-of-Distribution Detection
Taisei Saito, Koretaka Ogata and Takafumi Hiroi
We propose GP-Adapter, a training-free framework that augments CLIP (Contrastive
Language-Image Pre-training) with Gaussian Process (GP) uncertainty modeling for few-shot classification and out-of-distribution (OOD) detection. While CLIP achieves strong zero-shot recognition, it yields deterministic similarity scores and offers limited uncertainty information, which is critical under distribution shift and data scarcity. GP-Adapter constructs modality-specific, class-wise one-class GPs on top of frozen CLIP
embeddings using an RBF kernel for image features and a linear kernel for text
prompts and fuses their predictive statistics to produce a variance-aware confidence score for OOD detection. The method requires no fine-tuning of the CLIP backbone and relies only on a small $K$-shot cache and lightweight hyperparameter selection, with
memory cost scaling as $O(CK^2)$ for $C$ classes and $K$ shots. Experiments on ImageNet and multiple OOD
benchmarks show that GP-Adapter provides competitive few-shot performance and consistently improves OOD detection when combined with
prompt-learning baselines, highlighting the complementarity between GP-based uncertainty modeling and
prompt learning. Overall, our results suggest that integrating probabilistic inference with large pre-trained
vision-
language models can improve reliability in low-data and distribution-shifted settings. Code is available at https://github.com/tms-byte/GP-Adapter
Read the full abstract ▾
Computer Vision
LARA: Latent Action Representation Alignment for Vision-Language-Action Models
Mengya Liu, Baoxiong Jia, Jiangyong Huang et al. (+2 more)
Visual-
language action (VLA) models enable
robots to predict actions directly from observations and
language instructions, but their performance depends on large-scale, high-quality data and is limited by the scarcity of real-world
robot action
datasets. To facilitate VLA model learning with abundant unlabeled human
videos, Latent Action Models (LAM) learn latent action representations from visual dynamics to provide additional supervision for VLA learning. However, LAM and VLA are typically trained separately, leaving LAM ungrounded during VLA training and VLA models constrained by frozen LAM representations. To address these issues, we propose Latent Action Representation
Alignment (LARA), a plug-and-play framework that jointly optimizes LAM and VLA via representation
alignment. This enables reciprocal benefits where LAMs learn with action trajectories to avoid spurious visual changes, while VLAs are regularized by forward dynamics learned within LAMs to reduce
hallucinations of functionally ineffective trajectories. We demonstrate LARA versatility and effectiveness for pre-training, post-training enhancement of pre-trained VLA models, and LAM refinement, achieving an average of ~10%, ~5%, and ~15% improvement over 3 simulation and 1 meticulously designed real-world robotic manipulation
benchmarks.
Read the full abstract ▾
All authors Mengya Liu, Baoxiong Jia, Jiangyong Huang, Jingze Zhang, Siyuan Huang
Computer Vision
Detecting Temporally Localized Manipulations in Authentic Video Streams
Okan Umur, Ali Emre Güşlü and Ibrahim Delibasoglu
The rapid advancement of
video editing and generative artificial intelligence technologies has made realistic
video manipulation increasingly accessible. Although existing
datasets have significantly advanced research in deepfake detection, object removal, and
video inpainting, they do not adequately model scenarios in which a short manipulated segment is inserted into an otherwise authentic
video and the original
video continues afterward. In this study, we review representative
datasets from the literature, analyze their characteristics, and discuss their limitations with respect to temporally localized realistic manipulation detection. Based on this analysis, we motivate the need for a new
dataset specifically designed for authentic
videos containing short and highly realistic manipulated intervals. Finally, we evaluate two complementary approaches on our custom-curated test set to establish an initial
benchmark for this challenging scenario. The first employs a linear probe on DINOv3 features, assessed under three thresholding strategies. The second leverages DINOv3 features with a consecutive frame similarity-based method to detect temporal manipulation boundaries. Together, these experiments provide an initial
benchmark for partially manipulated
video detection and highlight the need for content-adaptive thresholding mechanisms. The
dataset, code, and supplementary materials are publicly available at https://github.com/OkanUmur/temporally-localized-
video-manipulation-detection.
Read the full abstract ▾
Computer Vision
An Adaptive Data cleaning Framework for Noisy Label Detection
Chen-Hsuan Fang, Wei-Hsinag Chen, Pin-Hsuan Yu et al. (+2 more)
Deep neural networks (DNNs) excel in computer
vision tasks given large annotated
datasets. In real-world applications, however, labels are often corrupted by ambiguity, human error, or dynamic environments. Over-parameterized DNNs easily memorize these noisy labels during training, degrading model accuracy and generalization. Existing data-cleaning and sample-selection strategies often rely on manually specified thresholds, prior knowledge of the noise ratio, or a single metric (either learning dynamics or
geometric structure), making them unstable in complex data regimes. This paper proposes a self-adaptive data-cleaning framework that integrates local, global, and learning dynamics cues for robust noisy-label detection. Samples are mapped into a unified low-dimensional feature space through a modular feature concatenation paradigm. We provide two instantiations: a 2D metric integrating class-adaptive KNN-based local disagreement with k-means-based global centroid distance, and a 3D multi-metric that additionally incorporates a z-normalized score. Unlike conventional 1D Gaussian Mixture Models applied to a single scalar metric, our framework performs multi-metric clustering on the feature space to adaptively partition samples into clean-dominant and noise-dominant components without requiring manual thresholds or noise priors. Experiments on CIFAR-10, MNIST, and ImageNet-100 with 5% to 40% symmetric label noise show high recall across settings, including near-perfect recall (>=98%) on ImageNet-100 at 40% noise. Subsequent training yields accuracy gains across evaluated settings, especially under severe corruption on ImageNet-100. These findings suggest that multi-metric integration provides a threshold-free, practical, and low-tuning strategy for noisy label detection.
Read the full abstract ▾
All authors Chen-Hsuan Fang, Wei-Hsinag Chen, Pin-Hsuan Yu, Jung-Hua Wang, Tsung-Wei Pan
Computer Vision
AsyncPatch Diffusion: spatially-flexible image generation
Samuele Papa, Valentin De Bortoli, Guillaume Couairon et al. (+3 more)
Standard
diffusion models corrupt an entire sample with a single shared noise level, forcing all spatial regions to follow the same denoising trajectory. We introduce AsyncPatch
Diffusion, a joint-
diffusion framework that assigns distinct noise levels to different input dimensions, such as image pixels, or latent
tokens. We show how this asynchronous corruption defines a valid generative process while supporting a richer family of spatially heterogeneous denoising trajectories, and prove the first valid ELBO for this process. We show that a single pretrained model can perform spatially adaptive generation, where different regions are denoised on different schedules. A key challenge is training: naive independent noise-level sampling overemphasizes highly heterogeneous configurations and underrepresents homogeneous noise levels, that are crucial during sampling. We address this with a controlled noise-level sampler that regulates both the average corruption level and its spatial variability. AsyncPatch achieves generation quality comparable to conventional
diffusion on ImageNet 256 and LSUN, while being natively suited for inpainting without task-specific fine-tuning. We further introduce input guidance, which uses clean or partially corrupted regions to guide the generation of unknown regions, improving local consistency and texture matching. Finally, we demonstrate adaptive generation strategies including uncertainty-guided acceleration and autoregressive sampling.
Read the full abstract ▾
All authors Samuele Papa, Valentin De Bortoli, Guillaume Couairon, Daniel Sýkora, Romuald Elie, Klaus Greff
Computer Vision · Cross-listed from eess.IV
Beyond Universality: The GCC-FER Dataset and Culture-Aware Adaptation for Dynamic Facial Expression Recognition
Sonalika Singh, Jyotirindra Dandapat, Avishi Razdan et al. (+3 more)
Dynamic Facial Expression Recognition (DFER) is a key enabling technology in affective computing, human-computer interaction, and intelligent multimedia systems. Despite the significant influence of cultural nuances on FER performance, most existing FER systems assume that emotional expressions are universally consistent across populations. This variation can be attributed to systematic differences in facial muscle activation patterns across cultures. A major challenge in advancing cross-cultural FER lies in the scarcity of culturally diverse
benchmark datasets. To address this, a new hybrid multicultural
video dataset termed Global Cross-Cultural Facial Expression Recognition (GCC-FER) is introduced. GCC-FER comprises 23,934
video samples spanning four cultural groups (African, Caucasian, East Asian, and South Asian) across seven basic expressions, combining psychologically supervised in-house data collection for underrepresented populations with rigorous ethnicity filtering of existing sources. To the best of our knowledge, GCC-FER is the first large-scale global cross-cultural DFER
dataset designed to address these demographic gaps. Leveraging this
dataset, behaviorally grounded cultural priors are derived for each cultural group and a global prior for practical deployment. A Culture-Aware FER (CA-FER) system is proposed to mitigate cultural bias by adaptively recalibrating latent facial representations. Extensive experiments on GCC-FER and DFEW demonstrate that the proposed system consistently improves FER performance across multicultural settings.
Read the full abstract ▾
All authors Sonalika Singh, Jyotirindra Dandapat, Avishi Razdan, Kshipra V. Moghe, Puneet Gupta, Lalan Kumar
Computer Vision · Cross-listed from cs.LG
Constructing VAE Latent Spaces with Prescribed Topology
Jilles S. van Hulst, Jakub M. Tomczak, W.P.M.H. Heemels et al. (+1 more)
Variational autoencoders (VAEs) learn low-dimensional latent representations of high-dimensional data. When the data lies on a
manifold with non-Euclidean
topology, the standard Gaussian prior introduces a topological mismatch that degrades reconstruction quality and prevents faithful representation. We present a constructive mathematical framework that resolves this mismatch for all
manifolds that admit a product covering space. These are
manifolds expressible as products of elementary factors (circles, intervals, or lines) or as quotients of such products by a finite
symmetry group. The class includes cylinders,
tori, Möbius strips, Klein bottles, and real projective spaces. Factorized distributions over the elementary factors yield product
topologies with closed-form, decoupled KL divergences, so that each latent factor can be shaped independently while keeping training tractable. We catalogue reparametrizable encoder-prior pairs for periodic, bounded, and unbounded supports, and provide coordinate transformations that allow standard neural networks to output non-Euclidean parameters with smooth gradients. For quotient
manifolds, the decoder receives group-invariant features of the covering-space coordinates, so that identified points produce identical outputs. Anchor constraints fix the coordinate system relative to the data or create soft topological holes. Experiments on synthetic
manifolds and real-image
datasets (rotated and cyclically shifted MNIST) confirm that a
topology-matched prior aligns KL regularization with the data
manifold. The resulting
topology-aware models outperform the Gaussian baseline at all practically relevant regularization strengths. The code is available at https://github.com/JvHulst/VAE-
Topology.
Read the full abstract ▾
All authors Jilles S. van Hulst, Jakub M. Tomczak, W.P.M.H. Heemels, Duarte J. Antunes
Computer Vision
TrioPose: Native Triple-Stream Diffusion Transformers for Pose-Guided Text-to-Image Generation
Dian Gu and Zhengyi Yang
Pose-guided text-to-image generation often suffers from limb distortions and feature crosstalk in complex multi-person scenarios. While existing UNet-based adapters struggle with long-range spatial dependencies, emerging
Multimodal Diffusion Transformers (MM-DiTs) offer superior global modeling. However, naive signal concatenation in MM-DiTs severely disrupts pre-trained latent distributions. To address this, we propose TrioPose, a native pose-driven framework built upon the SD3.5M architecture. Specifically, we introduce a Triple-Stream Pose-Aware DiT (TSPA-DiT) that treats pose as an independent modality. It employs layer-wise activation and zero-initialized dual-residual injection to smoothly enforce
geometric constraints while preserving pre-trained latent stability. To resolve severe multi-instance occlusions, we design a Learnable Relational Bias Mask that categorizes topological connectivity into fine-grained physical states, mapping them into continuous
attention soft constraints to effectively decouple inter-instance interference. Furthermore, a Pose-Guided Spatial Loss Weighting strategy modulates the native
diffusion objective using heatmap-derived error maps, focusing anatomical supervision strictly on distortion-prone regions. Extensive experiments demonstrate that TrioPose achieves state-of-the-art performance across challenging
benchmarks, including Human-Art, CrowdPose, and OCHuman. Notably, it attains an AP of $64.33$ on Human-Art, representing a $30\%$ improvement over prior arts, while setting new standards for visual fidelity and text-image semantic
alignment in complex multi-human generation.
Read the full abstract ▾
Computer Vision
STREAM: Stochastic Riemannian Flow Matching with Anisotropic Decoder for Digital Histopathology Image Generation
Won June Cho, Daeky Jeong, Hyeongyeol Lim et al. (+1 more)
Synthetic histopathology image generation addresses patient-
privacy concerns and the growing data demands of foundation models. Existing state-of-the-art histopathology generative models use pretrained
Vision Foundation Models (VFMs) as conditioning signals. We show this yields conditioning-dominated diversity: on TCGA-BRCA, 62-75% of their output diversity is attributable to the conditioning signal rather than the learned latent space, while de novo synthesis still requires a VFM at inference. We instead use histopathology VFMs as the latent space itself: their patch
tokens are $\ell_2$-normalized on the unit hypersphere $\mathcal{S}^{d-1}$ with strong angular dominance and intrinsic
curvature, motivating a Riemannian formulation. We present STREAM, the first framework to apply Riemannian flow matching in the histopathology domain, in two stages: 1) a bridge-type
stochastic perturbation that establishes per-
token rectifiability on $\mathcal{S}^{d-1}$ for training a
Diffusion Transformer, and 2) a novel decoder training design whose noise covariance is anisotropic in the left-singular basis of the per-
token tangent-projected velocity-field Jacobian, spending a large robustness budget on its low-response directions and a small one on its high-response directions. Across TCGA-BRCA and TCGA-COADREAD, STREAM achieves state-of-the-art gFID and ranks first on nearly all histopathology-specific metrics as well. Code and a public gallery of generated images are available at https://chokevin8.github.io/STREAM-Patho/.
Read the full abstract ▾
All authors Won June Cho, Daeky Jeong, Hyeongyeol Lim, Hongjun Yoon
Computer Vision
ForensicConcept: Transferable Forensic Concepts for AIGI Detection
Menyanshu Zhou, Ziyin Zhou, Ke Sun et al. (+4 more)
AI-generated image detectors achieve high accuracy on in-distribution data but often fail on unseen generators. A key obstacle to understanding this failure is the black-box nature of current detectors: they do not reveal which evidence drives their decisions. We propose ForensicConcept, a framework that extracts explicit forensic concepts from detectors and enables their transfer across backbones. Our method localizes decision-critical patches via
Transformer attribution, clusters them into a compact concept codebook, and uses a concept-aligned projection to produce auditable evidence readouts. Motivated by prior studies showing that DINO representations can guide
diffusion generation and exhibit concept-level correspondence with
diffusion features, we introduce a generation-trace reference based on CleanDIFT
diffusion features and quantify backbone-trace
alignment via neighborhood-structure consistency (CKNNA). We further propose concept codebook injection to transfer
diffusion-derived concepts into target backbones. Experiments on GenImage, GAN-family, and Chameleon
benchmarks show consistent improvements over prior methods. We also find that CKNNA
alignment predicts transfer effectiveness, providing a principled explanation for why some backbones yield more transferable forensic evidence than others.
Read the full abstract ▾
All authors Menyanshu Zhou, Ziyin Zhou, Ke Sun, Yunpeng Luo, Jiayi Ji, Xiaoshuai Sun, Rongrong Ji
Computer Vision · Cross-listed from cs.AI
Hierarchical Semantic-Constrained Heterogeneous Graph for Audio-Visual Event Localization
Zhe Yang, Ruyi Zhang, Hongtao Chen et al. (+4 more)
Open-vocabulary audio-visual event localization (OV-AVEL) jointly models audio-visual cues to recognize and temporally localize events, including categories unseen during training. Existing methods primarily learn joint audio-visual representations in Euclidean space, but still face two significant challenges. First, the lack of supervision signals for unseen categories makes it difficult to maintain audio-visual consistency across multiple temporal scales. Second, the lack of hierarchical constraints between segment- and
video-level semantics prevents the model from establishing semantic consistency across different levels. To address these challenges, we propose a hierarchical semantic constrained heterogeneous graph (HSCHG) for audio-visual event localization framework. We first construct a heterogeneous hierarchical graph in Euclidean space, which includes audio and visual segment nodes and their corresponding
video-level nodes. We use multi-directional temporal edges to capture complete temporal information within each modality. Simultaneously, we employ a dual-threshold filtering gated fusion strategy, introducing cross-modal information only when the
alignment confidence is high. Furthermore, we introduce bidirectional semantic constraints between segment- and
video-level representations to achieve semantic consistency across different levels. Based on this, we map the multi-level audio-visual representations and text prototypes uniformly into
hyperbolic space. We use a hierarchical entailment regularization loss to characterize the hierarchical relationships between
videos and segments. Extensive experimental results show that our method outperforms existing methods on the OV-AVEL
benchmark. Ablation studies further validate the effectiveness of our method.
Read the full abstract ▾
All authors Zhe Yang, Ruyi Zhang, Hongtao Chen, Wenrui Li, Hengyu Man, Wangmeng Zuo, Xiaopeng Fan
Computer Vision
Never Seen Before: Benchmarking Genuine Zero-Shot Composed Image Retrieval with Consistent Video-Sourced Datasets
Zhenyu Yang, Zemin Du, Shengsheng Qian et al. (+1 more)
Zero-Shot Composed Image
Retrieval (ZS-CIR) aims to retrieve a target image based on a query composed of a reference image and a relative caption without training samples. Existing ZS-CIR
datasets often suffer from complete irrelevance between reference and target images due to noisy image sources, and do not achieve a true zero-shot scenario as they use public image
datasets that models like CLIP have been trained on. To tackle these challenges, we introduce ZeroSight, a novel
benchmark for ZS-CIR. It includes a
dataset with consistent reference-target pairs sourced from
videos, a data construction pipeline, and evaluation methods that consider the ranking of multiple positive and negative target images. We ensure visually and semantically consistent reference-target pairs by extracting frames from a single
video and generating relative captions using LLM-assisted methods. To ensure a true zero-shot scenario, we use
video data published after March 31, 2022, ensuring it was not included in CLIP's pre-training data. Additionally, we propose a training-free MLLM-driven method, SC4CIR (Symmetric Consistency for CIR), which can effectively identify hard negative targets through 3 symmetric consistency checks. This method is plug-and-play, seamlessly integrating with various CIR methods and significantly improving performance. Our experimental results from 27 methods reveal that current ZS-CIR
datasets and evaluation metrics result in inflated
retrieval performance, exaggerating the capabilities of CIR methods. Our
benchmark and models can be accessed at https://github.com/sotayang/ZeroSight.
Read the full abstract ▾
All authors Zhenyu Yang, Zemin Du, Shengsheng Qian, Changsheng Xu
Computer Vision
GuideCAD: A Lightweight Multimodal Framework for 3D CAD Model Generation via Prefix Embedding
Minseong Kim, Jinyeong Park, Sungho Park et al. (+1 more)
Multi-modal approaches used for 3D CAD generation require substantial computational resources, necessitating efficient training. To address this, we propose GuideCAD, which leverages semantically rich visual-textual representations having only a small number of trainable parameters to generate 3D CAD models. Specifically, GuideCAD uses a mapping network that converts image
embeddings into prefix
embeddings, enabling a pretrained large
language model to integrate visual and textual information. As a result, a
transformer-based decoder predicts the construction sequence using the visual-textual
embeddings in order to generate the 3D CAD model. For experimental evaluation, we construct a new
dataset, referred to as GuideCAD, which consists of text-image pairs. Each pair includes a text
prompt that represents a 3D CAD construction sequence and its corresponding 3D CAD image. Our experimental results show that GuideCAD generates comparably high-quality 3D CAD models while using approximately four times fewer parameters and achieving twice the training efficiency compared to fine-tuning approaches. We have released the source code and
dataset for our method at: https://github.com/mskimS2/GuideCAD
Read the full abstract ▾
All authors Minseong Kim, Jinyeong Park, Sungho Park, Jibum Kim
Computer Vision · Cross-listed from stat.AP
An Integrated Roadside Sensing and Communication Framework for Vulnerable Road User Safety at Signalized Intersections
Parvez Anowar
Vulnerable road users (VRUs) account for approximately half of urban traffic deaths globally, with intersections concentrating a disproportionate share of these casualties. Recent reviews of sensing technology for VRU protection have cataloged dozens of single-sensor and dual-sensor deployments, yet none of the surveyed systems couples multi-modal sensing with edge-side near-miss analytics and bidirectional vehicle-to-everything (V2X) and pedestrian-to-everything (P2X) messaging in a single intersection cabinet. This paper presents an integrated framework for VRU protection at signalized intersections, combining
LiDAR, radar, RGB camera, and thermal camera at the perception layer, edge-based prediction and surrogate-safety analytics at the computation layer, V2X and P2X messaging at the communication layer, and adaptive signal control at the actuation layer. The framework is grounded in an empirical case study using R-LiViT, the first publicly released roadside
LiDAR-Visual-Thermal
dataset, which provides 200 multi-modal sequences and 2,400 annotated RGB-T frames at three German intersections. Analysis of 53,319 detection annotations reveals that VRUs comprise approximately 49% of all road-user observations, that day-to-night density drops by 38% for pedestrians and 45% for vehicles while the night distribution shows a higher close-proximity share, that per-frame close-proximity event counts vary approximately 10-fold across the eight unique locations at three intersections, and that 83% of pedestrian bounding boxes are small in image space, indicating that VRUs are typically far from any single sensor. These findings support multi-modal sensing, edge-side analytics, and adaptive context-sensitive deployment rather than uniform single-sensor solutions.
Read the full abstract ▾
Computer Vision
Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding
Zhenyu Yang, Kairui Zhang, Shengsheng Qian et al. (+2 more)
Online
Video Large
Language Models (
Video-LLMs) have advanced toward seamless human-AI interaction through frame-by-frame processing and proactive responding. However, a critical challenge remains in streaming scenarios: existing models typically pause
video perception while generating responses, breaking real-time
video-
language synchrony and causing stutters. To address this, we introduce a novel paradigm for online
video understanding: Streaming
Video-
Language Synchrony (SVLS), and present LyraV, a live streaming assistant built upon a hierarchical control framework with two core innovations. First, the Frame-Driven Transition Controller (FDTC), a training-free verification-based finite-state machine, makes high-level semantic decisions on when to continue speaking, start a new response, or stay silent. Second, the Streaming
Token Pacer (SToP), a plug-and-play lightweight predictive module, dynamically adapts the
language generation rate to match the pace of the visual content. Concretely, LyraV performs per-frame incremental, sub-budget decoding: within each frame interval it emits only a small chunk of
tokens that fits the real-time budget, so perception is never blocked for a full sentence. Together, these components enable LyraV to seamlessly interleave incoming
video frames with generated word
tokens, achieving a fine-grained synchrony. Extensive experiments conducted on five online and three offline
benchmarks demonstrate that LyraV preserves the backbone's general understanding ability while substantially improving streaming synchrony and narrative fluency, delivering a 98.29% synchrony with
video playback and a real-time processing speed of 3.89 FPS. Interestingly, we observe an empirical capability in LyraV: dynamic
reasoning over streaming
tokens, enabling continuous interpretation and "thinking" alongside visual input.
Read the full abstract ▾
All authors Zhenyu Yang, Kairui Zhang, Shengsheng Qian, Weiming Dong, Changsheng Xu
Computer Vision · Cross-listed from eess.IV
DaX: Learning General Pathology Representations Across Scales
Bokai Zhao, Yiyang Zhang, Long Bai et al. (+3 more)
Computational pathology requires visual representations that transfer across diverse clinical endpoints and remain robust to variation in magnification, staining, scanner type, slide preparation, and input resolution. We present DaX, a pathology
vision foundation model that adapts DINOv3-style self-supervised learning to whole-slide histopathology. DaX is initialized from natural-image DINOv3 weights and incorporates continuous magnification training, cross-scale tissue views, orientation-agnostic and acquisition-robust augmentation, multi-input-size training, and Gram-anchored dense consistency. These designs aim to connect local cellular morphology with global tissue architecture while stabilizing dense
token-level representations across input scales. We further construct a WSI-level
benchmark comprising 161 clinically meaningful tasks from 44 public
datasets, covering 28,182 patients and 34,394 slides across four clinical domains and nine task categories. All models are evaluated under a fixed patient-level cross-validation protocol with fold-level statistical ranking, enabling reproducible comparisons that are less sensitive to split-dependent variation. Across this
benchmark, DaX achieves the highest mean performance across tasks and consistently strong task-level ranking scores, with gains spanning diagnostic pathology, biomarker and molecular profiling, tissue/specimen context, and risk, response, and prognosis. These results support DaX as a transferable visual encoder for computational pathology and provide a standardized evaluation framework for future pathology foundation models. Project page: https://alibaba-damo-academy.github.io/DaX/benchboard/.
Read the full abstract ▾
All authors Bokai Zhao, Yiyang Zhang, Long Bai, Tai Ma, Hanqing Chao, Minfeng Xu
Computer Vision
CL-CLIP: CLIP-Based Continual Learning Framework with Cost-Volume Category Decoupling for Object Detection
Zihan Liu, Yuguang Yang, Shengjie Su et al. (+5 more)
Continual Object Detection (COD) requires a detector to acquire new categories over time while preserving previously learned ones. This goal is closely related to open-vocabulary detection, since both settings require
reasoning over categories that are not fully covered by the annotations available at the current training stage. Recent CLIP-based open-vocabulary detectors have shown strong zero-shot generalization, and frameworks such as F-ViT demonstrate that
vision-
language pretraining can provide powerful zero-shot detection ability for unseen categories. However, real-world deployments cannot remain purely zero-shot: once these detectors are continually updated on newly introduced categories, they suffer severe catastrophic forgetting and quickly lose their previously calibrated detection ability. We therefore propose CL-CLIP, a CLIP-based COD framework that equips open-vocabulary detectors with better continual learning ability through cost-volume-guided category decoupling. Specifically, following CAT-Seg, we compute a CLIP image-text similarity cost volume, defined as dense category-wise response maps between visual
tokens and class text
embeddings. This zero-shot spatial prior decomposes shared region features into class-specific pathways, which are then processed by a Multi-Expert RoI head. Extensive experiments on PASCAL VOC and MS-COCO show that CL-CLIP substantially improves the F-ViT baseline under continual fine-tuning and achieves competitive performance with existing continual object detectors, especially in adapting to newly introduced categories while preserving competitive base-class performance.
Read the full abstract ▾
All authors Zihan Liu, Yuguang Yang, Shengjie Su, Jianing Pang, Linlin Yang, Chunyu Xie, Nikolai Yu. Zolotykh, Baochang Zhang
Computer Vision
From Vision to Text: A Compact Multimodal Approach for Robust, Cross-Domain Presentation Attack Detection on ID Cards
Qingwen Zeng, Juan E. Tapia, Sneha Das et al. (+1 more)
Cross-domain shifts challenge Presentation Attack Detection (PAD) on ID Cards, given the restricted data available due to
privacy concerns. This work proposes a compact
multimodal model, based on new generative and discriminative blocks, which combines visual and textual data for PAD on genuine and synthetic ID images. While
multimodal models exhibit strong generalisation after supervised fine-tuning, they fail in zero-shot settings. Our findings underscore that model capacity and real-world data are essential for reliable PAD, while existing synthetic
datasets may not reflect real-world challenges. We argue for a re-evaluation of synthetic data as a
benchmark and emphasise the need for more realistic, diverse
datasets to advance PAD research.
Read the full abstract ▾
All authors Qingwen Zeng, Juan E. Tapia, Sneha Das, Christoph Busch
Computer Vision
MVSegNet: A Lightweight Boundary-Aware Network for Fetal Lateral Ventricle Segmentation and Atrial Width Estimation in Prenatal Ultrasound
Arafat Hossain Sayem
Fetal ventriculomegaly is assessed by measuring the atrial width of the lateral ventricle in prenatal ultrasound. Accurate
segmentation is essential for this measurement, but acoustic shadowing, speckle noise, and poor contrast make it difficult. We developed MVSegNet, a lightweight encoder-decoder network combining multi-scale feature extraction and boundary-aware refinement. The model was trained and evaluated on 584 expert-annotated transventricular ultrasound frames using a 70/15/15 split. Performance was compared against six
segmentation baselines using overlap, boundary, and measurement metrics. MVSegNet achieved a Dice score of 80.79%, IoU of 68.47%, Hausdorff distance of 4.07 mm, and atrial width mean absolute error of 3.40 mm. The model contains 2.31 million parameters and runs at 165.6 frames per second on an NVIDIA T4 GPU. MVSegNet outperformed all evaluated baselines on boundary and measurement metrics while maintaining low computational cost, supporting its use in automated fetal ultrasound analysis.
Read the full abstract ▾