Computer Vision
SCHEMA for Gemini 3 Pro Image: A Structured Methodology for Controlled AI Image Generation on Google's Native Multimodal Model
Luca Cazzaniga
This paper presents SCHEMA (Structured Components for Harmonized Engineered Modular Architecture), a structured
prompt engineering methodology specifically developed for Google Gemini 3 Pro Image. Unlike generic
prompt guidelines or model-agnostic tips, SCHEMA is an engineered framework built on systematic professional practice encompassing 850 verified API predictions within an estimated corpus of approximately 4,800 generated images, spanning six professional domains: real estate photography, commercial product photography, editorial content, storyboards, commercial campaigns, and information design. The methodology introduces a three-tier progressive system (BASE, MEDIO, AVANZATO) that scales practitioner control from exploratory (approximately 5%) to directive (approximately 95%), a modular label architecture with 7 core and 5 optional structured components, a decision tree with explicit routing rules to alternative tools, and systematically documented model limitations with corresponding workarounds. Key findings include an observed 91% Mandatory compliance rate and 94% Prohibitions compliance rate across 621 structured
prompts, a comparative batch consistency test demonstrating substantially higher inter-generation coherence for structured
prompts, independent practitioner validation (n=40), and a dedicated Information Design validation demonstrating >95% first-generation compliance for spatial and typographical control across approximately 300 publicly verifiable infographics. Previously published on Zenodo (doi:10.5281/zenodo.18721380).
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CR
PrivacyBench: Privacy Isn't Free in Hybrid Privacy-Preserving Vision Systems
Nnaemeka Obiefuna, Samuel Oyeneye, Similoluwa Odunaiya et al. (+2 more)
Privacy preserving machine learning deployments in sensitive deep learning applications; from medical imaging to autonomous systems; increasingly require combining multiple techniques. Yet, practitioners lack systematic guidance to assess the synergistic and non-additive interactions of these hybrid configurations, relying instead on isolated technique analysis that misses critical system level interactions. We introduce PrivacyBench, a benchmarking framework that reveals striking failures in
privacy technique combinations with severe deployment implications. Through systematic evaluation across ResNet18 and ViT models on medical
datasets, we uncover that FL + DP combinations exhibit severe convergence failure, with accuracy dropping from 98% to 13% while compute costs and energy consumption substantially increase. In contrast, FL + SMPC maintains near-baseline performance with modest overhead. Our framework provides the first systematic platform for evaluating
privacy-utility-cost trade-offs through automated YAML configuration, resource monitoring, and reproducible experimental protocols. PrivacyBench enables practitioners to identify problematic technique interactions before deployment, moving
privacy-preserving computer
vision from ad-hoc evaluation toward principled systems design. These findings demonstrate that
privacy techniques cannot be composed arbitrarily and provide critical guidance for robust deployment in resource-constrained environments.
Read the full abstract ▾
All authors Nnaemeka Obiefuna, Samuel Oyeneye, Similoluwa Odunaiya, Iremide Oyelaja, Steven Kolawole
Computer Vision
Beyond Stationarity: Rethinking Codebook Collapse in Vector Quantization
Hao Lu, Onur C. Koyun, Yongxin Guo et al. (+3 more)
Vector Quantization (VQ) underpins many modern generative frameworks such as VQ-VAE, VQ-GAN, and latent
diffusion models. Yet, it suffers from the persistent problem of codebook collapse, where a large fraction of code vectors remains unused during training. This work provides a new theoretical explanation by identifying the nonstationary nature of encoder updates as the fundamental cause of this phenomenon. We show that as the encoder drifts, unselected code vectors fail to receive updates and gradually become inactive. To address this, we propose two new methods: Non-Stationary Vector Quantization (NSVQ), which propagates encoder drift to non-selected codes through a kernel-based rule, and
Transformer-based Vector Quantization (TransVQ), which employs a lightweight mapping to adaptively transform the entire codebook while preserving convergence to the k-means solution. Experiments on the CelebA-HQ
dataset demonstrate that both methods achieve near-complete codebook utilization and superior reconstruction quality compared to baseline VQ variants, providing a principled and scalable foundation for future VQ-based generative models. The code is available at: https://github.com/CAIR- LAB- WFUSM/NSVQ-TransVQ.git
Read the full abstract ▾
All authors Hao Lu, Onur C. Koyun, Yongxin Guo, Zhengjie Zhu, Abbas Alili, Metin Nafi Gurcan
Computer Vision
SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse World
Jungho Kim, Jiyong Oh, Seunghoon Yu et al. (+3 more)
The end-to-end (E2E) paradigm, which maps sensor inputs directly to driving decisions, has recently attracted significant
attention due to its unified modeling capability and scalability. However, ensuring safety in this unified framework remains one of the most critical challenges. In this work, we propose SafeDrive, an E2E planning framework designed to perform explicit and interpretable safety
reasoning through a trajectory-conditioned Sparse World Model. SafeDrive comprises two complementary networks: the Sparse World Network (SWNet) and the Fine-grained
Reasoning Network (FRNet). SWNet constructs trajectory-conditioned sparse worlds that simulate the future behaviors of critical dynamic
agents and road entities, providing interaction-centric representations for downstream
reasoning. FRNet then evaluates
agent-specific collision risks and temporal adherence to drivable regions, enabling precise identification of safety-critical events across future timesteps. SafeDrive achieves state-of-the-art performance on both open-loop and closed-loop
benchmarks. On NAVSIM, it records a PDMS of 91.6 and an EPDMS of 87.5, with only 61 collisions out of 12,146 scenarios (0.5%). On Bench2Drive, SafeDrive attains a 66.8% driving score.
Read the full abstract ▾
All authors Jungho Kim, Jiyong Oh, Seunghoon Yu, Hongjae Shin, Donghyuk Kwak, Jun Won Choi
Computer Vision
PhysConvex: Physics-Informed 3D Dynamic Convex Radiance Fields for Reconstruction and Simulation
Dan Wang, Xinrui Cui, Serge Belongie et al. (+1 more)
Reconstructing and simulating dynamic 3D scenes with both visual realism and physical consistency remains a fundamental challenge. Existing neural representations, such as NeRFs and 3DGS, excel in appearance reconstruction but struggle to capture complex material deformation and dynamics. We propose PhysConvex, a Physics-informed 3D Dynamic Convex Radiance Field that unifies visual rendering and physical simulation. PhysConvex represents deformable radiance fields using physically grounded convex primitives governed by continuum mechanics. We introduce a boundary-driven dynamic convex representation that models deformation through vertex and surface dynamics, capturing spatially adaptive, non-uniform deformation, and evolving boundaries. To efficiently simulate complex geometries and heterogeneous materials, we further develop a reduced-order convex simulation that advects dynamic convex fields using neural skinning eigenmodes as shape- and material-aware deformation bases with time-varying reduced DOFs under Newtonian dynamics. Convex dynamics also offers compact, gap-free volumetric coverage, enhancing both
geometric efficiency and simulation fidelity. Experiments demonstrate that PhysConvex achieves high-fidelity reconstruction of geometry, appearance, and physical properties from
videos, outperforming existing methods.
Read the full abstract ▾
All authors Dan Wang, Xinrui Cui, Serge Belongie, Ravi Ramamoorthi
Computer Vision · Cross-listed from astro-ph.GA
Characterization of Residual Morphological Substructure Using Supervised and Unsupervised Deep Learning
Kameswara Bharadwaj Mantha, Daniel H. McIntosh, Cody Ciaschi et al. (+9 more)
Automated characterization of
galactic substructure is an essential step in understanding the transformative physical processes driving
galaxy evolution. In this study, we investigate the application of deep learning (DL) frameworks to characterize different
galactic substructures hosted within parametric light-profile subtracted “residual” images of a large sample
galaxies from the CANDELS survey. We develop a supervised Convolutional Neural Network (CNN) and unsupervised Convolutional Variational Autoencoder (CvAE) and train it on the single-Sérsic profile fitting based residual images of $10,046$ bright and massive
galaxies ($H<24.5\,{\rm mag}$ and $M_{\rm stellar} \geq 10^{9.5}\,M_{\odot}$) spanning $1<z<3$, in conjunction with their visual-based classification labels indicating the nature of residual substructures hosted within them. Using our unique data preprocessing approach, we prepare our residual images such that the inputs to our DL networks comprise only “
galaxy of interest”, and augment them such that our sample span uniformly across different residual characteristics. We assess the latent space of the CNN and CvAE using Principle Component Analysis (PCA) along with independently quantified metrics of residual strength (significant pixel flux $SPF$, Bumpiness, and Residual Flux Fraction). We also employ an unsupervised Gaussian Mixture Modeling (GMM) based clustering scheme with Support Vector Classification (SVC) to identify groupings in PCA space that correspond to similar residual substructure. We find that our supervised CNN latent features in PCA space correlate with the $SPF$ values and distinguish between qualitatively strong and weak residual substructures. While our unsupervised CvAE latent space also correlates with visual and quantitative residual characteristics, but lacks clear discriminatory power when characterizing different residual substructures.
Read the full abstract ▾
All authors Kameswara Bharadwaj Mantha, Daniel H. McIntosh, Cody Ciaschi, Rubyet Evan, Luther Landry, Henry C. Ferguson, Camilla Pacifici, Joel Primack, Nimish Hathi, Anton Koekemoer, Yicheng Guo, The CANDELS Collaboration
Computer Vision
SceneTok: A Compressed, Diffusable Token Space for 3D Scenes
Mohammad Asim, Christopher Wewer and Jan Eric Lenssen
We present SceneTok, a novel tokenizer for encoding view sets of scenes into a compressed and diffusable set of unstructured
tokens. Existing approaches for 3D scene representation and generation commonly use 3D data structures or view-aligned fields. In contrast, we introduce the first method that encodes scene information into a small set of permutation-invariant
tokens that is disentangled from the spatial grid. The scene
tokens are predicted by a multi-view tokenizer given many context views and rendered into novel views by employing a light-weight rectified flow decoder. We show that the compression is 1-3 orders of magnitude stronger than for other representations while still reaching state-of-the-art reconstruction quality. Further, our representation can be rendered from novel trajectories, including ones deviating from the input trajectory, and we show that the decoder gracefully handles uncertainty. Finally, the highly-compressed set of unstructured latent scene
tokens enables simple and efficient scene generation in 5 seconds, achieving a much better quality-speed trade-off than previous paradigms.
Read the full abstract ▾
Computer Vision
FOCA: Frequency-Oriented Cross-Domain Forgery Detection, Localization and Explanation via Multi-Modal Large Language Model
Zhou Liu, Tonghua Su, Hongshi Zhang et al. (+4 more)
Advances in image tampering techniques, particularly generative models, pose significant challenges to media verification, digital forensics, and public trust. Existing image forgery detection and localization (IFDL) methods suffer from two key limitations: over-reliance on semantic content while neglecting textural cues, and limited interpretability of subtle low-level tampering traces. To address these issues, we propose FOCA, a
multimodal large
language model-based framework that integrates discriminative features from both the RGB spatial and frequency domains via a cross-
attention fusion module. This design enables accurate forgery detection and localization while providing explicit, human-interpretable cross-domain explanations. We further introduce FSE-Set, a large-scale
dataset with diverse authentic and tampered images, pixel-level masks, and dual-domain annotations. Extensive experiments show that FOCA outperforms state-of-the-art methods in detection performance and interpretability across both spatial and frequency domains.
Read the full abstract ▾
All authors Zhou Liu, Tonghua Su, Hongshi Zhang, Fuxiang Yang, Donglin Di, Yang Song, Lei Fan
Computer Vision
Disentangle to create: Architecture-level disentangled diffusion model for few-shot Chinese font generation
Jie Li, Suorong Yang, Jian Zhao et al. (+1 more)
Few-shot Chinese font generation aims to synthesize new characters in a target style using only a handful of reference images. Achieving accurate content rendering and faithful style transfer requires effective disentanglement between content and style. However, most existing approaches achieve only feature-level disentanglement and allow the generator to re-entangle these features, leading to content distortion and degraded style fidelity. To address this, we propose the Architecture-Level Disentangled
Diffusion Model (ALD-Font) with an asymmetric design. The U-Net backbone takes SimSun-style glyphs as content templates, concatenated with noisy latents to construct a spatial canvas. Style features extracted from target-style images are injected as modulation signals to guide generation. Based on theoretical validation of disentanglement effectiveness, we adopt a parameter-efficient fine-tuning (PEFT) strategy that updates only the style-related modules. This allows the model to better adapt to new styles while avoiding overfitting to the reference images' content under the few-shot setting. Additionally, we train a Background Noise Removal (BNR) module in the pixel space to remove background noise in complex stroke regions. We further introduce the Grey and OCR metrics to evaluate the content quality of generated characters. Experimental results show that ALD-Font achieves significantly higher style fidelity while maintaining comparable content accuracy to existing state-of-the-art methods. The implementation is available at https://github.com/JayLee1002/ALD-Font.
Read the full abstract ▾
All authors Jie Li, Suorong Yang, Jian Zhao, Furao Shen
Computer Vision
BiMotion: B-spline Motion for Text-guided Dynamic 3D Character Generation
Miaowei Wang, Qingxuan Yan, Zhi Cao et al. (+4 more)
Text-guided dynamic 3D character generation has advanced rapidly, yet producing high-quality motion that faithfully reflects rich textual descriptions remains challenging. Existing methods tend to generate limited sub-actions or incoherent motion due to fixed-length temporal inputs and discrete frame-wise representations that fail to capture rich motion semantics. We address these limitations by representing motion with continuous differentiable B-spline curves, enabling more effective motion generation without modifying the capabilities of the underlying generative model. Specifically, our closed-form,
Laplacian-regularized B-spline solver efficiently compresses variable-length motion sequences into compact representations with a fixed number of control points. Further, we introduce a normal-fusion strategy for input shape adherence along with correspondence-aware and local-
rigidity losses for motion-restoration quality. To train our model, we collate BIMO, a new
dataset containing diverse variable-length 3D motion sequences with rich, high-quality text annotations. Extensive evaluations show that our feed-forward framework BiMotion generates more expressive, higher-quality, and better
prompt-aligned motions than existing state-of-the-art methods, while also achieving faster generation. Our project page is at: https://wangmiaowei.github.io/BiMotion.github.io/.
Read the full abstract ▾
All authors Miaowei Wang, Qingxuan Yan, Zhi Cao, Yayuan Li, Oisin Mac Aodha, Jason J. Corso, Amir Vaxman
Computer Vision
Enhancing 3D LiDAR Segmentation by Shaping Dense and Accurate 2D Semantic Predictions
Xiaoyu Dong, Tiankui Xian, Wanshui Gan et al. (+1 more)
Semantic
segmentation of 3D
LiDAR point clouds is important in urban remote sensing for understanding real-world street environments. This task, by projecting
LiDAR point clouds and 3D semantic labels as sparse maps, can be reformulated as a 2D problem. However, the intrinsic sparsity of the projected
LiDAR and label maps can result in sparse and inaccurate intermediate 2D semantic predictions, which in return limits the final 3D accuracy. To address this issue, we enhance this task by shaping dense and accurate 2D predictions. Specifically, we develop a multi-modal
segmentation model, MM2D3D. By leveraging camera images as auxiliary data, we introduce cross-modal guided filtering to overcome label map sparsity by constraining intermediate 2D semantic predictions with dense semantic relations derived from the camera images; and we introduce dynamic cross pseudo supervision to overcome
LiDAR map sparsity by encouraging the 2D predictions to emulate the dense distribution of the semantic predictions from the camera images. Experiments show that our techniques enable our model to achieve intermediate 2D semantic predictions with dense distribution and higher accuracy, which effectively enhances the final 3D accuracy. Comparisons with previous methods demonstrate our superior performance in both 2D and 3D spaces.
Read the full abstract ▾
All authors Xiaoyu Dong, Tiankui Xian, Wanshui Gan, Naoto Yokoya
Computer Vision
Similarity-as-Evidence: Calibrating Overconfident VLMs for Interpretable and Label-Efficient Medical Active Learning
Zhuofan Xie, Zishan Lin, Jinliang Lin et al. (+3 more)
Active Learning (AL) reduces annotation costs in medical imaging by selecting only the most informative samples for labeling, but suffers from cold-start when labeled data are scarce.
Vision-
Language Models (VLMs) address the cold-start problem via zero-shot predictions, yet their temperature-scaled softmax outputs treat text-image similarities as deterministic scores while ignoring inherent uncertainty, leading to overconfidence. This overconfidence misleads sample selection, wasting annotation budgets on uninformative cases. To overcome these limitations, the Similarity-as-Evidence (SaE) framework calibrates text-image similarities by introducing a Similarity Evidence Head (SEH), which reinterprets the similarity vector as evidence and parameterizes a Dirichlet distribution over labels. In contrast to a standard softmax that enforces confident predictions even under weak signals, the Dirichlet formulation explicitly quantifies lack of evidence (vacuity) and conflicting evidence (dissonance), thereby mitigating overconfidence caused by rigid softmax normalization. Building on this, SaE employs a dual-factor acquisition strategy: high-vacuity samples (e.g., rare diseases) are prioritized in early rounds to ensure coverage, while high-dissonance samples (e.g., ambiguous diagnoses) are prioritized later to refine boundaries, providing clinically interpretable selection rationales. Experiments on ten public medical imaging
datasets with a 20% label budget show that SaE attains state-of-the-art macro-averaged accuracy of 82.57%. On the representative BTMRI
dataset, SaE also achieves superior calibration, with a negative log-likelihood (NLL) of 0.425.
Read the full abstract ▾
All authors Zhuofan Xie, Zishan Lin, Jinliang Lin, Jie Qi, Shaohua Hong, Shuo Li
Computer Vision · Cross-listed from eess.IV
TIACam: Text-Anchored Invariant Feature Learning with Auto-Augmentation for Camera-Robust Zero-Watermarking
Abdullah All Tanvir, Agnibh Dasgupta and Xin Zhong
Camera recapture introduces complex optical degradations, such as perspective warping, illumination shifts, and Moiré interference, that remain challenging for deep watermarking systems. We present TIACam, a text-anchored invariant feature learning framework with auto-augmentation for camera-robust zero-watermarking. The method integrates three key innovations: (1) a learnable auto-augmentor that discovers camera-like distortions through differentiable
geometric, photometric, and Moiré operators; (2) a text-anchored invariant feature learner that enforces semantic consistency via cross-modal
adversarial alignment between image and text; and (3) a zero-watermarking head that binds binary messages in the invariant feature space without modifying image pixels. This unified formulation jointly optimizes invariance, semantic
alignment, and watermark recoverability. Extensive experiments on both synthetic and real-world camera captures demonstrate that TIACam achieves state-of-the-art feature stability and watermark extraction accuracy, establishing a principled bridge between
multimodal invariance learning and physically robust zero-watermarking.
Read the full abstract ▾
Computer Vision
Joint Post-Training Quantization of Vision Transformers with Learned Prompt-Guided Data Generation
Shile Li, Markus Karmann and Onay Urfalioglu
We present a framework for end-to-end joint quantization of
Vision Transformers trained on ImageNet for the purpose of image classification. Unlike prior post-training or block-wise reconstruction methods, we jointly optimize over the entire set of all layers and inter-block dependencies without any labeled data, scaling effectively with the number of samples and completing in just one hour on a single GPU for ViT-small. We achieve state-of-the-art W4A4 and W3A3 accuracies on ImageNet and, to the best of our knowledge, the first PTQ results that maintain strong accuracy on ViT, DeiT, and Swin-T models under extremely low-bit settings (W1.58A8), demonstrating the potential for efficient edge deployment. Furthermore, we introduce a data-free calibration strategy that synthesizes diverse, label-free samples using Stable
Diffusion Turbo guided by learned multi-mode
prompts. By encouraging diversity in both the learned
prompt embeddings and the generated image features, our data-free approach achieves performance on par with real-data ImageNet calibration and surpasses simple text-
prompt baselines such as "a <adjective> photo of <adjective> <cls>".
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Hyperbolic Busemann Neural Networks
Ziheng Chen, Bernhard Schölkopf and Nicu Sebe
Hyperbolic spaces provide a natural geometry for representing hierarchical and tree-structured data due to their exponential volume growth. To leverage these benefits, neural networks require intrinsic and efficient components that operate directly in
hyperbolic space. In this work, we lift two core components of neural networks, Multinomial Logistic Regression (MLR) and Fully Connected (FC) layers, into
hyperbolic space via Busemann functions, resulting in Busemann MLR (BMLR) and Busemann FC (BFC) layers with a unified mathematical interpretation. BMLR provides compact parameters, a point-to-horosphere distance interpretation, batch-efficient computation, and a Euclidean limit, while BFC generalizes FC and activation layers with comparable complexity. Experiments on image classification, genome sequence learning, node classification, and link prediction demonstrate improvements in effectiveness and efficiency over prior
hyperbolic layers. The code is available at https://github.com/GitZH-Chen/HBNN.
Read the full abstract ▾
Computer Vision
Open-Vocabulary Domain Generalization in Urban-Scene Segmentation
Dong Zhao, Qi Zang, Nan Pu et al. (+3 more)
Domain Generalization in Semantic
Segmentation (DG-SS) aims to enable
segmentation models to perform robustly in unseen environments. However, conventional DG-SS methods are restricted to a fixed set of known categories, limiting their applicability in open-world scenarios. Recent progress in
Vision-
Language Models (VLMs) has advanced Open-Vocabulary Semantic
Segmentation (OV-SS) by enabling models to recognize a broader range of concepts. Yet, these models remain sensitive to domain shifts and struggle to maintain robustness when deployed in unseen environments, a challenge that is particularly severe in urban-driving scenarios. To bridge this gap, we introduce Open-Vocabulary Domain Generalization in Semantic
Segmentation (OVDG-SS), a new setting that jointly addresses unseen domains and unseen categories. We introduce the first
benchmark for OVDG-SS in autonomous driving, addressing a previously unexplored problem and covering both synthetic-to-real and real-to-real generalization across diverse unseen domains and unseen categories. In OVDG-SS, we observe that domain shifts often distort text-image correlations in pre-trained VLMs, which hinders the performance of OV-SS models. To tackle this challenge, we propose S2-Corr, a state-space-driven text-image correlation refinement mechanism that mitigates domain-induced distortions and produces more consistent text-image correlations under distribution changes. Extensive experiments on our constructed
benchmark demonstrate that the proposed method achieves superior cross-domain performance and efficiency compared to existing OV-SS approaches.
Read the full abstract ▾
All authors Dong Zhao, Qi Zang, Nan Pu, Wenjing Li, Nicu Sebe, Zhun Zhong
Computer Vision
DUET-VLM: Dual stage Unified Efficient Token reduction for VLM Training and Inference
Aditya Kumar Singh, Hitesh Kandala, Pratik Prabhanjan Brahma et al. (+2 more)
Vision-
language models (VLMs) have achieved remarkable
multimodal understanding and
reasoning capabilities, yet remain computationally expensive due to dense visual tokenization. Existing efficiency approaches either merge redundant visual
tokens or drop them progressively in
language backbone, often trading accuracy for speed. In this work, we propose DUET-VLM, a versatile plug-and-play dual compression framework that consists of (a)
vision-only redundancy aware compression of
vision encoder's output into information-preserving
tokens, followed by (b) layer-wise, salient text-guided dropping of visual
tokens within the
language backbone to progressively prune less informative
tokens. This coordinated
token management enables aggressive compression while retaining critical semantics. On LLaVA-1.5-7B, our approach maintains over 99% of baseline accuracy with 67% fewer
tokens, and still retains >97% even at 89% reduction. With this dual-stage compression during training, it achieves 99.7% accuracy at 67% and 97.6% at 89%, surpassing prior SoTA visual
token reduction methods across multiple
benchmarks. When integrated into
Video-LLaVA-7B, it even surpasses the baseline – achieving >100% accuracy with a substantial 53.1%
token reduction and retaining 97.6% accuracy under an extreme 93.4% setting. These results highlight end-to-end training with DUET-VLM, enabling robust adaptation to reduced visual (image/
video) input without sacrificing accuracy, producing compact yet semantically rich representations within the same computational budget. Our code is available at https://github.com/AMD-AGI/DUET-VLM.
Read the full abstract ▾
All authors Aditya Kumar Singh, Hitesh Kandala, Pratik Prabhanjan Brahma, Zicheng Liu, Emad Barsoum
Computer Vision
Echoes of ownership: Adversarial-guided dual injection for copyright protection in MLLMs
Chengwei Xia, Fan Ma, Ruijie Quan et al. (+3 more)
With the rapid deployment of
multimodal large
language models (MLLMs), disputes regarding model ownership have become increasingly frequent, raising significant concerns about intellectual property protection. In this paper, we propose a framework for generating copyright triggers for MLLMs, enabling model publishers to embed verifiable ownership information into the model. The goal is to construct trigger images that elicit ownership-related textual responses exclusively in fine-tuned derivatives, while remaining inert in other non-derivative models. Our method constructs a tracking trigger image by treating the image as a learnable tensor, performing
adversarial optimization with dual-injection of ownership-relevant semantic information. The first injection is achieved by enforcing textual consistency between the output of an auxiliary MLLM and a predefined ownership-relevant target text; the consistency loss is backpropagated to inject this ownership-related information into the image. The second injection is performed at the semantic-level by minimizing the distance between the CLIP features of the image and those of the target text. Furthermore, we introduce an additional
adversarial training stage involving the auxiliary model. It is specifically trained to resist generating ownership-relevant target text, thereby enhancing robustness in heavily fine-tuned derivative models. Extensive experiments demonstrate the effectiveness of our dual-injection approach in tracking model lineage under various fine-tuning and domain-shift scenarios. Code is at https://github.com/kunzhan/AGDI
Read the full abstract ▾
All authors Chengwei Xia, Fan Ma, Ruijie Quan, Yunqiu Xu, Kun Zhan, Yi Yang
Computer Vision
Detecting AI-Generated Forgeries via Iterative Manifold Deviation Amplification
Jiangling Zhang, Shuxuan Gao, Bofan Liu et al. (+4 more)
The proliferation of highly realistic AI-generated images poses critical challenges for digital forensics, demanding precise pixel-level localization of manipulated regions. Existing methods predominantly learn discriminative patterns of specific forgeries and often struggle with novel manipulations as editing techniques continue to evolve. We propose the Iterative Forgery Amplifier Network (IFA-Net), which shifts from learning "what is fake" to modeling "what is real". Grounded in the principle that all manipulations deviate from the natural image
manifold, IFA-Net leverages a frozen Masked Autoencoder (MAE) pretrained on real images as a universal realness prior. Our framework operates through a two-stage closed-loop process: an initial Dual-Stream
Segmentation Network (DSSN) fuses the original image with MAE reconstruction residuals for coarse localization, followed by a Task-Adaptive Prior Injection (TAPI) module that converts this coarse prediction into guiding
prompts to steer the MAE decoder and amplify reconstruction failures in suspicious regions for precise refinement. Extensive experiments on four
diffusion-based inpainting
benchmarks show that IFA-Net achieves an average improvement of 6.5% in IoU and 8.1% in F1-score over the second-best method, while demonstrating strong generalization to traditional manipulation types.
Read the full abstract ▾
All authors Jiangling Zhang, Shuxuan Gao, Bofan Liu, Siqiang Feng, Jirui Huang, Yaxiong Chen, Ziyu Chen
Computer Vision
CLAP Convolutional Lightweight Autoencoder for Plant Disease Classification
Asish Bera, Subhajit Roy and Sudiptendu Banerjee
Convolutional neural networks have remarkably progressed the performance of distinguishing plant diseases, severity grading, and nutrition deficiency prediction using leaf images. However, these tasks become more challenging in a realistic in-situ field condition. Often, a traditional machine learning model may fail to capture and interpret discriminative characteristics of plant health, growth and diseases due to subtle variations within leaf subcategories. A few deep learning methods have used additional preprocessing stages or network modules to address the problem, whereas several other methods have utilized pre-trained backbone CNNs, most of which are computationally intensive. Therefore, to address the challenge, we propose a lightweight autoencoder using separable convolutional layers in its encoder decoder blocks. A sigmoid gating is applied for refining the prowess of the encoders feature discriminability, which is improved further by the decoder. Finally, the feature maps of the encoder decoder are combined for rich feature representation before classification. The proposed Convolutional Lightweight Autoencoder for Plant disease classification, called CLAP, has been experimented on three public plant
datasets consisting of cassava, tomato, maize, groundnut, grapes, etc. for determining plant health conditions. The CLAP has attained improved or competitive accuracies on the Integrated Plant Disease, Groundnut, and CCMT
datasets balancing a tradeoff between the performance, and little computational cost requiring 5 million parameters. The training time is 20 milliseconds and inference time is 1 ms per image.
Read the full abstract ▾
Computer Vision
IDperturb: Enhancing Variation in Synthetic Face Generation via Angular Perturbation
Fadi Boutros, Eduarda Caldeira, Tahar Chettaoui et al. (+1 more)
Synthetic data has emerged as a practical alternative to authentic face
datasets for training face recognition (FR) systems, especially as
privacy and legal concerns increasingly restrict the use of real biometric data. Recent advances in identity-conditional
diffusion models have enabled the generation of photorealistic and identity-consistent face images. However, many of these models suffer from limited intra-class variation, an essential property for training robust and generalizable FR models. In this work, we propose IDPERTURB, a simple yet effective
geometric-driven sampling strategy to enhance diversity in synthetic face generation. IDPERTURB perturbs identity
embeddings within a constrained angular region of the unit hyper-sphere, producing a diverse set of
embeddings without modifying the underlying generative model. Each perturbed
embedding serves as a conditioning vector for a pre-trained
diffusion model, enabling the synthesis of visually varied yet identity-coherent face images suitable for training generalizable FR systems. Empirical results demonstrate that training FR on
datasets generated using IDPERTURB yields improved performance across multiple FR
benchmarks, compared to existing synthetic data generation approaches.
Read the full abstract ▾
All authors Fadi Boutros, Eduarda Caldeira, Tahar Chettaoui, Naser Damer
Computer Vision
Spatial-Temporal State Propagation Autoregressive Model for 4D Object Generation
Liying Yang, Jialun Liu, Jiakui Hu et al. (+5 more)
Generating high-quality 4D objects with spatial-temporal consistency is still formidable. Existing
diffusion-based methods often struggle with spatial-temporal inconsistency, as they fail to leverage outputs from all previous timesteps to guide the generation at the current timestep. Therefore, we propose a Spatial-Temporal State Propagation AutoRegressive Model (4DSTAR), which generates 4D objects maintaining temporal-spatial consistency. 4DSTAR formulates the generation problem as the prediction of
tokens that represent the 4D object. It consists of two key components: (1) The dynamic spatial-temporal state propagation autoregressive model (
STAR) is proposed, which achieves spatial-temporal consistent generation. Unlike standard autoregressive models,
STAR divides prediction
tokens into groups based on timesteps. It models long-term dependencies by propagating spatial-temporal states from previous groups and utilizes these dependencies to guide generation at the next timestep. To this end, a spatial-temporal container is proposed, which dynamically updating the effective spatial-temporal state features from all historical groups, then updated features serve as conditional features to guide the prediction of the next
token group. (2) The 4D VQ-VAE is proposed, which implicitly encodes the 4D structure into discrete space and decodes the discrete
tokens predicted by
STAR into temporally coherent dynamic 3D Gaussians. Experiments demonstrate that 4DSTAR generates spatial-temporal consistent 4D objects, and achieves performance competitive with
diffusion models.
Read the full abstract ▾
All authors Liying Yang, Jialun Liu, Jiakui Hu, Chenhao Guan, Haibin Huang, Fangqiu Yi, Chi Zhang, Yanyan Liang
Computer Vision · Cross-listed from cs.LG
Bayesian Lottery Ticket Hypothesis
Nicholas Kuhn, Arvid Weyrauch, Lars Heyen et al. (+3 more)
Bayesian neural networks (BNNs) are a useful tool for uncertainty quantification, but require substantially more computational resources than conventional neural networks. For non-
Bayesian networks, the Lottery Ticket Hypothesis (LTH) posits the existence of sparse subnetworks that can train to the same or even surpassing accuracy as the original dense network. Such sparse networks can lower the demand for computational resources at inference, and during training. The existence of the LTH and corresponding sparse subnetworks in BNNs could motivate the development of sparse training
algorithms and provide valuable insights into the underlying training process. Towards this end, we translate the LTH experiments to a
Bayesian setting using common computer
vision models. We investigate the defining characteristics of
Bayesian lottery tickets, and extend our study towards a transplantation method connecting BNNs with deterministic Lottery Tickets. We generally find that the LTH holds in BNNs, and winning tickets of matching and surpassing accuracy are present independent of model size, with degradation at very high sparsities. However, the
pruning strategy should rely primarily on magnitude, secondly on standard deviation. Furthermore, our results demonstrate that models rely on mask structure and weight initialization to varying degrees.
Read the full abstract ▾
All authors Nicholas Kuhn, Arvid Weyrauch, Lars Heyen, Achim Streit, Markus Götz, Charlotte Debus
Computer Vision
Robust Self-Supervised Cross-Modal Super-Resolution against Real-World Misaligned Observations
Xiaoyu Dong, Jiahuan Li, Ziteng Cui et al. (+1 more)
Cross-modal super-resolution (SR) on real-world misaligned data is challenging, as only unlabeled low-resolution (LR) source and high-resolution (HR) guide images with complex spatial misalignment are available. Previous methods either rely on simulated training data or adopt suboptimal
alignment strategies that overlook cross-modal dependencies, limiting their practical performance. To address these issues, we propose RobSelf, a self-supervised model that jointly optimizes a misalignment-aware feature translator and a content-aware reference filter online. The translator resolves unsupervised cross-modal and cross-resolution
alignment via weakly-supervised, misalignment-aware translation, yielding an aligned guide feature. Guided by this feature, the filter performs reference-based discriminative self-enhancement on the source, enabling SR prediction with high resolution and high fidelity. Experiments on synthesized data and collected real-world data demonstrate that RobSelf achieves state-of-the-art performance, outperforming existing self-supervised and supervised methods. Moreover, it achieves superior efficiency, being up to 15.3$\times$ faster than prior self-supervised methods.
Read the full abstract ▾
All authors Xiaoyu Dong, Jiahuan Li, Ziteng Cui, Naoto Yokoya
Computer Vision
HeRO: Hierarchical 3D Semantic Representation for Pose-aware Object Manipulation
Chongyang Xu, Shen Cheng, Haipeng Li et al. (+3 more)
Imitation learning for robotic manipulation has progressed from 2D image policies to 3D representations that explicitly encode geometry. Yet purely
geometric policies often lack explicit part-level semantics, which are critical for pose-aware manipulation (e.g., distinguishing a shoe's toe from heel). In this paper, we present HeRO, a
diffusion-based policy that couples geometry and semantics via hierarchical semantic fields. HeRO employs dense semantics lifting to fuse discriminative, geometry-sensitive features from DINOv2 with the smooth, globally coherent correspondences from Stable
Diffusion, yielding dense features that are both fine-grained and spatially consistent. These features are processed and partitioned to construct a global field and a set of local fields. A hierarchical conditioning module conditions the generative denoiser on global and local fields using permutation-invariant network architecture, thereby avoiding order-sensitive bias and producing a coherent control policy for pose-aware manipulation. In various tests, HeRO establishes a new state-of-the-art, improving success on Place Dual Shoes by 12.3% and averaging 6.5% gains across six challenging pose-aware tasks. Code is available at https://github.com/Chongyang-99/HeRO.
Read the full abstract ▾
All authors Chongyang Xu, Shen Cheng, Haipeng Li, Haoqiang Fan, Ziliang Feng, Shuaicheng Liu
Computer Vision
Learning Multi-Modal Prototypes for Cross-Domain Few-Shot Object Detection
Wanqi Wang, Jingcai Guo, Yuxiang Cai et al. (+1 more)
Cross-Domain Few-Shot Object Detection (CD-FSOD) aims to detect novel classes in unseen target domains given only a few labeled examples. While open-vocabulary detectors built on
vision-
language models (VLMs) transfer well, they depend almost entirely on text
prompts, which encode domain-invariant semantics but miss domain-specific visual information needed for precise localization under few-shot supervision. We propose a dual-branch detector that Learns Multi-modal Prototypes, dubbed LMP, by coupling textual guidance with visual exemplars drawn from the target domain. A Visual Prototype Construction module aggregates class-level prototypes from support RoIs and dynamically generates hard-negative prototypes in query images via jittered boxes, capturing distractors and visually similar backgrounds. In the visual-guided branch, we inject these prototypes into the detection pipeline with components mirrored from the text branch as the starting point for training, while a parallel text-guided branch preserves open-vocabulary semantics. The branches are trained jointly and ensembled at inference by combining semantic abstraction with domain-adaptive details. On six cross-domain
benchmark datasets and standard 1/5/10-shot settings, our method achieves state-of-the-art or highly competitive mAP.
Read the full abstract ▾
All authors Wanqi Wang, Jingcai Guo, Yuxiang Cai, Zhi Chen
Computer Vision
Rethinking Preference Alignment for Diffusion Models with Classifier-Free Guidance
Zhou Jiang, Yandong Wen and Zhen Liu
Aligning large-scale text-to-image
diffusion models with nuanced human preferences remains challenging. While direct preference optimization (DPO) is simple and effective, large-scale finetuning often shows a generalization gap. We take inspiration from test-time guidance and cast preference
alignment as classifier-free guidance (CFG): a finetuned preference model acts as an external control signal during sampling. Building on this view, we propose a simple method that improves
alignment without retraining the base model. To further enhance generalization, we decouple preference learning into two modules trained on positive and negative data, respectively, and form a contrastive guidance vector at inference by subtracting their predictions (positive minus negative), scaled by a user-chosen strength and added to the base prediction at each step. This yields a sharper and controllable
alignment signal. We evaluate on Stable
Diffusion 1.5 and Stable
Diffusion XL with Pick-a-Pic v2 and HPDv3, showing consistent quantitative and qualitative gains.
Read the full abstract ▾
Computer Vision
MaskDiME: Adaptive Masked Diffusion for Precise and Efficient Visual Counterfactual Explanations
Changlu Guo, Anders Nymark Christensen, Anders Bjorholm Dahl et al. (+1 more)
Visual counterfactual explanations aim to reveal the minimal semantic modifications that can alter a model's prediction, providing causal and interpretable insights into deep neural networks. However, existing
diffusion-based counterfactual generation methods are often computationally expensive, slow to sample, and imprecise in localizing the modified regions. To address these limitations, we propose MaskDiME, a simple, fast, yet effective
diffusion framework that unifies semantic consistency and spatial precision through localized sampling. Our approach adaptively focuses on decision-relevant regions to achieve localized and semantically consistent counterfactual generation while preserving high image fidelity. Our training-free framework, MaskDiME, performs inference over 30x faster than the baseline and achieves comparable or state-of-the-art performance across five
benchmark datasets spanning diverse visual domains, establishing a practical and generalizable solution for efficient counterfactual explanation.
Read the full abstract ▾
All authors Changlu Guo, Anders Nymark Christensen, Anders Bjorholm Dahl, Morten Rieger Hannemose
Computer Vision
Initialization matters in few-shot adaptation of vision-language models for histopathological image classification
Pablo Meseguer, Rocío del Amor and Valery Naranjo
Vision language models (VLM) pre-trained on
datasets of histopathological image-caption pairs enabled zero-shot slide-level classification. The ability of VLM image encoders to extract discriminative features also opens the door for supervised fine-tuning for whole-slide image (WSI) classification, ideally using few labeled samples. Slide-level prediction frameworks require the incorporation of multiple instance learning (MIL) due to the gigapixel size of the WSI. Following patch-level feature extraction and aggregation, MIL frameworks rely on linear classifiers trained on top of the slide-level aggregated features. Classifier weight initialization has a large influence on Linear Probing performance in efficient transfer learning (ETL) approaches based on few-shot learning. In this work, we propose Zero-Shot Multiple-Instance Learning (ZS-MIL) to address the limitations of random classifier initialization that underperform zero-shot prediction in MIL problems. ZS-MIL uses the class-level
embeddings of the VLM text encoder as the classification layer's starting point to compute each sample's bag-level probabilities. Through multiple experiments, we demonstrate the robustness of ZS-MIL compared to well-known weight initialization techniques both in terms of performance and variability in an ETL few-shot scenario for subtyping prediction.
Read the full abstract ▾
Computer Vision
A high-resolution nationwide urban village mapping product for 342 Chinese cities based on foundation models
Lubin Bai, Sheng Xiao, Ziyu Yin et al. (+4 more)
Urban Villages (UVs) represent a distinctive form of high-density informal settlement embedded within China's rapidly urbanizing cities. Accurate identification of UVs is critical for urban governance, renewal, and sustainable development. But due to the pronounced heterogeneity and diversity of UVs across China's vast territory, a consistent and reliable nationwide
dataset has been lacking. In this work, we present GeoLink-UV, a high-resolution nationwide UV mapping product that clearly delineates the locations and boundaries of UVs in 342 Chinese cities. The
dataset is derived from multisource geospatial data, including optical remote sensing images and geo-vector data, and is generated through a foundation model-driven mapping framework designed to address the generalization issues and improve the product quality. A geographically stratified accuracy assessment based on independent samples from 28 cities confirms the reliability and scientific credibility of the nationwide
dataset across heterogeneous urban contexts. Based on this nationwide product, we reveal substantial interregional disparities in UV prevalence and spatial configuration. On average, UV areas account for 8 % of built-up land, with marked clustering in central and south China. Building-level analysis further confirms a consistent low-rise, high-density development pattern of UVs nationwide, while highlighting regionally differentiated morphological characteristics. The GeoLink-UV
dataset provides an open and systematically validated geospatial foundation for urban studies, informal settlement monitoring, and evidence-based urban renewal planning, and contributes directly to large-scale assessments aligned with Sustainable Development Goal 11. The GeoLink-UV
dataset introduced in this article is freely available at https://doi.org/10.5281/zenodo.18688062.
Read the full abstract ▾
All authors Lubin Bai, Sheng Xiao, Ziyu Yin, Haoyu Wang, Siyang Wu, Xiuyuan Zhang, Shihong Du