Computer Vision
Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learning
Juekai Lin, Yun Zhu, Honglin Lin et al. (+6 more)
Graphics Program Synthesis is pivotal for interpreting and editing visual data, effectively facilitating the reverse-engineering of static visuals into editable TikZ code. While TikZ is the de facto standard for scientific schematics due to its programmatic flexibility, its requirement for rigorous spatial precision presents a significant challenge for
Multimodal Large
Language Models. Progress is currently stifled by two primary gaps: (1) Data Quality Gap: existing image-TikZ corpora often lack strict executability and reliable visual
alignment; (2) Evaluation Gap: a lack of
benchmarks for both structural and visual fidelity. To address these, we present a closed-loop framework featuring: SciTikZ-230K, a large-scale, high-quality
dataset from our Execution-Centric Data Engine covering 11 diverse scientific disciplines; SciTikZ-Bench, a multifaceted
benchmark spanning from basic
geometric constructs to intricate hierarchical schematics to evaluate both visual fidelity and structural logic. To further broaden the scope of visual-code optimization methodology, we introduce a novel Dual Self-Consistency Reinforcement Learning optimization paradigm, which utilizes Round-Trip Verification to penalize degenerate code and boost overall self-consistency. Empowered by these, our trained model SciTikZer-8B achieves state-of-the-art performance, consistently outperforming proprietary
giants like Gemini-2.5-Pro and massive models like Qwen3-VL-235B-A22B-Instruct.
Read the full abstract ▾
All authors Juekai Lin, Yun Zhu, Honglin Lin, Sijing Li, Tianwei Lin, Zheng Liu, Xiaoyang Wang, Wenqiao Zhang, Lijun Wu
Computer Vision
Graph-PiT: Enhancing Structural Coherence in Part-Based Image Synthesis via Graph Priors
Junbin Zhang, Meng Cao, Feng Tan et al. (+2 more)
Achieving fine-grained and structurally sound controllability is a cornerstone of advanced visual generation. Existing part-based frameworks treat user-provided parts as an unordered set and therefore ignore their intrinsic spatial and semantic relationships, which often results in compositions that lack structural integrity. To bridge this gap, we propose Graph-PiT, a framework that explicitly models the structural dependencies of visual components using a graph prior. Specifically, we represent visual parts as nodes and their spatial-semantic relationships as edges. At the heart of our method is a Hierarchical Graph Neural Network (HGNN) module that performs bidirectional message passing between coarse-grained part-level super-nodes and fine-grained IP+
token sub-nodes, refining part
embeddings before they enter the generative pipeline. We also introduce a graph
Laplacian smoothness loss and an edge-reconstruction loss so that adjacent parts acquire compatible, relation-aware
embeddings. Quantitative experiments on controlled synthetic domains (character, product, indoor layout, and jigsaw), together with qualitative transfer to real web images, show that Graph-PiT improves structural coherence over vanilla PiT while remaining compatible with the original IP-Prior pipeline. Ablation experiments confirm that explicit relational
reasoning is crucial for enforcing user-specified adjacency constraints. Our approach not only enhances the plausibility of generated concepts but also offers a scalable and interpretable mechanism for complex, multi-part image synthesis. The code is available at https://github.com/wolf-bailang/Graph-PiT.
Read the full abstract ▾
All authors Junbin Zhang, Meng Cao, Feng Tan, Yikai Lin, Yuexian Zou
Computer Vision · Cross-listed from cs.DC
CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inference
Yulin Zou, Wenyan Chen, Yan Chen et al. (+6 more)
Continuous inference over concurrent
video streams imposes substantial compute and
memory demands on
vision-
language model (VLM) serving. Streaming inference uses sliding windows to maintain a bounded context of recent
video, but processing each window independently repeats visual encoding and large
language model (LLM) prefilling for similar and overlapping content. Existing optimizations provide limited coordination across these stages and often rely on model-specific training, profiling, or model-generated signals. We present CodecSight, a streaming VLM serving system that uses codec metadata as shared runtime guidance across visual encoding and LLM prefilling, without model-specific training or offline profiling. Codec-derived change signals guide patch
pruning before visual encoding, reducing both visual computation and the number of downstream visual
tokens. Codec-defined frame types guide selective key-value (KV) refresh across windows, while positional correction enables reuse of the remaining cached keys. Across three VLMs and four
video workloads, our vLLM-based implementation supports up to $3.3\times$ as many concurrent streams and achieves up to a $5.3\times$ speedup in average time-to-first-
token relative to the state-of-the-art baselines. It also reduces executed FLOPs by up to 93%, with a maximum task-quality decrease of 4.64 percentage points.
Read the full abstract ▾
All authors Yulin Zou, Wenyan Chen, Yan Chen, Anya Rajan, JooYoung Park, Shivaraman Nitin, Luo Tao, Francisco Romero, Dmitrii Ustiugov
Computer Vision
Toward Aristotelian Medical Representations: Backpropagation-Free Layer-wise Analysis for Interpretable Generalized Metric Learning on MedMNIST
Michael Karnes and Alper Yilmaz
While deep learning has achieved remarkable success in medical imaging, the "black-box" nature of backpropagation-based models remains a significant barrier to clinical adoption. To bridge this gap, we propose Aristotelian Rapid Object Modeling (A-ROM), a framework built upon the Platonic Representation Hypothesis (PRH). This hypothesis posits that models trained on vast, diverse
datasets converge toward a universal and objective representation of reality. By leveraging the generalizable metric space of pretrained
Vision Transformers (ViTs), A-ROM enables the rapid modeling of novel medical concepts without the computational burden or opacity of further gradient-based fine-tuning. We replace traditional, opaque decision layers with a human-readable concept dictionary and a k-Nearest Neighbors (kNN) classifier to ensure the model's logic remains interpretable. Experiments on the MedMNIST v2 suite demonstrate that A-ROM delivers performance competitive with standard
benchmarks while providing a simple and scalable, "few-shot" solution that meets the rigorous transparency demands of modern clinical environments.
Read the full abstract ▾
Computer Vision
OmniCamera: A Unified Framework for Multi-task Video Generation with Arbitrary Camera Control
Yukun Wang, Ruihuang Li, Jiale Tao et al. (+7 more)
Video fundamentally intertwines two crucial axes: the dynamic content of a scene and the camera motion through which it is observed. However, existing generation models often entangle these factors, limiting independent control. In this work, we introduce OmniCamera, a unified framework designed to explicitly disentangle and command these two dimensions. This compositional approach enables flexible
video generation by allowing arbitrary pairings of camera and content conditions, unlocking unprecedented creative control. To overcome the fundamental challenges of modality conflict and data scarcity inherent in such a system, we present two key innovations. First, we construct OmniCAM, a novel hybrid
dataset combining curated real-world
videos with synthetic data that provides diverse paired examples for robust multi-task learning. Second, we propose a Dual-level Curriculum Co-Training strategy that mitigates modality interference and synergistically learns from diverse data sources. This strategy operates on two levels: first, it progressively introduces control modalities by difficulties (condition-level), and second, trains for precise control on synthetic data before adapting to real data for photorealism (data-level). As a result, OmniCamera achieves state-of-the-art performance, enabling flexible control for complex camera movements while maintaining superior visual quality.
Read the full abstract ▾
All authors Yukun Wang, Ruihuang Li, Jiale Tao, Shiyuan Yang, Liyi Chen, Zhantao Yang, Handz, Yulan Guo, Shuai Shao, Qinglin Lu
Computer Vision
Is CLIP Cross-Eyed? Revealing and Mitigating Center Bias in the CLIP Family
Oscar Chew, Hsiao-Ying Huang, Kunal Jain et al. (+3 more)
Recent research has shown that contrastive
vision-
language models such as CLIP often lack fine-grained understanding of visual content. While a growing body of work has sought to address this limitation, we identify a distinct failure mode in the CLIP family, which we term center bias, that persists even in recent model variants. Specifically, CLIP tends to disproportionately focus on the central region of an image, overlooking important objects located near the boundaries. This limitation is fundamental as failure to recognize relevant objects makes it difficult to perform any sophisticated tasks that depend on those objects. To understand the underlying causes of the limitation, we conduct analyses from both representation and
attention perspectives. Using interpretability methods, i.e.,
embedding decomposition and
attention map analysis, we find that relevant concepts especially those associated with off-center objects vanish from the model's
embedding in the final representation due to information loss during the aggregation of visual
embeddings, particularly the reliance on pooling mechanisms. Finally, we show that this bias can be alleviated with training-free strategies such as visual prompting and
attention redistribution by redirecting models'
attention to off-center regions.
Read the full abstract ▾
All authors Oscar Chew, Hsiao-Ying Huang, Kunal Jain, Tai-I Chen, Khoa D Doan, Kuan-Hao Huang
Computer Vision
HumANDiff: Articulated Noise Diffusion for Motion-Consistent Human Video Generation
Tao Hu and Varun Jampani
Despite tremendous recent progress in human
video generation, generative
video diffusion models still struggle to capture the dynamics and physics of human motions faithfully. In this paper, we propose a new framework for human
video generation, HumANDiff, which enhances the human motion control with three key designs: 1) Articulated motion-consistent noise sampling that correlates the spatiotemporal distribution of latent noise and replaces the unstructured random Gaussian noise with 3D articulated noise sampled on the dense surface
manifold of a statistical human body template. It inherits body
topology priors for spatially and temporally consistent noise sampling. 2) Joint appearance-motion learning that enhances the standard training objective of
video diffusion models by jointly predicting pixel appearances and corresponding physical motions from the articulated noises. It enables high-fidelity human
video synthesis, e.g., capturing motion-dependent clothing wrinkles. 3)
Geometric motion consistency learning that enforces physical motion consistency across frames via a novel
geometric motion consistency loss defined in the articulated noise space. HumANDiff enables scalable controllable human
video generation by fine-tuning
video diffusion models with articulated noise sampling. Consequently, our method is agnostic to
diffusion model design, and requires no modifications to the model architecture. During inference, HumANDiff enables image-to-
video generation within a single framework, achieving intrinsic motion control without requiring additional motion modules. Extensive experiments demonstrate that our method achieves state-of-the-art performance in rendering motion-consistent, high-fidelity humans with diverse clothing styles. Project page: https://taohuumd.github.io/projects/HumANDiff/
Read the full abstract ▾
Computer Vision
Multi-Modal Landslide Detection from Sentinel-1 SAR and Sentinel-2 Optical Imagery Using Multi-Encoder Vision Transformers and Ensemble Learning
Ioannis Nasios
Landslides represent a major geohazard with severe impacts on human life, infrastructure, and ecosystems, underscoring the need for accurate and timely detection approaches to support disaster risk reduction. This study proposes a modular, multi-model framework that fuses Sentinel-2 optical imagery with Sentinel-1 Synthetic Aperture Radar (SAR) data, for robust landslide detection. The methodology leverages multi-encoder
vision transformers, where each data modality is processed through separate lightweight pretrained encoders, achieving strong performance in landslide detection. In addition, the integration of multiple models, particularly the combination of neural networks and gradient boosting models (LightGBM and XGBoost), demonstrates the power of ensemble learning to further enhance accuracy and robustness. Derived spectral indices, such as NDVI, are integrated alongside original bands to enhance sensitivity to vegetation and surface changes. The proposed methodology achieves a state-of-the-art F1 score of 0.919 on landslide detection, addressing a patch-based classification task rather than pixel-level
segmentation and operating without pre-event Sentinel-2 data, highlighting its effectiveness in a non-classical change detection setting. It also demonstrated top performance in a machine learning competition, achieving a strong balance between precision and recall and highlighting the advantages of explicitly leveraging the complementary strengths of optical and radar data. The conducted experiments and research also emphasize scalability and operational applicability, enabling flexible configurations with optical-only, SAR-only, or combined inputs, and offering a transferable framework for broader natural hazard monitoring and environmental change applications. Full training and inference code can be found in https://github.com/IoannisNasios/sentinel-landslide-cls.
Read the full abstract ▾
Computer Vision
Mixture-of-Modality-Experts with Holistic Token Learning for Fine-Grained Multimodal Visual Analytics in Driver Action Recognition
Tianyi Liu, Yiming Li, Wenqian Wang et al. (+4 more)
Robust
multimodal visual analytics remains challenging when heterogeneous modalities provide complementary but input-dependent evidence for decision-making.Existing
multimodal learning methods mainly rely on fixed fusion modules or predefined cross-modal interactions, which are often insufficient to adapt to changing modality reliability and to capture fine-grained action cues. To address this issue, we propose a Mixture-of-Modality-Experts (MoME) framework with a Holistic
Token Learning (HTL) strategy. MoME enables adaptive collaboration among modality-specific experts, while HTL improves both intra-expert refinement and inter-expert knowledge transfer through class
tokens and spatio-temporal
tokens. In this way, our method forms a knowledge-centric
multimodal learning framework that improves expert specialization while reducing ambiguity in
multimodal fusion.We validate the proposed framework on driver action recognition as a representative
multimodal understanding taskThe experimental results on the public
benchmark show that the proposed MoME framework and the HTL strategy jointly outperform representative single-modal and
multimodal baselines. Additional ablation, validation, and visualization results further verify that the proposed HTL strategy improves subtle
multimodal understanding and offers better interpretability.
Read the full abstract ▾
All authors Tianyi Liu, Yiming Li, Wenqian Wang, Jiaojiao Wang, Chen Cai, Yi Wang, Kim-Hui Yap
Computer Vision
Leveraging Image Editing Foundation Models for Data-Efficient CT Metal Artifact Reduction
Ahmet Rasim Emirdagi, Süleyman Aslan, M{\i}sra Yavuz et al. (+5 more)
Metal artifacts from high-attenuation implants severely degrade CT image quality, obscuring critical anatomical structures and posing a challenge for standard deep learning methods that require extensive paired training data. We propose a paradigm shift: reframing artifact reduction as an in-context
reasoning task by adapting a general-purpose
vision-
language diffusion foundation model via parameter-efficient Low-Rank Adaptation (LoRA). By leveraging rich visual priors, our approach achieves effective artifact suppression with only 16 to 128 paired training examples reducing data requirements by two orders of magnitude. Crucially, we demonstrate that domain adaptation is essential for
hallucination mitigation; without it, foundation models interpret streak artifacts as erroneous natural objects (e.g., waffles or petri dishes). To ground the restoration, we propose a multi-reference conditioning strategy where clean anatomical exemplars from unrelated subjects are provided alongside the corrupted input, enabling the model to exploit category-specific context to infer uncorrupted anatomy. Extensive evaluation on the AAPM CT-MAR
benchmark demonstrates that our method achieves state-of-the-art performance on perceptual and radiological-feature metrics . This work establishes that foundation models, when appropriately adapted, offer a scalable alternative for interpretable, data-efficient medical image reconstruction. Code is available at https://github.com/ahmetemirdagi/CT-EditMAR.
Read the full abstract ▾
All authors Ahmet Rasim Emirdagi, Süleyman Aslan, M{\i}sra Yavuz, Görkay Aydemir, Yunus Bilge Kurt, Nasrin Rahimi, Burak Can Biner, M. Ak{\i}n Y{\i}lmaz
Computer Vision
SonoSelect: Efficient Ultrasound Perception via Active Probe Exploration
Yixin Zhang, Yunzhong Hou, Longqi Li et al. (+3 more)
Ultrasound perception typically requires multiple scan views through probe movement to reduce diagnostic ambiguity, mitigate acoustic occlusions, and improve anatomical coverage. However, not all probe views are equally informative. Exhaustively acquiring a large number of views can introduce substantial redundancy, increase scanning and processing costs. To address this, we define an active view exploration task for ultrasound and propose SonoSelect, an ultrasound-specific method that adaptively guides probe movement based on current observations. Specifically, we cast ultrasound active view exploration as a sequential decision-making problem. Each new 2D ultrasound view is fused into a 3D spatial
memory of the observed anatomy, which guides the next probe position. On top of this formulation, we propose an ultrasound-specific objective that favors probe movements with greater organ coverage, lower reconstruction uncertainty, and less redundant scanning. Experiments on the ultrasound simulator show that SonoSelect achieves promising multi-view organ classification accuracy using only 2 out of N views. Furthermore, for a more difficult kidney cyst detection task, it reaches 54.56% kidney coverage and 35.13% cyst coverage, with short trajectories consistently centered on the target cyst.
Read the full abstract ▾
All authors Yixin Zhang, Yunzhong Hou, Longqi Li, Zhenyue Qin, Yang Liu, Yue Yao
Computer Vision
Saliency-Guided Representation with Consistency Policy Learning for Visual Unsupervised Reinforcement Learning
Jingbo Sun, Qichao Zhang, Songjun Tu et al. (+5 more)
Zero-shot unsupervised reinforcement learning (URL) offers a promising direction for building generalist
agents capable of generalizing to unseen tasks without additional supervision. Among existing approaches, successor representations (SR) have emerged as a prominent paradigm due to their effectiveness in structured, low-dimensional settings. However, SR methods struggle to scale to high-dimensional visual environments. Through empirical analysis, we identify two key limitations of SR in visual URL: (1) SR objectives often lead to suboptimal representations that attend to dynamics-irrelevant regions, resulting in inaccurate successor measures and degraded task generalization; and (2) these flawed representations hinder SR policies from modeling multi-modal skill-conditioned action distributions and ensuring skill controllability. To address these limitations, we propose Saliency-Guided Representation with Consistency Policy Learning (SRCP), a novel framework that improves zero-shot generalization of SR methods in visual URL. SRCP decouples representation learning from successor training by introducing a saliency-guided dynamics task to capture dynamics-relevant representations, thereby improving successor measure and task generalization. Moreover, it integrates a fast-sampling consistency policy with URL-specific classifier-free guidance and tailored training objectives to improve skill-conditioned policy modeling and controllability. Extensive experiments on 16 tasks across 4
datasets from the ExORL
benchmark demonstrate that SRCP achieves state-of-the-art zero-shot generalization in visual URL and is compatible with various SR methods.
Read the full abstract ▾
All authors Jingbo Sun, Qichao Zhang, Songjun Tu, Xing Fang, Yupeng Zheng, Haoran Li, Ke Chen, Dongbin Zhao
Computer Vision
Appearance Decomposition Gaussian Splatting for Multi-Traversal Reconstruction
Yangyi Xiao, Siting Zhu, Baoquan Yang et al. (+3 more)
Multi-traversal scene reconstruction is important for high-fidelity autonomous driving simulation and digital twin construction. This task involves integrating multiple sequences captured from the same geographical area at different times. In this context, a primary challenge is the significant appearance inconsistency across traversals caused by varying illumination and environmental conditions, despite the shared underlying geometry. This paper presents ADM-GS (Appearance Decomposition Gaussian Splatting for Multi-Traversal Reconstruction), a framework that applies an explicit appearance decomposition to the static background to alleviate appearance
entanglement across traversals. For the static background, we decompose the appearance into traversal-invariant material, representing intrinsic material properties, and traversal-dependent illumination, capturing lighting variations. Specifically, we propose a neural light field that utilizes a frequency-separated hybrid encoding strategy. By incorporating surface normals and explicit reflection vectors, this design separately captures low-frequency diffuse illumination and high-frequency specular reflections. Quantitative evaluations on the Argoverse 2 and Waymo Open
datasets demonstrate the effectiveness of ADM-GS. In multi-traversal experiments, our method achieves a +0.98 dB PSNR improvement over existing latent-based baselines while producing more consistent appearance across traversals. Code will be available at https://github.com/IRMVLab/ADM-GS.
Read the full abstract ▾
All authors Yangyi Xiao, Siting Zhu, Baoquan Yang, Tianchen Deng, Yongbo Chen, Hesheng Wang
Computer Vision
Selective Aggregation of Attention Maps Improves Diffusion-Based Visual Interpretation
Jungwon Park, Jungmin Ko, Dongnam Byun et al. (+1 more)
Numerous studies on text-to-image (T2I) generative models have utilized cross-
attention maps to boost application performance and interpret model behavior. However, the distinct characteristics of
attention maps from different
attention heads remain relatively underexplored. In this study, we show that selectively aggregating cross-
attention maps from heads most relevant to a target concept can improve visual interpretability. Compared to the
diffusion-based
segmentation method DAAM, our approach achieves higher mean IoU scores. We also find that the most relevant heads capture concept-specific features more accurately than the least relevant ones, and that selective aggregation helps diagnose
prompt misinterpretations. These findings suggest that
attention head selection offers a promising direction for improving the interpretability and controllability of T2I generation.
Read the full abstract ▾
All authors Jungwon Park, Jungmin Ko, Dongnam Byun, Wonjong Rhee
Computer Vision
AICA-Bench: Holistically Examining the Capabilities of VLMs in Affective Image Content Analysis
Dong She, Xianrong Yao, Liqun Chen et al. (+3 more)
Vision-
Language Models (VLMs) have demonstrated strong capabilities in perception, yet holistic Affective Image Content Analysis (AICA), which integrates perception,
reasoning, and generation into a unified framework, remains underexplored. To address this gap, we introduce AICA-Bench, a comprehensive
benchmark with three core tasks:
Emotion Understanding (EU),
Emotion Reasoning (ER), and
Emotion-Guided Content Generation (EGCG). We evaluate 23 VLMs and identify two major limitations: weak intensity calibration and shallow open-ended descriptions. To address these issues, we propose Grounded Affective Tree (GAT) Prompting, a training-free framework that combines visual scaffolding with hierarchical
reasoning. Experiments show that GAT reduces intensity errors and improves descriptive depth, providing a strong baseline for future research on affective
multimodal understanding and generation.
Read the full abstract ▾
All authors Dong She, Xianrong Yao, Liqun Chen, Jinghe Yu, Yang Gao, Zhanpeng Jin
Computer Vision
Physics-Aware Video Instance Removal Benchmark
Zirui Li, Xinghao Chen, Lingyu Jiang et al. (+5 more)
Video Instance Removal (VIR) requires removing target objects while maintaining background integrity and physical consistency, such as specular reflections and illumination interactions. Despite advancements in text-guided editing, current
benchmarks primarily assess visual plausibility, often overlooking the physical causalities, such as lingering shadows, triggered by object removal. We introduce the Physics-Aware
Video Instance Removal (PVIR)
benchmark, featuring 95 high-quality
videos annotated with instance-accurate masks and removal
prompts. PVIR is partitioned into Simple and Hard subsets, the latter explicitly targeting complex physical interactions. We evaluate four representative methods, PISCO-Removal, UniVideo, DiffuEraser, and CoCoCo, using a decoupled human evaluation protocol across three dimensions to isolate semantic, visual, and spatial failures: instruction following, rendering quality, and edit exclusivity. Our results show that PISCO-Removal and UniVideo achieve state-of-the-art performance, while DiffuEraser frequently introduces blurring artifacts and CoCoCo struggles significantly with instruction following. The persistent performance drop on the Hard subset highlights the ongoing challenge of recovering complex physical side effects.
Read the full abstract ▾
All authors Zirui Li, Xinghao Chen, Lingyu Jiang, Dengzhe Hou, Fangzhou Lin, Kazunori Yamada, Xiangbo Gao, Zhengzhong Tu
Computer Vision
Automatic dental superimposition of 3D intraorals and 2D photographs for human identification
Antonio D. Villegas-Yeguas, Xavier Abreau-Freire, Guillermo R-García et al. (+5 more)
Dental comparison is considered a primary identification method, at the level of fingerprints and DNA profiling. One crucial but time-consuming step of this method is the morphological comparison. One of the main challenges to apply this method is the lack of ante-mortem medical records, specially on scenarios such as migrant death at the border and/or in countries where there is no universal healthcare. The availability of photos on social media where teeth are visible has led many odontologists to consider morphological comparison using them. However, state-of-the-art proposals have significant limitations, including the lack of proper modeling of perspective distortion and the absence of objective approaches that quantify morphological differences. Our proposal involves a 3D (post-mortem scan) - 2D (ante-mortem photos) approach. Using computer
vision and optimization techniques, we replicate the ante-mortem image with the 3D model to perform the morphological comparison. Two automatic approaches have been developed: i) using paired landmarks and ii) using a
segmentation of the teeth region to estimate camera parameters. Both are capable of obtaining very promising results over 20,164 cross comparisons from 142 samples, obtaining mean ranking values of 1.6 and 1.5, respectively. These results clearly outperform filtering capabilities of automatic dental chart comparison approaches, while providing an automatic, objective and quantitative score of the morphological correspondence, easily to interpret and analyze by visualizing superimposed images.
Read the full abstract ▾
All authors Antonio D. Villegas-Yeguas, Xavier Abreau-Freire, Guillermo R-García, Andrea Valsecchi, Teresa Pinho, Daniel Pérez-Mongiovi, Oscar Ibáñez, Oscar Cordón
Computer Vision
Neural Network Pruning via QUBO Optimization
Osama Orabi, Artur Zagitov, Hadi Salloum et al. (+3 more)
Neural network
pruning can be formulated as a combinatorial optimization problem, yet most existing approaches rely on greedy heuristics that ignore complex interactions between filters. Formal optimization methods such as Quadratic Unconstrained Binary Optimization (QUBO) provide a principled alternative but have so far underperformed due to oversimplified objective formulations based on metrics like the L1-norm. In this work, we propose a unified Hybrid QUBO framework that bridges heuristic importance estimation with global combinatorial optimization. Our formulation integrates gradient-aware sensitivity metrics - specifically first-order Taylor and second-order Fisher information - into the linear term, while utilizing data-driven activation similarity in the quadratic term. This allows the QUBO objective to jointly capture individual filter relevance and inter-filter functional redundancy. We further introduce a dynamic capacity-driven search to strictly enforce target sparsity without distorting the optimization landscape. Finally, we employ a two-stage pipeline featuring a Tensor-Train (TT) Refinement stage - a gradient-free optimizer that fine-tunes the QUBO-derived solution directly against the true evaluation metric. Experiments on the SIDD image denoising
dataset demonstrate that the proposed Hybrid QUBO significantly outperforms both greedy Taylor
pruning and traditional L1-based QUBO, with TT Refinement providing further consistent gains at appropriate combinatorial scales. This highlights the potential of hybrid combinatorial formulations for robust, scalable, and interpretable neural network compression.
Read the full abstract ▾
All authors Osama Orabi, Artur Zagitov, Hadi Salloum, Viktor A. Lobachev, Kasymkhan Khubiev, Yaroslav Kholodov
Computer Vision
Reading Between the Pixels: An Inscriptive Jailbreak Attack on Text-to-Image Models
Zonghao Ying, Haowen Dai, Lianyu Hu et al. (+5 more)
Modern text-to-image (T2I) models can now render legible, paragraph-length text, enabling a fundamentally new class of misuse. We identify and formalize the inscriptive jailbreak, where an adversary coerces a T2I system into generating images containing harmful textual payloads (e.g., fraudulent documents) embedded within visually benign scenes. Unlike traditional depictive jailbreaks that elicit visually objectionable imagery, inscriptive attacks weaponize the text-rendering capability itself. Because existing jailbreak techniques are designed for coarse visual manipulation, they struggle to bypass multi-stage safety filters while maintaining character-level fidelity. To expose this vulnerability, we propose Etch, a black-box attack framework that decomposes the
adversarial prompt into three functionally orthogonal layers: semantic camouflage, visual-spatial anchoring, and typographic encoding. This decomposition reduces joint optimization over the full
prompt space to tractable sub-problems, which are iteratively refined through a zero-order loop. In this process, a
vision-
language model critiques each generated image, localizes failures to specific layers, and prescribes targeted revisions. Extensive evaluations across 7 models on the 2
benchmarks demonstrate that Etch achieves an average attack success rate of 65.57% (peaking at 91.00%), significantly outperforming existing baselines. Our results reveal a critical blind spot in current T2I safety
alignments and underscore the urgent need for typography-aware defense
multimodal mechanisms.
Read the full abstract ▾
All authors Zonghao Ying, Haowen Dai, Lianyu Hu, Zonglei Jing, Quanchen Zou, Yaodong Yang, Aishan Liu, Xianglong Liu
Computer Vision
Learn to Rank: Visual Attribution by Learning Importance Ranking
David Schinagl, Christian Fruhwirth-Reisinger, Alexander Prutsch et al. (+2 more)
Interpreting the decisions of complex computer
vision models is crucial to establish trust and accountability, especially in safety-critical domains. An established approach to interpretability is generating visual attribution maps that highlight regions of the input most relevant to the model's prediction. However, existing methods face a three-way trade-off. Propagation-based approaches are efficient, but they can be biased and architecture-specific. Meanwhile, perturbation-based methods are causally grounded, yet they are expensive and for
vision transformers often yield coarse, patch-level explanations. Learning-based explainers are fast but usually optimize surrogate objectives or distill from heuristic teachers. We propose a learning scheme that instead optimizes deletion and insertion metrics directly. Since these metrics depend on non-differentiable sorting and ranking, we frame them as permutation learning and replace the hard sorting with a differentiable relaxation using Gumbel-Sinkhorn. This enables end-to-end training through attribution-guided perturbations of the target model. During inference, our method produces dense, pixel-level attributions in a single forward pass with optional, few-step gradient refinement. Our experiments demonstrate consistent quantitative improvements and sharper, boundary-aligned explanations, particularly for
transformer-based
vision models. Code and pretrained models are available at https://github.com/dschinagl/AHA.
Read the full abstract ▾
All authors David Schinagl, Christian Fruhwirth-Reisinger, Alexander Prutsch, Samuel Schulter, Horst Possegger
Computer Vision
WikiSeeker: Rethinking the Role of Vision-Language Models in Knowledge-Based Visual Question Answering
Yingjian Zhu, Xinming Wang, Kun Ding et al. (+3 more)
Multi-modal
Retrieval-Augmented Generation (RAG) has emerged as a highly effective paradigm for Knowledge-Based Visual Question Answering (KB-VQA). Despite recent advancements, prevailing methods still primarily depend on images as the
retrieval key, and often overlook or misplace the role of
Vision-
Language Models (VLMs), thereby failing to leverage their potential fully. In this paper, we introduce WikiSeeker, a novel multi-modal RAG framework that bridges these gaps by proposing a multi-modal retriever and redefining the role of VLMs. Rather than serving merely as answer generators, we assign VLMs two specialized
agents: a Refiner and an Inspector. The Refiner utilizes the capability of VLMs to rewrite the textual query according to the input image, significantly improving the performance of the
multimodal retriever. The Inspector facilitates a decoupled generation strategy by selectively routing reliable retrieved context to another LLM for answer generation, while relying on the VLM's internal knowledge when
retrieval is unreliable. Extensive experiments on EVQA, InfoSeek, and M2KR demonstrate that WikiSeeker achieves state-of-the-art performance, with substantial improvements in both
retrieval accuracy and answer quality. Our code will be released on https://github.com/zhuyjan/WikiSeeker.
Read the full abstract ▾
All authors Yingjian Zhu, Xinming Wang, Kun Ding, Ying Wang, Bin Fan, Shiming Xiang
Computer Vision
EfficientMonoHair: Fast Strand-Level Reconstruction from Monocular Video via Multi-View Direction Fusion
Da Li, Dominik Engel, Deng Luo et al. (+1 more)
Strand-level hair geometry reconstruction is a fundamental problem in virtual human modeling and the digitization of hairstyles. However, existing methods still suffer from a significant trade-off between accuracy and efficiency. Implicit neural representations can capture the global hair shape but often fail to preserve fine-grained strand details, while explicit optimization-based approaches achieve high-fidelity reconstructions at the cost of heavy computation and poor scalability. To address this issue, we propose EfficientMonoHair, a fast and accurate framework that combines the implicit neural network with multi-view
geometric fusion for strand-level reconstruction from
monocular video. Our method introduces a fusion-patch-based multi-view optimization that reduces the number of optimization iterations for point cloud direction, as well as a novel parallel hair-growing strategy that relaxes voxel occupancy constraints, allowing large-scale strand tracing to remain stable and robust even under inaccurate or noisy orientation fields. Extensive experiments on representative real-world hairstyles demonstrate that our method can robustly reconstruct high-fidelity strand geometries with accuracy. On synthetic
benchmarks, our method achieves reconstruction quality comparable to state-of-the-art methods, while improving runtime efficiency by nearly an order of magnitude.
Read the full abstract ▾
All authors Da Li, Dominik Engel, Deng Luo, Ivan Viola
Computer Vision · Cross-listed from cs.CR
BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelines
Bo Ma, Jinsong Wu and Weiqi Yan
In LLM
agent pipelines,
prompt privacy risk propagates beyond a single model call: raw user content enters
retrieval queries,
memory writes, tool arguments, OCR-derived text, and logs, and every downstream copy inherits what the first write contained. Existing de-identification pipelines protect document boundaries but not this cross-stage surface. We present BodhiPromptShield, a policy-aware mediation layer that detects sensitive spans before they propagate, replaces each with a typed placeholder, a semantic abstraction, or a secure symbolic
token under a configured policy, and defers restoration to authorized execution boundaries. We evaluate it under one protocol against Presidio, Casper-style sanitization, an LLM sanitizer, and
transformer and learned detectors, on 300 AI4
Privacy documents, 493 PrivacyLens trajectories, 200 PrivacyLens tasks scored by that
benchmark's own judge, and AgentDojo tasks under injection. Three findings result. Identifier propagation is controllable: residual exposure falls to 7.4% on AI4
Privacy and 1.8% on PrivacyLens, and exact identifiers in an
agent's final action fall from 13.7% to 2.1-3.1%. Restoration timing governs what every stage upstream of the authorized boundary sees: deferring it leaves 1.6% of protected values readable in the released context against 51.0%, and 2.7% against 4.8% in what the
agent emits, for 0.11 helpfulness points. Measuring factual disclosure is harder: a word-overlap metric and an LLM judge both report that mediation leaves facts intact, and both disagree with blind human annotation (kappa = 0.25 and 0.09). The human labels reverse that: inferability falls from 100% to 24-53% under mediation, so semantic-leakage measures need human validation before they are trusted. These are systems results on English text with open-weight models, not formal guarantees.
Read the full abstract ▾
Computer Vision
Sparse Gain Radio Map Reconstruction With Geometry Priors and Uncertainty-Guided Measurement Selection
Zhihan Zeng, Ning Wei, Kaihe Wang et al. (+4 more)
Radio maps are important for environment-aware wireless communication, network planning, and
radio resource optimization. However, dense
radio map construction remains challenging when only a limited number of measurements are available, especially in complex urban environments with strong blockages, irregular geometry, and restricted sensing accessibility. Existing methods have explored interpolation, low-rank cartography, deep completion, and channel knowledge map (CKM) construction, but many of these methods insufficiently exploit explicit
geometric priors or overlook the value of predictive uncertainty for subsequent sensing. In this paper, we study sparse gain
radio map reconstruction from a geometry-aware and active sensing perspective. We first construct UrbanRT-RM, a controllable ray-tracing
benchmark with diverse urban layouts, multiple base-station deployments, and multiple sparse sampling modes. We then propose GeoUQ-GFNet, a lightweight network that jointly predicts a dense gain
radio map and a spatial uncertainty map from sparse measurements and structured scene priors. The predicted uncertainty is further used to guide active measurement selection under limited sensing budgets. Extensive experiments show that our proposed GeoUQ-GFNet method achieves strong and consistent reconstruction performance across different scenes and transmitter placements generated using UrbanRT-RM. Moreover, uncertainty-guided querying provides more effective reconstruction improvement than non-adaptive sampling under the same additional measurement budget. These results demonstrate the effectiveness of combining geometry-aware learning, uncertainty estimation, and
benchmark-driven evaluation for sparse
radio map reconstruction in complex urban environments.
Read the full abstract ▾
All authors Zhihan Zeng, Ning Wei, Kaihe Wang, Phee Lep Yeoh, Fei Xu, Yue Xiu, Zhongpei Zhang
Computer Vision
RHVI-FDD: A Hierarchical Decoupling Framework for Low-Light Image Enhancement
Junhao Yang, Bo Yang, Hongwei Ge et al. (+3 more)
Low-light images often suffer from severe noise, detail loss, and color distortion, which hinder downstream multimedia analysis and
retrieval tasks. The degradation in low-light images is complex: luminance and chrominance are coupled, while within the chrominance, noise and details are deeply entangled, preventing existing methods from simultaneously correcting color distortion, suppressing noise, and preserving fine details. To tackle the above challenges, we propose a novel hierarchical decoupling framework (RHVI-FDD). At the macro level, we introduce the RHVI transform, which mitigates the estimation bias caused by input noise and enables robust luminance-chrominance decoupling. At the micro level, we design a Frequency-Domain Decoupling (FDD) module with three branches for further feature separation. Using the Discrete Cosine Transform, we decompose chrominance features into low, mid, and high-frequency bands that predominantly represent global tone, local details, and noise components, which are then processed by tailored expert networks in a divide-and-conquer manner and fused via an adaptive gating module for content-aware fusion. Extensive experiments on multiple low-light
datasets demonstrate that our method consistently outperforms existing state-of-the-art approaches in both objective metrics and subjective visual quality.
Read the full abstract ▾
All authors Junhao Yang, Bo Yang, Hongwei Ge, Yanchun Liang, Heow Pueh Lee, Chunguo Wu
Computer Vision
Sparsity-Aware Voxel Attention and Foreground Modulation for 3D Semantic Scene Completion
Yu Xue, Longjun Gao, Yuanqi Su et al. (+2 more)
Monocular Semantic Scene Completion (SSC) aims to reconstruct complete 3D semantic scenes from a single RGB image, offering a cost-effective solution for autonomous driving and robotics. However, the inherently imbalanced nature of voxel distributions, where over 93% of voxels are empty and foreground classes are rare, poses significant challenges. Existing methods often suffer from redundant emphasis on uninformative voxels and poor generalization to long-tailed categories. To address these issues, we propose VoxSAMNet (Voxel Sparsity-Aware Modulation Network), a unified framework that explicitly models voxel sparsity and semantic imbalance. Our approach introduces: (1) a Dummy Shortcut for Feature Refinement (DSFR) module that bypasses empty voxels via a shared dummy node while refining occupied ones with deformable
attention; and (2) a Foreground Modulation Strategy combining Foreground Dropout (FD) and Text-Guided Image Filter (TGIF) to alleviate overfitting and enhance class-relevant features. Extensive experiments on the public
benchmarks SemanticKITTI and SSCBench-KITTI-360 demonstrate that VoxSAMNet achieves state-of-the-art performance, surpassing prior
monocular and stereo baselines with mIoU scores of 18.2% and 20.2%, respectively. Our results highlight the importance of sparsity-aware and semantics-guided design for efficient and accurate 3D scene completion, offering a promising direction for future research.
Read the full abstract ▾
All authors Yu Xue, Longjun Gao, Yuanqi Su, HaoAng Lu, Xiaoning Zhang
Computer Vision
PDMP: Rethinking Balanced Multimodal Learning via Performance-Dominant Modality Prioritization
Shicai Wei, Chunbo Luo, Qiang Zhu et al. (+1 more)
Multimodal learning has attracted increasing
attention due to its practicality. However, it often suffers from insufficient optimization, where the
multimodal model underperforms even compared to its unimodal counterparts. Existing methods attribute this problem to the imbalanced learning between modalities and solve it by gradient modulation. This paper argues that balanced learning is not the optimal setting for
multimodal learning. On the contrary, imbalanced learning driven by the performance-dominant modality that has superior unimodal performance can contribute to better
multimodal performance. And the under-optimization problem is caused by insufficient learning of the performance-dominant modality. To this end, we propose the Performance-Dominant Modality Prioritization (PDMP) strategy to assist
multimodal learning. Specifically, PDMP firstly mines the performance-dominant modality via the performance ranking of the independently trained unimodal model. Then PDMP introduces asymmetric coefficients to modulate the gradients of each modality, enabling the performance-dominant modality to dominate the optimization. Since PDMP only relies on the unimodal performance ranking, it is independent of the structures and fusion methods of the
multimodal model and has great potential for practical scenarios. Finally, extensive experiments on various
datasets validate the superiority of PDMP.
Read the full abstract ▾
All authors Shicai Wei, Chunbo Luo, Qiang Zhu, Yang Luo
Computer Vision
Beyond the Beep: Scalable Collision Anticipation and Real-Time Explainability with BADAS-2.0
Roni Goldshmidt, Hamish Scott, Lorenzo Niccolini et al. (+1 more)
We present BADAS-2.0, the second generation of our collision anticipation system, building on BADAS-1.0, which showed that fine-tuning V-JEPA2 on large-scale ego-centric dashcam data outperforms both academic baselines and production ADAS systems. BADAS-2.0 advances the state of the art along three axes. (i) Long-tail
benchmark and accuracy: We introduce a 10-group long-tail
benchmark targeting rare and safety-critical scenarios. To construct it, BADAS-1.0 is used as an active oracle to score millions of unlabeled drives and surface high-risk candidates for annotation. Combined with Nexar's Atlas platform for targeted data collection, this expands the
dataset from 40k to 178,500 labeled
videos (~2M clips), yielding consistent gains across all subgroups, with the largest improvements on the hardest long-tail cases. (ii) Knowledge
distillation to edge: Domain-specific self-supervised pre-training on 2.25M unlabeled driving
videos enables
distillation into compact models, BADAS-2.0-Flash (86M) and BADAS-2.0-Flash-Lite (22M), achieving 7-12x speedup with near-parity accuracy, enabling real-time edge deployment. (iii) Explainability: BADAS-2.0 produces real-time object-centric
attention heatmaps that localize the evidence behind predictions. BADAS-Reason extends this with a
vision-
language model that consumes the last frame and heatmap to generate driver actions and structured textual
reasoning. Inference code and evaluation
benchmarks are publicly available.
Read the full abstract ▾
All authors Roni Goldshmidt, Hamish Scott, Lorenzo Niccolini, Hernan Matzner
Computer Vision
Improving Controllable Generation: Faster Training and Better Performance via $x_0$-Supervision
Amadou S. Sangare, Adrien Maglo, Mohamed Chaouch et al. (+1 more)
Text-to-Image (T2I)
diffusion/flow models have recently achieved remarkable progress in visual fidelity and text
alignment. However, they remain limited when users need to precisely control image layouts, something that natural
language alone cannot reliably express. Controllable generation methods augment the initial T2I model with additional conditions that more easily describe the scene. Prior works straightforwardly train the augmented network with the same loss as the initial network. Although natural at first glance, this can lead to very long training times in some cases before convergence. In this work, we revisit the training objective of controllable
diffusion models through a detailed analysis of their denoising dynamics. We show that direct supervision on the clean target image, dubbed $x_0$-supervision, or an equivalent re-weighting of the
diffusion loss, yields faster convergence. Experiments on multiple control settings demonstrate that our formulation accelerates convergence by up to 2$\times$ according to our novel metric (mean Area Under the Convergence Curve - mAUCC), while also improving both visual quality and conditioning accuracy. Our code is available at https://github.com/CEA-LIST/x0-supervision
Read the full abstract ▾
All authors Amadou S. Sangare, Adrien Maglo, Mohamed Chaouch, Bertrand Luvison
Computer Vision
SVC 2026: the Second Multimodal Deception Detection Challenge and the First Domain Generalized Remote Physiological Measurement Challenge
Dongliang Zhu, Zhiyi Niu, Bo Zhao et al. (+14 more)
Subtle visual signals, although difficult to perceive with the naked eye, contain important information that can reveal hidden patterns in visual data. These signals play a key role in many applications, including biometric security, multimedia forensics, medical diagnosis, industrial inspection, and affective computing. With the rapid development of computer
vision and representation learning techniques, detecting and interpreting such subtle signals has become an emerging research direction. However, existing studies often focus on specific tasks or modalities, and models still face challenges in robustness, representation ability, and generalization when handling subtle and weak signals in real-world environments. To promote research in this area, we organize the Subtle visual Challenge, which aims to learn robust representations for subtle visual signals. The challenge includes two tasks: cross-domain
multimodal deception detection and remote photoplethysmography (rPPG) estimation. We hope that this challenge will encourage the development of more robust and generalizable models for subtle visual understanding, and further advance research in computer
vision and
multimodal learning. A total of 22 teams submitted their final results to this workshop competition, and the corresponding baseline models have been released on the \href{https://sites.google.com/view/svc-cvpr26}{MMDD2026 platform}\footnote{https://sites.google.com/view/svc-cvpr26}
Read the full abstract ▾
All authors Dongliang Zhu, Zhiyi Niu, Bo Zhao, Jiajian Huang, Shuo Ye, Xun Lin, Hui Ma, Taorui Wang, Jiayu Zhang, Chunmei Zhu, Junzhe Cao, Yingjie Ma, Rencheng Song, Albert Clapés, Sergio Escalera, Dan Guo, Zitong Yu