Computer Vision
HeBA: Heterogeneous Bottleneck Adapters for Robust Vision-Language Models
Md Jahidul Islam
Adapting large-scale
Vision-
Language Models (VLMs) like CLIP to downstream tasks often suffers from a "one-size-fits-all" architectural approach, where visual and textual
tokens are processed uniformly by wide, generic adapters. We argue that this homogeneity ignores the distinct structural nature of the modalities – spatial locality in images versus semantic density in text. To address this, we propose HeBA (Heterogeneous Bottleneck Adapter), a unified architectural framework that introduces modality-specific structural inductive biases. HeBA departs from conventional designs through three key architectural innovations: (1) Heterogeneity: It processes visual
tokens via 2D depthwise-separable convolutions to preserve spatial correlations, while distinctively processing text
tokens via dense linear projections to capture semantic relationships; (2) Bottleneck Regularization: Unlike standard expanding adapters, HeBA employs a compression bottleneck (D -> D/4) that explicitly forces the model to learn compact, robust features and acts as a structural regularizer; and (3) Active Gradient Initialization: We challenge the restrictive zero-initialization paradigm, utilizing a Kaiming initialization strategy that ensures sufficient initial gradient flow to accelerate convergence without compromising the frozen backbone's pre-trained knowledge. Extensive experiments demonstrate that HeBA's architecturally specialized design achieves superior stability and accuracy, establishing a new state-of-the-art on 11 few-shot
benchmarks. Code is available at https://github.com/Jahid12012021/VLM-HeBA.
Read the full abstract ▾
Computer Vision
Efficient Brood Cell Detection in Layer Trap Nests for Bees and Wasps: Balancing Labeling Effort and Species Coverage
Chenchang Liu, Felix Fornoff, Annika Grasreiner et al. (+3 more)
Monitoring cavity-nesting wild bees and wasps is vital for biodiversity research and conservation. Layer trap nests (LTNs) are emerging as a valuable tool to study the abundance and species richness of these insects, offering insights into their nesting activities and ecological needs. However, manually evaluating LTNs to detect and classify brood
cells is labor-intensive and time-consuming. To address this, we propose a deep learning based approach for efficient brood
cell detection and classification in LTNs. LTNs present additional challenges due to densely packed brood
cells, leading to a high labeling effort per image. Moreover, we observe a significant imbalance in class distribution, with common species having notably more occurrences than rare species. Comprehensive labeling of common species is time-consuming and exacerbates data imbalance, while partial labeling introduces data incompleteness which degrades model performance. To reduce labeling effort and mitigate the impact of unlabeled data, we introduce a novel Constrained False Positive Loss (CFPL) strategy. CFPL dynamically masks predictions from unlabeled data, preventing them from interfering with the classification loss during training. Experimental results demonstrate that our method improves detection performance, balances model accuracy and labeling effort, while also mitigating class imbalance.
Read the full abstract ▾
All authors Chenchang Liu, Felix Fornoff, Annika Grasreiner, Patrick Maeder, Henri Greil, Marco Seeland
Computer Vision
Mixture of Style Experts for Diverse Image Stylization
Shihao Zhu, Ziheng Ouyang, Yijia Kang et al. (+5 more)
Diffusion-based stylization has advanced significantly, yet existing methods are limited to color-driven transformations, neglecting complex semantics and material details. We introduce StyleExpert, a semantic-aware framework based on the Mixture of Experts (MoE). Our framework employs a unified style encoder, trained on our large-scale
dataset of content-style-stylized triplets, to embed diverse styles into a consistent latent space. This
embedding is then used to condition a similarity-aware gating mechanism, which dynamically routes styles to specialized experts within the MoE architecture. Leveraging this MoE architecture, our method adeptly handles diverse styles spanning multiple semantic levels, from shallow textures to deep semantics. Extensive experiments show that StyleExpert outperforms existing approaches in preserving semantics and material details, while generalizing to unseen styles. Our code and collected images are available at the project page: https://hh-lg.github.io/StyleExpert-Page/.
Read the full abstract ▾
All authors Shihao Zhu, Ziheng Ouyang, Yijia Kang, Qilong Wang, Mi Zhou, Bo Li, Ming-Ming Cheng, Qibin Hou
Computer Vision
BUSSARD: Normalizing Flows for Bijective Universal Scene-Specific Anomalous Relationship Detection
Melissa Schween, Mathis Kruse and Bodo Rosenhahn
We propose Bijective Universal Scene-Specific Anomalous Relationship Detection (BUSSARD), a normalizing flow-based model for detecting anomalous relations in scene graphs, generated from images. Our work follows a
multimodal approach,
embedding object and relationship
tokens from scene graphs with a
language model to leverage semantic knowledge from the real world. A normalizing flow model is used to learn bijective transformations that map object-relation-object triplets from scene graphs to a simple base distribution (typically Gaussian), allowing anomaly detection through likelihood estimation. We evaluate our approach on the SARD
dataset containing office and dining room scenes. Our method achieves around 10% better AUROC results compared to the current state-of-the-art model, while simultaneously being five times faster. Through ablation studies, we demonstrate superior robustness and universality, particularly regarding the use of synonyms, with our model maintaining stable performance while the baseline shows 17.5% deviation. This work demonstrates the strong potential of learning-based methods for relationship anomaly detection in scene graphs. Our code is available at https://github.com/mschween/BUSSARD .
Read the full abstract ▾
Computer Vision
FlowComposer: Composable Flows for Compositional Zero-Shot Learning
Zhenqi He, Lin Li and Long Chen
Compositional zero-shot learning (CZSL) aims to recognize unseen attribute-object compositions by recombining primitives learned from seen pairs. Recent CZSL methods built on
vision-
language models (VLMs) typically adopt parameter-efficient fine-tuning (PEFT). They apply visual disentanglers for decomposition and manipulate
token-level
prompts or prefixes to encode compositions. However, such PEFT-based designs suffer from two fundamental limitations: (1) Implicit Composition Construction, where composition is realized only via
token concatenation or branch-wise
prompt tuning rather than an explicit operation in the
embedding space; (2) Remained Feature
Entanglement, where imperfect disentanglement leaves attribute, object, and composition features mutually contaminated. Together, these issues limit the generalization ability of current CZSL models. In this paper, we are the first to systematically study flow matching for CZSL and introduce FlowComposer, a model-agnostic framework that learns two primitive flows to transport visual features toward attribute and object text
embeddings, and a learnable Composer that explicitly fuses their velocity fields into a composition flow. To exploit the inevitable residual
entanglement, we further devise a leakage-guided augmentation scheme that reuses leaked features as auxiliary signals. We thoroughly evaluate FlowComposer on three public CZSL
benchmarks by integrating it as a plug-and-play component into various baselines, consistently achieving significant improvements.
Read the full abstract ▾
Computer Vision
MLLM-based Textual Explanations for Face Comparison
Redwan Sony, Anil K Jain and Arun Ross
Multimodal Large
Language Models (MLLMs) have recently been proposed as a means to generate natural-
language explanations for face recognition decisions. While such explanations facilitate human interpretability, their reliability on unconstrained face images remains underexplored. In this work, we systematically analyze MLLM-generated explanations for the unconstrained face verification task on the challenging IJB-S
dataset, with a particular focus on extreme pose variation and surveillance imagery. Our results show that even when MLLMs produce correct verification decisions, the accompanying explanations frequently rely on non-verifiable or hallucinated facial attributes that are not supported by visual evidence. We further study the effect of incorporating information from traditional face recognition systems, viz., scores and decisions, alongside the input images. Although such information improves categorical verification performance, it does not consistently lead to faithful explanations. To evaluate the explanations beyond decision accuracy, we introduce a likelihood-ratio-based framework that measures the evidential strength of textual explanations. Our findings highlight fundamental limitations of current MLLMs for explainable face recognition and underscore the need for a principled evaluation of reliable and trustworthy explanations in biometric applications. Code is available at https://github.com/redwankarimsony/LR-MLLMFR-Explainability.
Read the full abstract ▾
Computer Vision
TCATSeg: A Tooth Center-Wise Attention Network for 3D Dental Model Semantic Segmentation
Qiang He, Wentian Qu, Jiajia Dai et al. (+9 more)
Accurate semantic
segmentation of 3D dental models is essential for digital dentistry applications such as orthodontics and dental implants. However, due to complex tooth arrangements and similarities in shape among adjacent teeth, existing methods struggle with accurate
segmentation, because they often focus on local geometry while neglecting global contextual information. To address this, we propose TCATSeg, a novel framework that combines local
geometric features with global semantic context. We introduce a set of sparse yet physically meaningful superpoints to capture global semantic relationships and enhance
segmentation accuracy. Additionally, we present a new
dataset of 400 dental models, including pre-orthodontic samples, to evaluate the generalization of our method. Extensive experiments demonstrate that TCATSeg outperforms state-of-the-art approaches.
Read the full abstract ▾
All authors Qiang He, Wentian Qu, Jiajia Dai, Changsong Lei, Shaofeng Wang, Feifei Zuo, Yajie Wang, Yaqian Liang, Xiaoming Deng, Cuixia Ma, Yong-Jin Liu, Hongan Wang
Computer Vision
ACPV-Net: All-Class Polygonal Vectorization for Seamless Vector Map Generation from Aerial Imagery
Weiqin Jiao, Hao Cheng, George Vosselman et al. (+1 more)
We tackle the problem of generating a complete vector map representation from aerial imagery in a single run: producing polygons for all land-cover classes with shared boundaries and without gaps or overlaps. Existing polygonization methods are typically class-specific; extending them to multiple classes via per-class runs commonly leads to topological inconsistencies, such as duplicated edges, gaps, and overlaps. We formalize this new task as All-Class Polygonal Vectorization (ACPV) and release the first public
benchmark, Deventer-512, with standardized metrics jointly evaluating semantic fidelity,
geometric accuracy, vertex efficiency, per-class topological fidelity and global topological consistency. To realize ACPV, we propose ACPV-Net, a unified framework introducing a novel Semantically Supervised Conditioning (SSC) mechanism coupling semantic perception with
geometric primitive generation, along with a topological reconstruction that enforces shared-edge consistency by design. While enforcing such strict topological constraints, ACPV-Net surpasses all class-specific baselines in polygon quality across classes on Deventer-512. It also applies to single-class polygonal vectorization without any architectural modification, achieving the best-reported results on WHU-Building. Data, code, and models will be released at: https://github.com/HeinzJiao/ACPV-Net.
Read the full abstract ▾
All authors Weiqin Jiao, Hao Cheng, George Vosselman, Claudio Persello
Computer Vision
Rationale Matters: Learning Transferable Rubrics via Proxy-Guided Critique for VLM Reward Models
Weijie Qiu, Dai Guan, Junxin Wang et al. (+6 more)
Generative reward models (GRMs) for
vision-
language models (VLMs) often evaluate outputs via a three-stage pipeline: rubric generation, criterion-based scoring, and a final verdict. However, the intermediate rubric is rarely optimized directly. Prior work typically either treats rubrics as incidental or relies on expensive LLM-as-judge checks that provide no differentiable signal and limited training-time guidance. We propose Proxy-GRM, which introduces proxy-guided rubric verification into Reinforcement Learning (RL) to explicitly enhance rubric quality. Concretely, we train lightweight proxy
agents (Proxy-SFT and Proxy-RL) that take a candidate rubric together with the original query and preference pair, and then predict the preference ordering using only the rubric as evidence. The proxy's prediction accuracy serves as a rubric-quality reward, incentivizing the model to produce rubrics that are internally consistent and transferable. With ~50k data samples, Proxy-GRM reaches state-of-the-art results on the VL-Reward Bench,
Multimodal Reward Bench, and MM-RLHF-Reward Bench, outperforming the methods trained on four times the data. Ablations show Proxy-SFT is a stronger verifier than Proxy-RL, and implicit reward aggregation performs best. Crucially, the learned rubrics transfer to unseen evaluators, improving reward accuracy at test time without additional training. Our code is available at https://github.com/Qwen-Applications/Proxy-GRM.
Read the full abstract ▾
All authors Weijie Qiu, Dai Guan, Junxin Wang, Zhihang Li, Yongbo Gai, Mengyu Zhou, Erchao Zhao, Xiaoxi Jiang, Guanjun Jiang
Computer Vision
FSMC-Pose: Frequency and Spatial Fusion with Multiscale Self-calibration for Cattle Mounting Pose Estimation
Fangjing Li, Zhihai Wang, Xinxin Ding et al. (+5 more)
Mounting posture is an important visual indicator of estrus in dairy cattle. However, achieving reliable mounting pose estimation in real-world environments remains challenging due to cluttered backgrounds and frequent inter-animal occlusion. We present FSMC-Pose, a top-down framework that integrates a lightweight frequency-spatial fusion backbone, CattleMountNet, and a multiscale self-calibration head, SC2Head. Specifically, we design two algorithmic components for CattleMountNet: the Spatial Frequency Enhancement Block (SFEBlock) and the Receptive Aggregation Block (RABlock). SFEBlock separates cattle from cluttered backgrounds, while RABlock captures multiscale contextual information. The Spatial-Channel Self-Calibration Head (SC2Head) attends to spatial and channel dependencies and introduces a self-calibration branch to mitigate structural misalignment under inter-animal overlap. We construct a mounting
dataset, MOUNT-Cattle, covering 1176 mounting instances, which follows the COCO format and supports drop-in training across pose estimation models. Using a comprehensive
dataset that combines MOUNT-Cattle with the public NWAFU-Cattle
dataset, FSMC-Pose achieves higher accuracy than strong baselines, with markedly lower computational and parameter costs, while maintaining real-time inference on commodity GPUs. Extensive experiments and qualitative analyses show that FSMC-Pose effectively captures and estimates cattle mounting pose in complex and cluttered environments.
Dataset and code are available at https://github.com/elianafang/FSMC-Pose.
Read the full abstract ▾
All authors Fangjing Li, Zhihai Wang, Xinxin Ding, Haiyang Liu, Ronghua Gao, Rong Wang, Yao Zhu, Ming Jin
Computer Vision
On the Transfer of Collinearity to Computer Vision
Frederik Beuth and Danny Kowerko
Collinearity is a visual perception phenomenon in the human
brain that amplifies spatially aligned edges arranged along a straight line. However, it is vague for which purpose humans might have this principle in the real-world, and its utilization in computer
vision and engineering applications even is a largely unexplored field. In this work, our goal is to transfer the collinearity principle to computer
vision, and we explore the potential usages of this novel principle for computer
vision applications. We developed a prototype model to exemplify the principle, then tested it systematically, and benchmarked it in the context of four use cases. Our cases are selected to spawn a broad range of potential applications and scenarios: sketching the combination of collinearity with deep learning (case I and II), using collinearity with saliency models (case II), and as a feature detector (case I). In the first use case, we found that collinearity is able to improve the fault detection of wafers and obtain a performance increase by a factor 1.24 via collinearity (decrease of the error rate from 6.5% to 5.26%). In the second use case, we test the defect recognition in nanotechnology materials and achieve a performance increase by 3.2x via collinearity (deep learning, error from 21.65% to 6.64%), and also explore saliency models. As third experiment, we cover occlusions; while as fourth experiment, we test ImageNet and observe that it might not be very beneficial for ImageNet. Therefore, we can assemble a list of scenarios for which collinearity is beneficial (wafers, nanotechnology, occlusions), and for what is not beneficial (ImageNet). Hence, we infer collinearity might be suitable for industry applications as it helps if the image structures of interest are man-made because they often consist of lines. Our work provides another tool for CV, hope to capture the power of human processing.
Read the full abstract ▾
Computer Vision · Cross-listed from q-bio.QM
HistoAtlas: A Pan-Cancer Morphology Atlas Linking Histomics to Molecular Programs and Clinical Outcomes
Pierre-Antoine Bannier
We present HistoAtlas, a pan-cancer computational atlas that extracts 38 interpretable histomic features from 6,745 diagnostic H&E slides across 21 TCGA cancer types and systematically links every feature to survival, gene expression, somatic mutations, and immune subtypes. All associations are
covariate-adjusted, multiple-testing corrected, and classified into evidence-strength tiers. The atlas recovers known biology, from immune infiltration and prognosis to proliferation and kinase signaling, while uncovering compartment-specific immune signals and morphological subtypes with divergent outcomes. Every result is spatially traceable to tissue compartments and individual
cells, statistically calibrated, and openly queryable. HistoAtlas enables systematic, large-scale biomarker discovery from routine H&E without specialized staining or sequencing. Data and an interactive web atlas are freely available at https://histoatlas.com .
Read the full abstract ▾
Computer Vision
REFORGE: Multi-modal Attacks Reveal Vulnerable Concept Unlearning in Image Generation Models
Yong Zou, Haoran Li, Fanxiao Li et al. (+5 more)
Recent progress in image generation models (IGMs) enables high-fidelity content creation but also amplifies risks, including the reproduction of copyrighted content and the generation of offensive content. Image Generation Model Unlearning (IGMU) mitigates these risks by removing harmful concepts without full retraining. Despite growing
attention, the robustness under
adversarial inputs, particularly image-side threats in black-box settings, remains underexplored. To bridge this gap, we present REFORGE, a black-box red-teaming framework that evaluates IGMU robustness via
adversarial image
prompts. REFORGE initializes stroke-based images and optimizes perturbations with a cross-
attention-guided masking strategy that allocates noise to concept-relevant regions, balancing attack efficacy and visual fidelity. Extensive experiments across representative unlearning tasks and defenses demonstrate that REFORGE significantly improves attack success rate while achieving stronger semantic
alignment and higher efficiency than involved baselines. These results expose persistent vulnerabilities in current IGMU methods and highlight the need for robustness-aware unlearning against multi-modal
adversarial attacks. Our code is at: https://github.com/Imfatnoily/REFORGE.
Read the full abstract ▾
All authors Yong Zou, Haoran Li, Fanxiao Li, Shenyang Wei, Yunyun Dong, Li Tang, Wei Zhou, Renyang Liu
Computer Vision
Face2Scene: Using Facial Degradation as an Oracle for Diffusion-Based Scene Restoration
Amirhossein Kazerouni, Maitreya Suin, Tristan Aumentado-Armstrong et al. (+6 more)
Recent advances in image restoration have enabled high-fidelity recovery of faces from degraded inputs using reference-based face restoration models (Ref-FR). However, such methods focus solely on facial regions, neglecting degradation across the full scene, including body and background, which limits practical usability. Meanwhile, full-scene restorers often ignore degradation cues entirely, leading to underdetermined predictions and visual artifacts. In this work, we propose Face2Scene, a two-stage restoration framework that leverages the face as a perceptual oracle to estimate degradation and guide the restoration of the entire image. Given a degraded image and one or more identity references, we first apply a Ref-FR model to reconstruct high-quality facial details. From the restored-degraded face pair, we extract a face-derived degradation code that captures degradation attributes (e.g., noise, blur, compression), which is then transformed into multi-scale degradation-aware
tokens. These
tokens condition a
diffusion model to restore the full scene in a single step, including the body and background. Extensive experiments demonstrate the superior effectiveness of the proposed method compared to state-of-the-art methods.
Read the full abstract ▾
All authors Amirhossein Kazerouni, Maitreya Suin, Tristan Aumentado-Armstrong, Sina Honari, Amanpreet Walia, Iqbal Mohomed, Konstantinos G. Derpanis, Babak Taati, Alex Levinshtein
Computer Vision
VideoMatGen: PBR Materials through Joint Generative Modeling
Jon Hasselgren, Zheng Zeng, Milos Hasan et al. (+1 more)
We present a method for generating physically-based materials for 3D shapes based on a
video diffusion transformer architecture. Our method is conditioned on input geometry and a text description, and jointly models multiple material properties (base color, roughness,
metallicity, height map) to form physically plausible materials. We further introduce a custom variational auto-encoder which encodes multiple material modalities into a compact latent space, which enables joint generation of multiple modalities without increasing the number of
tokens. Our pipeline generates high-quality materials for 3D shapes given a text
prompt, compatible with common content creation tools.
Read the full abstract ▾
All authors Jon Hasselgren, Zheng Zeng, Milos Hasan, Jacob Munkberg
Computer Vision
Understanding Cell Fate Decisions with Temporal Attention
Florian Bürger, Martim Dias Gomes, Adrián E. Granada et al. (+2 more)
Understanding non-genetic determinants of
cell fate is critical for developing and improving cancer therapies, as genetically identical
cells can exhibit divergent outcomes under the same treatment conditions. In this work, we present a deep learning approach for
cell fate prediction from raw long-term live-
cell recordings of cancer
cell populations under chemotherapeutic treatment. Our
Transformer model is trained to predict
cell fate directly from raw image sequences, without relying on predefined morphological or molecular features. Beyond classification, we introduce a comprehensive explainability framework for interpreting the temporal and morphological cues guiding the model's predictions. We demonstrate that prediction of
cell outcomes is possible based on the
video only, our model achieves balanced accuracy of 0.94 and an F1-score of 0.93.
Attention and masking experiments further indicate that the signal predictive of the
cell fate is not uniquely located in the final frames of a
cell trajectory, as reliable predictions are possible up to 10 h before the event. Our analysis reveals distinct temporal distribution of predictive information in the mitotic and apoptotic sequences, as well as the role of
cell morphology and p53 signaling in determining
cell outcomes. Together, these findings demonstrate that
attention-based temporal models enable accurate
cell fate prediction while providing biologically interpretable insights into non-genetic determinants of cellular decision-making. The code is available at https://github.com/bozeklab/
Cell-Fate-Prediction.
Read the full abstract ▾
All authors Florian Bürger, Martim Dias Gomes, Adrián E. Granada, Noémie Moreau, Katarzyna Bozek
Computer Vision
Jiale Song, Jiaxin Luo, Xue-song Tang et al. (+2 more)
Read the full abstract ▾
All authors Jiale Song, Jiaxin Luo, Xue-song Tang, Kuangrong Hao, Mingbo Zhao
Computer Vision
CompDiff enables fair and zero shot medical image generation across demographic intersections through compositional diffusion
Mahmoud K. Ibrahim, Bart Elen, Chang Sun et al. (+5 more)
Medical image generators trained on imbalanced data can fail at demographic intersections absent from training. We introduce CompDiff, which encodes age, sex and race separately and composes supervised demographic
tokens alongside clinical text. Across chest radiographs and fundus images, CompDiff improves overall and subgroup fidelity relative to
prompt conditioning (RoentGen-v2) and loss reweighting (FairDiffusion). It generalises in zero-shot generation to 16 chest X-ray intersections excluded from training, achieving the lowest mean FID-RadImageNet in every intersection. In a blinded reader study of these unseen intersections, two radiologists gave CompDiff the highest mean scores among generators for anatomical realism and agreement with the clinical impression, and selected its images most often as the most realistic. Pretraining with CompDiff images improved downstream classification, while CompDiff audit cohorts reduced estimation error on rare intersections. These findings support compositional demographic conditioning for extending medical image synthesis to underserved populations. Code: https://github.com/mahmoudibrahim98/CompDiff
Read the full abstract ▾
All authors Mahmoud K. Ibrahim, Bart Elen, Chang Sun, Ahmad Jiblawi, Mohammed M. Saleh, Maryam K. Ibrahim, Gökhan Ertaylan, Michel Dumontier
Computer Vision
Bridging the Simulation-to-Reality Gap in Electron Microscope Calibration via VAE-EM Estimation
Jilles S. van Hulst, W.P.M.H. Heemels and Duarte J. Antunes
Electron microscopy has enabled many scientific breakthroughs across multiple fields. A key challenge is the tuning of microscope parameters based on images to overcome optical aberrations that deteriorate image quality. This calibration problem is challenging due to the high-dimensional and noisy nature of the diagnostic images, and the fact that optimal parameters cannot be identified from a single image. We tackle the calibration problem for Scanning Transmission Electron Microscopes (STEM) by employing variational autoencoders (VAEs), trained on simulated data, to learn low-dimensional representations of images, whereas most existing methods extract only scalar values. We then simultaneously estimate the model that maps calibration parameters to encoded representations and the optimal calibration parameters using an expectation maximization (EM) approach. This joint estimation explicitly addresses the simulation-to-reality gap inherent in data-driven methods that train on simulated data from a digital twin. We leverage the known
symmetry property of the optical system to establish global identifiability of the joint estimation problem, ensuring that a unique optimum exists. We demonstrate that our approach is substantially faster and more consistent than existing methods on a real STEM, achieving a 2x reduction in estimation error while requiring fewer observations. This represents a notable advance in automated STEM calibration and demonstrates the potential of VAEs for information compression in images. Beyond microscopy, the VAE-EM framework applies to inverse problems where simulated training data introduces a reality gap and where non-injective mappings would otherwise prevent unique solutions.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CR
SAMSEM – A Generic and Scalable Approach for IC Metal Line Segmentation
Christian Gehrmann, Jonas Ricker, Simon Damm et al. (+5 more)
In light of globalized hardware supply chains, the assurance of hardware components has gained significant interest, particularly in cryptographic applications and high-stakes scenarios. Identifying metal lines on scanning electron microscope (SEM) images of integrated
circuits (ICs) is one essential step in verifying the absence of malicious circuitry in chips manufactured in untrusted environments. Due to varying manufacturing processes and technologies, such verification usually requires tuning parameters and
algorithms for each target IC. Often, a machine learning model trained on images of one IC fails to accurately detect metal lines on other ICs. To address this challenge, we create SAMSEM by adapting Meta's Segment Anything Model 2 (SAM2) to the domain of IC metal line
segmentation. Specifically, we develop a multi-scale
segmentation approach that can handle SEM images of varying sizes, resolutions, and magnifications. Furthermore, we deploy a
topology-based loss alongside pixel-based losses to focus our
segmentation on electrical connectivity rather than pixel-level accuracy. Based on a hyperparameter optimization, we then fine-tune the SAM2 model to obtain a model that generalizes across different technology nodes, manufacturing materials, sample preparation methods, and SEM imaging technologies. To this end, we leverage an unprecedented
dataset of SEM images obtained from 48 metal layers across 14 different ICs. When fine-tuned on seven ICs, SAMSEM achieves an error rate as low as 0.72% when evaluated on other images from the same ICs. For the remaining seven unseen ICs, it still achieves error rates as low as 5.53%. Finally, when fine-tuned on all 14 ICs, we observe an error rate of 0.62%. Hence, SAMSEM proves to be a reliable tool that significantly advances the frontier in metal line
segmentation, a key challenge in post-manufacturing IC verification.
Read the full abstract ▾
All authors Christian Gehrmann, Jonas Ricker, Simon Damm, Deruo Cheng, Julian Speith, Yiqiong Shi, Asja Fischer, Christof Paar
Computer Vision
Rethinking Pose Refinement in 3D Gaussian Splatting under Pose Prior and Geometric Uncertainty
Mangyu Kong, Jaewon Lee, Seongwon Lee et al. (+1 more)
3D Gaussian Splatting (3DGS) has recently emerged as a powerful scene representation and is increasingly used for visual localization and pose refinement. However, despite its high-quality differentiable rendering, the robustness of 3DGS-based pose refinement remains highly sensitive to both the initial camera pose and the reconstructed geometry. In this work, we take a closer look at these limitations and identify two major sources of uncertainty: (i) pose prior uncertainty, which often arises from regression or
retrieval models that output a single deterministic estimate, and (ii)
geometric uncertainty, caused by imperfections in the 3DGS reconstruction that propagate errors into PnP solvers. Such uncertainties can distort reprojection geometry and destabilize optimization, even when the rendered appearance still looks plausible. To address these uncertainties, we introduce a relocalization framework that combines Monte Carlo pose sampling with Fisher Information-based PnP optimization. Our method explicitly accounts for both pose and
geometric uncertainty and requires no retraining or additional supervision. Across diverse indoor and outdoor
benchmarks, our approach consistently improves localization accuracy and significantly increases stability under pose and depth noise.
Read the full abstract ▾
All authors Mangyu Kong, Jaewon Lee, Seongwon Lee, Euntai Kim
Computer Vision
An Approximate Graph Elicits Detonation Lattice
Vansh Sharma and Venkat Raman
This study presents a novel
algorithm based on graph theory for the precise
segmentation and measurement of detonation
cells from 3D pressure traces, termed detonation
lattices, addressing the limitations of manual and primitive 2D edge detection methods prevalent in the field. Using a
segmentation model, the proposed training-free
algorithm is designed to accurately extract cellular patterns, a longstanding challenge in detonations research. First, the efficacy of
segmentation phase on two synthetic
datasets is evaluated with an error of 2%. Next, 3D simulation data is used to establish performance of the graph-based workflow. The results of statistics and joint probability densities show oblong
cells aligned with the wave propagation axis with 17% deviation, whereas larger dispersion in volume reflects cubic amplification of linear variability. Although the framework is robust, it remains challenging to reliably segment and quantify highly complex cellular patterns. However, the graph-based formulation generalizes across diverse cellular geometries, positioning it as a practical tool for detonation analysis and a strong foundation for future extensions in triple-point collision studies.
Read the full abstract ▾
Computer Vision
VIEW2SPACE: Studying Multi-View Visual Reasoning from Sparse Observations
Fucai Ke, Zhixi Cai, Boying Li et al. (+6 more)
Multi-view visual
reasoning is essential for intelligent systems that must understand complex environments from sparse and discrete viewpoints, yet existing research has largely focused on single-image or temporally dense
video settings. In real-world scenarios,
reasoning across views requires integrating partial observations without explicit guidance, while collecting large-scale multi-view data with accurate
geometric and semantic annotations remains challenging. To address this gap, we leverage physically grounded simulation to construct diverse, high-fidelity 3D scenes with precise per-view metadata, enabling scalable data generation that remains transferable to real-world settings. Based on this engine, we introduce VIEW2SPACE, a multi-dimensional
benchmark for sparse multi-view
reasoning, together with a scalable, disjoint training split supporting millions of grounded question-answer pairs. Using this
benchmark, a comprehensive evaluation of state-of-the-art
vision-
language and spatial models reveals that multi-view
reasoning remains largely unsolved, with most models performing only marginally above random guessing. We further investigate whether training can bridge this gap. Our proposed Grounded Chain-of-Thought with Visual Evidence substantially improves performance under moderate difficulty, and generalizes to real-world data, outperforming existing approaches in cross-
dataset evaluation. We further conduct difficulty-aware scaling analyses across model size, data scale,
reasoning depth, and visibility constraints, indicating that while
geometric perception can benefit from scaling under sufficient visibility, deep compositional
reasoning across sparse views remains a fundamental challenge.
Read the full abstract ▾
All authors Fucai Ke, Zhixi Cai, Boying Li, Long Chen, Beibei Lin, Weiqing Wang, Pari Delir Haghighi, Gholamreza Haffari, Hamid Rezatofighi
Computer Vision
Unlearning for One-Step Generative Models via Unbalanced Optimal Transport
Hyundo Choi, Junhyeong An, Jinseong Park et al. (+1 more)
Recent advances in one-step generative frameworks, such as flow map models, have significantly improved the efficiency of image generation by learning direct noise-to-data mappings in a single forward pass. However, machine unlearning for ensuring the safety of these powerful generators remains entirely unexplored. Existing
diffusion unlearning methods are inherently incompatible with these one-step models, as they rely on a multi-step iterative denoising process. In this work, we propose UOT-Unlearn, a novel plug-and-play class unlearning framework for one-step generative models based on the Unbalanced Optimal Transport (UOT). Our method formulates unlearning as a principled trade-off between a forget cost, which suppresses the target class, and an $f$-divergence penalty, which preserves overall generation fidelity via relaxed marginal constraints. By leveraging UOT, our method enables the probability mass of the forgotten class to be smoothly redistributed to the remaining classes, rather than collapsing into low-quality or noise-like samples. Experimental results on CIFAR-10 and ImageNet-256 demonstrate that our framework achieves superior unlearning success (PUL) and retention quality (u-FID), significantly outperforming baselines.
Read the full abstract ▾
All authors Hyundo Choi, Junhyeong An, Jinseong Park, Jaewoong Choi
Computer Vision
Cue-guided dual-path reconstruction with adaptive multi-scale response refinement for low-light image enhancement
Yicui Shi, Yuhan Chen, Xiangfei Huang et al. (+5 more)
Low-light image enhancement (LLIE) supports optical sensing and scene observation when limited
photon capture and sensor noise degrade image quality. However, existing methods can introduce colour deviations or blur local details in the enhanced images. To address this challenge, we propose CueDR-Net, a dual-path network that combines brightness adjustment with local content reconstruction through bidirectional cue interaction and adaptive multi-scale refinement. Specifically, a fixed extractor derives luminance variation, colour, and texture cue maps from the input image. These maps are encoded in a cue stream that exchanges features with the image stream through bidirectional
attention. The exchanged features support two paths for brightness adjustment and local content reconstruction. The brightness adjustment path predicts pixel wise adjustment maps that control iterative intensity changes in the input image. In parallel, the residual path uses the exchanged features to predict an additive correction for local content reconstruction. To capture weak structures across scales, Adaptive Multi-Scale Response Refinement Convolution (AMRRConv) refines the features used for both predictions. It fuses learnable local transition responses with multi-scale features from factorized three-plane filtering using pixel wise weights. The adjusted image is added to the correction before being mapped to the enhanced image. Evaluations on LOLv1, LSRW-HUAWEI, and LSRW-NIKON demonstrate competitive enhancement performance, supported by ablation studies. AMRRConv also achieves competitive low-light detection performance with a small neck parameter count when integrated into YOLOv11.
Read the full abstract ▾
All authors Yicui Shi, Yuhan Chen, Xiangfei Huang, Zhenguo Wang, Wenxuan Yu, Ying Fang, Jie Li, Guofa Li
Computer Vision
GAP-MLLM: Geometry-Aligned Pre-training for Activating 3D Spatial Perception in Multimodal Large Language Models
Jiaxin Zhang, Junjun Jiang, Haijie Li et al. (+3 more)
Multimodal Large
Language Models (MLLMs) demonstrate exceptional semantic
reasoning but struggle with 3D spatial perception when restricted to pure RGB inputs. Despite leveraging implicit
geometric priors from 3D reconstruction models, image-based methods still exhibit a notable performance gap compared to methods using explicit 3D data. We argue that this gap does not arise from insufficient
geometric priors, but from a misalignment in the training paradigm: text-dominated fine-tuning fails to activate
geometric representations within MLLMs. Existing approaches typically resort to naive feature concatenation and optimize directly for downstream tasks without geometry-specific supervision, leading to suboptimal structural utilization. To address this limitation, we propose GAP-MLLM, a Geometry-Aligned Pre-training paradigm that explicitly activates structural perception before downstream adaptation. Specifically, we introduce a visual-prompted joint task that compels the MLLMs to predict sparse pointmaps alongside semantic labels, thereby enforcing
geometric awareness. Furthermore, we design a multi-level progressive fusion module with a
token-level gating mechanism, enabling adaptive integration of
geometric priors without suppressing semantic
reasoning. Extensive experiments demonstrate that GAP-MLLM significantly enhances
geometric feature fusion and consistently enhances performance across 3D visual grounding, 3D dense captioning, and 3D
video object detection tasks.
Read the full abstract ▾
All authors Jiaxin Zhang, Junjun Jiang, Haijie Li, Youyu Chen, Kui Jiang, Dave Zhenyu Chen
Computer Vision
Evo-Retriever: LLM-Guided Curriculum Evolution with Viewpoint-Pathway Collaboration for Multimodal Document Retrieval
Weiqing Li, Jinyue Guo, Yaqi Wang et al. (+4 more)
Visual-
language models (VLMs) excel at data mappings, but real-world document heterogeneity and unstructuredness disrupt the consistency of cross-modal
embeddings. Recent late-interaction methods enhance image-text
alignment through multi-vector representations, yet traditional training with limited samples and static strategies cannot adapt to the model's dynamic
evolution, causing cross-modal
retrieval confusion. To overcome this, we introduce Evo-Retriever, a
retrieval framework featuring an LLM-guided curriculum
evolution built upon a novel Viewpoint-Pathway collaboration. First, we employ multi-view image
alignment to enhance fine-grained matching via multi-scale and multi-directional perspectives. Then, a bidirectional contrastive learning strategy generates "hard queries" and establishes complementary learning paths for visual and textual disambiguation to rebalance supervision. Finally, the model-state summary from the above collaboration is fed into an LLM meta-controller, which adaptively adjusts the training curriculum using expert knowledge to promote the model's
evolution. On ViDoRe V2 and MMEB (VisDoc), Evo-Retriever achieves state-of-the-art performance, with nDCG@5 scores of 65.2% and 77.1%.
Read the full abstract ▾
All authors Weiqing Li, Jinyue Guo, Yaqi Wang, Haiyang Xiao, Yuewei Zhang, Guohua Liu, Hao Henry Wang
Computer Vision
TinyGLASS: Real-Time Self-Supervised In-Sensor Anomaly Detection
Pietro Bonazzi, Rafael Sutter, Luigi Capogrosso et al. (+2 more)
Anomaly detection plays a key role in industrial quality control, where defects must be identified despite the scarcity of labeled faulty samples. Recent self-supervised approaches, such as GLASS, learn normal visual patterns using only defect-free data and have shown strong performance on industrial
benchmarks. However, their computational requirements limit their deployment on resource-constrained edge platforms, especially within in-sensor processing architectures. This work introduces TinyGLASS, a lightweight adaptation of the GLASS framework designed for real-time edge and in-sensor anomaly detection. The proposed architecture replaces the original WideResNet-50 backbone with a compact ResNet-18 and introduces deployment-based modifications that enable static graph tracing and INT8 quantization. We evaluate the proposed approach on the Sony IMX500 intelligent
vision sensor, exploiting the in-sensor processor using the Sony Model Compression Toolkit. In addition to evaluating performance on the MVTec-AD
benchmark, we investigate robustness to contaminated training data and introduce a custom industrial
dataset, named MMS
Dataset, for cross-device evaluation. Experimental results show that TinyGLASS achieves 8.6x parameter compression while maintaining competitive detection performance, reaching 94.2% image-level AUROC on MVTec-AD and operating at 20 FPS within the 8 MB
memory constraints of the IMX500 platform. System profiling showcases low power consumption (4.0 mJ per inference), real-time end-to-end throughput (20 FPS), and high energy efficiency (470 GMAC/J). Furthermore, the model demonstrates stable performance under moderate levels of training data contamination.
Read the full abstract ▾
All authors Pietro Bonazzi, Rafael Sutter, Luigi Capogrosso, Mischa Buob, Michele Magno
Computer Vision
ProgressiveAvatars: Progressive Animatable 3D Gaussian Avatars
Kaiwen Song, Jinkai Cui and Juyong Zhang
In practical real-time XR and telepresence applications, network and computing resources fluctuate frequently. Therefore, a progressive 3D representation is needed. To this end, we propose ProgressiveAvatars, a progressive avatar representation built on a hierarchy of 3D Gaussians grown by adaptive implicit subdivision on a template mesh. 3D Gaussians are defined in face-local coordinates to remain animatable under varying expressions and head motion across multiple detail levels. The hierarchy expands when screen-space signals indicate a lack of detail, allocating resources to important areas. Leveraging importance ranking, ProgressiveAvatars supports incremental loading and rendering, adding new Gaussians as they arrive while preserving previous content, thus achieving smooth quality improvements across varying bandwidths. ProgressiveAvatars enables progressive delivery and progressive rendering under fluctuating network bandwidth and varying compute and
memory resources.
Read the full abstract ▾
Computer Vision
Unified Removal of Raindrops and Reflections: A New Benchmark and A Novel Pipeline
Xingyu Liu, Zewei He, Yu Chen et al. (+4 more)
When capturing images through glass surfaces or windshields on rainy days, raindrops and reflections frequently co-occur to significantly reduce the visibility of captured images. This practical problem lacks
attention and needs to be resolved urgently. Prior de-raindrop, de-reflection, and all-in-one models have failed to address this composite degradation. To this end, we first formally define the unified removal of raindrops and reflections (UR$^3$) task for the first time and construct a real-shot
dataset, namely RainDrop and ReFlection (RDRF), which provides a new
benchmark with substantial, high-quality, diverse image pairs. Then, we propose a novel
diffusion-based framework (i.e., DiffUR$^3$) with several target designs to address this challenging task. By leveraging the powerful generative prior, DiffUR$^3$ successfully removes both types of degradations. Extensive experiments demonstrate that our method achieves state-of-the-art performance on our
benchmark and on challenging in-the-wild images.
Read the full abstract ▾
All authors Xingyu Liu, Zewei He, Yu Chen, Chunyu Zhu, Zixuan Chen, Xing Luo, Zhe-Ming Lu