Computer Vision
Rethinking Image-to-3D Generation with Sparse Queries: Efficiency, Capacity, and Input-View Bias
Zhiyuan Xu, Jiuming Liu, Yuxin Chen et al. (+3 more)
We present SparseGen, a novel framework for efficient image-to-3D generation, which exhibits low input-view bias while being significantly faster. Unlike traditional approaches that rely on dense volumetric grids, triplanes, or pixel-aligned primitives, we model scenes with a compact sparse set of learned 3D anchor queries and a learned expansion operator that decodes each transformed query into a small local set of 3D Gaussian primitives. Trained under a rectified-flow reconstruction objective without 3D supervision, our model learns to allocate representation capacity where geometry and appearance matter, achieving significant reductions in
memory and inference time while preserving multi-view fidelity. We introduce quantitative measures of input-view bias and utilization to show that sparse queries reduce overfitting to conditioning views while being representationally efficient. Our results argue that sparse set-latent expansion is a principled, practical alternative for efficient 3D generative modeling.
Read the full abstract ▾
All authors Zhiyuan Xu, Jiuming Liu, Yuxin Chen, Masayoshi Tomizuka, Chenfeng Xu, Chensheng Peng
Computer Vision
Context Sensitivity Improves Human-Machine Visual Alignment
Frieda Born, Tom Neuhäuser, Lukas Muttenthaler et al. (+6 more)
Modern machine learning models typically represent inputs as fixed points in a high-dimensional
embedding space. While this approach has been proven powerful for a wide range of downstream tasks, it fundamentally differs from the way humans process information. Because humans are constantly adapting to their environment, they represent objects and their relationships in a highly context-sensitive manner. To address this gap, we propose a method for context-sensitive similarity computation from neural network
embeddings, applied to modeling a triplet odd-one-out task with an anchor image serving as simultaneous context. Modeling context enables us to achieve up to a 15% improvement in odd-one-out accuracy over a context-insensitive model. We find that this improvement is consistent across both original and "human-aligned"
vision foundation models.
Read the full abstract ▾
All authors Frieda Born, Tom Neuhäuser, Lukas Muttenthaler, Brett D. Roads, Bernhard Spitzer, Andrew K. Lampinen, Matt Jones, Klaus-Robert Müller, Michael C. Mozer
Computer Vision
PostureObjectstitch: Anomaly Image Generation Considering Assembly Relationships in Industrial Scenarios
Zebei Tong, Hongchang Chen, Yujie Lei et al. (+7 more)
Image generation technology can synthesize condition-specific images to supplement real-world industrial anomaly data and enhance anomaly detection model performance. Existing generation techniques rarely account for the pose and orientation of industrial components in assembly, making the generated images difficult to utilize for downstream application. To solve this, we propose a novel image synthesis approach, called PostureObjectStitch, that achieves accurate generation to meet the requirement of industrial assembly. A condition decoupling approach is introduced to separate input multi-view images into high-frequency, texture, and RGB features. The feature temporal modulation mechanism adapts these features across
diffusion model time-steps, enabling progressive generation from coarse to fine details while maintaining consistency. To ensure semantic accuracy, we introduce a conditional loss that enhances critical industrial elements and a
geometric prior that guides component positioning for correct assembly relationships. Comprehensive experimental results on the MureCom
dataset, our newly contributed DreamAssembly
dataset, and the downstream application validate the outstanding performance of our method.
Read the full abstract ▾
All authors Zebei Tong, Hongchang Chen, Yujie Lei, Gang Chen, Yushi Liu, Zhi Zheng, Hao Chen, Jieming Zhang, Ying Li, Dongpu Cao
Computer Vision
Any3DAvatar: Fast and High-Quality Full-Head 3D Avatar Reconstruction from Single Portrait Image
Yujie Gao, Yao Xiao, Xiangnan Zhu et al. (+4 more)
Reconstructing a complete 3D head from a single portrait remains challenging because existing methods still face a sharp quality-speed trade-off: high-fidelity pipelines often rely on multi-stage processing and per-subject optimization, while fast feed-forward models struggle with complete geometry and fine appearance details. To bridge this gap, we propose Any3DAvatar, a fast and high-quality method for single-image 3D Gaussian head avatar generation, whose fastest setting reconstructs a full head in under one second while preserving high-fidelity geometry and texture. First, we build AnyHead, a unified data suite that combines identity diversity, dense multi-view supervision, and realistic accessories, filling the main gaps of existing head data in coverage, full-head geometry, and complex appearance. Second, rather than sampling unstructured noise, we initialize from a Plücker-aware structured 3D Gaussian scaffold and perform one-step conditional denoising, formulating full-head reconstruction into a single forward pass while retaining high fidelity. Third, we introduce auxiliary view-conditioned appearance supervision on the same latent
tokens alongside 3D Gaussian reconstruction, improving novel-view texture details at zero extra inference cost. Experiments show that Any3DAvatar outperforms prior single-image full-head reconstruction methods in rendering fidelity while remaining substantially faster.
Read the full abstract ▾
All authors Yujie Gao, Yao Xiao, Xiangnan Zhu, Ya Li, Yiyi Zhang, Liqing Zhang, Jianfu Zhang
Computer Vision
DiffMagicFace: Identity Consistent Facial Editing of Real Videos
Huanghao Yin, Shenkun Xu, Kanle Shi et al. (+2 more)
Text-conditioned image editing has greatly benefitted from the advancements in Image
Diffusion Models. However, extending these techniques to facial
video editing introduces challenges in preserving facial identity throughout the source
video and ensuring consistency of the edited subject across frames. In this paper, we introduce DiffMagicFace, a unique
video editing framework that integrates two fine-tuned models for text and image control. These models operate concurrently during inference to produce
video frames that maintain identity features while seamlessly aligning with the editing semantics. To ensure the consistency of the edited
videos, we develop a
dataset comprising images showcasing various facial perspectives for each edited subject. The creation of a data set is achieved through rendering techniques and the subsequent application of optimization
algorithms. Remarkably, our approach does not depend on
video datasets but still delivers high-quality results in both consistency and content. The excellent effect holds even for complex tasks like talking head
videos and distinguishing closely related categories. The
videos edited using our framework exhibit parity with
videos that are made using traditional rendering software. Through comparative analysis with current state-of-the-art methods, our framework demonstrates superior performance in both visual appeal and quantitative metrics.
Read the full abstract ▾
All authors Huanghao Yin, Shenkun Xu, Kanle Shi, Junhai Yong, Bin Wang
Computer Vision
A Resource-Efficient Hybrid CNN-LSTM network for image-based bean leaf disease classification
Hye Jin Rhee and Joseph Damilola Akinyemi
Accurate and resource-efficient automated diagnosis is a cornerstone of modern agricultural expert systems. While Convolutional Neural Networks (CNNs) have established
benchmarks in plant pathology, their ability to capture long-range spatial dependencies is often limited by standard pooling layers, and their high
memory footprint hinders deployment on portable devices. This paper proposes a lightweight hybrid CNN-LSTM system for bean leaf disease classification. By integrating an LSTM layer to model the spatial-sequential relationships within feature maps, our hybrid architecture achieves a 94.38% accuracy while maintaining an exceptionally small footprint of 1.86 MB; a 70% reduction in size compared to traditional CNN-based systems. Furthermore, we provide a systematic evaluation of image augmentation strategies, demonstrating that tailored transformations are superior to generic combinations for maintaining the integrity of diagnostic patterns. Results on the $\textit{ibean}$
dataset confirm that the proposed system achieves state-of-the-art F1 scores of 99.22% with EfficientNet-B7+LSTM, providing a robust and scalable framework for real-time agricultural decision support in resource-constrained environments. The code and augmented
datasets used in this study are publicly available on this $\href{https://github.com/HJin-R/bean_disease}{Github}$ repo.
Read the full abstract ▾
Computer Vision
Gaslight, Gatekeep, V1-V3: Early Visual Cortex Alignment Shields Vision-Language Models from Sycophantic Manipulation
Arya Shah, Vaibhav Tripathi, Mayank Singh et al. (+1 more)
Vision-
language models are increasingly deployed in high-stakes settings, yet their susceptibility to sycophantic manipulation remains poorly understood, particularly in relation to how these models represent visual information internally. Whether models whose visual representations more closely mirror human neural processing are also more resistant to
adversarial pressure is an open question with implications for both neuroscience and AI safety. We investigate this question by evaluating 12 open-weight
vision-
language models spanning 6 architecture families and a 40$\times$ parameter range (256M–10B) along two axes:
brain alignment, measured by predicting fMRI responses from the Natural Scenes
Dataset across 8 human subjects and 6 visual cortex regions of interest, and sycophancy, measured through 76,800 two-turn gaslighting
prompts spanning 5 categories and 10 difficulty levels. Region-of-interest analysis reveals that
alignment specifically in early visual cortex (V1–V3) is a reliable negative predictor of sycophancy ($r = -0.441$, BCa 95% CI $[-0.740, -0.031]$), with all 12 leave-one-out correlations negative and the strongest effect for existence denial attacks ($r = -0.597$, $p = 0.040$). This anatomically specific relationship is absent in higher-order category-selective regions, suggesting that faithful low-level visual encoding provides a measurable anchor against
adversarial linguistic override in
vision-
language models. We release our code on \href{https://github.com/aryashah2k/Gaslight-Gatekeep-Sycophantic-Manipulation}{GitHub} and
dataset on \href{https://huggingface.co/
datasets/aryashah00/Gaslight-Gatekeep-V1-V3}{Hugging Face}
Read the full abstract ▾
All authors Arya Shah, Vaibhav Tripathi, Mayank Singh, Chaklam Silpasuwanchai
Computer Vision
DRG-Font: Dynamic Reference-Guided Few-shot Font Generation via Contrastive Style-Content Disentanglement
Rejoy Chakraborty, Prasun Roy, Saumik Bhattacharya et al. (+1 more)
Few-shot Font Generation aims to generate stylistically consistent glyphs from a few reference glyphs. However, capturing complex font styles from a few exemplars remains challenging, and the existing methods often struggle to retain discernible local characteristics in generated samples. This paper introduces DRG-Font, a contrastive font generation strategy that learns complex glyph attributes by decomposing style and content
embedding spaces. For optimal style supervision, the proposed architecture incorporates a Reference Selection (RS) Module to dynamically select the best style reference from an available pool of candidates. The network learns to decompose glyph attributes into style and shape priors through a Multi-scale Style Head Block (MSHB) and a Multi-scale Content Head Block (MCHB). For style adaptation, a Multi-Fusion Upsampling Block (MFUB) produces the target glyph by combining the reference style prior and target content prior. The proposed method demonstrates significant improvements over state-of-the-art approaches across multiple visual and analytical
benchmarks.
Read the full abstract ▾
All authors Rejoy Chakraborty, Prasun Roy, Saumik Bhattacharya, Umapada Pal
Computer Vision
Artificial intelligence application in lymphoma diagnosis with Vision Transformer using weakly supervised training
Nghia Nguyen, Amer Wahed, Andy Quesada et al. (+8 more)
Vision transformers (ViT) have been shown to allow for more flexible feature detection and can outperform convolutional neural network (CNN) when pre-trained on sufficient data. Due to their promising feature detection capabilities, we deployed ViTs for morphological classification of anaplastic large
cell lymphoma (ALCL) versus classic Hodgkin lymphoma (cHL). We had previously designed a ViT model which was trained on a small
dataset of 1,200 image patches in fully supervised training. That model achieved a diagnostic accuracy of 100% and an F1 score of 1.0 on the independent test set. Since fully supervised training is not a practical method due to lack of expertise resources in both the training and testing phases, we conducted a recent study on a modified approach to training data (weakly supervised training) and show that labeling training image patch automatically at the slide level of each whole-slide-image is a more practical solution for clinical use of
Vision Transformer. Our ViT model, trained on a larger
dataset of 100,000 image patches, yields evaluation metrics with significant accuracy, F1 score, and area under the curve (AUC) at 91.85%, 0.92, and 0.98, respectively. These are respectable values that qualify this ViT model, with weakly supervised training, as a suitable tool for a deep learning module in clinical model development using automated image patch extraction.
Read the full abstract ▾
All authors Nghia Nguyen, Amer Wahed, Andy Quesada, Yasir Ali, Hanadi El Achi, Y. Helen Zhang, Jocelyn Ursua, Alex Banerjee, Sahib Kalra, L. Jeffrey Medeiros, Jie Xu
Computer Vision
From Synchrony to Sequence: Exo-to-Ego Generation via Interpolation
Mohammad Mahdi, Nedko Savov, Danda Pani Paudel et al. (+1 more)
Exo-to-Ego
video generation aims to synthesize a first-person
video from a synchronized third-person view and corresponding camera poses. While paired supervision is available, synchronized exo-ego data inherently introduces substantial spatio-temporal and
geometric discontinuities, violating the smooth-motion assumptions of standard
video generation
benchmarks. We identify this synchronization-induced jump as the central challenge and propose Syn2Seq-Forcing, a sequential formulation that interpolates between the source and target
videos to form a single continuous signal. By reframing Exo2Ego as sequential signal modeling rather than a conventional condition-output task, our approach enables
diffusion-based sequence models, e.g.
Diffusion Forcing
Transformers (DFoT), to capture coherent transitions across frames more effectively. Empirically, we show that interpolating only the
videos, without performing pose interpolation already produces significant improvements, emphasizing that the dominant difficulty arises from spatio-temporal discontinuities. Beyond immediate performance gains, this formulation establishes a general and flexible framework capable of unifying both Exo2Ego and Ego2Exo generation within a single continuous sequence model, providing a principled foundation for future research in cross-view
video synthesis.
Read the full abstract ▾
All authors Mohammad Mahdi, Nedko Savov, Danda Pani Paudel, Luc Van Gool
Computer Vision
PBE-UNet: A light weight Progressive Boundary-Enhanced U-Net with Scale-Aware Aggregation for Ultrasound Image Segmentation
Chen Wang, Yixin Zhu, Yongbin Zhu et al. (+5 more)
Accurate lesion
segmentation in ultrasound images is essential for preventive screening and clinical diagnosis, yet remains challenging due to low contrast, blurry boundaries, and significant scale variations. Although existing deep learning-based methods have achieved remarkable performance, these methods still struggle with scale variations and indistinct tumor boundaries. To address these challenges, we propose a progressive boundary enhanced U-Net (PBE-UNet). Specially, we first introduce a scale-aware aggregation module (SAAM) that dynamically adjusts its receptive field to capture robust multi-scale contextual information. Then, we propose a boundary-guided feature enhancement (BGFE) module to enhance the feature representations. We find that there are large gaps between the narrow boundary and the wide
segmentation error areas. Unlike existing methods that treat boundaries as static masks, the BGFE module progressively expands the narrow boundary prediction into broader spatial
attention maps. Thus, broader spatial
attention maps could effectively cover the wider
segmentation error regions and enhance the model's focus on these challenging areas. We conduct expensive experiments on four
benchmark ultrasound
datasets, BUSI,
Dataset B, TN3K, and BP. The experimental results how that our proposed PBE-UNet outperforms state-of-the-art ultrasound image
segmentation methods. The code is at https://github.com/cruelMouth/PBE-UNet.
Read the full abstract ▾
All authors Chen Wang, Yixin Zhu, Yongbin Zhu, Fengyuan Shi, Qi Li, Jun Wang, Zuozhu Liu, Keli Hu
Computer Vision
Temporally Consistent Long-Term Memory for 3D Single Object Tracking
Jaejoon Yoo, SuBeen Lee, Yerim Jeon et al. (+2 more)
3D Single Object Tracking (3D-SOT) aims to localize a target object across a sequence of
LiDAR point clouds, given its 3D bounding box in the first frame. Recent methods have adopted a
memory-based approach to utilize previously observed features of the target object, but remain limited to only a few recent frames. This work reveals that their temporal capacity is fundamentally constrained to short-term context due to severe temporal feature inconsistency and excessive
memory overhead. To this end, we propose a robust long-term 3D-SOT framework, ChronoTrack, which preserves the temporal feature consistency while efficiently aggregating the diverse target features via long-term
memory. Based on a compact set of learnable
memory tokens, ChronoTrack leverages long-term information through two complementary objectives: a temporal consistency loss and a
memory cycle consistency loss. The former enforces feature
alignment across frames, alleviating temporal drift and improving the reliability of proposed long-term
memory. In parallel, the latter encourages each
token to encode diverse and discriminative target representations observed throughout the sequence via
memory-point-
memory cyclic walks. As a result, ChronoTrack achieves new state-of-the-art performance on multiple 3D-SOT
benchmarks, demonstrating its effectiveness in long-term target modeling with compact
memory while running at real-time speed of 42 FPS on a single RTX 4090 GPU. The code is available at https://github.com/ujaejoon/ChronoTrack
Read the full abstract ▾
All authors Jaejoon Yoo, SuBeen Lee, Yerim Jeon, Miso Lee, Jae-Pil Heo
Computer Vision · Cross-listed from cs.RO
Failure Identification in Imitation Learning Via Statistical and Semantic Filtering
Quentin Rolland, Fabrice Mayran de Chamisso and Jean-Baptiste Mouret
Imitation learning (IL) policies in robotics deliver strong performance in controlled settings but remain brittle in real-world deployments: rare events such as hardware faults, defective parts, unexpected human actions, or any state that lies outside the training distribution can lead to failed executions.
Vision-based Anomaly Detection (AD) methods emerged as an appropriate solution to detect these anomalous failure states but do not distinguish failures from benign deviations. We introduce FIDeL (Failure Identification in Demonstration Learning), a policy-independent failure detection module. Leveraging recent AD methods, FIDeL builds a compact representation of nominal demonstrations and aligns incoming observations via optimal transport matching to produce anomaly scores and heatmaps. Spatio-temporal thresholds are derived with an extension of conformal prediction, and a
Vision-
Language Model (VLM) performs semantic filtering to discriminate benign anomalies from genuine failures. We also introduce BotFails, a
multimodal dataset of real-world tasks for failure detection in robotics. FIDeL consistently outperforms state-of-the-art baselines, yielding +5.30% percent AUROC in anomaly detection and +17.38% percent failure-detection accuracy on BotFails compared to existing methods.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CY
Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarking
Alexander Nemecek, Osama Zafar, Yuqiao Xu et al. (+2 more)
Watermarking is becoming the default mechanism for AI content authentication, with governance policies and frameworks referencing it as infrastructure for content provenance. Yet across text, image, and audio modalities, watermark signal strength, detectability, and robustness depend on statistical properties of the content itself, properties that vary systematically across
languages, cultural visual traditions, and demographic groups. We examine how this content dependence creates modality-specific pathways to bias. Reviewing the major watermarking
benchmarks across modalities, we find that, with one exception, none report performance across
languages, cultural content types, or population groups. To address this, we propose three concrete evaluation dimensions for pluralistic watermark benchmarking: cross-lingual detection parity, culturally diverse content coverage, and demographic disaggregation of detection metrics. We argue that watermarking is part of the pluralistic
alignment pipeline and should be held to the same evaluation standards. We connect this to governance frameworks currently mandating watermarking deployment without requiring
fairness evaluation. Our position is that evaluation must precede deployment, and that the same bias
auditing requirements applied to AI models should extend to the verification layer.
Read the full abstract ▾
All authors Alexander Nemecek, Osama Zafar, Yuqiao Xu, Wenbiao Li, Erman Ayday
Computer Vision
Design and Behavior of Sparse Mixture-of-Experts Layers in CNN-based Semantic Segmentation
Svetlana Pavlitska, Haixi Fan, Konstantin Ditschuneit et al. (+1 more)
Sparse mixture-of-experts (MoE) layers have been shown to substantially increase model capacity without a proportional increase in computational cost and are widely used in
transformer architectures, where they typically replace feed-forward network blocks. In contrast, integrating sparse MoE layers into convolutional neural networks (CNNs) remains inconsistent, with most prior work focusing on fine-grained MoEs operating at the filter or channel levels. In this work, we investigate a coarser, patch-wise formulation of sparse MoE layers for semantic
segmentation, where local regions are routed to a small subset of convolutional experts. Through experiments on the Cityscapes and BDD100K
datasets using encoder-decoder and backbone-based CNNs, we conduct a design analysis to assess how architectural choices affect routing dynamics and expert specialization. Our results demonstrate consistent, architecture-dependent improvements (up to +3.9 mIoU) with little computational overhead, while revealing strong design sensitivity. Our work provides empirical insights into the design and internal dynamics of sparse MoE layers in CNN-based dense prediction. Our code is available at https://github.com/KASTEL-MobilityLab/moe-layers/.
Read the full abstract ▾
All authors Svetlana Pavlitska, Haixi Fan, Konstantin Ditschuneit, J. Marius Zöllner
Computer Vision · Cross-listed from cs.CL
MedRCube: A Multidimensional Framework for Fine-Grained and In-Depth Evaluation of MLLMs in Medical Imaging
Zhijie Bao, Fangke Chen, Licheng Bao et al. (+4 more)
The potential of
Multimodal Large
Language Models (MLLMs) in domain of medical imaging raise the demands of systematic and rigorous evaluation frameworks that are aligned with the real-world medical imaging practice. Existing practices that report single or coarse-grained metrics are lack the granularity required for specialized clinical support and fail to assess the reliability of
reasoning mechanisms. To address this, we propose a paradigm shift toward multidimensional, fine-grained and in-depth evaluation. Based on a two-stage systematic construction pipeline designed for this paradigm, we instantiate it with MedRCube. We
benchmark 33 MLLMs, Lingshu-32B achieve top-tier performance. Crucially, MedRCube exposes a series of pronounced insights inaccessible under prior evaluation settings. Furthermore, we introduce a credibility evaluation subset to quantify
reasoning credibility, uncover a highly significant positive association between shortcut behavior and diagnostic task performance, raising concerns for clinically trustworthy deployment. The resources of this work can be found at https://github.com/F1mc/MedRCube.
Read the full abstract ▾
All authors Zhijie Bao, Fangke Chen, Licheng Bao, Chenhui Zhang, Wei Chen, Jiajie Peng, Zhongyu Wei
Computer Vision
ClipGStream: Clip-Stream Gaussian Splatting for Any Length and Any Motion Multi-View Dynamic Scene Reconstruction
Jie Liang, Jiahao Wu, Chao Wang et al. (+7 more)
Dynamic 3D scene reconstruction is essential for immersive media such as VR, MR, and XR, yet remains challenging for long multi-view sequences with large-scale motion. Existing dynamic Gaussian approaches are either Frame-Stream, offering scalability but poor temporal stability, or Clip, achieving local consistency at the cost of high
memory and limited sequence length. We propose ClipGStream, a hybrid reconstruction framework that performs stream optimization at the clip level rather than the frame level. The sequence is divided into short clips, where dynamic motion is modeled using clip-independent spatio-temporal fields and residual anchor compensation to capture local variations efficiently, while inter-clip inherited anchors and decoders maintain structural consistency across clips. This Clip-Stream design enables scalable, flicker-free reconstruction of long dynamic
videos with high temporal coherence and reduced
memory overhead. Extensive experiments demonstrate that ClipGStream achieves state-of-the-art reconstruction quality and efficiency. The project page is available at: https://liangjie1999.github.io/ClipGStreamWeb/
Read the full abstract ▾
All authors Jie Liang, Jiahao Wu, Chao Wang, Jiayu Yang, Xiaoyun Zheng, Kaiqiang Xiong, Zhanke Wang, Jinbo Yan, Feng Gao, Ronggang Wang
Computer Vision
ReConText3D: Replay-based Continual Text-to-3D Generation
Muhammad Ahmed Ullah Khan, Muhammad Haris Bin Amir, Didier Stricker et al. (+1 more)
Continual learning enables models to acquire new knowledge over time while retaining previously learned capabilities. However, its application to text-to-3D generation remains unexplored. We present ReConText3D, the first framework for continual text-to-3D generation. We first demonstrate that existing text-to-3D models suffer from catastrophic forgetting under incremental training. ReConText3D enables generative models to incrementally learn new 3D categories from textual descriptions while preserving the ability to synthesize previously seen assets. Our method constructs a compact and diverse replay
memory through text-
embedding k-Center selection, allowing representative rehearsal of prior knowledge without modifying the underlying architecture. To systematically evaluate continual text-to-3D learning, we introduce Toys4K-CL, a
benchmark derived from the Toys4K
dataset that provides balanced and semantically diverse class-incremental splits. Extensive experiments on the Toys4K-CL
benchmark show that ReConText3D consistently outperforms all baselines across different generative backbones, maintaining high-quality generation for both old and new classes. To the best of our knowledge, this work establishes the first continual learning framework and
benchmark for text-to-3D generation, opening a new direction for incremental 3D generative modeling. Project page is available at: https://mauk95.github.io/ReConText3D/.
Read the full abstract ▾
All authors Muhammad Ahmed Ullah Khan, Muhammad Haris Bin Amir, Didier Stricker, Muhammad Zeshan Afzal
Computer Vision
Granularity-Aware Transfer for Tree Instance Segmentation in Synthetic and Real Forests
Pankaj Deoli, Atef Tej, Anmol Ashri et al. (+2 more)
We address the challenge of synthetic-to-real transfer in forestry perception where real data have only coarse Tree labels while synthetic data provide fine-grained trunk/crown annotations. We introduce MGTD, a mixed-granularity
dataset with 53k synthetic and 3.6k real images, and a four-stage protocol isolating domain shift and granularity mismatch. Our core contribution is granularity-aware
distillation, which transfers structural priors from fine-grained synthetic teachers to a coarse-label student via logit-space merging and mask unification. Experiments show consistent mask AP gains, especially for small/distant trees, establishing a testbed for Sim-Real transfer under label granularity constraints.
Read the full abstract ▾
All authors Pankaj Deoli, Atef Tej, Anmol Ashri, Anandatirtha JS, Karsten Berns
Computer Vision
SLQ: Bridging Modalities via Shared Latent Queries for Retrieval with Frozen MLLMs
Haoran Lou, Ziyan Liu, Chunxiao Fan et al. (+6 more)
Multimodal Large
Language Models (MLLMs) possess intrinsic
reasoning and world-knowledge capabilities, yet adapting them for dense
retrieval remains challenging. Existing approaches rely on invasive parameter updates, such as full fine-tuning and LoRA, which may disrupt the pre-trained semantic space and impair the structured knowledge essential for
reasoning. To address this, we propose SLQ, a parameter-efficient tuning framework that adapts MLLMs for
retrieval while keeping the backbone entirely frozen. SLQ introduces a small set of Shared Latent Queries that are appended to both text and image
tokens, leveraging the model's native causal
attention to aggregate
multimodal context into a unified
embedding space. Furthermore, to better evaluate
retrieval beyond superficial pattern matching, we construct KARR-Bench, a
benchmark designed for knowledge-aware
reasoning retrieval. Extensive experiments show that SLQ outperforms full fine-tuning and LoRA on COCO and Flickr30K, while achieving competitive performance on MMEB and yielding substantial gains on KARR-Bench, validating that preserving the pre-trained representations via non-invasive adaptation is an effective strategy for MLLM-based
retrieval. The code is available under: https://github.com/CnFaker/SLQ.
Read the full abstract ▾
All authors Haoran Lou, Ziyan Liu, Chunxiao Fan, Yuexin Wu, Yue Ming, Hao Wu, Kai Zuo, Yibo Chen, Xu Tang
Computer Vision
Med-CAM: Minimal Evidence for Explaining Medical Decision Making
Pirzada Suhail, Aditya Anand and Amit Sethi
Reliable and interpretable decision-making is essential in medical imaging, where diagnostic outcomes directly influence patient care. Despite advances in deep learning, most medical AI systems operate as opaque black boxes, providing little insight into why a particular diagnosis was reached. In this paper, we introduce Med-CAM, a framework for generating minimal and sharp maps as evidence-based explanations for Medical decision making via Classifier Activation Matching. Med-CAM trains a
segmentation network from scratch to produce a mask that highlights the minimal evidence critical to model's decision for any seen or unseen image. This ensures that the explanation is both faithful to the network's behaviour and interpretable to clinicians. Experiments show, unlike prior spatial explanation methods, such as Grad-CAM and
attention maps, which yield only fuzzy regions of relative importance, Med-CAM with its superior spatial awareness to shapes, textures, and boundaries, delivers conclusive, evidence-based explanations that faithfully replicate the model's prediction for any given image. By explicitly constraining explanations to be compact, consistent with model activations, and diagnostic
alignment, Med-CAM advances transparent AI to foster clinician understanding and trust in high-stakes medical applications such as pathology and radiology.
Read the full abstract ▾
Computer Vision
Beyond Voxel 3D Editing: Learning from 3D Masks and Self-Constructed Data
Yizhao Xu, Hongyuan Zhu, Caiyun Liu et al. (+6 more)
3D editing refers to the ability to apply local or global modifications to 3D assets. Effective 3D editing requires maintaining semantic consistency by performing localized changes according to
prompts, while also preserving local invariance so that unchanged regions remain consistent with the original. However, existing approaches have significant limitations: multi-view editing methods incur losses when projecting back to 3D, while voxel-based editing is constrained in both the regions that can be modified and the scale of modifications. Moreover, the lack of sufficiently large editing
datasets for training and evaluation remains a challenge. To address these challenges, we propose a Beyond Voxel 3D Editing (BVE) framework with a self-constructed large-scale
dataset specifically tailored for 3D editing. Building upon this
dataset, our model enhances a foundational image-to-3D generative architecture with lightweight, trainable modules, enabling efficient injection of textual semantics without the need for expensive full-model retraining. Furthermore, we introduce an annotation-free 3D masking strategy to preserve local invariance, maintaining the integrity of unchanged regions during editing. Extensive experiments demonstrate that BVE achieves superior performance in generating high-quality, text-aligned 3D assets, while faithfully retaining the visual characteristics of the original input.
Read the full abstract ▾
All authors Yizhao Xu, Hongyuan Zhu, Caiyun Liu, Tianfu Wang, Keyu Chen, Sicheng Xu, Jiaolong Yang, Nicholas Jing Yuan, Qi Zhang
Computer Vision
From Pixels to Nucleotides: End-to-End Token-Based Video Compression for DNA Storage
Cihan Ruan, Lebin Zhou, Bingqing Zhao et al. (+8 more)
DNA-based storage has emerged as a promising approach to the global data crisis, offering molecular-scale density and millennial-scale stability at low maintenance cost. Over the past decade, substantial progress has been made in storing text, images, and files in DNA – yet
video remains an open challenge. The difficulty is not merely technical: effective
video DNA storage requires co-designing compression and molecular encoding from the ground up, a challenge that sits at the intersection of two fields that have largely evolved independently. In this work, we present HELIX, the first end-to-end neural network jointly optimizing
video compression and DNA encoding – prior approaches treat the two stages independently, leaving biochemical constraints and compression objectives fundamentally misaligned. Our key insight:
token-based representations naturally align with DNA's quaternary alphabet – discrete semantic units map directly to ATCG bases. We introduce TK-SCONE (
Token-Kronecker Structured Constraint-Optimized Neural Encoding), which achieves 1.91 bits per nucleotide through Kronecker-structured mixing that breaks spatial correlations and FSM-based mapping that guarantees biochemical constraints. Unlike two-stage approaches, HELIX learns
token distributions simultaneously optimized for visual quality, prediction under masking, and DNA synthesis efficiency. This work demonstrates for the first time that learned compression and molecular storage converge naturally at
token representations – suggesting a new paradigm where neural
video codecs are designed for biological substrates from the ground up.
Read the full abstract ▾
All authors Cihan Ruan, Lebin Zhou, Bingqing Zhao, Rongduo Han, Qiming Yuan, Chenchen Zhu, Linyi Han, Liang Yang, Wei Wang, Wei Jiang, Nam Ling
Computer Vision · Cross-listed from cond-mat.mes-hall
Automatic Charge State Tuning of 300 mm FDSOI Quantum Dots Using Neural Network Segmentation of Charge Stability Diagram
Peter Samaha, Amine Torki, Ysaline Renaud et al. (+4 more)
Tuning of gate-defined
semiconductor quantum dots (QDs) is a major bottleneck for scaling
spin qubit technologies. We present a deep learning (DL) driven, semantic-
segmentation pipeline that performs charge auto-tuning by locating transition lines in full charge stability diagrams (CSDs) and returns gate voltage targets for the single charge regime. We assemble and manually annotate a large, heterogeneous
dataset of 1015 experimental CSDs measured from silicon QD devices, spanning nine design geometries, multiple wafers, and fabrication runs. A U-Net style convolutional neural network (CNN) with a MobileNetV2 encoder is trained and validated through five-fold group cross validation. Our model achieves an overall offline tuning success of 80.0% in locating the single-charge regime, with peak performance exceeding 88% for some designs. We analyze dominant failure modes and propose targeted mitigations. Finally, wide-range diagram
segmentation also naturally enables scalable physic-based feature extraction that can feed back to fabrication and design workflows and outline a roadmap for real-time integration in a cryogenic wafer prober. Overall, our results show that neural network (NN) based wide-diagram
segmentation is a practical step toward automated, high-throughput charge tuning for silicon QD
qubits.
Read the full abstract ▾
All authors Peter Samaha, Amine Torki, Ysaline Renaud, Sam Fiette, Emmanuel Chanrion, Pierre-Andre Mortemousque, Yann Beilliard
Computer Vision
VRAG-DFD: Verifiable Retrieval-Augmentation for MLLM-based Deepfake Detection
Hui Han, Shunli Wang, Yandan Zhao et al. (+2 more)
In Deepfake Detection (DFD) tasks, researchers proposed two types of MLLM-based methods: complementary combination with small DFD detectors, or static forgery knowledge injection. The lack of professional forgery knowledge hinders the performance of these DFD-MLLMs. To solve this, we deeply considered two insightful issues: How to provide high-quality associated forgery knowledge for MLLMs? AND How to endow MLLMs with critical
reasoning abilities given noisy reference information? Notably, we attempted to address above two questions with preliminary answers by leveraging the combination of
Retrieval-Augmented Generation (RAG) and Reinforcement Learning (RL). Through RAG and RL techniques, we propose the VRAG-DFD framework with accurate dynamic forgery knowledge
retrieval and powerful critical
reasoning capabilities. Specifically, in terms of data, we constructed two
datasets with RAG: Forensic Knowledge Database (FKD) for DFD knowledge annotation, and Forensic Chain-of-Thought
Dataset (F-CoT), for critical CoT construction. In terms of model training, we adopt a three-stage training method (
Alignment->SFT->GRPO) to gradually cultivate the critical
reasoning ability of the MLLM. In terms of performance, VRAG-DFD achieved SOTA and competitive performance on DFD generalization testing.
Read the full abstract ▾
All authors Hui Han, Shunli Wang, Yandan Zhao, Taiping Yao, Shouhong Ding
Computer Vision
ESCAPE: Episodic Spatial Memory and Adaptive Execution Policy for Long-Horizon Mobile Manipulation
Jingjing Qian, Zeyuan He, Chen Shi et al. (+2 more)
Coordinating
navigation and manipulation with robust performance is essential for
embodied AI in complex indoor environments. However, as tasks extend over long horizons, existing methods often struggle due to catastrophic forgetting, spatial inconsistency, and rigid execution. To address these issues, we propose ESCAPE (Episodic Spatial
Memory Coupled with an Adaptive Policy for Execution), operating through a tightly coupled perception-grounding-execution workflow. For robust perception, ESCAPE features a Spatio-Temporal Fusion Mapping module to autoregressively construct a depth-free, persistent 3D spatial
memory, alongside a
Memory-Driven Target Grounding module for precise interaction mask generation. To achieve flexible action, our Adaptive Execution Policy dynamically orchestrates proactive global
navigation and reactive local manipulation to seize opportunistic targets. ESCAPE achieves state-of-the-art performance on the ALFRED
benchmark, reaching 65.09% and 60.79% success rates in test seen and unseen environments with step-by-step instructions. By reducing redundant exploration, our ESCAPE attains substantial improvements in path-length-weighted metrics and maintains robust performance (61.24% / 56.04%) even without detailed guidance for long-horizon tasks.
Read the full abstract ▾
All authors Jingjing Qian, Zeyuan He, Chen Shi, Lei Xiao, Li Jiang
Computer Vision
What Are We Really Measuring? Rethinking Dataset Bias in Web-Scale Natural Image Collections via Unsupervised Semantic Clustering
Amir Hossein Saleknia and Mohammad Sabokrou
In computer
vision, a prevailing method for quantifying
dataset bias is to train a model to distinguish between
datasets. High classification accuracy is then interpreted as evidence of meaningful semantic differences. This approach assumes that standard image augmentations successfully suppress low-level, non-semantic cues, and that any remaining performance must therefore reflect true semantic divergence. We demonstrate that this fundamental assumption is flawed within the domain of large-scale natural image collections. High classification accuracy is often driven by resolution-based artifacts, which are structural fingerprints arising from native image resolution distributions and interpolation effects during resizing. These artifacts form robust,
dataset-specific signatures that persist despite conventional image corruptions. Through controlled experiments, we show that models achieve strong
dataset classification even on non-semantic, procedurally generated images, proving their reliance on superficial cues. To address this issue, we revisit this decades-old idea of
dataset separability, but not with supervised classification. Instead, we introduce an unsupervised approach that measures true semantic separability. Our framework directly assesses semantic similarity by clustering semantically-rich features from foundational
vision models, deliberately bypassing supervised classification on
dataset labels. When applied to major web-scale
datasets, the primary focus of this work, the high separability reported by supervised methods largely vanishes, with clustering accuracy dropping to near-chance levels. This reveals that conventional classification-based evaluation systematically overstates semantic bias by an overwhelming margin.
Read the full abstract ▾
Computer Vision
VGGT-Segmentor: Geometry-Enhanced Cross-View Segmentation
Yulu Gao, Bohao Zhang, Zongheng Tang et al. (+3 more)
Instance-level object
segmentation across disparate
egocentric and exocentric views is a fundamental challenge in visual understanding, critical for applications in
embodied AI and remote collaboration. This task is exceptionally difficult due to severe changes in scale, perspective, and occlusion, which destabilize direct pixel-level matching. While recent geometry-aware models like VGGT provide a strong foundation for feature
alignment, we find they often fail at dense prediction tasks due to significant pixel-level projection drift, even when their internal object-level
attention remains consistent. To bridge this gap, we introduce VGGT-Segmentor (VGGT-S), a framework that unifies robust
geometric modeling with pixel-accurate semantic
segmentation. VGGT-S leverages VGGT's powerful cross-view feature representation and introduces a novel Union
Segmentation Head. This head operates in three stages: mask
prompt fusion, point-guided prediction, and iterative mask refinement, effectively translating high-level feature
alignment into a precise
segmentation mask. Furthermore, we propose a single-image self-supervised training strategy that eliminates the need for paired annotations and enables strong generalization. On the Ego-Exo4D
benchmark, VGGT-S sets a new state-of-the-art, achieving 67.7% and 68.0% average IoU for Ego to Exo and Exo to Ego tasks, respectively, significantly outperforming prior methods. Notably, our correspondence-free pretrained model surpasses most fully-supervised baselines, demonstrating the effectiveness and scalability of our approach. Code is publicly available at: https://github.com/buaa-colalab/VGGT-S.
Read the full abstract ▾
All authors Yulu Gao, Bohao Zhang, Zongheng Tang, Jitong Liao, Wenjun Wu, Si Liu
Computer Vision
Dehaze-then-Splat: Generative Dehazing with Physics-Informed 3D Gaussian Splatting for Smoke-Free Novel View Synthesis
Boss Chen and Hanqing Wang
We present Dehaze-then-Splat, a two-stage pipeline for multi-view smoke removal and novel view synthesis developed for Track~2 of the NTIRE 2026 3D Restoration and Reconstruction Challenge. In the first stage, we produce pseudo-clean training images via per-frame generative dehazing using Nano Banana Pro, followed by brightness normalization. In the second stage, we train 3D Gaussian Splatting (3DGS) with physics-informed auxiliary losses – depth supervision via Pearson correlation with pseudo-depth, dark channel prior regularization, and dual-source gradient matching – that compensate for cross-view inconsistencies inherent in frame-wise generative processing. We identify a fundamental tension in dehaze-then-reconstruct pipelines: per-image restoration quality does not guarantee multi-view consistency, and such inconsistency manifests as blurred renders and structural instability in downstream 3D reconstruction.Our analysis shows that MCMC-based densification with early stopping, combined with depth and haze-suppression priors, effectively mitigates these artifacts. On the Akikaze validation scene, our pipeline achieves 20.98\,dB PSNR and 0.683 SSIM for novel view synthesis, a +1.50\,dB improvement over the unregularized baseline.
Read the full abstract ▾
Computer Vision
Efficient Multi-View 3D Object Detection by Dynamic Token Selection and Fine-Tuning
Danish Nazir, Antoine Hanna-Asaad, Lucas Görnhardt et al. (+3 more)
Existing multi-view three-dimensional (3D) object detection approaches widely adopt large-scale pre-trained
vision transformer (ViT)-based foundation models as backbones, being computationally complex. To address this problem, current state-of-the-art (SOTA) ToC3D for efficient multi-view ViT-based 3D object detection employs ego-motion-based relevant
token selection. However, there are two key limitations: (1) The fixed layer-individual
token selection ratios limit computational efficiency during both training and inference. (2) Full end-to-end retraining of the ViT backbone is required for the multi-view 3D object detection method. In this work, we propose an image
token compensator combined with a
token selection for ViT backbones to accelerate multi-view 3D object detection. Unlike ToC3D, our approach enables dynamic layer-wise
token selection within the ViT backbone. Furthermore, we introduce a parameter-efficient fine-tuning strategy, which trains only the proposed modules, thereby reducing the number of fine-tuned parameters from more than $300$ million (M) to only $1.6$ M. Experiments on the large-scale NuScenes
dataset across three multi-view 3D object detection approaches demonstrate that our proposed method decreases computational complexity (GFLOPs) by $48\%$ ... $55\%$, inference latency (on an NVIDIA-GV100 GPU) by $9\%$ ... $25\%$, while still improving mean average precision by $1.0\%$ ... $2.8\%$ absolute and NuScenes detection score by $0.4\%$ ... $1.2\%$ absolute compared to so-far SOTA ToC3D.
Read the full abstract ▾
All authors Danish Nazir, Antoine Hanna-Asaad, Lucas Görnhardt, Jan Piewek, Thorsten Bagdonat, Tim Fingscheidt