Computer Vision
Shallow- and Deep-fake Image Manipulation Localization Using Vision Mamba and Guided Graph Neural Network
Junbin Zhang, Hamid Reza Tohidypour, Yixiao Wang et al. (+1 more)
Image manipulation localization is a critical research task, given that forged images may have a significant societal impact of various aspects. Such image manipulations can be produced using traditional image editing tools (known as "shallowfakes") or advanced artificial intelligence techniques ("deepfakes"). While numerous studies have focused on image manipulation localization on either shallowfake images or deepfake
videos, few approaches address both cases. In this paper, we explore the feasibility of using a deep learning network to localize manipulations in both shallow- and deep-fake images, and proposed a solution for such purpose. To precisely differentiate between authentic and manipulated pixels, we leverage the
Vision Mamba network to extract feature maps that clearly describe the boundaries between tampered and untouched regions. To further enhance this separation, we propose a novel Guided Graph Neural Network (G-GNN) module that amplifies the distinction between manipulated and authentic pixels. Our evaluation results show that our proposed method achieved higher inference accuracy compared to other state-of-the-art methods.
Read the full abstract ▾
All authors Junbin Zhang, Hamid Reza Tohidypour, Yixiao Wang, Panos Nasiopoulos
Computer Vision · Cross-listed from eess.IV
Comparative Analysis of Binarization Methods For Medical Image Hashing On Odir Dataset
Nedim Muzoglu
In this study, we evaluated four binarization methods. Locality-Sensitive Hashing (LSH), Iterative Quantization (ITQ), Kernel-based Supervised Hashing (KSH), and Supervised Discrete Hashing (SDH) on the ODIR
dataset using deep feature
embeddings. Experimental results show that SDH achieved the best performance, with an mAP@100 of 0.9184 using only 32-bit codes, outperforming LSH, ITQ, and KSH. Compared with prior studies, our method proved highly competitive: Fang et al. reported 0.7528 (Fundus-iSee, 48 bits) and 0.8856 (ASOCT-Cataract, 48 bits), while Wijesinghe et al. achieved 94.01 (KVASIR, 256 bits). Despite using significantly fewer bits, our SDH-based framework reached
retrieval accuracy close to the state-of-the-art. These findings demonstrate that SDH is the most effective approach among those tested, offering a practical balance of accuracy, storage, and efficiency for medical image
retrieval and device inventory management.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Normalized Conditional Mutual Information Surrogate Loss for Deep Neural Classifiers
Linfeng Ye, Zhixiang Chi, Konstantinos N. Plataniotis et al. (+1 more)
In this paper, we propose a novel information theoretic surrogate loss; normalized conditional mutual information (NCMI); as a drop in alternative to the de facto cross-
entropy (CE) for training deep neural network (DNN) based classifiers. We first observe that the model's NCMI is inversely proportional to its accuracy. Building on this insight, we introduce an alternating
algorithm to efficiently minimize the NCMI. Across image recognition and whole-slide imaging (WSI) subtyping
benchmarks, NCMI-trained models surpass state of the art losses by substantial margins at a computational cost comparable to that of CE. Notably, on ImageNet, NCMI yields a 2.77% top-1 accuracy improvement with ResNet-50 comparing to the CE; on CAMELYON-17, replacing CE with NCMI improves the macro-F1 by 8.6% over the strongest baseline. Gains are consistent across various architectures and batch sizes, suggesting that NCMI is a practical and competitive alternative to CE.
Read the full abstract ▾
All authors Linfeng Ye, Zhixiang Chi, Konstantinos N. Plataniotis, En-hui Yang
Computer Vision · Cross-listed from physics.med-ph
A Green Solution for Breast Region Segmentation Using Deep Active Learning
Sam Narimani, Solveig Roth Hoff, Kathinka Dæhli Kurz et al. (+3 more)
Purpose: Annotation of medical breast images is an essential step toward better diagnostic but a time consuming task. This research aims to focus on different selecting sample strategies within deep active learning on Breast Region
Segmentation (BRS) to lessen computational cost of training and effective use of resources. Methods: The Stavanger breast MRI
dataset containing 59 patients was used in this study, with FCN-ResNet50 adopted as a sustainable deep learning (DL) model. A novel sample selection approach based on Breast Anatomy Geometry (BAG) analysis was introduced to group data with similar informative features for DL. Patient positioning and Breast Size were considered the key selection criteria in this process. Four selection strategies including Random Selection, Nearest Point, Breast Size, and a hybrid of all three strategies were evaluated using an active learning framework. Four training data proportions of 10%, 20%, 30%, and 40% were used for model training, with the remaining data reserved for testing. Model performance was assessed using Dice score, Intersection over Union, precision, and recall, along with 5-fold cross-validation to enhance generalizability. Results: Increasing the training data proportion from 10% to 40% improved
segmentation performance for nearly all strategies, except for Random Selection. The Nearest Point strategy consistently achieved the lowest carbon footprint at 30% and 40% data proportions. Overall, combining the Nearest Point strategy with 30% of the training data provided the best balance between
segmentation performance, efficiency, and environmental sustainability. Keywords: Deep Active Learning, Breast Region
Segmentation, Human-center analysis
Read the full abstract ▾
All authors Sam Narimani, Solveig Roth Hoff, Kathinka Dæhli Kurz, Kjell-Inge Gjesdal, Jürgen Geisler, Endre Grøvik
Computer Vision
MovieRecapsQA: A Multimodal Open-Ended Video Question-Answering Benchmark
Shaden Shaar, Bradon Thymes, Sirawut Chaixanien et al. (+2 more)
Understanding real-world
videos such as movies requires integrating visual and dialogue cues. Yet existing VideoQA
benchmarks struggle to capture this
multimodal reasoning and, given the difficulty of evaluating free-form answers, largely resort to simple multiple choice questions. We introduce a novel open-ended
multimodal VideoQA
benchmark, MovieRecapsQA, created using movie recap
videos – a distinctive type of YouTube content that summarizes a film via a voiceover description of key clips from the movie (recap
video). From the transcribed voiceover (recap summary) of 60 recap
videos, we generate $\approx$8.2K questions along with the necessary “facts” expected in each answer; the former facilitates the creation of questions that require mutimodal
reasoning and the latter allow the construction of a reference-free evaluation metric that can be applied to open-ended responses. To our knowledge, this is the first reference-free open-ended VideoQA
benchmark. The
benchmark allows each question to be evaluated in different input
video settings: given (a) the full-length movie, (b) the full ($\approx$11 min) recap
video (visual only), (c) $\approx$14 min of aligned movie scenes, i.e, movie scenes relevant to the question, and (d) $\approx$1.2 min of aligned recap
video scenes. In all cases, the text of any associated movie dialogue is provided. Each question is categorized by the modality required to answer it – visual, dialogue, or both – enabling fine-grained evaluation of
multimodal capabilities. We
benchmark (setting (d)) seven state-of-the-art MLLMs and find that (i) only our reference-free metric produces meaningful human-aligned model separation; (ii)
vision-centric questions yield the lowest scores across all models; (iii) removing visual input often improves model factuality; and (iv) the primary bottleneck is visual perception, not visual
reasoning.
Read the full abstract ▾
All authors Shaden Shaar, Bradon Thymes, Sirawut Chaixanien, Claire Cardie, Bharath Hariharan
Computer Vision
CT Scans As Video: Efficient Intracranial Hemorrhage Detection Using Multi-Object Tracking
Amirreza Parvahan, Mohammad Hoseyni, Javad Khoramdel et al. (+1 more)
Automated analysis of volumetric medical imaging on edge devices is severely constrained by the high
memory and computational demands of 3D Convolutional Neural Networks (CNNs). This paper develops a lightweight computer
vision framework that reconciles the efficiency of 2D detection with the necessity of 3D context by reformulating volumetric Computer
Tomography (CT) data as sequential
video streams. This
video-viewpoint paradigm is applied to the time-sensitive task of Intracranial Hemorrhage (ICH) detection using the Hemorica
dataset. To ensure operational efficiency, we benchmarked multiple generations of the YOLO architecture (v8, v10, v11 and v12) in their Nano configurations, selecting the version with the highest mAP@50 to serve as the slice-level backbone. A ByteTrack
algorithm is then introduced to enforce anatomical consistency across the $z$-axis. To address the initialization lag inherent in
video trackers, a hybrid inference strategy and a spatiotemporal consistency filter are proposed to distinguish true pathology from transient prediction noise. Experimental results on independent test data demonstrate that the proposed framework serves as a rigorous temporal validator, increasing detection Precision from 0.703 to 0.779 compared to the baseline 2D detector, while maintaining high sensitivity. By approximating 3D contextual
reasoning at a fraction of the computational cost, this method provides a scalable solution for real-time patient prioritization in resource-constrained environments, such as mobile stroke units and IoT-enabled remote clinics.
Read the full abstract ▾
All authors Amirreza Parvahan, Mohammad Hoseyni, Javad Khoramdel, Amirhossein Nikoofard
Computer Vision
PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understanding
Souhail Hadgi, Bingchen Gong, Ramana Sundararaman et al. (+4 more)
Current foundation models for 3D shapes excel at global tasks (
retrieval, classification) but transfer poorly to local part-level
reasoning. Recent approaches leverage
vision and
language foundation models to directly solve dense tasks through multi-view renderings and text queries. While promising, these pipelines require expensive inference over multiple renderings, depend heavily on large
language-model (LLM)
prompt engineering for captions, and fail to exploit the inherent 3D geometry of shapes. We address this gap by introducing an encoder-only 3D model that produces
language-aligned patch-level features directly from point clouds. Our pre-training approach builds on existing data engines that generate part-annotated 3D shapes by pairing multi-view SAM regions with VLM captioning. Using this data, we train a point cloud
transformer encoder in two stages: (1)
distillation of dense 2D features from visual encoders such as DINOv2 into 3D patches, and (2)
alignment of these patch
embeddings with part-level text
embeddings through a multi-positive contrastive objective. Our 3D encoder achieves zero-shot 3D part
segmentation with fast single-pass inference without any test-time multi-view rendering, while significantly outperforming previous rendering-based and feed-forward approaches across several 3D part
segmentation benchmarks. Project website: https://souhail-hadgi.github.io/patchalign3dsite/
Read the full abstract ▾
All authors Souhail Hadgi, Bingchen Gong, Ramana Sundararaman, Emery Pierson, Lei Li, Peter Wonka, Maks Ovsjanikov
Computer Vision
Don't Mind the Gaps: Implicit Neural Representations for Resolution-Agnostic Retinal OCT Analysis
Bennet Kahrs, Julia Andresen, Fenja Falta et al. (+3 more)
Routine clinical imaging of the retina using optical coherence
tomography (OCT) is performed with large slice spacing, resulting in highly anisotropic images and a sparsely scanned retina. Most learning-based methods circumvent the problems arising from the anisotropy by using 2D approaches rather than performing volumetric analyses. These approaches inherently bear the risk of generating inconsistent results for neighboring B-scans. For example, 2D retinal layer
segmentations can have irregular surfaces in 3D. Furthermore, the typically used convolutional neural networks are bound to the resolution of the training data, which prevents their usage for images acquired with a different imaging protocol. Implicit neural representations (INRs) have recently emerged as a tool to store voxelized data as a continuous representation. Using coordinates as input, INRs are resolution-agnostic, which allows them to be applied to anisotropic data. In this paper, we propose two frameworks that make use of this characteristic of INRs for dense 3D analyses of retinal OCT volumes. 1) We perform inter-B-scan interpolation by incorporating additional information from en-face modalities, that help retain relevant structures between B-scans. 2) We create a resolution-agnostic retinal atlas that enables general analysis without strict requirements for the data. Both methods leverage generalizable INRs, improving retinal shape representation through population-based training and allowing predictions for unseen cases. Our resolution-independent frameworks facilitate the analysis of OCT images with large B-scan distances, opening up possibilities for the volumetric evaluation of retinal structures and pathologies.
Read the full abstract ▾
All authors Bennet Kahrs, Julia Andresen, Fenja Falta, Monty Santarossa, Heinz Handels, Timo Kepp
Computer Vision
A Spatio-Temporal Deep Learning Approach For High-Resolution Gridded Monsoon Prediction
Parashjyoti Borah, Sanghamitra Sarkar and Ranjan Phukan
The Indian Summer Monsoon (ISM) is a critical
climate phenomenon, fundamentally impacting the agriculture, economy, and water security of over a billion people. Traditional long-range
forecasting, whether statistical or dynamical, has predominantly focused on predicting a single, spatially-averaged seasonal value, lacking the spatial detail essential for regional-level resource management. To address this gap, we introduce a novel deep learning framework that reframes gridded monsoon prediction as a spatio-temporal computer
vision task. We treat multi-variable, pre-monsoon atmospheric and oceanic fields as a sequence of multi-channel images, effectively creating a
video-like input tensor. Using 85 years of ERA5 reanalysis data for predictors and IMD rainfall data for targets, we employ a Convolutional Neural Network (CNN)-based architecture to learn the complex mapping from the five-month pre-monsoon period (January-May) to a high-resolution gridded rainfall pattern for the subsequent monsoon season. Our framework successfully produces distinct forecasts for each of the four monsoon months (June-September) as well as the total seasonal average, demonstrating its utility for both intra-seasonal and seasonal outlooks.
Read the full abstract ▾
Computer Vision
Evaluating the Diagnostic Classification Ability of Multimodal Large Language Models: Insights from the Osteoarthritis Initiative
Li Wang, Xi Chen, XiangWen Deng et al. (+4 more)
Multimodal large
language models (MLLMs) show promising performance on medical visual question answering (VQA) and report generation, but these generation and explanation abilities do not reliably transfer to disease-specific classification. We evaluated MLLM architectures on knee osteoarthritis (OA) radiograph classification, which remains underrepresented in existing medical MLLM
benchmarks, even though knee OA affects an estimated 300 to 400 million people worldwide. Through systematic ablation studies manipulating the
vision encoder, the connector, and the large
language model (LLM) across diverse training strategies, we measured each component's contribution to diagnostic accuracy. In our classification task, a trained
vision encoder alone could outperform full MLLM pipelines in classification accuracy and fine-tuning the LLM provided no meaningful improvement over
prompt-based guidance. And LoRA fine-tuning on a small, class-balanced
dataset (500 images) gave better results than training on a much larger but class-imbalanced set (5,778 images), indicating that data balance and quality can matter more than raw scale for this task. These findings suggest that for domain-specific medical classification, LLMs are more effective as interpreters and report generators rather than as primary classifiers. Therefore, the MLLM architecture appears less suitable for medical image diagnostic classification tasks that demand high certainty. We recommend prioritizing
vision encoder optimization and careful
dataset curation when developing clinically applicable systems.
Read the full abstract ▾
All authors Li Wang, Xi Chen, XiangWen Deng, HuaHui Yi, ZeKun Jiang, Kang Li, Jian Li
Computer Vision
Understanding Pure Textual Reasoning for Blind Image Quality Assessment
Yuan Li and Shin'ya Nishida
Textual
reasoning has recently been widely adopted in Blind Image Quality Assessment (BIQA). However, it remains unclear how textual information contributes to quality prediction and to what extent text can represent the score-related image contents. This work addresses these questions from an information-flow perspective by comparing existing BIQA models with three paradigms designed to learn the image-text-score relationship: Chain-of-Thought, Self-Consistency, and Autoencoder. Our experiments show that the score prediction performance of the existing model significantly drops when only textual information is used for prediction. Whereas the Chain-of-Thought paradigm introduces little improvement in BIQA performance, the Self-Consistency paradigm significantly reduces the gap between image- and text-conditioned predictions, narrowing the PLCC/SRCC difference to 0.02/0.03. The Autoencoder-like paradigm is less effective in closing the image-text gap, yet it reveals a direction for further optimization. These findings provide insights into how to improve the textual
reasoning for BIQA and high-level
vision tasks.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
WebGym: Scaling Training Environments for Visual Web Agents with Realistic Tasks
Hao Bai, Alexey Taymanov, Tong Zhang et al. (+2 more)
We present WebGym, the largest-to-date open-source environment for training realistic visual web
agents. Real websites are non-stationary and diverse, making artificial or small-scale task sets insufficient for robust policy learning. WebGym contains nearly 300,000 tasks with rubric-based evaluations across diverse, real-world websites and difficulty levels. We train
agents with a simple reinforcement learning (RL) recipe, which trains on the
agent's own interaction traces (rollouts), using task rewards as feedback to guide learning. To enable scaling RL, we speed up sampling of trajectories in WebGym by developing a high-throughput asynchronous rollout system, designed specifically for web
agents. Our system achieves a 4-5x rollout speedup compared to naive implementations. Second, we scale the task set breadth, depth, and size, which results in continued performance improvement. Fine-tuning a strong base
vision-
language model, Qwen-3-VL-8B-Instruct, on WebGym results in an improvement in success rate on an out-of-distribution test set from 26.2% to 42.9%, significantly outperforming
agents based on proprietary models such as GPT-4o and GPT-5-Thinking that achieve 27.1% and 29.8%, respectively. This improvement is substantial because our test set consists only of tasks on websites never seen during training, unlike many other prior works on training visual web
agents.
Read the full abstract ▾
All authors Hao Bai, Alexey Taymanov, Tong Zhang, Aviral Kumar, Spencer Whitehead
Computer Vision
TAP-ViTs: Task-Adaptive Pruning for On-Device Deployment of Vision Transformers
Zhibo Wang, Zuoyuan Zhang, Xiaoyi Pang et al. (+4 more)
Vision Transformers (ViTs) have demonstrated strong performance across a wide range of
vision tasks, yet their substantial computational and
memory demands hinder efficient deployment on resource-constrained mobile and edge devices.
Pruning has emerged as a promising direction for reducing ViT complexity. However, existing approaches either (i) produce a single pruned model shared across all devices, ignoring device heterogeneity, or (ii) rely on fine-tuning with device-local data, which is often infeasible due to limited on-device resources and strict
privacy constraints. As a result, current methods fall short of enabling task-customized ViT
pruning in
privacy-preserving mobile computing settings. This paper introduces TAP-ViTs, a novel task-adaptive
pruning framework that generates device-specific pruned ViT models without requiring access to any raw local data. Specifically, to infer device-level task characteristics under
privacy constraints, we propose a Gaussian Mixture Model (GMM)-based metric
dataset construction mechanism. Each device fits a lightweight GMM to approximate its private data distribution and uploads only the GMM parameters. Using these parameters, the cloud selects distribution-consistent samples from public data to construct a task-representative metric
dataset for each device. Based on this proxy
dataset, we further develop a dual-granularity importance evaluation-based
pruning strategy that jointly measures composite
neuron importance and adaptive layer importance, enabling fine-grained, task-aware
pruning tailored to each device's computational budget. Extensive experiments across multiple ViT backbones and
datasets demonstrate that TAP-ViTs consistently outperforms state-of-the-art
pruning methods under comparable compression ratios.
Read the full abstract ▾
All authors Zhibo Wang, Zuoyuan Zhang, Xiaoyi Pang, Qile Zhang, Xuanyi Hao, Shuguo Zhuo, Peng Sun
Computer Vision · Cross-listed from eess.IV
Deep Learning Superresolution for 7T Knee MR Imaging: Impact on Image Quality and Diagnostic Performance
Pinzhen Chen, Libo Xu, Boyang Pan et al. (+8 more)
Background: Deep learning superresolution (SR) may enhance musculoskeletal MR image quality, but its diagnostic value in knee imaging at 7T is unclear. Objectives: To compare image quality and diagnostic performance of SR, low-resolution (LR), and high-resolution (HR) 7T knee MRI. Methods: In this prospective study, 42 participants underwent 7T knee MRI with LR (0.8*0.8*2 mm3) and HR (0.4*0.4*2 mm3) sequences. SR images were generated from LR data using a Hybrid
Attention Transformer model. Three radiologists assessed image quality, anatomic conspicuity, and detection of knee pathologies. Arthroscopy served as reference in 10 cases. Results: SR images showed higher overall quality than LR (median score 5 vs 4, P<.001) and lower noise than HR (5 vs 4, P<.001). Visibility of cartilage, menisci, and ligaments was superior in SR and HR compared to LR (P<.001). Detection rates and diagnostic performance (sensitivity, specificity, AUC) for intra-articular pathology were similar across image types (P>=.095). Conclusions: Deep learning superresolution improved subjective image quality in 7T knee MRI but did not increase diagnostic accuracy compared with standard LR imaging.
Read the full abstract ▾
All authors Pinzhen Chen, Libo Xu, Boyang Pan, Jing Li, Yuting Wang, Ran Xiong, Xiaoli Gou, Long Qing, Wenjing Hou, Nan-jie Gong, Wei Chen
Computer Vision
ExposeAnyone: Personalized Audio-to-Expression Diffusion Models Are Robust Zero-Shot Face Forgery Detectors
Kaede Shiohara, Toshihiko Yamasaki and Vladislav Golyanik
Detecting unknown deepfake manipulations remains one of the most challenging problems in face forgery detection. Current state-of-the-art approaches fail to generalize to unseen manipulations, as they primarily rely on supervised training with existing deepfakes or pseudo-fakes, which leads to overfitting to specific forgery patterns. In contrast, self-supervised methods offer greater potential for generalization, but existing work struggles to learn discriminative representations only from self-supervision. In this paper, we propose ExposeAnyone, a fully self-supervised approach based on a
diffusion model that generates expression sequences from audio. The key idea is, once the model is personalized to specific subjects using reference sets, it can compute the identity distances between suspected
videos and personalized subjects via
diffusion reconstruction errors, enabling person-of-interest face forgery detection. Extensive experiments demonstrate that 1) our method outperforms the previous state-of-the-art method by 4.22 percentage points in the average AUC on DF-TIMIT, DFDCP, KoDF, and IDForge
datasets, 2) our model is also capable of detecting Sora2-generated
videos, where the previous approaches perform poorly, and 3) our method is highly robust to corruptions such as blur and compression, highlighting the applicability in real-world face forgery detection.
Read the full abstract ▾
Computer Vision
VINO: A Unified Visual Generator with Interleaved OmniModal Context
Junyi Chen, Tong He, Zhoujie Fu et al. (+3 more)
We present VINO, a unified visual generator that performs image and
video generation and editing within a single framework. Instead of relying on task-specific models or independent modules for each modality, VINO uses a shared
diffusion backbone that conditions on text, images and
videos, enabling a broad range of visual creation and editing tasks under one model. Specifically, VINO couples a
vision-
language model (VLM) with a
Multimodal Diffusion Transformer (MMDiT), where
multimodal inputs are encoded as interleaved conditioning
tokens, and then used to guide the
diffusion process. This design supports multi-reference grounding, long-form instruction following, and coherent identity preservation across static and dynamic content, while avoiding modality-specific architectural components. To train such a unified system, we introduce a multi-stage training pipeline that progressively expands a
video generation base model into a unified, multi-task generator capable of both image and
video input and output. Across diverse generation and editing
benchmarks, VINO demonstrates strong visual quality, faithful instruction following, improved reference and attribute preservation, and more controllable multi-identity edits. Our results highlight a practical path toward scalable unified visual generation, and the promise of interleaved, in-context computation as a foundation for general-purpose visual creation.
Read the full abstract ▾
All authors Junyi Chen, Tong He, Zhoujie Fu, Pengfei Wan, Kun Gai, Weicai Ye
Computer Vision
Talk2Move: Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenes
Jing Tan, Zhaoyang Zhang, Yantao Shen et al. (+6 more)
We introduce Talk2Move, a reinforcement learning (RL) based
diffusion framework for text-instructed spatial transformation of objects within scenes. Spatially manipulating objects in a scene through natural
language poses a challenge for
multimodal generation systems. While existing text-based manipulation methods can adjust appearance or style, they struggle to perform object-level
geometric transformations-such as translating, rotating, or resizing objects-due to scarce paired supervision and pixel-level optimization limits. Talk2Move employs Group Relative Policy Optimization (GRPO) to explore
geometric actions through diverse rollouts generated from input images and lightweight textual variations, removing the need for costly paired data. A spatial reward guided model aligns
geometric transformations with linguistic description, while off-policy step evaluation and active step sampling improve learning efficiency by focusing on informative transformation stages. Furthermore, we design object-centric spatial rewards that evaluate displacement, rotation, and scaling behaviors directly, enabling interpretable and coherent transformations. Experiments on curated
benchmarks demonstrate that Talk2Move achieves precise, consistent, and semantically faithful object transformations, outperforming existing text-guided editing approaches in both spatial accuracy and scene coherence.
Read the full abstract ▾
All authors Jing Tan, Zhaoyang Zhang, Yantao Shen, Jiarui Cai, Shuo Yang, Jiajun Wu, Wei Xia, Zhuowen Tu, Stefano Soatto
Computer Vision
Meta-Learning Guided Pruning for Few-Shot Plant Pathology on Edge Devices
Mohammed Mudassir Uddin, Shahnawaz Alam, Mohammed Kaif Pasha et al. (+3 more)
Farmers in remote areas need quick and reliable methods for identifying plant diseases, yet they often lack access to laboratories or high-performance computing resources. Deep learning models can detect diseases from leaf images with high accuracy, but these models are typically too large and computationally expensive to run on low-cost edge devices such as Raspberry Pi. Furthermore, collecting thousands of labeled disease images for training is both expensive and time-consuming. This paper addresses both challenges by combining neural network
pruning, removing unnecessary parts of the model, with few-shot learning, which enables the model to learn from limited examples. This paper proposes Disease-Aware Channel Importance Scoring (DACIS), a method that identifies which parts of the neural network are most important for distinguishing between different plant diseases, integrated into a three-stage Prune-then-Meta-Learn-then-Prune (PMP) pipeline. Experiments on PlantVillage and PlantDoc
datasets demonstrate that the proposed approach reduces model size by 78% while maintaining 92.3% of the original accuracy, with the compressed model running at 7 frames per second on a Raspberry Pi 4, making real-time field diagnosis practical for smallholder farmers.
Read the full abstract ▾
All authors Mohammed Mudassir Uddin, Shahnawaz Alam, Mohammed Kaif Pasha, Dr Tasneem Bano Rehman, Dr Fahmina Taranum, Afroze Begum
Computer Vision
Joint Semantic and Rendering Enhancements in 3D Gaussian Modeling with Anisotropic Local Encoding
Jingming He, Chongyi Li, Shiqi Wang et al. (+1 more)
Recent works propose extending 3DGS with semantic feature vectors for simultaneous semantic
segmentation and image rendering. However, these methods often treat the semantic and rendering branches separately, relying solely on 2D supervision while ignoring the 3D Gaussian geometry. Moreover, current adaptive strategies adapt the Gaussian set depending solely on rendering gradients, which can be insufficient in subtle or textureless regions. In this work, we propose a joint enhancement framework for 3D semantic Gaussian modeling that synergizes both semantic and rendering branches. Firstly, unlike conventional point cloud shape encoding, we introduce an anisotropic 3D Gaussian Chebyshev descriptor using the Laplace-Beltrami operator to capture fine-grained 3D shape details, thereby distinguishing objects with similar appearances and reducing reliance on potentially noisy 2D guidance. In addition, without relying solely on rendering gradient, we adaptively adjust Gaussian allocation and spherical harmonics with local semantic and shape signals, enhancing rendering efficiency through selective resource allocation. Finally, we employ a cross-scene knowledge transfer module to continuously update learned shape patterns, enabling faster convergence and robust representations without relearning shape information from scratch for each new scene. Experiments on multiple
datasets demonstrate improvements in
segmentation accuracy and rendering quality while maintaining high rendering frame rates.
Read the full abstract ▾
All authors Jingming He, Chongyi Li, Shiqi Wang, Sam Kwong
Computer Vision
BEDS : Bayesian Emergent Dissipative Structures : A Formal Framework for Continuous Inference Under Energy Constraints
Laurent Caraffa
We introduce BEDS (
Bayesian Emergent Dissipative Structures), a formal framework for analyzing inference systems that must maintain beliefs continuously under energy constraints. Unlike classical computational models that assume perfect
memory and focus on one-shot computation, BEDS explicitly incorporates dissipation (information loss over time) as a fundamental constraint. We prove a central result linking energy, precision, and dissipation: maintaining a belief with precision $\tau$ against dissipation rate $\gamma$ requires power $P \geq \gamma k_{\rm B} T / 2$, with scaling $P \propto \gamma \cdot \tau$. This establishes a fundamental thermodynamic cost for continuous inference. We define three classes of problems – BEDS-attainable, BEDS-maintainable, and BEDS-crystallizable – and show these are distinct from classical decidability. We propose the Gödel-Landauer-Prigogine
conjecture, suggesting that closure pathologies across formal systems, computation, and thermodynamics share a common structure.
Read the full abstract ▾
Computer Vision
Fusion2Print: Deep Flash-Non-Flash Fusion for Contactless Fingerprint Matching
Roja Sahoo and Anoop Namboodiri
Contactless fingerprint recognition offers a hygienic and convenient alternative to contact-based systems, enabling rapid acquisition without latent prints, pressure artifacts, or hygiene risks. However, contactless images often show degraded ridge clarity due to illumination variation, subcutaneous skin discoloration, and specular reflections. Flash captures preserve ridge detail but introduce noise, whereas non-flash captures reduce noise but lower ridge contrast. We propose Fusion2Print (F2P), the first framework to systematically capture and fuse paired flash-non-flash contactless fingerprints. We construct a custom paired
dataset, FNF Database, and perform manual flash-non-flash subtraction to isolate ridge-preserving signals. A lightweight
attention-based fusion network also integrates both modalities, emphasizing informative channels and suppressing noise, and then a U-Net enhancement module produces an optimally weighted grayscale image. Finally, a deep
embedding model with cross-domain compatibility, generates discriminative and robust representations in a unified
embedding space compatible with both contactless and contact-based fingerprints for verification. F2P enhances ridge clarity and achieves superior recognition performance (AUC=0.999, EER=1.12%) over single-capture baselines (Verifinger, DeepPrint).
Read the full abstract ▾
Computer Vision
Prithvi-Complimentary Adaptive Fusion Encoder (CAFE): unlocking full-potential for flood inundation mapping
Saurabh Kaushik, Lalit Maurya and Beth Tellman
Geo-Foundation Models (GFMs), have proven effective in diverse downstream applications, including semantic
segmentation, classification, and regression tasks. However, in case of flood mapping using Sen1Flood11
dataset as a downstream task, GFMs struggles to outperform the baseline U-Net, highlighting model's limitation in capturing critical local nuances. To address this, we present the Prithvi-Complementary Adaptive Fusion Encoder (CAFE), which integrate Prithvi GFM pretrained encoder with a parallel CNN residual branch enhanced by Convolutional
Attention Modules (CAM). Prithvi-CAFE enables fast and efficient fine-tuning through adapters in Prithvi and performs multi-scale, multi-level fusion with CNN features, capturing critical local details while preserving long-range dependencies. We achieve state-of-the-art results on two comprehensive flood mapping
datasets: Sen1Flood11 and FloodPlanet. On Sen1Flood11 test data, Prithvi-CAFE (IoU 83.41) outperforms the original Prithvi (IoU 82.50) and other major GFMs (TerraMind 82.90, DOFA 81.54, spectralGPT: 81.02). The improvement is even more pronounced on the hold-out test site, where Prithvi-CAFE achieves an IoU of 81.37 compared to the baseline U-Net (70.57) and original Prithvi (72.42). On FloodPlanet, Prithvi-CAFE also surpasses the baseline U-Net and other GFMs, achieving an IoU of 64.70 compared to U-Net (60.14), Terramind (62.33), DOFA (59.15) and Prithvi 2.0 (61.91). Our proposed simple yet effective Prithvi-CAFE demonstrates strong potential for improving
segmentation tasks where multi-channel and multi-modal data provide complementary information and local details are critical. The code is released on \href{https://github.com/Sk-2103/Prithvi-CAFE}{Prithvi-CAFE Github}
Read the full abstract ▾
Computer Vision
360DVO: Deep Visual Odometry for Monocular 360-Degree Camera
Xiaopeng Guo, Yinzhe Xu, Huajian Huang et al. (+1 more)
Monocular omnidirectional visual odometry (OVO) systems leverage 360-degree cameras to overcome field-of-view limitations of perspective VO systems. However, existing methods, reliant on handcrafted features or photometric objectives, often lack robustness in challenging scenarios, such as aggressive motion and varying illumination. To address this, we present 360DVO, the first deep learning-based OVO framework. Our approach introduces a distortion-aware spherical feature extractor (DAS-Feat) that adaptively learns distortion-resistant features from 360-degree images. These sparse feature patches are then used to establish constraints for effective pose estimation within a novel omnidirectional differentiable bundle adjustment (ODBA) module. To facilitate evaluation in realistic settings, we also contribute a new real-world OVO
benchmark. Extensive experiments on this
benchmark and public synthetic
datasets (TartanAir V2 and 360VO) demonstrate that 360DVO surpasses state-of-the-art baselines (including 360VO and OpenVSLAM), improving robustness by 50% and accuracy by 37.5%. Homepage: https://chris1004336379.github.io/360DVO-homepage
Read the full abstract ▾
All authors Xiaopeng Guo, Yinzhe Xu, Huajian Huang, Sai-Kit Yeung
Computer Vision
SortWaste: A Densely Annotated Dataset for Object Detection in Industrial Waste Sorting
Sara Inácio, Hugo Proença and João C. Neves
The increasing production of waste, driven by population growth, has created challenges in managing and recycling materials effectively. Manual waste sorting is a common practice; however, it remains inefficient for handling large-scale waste streams and presents health risks for workers. On the other hand, existing automated sorting approaches still struggle with the high variability, clutter, and visual complexity of real-world waste streams. The lack of real-world
datasets for waste sorting is a major reason automated systems for this problem are underdeveloped. Accordingly, we introduce SortWaste, a densely annotated object detection
dataset collected from a Material Recovery Facility. Additionally, we contribute to standardizing waste detection in sorting lines by proposing ClutterScore, an objective metric that gauges the scene's hardness level using a set of proxies that affect visual complexity (e.g., object count, class and size
entropy, and spatial overlap). In addition to these contributions, we provide an extensive
benchmark of state-of-the-art object detection models, detailing their results with respect to the hardness level assessed by the proposed metric. Despite achieving promising results (mAP of 59.7% in the plastic-only detection task), performance significantly decreases in highly cluttered scenes. This highlights the need for novel and more challenging
datasets on the topic.
Read the full abstract ▾
Computer Vision
Rank-based Geographical Regularization: Revisiting Contrastive Self-Supervised Learning for Multispectral Remote Sensing Imagery
Tom Burgert, Leonard Hackel, Paolo Rota et al. (+1 more)
Self-supervised learning (SSL) has become a powerful paradigm for learning from large, unlabeled
datasets, particularly in computer
vision (CV). However, applying SSL to multispectral remote sensing (RS) images presents unique challenges and opportunities due to the geographical and temporal variability of the data. In this paper, we introduce GeoRank, a novel regularization method for contrastive SSL that improves upon prior techniques by directly optimizing spherical distances to embed geographical relationships into the learned feature space. GeoRank outperforms or matches prior methods that integrate geographical metadata and consistently improves diverse contrastive SSL
algorithms (e.g., BYOL, DINO). Beyond this, we present a systematic investigation of key adaptations of contrastive SSL for multispectral RS images, including the effectiveness of data augmentations, the impact of
dataset cardinality and image size on performance, and the task dependency of temporal views. Code is available at https://github.com/tomburgert/georank.
Read the full abstract ▾
All authors Tom Burgert, Leonard Hackel, Paolo Rota, Begüm Demir
Computer Vision
InfiniteVGGT: Visual Geometry Grounded Transformer for Endless Streams
Shuai Yuan, Yantai Yang, Xiaotian Yang et al. (+4 more)
The grand
vision of enabling persistent, large-scale 3D visual geometry understanding is shackled by the irreconcilable demands of scalability and long-term stability. While offline models like VGGT achieve inspiring geometry capability, their batch-based nature renders them irrelevant for live systems. Streaming architectures, though the intended solution for live operation, have proven inadequate. Existing methods either fail to support truly infinite-horizon inputs or suffer from catastrophic drift over long sequences. We shatter this long-standing dilemma with InfiniteVGGT, a causal visual geometry
transformer that operationalizes the concept of a rolling
memory through a bounded yet adaptive and perpetually expressive KV cache. Capitalizing on this, we devise a training-free,
attention-agnostic
pruning strategy that intelligently discards obsolete information, effectively “rolling” the
memory forward with each new frame. Fully compatible with FlashAttention, InfiniteVGGT finally alleviates the compromise, enabling infinite-horizon streaming while outperforming existing streaming methods in long-term stability. The ultimate test for such a system is its performance over a truly infinite horizon, a capability that has been impossible to rigorously validate due to the lack of extremely long-term, continuous
benchmarks. To address this critical gap, we introduce the Long3D
benchmark, which, for the first time, enables a rigorous evaluation of continuous 3D geometry estimation on sequences about 10,000 frames. This provides the definitive evaluation platform for future research in long-term 3D geometry understanding. Code is available at: https://github.com/AutoLab-SAI-SJTU/InfiniteVGGT
Read the full abstract ▾
All authors Shuai Yuan, Yantai Yang, Xiaotian Yang, Xupeng Zhang, Zhonghao Zhao, Lingming Zhang, Zhipeng Zhang
Computer Vision
TopoLoRA-SAM: Topology-Aware Parameter-Efficient Adaptation of Foundation Segmenters for Thin-Structure and Cross-Domain Binary Semantic Segmentation
Salim Khazem
Foundation
segmentation models such as the Segment Anything Model (SAM) exhibit strong zero-shot generalization through large-scale pretraining, but adapting them to domain-specific semantic
segmentation remains challenging, particularly for thin structures (e.g., retinal vessels) and noisy modalities (e.g., SAR imagery). Full fine-tuning is computationally expensive and risks catastrophic forgetting. We propose TopoLoRA-SAM, a
topology-aware and parameter-efficient adaptation framework for binary semantic
segmentation. TopoLoRA-SAM injects Low-Rank Adaptation (LoRA) into the frozen ViT encoder, augmented with a lightweight spatial convolutional adapter and optional
topology-aware supervision via differentiable clDice. We evaluate our approach on five
benchmarks spanning retinal vessel
segmentation (DRIVE, STARE, CHASE\_DB1), polyp
segmentation (Kvasir-SEG), and SAR sea/land
segmentation (SL-SSDD), comparing against U-Net, DeepLabV3+, SegFormer, and Mask2Former. TopoLoRA-SAM achieves the best retina-average Dice and the best overall average Dice across
datasets, while training only 5.2% of model parameters ($\sim$4.9M). On the challenging CHASE\_DB1
dataset, our method substantially improves
segmentation accuracy and robustness, demonstrating that
topology-aware parameter-efficient adaptation can match or exceed fully fine-tuned specialist models. Code is available at : https://github.com/salimkhazem/Seglab.git
Read the full abstract ▾
Computer Vision
DiffProxy: Multi-View Human Mesh Recovery via Diffusion-Generated Dense Proxies
Renke Wang, Zhenyu Zhang, Ying Tai et al. (+2 more)
Precise human mesh recovery (HMR) from multi-view images remains challenging: end-to-end methods produce entangled errors hard to localize, while fitting-based methods rely on sparse keypoints that provide limited surface constraints. We observe that the true bottleneck lies in the quality of intermediate representations, and that dense pixel-to-surface correspondences can be effectively generated by repurposing pre-trained
diffusion models with rich visual priors. We propose DiffProxy, a Stable-
Diffusion-based framework trained on large-scale synthetic data with pixel-perfect annotations. A multi-conditional proxy generator predicts dense correspondences from multi-view images, providing uniform surface constraints that enable precise fitting. Hand refinement feeds enlarged hand crops alongside full-body images for fine-grained detail, while test-time scaling exploits
diffusion stochasticity to estimate per-pixel uncertainty. Trained only on synthetic data, DiffProxy achieves state-of-the-art results on five diverse real-world
benchmarks. Project page: https://wrk226.github.io/DiffProxy.html
Read the full abstract ▾
All authors Renke Wang, Zhenyu Zhang, Ying Tai, Jun Li, Jian Yang
Computer Vision
VAR RL Done Right: Tackling Asynchronous Policy Conflicts in Visual Autoregressive Generation
Shikun Sun, Liao Qu, Huichao Zhang et al. (+8 more)
Visual generation is dominated by three paradigms: AutoRegressive (AR),
diffusion, and Visual AutoRegressive (VAR) models. Unlike AR and
diffusion, VARs operate on heterogeneous input structures across their generation steps, which creates severe asynchronous policy conflicts. This issue becomes particularly acute in reinforcement learning (RL) scenarios, leading to unstable training and suboptimal
alignment. To resolve this, we propose a novel framework to enhance Group Relative Policy Optimization (GRPO) by explicitly managing these conflicts. Our method integrates three synergistic components: 1) a stabilizing intermediate reward to guide early-stage generation; 2) a dynamic time-step reweighting scheme for precise credit assignment; and 3) a novel mask propagation
algorithm, derived from principles of Reward Feedback Learning (ReFL), designed to isolate optimization effects both spatially and temporally. Our approach demonstrates significant improvements in sample quality and objective
alignment over the vanilla GRPO baseline, enabling robust and effective optimization for VAR models.
Read the full abstract ▾
All authors Shikun Sun, Liao Qu, Huichao Zhang, Yiheng Liu, Yangyang Song, Xian Li, Xu Wang, Yi Jiang, Daniel K. Du, Xinglong Wu, Jia Jia
Computer Vision · Cross-listed from cs.LG
Neuro-Channel Networks: A Multiplication-Free Architecture by Biological Signal Transmission
Emrah Mete and Emin Erkan Korkmaz
The rapid proliferation of Deep Learning is increasingly constrained by its heavy reliance on high-performance hardware, particularly Graphics Processing Units (GPUs). These specialized accelerators are not only prohibitively expensive and energy-intensive but also suffer from significant supply scarcity, limiting the ubiquity of Artificial Intelligence (AI) deployment on edge devices. The core of this inefficiency stems from the standard artificial perceptron's dependence on intensive
matrix multiplications. However, biological nervous systems achieve unparalleled efficiency without such arithmetic intensity; synaptic signal transmission is regulated by physical ion channel limits and chemical neurotransmitter levels rather than a process that can be analogous to arithmetic multiplication. Inspired by this biological mechanism, we propose Neuro-Channel Networks (NCN), a novel multiplication-free architecture designed to decouple AI from expensive hardware dependencies. In our model, weights are replaced with Channel Widths that physically limit the signal magnitude, while a secondary parameter acts as a Neurotransmitter to regulate Signal Transmission based on sign logic. The forward pass relies exclusively on addition, subtraction, and bitwise operations (minimum, sign), eliminating floating-point multiplication entirely. In this proof-of-concept study, we demonstrate that NCNs can solve non-linearly separable problems like XOR and the Majority function with 100% accuracy using standard backpropagation, proving their capability to form complex decision boundaries without multiplicative weights. This architecture offers a highly efficient alternative for next-generation neuromorphic hardware, paving the way for running complex models on commodity CPUs or ultra-low-power chips without relying on costly GPU clusters.
Read the full abstract ▾