Peer Revenue Now accepting positions

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Computer Science Computer Vision

cs.CV Watch Computer Vision

Monday 10 August

Computer Vision

GeoAI-based post-segmentation quality validation of building footprints via spatial feature engineering

Shah Imran Ahsan Chowdhury, Kazi Jihadur Rashid, Rajsree Das Tuli et al. (+2 more)

Deep learning-based building footprint extraction from high-resolution imagery often produces topologically inconsistent vectors unfit for direct GIS database ingestion. To address this, we present a multidomain GeoAI quality control framework that automates error detection to systematically purify vector footprint databases. Candidate footprints were generated across five UAV survey sites in Bangladesh using U-Net (ResNet-34) and SAM-LoRA (ViT-B). The extracted raster masks were vectorized, geometrically regularized, and consolidated under a spatial-exclusivity constraint to eliminate duplicate representations. We used twenty-four predictors capturing geometric, spatial-contextual, and raster-derived spectral and texture properties. Machine Learning (ML) classifiers were trained on a development partition (Sites B-D) and rigorously validated on a spatially independent test set (Site E) excluded from hyperparameter tuning and class balancing. The experimental results demonstrate that geometric and spatial-contextual predictors using Decision Tree (DT) provide the most effective discriminatory evidence for identifying object-level boundary deformations. DT achieved an accuracy of 95.31%, an F1-score of 91.06%, and a Matthews correlation coefficient (MCC) of 0.880 on the unseen testing site. At the database level, this framework successfully identified 87.34% of erroneous footprints while maintaining 98.31% of acceptable structures, reducing the residual error proportion from 27.32% to 4.62% and improving final database purity to 95.38%. This translates into a relative error reduction of 83.09%. The findings indicate that post-segmentation object-level ML provides a highly transferable, robust mechanism for automated quality assurance in production-ready geographic information system (GIS) workflows.
Read the full abstract

All authors Shah Imran Ahsan Chowdhury, Kazi Jihadur Rashid, Rajsree Das Tuli, Rahul Saha, Bulbul Ahammad

Computer Vision · Cross-listed from cs.CR

Diversity Matters: Distributional Feature Coverage Sample Selection for Data-Efficient Backdoor Attacks

Yi Yang, Xiaoke Chen, Jinyang Huang et al. (+6 more)

Backdoor attacks compromise training data so that a model retains clean accuracy but predicts an attacker-chosen target on triggered inputs. At very low poisoning rates, only a few samples convey the trigger–target association, making poison-sample selection critical. Existing methods typically rank candidates using per-sample scores, which can select redundant samples from similar semantic regions, and many require task-specific surrogate training. We propose Distributional Feature Coverage Sample Selection (DFCS), a training-free, trigger-agnostic method that clusters fixed pretrained features into one region per poisoning slot and selects the centroid-nearest sample from each region. A local first-order analysis relates this allocation to feature-coverage and representative-mass terms. Across BadNets and Blended attacks on CIFAR-10, Tiny-ImageNet, and Imagenette, DFCS achieves the highest mean attack success rate among seven selectors in all six dataset–attack settings, averaging $96.30\%$ and exceeding the strongest comparator in each setting by 4.60 percentage points on average while preserving clean accuracy. These results support distributional feature coverage as an effective selection principle for low-budget dirty-label backdoor attacks.
Read the full abstract

All authors Yi Yang, Xiaoke Chen, Jinyang Huang, Feng-Qi Cui, Yu-Tong Guo, Jia-Cheng Zhao, Haiming Jin, Xiaokang Zhou, Meng Li

Computer Vision · Cross-listed from cs.CL

Bridging the Gap Between Semantics and Reconstruction:Unifying Sign Language Translation and Production

Xiao Liu, Shiwei Gan, Yafeng Yin et al. (+5 more)

Recent advances in sign language (SL) research have shown a trend toward unifying multiple sign language understanding (SLU) subtasks, such as isolated sign language recognition (ISLR), continuous sign language recognition (CSLR), and sign language translation (SLT), within a single framework, leading to substantial progress. Meanwhile, sign language production (SLP), which generates sign sequences from text, has also attracted growing attention. This naturally raises an important question: can sign language understanding and production be unified within a single framework? Compared with unifying SLU subtasks, this problem is substantially more challenging. Existing SLU tasks largely share the same direction of mapping, namely from sign inputs to linguistic outputs, whereas SLT and SLP lie in opposite directions of sign-text mapping. A unified framework must therefore address two key challenges: (1) bridging the modality gap between continuous sign motions and discrete text tokens through a shared sign tokenizer that supports both linguistic abstraction and motion reconstruction; and (2) learning a single conditional autoregressive model that can take either sign or text as input and generate the corresponding target sequence in the opposite modality. To this end, we propose Uni-SLTP, a unified framework for SLT and SLP with two key components: (1) a shared sign tokenizer that converts sign sequences into discrete tokens and latent representations, capturing both semantic and reconstructive information; and (2) a unified autoregressive generation model that formulates both tasks as conditional sequence generation. Experiments on widely used public datasets show that Uni-SLTP achieves superior motion accuracy for SLP while maintaining competitive SLT performance.
Read the full abstract

All authors Xiao Liu, Shiwei Gan, Yafeng Yin, Jiaxin Yin, Bowen Guo, Yaqi Sun, Zhiwei Jiang, Lei Xie

Computer Vision

SignLlama: Enhancing Gloss-free Sign Language Translation by Prioritizing Visual Features for LLMs

Shiwei Gan, Xiao Liu, Yafeng Yin et al. (+5 more)

Large Language Models (LLMs) have achieved remarkable success across a wide range of tasks. However, fine-tuning LLMs for Gloss-Free Sign Language Translation (GFSLT) remains a challenge. In this paper, we investigate how to effectively adapt LLMs to the GFSLT task. We show that there are two key issues that need to be solved: (1) the inherent distributional gap between visual feature inputs and text feature inputs makes it difficult for LLMs to interpret visual inputs; and (2) existing approaches typically concatenate visual and textual features in an autoregressive framework, which leads to the model overemphasizing textual inputs and deprioritizing visual cues, as LLMs are pretrained predominantly on text-centric data. To address the first challenge, we propose a simple yet effective method named Filtered Pseudo-Gloss CTC Pretraining, which leverages filtered pseudo-gloss sequences generated from text sequences to supervise the training of the visual backbone. To tackle the second issue, we introduce a Visual-Prioritized Distillation training strategy. Specifically, we define a visual-only prediction path in which text inputs are masked, and the model is required to generate the target sequence relying solely on visual inputs. To guide this path, the outputs from the standard visual-textual prediction are then distilled into the visual-only prediction path, encouraging the model to prioritize visual features. Comprehensive experiments and qualitative analyses demonstrate the effectiveness of the proposed model. The proposed SignLlama achieves very competitive performance on multiple datasets for GFSLT tasks, without using any extra modalities or external sign language datasets for pretraining.
Read the full abstract

All authors Shiwei Gan, Xiao Liu, Yafeng Yin, Zhiwei Jiang, Bowen Guo, Lie Xie, Sanglu Lu, Hongkai Wen

Computer Vision

DeepFreqMark: End-To-End Learnable Frequency-Domain Watermarking with Spherical Attack Simulation for Latent Diffusion Models

Chen-Hsiu Huang, Mario Köppen and Ja-Ling Wu

The proliferation of AI-generated images produced by Latent Diffusion Models (LDMs) has raised critical concerns regarding copyright infringement and misinformation. Although existing frequency-domain watermarking methods embed handcrafted geometric patterns into the initial latent noise prior to generation, they suffer from limited capacity and rigid pattern designs. We propose DeepFreqMark, an end-to-end learnable frequency-domain watermarking framework that replaces manual pattern engineering with a neural message encoder and decoder. To circumvent the computational bottleneck caused by Denoising Diffusion Implicit Model (DDIM) inversion during training, we introduce a Spherical Linear Interpolation (Slerp)-based attack simulation. This approach operates directly on the noise latent while strictly preserving the Gaussian variance. Extensive experiments demonstrate that DeepFreqMark achieves significantly lower Bit Error Rates (BER) than baseline methods under real-world attacks and scales to 256 bits message capacity. Our source code is available at https://github.com/chenhsiu48/DeepFreqMark.
Read the full abstract

Computer Vision

Detecting Clear Contact Lenses for Iris Recognition: A Two-Stage Mask-Guided Attention Approach

Parisa Farmanifard and Arun Ross

This work focuses on the impact and detection of clear contact lenses in the context of iris recognition. While the detection of cosmetic or patterned contact lenses has been extensively studied under the presentation attack detection (PAD) paradigm, clear prescription contact lenses, that are typically transparent, have received comparatively less attention despite their widespread use. Unlike patterned lenses, clear lenses introduce no salient texture artifact, making them difficult to detect and are often assumed to have no impact on iris recognition. We first examine this assumption using the commercial VeriEye matcher on four benchmark datasets and show that clear lenses marginally degrade genuine match scores and increase verification error. We then propose a two-stage contact-lens detection framework. Stage~1 uses an existing PAD model to identify patterned lenses, while Stage~2 focuses on the more challenging clear-lens versus no-lens distinction using a ConvNeXt-Base model equipped with Mask-Guided Spatial Attention (MGSA). The proposed MGSA module incorporates a Hough-derived anatomical ROI mask together with learned spatial attention and Squeeze-and-Excitation channel recalibration, allowing the network to focus on subtle limbal cues associated with clear lens wear. Across four datasets, the full pipeline consisting of both patterned and clear contact lens detection achieves between 90.0%–98.8% accuracy. Finally, we introduce a z-score calibration method that adjusts VeriEye match scores when a clear lens is detected in the input images. This calibration reduces EER by 4.1%–28.3% across datasets, demonstrating that reliable clear contact lens detection can directly improve iris verification performance.
Read the full abstract

Computer Vision · Cross-listed from cs.AI

CoRe-UIE: Rethinking Coexisting and Region-wise Degradation for Underwater Image Enhancement

Weifeng Kong, Chenghao Xu, Lin Chen et al. (+2 more)

Underwater images often suffer from diverse and coexisting degradations, including color distortion, scattering haze, texture attenuation, and uneven illumination. These degradations vary across regions and may coexist locally, making conventional uniform restoration difficult to adapt to different degradation patterns. To address this problem, we propose Coexisting and Region-wise Degradation for Underwater Image Enhancement (CoRe-UIE), a degradation-oriented expert collaboration framework. CoRe-UIE combines a content-preserving shared expert with four shared-backbone routed experts for color correction, scattering suppression, texture recovery, and illumination protection. The routed experts share the same architecture but have independent parameters, and are assigned to different regions through input-derived degradation cues and region-adaptive Top-\(k\) routing. We further introduce a Hilbert–Schmidt Independence Criterion (HSIC)-based representation constraint to reduce statistical dependence among expert features and alleviate redundant expert responses. Experiments on UIEB, LSUI, and U45 demonstrate that CoRe-UIE achieves competitive quantitative performance and visually balanced enhancement under diverse underwater degradation conditions.
Read the full abstract

All authors Weifeng Kong, Chenghao Xu, Lin Chen, Ziheng Cao, Guanying Huo

Sunday 9 August

Computer Vision

MGRL-RSCC: Multi-Granularity Reward Reinforcement Learning for Fine-Grained Remote Sensing Change Captioning

Futian Wang, Mengqi Wang, Xiao Wang et al. (+4 more)

Remote Sensing Change Captioning (RSCC), which aims to generate accurate and detailed linguistic descriptions of ground object variations from bi-temporal remote sensing images, is a critical and challenging task in intelligent remote sensing interpretation. The mainstream autoregressive training paradigm faces severe exposure bias and train-test distribution mismatch, resulting in cumulative generation errors. They tend to produce conservative and template-fixed captions while ignoring subtle scene change details. To address these challenges, this paper proposes a novel multi-granularity reward reinforcement learning paradigm, termed MGRL-RSCC. Specifically, we first leverage a CNN and hierarchical self-attention module to extract and enhance visual features from bi-temporal remote sensing images. A Transformer decoder is then utilized to complete visual-to-linguistic translation. Different from existing methods, we design a dual-decoding strategy and a two-stage joint optimization scheme, which combines token-level supervised learning via greedy decoding and multi-granularity reward-driven self-critical reinforcement learning via sampling decoding. We further construct three complementary reward functions covering linguistic fluency, change state consistency, and structural-semantic relevance to comprehensively optimize caption quality and alleviate false and missing change descriptions. Extensive experiments on multiple public RSCC benchmark datasets demonstrate that the proposed MGRL-RSCC effectively mitigates exposure bias and conservative generation problems in traditional autoregressive methods. The source code and pre-trained models will be released on https://github.com/Event-AHU/MGRL-RSCC
Read the full abstract

All authors Futian Wang, Mengqi Wang, Xiao Wang, Wentao Wu, Haowen Wang, Zhicheng Zhao, Jin Tang

Computer Vision

Moving6DPoSe: A Multimodal Database for Monocular 6D Pose Estimation and Segmentation of Moving Objects

Ignacio Bugueno-Cordova, Javier Ruiz-del-Solar and Rodrigo Verschae

Estimating the 6D pose of moving objects remains challenging due to motion blur and the limited temporal resolution of conventional frame-based cameras. Existing event-based datasets further provide limited sensing modalities, annotations, and motion scenarios. We introduce Moving6DPoSe, a multimodal database comprising two complementary subsets: Moving6DPoSe-R with real-world recordings and Moving6DPoSe-S with synthetic sequences generated from the same objects. The dataset contains 16 scanned objects and 1,702 real and synthetic rosbags spanning multiple motion scenarios, with annotations for semantic segmentation, object detection, and monocular 6D pose estimation. We further provide baseline results for all three tasks across frame and event-based modalities. Experimental results show that event-based representations achieve more robust moving-object segmentation than conventional RGB images, while monocular orientation estimation remains challenging, highlighting the potential of Moving6DPoSe for moving-object perception research.
Read the full abstract

Computer Vision · Cross-listed from cs.RO

Protection Levels for Vision-Based Pose Estimation

Olivia Beyer Bruvik, Romeo Valentin, Marc R. Schlichting et al. (+2 more)

Vision-based navigation complements Global Navigation Satellite Systems, but certification demands integrity guarantees that account for faulty measurements. Previous work presented a probabilistic computer vision pipeline for runway-based pose estimation with fault detection inspired by Receiver Autonomous Integrity Monitoring. This work extends that framework by deriving protection levels, which provide probabilistic bounds on pose error that remain valid under undetected faults. We present an algorithm for computing protection levels for the nonlinear Perspective-$n$-Point problem applied to an aviation setting. The algorithm covers all six degrees of freedom of the aircraft pose (position and orientation) directly. We analyze the effect of measurement redundancy, pixel-level prediction uncertainty, and runway distance on the resulting protection levels. To make the results tangible, we demonstrate tradeoffs in the protection levels on an illustrative runway example.
Read the full abstract

All authors Olivia Beyer Bruvik, Romeo Valentin, Marc R. Schlichting, Don Walker, Mykel J. Kochenderfer

Computer Vision

Fourier Self-Supervision for Fine-Grained Generalized Category Discovery

Sarah Rastegar, Mina Ghadimi Atigh, Pascal Mettes et al. (+2 more)

Generalized Category Discovery aims to recognize known categories while identifying novel ones within unlabeled data. Existing methods, typically based on self-supervision and contrastive learning, often struggle to capture fine-grained distinctions, relying on superficial visual cues rather than the intrinsic attributes humans use for categorization. We introduce Fourier Self-Supervision, that leverages the Fourier transform of images to enhance the discrimination of subtle differences and support the discovery of new categories. Our method employs a dual frequency filtering strategy: a low-pass filter first extracts broad, abstract attributes that capture high-level category information, while a high-pass filter emphasizes fine details such as edges and textures that are essential for fine-grained recognition. Each operates on a dedicated latent space, and their overlapping representations together yield a richer, more complete feature space. This dual-frequency approach not only refines feature extraction to identify novel categories, but also strengthens the model's discriminative power in fine-grained category discovery. Experiments on multiple fine-grained datasets show that incorporating Fourier Self-Supervision outperforms state-of-the-art methods, even when the number of classes is unknown, demonstrating its effectiveness for Generalized Category Discovery. Our code is available at: https://github.com/SarahRastegar/FourEx.
Read the full abstract

All authors Sarah Rastegar, Mina Ghadimi Atigh, Pascal Mettes, Yuki M. Asano, Cees G. M. Snoek

Computer Vision

RMR-P: Road Metadata-Aware Restoration for Pavement Inspection

Amir Ghorbani, Amirali K. Gostar, WeiQin Chuah et al. (+4 more)

Road-surface images captured by vehicle-mounted cameras are often degraded by motion blur, defocus, poor illumination, and noise due to vehicle motion, camera limitations, and varying environmental conditions. These degradations can obscure thin cracks and pothole boundaries that are critical for accurate road-defect detection. This paper presents RMR-P, a restoration network designed to recover defect-relevant information from degraded road images. It estimates degradation characteristics from the input image and can optionally incorporate external degradation parameters to guide restoration. To evaluate whether the recovered information improves downstream detection, a clean-trained YOLO11s detector is applied to degraded and restored images without further modification. Experiments on the IVCNZ and PCM datasets, with known synthetic degradation parameters provided as conditioning information, demonstrate that RMR-P achieves the highest mAP50 in seven of eight held-out degradation conditions, including improvements from 0.140 to 0.427 under IVCNZ motion blur and from 0.060 to 0.233 under PCM defocus. Moreover, our ablation studies show that preserving fine pavement details (detail-preserving pathway) provides the largest contribution to defect-detection improvement, while degradation conditioning and task-guided training offer complementary benefits.
Read the full abstract

All authors Amir Ghorbani, Amirali K. Gostar, WeiQin Chuah, Vahid Ghorbani, Aidan Blair, Reza Hoseinnezhad, Alireza Bab-Hadiashar

Computer Vision

Damage Classification for 3D Point Cloud Data via 3D Data Analysis and Vision Foundation Model-based 2D Projections

Evan Perez, Kalelo Dukuray, Erika Ardiles-Cruz et al. (+1 more)

Fine-grained damage classification of 3D point cloud data (PCD) remains a persistent challenge, constrained by high computational demands and limited labeled data. This study examines two methods: 3D PCD-based damage assessment (3PDA) algorithm and 2D projection damage assessment (2PDA) In our 3PDA analysis algorithm, TDA is used to derive compact representations of 3D PCD segmented by pointNet, which are then integrated with anomaly detection algorithms to quantify structural degradation. We show that TDA effectively compresses geometric structure from VFM-segmented components into discriminative feature vectors and that anomaly detection models can reliably distinguish components with varying damage severity using only 3D PCD inputs. In the 2D projection analysis algorithm, we leverage large VFMs for granular damage detection by projecting 3D PCD into 2D views. These projections allow VFM based models to achieve competitive classification performance while requiring only a fraction of the computational cost associated with full 3D data processing. Our results demonstrate that 2D VFM pipelines in 2PDA can perform strongly on fine-grained damage classification tasks, highlighting their viability as lightweight, resource-efficient alternatives to traditional 3PDA architectures. Comparative evaluation shows that the 3PDA attains higher accuracy but only for a narrow subset of object geometries and at substantially higher computational cost due to its reliance on TDA and the scarcity of high-fidelity 3D datasets. In contrast, the 2PDA algorithm yields slightly lower accuracy but offers an order of magnitude reduction in time complexity and generalizes across a far broader range of object categories.
Read the full abstract

All authors Evan Perez, Kalelo Dukuray, Erika Ardiles-Cruz, Jie Wei

Computer Vision

Topology-Aware Global-Local Mamba Networks for Palm Vein Biometrics

Zhengxi Wu, Felix Marattukalam and Waleed H. Abdulla

Palm-vein recognition is a fine-grained biometric task in which both local vascular texture and the global layout of the vessel tree carry discriminative information, while public datasets remain limited. We propose a topology-aware global-local backbone that combines multi-scale local features, a structureguided directional stream built on a fixed Sobel-magnitude edge prior, and a four-direction state-space scan global pathway within six Topology-Aware Blocks. A staged gated fusion integrates local, structural, and global representations in that order. On HKPUNIR, our method achieves 99.13% top-1 accuracy and 0.08% EER with 7.2 M parameters; on VERA Palm Vein, it achieves 92.42% accuracy and 0.61% EER. Across both datasets it attains the lowest EER among ResNet50, Vim-S, ViT-S, and GLVM at the smallest parameter count, while GLVM remains the strongest in top-1 accuracy and the cheapest in FLOPs. Code is available upon request.
Read the full abstract

Computer Vision

EndoMD-SLAM: Endoscopic Gaussian Splatting SLAM under Optical Degradation with Memory and Static-Transient Decomposition

Nuo Chen, Kangqi Ni, Lulin Liu et al. (+5 more)

Dense 3D reconstruction is critical for clinical endoscopic navigation and documentation. While Gaussian Splatting SLAM systems show promise in this domain, they fundamentally rely on strict multi-view photometric consistency. In routine procedures, this assumption is severely violated by intermittent optical degradations like moving debris and water flushing. Standard systems erroneously fuse these cameraattached artifacts into the persistent 3D geometry, causing severe tracking drift and irreversible map corruption. To address this limitation, we propose EndoMD-SLAM, a framework designed to maintain stability under optical degradation through specialized tracking and mapping mechanisms. On the tracking side, a memory-driven gating mechanism detects unreliable observations to suspend map updates and utilizes historical keyframes for drift-aware relocalization. On the mapping side, a self-supervised static-transient decomposition isolates visual contaminants into a dedicated transient field. This explicit separation prevents artifacts from structurally entangling with the persistent anatomical map. We curate a degradationfocused benchmark from colonoscopy videos to systematically evaluate these failure modes. Extensive experiments show that while standard baselines fail under severe optical degradation, EndoMD-SLAM preserves geometric integrity, reducing absolute trajectory error by 91% and improving rendering fidelity by 9.9 dB PSNR.
Read the full abstract

All authors Nuo Chen, Kangqi Ni, Lulin Liu, Joga Ivatury, Ying Ding, Farshid Alambeigi, Tianlong Chen, Zhiwen Fan

Computer Vision

Can Webcam Gaze Constrain Mesa-Objectives in Driving Models? An Instrument Precision Analysis

Lennox Anderson, Ahmed Boutar, Jonah Mulcrone et al. (+1 more)

Current hazard detection systems in autonomous driving may develop mesa objectives, learned internal goals that achieve high training performance through spurious correlations rather than genuine hazard recognition. We investigate whether human gaze patterns, captured via webcam-based eye tracking (WebGazer.js), can serve as privileged information to constrain mesa-objective formation. We collected 137,663 frame-level gaze samples synchronized with hazard annotations across 388 real dashcam clips, then test this hypothesis across two calibration protocols (9-point/45-click and 11-point/440-click), two model architectures (Random Forest and causal Transformer), and five random seeds per experiment with paired t-tests. No experiment yields a statistically significant improvement from gaze (p = 0.919, 0.578, and 0.667 respectively). A geometric analysis reveals the root cause: WebGazer's reported error (~130-257 px depending on configuration) exceeds 93% of detected hazard object sizes (median 36 px), rendering object-level gaze attribution physically impossible at this instrument precision.
Read the full abstract

All authors Lennox Anderson, Ahmed Boutar, Jonah Mulcrone, Tal Erez

Computer Vision

Zero-shot 2D Grounding with Novel Affordance Types

Haomeng Zhang and Raymond A. Yeh

2D affordance grounding aims to locate the region of an object that a human can interact with. Existing research focuses on recognizing affordance types seen during training and does not study models' ability to generalize to novel affordances, which is crucial for real-world applications. We propose the task of zero-shot 2D grounding with novel affordance types (NAT) and introduce the NAT benchmarks. We then propose AffordAnything, a training-free method that leverages segmentation cues, motivated by the strong correlation between affordance regions and object subparts. To further improve performance, we develop AffordAnything+, a trainable variant that learns to combine these cues. On the proposed AGD20K-NAT benchmark, our best model AffordAnything+ achieves a substantial improvement of 12.3% (absolute) in IoU@0.4 over the SOTA affordance grounding method, OOAL.
Read the full abstract

Computer Vision

From Noise to Meaning: Meaningful Secret Sharing with Tamper Detection for Facial Recognition

Ajnas Muhammed, Iurii Medvedev and Nuno Gonçalves

Popularity of AI-based face recognition system directly demands protection of sensitive biometric data used for training. Visual secret sharing is an interesting idea, as it splits facial images into secret shares that look random and spread across many institutions. However, these shares look like noise and can easily spark suspicion and recognized as encrypted content. This makes them open to targeted collection and harvest-now-decrypt-later attacks. Additionally, visual secret sharing does not detect tampering, allowing attackers to modify shares and threaten the integrity of reconstruction. In this paper, we introduce a new method that turns distracting noise-like secret shares into visually appealing cover images with additional cryptographic tamper detection. The proposed technique works with visual secret sharing and introduces cover images to embed the shares using adaptive least significant bit steganography. Here, cover images with perceptual transparency are used to store secret shares while guaranteeing complete privacy. A two layer authentication using strong digital watermarking and cryptographic hashing is used to protect the integrity of shares. The proposed technique shows high resilience in stopping bit-flipping, cropping, and substitution attacks. Extensive experiments on multiple public face datasets show that the technique shows better FR accuracy, while eliminating share conspicuousness and guaranteeing integrity. The proposed framework sets a new standard for protecting facial data in such a way that privacy, security, and integrity are protected.
Read the full abstract

Computer Vision

ToolVision: Learning When and How to Use Visual Tools with Capability-Aligned Supervision

Delin Mao, Chenghao Sun, Jingwei Song et al. (+2 more)

Thinking with images allows a multimodal model to compensate for limited perception by invoking visual tools through code. Yet the prevailing SFT-then-RL recipe creates a different supervision misalignment at each stage. SFT is expected to teach how to use tools, but trajectories from stronger teachers may succeed through perceptual capabilities that a smaller student cannot reliably reproduce or exploit, causing the student to imitate tool-call patterns without learning how to make them useful. RL is expected to teach when to use tools, but outcome-only rewards make fallible tool execution a liability and suppress tool use, whereas a blanket bonus for every correct tool-using trajectory encourages valid but ineffective operations. To address these two misalignments, we introduce ToolVision. During SFT, a multi-agent pipeline explores candidate trajectories, and a committee including student-scale models scores stepwise evidence gain to rank and prune the search branches. Only successfully executed trajectories with correct final answers are retained for SFT. Before RL, ToolVision compares the learner's performance with and without tools, then rewards successful tool use only on questions where tools provide a clear benefit. Both signals are constructed automatically from public task data without additional human annotations of tool use or necessity. ToolVision-8B improves over its base on all seven main benchmarks, surpasses Thyme-7B, CodeVision-8B, and CodeDance-7B on all three high-resolution benchmarks, and outperforms Qwen3-VL-32B-Thinking on V* and HRBench 8K. We will publicly release the datasets and source code.
Read the full abstract

All authors Delin Mao, Chenghao Sun, Jingwei Song, Chishui Chen, Linfeng Zhang

Computer Vision

From Recovery to Drop-off: How Action Post-training Reduces a VLM's Late-Layer Depth Decodability

Alexander Hackett, Arnaud Denis-Remillard and Axel Cassou

How much of a vision-language model's (VLM) spatial understanding remains after the action post-training process of building a vision-language-action model (VLA)? We probe depth perception, a primitive of spatiogeometric understanding, from every decoder layer of a weight-matched open-source base VLM/VLA pair: Molmo2-ER and MolmoAct2-LIBERO. First, the VLA decodes depth worse at every layer, a persistent gap we call the floor. Second, the degradation is not uniform: while the base VLM's depth decodability improves through its final layers, the VLA's collapses, an additional late-layer drop we call the cliff. We causally localize the cliff to late-layer MLP interference: ablating the late-layer MLP writes recovers the majority of the terminal decodability cliff, while matched attention ablations and the same intervention in the weight-matched base VLM produce no comparable recovery. A module-level decomposition explains this dissociation: the base VLM carries depth most accessibly in accumulated MLP writes, whereas action post-training collapses depth decodability in the late accumulated writes.
Read the full abstract

Computer Vision

City Sentinel: A Unified AI-Based Smart Surveillance Framework for Real-Time Multi-Threat Detection Using Deep Learning

Hanan Syed Shabir, Noor Fatima, Safia Baloch et al. (+1 more)

Rapid urbanization has increased the need for surveillance systems that can monitor multiple public safety risks at the same time. Traditional systems often use separate solutions for facial recognition, vehicle identification, fire detection, and behavioral analysis, resulting in fragmented infrastructure and multiple interfaces for operators to manage. This paper presents City Sentinel, a unified AI-based surveillance framework that integrates six detection capabilities into one scalable platform: facial recognition, automatic number plate recognition (ANPR), fire and smoke detection, weapon and knife detection, violence detection, and road accident detection. The system combines a Next.js operator dashboard, FastAPI backend, cloud-based PostgreSQL event storage, InsightFace and YOLOv8 vision models, and EasyOCR for plate recognition. Camera streams are processed through dedicated inference workers using RTSP. On a workstation equipped with an NVIDIA RTX 3060 GPU, the system achieves a median end-to-end latency of 743 ms and supports four concurrent RTSP streams within a two-second latency limit. It achieves a 91.2% face-match rate, 85.7% plate-reading accuracy, and mAP@0.5 scores of 0.846 to 0.889 across the fire, knife, and weapon detection modules. In user-acceptance testing, operators could enroll a new identity in under one minute and identify a flagged person from live footage in an average of 12 seconds. The results demonstrate that a modular, open-source, multi-model architecture can provide broad surveillance coverage, cloud-based auditability, and flexibility for adding new detection capabilities while maintaining practical real-time performance.
Read the full abstract

All authors Hanan Syed Shabir, Noor Fatima, Safia Baloch, Masroor Hussain

Computer Vision

AeroReformer2: Spoken-Query Referring Segmentation for Aerial Images

Rui Li, Chenxi Duan and Haoyang Yang

Spoken language offers a natural, hands-free interface for specifying an arbitrary target in dense remote-sensing imagery, yet existing referring remote-sensing image segmentation benchmarks accept only written expressions. To bridge this gap, we introduce \dataset, a spoken-query benchmark derived from RISBench that adds accent- and voice-diverse speech while preserving the original image, mask, and data splits. Its hard evaluation sets combine rotor, wind, and mixed interference with three signal-to-noise levels. We also propose \model, an efficient bilateral network that combines a boundary-preserving visual path with token-preserving speech encoding, kernel linear cross-modal attention, and a resolution refinement head. The design conditions visual features at two scales without materializing a dense speech–visual affinity matrix, then restores fine boundaries using high-resolution visual features. On the clean test split, \model with Swin-Base achieves 62.09% mean intersection over union (mIoU) and 68.22% overall intersection over union (oIoU), outperforming the strongest audio-adapted remote-sensing baseline by 5.38 and 2.08 percentage points, respectively. It retains the best hard-set mIoU at 54.09%. To the best of our knowledge, this is the first benchmark and model study of full-sentence spoken-query referring segmentation for remote-sensing imagery. The code will be made publicly available.
Read the full abstract

Computer Vision

Sparse Attention to Emotion: Efficient Facial Emotion Recognition via Token Reduction

Aya Manel Zitouni, Aicha Zenakhri, Karim Haroun et al. (+1 more)

Facial Emotion Recognition (FER) is an important task that has significant implications across various fields such as biometrics, health, and human-computer interaction. Current Vision Transformer-based approaches display quadratic complexity $\mathcal{O}(N^2)$, with N being the input sequence length, making them cumbersome to deploy at the edge. In this paper, we hypothesize that the FER task does not necessarily require all facial information to correctly interpret emotional states, as specific regions such as the eyes, the mouth, and parts of the cheeks carry discriminative information that can be sufficient to recognize emotions. Based on this, we propose Sparse Attention to Emotion (SAE), a model that discards image tokens that have no added value to the emotional context, while preserving good accuracy and achieving a significant gain in computational cost. Surprisingly, even after suppressing 90% of the image tokens, our model achieves competitive accuracy to state of the art methods at much lower cost, providing a lightweight Facial Emotion Recognition approach. Experimental results demonstrate that SAE achieves new state of the art results on the RAF-DB dataset while reducing the computational complexity by up to 90%.
Read the full abstract

All authors Aya Manel Zitouni, Aicha Zenakhri, Karim Haroun, Larbi Boubchir

Computer Vision

Zero-Shot Traffic Accident Detection via a Coarse-to-Fine VLM-Tracking Pipeline

Dipit Saha, Shah Mohammad Abdul Mannan, Mohammad Raihan Rashid et al. (+2 more)

Traffic surveillance cameras capture accidents continuously, yet converting raw CCTV footage into structured event records that pinpoint when, where, and what type of collision occurred remains unsolved at scale. The ACCIDENT @ CVPR benchmark evaluates exactly this joint prediction under a strict constraint: no labeled real-world training data is available. We introduce a training-free, two-pass coarse-to-fine pipeline that pairs a frozen Qwen3-VL-32B-Instruct vision-language model with YOLO11x object detection and BoT-SORT tracking. A first pass sparsely samples the full clip to anchor the collision moment in time; a second pass re-examines a tight window around that estimate using frames annotated with stable vehicle identities and normalized bounding-box coordinates, which gives the model both a visual overlay and an explicit numeric description of the same scene. On the official 2,027-clip real-CCTV test set, our system achieves a three-way harmonic mean score of 0.504, surpassing all organizer-published baselines including the best multi-model ensemble (0.412) by a 22% relative margin.
Read the full abstract

All authors Dipit Saha, Shah Mohammad Abdul Mannan, Mohammad Raihan Rashid, Ruwad Naswan, Ahnaf Tahmid

Computer Vision

Toward Mask Annotation-Free Surgical Instrument Segmentation from Endoscopic Images Using Text-Prompted Segment Anything Model 3 (SAM3)

Nakul Poudel, Richard Simon and Cristian A. Linte

Surgical instrument segmentation is a fundamental task for computer-assisted interventions, yet most existing methods rely on pixel-level annotations or manual spatial prompts, which limit scalability and automation. The recently introduced Segment Anything Model 3 (SAM3) offers a pathway to annotation-free, automatic segmentation via text-based prompting; however, the instrument name as a text prompt could not be directly used due to a large domain gap. To overcome these limitations, we propose a two-stage framework that achieves instance-level segmentation without requiring ground truth masks or manual interaction. In the first stage, we leverage a natural-language-aligned generic prompt - "tool" - to produce binary masks using SAM3's zero-shot capability. In the second stage, these masks are extended to instance-level by integrating a vision-language model (Qwen) that is fine-tuned on SAM3-generated masked regions for instrument classification. We evaluate our approach on the EndoVis 2017 and 2018 datasets. Results show that, while our two-stage approach does not reach the performance of current fully supervised methods, it significantly outperforms the direct use of SAM3 for instance-level instrument segmentation with text prompts. Overall, our findings highlight both the limitations and potential of SAM3, suggesting a promising direction toward annotation-free surgical instrument segmentation.
Read the full abstract

Computer Vision

SLAP: Selective Local Vision-Language Alignment for Fish Re-Identification via Partial Optimal Transport

Cigdem Beyan, Tonje Knutsen Sordalen and Kim Tallaksen Halvorsen

Individual fish re-identification (ReID) is a fine-grained recognition problem in which identity-discriminative cues are often localized to specific body regions rather than distributed uniformly across the animal. Nevertheless, recent CLIP-based ReID methods rely predominantly on global image-text alignment, allowing background and weakly discriminative regions to contribute to cross-modal supervision. We propose a selective local vision-language alignment framework that establishes localized correspondences between visual patch embeddings and multiple identity-aware prompt embeddings through Partial Optimal Transport (POT). Rather than enforcing exhaustive correspondence, POT enables selective matching between visual patches and prompt embeddings, allowing the model to emphasize the strongest cross-modal correspondences while avoiding forced alignment of weakly matching regions, thereby yielding more discriminative visual representations for retrieval. The framework is trained end-to-end, while only the adapted visual encoder is retained during inference. Experiments on the longitudinal Symphodus melops dataset demonstrate consistent improvements over recent CLIP-based ReID methods under both closed-set and open-set evaluation protocols. Additional evaluations on other datasets further demonstrate the generalization capability of the proposed method across diverse marine ReID benchmarks.
Read the full abstract

Computer Vision · Cross-listed from cs.RO

SG-WAM: Text-Grounded and Spatial-aware Semantic Guidance for World-Action Models

Junjie He, Junfeng Li, Zhide Zhong et al. (+9 more)

World-Action Models (WAMs) have emerged as a promising paradigm for robotic manipulation. However, most existing WAMs generate future videos and actions by relying mainly on visual cues rather than language instructions, since off-the-shelf text encoders embed instructions independently of visual observations. As a result, the videos predicted by these WAMs are often semantically misaligned with their corresponding language instructions, which degrades the accuracy of the predicted actions. To overcome this limitation, we propose SG-WAM, a semantic guidance method for world-action models that leverages a vision-language model (VLM) as a semantic planner to enhance the instruction-grounding capacity of world-action models. Specifically, we train a VLM-based planner to predict text-grounded and spatial-aware semantic foresight. The text-grounded semantic foresight grounds the instruction by identifying the correct target objects, and the spatial-aware semantic foresight provides the scene geometry for precise manipulation. We then inject this foresight into the world-action model as high-level semantic guidance, ensuring that both future-video generation and action prediction faithfully follow the language instruction. Extensive experiments in simulation and the real world demonstrate the superiority of our semantic guidance method, showcasing precise manipulation and strong instruction-following capabilities.
Read the full abstract

All authors Junjie He, Junfeng Li, Zhide Zhong, Haodong Yan, Ruixin Li, Yangyang Zheng, Jiaguan Zhu, Tianran Zhang, Yuqiao Du, Wen Chen, Shunbo Zhou, Haoang Li

Computer Vision

Visual Token Codec: Unleashing Spatial Redundancy for ViT Feature Coding

Donghui Feng, Fengxi Zhang, Changsheng Gao et al. (+6 more)

Distributed deployment of large vision foundation models often partitions a ViT backbone and exchanges intermediate token features between computing nodes, making efficient feature compression critical under bandwidth and computation constraints. Existing ViT feature codecs typically flatten heterogeneous global and patch tokens into an L x C pseudo image, causing entropy models to mainly capture sequence-axis dependencies while overlooking the native two-dimensional patch-grid structure. In this paper, we show that ViT patch tokens retain strong local spatial correlations on the original grid. To exploit this structural prior, we propose the Visual Token Codec (VTC), a dual-path learned codec that separates global and patch tokens into dedicated coding paths. Global tokens are compressed with a lightweight factorized prior, whereas patch tokens are encoded on the patch-token grid using a spatial-channel context entropy model. To support intermediate-layer compression and practical rate adaptation, VTC further incorporates feature-matching supervision after subsequent ViT blocks and variable-rate modules within a single codec. Experiments on DINOv2 and SAM3 show that VTC consistently outperforms representative ViT feature coding baselines on classification, segmentation, and detection tasks. At 90% of uncompressed-feature performance, VTC reduces bitrate by 15.7x-37.4x across these tasks. We further provide intermediate-layer rate-utility analyses for practical transmission- and storage-oriented deployment scenarios.
Read the full abstract

All authors Donghui Feng, Fengxi Zhang, Changsheng Gao, Wenhan Yang, Qi Wang, Qunshan Gu, Hongwei Hu, Zhengxue Cheng, Li Song

Computer Vision

LogiShot: Logically Coherent Cross-Shot Video Generation

Shuai Guo, Yuhang Yang, Zeyu Zhang et al. (+4 more)

Generating cross-shot videos that are logically connected is essential for content creation. Currently, most cross-shot video-generation workflows, such as short-drama production, still rely on isolated textual scripts or explicit reference images to specify the generated content. Consequently, when user instructions are underspecified or ambiguous, a generated clip may appear visually plausible on its own but fail to align with the overall narrative, leading to disjointed content. We argue that achieving cross-shot logical coherence in video generation requires establishing logical connections across shots and maintaining visual consistency. To this end, we propose LogiShot, which incorporates information through two complementary paths: 1) LogiShot jointly encodes the context video and other conditioning signals, yielding dense multimodal cues that provide visual-semantic evidence for cross-shot generation; 2) the model maintains a visual memory of the context video throughout generation to preserve visual consistency across shots. Additionally, we construct a dataset with 110K samples and a dedicated benchmark for evaluating cross-shot logical coherence. Experiments demonstrate that LogiShot consistently outperforms existing baselines in terms of logical coherence across multiple shots. Model and data will be made publicly available.
Read the full abstract

All authors Shuai Guo, Yuhang Yang, Zeyu Zhang, Pengfei Yu, Wei Zhai, Yang Cao, Zheng-Jun Zha

Computer Vision

MRI super-resolution in ten sampling steps using a diffusion bridge model

Mojtaba Safari, Hang Yu, Zach Eidex et al. (+10 more)

Objective. MRI provides excellent soft-tissue contrast, but long acquisition times can cause patient discomfort and lead to motion artifacts, forcing a trade-off between spatial resolution and scan time. Diffusion-based super-resolution (SR) reconstructs high-resolution (HR) images from low-resolution (LR) inputs, but typically needs many sampling steps and initializes from a Gaussian prior ill-suited to image restoration. We developed an efficient diffusion framework that reconstructs HR MRI directly from LR data. Approach. We propose super-resolution diffusion bridge model (SR-DBM), a super-resolution diffusion bridge model that casts SR as a stochastic transport between the LR and HR image distributions. Through a Doob's h-transform of a mean-reverting stochastic differential equation, SR-DBM pins the process to the paired HR and LR images at its endpoints, initializing reconstruction from the measured anatomy rather than from Gaussian noise. The HR image is recovered by a deterministic reverse trajectory in which a network predicts the clean image at each of only ten sampling steps. We evaluated SR-DBM on ultra-high-field 7T brain T1 MP2RAGE maps and pelvic T2-weighted prostate images against nine comparison methods using PSNR, SSIM, GMSD, and LPIPS. Main results. SR-DBM attained the highest PSNR and SSIM and the lowest GMSD on both datasets (brain: 27.66+-1.52 dB, 0.96+-0.02, 7.96+-1.86$; prostate: 27.87+-2.29 dB, 0.80+-0.05, 8.38+- 1.44), with statistically significant gains over every comparison method (two-sided Wilcoxon signed-rank test with Holm correction, p<0.05). The strongest baseline, SR-EMamba, ranked second. Qualitatively, SR-DBM produced the smallest residual errors and best preserved fine structures and lesions.
Read the full abstract

All authors Mojtaba Safari, Hang Yu, Zach Eidex, Mingzhe Hu, Ryan J. Sanford, Alexandru Florea, Shansong Wang, Chih-Wei Chang, Erik H Middlebrooks, Aditya Juloori, Stanley L. Liauw, Ralph Weichselbaum, Xiaofeng Yang