Computer Vision
CG-HAF: An Interpretable Global-Local Lesion-Burden Fusion Framework for Ordinal Acne Severity Grading in Agentic Skincare Support
Muhammad Muhtasim Shahriar, Md. Naimur Asif Borno, Saad Aloteibi et al. (+1 more)
Ordinal acne severity grading requires distinguishing visually similar neighboring grades while jointly weighing holistic facial appearance and localized lesion burden - evidence that most existing approaches collapse into a single opaque representation. We introduce CG-HAF, a global-local fusion framework that instead keeps this evidence explicit: averaged holistic severity probabilities from independently trained classifiers are combined with structured lesion-burden descriptors from an object detector (lesion count, detection confidence, lesion area) into a compact representation, from which a lightweight, interpretable classifier produces the final grade. On a widely used
benchmark, this fusion yields a clear, statistically supported improvement over global-evidence-only baselines, with the largest gains on the most severe cases. Testing on an independent
dataset with a different grading standard shows that strong within-
dataset performance does not transfer automatically, and a follow-up diagnostic attributes much of this gap to mismatched grading criteria rather than detection failure alone. These findings support interpretable global-local fusion as an effective strategy for ordinal acne grading while highlighting criterion
alignment as key to cross-
dataset portability, with a further illustration of how the resulting severity signal can support transparent, non-diagnostic decision-making in skincare applications.
Read the full abstract ▾
All authors Muhammad Muhtasim Shahriar, Md. Naimur Asif Borno, Saad Aloteibi, Mohammad Ali Moni
Computer Vision
CytoSPM: Open-Vocabulary Cytopathology Detection with Structured Prompt Bank
Wenjie Li, Zishan Xu, Jinyang Huang et al. (+3 more)
Cytopathology detection requires open-vocabulary recognition because cellular categories are fine-grained, long-tailed, and continuously evolving across different organ systems. However, existing cytology detectors are mostly single-domain and closed-set, and there is still no unified
benchmark for evaluating open-vocabulary cytopathology detection. We present PentaCyto, a multi-domain
benchmark covering cervical, urinary, respiratory, serous fluid, and thyroid cytology, with 24 base categories and 9 held-out novel categories. Each category is associated with structured cytomorphology
prompts that describe diagnostic morphological attributes and provide clinically grounded textual knowledge. We further propose CytoSPM, an efficient detector based on a decoupled two-stage design. It first extracts reusable class-agnostic visual representations, and then performs class-aware structural
prompt matching with class names and cytomorphology
prompts. On PentaCyto, CytoSPM outperforms existing methods in novel-category detection and open-vocabulary detection while maintaining efficient inference.
Read the full abstract ▾
All authors Wenjie Li, Zishan Xu, Jinyang Huang, Zhengxin Nie, Shichao Kan, Yixiong Liang
Computer Vision
UniAR: A Unified Framework for Autism Recognition Enhanced by Multi-View Prompt Learning
Lei Xin, Zeheng Wang, Jiayin Zhu et al. (+6 more)
Autism Spectrum Disorder (ASD) is a complex neurodevelopmental disorder for which early and accurate diagnosis is critical to improving long-term developmental outcomes. However, existing ASD recognition methods are often constrained by the scarcity of diagnostic text data, forcing them to rely mainly on visual analysis and limiting their ability to model clinically meaningful semantic
reasoning. To address this challenge, we propose UniAR, a unified framework enhanced by multi-granularity
prompt learning for robust ASD recognition under heterogeneous data variations. Specifically, UniAR leverages a large
multimodal model to generate hierarchical diagnostic descriptions at the word, phrase, and sentence levels, compensating for the lack of paired clinical reports. To align the generated semantics with visual evidence, we further design a Mixture-of-Experts-based Multi-Scale
Alignment Module, which dynamically matches vector-quantized visual prototypes with semantic representations at corresponding granularities. Extensive experiments on four
benchmarks covering
brain MRI and facial expression scenarios show that UniAR consistently outperforms existing state-of-the-art methods, achieving average accuracies of 75.9% on MRI
benchmarks and 91.6% on facial
benchmarks, while improving average Accuracy on MRI
benchmarks by 1.5 percentage points and average Accuracy on facial
benchmarks by 1.2 percentage points over baselines. These results demonstrate that UniAR offers a robust and interpretable framework for ASD screening under semantic scarcity.
Read the full abstract ▾
All authors Lei Xin, Zeheng Wang, Jiayin Zhu, Shihong Huang, Fanhu Zeng, Changjiang Jiang, Dengbo He, Yutao Yue, Zhenglun Kong
Computer Vision
MoTop: Motion-Topological Model For Micro AU Detection
Huai-Qian Khor, Mengting Wei, Yante Li et al. (+2 more)
Facial micro-expressions are spontaneous, brief, and subtle facial movements that reveal suppressed
emotions in high-stakes environments. In contrast to classic expression analysis, detecting action unit (AU) yields a finer representation of facial movements, serving as a preliminary step before defining expression classes and other downstream tasks. Therefore, it represents a crucial upstream task in facial analysis, and improving an AU detection module increases the precision of facial analysis. Despite that, detecting AU is challenging because of the constrictive nature of the AU activation regions, leading to confusion among different AUs known as AU ambiguity. To model the fine-scale changes, we propose MoTop, a motion-topological model that is augmented with a learnable motion context, yielding regional soft guidance for facial activity, followed by facial landmarks that capture the fine-scale topological changes of micro AUs. To increase the micro facial landmark representations, we amplify the encoded facial landmark transitions via linear extrapolation, thereby increasing the spatial proximity of landmarks and enhancing the low-intensity landmark dynamics. In addition, we design anatomical facial clusters that enhance the hierarchical representation, facilitating multi-scale modelling of facial geometry and improving micro-topological representations. With these contributions, we have achieved state-of-the-art performance on the CD6ME protocol for the micro AU detection task.
Read the full abstract ▾
All authors Huai-Qian Khor, Mengting Wei, Yante Li, Chu Kiong Loo, Guoying Zhao
Computer Vision
Gauss What You Need: Compact Gaussian Splatting Across Scene Scales
Afif Boudaoud, Jiayi Liu, Alexandru Calotoiu et al. (+1 more)
3D Gaussian Splatting reconstructs a scene as a collection of Gaussian primitives from a set of posed photographs called the capture. The number of primitives used to represent the scene affects reconstruction quality, storage, and rendering cost. How to select this number automatically across capture scales remains unresolved: configurations effective on standard
benchmarks can leave larger captures with too few Gaussians to reconstruct fine details. We observe that the surface to represent, given by the capture's extent and resolution, is known before training, whereas its content complexity becomes apparent during training, through the reconstruction quality on the training views. We introduce TangoGS, which combines capture-derived model sizing with training-based adaptation: the capture determines the scale of the model, and training feedback determines its final size within that scale. Before training, TangoGS derives a learning allowance for model growth from the capture's total pixels after discounting views that re-observe the same scene points. During training, reconstruction quality guides how many Gaussians to add and remove. On 13 standard
benchmark scenes, TangoGS matches the mean PSNR of the best-performing evaluated baseline, LeGS, with $48\%$ fewer Gaussians. On eight large captures, the same configuration automatically scales to larger models when necessary, achieving the highest mean PSNR among evaluated methods: $0.54$ dB above the runner-up with $2.3\times$ as many Gaussians. Together, capture-derived learning allowances and training-quality guided density control enable a state-of-the-art quality–size compromise across scene scales without retuning.
Read the full abstract ▾
All authors Afif Boudaoud, Jiayi Liu, Alexandru Calotoiu, Torsten Hoefler
Computer Vision
Geometric Inconsistency Localization in Multi-View Image Sets
Xander Staelens, Albéric Loos, Bert Ramlot et al. (+3 more)
Novel view synthesis (NVS) models can produce realistic new views of the same scene from different viewpoints. However, these generated views are not always geometrically consistent with one another. Multi-view (MV) consistency has shown promise as a tool for evaluating these NVS models. Its potential for multimedia forensics, however, remains largely unexplored, particularly for localizing
geometric inconsistencies across wide-baseline image pairs. To enable research in this direction, we introduce DeformView, a wide-baseline MV
dataset with pixel-level annotations of
geometric inconsistencies. Using DeformView, we evaluate state-of-the-art MV consistency-scoring methods and show that approaches developed for NVS evaluation transfer poorly to the forensic task of
geometric inconsistency localization. To address this limitation, we propose DEFECt3R, a lightweight learning-based classifier that uses cross-view feature relationships to localize
geometric inconsistencies at the pixel level. By learning from explicit supervision, including hard negatives from geometrically consistent yet deformed views, DEFECt3R improves localization performance and substantially reduces false positives compared to existing consistency-scoring methods. Ablation experiments further show that both feature representations and correspondence quality contribute to localization performance. Overall, our findings demonstrate that MV
geometric consistency is a promising yet underexplored signal for multimedia forensics and establish a
benchmark and baseline for
geometric inconsistency localization in wide-baseline MV image pairs. Code and
dataset are available at https://github.com/IDLabMedia/DeformView-DEFECt3R
Read the full abstract ▾
All authors Xander Staelens, Albéric Loos, Bert Ramlot, Hannes Mareen, Peter Lambert, Glenn Van Wallendael
Computer Vision
WeaveAgent: A Two-Stage Tool-Routing Agent for Ultra-High-Resolution Remote Sensing Imagery
Zhongyu Pang
Problem. Ultra-high-resolution (UHR) remote sensing with vague user intents has two bottlenecks: visual
tokens are expensive, and tool calling must be format-reliable (pretrained models emit zero tool calls zero-shot). Method. WeaveAgent, a two-stage tool-routing
agent, decouples routing from visual perception. Stage A is routing-first: emission is trained, not elicited. Stage B executes conditionally: intrinsic queries enter visual answering (full-scene thumbnail; a WeaveEarth-style evidence board as an optional fixed-budget, approx. 5k-
token compression interface); extrinsic queries execute tool call on original full-resolution imagery, answering from tool observations in a second, observation-masked round. Training:
alignment SFT, then GRPO under reward R_WA2. Results.
Alignment SFT lifts extrinsic routing from 0% to 80.75% (323/400); GRPO suppresses 9 intrinsic mis-emissions while tool selection is unchanged. The trained 2B system does not beat the zero-shot 8B baseline overall (0.263 vs. 0.250), a diagnostic contribution. Oracle attribution separates two repair ingredients: loading the observation into context lifts extrinsic answer accuracy from 0.025 to 0.425 under marker-free cross-mode returns, and the two-turn SFT stage adds a further +9.3 points to 0.518 at a small routing cost. A +/- image ablation shows emission suppression is visually grounded, and a query-register
matrix shows LLM-rewritten queries cost trained checkpoints 2-11 points. Scope. All training and evaluation use the 5,000 / 3,273 / 1,000-record VagueUHR corpus (600 intrinsic + 400 tool-requiring; the base seeds synthesis and is not used for optimization). Single-pass evidence construction runs at 7.31 s per image on an RTX 4090. Code, data, and evaluation protocols will be released.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Enabling a Unified Cross-Domain Representation for Two-Finger Gripper Manipulation via Interaction-Centric Modeling
Guanlin Li, Shifeng Bao, Yihan Zhao et al. (+6 more)
Achieving robust cross-embodiment generalization in imitation learning demands overcoming a critical representation flaw that inextricably entangles task semantics with hardware-specific visual geometry. We propose an interaction-centric framework that leverages the shared structure of two-finger grippers via a parameterized universal gripper abstraction, yielding a canonical gripper-frame representation. Given
language and RGB-D observations, a VLM infers the subtask and grounds an interaction triplet (gripper, held, target), while SAM~2.1 tracks masks to reduce VLM queries. We design concise hybrid features that combine target/collision artificial potential fields for global guidance with segmented gripper-frame point clouds for local geometry, and use a Flow-Matching
Transformer to predict smooth 7-DoF action chunks. Experiments in simulation and real-world tasks demonstrate that ours is the first imitation learning approach to simultaneously achieve competitive
benchmark scores and extreme cross-embodiment/cross-viewpoint zero-shot sim-to-real transfer to completely distinct, heterogeneous
robot platforms.
Read the full abstract ▾
All authors Guanlin Li, Shifeng Bao, Yihan Zhao, Haitao Shen, Haoyang Li, Chen Zhao, Tong Yang, Jie Tang, Jing Zhang
Computer Vision · Cross-listed from cs.CE
FlatClip: Reusing Image Foundation Models for fMRI Representation Learning via Cortical Flatmaps
Mo Wang, Wenhao Ye, Zihan Ning et al. (+4 more)
Can the spatial organization of cortical activity help transfer frozen image representations to fMRI prediction? We introduce FlatClip, which renders cortical activity as anatomy-linked flatmaps and extracts features with a frozen SigLIP2 encoder. Only a lightweight downstream probe is trained, without fMRI-specific encoder pretraining or fine-tuning. Across four resting-state
benchmarks, FlatClip outperforms the evaluated ROI-level baselines on HCP and ADNI, but is weaker on PPMI and below the strongest voxel-level models overall. On NSD visual-fMRI decoding, restricting inputs to visual or NSD-provided task-active cortex improves recognition over whole-cortex inputs. Spatial perturbations reduce performance with both retrained and fixed readouts, and anatomy-linked arrangements outperform cortical-pixel permutations across three colormaps. Together, these results demonstrate a practical route for reusing image-pretrained features in fMRI and highlight cortical arrangement and input-region selection as important design choices. Code is available at https://github.com/OneMore1/FlatClip.
Read the full abstract ▾
All authors Mo Wang, Wenhao Ye, Zihan Ning, Jiayu Zuo, Junfeng Xia, Hongkai Wen, Quanying Liu
Computer Vision
Preserve-and-Compose Training for Composed Image Retrieval
Sehyun Kwon
Composed image
retrieval (CIR) aims to retrieve images that satisfy a user-specified modification while preserving relevant visual content from a reference image. Collecting target images for this purpose is costly, motivating zero-shot CIR methods that instead use target captions as supervision. However, target captions may omit source details that should be preserved. We therefore propose, Preserve-and-Compose Training, which complements target-caption supervision with visual evidence from the source image. PACT learns from image–text–text (ITT) triplets without target images or gallery updates, aligning composed queries with target captions while preserving source evidence through visual supervision. We further introduce Chord scoring, which combines target similarity with source-relative directional agreement in the frozen image space. Results across four ZS-CIR
benchmarks show that combining target-caption supervision with source-image evidence leads to strong
retrieval performance across
datasets, backbone scales, and external galleries. The code is available on https://github.com/sehyunkwon/PACT.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Enhancing Photogrammetric Digital Surface Models with Pretrained Diffusion Models and Multimodal Conditioning
Antoine Lorentz, Stéphane May, Valentine Bellet et al. (+2 more)
Large-scale Digital Surface Models (DSMs) can be produced cost-effectively from satellite images via stereo-photogrammetry. However, the resulting 3D maps are often contaminated by noise, outliers, and voids. On the other hand, aerial
LiDAR provides high-accuracy elevation measurements at a substantially higher cost. In this work, we study
diffusion models conditioned both on photogrammetric DSMs and Pléiades imagery to refine vertically co-registered DSMs. We introduce a modified Stable
Diffusion 3 architecture with a pruned text stream and a patch-wise normalization strategy, enabling stable training on
LiDAR data and transfer from natural images to elevation maps. Experiments in French cities demonstrate that
multimodal conditioning improves elevation accuracy, reducing Dense Urban RMSE from 6.00 to 3.45 m in the in-context cities and from 4.16 to 2.77 m in the held-out city of Bordeaux.
Read the full abstract ▾
All authors Antoine Lorentz, Stéphane May, Valentine Bellet, Dawa Derksen, Bastien Nespoulous
Computer Vision
Light Field Primitive for Novel View Synthesis
Liang Chen, Jiahui Ning, Xun Jiang et al. (+4 more)
We present Light Field Primitives (LFP), a formulation for novel view synthesis that replaces the dense ray database with a compact set of differentiable primitives in the classical two-plane parameterization. Each primitive condenses a group of rays into one learned record, and its response to a query is governed by how closely that query belongs to the group. Rendering a camera ray then reduces to compositing all responses it elicits, and a scene can be optimized directly from posed images and rendered in real time with rays. Beyond its competitive performance on standard
benchmarks, the main advantage of LFP is structural: its primitives reside directly in the 4D ray space, so optical and appearance effects that are already operations on the light field become behaviors of a single shared renderer. With minimal changes to that renderer, LFP supports multi-scale anti-aliasing, defocus deblurring with refocusing, rendering for fisheye cameras, and even transparent object reconstruction with ray refraction, matching specialized frameworks that devote substantial machinery to these effects.
Read the full abstract ▾
All authors Liang Chen, Jiahui Ning, Xun Jiang, Xing Xu, Jimmy Ren, Fenglei Fan, Heng Tao Shen
Computer Vision
Who Says What: Symbolic Trimodal Binding Mechanisms in Audio-Visual LLMs
Jihoo Jung, Youngjoon Jang and Joon Son Chung
Current Audio-Visual LLMs (AVLLMs) struggle with
reasoning over
videos featuring multi-speaker dialogues. In such
videos, resolving "who says what" is crucial, which necessitates trimodal (text-audio-visual) binding. Motivated by these challenges, we systematically investigate how this trimodal binding is achieved in AVLLMs. Specifically, we identify emergent symbolic trimodal binding mechanisms in AVLLMs that utilize modality-specific symbolic variables. By encoding auditory and visual components into symbolic variables-capturing temporal utterance sequences and spatial entity coordinates, respectively-the model establishes cross-modal linking within this abstract space. Crucially, we reveal that when trimodal binding fails, the breakdown predominantly stems from misaligned audio-visual connections. To overcome this bottleneck, we introduce an audio-visual prompting method utilizing an off-the-shelf Active Speaker Detection (ASD) model. By simply overlaying visual bounding boxes on active speakers, this training-free approach yields immediate performance gains across four conversation-centric
benchmarks. Moreover, lightweight fine-tuning of fewer than 300 steps on these ASD-prompted-
videos extends these gains to three general AV
benchmarks, suggesting the generalizability of our method.
Read the full abstract ▾
Computer Vision
TaskIR: Task-Driven Image Restoration via Degradation Adaptation and Task Feedback
Yanjie Tu, Qingsen Yan, Axi Niu et al. (+4 more)
Task-driven image restoration aims to improve both image quality and downstream task performance. However, existing methods predominantly focus on single degradation type and struggle to handle the diverse degradations encountered in real-world scenarios. Different degradations impose distinct restoration demands, and insufficient restoration may leave residual degradations and artifacts that impair object boundaries and semantic cues, thereby compromising downstream task performance. To address these challenges, we propose TaskIR, a two-stage task-driven unified image restoration framework that integrates degradation-adaptive restoration with task feedback refinement. In Stage I, a Degradation Representation Module (DRM) extracts degradation representations, enabling a Degradation-Guided
Transformer Block (DGTB) to dynamically modulate feature transformations for adaptive restoration. In Stage II, a Task-to-Restoration Feedback Generation module (TRFG) transforms heterogeneous task features into restoration feedback by modeling task-representation discrepancies associated with the current restoration. Subsequently, a Selective Task Feedback Refinement module (STFR) assesses feedback relevance and selectively refines intermediate restoration features to mitigate interference with well-restored content. Extensive experiments demonstrate that TaskIR achieves competitive restoration quality and downstream task performance across diverse degradations and tasks.
Read the full abstract ▾
All authors Yanjie Tu, Qingsen Yan, Axi Niu, Wenxuan Cai, Tao Hu, Wei Dong, Haokui Zhang
Computer Vision
ReG-SAM: Reference Graph-Driven SAM for 2D Foundational Vessel Segmentation
Donghang Lyu, Zichen Zhang, Oleh Dzyubachyk et al. (+1 more)
Vessel
segmentation in medical images is essential for many clinical tasks, ranging from diagnosis to treatment planning. However, it remains challenging due to complex vascular morphology and diverse imaging conditions. Existing deep learning methods rarely aim at building a generalizable vessel segmentor across anatomies and modalities. While the Seg- ment Anything Model (SAM) has shown promise for med- ical image
segmentation, its original design does not fully exploit vascular morphology and struggles with fine-grained vascular structures, leading to suboptimal performance. In this paper, we propose ReG-SAM, a SAM-based framework tailored to 2D vessel
segmentation that leverages reference graph set for enhancing vascular representations. Specifically, we introduce two modality-aware representations derived from the reference masks: graph
prompt embeddings (GPEs) that encode global spatial features from graphs, and vascu- lar prototype
embeddings (VPEs) that capture fine-grained modality-specific vessel characteristics from multi-scale fea- ture maps and vascular masks. Since both require vascular masks that are unavailable during inference and require robust modality-aware vascular feature representations, we construct a modality-wise vascular database and develop two reference graph-guided representation learning schemes for estimating GPEs and VPEs using samples from the database rather than ground-truth masks. Extensive experiments across 19
datasets demonstrate that ReG-SAM consistently outperforms existing baselines, even those using manual
prompts, particularly on challenging thin vessels.
Read the full abstract ▾
All authors Donghang Lyu, Zichen Zhang, Oleh Dzyubachyk, Marius Staring
Computer Vision
HyperErase: Scale-Calibrated Hypernetwork for Multi-Concept Erasure in Text-to-Image Models
Yi Sun, Xinhao Zhong, Zhiqi Zhang et al. (+3 more)
Recent advances in text-to-image (T2I) generation have substantially improved visual synthesis, but have also raised increasing safety concerns due to their potential to generate harmful or undesirable content. Existing concept erasure methods predominantly follow a static weight paradigm, producing a single frozen adapter that struggles to adapt to diverse
prompt variations and suffers from parameter interference when scaling to multiple concepts. We propose HyperErase, a framework for concept erasure based on hypernetwork-driven
prompt-conditioned parameter synthesis. Our approach first reframes concept erasure as
prompt-conditioned parameter amortization and trains a hypernetwork to map textual descriptions to
prompt-specific LoRA updates, eliminating the need for per-
prompt gradient optimization or manual LoRA merging. To further improve the stability and precision of synthesized adapters, we develop a decoupled rectification strategy, which disentangles LoRA
tokens into pattern and scale subspaces, applies a square-root transform to curb multiplicative over-scaling, and leverages teacher-derived canonical priors for inference-time correction. Extensive experiments across major concept categories demonstrate that HyperErase consistently improves the trade-off between erasure effectiveness, image quality, and semantic
alignment, achieving performance comparable to gold-standard single-concept baselines. Furthermore, the resulting models can provide specialized LoRAs for each input
prompt variation in a single forward pass without requiring gradient updates during inference. These principled and flexible framework offers a new paradigm for concept erasure in T2I models.
Read the full abstract ▾
All authors Yi Sun, Xinhao Zhong, Zhiqi Zhang, Yimin Zhou, Junhao Li, Yuxia Qiao
Computer Vision
FedHisto-PAST: Parameter-Efficient Stain-Aware Federated Learning for Cross-Site Lung Histopathology Classification
Muhammad Muhtasim Shahriar, M. M. Golam Hafiz, Saad Aloteibi et al. (+1 more)
Cross-site lung histopathology classification must account for stain variation, non-IID client data, missing classes, and the cost of adapting large pathology encoders. This study evaluates FedHisto-PAST v2 for three-way classification of adenocarcinoma (ACA), Normal, and squamous
cell carcinoma (SCC). FedHisto-PAST v2 combines a frozen HIBOU-B foundation model with parameter-efficient adaptation, stain-conditioned paired-view prediction and feature consistency, reliability-aware prototype learning, and adaptive
federated aggregation. Experiments used a five-client, non-IID, raw-data-local simulation with fixed internal evaluation, client-level analysis, component ablations, communication accounting, and a development-influenced exploratory LungHist700 cohort. All principal methods achieved near- ceiling internal performance, which limited discrimination on the fixed split. On LungHist700, FedHisto- PAST v2 achieved a Macro-F1 of 0.728560 and a balanced accuracy of 0.730454. Higher recognition of Normal and SCC was accompanied by lower ACA recall, and calibration remained imperfect. Prediction-level consistency was the only component with a clearly supported independent contribution in the external ablation analysis. Feature consistency and prototype regularization showed no conclusive independent overall gains in Macro-F1. The framework updated 1.253841% of the model parameters. The results provide exploratory cross-
dataset evidence for stain-aware, parameter-efficient federation; they do not establish formal
privacy, patient-level independence, prospective deployment, or clinical validation.
Read the full abstract ▾
All authors Muhammad Muhtasim Shahriar, M. M. Golam Hafiz, Saad Aloteibi, Mohammad Ali Moni
Computer Vision
Seeing Semantic Shift: Difference-Aware Sentence-Level Temporal Segmentation of Sign Language Videos
Bowen Guo, Shiwei Gan, Yafeng Yin et al. (+4 more)
Recent advances in sign
language understanding have achieved impressive success on short, single-sentence
videos, yet their performance drops sharply when applied to long, continuous sign
language videos. To bridge this gap, we focus on a challenging and realistic setting: Visual-only Sentence-level Sign
Language Segmentation (Vis-SSLS), which aims to partition continuous sign
language videos into non-overlapping sentence-level segments without any caption assistance, serving as a crucial prerequisite for downstream recognition and translation tasks. However, sentence transitions in sign
language are often smooth and visually ambiguous, lacking explicit pauses or posture resets. As a result, static frame representations may fail to capture the subtle temporal changes that indicate sentence boundaries. To address this challenge, we propose SignShift, a difference-aware
segmentation framework that explicitly models frame-to-frame feature variation as semantic cues for sentence boundary detection. First, to model the feature variation, we design a Temporal Difference Module, which incorporates full-frame, facial, and hand cues, and employs inter-frame differencing to learn multi-scale temporal variations that capture both fine-grained local kinematics and global semantic transitions. Second, to mitigate over- and under-
segmentation issues, we design a Segment Count Prediction module, which predicts the number of sentences to guide boundary selection. Extensive experiments on
benchmark datasets demonstrate that SignShift substantially outperforms existing methods, validating its effectiveness.
Read the full abstract ▾
All authors Bowen Guo, Shiwei Gan, Yafeng Yin, Xiao Liu, Kuizhuang Liu, Zhiwei Jiang, Lei Xie
Computer Vision
Pocket-STVG: lightweight architecture for Spatio-Temporal Video Grounding
Alberto Presta, Michal Byra, Grzegorz Stefański et al. (+3 more)
Spatio-Temporal
Video Grounding (STVG) aims to localize the spatio-temporal tube in a
video corresponding to a natural
language query. While recent methods achieve strong performance in fully supervised, weakly supervised, and zero-shot settings, they typically rely on computationally expensive architectures, complex training pipelines, or
multimodal large
language models. We present Pocket-STVG (P-STVG), a lightweight cascade architecture that addresses STVG by combining efficient pre-trained components instead of large end-to-end models. P-STVG integrates a temporal-aware
video encoder based on MobileViCLIP, a spatial encoder-decoder derived from MDETR, and a shared aligned text encoder. Temporal localization is performed through either a lightweight 1D U-Net or a simple thresholding strategy, enabling the same framework to operate in both weakly supervised and zero-shot settings. Furthermore,
video representations are precomputed independently of the query, yielding an indexing-friendly pipeline for efficient inference and large-scale
video collections. Despite requiring fewer than 90M parameters, P-STVG performs on par with weakly supervised methods and improves on earlier zero-shot approaches at a fraction of their
memory and computational cost, establishing a favorable performance-efficiency trade-off for STVG.
Read the full abstract ▾
All authors Alberto Presta, Michal Byra, Grzegorz Stefański, Karol Szurkowski, Eryk Kołodziejczyk, Krzysztof Arendt
Computer Vision
Double-stream registration with pyramid fusion for HDR video with alternating exposures
Onofre Martorell, Ivan Pereira-Sánchez, Antoni Fuentes et al. (+1 more)
High dynamic range (HDR)
video reconstruction from al\-ter\-na\-ting-exposure sequences remains challenging, especially in regions with extreme luminance variation. We propose a novel HDR reconstruction framework based on dual-stream registration and accurate pyramid fusion. Given three consecutive frames, our method computes optical flow directly with the central frame, while introducing a complementary midpoint displacement strategy to handle cases with severe overexposition. A pyramid fusion stage then merges the resulting radiance and LDR images into a final HDR output. Experimental results demonstrate that our approach consistently outperforms state-of-the-art methods.
Read the full abstract ▾
All authors Onofre Martorell, Ivan Pereira-Sánchez, Antoni Fuentes, Antoni Buades
Computer Vision
Jiangning Wei, Yuan Yao, Miaomiao Cui et al. (+4 more)
Spatial
reasoning with metric constraints requires linking objects to
geometric measurements and preserving their numerical content during
language reasoning. We present DepthEvidence, a 4B model that uses its own dense metric predictions as object-grounded evidence for
language generation. A camera-conditioned decoder predicts full-resolution metric depth using multi-scale visual features and high-resolution RGB refinement. A dense-to-
language interface converts predicted depths and decoder features into object-aligned continuous geometry
tokens anchored to object identifiers.
Geometric supervision encourages metric information to remain recoverable before and after
language-context interaction, while instruction tuning supports object measurement and compositional
reasoning. We introduce a Depth-VQA
benchmark evaluating object-depth queries, relative comparisons, and decisions combining spatial and numerical constraints. Across nine
datasets, DepthEvidence achieves the highest average dense $\delta_1$ among evaluated methods, competitive with specialized
estimators. It also leads the evaluated methods in instance-level metric depth estimation and overall accuracy on both relative and metric
reasoning tracks, while broadly preserving general VQA performance and improving spatial understanding relative to the base model.
Read the full abstract ▾
All authors Jiangning Wei, Yuan Yao, Miaomiao Cui, Mingsheng Li, Humen Zhong, Shuai Bai, Zhibo Yang
Computer Vision
Band-Selection Stability and Semantic Segmentation Performance: A Study on Hyperspectral City
Jiarong Li, Imad Ali Shah, Enda Ward et al. (+3 more)
Resource constraints make high-dimensional hyperspectral imaging challenging in autonomous perception, motivating the use of band selection methods. However, the sensitivity of band-selection methods to sampled data and their relationship to semantic
segmentation models (SSMs) remain underexplored. This study evaluates six band selection methods on ten independently sampled, class-balanced region-of-interest (ROI) sets, yielding 60 top-25 band subsets from the Hyperspectral City V2 (128 bands: 450-950nm)
dataset. Top-$K$ bands ($K\in\{3,5, ... 13\}$) from the first three ROI sets are evaluated with three SSMs against the corresponding 128-band baseline. Experiments show that intra-method stability is method-dependent: Sim-LP shows the highest stability (pairwise Jaccard similarity) and, together with JMIM+CSNR, yields the best
segmentation results. Top-$K$ based SSMs remain competitive with baselines, with gains of up to 2.01 mIoU and 1.72 mF1 points, and 18-22x faster CPU inference for $K=9$. However, performance does not improve monotonically with $K$, and stability shows no consistent association with SSM performance. These findings suggest that intra-method stability is informative but an unreliable indicator of downstream
segmentation performance, highlighting the need to evaluate band-selection methods across repeated samples, subset sizes, and SSMs.
Read the full abstract ▾
All authors Jiarong Li, Imad Ali Shah, Enda Ward, Martin Glavin, Edward Jones, Brian Deegan
Computer Vision · Cross-listed from eess.IV
Quantum Diffusion Models for Medical Image Analysis
Francesco Aldo Venturelli, Stefano Martina, Marco Parigi et al. (+3 more)
Quantum Machine Learning is a novel field of research aimed at devising machine learning approaches exploiting principles of
quantum mechanics, such as superposition,
entanglement and interference. In this context, we present a scalable hybrid
Quantum Diffusion Model, and evaluate its use for medical image analysis. Specifically, our method is based on a Discrete-Time
Quantum Walk
algorithm, executed on a real
quantum device, to model the forward dynamics of the
diffusion model. For the backward step of the
diffusion model, we devise and evaluate a classical learning model, which is used to reversely denoise the data. In contrast with other existing attempts at applying
quantum machine learning for image analysis tasks, severely limited by the size of existing
quantum devices, our method allows to process real-world large size medical data. In particular, we present results on grayscale and RGB images, as well as 3D volumes of moderate sizes. We
benchmark our results by reproducing an alternative classical counterpart model, based on
diffusion models on discrete state spaces. By doing so, we compare the generation capabilities of both models in terms of three distinct state-of-the-art metrics in the field of image generation, showing the competitive, promising results of our approach.
Read the full abstract ▾
All authors Francesco Aldo Venturelli, Stefano Martina, Marco Parigi, Filippo Caruso, Alba Cervera-Lierta, Miguel A. González Ballester
Computer Vision
Where Compute Matters: Heterogeneous Attention for Efficient Video Diffusion
Olga Zatsarynna, Denis Korzhenkov, Juergen Gall et al. (+2 more)
Efficient
video generation requires reducing the quadratic cost of self-
attention over long spatio-temporal
token sequences. Existing efficient-
attention methods typically apply the same computation pattern to every
token, even though denoising difficulty varies substantially across
video regions and evolves throughout the generation process. We introduce HetA-DiT, a heterogeneous
attention mechanism that adaptively allocates computation according to
token difficulty. A lightweight uncertainty branch predicts a
token-wise estimate of denoising difficulty, which is used to route uncertain
tokens through dense global
attention while processing more reliable
tokens with efficient local
attention. The resulting routing is content- and timestep-adaptive, retains global context where it matters most, and provides a single parameter for controlling the quality-efficiency trade-off. HetA-DiT is compatible with few-step distribution-matching
distillation and introduces no additional
Transformer evaluation at inference time by reusing uncertainty estimates from the preceding denoising step. We evaluate the method on DMD-distilled Wan2.2-5B and Wan2.1-1.3B models. Across VBench, VBench-2.0, and human preference evaluation, HetA-DiT maintains competitive generation quality while routing only approximately 20% of
tokens through dense
attention.
Read the full abstract ▾
All authors Olga Zatsarynna, Denis Korzhenkov, Juergen Gall, Amir Habibian, Mohsen Ghafoorian
Computer Vision
Exploiting Spatial Structure for Transductive Few-Shot Classification of Whole-Slide Images
Tiffanie Godelaine, Manon Dausort, Karim El Khoury et al. (+3 more)
Automating the analysis of whole-slide images (WSIs), a key step in cancer diagnosis, has high clinical value, as it can reduce pathologist's workload while improving diagnosis accuracy. Recently,
vision-
language models have shown promising performance for patch-level classification without requiring any annotation, yet these zero-shot (ZS) predictions remain noisy on fine-grained tasks and must be further refined. A promising direction is to refine all predictions jointly, i.e., a transductive approach. However, most existing methods are not tailored to WSIs. We thus propose SlideTIM, an adaptation to WSIs of the recent transductive approach LC-TIM, which introduces a combined spatial–latent regularizer together with a prior on the patch class distribution. The former enforces spatially and semantically close patches to receive the same predictions, while the prior calibrates the predicted class proportions. Together, they address the complex spatial organization and the strong class imbalance of WSIs. Evaluated on four histology
datasets, SlideTIM consistently outperforms all TIM variants, improving the macro-F1 by +8.1pp over the best competing baseline at 1 shot. Compared to the ZS, it raises the macro-F1 by +19.4pp at 1 shot. The code will be made available after submission.
Read the full abstract ▾
All authors Tiffanie Godelaine, Manon Dausort, Karim El Khoury, Benoît Gérin, Benoît Macq, Christophe De Vleeschouwer
Computer Vision · Cross-listed from cs.MM
TempQ-Jail: Query-Constrained Candidate Ranking for Text-to-Video Jailbreak Attacks
Tianmeng Fang, Jiancheng Wang, Chen Wang et al. (+4 more)
Existing text-to-
video (T2V) jailbreak methods mainly seek more effective or stealthier attack candidates. In guarded T2V systems, however,
video generation and security evaluation are costly, so an attacker often cannot test a large candidate pool. We therefore formulate T2V jailbreak as a query-constrained candidate allocation and ranking problem and propose TempQ-Jail. The method combines heterogeneous attack mechanisms to expand candidate coverage, estimates each candidate's end-to-end attack value from security-gate passage, dangerous visual generation, preservation of the original intent, and temporal validity, and ranks candidates so that high-value attacks appear early in a limited query trajectory. We evaluate TempQ-Jail on CogVideoX-5B using 70 common viable intents derived from T2VSafetyBench and compare it with six representative T2V jailbreak methods under a unified protocol. TempQ-Jail achieves TP-ASR@5 and TP-ASR@10 of 48.9% and 65.4%, improving over the strongest baselines by 4.6 and 4.0 percentage points, respectively. It also obtains the highest AUC-TP (0.469) and the lowest AvgQ (6.3). Analyses of query trajectories, candidate allocation, failure attribution, and ablations show that TempQ-Jail more effectively identifies and prioritises candidates with complete attack potential under limited query budgets.
Read the full abstract ▾
All authors Tianmeng Fang, Jiancheng Wang, Chen Wang, Liming Wang, Wei Wang, Jiayang Liu, Xiaochun Cao
Computer Vision
Refining Cytology Predictions with Conditional Random Fields
Manon Dausort, Tiffanie Godelaine, Karim El Khoury et al. (+3 more)
Vision-
language models (VLMs) achieve strong zero-shot (ZS) classification on histology images but do not perform as well on cytology, whose stains and
cell morphology differ markedly compared to histology. Conditional random fields (CRFs) can refine noisy VLM predictions by propagating information across patches, but existing CRF frameworks were designed for histopathology and do not transfer to cytology
datasets, released as independent patch pools spanning multiple staining protocols. We introduce CytoCRF, which adapts the pairwise terms to cytology by targeting chromatin and cytology-specific staining, and further enrich the neighborhood of each potential term by combining multiple backbones. Across ten cytology
datasets, CytoCRF outperforms existing CRF frameworks at every annotation budget, reaching +13.6 percentage points over the best baseline and +33.7 over ZS with only 50 annotations. Combining information from multiple backbones brings further gains, showing that the neighborhood
topology matters more than the pairwise potential computed over it.
Read the full abstract ▾
All authors Manon Dausort, Tiffanie Godelaine, Karim El Khoury, Maxime Zanella, Christophe De Vleeschouwer, Benoît Macq
Computer Vision
TRACKGRAPH: Online Open-Vocabulary 3D Scene Graphs via Image-Space Tracking
Peder Borge Hellesylt, Albert Gassol Puigjaner, Kostas Alexis et al. (+1 more)
Open-vocabulary 3D maps enable
robots to reason about previously unknown environments using natural
language. However, existing systems typically segment every incoming image, associate detections with persistent 3D segments, and frequently perform costly
Vision-
Language (VL) inference. We present TRACKGRAPH, an online open-vocabulary system that maintains short-term 2D mask identity directly in the image stream before fusing segments into 3D. FastSAM masks and CLIP features are computed at sparse keyframes, while dense DINOv3 features are used to propagate masks at a high rate in between. The resulting tracked masks are fused into a class-agnostic 3D segment layer within a hierarchical scene graph, with 3D association handling tracking interruptions and long-term revisits. Compact multi-view CLIP
embeddings enable open-vocabulary
retrieval. Across Replica, ScanNet++, and HM3D, TRACKGRAPH achieves competitive open-vocabulary
segmentation and
retrieval against state-of-the-art mapping methods, including the highest synonym frequency on Replica (0.50). On the same NVIDIA A100, it is 1.7x faster and uses 3.3x less GPU
memory than ViT-H OVI-MAP. Real-world quadruped deployments demonstrate onboard scene graph construction and object search at 7.5Hz, while recorded drone data is used to test the method under aerial viewpoints.
Read the full abstract ▾
All authors Peder Borge Hellesylt, Albert Gassol Puigjaner, Kostas Alexis, Annette Stahl
Computer Vision · Cross-listed from cs.CR
Can Pixels Alone Reveal Image Origin? Minimax Limits and Learnable Interfaces for Passive Provenance
Kai Yao
Passive image provenance asks whether pixels alone can reveal where an image came from: a human, an aggregate AI class, or a particular generator. This becomes a robustness problem once a source image can be edited before the verifier sees it. We study the problem as source–target verification under
adversarial distribution shift. Our first result gives the exact best-case limit for any image-only verifier: the largest robust target-acceptance gap equals the minimum total-variation distance between the target distribution and the set of attacked source distributions. This quantity depends on the source, target, and edit class, not on the verifier architecture. Our second result explains why deployed public verifiers can fail before this statistical limit is reached. If the verifier can be emulated on the attack region to error $\varepsilon$, then a surrogate black-box attack reaches target acceptance within $2\varepsilon$ plus optimization error of the white-box optimum; score-revealing logistic and softmax heads over public features are identifiable, and approximate score access gives stable recovery bounds. A finite-state experiment checks the minimax identity where both sides are computable. On same-
prompt real/
diffusion benchmarks, the evaluated public CLIP verifiers fail under targeted pixel attacks, while a ResNet-18 victim exhibits partial fake-to-real transfer. Binary feedback with abstention reduces measured attack success, but positive empirical gap upper bounds do not establish robustness. These results motivate separate evaluation of the source–target statistical ceiling and the information released by a deployed verifier.
Read the full abstract ▾
Computer Vision
FLIP: Final Layer Inference-Time Probing for Vision-Language Models
Drandreb Earl O. Juanico and Rowel O. Atienza
We present FLIP, a final-layer inference-time probe for testing whether a logit-facing intervention site in an open-weight
vision-
language model (VLM) supports structured, task-linked computation rather than generic perturbation. Behavioral change under internal intervention is otherwise mechanistically ambiguous: it may reflect improved use of visual evidence, generic output instability, or outright degradation. FLIP applies elementwise flooring to the final normalized hidden state before logit computation, leaving parameters,
prompts, and decoding unchanged. On a controlled detection/counting probe, sweeping intervention strength reveals three regions: negligible change, a bounded interior regime in which detection recall at IoU 0.50 ($R_{50}$) improves while tolerant counting error ($\mathcal{E}_{\mathrm{count}}$) falls, and over-suppression. We formalize a four-criterion probe-and-sweep protocol for disciplining the interpretation of intervention effects: regime structure, grounding-proxy
alignment, feature-coherence dependence, and failure to reproduce the same positive regime on a performance-based negative control. The post-normalization state passed to the output head is the logit-facing instantiation of this test; under a non-targeted flooring sweep it satisfies the full protocol. Raw decoder-layer interventions, including the last-block output before final normalization, and the singleton-pair left/right control fail to reproduce the Final-site signature, while same-site operators and multiple VLMs replicate it. FLIP is therefore a validation step for intervention-based mechanistic interpretability, not a steering method. Repo at https://github.com/earl-juanico/flip-qwen3vl/
Read the full abstract ▾