Computer Vision
Risk-Aware Robust Learning: Reducing Clinical Risk under Label Noise in Medical Image Classification
Maycon R. S. Pereira and Filipe R. Cordeiro
Noisy labels are a pervasive challenge in medical image classification, where annotation errors arise from inter-observer variability and diagnostic ambiguity. Although several noise-robust learning methods have been proposed, their evaluation predominantly relies on accuracy-oriented metrics, overlooking the clinical implications of asymmetric error costs. In medical diagnosis, a false negative (missed disease) carries substantially higher consequences than a false positive (false alarm), as delayed treatment can directly impact patient outcomes. In this work, we investigate whether noise-robust training methods preserve clinical safety under label noise. We conduct a systematic risk-aware evaluation of the state-of-the-art noise-robust methods Coteaching, DivideMix, UNICON, and a GMM-based filtering approach on binarized DermaMNIST and PathMNIST
datasets under clean and label noise rates of 20%, and 40%. Beyond balanced accuracy, we adopt a cost-sensitive Global Risk formulation that explicitly penalizes false negatives. Our analysis reveals that the robustness of state-of-the-art methods does not guarantee clinical safety. Furthermore, we demonstrate that integrating cost-sensitive optimization into noise-robust training significantly reduces clinical risk, while mantaining model utility. These findings demonstrate that noise-robust learning must be evaluated through a clinical risk lens, and that combining robust training with cost-sensitive optimization can meaningfully reduce risk in noisy-label medical imaging scenarios.
Read the full abstract ▾
Computer Vision
Empirical Ablation and Ensemble Optimization of a Convolutional Neural Network for CIFAR-10 Classification
Naser Khatti Dizabadi
Convolutional neural networks (CNNs) remain a central approach in image classification, but their performance depends strongly on architectural and training choices. This paper presents an empirical ablation-based study of CNN optimization for the CIFAR-10
benchmark. The study evaluates 17 progressive modifications involving training duration, learning-rate scheduling, dropout configuration, pooling strategy, network depth, filter arrangement, and dense-layer design. The goal is to identify which changes improve generalization and which increase complexity without improving performance. The baseline model achieved 79.5% test accuracy. Extending training duration improved performance steadily, whereas several structural redesigns reduced accuracy despite greater architectural variation. Based on the strongest individual configurations, a weighted ensemble was constructed, achieving 86.38% accuracy in the reduced-data setting and 89.23% when trained using the full CIFAR-10
dataset. These results suggest that performance gains in CNN-based classification depend less on indiscriminate increases in depth or parameter count than on careful empirical selection of training and architectural modifications. The study therefore highlights the practical value of ablation-oriented optimization and ensemble learning for small-image classification.
Read the full abstract ▾
Computer Vision
Ashish Seth, Xinhao Mei, Changsheng Zhao et al. (+9 more)
Egocentric videos provide a distinctive setting in which sound serves as crucial cues to understand user activities and surroundings, particularly when visual information is unstable or occluded due to continuous camera movement. State-of-the-art large audio-visual
language models (AV-LLMs) can generate
multimodal descriptions. However, we show in this work that they are prone to audio
hallucinations, often inferring sounds from visual cues that are visible but not heard. We present a systematic and automatic evaluation framework for analyzing audio
hallucinations in
egocentric video through a targeted question-answering (Q/A) protocol. We curate a
dataset of 300
egocentric videos and design 1,000 sound-focused questions to probe model outputs. To characterize
hallucinations, we propose a grounded taxonomy that distinguishes between foreground action sounds from the user activities and background ambient sounds. Our evaluation shows that advanced AV-LLMs, such as Qwen2.5 Omni, exhibit high
hallucination rates, achieving only 27.3% and 39.5% accuracy on Q/As related to foreground and background sounds, respectively. With this work, we highlight the need to measure the reliability of
multimodal responses, emphasizing that robust evaluation of
hallucinations is essential to develop reliable AV-LLMs.
Read the full abstract ▾
All authors Ashish Seth, Xinhao Mei, Changsheng Zhao, Varun Nagaraja, Ernie Chang, Gregory P. Meyer, Gael Le Lan, Yunyang Xiong, Vikas Chandra, Yangyang Shi, Dinesh Manocha, Zhipeng Cai
Computer Vision
Latent Inter-Frame Pruning: A Training-Free Method Bridging Traditional Video Compression and Modern Diffusion Transformers for Efficient Generation
Dennis Menn and Chih-Hsien Chou
Video generation, while capable of generating realistic
videos, is computationally expensive and slow, prohibiting real-time applications. In this paper, we observe that
video latents encoded via an autoencoder under the Latent
Diffusion Model (LDM) framework contain redundancy along the temporal axis. Analogous to how traditional
video compression
algorithms avoid transmitting redundant frame data, we propose the Latent Inter-frame
Pruning framework to prune (skip the re-computation of) duplicated latent patches, thereby reducing computational burden and increasing throughput. However, direct
pruning results in visual artifacts due to the discrepancy between full-sequence training and pruned inference. To resolve these artifacts, we propose an
Attention Recovery mechanism to bridge the train-inference gap. With our proposed method, we increase
video editing throughput by 1.44$\times$, achieving 12.44 FPS on an NVIDIA RTX 6000 while maintaining
video quality. We hope our work inspires further research into integrating traditional
video compression methods with modern
video generation pipelines. This work is a preliminary work on Training-free Latent Inter-Frame
Pruning with
Attention Recovery.
Read the full abstract ▾
Computer Vision
Focus on What Matters: Two-Stage ROI-Aware Refinement for Anatomy-Preserving Fetal Ultrasound Reconstruction
Ines Abbes, Mahmood Alzubaidi, Mowafa Househ et al. (+3 more)
Measurement-critical ultrasound tasks often depend on a small anatomical region, making global reconstruction metrics an unreliable proxy for clinical fidelity. We propose an ROI-aware representation learning framework and instantiate it for first-trimester nuchal translucency (NT) screening under multi-hospital domain shift. A two-phase convolutional autoencoder (CAE) first learns a globally faithful 128-D latent code via MS-SSIM, then refines the NT ROI using intensity (L1) and normalized Sobel-edge constraints. To combine these heterogeneous objectives without manual tuning, we initialize loss weights via gradient-based calibration from per-term gradient magnitudes. Under strict hospital-wise evaluation with one hospital held out, ROI refinement improves both global and measurement-relevant quality: on the standard dev split it increases PSNR by +0.27 dB (val) and +0.29 dB (held-out test), reduces ROI MAE by 8.87% (val) and 6.43% (held-out test), and reduces ROI Edge-MAE by 11.10% on source hospitals and 4.90% on the unseen hospital. Beyond reconstruction, frozen-latent probes provide additional evidence of generalization: hospital provenance becomes less confidently predictable on the unseen site (0.556 to 0.541 max-softmax; 0.684 to 0.688
entropy) while OOD detection remains strong across site-held-out protocols (Mahalanobis AUROC up to 0.9956, with modest KNN gains in challenging splits). The same ROI-aware refinement principle is anatomy-agnostic and can be adopted for other fetal biometry targets (e.g., crown-rump length (CRL), nasal bone (NB)) and broader medical imaging settings where small ROIs dominate clinical decisions.
Read the full abstract ▾
All authors Ines Abbes, Mahmood Alzubaidi, Mowafa Househ, Khalid Alyafei, Marco Agus, Samir Brahim Belhaouari
Computer Vision
Mapping License Plate Recoverability Under Extreme Viewing Angles for Opportunistic Urban Sensing
Igor Adamenko, Orpaz Ben Aharon, Yehudit Aperstein et al. (+1 more)
Urban environments contain many imaging sensors built for specific purposes, including ATM, body-worn, CCTV, and dashboard cameras. Under the opportunistic sensing paradigm, these sensors can be repurposed for secondary inference tasks such as license plate recognition. Yet objects of interest in such im-agery are often noisy, low-resolution, and captured from extreme viewpoints. Recent advances in AI-based restoration can recover useful information even from severely degraded images. A central challenge is de-termining which distortion parameters allow reliable recovery and which lead to inference failure. This paper introduces recoverability maps, a task-agnostic method for quantifying this boundary. The method combines a dense synthetic sweep of degradation parameters with two summary measures: boundary ar-ea-under-curve, which estimates the recoverable fraction of the parameter space, and a reliability score, which captures the frequency and severity of failures within that region. We demonstrate the method on li-cense plate recognition from highly angled views under realistic camera artifacts. Several restoration archi-tectures are trained and evaluated, including U-Net, Restormer, Pix2Pix, and SR3
diffusion. The best model recovers about 93% of the parameter space. Similar results across models suggest that sensing geometry, ra-ther than architecture, sets the limit of recovery.
Read the full abstract ▾
All authors Igor Adamenko, Orpaz Ben Aharon, Yehudit Aperstein, Alexander Apartsin
Computer Vision
ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstruction
Zichun Guo, Yuling Shi, Wenhao Zeng et al. (+6 more)
Multimodal Large
Language Models (MLLMs) have achieved remarkable performance in Visually Rich Document Understanding (VRDU) tasks, but their capabilities are mainly evaluated on pristine, well-structured document images. We consider content restoration from shredded fragments, a challenging VRDU setting that requires integrating visual pattern recognition with semantic
reasoning under significant content discontinuities. To facilitate systematic evaluation of complex VRDU tasks, we introduce ShredBench, a
benchmark supported by an automated generation pipeline that renders fragmented documents directly from Markdown. The proposed pipeline ensures evaluation validity by allowing the flexible integration of latest or unseen textual sources to prevent training data contamination. ShredBench assesses four scenarios (English, Chinese, Code, Table) with three fragmentation granularities (8, 12, 16 pieces). Empirical evaluations on state-of-the-art MLLMs reveal a significant performance gap: The method is effective on intact documents; however, once the document is shredded, restoration becomes a significant challenge, with NED dropping sharply as fragmentation increases. Our findings highlight that current MLLMs lack the fine-grained cross-modal
reasoning required to bridge visual discontinuities, identifying a critical gap in robust VRDU research.
Read the full abstract ▾
All authors Zichun Guo, Yuling Shi, Wenhao Zeng, Chao Hu, Haotian Lin, Terry Yue Zhuo, Jiawei Chen, Xiaodong Gu, Wenping Ma
Computer Vision
Bringing a Personal Point of View: Evaluating Dynamic 3D Gaussian Splatting for Egocentric Scene Reconstruction
Jan Warchocki, Xi Wang, Jonas Kulhanek et al. (+1 more)
Egocentric video provides a unique view into human perception and interaction, with growing relevance for augmented reality, robotics, and assistive technologies. However, rapid camera motion and complex scene dynamics pose major challenges for 3D reconstruction from this perspective. While 3D Gaussian Splatting (3DGS) has become a state-of-the-art method for efficient, high-quality novel view synthesis, variants, that focus on reconstructing dynamic scenes from
monocular video are rarely evaluated on
egocentric video. It remains unclear whether existing models generalize to this setting or if
egocentric-specific solutions are needed. In this work, we evaluate dynamic
monocular 3DGS models on
egocentric and exocentric
video using paired ego-exo recordings from the EgoExo4D
dataset. We find that reconstruction quality is consistently lower in
egocentric views. Analysis reveals that the difference in reconstruction quality, measured in peak signal-to-noise ratio, stems from the reconstruction of static, not dynamic, content. Our findings underscore current limitations and motivate the development of
egocentric-specific approaches, while also highlighting the value of separately evaluating static and dynamic regions of a
video.
Read the full abstract ▾
All authors Jan Warchocki, Xi Wang, Jonas Kulhanek, Jan van Gemert
Computer Vision
VitaminP: cross-modal learning enables whole-cell segmentation from routine histology
Yasin Shokrollahi, Karina B. Pinao Gonzales, Elizve N. Barrientos Toro et al. (+5 more)
Accurate whole-
cell and
nuclear segmentation is essential for precision pathology and spatial omics, yet routine hematoxylin and eosin (H&E) staining provides limited cytoplasmic contrast, restricting analyses to
nuclei. Multiplex immunofluorescence (mIF) facilitates precise whole-
cell delineation but remains constrained by cost and accessibility. We introduce VitaminP, a cross-modal learning framework enabling whole
cell segmentation from H&E images. By learning from paired H&E-mIF data, VitaminP transfers molecular boundary information from mIF to overcome cytoplasmic contrast in H&E, establishing cross-modal supervision as a general strategy for recovering missing biological structure. We train VitaminP on 14 public
datasets covering 34 cancer types and over 7 million instances, integrating publicly available labels with extensive annotations generated in this study, forming one of the largest resources for
segmentation. VitaminP outperforms four state-of-the-art methods and generalizes to unseen
datasets, including an in-house
dataset spanning 24 rare cancer types. We further developed VitaminPScope, an open-source platform providing an interface for scalable inference and enabling broad adoption.
Read the full abstract ▾
All authors Yasin Shokrollahi, Karina B. Pinao Gonzales, Elizve N. Barrientos Toro, Paul Acosta, Patient Mosaic Team, Pingjun Chen, Yinyin Yuan, Xiaoxi Pan
Computer Vision · Cross-listed from cs.LG
Chih-Chung Hsu, Xin-Di Ma, Wo-Ting Liao et al. (+1 more)
Existing
attention accelerators often trade exact softmax semantics, depend on fused Tensor Core kernels, or incur sequential depth that limits FP32 throughput on long sequences. We present ELSA, an algorithmic reformulation of online softmax
attention that (i)~preserves exact softmax semantics in real arithmetic with a provable $\mathcal{O}(u\log n)$ FP32 relative error bound; (ii)~casts the online softmax update as a prefix scan over an associative monoid $(m,S,W)$, yielding $O(n)$ extra
memory and $O(\log n)$ parallel depth; and (iii)~is Tensor-Core independent, implemented in Triton and CUDA C++, and deployable as a drop-in replacement requiring no retraining or weight modification. Unlike FlashAttention-2/3, which rely on HMMA/GMMA Tensor Core instructions and provide no compatible FP32 path, ELSA operates identically on A100s and resource-constrained edge devices such as Jetson TX2 – making it the only hardware-agnostic exact-
attention kernel that reduces parallel depth to $O(\log n)$ at full precision. On A100 FP32
benchmarks (1K–16K
tokens), ELSA delivers $1.3$–$3.5\times$ speedup over
memory-efficient SDPA and $1.97$–$2.27\times$ on BERT; on Jetson TX2, ELSA achieves $1.5$–$1.6\times$ over Math (64–900
tokens), with $17.8$–$20.2\%$ throughput gains under LLaMA-13B offloading at $\ge$32K. In FP16, ELSA approaches hardware-fused baselines at long sequences while retaining full FP32 capability, offering a unified kernel for high-precision inference across platforms. Our code and implementation are available at https://github.com/ming053l/ELSA.
Read the full abstract ▾
All authors Chih-Chung Hsu, Xin-Di Ma, Wo-Ting Liao, Chia-Ming Lee
Computer Vision
MuSS: A Large-Scale Dataset and Cinematic Narrative Benchmark for Multi-Shot Subject-to-Video Generation
Haojie Zhang, Di Wu, Bingyan Liu et al. (+5 more)
While
video foundation models excel at single-shot generation, real-world cinematic storytelling inherently relies on complex multi-shot sequencing. Further progress is constrained by the absence of
datasets that address three core challenges: authentic narrative logic, spatiotemporal text-
video alignment conflicts, and the "copy-paste" dilemma prevalent in Subject-to-
Video (S2V) generation. To bridge this gap, we introduce MuSS, a large-scale, dual-track
dataset tailored for multi-shot
video and S2V generation. Sourced from over 3,000 movies, MuSS explicitly supports both complex montage transitions and subject-centric narratives. To construct this
dataset, we pioneer a progressive captioning pipeline that eliminates contextual conflicts by ensuring local shot-level accuracy before enforcing global narrative coherence. Crucially, we implement a cross-shot matching mechanism to fundamentally eradicate the S2V copy-paste shortcut. Alongside the
dataset, we propose the Cinematic Narrative
Benchmark, featuring a visual-logic-driven paradigm and a novel Anti-Copy-Paste Variance (ACP-Var) metric to rigorously assess continuous storytelling and 3D structural consistency. Extensive experiments demonstrate that while current baselines struggle with continuous narrative logic or degenerate into trivial 2D sticker generators, our MuSS-augmented model achieves state-of-the-art narrative effectiveness and cross-shot identity preservation.
Read the full abstract ▾
All authors Haojie Zhang, Di Wu, Bingyan Liu, Linjie Zhong, Yuancheng Wei, Xingsong Ye, Nanqing Liu, Yaling Liang
Computer Vision
MIRAGE: A Micro-Interaction Relational Architecture for Grounded Exploration in Multi-Figure Artworks
Jui-Cheng Chiu, Yu-Chao Wang, Shengyang Luo et al. (+4 more)
Appreciating multi-figure paintings requires understanding how characters relate through subtle cues like gaze
alignment, gesture, and spatial arrangement. We present MIRAGE, an evidence-centric framework designed to scaffold the exploration of these "micro-interactions" in multi-figure artworks. While such cues are essential for deep narrative appreciation, they are often distributed across complex scenes and difficult for viewers to systematically identify. Existing
vision-
language models (VLMs) frequently fail to provide reliable assistance, offering ungrounded interpretations that lack traceable visual evidence. MIRAGE addresses this by constructing a structured intermediate representation capturing identities, pose cues, and gaze hypotheses. However, the challenge extends beyond extracting these cues to coordinating them during interpretation. Without an explicit mechanism to organize and reconcile relational evidence, models often collapse multiple interaction hypotheses into a single unstable or weakly grounded narrative, even when low-level signals are available. This representation allows users to verify how high-level interpretations are anchored in low-level visual facts. By separating spatial grounding from narrative generation, MIRAGE enables users to inspect and reason about figure-to-figure relationships through a verifiable evidence layer. We evaluate MIRAGE against painting-only VLM baselines using a blind assessment protocol. Results show that MIRAGE significantly improves identity consistency, reduces relational
hallucinations, and increases the coverage of subtle interactions. These findings suggest that structured grounding can serve as a critical interaction control layer, providing the necessary scaffolding for a more reliable, transparent, and human-led understanding of complex visual narratives.
Read the full abstract ▾
All authors Jui-Cheng Chiu, Yu-Chao Wang, Shengyang Luo, Tongyan Wang, Qi Yang, Nabin Khanal, Yingjie Victor Chen
Computer Vision
ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents
Fanqing Meng, Lingxiao Du, Zijian Wu et al. (+46 more)
Language-model
agents are increasingly used as persistent coworkers that assist users across multiple working days. During such workflows, the surrounding environment may change independently of the
agent: new emails arrive, calendar entries shift, knowledge-base records are updated, and evidence appears across images, scanned PDFs, audio,
video, and spreadsheets. Existing
benchmarks do not adequately evaluate this setting because they typically run within a single static episode and remain largely text-centric. We introduce \bench{}, a
benchmark for coworker
agents built around multi-turn multi-day tasks, a stateful sandboxed service environment whose state evolves between turns, and rule-based verification. The current release contains 100 tasks across 13 professional scenarios, executed against five stateful sandboxed services (filesystem, email, calendar, knowledge base, spreadsheet) and scored by 1537 deterministic Python checkers over post-execution service state; no LLM-as-judge is invoked during scoring. We
benchmark seven frontier
agent systems. The strongest model reaches 75.8 weighted score, but the best strict Task Success is only 20.0%, indicating that partial progress is common while complete end-to-end workflow completion remains rare. Turn-level analysis shows that performance drops after the first exogenous environment update, highlighting adaptation to changing state as a key open challenge. We release the
benchmark, evaluation harness, and construction pipeline to support reproducible coworker-
agent evaluation.
Read the full abstract ▾
All authors Fanqing Meng, Lingxiao Du, Zijian Wu, Guanzheng Chen, Xiangyan Liu, Jiaqi Liao, Chonghe Jiang, Zhenglin Wan, Jiawei Gu, Pengfei Zhou, Rui Huang, Ziqi Zhao, Shengyuan Ding, Ailing Yu, Bo Peng, Bowei Xia, Hao Sun, Haotian Liang, Ji Xie, Jiajun Chen, Jiajun Song, Liu Yang, Ming Xu, Qionglin Qiu, Runhao Fu, Shengfang Zhai, Shijian Wang, Tengfei Ma, Tianyi Wu, Weiyang Jin, Yan Wang, Yang Dai, Yao Lai, Youwei Shu, Yue Liu, Yunzhuo Hao, Yuwei Niu, Jinkai Huang, Jiayuan Zhuo, Zhennan Shen, Linyu Wu, Hannah Yao, Charles Chen, Cihang Xie, Yuyin Zhou, Jiaheng Zhang, Zeyu Zheng, Mengkang Hu, Michael Qizhe Shieh
Computer Vision
From Noisy Historical Maps to Time-Series Oil Palm Mapping Without Annotation in Malaysia and Indonesia (2020-2024)
Nuttaset Kuapanich, Juepeng Zheng, Bohan Shi et al. (+6 more)
Accurate monitoring of oil palm plantations is critical for balancing economic development with environmental conservation in Southeast Asia. However, existing plantation maps often suffer from low spatial resolution and a lack of recent temporal coverage, impeding effective surveillance of rapid land-use changes. In this study, we propose a deep learning framework to generate 10-meter resolution oil palm plantation maps for Indonesia and Malaysia from 2020 to 2024, utilizing Sentinel-2 imagery without requiring new manual annotations. To address the resolution mismatch between coarse 100-meter historical labels and 10-meter imagery, we employ a U-Net architecture optimized with Determinant-based Mutual Information (DMI). This approach effectively mitigates the influence of label noise. We validated our method against 2,058 manually verified points, achieving overall accuracies of 70.64%, 63.53%, and 60.06% for the years 2020, 2022, and 2024, respectively. Our comprehensive analysis reveals that oil palm coverage in the region peaked in 2022 before experiencing a decline in 2024. Furthermore, land cover transition analysis highlights a concerning trajectory of plantation expansion into flooded vegetation areas, despite a general stabilization in rotations with other crop types. These high-resolution maps provide essential data for monitoring sustainability commitments and deforestation dynamics in the region, and the generated
datasets are made publicly available at https://doi.org/10.5281/zenodo.17768444.
Read the full abstract ▾
All authors Nuttaset Kuapanich, Juepeng Zheng, Bohan Shi, Jiaying Liu, Jiayin Jiang, Jiatao Huang, Shenghan Tan, Qingmei Li, Haohuan Fu
Computer Vision
Edit Where You Mean: Region-Aware Adapter Injection for Mask-Free Local Image Editing
Honghao Cai, Xiangyuan Wang, Yunhao Bai et al. (+8 more)
Large
diffusion transformers (DiTs) follow global editing instructions well but consistently leak local edits into unrelated regions, because joint-
attention architectures offer no explicit channel telling the network where to apply the edit. We introduce AdaptEdit, a co-trained, instruction- and region-aware adapter framework that retro-fits a frozen DiT into a precise local editor without modifying its backbone weights. A lightweight Block Adapter at every
transformer block injects a structured condition stream that factorizes what to edit (instruction semantics) from where to edit (spatial mask); a learned SpatialGate routes the adapter signal selectively into the edit region while keeping the rest of the image near-identical to the source; and a Region-Aware Loss focuses the training objective on the changing pixels. Because these components make the backbone's internal representation mask-aware end-to-end, a thin MaskPredictor head trained jointly with the editor can ground the edit region directly from the instruction and source image – eliminating any user-mask requirement at deployment. We evaluate on two complementary
benchmarks: MagicBrush (paired ground-truth targets) to measure pixel-level preservation and edit accuracy, and Emu-Edit Test (no ground-truth images, 9 diverse edit categories) to stress-test instruction following and generalization across edit types. On both, AdaptEdit achieves state-of-the-art results, simultaneously outperforming mask-free and oracle-mask baselines. A seven-variant ablation cleanly isolates the contribution of each component.
Read the full abstract ▾
All authors Honghao Cai, Xiangyuan Wang, Yunhao Bai, Haohua Chen, Tianze Zhou, Runqi Wang, Wei Zhu, Yibo Chen, Xu Tang, Yao Hu, Zhen Li
Computer Vision
DynProto: Dynamic Prototype Evolution for Out-of-Distribution Detection
Yanqi Wu, Xinhua Lu, Runhe Lai et al. (+4 more)
Recent studies show that using potential out-of-distribution (OOD) labels from large corpora as auxiliary information can improve OOD detection in
vision-
language models (VLMs). However, these methods often fail when real-world OOD samples fall outside the predefined OOD label set. To address this limitation, we propose DynProto, a novel approach that learns OOD prototypes dynamically during testing using only in-distribution (ID) information. DynProto is inspired by a key observation: OOD samples predicted as the same ID class tend to cluster in the feature space. With this insight, we leverage easy-to-detect OOD samples as “anchors” to find their harder-to-detect, similar counterparts. To this end, DynProto introduces two modules: Coarse OOD Pattern Capturing Module caches OOD patterns that are easily confused with each ID class during testing, and Fine-grained OOD Pattern Refinement Module subsequently clusters these patterns within each cache and aggregates them into representative OOD prototypes. By measuring similarity to ID and dynamic OOD prototypes, DynProto enables accurate OOD detection. DynProto significantly outperforms prior methods across multiple
benchmarks. On ImageNet OOD
benchmark, DynProto reduces FPR95 by 11.60% and improves AUROC by 4.70%. Moreover, the framework is architecture-agnostic and can be integrated into various backbones.
Read the full abstract ▾
All authors Yanqi Wu, Xinhua Lu, Runhe Lai, Qichao Chen, Jia-Xin Zhuang, Wei-Shi Zheng, Ruixuan Wang
Computer Vision
ESIA: An Energy-Based Spatiotemporal Interaction-Aware Framework for Pedestrian Intention Prediction
Yanping Wu, Meiting Dang, Lin Wu et al. (+3 more)
Recent advances in autonomous driving have motivated research on pedestrian intention prediction, which aims to infer future crossing decisions and actions by modeling temporal dynamics, social interactions, and environmental context. However, existing studies remain constrained by oversimplified multi-
agent interaction patterns, opaque
reasoning logic, and a lack of global consistency in behavioral predictions, which compromise both robustness and interpretability. In this work, we propose ESIA (Energy-based Spatiotemporal Interaction-Aware framework), a novel Conditional Random Field (CRF)-based paradigm. We cast the intention prediction task as a structured prediction problem over a unified graph-based representation, treating pedestrians and the environment as spatiotemporal nodes. To characterize their distinct roles, we assign unary potentials to nodes to capture individual intentions, and pairwise potentials to edges to encode social and environmental interactions. These potentials are integrated into a unified global energy function to ensure scene-level consistency across behavioral predictions. To further constrain inference without ground-truth supervision, we introduce structural consistency terms to penalize logical contradictions. This optimization is efficiently solved via a novel Unary-Seeded Simulated Annealing (U-SSA)
algorithm, which leverages high-confidence unary priors to rapidly converge to a high-quality solution. Extensive experiments on standard
benchmarks demonstrate that ESIA achieves state-of-the-art performance with improved interpretability over existing methods.
Read the full abstract ▾
All authors Yanping Wu, Meiting Dang, Lin Wu, Edmond S. L. Ho, Zhenghua Chen, Chongfeng Wei
Computer Vision
Zoom In, Reason Out: Efficient Far-field Anomaly Detection in Expressway Surveillance Videos via Focused VLM Reasoning Guided by Bayesian Inference
Xiaowei Mao, Bowen Sui, Weijie Zhang et al. (+7 more)
Expressway
video anomaly detection is important for traffic safety, but remains challenging across diverse scenes, particularly for far-field vehicles with subtle abnormal motion.
Vision-
Language Models (VLMs) provide strong semantic
reasoning capabilities, yet processing full frames can dilute evidence from distant targets and introduce substantial computational overhead. To address these challenges, we propose VIBES, an asynchronous framework that uses
Bayesian inference to guide focused VLM
reasoning. Specifically, an online kinematics-guided
Bayesian inference module continuously estimates a context-dependent normal-motion distribution from vehicle trajectories and updates its probabilistic boundaries. Deviations from these boundaries produce asynchronous triggers that localize candidate anomalies in time and space. Instead of processing continuous full-frame
video, the VLM reasons only over selected frames and localized visual regions associated with the triggers, reducing irrelevant visual content and unnecessary inference. Extensive experiments show that VIBES improves far-field anomaly detection and semantic interpretation while achieving real-time processing efficiency across diverse expressway conditions.
Read the full abstract ▾
All authors Xiaowei Mao, Bowen Sui, Weijie Zhang, Yawen Yang, Shengnan Guo, Shilong Zhao, Jiaqi Lin, Tingrui Wu, Youfang Lin, Huaiyu Wan
Computer Vision
Caries DETR: Tooth Structure-aware Prior and Lesion-aware Dynamic Loss Refinement for DETR Based Caries Detection
Xuefen Liu, Xinquan Yang, Mianjie Zheng et al. (+5 more)
As dental caries appear as subtle, low-contrast lesions in intraoral imaging, existing deep learning models face significant challenges in the early detection of caries. While recent
Transformer-based detectors have shown promising results in natural images, they often fail to capture the domain-specific anatomical priors crucial for dental caries detection. In this paper, we propose Caries-DETR, a specialized
Transformer framework for caries detection in intraoral images. A Tooth Structure-aware Query Initialization (TSQI) is designed, leveraging large-scale intraoral photograph pre-training and a structure perception branch (SPB) to integrate high-frequency structural priors, guiding the model to focus on anatomically significant lesion areas. Furthermore, we design a Lesion-aware Dynamic Loss Refinement (LDLR) to implement quality-driven hard mining through adaptive loss reweighting based on lesion size, anatomical relevance, and prediction quality, optimizing detection for subtle lesions. Extensive experiments on two public
datasets (i.e., AlphaDent and DentalAI) demonstrate that Caries-DETR achieves a state-of-the-art performance compared to existing methods and exhibits good generalization and robustness. Code and data at https://github.com/XuefenLiu-SZU/Caries-DETR}{https://github.com/XuefenLiu-SZU/Caries-DETR.
Read the full abstract ▾
All authors Xuefen Liu, Xinquan Yang, Mianjie Zheng, Kun Tang, Xuguang Li, Xiaoqi Guo, Linlin Shen, He Meng
Computer Vision
ZID-Net: Zero-Inference Diffusion Prior Decoupling Network for Single Image Dehazing
Xinheng Li, Minghao Chen, Mengqing Wu et al. (+2 more)
Single image dehazing is often constrained by a trade-off between restoration quality and computational efficiency. While efficient, CNNs struggle to learn robust priors for dense and non-homogeneous haze. Conversely,
diffusion models provide strong generative priors but suffer from severe inference latency and sampling instability. To address these limitations, we propose ZID-Net, a novel framework that explicitly decouples
diffusion supervision from feed-forward inference. For efficient inference, we design a frequency-spatial decoupled feed-forward backbone. Within this backbone, a Channel-Spatial
Laplacian Mask (CSLM) filters haze-amplified noise to extract purified structural details, while Lightweight Global Context Blocks (LGCBs) establish long-range spatial dependencies to capture global haze variations. A Dynamic Feature Arbitration Block (DFAB) then adaptively fuses these semantic and structural features for robust reconstruction. To provide the backbone with degradation-aware priors without increasing inference cost, we introduce a Zero-Inference Prior Propagation Head (ZI-PPH) during training. ZI-PPH leverages a conditional
diffusion process to predict residual noise and regularize the shared representation. By discarding the
diffusion branch at test time, ZID-Net integrates the representation-shaping benefits of
diffusion supervision into a pure feed-forward architecture. Extensive experiments on synthetic, real-world, and remote-sensing
benchmarks demonstrate robust restoration across diverse haze conditions, with particularly strong performance on StateHaze1k, where ZID-Net achieves an average PSNR gain of 3.06 dB while maintaining efficient deterministic inference at 19.35 ms.
Read the full abstract ▾
All authors Xinheng Li, Minghao Chen, Mengqing Wu, Yan Liu, Guanying Huo
Computer Vision
Weakly Supervised Multicenter Nancy Index Scoring in Ulcerative Colitis Using Foundation Models
Adam Kukučka, Ondřej Fabián, Vít Musil et al. (+1 more)
Histologic assessment of ulcerative colitis (UC) activity is an important endpoint in clinical trials and routine care, but manual grading with indices such as the Nancy histological index (NHI) is time-consuming and prone to observer variability. While computational pathology methods can automate scoring, many approaches depend on dense region-level annotations, which are costly to obtain, particularly in heterogeneous, multicenter cohorts. We propose a weakly supervised multiple instance learning (MIL) approach for whole-slide images that learns from case- and slide-level NHI labels, leveraging foundation models. Our method targets clinically relevant endpoints, including neutrophilic activity and derived Nancy-low/high groupings, enabling full five-grade NHI prediction. On a multicenter
dataset of H&E-stained colon biopsies from three hospitals (2019-2025), we evaluate multiple foundation model encoders and aggregation strategies. We find that foundation model choice and resolution substantially affect performance, with Virchow2 providing the most consistent gains, and that a simple ensembling rule improves five-grade NHI prediction compared to a hierarchical gating baseline. Overall, our results demonstrate that weakly supervised MIL with modern foundation-model representations can provide robust, interpretable UC histology activity assessment in realistic multicenter settings.
Read the full abstract ▾
All authors Adam Kukučka, Ondřej Fabián, Vít Musil, Tomáš Brázdil
Computer Vision
A Pose-only Geometric Constraint for Multi-Camera Pose Adjustment
Shunkun Liang, Banglei Guan, Bin Li et al. (+2 more)
Multi-camera systems offer rich observation capabilities for visual
navigation and 3D scene reconstruction; however, the resulting feature redundancy often compromises computational efficiency. This challenge is particularly pronounced during bundle adjustment, where the non-linear optimization of both system poses and scene points incurs substantial computational overhead. To address this challenge, this paper introduces a pose-only
geometric constraint for multi-camera systems and proposes a corresponding pose adjustment
algorithm. Specifically, we use generalized camera model to establish a unified representation of the multi-camera system. Building upon this model, we formulate the multi-camera pose-only constraint, which implicitly represents a 3D scene point using two base observations and their associated poses, thereby achieving a pose-only representation of the projection geometry. Subsequently, we introduce a multi-camera pose adjustment
algorithm that eliminates 3D points from the parameter space, thereby achieving efficient and focused pose optimization. Experimental results on both synthetic and real-world
datasets demonstrate that the proposed
algorithm outperforms baseline bundle adjustment methods in computational efficiency, while maintaining or even improving pose estimation accuracy.
Read the full abstract ▾
All authors Shunkun Liang, Banglei Guan, Bin Li, Qifeng Yu, Yang Shang
Computer Vision · Cross-listed from cs.CL
Agri-CPJ: A Training-Free Explainable Framework for Agricultural Pest Diagnosis Using Caption-Prompt-Judge and LLM-as-a-Judge
Wentao Zhang, Qi Zhang, Mingkun Xu et al. (+6 more)
Crop disease diagnosis from field photographs faces two recurring problems: models that score well on
benchmarks frequently hallucinate species names, and when predictions are correct, the
reasoning behind them is typically inaccessible to the practitioner. This paper describes Agri-CPJ (Caption-
Prompt-Judge), a training-free few-shot framework in which a large
vision-
language model first generates a structured morphological caption, iteratively refined through multi-dimensional quality gating, before any diagnostic question is answered. Two candidate responses are then generated from complementary viewpoints, and an LLM judge selects the stronger one based on domain-specific criteria. Caption refinement is the component with the largest individual impact: ablations confirm that skipping it consistently degrades downstream accuracy across both models tested. On CDDMBench, pairing GPT-5-Nano with GPT-5-mini-generated captions yields +22.7 pp in disease classification and +19.5 points in QA score over no-caption baselines. Evaluated without modification on AgMMU-MCQs, GPT-5-Nano reached 77.84% and Qwen-VL-Chat reached 64.54%, placing them at or above most open-source models of comparable scale despite the format shift from open-ended to multiple-choice. The structured caption and judge rationale together constitute a readable audit trail: a practitioner who disagrees with a diagnosis can identify the specific caption observation that was incorrect. Code and data are publicly available https://github.com/CPJ-Agricultural/CPJ-Agricultural-Diagnosis
Read the full abstract ▾
All authors Wentao Zhang, Qi Zhang, Mingkun Xu, Mu You, Henghua Shen, Zhongzhi He, Keyan Jin, Derek F. Wong, Tao Fang
Computer Vision · Cross-listed from cs.GR
Personalizing Causal Audio-Driven Facial Motion via Dynamic Multi-modal Retrieval
Xuangeng Chu, Yu Han, Wei Mao et al. (+1 more)
Audio-driven facial animation is essential for immersive digital interaction, yet existing frameworks struggle to reconcile real-time streaming with high-fidelity personalization. Current methods either rely on latency-inducing audio look-ahead, or ask users to record scripted calibration sequences to pre-encode static identity
embeddings that fail to capture dynamic idiosyncrasies. We present an end-to-end framework for personalized audio-driven facial motion generation, supporting causal, zero-lookahead streaming. We introduce two key innovations: (1) a causal multi-resolution motion tokenizer that captures both global temporal context and high-frequency articulatory details, and (2) a multi-modal style retriever that extracts stylistic priors from unstructured reference libraries by jointly querying ongoing audio and motion. Unlike prior
retrieval mechanisms restricted to curated, fixed-size, or audio-only style banks, our design accepts arbitrary footage of the target identity, enabling personalization from a handful of casually recorded clips. By integrating these components, our method outperforms state-of-the-art approaches in lip-sync accuracy, identity consistency, and perceived realism, while preserving the streaming constraints of real-time telepresence. Code is available at https://github.com/xg-chu/Fallingwater.
Read the full abstract ▾
All authors Xuangeng Chu, Yu Han, Wei Mao, Shih-En Wei
Computer Vision
Do Protective Perturbations Really Protect Portrait Privacy under Real-world Image Transformations?
Ruiqing Sun, Xingshan Yao, Zhijing Wu et al. (+6 more)
Proactive defense methods protect portrait images from unauthorized editing or talking face generation (TFG) by introducing pixel-level protective perturbations, and have attracted increasing
attention for
privacy protection. In real-world use, images inevitably undergo sequences of benign operations during display and dissemination, such as resizing and color compression, which directly alter pixel values. Existing studies and robustness defenses mainly examine individual transformations in isolation, while sequentially composed transformations remain underexplored. To address this gap, we systematically evaluate whether representative proactive defenses against unauthorized image editing using GANs and
diffusion models, as well as talking face generation, remain effective under sequential image transformations. The evaluated methods span general-purpose and portrait-specific defenses and are assessed qualitatively and quantitatively. Experimental results show that pixel-level perturbation-based defenses struggle to withstand sequences of common image transformations, posing a risk of failure in real-world applications. To further demonstrate that this vulnerability can be exploited at low cost, we introduce Transformation-Induced Purification via Region-wise Super-Resolution (TIP-RSR), a simple training-free framework combining sequential image transformations with off-the-shelf restoration models. TIP-RSR efficiently purifies protective perturbations while preserving image fidelity and requiring substantially less computation than
diffusion-based purification methods. These findings expose a practical vulnerability in current proactive portrait defenses and highlight the need to account for sequential real-world transformations when designing future protection mechanisms. Our code is publicly available at https://github.com/Richen7418/TIP-RSR.
Read the full abstract ▾
All authors Ruiqing Sun, Xingshan Yao, Zhijing Wu, Tian Lan, Chenhao Cui, Huiyang Zhao, Jialing Shi, Chen Yang, Xianling Mao
Computer Vision
Reading in the Dark: Low-light Scene Text Recognition
Xuanshuo Fu, Lei Kang, Ernest Valveny et al. (+2 more)
Accurate text recognition in low-light environments is essential for intelligent systems in applications ranging from autonomous vehicles to smart surveillance. However, challenges such as poor illumination and noise interference remain underexplored. To address this gap, we introduce LSTR, a large-scale Low-light Scene Text Recognition
dataset comprising 11,273 low-light images generated from well-lit
datasets (ICDAR2015, IIIT5K, and WordArt), along with ESTR, which includes 60 real nighttime street-scene images in English and Spanish for exclusive evaluation. We explore two solution strategies: (1) employing Optical Character Recognition (OCR) models with fine-tuning and LoRA-based fine-tuning and (2) a joint training strategy that integrates a low-light image enhancement (LLIE) module with an OCR model. In particular, we propose a novel re-render LLIE (RLLIE) module, which demonstrates improved performance on real-world data. Through extensive experimentation, we analyze various training strategies and address a key research question: How bright is bright enough for effective scene text recognition? Our results indicate that standalone LLIE or OCR models perform inadequately under low-light conditions, highlighting the advantages of specialized, jointly trained text-centric approaches. Additionally, we provide a comprehensive
benchmark to support future research in robust low-light scene text recognition. https://huggingface.co/
datasets/lumimusta/Low-light_Scene_Text_
Dataset.
Read the full abstract ▾
All authors Xuanshuo Fu, Lei Kang, Ernest Valveny, Dimosthenis Karatzas, Javier Vazquez-Corral
Computer Vision
Learning to Decipher from Pixels: A Case Study of Copiale
Lei Kang, Giuseppe De Gregorio, Raphaela Heil et al. (+2 more)
Historical encrypted manuscripts require both paleographic interpretation of cipher symbols and cryptanalytic recovery of plaintext. Most existing computational workflows rely on a transcription-first paradigm, in which handwritten symbols are transcribed prior to decipherment. This intermediate step is labor-intensive, error-prone, and not always aligned with the goal of direct plaintext recovery. We propose an end-to-end, transcription-free approach that directly maps handwritten cipher images to plaintext. Using the Copiale cipher as a case study, we introduce the first text-line-level
dataset pairing cipher images with German plaintext. We show that pretraining on generic handwriting data followed by cipher-specific fine-tuning substantially improves decipherment accuracy. Our results demonstrate that transcription-free image-to-plaintext decipherment is both feasible and effective for historical substitution ciphers, offering a simplified and scalable alternative to traditional pipelines. https://github.com/leitro/Decipher-from-Pixels-Copiale
Read the full abstract ▾
All authors Lei Kang, Giuseppe De Gregorio, Raphaela Heil, Alicia Fornés, Beáta Megyesi
Computer Vision · Cross-listed from eess.IV
GS-DOT: Gaussian splatting-based image reconstruction for diffuse optical tomography
Jingjing Jiang
This work presents GS-DOT, a novel image reconstruction framework based on Gaussian Splatting (GS) for diffuse optical
tomography (DOT). Inspired by GS for rendering applications, absorption coefficients are represented as a sparse sum of anisotropic Gaussian primitives optimized to fit measured time-resolved point-spread functions through analytic gradients and Adam optimization. This is the first adaptation of GS
algorithms in the
photon diffusion regime, where the ray transport function is replaced by the
diffusion functions to enable accurate modeling of light transport in highly scattering media. Validation on synthetic tissue models demonstrate high accuracy in localization and quantification of reconstructed absorption maps for both clean and noisy signals. GS-DOT has demonstrated high robustness to noise and showed a huge reduction in
memory demand.
Read the full abstract ▾
Computer Vision
Preserve, Then Resolve: Many-to-Many Association and Robust Estimation with General-Purpose Visual Features
Haodong Jiang, Mingzhe Li and Junfeng Wu
The semantic transferability of general-purpose visual features does not guarantee
geometric consistency across images. Using frozen DINOv3 features, we show that geometrically correct correspondences often fall below rank one in cosine similarity yet remain within a small top-$K$ candidate set. This motivates a preserve-then-resolve design: we retain multiple candidates in a many-to-many (m-to-m) association graph and defer their disambiguation to robust estimation. We study m-to-m robust estimation from a probabilistic perspective. We interpret the existing Matching Cardinality Maximization (MCM) mechanism as a dominant-cardinality approximation to likelihood maximization and propose a faster, real-valued mechanism called Harmonic Consensus Maximization (HCM). A two-stage LO-RANSAC uses HCM for candidate sourcing and MCM for graph-aware selection. We evaluate end-task gains in relative-pose estimation, where our pipeline consistently improves rank-one baselines for DINOv2, DINOv3, V-JEPA 2.1, and SigLIP 2. Code is available at https://github.com/LIAS-CUHKSZ/preserve_then_resolve.
Read the full abstract ▾
Computer Vision
HAC: Parameter-Efficient Hyperbolic Adaptation of CLIP for Zero-Shot VQA
Francesco Dibitonto, Cigdem Beyan and Vittorio Murino
Recent advances in representation learning have shown that
hyperbolic geometry can offer a more expressive alternative to the Euclidean
embeddings used in CLIP models, capturing hierarchical structures and leading to better-organized representations. However, current
hyperbolic CLIP variants are trained entirely from scratch, which is computationally expensive and resource-intensive. In this work, we propose HAC (
Hyperbolic Adaptation of CLIP), a parameter-efficient framework that enables pretrained CLIP models to transition into
hyperbolic space via lightweight fine-tuning. We apply HAC to Visual Question Answering (VQA), where models must interpret visual elements and align them with textual queries. Notably, HAC's training is performed on a
dataset with no overlap with any VQA
benchmark, resulting in a strict zero-shot evaluation paradigm that underscores HAC's task-agnostic adaptability. We evaluate HAC across a diverse suite of VQA
benchmarks spanning General,
Reasoning, and OCR categories. Both HAC-S (small) and HAC-B (medium) consistently surpass Euclidean baselines and prior
hyperbolic approaches, with HAC-B delivering up to a +1.9 point average improvement over CLIP-B on
reasoning-intensive tasks. Our code is available at https://github.com/fdibiton/HAC
Read the full abstract ▾