Computer Vision
LeakageBench: Document-Level Leakage Risk for Redacting Personally Identifiable Information in Document Images
Vishnu Prasad Vijaya Kumar, Santhosh Venkatesh and Ivan P. Yamshchikov
Real-world personally identifiable information (PII) redaction often operates on document images—scans, screenshots, and PDF renderings—where OCR errors, layout structure, and visual noise determine whether sensitive information is actually removed. Existing PII
benchmarks are mostly text-centric and do not measure document-level redaction risk: a page remains unsafe if even one identifier is missed. We introduce LeakageBench, a challenge set of 500 document images with 11,954 GDPR-aligned PII annotations spanning direct identifiers, linkage keys, and contextual re-identification surfaces. We evaluate generic OCR pipelines, commercial and task-adapted OCR-dependent detectors, and OCR-free
vision-
language models using entity-level F1, group-wise leakage, and document-level leakage metrics. Code Interpreter raises GPT-5.5 localization F1 from 0.090 to 0.249, but critical page-level leakage remains 0.968. These results show that stronger detection and tool assistance improve localization without making most pages safe for release. LeakageBench provides a diagnostic
benchmark for high-recall, spatially grounded PII redaction in document images.
Read the full abstract ▾
Computer Vision
TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval
Uicheol Jung, Juyoung Hong, Hojung Kwon et al. (+1 more)
Text-
Video Retrieval (TVR) retrieves
videos that match a natural-
language query, but extending image-text models such as CLIP to
videos is fundamentally limited by the lack of temporal modeling.
Videos exhibit frame-wise heterogeneity in appearance and motion, and compressing all frames into a single representation often obscures temporal structure and semantic transitions. To address this, we propose Temporal-Aware Mixture-of-Experts for Text-
Video Retrieval (TAME), a CLIP-based framework that jointly models frame-level structure and temporal relations. First, we integrate sparse Mixture-of-Experts (MoE) layers into both CLIP encoders and apply frame-consistent routing on the
vision branch so that experts specialize according to frame-level visual patterns while preserving the original
vision-
language alignment. Second, we introduce Frame-Temporal (FT)
tokens that aggregate global cross-frame information and feed it back to each frame, enabling the visual encoder to capture long-range temporal dependencies without harming local details. Third, we design a Cross-Temporal Interaction and Aggregation (CTIA) module that refines frame-wise sentence-
video similarities through staged temporal filtering and fusion. Experiments on standard TVR
benchmarks show that TAME consistently improves over CLIP-based baselines. On MSR-VTT, it improves R@1 by 4.0 over CLIP4Clip, and also achieves consistent gains on DiDeMo, MSVD, LSMDC, and ActivityNet. The code is available at https://github.com/sejong-rcv/TAME.
Read the full abstract ▾
All authors Uicheol Jung, Juyoung Hong, Hojung Kwon, Yukyung Choi
Computer Vision
Lightweight Adaptation of General-Purpose VLMs for Multispectral and SAR Image Understanding
Shanji Liu, Kelu Yao, Junxiao Xue et al. (+5 more)
General-purpose
vision-
language models (VLMs) now support strong visual recognition, instruction following, and generation. However, most pretrained visual encoders are built around three-channel natural images and do not directly accommodate observations such as native multispectral measurements or synthetic aperture radar (SAR). Adapting VLMs to these sensors typically requires dedicated encoders and domain pretraining, slowing the reuse of stronger general-purpose checkpoints. We show that the multi-image interface of general-purpose VLMs offers a lightweight alternative. Our protocol renders each observation as five optical views and one SAR view, names them in the
prompt, and adapts the
language network and selected visual
transformer blocks with LoRA. This exposes band composites, spectral indices, and radar backscatter through an existing visual interface. For land-cover recognition, structured supervision couples predicted classes with sensor evidence. We further construct preference pairs in which a true label is omitted while its supporting evidence is retained, encouraging complete predictions that remain consistent with the observations. On a balanced six-class land-cover
benchmark derived from BigEarthNet-v2, the adapted Qwen3-VL reaches 0.8275 micro F1. The same input and adaptation protocol improves all four tested VLM architectures and transfers to Sen1Floods11 flood verification and BigEarthNet.txt captioning. Image removal and mismatch controls show that the adapted models use the supplied sensor observations. Together, these results demonstrate that VLMs can be repurposed for multispectral and SAR tasks through rendered inputs and compact LoRA adaptation, without training a new foundation model.
Read the full abstract ▾
All authors Shanji Liu, Kelu Yao, Junxiao Xue, Chenghui Lv, Xiangyang Miao, Yekai Huang, Yaying Chen, Chao Li
Computer Vision
CC-4DGS: Computational Deformation and Point-Cloud Compression for Storage-Efficient Dynamic Gaussian Splatting
Kyungdae Park and Chae Eun Rhee
Dynamic four-dimensional (4D) Gaussian Splatting has emerged as a powerful explicit representation for high-quality view synthesis, yet existing methods still require tens to hundreds of megabytes per scene due to their heavy reliance on large multi-resolution hash tables and high-dimensional Gaussian attributes. This paper presents CC-4DGS, a storage-efficient and scalable framework that rethinks both deformation modeling and canonical attribute storage. First, we introduce a computational deformation field (CDF) that replaces large multi-resolution learnable hash tables with deterministic dense hash encoding and compact neural decoders, enabling on-the-fly synthesis of deformation features while reducing deformation storage to only 1–3 MB per scene. Second, we propose a compression of canonical point-cloud attributes (CCA) pipeline that compresses high-dimensional spherical harmonic appearance terms and auxiliary Gaussian attributes via conditional autoencoding, selective quantization, and residual codebooks, achieving 3–5$\times$ point-cloud reduction with negligible quality loss. Together, these components yield a unified representation that preserves real-time rendering performance while reducing total storage to 20–30 MB. Extensive experiments across the N3DV and Technicolor Light Field
datasets demonstrate that CC-4DGS achieves reconstruction accuracy comparable to state-of-the-art methods such as Swift4D, while offering significantly improved storage efficiency and favorable runtime-
memory trade-offs.
Read the full abstract ▾
Computer Vision
Progressive Pseudo-Label Optimization for Point-Supervised Change Detection
Hailong Ning, Hao Wang, Yimeng Wang et al. (+3 more)
Point-supervised change detection (PS-CD) aims to identify pixel-level changes between bi-temporal images using only sparsely annotated points. Although point annotations substantially reduce labeling costs, their limited spatial coverage often results in incomplete and noisy pseudo-labels. To address this issue, we propose a two-stage framework that introduces SAM2 priors into PS-CD and progressively adapts them to the target task. In Stage I, SAM2 generates object-aware candidate masks from point annotations on the bi-temporal images, and a bi-temporal mask selection strategy is designed to convert generic
segmentation responses into more reliable change pseudo-labels. Subsequently, a lightweight CNN refinement module with an uncertainty-aware loss is employed to improve boundary quality and local structural consistency. In Stage II, we construct a teacher-student self-training framework in which the teacher is updated by exponential moving average and periodically refreshes the pseudo-labels. This design establishes a closed-loop optimization process that alternates between pseudo-label refinement and model re-optimization. Experiments on three
benchmark datasets, including WHU-CD, LEVIR-CD, and SYSU-CD, demonstrate that the proposed method outperforms previous weakly supervised approaches on most
benchmarks and remains competitive with several fully supervised methods.
Read the full abstract ▾
All authors Hailong Ning, Hao Wang, Yimeng Wang, Tao Lei, Renwei Dian, Asoke K. Nandi
Computer Vision
World-Coherent Decoding: Self-Verifying Test-Time Planning for World Action Models
Chuhan Zhang, Seiji Ito, Kenta Hoshino et al. (+2 more)
World Action Models (WAMs) aim to control
robots by stochastically generating visual futures and then decoding actions, but empirical observations indicate that the results can strongly depend on which future is selected. We propose World-Coherent-Decoding (WCD), a self-verifying test-time planning framework that treats WAM rollouts as falsifiable future–action hypotheses. At each decision step, WCD samples multiple candidates from a frozen WAM and ranks them using internal generative signals: flow-based
video surprisal for visual plausibility and action path effort for action-generation stability. After execution, the realized observation audits the selected imagination, yielding an imagination–reality mismatch that trains a lightweight online predictor for future candidate selection. Thus, WCD converts delayed self-verification into pre-execution reliability estimation without updating the backbone model. On RoboTwin 2.0, WCD improves Hard success under limited randomized-scene supervision from $55.80\%$ to $60.90\%$, with a $+16.43$ gains on Horizon-3 tasks, and shows qualitative robustness on real Franka visual-shift tests. These results highlight a simple principle: test-time scaling for WAMs depends less on sampling more futures than on selecting reliable ones.
Read the full abstract ▾
All authors Chuhan Zhang, Seiji Ito, Kenta Hoshino, Satoshi Ikehata, Ikuro Sato
Computer Vision
Synergistic Information Disentanglement for Omni-modal Slide Representation Learning in Computational Pathology
Mingxin Liu, Chengfei Cai, Anwen Lu et al. (+5 more)
In computational pathology (CPath), developing omni-modal self-supervised learning (SSL) models that integrate histology, genomics, and clinical reports enables transferable representation learning for whole slide images (WSIs). Existing approaches implicitly force heterogeneous modalities into a uniform latent space by contrastive
alignment, causing modality collapse where unique, synergistic diagnostic signals (termed as $\mathrm{\Phi}$) are discarded in favor of trivial redundancy. We hypothesize that the strongest task-agnostic SSL training signal stems from distilling the synergistic interactions over merely aligning shared redundancy. To this end, we introduce \textsc{$\mathrm{\Phi}$-Omni}, a synergistic information disentanglement framework grounded in Partial Information Decomposition (PID) theory for slide representation learning. Unlike standard contrastive approaches, \textsc{$\mathrm{\Phi}$-Omni} employs a Synergistic Information Bottleneck (SIB) regulated by the proposed $\mathrm{\Phi}\text{ID}$ objective, which explicitly suppresses marginal redundancy while maximizing irreducible synergy, thereby distilling high-order cross-modal interactions. Following pretraining on breast ($n$=1031) and lung ($n$=919) cohorts, \textsc{$\mathrm{\Phi}$-Omni} demonstrates superior few-shot performance across five independent external
datasets spanning eight tasks compared to supervised and SSL baselines. Source code is available here.
Read the full abstract ▾
All authors Mingxin Liu, Chengfei Cai, Anwen Lu, Pengbo Xu, Jun Li, Jinze Li, Depin Chen, Jun Xu
Computer Vision
Disease Burden over Skin Tone: Decomposing the Dermatology-AI Generalization Gap
Nirajan Kunwor, Sanjaya Poudel, Quoc-Huy Trinh et al. (+2 more)
Dermatology artificial intelligence (AI) models are predominantly trained on light-skinned, cancer-focused image collections, yet they are increasingly proposed for deployment in resource-constrained settings where patients differ from training populations along two confounded axes: skin tone and disease distribution. We investigate whether poor generalization is primarily caused by skin-tone underrepresentation or disease-distribution shift. We evaluate a cancer-trained baseline (ResNet-50 fine-tuned on HAM10000 and ISIC 2019), two dermatology foundation models (DermLIP and MONET), and a general-purpose
vision model (DINOv3) as frozen feature extractors. Models are evaluated on a tone-stratified disease-matched
dataset (Diverse Dermatology Images, DDI) and a disease-shifted tone-diverse
dataset (Skin Condition Image Network, SCIN). Our results show that disease-distribution shift contributes more than skin tone in the evaluated settings. The cancer baseline decreases from 0.62 to 0.21 balanced accuracy when transferred to unfamiliar clinical conditions, while the within-disease skin-tone gap is smaller (0.10-0.18) and inconsistent. Label-free representation analysis shows that this failure reflects a representational limitation rather than only missing output labels: cancer-specialized features poorly cluster unfamiliar conditions (kNN purity lift +0.06 over chance), whereas dermatology-pretrained features retain stronger transferable structure (+0.23). Finally, we show that representation quality predicts recoverable performance under lightweight adaptation. Starting from dermatology foundation models, approximately ten labeled examples per clinical category recover most attainable performance. We release the evaluation protocol and code to support reproducible
auditing of dermatology AI generalization.
Read the full abstract ▾
All authors Nirajan Kunwor, Sanjaya Poudel, Quoc-Huy Trinh, Jahidul Arafat, Sunil Kumar Gaire
Computer Vision · Cross-listed from cs.LG
A Unified Rate-Distortion Perspective on Vector, Product, and Scalar Quantization
Xianghong Fang, Wenlong Mou, Yuan Yuan et al. (+2 more)
Discrete visual tokenization, predominantly driven by vector, scalar, and product quantization, lacks a unified conceptual framework for understanding quantization tradeoffs. In this paper, we propose a unified rate–distortion perspective on modern discrete visual tokenization. By viewing quantization as lossy compression, we characterize the nominal fixed-length coding rate through
token count and codebook size, and quantization error as the distortion. Within this framework, we resolve three central questions. First, we theoretically and empirically show that minimizing distortion, rather than maximizing codebook utilization, is the primary intrinsic objective for reconstruction fidelity, with a direct connection to the STE-induced gradient discrepancy. Second, we establish two critical
fairness conditions for intrinsic quantization comparison: controlling latent feature statistics and enforcing identical coding rates. Third, under these conditions, we recover the VQ–PQ–SQ distortion hierarchy in modern visual tokenization and show empirically that modern VQ methods achieve the lowest distortion. This work provides a foundational rate–distortion reframing of modern discrete visual tokenization, resolves ambiguities in quantizer evaluation, and provides a controlled framework for isolating intrinsic quantization effectiveness under fixed-rate constraints.
Read the full abstract ▾
All authors Xianghong Fang, Wenlong Mou, Yuan Yuan, Dehan Kong, Tim G. J. Rudner
Computer Vision · Cross-listed from cs.LG
Federated LoRA Adaptation of BiomedCLIP Across Four International Chest X-Ray Cohorts
Sanjaya Poudel, Nirajan Kunwor, Manish Dhakal et al. (+2 more)
Federated learning (FL) lets institutions train a shared model without exchanging data, and Low-Rank Adaptation (LoRA) makes this practical at scale by communicating only compact low-rank updates. Biomedical imaging is a compelling setting for this combination: patient data are archived behind
privacy regulations, and institutions differ widely in scanners, protocols, and compute. Such heterogeneity raises the question of how
federated LoRA updates should be aggregated, increasingly pressing as
multimodal vision-
language models become central to medical image analysis. We
benchmark federated Parameter-efficient fine-tuning (PEFT) of BiomedCLIP for chest radiograph classification across four public cohorts on three continents (USA, Vietnam, Spain).
Federated LoRA adaptation improves shared-class AUC on all four cohorts over the unadapted BiomedCLIP backbone (mean 0.687 to 0.802), showing that the gains come from
federated adaptation rather than from the pretrained model's zero-shot ability. Relative to isolated single-cohort training, federation improves the weaker cohorts while largely preserving the strongest and approaches a centralized reference (0.812) that pools all data. The singular value decomposition (SVD)-based product-space aggregation introduced by FlexLoRA is essential to this gain (naive factor averaging drops mean AUC by 0.097), whereas a drift-correcting optimizer (FedProx) shows no benefit over FedAvg in our single-seed runs, consistent with LoRA's low-rank updates already limiting client drift. Biomedical
vision-
language models can thus be adapted collaboratively across heterogeneous, geographically distributed institutions without centralizing data.
Read the full abstract ▾
All authors Sanjaya Poudel, Nirajan Kunwor, Manish Dhakal, Debesh Jha, Sunil Kumar Gaire
Computer Vision
Evidence-Guided Detection, Localization and Explanation for Text-Centric Image Forensics
Peifeng Liu, Bin Li, Qingsong Zhang et al. (+3 more)
The rapid progress of AIGC has made text-centric image manipulation increasingly accessible, creating new forensic challenges that require not only authenticity detection but also spatial grounding and evidence-based explanation. This paper presents our solution to the GenText-Forensics Challenge at ACM Multimedia 2026. We propose an evidence-guided detector-localizer-reasoner system, where an image-level detector provides a global authenticity prior, a dedicated localizer extracts tampered regions as spatial grounding evidence, and an MLLM-based reasoner generates structured forensic reports grounded in this expert forensic evidence. These modules are connected through a cascaded evidence flow: the detector gates the subsequent localization and prompting process, the localizer converts tamper responses into grounding boxes, and the reasoner is trained to synthesize the detector decision and localized evidence into the final report. As a key part of our method, we introduce iterative difficulty-aware mining to improve localization quality and apply report-mask consistency post-processing to align report grounding with predicted masks. On the official hidden test set, our system achieves a final score of 0.638 and ranks second in the challenge, validating the effectiveness of the proposed evidence-guided system. The code is available at https://github.com/peifengLiu42/ACMMM26-evidence-guided-detector-localizer-reasoner-system.
Read the full abstract ▾
All authors Peifeng Liu, Bin Li, Qingsong Zhang, Yangxin Yu, Leqing Chen, Xiaoye Qiu
Computer Vision
Rendering-in-the-Loop: An Execution-Driven Agent for Interactive Web Development
Yilong Guo, Hanqi Chen, Zixiao Ye et al. (+3 more)
Multimodal large
language models have achieved remarkable progress in front-end web development, generating interactive webpages from
multimodal references such as screenshots and interaction
videos. However, existing work largely emphasizes visual metrics such as aesthetics and layout similarity, while overlooking the more critical validation of interactive functionality. We present RILA, an execution-driven
agent that puts browser rendering in the loop, iteratively editing generated code from runtime interaction feedback. RILA introduces an Action Interaction Verification (AIV) module that replays the reference interaction trajectory on the generated webpage to collect grounded execution-aware observations, and an Execution-aware Rendering Score (ERS) that jointly measures interaction correctness and visual fidelity to guide iterative optimization. We further build an execution-verified data synthesis pipeline that produces diverse, high-quality training data, offering gains complementary to inference-time optimization. On IWR-Bench, RILA consistently improves both interaction and visual fidelity across foundation models. Notably, with our training pipeline, RILA lifts the compact Qwen3.5-9B backbone from 40.40% to 57.52%, surpassing far larger one-shot generators, including the 1T-parameter Kimi-K2.6 (55.61%) and the proprietary GPT-5.5 (55.74%).
Read the full abstract ▾
All authors Yilong Guo, Hanqi Chen, Zixiao Ye, Guanzhong Wang, Chen Yu, Zeyu Chen
Computer Vision · Cross-listed from cs.LG
Zhe Wang, Sijie Chen, Yiming Luo et al. (+2 more)
We present TropicalCycloneNext (TC-Next), a
multimodal deep learning model that forecasts tropical cyclone track and intensity at $6-24$h leads by leveraging a foundation model's forecast of atmospheric kinematic and thermodynamic fields, along with GridSat infrared satellite imagery. Trained only on GraphCast forecasts over the Western Pacific (WP), yet reliant only on generic atmospheric variables, TC-Next on GraphCast lowers track error by $15-44\%$ and intensity error by a factor of $3-6$ relative to the rule-based tracker TempestExtremes; applied without retraining to Pangu-Weather and IFS HRES forecasts, it stays ahead on both. Applied zero-shot to the generic weather fields of WeatherNext Cyclones on the 2025 WP season, it attains lower average intensity error at every lead time, and lower or comparable average track error, than that model's specialized direct tracker in a deterministic comparison. Ablations show that the satellite branch lowers track error at every lead time and intensity error at longer leads. TC-Next thus gives a common, independently verifiable basis for comparing the AI forecasts that inform cyclone-warning and
climate-adaptation decisions as best available science.
Read the full abstract ▾
All authors Zhe Wang, Sijie Chen, Yiming Luo, Daehyun Kim, Chien-Yi Chang
Computer Vision
KSG-Net: Key-Sparse and Global-Context Learning for Maritime 3D Ship Detection
Zhouyuan Huai, Meiqi Wan, Yan Yang et al. (+4 more)
Accurate 3D ship detection in maritime environments is critical for autonomous
navigation, yet remains challenging due to large-scale vessel variations, sparse point clouds of small vessels, and severe sea-clutter interference. Existing methods, primarily based on 2D features or dense representations, struggle to balance detection accuracy and computational efficiency, while sparse 3D detectors designed for road scenes generalize poorly to maritime scenarios. This paper focuses on two key challenges in maritime
LiDAR perception: weak feature representation for small and sparse vessels, and insufficient global structural modeling for large vessels due to the limited receptive field of local sparse convolutions. To address these issues, we propose KSG-Net, a Key-Sparse and Global-Context learning network for maritime 3D ship detection. The core idea is to jointly enhance local discriminative features and global structural awareness within a unified fully sparse detection framework. Specifically, a Key Sparse Multi-scale Aggregation (KSMA) module is designed to enhance the representation of small and sparse vessels by selecting informative key voxels and aggregating cross-scale neighborhood features. Furthermore, a Global Context Aggregation (GCA) module is introduced to capture long-range
geometric dependencies through scene-level context modeling with gated residual interactions, thereby improving the representation of large vessels. Extensive experiments on the Thames River vessel
dataset and simulated
datasets demonstrate that KSG-Net consistently outperforms existing methods in multi-scale vessel detection and exhibits strong robustness in complex maritime environments.
Read the full abstract ▾
All authors Zhouyuan Huai, Meiqi Wan, Yan Yang, Minshi Chen, Xin Yuan, Wei Wang, Xiao Wang
Computer Vision
DPA: Decoupling Product-Agnostic Anomaly Representations for Zero-shot Anomaly Generation
Hang Yao, Yansheng Fu, Ming Liu et al. (+4 more)
Industrial anomaly detection benefits from anomaly samples, yet newly deployed products typically provide only normal images, making anomaly samples difficult to collect. Zero-shot anomaly generation offers a promising solution which avoids collection of target-product anomalies. However, existing methods mainly rely on texture images or text descriptions as anomaly sources, which often produce unrealistic anomalies. Observing that similar anomalies can recur across different products, we propose anomaly transfer-based zero-shot generation, which reuses real anomalies from existing source products, making target-product anomalies no longer necessary to generate realistic anomalious samples for unseen target products. Since not every anomaly type suits the target product, an anomaly type filtering mechanism first selects plausible source types. To transfer selected anomaly, we propose DPA, a
diffusion-based framework that decouples product-agnostic anomaly representations. Instead of directly extracting anomaly representations, DPA learns product-irrelevant anomaly
embeddings through training with the mismatched data pair, enabling transferable anomaly concept learning across products. Furthermore, we design an adaptive mask-guided pipeline that leverages adaptive masks to control the positional and
geometric plausibility of generated anomalies during generation. A training-free anomaly labeling module is further introduced to produce pixel-level annotations aligned with generated anomalies. Extensive experiments on MVTec-AD, VisA, and a dedicated anomaly-transfer
benchmark demonstrate that the proposed setting and DPA generate more realistic anomalies and significantly improve downstream anomaly detection performance under both zero-shot and few-shot settings. Source code and models will be released.
Read the full abstract ▾
All authors Hang Yao, Yansheng Fu, Ming Liu, Zifei Yan, Yanli Ji, Hongzhi Zhang, Wangmeng Zuo
Computer Vision
LaST-SR: Laplace-Inspired Steady-Transient Complex-Frequency Decomposition for Single Image Super-Resolution
Linhao Li, Zhaojie Pan and Langkun Chen
Single-image super-resolution (SISR) requires global context modeling for structurally consistent reconstruction.
Fourier operators are increasingly adopted for global feature modeling. However, their periodic spectral bases constrain the representation of localized aperiodic variations, limiting the recovery of irregular structures and fine details. In dynamical systems, the Laplace neural operator extends
Fourier modes to complex frequencies and decomposes the output signal into complementary steady-state and transient responses to jointly model periodic and aperiodic information. We derive, for the first time, an approximate steady-transient decomposition for two-dimensional feature maps, providing an analytical basis for the proposed complex-frequency decomposition. Accordingly, we propose LaST-SR, centered on a Complex-Frequency Decomposition module that couples a global full-spectrum
Fourier branch for image-wide dependencies and long-range structural consistency with a window-conditioned local complex-frequency branch for localized, content-dependent aperiodic variations. To fuse the resulting features, we further design a Steady-Transient Collaborative Aggregation module for cross-branch interaction and joint aggregation. Experiments on five
benchmarks show that LaST-SR achieves the best PSNR/SSIM among the compared methods for $\times2$ and $\times4$ SISR. Ablation studies further validate the effectiveness of the proposed architecture and its key modeling mechanisms.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents
Zhuoran Yu, Le Thien Phuc Nguyen, Jaden Park et al. (+5 more)
Multimodal Large
Language Models (MLLMs) have achieved strong performance on structured visual understanding tasks such as chart and document question answering. However, existing
benchmarks typically evaluate these domains in isolation, leaving underexplored a key capability: whether models can use textual context to determine how chart evidence should be selected, interpreted, and aggregated. We introduce DocHop, a
benchmark for integrated chart–context
reasoning in document-style images. In DocHop, the document narrative specifies multi-step compositional constraints, while charts provide the corresponding data values. Questions are grounded on a semantic reference label defined in the narrative, requiring models to resolve target entities from context before aggregating evidence across multiple charts. To enable systematic evaluation, we construct DocHop via a
stochastic logic-first generation pipeline with controllable
reasoning depth and visual density, covering 2,074 examples across six task categories. Experiments on a wide range of proprietary and open-source MLLMs show a substantial gap to human performance: annotators achieve over 90% accuracy, while the best model reaches only 62.83%.
Reasoning-enhanced models consistently show improved results, but performance degrades as
reasoning complexity increases. Overall, DocHop provides a controlled testbed for challenging multi-hop document
reasoning.
Read the full abstract ▾
All authors Zhuoran Yu, Le Thien Phuc Nguyen, Jaden Park, Xinyi Gu, Zexue He, Soochahn Lee, Rogerio Feris, Yong Jae Lee
Computer Vision
Test-Time Logit Prompting for Source-Free Missing Modality Adaptation
Taixi Chen and Nancy Guo
Vision-
language models (VLMs) have achieved remarkable performance by leveraging complementary information from large-scale image-text pairs. However, missing-modality inputs are commonly encountered during real-world deployment, often leading to significant performance degradation. Existing methods primarily enhance model robustness by learning modality compensation strategies from source training data. However, their reliance on source training data makes them difficult to apply when original data are unavailable due to
privacy, storage, or accessibility constraints, such as clinical applications and personalized AI services. This raises an important yet underexplored question: can VLMs be efficiently adapted at test time for visual recognition with missing modalities without accessing source training data? To this end, we propose Test-Time Logit Prompting (TLP), a lightweight source-free test-time adaptation framework for visual recognition with missing modalities. To address missing-induced prediction shifts, TLP optimizes logit
prompts with uncertainty-aware adjustment and modality-complete consistency regularization, adaptively adjusting prediction confidence while preserving semantic consistency. Extensive experiments across diverse
vision-
language benchmarks demonstrate that TLP consistently enhances recognition performance under missing-modality scenarios, achieving up to 8% improvements while requiring only hundreds of tunable parameters and a few test-time optimization steps.
Read the full abstract ▾
Computer Vision
SelfLift: Accelerating Few-Step Diffusion via Self-Recovering Resolution Transition
Tingyan Wen, Chenqian Yan, Xurui Peng et al. (+4 more)
Few-step
diffusion models substantially compress temporal computation, making the spatial cost of each model evaluation an increasingly dominant source of inference latency. Progressive-resolution inference reduces this cost by performing early denoising at low resolution and reserving high-resolution computation for refinement. However, existing methods typically lift intermediate latents directly and rely on subsequent steps to absorb the induced distribution mismatch. In the few-step regime, the limited recovery budget leaves these errors as visible artifacts, constraining how late the transition can occur and, consequently, how efficiently it can be performed. We introduce SelfLift, a self-recovering progressive-resolution framework that derives both transition-repair signals and trajectory-aligned supervision from the generative model itself. SelfLift-zero proposes a training-free Artifact-Aware Consistency Lift, using disagreement between direct latent lifting and pixel-VAE re-encoding as both a localized artifact-risk signal and a model-native correction direction. It enables reliable late transitions without external super-resolution, extra denoiser evaluations, or sampling-schedule modifications. Building on this robust transition, SelfLift-rich performs On-Policy Self Recovery on student-visited states, transferring dense high-resolution guidance from an internal self-teacher while remaining aligned with the altered progressive-resolution dynamics. Across FLUX.2-Klein and Z-Image-Turbo, SelfLift reduces end-to-end latency by 41.5% and 44.1%, respectively. Combined with timestep
distillation, it delivers overall speedups of 29.61x and 19.21x over the corresponding 50-step models while preserving competitive generation quality, establishing a stronger speed-quality frontier for few-step
diffusion.
Read the full abstract ▾
All authors Tingyan Wen, Chenqian Yan, Xurui Peng, Xiazhang Fang, Shuai Wang, Xueqian Wang, Songwei Liu
Computer Vision
Detecting Object Hallucinations in Large Vision-Language Models via Cross-Modal Attention Drifts and Mask-Based Verification
Xuanbing Wen, Boxu Chen, Le Yang et al. (+4 more)
Despite recent advances in large
vision-
language models (LVLMs), object
hallucination remains a major barrier to their reliable deployment. Existing detection methods often characterize visual grounding using
attention from individual layers, leaving its
evolution across layers underexplored. We propose CADMP, a lightweight object
hallucination detection framework that combines adjacent-layer cross-modal
attention drift with prediction sensitivity to targeted visual masking. During decoding, CADMP quantifies distributional changes between consecutive cross-modal
attention maps to capture abrupt transitions in visual grounding. It then selects the transition with the largest drift, locates the corresponding visually relevant regions, and measures the change in prediction probability after masking these regions. These two signals provide complementary evidence:
attention drift characterizes the stability of internal visual grounding, while probability variation verifies whether a prediction truly depends on the identified visual evidence. A lightweight detector integrates both signals to identify hallucinated predictions. Experiments on multiple
benchmarks and representative open-source LVLMs demonstrate that CADMP achieves consistently competitive detection performance. Ablation studies further confirm the complementary contributions of adjacent-layer drift modeling and mask-based grounding verification.
Read the full abstract ▾
All authors Xuanbing Wen, Boxu Chen, Le Yang, Jiakai Wang, Zhengyu Zhao, Chenhao Lin, Chao Shen
Computer Vision · Cross-listed from cs.LG
Source-Free Class Relearning: Diagnosing Forgetting in Class Unlearning
Zahra Dehghani, Pablo Piantanida and Mohammadhadi Shateri
Class unlearning aims to remove a model's ability to recognize designated forget classes while preserving performance on retain classes. However, low forget accuracy after unlearning does not necessarily mean the class structure has been erased. Approximate unlearning methods can alter classifier decision boundaries while leaving recoverable structure in the representation. Prior work has shown that forget classes can be recovered, but existing approaches require real forget or retain samples, auxiliary data, or reference checkpoints. We study class relearning in a strictly source-free setting, asking whether a forget class can be recovered through a classifier-head update using only the unlearned model. Our approach rests on a theoretical analysis establishing a sufficient
alignment condition under which a single gradient step on a synthetic probe set increases the expected logit margin of the forget class. Building on this, we propose a white-box Source-Free Relearning Audit (SFRA), which generates candidate
embeddings in representation space and uses model-guided confidence filtering to construct high-confidence retain probes and low-confidence boundary-adjacent probes that are relabelled as the forget class. Gaussian sampling and Softmax confidence are used by default, while ablations with alternative proposal distributions and uncertainty criteria show that recoverability is not specific to these choices. To quantify recoverability, we introduce the Relearning Score (RS), which jointly measures forget-class recovery and retain-accuracy preservation, and report class-matched $\Delta$RS relative to a retrained reference. Experiments on CIFAR-10, CIFAR-100, and TinyImageNet with ResNet-18, ViT-B/16, and Swin-T show that several unlearning methods exhibit substantial source-free recoverability, and that for a subset of methods this recoverability exceeds the matched retrained reference.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Perceptually Regularized Diffusion Model for Image Super-Resolution
Chuxiangbo Wang, Pavithra Venkatachalapathy, Ying Liang et al. (+4 more)
Image super-resolution, which aims to reconstruct high-resolution images from their low-resolution observations, is fundamental to medical imaging, remote sensing, surveillance, microscopy, and scientific visualization. Traditional model-based methods formulate super-resolution as an inverse problem with hand-crafted regularization priors. While interpretable and theoretically grounded, they rely on fixed assumptions and require computationally intensive iterative solvers. Deep learning methods offer data-driven flexibility by learning nonlinear mappings from low- to high-resolution images, among which
diffusion models have achieved particularly impressive perceptual quality. However, the standard
diffusion training objective is a pixel-domain noise-prediction loss that does not explicitly enforce perceptual fidelity, which can lead to oversmoothing and loss of fine image structure. To address these limitations, we propose a perceptually regularized
diffusion framework that incorporates prior knowledge through perceptual-loss-based regularization, improving training convergence and encouraging the recovery of meaningful image features. Experiments on
benchmark datasets demonstrate improved perceptual quality and competitive distortion metrics, highlighting the effectiveness of regularization for
diffusion-based super resolution.
Read the full abstract ▾
All authors Chuxiangbo Wang, Pavithra Venkatachalapathy, Ying Liang, Min Wang, Jing Qin, Yifei Lou, Weihong Guo
Computer Vision
POI-Loc: A Fine-Grained POI Localization Benchmark and an Asymmetric Global-to-Local Matching Method
Lu Han, Xiting Sun, Hao Wang et al. (+4 more)
Point-of-interest (POI) localization matches user-provided storefront close-ups to the same shops in wide, geo-tagged vehicle-mounted street views. POIs may change while the surrounding scene stays similar, so scene-level recognition alone cannot establish POI identity. Differences in target scale and capture domains further challenge matching. We introduce POI-Loc, to our knowledge the first
benchmark dedicated to this asymmetric, fine-grained POI localization task. Many visual place recognition methods represent each image with a single global vector, which tends to dilute fine-grained features of small storefronts amid background clutter. We propose GLAM (Global-to-Local Asymmetric Matching) to combine global and local evidence. In stage one, a single
attention-pooled query probe is matched against compact reference region
tokens via learnable soft top-k interaction, with the resulting local similarity fused with global similarity for
retrieval. Stage two reuses query region
tokens before
attention pooling and stored reference
tokens for mutual-nearest-neighbor re-ranking. GLAM surpasses both global and two-stage baselines on Recall@1/5/10 and mAP, with about $5\times$ smaller re-ranking features and $280\times$ lower per-pair matching cost than FoL. The
benchmark and code will be released at https://github.com/roadhan/glam.
Read the full abstract ▾
All authors Lu Han, Xiting Sun, Hao Wang, Zhiqiang Cao, Ruihuan Du, Ziquan Zeng, Chunlong Lv
Computer Vision
InstEditSeg: Instruction-Driven Image Editing for Polyp and Skin Lesion Segmentation
Ziquan Liu, Zhewei Zhu and Xuyang Shi
Accurate
segmentation of polyps and skin lesions is pivotal for clinical diagnosis, yet existing methods struggle with low contrast, ambiguous boundaries, and cross-domain distribution discrepancies. Discriminative networks and most
diffusion-based
segmentation approaches predict standalone binary masks, leaving the visual priors of large-scale pretrained generative models largely unexploited. We propose InstEditSeg, a unified generative framework that reformulates medical
segmentation as an instruction-driven image editing problem. Instead of emitting a mask, the model renders a color-coded overlay on the original image, conditioned on a textual instruction, so that the edited output aligns with the natural image distribution learned by latent
diffusion models and mitigates the domain gap between natural and medical imagery. To recover fine anatomical structures, we introduce DINOv3 as an auxiliary visual encoder and a DINO Feature Guidance Block that builds a multi-scale feature pyramid. The pyramid is fused into the
diffusion U-Net by channel concatenation and zero-initialized convolution so that hierarchical discriminative priors can be injected without perturbing the pretrained weights. A dual-branch classifier-free guidance strategy requiring only two forward passes per denoising step reduces inference cost. On polyp and skin lesion
benchmarks the framework achieves accuracy competitive with strong discriminative baselines, and it further demonstrates concrete advantages of the generative formulation: notably better cross-domain generalization on unseen data, more complete multi-lesion
segmentation, instruction-conditioned task control, and sampling flexibility. We also analyze the strengths and limitations of the paradigm, including its color sensitivity and unsupported attribute-conditioned selection. Code is available at: https://github.com/wincharm001/InstEditSeg.
Read the full abstract ▾
Computer Vision
InsightSeg: Reusing Correction Insights for Guideline-Consistent Segmentation
Vanshika Vats, Ashwani Rathee and James Davis
Guideline-consistent semantic
segmentation requires more than category recognition, as real-world labeling policies demand fine-grained, task-specific decisions. Recent multi-
agent refinement systems improve compliance with such textual guidelines by detecting and correcting errors. However, they are stateless: feedback from the critiquing
agent is discarded, causing the same guideline-specific mistakes to be repeatedly rediscovered and corrected across the
dataset at the cost of additional refinement. We introduce InsightSeg, an episodic
memory mechanism that converts successful correction episodes into reusable, visually grounded insights. A meta-analyzer distills each qualifying episode into directive natural-
language insights and anchors them to the local image regions that caused the error using patch-level visual concept vectors. On subsequent images, these concepts are matched against dense patch
embeddings to retrieve relevant insights, which condition the segmenting
agent before making its first prediction. This shifts the system from correcting recurring errors to preventing them, improving
segmentation quality before any refinement occurs. Across Waymo and Cityscapes, InsightSeg improves both first-pass and final guideline-consistent
segmentation performance while requiring fewer refinement steps, demonstrating that multi-
agent refinement can become more accurate and efficient by drawing on past correction experience.
Read the full abstract ▾
Computer Vision
Who Drives the Probability Game of VLMs? A Temporal Causal Drive Evaluation Framework
Shuyao Xiao, Shengling Wang, Haoyu Niu et al. (+4 more)
Vision-
language models (VLMs) are increasingly evaluated on complex image and
video understanding tasks, yet conventional metrics primarily assess final-answer quality and reveal little about how different information sources shape the generation process. We propose a causal and temporal evaluation framework that traces the evolving roles of visual input, question text, and generated prefixes during autoregressive decoding. Grounded in a Structural Causal Model, we use interventions and
backdoor adjustment to derive three step-indexed causal-drive metrics—Visual Causal Drive (VCD), Question Causal Drive (QCD), and Prefix Causal Drive (PCD)—for characterizing source-specific generation patterns without requiring reference answers. Experiments on Qwen3-VL-8B-Instruct across MAVIS, LLaVA-
Video-178K, and MiraData, together with cross-model validation on InternVL2-8B, reveal a consistent transition from stronger early question and visual guidance toward increasing reliance on generated prefixes. Randomized-intervention validation shows that QCD and PCD reduce recovery error over observational PMI baselines by 34.8% and 47.1%, respectively. On VLMBias, the prefix–visual imbalance score achieves 0.767 AUROC and 0.873 AUPRC for distinguishing prior-driven from visually grounded generations. These results show that causal-drive trajectories provide complementary source-level diagnostics for
multimodal generation.
Read the full abstract ▾
All authors Shuyao Xiao, Shengling Wang, Haoyu Niu, Ke Chao, Changwei Xu, Xinran Duan, Chaoyong Jiang
Computer Vision
Linear Fusion MultiDiffusion for Fast Training-Free Spherical Panorama Generation
Akio Hayakawa, Yusuke Mukuta and Tatsuya Harada
We propose LF-MultiDiffusion, a training-free panorama generation method that extends MultiDiffusion to support linear projections between target and reference image spaces. Our key idea is to reformulate latent aggregation as a regularized least-squares problem and solve it efficiently with a Krylov-based iterative solver inside the denoising loop. This formulation enables denser and more natural mappings than prior training-free methods, yielding more stable generation with far fewer perspective views. As a result, LF-MultiDiffusion reduces the number of image generator evaluations during denoising and significantly improves inference efficiency. Experiments show that LF-MultiDiffusion achieves better visual quality, text
alignment, and panoramic consistency than the strongest training-free baseline, while providing a 15.36$\times$ speedup. Our project page is available at: https://ahykw.github.io/lfmd.
Read the full abstract ▾
Computer Vision
Morphology signal in whole slide image foundation models can automatically triage slides
Ayushi Sinha, Shashank Yadav, Benjamin Holmes et al. (+9 more)
Patient exams in the cancer diagnosis and staging process typically generate several whole slide images (WSIs). One of the initial steps in training models on WSI data is identifying one or a few slides containing tumor or other diagnostic biomarkers necessary for downstream prediction tasks such as estimating recurrence risk or progression-free survival. This step requires tedious manual curation by experienced pathologists. Many published
datasets make the artificial assumption of 1 slide per patient. Alternatively, all slides per patient may be used for model training, which may dilute the signal from the few slides containing tumor or other relevant information. In this paper, we present a pipeline to overcome these challenges using publicly available WSI foundation models (FMs). Our evaluations show that ranking WSIs based on predictions from zero-shot classification using WSI FMs accurately identifies slides with the most tumor, indicating that WSI FMs contain sufficient morphology signal to automatically triage slides. We also present a formulation for ranked evaluation to
benchmark FM performance in slide triage. We show, on multiple
datasets, that tumor slides are identified in the top-2 ranked slides for patients with up to 43 slides.
Read the full abstract ▾
All authors Ayushi Sinha, Shashank Yadav, Benjamin Holmes, Pravat Das, Aaron W. Bogan, James S. Lewis Jr., Santiago Romero-Brufau, Andrew Y. K. Foong, Scott H. Kaufmann, Kathryn M. Van Abel, David M. Routman, Michael R. Lucas
Computer Vision
Aggregating Neighbor Embedding Projection and Rank-Based Manifold Learning for Image Retrieval
Vinicius Atsushi Sato Kawai, Gustavo Rosseto Leticio, Lucas Pascotti Valem et al. (+1 more)
Content-based image
retrieval (CBIR) has advanced significantly with deep learning, yet effectively ranking similar images remains challenging, particularly in high-dimensional feature spaces, where pairwise distances often fail to capture contextual relationships and the semantic gap between visual features and high-level concepts persists.
Manifold learning and rank-based refinement methods have emerged as complementary strategies, respectively improving feature representations and exploiting contextual information embedded in ranked lists, such as neighborhood relationships among images. However, combining these projection-based and rank-based strategies to exploit their complementary properties remains a challenging research problem. To address this, we propose a framework that combines neighbor
embedding projections with rank-based
manifold learning through rank aggregation. Uniform
Manifold Approximation and Projection (UMAP) generates alternative low-dimensional feature representations, and ranked lists obtained from UMAP projections and rank-based re-ranking methods are combined using the Borda Count aggregation strategy. Experiments were conducted on several public
datasets using deep learning features extracted from ResNet152, Swin
Transformer, and DINOv2 models. Results show that the proposed approach improves
retrieval effectiveness in several scenarios, particularly when the baseline representation struggles to achieve high precision. The aggregation strategy also often improves the quality of top-ranked positions, leading to competitive Mean Average Precision (MAP) and Precision values across different
datasets and feature extractors. These findings suggest that combining projection-based and rank-based
manifold learning strategies through rank aggregation can provide complementary contextual information for image
retrieval tasks.
Read the full abstract ▾
All authors Vinicius Atsushi Sato Kawai, Gustavo Rosseto Leticio, Lucas Pascotti Valem, Daniel Carlos Guimarães Pedronette
Computer Vision · Cross-listed from eess.IV
Data-Efficient Networks for Multi-Contrast MRI Reconstruction based on a Generalized Content/Style Prior
Chinmay Rao, Efe Il{\i}cak, Matthias J.P. van Osch et al. (+5 more)
Multi-contrast MR scans contain redundant structural information that can be leveraged during reconstruction and potentially accelerate acquisition times. This idea has inspired end-to-end guided reconstruction models, leveraging one or more contrasts to guide the reconstruction of a different contrast. However, these models require large paired multi-contrast raw
datasets for training, limiting their application in low-data regimes. In this work, we propose a modular framework, namely CoSMo-RecNet, for learning guided reconstruction models in the low-data regime. At its core is a reusable multi-contrast representation based on a content/style model, which can be learned from large-scale, publicly accessible, unpaired multi-contrast image
datasets, without available k-space data. Using this frozen model as a multi-contrast prior and using a set of reference contrasts, the reconstruction problem reduces to a much simpler refinement problem that can be solved by a lightweight unrolled network and thus learned from small, task-specific reconstruction
datasets. We demonstrate the efficacy of CoSMo-RecNet by evaluating it on the low-field 0.3 T M4Raw
dataset, showing stable reconstruction quality on decreasing the raw training data budget. CoSMo-RecNet achieved higher reconstruction quality with 5 training subjects or lower compared to a parameter-count-matched MoDL trained on 100 subjects. On a data-limited and severely out-of-distribution ultra-low-field 47 mT Halbach scanner
dataset, CoSMo-RecNet was superior to other viable strategies, including classical reconstruction, transfer learning, and zero-shot reconstruction.
Read the full abstract ▾
All authors Chinmay Rao, Efe Il{\i}cak, Matthias J.P. van Osch, Mariya Doneva, Laurens Beljaards, Navid Jabarimani, Nicola Pezzotti, Marius Staring