Computer Vision
GaussVid: Sparse-View Gaussian Splatting with 3D-Aware Video Diffusion Priors
Xinhui Liu, Can Wang, Wei Jiang et al. (+2 more)
3D Gaussian Splatting (3DGS) has achieved remarkable success in novel view synthesis; however, reconstructions under sparse views often exhibit noticeable artifacts. While recent
video diffusion models provide strong spatio-temporal priors for 3DGS restoration, directly fine-tuning them for restoration is suboptimal, as they lack awareness of the underlying multi-camera geometry, resulting in multi-view inconsistencies. In this work, we propose a novel 3D-aware
video restoration framework designed to enhance the quality of sparse 3DGS reconstruction. Specifically, we construct a large-scale 3DGS
video dataset to enable specialized fine-tuning. To bridge the gap between 2D
video generation and 3D multi-view constraints, we introduce a camera-conditioned
geometric prior. By using the first and last frames as boundary anchors and encoding the corresponding camera relationships, we explicitly inject spatial structure into the
video generation pipeline. This boundary-anchored, camera-aware prior guides the network toward geometrically grounded restoration that remains coherent across viewpoints. Extensive experiments show that, among
video-prior restoration methods, our approach attains the best pixel- and structure-level fidelity (PSNR/SSIM) and improves multi-view consistency, while remaining competitive in perceptual quality (LPIPS).
Read the full abstract ▾
All authors Xinhui Liu, Can Wang, Wei Jiang, Wei Wang, Dong Xu
Computer Vision
BC-IHV: Conditioning the Color Space for Stable Rectified-Flow Low-Light Enhancement
Yi Ai, Zheng Chen, Yuanhao Cai et al. (+2 more)
Low-light image enhancement (LLIE) must correct ambiguous exposure without overwriting structure already supported by the input. Generative transport can model exposure ambiguity; however, its flexibility may also alter observable geometry and chromatic content. Moreover, fixed invertible color coordinates are usually treated only as representations, although their inverse mappings reshape the RGB-domain gradients received by the enhancement network. To address these issues, we propose Structure-Anchored Rectified Flow (SA-RF), which maintains correspondence through separate chromaticity/intensity stems, a scale-matched condition pyramid, and HybridAda. HybridAda assigns location-specific
retrieval to spatial cross-
attention and global exposure modulation to pooled AdaLN. We further introduce BC-IHV, a learnable Box–Cox polar color space whose analytically invertible intensity mapping controls the inverse-gradient dynamic range through a single exponent. This allows the representation to balance dark-range expansion and gradient conditioning instead of adopting a fixed linear or logarithmic law. Experiments on three LOL
benchmarks, blind image-quality evaluation, and cross-
dataset tests demonstrate consistent reconstruction and perceptual advantages over the sota. Controlled studies further support the effectiveness of both the proposed framework and color representation.
Read the full abstract ▾
All authors Yi Ai, Zheng Chen, Yuanhao Cai, Yulun Zhang, Xiaokang Yang
Computer Vision
FIRM-Video: Check Before You Score for Reliable Text-to-Video Reward Modeling
Peiyuan Zhang, Xiangyu Zhao, Hongbo Liu et al. (+8 more)
Reliable reward models are essential for text-to-
video evaluation and
alignment. However, the trade-off between evaluation accuracy and inference efficiency places high demands on the quality of training supervision. Existing approaches often rely on holistic judges with fixed rubrics or open-ended
reasoning, leading to incomplete inspection, unfaithful justification, and entangled attribution. We introduce FIRM-
Video, a unified checklist-driven data construction framework based on a check-before-score principle: construct dimension-specific checklists, verify each criterion against temporal visual evidence, and aggregate only verified decisions. For Instruction Following, FIRM-
Video decomposes
prompts into weighted atomic requirements; for World Coherence, it constructs
prompt-calibrated, target-specific checks grounded in visible entities and actions; and for Perceptual Quality, it applies a generic taxonomy of visual defects. The verified criteria and scores are further transformed into natural-
language analyses for end-to-end reward modeling. Subsequently, we construct FIRM-
Video-90K with 88,044 dimension-specific instances from 29,348
videos, and introduce FIRM-
Video-Bench with 750 point-wise human annotations across 250
videos. The Qwen3-VL-based FIRM-
Video-8B achieves the best overall MAE on FIRM-
Video-Bench while consistently delivering the highest VBench Total, Quality, and Semantic Scores in Best-of-8 sampling across three
video generators.
Read the full abstract ▾
All authors Peiyuan Zhang, Xiangyu Zhao, Hongbo Liu, Xiaoxing Hu, Mingxin Liu, Shuran Ma, Yunhang Shen, Jian Hu, Haihan Gao, Haoyu Cao, Xue Yang
Computer Vision
Towards Bitstream-corrupted Harsh Visual Understanding: Through Bitstream Language Modeling as Robust Semantic Priors
Chaoran Huang, Fangcheng Li, Tianyi Liu et al. (+2 more)
Bitstream-corrupted Harsh Visual Understanding (BcHVU) aims to understand harshly degraded
videos originally decoded from a severely corrupted bitstream in real-world multimedia communication. The ill-posed nature of BcHVU poses a major challenge for existing
vision models, as even subtle bitstream corruption can lead to irreversible pixel distortion and significant semantic loss. To address these challenges in BcHVU, we propose Bitstream
Language Modeling as Robust Semantic Priors (BLMSP), a framework for learning and injecting bitstream-native semantic cues. Our proposed BLMSP framework learns to extract bitstream-native semantic cues by bitstream
language modeling, and leverages them as priors by injecting into off-the-shelf
vision models of BcHVU tasks. Specifically, we present a
Video Bitstream Byte Model (VBBM) that integrates byte-level modeling and cross-codec semantic
distillation, enabling it to interpret robust semantics from byte sequences in multiple corrupted bitstream formats. The learned bitstream semantics are leveraged as robust priors and fused into BcHVU model backbones for improving the quality of
video restoration, captioning, and human pose estimation. To train BLMSP, we construct a large-scale multi-source Corrupted-bitstream Harsh-
video Paired (CHP)
dataset containing 607k corrupted bitstream segments and 287k paired harsh
video clips. Extensive experimental results show that the learned bitstream priors improve
video restoration, captioning, and human pose estimation by 2.51 dB in PSNR, 0.20 in CIDEr, and 0.18 in PCK@0.2 on average, respectively. These results demonstrate that corrupted bitstream can serve as robust semantic priors in solving pixel distortion and semantic loss in BcHVU.
Read the full abstract ▾
All authors Chaoran Huang, Fangcheng Li, Tianyi Liu, Wenyang Liu, Kejun Wu
Computer Vision
Towards Alias-Free 4D Gaussian Representations with Motion-Aware Filtering
Ankit Dhiman, Kunal A Kathare, Pranav Vignesh et al. (+2 more)
Novel-view synthesis of dynamic scenes, crucial for AR/VR applications, remains a challenging problem. Recent methods adapt representations like 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF) for dynamic scenes by incorporating time as the fourth dimension (4D representations). These 4D representations still suffer from aliasing artifacts, especially when generating novel views from divergent viewpoints (zoom-in/zoom-out operations). While using 3D smoothing filters like those proposed in Mip-Splatting might seem like a possible solution, they fail to account for local motion and also exhibit aliasing. To address this, we propose a motion-aware 3D smoothing filter specifically designed for 4D representations. Our approach adapts the filter strength based on local motion information, effectively mitigating aliasing without compromising rendering quality. This is achieved by estimating the joint density function of time and focal-to-depth ratio using a non-parametric estimation method. During inference, we sample from this joint distribution to determine the appropriate smoothing filter. This flexible strategy can be integrated with various 4D representations. Our evaluations on standard
datasets demonstrate superior performance compared to state-of-the-art methods.
Read the full abstract ▾
All authors Ankit Dhiman, Kunal A Kathare, Pranav Vignesh, Lokesh R Boregowda, Venkatesh Babu Radhakrishnan
Computer Vision · Cross-listed from cs.AI
VisAdj: Learning Adjacency Matrices from Node-Link Images
Jiahao Xie and Guangmo Tong
Learning adjacency
matrices from node-link images is a fundamental problem for recovering structured graph information from visual observations. Existing methods typically rely on fixed KNN-based heuristics for candidate edge selection and fail to capture dependencies among edges. To overcome these limitations, we propose VisAdj, a new framework for
topology-aware adjacency prediction. VisAdj introduces an
attention-sparse neighbor sampler to adaptively select a high-recall set of candidate node pairs and performs joint edge inference using a line-graph
transformer that treats candidate edges as
tokens and explicitly models dependencies among incident edges. Extensive experiments on synthetic graphs, road networks, and vessel images demonstrate that VisAdj consistently outperforms existing baselines by clear margins.
Read the full abstract ▾
Computer Vision
PatchGate: Narrowing the Verbalization Gap with Intrinsic Object Inventories in Frozen Vision-Language Models
Jihyung Ko, Eunji Jung, Hyeongsub Kim et al. (+4 more)
Reliable image captioning in
Vision-
Language Models (VLMs) requires captions to be both precise and complete, avoiding unsupported object mentions while covering visible objects. Existing training-free methods primarily address the former requirement, suppressing unsupported object words by intervening on model-predicted mentions during generation. Because they operate only on objects the model is already likely to mention, visible objects omitted from the output remain difficult to recover. We propose PatchGate, a training-free framework that extracts
prompt-free object evidence intrinsic to a frozen VLM before generation and uses it to narrow the gap between an intrinsic object set and final object mentions. In the first stage, Visual Evidence eXtraction (VEX) reads patch-level lexical evidence from the latter half of LM decoder layers and constructs an image-conditioned object set without any task
prompt. In the second stage, Visual-Evidence Inclusion-Exclusion Decoding (VIED) uses this object evidence to calibrate decoding logits, promoting evidence-supported but under-verbalized objects and suppressing weakly supported but over-verbalized objects. On AMBER, PatchGate improves both sides of object-level reliability, increasing visible-object coverage from 49.4 to 56.0 (+13.4%) and reducing object
hallucination by lowering CHAIR from 7.5 to 6.6 (-12.0%), without external detectors or fine-tuning and with one extra forward pass.
Read the full abstract ▾
All authors Jihyung Ko, Eunji Jung, Hyeongsub Kim, Ziseok Lee, Jae Won Cho, Sanghyun Jo, Kyungsu Kim
Computer Vision
Through the Schrödinger Bridge: Benchmarking Antemortem Image Restoration from Postmortem Autolysis to Enhance Forensic Diagnostics
Shuang Hao, Jiacheng Yue, Yaxuan Zhao et al. (+4 more)
Forensic histopathology, essential for determining cause of death and disease diagnosis, is severely impeded by postmortem autolysis, i.e., an irreversible,
stochastic degradation process that distorts tissue morphology and introduces diagnostic subjectivity, thereby underscoring the value of restoring autolyzed images to a diagnostically plausible, pre-autolysis state for improving objectivity in forensic practice. This restoration task is fundamentally challenging due to the large, non-deterministic morphological changes caused by autolysis and the infeasibility of pixel-wise paired data, which invalidates assumptions underlying supervised and cycle/structure-consistent unpaired translation methods. To address this, we formalize forensic histopathology autolysis restoration as a new task: under unpaired supervision, transform postmortem images with severe autolysis into diagnostically meaningful “antemortem” representations. We contribute AutoPath, the first homologous yet unpaired
dataset for this problem, constructed by splitting specimens into adjacent tissue blocks—one processed immediately, the other exposed to induce autolysis—yielding nearly ten thousand $10\times$ patches from 69 cases with varying liver conditions. We further frame the problem as a Schrödinger Bridge between the autolyzed and non-autolyzed distributions, offering a principled approach to modeling
stochastic, severe morphological degradation. Critically, we demonstrate the misalignment of generic image-level generative metrics (e.g., FID) with diagnostic utility and propose a forensically grounded, slide-level diagnostic distribution consistency evaluation. Overall, this work establishes a reproducible
benchmark (encompassing task definition, a real-world
dataset, and an evaluation methodology) toward rigorous and practically meaningful progress in autolysis restoration for forensic pathology.
Read the full abstract ▾
All authors Shuang Hao, Jiacheng Yue, Yaxuan Zhao, Fan Wang, Jianhua Ma, Erwen Huang, Chunfeng Lian
Computer Vision · Cross-listed from cs.LG
MSM-Mem: A Universal Medical Structured Multimodal Memory Framework for Medical AI Agents
Md Asaduzzaman Jabin, Khoa Le, Lin Zhao et al. (+1 more)
Clinical decision-making is inherently experience-driven: physicians progressively refine their
reasoning by synthesizing patient history,
multimodal observations, and prior diagnostic experiences across interactions. In contrast, current
multimodal large
language model (MLLM)-based medical AI
agents largely operate as stateless inference systems, generating decisions independently for each interaction without retaining or internalizing experiential knowledge. This discrepancy limits their ability to progressively improve
reasoning reliability through usage and adapt to longitudinal patient contexts in real-world clinical workflows. In this study, we propose Medical Structured
Multimodal Memory (MSM-Mem), an agentic
memory framework that enables medical AI
agents to evolve through accumulated clinical experiences. MSM-Mem organizes heterogeneous clinical experiences into semantic, episodic, and visual
memory and incrementally updates them during inference, allowing the
agent to retrieve prior experiences to inform current
reasoning and progressively refine decision-making over time. Evaluations on MoE-LLaVA backbones demonstrate consistent performance improve- ments with further gains observed through continued usage. In general, MSM-Mem offers a viable pathway toward medical AI
agents capable of evolving their
reasoning competence in a manner analogous to the way clinicians learn from practice over time.
Read the full abstract ▾
All authors Md Asaduzzaman Jabin, Khoa Le, Lin Zhao, Tianming Liu
Computer Vision
FlashReg: GPU-Accelerated 3-Clique Point Cloud Registration for Real-Time Correspondence-to-Pose Estimation
Ziyang Yu, Xiang Li, Qiong Chang et al. (+1 more)
Graph-based point cloud registration achieves high robustness by identifying geometrically consistent correspondence sets, but constructing second-order compatibility graphs and enumerating candidate cliques remain compute- and
memory-intensive. This work presents FlashReg, a GPU-oriented correspondence-to-pose
estimator that avoids materializing the dense scored second-order graph. Its Fast First- and Second-Order Graph (FFSOG) construction builds a capacity-bounded sparse second-order graph directly from the binary first-order graph. A dataflow-optimized three-node clique (3-clique) search then selects pivots from compact per-row candidate pools and enumerates triples through sorted sparse-neighborhood intersections. Across indoor and outdoor
benchmarks, FlashReg reduces correspondence-to-pose latency by 2–3x relative to TurboReg at comparable registration recall, while using about 50% of its peak allocated tensor
memory on an embedded GPU. These results make FlashReg suitable as a high-throughput registration backend within onboard perception pipelines.
Read the full abstract ▾
All authors Ziyang Yu, Xiang Li, Qiong Chang, Jun Miyazaki
Computer Vision
SAFE-G: Structure-aware Faithful Evidence-guided Generation for Knowledge-based Visual Question Answering
Long Shu, Shuochen Liu, Wei Chen et al. (+4 more)
Knowledge-based Visual Question Answering (KB-VQA) aims to answer queries that necessitate
reasoning over external knowledge sources beyond the visual content. Typically, current methods fuse
multimodal features to retrieve external information, subsequently leveraging
Multimodal Large
Language Models (MLLMs) to derive answers from the retrieved evidence. However, these methods often struggle to capture structural associations within complex contexts to effectively filter noise. Furthermore, they frequently fail to ensure that the
reasoning process remains strictly faithful to the retrieved evidence. To address these challenges, we propose SAFE-G, a Structure-Aware Faithful Evidence-guided Generation framework, which enables precise evidence localization and trustworthy
reasoning. Specifically, we first employ a coarse-grained hybrid search fusing visual and textual modalities to recall candidate documents, and subsequently implement a structure-aware fine-grained graph
retrieval that captures structural dependencies to filter noise and pinpoint precise evidence. Moreover, we introduce a reinforcement learning (RL) strategy with an evidence-grounded reward that assigns credit to correct answers only when the selected evidence is correct. This strict
alignment constraint compels the model to anchor its response in the retrieved context, effectively enhancing its capability to locate evidence via
multimodal features and perform faithful
reasoning. Extensive experiments on the Encyclopedic-VQA and InfoSeek
benchmarks demonstrate that SAFE-G outperforms prior methods by a margin of 8.9% and 3.5%, substantially enhancing the overall
reasoning accuracy. Our source code is publicly available at: https://github.com/MINE-USTC/SAFE-G.
Read the full abstract ▾
All authors Long Shu, Shuochen Liu, Wei Chen, Junda Lin, Zhi Zheng, Huijun Hou, Tong Xu
Computer Vision · Cross-listed from cs.AI
HIRA: A Human-in-the-Loop Retrieval-Augmented Cascade for Document Classification in Regulated Industries
Shangxuan Tian, Yanhui Chen and Carlos Queiroz
Document classification in regulated industries is constrained by data residency, limited cold-start labels, scarce review capacity, and costly model-governance procedures. We present HIRA, a training-free, on-premises
retrieval-augmented cascade for document classification in regulated deployments that combines BM25 over OCR text, dense text
embeddings, and image-level representations through validation-calibrated weighted reciprocal-rank fusion. Confident documents are classified directly by
retrieval; uncertain or visually confusable documents are passed to a locally hosted LLM verifier, which receives the OCR text, retrieved exemplars, label descriptions, and confusion-specific terms. When the verifier remains uncertain, the document is sent to human review. Each correction is stored as a margin-weighted
retrieval exemplar and updates a Dirichlet-smoothed confusion graph, letting the system improve without updating model weights. On a private 80-class trade-finance corpus, HIRA processes the full 30,233-document production stream while requesting human correction for only 1,945 documents (6.4%), improving Macro-F1 from 0.6218 to 0.8548. On the corrected Tobacco-3482
benchmark, HIRA reaches 0.9423 Macro-F1 with a locally hosted DeepSeek-R1-Distill-Qwen-32B verifier, 17.4 percentage points above the zero-shot LLM baseline, while invoking the verifier for only about 40% of documents and reducing LLM calls by approximately 60%. With 518 human corrections (24.8% of the pool), HIRA matches the fully labelled pool oracle, in which all 2,086 pool documents are indexed with their ground-truth labels. These results show that selective human feedback and
retrieval-
memory adaptation can be a practical alternative to repeated model retraining for long-tail document classification in regulated deployments.
Read the full abstract ▾
Computer Vision
UniDiffFusion: A Unified Diffusion Framework for Multi-Task and Degradation-Robust Image Fusion
Xingxin Xu, Siqi Zhao, Xin Li et al. (+3 more)
General image fusion aims to integrate complementary information from multiple source images, but existing methods often rely on task-specific models and struggle to maintain robust performance under diverse degradation conditions. In this paper, we propose UniDiffFusion, a unified
diffusion framework for multi-task and degradation-robust image fusion. UniDiffFusion leverages the strong generative prior of a pretrained
diffusion model to establish a shared fusion backbone across heterogeneous fusion tasks, while introducing task- and degradation-aware conditional adaptation to accommodate their distinct information-selection requirements. Specifically, we employ task
prompt modulation to progressively adapt the shared
diffusion representations to different fusion objectives, and develop a degradation
prompt router to dynamically retrieve degradation-aware priors and restore corrupted source features before fusion. Furthermore, an application
prompt bank is introduced to incorporate task-oriented semantic guidance for downstream applications, such as object detection and semantic
segmentation, without altering the shared fusion and restoration pathways. The proposed framework is trained in a progressive manner to decouple fusion learning, degradation-aware restoration, and application-specific adaptation, thereby reducing interference among heterogeneous objectives. Extensive experiments on visible-infrared, multi-exposure, and multi-focus image fusion demonstrate that UniDiffFusion achieves superior fusion quality and robustness under both clean and degraded conditions. Moreover, UniDiffFusion consistently improves downstream detection and semantic
segmentation performance, demonstrating its effectiveness as a unified
diffusion framework for both perceptual fusion and task-oriented
vision.
Read the full abstract ▾
All authors Xingxin Xu, Siqi Zhao, Xin Li, Xinjie Yao, Yiming Sun, Pengfei Zhu
Computer Vision
DefaultShift: Auditing Semantic Default Shift in Accelerated Text-to-Image Models
Xuanhua Yin, Chuanzhi Xu, Shunqi Mao et al. (+2 more)
Few-step text-to-image models increasingly replace slower generators, yet acceleration can silently change distributions over unspecified attributes even when individual outputs remain plausible and aligned. We call these distributions semantic defaults and their change under replacement semantic default shift. Existing quality, preference, and diversity evaluations do not test whether a replacement preserves its reference model's semantic defaults. We introduce DefaultShift, a paired audit that labels repeated samples with closed semantic vocabularies, measures probability-mass movement, and separates interpretable ranking from confirmatory cross-fit inference. Across 14 reference and replacement pairs, adjusted color discrepancies range from 0.054 to 0.303 with recipe-specific directions. A 1,000-image human audit reproduces the ordering. We further introduce DefaultShift-Select, an offline calibration method that reduces human-measured shift by 10.3 percent to 35.1 percent across Turbo, DMD2, and FLUX without material quality loss. Under balanced evaluation, selected data recover 4.3 accuracy points and 7.5 worst-group points over uncalibrated replacement data. DefaultShift makes semantic preservation under acceleration measurable and actionable.
Read the full abstract ▾
All authors Xuanhua Yin, Chuanzhi Xu, Shunqi Mao, Wei Guo, Weidong Cai
Computer Vision
SpatialDiff: 3D-Aware Object Movement via Implicit Spatial Modeling
Zheng Liu, Zijian He, Huiguo He et al. (+5 more)
Recent advances in image editing allow impressive manipulation of objects, existing methods still struggle to handle spatial movement in complex scenes, such as objects span different depth layers or are partially occluded. Most image editing methods focus solely on prior information from 2D
datasets, emphasizing planar features while lacking support for spatial structures. Even approaches that incorporate explicit positional information fail to capture true 3D spatial relationships, thus limiting accurate object movement in complex scenes. In this paper, we present SpatialDiff, a method that effectively captures 3D spatial structures, enabling precise and consistent object movements in complex scenes. Our core innovations are twofold: (1) Implicit 3D Spatial Modeling, which introduces 3D prior knowledge and enables the model to internally build a comprehensive understanding of the three-dimensional spatial structure; and (2) Global Spatial Supervision, which constrains the latent spatial features to enable the model to perceive changes in object spatial positions caused by editing operations. Experimental results demonstrate that our method significantly improves the accuracy and fidelity of spatial movement in complex scenes.
Read the full abstract ▾
All authors Zheng Liu, Zijian He, Huiguo He, Weizhi Zhong, Yejun Tang, Huan Yang, Kun Gai, Guanbin Li
Computer Vision
LiteEvent-AE: Lightweight Autoencoder for Event-Based Vision on Low-Latency Energy-Constrained Edge Devices
Riadul Islam, Joey Mule, Dhandeep Challagundla et al. (+3 more)
Event-based
vision has emerged as a promising paradigm for energy-aware artificial intelligence (AI), offering sparse, low-latency visual signals that reduce redundant data processing and support sustainable edge computing. However, the asynchronous and noise-prone nature of event streams creates challenges for conventional deep learning models, which are often too computationally intensive for low-power embedded platforms. This work presents a compact and configurable event-driven autoencoder that efficiently compresses neuromorphic data while preserving essential spatiotemporal structure for downstream inference. The architecture integrates lightweight convolutional encoding with robust performance under adaptive event thresholding and a minimal classifier head, enabling substantial reductions in computational cost without degrading recognition fidelity. Extensive evaluations on the Smart Event Face
Dataset (SEFD) and Event-Based Crossing
Dataset (EBCD) show that the proposed framework achieves competitive or superior accuracy compared to YOLOv9 while requiring up to 35.6$\times$ fewer parameters. To assess real-world sustainability, the model is deployed on resource-constrained hardware: a Raspberry Pi 4B and a NVIDIA Jetson Nano. On NVIDIA Jetson Nano, it delivers real-time throughput of 44.8 FPS. On a Raspberry Pi 4B CPU, the 50% autoencoder classifier consumes 16.19 J for the evaluated inference workload, corresponding to approximately 726.3$\times$ lower energy consumption than YOLOv9 under the same evaluation protocol. These results demonstrate the potential of compact event-driven models to advance environmentally conscious, low-power AI systems for high-speed perception in autonomous, mobile, and embedded computing environments.
Read the full abstract ▾
All authors Riadul Islam, Joey Mule, Dhandeep Challagundla, Shahmir Rizvi, Sean Carson, Rachit Saini
Computer Vision
Learning to Look Again: Loss-Gap Supervision for Free-form Crop Routing in Vision-Language Models
Jinchang Zhu, Rong Fu, Yi Ding et al. (+3 more)
Vision-
language models (VLMs) fail many detail-centric questions for a concrete reason: the answer is visible in the image, yet lost after the image is compressed into a low-resolution global view. Allocating more visual
tokens to every query improves some OCR and document cases, but it spends computation indiscriminately and can disturb tasks that rely on global context. We propose GapSight, a framework for learning visual re-reading: a VLM first takes a global glance, then selectively returns to a free-form region when the question calls for local evidence. The supervision comes from the target model's own failure signal. Offline, we compare answer loss or multiple-choice option margin under a global-only view and candidate crop-augmented views; crops that improve the target answer become model-specific review labels. A lightweight free-form crop router distills these labels into a one-shot inference policy that predicts whether to review, expected utility, and a continuous crop box from the global state. Across LLaVA-1.5-7B, InternVL2.5-8B, and Qwen2-VL-2B-Instruct, GapSight improves the Base no-zoom baseline on six
benchmarks spanning OCR, documents, charts, infographics, VStarBench, and MME-RealWorld-Lite. On InternVL2.5-8B, GapSight raises the six-
benchmark average from 52.25 to 64.29, above CropVLM (57.16), ViCrop (55.84), and ZoomRefine (54.43). Mechanism analyses show that the router rescues concrete wrong answers, adapts its action rate by task, and forms a favorable
token-performance profile. These results position loss-gap supervision as a practical route to teaching VLMs when and where to look again.
Read the full abstract ▾
All authors Jinchang Zhu, Rong Fu, Yi Ding, Chenghao Wu, Ying Liu, Menglin Yang
Computer Vision · Cross-listed from cs.AI
What Does CLIP Learn for Regional Geolocalization? Probing Visual Cues and Scene Configuration After Adaptation
Changyu Lee, Yeonsoo Park, Abdullah Alfarrarjeh et al. (+1 more)
Large collections of street-view imagery provide rich visual information about urban environments, but extracting fine-grained geographic information from such data remains challenging. In particular, fine-grained regional geolocalization is challenging because nearby areas often share coarse geographic cues. We study regional geolocalization within a metropolitan area and ask whether pretrained CLIP features are sufficient for regional discrimination, and what visual information supports performance after adaptation. Using 9,085 street-view images from eight Greater Los Angeles regions, we compare zero-shot CLIP, frozen-encoder readouts, partial encoder updating, Low-Rank Adaptation (LoRA), and full fine-tuning. Frozen readouts remain near the 39.03% zero-shot accuracy, whereas encoder adaptation achieves 75.94-82.10%. Full fine-tuning also reduces the mean distance to the predicted region center from 12.30 km to 3.86 km. We probe these gains through semantic cue removal, appearance reduction using edge maps and blur, and scene-configuration disruption using patch scrambling. Adapted models achieve higher edge and blur accuracy and switch 42.92-45.56% of predictions after scrambling, compared with 10.79-14.60% for frozen methods. However, adaptation does not improve the fraction of performance retained after appearance reduction, while vegetation and sky remain influential. A Caltech101 control further shows that scrambling sensitivity is not unique to geolocalization. Overall, encoder adaptation substantially improves nearby-region discrimination and is associated with greater sensitivity to intact scene configuration, without evidence that coarse structure alone becomes sufficient for prediction. These conclusions concern viewpoint variation near known locations rather than geographically disjoint generalization.
Read the full abstract ▾
All authors Changyu Lee, Yeonsoo Park, Abdullah Alfarrarjeh, Seon Ho Kim
Computer Vision · Cross-listed from cs.LG
How Architecture and Training Affect TPC Representations Across Experiments
Tyler Wheeler, Michelle P. Kuchera, Raghuram Ramanujan et al. (+12 more)
Deep-learning efforts have increasingly shifted toward foundation model approaches. In experimental physics, this allows models and learned representations to be reused beyond the experiments in which they were developed. This work evaluates the reusability of representations across experiments and detector systems using probes on frozen encoders. These probes reveal task-relevant structure before downstream adaptation, complementing fine-tuning. Together with random-weight controls, they distinguish contributions from architecture and encoder training that downstream performance alone cannot resolve. Time projection chamber (TPC) data provide a useful testbed because events from TPC systems can be represented as variable-length sparse tensors, while detector geometries, event
topologies, and scientific tasks can differ substantially. We investigate whether fixed-dimensional TPC event representations can be reused across classification tasks, experiments, and detector systems. Sparse ResNet and PointNet-style encoders produce 512-dimensional
embeddings for four
datasets from the GADGET II TPC and AT-TPC. Randomly initialized encoders isolate the contribution from architecture before supervised training. We then train each encoder on a classification task, freeze its parameters, and train a linear or nonlinear probe for each downstream task. We find that this architecture-induced structure remains useful across experiments and detector systems. The randomly initialized PointNet-style representation is highly informative on several tasks. The two architectures organize their
embedding spaces differently, but neither exhibits a large, systematic loss of utility cross-detector. These results show that architecture is a major source of task-relevant structure in TPC
embeddings and should be treated explicitly when assessing representation learning and developing reusable detector models.
Read the full abstract ▾
All authors Tyler Wheeler, Michelle P. Kuchera, Raghuram Ramanujan, William Sieland, Ryan Krupp, Yassid Ayyad, Daniel Bazin, Connor L. Cross, Hoi Yan Ian Heung, Andrew J. Jones, Ruchi Mahajan, Saiprasad Ravishankar, Pranjal Singh, Benjamin Votaw, Chris Wrede
Computer Vision
Fidelity-Diversity-Consistency (FDC): Data Pruning for Remote Sensing Change Detection
Dongyao Zhu and Ranga Raju Vatsavai
Despite the success of data
pruning (DP) in reducing training data sizes and improving downstream model performance in classification and
segmentation tasks, its potential in remote sensing change detection remains unexplored. For the first time, we
benchmark six representative DP methods across building- and forest-change
datasets, CNN- and
transformer-based models, and three
pruning budgets, and show that existing baselines yield no reliable advantage over random selection. Notably, even the strongest evaluated baseline, Feature Diversity, is matched or exceeded by $\sim$33% of randomly sampled subsets. To understand the underlying mechanism, we conduct a systematic regression study over 540 randomly sampled data subsets, characterizing each with four descriptors covering label statistics, image diversity, and feature-space geometry. Random Forest models show that change distribution fidelity is the most prominent factor in determining the quality of change detection data subsets, a property absent from the existing
pruning literature. Our analyses further show that pixel-wise image diversity and label-feature consistency are secondary factors. We translate these findings into Fidelity-Diversity-Consistency (FDC), a simple two-stage
pruning method that shows consistent improvements over existing baselines across change detection
benchmarks and backbones, especially at lower
pruning ratios. Code is available at \href{https://github.com/ddydyd32/fidelity-diversity-consistency}{https://github.com/ddydyd32/fidelity-diversity-consistency}.
Read the full abstract ▾
Computer Vision
Calibrate What You SHIP: Post-Selection Risk Control for Verifier-Guided Text-to-Image Generation
Xuanhua Yin, Shunqi Mao, Wei Guo et al. (+2 more)
Verifier-guided text-to-image systems increasingly use test-time search to select, refine, or stop among multiple candidates, yet release thresholds are often calibrated on individual images. This creates a candidate-to-policy calibration mismatch: search changes both which
prompts receive an output and which candidate is released, so candidate-level risk control need not imply control of released-output risk. We formalize this estimand shift through
prompt reweighting and within-
prompt selection, and introduce SHIP, Selection-aware Held-out calibration of Inference Policies. SHIP runs or replays the complete deployed policy on held-out
prompts, evaluates the image it actually releases using an independent target judge, and selects the most permissive threshold whose risk upper bound satisfies a prescribed budget. For replayable policies with a prespecified threshold grid, simultaneous confidence control provides finite-sample validity. Experiments across fixed, sequential, and adaptive T2I inference procedures show that policy-level calibration recovers lower-risk operating points while exposing policy-dependent tradeoffs among risk, coverage, and compute. On GenEval2 with FLUX at N=16, a pooled-candidate threshold yields released risk 0.310, whereas SHIP reduces it to 0.162. Across 200 cached-stream splits, the fixed-grid certificate has no target crossing. Reliable inference-time scaling therefore requires calibrating the output distribution induced by the complete deployed policy.
Read the full abstract ▾
All authors Xuanhua Yin, Shunqi Mao, Wei Guo, Chuanzhi Xu, Weidong Cai
Computer Vision
The Plan, Not the Decoder: Diagnosing and Repairing Compositional Failure in Reasoning-Augmented Text-to-Image Generation
Ashritha Gonuguntla
Reasoning-augmented text-to-image models such as GoT-R1 emit an explicit textual plan - object names, attributes, and bounding boxes - before generating image
tokens. When such a model fails a compositional
prompt, is the plan wrong, or is the plan right and the decoder unfaithful? Because the plan is machine-readable it can be edited before decoding, which makes the two separable. We first validate the ruler. Swapping the two bounding boxes inside the model's own chain demonstrably flips the generated layout: detector-based accuracy falls 0.75 -> 0.48 (p<1e-3), while a widely used VQA-based spatial metric rises. A five-rater human study agrees with the detector on 81% of items and with the VQA judge on 57%. All spatial results therefore use
geometric scoring. Under sound measurement the decoder is a faithful executor: 94% of generated layouts realize the planned relation, and object-box binding survives reordering of the plan's object segments. The planner is the bottleneck. It writes wrong relations for phrasing-dependent reasons - 98% accuracy on "left" against 54% on "right" for semantically identical layouts, a raster-order bias we isolate with a mention-order control - and cluttered geometry that the decoder faithfully reproduces. Editing the plan therefore fixes the image without retraining: symbolic verification with resampling gives +5.0 points (p<1e-3), minimal in-place repair +6.0 (p=.02), rewriting only box geometry +10.7 (p<1e-4), and replacing the plan outright +13.3 (p=1e-4). Gains are indifferent to the plan's prose style and to its likelihood under the planner, but not to its geometry. Modular planner-decoder designs are therefore viable, provided the plan is internally consistent: box-text contradictions induce object duplication and identity fusion. We release the plan-fidelity evaluation protocol, all plans, and 12k generated images.
Read the full abstract ▾
Computer Vision
StereoDiffuer: Diffusion-based Progressive Geometry Modeling with Saliency Attention Perception for Stereo Matching
Bohan Li
With the advance of deep neural networks, the quality of disparity maps obtained through stereo matching has steadily improved. However, existing stereo matching methods still struggle to preserve fine-grained
geometric details, resulting in blurred edges and over-smoothed predictions in challenging regions. To address these limitations, we propose StereoDiffuer, an iterative
diffusion-based stereo matching framework that explicitly models
geometric details and progressively refines disparity estimates. The framework incorporates a Saliency
Attention Perception (SAP) module to extract salient
geometric cues, including object boundaries, thin structures, and sharp edges. Confidence-guided SAP features are combined with the initial disparity estimate to condition an iterative denoising
diffusion process, which corrects residual disparity errors and restores
geometric details suppressed during cost-volume regularization and upsampling. Experimental results on the Scene Flow and KITTI
benchmarks demonstrate the effectiveness of the proposed framework and its competitive performance relative to the compared stereo matching methods.
Read the full abstract ▾
Computer Vision
Emotion Intensity Matters: Generating Realistic Expressions in Virtual Humans with CVAEs
Vitor Miguel Xavier Peres, Lara Volpato, Gabriel Ferri Scnheider et al. (+1 more)
Generating expressive facial behavior in virtual humans (VHs) remains a central challenge in affective computing and character animation. This paper presents a novel approach based on Conditional Variational Autoencoders (CVAEs), trained on real human facial expression data, to synthesize controllable emotional expressions at varying intensities. Using a
dataset comprising six basic
emotions represented at two intensity levels (low and high), we train a CVAE model to generate synthetic facial expression data while preserving semantic consistency with real human expressions. Despite the limited amount of training data (only 7,680 facial expression samples), the proposed approach learns meaningful latent representations and generates coherent emotional variations. Our method enables control over emotional intensity, making it suitable for animating virtual characters without requiring actor performances or manual artistic intervention. Our research aimed to evaluate whether the method (CVAE) preserves the characteristics associated with the different intensity levels present in the
dataset. Results show that the proposed model preserves key expressive characteristics across intensity levels while supporting generalization across emotional intensity levels, contributing to the creation of emotionally expressive virtual characters from relatively small
datasets.
Read the full abstract ▾
All authors Vitor Miguel Xavier Peres, Lara Volpato, Gabriel Ferri Scnheider, Soraia Raupp Musse