Computer Vision
Beyond Fixed Luminance: Towards Panchromatic and Orthochromatic Image Colorization
Swarnim Maheshwari, Syed Imam Ali and Vineeth N. Balasubramanian
Most image colorization systems operate in $Lab$ space by predicting chroma ($ab$) while preserving an input-derived luminance channel ($L$). While effective on standard
benchmarks, this fixed-luminance design restricts brightness changes and becomes unreliable when grayscale formation deviates from natural-image luminance, as in historical orthochromatic photography. We propose a luminance-agnostic colorization framework that formulates colorization as full-RGB image editing using a foundation image-editing model. To bridge modern panchromatic and historical orthochromatic conditions, we introduce a mixed grayscale objective that trains the model under both standard luminance grayscale and a red-insensitive grayscale formation. Experiments on COCO, ImageNet, and a multi-instance
benchmark show that our method is competitive on standard grayscale inputs and substantially more robust under orthochromatic inputs, with qualitative comparisons and a human study indicating fewer visible color artifacts.
Read the full abstract ▾
Computer Vision
Lancheng Gao, Ziheng Jia, Shengyan Li et al. (+4 more)
Understanding both expressed and evoked
emotions is critical for
multimodal large
language models (MLLMs) to achieve comprehensive affect-aware interactions. However, existing
benchmarks typically examine expressed and evoked
emotions in isolation or are constrained to coarse-grained and incomplete affective characterizations. To bridge this gap, we introduce E$^3$mo-Bench, a scalable
benchmark comprising $12{,}314$ question-answer pairs across $2{,}524$
videos with predefined affective perspectives. It evaluates evoked and expressed
emotion understanding via $3$ complementary tasks:
emotion perception, open-vocabulary recognition, and valence-arousal-dominance (VAD) assessment. To efficiently scale reliable continuous annotations, we propose
Bayesian Pairwise
Alignment, which aggregates sparse, low-burden pairwise judgments into anchor-referenced VAD estimates. Furthermore, we develop E$^3$mo-Score, a training-free
agent that aggregates complementary judgments from a five-model committee to improve VAD estimation. Extensive experiments validate the effectiveness of our framework and expose a pronounced performance skew between evoked and expressed
emotion paradigms. These findings, coupled with MLLMs' persistent deficits in fine-grained recognition and dimensional assessment, chart a clear course for advancing
multimodal emotional intelligence.
Read the full abstract ▾
All authors Lancheng Gao, Ziheng Jia, Shengyan Li, Zixuan Xing, Jiarui Wang, Huiyu Duan, Xiongkuo Min
Computer Vision
MVTrack: Ultrafast Appearance-Free Moving Object Tracking from Compressed Bitstreams
Iñaki Erregue, Kamal Nasrollahi and Sergio Escalera
Deploying modern
video trackers at scale is bottlenecked by the computational cost of RGB-based object detectors. To this end, we present MVTrack, an ultrafast tracker for moving objects that operates directly on H.264 bitstreams. MVTrack combines MVDet, a lightweight detector for motion vector fields, with MVLink, a minimalist kinematic association module. On VIRAT, MVTrack outperforms YOLO26n while using 60$\times$ fewer parameters, requiring 40$\times$ fewer FLOPs, and reducing CPU latency by 8.6$\times$. These results demonstrate that compressed
video data alone can enable accurate and scalable surveillance tracking, thereby bypassing the need for pixel reconstruction.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Compositional Benchmark Synthesis for Hierarchical Human Action Recognition
Farnaz Soleimani, Abdelghani Chibani, Yacine Amirat et al. (+1 more)
Recognizing human behavior across levels of abstraction, from atomic actions to long-horizon intentions, requires data annotated along a semantic hierarchy. Large corpora provide isolated, atomically labeled clips without temporal composition, whereas recorded composite-activity corpora offer shallow, domain-narrow, fixedhierarchies. A
benchmark-generation and evaluation frameworkis proposed that synthesizes a four-level hierarchical-intention
benchmark, spanning actions, activities, low-level intentions (LLIs), and high-level intentions (HLIs), from a flat single-label action corpus while retaining real pre-extracted features at the action level. Episodes are assembled by a transition model under a subject-consistency constraint, and a coverage-aware sampler reduces the subject usage Gini from 0.566 to 0.248. Synthesizing such a
benchmark raises a circular-supervision risk that recorded
datasets avoid: if the rules generating the episodes also govern the evaluation, models can succeed by recovering the generator rather than through genuine
reasoning. Validity is addressed by design, holding sequence-generation rules disjoint from the first-order-logic rules used at evaluation. The instantiation yields 15,002 episodes. Four reference baselines from different model families characterize difficulty, not as recognition methods. A compositional held-out gap of 0.13 to 0.17 macro-F1 appears across all baselines, including a graph-aware model that recognizes best yet does not close the gap, indicating a structural property of the
benchmark rather than a model artifact. A logic-free baseline still violates the held-out semantic rules above their intrinsic data rate, and the order-destroying control changes macro-F1 within seed variation, serving as a generator-consistency check. Theontology, transition model, and generator are released so the
benchmark can beregenerated and extended.
Read the full abstract ▾
All authors Farnaz Soleimani, Abdelghani Chibani, Yacine Amirat, Ghazaleh Khodabandelou
Computer Vision
FADE: From Passive Verification to Active Discovery in Counterfactual Video Understanding
Fufangchen Zhao, Jinhu Fu, Jiachen Lei et al. (+3 more)
Counterfactual
video understanding evaluates whether models grasp physical and commonsense regularities. However, existing multiple-choice question (MCQ)
benchmarks inadvertently leak target events through their questions and candidate options. This reduces the core challenge from active discovery to text-guided verification. In this paper, we present FADE, an effective training framework for counterfactual discovery and explanation. Our method is built on an evidence-first, two-stage training paradigm. First, evidence-internalized supervised fine-tuning grounds the model's predictions in decisive visual anomalies. Second, we apply a fading-anchor reinforcement learning strategy that progressively removes textual guidance, compelling the model to independently discover and explain evidence. To rigorously evaluate this capability, we also introduce an effective pipeline that converts existing MCQ
datasets into aligned MCQ, open-ended question answering (OQA), and captioning tasks without requiring additional data curation. Our simple approach yields strong results. Using Qwen3-VL-8B as the baseline, FADE achieves state-of-the-art strict paired scores across all three tasks on DualityVidQA-test and IPV-Bench, outperforming GPT-5.6. In specific, when transitioning from constrained MCQs to unconstrained OQA and captioning, our model demonstrates remarkable robustness. Its performance retention is 90.4% and 67.4% on DualityVidQA-test-substantially higher than the 48.1% and 30.7% retained by GPT-5.6. We hope this simple framework can serve as a solid baseline for future research in unconstrained counterfactual
video understanding.
Read the full abstract ▾
All authors Fufangchen Zhao, Jinhu Fu, Jiachen Lei, Jiahong Wu, Xiangxiang Chu, Danfeng Yan
Computer Vision
Where To Look? : Causal Tracing of Vision Encoders in VLM
Naren Kumar S, Tirth Bhatt and Mayank Singh
Vision-
language models can describe an image with remarkable accuracy, yet a more fundamental question remains unanswered: what visual information actually drives their answers? In this work, we investigate this question through causal tracing, and we observe that highly causal
vision tokens often lie outside the target region. Extending the analysis to larger
vision-
language models reveals a similar pattern across models and corruption settings, suggesting that strong
multimodal performance does not necessarily imply spatially localized causal representations. We further investigate: can these models preserve visual structure when appearance cues are removed? and find that visual cues are exploited to understand visual structures. Together, our experiments expose a gap between seeing, using, and
reasoning over visual structure, and provide a causal framework for studying how visual information is transformed, preserved, and ultimately used by modern
vision-
language models.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Embodied Multimodal Grounding for Open-Vocabulary Mobile Manipulation via Semantic 3D Gaussian Splatting
Huosen Ou, Dongni Song, Yuncong Wang et al. (+2 more)
Embodied mobile manipulation requires
language, visual observations, three-dimensional scene structure, and action feasibility to be aligned before execution. We study open-vocabulary target grounding with few-shot manipulation in local household workspaces and present an
embodied multimodal grounding framework that integrates active multi-view Semantic 3D Gaussian Splatting (Semantic-3DGS), reachability-aware base positioning, and a
diffusion-based
vision-
language-action policy. A task-driven local Semantic-3DGS serves as a shared interface across active sensing,
language-conditioned 3D localization, obstacle-aware scene
reasoning, base preparation, and semantic conditioning of the action model. To preserve pretrained action priors, the 3D semantic cues are injected only into the late action-expert blocks. In expanded 50-trial real-
robot evaluations against representative
vision-
language-action (VLA) approaches, the full system achieves 60% long-horizon success compared with 40% for PointVLA and 28% for DexVLA, and reaches 74% success in heavily cluttered manipulation compared with 52% for the single-view variant and 46% for PointVLA. It also maintains 75% success under a 75 cm height shift and eliminates photo-induced false grasps. These results indicate that explicit, refreshable 3D semantic grounding can improve robustness under clutter, occlusion, viewpoint variation, and embodiment constraints.
Read the full abstract ▾
All authors Huosen Ou, Dongni Song, Yuncong Wang, Tao Zhou, Yiding Ji
Computer Vision
Beyond Pixels: From Video Priors to 4D Worlds
Zihao Liu, Xiaolong Shen, Zhenglin Zhou et al. (+2 more)
4D generation synthesizes dynamic 3D scenes from conditions such as text or images. Existing methods either reconstruct generated RGB
videos with a separate 4D model or adapt a particular
video generator to predict geometry directly. The former suffers from distribution mismatch and error propagation, whereas the latter ties 4D prediction to a specific generator and may require retraining when the generator or conditioning regime changes. We ask whether the final denoised latents of
video models that share a variational autoencoder (VAE) can instead provide a reusable interface to explicit 4D prediction. Building on this insight, we introduce direct latent-to-4D generation and instantiate it as Latent-to-4D, which bypasses RGB by aligning a
video latent with the
token grid of a pretrained 4D decoder and refining it through frame-wise and global spatiotemporal
attention. Trained on roughly 1K existing reconstruction clips, a single checkpoint transfers unchanged across multiple
video diffusion transformers within the same VAE family. On Text4D-200 and I4D-200, Latent-to-4D surpasses matched same-latent Wan+4RC cascades in projection-based DINO-F1 by 2.88–3.45 and 5.81 points, respectively, while also being preferred by human raters for geometry, temporal stability, and overall quality.
Read the full abstract ▾
All authors Zihao Liu, Xiaolong Shen, Zhenglin Zhou, Ruijie Quan, Yi Yang
Computer Vision
Rethinking LLM Verification: Evidence Structure, Uncertainty, and Selective Refinement
Uma Ranjan, Kunal Tilaganji, Aditya Koul et al. (+9 more)
Large
language models (LLMs) often rely on shortcuts rather than systematic
reasoning, raising safety concerns in medical applications. Allowing models to abstain when uncertain improves reliability but introduces a coverage accuracy tradeoff. We propose a two-stage framework for medical hypothesis verification in multiple-choice settings that manages this tradeoff through targeted ontology grounding, applied only when the model abstains. We show that abstention is not random but reflects genuine uncertainty, with abstained predictions associated with lower confidence. Across two frontier models (GPT-5.5, accessed via the Azure OpenAI API, and DeepSeek-R1), the proposed framework improves question-level accuracy by 9.6 percentage points (82.9% to 92.5%) and hypothesis-level accuracy by 4.2 percentage points (92.0% to 96.2%). Our experiments conducted on MedReason and MedQA show that abstention can be repurposed as a control signal for selective
reasoning refinement, achieving knowledge-graph-level performance without explicit knowledge graph construction.
Read the full abstract ▾
All authors Uma Ranjan, Kunal Tilaganji, Aditya Koul, Anurag Mahipal, Dashpreet Singh, Hriday Rana, Manan Jain, Sidharth Gupta, Ajo Babu George, Vineeth Balasubramanian, Nagarajan Natarajan, Amit Sharma
Computer Vision
InterPruner: Interactive Structured Pruning via Taylor-Implicit Criterion and Language-Prior Modulator for Multimodal Object Detection
Qi Ming, Zihan Yang, Shaoguang Huang et al. (+6 more)
Multimodal object detection proves effective in remote sensing, especially the RGB-Infrared paradigm. The parallel feature extractors provide rich
multimodal information for robust detection, yet introduce substantial channel redundancy and computational overhead. Existing
pruning methods can reduce channel redundancy, but they are designed for unimodal backbones, overlooking cross-modal interactions and dynamic scene-wise redundancy. In this paper, we propose InterPruner, the first interactive structured channel
pruning framework for RGB-infrared object detectors. Specifically, we first derive a Taylor-Implicit Criterion(TIC) to quantify channel importance via high-order Taylor expansion and the implicit function
theorem. Then, a Modality Interaction Redundancy Analyzer (MIRA) identifies redundant channels via mutual compensability assessment. Finally, a Scene-Prior Channel Anchor (SPCA) uses
language priors as semantic anchors to measure channel-scene relevance for dynamic channel importance estimation. Cross-modality channel
pruning for RGB-Infrared detection is yet unexplored. Extensive experiments on RGB-infrared object detection
dataset demonstrate that InterPruner maintains high performance with negligible degradation. Specifically, it even achieves a 0.6% mAP increase on the FLIR
dataset when
pruning 50% of the channels. Code will be available on GitHub to facilitate future work.
Read the full abstract ▾
All authors Qi Ming, Zihan Yang, Shaoguang Huang, Si Sun, Hanqing Zhang, Nanqing Liu, Jiahui Lv, Juan Fang, Aleksandra Pizurica
Computer Vision
Grid-Preserving Knowledge Distillation: Transferring Convolutional Inductive Bias to Vision Transformers under Data Scarcity
Junyong Choi, Cheolhyeon Park and Jaehoon Cho
Vision Transformers demonstrate remarkable global modeling capacity but often underperform in data-scarce regimes. Distilling convolutional inductive biases from a CNN teacher provides an effective remedy while leaving the deployed model unchanged. However, general-purpose feature
distillation transfers little in this setting. In CNN-to-CNN
distillation, pooling, flattening, and logit-space projections remove the spatial grid that encodes locality and translation equivariance. Unlike a convolutional student, a ViT cannot readily reconstruct this structure on its own. In this paper, we propose iBKD, a
distillation framework that preserves the spatial grid throughout the entire transfer process. Its core module, the Inductive Bias
Attention Module, aggregates features from all student layers onto the teacher's grid using learned weights. It then enhances structural cues through channel and deformable spatial
attention and injects them via convolutional cross-
attention operating directly between spatial grids rather than
token sets. The module is used only during training, leaving the deployed model as an unmodified ViT with no inference overhead. Across seven
Transformer backbones and six data-scarce
benchmarks, iBKD consistently outperforms both locality-guidance methods and general knowledge
distillation baselines, with its advantage increasing as the amount of training data decreases.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Ex-Omni-2D: Expressive Omni-Modal Dialogue Models with Native Visual Presence
Haoyu Zhang, Zhipeng Li, Xiaoying Tang et al. (+2 more)
Omni-modal dialogue models can understand
multimodal inputs and synthesize spoken replies, but a spoken answer still leaves the
agent visually absent. We introduce Ex-Omni-2D, a framework that answers a
multimodal query with coordinated text, personalized
speech, and reference-conditioned
video. The dialogue model first writes a structured Visual Thought Plan (VTP) for scene,
emotion, and motion, then generates the response text and multi-codebook
speech units. These
speech units are decoded into audio and aligned with
video frames, giving the
speech and avatar modules a common timing signal while allowing them to learn from different data sources. The
video module is trained as a full-sequence Teacher conditioned on reference appearance, VTP semantics, and frame-aligned
speech units. We further explore to distill it into a few-step block-causal Streaming Student; its Prefix Streaming mechanism carries the previous clean latent into the next chunk and is analyzed as a partial mitigation for late-chunk subject drift. At $400\times720$/$720\times400$, the four-step four-GPU Student provides incremental output with lower startup latency than the full-sequence Teacher.
Read the full abstract ▾
All authors Haoyu Zhang, Zhipeng Li, Xiaoying Tang, Tianshu Yu, Yiwen Guo
Computer Vision
Compact Feed-Forward 3D Gaussians via Saliency-Guided Primitive Merging
Tim-Felix Faasch, Jochen Kall and Cyrill Stachniss
3D scene reconstruction, modeling, and rendering are highly relevant for numerous tasks, and 3D Gaussian splatting has become a standard choice in this context. Its feed-forward variants provide fast reconstruction from sparse input views but often produce per-pixel primitives, leading to highly redundant and thus inefficient representations. We present a structure-aware merging pipeline that takes per-pixel primitives from any feed-forward method and consolidates them into a compact, content-adaptive Gaussian set while largely retaining visual quality at just $\frac{1}{20}^\text{th}$ of the Gaussians of a per-pixel method. We group spatially coherent Gaussians of similar appearance into variable-size clusters via adaptive superpixel
segmentation guided by a saliency map, which allocates fine segments to textured regions and coarse segments to homogeneous areas. We compress each cluster into a compact latent representation through a learned encoder, then match and consolidate representations across views based on
geometric overlap and feature similarity via a learned merger. A level-of-detail decoder then produces the final Gaussians at a controllable resolution, enabling a flexible quality-efficiency trade-off at inference. As a post-processing module, the pipeline is backbone-agnostic, leveraging the strengths of existing feed-forward methods. This leads to better and more robust quality than achieved by previous approaches that target a reduction in primitive count, while providing a highly compact representation, that can be rendered efficiently.
Read the full abstract ▾
Computer Vision
Self-Geometry: GT-Free and Plug-and-Play Test-Time Adaptation for Geometrically Consistent 3D Vision Foundation Models
Seokhyun Youn, Dahyeon Kye, Sung-Ho Bae et al. (+1 more)
Recent
Vision Foundation Models (VFMs) predict depth, camera pose, and pointmap in a single forward pass without per-scene optimization, achieving strong generalization. However, enforcing explicit multi-view
geometric consistency, e.g., through bundle adjustment, is computationally costly and is thus not imposed during VFM pretraining, so such inconsistency can arise. To address this, implicit self-consistency derived from model outputs (e.g., pointmaps, features), though enforced at test-time in prior work, delivers inherently limited performance gain, especially on scenes where the pretrained VFM is highly inaccurate. In contrast to this implicit signal, we propose Self-Geometry, a plug-and-play test-time adaptation pipeline that directly imposes explicit multi-view
geometric constraints using 2D pixel correspondences as pseudo ground-truth. Our proposed Self-Geometry consists of
Geometric Disentanglement Optimization, which combines Multi-View Consistency and Epipolar Consistency losses with Gradient Disentanglement to prevent gradient conflict; Frame Angular-Neighbor, a view sampler based on SO(3) geodesic distances for lightly imposing these constraints; and Lightweight TTA, which adapts VFMs via LoRA. Our method achieves consistent improvements in both pose and geometry estimation across six VFMs (VGGT, $\pi^3$, DA3-
Giant/Large/Base/Small) and four
benchmarks (7Scenes, ETH3D, ScanNet++, HiRoom).
Read the full abstract ▾
All authors Seokhyun Youn, Dahyeon Kye, Sung-Ho Bae, Jihyong Oh
Computer Vision
MMArt: A Multi-Perspective Multimodal Dataset for Visual Art Understanding
Shuai Wang, Wangyuan Ding, Yixian Shen et al. (+5 more)
Recent
vision-
language models demonstrate impressive general visual understanding, yet their art interpretation remains shallow: they describe surface content but struggle with formal analysis, grounded historical interpretation, or affective characterization. We argue this is not only a model but also a
dataset limitation. Existing art
datasets are single perspective resources, where no
dataset provides narrative, formal, emotional, and historical perspectives simultaneously for the same artworks. We introduce MMArt, a large-scale
dataset of 74,234 WikiArt paintings, each annotated with four independently annotated perspectives plus a harmonized unified caption, produced by specialized
vision-
language models or human annotation and validated through complementary quality evaluations. Two complementarity analyses establish that perspectives encode genuinely distinct information. A generative analysis shows that formal analysis descriptions best preserve compositional style, and historical descriptions carry strong affective signal in reconstructed images. A discriminative
retrieval analysis reveals task-asymmetry: narrative descriptions drive
retrieval (R@1 = 44.0%), while formal descriptions, strongest for reconstruction, are nearly nondiscriminative at
retrieval scale (R@1 = 7.8%). Leave-one-out analysis further confirms that historical descriptions are the least replaceable perspective across both tasks. Together, the two analyses establish that no single perspective suffices for all tasks, directly motivating MMArt multi-perspective design. The
dataset, code, and additional information are available at https://shuaiwang97.github.io/MMArt/.
Read the full abstract ▾
All authors Shuai Wang, Wangyuan Ding, Yixian Shen, Jia-Hong Huang, Stevan Rudinac, Monika Kackovic, Nachoem Wijnberg, Marcel Worring
Computer Vision
Embedding Rotation Invariance for Provable Multi-Oriented Scene Text Recognition
Zhibin Ma, Pengwen Dai, Yi Liu et al. (+3 more)
Multi-oriented text is ubiquitous in real-world scenes and remains a major challenge for scene text recognition (STR). Existing rotation-aware methods explicitly estimate text orientation. However, due to the lack of theoretical guarantees, they are prone to error accumulation, increased computational cost, and strong reliance on data. In this work, we incorporate rotation invariance into the STR framework to address these limitations. Specifically, we adopt an encoder-decoder architecture,
embedding rotation equivariance in the encoder and rotation invariance in the decoder to construct a fully rotation-invariant network. On the decoder side, we first identify and prove the rotation-invariant property of the cross-
attention mechanism and use it to formulate a rotation-invariant text decoder that maps visual features to output text in a rotation-invariant manner. On the encoder side, we propose a rotation-equivariant local-global extraction network that integrates deep equivariant convolutions with self-
attention, enabling rotation-equivariant feature extraction while modeling inter-character dependencies and preserving fine-grained visual details. By integrating the encoder and decoder, we obtain an end-to-end Rotation-Invariant Scene Text Recognition network (RISTER). RISTER provides rotation invariance with theoretical guarantees, enhancing robustness on multi-oriented samples without introducing additional inference computation or relying on data-driven orientation correction. Experiments show that RISTER achieves state-of-the-art performance on both standard and multi-oriented
benchmarks, surpassing the second-best model by 4.0 percent in accuracy on the general multi-oriented
dataset.
Read the full abstract ▾
All authors Zhibin Ma, Pengwen Dai, Yi Liu, Xugong Qin, Chenyun Yu, Xiaochun Cao
Computer Vision
Visual Geometry Foundation-Aware Gaussians for Single-Frame Surround-View Driving Reconstruction
Junhong Lin, Jinlong Wang, Xianda Guo et al. (+6 more)
Single-frame surround-view reconstruction faces severe
geometric instability and rendering artifacts due to minimal inter-camera overlap. While existing methods rely on complex decoders or auxiliary cues, they remain bottlenecked by the weak
geometric capacity of upstream features. We argue that leveraging pretrained visual geometry priors strengthens upstream representations and alleviates the
geometric ambiguity in sparse surround views. To this end, we propose VGGD, a visual geometry foundation-aware 3D Gaussian Splatting framework for feed-forward surround-view driving reconstruction, which shifts
geometric modeling to the frontend and adapts foundation priors to the driving camera setting. First, VGGD leverages VGGT to provide transferable multi-view
geometric prior
tokens. Next, we introduce a Dual-Path Neck to decouple geometry-consistent and appearance-aware representations, improving appearance completion in weakly observed regions. We further apply Scale Warmup to stabilize early geometry learning and suppress scale drift under ego-pose changes. Finally, we use a hybrid pixel–volume Gaussian decoder to produce a renderable 3D Gaussian scene for novel-view synthesis. Experiments on the nuScenes single-frame
benchmark show that VGGD achieves the best overall rendering quality among the compared methods and improves relative
geometric consistency.
Read the full abstract ▾
All authors Junhong Lin, Jinlong Wang, Xianda Guo, Yanlun Peng, Wei Zheng, Guoqing Liu, Hanli Wang, Tiesong Zhao, Wei Gao
Computer Vision
Bridging Severe Cross-Modal Misalignment: End-to-End Visible-Infrared Object Detection via Explicit Feature-Domain Affine Registration
Qi Ming, Yuyang Wang, Mingjing Zhao et al. (+7 more)
Visible-infrared object detection relies on complementary RGB and thermal cues, but its performance is often degraded by cross-modal spatial misalignment. Most existing methods rely on implicit feature adaptation to handle weakly misaligned scenarios, while large-offset
geometric discrepancies remain insufficiently addressed. In this paper, we propose a Joint Feature-domain Registration and Detection network (JFRDet), an end-to-end visible-infrared oriented object detector tailored for severely cross-modal
geometric discrepancies. JFRDet introduces a Cross-Modal Affine
Alignment (CMAA) module to estimate an image-level affine transformation for explicit multi-level feature
alignment. Note that illumination changes directly affect the reliability of RGB cues, an Illumination-Guided Complementary Fusion (IGCF) module adaptively exploits modality reliability under varying illumination conditions for cross-modal fusion. Then, an
Alignment Quality-Consistency Gating (AQCG) strategy stabilizes joint optimization by modulating detection supervision according to
alignment reliability and gradient consistency. We further construct DroneVehicle Misaligned (DVMA), a
benchmark for evaluating visible-infrared oriented object detection under severe cross-modal
geometric misalignment. The proposed JFRDet achieves 69.7% $\mathrm{mAP}_{50}$ on DVMA, which represents state-of-the-art (SOTA) performance. The code and
dataset will be available on GitHub.
Read the full abstract ▾
All authors Qi Ming, Yuyang Wang, Mingjing Zhao, Yifan Xiao, Zhixin Guo, Zhiqiang Zhou, Peng Sun, Juan Fang, Fuqiang Yang, Xudong Zhao
Computer Vision
Chartography: A Benchmark for Professional Chart Understanding
Suhaas Garre, Chris Mutty, Sushant Mehta et al. (+1 more)
Professionals across medicine, engineering, finance, manufacturing, and the sciences often make consequential decisions from charts. Existing chart
benchmarks do not sufficiently measure this ability: they are dominated by bar, line, and pie formats, rely on shorter
reasoning chains, and are nearing saturation, with frontier models already scoring 80-90%. We introduce Chartography, a
benchmark of 100 tasks that pair charts drawn from professional practice, in domain-specific formats that standard chart
benchmarks rarely include, with questions written by professionals who read these charts for a living and independently verified by three additional experts. In an evaluation of 30 frontier-model configurations (20 scored trials per task), the best configuration reaches only 45.0% mean pass@1; the remainder span 9.0-39.5%. Failures concentrate in visual perception: models can miss nuanced features, misread values along sparsely labeled axes, mishandle projected 3D geometry, and violate domain conventions encoded in the chart. We release all tasks, images, provenance metadata, and evaluation code.
Read the full abstract ▾
All authors Suhaas Garre, Chris Mutty, Sushant Mehta, Edwin Chen
Computer Vision · Cross-listed from cs.AI
VERDICT: Training-Free Step-Wise Verification of Multimodal Reasoning via Disagreement-Aware Consensus
Rohit Sinha, Kunal Tilaganji, Tanuja Ganu et al. (+3 more)
Multimodal large
language models often generate
reasoning chains containing subtle errors that lead to incorrect answers. Current verification approaches have notable limitations. Existing approaches either require expensive labelled supervision with inconsistent cross-task performance or aggregate scores from multiple sources by simple aggregations, missing a key insight: when these scores disagree, that disagreement itself carries important information about whether a
reasoning step is truly valid or not. We formalise this as a coupled scoring problem among disparate, frozen verifiers, interpretable as a coordination game with a unique closed-form
equilibrium where agreement signals valid steps while disagreement reveals instability. Towards this end, we propose a training-free domain-agnostic step-wise verification approach we call VERDICT: VERification via Disagreement-Informed Coupled Thresholding. To our knowledge, VERDICT is the first training-free verifier that makes the structure of cross-modal disagreement explicit and actionable. It computes consensus scores through a closed-form solution, enabling both disagreement-aware filtering and stability-conscious ranking of
reasoning steps. Evaluated across six
benchmarks, \method consistently improves over the base model by up to +5.95%, and performs competitively with domain-specific critics that demand extensive supervision, demonstrating that cross-modal agreement provides robust verification signals without task-specific adaptation and Training-Free Verification
Read the full abstract ▾
All authors Rohit Sinha, Kunal Tilaganji, Tanuja Ganu, Nagarajan Natarajan, Amit Sharma, Vineeth Balasubramanian
Computer Vision
Cross-View Sequential Visual Localization with Spatio-Temporal Context Modeling for Autonomous Driving
Jiaping Wang, Shaobo Li and Zhen Wang
Continuous and reliable localization is essential for autonomous driving. Cross-view visual localization matches ground images with satellite maps, providing complementary localization cues for pipelines that depend on Global
Navigation Satellite System (GNSS) signals and high-definition (HD) maps. Most existing cross-view visual localization methods process each frame independently, leaving temporal information underused and limiting accuracy under dynamic occlusion, illumination variation, and repetitive textures. This study proposes a temporal-context-enhanced framework for cross-view sequence visual localization. The proposed recurrent cross-frame module aggregates historical context from the previous state to enhance the coarse ground feature of each current frame. These enhanced features facilitate satellite candidate-region classification, while hierarchical fine-grained features enable precise local offset estimation. On the CVIS
dataset, the proposed method reduces mean localization error from 3.80 m to 1.57 m and increases R@1 m from 8.14% to 40.22%. Direct transfer to KITTI-CVL achieves a mean error of 2.61 m, with target-domain fine-tuning further reducing the mean error to 2.27 m. Zero-shot field experiments on a real-world vehicle achieve a mean error of 2.84 m and R@5 m of 96.86%. These results demonstrate that temporal context enhancement significantly improves cross-view localization accuracy and supports robust deployment on public
benchmarks and real-world roads.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Retrieval-Augmented Vision Foundation Models for Robust Leukemia Cell Classification across Multiple Microscopy Datasets
Carlos Zamora, Hiram Zuniga, Ulises Orozco-Rosas et al. (+1 more)
Leukemia
cell image classification is challenged by real-world domain shifts from acquisition, staining, illumination, and site protocols, causing single-
dataset models to generalize poorly in real clinical scenarios. This work presents a robust framework for leukemia classification across multiple heterogeneous
datasets using a two-stage pipeline with a pretrained
vision foundation model. Stage 1 performs binary classification (leukemia vs. non-leukemia) and is trained using 122,167 single-
cell images. Stage 2 is conditionally applied to Stage 1 positives to perform subtype classification into Acute Lymphoblastic Leukemia (ALL) and Acute Myeloid Leukemia (AML), trained using 69,400 single-
cell images. Labels are harmonized across five heterogeneous
datasets to enable cross-
dataset training, and performance is evaluated on a held-out
dataset protocol to assess domain-shift generalization. Within this pipeline, three encoders are benchmarked (DinoBloom, pretrained on single-
cell images; BiomedCLIP, pretrained on biomedical data; and CLIP as a general-purpose model) under linear probing, Low-Rank Adaptation (LoRA), and a
Retrieval-Augmented Classification (RAC) module that retrieves the top-k most similar
cell images to provide cytomorphological grounding. The objective is to quantify how much domain-specific pretraining contributes to performance under domain shift, and whether cost-effective adaptation and
retrieval can be a viable alternative to expensive domain-specialized pretraining. The held-out protocol additionally serves as a diagnostic tool, revealing when classification performance is attributable to
dataset-specific artifacts rather than to cytomorphological features.
Read the full abstract ▾
All authors Carlos Zamora, Hiram Zuniga, Ulises Orozco-Rosas, Kenia Picos
Computer Vision
PolypVision: A Three-Stage Hierarchical Deep Learning Framework for Classification and Segmentation of Colorectal Polyps
Hamidreza Bolhasani, Hamidreza Rastad, Amir Mohammad Akbari et al. (+4 more)
Colorectal cancer (CRC) remains one of the leading causes of cancer-related mortality worldwide, predominantly arising from precancerous polyps. Accurate detection,
segmentation, and endoscopic and histological classification of colorectal polyps are crucial for timely clinical intervention. In this study, we present PolypVision, a three-stage hierarchical deep learning framework that sequentially performs: (Stage 1) binary classification of polyps as adenomatous or hyperplastic, with simultaneous Paris and JNet classification, using EfficientNetV2-M with Focal Loss; (Stage 2) polyp
segmentation with recommended resection method using a UNet++ decoder with the Stage 1 backbone as encoder, optimized with Dice and BCE losses; and (Stage 3) adenoma subtype classification (tubular, tubulovillous, villous) using EfficientNetV2-M with transfer learning from Stage 2. Evaluated on three public
datasets – PolypGen, Kvasir-SEG, and CVC-ClinicDB – PolypVision achieves an AUC of approximately 0.99 for frame classification and a detection mAP@50 of 94.4% on Kvasir-SEG, outperforming or matching state-of-the-art methods. Gradient-weighted Class Activation Maps (Grad-CAM) confirm that the model attends to clinically relevant lesion features. The framework is device-independent, operating across diverse endoscopic imaging systems without hardware-specific adaptation. These results demonstrate that a hierarchical, transfer-learning-driven pipeline with task-specific loss functions offers a robust, device-independent, and clinically meaningful approach to automated colorectal polyp analysis. PolypVision is freely available as a web application at https://polypvision.com, a DataBioX initiative, with a free usage tier open to all users.
Read the full abstract ▾
All authors Hamidreza Bolhasani, Hamidreza Rastad, Amir Mohammad Akbari, Mohammad Tashakoripour, Parnian Asadollahi, Ata Khodami, Mojgan Forootan
Computer Vision
Precise Top-Layer Fabric Segmentation for Fabric Destacking with Edge- and Shape-Aware Deep Networks
Wenbo Dong, Dipankar Bhattacharya, Akinari Kobayashi et al. (+6 more)
Fabric destacking requires precise
segmentation of the topmost fabric layer, a task complicated by subtle fabric boundaries and high visual similarity between fabric layers. Existing semantic and edge-based
segmentation approaches often struggle with these complexities, limiting the performance of robotic manipulation for different tasks. In this work, a novel
segmentation training architecture tailored for top-layer fabric
segmentation in stacked fabrics is proposed. The method extends the classical encoder-decoder framework by introducing two specialized branches - an edge-aware branch and a shape-aware branch - that are used to supervise the backbone network for better tuning. The edge-aware branch enhances boundary delineation, while the shape-aware branch guides the network to capture and align the overall fabric shape with reference masks derived from Computer Aided Design (CAD) models. Experiments on a real-world fabric
dataset demonstrate that the training approach outperforms established baselines, verifying the effectiveness of the multi-branch design through both quantitative results and ablation studies.
Read the full abstract ▾
All authors Wenbo Dong, Dipankar Bhattacharya, Akinari Kobayashi, Akira Seino, Fuyuki Tokuda, Xuzhao Huang, Kai Tang, Norman C. Tien, Kazuhiro Kosuge
Computer Vision · Cross-listed from cs.IR
DistilVDR: A Compact End-to-End Visual Document Retriever via Dual-Student Distillation
Zhuchenyang Liu, Ziyi Wang, Yao Zhang et al. (+1 more)
Visual document
retrieval (VDR) is dominated by multi-billion-parameter models that are slow to index at full corpus scale and expensive to serve. Prior compression routes either train a smaller multi-vector encoder from scratch or distil only the query side; neither yields a compact single-vector retriever end-to-end. We present DistilVDR, a 524M end-to-end VDR system distilled bilaterally from a single 8B
vision-
language teacher under a pointwise cosine
alignment loss. All supervision comes from the frozen teacher's
embedding space, which was itself trained with relevance supervision, so the student objective needs no relevance labels, negative sampling, or contrastive term. We match VDR's text-query and image-document input asymmetry with an asymmetric encoder-only student that concentrates visual capacity on the document side and keeps the query side at 70M parameters. We release two variants that share the same encoders and training and differ only in the document encoder's visual-tile budget: DistilVDR-HiRes attains 61.74 average NDCG@5 on ViDoRe v1+v2+v3 (86.9% of the 8B teacher) and leads every reproduced sub-1B baseline on the high-resolution-sensitive v3
benchmark, while DistilVDR-Fast attains 59.98 at a 3 times smaller visual-
token budget. Both variants store one million documents in a 15.6 times smaller index than the strongest sub-1B multi-vector baseline and index the corpus an order of magnitude faster. The code is available at https://github.com/Ryenhails/NanoVDR.
Read the full abstract ▾
All authors Zhuchenyang Liu, Ziyi Wang, Yao Zhang, Yu Xiao
Computer Vision
MedUP: Awakening Unified Understanding and Perception in Medical Vision-Language Models
Yuan Wang, Hualiang Wang, Yixin Chen et al. (+6 more)
Medical
Vision-
Language Models (Med-VLMs) excel at verbalizing visual content, yet precise visual perception,
segmentation, and grounding remain challenging. Existing approaches either verbalize regions as coordinate strings or rely on external modules that decouple perception from understanding, creating representation gaps for region-
language alignment. We present MedUP, a Med-VLM that natively unifies perception and understanding within a shared
token space. At its core lies UniMedTok, a region tokenizer that encodes masks as discrete
tokens in the LLM vocabulary, enabling the model to seamlessly interleave mask
tokens with text. We curate UniMed-Train, a 1.84M-instance corpus spanning text-guided
segmentation, region-grounded understanding, medical VQA and CoT-based
segmentation, and introduce UniMed-Bench for unified evaluation. Extensive experiments show that MedUP outperforms native, agentic, and dual-decoder Med-VLMs across all tasks while remaining competitive with specialist segmentors, demonstrating the strong potential of unified understanding and perception modeling.
Read the full abstract ▾
All authors Yuan Wang, Hualiang Wang, Yixin Chen, Songtao Jiang, Shujian Gao, Jiaming Lin, Siming Fu, Jian Wu, Zuozhu Liu
Computer Vision
InSight-doc: Agentic Visual Perception for Long-Document Understanding
Kaican Li, Weiyan Xie, Lewei Yao et al. (+4 more)
Long-document understanding often requires
reasoning over many visually rich pages, making inference costly and prone to context rot. In this work, we propose InSight-doc, an agentic visual perception framework that treats visual resolution as an adaptive
reasoning-time resource. InSight-doc starts from low resolution and selectively zooms into high-resolution regions for finer evidence, without relying on any external retriever. To train such an
agent, we construct an active-perception corpus of 17.9K high-quality SFT examples with region-level zoom-in trajectories, accompanied by 19.2K hard RL examples. Through SFT+RL, InSight-doc-8B improves the baseline by 4.3–16.4 accuracy points over document VQA
benchmarks. On long documents, it reduces
hallucination by more than 40% and inference latency by 41%–68% while maintaining an accuracy lead. Our code,
datasets, and model are released at https://github.com/m-Just/InSight-doc .
Read the full abstract ▾
All authors Kaican Li, Weiyan Xie, Lewei Yao, Jiannan Wu, Lanqing Hong, Yongxiang Huang, Nevin L. Zhang
Computer Vision
Gaussian Sculpting: End-to-End Controllable Surface Reconstruction via Field Optimization
Ke Jiaxin, Juncheng Liu, Yi Wang et al. (+4 more)
3D Gaussian Splatting (3DGS) has recently enabled real-time novel view synthesis with impressive quality. However, it struggles to recover accurate surfaces under limited viewpoints and due to the inherent irregularity of Gaussian primitives. The resulting
geometric errors are notoriously difficult to correct manually. To address these issues, we propose Gaussian Sculpting, a fully differentiable end-to-end framework for high-quality surface reconstruction. Our key insight is to anchor Gaussians onto an evolving differentiable surface, allowing them to guide signed distance field (SDF) optimization instead of extracting the surface only during post-processing. To enable stable gradient isolation during joint optimization, we design a bi-level training strategy in which the outer loop optimizes the geometry represented by the SDF, while the inner loop updates the Gaussians with the geometry fixed. We further impose constraints on Gaussian parameters to ensure consistency with the underlying surface, thereby improving both
geometric and appearance fidelity during optimization. In addition, we introduce a multi-resolution subdivision scheme based on octree-like partitioning to preserve fine details while reducing
memory consumption. Experiments on object-level scenes demonstrate that our method effectively removes redundant surfaces, recovers missing structures caused by limited viewpoints, and achieves strong reconstruction quality even at relatively low resolutions.
Read the full abstract ▾
All authors Ke Jiaxin, Juncheng Liu, Yi Wang, Zhouhui Lian, Bin Liu, Shengfa Wang, Xiangjia He
Computer Vision · Cross-listed from cs.RO
BooST: Bridging Semantics and Motions for Efficient Skill Transfer
Jusuk Lee, Daesol Cho, Jonghun Shin et al. (+4 more)
Skill abstraction—the process of learning reusable and temporally extended behaviors—has emerged as a key paradigm for improving sample efficiency and generalization in
robot learning. For efficient skill transfer to real
robots, learned skills must generalize across tasks and domains, remain robust to visual and dynamic perturbations, and be efficient enough for practical deployment. However, existing methods typically satisfy only a subset of these properties, as they capture either high-level semantic intent (what) or low-level motion dynamics (how). This incomplete skill transfer yields weak priors for policy learning, thereby demanding substantial in-domain data for downstream adaptation. To address these challenges, we introduce BooST, a two-stage framework that explicitly bridges semantics and motions to satisfy all three desiderata. BooST first leverages a cross-modal VQ-VAE to capture both semantic intent and motion dynamics, yielding a unified skill representation. It then distills this representation into a lightweight policy for efficient downstream adaptation to new tasks. Extensive experiments across simulation and real-
robot settings demonstrate that BooST achieves superior few-shot adaptation, cross-domain skill transfer, and robustness to dynamic visual distractors, while maintaining a lightweight yet expressive design suitable for real-world deployment.
Read the full abstract ▾
All authors Jusuk Lee, Daesol Cho, Jonghun Shin, Seungyeon Yoo, Jonghae Park, Taekbeom Lee, H. Jin Kim
Computer Vision
Rethinking Data Efficiency in Industrial Dense Prediction: Pretraining Coherence, Not Inductive Bias, Determines ViTs Low-Data Advantage
Haoran Sui and Yaoyuan Jia
Vision Transformers (ViTs) are widely believed to require more labeled data than CNNs for industrial dense prediction. Through controlled experiments on four industrial
datasets, we show that the data-efficiency gap stems from pretraining incoherence, which refers to the statistical mismatch between ImageNet-pretrained ViT backbones and COCO-pretrained CNN necks, rather than from inherent self-
attention deficits. We characterize the cross-architecture feature gap and propose a lightweight AlignBlock family for pyramid-level feature recalibration. Our core finding empirically identifies a data-efficiency frontier: for domain-proximal scenes with >= 200 samples, Swin-Graft surpasses YOLOv11x (terminal 703-shot: 0.973 vs 0.956 mAP@50); for domain-distant scenes, CNNs retain advantage (hook 141-shot: 0.900 vs 0.600 mAP@50). Grafted neck weights yield up to 2.5x the mAP of a randomly initialized neck.
Read the full abstract ▾