Computer Vision
When Should a Network Emit Geometry, and When Should It Detect It? Readout, Reconciliation, and Representation in Floorplan Vectorization
He Zhang
A network trained to recover the walls, openings, and rooms of a rasterized floorplan can produce its output in two ways: by emitting the geometry as an autoregressive coordinate sequence, or by detecting it on dense junction and centerline heatmaps and assembling a graph. We compare the two readouts on the same trained network. On real scans (CubiCasa5K) detection is better on every wall measure (+2.7 wall F1 at tolerance 0.05, +5.1 at 0.015; paired bootstrap intervals exclude zero), and reading an opening heatmap the decoder never used raises opening F1 by 2.6x without retraining. Within real scans the readout's advantage grows with plan size and reverses on small plans; on clean vector renders sequence decoding is better by 5 to 8 points where its training covered the render style, while under full domain shift the readout, given calibrated thresholds, stays ahead; neither ink density nor plan size explains the reversal. With matched data and recipe, a room-centric system with a reconciliation step and a wall-first sequence model reach comparable wall quality, so the output representation matters less than is usually assumed. A prior from the other family helps at the output but not at the input: deterministic fusion of the two outputs raises wall F1 by 7 points, whereas conditioning one model on the other's output gives no gain in three forms, including two ground-truth-content controls. We also provide an edit-cost metric that scores a draft by the human work needed to correct it, corrected CubiCasa5K annotations, and ResPlan-FP, a CC BY 4.0
benchmark of 16,998 plans with frozen splits and three baseline tracks. Code, the
benchmark, and the corrected annotations are available at https://github.com/Cyprinus12138/fpvec-lab
Read the full abstract ▾
Computer Vision
A Dual-Transformer for Multi-Camera View Recommendation
Josep Cabacas-Maso, Carles Ventura and Ismael Benito-Altamirano
Multi-camera systems are foundational to modern media production, and multi-camera editing is a critical task. This involves the proper selection of the appropriate camera view at each moment. In this paper, we propose a novel Dual-
Transformer architecture with Cross-
Attention that heavily outperformed the current SOTA models over the TVMCE
dataset (TV Shows Multicamera Editing
dataset). Our model decouples these tasks: (1) a dedicated temporal encoder first processes the sequence of past frames to build a rich
memory of the recent history, and (2) the candidate camera views then act as queries to this
memory via a cross-
attention module, allowing each candidate to independently interrogate the historical context and find the most relevant information for its own evaluation. Our approach achieved 56.60% Precision@0.5, representing a substantial improvement over the prior best result of 37.16%. We further conducted an ablation study exploring the use of lightweight backbone architectures, where the SwinV2 backbone yielded the best performance, achieving 69.65% Precision@0.5. Using this best-performing configuration, we then investigated the feasibility of adapting the model to replicate the editing style of a specific human editor. To this end, we fine-tuned the model using varying proportions of the initial segment of a target
video. Our results demonstrate that even with only 20% of the
video used for fine-tuning, the model exhibited measurable improvements in Precision@0.5, indicating strong potential for data-efficient personalization of editing style adapted to each individual TV show or producer.
Read the full abstract ▾
Computer Vision
V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning
Shulin Tian, Minglun Li, Yuhao Dong et al. (+6 more)
Vision-
language models can produce fluent answers that are insufficiently grounded in the visual evidence: a single unsupported object, chart value, or intermediate inference can undermine an otherwise plausible response. We argue that this is a credit-assignment failure in
multimodal post-training. Scalar outcome rewards indicate whether an answer is acceptable, but do not identify which visual facts are grounded, which
reasoning steps are valid, or which instruction constraints are missed. We introduce Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness (VF),
Reasoning Consistency (RC), and Instruction Following (IF). The resulting rubric items provide structured partial credit and localize rubric credit when supporting evidence spans are available. We first obtain an SFT checkpoint by fine-tuning Qwen3-VL-8B-Instruct on the public OpenMMReasoner-SFT-874K corpus, adapting OpenMMReasoner's cold-start data recipe. We construct V-Rubrics 50K, a 50,248-example training set from 17 visually grounded sources, by applying rule-based filters before deriving example difficulty from rejection-sampling scores and then annotating every example with Gemini-3-Pro under the same structured
prompt and protocol. We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit. Experiments show that our rubricbased GRPO improves over both the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded
reasoning benchmarks. The results show rubrics as a useful reward abstraction for visual post-training.
Read the full abstract ▾
All authors Shulin Tian, Minglun Li, Yuhao Dong, Hao Ding, Jiarui Yao, Haiwen Diao, Jingkang Yang, Hongyuan Zhu, Ziwei Liu
Computer Vision
MLLMCLIP: Feature-Level Distillation of MLLM for Robust Vision-Language Representations
Jongsuk Kim, Qiyu Wu, Zhuoyuan Mao et al. (+3 more)
Pretrained
vision-
language models such as CLIP excel at zero-shot recognition but often fail at compositionality, particularly attribute-object and relational structures. Recent studies mitigate this issue by augmenting training with synthetic hard negatives generated by a cascade of large
language models and text-to-image models, which incurs substantial pipeline overhead. We instead propose MLLMCLIP, a heterogeneous
distillation framework that transfers
multimodal knowledge directly from a generative
Multimodal Large
Language Model (MLLM) teacher into a discriminative CLIP student, bypassing synthetic data entirely. To bridge the architectural mismatch between the two paradigms, we introduce an
attention-based per-layer
token selection and a CKA-based
distillation loss. Compared to prior CLIP-enhancement methods, MLLMCLIP achieves state-of-the-art compositional accuracy while delivering consistent gains on standard zero-shot classification and image-text
retrieval, showing that feature-level
distillation strengthens both compositional and general
vision-
language representation capability.
Read the full abstract ▾
All authors Jongsuk Kim, Qiyu Wu, Zhuoyuan Mao, Hiromi Wakaki, Junmo Kim, Yuki Mitsufuji
Computer Vision
CrossMambaTuning: Synergistic Spatial and Cross-Layer Adaptation for Machine Vision Compression
Haobo Xiong, Shaobo Liu, Kai Liu et al. (+1 more)
To reduce deployment cost and retraining overhead, adapting pretrained learned image compression (LIC) models to downstream machine
vision tasks has attracted growing
attention. However, existing methods typically insert fine-tuning modules independently into frozen backbones, lacking explicit mechanisms for cross-layer coordination. To address this limitation, we propose a novel framework named CrossMambaTuning, which integrates State Space Models with cross-layer interaction mechanisms for parameter-efficient fine-tuning. Specifically, we design an efficient Mamba adapter equipped with task-specific
prompts and multi-scale branching to precisely capture both local features and global dependencies. Furthermore, we introduce a Scale-Invariant Cross-Layer Adapter (SICA) utilizing a parameter-sharing strategy to fuse task information across different scales and reduce redundancy. Extensive experiments demonstrate that CrossMambaTuning achieves state-of-the-art (SOTA) performance on multiple machine
vision tasks, reducing parameter overhead by 72% compared to SOTA methods. Code is available at https://github.com/rsr1123/CrossMambaTuning.
Read the full abstract ▾
All authors Haobo Xiong, Shaobo Liu, Kai Liu, Chongyang Ding
Computer Vision
AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research
Xintong Zhang, Xiaomeng Fan, Shilin Yan et al. (+7 more)
Video deep research aims to answer questions by jointly understanding
video content and retrieving external knowledge from the open Web. However, these tasks often require diverse and multi-step tool-use trajectories, which may lead to unnecessary
video tool calls, incurring additional costs and failure risk. In this paper, we propose AdaVDR, an adaptive
video deep research
agent with adaptive tool use and reflection, which selects tools based on task and model capabilities while backtracking when needed. To this end, we develop a data construction pipeline that builds adaptive tool-use trajectories according to task types and model capabilities. Specifically, we construct QA pairs that require both
video and external information, and organize the information acquisition processes into task-specific tool-use trajectories. We then apply model-conditioned tool necessity filtering to remove tools that can be bypassed using
video understanding and internal knowledge of the model. Based on this pipeline, we construct a training
dataset with adaptive trajectories and perform SFT on them to initialize these capabilities of AdaVDR. To further improve adaptive capabilities, we propose NASA-GRPO, a necessity-aware segment-guided credit assignment method, which evaluates segment-guided evidence contribution and tool necessity to construct normalized tool-level rewards. We also construct VDR-EE, a
benchmark built using the proposed pipeline. Experiments show that AdaVDR outperforms the base model by 15.20 and 29.00 percentage points on VDR-EE and the existing VideoDR, respectively, demonstrating its superiority.
Read the full abstract ▾
All authors Xintong Zhang, Xiaomeng Fan, Shilin Yan, Ekko He, Zicheng Liu, Zijian Zou, Guannan Zhang, Yuwei Wu, Zhi Gao, Hongwei Xue
Computer Vision
CropCop: An Auditable 120-Class Plant-Health Model from Benchmark Reconstruction to a Quantised Runtime Artifact
Rana Muhammad Ahmed and Sabahat Abbas
A plant-health score can appear precise while resting on duplicated image families, a long-tailed label space, or a runtime file that was never evaluated. We present CropCop, a closed-set recognition system spanning 120 operational plant-health classes and an evidence chain from corpus reconstruction to direct execution of the final quantised artifact. Starting from 117,546 audited images, we rejected the inherited partition after confirming 3,233 duplicate relationships across split boundaries and froze a 109,107-image
benchmark with zero crossings among the audited trusted leakage groups and a 151.7 largest-to-smallest class ratio. A fully fine-tuned DINOv3 ConvNeXt-Tiny reference achieved 98.51% accuracy and 96.87% macro-F1 on the locked internal test. A compact MobileNetV4 Conv-Medium derivative achieved 98.46% accuracy and 96.27% macro-F1 without being presented as evidence for a new
distillation method. Validation-only post-training quantisation selected dynamic activations with per-channel weights, and the final 22.60 MiB ExecuTorch/XNNPACK PTE achieved 98.46% accuracy and 96.23% macro-F1 when executed directly. Only six of 16,363 top-1 decisions changed between the converted INT8 graph and the PTE, while paired analysis showed a modest class-balanced loss; an exploratory post hoc fruit-label slice localized a larger recall decline than aggregate accuracy revealed. CropCop establishes strong leakage-controlled internal recognition and software-runtime fidelity; it does not establish performance on unseen farms, camera pipelines, or physical Android hardware.
Read the full abstract ▾
Computer Vision
Video-IFBench: Evaluating Instruction Following of Multimodal LLMs in Video Understanding Scenarios
Hongbo Liu, Peixian Chen, Sihan Liu et al. (+12 more)
Multimodal Large
Language Models (MLLMs) have shown strong performance in
video understanding. However, their ability to follow instructions in this domain remains under-explored. Real-world
video understanding requires models not only to interpret
video content correctly, but also to satisfy diverse user-specified constraints. Existing
benchmarks focus primarily on task accuracy rather than instruction adherence, leaving this capability insufficiently evaluated. To address this gap, we introduce
Video-IFBench, a comprehensive
benchmark for evaluating instruction following in
video understanding, where models must satisfy diverse user-specified constraints, including those grounded in visual and audio content. We develop an instruction taxonomy with four templates, including single-task, multi-task, selection, and nested instructions, covering 32 task types and 39 manually designed constraint categories spanning both semantic and format requirements. To reduce annotation cost, we build a semi-automatic data construction pipeline that combines MLLMs, programmatic processing, and human verification, resulting in 1.5K samples. We conduct a large-scale evaluation of more than 20 recent MLLMs and show that
video instruction following remains challenging for current models, especially for instructions with many constraints, semantic constraints, or complex conditional structures that require selecting the correct branch or path based on
video content. We hope our work will facilitate future research on instruction following in
video understanding scenarios.
Read the full abstract ▾
All authors Hongbo Liu, Peixian Chen, Sihan Liu, Peiyuan Zhang, Kai Zou, Dian Zheng, Xiaoxing Hu, Yuhao Dong, Mengdan Zhang, Yunhang Shen, Haoyu Cao, Wei Liu, Weibo Gu, Xing Sun, Shengjie Zhao
Computer Vision
Asymmetric Cross-Modal Fine-Grained Visual Categorization: ACF-Net and the BirdPro Benchmark
Bohan Deng, Shuo Ye and Zitong Yu
Audio-visual cross-modal Fine-Grained Visual Categorization (FGVC) aims to identify fine-grained categories by jointly leveraging visual and auditory information. However, FGVC under asymmetric cross-modal scenarios has received limited
attention, where paired
video and audio are not strictly synchronized and may not even correspond to the same individual or moment. Such weak and ambiguous cross-modal correspondence poses substantial challenges to effective representation learning and modality
alignment. To address these issues, we propose ACF-Net, a novel optical flow-guided framework for asymmetric audio-visual fine-grained learning. ACF-Net consists of two key modules: Optical Flow-Guided Motion (OFGM) and Asymmetric CrossModal Adaptive Fusion (ACAF). OFGM captures motion-sensitive visual cues and suppresses irrelevant background interference, thereby enhancing discriminative dynamic representations in
videos. ACAF estimates modality reliability under weakly matched audio-
video pairs and performs uncertainty-aware adaptive fusion to improve category-level recognition robustness. To support research on asymmetric cross-modal FGVC, we further construct BirdPro, a new bird-oriented audio-visual
benchmark, since existing
datasets often lack large-scale category-level audio-
video associations under non-strict temporal and instance correspondence. BirdPro contains 1,919 audio recordings and 11,965
videos covering 194 bird species. Extensive experiments show that ACF-Net achieves the best results compared with representative baseline methods, outperforming the strongest baselines by 2.97% and 1.92% in the fused and mismatched settings, respectively.
Read the full abstract ▾
Computer Vision
OpenVeinNet: Robust Open-Set Finger Vein Verification with Dynamic Snake Convolution and Graph Learning
Sushrut Patwardhan and Raghavendra Ramachandra
Finger vein verification is a promising biometric modality for secure authentication because vascular patterns are internal, difficult to observe externally, and relatively resistant to presentation attacks. However, reliable verification remains challenging in open-set settings, where test identities are unseen during training and non-enrolled probes must be rejected at inference. This paper presents OpenVeinNet, a finger vein verification framework designed for cross-
dataset and open-set evaluation. The proposed model combines Dynamic Snake Convolution with graph-based feature modelling. Dynamic Snake Convolution extracts local curvilinear and tubular vein structures using adaptive sampling, while the graph convolutional backbone models long-range topological relationships between vein regions. To improve the discriminative quality of the
embedding space, we introduce a Centroid Angular Hybrid Loss, which jointly encourages intra-class compactness and inter-class angular separation for cosinesimilaritybased verification. Experiments are conducted on five public finger vein
datasets: FV-300, MMCBNU, FV-USM, PolyU, and VERA. The method is evaluated using leaveonedatasetout training under both enrolmentbased unknownrejection and fullsubject verification protocols, and is compared with handcrafted and recent deep learning-based baselines. The results show that OpenVeinNet achieves strong cross-
dataset generalisation, consistently low equal error rates, and competitive true accept rates at fixed false accept rate operating points. Ablation studies further confirm the individual and combined contributions of adaptive tubular feature extraction, graph-based relational modelling, and the proposed loss function. These findings indicate that explicitly modelling local vein geometry, global vascular relationships, and angularly compact
embeddings is effective for openset finger vein verification.
Read the full abstract ▾
Computer Vision
Pose-Anchored Optical Flow for Low-Latency Human Action Anticipation in Human-Robot Teaming
Lewis de Zoete Grundy, Chris McCarthy and Christopher Fluke
Human-
robot interaction (HRI) requires
robots to interpret human actions early in their execution in order to respond safely, efficiently, and naturally. However, many existing approaches to human action recognition rely either on sparse skeletal representations, which lack fine-grained motion cues, or dense optical flow, which can be computationally expensive for low-latency perception pipelines. In this paper, we propose PoseOFF, a pose-anchored optical flow representation that captures local motion information around human joints to support earlier human intent understanding. By conditioning motion feature extraction on human pose, PoseOFF encodes localised motion dynamics at semantically meaningful body locations, forming a structured motion representation that is explicitly aligned with human kinematics. We evaluate PoseOFF across multiple
benchmark datasets and backbone architectures for action anticipation, demonstrating consistent improvements in recognition accuracy, particularly at early observation ratios. Our results show that PoseOFF enables models to achieve comparable or improved performance while observing less of the action sequence, highlighting its effectiveness for early prediction. Importantly, these gains are achieved without requiring full-frame motion processing, making the approach practical for real-time and resource-constrained settings. These findings suggest that pose-centred motion representations such as PoseOFF can enhance the ability of interactive
robot systems to infer human actions earlier, supporting more responsive and anticipatory behaviour in human-
robot interaction scenarios.
Read the full abstract ▾
Computer Vision
SMART: MLLM-guided Temporal Alignment for Unifying Sign Language Recognition and Spotting
Eunjee Choi, JungHoon Sung, Seongwhan Cho et al. (+2 more)
Continuous sign
language recognition (CSLR) aims to recognize gloss sequences from unsegmented sign
videos under weak sequence-level supervision. However, existing methods rely on sentence-level gloss annotations, providing limited temporal and semantic guidance for fine-grained representation learning. Conventional
video-text
alignment also requires large batch sizes, making it inefficient for
memory-intensive sign
language video training. In this work, we propose SMART, an MLLM-guided temporal
alignment framework for joint sign recognition and spotting. SMART uses MLLMgenerated motion descriptions as auxiliary semantic cues and performs stable videotext
alignment under small-batch training. To improve temporal representation learning, we introduce a Multi-Scale Temporal Adapter that models temporal interactions during
transformer encoding. For dense temporal localization, SMART incorporates CSFormer, a CSLR-guided spotting module that injects recognition-derived gloss evidence into a boundary-aware spotting network. This unified framework enables CSLR features to benefit spotting, while spotting supervision complements weak CTC-based recognition. Experiments on four sign
language benchmarks, including PHOENIX14-T, CSL-Daily, Large-scale KSL, and Disaster and Safety KSL
datasets, demonstrate the effectiveness of SMART across both recognition and spotting tasks.
Read the full abstract ▾
All authors Eunjee Choi, JungHoon Sung, Seongwhan Cho, Chu Xin, Younggeun Choi
Computer Vision
Semi-Supervised Adaptation of Vision-Language Models for Image Classification
Mohamed L. Mekhalfi, Mohamad M. Al Rahhal, Yakoub Bazi et al. (+4 more)
Vision-
language models like CLIP have shown sig- nificant potential in handling natural images, yet their perfor- mance is often limited by the distinct characteristics of satellite imagery. While parameter-efficient adaptation techniques exist, their efficacy is frequently limited by the scarcity of annotated samples. In this letter, we propose Self-Evolutionary CLIP (SE- CLIP), a semi-supervised framework designed for recursive label mining in scene classification. The approach follows a dual-phase pipeline, where an initial warm-up on a few annotated seeds is followed by a recursive discovery phase that iteratively identifies high-confidence samples from unlabeled pools. To maintain the integrity of the evolving support set, we employ a class-balanced selection strategy that prevents the model from being dominated by easily learned categories. Results on the UCM and NWPU
benchmarks indicate that SE-CLIP significantly outperforms existing semi-supervised approaches. The framework provides a viable solution for adapting VLMs to the remote sensing domain with minimal human intervention.
Read the full abstract ▾
All authors Mohamed L. Mekhalfi, Mohamad M. Al Rahhal, Yakoub Bazi, Salah E. Khenfer, Mingdeng Shi, Hua Zou, Mansour Zuair
Computer Vision
Gaussian Splatting Underwater: A Controlled Cross-Regime Study
Olaya Álvarez-Tuñón and Stella Graßhof
The
underwater environment is challenging for 3D reconstruction, because particles suspended in the water scatter and diffuse light, turbidity varies, absorption depends on wavelength, and illumination is rarely uniform. Methods based on Gaussian splatting have generally been developed for conditions that allow good image quality, and have primarily been tested on relatively shallow water. This paper examines how well Gaussian splatting performs across publicly available
underwater datasets representing different degrees of turbidity, loss of illumination, and colour attenuation, together with an industrial survey. Five systems with public code are run under one protocol, with shared poses, initialisation, budget, and evaluator, to establish their relative advantages, disadvantages, and limitations. What these methods can do turns out to depend more on the setup than on the architecture. Water clarity binds upstream of rendering, since structure-from-motion registers 99.5 % of frames in clear water and 0.0 % at 12 NTU. Illumination geometry decides whether a medium model helps at all: under an artificial light that moves with the camera, medium-blind splatting beats both medium-aware systems. On the survey the
benchmark's photometric leader comes last, beaten on geometry by a restoration pre-pass in front of vanilla 3DGS—and none of it is visible in the scores the field reports. Scene builds, per-run configurations, and evaluation code are released at https://github.com/olayasturias/uw3dgs
Read the full abstract ▾
Computer Vision
DeCO: Discriminative Evidence Composition for Fine-Grained Dataset Distillation
Chuixuan Fan, Guang Li, Shijie Wang et al. (+5 more)
Dataset distillation compresses a large training set into a compact synthetic set while preserving its downstream utility. However, existing methods primarily preserve global image statistics and may overlook the localized evidence essential for fine-grained visual classification (FGVC), such as object parts, subtle textures, and region-specific structures. We formulate fine-grained
dataset distillation as budgeted discriminative-evidence preservation and propose Discriminative Evidence Composition (DeCO). DeCO uses
attention rollout from a pretrained TransFG teacher to identify informative patches, applies spatial diversification to reduce redundant coverage, and organizes the resulting regions into class-wise evidence banks. Multiple same-class regions are then packed into compact grid-composed images. The teacher is used only for
dataset construction, whereas downstream students are trained with standard hard-label supervision without teacher logits. Experiments on CUB-200-2011, FGVC-Aircraft, and Stanford Cars show that DeCO consistently outperforms representative coreset and
dataset-
distillation baselines under different IPC budgets.
Read the full abstract ▾
All authors Chuixuan Fan, Guang Li, Shijie Wang, Dongzhan Zhou, Baoli Sun, Takahiro Ogawa, Miki Haseyama, Zhihui Wang
Computer Vision
4DStreamCtrl: Interactive Video Generation with Online 4D Control
Shiqian Li, Chenguo Lin, Zhiguang Liu et al. (+4 more)
Generative
video models now synthesize footage nearly indistinguishable from reality. Their promise as interactive tools hinges on fine-grained control of how objects and the camera move over time, yet each existing approach captures only part of this: camera-parameter methods steer the viewpoint but cannot move objects, 2D-trajectory methods act in the image plane and ignore depth and occlusion, and recent 3D methods add geometry but run only offline at a fixed length. In particular, none combines 3D-consistent control of both camera and objects with real-time, streaming generation. Here we show that camera motion, object trajectories, and depth can be unified into a single 3D point-track representation, from which one model performs joint camera and object control, depth editing, and motion transfer in a single forward pass. To learn this interface at scale, we mine in-the-wild
video for 3D motion supervision, yielding OpenVidHD-Motion3D, and encode it with a lightweight
Geometric Motion Head that plugs into a pretrained
video diffusion model. Because this encoder is temporally separable, we distill the model into a causal streaming student that generates arbitrarily long
video in four denoising steps at
memory independent of length. This unified design surpasses prior camera-only, 2D, and offline-3D methods in motion-control precision while covering modalities they address only in isolation. 4DStreamCtrl runs at 20 FPS on a single high-end GPU for 480p
video and stays temporally coherent over hundreds of frames, enabling, to our knowledge, interactive 4D-controllable streaming generation for the first time. More broadly, grounding generation in explicit 3D geometry with efficient causal inference points toward interactive world models with closed-loop spatiotemporal control, from controllable simulators to real-time visual imagination for
embodied agents.
Read the full abstract ▾
All authors Shiqian Li, Chenguo Lin, Zhiguang Liu, Yu Tang, Jiarong Ou, Rui Chen, Yixin Zhu
Computer Vision
PAGS: Autofocusing Photoacoustic Tomography via Speed-of-Sound-Adaptive Gaussian Splatting
Jiarui Ge, Jintao Ma, Bangxu Fan et al. (+4 more)
Photoacoustic computed
tomography (PACT) combines optical absorption contrast with acoustic detection for high-resolution deep-tissue imaging. A persistent challenge is that unknown speed-of-sound (SoS) heterogeneity changes acoustic time-of-flight, causing defocusing artifacts when reconstruction assumes a uniform SoS. Existing SoS-adaptive methods either rely on calibrated acoustic priors or optimize dense physical medium models, which becomes expensive and difficult to scale in 3D. We propose PAGS, a differentiable framework for blind autofocusing PACT via speed-of-sound-adaptive Gaussian splatting. PAGS represents the initial pressure field with sparse Gaussian photoacoustic (PA) sources and replaces explicit medium recovery with a compact anisotropic path-averaged SoS (ASoS) field parameterized by spherical harmonic probes. This latent propagation field directly controls source-to-transducer arrival-time
alignment, while an analytic Gaussian acoustic projection maps the source representation to transducer signals efficiently. The resulting closed-loop signal-domain optimization jointly updates the Gaussian PA source parameters and the ASoS field from measured data, without calibrated SoS priors. Experiments on simulated and physical phantom data demonstrate improved reconstruction sharpness under heterogeneous acoustic media, robustness to sparse-view sampling, and computational benefits from the analytic Gaussian projection.
Read the full abstract ▾
All authors Jiarui Ge, Jintao Ma, Bangxu Fan, Jinyan Zhang, Xiaokang Yang, Shuai Na, Xiaoyun Yuan
Computer Vision
Automatic weld seam segmentation for industrial quality control: a comparison of RGB and polarimetric imaging with CNN and transformer architectures
Simone Garbin, Leonardo Venturoso and Marco Todescato
Visual inspection of welded assemblies remains one of the least automated stages in many industrial production processes, still depending largely on the experience of human operators and thus subject to inter-operator variability; the manufacturing of special-purpose machinery cabins, the setting of this study, is one representative case. This work evaluates the feasibility of automatic weld seam
segmentation from RGB and polarimetric imagery, comparing controlled laboratory acquisitions with images captured under real, uncontrolled conditions. Convolutional neural network (CNN) architectures and
transformer-based architectures are benchmarked under a unified, threshold-independent protocol, training each CNN with three random seeds to separate genuine effects from seed noise. In controlled RGB conditions, CNN models reach a mean mask mAP50 of up to 0.87, but drop to 0.22-0.48 under uncontrolled acquisition, showing that the acquisition setup is a first-order component of the inspection system. Polarimetric imaging with
alignment-preserving
geometric augmentation localizes previously unseen welds with a mean mask mAP50 up to 0.93: on par with, rather than ahead of, the best controlled-RGB result, but reaching that accuracy on uncontrolled RGB without requiring acquisition control. The clearest architectural finding concerns viewpoint robustness. In-distribution,
transformers and CNNs are broadly comparable; but under a test-time viewpoint shift, the
transformer models, and RF-DETR in particular, retain high accuracy while every CNN collapses. The gap holds across three seeds and a resolution-matched control, pointing to architecture rather than training resolution. Within the CNN family, capacity brings no reliable in-distribution gain once seed variance is accounted for: small CNNs suffice for fixed viewpoints,
transformers for variable ones.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
GLOSS: Geometric Local Self-Similarity Learning for Faithful Reference-Guided Texture Fill
Chenyue Cai, Anita Hu, James Lucas et al. (+2 more)
Using conditional image generators, texture artists can explore many single-view looks for an existing 3D shape. Despite impressive progress, state-of-the-art generative methods still struggle to generate a full object texture while closely adhering to fine scale
geometric detail and single view references, leaving little room for artists guidance. Furthermore, current automatic models lack the flexibility for artist to explore multiple textures from varied sources in an interactive and controllable manner. Unlike methods trained on large 3D
datasets that generate full object textures from global guidance, our work explores a local and less data-hungry approach to texture with explicit artist control. We leverage the
geometric self-similarity and geometry-texture correlation existing in many natural and man-made shapes; and train a shape-specific local texture generation and completion model. This model learns from existing image model priors and a single 3D shape, and is guided by attending to a set of geometry-aware reference patches. The trained shape-specific network can transfer any novel reference to the full target object texture through patchwise inpainting. We show improved or comparable quality to strong image-conditioned texture generation baselines, suggesting local texturing as a promising research direction. Our model also enables local geometry-conditioned texture inpainting, guided by artist-selected references, and generalizes to PBR materials and unseen meshes for texture transfer. We piloted our novel texture fill capability as a Blender addon with several 3D texturing professionals who reported positive feedback on the model's controllability, practical usefulness, and creative affordances.
Read the full abstract ▾
All authors Chenyue Cai, Anita Hu, James Lucas, Szymon Rusinkiewicz, Masha Shugrina
Computer Vision
VGA-BenchV2: An Expanded Unified Benchmark and Multi-Model Framework for Evaluating Video Aesthetics and Generation Quality
Longteng Jiang, DanDan Zheng, Qianqian Qiao et al. (+7 more)
We introduce VGA-BenchV2, an extended human-aligned
benchmark and optimization framework for jointly evaluating and improving
video generation quality and aesthetic value. Built upon VGA-Bench, VGA-BenchV2 preserves the original fine-grained taxonomy with two primary dimensions-Aesthetic and Generation-and 52 sub-dimensions. Guided by this taxonomy, we curate 1,016 diverse
prompts and collect over 60,000
videos generated by 12 mainstream
video generation models. More importantly, VGA-BenchV2 substantially expands human-labeled supervision by adding 36,000 task-level annotations, including 16,200 for aesthetic quality, 13,200 for aesthetic tagging, and 6,600 for generation quality, corresponding to 13.46x, 11.15x, and 1.55x scale-ups over VGA-Bench, respectively. Leveraging this enlarged annotation corpus, we develop a hybrid evaluator architecture consisting of VAQA-Net for continuous aesthetic scoring and two Qwen-based Large
Vision-
Language Model evaluators, VTag-Net and VGQA-Net, for aesthetic tagging and generation quality assessment. Extensive experiments demonstrate strong
alignment with human judgments across diverse generation models. Beyond evaluation, VGA-BenchV2 further introduces an evaluation-to-optimization pipeline, where the learned aesthetic evaluator serves as a reward model for reinforcement learning-based generator fine-tuning. This closes the loop from
benchmark construction and human supervision to automated evaluation and model optimization, enabling
video generators to improve not only in realism but also in aesthetic quality and human preference
alignment. Resources are available at https://huggingface.co/
datasets/BestiVictoryLab/VGA-Bench.
Read the full abstract ▾
All authors Longteng Jiang, DanDan Zheng, Qianqian Qiao, Heng Huang, Huaye Wang, Yihang Bo, Bao Peng, Jingdong Chen, Jun Zhou, Xin Jin
Computer Vision
Saliency-Depth Conditioning for Zero-Shot Segmentation of Communication-Tower Components in Cluttered UAV Imagery
Ali Lesani, Chul Min Yeum and Su-Min Kang
Fine-grained
segmentation of communication-tower components in
UAV imagery is essential for automated inspection, yet task-specific models are hard to develop due to limited instance-level annotations. Zero-shot
segmentation models offer a promising alternative, but in cluttered scenes, visually similar background structures interfere with component localization, causing missed instances and false positives. We propose a model-agnostic saliency-depth foreground-conditioning strategy combining appearance-based saliency with
monocular relative depth to construct a coarse tower prior and suppress irrelevant content. We integrate this module with Grounded-SAM and SAM 3, yielding SD-Grounded-SAM and SD-SAM 3. SD-Grounded-SAM further applies
geometric and depth-aware box refinement before mask generation, while SD-SAM 3 relies on SAM 3's internal setup. On TOW-300, a
dataset of 340 communication-tower
UAV images, our strategy improves both baselines: SD-SAM 3 achieves the strongest instance-
segmentation performance, while SD-Grounded-SAM produces fewer false positives. Ablations confirm complementary gains from saliency, depth, and box refinement, improving robustness in cluttered scenes.
Read the full abstract ▾
Computer Vision
Bootstrapping a 4D LiDAR Annotation Tool from Video Foundation Models
Jihun Kim, Hyun-Kurl Jang, Hyemin Yang et al. (+3 more)
Progress in 4D
LiDAR segmentation is bottlenecked by data. Assigning temporally consistent labels across sparse point cloud sequences is costly and hard to scale, and every new task or domain tends to demand fresh dense annotation. This motivates a simple question of whether high-quality
LiDAR training data can be produced automatically, without any human labeling. To this end, we introduce
LiDAR-SAM2, a framework that turns a 2D
video foundation model, SAM2, into a scalable source of supervision for the 4D
LiDAR domain. On the data side, it automatically generates temporally coherent
LiDAR-level labels from SAM2
video masks through multi-view projection and spatio-temporal aggregation. On the modeling side, a tailored modality interface and a two-stage learning objective adapt SAM2's
video segmentation kernel to spatio-temporal
LiDAR structure, so that a single click per object yields a consistent mask track across the sequence. Trained with no human
LiDAR annotation,
LiDAR-SAM2 produces semantic and panoptic labels on SemanticKITTI that approach the quality of full human annotation from only a few points, and models trained on these labels approach the performance of full ground-truth supervision. This positions
LiDAR-SAM2 as a scalable labeling tool that substantially reduces the annotation burden for 3D and 4D scene understanding.
Read the full abstract ▾
All authors Jihun Kim, Hyun-Kurl Jang, Hyemin Yang, Jinnyeong Yang, Hyeokjun Kweon, Kuk-Jin Yoon
Computer Vision
AdaptiveEmbed: Sample-Adaptive Multi-Vector Representation for Multimodal Retrieval
Xinze Liu, Lei Yang, Dayan Wu et al. (+7 more)
Multi-vector representations have emerged as an effective paradigm for
multimodal retrieval, representing each sample with multiple complementary
embeddings to capture fine-grained cross-modal information. However, existing approaches typically employ a fixed representation capacity, assigning the same number of vectors to all samples regardless of their individual
retrieval demands. Such a fixed-capacity formulation overlooks the fact that different samples may require different amounts of representation capacity for effective
retrieval. In this work, we introduce Sample-Adaptive Multi-Vector Representation (SAMVR), a new problem setting for
multimodal retrieval that studies how multi-vector representation capacity can be allocated at the sample level. Under SAMVR, each sample is represented by a content-adaptive
embedding set (CAES), whose capacity is determined according to the sample-specific
retrieval utility of additional representation vectors. To instantiate SAMVR, we propose AdaptiveEmbed, a unified framework for learning sample-adaptive multi-vector representations. AdaptiveEmbed learns structured multi-vector representations through Multi-Group Contrastive Learning (MGCL) with the symmetric set-to-set similarity (SetSim), and further employs Utility Policy Optimization (UPO) to determine sample-specific representation capacity via Marginal Utility Allocation (MUA). Experiments across
multimodal retrieval benchmarks involving image, text,
video, and audio show that sample-adaptive capacity allocation achieves overall better
retrieval performance than fixed-capacity multi-vector representations, validating the effectiveness of SAMVR for
multimodal retrieval. These results establish SAMVR as a viable formulation for adaptive capacity allocation in multi-vector
multimodal retrieval.
Read the full abstract ▾
All authors Xinze Liu, Lei Yang, Dayan Wu, Hengjie Zhu, Zihao Zhang, Hanqi Wu, Tianzhu Hu, Peng Fu, Zheng Lin, Weiping Wang
Computer Vision · Cross-listed from eess.SP
Token-Oriented Semantic Communication with Pretrained Vision Transformers
Jiwoong Im, Minwoo Kim, Jaeho Lee et al. (+2 more)
Token communications realize the semantic communication principle at the granularity of
transformer tokens, providing a promising direction for client–server collaborative inference in resource-constrained edge systems. However, directly transmitting
token embeddings presents two practical challenges: substantial communication cost and limited interoperability across model-specific
token embedding spaces. To address these challenges, we propose a
token-oriented semantic communication framework. In this framework,
token-level task relevance determines which compressed image latents are transmitted, enabling
token-granular transmission without directly transmitting
token embeddings. The framework is modular, coordinating three pretrained components—a lightweight client-side
vision transformer (ViT), a learned image compression (LIC) model, and a large server-side ViT—without end-to-end training. The key enabler is the one-to-one spatial
alignment between ViT patch
tokens and the LIC latent vectors, which allows
token-level task relevance to directly determine which latent vectors are transmitted. Building on this
alignment,
token-aligned LIC selectively transmits task-relevant latents, layer-selective
attention rollout estimates
token relevance from a selected range of
attention layers in a single forward pass, and surrogate
token substitution adapts the frozen server model by optimizing a single learnable
token. Experiments on ImageNet show that the proposed framework achieves a more favorable rate–accuracy trade-off than recent semantic communication schemes, hand-crafted codecs, and task-agnostic LIC models.
Read the full abstract ▾
All authors Jiwoong Im, Minwoo Kim, Jaeho Lee, Yo-Seb Jeon, Yongjune Kim
Computer Vision
PIVOT: A Multi-Trajectory Dataset and Testbed for Pose, Intrinsics, and Novel Viewpoint Evaluation in Real-World 3D Reconstruction
Mary Raymond
Neural radiance fields (NeRFs), 3D Gaussian Splatting (3DGS), and related novel-view synthesis methods are commonly evaluated under capture and reconstruction conditions cleaner than those encountered by
robots, drones, and autonomous systems.
Benchmarks often rely on reconstruction-friendly trajectories, optimized camera poses and intrinsics, and held-out views sampled from trajectories represented during training. These assumptions can obscure performance with measured poses, reusable camera calibration, and structurally different camera paths. We introduce PIVOT (Pose, Intrinsics and Viewpoint Oriented Testbed), a multi-trajectory
dataset, processing pipeline, and evaluation framework for independently studying these factors. PIVOT captures each scene using diverse camera trajectories and retains, where available, both sensor-derived measured poses and COLMAP-optimized poses, together with calibrated and optimized camera intrinsics. It defines three
benchmark families: (1) seen versus unseen trajectory novel-view generalization, (2) measured versus optimized pose sensitivity, and (3) calibrated versus optimized intrinsics sensitivity. We also introduce a directed pose-space Chamfer distance to quantify how well training poses cover an evaluation trajectory. PIVOT v1 contains five real-world scenes captured with a DJI Mini 4 Pro and provides an open processing and Nerfstudio-based evaluation toolchain.
Benchmark results show a consistent quality gap between held-out views on represented trajectories and unseen trajectories, as well as substantial sensitivity to pose source and camera intrinsics.
Read the full abstract ▾
Computer Vision
Efficient Training with Foresight: Multi-Token Auxiliary Supervision for Autoregressive Image Generation
Guo Niu, Xiongfei Yao, Teng Wang et al. (+1 more)
Autoregressive (AR) image generation has shown strong potential for scalable high-fidelity synthesis by modeling images as discrete
token sequences. However, traditional next
token prediction (NTP) continues to suffer from sparse and myopic supervision, insufficiently discriminative representations, and high training cost caused by dense computation over the full
token sequence. To address these issues, we propose multi-
token autoregressive (MTAR), a unified training framework that improves autoregressive image generation from three aspects: prediction objectives, representation regularization, and training efficiency. Specifically, MTAR introduces multi-
token prediction (MTP) to alleviate the sparsity and myopia of traditional NTP by imposing joint supervision on multiple future
tokens; employs
token-level contrastive regularization (TCR) to explicitly enhance the separability of sampled
token representations and thereby improve representation discriminability; and incorporates semantic dropping (SD) as a semantics-aware training acceleration strategy to reduce redundant computation on low-information
tokens while preserving informative learning signals. All three components are applied only during training and introduce no additional overhead during autoregressive inference. On ImageNet, MTAR achieves a better balance between generation quality and training efficiency. Compared with LlamaGen, MTAR achieves up to 0.95 lower FID and 39% faster training. Moreover, even with only 1/3 of the training iterations, it still attains performance comparable to or better than the baseline, substantially reducing training time.
Read the full abstract ▾
All authors Guo Niu, Xiongfei Yao, Teng Wang, Nannan Zhu
Computer Vision · Cross-listed from cs.CY
GGSS: Geodesic-Gated Spherical Steering for Inference-Time Debiasing of Generative Vision-Language Models
Yiqun Sun, Junyu Chen, Pengfei Wei et al. (+1 more)
Generative
vision-
language models (VLMs) are increasingly used in human-centered settings, yet they can produce demographically biased outputs even when images differ only in controlled attributes such as perceived race or gender. However, existing inference-time debiasers were largely designed for static
embeddings or CLIP-like models rather than generative VLMs. We propose GGSS—Geodesic-Gated Spherical Steering—a norm-preserving intervention that discovers a counterfactual bias subspace on the unit hypersphere, steers visual
tokens along geodesic arcs, and uses an adaptive gate to focus correction on
tokens that carry stronger demographic signal. We evaluate four generative VLMs against ten adapted inference-time debiasing baselines and
prompt-based mitigation under a single operating-point protocol across categorical, pairwise, and occupation-gender bias tests, while also measuring general visual-
language capability. GGSS achieves the lowest average bias on all four models, significant on three of four backbones under paired permutation tests, while preserving MMStar accuracy within +/- 0.6 p.p. of the unsteered baseline. Code is available at https://github.com/dukesun99/GGSS.
Read the full abstract ▾
All authors Yiqun Sun, Junyu Chen, Pengfei Wei, Lawrence B. Hsieh
Computer Vision
Capacity Overflow: A Blind Spot for Backdoor Attacks in Vision MoE
Xiaocheng Zou, Tiancheng Zheng, Xiaolin Xu et al. (+1 more)
Mixture-of-Experts (MoE) has become a prevalent paradigm for scaling
Vision Transformers efficiently. To ensure computational scalability and prevent expert overload,
Vision MoE architectures employ a capacity-bounded
token dispatch mechanism, where each expert's processing budget depends on the inference batch size. This work identifies this batch-dependent behavior as an overlooked attack surface, and proposes a stealthy supply-chain
backdoor attack that exploits this property through a three-phase framework. First, we inject a
backdoor into an early MoE layer. Second, we train a neutralizer in a deeper MoE layer that suppresses the
backdoor under normal capacity. Third, we configure a batch-adaptive capacity factor that preserves high capacity for small batches while reducing it for large batches, naturally disabling the neutralizer via
token overflow at deployment-scale batch sizes. The attack remains in dormant mode during small-batch security audits and enters activation mode during large-batch deployment. Experiments on V-MoE and Swin-MoE across ImageNet-100 and GTSRB demonstrate activation-mode attack success rates of 76-87% with dormant-mode ASR below 9%, while evading Neural Cleanse, STRIP, Fine-
Pruning, and Activation Clustering. Our findings reveal a fundamental security risk arising from batch-dependent execution in scalable
Vision MoE architectures.
Read the full abstract ▾
All authors Xiaocheng Zou, Tiancheng Zheng, Xiaolin Xu, Ruyi Ding
Computer Vision
RSFusionDet: Underwater RGB-Sonar Multimodal Object Detection
Zhuoyan Liu, Yihan Wang, Bo Wang et al. (+2 more)
Underwater unimodal object detection faces many challenges in sensor imaging, such as optical images limited by
underwater noise and visible distance, and sonar images limited by less object structural information. While, optical images have rich object structural information, and sonar images are less affected by
underwater noise and have a longer visible distance. Optical (RGB modality) and sonar (Sonar modality) images have complementary information
underwater. In this paper, we create an RGB-Sonar
multimodal object detection
dataset, RGB-Sonar Fusion (RSFusion) and propose evaluation metrics for the
benchmark. And we propose the RGB-Sonar Fusion Detector (RSFusionDet) with a new RGB-Sonar
multimodal object detection result expression for RGB-Sonar
multimodal object detection. We analyze the features of RGB and Sonar modal information, and design a Cross-
Attention Fusion (CAFusion) module to fuse RGB-Sonar spatial misalignment features and Object Matching Head (OMHead) with Loss (OMLoss) to match identical objects in RGB-Sonar modalities. Our RSFusionDet achieves 76.4/48.6 AP (RGB/Sonar) for object detection and 83.4 \(\text{F1-Score}_{match}\) for object matching, on RSFusion, which outperforms other object detection models. Compared with the DINO baseline, our method improves by 0.7/1.4 AP (RGB/Sonar) while simultaneously providing reliable cross-modal object matching. The code and
datasets are publicly available at https://github.com/LEFTeyex/RSFusionDet.
Read the full abstract ▾
All authors Zhuoyan Liu, Yihan Wang, Bo Wang, Bing Wang, Ye Li
Computer Vision
FlashNormal: Detailed Surface Normal Estimation from Flash and No-Flash Images
Ruiyang Chen, Feiran Li, Heng Guo et al. (+1 more)
High-quality surface normal estimation is preferred for detailed surface shape recovery and image editing. Existing single image-based methods, though being a practical setup, often struggle to recover fine surface details and are sensitive to inherent shape-reflectance ambiguity. While photometric stereo achieves high-fidelity surface normal estimation from images under varying lights, its applicability is strictly limited by requiring a multi-illumination capture setup. To this end, we propose FlashNormal, a
diffusion-based surface normal
estimator from flash/no-flash image pairs. While retaining high practicability on modern smartphones, our proposal takes advantage of flash-induced shading variations, and leverages
curvature-guided detail enhancement strategy, improving surface detail recovery and mitigating shape-reflectance ambiguity effectively. To evaluate our proposed method, we further present EvalFlash, the first real-world flash/no-flash evaluation
dataset containing 20 objects aligned with ground-truth surface normals for quantitative benchmarking. Extensive experiments demonstrate the effectiveness of FlashNormal over state-of-the-art single image-based methods and show a significant out-performance over flash/no-flash-based normal estimation method on EvalFlash.
Read the full abstract ▾
All authors Ruiyang Chen, Feiran Li, Heng Guo, Zhanyu Ma