Computer Vision
MotionSpaceFlow: Representation-Aware Flow Matching in Direct Motion Space
Qing Yu and Kent Fujiwara
Recent advances in
diffusion and flow models have substantially improved text-driven human motion generation. Yet most methods generate in low-dimensional, temporally downsampled latent spaces learned primarily for reconstruction, a bottleneck that can limit generation quality and preclude direct manipulation of individual frames and joints. We introduce MotionSpaceFlow (MSFlow), a representation-aware flow-matching framework that predicts clean motion directly in continuous motion space without a learned encoder or decoder. To account for the anisotropic structure of direct motion representations, we propose representation-aware noise scaling and show how the initial Gaussian source scale governs the covariance of intermediate probability-path marginals. We further introduce a Representation-Aware
Multimodal Diffusion Transformer (RA-MMDiT), which jointly updates
token-level
language and full-resolution motion features through joint
attention while adapting temporal information flow to the motion representation: causal
attention for incremental features defined by frame-to-frame changes, and bidirectional
attention for global features such as absolute joint coordinates. Across different
datasets and motion representations, MSFlow achieves state-of-the-art text-to-motion performance. Its global representation variant additionally enables zero-shot, inference-time control over any joint or frame through projection sampling without control-conditioned training, delivering leading motion quality with exact constraint satisfaction.
Read the full abstract ▾
Computer Vision
CRF Loss is How Networks Should Learn Boundaries in Weakly Supervised Segmentation
Joshua Li and Yuri Boykov
Weakly Supervised Semantic Segmentation (WSSS) learns pixel-level predictions from image-level tags. Recent work focuses on improving coarse CAMs extracted from large
vision-
language models (commonly CLIP), but does little to improve their accuracy along segment boundaries. That job is instead delegated to a post-processing method like DenseCRF. However, because DenseCRF relies on low-level colour cues, it can flip correct labels to incorrect ones when neighbouring pixels share similar colours. SAM has recently been adopted as a natural alternative, yet it simply takes on DenseCRF's role as an intermediate "refinement" step that outputs one-hot pseudo-labels in prior work. By discarding the valuable uncertainty in CAMs, these one-hot pseudo-labels turn borderline errors into confidently wrong targets. Our key insight is that CAMs should supervise training alongside SAM boundaries, each through its own loss, rather than being fused together into a single hard target. Inspired by CRF potentials, we propose a framework that disentangles soft pseudo-labels as unary supervision and binary edge maps as pairwise supervision. We realize our framework in a single-stage model, DS-CRF, using CAMs from dino.txt and boundaries from SAM. DS-CRF sets a new state-of-the-art of 56.5% mIoU on MS COCO.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
Decision Readouts for Text-Mediated Video Anomaly Detection: An Exploratory Evaluation of Jev and Qwen
Xukui Qin, Youting Wang, Xinjie He et al. (+4 more)
How much does the decision readout matter when
video-derived textual evidence is held fixed? We evaluate Jev typed decisions and three Qwen readouts on a sparse development sample of 40
videos and 400 target anchors from UCF-Crime and XD-Violence, each presented as a summary and ordered captions. Each
dataset contributes 20 source groups and 200 anchors, including only 10 and 37 positives, respectively. The original five-backend pilot requested 4,000 predictions; Jev Choice returned 776 valid responses out of 800 under the study's strict numerical policy, blocking its full-coverage quality comparison. On XD captions, Jev Noul achieved 75.99% average precision versus 48.47% for Qwen generated probability and 57.81% for the stronger local ordinal-likelihood expectation. The latter paired difference was 18.18 percentage points (95% source-group bootstrap interval 5.53-31.50). UCF did not show a corresponding advantage: caption ROC-AUC was 52.26% for Noul and 65.95% for ordinal likelihood. Both probability readouts had higher, hence worse, UCF Brier scores than the evaluation-prevalence reference of 0.0475. We additionally audit historical LAVAD scores at exactly matched anchors and distinguish response structure from numerical consistency. A binary-likelihood control is missing. These exploratory offline results characterize ranking, probability quality and interface failures; they establish neither a causal typed-interface benefit nor general superiority, calibration or end-to-end acceleration.
Read the full abstract ▾
All authors Xukui Qin, Youting Wang, Xinjie He, Ziyang Luo, Runxiong Wu, Yan-Syuan Chen, Zhongyao Chu
Computer Vision
Enhanced Video Text Editing with Trajectory-Aligned Glyph Rendering
Shulian Zhang, Xiangyu Shu, Wenbo Li et al. (+2 more)
Video text editing aims to replace or add text in a
video while keeping the rest of the
video unchanged, which requires the edited text to be correct in every frame and to move coherently with the scene. Despite the remarkable progress of
video diffusion models, they struggle to reproduce exact stroke structures and often produce garbled or wrong characters, especially for characters with complex strokes. To address this, we propose a trajectory-aligned glyph rendering reference that provides explicit per-frame glyph guidance following the position and perspective of the text, and a depth-normalized recognizer feature supervision that supervises the generated text on multi-depth features of a frozen text recognizer with per-depth normalized errors, targeting stroke errors overlooked by the
diffusion loss. We further build VTEdit, a
benchmark of 288 real-scene clips with 440 annotated text trajectories covering text replacement and text addition, which will be publicly released to facilitate future research. Experiments on VTEdit show that our method outperforms image text editing methods,
video editing methods, and commercial models in text accuracy and background preservation, achieving a sentence accuracy of 0.9408, and receives the highest preference in a user study.
Read the full abstract ▾
All authors Shulian Zhang, Xiangyu Shu, Wenbo Li, Jian Chen, Yong Guo
Computer Vision
AGILE-GS: Anchor-Guided Fast Next-Best-View Selection for Active 3D Gaussian Splatting
Amirhossein Mollaei Khass and Nader Motee
Radiance fields need hundreds of views, and their placement matters as much as their number. Next-best-view (NBV) selection for 3D Gaussian Splatting (3DGS) usually scores every candidate in the pool and keeps one. Searching for information and choosing a camera, however, are separable problems. We present AGILE-GS, an anchor-guided NBV method that separates the two. A virtual anchor pose is optimized on SE(3) by Riemannian gradient ascent on expected information gain. It need not be reachable or in the pool; it marks where the model is most uncertain. Candidates are scored against the anchor's viewing geometry, and a greedy ridge-leverage step distills the pool into a small, non-redundant shortlist without rendering any candidate. The shortlist can be used in two ways. AGILE-GS takes the first view on it as the next view, so no Fisher information is computed for any candidate. AGILE-GS+ computes the Fisher information gain of each shortlisted view and picks the best, so the expensive evaluation runs on a handful of views rather than the whole pool. On standard
benchmarks and in closed-loop embodied acquisition, both match or exceed existing baselines while cutting selection latency by one to two orders of magnitude.
Read the full abstract ▾
Computer Vision
Natural Image Autoencoder-Based fMRI Representations for Trait and State Prediction
Juhyeon Park, Yeonwoo Kim, Peter Yongho Kim et al. (+5 more)
Foundation models pre-trained on large-scale fMRI
datasets have shown strong downstream performance, but at substantial data and computation cost. To investigate how much fMRI-specific pre-training is actually needed for such performance, we introduce FReD, which derives fMRI representations from a frozen Deep Compression AutoEncoder (DCAE) pre-trained exclusively on natural images and pairs them with a task specific readout. For trait prediction, FReD summarizes frame-wise representations by their temporal mean and log-standard deviation and applies linear probing, with late fusion across two normalization schemes. For state prediction, it represents each frame as a single
token and models temporal dependencies with a shallow
Transformer. Across four resting-state
datasets spanning six trait-prediction targets, linear probes on frozen DCAE features generally outperform those on fMRI foundation model representations and remain competitive with fully fine-tuned fMRI foundation models. On three task-fMRI state-prediction tasks, a temporal readout on DCAE features performs comparably to the strongest foundation models evaluated. A Gaussian injection analysis further shows that localized signal changes are recovered more accurately from the frozen DCAE features than from the evaluated foundation-model representations. Together, these results show that strong performance on current fMRI
benchmarks is possible without fMRI-specific representation pre-training, making frozen natural-image features as a useful baseline for assessing its added value.
Read the full abstract ▾
All authors Juhyeon Park, Yeonwoo Kim, Peter Yongho Kim, Yansen Wang, Mingqing Xiao, Dongqi Han, Dongsheng Li, Taesup Moon
Computer Vision · Cross-listed from cs.LG
WorldGraph: Graph-Native World Modeling
Zezhong Ding, Yipeng Li and Xike Xie
World models infer latent states of an environment to capture its underlying dynamics and predict future
evolution. Many real-world environments, however, are inherently relational and observed as evolving graphs, where entities, relations, and their properties change over time. Prior graph-related world models use graph structures to organize internal states or support task-specific
reasoning, rather than treating an evolving graph itself as the modeled world. We instead study graph world modeling (GWM), where graph
evolution itself constitutes the world dynamics. We formulate graph world modeling over observed graph
evolution, latent graph states, and heterogeneous graph-transition predictions. Based on this formulation, we construct GWM-Zero, a
benchmark covering node-, edge-, and graph-level transitions over eight temporal graph
datasets. We propose WorldGraph, which combines a state-aware graph
transformer for multi-granularity structural and transition-conditioned
evolution modeling with transition-aware GRPO using dynamic grouping and structure-aware verifiable rewards. Extensive experiments on GWM-Zero show that WorldGraph consistently outperforms representative graph representation, temporal graph learning, graph pretraining, and graph world-model baselines across all three transition granularities.
Read the full abstract ▾
Computer Vision
Functional Hand Type Prior for 3D Hand Pose Estimation and Action Recognition from Egocentric View Monocular Videos
Wonseok Roh, Seung Hyun Lee, Won Jeong Ryoo et al. (+5 more)
Current methods for egocentric view action recognition often face challenges in perceiving dynamic hand movements relying solely on geometrical or physical information. In this work, we effectively address this problem by gaining insights into the correlation between functional hand configurations and objects, which improves the detailed interpretation of real-world scenarios. To this end, we introduce a practical taxonomy of hand types based on the functioning perspective and utilize it for per-frame hand type labeling on existing
datasets. We also propose a novel hand action recognition framework considering semantic details of the hand type as prior. This approach boosts the network's understanding of the continuous hand interaction throughout the action sequence. Our whole pipeline consists of three main modules: (1) Feature Extraction, (2) Egocentric Knowledge Module, which estimates 3D hand pose, object category, and hand type leveraging short-term cues, and (2) Egocentric Action Module, which aggregates per-frame knowledge, including text
embeddings of hand type, over a longer time. In our extensive experiments with large-scale
benchmarks, FPHA and H2O, our model outperforms current state-of-the-art methods, demonstrating its superior performance.
Read the full abstract ▾
All authors Wonseok Roh, Seung Hyun Lee, Won Jeong Ryoo, Jakyung Lee, Gyeongrok Oh, Sooyeon Hwang, Hyung-gun Chi, Sangpil Kim
Computer Vision
Antonio Jun, Haoshui Yu, Zhengyi Lu et al. (+2 more)
Vision-
Language Models (VLMs) are far more reliable at recognizing what appears in a
video than at
reasoning about its spatial and temporal properties, such as metric distances, object dimensions, and consistent object identities across frames. We present
Geometric Code, a perception-to-geometry pipeline that computes explicit spatial structure from
video and supplies it to VLMs as context to augment
reasoning. A perception layer segments and classifies objects and recovers depth, camera pose, and intrinsics from monocular RGB
video. A deterministic
geometric engine then back-projects, merges, and cleans these outputs into a spatial code, including per-object positions, dimensions, counts, inter-object distances, appearance order, and room geometry. The code is serialized into VLMs'
prompts, either alongside the
video or replacing it entirely. Specifically, there is no component trained or fine-tuned in our approach. On VSI-Bench, augmenting 2B and 4B open models with the spatial code improves average accuracy by +4.1 points over the frames-only baseline, with the largest gains on numeric estimation tasks such as absolute distance (+24.1 points). The results suggest that explicitly computed geometry, delivered through the
language channel, recovers spatial competence that small VLMs cannot extract from pixels alone.
Read the full abstract ▾
All authors Antonio Jun, Haoshui Yu, Zhengyi Lu, Huirong Fu, Yao Qiang
Computer Vision
CAST: Reconstruction-Coupled Acceleration of Interactive World Models
Leyang Chen, Junyi Wu, Fanqing Kong et al. (+2 more)
Interactive world models must respond quickly to controls while preserving scene consistency. Existing acceleration methods can miss heterogeneous control responses and spatial transport when recovering skipped features. We observe that interaction-induced feature changes correlate with approximation error, while low-frequency interpolation errors are phase-sensitive and show more predictable phase progression. These findings motivate CAST, a reconstruction-coupled inference framework. CAST selects anchors by interaction sensitivity and cross-layer coverage, reconstructs skipped residuals with frequency- and confidence-aware Phase-Aware Reconstruction (PAR), and coordinates historical KV routing according to downstream reconstruction responsibility. On
Matrix-Game 3.0 and HY-World 1.5, CAST achieves 2.15x and 3.48x speedups, respectively, while maintaining visual quality close to Native (Figure 1). It also attains the highest VBench scores among compared methods and leads non-native baselines on seven and six of thirteen WorldMark dimensions, demonstrating a balance of generation speed, visual quality, and interactive responsiveness under real-time control. Code is available at https://github.com/lokiniuniu/CAST.
Read the full abstract ▾
All authors Leyang Chen, Junyi Wu, Fanqing Kong, Shaoqiu Zhang, Yulun Zhang
Computer Vision
Beyond Geometry: Benchmarking and Consistency Reasoning for 3D Logical Anomaly Detection
Zhiqiang Qin, He Xie, Junfei Yi et al. (+5 more)
Existing 3D industrial anomaly detection mainly targets local
geometric deviations. In contrast, many industrial anomalies violate object-level design or assembly rules, which we define as 3D logical anomalies. To address these challenges, we introduce the Industrial Logical Anomaly Detection
Dataset (ILGAD), the first scalable
benchmark dedicated to logical anomalies in industrial point clouds. ILGAD contains 2,774 samples from 15 categories with point-level annotations and covers existence, specification, pose, and assembly-state errors. To detect such 3D logical anomalies, we propose a consistency
reasoning framework that assesses whether local geometry, structure coverage, and spatial relations conform to the normal design. The framework detects
geometric changes, unsupported expected structures, and abnormal local arrangements. Experiments on ILGAD, Anomaly-ShapeNet, and IEC3D demonstrate superior object-level detection and point-level localization, showing that the framework effectively detects logical anomalies and generalizes to conventional
geometric defects.
Read the full abstract ▾
All authors Zhiqiang Qin, He Xie, Junfei Yi, Yang Yang, Hao Wang, Yunkang Cao, Hui Zhang, Yaonan Wang
Computer Vision
Analytical and Convolutional Neural Network-Based Motion-Vector Propagation for Efficient Video Object Detection
Ashiyana Abdul Majeed, Mahmoud Meribout and Neethu Joseph
Continuous
video analytics requires accurate localization at low latency within embedded power budgets. This paper presents a hardware-software design methodology that reuses codec motion vectors (MVs) between detector invocations. Two alternative models support translation and scale changes: analytical motion-vector propagation (Analytical-MV) and learned propagation using a convolutional neural network (CNN) (CNN-MV). The learned model uses convolutional operations and independent object updates suited to parallel execution on an edge graphics processing unit (GPU). Analytical-MV combines a harmonic-mean precision-recall score (F1) of 0.909 with a mean end-to-end latency of 9.03 ms and an energy consumption of 0.177 J per frame, yielding the lowest latency and energy among the evaluated configurations. Relative to detection on every frame, it reduces mean latency by 25.9% and energy per frame by 36.4%. CNN-MV offers a different trade-off: its fastest configuration raises recall from 0.871 for Analytical-MV to 0.890 and lowers mean power from 19.64 to 17.32 W, while achieving a latency of 18.42 ms and an energy consumption of 0.319 J per frame. It is therefore useful when recall or operating power is more important than minimum latency and energy. Execution on a deep learning accelerator (DLA) further reduces time-averaged GPU utilization relative to GPU execution. Host-processing optimization substantially improves both latency and energy, demonstrating the value of jointly designing temporal models and their execution pipelines.
Read the full abstract ▾
Computer Vision
PrefLUT: Reusable and Refinable Personalized Color Editing from Pairwise Preferences
Chuanzhi Xu, Langyi Chen, Chengkun Yue et al. (+5 more)
Photographic color editing is inherently personal: the same image can appear too warm, too muted, or already satisfactory to different users. Most lookup table (LUT) and reference-guided methods target a specified appearance rather than model persistent preferences from repeated user choices. To address this gap, we introduce PrefLUT, a reusable and refinable user-preference modeling framework for deployable 3D LUTs, encoding ordered preferred/non-preferred image pairs into a lightweight Reusable User Profile that is reused across queries and refined using additional user preference pairs, without per-user optimization. A Query-Conditioned LUT Predictor combines this profile with each image to predict a LUT latent vector and edit strength. An Identity-Residual LUT Decoder and Edit-Strength Controller then produce an exportable 3D LUT. Experiments on three
datasets demonstrate effective personalized editing and general-purpose enhancement. Each quantized profile requires only 260 bytes, and editing takes 1.365 ms/image on an RTX 5090 GPU. We also introduce the Preference-Conditioning Verification Protocol (PCVP), an evaluation protocol to verify whether personalized image edits depend on user preferences and the query image through controlled changes to user profiles, preference orders, pair correspondences, and query images.
Read the full abstract ▾
All authors Chuanzhi Xu, Langyi Chen, Chengkun Yue, Xuanhua Yin, Boyu Wei, Qingwen Zeng, Zihan Deng, Weidong Cai
Computer Vision
SpatialSkill: Self-Evolving Skills for Cross-View Spatial Reasoning
Ruifan Zuo, Guocheng Hu, Wanshui Gan et al. (+3 more)
Cross-view spatial
reasoning requires a model to align different viewpoints into a coherent spatial representation, yet this ability remains challenging for
vision-
language models despite being natural to humans. Existing methods typically improve spatial
reasoning by updating model weights, which keeps the acquired knowledge implicit and tied to a specific backbone. We propose SpatialSkill, a weight-update-free framework that enables a frozen
vision-
language model to accumulate explicit natural-
language reasoning skills from offline trajectories. Unlike symbolic tasks, perceptual skills cannot be reliably verified simply by executing them: a plausible spatial rule may lack visual support or require transformations that the frozen model cannot perform. SpatialSkill therefore admits candidate skills only after visual-grounding and executability checks, constrains manual
evolution to prevent harmful regressions, and routes skills by spatial-
reasoning category to reduce negative transfer. On CityCube, across four frozen executors, SpatialSkill yields consistent gains, and a 9B executor equipped with SpatialSkill surpasses the strongest closed-source reference in our evaluation. The skills are stored in a versioned natural-
language manual, making the
reasoning strategies explicit and auditable without modifying model parameters. Code at https://github.com/vindahi/SpatialSkill.
Read the full abstract ▾
All authors Ruifan Zuo, Guocheng Hu, Wanshui Gan, Junyi Wang, Xiang Lei, Tian Gan
Computer Vision · Cross-listed from cs.AI
SpecRegMatch: Robust Semi-Supervised Regression for Vehicle Interior Noise Prediction
Sejin Sim, Jinsoo Bae and Seoung Bum Kim
The rapid advancement of artificial intelligence has observed increased application in predicting vehicle interior noise levels within the automotive industry. However, the collection of labeled data for training models in this context involves significant costs. Previous studies in semi-supervised regression (SSR) have effectively mitigated the reliance on labeled data by incorporating unlabeled data. Nonetheless, these approaches often introduce a high computational cost due to the training of multiple models and data sampling. This study introduces SpecRegMatch, a novel SSR method aimed at addressing the computational cost associated with training by leveraging a single model, thus eliminating the need for multiple data samplings. SpecRegMatch integrates consistency regularization and information maximization to robustly train the model, achieved through various augmentations applied to both the
embedding vectors and predicted values. Experimental results demonstrate that SpecRegMatch achieves state-of-the-art performance across various scenarios, even when using a single model. It attains a remarkable performance, as indicated by an R^2 score of 0.434. This is especially noteworthy in scenarios where labeled data is scarce. You can access the code for our proposed method at https://github.com/sejin-sim/SpecRegMatch.
Read the full abstract ▾
Computer Vision
The Devil is in the Spectrum Bias: Spectrum-Balanced Feature Matching for Robust Representation Distillation
Kuniaki Saito and Yoshitaka Ushiku
Large visual foundation models have demonstrated remarkable transferability across a wide range of downstream tasks. To deploy such models efficiently, feature matching has become a popular knowledge
distillation approach that transfers teacher representations to smaller student models without requiring labeled data. However, we show that the conventional feature matching objective with L2-distance is inherently biased toward reconstructing dominant spectral directions of the teacher representation, while under-optimizing low-variance directions that often contain task-relevant information. To address this, we propose Spectrum-Balanced Feature Matching, SpecMatch, a simple objective that adaptively emphasizes under-optimized spectral directions while preserving the relative importance of dominant directions. SpecMatch is easy to implement and introduces negligible computational overhead. Extensive experiments on image recognition demonstrate that SpecMatch consistently improves downstream adaptation across diverse tasks, including image classification, anomaly detection, medical image analysis, and domain generalization. In particular, SpecMatch outperforms conventional feature matching in 40 of 42 teacher–student and training-setting combinations, while consistently improving over the original student model in all settings. We further demonstrate that the proposed objective generalizes beyond
vision, improving downstream performance across six
protein understanding tasks.
Read the full abstract ▾
Computer Vision
Advancing Wildlife Conservation through Multimodal Animal Re-Identification with Environmental Metadata
Yuzhuo Li, Di Zhao, Tingrui Qiao et al. (+3 more)
Identifying individual animals is crucial for effective wildlife monitoring and conservation efforts. Recent advancements in computer
vision have shown promise in animal re-identification (Animal ReID) by leveraging data from camera traps. However, existing Animal ReID
datasets rely exclusively on visual data, overlooking environmental metadata that ecologists have identified as highly correlated with animal behavior and identity, such as temperature and circadian rhythms. Meanwhile, modern
vision-
language models (VLMs) offer rich
multimodal reasoning capabilities, but existing resources underutilize their text-processing potential. To address these limitations, we propose MetaWild, a
multimodal Animal ReID
dataset comprising 20,890 images across six species, paired with environmental metadata extracted from embedded camera trap overlays and scene contexts. Additionally, to facilitate the use of metadata in existing ReID methods, we propose the Meta-Feature Adapter (MFA), a lightweight module that can be incorporated into existing VLM-based ReID methods, allowing ReID models to leverage both environmental metadata and visual information to improve ReID performance. Experiments on MetaWild show that combining baseline ReID models with MFA to incorporate metadata consistently improves performance compared to using visual information alone, validating the effectiveness of incorporating metadata in re-identification.
Read the full abstract ▾
All authors Yuzhuo Li, Di Zhao, Tingrui Qiao, Yihao Wu, Bo Pang, Yun Sing Koh
Computer Vision · Cross-listed from cs.RO
AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Driving
Haoran Zhu, Wancong Zhang, Yann LeCun et al. (+1 more)
Autonomous driving requires world models that can understand the physical world, reason and plan, and operate safely. In this paper, we first systematically evaluate existing action-conditioned joint-
embedding predictive architecture (JEPA) world models, including LeWM, DINO-WM, and JEPA-WM for end-to-end autonomous driving (E2EAD). To isolate world-model quality from policy learning, we employ a goal-conditioned zero-shot planning setting that evaluates these models using ground-truth future observations as goals, without training any driving policy. We find that existing JEPA-based world models are either accurate for driving but computationally expensive, or computationally efficient but insufficient for planning. To address this trade-off, we propose AD-E2E-JEPA, which introduces a SIGReg-regularized learnable projector applied to projected patch
embeddings. The projector reduces the number of planning patches by $16\times$ and the
embedding dimension by $4\times$, achieving a $100\times$ inference speedup while retaining planning performance, with a 0.8-second runtime for an 8-frame rollout over 256 candidate trajectories. Without training any driving policy, the world model itself reaches the goals located 20 meters away on average within the displacement of respectively 4.0/2.8 meters, using world-model rollouts over trajectory vocabularies of respectively 256/8,192 candidates. On the NAVSIMv2
benchmark, it achieves 67.3/72.9 EPDMS with multiplicative safety metrics and 84.1/86.5 EPDMS$^{\dagger}$ without them in goal-conditioned zero-shot planning. Experiments further show that the self-supervised pretrained projector improves downstream imitation learning performance from 80.2 to 85.4 EPDMS. The source code is available at https://github.com/HaoranZhuExplorer/AD-E2E-JEPA
Read the full abstract ▾
All authors Haoran Zhu, Wancong Zhang, Yann LeCun, Anna Choromanska
Computer Vision · Cross-listed from cs.AI
K-OPSD: Verifiable On-Policy Self-Distillation for Post-Training Vision-Language Models on AEC Drawings
Yunfei Bai, Enrico Chionna, Akash Amol et al. (+2 more)
Interpreting architecture, engineering, and construction (AEC) drawings is hard for general
Multimodal Large
Language Models (MLLMs) and
vision-
language models (VLMs). We introduce K-OPSD, a VLM post-training methodology for improving AEC drawing understanding. Building on On-Policy Self-
Distillation (OPSD) with verifiable supervision, we construct a teacher from the model's own best-of-N generations, certified by a process-level verifier, and rescue failed
prompts by resampling under a hint that exposes the verified answer. We then perform an on-policy model update by training on verified completions with a cross-
entropy inner-loss, outperforming the bounded
token-wise generalized Jensen-Shannon divergence (JSD) used by on-policy
distillation. Using K-OPSD, we fine-tune Qwen3-VL models on the AECV-Bench
dataset. The resulting models attain the top average judge score (0.819) and combined accuracy (0.738), achieving competitive results against open-source baseline models. The recipe transfers to the out-of-domain ArchCAD
dataset, where the 8B model gains most. We present the verifier suite and the continual learning and self-improving pipeline, our results provide preliminary evidence that verifier-guided self-
distillation is a promising route toward more reliable machine reading of architecture drawings.
Read the full abstract ▾
All authors Yunfei Bai, Enrico Chionna, Akash Amol, Kawaljit Singh KC, Joern Tinnemeyer
Computer Vision
WhiteCon: Semi-Supervised Domain Adaptation Regression Through Whitening Transform and Dual Consistency
Se Jin Sim and Seoung Bum Kim
Domain adaptation is crucial for addressing distributional shifts that degrade model performance across domains. While most existing research has centered on classification, semi-supervised domain adaptation regression (SSDAR) for continuous-output tasks remains largely unexplored, particularly in practical scenarios with limited labeled target data. To address this gap, we propose semi-supervised domain adaptation regression through whitening transform and dual consistency (WhiteCon), which combines domain-specific whitening transform (DWT) and dual consistency regularization to enhance training stability and domain adaptation. DWT reduces the variance of the model parameters by transforming the feature covariance
matrix into an identity
matrix, thus stabilizing training under ordinary least squares assumptions. In addition, variance consistency regularization, as part of dual consistency regularization, aligns the variances of weak, strong, and mixup-augmented features to improve resilience against augmentation-induced perturbations. Empirical evaluations on various
benchmark datasets under SSDAR settings demonstrate that the proposed WhiteCon achieves state-of-the-art performance compared to existing methods, effectively addressing domain shifts in regression tasks. The code for WhiteCon is available at https://github.com/sejin-sim/WhiteCon.
Read the full abstract ▾
Computer Vision
SNaP: One-Step Posterior Sampling for Noisy Inverse Problems
Shirin Shoushtari, Edward P. Chandler, Xiao Shi et al. (+1 more)
Diffusion and flow-matching models can produce high-quality posterior samples for inverse problems, but typically require tens to thousands of network evaluations per draw. MeanFlow enables one-step generation, yet applying it to inverse problems leaves no intermediate steps at which to enforce measurement consistency. We introduce SNaP, a one-step MeanFlow posterior sampler for linear inverse problems with Gaussian noise. Its central innovation is a measurement-adapted source: a Gaussian distribution whose mean and anisotropic covariance are determined by the measurement operator, observation, and noise level. The source anchors well-measured directions while preserving variation where the measurements are weak or uninformative. We show that the exact conditional flow transports this source to the true posterior. Across natural-image restoration and multi-coil MRI, SNaP produces diverse, high-quality samples with one network evaluation per draw, 30 to 2250 $\times$ faster than iterative samplers.
Read the full abstract ▾
All authors Shirin Shoushtari, Edward P. Chandler, Xiao Shi, Ulugbek S. Kamilov
Computer Vision · Cross-listed from cs.RO
Xuan Wang, Yinan Wu, Haoran Duan et al. (+1 more)
Vision-
Language-Action (VLA) models integrate pretrained
Vision-
Language Models (VLMs) with action heads for
robot control. Common action heads have distinct limitations: point regression provides only a point estimate of the action distribution, while standard flow-matching samplers require costly iterative sampling. To address these limitations, we unify regression and flow matching under a shared objective and extend it to derive a
quantile objective. This
quantile objective guides the design of our
Quantile Head, which predicts a median and positive gaps to form ordered marginal action
quantiles in one forward pass. These
quantiles support multiple sampling strategies without retraining and are jointly supervised to train the default median policy. Our local analysis of this joint supervision shows that, with calibrated nearby
quantiles, fixed gaps, and matched correction speed, direct median updates have lower variance than under median-only supervision. Experiments show that this jointly supervised median policy achieves the highest average success rates among the compared methods on LIBERO, LIBERO-Plus, LIBERO-Pro, and two real-
robot tasks, together with the shortest mean episode time among matched LIBERO baselines; code is available at https://github.com/xwangrs/
Quantile-Head-for-VLA.
Read the full abstract ▾
All authors Xuan Wang, Yinan Wu, Haoran Duan, Jungong Han
Computer Vision
ARCH-B: Architectural Representation, Comprehension and Hierarchy Benchmark
Kieran Sagar Parikh and Jose Luis Garcia del Castillo y Lopez
Multimodal models increasingly interpret visual environments, but their ability to recognize the same building across photographs, floor plans, elevations, sections, and renderings remains poorly characterized. We introduce ARCH-B, a
benchmark of 354 four-choice questions across 11 cross-representational archetypes, constructed from a building-linked corpus of 3.9 million architectural images using visually similar distractors, model-guided difficulty screening, and manual validation. We evaluate 25
multimodal models and collect 5,830 responses from non-expert human participants. Model accuracy ranges from 10.45% to 83.90%, compared with a human baseline of 35.35%. Models perform comparatively well on mixed-representation outlier detection and photograph matching, but remain weaker on floorplan-to-photograph correspondence. Human and model difficulty across archetypes is only weakly correlated (Spearman's (\rho=0.33)). Held-out evaluation confirms that the difficulty identified during screening generalizes beyond the curation models. ARCH-B provides a diagnostic evaluation of visual correspondence and representation transfer across architectural media.
Read the full abstract ▾
Computer Vision
SCOPD: Sparse-Context On-Policy Self-Distillation for Efficient Vision-Language Models
Ahmadreza Jeddi, Enming Zhang, Jasper Gerigk et al. (+12 more)
Reasoning vision-
language models (VLMs) process images and
videos as long sequences of visual
tokens, making inference expensive. Training-free
token pruning reduces this cost, but aggressive compression can sharply degrade performance, often attributed to irreversible loss of task-relevant visual information. We show that this explanation is incomplete. In a fixed-context Pass@K analysis, repeated sampling from the same pruned visual representation recovers many examples missed by greedy decoding, indicating that useful visual evidence can remain accessible but be used unreliably. We call this the representation-utilization gap. Motivated by this observation, we introduce SCOPD, a sparse-context on-policy self-
distillation framework in which a student generates
reasoning trajectories from pruned visual
tokens while a privileged full-context teacher supervises the same on-policy prefixes. SCOPD requires no ground-truth responses, architectural changes, or additional inference-time computation. We further introduce SCOPD+, which uses a small visual-budget intervention to identify visually sensitive response positions and selectively distill them. At 10% visual-
token retention, the Vanilla model retains 86.37% of its unpruned performance across 13
benchmarks. SCOPD raises this to 90.49%, while SCOPD+ further improves it to 92.43%. Across
token budgets,
benchmarks, and
pruning operators, our results show that efficient
reasoning depends not only on which visual information survives
pruning, but also on how reliably the model learns to use it.
Read the full abstract ▾
All authors Ahmadreza Jeddi, Enming Zhang, Jasper Gerigk, Hakki Karaimer, Mozhgan Nasr Azadani, Jiayun Luo, Minh Ngoc Le, Gholamali Aminian, Hugo Buurmeijer, Yongchao Chen, Leonid Sigal, Igor Gilitschenski, Konstantinos G. Derpanis, Marco Pavone, Babak Taati