Computer Vision · Cross-listed from cs.CL
A machine-readable catalogue of the Tsiolkovsky papers (fond 555, Archive of the Russian Academy of Sciences), and a way to measure how well its handwriting can be read
Vladimir Beskorovainyi
The personal archive of Konstantin Tsiolkovsky (1857-1935) is held as fond 555 of the Archive of the Russian Academy of Sciences. The archive scanned the fond and published the images, but with no queryable catalogue, no full-text search and no
dataset: the holdings can only be browsed one page at a time. This paper describes a machine-readable catalogue of all 2,019 files and 51,008 scans, a dating for 1,969 files taken from the archive's own descriptions, a page-level classification of every scan into handwriting and typescript, and a machine transcription of the fond in full: all 2,019 files and all 51,008 scans. It also reports a way to measure handwritten-text-recognition accuracy in an archive with no ground truth. Archives of the typewriter era often preserve one text twice, as manuscript and as a typed copy; transcribing both and comparing isolates the reading error, since source and pipeline are identical and only page difficulty differs. Across 1,759 such pairs from 224 files, two readings of a handwritten page agree on a median 37% of words and share a longest verbatim run of a median 10 words; the median is unchanged from the 294 pairs of the first version, on a sample six times larger. On two files that also have a published edition the estimate can be checked against ground truth: it is unbiased to within a percentage point and ranks pages as the truth does (rank correlation 0.92 where the edition is a faithful witness). This bounds use: two variants of one work here share 19% of words, below the rate at which two readings of a single page agree, so the redactions cannot be collated word by word at this quality. That negative result is reported as such, and the constraint is built into the tool.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Predictive Enhancement Calibration for Latent Breast MRI Virtual Contrast Enhancement
Qin Lei and Hao Wu
Virtual contrast enhancement (VCE) synthesizes enhanced breast MR images from pre-contrast acquisitions. Modern latent generators offer strong image priors, but their bounded natural-image autoencoders conflict with the non-canonical intensity scale of MRI. We show that the upper bound can alter radiomic fidelity before generation, while scaling source and target independently creates a coordinate inconsistency. We propose Predictive Enhancement Calibration (PEC), which represents each pair in a shared, case-adaptive coordinate during training and predicts its unavailable upper endpoint from the pre-contrast image at inference. We integrate PEC with a pretrained FLUX latent flow
transformer via parameter-efficient reference conditioning. Target round trips first isolate representation loss before generation; near-matched conditional models then compare PEC with fixed-wide and separate coordinates under comparable training budgets and backbone settings. On the fixed internal MAMA100 development cohort, PEC improves all eight point estimates in this source-only VCE setting, with paired evidence strongest for MSE and LPIPS.\noindentCode: https://github.com/tanlei0/pec-breast-mri-vce
Read the full abstract ▾
Computer Vision
SlimVLM: Sensitivity-aware Dynamic Structured Pruning with Adaptive Visual Token Selection for Efficient Vision-Language Models
Yaozhi Wen, Jialong Guo, Zhenliang Ni et al. (+2 more)
While
Vision-
Language Models (VLMs) have demonstrated remarkable performance in processing and understanding both text and images, their large parameter sizes lead to significant computational overhead, limiting their deployment on resource-constrained devices. While
pruning has been effective for compressing Large
Language Models (LLMs), directly applying it to VLMs leads to significant performance drops, largely due to redundant visual
tokens interfering with importance estimation. To this end, we propose SlimVLM, a structured
pruning framework designed to compress VLMs while preserving their task performance. We introduce an adaptive visual
token selection strategy for VLMs that leverages average text-to-visual
attention scores to assess the importance of visual
tokens, removing redundant ones during
pruning based on a set threshold, thereby optimizing the importance calculation. Recognizing the varying tolerance to sparsity across different modules, we also propose a Sensitivity-aware dynamic
pruning mechanism that determines the appropriate
pruning ratio for each module by calculating the linear reconstruction error between the outputs of the pruned and unpruned modules, ensuring overall performance stability. Experimental results show that SlimVLM outperforms existing methods across multiple
multimodal benchmarks, achieving state-of-the-art performance.
Read the full abstract ▾
All authors Yaozhi Wen, Jialong Guo, Zhenliang Ni, Han Shu, Xinghao Chen
Computer Vision
Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learning
Kejian Zhu, Zhuoran Jin, Dongqi Huang et al. (+4 more)
Recent works train
agents by constructing large-scale
multimodal environment pools. However, we find that simply increasing the number of
multimodal environments does not always benefit. We further analyze the limitations in current
multimodal environment distributions through a series of experiments. Based on these findings, we study how to build more effective training environment distributions from two dimensions: **diversity** and **difficulty structure**. For diversity, we propose **Ability-aware Environment Selection (AES)** to obtain diverse environment sets. For difficulty structure, we propose **Hierarchical Difficulty Curriculum (HDC)**, which organizes curriculum learning through two difficulty levels: harness weakening and state-scale progression. Experiments show that AES and HDC effectively improve
multimodal agent training.
Read the full abstract ▾
All authors Kejian Zhu, Zhuoran Jin, Dongqi Huang, Hongbang Yuan, Yupu Hao, Kang Liu, Jun Zhao
Computer Vision
Compass: Degradation-Simulated Reciprocal Learning with Lightweight Needle RWKV for Multimodal Crack Segmentation under Missing Modalities
Hui Liu, Chen Jia, Fan Shi et al. (+3 more)
In
multimodal crack
segmentation for industrial facilities, the key challenge is preventing missing modalities from degrading pixel-level performance while maintaining low computational cost. Existing methods struggle to address semantic degradation caused by missing modalities. We propose Compass, a lightweight network for robust crack
segmentation under arbitrary missing modalities. Compass comprises Degradation Simulation
Distillation (DSD), Needle Block, and Evidential
Topology-Preserving Fusion (ETPF). DSD constructs a degradation simulation stream that mimics more severe missing conditions and performs reciprocal
distillation with the original stream, decoupling complete perception from degradation adaptation. Within DSD, Feature-Aware Prototype Transmitter (FAPT) performs modality agnostic prototype-guided feature completion to maintain semantic integrity under incomplete modality conditions. As a lightweight backbone, Needle injects crack-direction cues into WKV modulation and combines connectivity-aware gating with anisotropic context probing for structure-aware modeling. ETPF fuses
multimodal features via Dempster-Shafer evidential combination with uncertainty-gated decoding, preserving crack
topology while suppressing unreliable features. Experiments on three
datasets demonstrate state-of-the-art (SOTA) performance under diverse missing modality scenarios. Even with 90% depth modality missing on CrackDepth, Compass achieves F1 of 0.8216 and mIoU of 0.8434 with only 2.58M parameters. The code is available at https://github.com/Karl1109/Compass.
Read the full abstract ▾
All authors Hui Liu, Chen Jia, Fan Shi, Xu Cheng, Mianzhao Wang, Shengyong Chen
Computer Vision
Test-Time Augmentation for Tabular-to-Image Classifiers under Distribution Shifts
Malena Loza, Felipe Grijalva, Eva Milara et al. (+3 more)
Tabular-to-image methods that convert tabular data into visual representations have emerged as a novel paradigm for leveraging the high performance of deep learning models. Despite their advantages, the robustness of these methods under distribution shifts remains under explored. Test-Time Augmentation (TTA) is an effective approach in image classification to improve model generalization and robustness, where predictions over multiple transformed views of each input are aggregated. This work evaluates the impact of TTA techniques on predictive performance under Out-Of-Distribution (OOD) for representations generated by tabular-to-image methods. Six tabular-to-image encoding methods were considered: TINTO, IGTD, DeepInsight, BIE, DistanceMatrix, Fotomics. Twenty-five TTA techniques were used, organized into six types:
Geometric, Photometric, Structural, Frequency/Encoding, Mixup, and Composite. We employed two
datasets from the TableShift
benchmark (HELOC and Voting) that provide in-distribution and OOD test subsets designed to evaluate the effect of distribution shifts on tabular data. The results indicate that TTA improves OOD performance, with composite and photometric strategies providing the best trade-off between robustness and variance. In contrast, frequency-domain transformations that alter the encoder's feature-to-intensity mapping consistently degrade performance. These findings highlight TTA as a promising approach for improving the robustness and generalization of classifiers trained on image representations derived from tabular data, particularly under distribution shifts.
Read the full abstract ▾
All authors Malena Loza, Felipe Grijalva, Eva Milara, Luis Bote-Curiel, Francisco J. Lara-Abelenda, David Chushig-Muzo
Computer Vision
S$^3$-Diff: Structural Semantic Synergy Diffusion Model for High Fidelity Super Resolution of Pathological Images
Jiaming Liang, QiHui Han, Guangye Ou et al. (+6 more)
Digital pathology relies on high-resolution whole slide images for accurate diagnosis, yet limitations in imaging devices, storage, and transmission often make lower-resolution pathology images more common in clinical workflows. Current super-resolution techniques often tend to smooth diagnostically relevant morphology, leading to over-smoothed textures and semantic drift that compromise downstream clinical interpretation. To this end, we develop the Structural Semantic Synergy
Diffusion Model (S3-Diff), a
diffusion framework for high-fidelity super-resolution of pathological images. The core of S3-Diff is Specimen-aware Structural Anchoring (SSA), which combines prognosis-aware tissue support extracted by a fixed SAM with LR-HR gradient discrepancies to generate a specimen-specific structural anchor to preserve pathological morphology. Concurrently, we introduce Structure-guided Semantic Fidelity Tuning (SSFT) to adapt DINOv3 representations using SSA-derived structural supervision. SSFT combines the adapted semantic energy with LR-derived edge and grayscale cues. The resulting control guides denoising to suppress
stochastic artifacts and maintain structural consistency. Extensive experimental results demonstrate that S3-Diff consistently outperforms state-of-the-art methods in both reconstruction quality and downstream survival analysis performance. The source code will be made public.
Read the full abstract ▾
All authors Jiaming Liang, QiHui Han, Guangye Ou, Jiawen Liu, Haolin Chen, Xi Zhong, Jiazhou Chen, Xiaoqi Sheng, Hongmin Cai
Computer Vision
IRIS: Visual-Semantic Binding for Forgery-Resistant Watermarking of Diffusion Images
Xiaoyan Feng, Zheng Gao, Tong Guan et al. (+4 more)
Most in-generation
diffusion watermarks embed patterns independent of the image that carries them, and attackers transplant the marks onto images the generator did not produce, resulting in forgery. Binding the mark to visual semantics prevents such transplantation, yet existing bindings anchor to a proxy image rather than the image they mark. Realizing visual-semantic binding inside generation faces two challenges. The mark derives from the image itself yet enters the sampling trajectory before that image exists, and may itself shift the semantics it binds. The binding also meets opposite sensitivity demands, breaking under semantic change while holding through common processing. We present IRIS, a training-free watermarking scheme that embeds an Intrinsic Ring Identifier from Semantics. IRIS reads a content code from the non-watermarked generated image, derives a one-time ring from the code and a secret key, returns to the final low-noise steps of the same trajectory and blends the ring in, after the semantics it binds are settled. To meet the opposite sensitivity demands, the code is read through a canonicalization shared between
embedding and detection, holding through common distortions and mild regeneration while flipping under semantic change. Detection recomputes the ring from the query image and the key alone, and the mark therefore fails on a foreign or spliced image, with acceptance tracking semantic displacement. On three
prompt datasets IRIS detects reliably and stays close to its same-seed non-watermarked counterpart, a fidelity prior in-generation marks do not reach. While forgeries transfer fixed-pattern marks and regeneration strips post-hoc marks, IRIS alone among the compared marks withstands both.
Read the full abstract ▾
All authors Xiaoyan Feng, Zheng Gao, Tong Guan, Rui Bao, Bokang Zeng, Xiaoyu Li, Jiaojiao Jiang
Computer Vision
MinerU.Chem: A High-Precision System for Optical Chemical Structure and Reaction Recognition
Haote Yang, Jiang Wu, Jingchao Wang et al. (+42 more)
In organic chemistry papers and patents, molecular structures, reaction schemes, and experimental conditions are often presented as molecular structure depictions, reaction diagrams, and complex tables or figures. Such information is difficult for general-purpose document parsing systems to directly convert into machine-readable data. This limits data production for organic chemistry knowledge base construction and for AI for Chemistry tasks such as reaction prediction, retrosynthesis, condition recommendation, molecular property prediction, and drug molecule design. This report introduces MinerU-Chem, a document parsing system for organic chemistry literature integrated into the MinerU online platform. Built on top of MinerU's general document parsing pipeline, MinerU-Chem adds five chemistry-specific modules: chemistry relevance filtering, molecular structure detection, molecule identifier extraction, molecular structure recognition, and reaction scheme parsing. Together, these modules convert organic-chemistry-related image regions in documents into a Molecule Summary List and a Reaction Summary List. For molecular structure recognition, MinerU-Chem uses CARBON (Complex Atomic Representation and Bonding Object Notation) as its core representation. CARBON enables recognition results to preserve both the visual layout of the original image and complex chemical semantics, while supporting the export of standard downstream formats such as MolFile and SMILES. On the SMILES-evaluable subset of MolRecBench-Wild (N=2,392), MinerU-Chem's molecular structure recognition module achieves a SMILES exact-match accuracy of 93.02%, outperforming the best evaluated comparison system, GPT-5.6-Sol (74.87%), by 18.15 percentage points. The system has been integrated into the MinerU online platform and is available at https://mineru.net/OpenSourceTools/Extractor .
Read the full abstract ▾
All authors Haote Yang, Jiang Wu, Jingchao Wang, Xingjian Wei, Lixin Ma, Linye Li, Chen Zhu, Xiaolong Wu, Yuheng Lu, Ziran Zhu, Junyuan Gao, Lingli Ge, Yuan Xu, Huijie Ao, QianQian Wu, Dechen Lin, Huaiyu Gu, Lu Chen, Shengxin Lu, ShaSha Wang, Yuanyuan Cao, Zhejia Yu, Ruijie Zhang, Zimai Tian, Jiaxing Sun, Yinfan Wang, Jiahe Song, Chuang Wang, Yubin Wang, Rui Nie, Hao Zheng, Bowen Jiang, Hongbin Lai, Yifan He, Chengjin Liu, Tingting Zhang, Liqun Wei, Lijun Wu, Bin Wang, Yuqiang Li, Guangyu Wang, Wei Li, Bowen Zhou, Dahua Lin, Conghui He
Computer Vision
GVCCTurbo: Rate-Compute Quality Scheduling for Codebook Driven Generative Compression
Ziyue Zeng, Dingjie Peng, Xun Su et al. (+1 more)
Codebook-driven generative compression uses a pretrained image or
video generator as a zero-shot visual prior and transmits compact codebook indices to guide reconstruction at ultra-low bitrate. Current codecs tie each finite-rate correction to a fresh prior evaluation, so shortening the sampler also removes correction slots that carry target-dependent information. We propose GVCCTurbo, a BPP-driven scheduler that separates expensive prior refreshes from codebook corrections: after calibrating an atom-count operating point and skip-gap ratio once per protocol, it maps a target codebook-payload bitrate to a trajectory length and refresh period, making BPP a schedule input instead of a fixed consequence of sampler length. The same endpoint-prediction and finite-rate steering interface covers GVCC-style rectified-flow
video and DDCM-style
diffusion image compression, preserving zero-training deployment and compatibility with future distilled priors. Native 1080p curves position the complete zero-shot codec in the ultra-low-bitrate regime. In a controlled 720p Wan-GVCC study, the scheduler cuts prior evaluations from 20 to 9 for a $\sim\!44\%$ measured decoding-time reduction shared across the whole schedule family, at a small shared LPIPS cost on high-motion content; within that family, uniform refresh thinning (pure-skip) is a boundary point, and the BPP-aware interior point trades $2.9\%$ fewer codebook-payload bits for consistently higher PSNR at comparable LPIPS. These results support BPP-to-compute scheduling as a controllable extension of sampler-length tuning, without requiring the allocated point to dominate every boundary point.
Read the full abstract ▾
All authors Ziyue Zeng, Dingjie Peng, Xun Su, Hiroshi Watanabe
Computer Vision
Detecting Pose Estimation Failures via Keypoint Self-Consistency
Robin Chan
One common approach to pose estimation involves predicting object keypoints in an image, followed by using Perspective-n-Point
algorithms to compute the object's rotation and translation relative to the camera. While rotations preserve object shapes, this property is often neglected in keypoint-based pose estimation methods, where keypoints are typically predicted independently from each other. As imprecise keypoint predictions negatively affects pose estimation accuracy, it also limits its reliability in downstream tasks. In this work, we explore whether such inaccurate pose estimates can be identified by simply examining spatial locations between 2D keypoints. We propose a set of hand-crafted
geometric features that capture the self-consistency of keypoint predictions, including pairwise distances, reprojection consistency, as well as render and mask consistency. Despite its simplicity, a logistic regression classifier trained on these features reliably detects pose estimation failures, outperforming confidence-based approaches like conformal keypoint predictions that rely solely on keypoint uncertainty.
Read the full abstract ▾
Computer Vision
How Many Labels Are Enough? ALDA: Active Learning Deployment Advisor for Medical Image Classification
Julia Machnio, Mads Nielsen and Mostafa Mehdipour Ghazi
Active learning (AL) promises to reduce the cost of medical imaging projects by lowering the number of clinical labels required. However, practical deployment requires committing to a sampling strategy before the full annotation budget is spent, and choosing the wrong strategy can increase rather than decrease costs. We propose Active-Learning Deployment Advisor (ALDA), a deployment-oriented framework for AL method selection under clinical performance constraints. Given a short pilot phase, ALDA fits a parametric learning-curve model to each candidate strategy, estimates whether that strategy is expected to reach a required clinical performance target, and predicts the number of expert annotations needed to do so. In addition to absolute annotation cost, ALDA introduces a deployment window that quantifies the sensitivity of this cost estimate to uncertainty in the clinical threshold. The final recommendation follows a risk-aware rule: among strategies with near-optimal predicted cost, ALDA prefers the strategy with the narrowest deployment window, the most robust to threshold revisions. Experiments on four medical imaging classification domains show that ALDA predicts the deployment-optimal method from a pilot of 15-30% of the intended budget and reduces annotation costs by up to 82% compared with a poor strategy choice. Rather than introducing a new sampling heuristic, ALDA provides a practical decision layer that answers a deployment-critical question: how many labels are enough?
Read the full abstract ▾
Computer Vision
From Multi-Resolution Cells to Gigapixel Whole Slide Images Foundation Model for Computational Pathology
Basit Alawode, Moshira Ali Abdalla, Dwarikanath Mahapatra et al. (+2 more)
Vision Transformers (ViTs) and their hierarchical variants have achieved strong performance in Computational Pathology (CPath). However, most are pre-trained on single-resolution Whole Slide Images (WSIs), limiting their generalization across arbitrary resolutions. Gigapixel WSIs inherently contain diagnostic patterns at multiple scales, including cellular morphologies, tissue architectures, and global context, mirroring how expert pathologists examine WSIs. We introduce Multi-Resolution Pyramid
Transformer (MRPT), a model that hierarchically aggregates multi-resolution information from cellular to tissue and WSI levels. MRPT employs a biologically meaningful Consecutive Cross-Resolution
Attention (CCRA) mechanism to capture scale-independent interactions and enforces multi-resolution semantic consistency by aligning
embeddings across resolutions, yielding robust and generalizable WSI representations. Pre-trained in a multi-resolution self-supervised manner on 624M patches, 2.4M regions, and 36K WSIs, MRPT learns rich coarse-to-fine histopathology features. Extensive experiments on 34 diverse
datasets show that MRPT surpasses recent foundation models and
Multimodal Large
Language Models (MLLMs) in cancer subtype classification, tissue phenotyping, and Visual Question Answering (VQA) for WSI understanding.
Read the full abstract ▾
All authors Basit Alawode, Moshira Ali Abdalla, Dwarikanath Mahapatra, Muzammal Naseer, Sajid Javed
Computer Vision · Cross-listed from cs.RO
Principles of Robot Autonomy
Daniele Gammelli, Joseph Lorenzetti, Katie Luo et al. (+2 more)
Autonomous
robots are moving rapidly from research labs into everyday life - on roads, in the air, in warehouses, and in space.
Robot autonomy is no longer solely an academic pursuit, but a collection of mature, field-tested methods and tools that practitioners rely on in real-world deployments. This book offers a clear, unified introduction to the methods that make this possible. Built on decades of teaching at Stanford, the text develops the core elements of modern autonomy stacks within a single conceptual framework, bridging classical robotics and modern physical AI. Every major topic is paired with hands-on Jupyter notebooks and implementation-driven exercises, so readers build practical intuition alongside theoretical understanding. The result is a principled, accessible, and deployment-aware foundation for anyone seeking to design, analyze, or contribute to the next generation of autonomous systems. This is a comprehensive resource for students, engineers, and researchers entering one of today's fastest-growing fields.
Read the full abstract ▾
All authors Daniele Gammelli, Joseph Lorenzetti, Katie Luo, Gioele Zardini, Marco Pavone
Computer Vision · Cross-listed from cs.RO
Lightweight 3D Object Detection via Mamba-Based Knowledge Distillation
Quoc Cuong Ninh, Huy Xuan Pham, Anh Tung Nguyen et al. (+1 more)
3D object detection using light detection and ranging (
LiDAR) sensors requires a balance between accuracy and computational efficiency for onboard perception in autonomous driving and robotic
navigation. Many existing
LiDAR-based detection methods employ complex architectures to extract features, integrating large amounts of contextual information to enhance accuracy. This often results in significant computational costs, leading to suboptimal performance on resource-constrained embedded devices. In this study, we propose a knowledge
distillation framework that transfers object-level voxel representations from a strong teacher model to lightweight student models through selective voxel-space feature
alignment. Taking advantage of the linear-time sequence model with selective state spaces (Mamba), we design a multi-branch Mamba teacher backbone and a box-aware feature transfer mechanism that aligns spatially corresponding voxel features between teacher and student networks through a Mamba-based projection module. Experimental results on both a public
dataset and real-world data show that our approach significantly reduces computational load while maintaining competitive accuracy compared with state-of-the-art methods.
Read the full abstract ▾
All authors Quoc Cuong Ninh, Huy Xuan Pham, Anh Tung Nguyen, Dinh Hoan Trinh
Computer Vision · Cross-listed from cs.RO
Continue or Replan? Bernoulli-Continuation Policy Learning for Adaptive Horizon Execution
Weichen Xu, Zhenhua Liu, Lin Luo et al. (+8 more)
Existing chunk-based
Vision-
Language-Action (VLA) models execute a fixed number of actions (i.e., execution horizon) before replanning, turning replanning into a task-agnostic periodic schedule that is independent of task progress. As a result, when no replanning boundary falls before a critical manipulation stage, it is executed from a stale chunk rather than a freshly replanned one. To address this limitation, we propose Bernoulli-Continuation Policy (BCP), a lightweight, plug-and-play framework for adaptive horizon execution that keeps the base VLA frozen. Given a fixed-length action chunk, its continuation head decomposes execution-horizon selection into a sequence of continue-or-replan decisions, which imposes an ordinal, prefix-sharing inductive bias over candidate horizons rather than treating them as independent classes. Since the optimal horizon for each chunk is not observable, we train this head with reinforcement learning from trajectory-level outcomes and introduce a Replanning-Efficiency Reward that jointly rewards task success and efficient VLA usage, discouraging the policy from collapsing to unnecessarily short horizons. On RoboTwin 2.0 with LingBot-VLA as the base policy, BCP improves the average success rate by +11.08% on 13 low-success tasks and from 89.88% to 93.94% (+4.06%) across all 50 tasks. Although trained only under the Clean setting, BCP generalizes to the Randomized setting, raising the average success rate by +4.06%. It also transfers to a different base policy $\pi_{0.5}$, achieving a better result on LIBERO (+1.7%) and, notably, on the harder LIBERO-PRO (+6.8%). On a real
robot, BCP lifts success from 74% to 92% and from 44% to 84% on two manipulation tasks. Meanwhile, its negligible overhead, combined with higher success, makes BCP's overall runtime even lower than the fixed-horizon baselines.
Read the full abstract ▾
All authors Weichen Xu, Zhenhua Liu, Lin Luo, Yaobo Liang, Chengtang Yao, Qingyu Mei, Jian Cao, Xixin Cao, Xing Zhang, Jiaolong Yang, Baining Guo
Computer Vision
MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification
Qiming Li, Shujie Hu, Haohan Liu et al. (+3 more)
Recent advances in Large
Vision-
Language Models (LVLMs) have demonstrated impressive capabilities in web UI generation. However, existing
benchmarks predominantly focus on single-turn full-page generation from scratch, overlooking the iterative workflow of real-world frontend engineering, where developers repeatedly reconstruct missing regions and modify localized elements within existing codebases. To bridge this gap, we introduce MT-Web2Code, the first
multimodal coding
benchmark for multi-turn Macro-Level Regional Reconstruction and Micro-Level Localized Modification, which contains 102 tasks spanning 16 vertical domains. To construct deterministic repair trajectories without costly turn-level human annotation, we develop a scalable Reverse-Corruption Trajectory Engine that iteratively injects structural and stylistic defects into golden pages. We further propose a dual-axis evaluation protocol that measures target-region fidelity and the preservation of unaffected content, where regional reconstruction is assessed by a 5-dimensional VLM-based rubric and localized modification by deterministic pixel-grounded
alignment. Experiments on 13 frontier coding
agents reveal that current
agents struggle to faithfully reconstruct target regions while preserving unaffected content, lack fine-grained visual-code
alignment for localized edits, and suffer from error snowballing over multiple turns. Beyond benchmarking, our deterministic evaluation metrics provide fine-grained feedback signals that may facilitate future research on training iterative UI coding
agents. Our evaluation code and data will soon be released.
Read the full abstract ▾
All authors Qiming Li, Shujie Hu, Haohan Liu, Xiaocheng Feng, Songxiang Liu, Guanglu Wan
Computer Vision
Hi-Token: Hierarchical Coordinate Tokenization for Generative Visual Grounding
Xiuyuan Zhu, Ke Lu, Kun Dong et al. (+6 more)
Generative
Vision-
Language Models (VLMs) express bounding boxes as coordinate sequences, whose
token identities often leave axis and place value implicit. We study how explicitly encoding these roles improves visual grounding. Hi-
Token assigns a distinct
token to each combination of axis, decimal place, and digit value, representing a box with 12
tokens drawn from 60 types. This design exposes coarse-to-fine coordinate structure and increases
token reuse relative to atomic location
tokens while retaining the existing VLM architecture. Hi-GAR complements this representation with a geometry-based reward for Group Relative Policy Optimization (GRPO), combining box overlap, coordinate accuracy at multiple scales, strict-IoU bonuses, and validity gating. Matched training comparisons show that Hi-
Token improves localization throughout the evaluated IoU range, with gains across three VLM backbones. Hi-
Token also outperforms an additionally tuned atomic-
token baseline on the RefCOCO family before reinforcement learning. Hi-GAR further improves localization and is used only during training. The resulting Hi-R1 model achieves higher values than strong specialist baselines on most reported metrics.
Token-frequency and
geometric diagnostics characterize the representation and reward stages. These results establish explicit axis and place-value encoding as an effective design for generative visual grounding. Project page: https://xyzzzh.github.io/Hi-
Token/
Read the full abstract ▾
All authors Xiuyuan Zhu, Ke Lu, Kun Dong, Siwen Jiao, Hao Wu, Zijin Du, Shun Mao, Dongming Zhang, Jian Xue
Computer Vision · Cross-listed from cs.MM
Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs
Haoqian Kang, Liupeng Li, Kuofeng Gao et al. (+5 more)
Reasoning in
Multimodal Large
Language Models (MLLMs) requires both fine-grained visual perception and rigorous logical deduction. Explicit text-based Chain-of-Thought (CoT) is computationally expensive and prone to visual
hallucinations, while existing latent
reasoning methods typically require costly training. Furthermore, directly adapting training-free LLM
reasoning mechanisms to the
multimodal setting yields unstable performance. We identify that this failure stems from their reliance on
token-level
entropy, which fundamentally conflates perceptual ambiguity (e.g., unclear visual details) with logical uncertainty (e.g., complex
reasoning steps). To overcome this bottleneck, we present a novel training-free inference strategy for MLLMs that explicitly decouples perception and
reasoning. We propose a novel metric, the
vision-to-text
attention ratio, to dynamically gauge the model's cognitive focus. Guided by this metric, our proposed framework,
Attention-Guided Switching (AGS), adaptively triggers latent
reasoning for perceptual
tokens to preserve high-fidelity visual information in the continuous space, while enforcing explicit text generation for logical
tokens to maintain structural anchoring. Extensive experiments demonstrate that our method achieves state-of-the-art performance, significantly improving both accuracy and inference efficiency by reducing autoregressive steps and latency. Code is released at https://github.com/swordAndSnow/MM26-AGS.
Read the full abstract ▾
All authors Haoqian Kang, Liupeng Li, Kuofeng Gao, Jinpeng Wang, Zhenyu Lu, Bin Chen, Ke Chen, Yaowei Wang
Computer Vision · Cross-listed from cs.RO
A Low-Cost Hybrid Reservoir Computing Model for Isolated Sign Language Video Recognition
Nitin Kumar Singh, Arie Rachmad Syulistyo, Yuichiro Tanaka et al. (+1 more)
Sign
language recognition (SLR) enhances communication between hearing and hearing-impaired individuals. Although deep learning (DL) has achieved promising performance in SLR, its high computational cost limits deployment on edge devices. To address this challenge, we propose a lightweight reservoir computing (RC)-based approach for SLR. In the proposed method, MediaPipe extracts body and hand keypoints to capture the spatial and temporal dynamics of gestures. These keypoints are then processed by a hybrid reservoir computing (HRC) architecture that combines deep reservoir computing (DRC) and bidirectional reservoir computing (BRC), transforming the input into a high-dimensional dynamic representation. A ridge regression model maps the final HRC state to class labels. This HRC-based SLR method achieved Top-1, Top-5, and Top-10 accuracies of 61.12%, 86.05%, and 92.56%, respectively, on the Word-Level American Sign
Language 100 (WLASL100)
video dataset, demonstrating competitive performance compared to deep learning-based approaches. Additionally, due to the lightweight nature of RC, the training time was drastically reduced to only a few seconds compared with DL-based methods such as Bi-GRU.This method offers low computational cost, showing its potential for deployment on edge devices.
Read the full abstract ▾
All authors Nitin Kumar Singh, Arie Rachmad Syulistyo, Yuichiro Tanaka, Hakaru Tamukoh
Computer Vision · Cross-listed from cs.LG
Stop Replacing Noise with Noise: Two-Source Reliability Assessment for Label Correction and Sample Reweighting in Label-Noise Learning
Wenxiao Fan and Kan Li
Refurbishment-based noisy-label learning mixes an observed label with a model-derived pseudo target, typically using one sample-wise cleanliness score to control both branches. This creates a hidden coupling: reducing trust in the observed label automatically increases trust in the pseudo target. We show that this complementarity can replace one unreliable signal with another because a pseudo target learned from corrupted supervision may reproduce the noise it is meant to correct. Our representation diagnostics provide a consistent account of this mismatch: noisy supervision redirects deeper layers more strongly, whereas shallower relations remain comparatively stable and provide information beyond the loss posterior. We therefore propose TRACE, a Two-Source Reliability Assessment framework for Label Correction and Sample Reweighting. TRACE assesses the observed label using loss fit, shallow relation stability, and prediction agreement, while separately assessing the pseudo target using model confidence. Its source-specific scores control target correction and supervision strength without assuming complementary reliability. Across synthetic and real-world noisy
benchmarks, TRACE improves representative refurbishment baselines and yields more reliable pseudo supervision.
Read the full abstract ▾
Computer Vision
Dual-domain U-Nets with embedded back projection operators for motion-resolved 4D CBCT reconstruction
Ivo Herzig, Pascal Paysan, Daniel Barco et al. (+8 more)
Four-dimensional cone beam CT (4D CBCT) is important for image-guided radiation therapy of thoracic cancers, but its use is limited by long scan times, causing high patient dose and motion/sparse-sampling artifacts. We propose a deep learning method for motion-resolved 4D CBCT reconstruction from conventional free-breathing scans, without a respiratory signal or explicit projection binning. Our CNN takes free-breathing 3D CBCT projections as input and predicts a static volume at maximum inhalation plus ten displacement vector fields (DVFs) spanning a breathing cycle. The network extends U-Net: the encoder acts on filtered projection stacks, the decoder acts in the volume domain, and skip connections are replaced with non-trainable back-projection functions at multiple resolutions to transfer features between domains. The model is trained on simulated CBCT scans and evaluated on 11 unseen simulated patients and 13 clinical free-breathing scans. Two additional models (60 s and 6 s scans) were evaluated by clinical experts on three and two scans, comparing single phases of our 4D reconstruction to reference 3D SART-TV images for tumor and esophagus visibility. Experts preferred our method for tumor visibility (59% vs. 36% no preference, 5% reference) and esophagus visibility (47% vs. 42%, 11%). On simulated data, image quality matched SART-TV (mean RMSE: -1.19 HU, PSNR: +0.09 dB, SSIM: -0.009) while enabling 4D reconstruction. On clinical scans, our method showed sharper dynamic structures (e.g., diaphragm) and fewer motion streak artifacts than traditional reconstruction. This non-patient-specific CNN predicts static volumes and full 4D respiratory motion models from a single free-breathing scan, without a respiratory surrogate or projection binning, reducing motion artifacts while adding motion-modeling capability.
Read the full abstract ▾
All authors Ivo Herzig, Pascal Paysan, Daniel Barco, Marc André Stadelmann, Frank-Peter Schilling, Igor Peterlik, Michal Walczak, Lijin Aryananda, Woo Sang Ahn, Rudolf Marcel Füchslin, Lukas Lichtensteiger
Computer Vision
SLAMFormer-$\infty$: Infinite SLAM Transformer for Unbounded Frontend and Backend Processing
Zhijian Fang, Weicheng Zheng, Yijun Yuan et al. (+7 more)
We introduce the Infinite SLAM
Transformer (SLAMFormer-$\infty$), the first
geometric transformer capable of supporting both long-range frontend and backend processing without an explicit distance bound. Instead of relying on a first-frame-anchored formulation, SLAMFormer-$\infty$ employs
memory conditions to define flexible coordinate systems and scales for input frames, enabling more expressive structural conditioning. Built upon this formulation, the frontend preserves efficient local computation, while the backend jointly optimizes long-range trajectories and scene geometry in a globally consistent manner. Experimental results demonstrate that SLAMFormer-$\infty$ achieves superior or highly competitive performance in both trajectory estimation and scene reconstruction across large-scale
datasets. Notably, SLAMFormer-$\infty$ generalizes to extremely long trajectories, successfully operating on sequences exceeding $17\mathrm{km}$.
Read the full abstract ▾
All authors Zhijian Fang, Weicheng Zheng, Yijun Yuan, Weibang Wang, Zhuoguang Chen, Chang Sun, Junhao Huang, Kenan Li, Minghui Qin, Hang Zhao
Computer Vision
OliveGemma: A 3 Billion Visual Language Model for Recognising the Mediterranean & European Diet
Dimitrios I. Zaridis, Traianos Tsiokris, Vasileios C. Pezoulas et al. (+6 more)
Image based dietary assessment offers a scalable alternative to self reported food diaries, yet fine-grained food recognition remains challenging due to high intra-class variability and visually similar dishes. This study presents OliveGemma, a
vision language model for recognising and
reasoning about Mediterranean and European cuisine. Built on the open-weight PaliGemma-2-3B architecture, OliveGemma is fine-tuned with LoRA on a unified corpus of 17,340 images from three European research project
datasets (MedGR, ODIN, and VIPPSTAR), reconciled into a vocabulary of 216 composed dish categories and paired with 102,642 instruction style question-answer items covering dish recognition, likely and visible ingredients, class boundary discrimination, visual evidence and overall visual food understanding. Under a 3-fold cross-validation scheme, OliveGemma achieves a top-1 accuracy of 92.96% +/- 0.91%, exceeding the strongest CNN baseline (DenseNet-121) by 7.31% and outperforming zero-shot frontier models with exact instructions and bounded classes including Gemini Flash 3 and 3.5, GPT-5.4 Mini, and Claude Haiku 4.6 by 8%, 46%, and 64% respectively. Furthermore, OliveGemma demonstrates competitive performance on Top-3 and Top-5 accuracy, being second best across CNNs and frontier models, surpassed only by DenseNet-121. In addition, OliveGemma achieves 90.79% +/- 1.3% Exact-Set on the likely ingredients of the food categories. These results demonstrate that PEFT adaptation of a small VLM can surpass substantially larger proprietary models on specialised food recognition. The model is publicly available at https://huggingface.co/JamesZar/OliveGemma-3B and the experiments and results can be found at https://github.com/tsiokris/OliveGemma.
Read the full abstract ▾
All authors Dimitrios I. Zaridis, Traianos Tsiokris, Vasileios C. Pezoulas, Daphni Plati, Eugenia Mylona, Eleni Georga, Nikos Tsiknakis, Antonis Sakellarios, Dimitrios I. Fotiadis
Computer Vision
SGFormer: Structure-Guided Transformer for Robust Local Feature Matching
Zhihua Xu, Runyu Zhu and Rongjun Qin
Local feature matching is a fundamental component of photogrammetry, enabling accurate image correspondence critical for tasks such as 3D reconstruction, stereo mapping, and visual localization. While recent detector-free matching methods, like LoFTR, have advanced the field, the global features obtained by leveraging the global-range modeling capacity of the unconstrained
attention mechanism compromise the model's
attention to the salient structures in certain scenarios. This limitation leads to a phenomenon we define as
attention divergence, wherein a portion of high-confidence matches are distributed outside the valid matching region (overlapping region), especially in scenes with large viewpoint variations. This occurs because similar features in irrelevant regions may receive equal weighting and consideration within the standard
Transformer, limiting matching reliability in challenging photogrammetric environments. To address this issue in feature matching, we propose SGFormer (Structure-Guided
Transformer), a novel structure-aware matching network that adaptively updates
attention on features near salient structure in overlapping regions. SGFormer employs a semi-dense coarse-to-fine pipeline and incorporates the proposed Triple-Structure-
Attention (TSA) module into the backbone net for extracting distinctive features. The TSA module utilizes shallow local features from early network layers to enhance the representation around salient structure, guiding subsequent
transformer stages to intensify the model's focus on regions with salient structure across the global scope. SGFormer, thereby reinforcing
attention to visually consistent areas while mitigating the influence of non-overlapping regions. Extensive experiments show that SGFormer significantly mitigates
attention divergence and improves matching accuracy.
Read the full abstract ▾
Computer Vision
HyperbolicDiffusion: Sharp & Scalable Tiled Generation on the Hyperbolic Plane
Hugo Caselles-Dupré
Planar tiled
diffusion denoises overlapping windows of one rectangular canvas. The
hyperbolic plane has no such canvas, and its area grows exponentially with radius. We introduce HyperbolicDiffusion, a training-free method for generating finite visual fields directly on the
hyperbolic plane H2. Our
Hyperbolic Blooming Cover reduces window placement to a compact dynamic program that runs in seconds while providing strong theoretical guarantees. Permanent surface IDs form a shared latent canvas: a standard
diffusion model denoises local windows, whose predictions are fused back onto H2. Because
curvature causes residual disagreement and blur at multi-window junctions, a geometry-derived second stage re-noises and repairs precisely those regions. The resulting fields are sharp, reprojectable, and consistent across viewpoints, providing a
prompt-driven generative counterpart to Escher's Circle Limit series.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.SD
Multi-Task Multi-Frame Visual Piano Transcription
Yonghyun Kim, Hoyeol Sohn, Juhan Nam et al. (+1 more)
Audio-based piano transcription performs well on onset, pitch, and velocity, but the sustain pedal lets sound persist long after key release, so audio systems predict pedal-extended offsets rather than physical key release. Yet existing Visual Piano Transcription (VPT) systems focus on onset detection from short
video windows, offset accuracy lags onset by a wide margin, and note-level velocity has not been reported. To address these gaps, we present V2N (
Video to Notes), the first complete VPT system: a shared temporal backbone feeds task-specific heads for onset, offset, key hold, and velocity, jointly trained with per-frame supervision rather than only at the window center. Ablations show that multi-task supervision enables offset and velocity prediction while improving onset accuracy; longer temporal context yields further improvements. V2N sets new state-of-the-art results on PianoVAM and R3.
Read the full abstract ▾
All authors Yonghyun Kim, Hoyeol Sohn, Juhan Nam, Alexander Lerch
Computer Vision
Adam J. Stewart, Heng Fang, Isaac A. Corley et al. (+1 more)
Earth observation is moving from foundation models that users must run themselves toward
embedding products that package model feature outputs as reusable data without needing to download and process the imagery used to generate them. Earth
embeddings are vectors that summarize locations, image patches, or pixels, letting users analyze compact features instead of repeatedly training or running large models on raw satellite imagery. This chapter explains the main types of Earth
embeddings, from implicit location encoders to explicit patch and pixel products, and compares their coverage, resolution, dimensionality, storage cost, licenses, and reproducibility. We review their use in land cover and crop mapping, ecological and hazard modeling, socioeconomic prediction, and semantic search, with evidence on when
embeddings improve on conventional features and when pooling, fusion, or spatial transfer limit performance. Two case studies show practical workflows for similarity search and land cover mapping. We close with guidance for choosing, evaluating, storing, compressing, and publishing
embeddings, and with open problems in oceanic and atmospheric coverage, uncertainty, and benchmarking.
Read the full abstract ▾
All authors Adam J. Stewart, Heng Fang, Isaac A. Corley, Xiao Xiang Zhu
Computer Vision
Distilled Roads: Generalisable Road Network Extraction Across Sensors, Resolutions, and Region
Sanayya, Rakshith Sathish and Ashwathi Nambiar
Road network
segmentation from satellite imagery remains challenging due to large geographic variation in road appearance, occlusions, and domain shifts introduced by differing resolutions and sensors. Existing models, typically trained under narrow resolution–region combinations, generalise poorly to unseen environments such as rural settings, regions with distinct road materials, or imagery from new satellite platforms, often producing broken or disconnected predictions. Adapting these models to new domains usually requires retraining or fine-tuning, which is costly and risks catastrophic forgetting. In this work, we reframe global road extraction as a continual adaptation problem rather than an architectural one. Our framework combines cross-resolution knowledge
distillation across a resolution-decreasing curriculum, multi-sensor training, and
topology-aware supervision, yielding a single model that generalises across $0.3-1.0$ m imagery from multiple satellite platforms across continents. On publicly available
benchmarks, including City-Scale and Global-Scale, our model outperforms state-of-the-art results by up to $22$ F1 points and $15$ APLS points, while remaining the most efficient, with $3\times$ faster inference. Our results suggest that improved robustness across diverse sub-meter satellite imagery can be achieved through targeted training strategies, such as data curricula,
distillation, and
topology-aware losses, rather than increasingly complex architectures.
Read the full abstract ▾
Computer Vision
SRAP: SVD-Refined Adversarial Perturbations for Imperceptible Face-Swap Defense
Sungwon Cho, Kwanghyun Ko and Myungjoo Kang
Deepfake technologies pose increasing threats to facial
privacy and identity security, motivating proactive defenses that protect facial images before misuse. Although
adversarial perturbations generated by projected gradient descent (PGD) can disrupt the identity representations used by face-swapping models, their visual quality is degraded by two characteristics: perturbations are distributed broadly over the image, including identity-insensitive regions, and they contain visually salient high-frequency components. We analyze these spatial and spectral inefficiencies through identity-sensitivity estimation and the singular-value decomposition (SVD) of PGD perturbations. Our analysis shows that later singular components contain a disproportionate amount of high-frequency energy, while the leading components preserve most of the perturbation energy and defense utility. Based on these observations, we propose SRAP, which combines per-channel truncated SVD refinement with an identity-importance mask at every optimization step. The SVD refinement suppresses high-rank, high-frequency residuals, while the mask restricts perturbations to locations that strongly influence identity representations. Experiments on CelebA-HQ and VGGFace2-HQ demonstrate that SRAP substantially improves protected-image fidelity across all reported metrics while maintaining competitive identity-disruption performance, yielding a favorable trade-off between face-swap defense and visual imperceptibility.
Read the full abstract ▾