Computer Vision
VIPA: Visual Informative Part Attention for Referring Image Segmentation
Yubin Cho, Hyunwoo Yu, Kyeongbo Kong et al. (+3 more)
Referring Image
Segmentation (RIS) aims to segment a target object described by a natural
language expression. Existing methods have evolved by leveraging the
vision information into the
language tokens. To more effectively exploit visual contexts for fine-grained
segmentation, we propose a novel Visual Informative Part
Attention (VIPA) framework for referring image
segmentation. VIPA leverages the informative parts of visual contexts, called a visual expression, which can effectively provide the structural and semantic visual target information to the network. This design reduces high-variance cross-modal projection and enhances semantic consistency in an
attention mechanism of the referring image
segmentation. We also design a visual expression generator (VEG) module, which retrieves informative visual
tokens via local-global linguistic context cues and refines the retrieved
tokens for reducing noise information and sharing informative visual attributes. This module allows the visual expression to consider comprehensive contexts and capture semantic visual contexts of informative regions. In this way, our framework enables the network's
attention to robustly align with the fine-grained regions of interest. Extensive experiments and visual analysis demonstrate the effectiveness of our approach. Our VIPA outperforms the existing state-of-the-art methods on four public RIS
benchmarks.
Read the full abstract ▾
All authors Yubin Cho, Hyunwoo Yu, Kyeongbo Kong, Kyomin Sohn, Bongjoon Hyun, Suk-Ju Kang
Computer Vision
GOT-JEPA: Generic Object Tracking with Model Adaptation and Occlusion Handling using Joint-Embedding Predictive Architecture
Shih-Fang Chen, Jun-Cheng Chen, I-Hong Jhuo et al. (+1 more)
The human visual system tracks objects by integrating current observations with previously observed information, adapting to target and scene changes, and
reasoning about occlusion at fine granularity. In contrast, recent generic object trackers are often optimized for training targets, which limits robustness and generalization in unseen scenarios, and their occlusion
reasoning remains coarse, lacking detailed modeling of occlusion patterns. To address these limitations in generalization and occlusion perception, we propose GOT-JEPA, a model-predictive pretraining framework that extends JEPA from predicting image features to predicting tracking models. Given identical historical information, a teacher predictor generates pseudo-tracking models from a clean current frame, and a student predictor learns to predict the same pseudo-tracking models from a corrupted version of the current frame. This design provides stable pseudo supervision and explicitly trains the predictor to produce reliable tracking models under occlusions, distractors, and other adverse observations, improving generalization to dynamic environments. Building on GOT-JEPA, we further propose OccuSolver to enhance occlusion perception for object tracking. OccuSolver adapts a point-centric point tracker for object-aware visibility estimation and detailed occlusion-pattern capture. Conditioned on object priors iteratively generated by the tracker, OccuSolver incrementally refines visibility states, strengthens occlusion handling, and produces higher-quality reference labels that progressively improve subsequent model predictions. Extensive evaluations on seven
benchmarks show that our method effectively enhances tracker generalization and robustness.
Read the full abstract ▾
All authors Shih-Fang Chen, Jun-Cheng Chen, I-Hong Jhuo, Yen-Yu Lin
Computer Vision
SAILS: Segment Anything with Incrementally Learned Semantics for Task-Invariant and Training-Free Continual Learning
Shishir Muralidhara, Didier Stricker and René Schuster
Continual learning remains constrained by the need for repeated retraining, high computational costs, and the persistent challenge of forgetting. These factors significantly limit the applicability of continuous learning in real-world settings, as iterative model updates require significant computational resources and inherently exacerbate forgetting. We present SAILS – Segment Anything with Incrementally Learned Semantics, a training-free framework for Class-Incremental Semantic
Segmentation (CISS) that sidesteps these challenges entirely. SAILS leverages foundational models to decouple CISS into two stages: Zero-shot region extraction using Segment Anything Model (SAM), followed by semantic association through prototypes in a fixed feature space. SAILS incorporates selective intra-class clustering, resulting in multiple prototypes per class to better model intra-class variability. Our results demonstrate that, despite requiring no incremental training, SAILS typically surpasses the performance of existing training-based approaches on standard CISS
datasets, particularly in long and challenging task sequences where forgetting tends to be most severe. By avoiding parameter updates, SAILS completely eliminates forgetting and maintains consistent, task-invariant performance. Furthermore, SAILS exhibits positive backward transfer, where the introduction of new classes can enhance performance on previous classes.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Universal Algorithm-Implicit Learning
Stefano Woerner, Seong Joon Oh and Christian F. Baumgartner
Current meta-learning methods are constrained to narrow task distributions with fixed feature and label spaces, limiting applicability. Moreover, the current meta-learning literature uses key terms like "universal" and "general-purpose" inconsistently and lacks precise definitions, hindering comparability. We introduce a theoretical framework for meta-learning which formally defines practical universality and introduces a distinction between
algorithm-explicit and
algorithm-implicit learning, providing a principled vocabulary for
reasoning about universal meta-learning methods. Guided by this framework, we present TAIL, a
transformer-based
algorithm-implicit meta-learner that functions across tasks with varying domains, modalities, and label configurations. TAIL features three innovations over prior
transformer-based meta-learners: random projections for cross-modal feature encoding, random injection label
embeddings that extrapolate to larger label spaces, and efficient inline query processing. TAIL achieves state-of-the-art performance on standard few-shot
benchmarks while generalizing to unseen domains. Unlike other meta-learning methods, it also generalizes to unseen modalities, solving text classification tasks despite training exclusively on images, handles tasks with up to 20$\times$ more classes than seen during training, and provides orders-of-magnitude computational savings over prior
transformer-based approaches.
Read the full abstract ▾
Computer Vision
Depth Completion as Parameter-Efficient Test-Time Adaptation
Bingxin Ke, Qunjie Zhou, Jiahui Huang et al. (+5 more)
We introduce CAPA, a parameter-efficient test-time optimization framework that adapts pre-trained 3D foundation models (FMs) for depth completion, using sparse
geometric cues. Unlike prior methods that train task-specific encoders for auxiliary inputs, which often overfit and generalize poorly, CAPA freezes the FM backbone. Instead, it updates only a minimal set of parameters using Parameter-Efficient Fine-Tuning (e.g. LoRA or VPT), guided by gradients calculated directly from the sparse observations available at inference time. This approach effectively grounds the foundation model's
geometric prior in the scene-specific measurements, correcting distortions and misplaced structures. For
videos, CAPA introduces sequence-level parameter sharing, jointly adapting all frames to exploit temporal correlations, improve robustness, and enforce multi-frame consistency. CAPA is model-agnostic, compatible with any ViT-based FM, and achieves state-of-the-art results across diverse condition patterns on both indoor and outdoor
datasets. Project page: research.nvidia.com/labs/dvl/projects/capa.
Read the full abstract ▾
All authors Bingxin Ke, Qunjie Zhou, Jiahui Huang, Xuanchi Ren, Tianchang Shen, Konrad Schindler, Laura Leal-Taixé, Shengyu Huang
Computer Vision
It's a Matter of Time: Three Lessons on Long-Term Motion for Perception
Willem Davison, Xinyue Hao and Laura Sevilla-Lara
Temporal information has long been considered to be essential for perception. While there is extensive research on the role of image information for perceptual tasks, the role of the temporal dimension remains less well understood: What can we learn about the world from long-term motion information? What properties does long-term motion information have for visual learning? We leverage recent success in point-track estimation, which offers an excellent opportunity to learn temporal representations and experiment on a variety of perceptual tasks. We draw 3 clear lessons: 1) Long-term motion representations contain information to understand actions, but also objects, materials, and spatial information, often even better than images. 2) Long-term motion representations generalize far better than image representations in low-data settings and in zero-shot tasks. 3) The very low dimensionality of motion information makes motion representations a better trade-off between GFLOPs and accuracy than standard
video representations, and used together they achieve higher performance than
video representations alone. We hope these insights will pave the way for the design of future models that leverage the power of long-term motion information for perception.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error
Farzan Farnia, Mohammad Jalali and Azim Ospanov
Deep generative models have achieved great success in producing high-quality samples, making them a central tool across machine learning applications. Beyond sample quality, an important yet less systematically studied question is whether trained generative models faithfully capture the diversity of the underlying data distribution. In this work, we address this question by directly comparing the diversity of samples generated by state-of-the-art models with that of test samples drawn from the target data distribution, using recently proposed reference-free
entropy-based diversity scores, Vendi and RKE. Across multiple
benchmark datasets, we find that test data consistently attains substantially higher Vendi and RKE diversity scores than the generated samples, suggesting a systematic downward diversity bias in modern generative models. To understand the origin of this bias, we analyze the finite-sample behavior of
entropy-based diversity scores and show that their expected values increase with sample size, implying that diversity estimated from finite training sets could inherently underestimate the diversity of the true distribution. As a result, optimizing the generators to minimize divergence to empirical data distributions would induce a loss of diversity. Finally, we discuss potential diversity-aware regularization and guidance strategies based on Vendi and RKE as principled directions for mitigating this bias, and provide empirical evidence suggesting their potential to improve the results.
Read the full abstract ▾
Computer Vision
Universal Image Immunization against Diffusion-based Image Editing via Semantic Injection
Chanhui Lee, Donggyu Choi, Seunghyun Shin et al. (+2 more)
Diffusion model advances have enabled powerful text-guided image editing, but also raise ethical and legal risks such as deepfakes and unauthorized use. To prevent these risks,
adversarial attack-based image immunization has emerged as a promising defense against AI-driven semantic manipulation. Yet, most existing approaches require image-specific optimization or additional neural networks at inference time, hindering scalability and practicality. In this paper, we propose the first universal
adversarial perturbation-based image immunization framework that generates a single, image-agnostic
adversarial perturbation specifically designed for
diffusion-based editing pipelines. Inspired by UAP used in targeted attacks, our method aims to generate a UAP that induces
diffusion models to misinterpret the input image as a specific semantic target. Simultaneously, it suppresses original content to misdirect the model's
attention during editing, thereby effectively blocking unauthorized edits by overwriting the image's original semantics via the UAP. Extensive experiments show that our method, as the first universal immunization approach, significantly outperforms several baselines in the UAP setting. Notably, despite the inherent difficulty of universal perturbations, our method achieves competitive or superior performance compared to image-specific methods under a more restricted perturbation budget, while also exhibiting strong black-box transferability across diverse
diffusion models.
Read the full abstract ▾
All authors Chanhui Lee, Donggyu Choi, Seunghyun Shin, Hae-Gon Jeon, Jeany Son
Computer Vision
MeFEm: Medical Face Embedding model
Yury Borets and Stepan Botman
We present MeFEm, a
vision model based on a modified Joint
Embedding Predictive Architecture (JEPA) for biometric and medical analysis from facial images. Key modifications include an axial stripe masking strategy to focus learning on semantically relevant regions, a circular loss weighting scheme, and the probabilistic reassignment of the CLS
token for high quality linear probing. Trained on a consolidated
dataset of curated images, MeFEm outperforms strong baselines like FaRL and Franca on core anthropometric tasks despite using significantly less data. It also shows promising results on Body Mass Index (BMI) estimation, evaluated on a novel, consolidated closed-source
dataset that addresses the domain bias prevalent in existing data. Model weights are available at https://huggingface.co/boretsyury/MeFEm , offering a strong baseline for future work in this domain.
Read the full abstract ▾
Computer Vision
Advances in Global Solvers for 3D Vision
Zhenjun Zhao, Heng Yang, Bangyan Liao et al. (+7 more)
Global solvers have emerged as a powerful paradigm for 3D
vision, offering certifiable solutions to nonconvex
geometric optimization problems traditionally addressed by local or heuristic methods. This survey presents the first systematic review of global solvers in
geometric vision, unifying the field through a comprehensive taxonomy of three core paradigms: Branch-and-Bound (BnB), Convex Relaxation (CR), and Graduated Non-Convexity (GNC). We present their theoretical foundations, algorithmic designs, and practical enhancements for robustness and scalability, examining how each addresses the fundamental nonconvexity of
geometric estimation problems. Our analysis spans ten core
vision tasks, from Wahba problem to bundle adjustment, revealing the optimality-robustness-scalability trade-offs that govern solver selection. We identify critical future directions: scaling
algorithms while maintaining guarantees, integrating data-driven priors with certifiable optimization, establishing standardized
benchmarks, and addressing societal implications for safety-critical deployment. By consolidating theoretical foundations, practical advances, and broader impacts, this survey provides a unified perspective and roadmap toward certifiable, trustworthy perception for real-world applications. A continuously-updated literature summary and companion code tutorials are available at https://github.com/ericzzj1989/Awesome-Global-Solvers-for-3D-
Vision.
Read the full abstract ▾
All authors Zhenjun Zhao, Heng Yang, Bangyan Liao, Yingping Zeng, Shaocheng Yan, Yingdong Gu, Peidong Liu, Yi Zhou, Haoang Li, Javier Civera
Computer Vision
SketchingReality: From Freehand Scene Sketches To Photorealistic Images
Ahmed Bourouis, Mikhail Bessmeltsev and Yulia Gryaditskaya
Recent years have witnessed remarkable progress in generative AI, with natural
language emerging as the most common conditioning input. As underlying models grow more powerful, researchers are exploring increasingly diverse conditioning signals, such as depth maps, edge maps, camera parameters, and reference images, to give users finer control over generation. Among different modalities, sketches are a natural and long-standing form of human communication, enabling rapid expression of visual concepts. Previous literature has largely focused on edge maps, often misnamed 'sketches', yet
algorithms that effectively handle true freehand sketches, with their inherent abstraction and distortions, remain underexplored. We pursue the challenging goal of balancing photorealism with sketch adherence when generating images from freehand input. A key obstacle is the absence of ground-truth, pixel-aligned images: by their nature, freehand sketches do not have a single correct
alignment. To address this, we propose a modulation-based approach that prioritizes semantic interpretation of the sketch over strict adherence to individual edge positions. We further introduce a novel loss that enables training on freehand sketches without requiring ground-truth pixel-aligned images. We show that our method outperforms existing approaches in both semantic
alignment with freehand sketch inputs and in the realism and overall quality of the generated images.
Read the full abstract ▾
Computer Vision
VIGIL: Tackling Hallucination Detection in Image Recontextualization
Joanna Wojciechowicz, Maria Łubniewska, Jakub Antczak et al. (+4 more)
We introduce VIGIL (Visual Inconsistency & Generative In-context Lucidity), a
benchmark dataset and framework that provides a fine-grained categorization of
hallucinations in the
multimodal image recontextualization task for large
multimodal models. Most existing methods treat
hallucinations as a single undifferentiated error. We instead decompose them into five categories, namely Object Visual Fidelity, Background Fidelity, Spatial and Instructional Fidelity, Physical and Integration Fidelity, and Object Omission. We propose a multi-stage detection pipeline that processes recontextualized images through specialized steps targeting all five categories with a coordinated set of open-source models. We evaluate the pipeline on the VIGIL
dataset of 1,269 manually annotated samples across five product domains and report macro-F1 per category. The decomposed pipeline reaches the best macro-F1 among open-source detectors. It returns a textual explanation for each detected error, which prior methods for this task do not provide. We openly release VIGIL, along with the detection pipeline and
benchmark code, through our GitHub repository: https://github.com/mlubneuskaya/vigil and Data repository: https://huggingface.co/
datasets/joannaww/VIGIL.
Read the full abstract ▾
All authors Joanna Wojciechowicz, Maria Łubniewska, Jakub Antczak, Justyna Baczyńska, Wojciech Gromski, Wojciech Kozłowski, Maciej Zieba
Computer Vision
VariViT: A Vision Transformer for Variable Image Sizes
Aswathi Varma, Suprosanna Shit, Chinmay Prabhakar et al. (+5 more)
Vision Transformers (ViTs) have emerged as the state-of-the-art architecture in representation learning, leveraging self-
attention mechanisms to excel in various tasks. ViTs split images into fixed-size patches, constraining them to a predefined size and necessitating pre-processing steps like resizing, padding, or cropping. This poses challenges in medical imaging, particularly with irregularly shaped structures like tumors. A fixed bounding box crop size produces input images with highly variable foreground-to-background ratios. Resizing medical images can degrade information and introduce artefacts, impacting diagnosis. Hence, tailoring variable-sized crops to regions of interest can enhance feature representation capabilities. Moreover, large images are computationally expensive, and smaller sizes risk information loss, presenting a computation-accuracy tradeoff. We propose VariViT, an improved ViT model crafted to handle variable image sizes while maintaining a consistent patch size. VariViT employs a novel positional
embedding resizing scheme for a variable number of patches. We also implement a new batching strategy within VariViT to reduce computational complexity, resulting in faster training and inference times. In our evaluations on two 3D
brain MRI
datasets, VariViT surpasses vanilla ViTs and ResNet in glioma genotype prediction and
brain tumor classification. It achieves F1-scores of 75.5% and 76.3%, respectively, learning more discriminative features. Our proposed batching strategy reduces computation time by up to 30% compared to conventional architectures. These findings underscore the efficacy of VariViT in image representation learning. Our code can be found here: https://github.com/Aswathi-Varma/varivit
Read the full abstract ▾
All authors Aswathi Varma, Suprosanna Shit, Chinmay Prabhakar, Daniel Scholz, Hongwei Bran Li, Bjoern Menze, Daniel Rueckert, Benedikt Wiestler
Computer Vision
YOLO26: A Comprehensive Architecture Overview and Key Improvements
Priyanto Hidayatullah and Refdinal Tubagus
You Only Look Once (YOLO) has been the prominent model for computer
vision in deep learning for a decade. This study explores the novel aspects of YOLO26, the most recent version in the YOLO series. The elimination of Distribution Focal Loss (DFL), implementation of End-to-End NMS-Free Inference, introduction of ProgLoss + Small-Target-Aware Label Assignment (STAL), and use of the MuSGD optimizer are the primary enhancements designed to improve inference speed, which is claimed to achieve a 43% boost in CPU mode. This is designed to allow YOLO26 to attain real-time performance on edge devices or those without GPUs. Additionally, YOLO26 offers improvements in many computer
vision tasks, including instance
segmentation, pose estimation, and oriented bounding box (OBB) decoding. We aim for this effort to provide more value than just consolidating information already included in the existing technical documentation. Therefore, we performed a rigorous architectural investigation into YOLO26, mostly using the source code available in its GitHub repository and its official documentation. The authentic and detailed operational mechanisms of YOLO26 are inside the source code, which is seldom extracted by others. The YOLO26 architectural diagram is shown as the outcome of the investigation. This study is, to our knowledge, the first one presenting the CNN-based YOLO26 architecture, which is the core of YOLO26. Our objective is to provide a precise architectural comprehension of YOLO26 for researchers and developers aspiring to enhance the YOLO model, ensuring it remains the leading deep learning model in computer
vision.
Read the full abstract ▾
Computer Vision
DriveFine: Refining-Augmented Masked Diffusion VLA for Precise and Robust Driving
Chenxu Dang, Sining Ang, Yongkang Li et al. (+7 more)
Vision-
Language-Action (VLA) models for autonomous driving increasingly adopt generative planners trained with imitation learning followed by reinforcement learning.
Diffusion-based planners suffer from modality
alignment difficulties, low training efficiency, and limited generalization.
Token-based planners are plagued by cumulative causal errors and irreversible decoding. In summary, the two dominant paradigms exhibit complementary strengths and weaknesses. In this paper, we propose DriveFine, a masked
diffusion VLA model that combines flexible decoding with self-correction capabilities. In particular, we design a novel plug-and-play block-MoE, which seamlessly injects a refinement expert on top of the generation expert. By enabling explicit expert selection during inference and gradient blocking during training, the two experts are fully decoupled, preserving the foundational capabilities and generic patterns of the pretrained weights, which highlights the flexibility and extensibility of the block-MoE design. Furthermore, we design a hybrid reinforcement learning strategy that encourages effective exploration of refinement expert while maintaining training stability. Extensive experiments on NAVSIM v1, v2, and Navhard
benchmarks demonstrate that DriveFine exhibits strong efficacy and robustness. The code will be released at https://github.com/MSunDYY/DriveFine.
Read the full abstract ▾
All authors Chenxu Dang, Sining Ang, Yongkang Li, Haochen Tian, Jie Wang, Guang Li, Hangjun Ye, Jie Ma, Long Chen, Yan Wang
Computer Vision
OmniVTON++: Training-Free Universal Virtual Try-On with Principal Pose Guidance
Zhaotong Yang, Yong Du, Shengfeng He et al. (+5 more)
Image-based Virtual Try-On (VTON) concerns the synthesis of realistic person imagery through garment re-rendering under human pose and body constraints. In practice, however, existing approaches are typically optimized for specific data conditions, making their deployment reliant on retraining and limiting their generalization as a unified solution. We present OmniVTON++, a training-free VTON framework designed for universal applicability. It addresses the intertwined challenges of garment
alignment, human structural coherence, and boundary continuity by coordinating Structured Garment Morphing for correspondence-driven garment adaptation, Principal Pose Guidance for step-wise structural regulation during
diffusion sampling, and Continuous Boundary Stitching for boundary-aware refinement, forming a cohesive pipeline without task-specific retraining. Experimental results demonstrate that OmniVTON++ achieves state-of-the-art performance across diverse generalization settings, including cross-
dataset and cross-garment-type evaluations, while reliably operating across scenarios and
diffusion backbones within a single formulation. In addition to single-garment, single-human cases, the framework supports multi-garment, multi-human, and anime character virtual try-on, expanding the scope of virtual try-on applications. The code is available at https://github.com/Jerome-Young/OmniVTON-PlusPlus.
Read the full abstract ▾
All authors Zhaotong Yang, Yong Du, Shengfeng He, Yuhui Li, Xinzhe Li, Yangyang Xu, Junyu Dong, Jian Yang
Computer Vision
MoRL: Reinforced Reasoning for Unified Motion Understanding and Generation
Hongpeng Wang, Zeyu Zhang, Wenhao Li et al. (+1 more)
Human motion understanding and generation are crucial for
vision and robotics but remain limited in
reasoning capability and test-time planning. We propose MoRL, a unified
multimodal motion model trained with supervised fine-tuning and reinforcement learning with verifiable rewards. Our task-specific reward design combines semantic
alignment and
reasoning coherence for understanding with physical plausibility and text-motion consistency for generation, improving both logical
reasoning and perceptual realism. To further enhance inference, we introduce Chain-of-Motion (CoM), a test-time
reasoning method that enables step-by-step planning and reflection. We also construct two large-scale CoT
datasets, MoUnd-CoT-140K and MoGen-CoT-140K, to align motion sequences with
reasoning traces and action descriptions. Experiments on HumanML3D and KIT-ML show that MoRL achieves significant gains over state-of-the-art baselines. Code: https://github.com/AIGeeksGroup/MoRL. Website: https://aigeeksgroup.github.io/MoRL.
Read the full abstract ▾
All authors Hongpeng Wang, Zeyu Zhang, Wenhao Li, Hao Tang
Computer Vision
Cross-view Domain Generalization via Geometric Consistency for LiDAR Semantic Segmentation
Jindong Zhao, Yuan Gao, Yang Xia et al. (+4 more)
Domain-generalized
LiDAR semantic
segmentation (LSS) seeks to train models on source-domain point clouds that generalize reliably to multiple unseen target domains, which is essential for real-world
LiDAR applications. However, existing approaches assume similar acquisition views (e.g., vehicle-mounted) and struggle in cross-view scenarios, where observations differ substantially due to viewpoint-dependent structural incompleteness and non-uniform point density. Accordingly, we formulate cross-view domain generalization for
LiDAR semantic
segmentation and propose a novel framework, termed CVGC (Cross-View
Geometric Consistency). Specifically, we introduce a cross-view
geometric augmentation module that models viewpoint-induced variations in visibility and sampling density, generating multiple cross-view observations of the same scene. Subsequently, a
geometric consistency module enforces consistent semantic and occupancy predictions across geometrically augmented point clouds of the same scene. Extensive experiments on six public
LiDAR datasets establish the first systematic evaluation of cross-view domain generalization for
LiDAR semantic
segmentation, demonstrating that CVGC consistently outperforms state-of-the-art methods when generalizing from a single source domain to multiple target domains with heterogeneous acquisition viewpoints. The source code will be publicly available at https://github.com/KintomZi/CVGC-DG
Read the full abstract ▾
All authors Jindong Zhao, Yuan Gao, Yang Xia, Sheng Nie, Jun Yue, Weiwei Sun, Shaobo Xia
Computer Vision
Error Patterns in Historical OCR: A Comparative Analysis of TrOCR and a Vision-Language Model
Ari Vesalainen, Eetu Mäkelä, Laura Ruotsalainen et al. (+1 more)
Optical Character Recognition (OCR) of eighteenth-century printed texts remains challenging due to degraded print quality, archaic glyphs, and non-standardized orthography. Although
transformer-based OCR systems and
Vision-
Language Models (VLMs) achieve strong aggregate accuracy, metrics such as Character Error Rate (CER) and Word Error Rate (WER) provide limited insight into their reliability for scholarly use. We compare a dedicated OCR
transformer (TrOCR) and a general-purpose
Vision-
Language Model (Qwen) on line-level historical English texts using length-weighted accuracy metrics and hypothesis driven error analysis. While Qwen achieves lower CER/WER and greater robustness to degraded input, it exhibits selective linguistic regularization and orthographic normalization that may silently alter historically meaningful forms. TrOCR preserves orthographic fidelity more consistently but is more prone to cascading error propagation. Our findings show that architectural inductive biases shape OCR error structure in systematic ways. Models with similar aggregate accuracy can differ substantially in error locality, detectability, and downstream scholarly risk, underscoring the need for architecture-aware evaluation in historical digitization workflows.
Read the full abstract ▾
All authors Ari Vesalainen, Eetu Mäkelä, Laura Ruotsalainen, Mikko Tolonen
Computer Vision
Architectural Insights for Post-Tornado Damage Recognition
Robinson Umeike, Thang Dao, Shane Crawford et al. (+7 more)
Rapid and accurate building damage assessment in the immediate aftermath of tornadoes is critical for coordinating life-saving search and rescue operations, optimizing emergency resource allocation, and accelerating community recovery. However, current automated methods struggle with the unique visual complexity of tornado-induced wreckage, primarily due to severe domain shift from standard pre-training
datasets and extreme class imbalance in real-world disaster data. To address these challenges, we introduce a systematic experimental framework evaluating 79 open-source deep learning models, encompassing both Convolutional Neural Networks (CNNs) and
Vision Transformers, across over 2,300 controlled experiments on our newly curated Quad-State Tornado Damage (QSTD)
benchmark dataset. Our findings reveal that achieving operational-grade performance hinges on a complex interaction between architecture and optimization, rather than architectural selection alone. Most strikingly, we demonstrate that optimizer choice can be more consequential than architecture: switching from Adam to SGD provided dramatic F1 gains of +25 to +38 points for
Vision Transformer and Swin
Transformer families, fundamentally reversing their ranking from bottom-tier to competitive with top-performing CNNs. Furthermore, a low learning rate of 1x10^(-4) proved universally critical, boosting average F1 performance by +10.2 points across all architectures. Our champion model, ConvNeXt-Base trained with these optimized settings, demonstrated strong cross-event generalization on the held-out Tuscaloosa-Moore Tornado Damage (TMTD)
dataset, achieving 46.4% Macro F1 (+34.6 points over baseline) and retaining 85.5% Ordinal Top-1 Accuracy despite temporal and sensor domain shifts.
Read the full abstract ▾
All authors Robinson Umeike, Thang Dao, Shane Crawford, John van de Lindt, Blythe Johnston, Wanting Wang, Trung Do, Ajibola Mofikoya, Sarbesh Banjara, Cuong Pham
Computer Vision
Efficient Text-Guided Convolutional Adapter for the Diffusion Model
Aryan Das, Koushik Biswas, Swalpa Kumar Roy et al. (+2 more)
We introduce the Nexus Adapters, novel text-guided efficient adapters to the
diffusion-based framework for the Structure Preserving Conditional Generation (SPCG). Recently, structure-preserving methods have achieved promising results in conditional image generation by using a base model for
prompt conditioning and an adapter for structure input, such as sketches or depth maps. These approaches are highly inefficient and sometimes require equal parameters in the adapter compared to the base architecture. It is not always possible to train the model since the
diffusion model is itself costly, and doubling the parameter is highly inefficient. In these approaches, the adapter is not aware of the input
prompt; therefore, it is optimal only for the structural input but not for the input
prompt. To overcome the above challenges, we proposed two efficient adapters, Nexus
Prime and Slim, which are guided by
prompts and structural inputs. Each Nexus Block incorporates cross-
attention mechanisms to enable rich
multimodal conditioning. Therefore, the proposed adapter has a better understanding of the input
prompt while preserving the structure. We conducted extensive experiments on the proposed models and demonstrated that the Nexus
Prime adapter significantly enhances performance, requiring only 8M additional parameters compared to the baseline, T2I-Adapter. Furthermore, we also introduced a lightweight Nexus Slim adapter with 18M fewer parameters than the T2I-Adapter, which still achieved state-of-the-art results. Code: https://github.com/arya-domain/Nexus-Adapters
Read the full abstract ▾
All authors Aryan Das, Koushik Biswas, Swalpa Kumar Roy, Badri Narayana Patro, Vinay Kumar Verma
Computer Vision
Scalable next-scale autoregression for medical image generation across anatomical regions
Zhicheng He, Yunpeng Zhao, Junde Wu et al. (+7 more)
Autoregressive pretraining has been key to the scalability of large
language models, yet medical generative foundation models remain predominantly based on
diffusion. Here we introduce MedVAR, the first foundation model for all-round medical image generation through autoregressive training, and find it offers improved generation quality and efficiency, stronger scalability, and broader adaptability to downstream clinical tasks than
diffusion-based models. A tokenizer trained on medical images and separate semantic and structural controls enable generation across six anatomical regions in computed
tomography and
magnetic resonance imaging. Trained on 438,905 slices from 40
datasets, including seven internal clinical centres, MedVAR generates images 12-19 times faster than 100-step
diffusion baselines. Generation quality improves with model size. Pretraining on generated images improves seven-centre hepatocellular carcinoma
segmentation Dice from 0.615 to 0.632, while reconstruction using MedVAR images approaches the volumetric
segmentation performance of fully sampled volumes. Membership inference reaches 3.54% sensitivity at a 1% false-positive rate, while copy detection performs near chance. These findings establish next-scale autoregression as a scalable and versatile approach to medical image generation and downstream analysis.
Read the full abstract ▾
All authors Zhicheng He, Yunpeng Zhao, Junde Wu, Ziwei Niu, Ziyue Wang, Bohan Li, Zijun Li, Lanfen Lin, Nan Liu, Yueming Jin
Computer Vision
MacNet: An End-to-End Manifold-Constrained Adaptive Clustering Network for Interpretable Whole Slide Image Classification
Mingrui Ma, Chentao Li, Pan Huang et al. (+1 more)
Whole slide images (WSIs) are the gold standard for pathological diagnosis and sub-typing. Current main-stream two-step frameworks employ offline feature encoders trained without domain-specific knowledge. Among them,
attention-based multiple instance learning (MIL) methods are outcome-oriented and offer limited interpretability. Clustering-based approaches can provide explainable decision-making process but suffer from high dimension features and semantically ambiguous centroids. To this end, we propose an end-to-end MIL framework that integrates Grassmann re-
embedding and
manifold adaptive clustering, where the
manifold geometric structure facilitates robust clustering results. Furthermore, we design a prior knowledge guiding proxy instance labeling and aggregation strategy to approximate patch labels and focus on pathologically relevant tumor regions. Experiments on multicentre WSI
datasets demonstrate that: 1) our cluster-incorporated model achieves superior performance in both grading accuracy and interpretability; 2) end-to-end learning refines better feature representations and it requires acceptable computation resources.
Read the full abstract ▾
All authors Mingrui Ma, Chentao Li, Pan Huang, Jing Qin
Computer Vision
Prototype Instance-semantic Disentanglement with Low-rank Regularized Subspace Clustering for WSIs Explainable Recognition
Chentao Li and Pan Huang
The tumor region plays a key role in pathological diagnosis. Tumor tissues are highly similar to precancerous lesions and non tumor instances often greatly exceed tumor instances in whole slide images (WSIs). These issues cause instance-semantic
entanglement in multi-instance learning frameworks, degrading both model representation capability and interpretability. To address this, we propose an end-to-end prototype instance semantic disentanglement framework with low-rank regularized subspace clustering, PID-LRSC, in two aspects. First, we use secondary instance subspace learning to construct low-rank regularized subspace clustering (LRSC), addressing instance
entanglement caused by an excessive proportion of non tumor instances. Second, we employ enhanced contrastive learning to design prototype instance semantic disentanglement (PID), resolving semantic
entanglement caused by the high similarity between tumor and precancerous tissues. We conduct extensive experiments on multicentre pathology
datasets, implying that PID-LRSC outperforms other SOTA methods. Overall, PID-LRSC provides clearer instance semantics during decision-making and significantly enhances the reliability of auxiliary diagnostic outcomes.
Read the full abstract ▾
Computer Vision
Aryan Das, Tanishq Rachamalla, Koushik Biswas et al. (+2 more)
We introduce a novel uncertainty-aware
multimodal segmentation framework that leverages both radiological images and associated clinical text for precise medical diagnosis. We propose a Modality Decoding
Attention Block (MoDAB) with a lightweight State Space Mixer (SSMix) to enable efficient cross-modal fusion and long-range dependency modelling. To guide learning under ambiguity, we propose the Spectral-Entropic Uncertainty (SEU) Loss, which jointly captures spatial overlap, spectral consistency, and predictive uncertainty in a unified objective. In complex clinical circumstances with poor image quality, this formulation improves model reliability. Extensive experiments on various publicly available medical
datasets, QATA-COVID19, MosMed++, and Kvasir-SEG, demonstrate that our method achieves superior
segmentation performance while being significantly more computationally efficient than existing State-of-the-Art (SoTA) approaches. Our results highlight the importance of incorporating uncertainty modelling and structured modality
alignment in
vision-
language medical
segmentation tasks. Code: https://github.com/arya-domain/UA-VLS
Read the full abstract ▾
All authors Aryan Das, Tanishq Rachamalla, Koushik Biswas, Swalpa Kumar Roy, Vinay Kumar Verma
Computer Vision
Gaussian Mesh Renderer for Lightweight Differentiable Rendering
Xinpeng Liu and Fumio Okura
3D Gaussian Splatting (3DGS) has enabled high-fidelity virtualization with fast rendering and optimization for novel view synthesis. On the other hand, triangle mesh models still remain a popular choice for surface reconstruction but suffer from slow or heavy optimization in traditional mesh-based differentiable renderers. To address this problem, we propose a new lightweight differentiable mesh renderer leveraging the efficient rasterization process of 3DGS, named Gaussian Mesh Renderer (GMR), which tightly integrates the Gaussian and mesh representations. Each Gaussian primitive is analytically derived from the corresponding mesh triangle, preserving structural fidelity and enabling the gradient flow. Compared to the traditional mesh renderers, our method achieves smoother gradients, which especially contributes to better optimization using smaller batch sizes with limited
memory. Our implementation is available in the public GitHub repository at https://github.com/huntorochi/Gaussian-Mesh-Renderer.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Revisiting the Platonic Representation Hypothesis: An Aristotelian View
Fabian Gröger, Shuo Wen and Maria Brbić
The Platonic Representation Hypothesis suggests that representations from neural networks are converging to a common statistical model of reality. We show that the existing metrics used to measure representational similarity are confounded by network scale: increasing model depth or width can systematically inflate representational similarity scores. To correct these effects, we introduce a permutation-based null-calibration framework that transforms any representational similarity metric into a calibrated score with statistical guarantees. We revisit the Platonic Representation Hypothesis with our calibration framework, which reveals a nuanced picture: the apparent convergence reported by global spectral measures largely disappears after calibration, while local neighborhood similarity, but not local distances, retains significant agreement across different modalities. Based on these findings, we propose the Aristotelian Representation Hypothesis: representations in neural networks are converging to shared local neighborhood relationships.
Read the full abstract ▾
Computer Vision
TikArt: Stabilizing Aperture-Guided Fine-Grained Visual Reasoning with Reinforcement Learning
Hao Ding, Zhichuan Yang, Weijie Ge et al. (+3 more)
Fine-grained visual
reasoning in
multimodal large
language models (MLLMs) is bottlenecked by single-pass global image encoding: key evidence often lies in tiny objects, cluttered regions, subtle markings, or dense charts. We present TikArt (Thinking Aperture), an aperture-guided
agent that formulates
multimodal reasoning as sequential evidence acquisition over regions of interest. TikArt follows a Think–Aperture–Observe (TAO) loop that interleaves
language reasoning with two aperture actions: Zoom, which extracts rectangular crops, and Segment, which invokes an off-the-shelf segmenter to produce object-centric mask-based views for irregular targets. A mandatory Observation step after every aperture action writes local evidence back into text, yielding interpretable aperture trajectories and persistent linguistic
memory. Built on Qwen3-VL-8B, TikArt is trained with GRPO-style reinforcement learning under a two-stage curriculum. To stabilize long-horizon tool-integrated learning, we introduce Relative Uncertainty Reduction (RUR), a dense reward computed by a frozen evaluator that favors evidence-building trajectories and mitigates degenerate tool use. Experiments on high-resolution
reasoning, general
multimodal understanding, and both referring and
reasoning-oriented
segmentation show consistent gains over the backbone, demonstrating that aperture-guided observation improves fine-grained visual
reasoning and transfers naturally to pixel-level grounding.
Read the full abstract ▾
All authors Hao Ding, Zhichuan Yang, Weijie Ge, Ziqin Gao, Chaoyi Lu, Lei Zhao
Computer Vision
CoCoDiff: Correspondence-Consistent Diffusion Model for Fine-grained Style Transfer
Wenbo Nie, Zixiang Li, Renshuai Tao et al. (+3 more)
Transferring visual style between images while preserving semantic correspondence between similar objects remains a central challenge in computer
vision. While existing methods have made great strides, most of them operate at global level but overlook region-wise and even pixel-wise semantic correspondence. To address this, we propose CoCoDiff, a novel training-free and low-cost style transfer framework that leverages pretrained latent
diffusion models to achieve fine-grained, semantically consistent stylization. We identify that correspondence cues within generative
diffusion models are under-explored and that content consistency across semantically matched regions is often neglected. CoCoDiff introduces a pixel-wise semantic correspondence module that mines intermediate
diffusion features to construct a dense
alignment map between content and style images. Furthermore, a cycle-consistency module then enforces structural and perceptual
alignment across iterations, yielding object and region level stylization that preserves geometry and detail. Despite requiring no additional training or supervision, CoCoDiff delivers state-of-the-art visual quality and strong quantitative results, outperforming methods that rely on extra training or annotations.
Read the full abstract ▾
All authors Wenbo Nie, Zixiang Li, Renshuai Tao, Bin Wu, Yunchao Wei, Yao Zhao
Computer Vision · Cross-listed from cs.AI
Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report v1.5
Dongrui Liu, Yi Yu, Jie Zhang et al. (+18 more)
To understand and identify the unprecedented risks posed by rapidly advancing artificial intelligence (AI) models, Frontier AI Risk Management Framework in Practice presents a comprehensive assessment of their frontier risks. As Large
Language Models (LLMs) general capabilities rapidly evolve and the proliferation of agentic AI, this version of the risk analysis technical report presents an updated and granular assessment of five critical dimensions: cyber offense, persuasion and manipulation, strategic deception, uncontrolled AI R&D, and self-replication. Specifically, we introduce more complex scenarios for cyber offense. For persuasion and manipulation, we evaluate the risk of LLM-to-LLM persuasion on newly released LLMs. For strategic deception and scheming, we add the new experiment with respect to emergent misalignment. For uncontrolled AI R&D, we focus on the “mis-
evolution” of
agents as they autonomously expand their
memory substrates and toolsets. Besides, we also monitor and evaluate the safety performance of OpenClaw during the interaction on the Moltbook. For self-replication, we introduce a new resource-constrained scenario. More importantly, we propose and validate a series of robust mitigation strategies to address these emerging threats, providing a preliminary technical and actionable pathway for the secure deployment of frontier AI. This work reflects our current understanding of AI frontier risks and urges collective action to mitigate these challenges.
Read the full abstract ▾
All authors Dongrui Liu, Yi Yu, Jie Zhang, Guanxu Chen, Qihao Lin, Hanxi Zhu, Lige Huang, Yijin Zhou, Peng Wang, Shuai Shao, Boxuan Zhang, Zicheng Liu, Jingwei Sun, Yu Li, Yuejin Xie, Jiaxuan Guo, Jia Xu, Chaochao Lu, Bowen Zhou, Xia Hu, Jing Shao