Computer Vision
ImLoc: Revisiting Visual Localization with Image-based Representation
Xudong Jiang, Fangjinhua Wang, Silvano Galliani et al. (+2 more)
Existing visual localization methods are typically either 2D image-based, which are easy to build and maintain but limited in effective
geometric reasoning, or 3D structure-based, which achieve high accuracy but require a centralized reconstruction and are difficult to update. In this work, we revisit visual localization with a 2D image-based representation and propose to augment each image with estimated depth maps to capture the
geometric structure. Supported by the effective use of dense matchers, this representation is not only easy to build and maintain, but achieves highest accuracy in challenging conditions. With compact compression and a GPU-accelerated LO-RANSAC implementation, the whole pipeline is efficient in both storage and computation and allows for a flexible trade-off between accuracy and highest
memory efficiency. Our method achieves a new state-of-the-art accuracy on various standard
benchmarks and outperforms existing
memory-efficient methods at comparable map sizes. Code will be available at https://github.com/cvg/Hierarchical-Localization.
Read the full abstract ▾
All authors Xudong Jiang, Fangjinhua Wang, Silvano Galliani, Christoph Vogel, Marc Pollefeys
Computer Vision · Cross-listed from eess.IV
Scanner-Induced Domain Shifts Undermine the Robustness of Pathology Foundation Models
Erik Thiringer, Fredrik K. Gustafsson, Kajsa Ledesma Eriksson et al. (+1 more)
Pathology foundation models (PFMs) have become central to computational pathology, aiming to offer general encoders for feature extraction from whole-slide images (WSIs). Despite strong
benchmark performance, PFM robustness to real-world technical domain shifts, such as variability from whole-slide scanner devices, remains poorly understood. We systematically evaluated the robustness of 14 PFMs to scanner-induced variability, including state-of-the-art models, earlier self-supervised models, and a baseline trained on natural images. Using a multiscanner
dataset of 384 breast cancer WSIs scanned on five devices, we isolated scanner effects independently from biological and laboratory confounders. Robustness is assessed via complementary unsupervised
embedding analyses and a set of clinicopathological supervised prediction tasks. Our results demonstrate that current PFMs are not invariant to scanner-induced domain shifts. Most models encode pronounced scanner-specific variability in their
embedding spaces. While AUC often remains stable, this masks a critical failure mode: scanner variability systematically alters the
embedding space and impacts calibration of downstream model predictions, resulting in scanner-dependent bias that can impact reliability in clinical use cases. We further show that robustness is not a simple function of training data scale, model size, or model recency. None of the models provided reliable robustness against scanner-induced variability. While the models trained on the most diverse data, here represented by
vision-
language models, appear to have an advantage with respect to robustness, they underperformed on downstream supervised tasks. We conclude that development and evaluation of PFMs requires moving beyond accuracy-centric
benchmarks toward explicit evaluation and optimisation of
embedding stability and calibration under realistic acquisition variability.
Read the full abstract ▾
All authors Erik Thiringer, Fredrik K. Gustafsson, Kajsa Ledesma Eriksson, Mattias Rantalainen
Computer Vision
ToTMNet: FFT-Accelerated Toeplitz Temporal Mixing Network for Lightweight Remote Photoplethysmography
Vladimir Frants, Sos Agaian and Karen Panetta
Remote photoplethysmography (rPPG) estimates a blood volume pulse (BVP) waveform from facial
videos captured by commodity cameras. Although recent deep models improve robustness compared to classical signal-processing approaches, many methods increase computational cost and parameter count, and
attention-based temporal modeling introduces quadratic scaling with respect to the temporal length. This paper proposes ToTMNet, a lightweight rPPG architecture that replaces temporal
attention with an FFT-accelerated Toeplitz temporal mixing layer. The Toeplitz operator provides full-sequence temporal receptive field using a linear number of parameters in the clip length and can be applied in near-linear time using circulant
embedding and FFT-based convolution. ToTMNet integrates the global Toeplitz temporal operator into a compact gated temporal mixer that combines a local depthwise temporal convolution branch with gated global Toeplitz mixing, enabling efficient long-range temporal filtering while only having 63k parameters. Experiments on two
datasets, UBFC-rPPG (real
videos) and SCAMPS (synthetic
videos), show that ToTMNet achieves strong heart-rate estimation accuracy with a compact design. On UBFC-rPPG intra-
dataset evaluation, ToTMNet reaches 1.055 bpm MAE with Pearson correlation 0.996. In a synthetic-to-real setting (SCAMPS to UBFC-rPPG), ToTMNet reaches 1.582 bpm MAE with Pearson correlation 0.994. Ablation results confirm that the gating mechanism is important for effectively using global Toeplitz mixing, especially under domain shift. The main limitation of this preprint study is the use of only two
datasets; nevertheless, the results indicate that Toeplitz-structured temporal mixing is a practical and efficient alternative to
attention for rPPG.
Read the full abstract ▾
Computer Vision
Diffusion-DRF: Free, Rich, and Differentiable Reward for Video Diffusion Fine-Tuning
Yifan Wang, Yanyu Li, Gordon Guocheng Qian et al. (+3 more)
Video diffusion alignment has been heavily relied on scalar rewards. These rewards are typically derived from learned reward models in human preference
datasets, requiring additional training and extensive collection. Moreover, scalar rewards provide coarse, global supervision, offering limited
prompt-generation mismatch credit assignment and making models prone to reward exploitation and unstable optimization. We propose
Diffusion-DRF, a free, rich, and differentiable reward framework for
video diffusion fine-tuning.
Diffusion-DRF employs a frozen, off-the-shelf
Vision-
Language Model (VLM) as the critic, eliminating the need for reward model training. Instead of relying on a single scalar reward, it decomposes each user
prompt into multi-dimensional questions with freeform dense VQA explanation queries, yielding information-rich feedback. By direct differentiable optimization over this rich feedback,
Diffusion-DRF achieves stable reward-based tuning without preference
datasets collection.
Diffusion-DRF achieves significant gains both quantitatively and qualitatively, outperforming state-of-the-art Flow-GRPO by 4.74% in overall performance on unseen VBench-2.0.
Read the full abstract ▾
All authors Yifan Wang, Yanyu Li, Gordon Guocheng Qian, Sergey Tulyakov, Yun Fu, Anil Kag
Computer Vision
Apollo: Unified Multi-Task Audio-Video Joint Generation
Jun Wang, Chunyu Qiang, Yuxin Guo et al. (+3 more)
Audio-
video joint generation has progressed rapidly, yet substantial challenges still remain. Non-commercial approaches still suffer audio-visual asynchrony, poor lip-
speech alignment, and unimodal degradation, which can be stemmed from weak audio-visual correspondence modeling, limited generalization, and scarce high-quality dense-caption data. To address these issues, we introduce Apollo and delve into three axes–model architecture, training strategy, and data curation. Architecturally, we adopt a single-tower design with unified DiT blocks and an Omni-Full
Attention mechanism, achieving tight audio-visual
alignment and strong scalability. Training-wise, we adopt a progressive multitask regime–random modality masking to joint optimization across tasks, and a multistage curriculum, yielding robust representations, strengthening A-V aligned world knowledge, and preventing unimodal collapse. For
datasets, we present the first large-scale audio-
video dataset with dense captions, and introduce a novel automated data-construction pipeline which annotates and filters millions of diverse, high-quality, strictly aligned audio-
video-caption triplets. Building on this, Apollo scales to large
datasets, delivering high-fidelity, semantically and temporally aligned, instruction-following generation in both joint and unimodal settings while generalizing robustly to out-of-distribution scenarios. Across tasks, it substantially outperforms prior methods by a large margin and achieves performance comparable to Veo 3, offering a unified, scalable path toward next-generation audio-
video synthesis.
Read the full abstract ▾
All authors Jun Wang, Chunyu Qiang, Yuxin Guo, Yiran Wang, Xijuan Zeng, Feng Deng
Computer Vision · Cross-listed from cs.RO
Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Test
Chun-Kai Fan, Xiaowei Chi, Xiaozhu Ju et al. (+18 more)
As world models gain momentum in
Embodied AI, an increasing number of works explore using
video foundation models as predictive world models for downstream
embodied tasks like 3D prediction or interactive generation. However, before exploring these downstream tasks,
video foundation models still have two critical questions unanswered: (1) whether their generative generalization is sufficient to maintain perceptual fidelity in the eyes of human observers, and (2) whether they are robust enough to serve as a universal prior for real-world
embodied agents. To provide a standardized framework for answering these questions, we introduce the
Embodied Turing Test
benchmark: WoW-World-Eval (Wow,wo,val). Building upon 609
robot manipulation data, Wow-wo-val examines five core abilities, including perception, planning, prediction, generalization, and execution. We propose a comprehensive evaluation protocol with 22 metrics to assess the models' generation ability, which achieves a high Pearson Correlation between the overall score and human preference (>0.93) and establishes a reliable foundation for the Human Turing Test. On Wow-wo-val, models achieve only 17.27 on long-horizon planning and at best 68.02 on physical consistency, indicating limited spatiotemporal consistency and physical
reasoning. For the Inverse Dynamic Model Turing Test, we first use an IDM to evaluate the
video foundation models' execution accuracy in the real world. However, most models collapse to $\approx$ 0% success, while WoW maintains a 40.74% success rate. These findings point to a noticeable gap between the generated
videos and the real world, highlighting the urgency and necessity of benchmarking World Model in
Embodied AI.
Read the full abstract ▾
All authors Chun-Kai Fan, Xiaowei Chi, Xiaozhu Ju, Hao Li, Yong Bao, Yu-Kai Wang, Lizhang Chen, Zhiyuan Jiang, Kuangzhi Ge, Ying Li, Weishi Mi, Qingpo Wuwu, Peidong Jia, Yulin Luo, Kevin Zhang, Zhiyuan Qin, Yong Dai, Sirui Han, Yike Guo, Shanghang Zhang, Jian Tang
Computer Vision
Pixel-Wise Multimodal Contrastive Learning for Remote Sensing Images
Leandro Stival, Ricardo da Silva Torres and Helio Pedrini
Satellites continuously generate massive volumes of data, particularly for Earth observation, including satellite image time series (SITS). However, most deep learning models are designed to process either entire images or complete time series sequences to extract meaningful features for downstream tasks. In this study, we propose a novel
multimodal approach that leverages pixel-wise two-dimensional (2D) representations to encode visual property variations from SITS more effectively. Specifically, we generate recurrence plots from pixel-based vegetation index time series (NDVI, EVI, and SAVI) as an alternative to using raw pixel values, creating more informative representations. Additionally, we introduce PIxel-wise
Multimodal Contrastive (PIMC), a new
multimodal self-supervision approach that produces effective encoders based on two-dimensional pixel time series representations and remote sensing imagery (RSI). To validate our approach, we assess its performance on three downstream tasks: pixel-level
forecasting and classification using the PASTIS
dataset, and land cover classification on the EuroSAT
dataset. Moreover, we compare our results to state-of-the-art (SOTA) methods on all downstream tasks. Our experimental results show that the use of 2D representations significantly enhances feature extraction from SITS, while contrastive learning improves the quality of representations for both pixel time series and RSI. These findings suggest that our
multimodal method outperforms existing models in various Earth observation tasks, establishing it as a robust self-supervision framework for processing both SITS and RSI. Code avaliable on
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training
Ziyun Zhang, Zezhou Wang, Xiaoyi Zhang et al. (+4 more)
GUI
agents that interact with graphical interfaces on behalf of users represent a promising direction for practical AI assistants. However, training such
agents is hindered by the scarcity of suitable environments. We present InfiniteWeb, a system that automatically generates functional web environments at scale for GUI
agent training. While LLMs perform well on generating a single webpage, building a realistic and functional website with many interconnected pages faces challenges. We address these challenges through unified specification, task-centric test-driven development, and a combination of website seed with reference design image to ensure diversity. Our system also generates verifiable task evaluators enabling dense reward signals for reinforcement learning. Experiments show that InfiniteWeb surpasses commercial coding
agents at realistic website construction, and GUI
agents trained on our generated environments achieve significant performance improvements on OSWorld and Online-Mind2Web, demonstrating the effectiveness of proposed system.
Read the full abstract ▾
All authors Ziyun Zhang, Zezhou Wang, Xiaoyi Zhang, Zongyu Guo, Jiahao Li, Bin Li, Yan Lu
Computer Vision · Cross-listed from cs.LG
MORPHFED: Federated Learning for Cross-institutional Blood Morphology Analysis
Gabriel Ansah, Eden Ruffell, Delmiro Fernandez-Reyes et al. (+1 more)
Automated blood morphology analysis can support hematological diagnostics in low- and middle-income countries (LMICs) but remains sensitive to
dataset shifts from staining variability, imaging differences, and rare morphologies. Building centralized
datasets to capture this diversity is often infeasible due to
privacy regulations and data-sharing restrictions. We introduce a
federated learning framework for white blood
cell morphology analysis that enables collaborative training across institutions without exchanging training data. Using blood films from multiple clinical sites, our
federated models learn robust, domain-invariant representations while preserving complete data
privacy. Evaluations across convolutional and
transformer-based architectures show that
federated training achieves strong cross-site performance and improved generalization to unseen institutions compared to centralized training. These findings highlight
federated learning as a practical and
privacy-preserving approach for developing equitable, scalable, and generalizable medical imaging AI in resource-limited healthcare environments.
Read the full abstract ▾
All authors Gabriel Ansah, Eden Ruffell, Delmiro Fernandez-Reyes, Petru Manescu
Computer Vision
GeoReason: Aligning Thinking And Answering In Remote Sensing Vision-Language Models Via Logical Consistency Reinforcement Learning
Wenshuai Li, Xiantai Xiang, Zixiao Wen et al. (+6 more)
The
evolution of Remote Sensing
Vision-
Language Models(RS-VLMs) emphasizes the importance of transitioning from perception-centric recognition toward high-level deductive
reasoning to enhance cognitive reliability in complex spatial tasks. However, current models often suffer from logical
hallucinations, where correct answers are derived from flawed
reasoning chains or rely on positional shortcuts rather than spatial logic. This decoupling undermines reliability in strategic spatial decision-making. To address this, we present GeoReason, a framework designed to synchronize internal thinking with final decisions. We first construct GeoReason-Bench, a logic-driven
dataset containing 4,000
reasoning trajectories synthesized from
geometric primitives and expert knowledge. We then formulate a two-stage training strategy: (1) Supervised Knowledge Initialization to equip the model with
reasoning syntax and domain expertise, and (2) Consistency-Aware Reinforcement Learning to refine deductive reliability. This second stage integrates a novel Logical Consistency Reward, which penalizes logical drift via an option permutation strategy to anchor decisions in verifiable
reasoning traces. Experimental results demonstrate that our framework significantly enhances the cognitive reliability and interpretability of RS-VLMs, achieving state-of-the-art performance compared to other advanced methods.
Read the full abstract ▾
All authors Wenshuai Li, Xiantai Xiang, Zixiao Wen, Guangyao Zhou, Ben Niu, Feng Wang, Lijia Huang, Qiantong Wang, Yuxin Hu
Computer Vision
Gen3R: 3D Scene Generation Meets Feed-Forward Reconstruction
Jiaxin Huang, Yuanbo Yang, Bangbang Yang et al. (+3 more)
We present Gen3R, a method that bridges the strong priors of foundational reconstruction models and
video diffusion models for scene-level 3D generation. We repurpose the VGGT reconstruction model to produce
geometric latents by training an adapter on its
tokens, which are regularized to align with the appearance latents of pre-trained
video diffusion models. By jointly generating these disentangled yet aligned latents, Gen3R produces both RGB
videos and corresponding 3D geometry, including camera poses, depth maps, and global point clouds. Experiments demonstrate that our approach achieves state-of-the-art results in single- and multi-image conditioned 3D scene generation. Additionally, our method can enhance the robustness of reconstruction by leveraging generative priors, demonstrating the mutual benefit of tightly coupling reconstruction and generative models.
Read the full abstract ▾
All authors Jiaxin Huang, Yuanbo Yang, Bangbang Yang, Lin Ma, Yuewen Ma, Yiyi Liao
Computer Vision
Analyzing Reasoning Consistency in Large Multimodal Models under Cross-Modal Conflicts
Zhihao Zhu, Jiafeng Liang, Shixin Jiang et al. (+5 more)
Large
Multimodal Models (LMMs) have demonstrated impressive capabilities in
video reasoning via Chain-of-Thought (CoT). However, the robustness of their
reasoning chains remains questionable. In this paper, we identify a critical failure mode termed textual inertia, where once a textual
hallucination occurs in the thinking process, models tend to blindly adhere to the erroneous text while neglecting conflicting visual evidence. To systematically investigate this, we propose the LogicGraph Perturbation Protocol that structurally injects perturbations into the
reasoning chains of diverse LMMs spanning both native
reasoning architectures and
prompt-driven paradigms to evaluate their self-reflection capabilities. The results reveal that models successfully self-correct in less than 10% of cases and predominantly succumb to blind textual error propagation. To mitigate this, we introduce Active Visual-Context Refinement, a training-free inference paradigm which orchestrates an active visual re-grounding mechanism to enforce fine-grained verification coupled with an adaptive context refinement strategy to summarize and denoise the
reasoning history. Experiments demonstrate that our approach significantly stifles
hallucination propagation and enhances
reasoning robustness.
Read the full abstract ▾
All authors Zhihao Zhu, Jiafeng Liang, Shixin Jiang, Jinlan Fu, Ming Liu, Guanglu Sun, See-Kiong Ng, Bing Qin
Computer Vision
Mind the Generative Details: Direct Localized Detail Preference Optimization for Video Diffusion Models
Zitong Huang, Kaidong Zhang, Yukang Ding et al. (+4 more)
Aligning text-to-
video diffusion models with human preferences is crucial for generating high-quality
videos. Existing Direct Preference Otimization (DPO) methods rely on multi-sample ranking and task-specific critic models, which is inefficient and often yields ambiguous global supervision. To address these limitations, we propose LocalDPO, a novel post-training framework that constructs localized preference pairs from real
videos and optimizes
alignment at the spatio-temporal region level. We design an automated pipeline to efficiently collect preference pair data that generates preference pairs with a single inference per
prompt, eliminating the need for external critic models or manual annotation. Specifically, we treat high-quality real
videos as positive samples and generate corresponding negatives by locally corrupting them with random spatio-temporal masks and restoring only the masked regions using the frozen base model. During training, we introduce a region-aware DPO loss that restricts preference learning to corrupted areas for rapid convergence. Experiments on Wan2.1 and CogVideoX demonstrate that LocalDPO consistently improves
video fidelity, temporal coherence and human preference scores over other post-training approaches, establishing a more efficient and fine-grained paradigm for
video generator
alignment.The code is available at https://github.com/1170300714/Local-DPO.
Read the full abstract ▾
All authors Zitong Huang, Kaidong Zhang, Yukang Ding, Chao Gao, Rui Ding, Ying Chen, Wangmeng Zuo
Computer Vision
Unsupervised Modular Adaptive Region Growing and RegionMix Classification for Wind Turbine Segmentation
Raül Pérez-Gonzalo, Riccardo Magro, Andreas Espersen et al. (+1 more)
Reliable operation of wind turbines requires frequent inspections, as even minor surface damages can degrade aerodynamic performance, reduce energy output, and accelerate blade wear. Central to automating these inspections is the accurate
segmentation of turbine blades from visual data. This task is traditionally addressed through dense, pixel-wise deep learning models. However, such methods demand extensive annotated
datasets, posing scalability challenges. In this work, we introduce an annotation-efficient
segmentation approach that reframes the pixel-level task into a binary region classification problem. Image regions are generated using a fully unsupervised, interpretable Modular Adaptive Region Growing technique, guided by image-specific Adaptive Thresholding and enhanced by a Region Merging process that consolidates fragmented areas into coherent segments. To improve generalization and classification robustness, we introduce RegionMix, an augmentation strategy that synthesizes new training samples by combining distinct regions. Our framework demonstrates state-of-the-art
segmentation accuracy and strong cross-site generalization by consistently segmenting turbine blades across distinct windfarms.
Read the full abstract ▾
All authors Raül Pérez-Gonzalo, Riccardo Magro, Andreas Espersen, Antonio Agudo
Computer Vision · Cross-listed from cs.RO
CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videos
Chubin Zhang, Jianan Wang, Zifeng Gao et al. (+5 more)
Generalist
Vision-
Language-Action models remain constrained by the scarcity of robotic data relative to the abundance of human
video demonstrations. Existing Latent Action Models attempt to use
video data but often suffer from visual
entanglement, encoding noise rather than manipulation skills. To address this limitation, we propose Contrastive Latent Action Pretraining (CLAP), a framework that first uses Act-VAE to learn an executable action-
token vocabulary from
robot trajectories and then aligns human visual transitions with this vocabulary through contrastive learning. This
alignment maps unlabeled human
videos into a physically grounded latent action space rather than reconstructing appearance. Building on the aligned
tokens, we train CLAP-NTP as an autoregressive VLA using
robot demonstrations and pseudo-labeled human
videos, preserving instruction following and object generalization. For deployment and target-domain adaptation, we further introduce a post-training strategy that combines CLAP-RF, a Rectified Flow action head for low-latency continuous action chunk prediction, with Knowledge Matching regularization to preserve pretrained semantic knowledge during fine-tuning. Extensive experiments show that CLAP achieves strong performance against competitive baselines while enabling effective skill transfer from human
videos to robotic execution.
Read the full abstract ▾
All authors Chubin Zhang, Jianan Wang, Zifeng Gao, Yue Su, Tianru Dai, Cai Zhou, Jiwen Lu, Yansong Tang
Computer Vision
Thinking with Frames: Generative Video Distortion Evaluation via Frame Reward Model
Yuan Wang, Borui Liao, Huijuan Huang et al. (+5 more)
Recent advances in
video reward models and post-training strategies have improved text-to-
video (T2V) generation. While these models typically assess visual quality, motion quality, and text
alignment, they often overlook key structural distortions, such as abnormal object appearances and interactions, which can degrade the overall quality of the generative
video. To address this gap, we introduce REACT, a frame-level reward model designed specifically for structural distortions evaluation in generative
videos. REACT assigns point-wise scores and attribution labels by
reasoning over
video frames, focusing on recognizing distortions. To support this, we construct a large-scale human preference
dataset, annotated based on our proposed taxonomy of structural distortions, and generate additional data using a efficient Chain-of-Thought (CoT) synthesis pipeline. REACT is trained with a two-stage framework: (1) supervised fine-tuning with masked loss for domain knowledge injection, followed by (2) reinforcement learning with Group Relative Policy Optimization (GRPO) and pairwise rewards to enhance
reasoning capability and align output scores with human preferences. During inference, a dynamic sampling mechanism is introduced to focus on frames most likely to exhibit distortion. We also present REACT-Bench, a
benchmark for generative
video distortion evaluation. Experimental results demonstrate that REACT complements existing reward models in assessing structutal distortion, achieving both accurate quantitative evaluations and interpretable attribution analysis.
Read the full abstract ▾
All authors Yuan Wang, Borui Liao, Huijuan Huang, Jinda Lu, Ouxiang Li, Kuien Liu, Meng Wang, Xiang Wang
Computer Vision
Padé Neurons for Efficient Neural Models
Onur Keleş and A. Murat Tekalp
Neural networks commonly employ the McCulloch-Pitts
neuron model, which is a linear model followed by a point-wise non-linear activation. Various researchers have already advanced inherently non-linear
neuron models, such as quadratic
neurons, generalized operational
neurons, generative
neurons, and super
neurons, which offer stronger non-linearity compared to point-wise activation functions. In this paper, we introduce a novel and better non-linear
neuron model called Padé
neurons (Paons), inspired by Padé approximants. Paons offer several advantages, such as diversity of non-linearity, since each Paon learns a different non-linear function of its inputs, and layer efficiency, since Paons provide stronger non-linearity in much fewer layers compared to piecewise linear approximation. Furthermore, Paons include all previously proposed
neuron models as special cases, thus any
neuron model in any network can be replaced by Paons. We note that there has been a proposal to employ the Padé approximation as a generalized point-wise activation function, which is fundamentally different from our model. To validate the efficacy of Paons, in our experiments, we replace classic
neurons in some well-known neural image super-resolution, compression, and classification models based on the ResNet architecture with Paons. Our comprehensive experimental results and analyses demonstrate that neural models built by Paons provide better or equal performance than their classic counterparts with a smaller number of layers. The PyTorch implementation code for Paon is open-sourced at https://github.com/onur-keles/Paon.
Read the full abstract ▾
Computer Vision
PosterVerse: A Full-Workflow Framework for Commercial-Grade Poster Generation with HTML-Based Scalable Typography
Junle Liu, Peirong Zhang, Yuyi Zhang et al. (+5 more)
Commercial-grade poster design demands the seamless integration of aesthetic appeal with precise, informative content delivery. Current automated poster generation systems face significant limitations, including incomplete design workflows, poor text rendering accuracy, and insufficient flexibility for commercial applications. To address these challenges, we propose PosterVerse, a full-workflow, commercial-grade poster generation method that seamlessly automates the entire design process while delivering high-density and scalable text rendering. PosterVerse replicates professional design through three key stages: (1) blueprint creation using fine-tuned LLMs to extract key design elements from user requirements, (2) graphical background generation via customized
diffusion models to create visually appealing imagery, and (3) unified layout-text rendering with an MLLM-powered HTML engine to guarantee high text accuracy and flexible customization. In addition, we introduce PosterDNA, a commercial-grade, HTML-based
dataset tailored for training and validating poster design models. To the best of our knowledge, PosterDNA is the first Chinese poster generation
dataset to introduce HTML typography files, enabling scalable text rendering and fundamentally solving the challenges of rendering small and high-density text. Experimental results demonstrate that PosterVerse consistently produces commercial-grade posters with appealing visuals, accurate text
alignment, and customizable layouts, making it a promising solution for automating commercial poster design. The code and model are available at https://github.com/wuhaer/PosterVerse.
Read the full abstract ▾
All authors Junle Liu, Peirong Zhang, Yuyi Zhang, Pengyu Yan, Hui Zhou, Xinyue Zhou, Fengjun Guo, Lianwen Jin
Computer Vision
FUSION: Full-Body Unified Motion Prior for Body and Hands via Diffusion
Enes Duran, Nikos Athanasiou, Muhammed Kocabas et al. (+2 more)
Hands are central to interacting with our surroundings and conveying gestures, making their inclusion essential for full-body motion synthesis. Despite this, existing human motion synthesis methods fall short: some ignore hand motions entirely, while others generate full-body motions only for narrowly scoped tasks under highly constrained settings. A key obstacle is the lack of large-scale
datasets that jointly capture diverse full-body motion with detailed hand articulation. While some
datasets capture both, they are limited in scale and diversity. Conversely, large-scale
datasets typically focus either on body motion without hands or on hand motions without the body. To overcome this, we curate and unify existing hand motion
datasets with large-scale body motion data to generate full-body sequences that capture both hand and body. We then propose the first
diffusion-based unconditional full-body motion prior, FUSION, which jointly models body and hand motion. Despite using a pose-based motion representation, FUSION surpasses state-of-the-art skeletal control models on the Keypoint Tracking task in the HumanML3D
dataset and achieves superior motion naturalness. Beyond standard
benchmarks, we demonstrate that FUSION can go beyond typical uses of motion priors through two applications: (1) generating detailed full-body motion including fingers during interaction given the motion of an object, and (2) generating Self-Interaction motions using an LLM to transform natural
language cues into actionable motion constraints. For these applications, we develop an optimization pipeline that refines the latent space of our
diffusion model to generate task-specific motions. Experiments on these tasks highlight precise control over hand motion while maintaining plausible full-body coordination. The code will be public.
Read the full abstract ▾
All authors Enes Duran, Nikos Athanasiou, Muhammed Kocabas, Michael J. Black, Omid Taheri
Computer Vision
ResTok: Learning Hierarchical Residuals in 1D Visual Tokenizers for Autoregressive Image Generation
Xu Zhang, Cheng Da, Huan Yang et al. (+3 more)
Existing 1D visual tokenizers for autoregressive (AR) generation largely follow the design principles of
language modeling, as they are built directly upon
transformers whose priors originate in
language, yielding single-hierarchy latent
tokens and treating visual data as flat sequential
token streams. However, this
language-like formulation overlooks key properties of
vision, particularly the hierarchical and residual network designs that have long been essential for convergence and efficiency in visual models. To bring "
vision" back to
vision, we propose the Residual Tokenizer (ResTok), a 1D visual tokenizer that builds hierarchical residuals for both image
tokens and latent
tokens. The hierarchical representations obtained through progressively merging enable cross-level feature fusion at each layer, substantially enhancing representational capacity. Meanwhile, the semantic residuals between hierarchies prevent information overlap, yielding more concentrated latent distributions that are easier for AR modeling. Cross-level bindings consequently emerge without any explicit constraints. To accelerate the generation process, we further introduce a hierarchical AR generator that substantially reduces sampling steps by predicting an entire level of latent
tokens at once rather than generating them strictly
token-by-
token. Extensive experiments demonstrate that restoring hierarchical residual priors in visual tokenization significantly improves AR image generation, achieving a gFID of 2.34 on ImageNet-256 with only 9 sampling steps. Code is available at https://github.com/Kwai-Kolors/ResTok.
Read the full abstract ▾
All authors Xu Zhang, Cheng Da, Huan Yang, Kun Gai, Ming Lu, Zhan Ma
Computer Vision
FocusUI: Efficient UI Grounding via Position-Preserving Visual Token Selection
Mingyu Ouyang, Kevin Qinghong Lin, Mike Zheng Shou et al. (+1 more)
Vision-
Language Models (VLMs) have shown remarkable performance in User Interface (UI) grounding tasks, driven by their ability to process increasingly high-resolution screenshots. However, screenshots are tokenized into thousands of visual
tokens (e.g., about 4700 for 2K resolution), incurring significant computational overhead and diluting
attention. In contrast, humans typically focus on regions of interest when interacting with UI. In this work, we pioneer the task of efficient UI grounding. Guided by practical analysis of the task's characteristics and challenges, we propose FocusUI, an efficient UI grounding framework that selects patches most relevant to the instruction while preserving positional continuity for precise grounding. FocusUI addresses two key challenges: (1) Eliminating redundant
tokens in visual encoding. We construct patch-level supervision by fusing an instruction-conditioned score with a rule-based UI-graph score that down-weights large homogeneous regions to select distinct and instruction-relevant visual
tokens. (2) Preserving positional continuity during visual
token selection. We find that general visual
token pruning methods suffer from severe accuracy degradation on UI grounding tasks due to broken positional information. We introduce a novel PosPad strategy, which compresses each contiguous sequence of dropped visual
tokens into a single special marker placed at the sequence's last index to preserve positional continuity. Comprehensive experiments on four grounding
benchmarks demonstrate that FocusUI surpasses GUI-specific baselines. On the ScreenSpot-Pro
benchmark, FocusUI-7B achieves a performance improvement of 3.7% over GUI-Actor-7B. Even with only 30% visual
token retention, FocusUI-7B drops by only 3.2% while achieving up to 1.44x faster inference and 17% lower peak GPU
memory.
Read the full abstract ▾
All authors Mingyu Ouyang, Kevin Qinghong Lin, Mike Zheng Shou, Hwee Tou Ng
Computer Vision · Cross-listed from eess.IV
A low-complexity method for efficient depth-guided image deblurring
Ziyao Yi, Diego Valsesia, Tiziano Bianchi et al. (+1 more)
Image deblurring is a challenging problem in imaging due to its highly ill-posed nature. Deep learning models have shown great success in tackling this problem but the quest for the best image quality has brought their computational complexity up, making them impractical on anything but powerful servers. Meanwhile, recent works have shown that mobile
Lidars can provide complementary information in the form of depth maps that enhance deblurring quality. In this paper, we introduce a novel low-complexity neural network for depth-guided image deblurring. We show that the use of the wavelet transform to separate structural details and reduce spatial redundancy as well as efficient feature conditioning on the depth information are essential ingredients in developing a low-complexity model. Experimental results show competitive image quality against recent state-of-the-art models while reducing complexity by up to two orders of magnitude.
Read the full abstract ▾
All authors Ziyao Yi, Diego Valsesia, Tiziano Bianchi, Enrico Magli
Computer Vision
HemBLIP: A Vision-Language Model for Interpretable Leukemia Cell Morphology Analysis
Julie van Logtestijn and Petru Manescu
Microscopic evaluation of white blood
cell morphology is central to leukemia diagnosis, yet current deep learning models often act as black boxes, limiting clinical trust and adoption. We introduce HemBLIP, a
vision language model designed to generate interpretable, morphology aware descriptions of peripheral blood
cells. Using a newly constructed
dataset of 14k healthy and leukemic
cells paired with expert-derived attribute captions, we adapt a general-purpose VLM via both full fine-tuning and LoRA based parameter efficient training, and
benchmark against the biomedical foundation model MedGEMMA. HemBLIP achieves higher caption quality and morphological accuracy, while LoRA adaptation provides further gains with significantly reduced computational cost. These results highlight the promise of
vision language models for transparent and scalable hematological diagnostics.
Read the full abstract ▾
Computer Vision
FLNet: Flood-Induced Agriculture Damage Assessment using Super Resolution of Satellite Images
Sanidhya Ghosal, Anurag Sharma, Sushil Ghildiyal et al. (+1 more)
Distributing government relief efforts after a flood is challenging. In India, the crops are widely affected by floods; therefore, making rapid and accurate crop damage assessment is crucial for effective post-disaster agricultural management. Traditional manual surveys are slow and biased, while current satellite-based methods face challenges like cloud cover and low spatial resolution. Therefore, to bridge this gap, this paper introduced FLNet, a novel deep learning based architecture that used super-resolution to enhance the 10 m spatial resolution of Sentinel-2 satellite images into 3 m resolution before classifying damage. We tested our model on the Bihar Flood Impacted Croplands
Dataset (BFCD-22), and the results showed an improved critical "Full Damage" F1-score from 0.83 to 0.89, nearly matching the 0.89 score of commercial high-resolution imagery. This work presented a cost-effective and scalable solution, paving the way for a nationwide shift from manual to automated, high-fidelity damage assessment.
Read the full abstract ▾
All authors Sanidhya Ghosal, Anurag Sharma, Sushil Ghildiyal, Mukesh Saini
Computer Vision · Cross-listed from eess.IV
Staged Voxel-Level Deep Reinforcement Learning for 3D Medical Image Segmentation with Noisy Annotations
Yuyang Fu, Xiuzhen Guo and Ji Shi
Deep learning has achieved significant advancements in medical image
segmentation. Currently, obtaining accurate
segmentation outcomes is critically reliant on large-scale
datasets with high-quality annotations. However, noisy annotations are frequently encountered owing to the complex morphological structures of organs in medical images and variations among different annotators, which can substantially limit the efficacy of
segmentation models. Motivated by the fact that medical imaging annotator can correct labeling errors during
segmentation based on prior knowledge, we propose an end-to-end Staged Voxel-Level Deep Reinforcement Learning (SVL-DRL) framework for robust medical image
segmentation under noisy annotations. This framework employs a dynamic iterative update strategy to automatically mitigate the impact of erroneous labels without requiring manual intervention. The key advancements of SVL-DRL over existing works include: i) formulating noisy annotations as a voxel-dependent problem and addressing it through a novel staged reinforcement learning framework which guarantees robust model convergence; ii) incorporating a voxel-level asynchronous advantage actor-critic (vA3C) module that conceptualizes each voxel as an autonomous
agent, which allows each
agent to dynamically refine its own state representation during training, thereby directly mitigating the influence of erroneous labels; iii) designing a novel action space for the
agents, along with a composite reward function that strategically combines the Dice value and a spatial continuity metric to significantly boost
segmentation accuracy while maintain semantic integrity. Experiments on three public medical image
datasets demonstrates State-of-The-Art (SoTA) performance under various experimental settings, with an average improvement of over 3% in both Dice and IoU scores.
Read the full abstract ▾
Computer Vision
Bayesian Monocular Depth Refinement via Neural Radiance Fields
Arun Muthukkumar
Monocular depth estimation has applications in many fields, such as autonomous
navigation and extended reality, making it an essential computer
vision task. However, current methods often produce smooth depth maps that lack the fine
geometric detail needed for accurate scene understanding. We propose MDENeRF, an iterative framework that refines
monocular depth estimates using depth information from Neural Radiance Fields (NeRFs). MDENeRF consists of three components: (1) an initial
monocular estimate for global structure, (2) a NeRF trained on perturbed viewpoints, with per-pixel uncertainty, and (3)
Bayesian fusion of the noisy
monocular and NeRF depths. We derive NeRF uncertainty from the volume rendering process to iteratively inject high-frequency fine details. Meanwhile, our
monocular prior maintains global structure. We demonstrate improvements on key metrics and experiments using indoor scenes from the SUN RGB-D
dataset.
Read the full abstract ▾
Computer Vision
IDESplat: Iterative Depth Probability Estimation for Generalizable 3D Gaussian Splatting
Wei Long, Haifeng Wu, Shiyin Jiang et al. (+3 more)
Generalizable 3D Gaussian Splatting aims to directly predict Gaussian parameters using a feed-forward network for scene reconstruction. Among these parameters, Gaussian means are particularly difficult to predict, so depth is usually estimated first and then unprojected to obtain the Gaussian sphere centers. Existing methods typically rely solely on a single warp to estimate depth probability, which hinders their ability to fully leverage cross-view
geometric cues, resulting in unstable and coarse depth maps. To address this limitation, we propose IDESplat, which iteratively applies warp operations to boost depth probability estimation for accurate Gaussian mean prediction. First, to eliminate the inherent instability of a single warp, we introduce a Depth Probability Boosting Unit (DPBU) that integrates epipolar
attention maps produced by cascading warp operations in a multiplicative manner. Next, we construct an iterative depth estimation process by stacking multiple DPBUs, progressively identifying potential depth candidates with high likelihood. As IDESplat iteratively boosts depth probability estimates and updates the depth candidates, the depth map is gradually refined, resulting in accurate Gaussian means. We conduct experiments on RealEstate10K, ACID, and DL3DV. IDESplat achieves outstanding reconstruction quality and state-of-the-art performance with real-time efficiency. On RE10K, it outperforms DepthSplat by 0.33 dB in PSNR, using only 10.7% of the parameters and 70% of the
memory. Additionally, our IDESplat improves PSNR by 2.95 dB over DepthSplat on the DTU
dataset in cross-
dataset experiments, demonstrating its strong generalization ability.
Read the full abstract ▾
All authors Wei Long, Haifeng Wu, Shiyin Jiang, Jinhua Zhang, Xinchun Ji, Shuhang Gu
Computer Vision
EvalBlocks: A Modular Pipeline for Rapidly Evaluating Foundation Models in Medical Imaging
Jan Tagscherer, Sarah de Boer, Lena Philipp et al. (+7 more)
Developing foundation models in medical imaging requires continuous monitoring of downstream performance. Researchers are burdened with tracking numerous experiments, design choices, and their effects on performance, often relying on ad-hoc, manual workflows that are inherently slow and error-prone. We introduce EvalBlocks, a modular, plug-and-play framework for efficient evaluation of foundation models during development. Built on Snakemake, EvalBlocks supports seamless integration of new
datasets, foundation models, aggregation methods, and evaluation strategies. All experiments and results are tracked centrally and are reproducible with a single command, while efficient caching and parallel execution enable scalable use on shared compute infrastructure. Demonstrated on five state-of-the-art foundation models and three medical imaging classification tasks, EvalBlocks streamlines model evaluation, enabling researchers to iterate faster and focus on model innovation rather than evaluation logistics. The framework is released as open source software at https://github.com/DIAGNijmegen/eval-blocks.
Read the full abstract ▾
All authors Jan Tagscherer, Sarah de Boer, Lena Philipp, Fennie van der Graaf, Dré Peeters, Joeran Bosma, Lars Leijten, Bogdan Obreja, Ewoud Smit, Alessa Hering
Computer Vision
From Brute Force to Semantic Insight: Performance-Guided Data Transformation Design with LLMs
Usha Shrestha, Dmitry Ignatov and Radu Timofte
Large
language models (LLMs) have achieved notable performance in code synthesis; however, data-aware augmentation remains a limiting factor, handled via heuristic design or brute-force approaches. We introduce a performance-aware, closed-loop solution in the NNGPT ecosystem of projects that enables LLMs to autonomously engineer optimal transformations by internalizing empirical performance cues. We fine-tune LLMs with Low-Rank Adaptation on a novel repository of more than 6,000 empirically evaluated PyTorch augmentation functions, each annotated solely by downstream model accuracy. Training uses pairwise performance ordering (better-worse transformations), enabling
alignment through empirical feedback without reinforcement learning, reward models, or symbolic objectives. This reduces the need for exhaustive search, achieving up to 600x times fewer evaluated candidates than brute-force discovery while maintaining competitive peak accuracy and shifting generation from random synthesis to task-aligned design. Ablation studies show that structured Chain-of-Thought prompting introduces syntactic noise and degrades performance, whereas direct prompting ensures stable optimization in performance-critical code tasks. Qualitative and quantitative analyses demonstrate that the model internalizes semantic performance cues rather than memorizing syntax. These results show that LLMs can exhibit task-level
reasoning through non-textual feedback loops, bypassing explicit symbolic rewards.
Read the full abstract ▾
Computer Vision
A Comparative Study of 3D Model Acquisition Methods for Synthetic Data Generation of Agricultural Products
Steven Moonen, Rob Salaets, Kenneth Batstone et al. (+2 more)
In the manufacturing industry, computer
vision systems based on artificial intelligence (AI) are widely used to reduce costs and increase production. Training these AI models requires a large amount of training data that is costly to acquire and annotate, especially in high-variance, low-volume manufacturing environments. A popular approach to reduce the need for real data is the use of synthetic data that is generated by leveraging computer-aided design (CAD) models available in the industry. However, in the agricultural industry these models are not readily available, increasing the difficulty in leveraging synthetic data. In this paper, we present different techniques for substituting CAD files to create synthetic
datasets. We measure their relative performance when used to train an AI object detection model to separate stones and potatoes in a bin picking environment. We demonstrate that using highly representative 3D models acquired by scanning or using image-to-3D approaches can be used to generate synthetic data for training object detection models. Finetuning on a small real
dataset can significantly improve the performance of the models and even get similar performance when less representative models are used.
Read the full abstract ▾
All authors Steven Moonen, Rob Salaets, Kenneth Batstone, Abdellatif Bey-Temsamani, Nick Michiels