Computer Vision
Think-as-You-See: Streaming Chain-of-Thought Reasoning for Large Vision-Language Models
Jialiang Zhang, Junlong Tong, Junyan Lin et al. (+4 more)
Large
Vision Language Models (LVLMs) exhibit strong Chain-of-Thought (CoT) capabilities, yet most existing paradigms assume full-
video availability before inference, a batch-style process misaligned with real-world
video streams where information arrives sequentially. Motivated by the streaming nature of
video data, we investigate two streaming
reasoning paradigms for LVLMs. The first, an interleaved paradigm, alternates between receiving frames and producing partial
reasoning but remains constrained by strictly ordered cache updates. To better match streaming inputs, we propose Think-as-You-See (TaYS), a unified framework enabling true concurrent
reasoning. TaYS integrates parallelized CoT generation, stream-constrained training, and stream-parallel inference. It further employs temporally aligned
reasoning units, streaming
attention masks and positional encodings, and a dual KV-cache that decouples visual encoding from textual
reasoning. We evaluate all paradigms on the Qwen2.5-VL family across representative
video CoT tasks, including event dynamics analysis, causal
reasoning, and thematic understanding. Experiments show that TaYS consistently outperforms both batch and interleaved baselines, improving
reasoning performance while substantially reducing time-to-first-
token (TTFT) and overall
reasoning delay. These results demonstrate the effectiveness of data-aligned streaming
reasoning in enabling efficient and responsive
video understanding for LVLMs. We release our code at https://github.com/EIT-NLP/StreamingLLM/tree/main/TaYS
Read the full abstract ▾
All authors Jialiang Zhang, Junlong Tong, Junyan Lin, Hao Wu, Yirong Sun, Yunpu Ma, Xiaoyu Shen
Computer Vision
Multimodal-Prior-Guided Importance Sampling for Hierarchical Gaussian Splatting in Sparse-View Novel View Synthesis
Kaiqiang Xiong, Zhanke Wang and Ronggang Wang
We present
multimodal-prior-guided importance sampling as the central mechanism for hierarchical 3D Gaussian Splatting (3DGS) in sparse-view novel view synthesis. Our sampler fuses complementary cues { – } photometric rendering residuals, semantic priors, and
geometric priors { – } to produce a robust, local recoverability estimate that directly drives where to inject fine Gaussians. Built around this sampling core, our framework comprises (1) a coarse-to-fine Gaussian representation that encodes global shape with a stable coarse layer and selectively adds fine primitives where the
multimodal metric indicates recoverable detail; and (2) a
geometric-aware sampling and retention policy that concentrates refinement on geometrically critical and complex regions while protecting newly added primitives in underconstrained areas from premature
pruning. By prioritizing regions supported by consistent
multimodal evidence rather than raw residuals alone, our method alleviates overfitting texture-induced errors and suppresses noise from pose/appearance inconsistencies. Experiments on diverse sparse-view
benchmarks demonstrate state-of-the-art reconstructions, with up to +0.3 dB PSNR on DTU.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Signals Are Present, Predictions Fail: Probing Diagram Representations in Large Vision-Language Models
Haruto Yoshida, Keito Kudo, Yoichi Aoki et al. (+4 more)
Large
vision-
language models (LVLMs) demonstrate strong performance on diagram understanding
benchmarks, yet they still struggle to understand relationships between elements, particularly those represented by directed edges between nodes. To investigate the causes of this limitation, we probe the internal representations of LVLMs using a synthetic diagram
dataset based on directed graphs. Our linear probing experiments reveal that signals about nodes and edges are linearly decodable from the hidden states of LVLMs, even for aspects that LVLMs struggle with in VQA, such as edge direction and connectivity between nodes. These results reveal a gap between internal signals and model outputs. We further perform causal interventions on the model's hidden states, showing that, for some aspects, the signals identified through probing causally contribute to the model's final predictions. These findings also hold on flowcharts, not only on synthetic diagrams.
Read the full abstract ▾
All authors Haruto Yoshida, Keito Kudo, Yoichi Aoki, Ryota Tanaka, Itsumi Saito, Keisuke Sakaguchi, Kentaro Inui
Computer Vision
Scale-invariant Gaussian derivative residual networks
Andrzej Perzanowski and Tony Lindeberg
Generalisation across image scales remains a fundamental challenge for deep networks, which often fail to handle images at scales not seen during training (the out-of-distribution problem). In this paper, we present provably scale-invariant Gaussian derivative residual networks (GaussDerResNets), constructed out of scale-covariant Gaussian derivative residual blocks coupled in cascade, aimed at addressing this problem. By adding residual skip connections to the previous notion of Gaussian derivative layers, deeper networks with substantially increased accuracy can be constructed, while preserving very good scale generalisation properties. Explicit proofs are provided for the underlying scale-covariant and scale-invariant properties in arbitrary dimensions. To analyse the ability of GaussDerResNets to generalise to new scales, we apply them on a new rescaled version of the STL-10
dataset, where training is done at a single fixed scale and evaluation is performed on copies of the test set, each rescaled to a distinct spatial scale, with scale factors extending over a range of 4. We also conduct similar systematic experiments on the rescaled versions of Fashion-MNIST and CIFAR-10
datasets, and the existing STIR
datasets. Experimentally, we demonstrate that the GaussDerResNets have strong scale generalisation and scale selection properties on all the four considered
datasets with scaling variations. In our ablation studies, we investigate different architectural variants of GaussDerResNets, demonstrating that basing the architecture on depthwise-separable convolutions reduces the number of parameters and computations, with reasonably maintained accuracy and scale generalisation. We conclude by outlining how the proposed GaussDerResNets can be extended to joint local spatial and scale selection, to address the topic of multi-object detection in a provably scale-invariant manner.
Read the full abstract ▾
Computer Vision
Toward Early Quality Assessment of Text-to-Image Diffusion Models
Huanlei Guo, Hongxin Wei and Bingyi Jing
Recent text-to-image (T2I)
diffusion and flow-matching models can produce highly realistic images from natural
language prompts. In practical scenarios, T2I systems are often run in a “generate–then–select” mode: many seeds are sampled and only a few images are kept for use. However, this pipeline is highly resource-intensive since each candidate requires tens to hundreds of denoising steps, and evaluation metrics such as CLIPScore and ImageReward are post-hoc. In this work, we address this inefficiency by introducing Probe-Select, a plug-in module that enables efficient evaluation of image quality within the generation process. We observe that certain intermediate denoiser activations, even at early timesteps, encode a stable coarse structure, object layout and spatial arrangement–that strongly correlates with final image fidelity. Probe-Select exploits this property by predicting final quality scores directly from early activations, allowing unpromising seeds to be terminated early. Across
diffusion and flow-matching backbones, our experiments show that early evaluation at only 20% of the trajectory accurately ranks candidate seeds and enables selective continuation. This strategy reduces sampling cost by over 60% while improving the quality of the retained images, demonstrating that early structural signals can effectively guide selective generation without altering the underlying generative model. Code is available at https://github.com/Guhuary/ProbeSelect.
Read the full abstract ▾
Computer Vision
BrandFusion: A Multi-Agent Framework for Seamless Brand Integration in Text-to-Video Generation
Zihao Zhu, Ruotong Wang, Siwei Lyu et al. (+2 more)
The rapid advancement of text-to-
video (T2V) models has revolutionized content creation, yet their commercial potential remains largely untapped. We introduce, for the first time, the task of seamless brand integration in T2V: automatically
embedding advertiser brands into
prompt-generated
videos while preserving semantic fidelity to user intent. This task confronts three core challenges: maintaining
prompt fidelity, ensuring brand recognizability, and achieving contextually natural integration. To address them, we propose BrandFusion, a novel multi-
agent framework comprising two synergistic phases. In the offline phase (advertiser-facing), we construct a Brand Knowledge Base by probing model priors and adapting to novel brands via lightweight fine-tuning. In the online phase (user-facing), five
agents jointly refine user
prompts through iterative refinement, leveraging the shared knowledge base and real-time contextual tracking to ensure brand visibility and semantic
alignment. Experiments on 18 established and 2 custom brands across multiple state-of-the-art T2V models demonstrate that BrandFusion significantly outperforms baselines in semantic preservation, brand recognizability, and integration naturalness. Human evaluations further confirm higher user satisfaction, establishing a practical pathway for sustainable T2V monetization.
Read the full abstract ▾
All authors Zihao Zhu, Ruotong Wang, Siwei Lyu, Min Zhang, Baoyuan Wu
Computer Vision
ScribeTokens: Fixed-Vocabulary Tokenization of Digital Ink
Douglass Wang
Digital ink – the coordinate stream captured from stylus or touch input – lacks a unified representation. Continuous vector representations produce long sequences and suffer from training instability, while existing
token representations require large vocabularies, face out-of-vocabulary issues, and underperform vectors on recognition. We propose ScribeTokens, a tokenization that decomposes pen movement into unit pixel steps. Together with two pen-state
tokens, this fixed 10-
token base vocabulary suffices to represent any digital ink and enables aggressive BPE compression. On handwritten text generation, ScribeTokens dramatically outperforms vectors (17.33% vs. 70.29% CER), showing
tokens are far more effective for generation. On recognition, ScribeTokens is the only
token representation to outperform vectors without pretraining. We further introduce next-ink-
token prediction as a self-supervised pretraining strategy, which consistently improves recognition across all
token-based models and accelerates convergence by up to 83x. With pretraining, ScribeTokens achieves the best recognition results across all representations on both
datasets (8.27% CER on IAM, 9.83% on DeepWriting).
Read the full abstract ▾
Computer Vision
Structure-Aware Text Recognition for Ancient Greek Critical Editions
Nicolas Angleraud, Antonia Karamolegkou, Benoît Sagot et al. (+1 more)
Recent advances in visual
language models (VLMs) have transformed end-to-end document understanding. However, their ability to interpret the complex layout semantics of historical scholarly texts remains limited. This paper investigates structure-aware text recognition for Ancient Greek critical editions, which have dense reference hierarchies and extensive marginal annotations. We introduce two novel resources: (i) a large-scale synthetic corpus of 185,000 page images generated from TEI/XML sources with controlled typographic and layout variation, and (ii) a curated
benchmark of real scanned editions spanning more than a century of editorial and typographic practices. Using these
datasets, we evaluate three state-of-the-art VLMs under both zero-shot and fine-tuning regimes. Our experiments reveal substantial limitations in current VLM architectures when confronted with highly structured historical documents. In zero-shot settings, most models significantly underperform compared to established off-the-shelf software. Nevertheless, the Qwen3VL-8B model achieves state-of-the-art performance, reaching a median Character Error Rate of 1.0% on real scans. These results highlight both the current shortcomings and the future potential of VLMs for structure-aware recognition of complex scholarly documents.
Read the full abstract ▾
All authors Nicolas Angleraud, Antonia Karamolegkou, Benoît Sagot, Thibault Clérice
Computer Vision
NOVA: Sparse Control, Dense Synthesis for Pair-Free Video Editing
Tianlin Pan, Jiayi Dai, Chenpu Yuan et al. (+7 more)
Recent
video editing models have achieved impressive results, but most still require large-scale paired
datasets. Collecting such naturally aligned pairs at scale remains highly challenging and constitutes a critical bottleneck, especially for local
video editing data. Existing workarounds transfer image editing to
video through global motion control for pair-free
video editing, but such designs struggle with background and temporal consistency. In this paper, we propose NOVA: Sparse Control & Dense Synthesis, a new framework for unpaired
video editing. Specifically, the sparse branch provides semantic guidance through user-edited keyframes distributed across the
video, and the dense branch continuously incorporates motion and texture information from the original
video to maintain high fidelity and coherence. Moreover, we introduce a degradation-simulation training strategy that enables the model to learn motion reconstruction and temporal consistency by training on artificially degraded
videos, thus eliminating the need for paired data. Our extensive experiments demonstrate that NOVA outperforms existing approaches in edit fidelity, motion preservation, and temporal coherence.
Read the full abstract ▾
All authors Tianlin Pan, Jiayi Dai, Chenpu Yuan, Zhengyao Lv, Binxin Yang, Hubery Yin, Chen Li, Jing Lyu, Caifeng Shan, Chenyang Si
Computer Vision
R3GW: Relightable 3D Gaussians for Outdoor Scenes in the Wild
Margherita Lea Corona, Wieland Morgenstern, Peter Eisert et al. (+1 more)
3D Gaussian Splatting (3DGS) has established itself as a leading technique for 3D reconstruction and novel view synthesis of static scenes, achieving outstanding rendering quality and fast training. However, the method does not explicitly model the scene illumination, making it unsuitable for relighting tasks. Furthermore, 3DGS struggles to reconstruct scenes captured in the wild by unconstrained photo collections featuring changing lighting conditions. In this paper, we present R3GW, a novel method that learns a relightable 3DGS representation of an outdoor scene captured in the wild. Our approach separates the scene into a relightable foreground and a non-reflective background (the sky), using two distinct sets of Gaussians. R3GW models view-dependent lighting effects in the foreground reflections by combining Physically Based Rendering with the 3DGS scene representation in a varying illumination setting. We evaluate our method quantitatively and qualitatively on the NeRF-OSR
dataset, offering state-of-the-art performance and enhanced support for physically-based relighting of unconstrained scenes. Our method synthesizes photorealistic novel views under arbitrary illumination conditions. Additionally, our representation of the sky mitigates depth reconstruction artifacts, improving rendering quality at the sky-foreground boundary
Read the full abstract ▾
All authors Margherita Lea Corona, Wieland Morgenstern, Peter Eisert, Anna Hilsmann
Computer Vision
VSearcher: Long-Horizon Multimodal Search Agent via Reinforcement Learning
Ruiyang Zhang, Qianguo Sun, Chao Song et al. (+2 more)
Large models are increasingly becoming autonomous
agents that interact with real-world environments and use external tools to augment their static capabilities. However, most recent progress has focused on text-only large
language models, which are limited to a single modality and therefore have narrower application scenarios. On the other hand,
multimodal large models, while offering stronger perceptual capabilities, remain limited to static knowledge and lack the ability to access and leverage up-to-date web information. In this paper, we propose VSearcher, turning static
multimodal model into
multimodal search
agent capable of long-horizon, multi-turn tool use in real-world web environments, including text search, image search, and web browsing, via reinforcement learning. Specifically, we introduce Iterative Injection Data Synthesis pipeline to generate large-scale, complex
multimodal QA questions, which are further filtered with comprehensive metrics to ensure high quality and sufficient difficulty. We then adopt an SFT-then-RL training pipeline to turn base
multimodal models to
agent capable of multi-turn tool calling in real-world web environments. Besides, we propose a
multimodal search
benchmark MM-SearchExam dedicated to evaluating search capabilities of
multimodal search
agents, which proves highly challenging for recent proprietary models. Extensive evaluations across multiple
multimodal search
benchmarks reveal effectiveness of our method. VSearcher achieves superior performance compared to recent
multimodal search
agents and even surpasses several proprietary models on
multimodal web search tasks.
Read the full abstract ▾
All authors Ruiyang Zhang, Qianguo Sun, Chao Song, Yiyan Qi, Zhedong Zheng
Computer Vision
Designing UNICORN: a Unified Benchmark for Imaging in Computational Pathology, Radiology, and Natural Language
Michelle Stegeman, Lena Philipp, Fennie van der Graaf et al. (+20 more)
Foundation models are changing the way we develop medical artificial intelligence. By learning broadly generalizable features across diverse data modalities, a single model can be rapidly adapted to address multiple modalities and tasks with minimal supervision. This potential comes with the urgent need to reliably
benchmark, understand and compare the performance and clinical impact of foundation models across data modalities and clinical tasks. We introduce UNICORN, a fundamentally new benchmarking concept for medical foundation models. UNICORN brings four main contributions to medical artificial intelligence. First, a framework that enables a one-to-many benchmarking approach, where a single foundation model is tested across multiple tasks and data modalities. Here, we populate it with 20 tasks across radiology, pathology, and clinical text, covering classification, detection,
segmentation, regression, and
vision-
language generation. Second, a publicly available evaluation platform that implements, for the first time, a two-step approach to run foundation models for data encoding followed by custom task-specific adaptation via few-shot learning and linear probing mechanisms. Third, we create a meta-model that combines state-of-the-art foundation models in pathology, radiology and
language with novel task-specific adapters that address all UNICORN tasks, which we refer to as Unicorn Model-0 (UM-0). Finally, we design a novel UNICORN score to
benchmark and compare model performance across all tasks. We present the results of UM-0 using sequestered test data from over 2,400 patients, 3,700
vision cases, and 2,400 clinical reports from 17 institutions across eight countries, spanning eight anatomical regions and four imaging modalities. Data, baselines, and evaluation platform are publicly accessible at unicorn.grand-challenge.org.
Read the full abstract ▾
All authors Michelle Stegeman, Lena Philipp, Fennie van der Graaf, Marina D'Amato, Clément Grisi, Luc Builtjes, Joeran S. Bosma, Judith Lefkes, Rianne A. Weber, James A. Meakin, Thomas Koopman, Anne Mickan, Mathias Prokop, Ewoud J. Smit, Frédérique Meeuwsen, Geert Litjens, Jeroen van der Laak, Bram van Ginneken, Maarten de Rooij, Henkjan Huisman, Colin Jacobs, Francesco Ciompi, Alessa Hering
Computer Vision
HiLoRA: Hierarchical Low-Rank Adaptation for Personalized Federated Learning
Zihao Peng, Nan Zou, Jiandian Zeng et al. (+4 more)
Vision Transformers (ViTs) have been widely adopted in
vision tasks due to their strong transferability. In
Federated Learning (FL), where full fine-tuning is communication heavy, Low-Rank Adaptation (LoRA) provides an efficient and communication-friendly way to adapt ViTs. However, existing LoRA-based
federated tuning methods overlook latent client structures in real-world settings, limiting shared representation learning and hindering effective adaptation to unseen clients. To address this, we propose HiLoRA, a hierarchical LoRA framework that places adapters at three levels: root, cluster, and leaf, each designed to capture global, subgroup, and client-specific knowledge, respectively. Through cross-tier orthogonality and cascaded optimization, HiLoRA separates update subspaces and aligns each tier with its residual personalized objective. In particular, we develop a LoRA-Subspace Adaptive Clustering mechanism that infers latent client groups via subspace similarity analysis, thereby facilitating knowledge sharing across structurally aligned clients. Theoretically, we establish a tier-wise generalization analysis that supports HiLoRA's design. Experiments on ViT backbones with CIFAR-100 and DomainNet demonstrate consistent improvements in both personalization and generalization.
Read the full abstract ▾
All authors Zihao Peng, Nan Zou, Jiandian Zeng, Guo Li, Ke Chen, Boyuan Li, Tian Wang
Computer Vision
ITO: Multi-View Alignment and Training-Time Fusion for Image-Text Pretraining
Hanpeng Liu, Yaqian Li, Zidan Wang et al. (+6 more)
Image–text contrastive pretraining has become a dominant paradigm for visual representation learning, yet existing methods often yield representations that remain partially organized by modality rather than by semantics. We propose ITO, a framework addressing this limitation through two complementary mechanisms with distinct roles.
Multimodal multiple
alignment enriches supervision by constructing diverse cross-modal correspondences from multi-view image augmentations, providing the primary source of discriminative gain. A lightweight training-time
multimodal fusion module then acts as a training-time regularizer, encouraging the encoders to produce features that are compatible under fusion. Crucially, the fusion module is discarded at inference, preserving the efficiency of standard dual-encoder architectures while incurring training-time overhead only. Extensive experiments across pretraining scales from millions to billions of image–text pairs show that ITO outperforms strong contrastive baselines at moderate scale and yields consistent gains over CLIP under identical data, backbone, and compute at the 100M–1B scale, on classification,
retrieval, and
multimodal benchmarks. Controlled comparisons show that these gains are not explained by additional training compute or image augmentation alone. Our analysis further reveals that the contribution of fusion grows with data scale and that it stabilizes optimization, mitigating the late-stage overfitting commonly observed in aggressive contrastive learning. Code is available at https://github.com/showstarpro/ITO.
Read the full abstract ▾
All authors Hanpeng Liu, Yaqian Li, Zidan Wang, Shuoxi Zhang, Zonglin Zhao, Zihao Bo, Rinyoichi Takezoe, Kaiwen Long, Kun He
Computer Vision
Seeing Clearly without Training: Mitigating Hallucinations in Multimodal LLMs for Remote Sensing
Yi Liu, Jing Zhang, Di Wang et al. (+3 more)
Multimodal large
language models (MLLMs) suffer from pronounced
hallucinations in remote sensing visual question-answering (RS-VQA), primarily caused by visual grounding failures in large-scale scenes or misinterpretation of fine-grained small targets. To systematically analyze these issues, we introduce RSHBench, a protocol-based
benchmark for fine-grained diagnosis of factual and logical
hallucinations. To mitigate grounding-induced factual
hallucinations, we further propose Relative
Attention-Driven Actively
Reasoning (RADAR), a training-free inference method that leverages intrinsic
attention in MLLMs to guide progressive localization and fine-grained local
reasoning at test time. Extensive experiments across diverse MLLMs demonstrate that RADAR consistently improves RS-VQA performance and reduces both factual and logical
hallucinations. Code and data will be publicly available at: https://github.com/MiliLab/RADAR
Read the full abstract ▾
All authors Yi Liu, Jing Zhang, Di Wang, Xiaoyu Tian, Haonan Guo, Bo Du
Computer Vision
iGVLM: Dynamic Instruction-Guided Vision Encoding for Question-Aware Multimodal Understanding
Hanpeng Liu, Yaqian Li, Zidan Wang et al. (+5 more)
Despite the success of Large
Vision–
Language Models (LVLMs), most existing architectures suffer from a representation bottleneck: they rely on static, instruction-agnostic
vision encoders whose visual representations are utilized in an invariant manner across different textual tasks. This
rigidity hinders fine-grained
reasoning where task-specific visual cues are critical. To address this issue, we propose iGVLM, a general framework for instruction-guided visual modulation. iGVLM introduces a decoupled dual-branch architecture: a frozen representation branch that preserves task-agnostic visual representations learned during pre-training, and a dynamic conditioning branch that performs affine feature modulation via Adaptive Layer Normalization (AdaLN). This design enables a smooth transition from general-purpose perception to instruction-aware
reasoning while maintaining the structural integrity and stability of pre-trained visual priors. Beyond standard
benchmarks, we introduce MM4, a controlled diagnostic probe for quantifying logical consistency under multi-query, multi-instruction settings. Extensive results show that iGVLM consistently enhances instruction sensitivity across diverse
language backbones, offering a plug-and-play paradigm for bridging passive perception and active
reasoning.
Read the full abstract ▾
All authors Hanpeng Liu, Yaqian Li, Zidan Wang, Shuoxi Zhang, Zihao Bo, Rinyoichi Takezoe, Kaiwen Long, Kun He
Computer Vision
MultiShadow: Multi-Object Shadow Generation for Image Compositing via Diffusion Model
Waqas Ahmed, Dean Diepeveen and Ferdous Sohel
Realistic shadow generation is crucial for achieving seamless image compositing, yet existing methods primarily focus on single-object insertion and often fail to generalize when multiple foreground objects are composited into a background scene. In practice, however, modern compositing pipelines and real-world applications often insert multiple objects simultaneously, necessitating shadows that are jointly consistent in terms of geometry, attachment, and location. In this paper, we address the under-explored problem of multi-object shadow generation, aiming to synthesize physically plausible shadows for multiple inserted objects. Our approach exploits the
multimodal capabilities of a pre-trained text-to-image
diffusion model. An image pathway injects dense, multi-scale features to provide fine-grained spatial guidance, while a text-based pathway encodes per-object shadow bounding boxes as learned positional
tokens and fuses them via cross-
attention. An
attention-
alignment loss further grounds these
tokens to their corresponding shadow regions. To support this task, we augment the DESOBAv2
dataset by constructing composite scenes with multiple inserted objects and automatically derive
prompts combining object category and shadow positioning information. Experimental results demonstrate that our method achieves state-of-the-art performance in both single and multi-object shadow generation settings.
Read the full abstract ▾
Computer Vision
Gated Differential Linear Attention: A Linear-Time Decoder for High-Fidelity Medical Segmentation
Hongbo Zheng, Afshin Bozorgpour, Dorit Merhof et al. (+1 more)
Medical image
segmentation requires models that preserve fine anatomical boundaries while remaining practical for clinical deployment.
Transformers capture long-range dependencies but incur quadratic
attention cost, whereas CNNs are efficient but less effective at global
reasoning. Linear
attention offers \(\mathcal{O}(N)\) scaling, but often produces diffuse feature aggregation that weakens boundary-sensitive prediction. We introduce a gated differential linear-
attention mixer for medical image
segmentation. Its global path, Gated Differential Linear
Attention (GDLA), performs differential subtraction between two kernelized
attention branches over complementary query/key subspaces to suppress redundant responses, and employs a data-dependent gate for
token refinement. A parallel local
token-mixing branch with depthwise convolution strengthens neighborhood interactions for better refinement, and the two branches are fused while preserving \(\mathcal{O}(N)\) complexity. When instantiated in a pretrained Pyramid
Vision Transformer (PVT)-based encoder–decoder model, \name achieves state-of-the-art results on the evaluated 2D medical
segmentation benchmarks spanning CT, MRI, ultrasound, and dermoscopy, with a favorable accuracy–efficiency trade-off over closely related baselines. The code is publicly available at \href{https://github.com/xmindflow/gdla}{https://github.com/xmindflow/gdla}.
Read the full abstract ▾
All authors Hongbo Zheng, Afshin Bozorgpour, Dorit Merhof, Minjia Zhang
Computer Vision
Cross-view geo-localization, Image retrieval, Multiscale geometric modeling, Frequency domain enhancement
Hongying Zhang and ShuaiShuai Ma
Cross-view geo-localization (CVGL) aims to establish spatial correspondences between images captured from significantly different viewpoints and constitutes a fundamental technique for visual localization in GNSS-denied environments. Nevertheless, CVGL remains challenging due to severe
geometric asymmetry, texture inconsistency across imaging domains, and the progressive degradation of discriminative local information. Existing methods predominantly rely on spatial domain feature
alignment, which is inherently sensitive to large scale viewpoint variations and local disturbances. To alleviate these limitations, this paper proposes the Spatial and Frequency Domain Enhancement Network (SFDE), which leverages complementary representations from spatial and frequency domains. SFDE adopts a three branch parallel architecture to model global semantic context, local
geometric structure, and statistical stability in the frequency domain, respectively, thereby characterizing consistency across domains from the perspectives of scene
topology, multiscale structural patterns, and frequency invariance. The resulting complementary features are jointly optimized in a unified
embedding space via progressive enhancement and coupled constraints, enabling the learning of cross-view representations with consistency across multiple granularities. Comprehensive experiments show that SFDE achieves competitive performance and in many cases even surpasses state-of-the-art methods, while maintaining a lightweight and computationally efficient design. {Our code is available at https://github.com/Mashuaishuai669/SFDE
Read the full abstract ▾
Computer Vision
Tensor Decomposition Structure Search Framework from an Interaction Perspective
Ting-Wei Zhou, Xi-Le Zhao, Sheng Liu et al. (+3 more)
Recently, tensor decompositions have attracted increasing
attention. Fundamentally, different interactions among factors induce distinct tensor decomposition structures (i.e., tensor decomposition). Identifying an appropriate interaction-induced tensor decomposition structure for given data is a fundamental yet challenging problem in tensor modeling, and remains largely under-explored. Existing tensor decomposition structure search methods are typically restricted to a predefined interaction family, such as tensor contraction. To address this problem, we suggest a tensor decomposition structure search framework (I-TSS) from an interaction perspective that can identify either a single structure beyond a predefined interaction family or a mixture of structures involving heterogeneous interaction families. Specifically, we first systematically review existing tensor decomposition structures from an interaction perspective and construct a heterogeneous interaction-induced candidate structure set. Based on it, we propose a unified energy-based rank estimation scheme for heterogeneous interaction-induced candidate structure set. We then introduce the top-$k$ gating mechanism with learnable gating scores to dynamically select a suitable single-candidate structure or combine suitable multi-candidate structures, thereby identifying the tensor decomposition structure in a data-adaptive manner. Theoretically, we derive an approximation error bound for I-TSS, thereby establishing its approximation capability. Extensive experiments on synthetic and real-world
datasets demonstrate that I-TSS consistently outperforms state-of-the-art tensor decomposition methods.
Read the full abstract ▾
All authors Ting-Wei Zhou, Xi-Le Zhao, Sheng Liu, Wei-Hao Wu, Yu-Bang Zheng, Deyu Meng
Computer Vision
From "What" to "How": Constrained Reasoning for Autoregressive Image Generation
Ruxue Yan, Xubo Liu, Wenya Guo et al. (+3 more)
Autoregressive image generation has seen recent improvements with the introduction of chain-of-thought and reinforcement learning. However, current methods merely specify "What" details to depict by rewriting the input
prompt, yet fundamentally fail to reason about "How" to structure the overall image. This inherent limitation gives rise to persistent issues, such as spatial ambiguity directly causing unrealistic object overlaps. To bridge this gap, we propose CoR-Painter, a novel framework that pioneers a "How-to-What" paradigm by introducing Constrained
Reasoning to guide the autoregressive generation. Specifically, it first deduces "How to draw" by deriving a set of visual constraints from the input
prompt, which explicitly govern spatial relationships, key attributes, and compositional rules. These constraints steer the subsequent generation of a detailed description "What to draw", providing a structurally sound and coherent basis for accurate visual synthesis. Additionally, we introduce a Dual-Objective GRPO strategy that specifically optimizes the textual constrained
reasoning and visual projection processes to ensure the coherence and quality of the entire generation pipeline. Extensive experiments on T2I-CompBench, GenEval, and WISE demonstrate that our method achieves state-of-the-art performance, with significant improvements in spatial metrics (e.g., +5.41% on T2I-CompBench).
Read the full abstract ▾
All authors Ruxue Yan, Xubo Liu, Wenya Guo, Zhengkun Zhang, Ying Zhang, Xiaojie Yuan
Computer Vision
MiM-DiT: MoE in MoE with Diffusion Transformers for All-in-One Image Restoration
Lingshun Kong, Jiawei Zhang, Zhengpeng Duan et al. (+6 more)
All-in-one image restoration is challenging because different degradation types, such as haze, blur, noise, and low-light, impose diverse requirements on restoration strategies, making it difficult for a single model to handle them effectively. In this paper, we propose a unified image restoration framework that integrates a dual-level Mixture-of-Experts (MoE) architecture with a pretrained
diffusion model. The framework operates at two levels: the Inter-MoE layer adaptively combines expert groups to handle major degradation types, while the Intra-MoE layer further selects specialized sub-experts to address fine-grained variations within each type. This design enables the model to achieve coarse-grained adaptation across diverse degradation categories while performing fine-grained modulation for specific intra-class variations, ensuring both high specialization in handling complex, real-world corruptions. Extensive experiments demonstrate that the proposed method performs favorably against the state-of-the-art approaches on multiple image restoration task.
Read the full abstract ▾
All authors Lingshun Kong, Jiawei Zhang, Zhengpeng Duan, Xiaohe Wu, Yueqi Yang, Xiaotao Wang, Dongqing Zou, Lei Lei, Jinshan Pan
Computer Vision
Intelligent Pathological Diagnosis of Gestational Trophoblastic Diseases via Visual-Language Deep Learning Model
Yuhang Liu, Yueyang Cang, Wenge Que et al. (+12 more)
The pathological diagnosis of gestational trophoblastic disease(GTD) takes a long time, relies heavily on the experience of pathologists, and the consistency of initial diagnosis is low, which seriously threatens maternal health and reproductive outcomes. We developed an expert model for GTD pathological diagnosis, named GTDoctor. GTDoctor can perform pixel-based lesion
segmentation on pathological slides, and output diagnostic conclusions and personalized pathological analysis results. We developed a software system, GTDiagnosis, based on this technology and conducted clinical trials. The retrospective results demonstrated that GTDiagnosis achieved a mean precision of over 0.91 for lesion detection in pathological slides (n=679 slides). In prospective studies, pathologists using GTDiagnosis attained a Positive Predictive Value of 95.59% (n=68 patients). The tool reduced average diagnostic time from 56 to 16 seconds per case (n=285 patients). GTDoctor and GTDiagnosis offer a novel solution for GTD pathological diagnosis, enhancing diagnostic performance and efficiency while maintaining clinical interpretability.
Read the full abstract ▾
All authors Yuhang Liu, Yueyang Cang, Wenge Que, Xinru Bai, Xingtong Wang, Kuisheng Chen, Jingya Li, Xiaoteng Zhang, Xinmin Li, Lixia Zhang, Pingge Hu, Qiaoting Xie, Peiyu Xu, Xianxu Zeng, Li Shi
Computer Vision
ShareVerse: Multi-Agent Consistent Video Generation for Shared World Modeling
Jiayi Zhu, Jianing Zhang, Yiying Yang et al. (+2 more)
This paper presents ShareVerse, a
video generation framework enabling multi-
agent shared world modeling, addressing the gap in existing works that lack support for unified shared world construction with multi-
agent interaction. ShareVerse leverages the generation capability of large
video models and integrates three key innovations: 1) A
dataset for large-scale multi-
agent interactive world modeling is built on the CARLA simulation platform, featuring diverse scenes, weather conditions, and interactive trajectories with paired multi-view
videos (front/ rear/ left/ right views per
agent) and camera data. 2) We propose a spatial concatenation strategy for four-view
videos of independent
agents to model a broader environment and to ensure internal multi-view
geometric consistency. 3) We integrate cross-
agent attention blocks into the pretrained
video model, which enable interactive transmission of spatial-temporal information across
agents, guaranteeing shared world consistency in overlapping regions and reasonable generation in non-overlapping regions. ShareVerse, which supports 49-frame large-scale
video generation, accurately perceives the position of dynamic
agents and achieves consistent shared world modeling.
Read the full abstract ▾
All authors Jiayi Zhu, Jianing Zhang, Yiying Yang, Wei Cheng, Xiaoyun Yuan
Computer Vision
FiDeSR: High-Fidelity and Detail-Preserving One-Step Diffusion Super-Resolution
Aro Kim, Myeongjin Jang, Chaewon Moon et al. (+3 more)
Diffusion-based approaches have recently driven remarkable progress in real-world image super-resolution (SR). However, existing methods still struggle to simultaneously preserve fine details and ensure high-fidelity reconstruction, often resulting in suboptimal visual quality. In this paper, we propose FiDeSR, a high-fidelity and detail-preserving one-step
diffusion super-resolution framework. During training, we introduce a detail-aware weighting strategy that adaptively emphasizes regions where the model exhibits higher prediction errors. During inference, low- and high-frequency adaptive enhancers further refine the reconstruction without requiring model retraining, enabling flexible enhancement control. To further improve the reconstruction accuracy, FiDeSR incorporates a residual-in-residual noise refinement, which corrects prediction errors in the
diffusion noise and enhances fine detail recovery. FiDeSR achieves superior real-world SR performance compared to existing
diffusion-based methods, producing outputs with both high perceptual quality and faithful content restoration. The source code will be released at: https://github.com/Ar0Kim/FiDeSR.
Read the full abstract ▾
All authors Aro Kim, Myeongjin Jang, Chaewon Moon, Youngjin Shin, Jinwoo Jeong, Sang-hyo Park
Computer Vision
ReCo-Diff: Residual-Conditioned Deterministic Sampling for Cold Diffusion in Sparse-View CT
Yong Eun Choi, Hyoung Suk Park, Kiwan Jeon et al. (+2 more)
Cold and generalized
diffusion models have recently shown strong potential for sparse-view CT reconstruction by explicitly modeling deterministic degradation processes. However, existing sampling strategies often rely on ad hoc sampling controls or fixed schedules, which remain sensitive to error accumulation and sampling instability. We propose ReCo-Diff, a residual-conditioned
diffusion framework that leverages observation residuals through residual-conditioned self-guided sampling. At each sampling step, ReCo-Diff first produces a null (unconditioned) baseline reconstruction and then conditions subsequent predictions on the observation residual between the predicted image and the measured sparse-view input. This residual-driven guidance provides continuous, measurement-aware correction while preserving a deterministic sampling schedule, without requiring heuristic interventions. Experimental results demonstrate that ReCo-Diff consistently outperforms existing cold
diffusion sampling baselines, achieving higher reconstruction accuracy, improved stability, and enhanced robustness under severe sparsity.
Read the full abstract ▾
All authors Yong Eun Choi, Hyoung Suk Park, Kiwan Jeon, Hyun-Cheol Park, Sung Ho Kang
Computer Vision
VisionCreator: A Native Visual-Generation Agentic Model with Understanding, Thinking, Planning and Creation
Jinxiang Lai, Zexin Lu, Jiajun He et al. (+11 more)
Visual content creation tasks demand a nuanced understanding of design conventions and creative workflows-capabilities challenging for general models, while workflow-based
agents lack specialized knowledge for autonomous creative planning. To overcome these challenges, we propose VisionCreator, a native visual-generation agentic model that unifies Understanding, Thinking, Planning, and Creation (UTPC) capabilities within an end-to-end learnable framework. Our work introduces four key contributions: (i) VisGenData-4k and its construction methodology using metacognition-based VisionAgent to generate high-quality creation trajectories with explicit UTPC structures; (ii) The VisionCreator agentic model, optimized through Progressive Specialization Training (PST) and Virtual Reinforcement Learning (VRL) within a high-fidelity simulated environment, enabling stable and efficient acquisition of UTPC capabilities for complex creation tasks; (iii) VisGenBench, a comprehensive
benchmark featuring 1.2k test samples across diverse scenarios for standardized evaluation of multi-step visual creation capabilities; (iv) Remarkably, our VisionCreator-8B/32B models demonstrate superior performance over larger closed-source models across multiple evaluation dimensions. Overall, this work provides a foundation for future research in visual-generation agentic systems.
Read the full abstract ▾
All authors Jinxiang Lai, Zexin Lu, Jiajun He, Rongwei Quan, Wenzhe Zhao, Qinyu Yang, Qi Chen, Qin Lin, Chuyue Li, Tao Gao, Yuhao Shan, Shuai Shao, Song Guo, Qinglin Lu
Computer Vision
Unifying Contrastive and Generative Objectives for Visual Understanding and Text-to-Image Generation
Chao Li, Tianhong Li, Sai Vidyaranya Nuthalapati et al. (+9 more)
Unifying text-image contrastive learning and text-to-image (T2I) generation in a single end-to-end model is challenging because the two objectives demand opposing masking regimes: contrastive
alignment needs near-complete visible
tokens, while masked generative modeling needs heavy corruption. We introduce DREAM, a unified framework that resolves this conflict through Masking Warmup, a schedule that shifts the center of the masking distribution over training, so low and high masking ratios coexist at every step. This co-exposure lets a single jointly-trained encoder serve both objectives. The resulting stable optimization unlocks Semantically Aligned Decoding at inference: the text encoder, trained against visual
embeddings at all masking ratios, can score partially generated images and select the best trajectory with as little as 12.5% of the image decoded, improving both FID and throughput. DREAM outperforms its single-objective baselines, CLIP and FLUID: on ImageNet linear-probing (+1.1%), 5-shot transfer (+4.1%), ADE20K
segmentation (+1.9%), and NYU depth estimation (+6.25%) over CLIP, and on CC12M FID (+6.2%) over FLUID while maintaining CLIP Score. Together, these gains show that text-image contrastive and generative objectives, when properly unified, are synergistic rather than competing.
Read the full abstract ▾
All authors Chao Li, Tianhong Li, Sai Vidyaranya Nuthalapati, Hong-You Chen, Satya Narayan Shukla, Jianpeng Cheng, Yonghuan Yang, Jun Xiao, Xiangjun Fan, Aashu Singh, Dina Katabi, Shlok Kumar Mishra
Computer Vision · Cross-listed from cs.CL
Evaluating Cross-Modal Reasoning Ability and Problem Characteristics with Multimodal Item Response Theory
Shunki Uebayashi, Kento Masui, Kyohei Atarashi et al. (+5 more)
Multimodal Large
Language Models (MLLMs) have recently emerged as general architectures capable of
reasoning over diverse modalities.
Benchmarks for MLLMs should measure their ability for cross-modal integration. However, current
benchmarks are filled with shortcut questions, which can be solved using only a single modality, thereby yielding unreliable rankings. For example, in
vision-
language cases, we can find the correct answer without either the image or the text. These low-quality questions unnecessarily increase the size and computational requirements of
benchmarks. We introduce a multi-modal and multidimensional item response theory framework (M3IRT) that extends classical IRT by decomposing both model ability and item difficulty into image-only, text-only, and cross-modal components. M3IRT estimates cross-modal ability of MLLMs and each question's cross-modal difficulty, enabling compact, high-quality subsets that better reflect
multimodal reasoning. Across 24 VLMs on three
benchmarks, M3IRT prioritizes genuinely cross-modal questions over shortcuts and preserves ranking fidelity even when 50% of items are artificially generated low-quality questions, thereby reducing evaluation cost while improving reliability. M3IRT thus offers a practical tool for assessing cross-modal
reasoning and refining
multimodal benchmarks.
Read the full abstract ▾
All authors Shunki Uebayashi, Kento Masui, Kyohei Atarashi, Han Bao, Hisashi Kashima, Naoto Inoue, Mayu Otani, Koh Takeuchi
Computer Vision
OmniFashion: Towards Generalist Fashion Intelligence via Multi-Task Vision-Language Learning
Zhengwei Yang, Andi Long, Hao Li et al. (+3 more)
Fashion intelligence spans multiple tasks, i.e.,
retrieval, recommendation, recognition, and dialogue, yet remains hindered by fragmented supervision and incomplete fashion annotations. These limitations jointly restrict the formation of consistent visual-semantic structures, preventing recent
vision-
language models (VLMs) from serving as a generalist fashion
brain that unifies understanding and
reasoning across tasks. Therefore, we construct FashionX, a million-scale
dataset that exhaustively annotates visible fashion items within an outfit and organizes attributes from global to part-level. Built upon this foundation, we propose OmniFashion, a unified
vision-
language framework that bridges diverse fashion tasks under a unified fashion dialogue paradigm, enabling both multi-task
reasoning and interactive dialogue. Experiments on multi-subtasks and
retrieval benchmarks show that OmniFashion achieves strong task-level accuracy and cross-task generalization, highlighting its offering of a scalable path toward universal, dialogue-oriented fashion intelligence.
Read the full abstract ▾
All authors Zhengwei Yang, Andi Long, Hao Li, Zechao Hu, Kui Jiang, Zheng Wang