Computer Vision
Xiuwei Chen, Wentao Hu, Yongxin Wang et al. (+8 more)
The integration of visual evidence has significantly enhanced the capabilities of large
multimodal models. However, this integration predominantly relies on generating discrete outputs (etc., code or box coordinates) to invoke external tools, a process that introduces rigid dependencies and substantial latency. To overcome these limitations, we propose {EVA} (LatEnt Visual StAtes), a novel framework that natively generates continuous latent visual representations. These internal representations manifest as an adaptive sequence of Latent\_slot
tokens, serving as intermediate visual thoughts during the
reasoning process. These Latent\_slot
tokens are then trained end-to-end with the discrete text
tokens. This co-optimization, notably, causes extreme policy deviation in the 'transition window' following the Latent\_slot
tokens. We develop D-GSPO (Decouple-GSPO) to target this root cause by decoupling the optimization of latent and discrete components. To support SFT, we construct EVA-230K, a high-quality text-image interleaved CoT
dataset encompassing a diverse range of real-world scenes, documents, charts and OCR tasks. Extensive experiments across multiple
benchmarks confirm that EVA achieves significant performance gains while enhancing inference efficiency.
Read the full abstract ▾
All authors Xiuwei Chen, Wentao Hu, Yongxin Wang, Zisheng Chen, Likui Zhang, Kun Xiang, Jianhua Han, Hui-Ling Zhen, Jingyuan Zou, Hang Xu, Xiaodan Liang
Computer Vision
FiCA: Feed-forward instant Gaussian Codec Avatars from a Single Portrait Image
Kim Youwang, Zhengyu Yang, Liuhao Ge et al. (+8 more)
We introduce FiCA, a Feed-forward, instant Gaussian Codec Avatar generation pipeline that creates lifelike avatars from a single portrait image. Generating a photorealistic and drivable avatar from just a single image is significantly challenging due to the limited visual information available to accurately infer the 3D appearance and geometry of human heads. To address this, we develop a novel system that combines human-centric
vision foundation models with a
diffusion model. This system is designed to fully exploit partial visual observations to generate lifelike human avatars. Our proposed
diffusion model learns a generative mapping from these partial observations to complete and authentic 3D mesh reconstruction. Additionally, we introduce a feed-forward mesh refinement network that enhances the fidelity and identity preservation of the generated avatars, eliminating the need for person-specific test-time optimization. By leveraging a universal prior model that decodes a generated mesh into a set of 3D Gaussians, we generate a photorealistic 3D Gaussian avatar, capable of being driven with novel expressions in real-time. Our experiments demonstrate that the avatars generated by our feed-forward approach faithfully represent diverse identities and surpass the visual quality of avatars produced by recent competing methods.
Read the full abstract ▾
All authors Kim Youwang, Zhengyu Yang, Liuhao Ge, Yu Rong, Timur Bagautdinov, Su Zhaoen, Nir Sopher, Jovan Popović, Teng Deng, Tae-Hyun Oh, Chen Cao
Computer Vision
Geometry-Instructed Video Editing
Chirui Chang, Xiaoyang Lyu, Yi-Hua Huang et al. (+7 more)
Object-level
geometric edits, including translating, rotating, scaling, duplicating, or removing an object, are routine operations in digital content creation (DCC) workflows, yet they remain unreliable in generative
video editing. The key challenge lies in specifying the target object's 3D state change unambiguously across viewpoint and time, while consistently updating geometry-dependent secondary effects such as shadows and reflections. We introduce GIVE, a geometry-instructed
video editing framework that represents edits through a unified object-state formulation. Two
video-aligned geometry streams describe the target object before and after editing: a depth-box encoding coarse 3D placement and extent, and an orientation-box providing an appearance-agnostic orientation cue. Together, these streams provide a compact pre/post
geometric specification for object-state transitions. To provide paired supervision for learning these edits, we build a scalable graphics-engine pipeline that executes object-level edit programs and renders controlled before/after pairs, isolating the intended
geometric edit while keeping secondary effects consistent with the transformation. Experimental results demonstrate that GIVE produces faithful
geometric edits with temporal coherence and consistent secondary effects across operators in a unified framework, and shows promising transfer to in-the-wild
videos. Project page: https://geometry-instructed-
video-editing.github.io/give/
Read the full abstract ▾
All authors Chirui Chang, Xiaoyang Lyu, Yi-Hua Huang, Haoru Tan, Shizhen Zhao, Yikang Ding, Jianmin Bao, Xin Tao, Pengfei Wan, Xiaojuan Qi
Computer Vision
MorVess: Morphology-Aware Pulmonary Vessel Segmentation Network
Fuyou Mao, Yifei Chen, Beining Wu et al. (+9 more)
Accurate pulmonary vessel
segmentation remains challenging due to the sparse, tortuous, and multi-scale nature of vascular structures, where small branches are easily lost and
topology integrity is difficult to preserve under voxel-wise supervision. Existing deep
segmentation models primarily optimize binary masks, lacking explicit
geometric constraints, thus struggling to recover continuous tubular morphology and fine vascular connectivity. In this study, we introduce MorVess, a morphology-aware
segmentation framework that integrates differentiable
geometric priors with large-scale foundation model adaptation to achieve fine-grained vascular parsing. MorVess jointly predicts vessel masks, distance maps, and thickness maps, providing explicit supervision for vascular boundaries, centerline consistency, and smooth diameter transitions. A lightweight 2.5D adapter bridges 3D spatial context and 2D SAM representations, while a global-local fusion block aggregates multi-level semantics and
geometric cues for high-fidelity
topology reconstruction. Across two challenging pulmonary CT
benchmarks, MorVess delivers superior Dice, clDice, and HD95 scores, substantially improving small-vessel recovery and global connectivity. These results demonstrate that
embedding geometric intelligence into pretrained
vision models offers a principled and scalable pathway toward precise vessel analysis and clinically reliable structural quantification. Our source code is available at https://github.com/MaoFuyou/MorVess.
Read the full abstract ▾
All authors Fuyou Mao, Yifei Chen, Beining Wu, Lixin Lin, Jinnan Dai, Zhiling Li, Yilei Chen, Yaqi Wang, Hao Zhang, Yan Tang, Huiyu Zhou, Feiwei Qin
Computer Vision
Inclusive Interactive Collisions for Multi-View Consistent Compositional 3D Generation
Chang Liu, Mingwen Shao, Xiang Lv et al. (+5 more)
Recent breakthroughs in 3D generation have advanced notably with the development of text-to-image
diffusion model. However, existing methods remain two practical challenges: (1) They primarily generate single 3D object, but struggle to generate multi-object compositional 3D assets due to the lack of the modeling for Gaussian primitives in reasonable interactions. (2) They often suffer from cross-view inconsistency during 3D optimization, as Score
Distillation Sampling inherently performs on each single view, inevitably resulting in cross-view
hallucinations. To solve above issues, we propose I2C-3D, a novel optimization-based method to generate multi-view consistent compositional 3D assets with reasonable interactions. Specifically, we propose an Inclusive Interactive Collisions strategy to guide Gaussian primitives appearing in reasonable interaction regions naturally, thereby ensuring objects in the compositional scene interact in a physically plausible and visually coherent way. Additionally, to enhance multi-view consistency, Multi-View Adaptive Score
Distillation Sampling is devised to distill multi-view consistency prior and layout prior from pre-trained
diffusion model by modulating
attention map of instance
token and spatial
token across viewpoints. Benefiting from above elaborate designs, I2C-3D not only generates high-fidelity multi-view consistent compositional 3D assets but also supports 3D editing flexibly, facilitating complex scene generation. Extensive experiments demonstrate our I2C-3D outperforms existing methods in generation quality and multi-view consistency.
Read the full abstract ▾
All authors Chang Liu, Mingwen Shao, Xiang Lv, Xinyuan Chen, Lingzhuang Meng, Qiao Zhang, Zhengyi Gong, Jinghao Hu
Computer Vision
Co-occurring Associated REtained concepts in Diffusion Unlearning
Miso Kim, Georu Lee, Yunji Kim et al. (+3 more)
Unlearning has emerged as a key technique to mitigate harmful content generation in
diffusion models. However, existing methods often remove not only the target concept, but also benign co-occurring concepts. As illustrated in Fig.1, unlearning nudity can unintentionally suppress the concept of person, preventing a model from generating images with person. We define these undesirably suppressed co-occurring concepts that must be preserved CARE (Co-occurring Associated REtained concepts). Then, we introduce the CARE score, a general metric that directly quantifies their preservation across unlearning tasks. With this foundation, we propose ReCARE (Robust erasure for CARE), a framework that explicitly safeguards CARE while erasing only the target concept. ReCARE automatically constructs the CARE-set, a curated vocabulary of benign co-occurring
tokens extracted from target images, and leverages this vocabulary during training for stable unlearning. Extensive experiments across various target concepts (Nudity, Van Gogh style, and Tench object) demonstrate that ReCARE achieves overall state-of-the-art performance in balancing robust concept erasure, overall utility, and CARE preservation.
Read the full abstract ▾
All authors Miso Kim, Georu Lee, Yunji Kim, Hoki Kim, Jinseong Park, Woojin Lee
Computer Vision
Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Sampling
Kun Zhang, Chenxin Fang, Tao Chen et al. (+4 more)
Long
video understanding remains a daunting challenge for
Multimodal Large
Language Models (MLLMs) due to the excessive computation and
memory footprint. Thus, keyframe selection is often adopted to mitigate this shortcoming, which however still suffers from low flexibility and high noise due to its hard sampling principle. In this paper, we define
video frame selection as a problem of Quasi-Gaussian Sampling, and propose an adaptive and training-free approach termed AdaQ. Inspired by the 3-$\sigma$ rule of Gaussian distribution, the objective of AdaQ is to achieve the optimal 3-$\sigma$ interval for different examples, i.e., a smaller 3-$\sigma$ interval for the local query and a larger one for the global query, thereby facilitating robust and adaptive frame sampling. To validate AdaQ, we apply it to four MLLMs with three
embedding models. The extensive experimental results not only show its obvious performance gains over the default MLLMs and the SOTA keyframe selection methods, e.g., helping Qwen3-VL-8B outperform GPT4o by 15.8% on average by using only 64 frames, but also confirm its superior robustness and high efficiency for long-
video understanding, e.g., only 1 hyper-parameter needs to be set.
Read the full abstract ▾
All authors Kun Zhang, Chenxin Fang, Tao Chen, Baiyang Song, Yunhang Shen, Yiyi Zhou, Rongrong Ji
Computer Vision
Deep Learning Approaches for 3D Medical Scene Completion: From Geometric Modeling to Generative Paradigms
Afifa Khaled, Said Jadid Abdulkadir and Majdy Mohamed Eltayeb Eltahir
Three-dimensional scene completion has evolved as a major problem in computer
vision and robotics, and its applications are diverse, including autonomous
navigation and augmented reality. In this study, a systematic review has been conducted to compile the research contributions made in the last ten years, i.e., 2016 to 2026, which has revolutionized the field from the voxel semantic completion paradigm represented by SSCNet to the latest paradigm that combines generative
diffusion priors with real-time rendering using a Gaussian splatting technique. The
evolution in representation paradigms, such as voxel grids, point learning, implicit neural fields,
transformer networks,
diffusion networks, and the latest paradigm based on rendering-aware 3D Gaussian primitives, has been discussed in this study. A comprehensive analysis has been carried out on the contributions made in the last ten years, and a taxonomy has been developed to provide a clear idea about the contributions made in the field. The study has also discussed the research contributions made in the field, along with the challenges that still need to be addressed. Finally, the study has presented a research agenda that will provide a clear idea about the directions that can be followed in the development of the next-generation system
Read the full abstract ▾
Computer Vision
Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring
Dominik Lindner, Johann Schmidt, Tom Siegl et al. (+2 more)
Pretrained
vision models often misclassify inputs that are rotated, scaled, or sheared, even though these affine transformations leave the object class unchanged. Robustness is usually restored either by building equivariance into the architecture or by retraining with augmentation, both of which require changing or retraining the model. Test-time canonicalization instead leaves the classifier untouched. It undoes the transformation of each input, mapping it to a canonical form near the training distribution before classification. Existing canonicalizers, however, rely on a narrow set of logit-based energy scores and bespoke search procedures, leaving the design space of scoring functions and optimizers unexplored. We reframe canonicalization as out-of-distribution (OOD) detection, which lets any OOD score serve as the energy minimized over transformations. Across
benchmarks ranging from handwritten characters and sketches to natural images and 3D point clouds, we systematically evaluate around twenty OOD scores and nine search
algorithms, finding that distance-based scores paired with random search and local refinement perform best overall. Because canonicalizing an already-aligned input can hurt accuracy, we add a gated mechanism that transforms an input only when its OOD score indicates this is needed, preserving most in-distribution accuracy while retaining the robustness gains on transformed inputs. Code is available at github.com/johschm/its.
Read the full abstract ▾
All authors Dominik Lindner, Johann Schmidt, Tom Siegl, Martin Becker, Sebastian Stober
Computer Vision
Tri-Efficient Transfer Learning for Point Cloud Videos
Yiding Sun, Dongxu Zhang, Jihua Zhu et al. (+6 more)
While point cloud foundation models have significantly advanced point cloud
video understanding, existing parameter-efficient fine-tuning (PEFT) methods still suffer from two critical limitations: prohibitive annotation costs for large-scale point cloud
datasets and severe
memory bottlenecks. In this paper, we aim to mine richer supervision signals from existing data rather than blindly scaling
datasets. A further key principle is that the
memory footprint of fine-tuning must be drastically reduced compared to full fine-tuning, which remains elusive for current PEFT techniques. Driven by these challenges, we identify three core desiderata: data-, parameter-, and
memory efficiency, and present PoinTriE, a unified framework that excels along all three dimensions. For pre-training, pseudo-motion trajectories are synthesized via rigid transformations, paired with text corpora and 2D projections derived from raw point clouds. We then propose a
Geometric-Motion Duality Network optimized via
multimodal contrastive learning, rigid rotation prediction, and motion distribution divergence to produce dense self-supervision. During fine-tuning, we freeze the pretrained backbone and only update a lightweight Spatio-temporal Side Network built with LoRA units. Equipped with a gradient flow masking strategy, PoinTriE simultaneously reduces
memory consumption and parameter overhead. Extensive experiments confirm that PoinTriE establishes new state-of-the-art results on action recognition and semantic
segmentation tasks.
Read the full abstract ▾
All authors Yiding Sun, Dongxu Zhang, Jihua Zhu, Haozhe Cheng, Zhengqiao Li, Pengcheng Li, Chaowei Fang, Yonghao Dong, Lin Chen
Computer Vision · Cross-listed from eess.IV
A Dual Edge Spatial Jacobian Image Graph for Interpretable Diabetic Retinopathy Grading
Inam Ullah, Imran Razzak and Shoaib Jameel
Automated diabetic retinopathy (DR) grading from colour fundus photographs can achieve strong predictive performance, but clinical interpretation requires more than an image-level label. It requires understanding how lesion evidence is distributed around retinal vessels and how this evidence relates to quantitative vascular biomarkers. We present a dual-edge spatial-Jacobian image graph for interpretable DR grading. Each fundus image is represented as a graph node with four aligned evidence streams: AutoMorph vessel information ($X_1$), DR-XAI-style lesion evidence maps ($X_2$), a 128-dimensional lesion-based contrastive image
embedding ($X_3$), and AutoMorph morphometric biomarkers ($X_4$). The spatial edge branch ($X_{12}$) encodes vessel-lesion geometry, while the Jacobian branch ($X_{34}$) models
embedding-biomarker sensitivity. Lightweight two-
token attention fuses both edge families into a final image graph. On 2,910 matched non-augmented APTOS images, the full graph achieves 0.8076 accuracy, 0.8312 quadratic weighted kappa, 0.5915 macro-F1, and 0.9330 adjacent-grade accuracy; referable DR reaches 0.9055 accuracy and 0.9711 AUROC. The framework is positioned as an explainable representation-learning tool for lesion-biomarker hypothesis generation, rather than as a deployment-ready clinical classifier. The code is available at https://github.com/Inamullah-Colab/dual-edge-dr-graph-xai.
Read the full abstract ▾
Computer Vision
Bin Chen, Yuxiang Cai, Yadan Luo et al. (+3 more)
Reducing visual
token redundancy is critical for accelerating
Multimodal Large
Language Models (MLLMs) without degrading cross-modal
reasoning performance. Existing
token pruning methods typically rely on single-layer signals, such as
attention scores or
token similarities, which overlook the cross-layer transformation of visual representations and may exhibit positional bias in
multimodal token sequences. To address this limitation, we propose a training-free
token pruning framework based on Cross-Layer Spectral
Evolution (CLSE). Instead of measuring
token importance from single-layer feature magnitudes, CLSE quantifies how
token representations evolve across
Transformer layers in the frequency domain. This
evolution reflects the transition from high-frequency structural details to low-frequency semantic abstractions. We observe that
tokens with stronger spectral redistribution across layers are more likely to be semantically active and should therefore be preserved. By modeling cross-layer
token dynamics, CLSE provides a stable importance criterion that mitigates positional bias. Extensive experiments on both image and
video benchmarks demonstrate that CLSE achieves a superior trade-off between efficiency and accuracy under aggressive
token reduction. Across multiple MLLMs, CLSE reduces FLOPs, KV cache
memory, and latency while maintaining competitive or improved performance.
Read the full abstract ▾
All authors Bin Chen, Yuxiang Cai, Yadan Luo, Yi Zhang, Jianwei Yin, Zhi Chen
Computer Vision
Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement
Xiangqian Zhao, Xinyang Jiang, Zhipeng Xu et al. (+5 more)
Foundation models trained on biased
datasets often rely on spurious correlations between target labels and non-causal attributes, resulting in poor generalization on minority groups. Bias mitigation remains challenging due to two fundamental issues. First, when group labels are unavailable, existing group-unsupervised methods typically infer spurious attributes implicitly from model behavior, making it difficult to identify spurious factors that are semantically aligned with real-world biases. Second, even with pseudo spurious supervision, most existing debiasing methods follow a single-branch design that operates within a single shared feature space, where target and spurious attributes are intrinsically entangled. To address the first challenge, we introduce Confidence-guided Bias Concept Mining (CBCM), which leverages
diffusion-disentangled, semantically grounded concept representations to identify reliable spurious attributes without attribute annotations. To address the second challenge, we propose Dual-branch Cross-projection Debiasing (DCD), a
prompt-tuning framework that separates target and spurious representations into two branches and explicitly removes spurious information through cross null-space projection while preserving target-relevant semantics. Extensive experiments on four
benchmark datasets show that our method achieves state-of-the-art worst group accuracy among group-unsupervised approaches, while tuning at most 0.22% of the model parameters. The source code is available in the supplementary materials.
Read the full abstract ▾
All authors Xiangqian Zhao, Xinyang Jiang, Zhipeng Xu, Lingfeng He, Zilong Wang, Dongsheng Li, De Cheng, Nannan Wang
Computer Vision
Accelerating Multimodal Large Language Models with Prior-Corrected Token Reduction
Zengjie Chen, Yuxiang Cai, Jingcai Guo et al. (+3 more)
Visual
token reduction has emerged as an effective strategy for accelerating
Multimodal Large
Language Models (MLLMs). Many existing methods prune
tokens by ranking text-visual
attention scores. However, we show that
attention is often dominated by a model-induced prior: even without textual instruction, MLLMs tend to focus on certain task-agnostic regions. Consequently, the
attention scores of instruction-conditioned
tokens are suppressed, increasing the risk that these
tokens are discarded during
pruning. To address this issue, we propose Prior-Corrected
Token Reduction (PriorTR), a training-free
token reduction method that explicitly separates task-conditioned
attention from the model-induced prior. PriorTR estimates the
attention map of the prior, and contrasts it with the task-conditioned
attention distribution to measure the additional usable information contributed by each visual
token. Importantly, PriorTR computes both the model-induced prior and the task-conditioned posterior within a single forward pass by introducing a null
token that serves as an instruction-agnostic probe in the
attention block. This design avoids duplicated propagation. Extensive experiments across multiple
multimodal benchmarks and MLLMs demonstrate that PriorTR consistently improves the trade-off between accuracy and efficiency over strong training-free baselines, particularly under aggressive
token budgets.
Read the full abstract ▾
All authors Zengjie Chen, Yuxiang Cai, Jingcai Guo, Taotao Cai, Jianwei Yin, Zhi Chen
Computer Vision
Differential Unfolding: Efficient Reconstruction for Video Snapshot Compressive Imaging
Yin-Ping Zhao, Muyuan Zhang, Jiaxi Li et al. (+2 more)
Although Deep Unfolding Networks (DUNs) are predominant in
Video Snapshot Compressive Imaging (SCI), they are still limited by a uniform design philosophy. The existing approaches stack high-complexity priors with the same structure repeatedly, neglecting the fact that the optimization paths gradually converge to a nearly static state. As a consequence, representation stagnation occurs, and much computational cost is wasted on small feature updates. To solve this inefficiency, we propose Differential Unfolding (DU), a heterogeneous framework that substitutes repeated work with a periodic schedule of heavy and lightweight stages. The core of DU is the Differential Evolutionary Framework (DEF), which divides the unfolding procedure into two complementary roles: structural anchoring and differential
evolution. In this method, high-parameter general stages are used sparsely to form high-fidelity feature foundations. On the other hand, lightweight differential stages use a Differential Representation Prior (DRP) to pass on and refine these foundational features by a differential mechanism. By integrating Differential Representation
Attention (DRA) for evolving
attention maps and a Differential Modulated FFN (DM-FFN) for feature rectification, DRP effectively models cross-stage variations with minimal overhead. By concentrating high-capacity computation on the sparsely-scheduled general stages and delegating incremental refinement to the lightweight differential stages, DU achieves a superior trade-off between accuracy and efficiency. Extensive experiments show that DU-9stg achieves state-of-the-art reconstruction quality, while DU-5stg surpasses all compared methods with the fewest parameters and FLOPs; compared with the unfolding method DADUN, DU-9stg needs about half the parameters and a quarter of the FLOPs. Code: https://github.com/Muyuan-Zhang/DU
Read the full abstract ▾
All authors Yin-Ping Zhao, Muyuan Zhang, Jiaxi Li, Jiancheng Zhang, Haijin Zeng
Computer Vision
Autonomous Video Generation with Counterfactual Controllability for Self-Evolving World Models
Xin Wang, Wenxuan Liu, Tongtong Feng et al. (+1 more)
Large-scale
video generation models are increasingly described as world models because they can learn rich spatiotemporal regularities from visual data. However, we argue that an ideal world model should benefit in a self-evolving generative character. Traditional visually plausible predictions alone are not enough to establish whether an imagined future is physically actionable for a particular
embodied agent, failing to provide informative feedback from environments for self-evolving improvement. To realize self-evolving world models, this article proposes the concept of autonomous
video generation, which is evaluated through counterfactual controllability, i.e., the ability to i) generate intervention-conditioned futures, ii) bind these future frames to embodiment constraints, iii) verify them under distribution shifts, and iv) distil surviving branches into compact variables for decision-making. We formalize a four-stage closed-loop optimization of Generation, Binding, Verification and
Distillation, together with four corresponding evaluation metrics: novelty, consistency, out-of-distribution (OOD) and efficiency. We further discuss two examples, i.e., drones and manipulators, as early
embodied testbeds where wind, sensing limits, actuation delay, contact dynamics and recovery constraints can be systematically perturbed and verified. The central claim is that the framework of autonomous
video generation for self-evolving world models should not be judged by
video fidelity alone, but by whether the generated frames improve valid action under counterfactual interventions and various embodiment constraints.
Read the full abstract ▾
All authors Xin Wang, Wenxuan Liu, Tongtong Feng, Wenwu Zhu
Computer Vision
Geometry-Aware Style Transfer in 3D Gaussian Splatting
Min Hyeok Bang, Jun Hyeong Kim, Seung-Wook Kim et al. (+1 more)
In this paper, we present a novel geometry-aware style transfer framework for 3D Gaussian splatting (3DGS) that simultaneously transfers appearance attributes and
geometric structures. Unlike prior works that primarily focus on color-based stylization and often overlook structural adaptation, our method explicitly incorporates geometry adaptation through a decoupled optimization scheme that alternately updates color and geometry parameters. This strategy alleviates potential interference between color and geometry updates, leading to stable and consistent scene-level geometry transformation. The decoupled optimization is enabled by the proposed geometry-aware contrastive feature matching (GCFM). GCFM integrates RGB, depth, and edge cues into a contrastive objective and is employed in both optimization phases to effectively transfer structural characteristics from style images to Gaussian primitives. Extensive experiments show that our approach achieves superior performance in both qualitative fidelity and quantitative metrics, significantly outperforming existing 3DGS-based stylization methods. Our code is available at \href{https://github.com/oweixx/gast}{https://github.com/oweixx/gast}.
Read the full abstract ▾
All authors Min Hyeok Bang, Jun Hyeong Kim, Seung-Wook Kim, Se-Ho Lee
Computer Vision
Sat2City v2: Native 3D City Asset Generation from a Single Satellite Image
Tongyan Hua, Dongli Wu, Jinjing Zhu et al. (+5 more)
Generating explicit textured 3D city assets from a single satellite image is important for urban simulation and digital twins. Most prior methods, however, learn 3D proxies optimized for rendering street-view images or
videos over prescribed viewpoints and trajectories, rather than producing explicit 3D assets. Our previous framework, Sat2City, took a first step toward this goal with task-specific cascaded sparse-voxel latent
diffusion conditioned on satellite-derived height maps. However, it relied on synthetic data and did not condition appearance on the satellite input. To address these limitations, we present Sat2City v2, a framework for generating explicit textured 3D city assets directly from real satellite images. We first construct a real-world
dataset of 16,027 geographically matched but weakly aligned satellite-image-textured-mesh pairs, comprising 14,651 training pairs and 1,376 held-out test pairs. Building on this
dataset, Sat2City v2 uses a native structured-latent 3D prior pretrained on curated assets to anchor geospatial adaptation to its asset
manifold. We empirically demonstrate that weakly aligned satellite images and regional textured 3D meshes can support asset-level conditional generation: learned
attention lets satellite
tokens steer geometry generation without calibrated pixel-to-surface correspondence, while the generated geometry anchors satellite-guided material synthesis. Sat2City v2 ranks first among the evaluated baselines on every reported metric across geometry accuracy, generative mesh quality, and satellite-to-asset feature
alignment.
Read the full abstract ▾
All authors Tongyan Hua, Dongli Wu, Jinjing Zhu, Yinrui Ren, Zhongcheng Hong, Ying-Cong Chen, Hui Xiong, Wufan Zhao
Computer Vision
Bengal-HP_RU: A Dataset of Bengal People For Head Pose Estimation
Md. Ahanaf Arif Khan, Md. Tawhidur Rahman, Sangeeta Biswas et al. (+4 more)
Existing head pose
datasets predominantly feature subjects of Western or East Asian origin, leaving South Asian populations, particularly Bengali individuals, largely underrepresented. We introduce Bengal-HP_RU, the first publicly available head pose
dataset centred on Bengali subjects, comprising 12,894 labelled head images annotated with continuous yaw, pitch, and roll values. Images were collected from Wikimedia Commons under free licences and processed through an automated pipeline followed by manual label correction. The
dataset is partitioned by Wikimedia uploader identity to prevent data contamination, yielding 10,494 training and 2,400 test images across 296 unique uploaders. Bengal-HP_RU exhibits substantial diversity in subject age, gender, occlusion, illumination, and background, reflecting realistic in-the-wild conditions. The
dataset is publicly available at https://doi.org/10.17632/xbw9kr37jb.2.
Read the full abstract ▾
All authors Md. Ahanaf Arif Khan, Md. Tawhidur Rahman, Sangeeta Biswas, Md. Iqbal Aziz Khan, Subrata Pramanik, Sanjoy Kumar Chakravarty, Bimal Kumar Pramanik
Computer Vision
Flood Mapping from RGB imagery using a Vision Foundation Model
Vladyslav Polushko, Tilman Bucher, Ronald Rösch et al. (+3 more)
Timely, high-resolution maps of flood extent around settlements are essential for emergency response and damage assessment. We consider airborne RGB imagery for flood mapping as it can be collected rapidly at low cost. To produce flood maps, deep learning models for water
segmentation are often used. CNN based and small
vision transformer models are used. However, they need much data for adaptation to a change of scenery, i.e., another flooding event.
Vision foundation models or large
vision transformers are known to generalize across domains. Recently, foundation models for Earth observation became available. They are pretrained on satellite data, whose spatial resolution, viewing geometry, and radiometry differ from nadir RGB imagery. Thus, adaptation is required. We investigate how a satellite-pretrained Earth observation foundation model can be adapted to centimeter-scale floodwater mapping from RGB imagery. Specifically, we fine-tune a model we call Prithvi-2.0-UPN consisting of the Prithvi-EO-2.0-600M
Vision Transformer combined with a UPerNet decoder for binary water
segmentation on two RGB
datasets (BlessemFlood21, NeuenahrFlood). In a first experiment we observe that Prithvi-2.0-UPN reaches state-of-the-art results on BlessemFlood21 and NeuenahrFlood, when trained on their
datasets. In a second experiment we show that Prithvi-2.0-UPN performs better than state-of-the-art baseline models for transfer to a new flood event (trained on BlessemFlood21, tested on NeuenahrFlood) in a zero-shot setting. However, the performance indicates room for improvement. In this respect, we investigate in a third experiment how performance improves when further fine-tuning the models with small shares of NeuenahrFlood training data: Prithvi-2.0-UPN improves the fastest and reaches almost the performance level when fully trained on NeuenahrFlood, indicating transfer capabilities.
Read the full abstract ▾
All authors Vladyslav Polushko, Tilman Bucher, Ronald Rösch, Thomas März, Markus Rauhut, Andreas Weinmann
Computer Vision
An LMM for Precisely Grounding Elements in Documents
Yijian Lu, Chuangxin Zhao, Kai Sun et al. (+3 more)
Visual grounding in documents is a crucial ability for Large
Multimodal Models (LMMs) in areas such as document understanding, deep research and document error detection. However, existing approaches exhibit poor grounding precision in text-rich document images, often failing to accurately locate the critical document elements needed for reliable
reasoning. To address this gap, we introduce PreciseDoc, an LMM specifically designed for precise element grounding and can be further optimized for Document VQA tasks. Specifically, to enhance the basic localization capability, we construct challenging training data by two pipelines capable of mass-producing high-quality documents with paired metadata of fine-grained coordinates, including synthetic hand-filled documents with camera effects. The model develops more real-world functions beyond straightforward localization of single text, such as locating personal information from CVs. Furthermore, we introduce a training paradigm for visual grounded
reasoning where the grounding and
reasoning are supervised jointly with reinforcement learning to improve the contribution of the grounded evidence. A comprehensive evaluation on various
benchmarks demonstrates the advantage of the proposed data and methods in document spatial grounding and document understanding.
Read the full abstract ▾
All authors Yijian Lu, Chuangxin Zhao, Kai Sun, Lei Hou, Ji Qi, Juanzi Li
Computer Vision
A Benchmark for Hallucination Detection in VLMs for Gastrointestinal Endoscopy
Aminu Lawal, Niyoj Oli, Sachin Acharya et al. (+3 more)
Vision-
language models (VLMs) are prone to
hallucination, which remains a major barrier to their safe deployment in clinical practice. To date, most
hallucination detection methods have been evaluated on radiology
benchmarks such as MIMIC-CXR and VQA-RAD, while gastrointestinal (GI) endoscopy remains largely underexplored. In this paper, we
benchmark nine
hallucination detection methods on the Gut-VLM
dataset, a GI diagnostic Visual Question Answering (VQA)
dataset with 4,392 test VQA pairs, across five VLMs (MedGemma-4B, MedGemma-27B, LLaVA-Med-7B, LLaVA-v1.6-7B, and Lingshu-32B). The methods span three categories: black-box methods (RadFlag, SelfCheckGPT-NLI), gray-box methods (AvgProb, AvgEnt, MaxProb, MaxEnt, Semantic
Entropy, and VASE), and a white-box method (ReXTrust). Our results show that ReXTrust, a white-box method, achieves the highest AUC across all five models, outperforming the strongest alternative method on each VLM by a statistically significant margin (paired permutation test, p < 0.001 in all cases), reaching a peak AUC of 93.0 on MedGemma-4B. White-box hidden-state access provides a consistent advantage of 19.5 AUC points on average (range: 9.5–33.5), with ReXTrust maintaining strong performance even on LLaVA-v1.6-7B (AUC 79.9), where black-box methods and clustering-based gray-box methods collapse to near-chance performance. Among non-white-box methods,
token-level gray-box statistics (MaxEnt, MaxProb) are the strongest alternatives, outperforming both clustering-based gray-box methods (Semantic
Entropy, VASE) and black-box approaches on average. We further identify confident confabulation, a failure mode in which models hallucinate with high inter-sample consistency or high
token-level probability, as a systemic failure for both consistency and uncertainty-based methods.
Read the full abstract ▾
All authors Aminu Lawal, Niyoj Oli, Sachin Acharya, Prashnna Gyawali, Maria Carmen Romano, Binod Bhattarai
Computer Vision
DramaDirector: Geometry-Guided Short Drama Generation
Hengji Zhou, Sijie Liu, Jianrun Chen et al. (+3 more)
Short dramas, with their rapid shot rhythms, dialogue-driven focus shifts, and demanding cinematographic grounding, pose challenges that
prompt-level or text-only
video generation pipelines struggle to meet. We study plot-to-short-drama generation, where a global plot and local context are transformed into visually grounded multi-shot
videos. We propose DramaDirector, a geometry-grounded framework that lets the planner borrow cinematographic geometry from a gallery of real short-drama shots indexed by depth and pose. DramaDirector decouples each shot into static visual and dynamic narrative conditions, trains the planner with schema-constrained SFT and GRPO under a learned text-visual
alignment reward, and retrieves depth-pose references to guide first-frame generation and image-to-
video synthesis. We also introduce DramaBoard, a
benchmark built from 35 live-action dramas, 2.8K episodes, and 81K shots, with structured storyboards and multi-dimensional evaluation protocols. Experiments show that DramaDirector improves over representative multi-
agent and
video generation baselines on faithfulness, consistency, and controllability. Our code is released at: https://github.com/iLearn-Lab/DramaDirector
Read the full abstract ▾
All authors Hengji Zhou, Sijie Liu, Jianrun Chen, Xingchen Zou, Lianghao Xia, Liqiang Nie
Computer Vision · Cross-listed from cs.RO
NavWM: A Unified Navigation World Model for Foresight-Driven Planning
Yanghong Mei, Longteng Guo, Ming-Ming Yu et al. (+3 more)
Conventional visual
navigation policies often struggle with myopic decision-making and mode collapse in complex environments. While world models offer a promising alternative, existing paradigms typically isolate perception, generation, and control, failing to capture their shared spatio-temporal dynamics. In this paper, we propose NavWM, a unified
navigation world model that seamlessly integrates latent world
reasoning,
multimodal action prediction, and controllable visual generation. At its core, NavWM leverages latent world
tokens to distill
geometric and semantic priors, endowing the
agent with robust structural understanding. To overcome the limitations of deterministic policies, we introduce an anchor-based
multimodal trajectory
forecasting framework that generates a diverse action space. This inherent diversity explicitly empowers the generative world model to act as a robust closed-loop planner, utilizing visual foresight to evaluate and select the optimal path. Extensive experiments across diverse robotics
datasets demonstrate that NavWM significantly advances the state-of-the-art, delivering remarkable improvements in both high-fidelity future state generation and zero-shot
navigation success.
Read the full abstract ▾
All authors Yanghong Mei, Longteng Guo, Ming-Ming Yu, Guiyu Zhao, Xingjian He, Jing Liu
Computer Vision
Beyond Bayer: Task-Optimal Sensor Co-Design for Robust Autonomous-Driving Segmentation
Reeshad Khan and John Gauch
Robust perception underpins autonomous driving, and most recent progress comes from scaling the model-larger backbones, foundation models, and cooperative multi-
agent fusion. We pursue a complementary, upstream question: what should the camera itself measure? Using a differentiable RAW-to-task pipeline, we decompose which sensor degrees of freedom benefit dense prediction. Learning the spectral colour-filter-array (CFA) weights is the dominant lever, improving mIoU by +0.017 (KITTI-360) and +0.023 (ACDC) over a fixed camera. In contrast, point-spread-function (optics) co-design is net-negative (-0.020 mIoU on KITTI-360) - a consequence of the data-processing inequality, which also bounds the task information that any downstream model, however large or cooperative, can recover. Noise co-optimisation is marginal, and counter to intuition enlarging the CFA tile beyond 2x2 consistently hurts, as the filters are confined to the rank three sRGB input. Because the intervention is at the sensor, the gains are model-agnostic; we validate robustness on ACDC's fog, night, rain, and snow, and conclude with a simple recipe: learn the 2x2 CFA weights and keep an identity PSF.
Read the full abstract ▾
Computer Vision
Universal Guideline-Driven Image Clustering via a Hybrid LLM Agent
Wenliang Zhong, Rob Barton, Lucas Goncalves et al. (+7 more)
Unifying image clustering across different clustering scenarios remains challenging due to fundamental gaps among tasks. We introduce a Guideline-Driven Image Clustering
Agent, the first universal framework that bridges these gaps through textual guidelines. To incorporate complex guidelines without task-specific training, we propose Generative Concept Proxy Modeling, which generates guideline-aware
embeddings via concept proxy extraction. For scenarios requiring automatic cluster discovery, we introduce LLM Traversal based on Minimum Spanning Tree that selectively applies LLM
reasoning for complex semantic judgments. Our method generalizes across diverse clustering scenarios spanning from general to fine-grained categorization, from global to local criteria, and from balanced to long-tail distributions. Our framework consistently outperforms specialized methods across diverse clustering tasks.
Read the full abstract ▾
All authors Wenliang Zhong, Rob Barton, Lucas Goncalves, Kushal Kumar, Feng Jiang, Hehuan Ma, Yuzhi Guo, Vidit Bansal, Karim Bouyarmane, Junzhou Huang
Computer Vision
Progressive Pixel-Neighborhood Deformable Cross-Attention for Multispectral Object Detection
Tian Qiu, Jifeng Shen and Xin Zuo
Effective cross-modal feature
alignment and interaction are central challenges in multispectral object detection. Although global cross-
attention provides strong long-range modeling ability, its quadratic complexity with respect to feature size limits deployment on resource-constrained platforms. We therefore propose Progressive Pixel-Neighborhood Deformable Cross-
Attention for multispectral feature fusion, termed PNAFusion. The proposed framework is motivated by two observations: weak misalignment between visible and thermal images is usually concentrated around local neighborhoods, and semantic correspondence across modalities often follows non-linear spatial mappings that fixed receptive fields cannot model well. To address these issues, PNAFusion incorporates local spatial priors into its architectural design to concentrate feature interaction and
alignment on the most relevant neighborhoods. Specifically, a Pixel-Neighborhood Cross-
Attention (PNCA) module is introduced to avoid redundant global feature matching and suppress background noise. Meanwhile, an Adaptive Deformable
Alignment (ADA) module captures non-linear spatial correspondences through learned pixel-wise offsets. These components are further integrated through an iterative feedback mechanism to progressively refine cross-modal feature
alignment. Experiments on FLIR, M3FD, and DroneVehicle show that PNAFusion achieves 84.2, 90.5, and 85.5 mAP@0.5, respectively, under the YOLOv5 detector, and further reaches 86.8 mAP@0.5 on FLIR and 90.8 mAP@0.5 on M3FD when transferred to Co-DETR. Efficiency analysis indicates that PNAFusion reduces allocated GPU
memory by 33.0% compared with ICAFusion and reduces theoretical FLOPs from 194.8 G to 156.4 G, although the deformable sampling and iterative refinement introduce additional latency. Our code will be available at https://github.com/DanielQiuTian/PNAFusion.
Read the full abstract ▾
Computer Vision
End-to-End Radar and Communication Modulation Recognition with Neuromorphic Computing
Xiaohu Li, Chongxiao Qu, Caiyong Lin et al. (+2 more)
Although deep learning-based methods can achieve high accuracy in automatic modulation recognition (AMR) tasks, their high computational cost makes it difficult to strike a balance between accuracy and power consumption, thereby limiting their application on resource-constrained platforms. Neuromorphic architectures that perform spike-driven inference with modest energy budgets have recently been explored for
vision and timeseries tasks. Motivated by these works, we propose EMRFormer, a novel end-to-end spiking nerural network (SNN) architecture that applies spike-driven
transformer to the constraints of neuromorphic hardware for AMR. The model incorporates an adaptive spike encoder and Integer Leaky Integrate-and-Fire
neurons to mitigate the degradation of effective information and enhance SNN representational capacity. By integrating spike-separable Convolution Neural Networks (SSCNN) into Spike-Driven
Transformers (SpikeFormer), EMRFormer effectively extracts multi-scale temporal features from the raw IQ waveforms. We validate our approach across various mainstream
datasets, the experimental results show that EMRFormer achieves state-of-the-art interms of accuracy, outperforming all the baselines. Furthermore, the model maintains strong performance in low signal-to-noise(SNR) environments and reduces theoretical energy consumption by over 90%. Finally, we evaluate our model on a KA200 neuromorphic chip. The results show that our model achieves up to 5 times reduction in power compared to running on a 3090 GPU or an Orin NX. This work demonstrates a promising pathway for AMR on resource-constrained devices.
Read the full abstract ▾
All authors Xiaohu Li, Chongxiao Qu, Caiyong Lin, Chenxiao Dou, Wei Hua
Computer Vision
Fabric Image Demoiréing Benchmark from Synthesis to Restoration
Pengchao Wei and Xiaojie Guo
Fabric moiré is a sampling-induced aliasing artifact caused by the interaction between fine textile patterns and camera sensor grids, producing structured interference that severely degrades image quality. Unlike screen-induced moiré, which stems from strictly periodic display
lattices, fabric moiré is intrinsically more challenging due to the broadband and semi-periodic nature of textile weaves. The heavy spectral overlap between intrinsic texture and aliasing components renders fabric demoiréing substantially more ill-posed. Consequently, existing models trained on screen moiré
datasets generalize poorly to these complex textile patterns. Despite its practical importance, fabric image demoiréing remains underexplored and lacks standardized
benchmarks. We present the first comprehensive
benchmark for fabric image demoiréing. To address the difficulty of acquiring pixel-aligned real-world pairs, we develop a physically motivated synthesis framework and construct a large-scale
dataset comprising 16,050 paired multi-resolution fabric images with controllable aliasing severity. Furthermore, we customize a baseline model, which establishes promising performance on the proposed
benchmark dataset with strong generalization ability. Our
benchmark provides a standardized platform for advancing research in fabric image demoiréing.
Read the full abstract ▾
Computer Vision
ObsGraph: Hierarchical Observation Representation for Embodied Reasoning and Exploration
Taekbeom Lee, Youngseok Jang, Jeonghwa Heo et al. (+2 more)
Embodied reasoning and exploration are increasingly considered crucial abilities for
robots operating in complex and unfamiliar environments. To accomplish tasks in such settings, an
agent must identify and acquire the information necessary for the task through exploration. We propose ObsGraph, an observation-centric hierarchical scene graph that unifies scene representation,
retrieval, and exploration. It retains visual evidence and organizes it into room-view-object layers: rooms provide coarse semantic anchors, views preserve contextual object covisibility, and objects store fine-grained details. On top of this representation, we perform coarse-to-fine hierarchical
retrieval under a bounded budget, and crucially use
retrieval outcomes to structure the exploration candidate space–activating room-level exploration, view refinement, or frontier exploration–thereby tightly coupling representation,
retrieval, and adaptive multi-scale exploration. Experiments across
embodied reasoning and exploration
benchmarks demonstrate improved success and efficiency, highlighting the benefits of structured scene representation and more targeted information gathering driven by identified evidence gaps.
Read the full abstract ▾
All authors Taekbeom Lee, Youngseok Jang, Jeonghwa Heo, Jeongjun Choi, H. Jin Kim