Computer Vision · Cross-listed from cs.RO
PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation
Wenlong Huang, Yu-Wei Chao, Arsalan Mousavian et al. (+4 more)
Humans anticipate, from a glance and a contemplated action of their bodies, how the 3D world will respond, a capability that is equally vital for robotic manipulation. We introduce PointWorld, a large pre-trained 3D world model that unifies state and action in a shared 3D space as 3D point flows: given one or few RGB-D images and a sequence of low-level
robot action commands, PointWorld forecasts per-pixel displacements in 3D that respond to the given actions. By representing actions as 3D point flows instead of embodiment-specific action spaces (e.g., joint positions), this formulation directly conditions on physical geometries of
robots while seamlessly integrating learning across embodiments. To train our 3D world model, we curate a large-scale
dataset spanning real and simulated robotic manipulation in open-world environments, enabled by recent advances in 3D
vision and simulated environments, totaling about 2M trajectories and 500 hours across a single-arm Franka and a bimanual
humanoid. Through rigorous, large-scale empirical studies of backbones, action representations, learning objectives, partial observability, data mixtures, domain transfers, and scaling, we distill design principles for large-scale 3D world modeling. With a real-time (0.1s) inference speed, PointWorld can be efficiently integrated in the model-predictive control (MPC) framework for manipulation. We demonstrate that a single pre-trained checkpoint enables a real-world Franka
robot to perform rigid-body pushing, deformable and articulated object manipulation, and tool use, without requiring any demonstrations or post-training and all from a single image captured in-the-wild. Project website at https://point-world.github.io/.
Read the full abstract ▾
All authors Wenlong Huang, Yu-Wei Chao, Arsalan Mousavian, Ming-Yu Liu, Dieter Fox, Kaichun Mo, Li Fei-Fei
Computer Vision
MVP: Enhancing Video Large Language Models via Self-supervised Masked Video Prediction
Xiaokun Sun, Zezhong Wu, Zewen Ding et al. (+1 more)
Reinforcement learning based post-training paradigms for
Video Large
Language Models (VideoLLMs) have achieved significant success by optimizing for visual-semantic tasks such as captioning or VideoQA. However, while these approaches effectively enhance perception abilities, they primarily target holistic content understanding, often lacking explicit supervision for intrinsic temporal coherence and inter-frame correlations. This tendency limits the models' ability to capture intricate dynamics and fine-grained visual causality. To explicitly bridge this gap, we propose a novel post-training objective: Masked
Video Prediction (MVP). By requiring the model to reconstruct a masked continuous segment from a set of challenging distractors, MVP forces the model to attend to the sequential logic and temporal context of events. To support scalable training, we introduce a scalable data synthesis pipeline capable of transforming arbitrary
video corpora into MVP training samples, and further employ Group Relative Policy Optimization (GRPO) with a fine-grained reward function to enhance the model's understanding of
video context and temporal properties. Comprehensive evaluations demonstrate that MVP enhances
video reasoning capabilities by directly reinforcing temporal
reasoning and causal understanding.
Read the full abstract ▾
All authors Xiaokun Sun, Zezhong Wu, Zewen Ding, Linli Xu
Computer Vision
I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing
Jinghan Yu, Junhao Xiao, Chenyu Zhu et al. (+9 more)
Existing text-guided image editing methods primarily rely on end-to-end pixel-level inpainting paradigm. Despite its success in simple scenarios, this paradigm still significantly struggles with compositional editing tasks that require precise local control and complex multi-object spatial
reasoning. This paradigm is severely limited by 1) the implicit coupling of planning and execution, 2) the lack of object-level control granularity, and 3) the reliance on unstructured, pixel-centric modeling. To address these limitations, we propose I2E, a novel "Decompose-then-Action" paradigm that revisits image editing as an actionable interaction process within a structured environment. I2E utilizes a Decomposer to transform unstructured images into discrete, manipulable object layers and then introduces a physics-aware
Vision-
Language-Action
Agent to parse complex instructions into a series of atomic actions via Chain-of-Thought
reasoning. Further, we also construct I2E-Bench, a
benchmark designed for multi-instance spatial
reasoning and high-precision editing. Experimental results on I2E-Bench and multiple public
benchmarks demonstrate that I2E significantly outperforms state-of-the-art methods in handling complex compositional instructions, maintaining physical plausibility, and ensuring multi-turn editing stability.
Read the full abstract ▾
All authors Jinghan Yu, Junhao Xiao, Chenyu Zhu, Jiaming Li, Jia Li, HanMing Deng, Xirui Wang, Guoli Jia, Jianjun Li, Xiang Bai, Bowen Zhou, Zhiyuan Ma
Computer Vision
HyperCOD: The First Challenging Benchmark and Baseline for Hyperspectral Camouflaged Object Detection
Shuyan Bai, Tingfa Xu, Peifu Liu et al. (+5 more)
RGB-based camouflaged object detection struggles in real-world scenarios where color and texture cues are ambiguous. While hyperspectral image offers a powerful alternative by capturing fine-grained spectral signatures, progress in hyperspectral camouflaged object detection (HCOD) has been critically hampered by the absence of a dedicated, large-scale
benchmark. To spur innovation, we introduce HyperCOD, the first challenging
benchmark for HCOD. Comprising 350 high-resolution hyperspectral images, It features complex real-world scenarios with minimal objects, intricate shapes, severe occlusions, and dynamic lighting to challenge current models. The advent of foundation models like the Segment Anything Model (SAM) presents a compelling opportunity. To adapt the Segment Anything Model (SAM) for HCOD, we propose HyperSpectral Camouflage-aware SAM (HSC-SAM). HSC-SAM ingeniously reformulates the hyperspectral image by decoupling it into a spatial map fed to SAM's image encoder and a spectral saliency map that serves as an adaptive
prompt. This translation effectively bridges the modality gap. Extensive experiments show that HSC-SAM sets a new state-of-the-art on HyperCOD and generalizes robustly to other public HSI
datasets. The HyperCOD
dataset and our HSC-SAM baseline provide a robust foundation to foster future research in this emerging area.
Read the full abstract ▾
All authors Shuyan Bai, Tingfa Xu, Peifu Liu, Yuhao Qiu, Huiyan Bai, Huan Chen, Yanyan Peng, Jianan Li
Computer Vision
RadDiff: Describing Differences in Radiology Image Sets with Natural Language
Xiaoxian Shen, Yuhui Zhang, Sahithi Ankireddy et al. (+5 more)
Understanding how two radiology image sets differ is critical for generating clinical insights and for interpreting medical AI systems. We introduce RadDiff, a
multimodal agentic system that performs radiologist-style comparative
reasoning to describe clinically meaningful differences between paired radiology studies. RadDiff builds on a proposer-ranker framework from VisDiff, and incorporates four innovations inspired by real diagnostic workflows: (1) medical knowledge injection through domain-adapted
vision-
language models; (2)
multimodal reasoning that integrates images with their clinical reports; (3) iterative hypothesis refinement across multiple
reasoning rounds; and (4) targeted visual search that localizes and zooms in on salient regions to capture subtle findings. To evaluate RadDiff, we construct RadDiffBench, a challenging
benchmark comprising 57 expert-validated radiology study pairs with ground-truth difference descriptions. On RadDiffBench, RadDiff achieves 47% accuracy, and 50% accuracy when guided by ground-truth reports, significantly outperforming the general-domain VisDiff baseline. We further demonstrate RadDiff's versatility across diverse clinical tasks, including COVID-19 phenotype comparison, racial subgroup analysis, and discovery of survival-related imaging features. Together, RadDiff and RadDiffBench provide the first method-and-
benchmark foundation for systematically uncovering meaningful differences in radiological data.
Read the full abstract ▾
All authors Xiaoxian Shen, Yuhui Zhang, Sahithi Ankireddy, Xiaohan Wang, Maya Varma, Henry Guo, Curtis Langlotz, Serena Yeung-Levy
Computer Vision
MATANet: A Multi-context Attention and Taxonomy-Aware Network for Fine-Grained Underwater Recognition of Marine Species
Donghwan Lee, Byeongjin Kim, Geunhee Kim et al. (+3 more)
Accurate fine-grained recognition of marine organisms is important for scalable biodiversity monitoring and ecological assessment using
underwater imagery. However, existing methods mainly focus on target appearance and make limited use of surrounding environmental cues and biological taxonomy. We propose the Multi-Context
Attention and Taxonomy-Aware Network (MATANet) for region-of-interest (ROI)-guided marine organism recognition. MATANet contains two complementary components. The Multi-Context Environmental
Attention Module uses the ROI representation as a query to aggregate spatial patch features from ROI-centered contextual views at multiple scales, enabling target-conditioned modeling of the surrounding environment. Level-wise auxiliary classifiers further incorporate higher taxonomic ranks during training, encouraging hierarchically consistent representations without changing the finest-label prediction space or inference procedure. On the official FathomNet 2025 Private test split, MATANet achieves a hierarchical distance of 1.570 with the base backbone and 1.423 with the large backbone, substantially outperforming the strongest
benchmark value of 2.603. On FishCLEF2015, MATANet achieves an accuracy of 0.793 and a hierarchical distance of 1.120, outperforming the strongest
benchmark values of 0.766 and 1.327, respectively. Ablation studies show that surrounding scene information provides complementary evidence beyond repeated multi-scale observations of the target and that target-conditioned aggregation outperforms direct multi-view concatenation. Additional experiments on FAIR1M v2.0 examine the applicability of the proposed design beyond
underwater imagery. In post-detection evaluation, MATANet improves fine-grained classification accuracy on matched detector-generated ROIs from 0.828 to 0.959 supporting its engineering applicability to automated marine monitoring.
Read the full abstract ▾
All authors Donghwan Lee, Byeongjin Kim, Geunhee Kim, Hyukjin Kwon, Nahyeon Maeng, Wooju Kim
Computer Vision
CSMCIR: CoT-Enhanced Symmetric Alignment with Memory Bank for Composed Image Retrieval
Zhipeng Qian, Zihan Liang, Yufei Ma et al. (+7 more)
Composed Image
Retrieval (CIR) enables users to search for target images using both a reference image and manipulation text, offering substantial advantages over single-modality
retrieval systems. However, existing CIR methods suffer from representation space fragmentation: queries and targets comprise heterogeneous modalities and are processed by distinct encoders, forcing models to bridge misaligned representation spaces only through post-hoc
alignment, which fundamentally limits
retrieval performance. This architectural asymmetry manifests as three distinct, well-separated clusters in the feature space, directly demonstrating how heterogeneous modalities create fundamentally misaligned representation spaces from initialization. In this work, we propose CSMCIR, a unified representation framework that achieves efficient query-target
alignment through three synergistic components. First, we introduce a Multi-level Chain-of-Thought (MCoT) prompting strategy that guides
Multimodal Large
Language Models to generate discriminative, semantically compatible captions for target images, establishing modal
symmetry. Building upon this, we design a symmetric dual-tower architecture where both query and target sides utilize the identical shared-parameter Q-Former for cross-modal encoding, ensuring consistent feature representations and further reducing the
alignment gap. Finally, this architectural
symmetry enables an
entropy-based, temporally dynamic
Memory Bank strategy that provides high-quality negative samples while maintaining consistency with the evolving model state. Extensive experiments on four
benchmark datasets demonstrate that our CSMCIR achieves state-of-the-art performance with superior training efficiency. Comprehensive ablation studies further validate the effectiveness of each proposed component.
Read the full abstract ▾
All authors Zhipeng Qian, Zihan Liang, Yufei Ma, Ben Chen, Huangyu Dai, Yiwei Ma, Jiayi Ji, Chenyi Lei, Han Li, Xiaoshuai Sun
Computer Vision
Towards Real-world Lens Active Alignment with Unlabeled Data via Domain Adaptation
Wenyong Li, Qi Jiang, Weijian Hu et al. (+5 more)
Active
Alignment (AA) is a key technology for the large-scale automated assembly of high-precision optical systems. Compared with labor-intensive per-model on-device calibration, a digital-twin pipeline built on optical simulation offers a substantial advantage in generating large-scale labeled data. However, complex imaging conditions induce a domain gap between simulation and real-world images, limiting the generalization of simulation-trained models. To address this, we propose augmenting a simulation baseline with minimal unlabeled real-world images captured at random misalignment positions, mitigating the gap from a domain adaptation perspective. We introduce Domain Adaptive Active
Alignment (DA3), which utilizes an autoregressive domain transformation generator and an
adversarial-based feature
alignment strategy to distill real-world domain information via self-supervised learning. This enables the extraction of domain-invariant image degradation features to facilitate robust misalignment prediction. Experiments on two lens types reveal that DA3 improves accuracy by 46% over a purely simulation pipeline. Notably, it approaches the performance achieved with precisely labeled real-world data collected on 3 lens samples, while reducing on-device data collection time by 98.7%. The results demonstrate that domain adaptation effectively endows simulation-trained models with robust real-world performance, validating the digital-twin pipeline as a practical solution to significantly enhance the efficiency of large-scale optical assembly.
Read the full abstract ▾
All authors Wenyong Li, Qi Jiang, Weijian Hu, Kailun Yang, Zhanjun Zhang, Wenjun Tian, Kaiwei Wang, Jian Bai
Computer Vision · Cross-listed from cs.CL
Visual Merit or Linguistic Crutch? A Close Look at DeepSeek-OCR
Yunhao Liang, Ruixuan Ying, Bo Li et al. (+7 more)
DeepSeek-OCR utilizes an optical 2D mapping approach to achieve high-ratio
vision-text compression, claiming to decode text
tokens exceeding ten times the input visual
tokens. While this suggests a promising solution for the LLM long-context bottleneck, we investigate a critical question: "Visual merit or linguistic crutch - which drives DeepSeek-OCR's performance?" By employing sentence-level and word-level semantic corruption, we isolate the model's intrinsic OCR capabilities from its
language priors. Results demonstrate that without linguistic support, DeepSeek-OCR's performance plummets from approximately 90% to 20%. Comparative benchmarking against 13 baseline models reveals that traditional pipeline OCR methods exhibit significantly higher robustness to such semantic perturbations than end-to-end methods. Furthermore, we find that lower visual
token counts correlate with increased reliance on priors, exacerbating
hallucination risks. Context stress testing also reveals a total model collapse around 10,000 text
tokens, suggesting that current optical compression techniques may paradoxically aggravate the long-context bottleneck. This study empirically defines DeepSeek-OCR's capability boundaries and offers essential insights for future optimizations of the
vision-text compression paradigm. We release all data, results and scripts used in this study at https://github.com/dududuck00/DeepSeekOCR.
Read the full abstract ▾
All authors Yunhao Liang, Ruixuan Ying, Bo Li, Hong Li, Kai Yan, Qingwen Li, Min Yang, Okamoto Satoshi, Zhe Cui, Shiwen Ni
Computer Vision
BREATH-VL: Vision-Language-Guided 6-DoF Bronchoscopy Localization via Semantic-Geometric Fusion
Qingyao Tian, Bingyu Yang, Huai Liao et al. (+4 more)
Vision-
language models (VLMs) have recently shown remarkable performance in
navigation and localization tasks by leveraging large-scale pretraining for semantic understanding. However, applying VLMs to 6-DoF endoscopic camera localization presents several challenges: 1) the lack of large-scale, high-quality, densely annotated, and localization-oriented
vision-
language datasets in real-world medical settings; 2) limited capability for fine-grained pose regression; and 3) high computational latency when extracting temporal features from past frames. To address these issues, we first construct BREATH
dataset, the largest in-vivo endoscopic localization
dataset to date, collected in the complex human airway. Building on this
dataset, we propose BREATH-VL, a hybrid framework that integrates semantic cues from VLMs with
geometric information from
vision-based registration methods for accurate 6-DoF pose estimation. Our motivation lies in the complementary strengths of both approaches: VLMs offer generalizable semantic understanding, while registration methods provide precise
geometric alignment. To further enhance the VLM's ability to capture temporal context, we introduce a lightweight context-learning mechanism that encodes motion history as linguistic
prompts, enabling efficient temporal
reasoning without expensive
video-level computation. Extensive experiments demonstrate that the
vision-
language module delivers robust semantic localization in challenging
surgical scenes. Building on this, our BREATH-VL outperforms state-of-the-art
vision-only localization methods in both accuracy and generalization, reducing translational error by 25.5% compared with the best-performing baseline, while achieving competitive computational latency.
Read the full abstract ▾
All authors Qingyao Tian, Bingyu Yang, Huai Liao, Xinyan Huang, Junyong Li, Dong Yi, Hongbin Liu
Computer Vision
TRec: Learning Hand-Object Interactions through 2D Point Track Motion
Dennis Holzmann and Sven Wachsmuth
We present a novel approach for hand-object action recognition that leverages 2D point tracks as an additional motion cue. While most existing methods rely on RGB appearance, human pose estimation, or their combination, our work demonstrates that tracking randomly sampled image points across
video frames can substantially improve recognition accuracy. Unlike prior approaches, we do not detect hands, objects, or interaction regions. Instead, we employ CoTracker to follow a set of randomly initialized points through each
video and use the resulting trajectories, together with the corresponding image frames, as input to a
Transformer-based recognition model. Surprisingly, our method achieves notable gains even when only the initial frame and the point tracks are provided, without incorporating the full
video sequence. Experimental results confirm that integrating 2D point tracks consistently enhances performance compared to the same model trained without motion information, highlighting their potential as a lightweight yet effective representation for hand-object action understanding.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
e5-omni: Explicit Cross-modal Alignment for Omni-modal Embeddings
Haonan Chen, Sicheng Gao, Radu Timofte et al. (+2 more)
Modern information systems often involve different types of items, e.g., a text query, an image, a
video clip, or an audio segment. This motivates omni-modal
embedding models that map heterogeneous modalities into a shared space for direct comparison. However, most recent omni-modal
embeddings still rely heavily on implicit
alignment inherited from pretrained
vision-
language model (VLM) backbones. In practice, this causes three common issues: (i) similarity logits have modality-dependent sharpness, so scores are not on a consistent scale; (ii) in-batch negatives become less effective over time because mixed-modality batches create an imbalanced hardness distribution; as a result, many negatives quickly become trivial and contribute little gradient; and (iii)
embeddings across modalities show mismatched first- and second-order statistics, which makes rankings less stable. To tackle these problems, we propose e5-omni, a lightweight explicit
alignment recipe that adapts off-the-shelf VLMs into robust omni-modal
embedding models. e5-omni combines three simple components: (1) modality-aware temperature calibration to align similarity scales, (2) a controllable negative curriculum with debiasing to focus on confusing negatives while reducing the impact of false negatives, and (3) batch whitening with covariance regularization to better match cross-modal geometry in the shared
embedding space. Experiments on MMEB-V2 and AudioCaps show consistent gains over strong bi-modal and omni-modal baselines, and the same recipe also transfers well to other VLM backbones. We release our model checkpoint at https://huggingface.co/Haon-Chen/e5-omni-7B.
Read the full abstract ▾
All authors Haonan Chen, Sicheng Gao, Radu Timofte, Tetsuya Sakai, Zhicheng Dou
Computer Vision
PhysVideoGenerator: Towards Physically Aware Video Generation via Latent Physics Guidance
Siddarth Nilol Kundur Satish, Devesh Jaiswal, Hongyu Chen et al. (+1 more)
Current
video generation models produce high-quality aesthetic
videos but often struggle to learn representations of real-world physics dynamics, resulting in artifacts such as unnatural object collisions, inconsistent
gravity, and temporal flickering. In this work, we propose PhysVideoGenerator, a proof-of-concept framework that explicitly embeds a learnable physics prior into the
video generation process. We introduce a lightweight predictor network, PredictorP, which regresses high-level physical features extracted from a pre-trained
Video Joint
Embedding Predictive Architecture (V-JEPA 2) directly from noisy
diffusion latents. These predicted physics
tokens are injected into the temporal
attention layers of a DiT-based generator (Latte) via a dedicated cross-
attention mechanism. Our primary contribution is demonstrating the technical feasibility of this joint training paradigm: we show that
diffusion latents contain sufficient information to recover V-JEPA 2 physical representations, and that multi-task optimization remains stable over training. This report documents the architectural design, technical challenges, and validation of training stability, establishing a foundation for future large-scale evaluation of physics-aware generative models.
Read the full abstract ▾
All authors Siddarth Nilol Kundur Satish, Devesh Jaiswal, Hongyu Chen, Abhishek Bakshi
Computer Vision
MGPC: Multimodal Network for Generalizable Point Cloud Completion With Modality Dropout and Progressive Decoding
Jiangyuan Liu, Yuhao Zhao, Hongxuan Ma et al. (+3 more)
Point cloud completion aims to recover complete 3D geometry from partial observations caused by limited viewpoints and occlusions. Existing learning-based works, including 3D Convolutional Neural Network (CNN)-based, point-based, and
Transformer-based methods, have achieved strong performance on synthetic
benchmarks. However, due to the limitations of modality, scalability, and generative capacity, their generalization to novel objects and real-world scenarios remains challenging. In this paper, we propose MGPC, a generalizable
multimodal point cloud completion framework that integrates point clouds, RGB images, and text within a unified architecture. MGPC introduces an innovative modality dropout strategy, a
Transformer-based fusion module, and a novel progressive generator to improve robustness, scalability, and
geometric modeling capability. We further develop an automatic data generation pipeline and construct MGPC-1M, a large-scale
benchmark with over 1,000 categories and one million training pairs. Extensive experiments on MGPC-1M and in-the-wild data demonstrate that the proposed method consistently outperforms prior baselines and exhibits strong generalization under real-world conditions.
Read the full abstract ▾
All authors Jiangyuan Liu, Yuhao Zhao, Hongxuan Ma, Zhe Liu, Jian Wang, Wei Zou
Computer Vision
VideoMemory: Toward Consistent Video Generation via Memory Integration
Jinsong Zhou, Yihua Du, Xinli Xu et al. (+7 more)
Maintaining consistent characters, props, and environments across multiple shots is a central challenge in narrative
video generation. Existing models can produce high-quality short clips but often fail to preserve entity identity and appearance when scenes change or when entities reappear after long temporal gaps. We present VideoMemory, an entity-centric framework that integrates narrative planning with visual generation through a Dynamic
Memory Bank. Given a structured script, a multi-
agent system decomposes the narrative into shots, retrieves entity representations from
memory, and synthesizes keyframes and
videos conditioned on these retrieved states. The Dynamic
Memory Bank stores explicit visual and semantic descriptors for characters, props, and backgrounds, and is updated after each shot to reflect story-driven changes while preserving identity. This
retrieval-update mechanism enables consistent portrayal of entities across distant shots and supports coherent long-form generation. To evaluate this setting, we construct a 54-case multi-shot consistency
benchmark covering character-, prop-, and background-persistent scenarios. Extensive experiments show that VideoMemory achieves strong entity-level coherence and high perceptual quality across diverse narrative sequences.
Read the full abstract ▾
All authors Jinsong Zhou, Yihua Du, Xinli Xu, Luozhou Wang, Zijie Zhuang, Yehang Zhang, Shuaibo Li, Xiaojun Hu, Bolan Su, Ying-cong Chen
Computer Vision
CrackSegFlow: Controllable Flow Matching Synthesis for Generalizable Crack Segmentation with a 50K Image-Mask Benchmark
Babak Asadi, Peiyang Wu, Mani Golparvar-Fard et al. (+1 more)
Defect
segmentation is central to computer
vision based inspection of infrastructure assets during both construction and operation. However, deployment remains limited due to scarce pixel-level labels and domain shift across environments. We introduce CrackSegFlow, a controllable Flow Matching synthesis method that renders synthetic images of cracks from masks with pixel-level
alignment. Our renderer combines
topology-preserving mask injection with edge gating to maintain thin-structure continuity. Class-conditional FM samples masks for
topology diversity, and CrackSegFlow renders aligned ground truth images from them. We further inject cracks onto crack-free backgrounds to diversify confounders and reduce false positives. Across five
datasets and using a CNN-
Transformer backbone, our results demonstrate that adding synthesized pairs improves in-domain performance by +5.37 mIoU and +5.13 F1, while target-guided cross-domain synthesis driven by target mask statistics adds +13.12 mIoU and +14.82 F1. We also release CSF-50K, a
benchmark dataset comprising 50,000 image-mask pairs.
Read the full abstract ▾
All authors Babak Asadi, Peiyang Wu, Mani Golparvar-Fard, Ramez Hajj
Computer Vision
MFC-RFNet: A Multi-scale Guided Rectified Flow Network for Radar Sequence Prediction
Wenjie Luo, Chuanhu Deng, Chaorong Li et al. (+2 more)
Accurate and high-resolution precipitation nowcasting from radar echo sequences is crucial for disaster mitigation and economic planning, yet it remains a significant challenge. Key difficulties include modeling complex multi-scale
evolution, correcting inter-frame feature misalignment caused by displacement, and efficiently capturing long-range spatiotemporal context without sacrificing spatial fidelity. To address these issues, we present the Multi-scale Feature Communication Rectified Flow (RF) Network (MFC-RFNet), a generative framework that integrates multi-scale communication with guided feature fusion. To enhance multi-scale fusion while retaining fine detail, a Wavelet-Guided Skip Connection (WGSC) preserves high-frequency components, and a Feature Communication Module (FCM) promotes bidirectional cross-scale interaction. To correct inter-frame displacement, a Condition-Guided Spatial Transform Fusion (CGSTF) learns spatial transforms from conditioning echoes to align shallow features. The backbone adopts rectified flow training to learn near-linear probability-flow trajectories, enabling few-step sampling with stable fidelity. Additionally, lightweight
Vision-RWKV (RWKV) blocks are placed at the encoder tail, the bottleneck, and the first decoder layer to capture long-range spatiotemporal dependencies at low spatial resolutions with moderate compute. Evaluations on four public
datasets (SEVIR, MeteoNet, Shanghai, and CIKM) demonstrate consistent improvements over strong baselines, yielding clearer echo morphology at higher rain-rate thresholds and sustained skill at longer lead times. These results suggest that the proposed synergy of RF training with scale-aware communication, spatial
alignment, and frequency-aware fusion presents an effective and robust approach for radar-based nowcasting.
Read the full abstract ▾
All authors Wenjie Luo, Chuanhu Deng, Chaorong Li, Rongyao Deng, Qiang Yang
Computer Vision
Shape Classification using Approximately Convex Segment Features
Bimal Kumar Ray
The existing object classification techniques based on descriptive features rely on object
alignment to compute the similarity of objects for classification. This paper replaces the necessity of object
alignment through sorting of feature. The object boundary is normalized and segmented into approximately convex segments and the segments are then sorted in descending order of their length. The segment length, number of extreme points in segments, area of segments, the base and the width of the segments - a bag of features - is used to measure the similarity between image boundaries. The proposed method is tested on
datasets and acceptable results are observed.
Read the full abstract ▾
Computer Vision
Systematic Evaluation of Depth Backbones and Semantic Cues for Monocular Pseudo-LiDAR 3D Detection
Samson Oseiwe Ajadalu
Monocular 3D object detection offers a low-cost alternative to
LiDAR, yet remains less accurate due to the difficulty of estimating metric depth from a single image. We systematically evaluate how depth backbones and feature engineering affect a
monocular Pseudo-
LiDAR pipeline on the KITTI validation split. Specifically, we compare NeWCRFs (supervised metric depth) against Depth Anything V2 Metric-Outdoor (Base) under an identical pseudo-
LiDAR generation and PointRCNN detection protocol. NeWCRFs yields stronger downstream 3D detection, achieving 10.50% AP$_{3D}$ at IoU$=0.7$ on the Moderate split using grayscale intensity (Exp~2). We further test point-cloud augmentations using appearance cues (grayscale intensity) and semantic cues (instance
segmentation confidence). Contrary to the expectation that semantics would substantially close the gap, these features provide only marginal gains, and mask-based sampling can degrade performance by removing contextual geometry. Finally, we report a depth-accuracy-versus-distance diagnostic using ground-truth 2D boxes (including Ped/Cyc), highlighting that coarse depth correctness does not fully predict strict 3D IoU. Overall, under an off-the-shelf
LiDAR detector, depth-backbone choice and
geometric fidelity dominate performance, outweighing secondary feature injection.
Read the full abstract ▾
Computer Vision
Unveiling Text in Challenging Stone Inscriptions: A Character-Context-Aware Patching Strategy for Binarization
Pratyush Jena, Amal Joseph, Arnav Sharma et al. (+1 more)
Binarization is a popular first step towards text extraction in historical artifacts. Stone inscription images pose severe challenges for binarization due to poor contrast between etched characters and the stone background, non-uniform surface degradation, distracting artifacts, and highly variable text density and layouts. These conditions frequently cause existing binarization techniques to fail and struggle to isolate coherent character regions. Many approaches sub-divide the image into patches to improve text fragment resolution and improve binarization performance. With this in mind, we present a robust and adaptive patching strategy to binarize challenging Indic inscriptions. The patches from our approach are used to train an
Attention U-Net for binarization. The
attention mechanism allows the model to focus on subtle structural cues, while our dynamic sampling and patch selection method ensures that the model learns to overcome surface noise and layout irregularities. We also introduce a carefully annotated, pixel-precise
dataset of Indic stone inscriptions at the character-fragment level. We demonstrate that our novel patching mechanism significantly boosts binarization performance across classical and deep learning baselines. Despite training only on single script Indic
dataset, our model exhibits strong zero-shot generalization to other Indic and non-indic scripts, highlighting its robustness and script-agnostic generalization capabilities. By producing clean, structured representations of inscription content, our method lays the foundation for downstream tasks such as script identification, OCR, and historical text analysis. Project page: https://ihdia.iiit.ac.in/shilalekhya-binarization/
Read the full abstract ▾
All authors Pratyush Jena, Amal Joseph, Arnav Sharma, Ravi Kiran Sarvadevabhatla
Computer Vision
Qianyu Guo, Jingrong Wu, Jieji Ren et al. (+2 more)
Few-shot
segmentation (FSS) aims to rapidly learn novel class concepts from limited examples to segment specific targets in unseen images, and has been widely applied in areas such as medical diagnosis and industrial inspection. However, existing studies largely overlook the complex environmental factors encountered in real world scenarios-such as illumination, background, and camera viewpoint-which can substantially increase the difficulty of test images. As a result, models trained under laboratory conditions often fall short of practical deployment requirements. To bridge this gap, in this paper, an environment-robust FSS setting is introduced that explicitly incorporates challenging test cases arising from complex environments-such as motion blur, small objects, and camouflaged targets-to enhance model's robustness under realistic, dynamic conditions. An environment robust FSS
benchmark (ER-FSS) is established, covering eight
datasets across multiple real world scenarios. In addition, an Adaptive
Attention Distillation (AAD) method is proposed, which repeatedly contrasts and distills key shared semantics between known (support) and unknown (query) images to derive class-specific
attention for novel categories. This strengthens the model's ability to focus on the correct targets in complex environments, thereby improving environmental robustness. Comparative experiments show that AAD improves mIoU by 3.3% - 8.5% across all
datasets and settings, demonstrating superior performance and strong generalization. The source code and
dataset are available at: https://github.com/guoqianyu-alberta/Adaptive-
Attention-
Distillation-for-FSS.
Read the full abstract ▾
All authors Qianyu Guo, Jingrong Wu, Jieji Ren, Weifeng Ge, Wenqiang Zhang
Computer Vision
Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions
Zhongbin Guo, Zhen Yang, Yushan Li et al. (+6 more)
Recent advancements in Spatial Intelligence (SI) have predominantly relied on
Vision-
Language Models (VLMs), yet a critical question remains: does spatial understanding originate from visual encoders or the fundamental
reasoning backbone? Inspired by this question, we introduce SiT-Bench, a novel
benchmark designed to evaluate the SI performance of Large
Language Models (LLMs) without pixel-level input, comprises over 3,800 expert-annotated items across five primary categories and 17 subtasks, ranging from
egocentric navigation and perspective transformation to fine-grained robotic manipulation. By converting single/multi-view scenes into high-fidelity, coordinate-aware textual descriptions, we challenge LLMs to perform symbolic textual
reasoning rather than visual pattern matching. Evaluation results of state-of-the-art (SOTA) LLMs reveals that while models achieve proficiency in localized semantic tasks, a significant "spatial gap" remains in global consistency. Notably, we find that explicit spatial
reasoning significantly boosts performance, suggesting that LLMs possess latent world-modeling potential. Our proposed
dataset SiT-Bench serves as a foundational resource to foster the development of spatially-grounded LLM backbones for future VLMs and
embodied agents. Our code and
benchmark will be released at https://github.com/binisalegend/SiT-Bench .
Read the full abstract ▾
All authors Zhongbin Guo, Zhen Yang, Yushan Li, Xinyue Zhang, Wenyu Gao, Jiacheng Wang, Chengzhi Li, Xiangrui Liu, Ping Jian
Computer Vision
Detecting AI-Generated Images via Distributional Deviations from Real Images
Yakun Niu, Yingjian Chen and Lei Zhang
The rapid advancement of generative models has significantly enhanced the quality of AI-generated images, raising concerns about misinformation and the erosion of public trust. Detecting AI-generated images has thus become a critical challenge, particularly in terms of generalizing to unseen generative models. Existing methods using frozen pre-trained CLIP models show promise in generalization but treat the image encoder as a basic feature extractor, failing to fully exploit its potential. In this paper, we perform an in-depth analysis of the frozen CLIP image encoder (CLIP-ViT), revealing that it effectively clusters real images in a high-level, abstract feature space. However, it does not truly possess the ability to distinguish between real and AI-generated images. Based on this analysis, we propose a Masking-based Pre-trained model Fine-Tuning (MPFT) strategy, which introduces a Texture-Aware Masking (TAM) mechanism to mask textured areas containing generative model-specific patterns during fine-tuning. This approach compels CLIP-ViT to attend to the "distributional deviations"from authentic images for AI-generated image detection, thereby achieving enhanced generalization performance. Extensive experiments on the GenImage and UniversalFakeDetect
datasets demonstrate that our method, fine-tuned with only a minimal number of images, significantly outperforms existing approaches, achieving up to 98.2% and 94.6% average accuracy on the two
datasets, respectively.
Read the full abstract ▾
Computer Vision
SpatiaLoc: Leveraging Multi-Level Spatial Enhanced Descriptors for Cross-Modal Localization
Tianyi Shang, Pengjie Xu, Zhaojun Deng et al. (+3 more)
Cross-modal localization using text and point clouds enables
robots to localize themselves via natural
language descriptions, with applications in autonomous
navigation and interaction between humans and
robots. In this task, objects often recur across text and point clouds, making spatial relationships the most discriminative cues for localization. Given this characteristic, we present SpatiaLoc, a framework utilizing a coarse-to-fine strategy that emphasizes spatial relationships at both the instance and global levels. In the coarse stage, we introduce a Bezier Enhanced Object Spatial Encoder (BEOSE) that models spatial relationships at the instance level using quadratic Bezier curves. Additionally, a Frequency Aware Encoder (FAE) generates spatial representations in the frequency domain at the global level. In the fine stage, an Uncertainty Aware Gaussian Fine Localizer (UGFL) regresses 2D positions by modeling predictions as Gaussian distributions with a loss function aware of uncertainty. Extensive experiments on KITTI360Pose demonstrate that SpatiaLoc significantly outperforms existing state-of-the-art (SOTA) methods.
Read the full abstract ▾
All authors Tianyi Shang, Pengjie Xu, Zhaojun Deng, Zhenyu Li, Zhicong Chen, Lijun Wu
Computer Vision
FEA-SLT: A Gloss-Free End-to-End Framework for Facial-Expression-Aware Sign Language Translation
Guobin Tu and Di Weng
Sign
Language Translation (SLT) is a challenging cross-modal task requiring joint modeling of manual articulations and non-manual signals. Existing gloss-free SLT methods effectively capture gestural dynamics but often underutilize facial expressions, which play crucial grammatical and disambiguating roles. This limitation can cause semantic degradation when distinct concepts share similar manual configurations. To address this issue, we propose FEA-SLT (**F**acial-**E**xpression-**A**ware **S**ign **L**anguage **T**ranslation), a gloss-free end-to-end framework that uses facial dynamics to provide complementary cues to manual signals. FEA-SLT employs a domain-transferred facial encoder to extract expression-sensitive representations and integrates them with manual features through a linguistically motivated *Facial-Expression-Aware Fusion* (FEAF) module. FEAF captures reciprocal dependencies between manual and facial channels via bidirectional modulation, enhancing syntactic fidelity. Experiments on PHOENIX14T and CSL-Daily show that FEA-SLT achieves state-of-the-art BLEU performance among gloss-free methods, while targeted analyses support improved translation of facial-sensitive utterances. Code is available at https://github.com/TuGuobin/FEA-SLT.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Persona-aware and Explainable Bikeability Assessment: A Vision-Language Model Approach
Yilong Dai, Ziyi Wang, Chenguang Wang et al. (+4 more)
Bikeability assessment is essential for advancing sustainable urban transportation and creating cyclist-friendly cities, and it requires incorporating users' perceptions of safety and comfort. Yet existing perception-based bikeability assessment approaches face key limitations in capturing the complexity of road environments and adequately accounting for heterogeneity in subjective user perceptions. This paper proposes a persona-aware
Vision-
Language Model framework for bikeability assessment with three novel contributions: (i) theory-grounded persona conditioning based on established cyclist typology that generates persona-specific explanations via chain-of-thought
reasoning; (ii) multi-granularity supervised fine-tuning that combines scarce expert-annotated
reasoning with abundant user ratings for joint prediction and explainable assessment; and (iii) AI-enabled data augmentation that creates controlled paired data to isolate infrastructure variable impacts. To test and validate this framework, we developed a panoramic image-based crowdsourcing system and collected 12,400 persona-conditioned assessments from 427 cyclists. Experiment results show that the proposed framework offers competitive bikeability rating prediction while uniquely enabling explainable factor attribution.
Read the full abstract ▾
All authors Yilong Dai, Ziyi Wang, Chenguang Wang, Kexin Zhou, Yiheng Qian, Susu Xu, Xiang Yan
Computer Vision
CloudMatch: Weak-to-Strong Consistency Learning for Semi-Supervised Cloud Detection
Jiayi Zhao, Changlu Chen, Jingsheng Li et al. (+2 more)
Due to the high cost of annotating accurate pixel-level labels, semi-supervised learning has emerged as a promising approach for cloud detection. In this paper, we propose CloudMatch, a semi-supervised framework that effectively leverages unlabeled remote sensing imagery through view-consistency learning combined with scene-mixing augmentations. An observation behind CloudMatch is that cloud patterns exhibit structural diversity and contextual variability across different scenes and within the same scene category. Our key insight is that enforcing prediction consistency across diversely augmented views, incorporating both inter-scene and intra-scene mixing, enables the model to capture the structural diversity and contextual richness of cloud patterns. Specifically, CloudMatch generates one weakly augmented view along with two complementary strongly augmented views for each unlabeled image: one integrates inter-scene patches to simulate contextual variety, while the other employs intra-scene mixing to preserve semantic coherence. This approach guides pseudolabel generation and enhances generalization. Extensive experiments show that CloudMatch achieves good performance, demonstrating its capability to utilize unlabeled data efficiently and advance semi-supervised cloud detection.
Read the full abstract ▾
All authors Jiayi Zhao, Changlu Chen, Jingsheng Li, Tianxiang Xue, Kun Zhan
Computer Vision
Physics-Constrained Cross-Resolution Enhancement Network for Optics-Guided Thermal UAV Image Super-Resolution
Zhicheng Zhao, Fengjiao Peng, Jinquan Yan et al. (+3 more)
Optics-guided thermal
UAV image super-resolution has attracted significant research interest due to its potential in all-weather monitoring applications. However, existing methods typically compress optical features to match thermal feature dimensions for cross-modal
alignment and fusion, which not only causes the loss of high-frequency information that is beneficial for thermal super-resolution, but also introduces physically inconsistent artifacts such as texture distortions and edge blurring by overlooking differences in the imaging physics between modalities. To address these challenges, we propose PCNet to achieve cross-resolution mutual enhancement between optical and thermal modalities, while physically constraining the optical guidance process via thermal conduction to enable robust thermal
UAV image super-resolution. In particular, we design a Cross-Resolution Mutual Enhancement Module (CRME) to jointly optimize thermal image super-resolution and optical-to-thermal modality conversion, facilitating effective bidirectional feature interaction across resolutions while preserving high-frequency optical priors. Moreover, we propose a Physics-Driven Thermal Conduction Module (PDTM) that incorporates two-dimensional heat conduction into optical guidance, modeling spatially-varying heat conduction properties to prevent inconsistent artifacts. In addition, we introduce a temperature consistency loss that enforces regional distribution consistency and boundary gradient smoothness to ensure generated thermal images align with real-world thermal radiation principles. Extensive experiments on VGTSR2.0 and DroneVehicle
datasets demonstrate that PCNet significantly outperforms state-of-the-art methods on both reconstruction quality and downstream tasks including semantic
segmentation and object detection.
Read the full abstract ▾
All authors Zhicheng Zhao, Fengjiao Peng, Jinquan Yan, Wei Lu, Chenglong Li, Jin Tang
Computer Vision
Semantic Belief-State World Model for 3D Human Motion Prediction
Sarim Chaudhry
Human motion prediction has traditionally been framed as a sequence regression problem where models extrapolate future joint coordinates from observed pose histories. While effective over short horizons this approach does not separate observation reconstruction with dynamics modeling and offers no explicit representation of the latent causes governing motion. As a result, existing methods exhibit compounding drift, mean-pose collapse, and poorly calibrated uncertainty when rolled forward beyond the training regime. Here we propose a Semantic Belief-State World Model (SBWM) that reframes human motion prediction as latent dynamical simulation on the human body
manifold. Rather than predicting poses directly, SBWM maintains a recurrent probabilistic belief state whose
evolution is learned independently of pose reconstruction and explicitly aligned with the SMPL-X anatomical parameterization. This
alignment imposes a structural information bottleneck that prevents the latent state from encoding static geometry or sensor noise, forcing it to capture motion dynamics, intent, and control-relevant structure. Inspired by belief-state world models developed for model-based reinforcement learning, SBWM adapts
stochastic latent transitions and rollout-centric training to the domain of human motion. In contrast to RSSM-based,
transformer, and
diffusion approaches optimized for reconstruction fidelity, SBWM prioritizes stable forward simulation. We demonstrate coherent long-horizon rollouts, and competitive accuracy at substantially lower computational cost. These results suggest that treating the human body as part of the world models state space rather than its output fundamentally changes how motion is simulated, and predicted.
Read the full abstract ▾
Computer Vision
G2P: Gaussian-to-Point Attribute Alignment for Boundary-Aware 3D Segmentation
Hojun Song, Chae-yeong Song, Jeong-hun Hong et al. (+5 more)
Point cloud
segmentation is critical for 3D scene understanding. However, sparse and irregular point distributions provide limited appearance evidence, making geometry-only features insufficient to distinguish objects with similar shapes but distinct appearances e.g., color, texture, and material. We propose Gaussian-to-Point (G2P), which transfers Gaussian attributes from 3D Gaussian Splatting to point clouds for more discriminative and appearance-consistent
segmentation. Our G2P addresses the misalignment between optimized Gaussians and original point geometry by establishing point-wise correspondences. By distilling opacity-derived visibility cues, we mitigate the
geometric ambiguity that limits existing models. Additionally, Gaussian scale attributes enable precise boundary localization in complex 3D scenes. Extensive experiments demonstrate that our approach achieves competitive performance on standard
benchmarks and shows notable improvements on geometrically challenging classes, without pretrained 2D features or
language supervision in our
segmentation pipeline.
Read the full abstract ▾
All authors Hojun Song, Chae-yeong Song, Jeong-hun Hong, Chaewon Moon, Soo Ye Kim, Yiyi Liao, Jaehyup Lee, Sang-hyo Park