Computer Vision
3D sans 3D Scans: Scalable Pre-training from Video-Generated Point Clouds
Ryousuke Yamada, Kohsuke Ide, Yoshihiro Fukuhara et al. (+4 more)
Despite recent progress in 3D self-supervised learning, collecting large-scale 3D scene scans remains expensive and labor-intensive. In this work, we investigate whether 3D representations can be learned from unlabeled
videos recorded without any real 3D sensors. We present
Laplacian-Aware Multi-level 3D Clustering with Sinkhorn-Knopp (LAM3C), a self-supervised framework that learns from
video-generated point clouds reconstructed from unlabeled
videos. We first introduce RoomTours, a
video-generated point cloud
dataset constructed by collecting room-walkthrough
videos from the web (e.g., real-estate tours) and generating 49,219 scenes using an off-the-shelf feed-forward reconstruction model. We also propose a noise-regularized loss that stabilizes representation learning by enforcing local
geometric smoothness and ensuring feature stability under noisy point clouds. Remarkably, without using any real 3D scans, LAM3C achieves better performance than previous self-supervised methods on indoor semantic and instance
segmentation. These results suggest that unlabeled
videos represent an abundant source of data for 3D self-supervised learning. Our source code is available at https://ryosuke-yamada.github.io/lam3c/.
Read the full abstract ▾
All authors Ryousuke Yamada, Kohsuke Ide, Yoshihiro Fukuhara, Hirokatsu Kataoka, Gilles Puy, Andrei Bursuc, Yuki M. Asano
Computer Vision
Toward Stable Semi-Supervised Remote Sensing Segmentation via Co-Guidance and Co-Fusion
Yi Zhou, Xuechao Zou, Shun Zhang et al. (+7 more)
Semi-supervised remote sensing (RS) image semantic
segmentation offers a promising solution to alleviate the burden of exhaustive annotation, yet it fundamentally struggles with pseudo-label drift, a phenomenon where confirmation bias leads to the accumulation of errors during training. In this work, we propose Co2S, a stable semi-supervised RS
segmentation framework that synergistically fuses priors from
vision-
language models and self-supervised models. Specifically, we construct a heterogeneous dual-student architecture comprising two distinct ViT-based
vision foundation models initialized with pretrained CLIP and DINOv3 to mitigate error accumulation and pseudo-label drift. To effectively incorporate these distinct priors, an explicit-implicit semantic co-guidance mechanism is introduced that utilizes text
embeddings and learnable queries to provide explicit and implicit class-level guidance, respectively, thereby jointly enhancing semantic consistency. Furthermore, a global-local feature collaborative fusion strategy is developed to effectively fuse the global contextual information captured by CLIP with the local details produced by DINOv3, enabling the model to generate highly precise
segmentation results. Extensive experiments on six popular
datasets demonstrate the superiority of the proposed method, which consistently achieves leading performance across various partition protocols and diverse scenarios. Project page is available at https://xavierjiezou.github.io/Co2S/.
Read the full abstract ▾
All authors Yi Zhou, Xuechao Zou, Shun Zhang, Kai Li, Shiying Wang, Jingming Chen, Congyan Lang, Tengfei Cao, Pin Tao, Yuanchun Shi
Computer Vision · Cross-listed from cs.SE
Interpretable Gallbladder Ultrasound Diagnosis: A Lightweight Web-Mobile Software Platform with Real-Time XAI
Fuyad Hasan Bhoyan, Prashanta Sarker, Parsia Noor Ethila et al. (+3 more)
Early and accurate detection of gallbladder diseases is crucial, yet ultrasound interpretation is challenging. To address this, an AI-driven diagnostic software integrates our hybrid deep learning model MobResTaNet to classify ten categories, nine gallbladder disease types and normal directly from ultrasound images. The system delivers interpretable, real-time predictions via Explainable AI (XAI) visualizations, supporting transparent clinical decision-making. It achieves up to 99.85% accuracy with only 2.24M parameters. Deployed as web and mobile applications using HTML, CSS, JavaScript, Bootstrap, and Flutter, the software provides efficient, accessible, and trustworthy diagnostic support at the point of care
Read the full abstract ▾
All authors Fuyad Hasan Bhoyan, Prashanta Sarker, Parsia Noor Ethila, Md. Emon Hossain, Md Kaviul Hossain, Md Humaion Kabir Mehedi
Computer Vision
An Architecture-Led Hybrid Report on Body Language Detection Project
Thomson Tong and Diba Darooneh
This report provides an architecture-led analysis of two modern
vision-
language models (VLMs), Qwen2.5-VL-7B-Instruct and Llama-4-Scout-17B-16E-Instruct, and explains how their architectural properties map to a practical
video-to-artifact pipeline implemented in the BodyLanguageDetection repository [1]. The system samples
video frames,
prompts a VLM to detect visible people and generate pixel-space bounding boxes with
prompt-conditioned attributes (
emotion by default), validates output structure using a predefined schema, and optionally renders an annotated
video. We first summarize the shared
multimodal foundation (visual tokenization,
Transformer attention, and instruction following), then describe each architecture at a level sufficient to justify engineering choices without speculative internals. Finally, we connect model behavior to system constraints: structured outputs can be syntactically valid while semantically incorrect, schema validation is structural (not
geometric correctness), person identifiers are frame-local in the current prompting contract, and interactive single-frame analysis returns free-form text rather than schema-enforced JSON. These distinctions are critical for writing defensible claims, designing robust interfaces, and planning evaluation.
Read the full abstract ▾
Computer Vision
With Great Context Comes Great Prediction Power: Classifying Objects via Geo-Semantic Scene Graphs
Ciprian Constantinescu and Marius Leordeanu
Humans effortlessly identify objects by leveraging a rich understanding of the surrounding scene, including spatial relationships, material properties, and the co-occurrence of other objects. In contrast, most computational object recognition systems operate on isolated image regions, devoid of meaning in isolation, thus ignoring this vital contextual information. This paper argues for the critical role of context and introduces a novel framework for contextual object classification. We first construct a Geo-Semantic Contextual Graph (GSCG) from a single
monocular image. This rich, structured representation is built by integrating a metric depth
estimator with a unified panoptic and material
segmentation model. The GSCG encodes objects as nodes with detailed
geometric, chromatic, and material attributes, and their spatial relationships as edges. This explicit graph structure makes the model's
reasoning process inherently interpretable. We then propose a specialized graph-based classifier that aggregates features from a target object, its immediate neighbors, and the global scene context to predict its class. Through extensive ablation studies, we demonstrate that our context-aware model achieves a classification accuracy of 73.4%, dramatically outperforming context-agnostic versions (as low as 38.4%). Furthermore, our GSCG-based approach significantly surpasses strong baselines, including fine-tuned ResNet models (max 53.5%) and a state-of-the-art
multimodal Large
Language Model (LLM), Llama 4 Scout, which, even when given the full image alongside a detailed description of objects, maxes out at 42.3%. These results on COCO 2017 train/val splits highlight the superiority of explicitly structured and interpretable context for object recognition tasks.
Read the full abstract ▾
Computer Vision
OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding
Wenyuan Huang, Zhenyu Zhang, Zhao Wang et al. (+4 more)
3D visual grounding aims to locate objects based on natural
language descriptions in 3D scenes. Existing supervised methods are limited by generalization and recent zero-shot methods typically rely on a predefined Object Lookup Table (OLT) to query Visual
Language Models (VLMs) for
reasoning about object locations via a single step grounding, which limits the applications in scenarios with undefined targets and complex queries. To address these problems, we present OpenGround, a novel zero-shot framework for open-world 3D visual grounding that remains compatible with recent zero-shot methods. OpenGround integrates Task-Chain Planning to decompose a query into a plan of context-to-target sub-goals for progressive grounding, and Context-Guided Perception to perceive novel objects online under context guidance from the task chain. We also propose a new
dataset named OpenTarget, which contains over 7000 object-description pairs to mimic open-world evaluation. Extensive experiments demonstrate that OpenGround achieves competitive performance on Nr3D, state-of-the-art on ScanRefer, and delivers a substantial 17.6% improvement on OpenTarget. Project Page at https://why-102.github.io/openground.io/.
Read the full abstract ▾
All authors Wenyuan Huang, Zhenyu Zhang, Zhao Wang, Zhou Wei, Ting Huang, Fang Zhao, Jian Yang
Computer Vision
A Low-Cost UAV Deep Learning Pipeline for Integrated Apple Disease Diagnosis,Freshness Assessment, and Fruit Detection
Soham Dutta, Soham Banerjee, Sneha Mahata et al. (+2 more)
Apple orchards require timely disease detection, fruit quality assessment, and yield estimation, yet existing
UAV-based systems address such tasks in isolation and often rely on costly multispectral sensors. This paper presents a unified, low-cost RGB-only
UAV-based orchard intelligent pipeline integrating ResNet50 for leaf disease detection, VGG 16 for apple freshness determination, and YOLOv8 for real-time apple detection and localization. The system runs on an ESP32-CAM and Raspberry Pi, providing fully offline on-site inference without cloud support. Experiments demonstrate 98.9% accuracy for leaf disease classification, 97.4% accuracy for freshness classification, and 0.857 F1 score for apple detection. The framework provides an accessible and scalable alternative to multispectral
UAV solutions, supporting practical precision agriculture on affordable hardware.
Read the full abstract ▾
All authors Soham Dutta, Soham Banerjee, Sneha Mahata, Anindya Sen, Sayantani Datta
Computer Vision
Reverse Personalization
Han-Wei Kung, Tuomas Varanka and Nicu Sebe
Recent text-to-image
diffusion models have demonstrated remarkable generation of realistic facial images conditioned on textual
prompts and human identities, enabling creating personalized facial imagery. However, existing
prompt-based methods for removing or modifying identity-specific features rely either on the subject being well-represented in the pre-trained model or require model fine-tuning for specific identities. In this work, we analyze the identity generation process and introduce a reverse personalization framework for face anonymization. Our approach leverages conditional
diffusion inversion, allowing direct manipulation of images without using text
prompts. To generalize beyond subjects in the model's training data, we incorporate an identity-guided conditioning branch. Unlike prior anonymization methods, which lack control over facial attributes, our framework supports attribute-controllable anonymization. We demonstrate that our method achieves a state-of-the-art balance between identity removal, attribute preservation, and image quality. Source code and data are available at https://github.com/hanweikung/reverse-personalization .
Read the full abstract ▾
Computer Vision
Spatial-aware Symmetric Alignment for Text-guided Medical Image Segmentation
Linglin Liao, Qichuan Geng and Yu Liu
Text-guided Medical Image
Segmentation has shown considerable promise for medical image
segmentation, with rich clinical text serving as an effective supplement for scarce data. However, current methods have two key bottlenecks. On one hand, they struggle to process diagnostic and descriptive texts simultaneously, making it difficult to identify lesions and establish associations with image regions. On the other hand, existing approaches focus on lesions description and fail to capture positional constraints, leading to critical deviations. Specifically, with the text "in the left lower lung", the
segmentation results may incorrectly cover both sides of the lung. To address the limitations, we propose the Spatial-aware Symmetric
Alignment (SSA) framework to enhance the capacity of referring hybrid medical texts consisting of locational, descriptive, and diagnostic information. Specifically, we propose symmetric optimal transport
alignment mechanism to strengthen the associations between image regions and multiple relevant expressions, which establishes bi-directional fine-grained
multimodal correspondences. In addition, we devise a composite directional guidance strategy that explicitly introduces spatial constraints in the text by constructing region-level guidance masks. Extensive experiments on public
benchmarks demonstrate that SSA achieves state-of-the-art (SOTA) performance, particularly in accurately segmenting lesions characterized by spatial relational constraints.
Read the full abstract ▾
Computer Vision
PoseStreamer: A Multi-modal Framework for 3D Tracking of Unseen Moving Objects
Huiming Yang, Linglin Liao, Fei Ding et al. (+2 more)
Six degree of freedom (6DoF) pose estimation for novel objects is a critical task in computer
vision, yet it faces significant challenges in high-speed and low-light scenarios where standard RGB cameras suffer from motion blur. While event cameras offer a promising solution due to their high temporal resolution, current 6DoF pose estimation methods typically yield suboptimal performance in high-speed object moving scenarios. To address this gap, we propose PoseStreamer, a robust multi-modal 6DoF pose estimation framework designed specifically on high-speed moving scenarios. Our approach integrates three core components: an Adaptive Pose
Memory Queue that utilizes historical orientation cues for temporal consistency, an Object-centric 2D Tracker that provides strong 2D priors to boost 3D center recall, and a Ray Pose Filter for
geometric refinement along camera rays. Furthermore, we introduce MoCapCube6D, a novel multi-modal
dataset constructed to
benchmark performance under rapid motion. Extensive experiments demonstrate that PoseStreamer not only achieves superior accuracy in high-speed moving scenarios, but also exhibits strong generalizability as a template-free framework for unseen moving objects.
Read the full abstract ▾
All authors Huiming Yang, Linglin Liao, Fei Ding, Sibo Wang, Zijian Zeng
Computer Vision
RealCamo: Boosting Real Camouflage Synthesis with Layout Controls and Textual-Visual Guidance
Chunyuan Chen, Yunuo Cai, Shujuan Li et al. (+3 more)
Camouflaged image generation (CIG) has recently emerged as an efficient alternative for acquiring high-quality training data for camouflaged object detection (COD). However, existing CIG methods still suffer from a substantial gap to real camouflaged imagery: generated images either lack sufficient camouflage due to weak visual similarity, or exhibit cluttered backgrounds that are semantically inconsistent with foreground targets. To address these limitations, we propose RealCamo, a novel out-painting-based framework for controllable realistic camouflaged image generation. RealCamo explicitly introduces additional layout controls to regulate global image structure, thereby improving semantic coherence between foreground objects and generated backgrounds. Moreover, we construct a
multimodal textual-visual condition by combining a unified fine-grained textual task description with texture-oriented background
retrieval, which jointly guides the generation process to enhance visual fidelity and realism. To quantitatively assess camouflage quality, we further introduce a background-foreground distribution divergence metric that measures the effectiveness of camouflage in generated images. Extensive experiments and visualizations demonstrate the effectiveness of our proposed framework.
Read the full abstract ▾
All authors Chunyuan Chen, Yunuo Cai, Shujuan Li, Weiyun Liang, Bin Wang, Jing Xu
Computer Vision
YOLO-IOD: Towards Real Time Incremental Object Detection
Shizhou Zhang, Xueqiang Lv, Yinghui Xing et al. (+4 more)
Current methods for incremental object detection (IOD) primarily rely on Faster R-CNN or DETR series detectors; however, these approaches do not accommodate the real-time YOLO detection frameworks. In this paper, we first identify three primary types of knowledge conflicts that contribute to catastrophic forgetting in YOLO-based incremental detectors: foreground-background confusion, parameter interference, and misaligned knowledge
distillation. Subsequently, we introduce YOLO-IOD, a real-time Incremental Object Detection (IOD) framework that is constructed upon the pretrained YOLO-World model, facilitating incremental learning via a stage-wise parameter-efficient fine-tuning process. Specifically, YOLO-IOD encompasses three principal components: 1) Conflict-Aware Pseudo-Label Refinement (CPR), which mitigates the foreground-background confusion by leveraging the confidence levels of pseudo labels and identifying potential objects relevant to future tasks. 2) Importancebased Kernel Selection (IKS), which identifies and updates the pivotal convolution kernels pertinent to the current task during the current learning stage. 3) Cross-Stage Asymmetric Knowledge
Distillation (CAKD), which addresses the misaligned knowledge
distillation conflict by transmitting the features of the student target detector through the detection heads of both the previous and current teacher detectors, thereby facilitating asymmetric
distillation between existing and newly introduced categories. We further introduce LoCo COCO, a more realistic
benchmark that eliminates data leakage across stages. Experiments on both conventional and LoCo COCO
benchmarks show that YOLO-IOD achieves superior performance with minimal forgetting.
Read the full abstract ▾
All authors Shizhou Zhang, Xueqiang Lv, Yinghui Xing, Qirui Wu, Di Xu, Chen Zhao, Yanning Zhang
Computer Vision
Wavelet-based Multi-View Fusion of 4D Radar Tensor and Camera for Robust 3D Object Detection
Runwei Guan, Jianan Liu, Shaofeng Liang et al. (+7 more)
4D millimeter-wave (mmWave) radar has been widely adopted in autonomous driving and
robot perception due to its low cost and all-weather robustness. However, point-cloud-based radar representations suffer from information loss due to multi-stage signal processing, while directly utilizing raw 4D radar tensors incurs prohibitive computational costs. To address these challenges, we propose WRCFormer, a novel 3D object detection framework that efficiently fuses raw 4D radar cubes with camera images via decoupled multi-view radar representations. Our approach introduces two key components: (1) A Wavelet
Attention Module embedded in a wavelet-based Feature Pyramid Network (FPN), which enhances the representation of sparse radar signals and image data by capturing joint spatial-frequency features, thereby mitigating information loss while maintaining computational efficiency. (2) A Geometry-guided Progressive Fusion mechanism, a two-stage query-based fusion strategy that progressively aligns multi-view radar and visual features through
geometric priors, enabling modality-agnostic and efficient integration without overwhelming computational overhead. Extensive experiments on the K-Radar
benchmark show that WRCFormer achieves state-of-the-art performance, surpassing the best existing model by approximately 2.4% in all scenarios and 1.6% in sleet conditions, demonstrating strong robustness in adverse weather.
Read the full abstract ▾
All authors Runwei Guan, Jianan Liu, Shaofeng Liang, Fangqiang Ding, Shanliang Yao, Xiaokai Bai, Daizong Liu, Tao Huang, Guoqiang Mao, Hui Xiong
Computer Vision
CLIP-Joint-Detect: End-to-End Joint Training of Object Detectors with Contrastive Vision-Language Supervision
Behnam Raoufi, Hossein Sharify, Mohamad Mahdee Ramezanee et al. (+2 more)
Conventional object detectors rely on cross-
entropy classification, which can be vulnerable to class imbalance and label noise. We propose CLIP-Joint-Detect, a simple and detector-agnostic framework that integrates CLIP-style contrastive
vision-
language supervision through end-to-end joint training. A lightweight parallel head projects region or grid features into the CLIP
embedding space and aligns them with learnable class-specific text
embeddings via InfoNCE contrastive loss and an auxiliary cross-
entropy term, while all standard detection losses are optimized simultaneously. The approach applies seamlessly to both two-stage and one-stage architectures. We validate it on Pascal VOC 2007+2012 using Faster R-CNN and on the large-scale MS COCO 2017
benchmark using modern YOLO detectors (YOLOv11), achieving consistent and substantial improvements while preserving real-time inference speed. Extensive experiments and ablations demonstrate that joint optimization with learnable text
embeddings markedly enhances closed-set detection performance across diverse architectures and
datasets.
Read the full abstract ▾
All authors Behnam Raoufi, Hossein Sharify, Mohamad Mahdee Ramezanee, Khosrow Hajsadeghi, Saeed Bagheri Shouraki
Computer Vision
Learning Where to Focus: Density-Driven Guidance for Detecting Dense Tiny Objects
Zhicheng Zhao, Xuanang Fan, Lingma Sun et al. (+2 more)
High-resolution remote sensing imagery increasingly contains dense clusters of tiny objects, the detection of which is extremely challenging due to severe mutual occlusion and limited pixel footprints. Existing detection methods typically allocate computational resources uniformly, failing to adaptively focus on these density-concentrated regions, which hinders feature learning effectiveness. To address these limitations, we propose the Dense Region Mining Network (DRMNet), which leverages density maps as explicit spatial priors to guide adaptive feature learning. First, we design a Density Generation Branch (DGB) to model object distribution patterns, providing quantifiable priors that guide the network toward dense regions. Second, to address the computational bottleneck of global
attention, our Dense Area Focusing Module (DAFM) uses these density maps to identify and focus on dense areas, enabling efficient local-global feature interaction. Finally, to mitigate feature degradation during hierarchical extraction, we introduce a Dual Filter Fusion Module (DFFM). It disentangles multi-scale features into high- and low-frequency components using a discrete cosine transform and then performs density-guided cross-
attention to enhance complementarity while suppressing background interference. Extensive experiments on the AI-TOD and DTOD
datasets demonstrate that DRMNet surpasses state-of-the-art methods, particularly in complex scenarios with high object density and severe occlusion.
Read the full abstract ▾
All authors Zhicheng Zhao, Xuanang Fan, Lingma Sun, Chenglong Li, Jin Tang
Computer Vision
ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving
Qihang Peng, Xuesong Chen, Chenye Yang et al. (+2 more)
Autonomous driving requires generating safe and reliable trajectories from complex
multimodal inputs. Traditional modular pipelines separate perception, prediction, and planning, while recent end-to-end (E2E) systems learn them jointly.
Vision-
language models (VLMs) further enrich this paradigm by introducing cross-modal priors and commonsense
reasoning, yet current VLM-based planners face three key challenges: (i) a mismatch between discrete text
reasoning and continuous control, (ii) high latency from autoregressive chain-of-thought decoding, and (iii) inefficient or non-causal planners that limit real-time deployment. We propose ColaVLA, a unified
vision-
language-action framework that transfers
reasoning from text to a unified latent space and couples it with a hierarchical, parallel trajectory decoder. The Cognitive Latent Reasoner compresses scene understanding into compact, decision-oriented meta-action
embeddings through ego-adaptive selection and only two VLM forward passes. The Hierarchical Parallel Planner then generates multi-scale, causality-consistent trajectories in a single forward pass. Together, these components preserve the generalization and interpretability of VLMs while enabling efficient, accurate and safe trajectory generation. Experiments on the nuScenes
benchmark show that ColaVLA achieves state-of-the-art performance in both open-loop and closed-loop settings with favorable efficiency and robustness.
Read the full abstract ▾
All authors Qihang Peng, Xuesong Chen, Chenye Yang, Shaoshuai Shi, Hongsheng Li
Computer Vision
JavisGPT: A Unified Multi-modal LLM for Sounding-Video Comprehension and Generation
Kai Liu, Jungang Li, Yuchong Sun et al. (+13 more)
This paper presents JavisGPT, the first unified
multimodal large
language model (MLLM) for joint audio-
video (JAV) comprehension and generation. JavisGPT has a concise encoder-LLM-decoder architecture, which has a SyncFusion module for spatio-temporal audio-
video fusion and synchrony-aware learnable queries to bridge a pretrained JAV-DiT generator. This design enables temporally coherent
video-audio understanding and generation from
multimodal instructions. We design an effective three-stage training pipeline consisting of
multimodal pretraining, audio-
video fine-tuning, and large-scale instruction-tuning, to progressively build
multimodal comprehension and generation from existing
vision-
language models. For instruction tuning, we construct JavisInst-Omni, a high-quality instruction
dataset with over 200K GPT-4o-curated audio-
video-text dialogues that cover diverse and multi-level comprehension and generation scenarios. On JAV comprehension and generation
benchmarks, our experiments show that JavisGPT outperforms existing MLLMs, particularly in complex and temporally synchronized settings.
Read the full abstract ▾
All authors Kai Liu, Jungang Li, Yuchong Sun, Shengqiong Wu, Jianzhang Gao, Daoan Zhang, Wei Zhang, Sheng Jin, Sicheng Yu, Geng Zhan, Jiayi Ji, Fan Zhou, Liang Zheng, Shuicheng Yan, Hao Fei, Tat-Seng Chua
Computer Vision · Cross-listed from cs.AI
HiSciBench: A Hierarchical Multi-disciplinary Benchmark for Scientific Intelligence from Reading to Discovery
Yaping Zhang, Qixuan Zhang, Xingquan Zhang et al. (+8 more)
The rapid advancement of large
language models (LLMs) and
multimodal foundation models has sparked growing interest in their potential for scientific research. However, scientific intelligence encompasses a broad spectrum of abilities ranging from understanding fundamental knowledge to conducting creative discovery, and existing
benchmarks remain fragmented. Most focus on narrow tasks and fail to reflect the hierarchical and multi-disciplinary nature of real scientific inquiry. We introduce HiSciBench, a hierarchical
benchmark designed to evaluate foundation models across five levels that mirror the complete scientific workflow: Scientific Literacy (L1), Literature Parsing (L2), Literature-based Question Answering (L3), Literature Review Generation (L4), and Scientific Discovery (L5). HiSciBench contains 8,735 carefully curated instances spanning six major scientific disciplines, including mathematics, physics, chemistry, biology, geography, and astronomy, and supports
multimodal inputs including text, equations, figures, and tables, as well as cross-lingual evaluation. Unlike prior
benchmarks that assess isolated abilities, HiSciBench provides an integrated, dependency-aware framework that enables detailed diagnosis of model capabilities across different stages of scientific
reasoning. Comprehensive evaluations of leading models, including GPT-5, DeepSeek-R1, and several
multimodal systems, reveal substantial performance gaps: while models achieve up to 69% accuracy on basic literacy tasks, performance declines sharply to 25% on discovery-level challenges. HiSciBench establishes a new standard for evaluating scientific Intelligence and offers actionable insights for developing models that are not only more capable but also more reliable. The
benchmark will be publicly released to facilitate future research.
Read the full abstract ▾
All authors Yaping Zhang, Qixuan Zhang, Xingquan Zhang, Zhiyuan Chen, Wenwen Zhuang, Yupu Liang, Lu Xiang, Yang Zhao, Jiajun Zhang, Yu Zhou, Chengqing Zong
Computer Vision
Hash Grid Feature Pruning
Yangzhi Ma, Bojun Liu, Jie Li et al. (+2 more)
Hash grids are widely used to learn an implicit neural field for Gaussian splatting, serving either as part of the
entropy model or for inter-frame prediction. However, due to the irregular and non-uniform distribution of Gaussian splats in 3D space, numerous sparse regions exist, rendering many features in the hash grid invalid. This leads to redundant storage and transmission overhead. In this work, we propose a hash grid feature
pruning method that identifies and prunes invalid features based on the coordinates of the input Gaussian splats, so that only the valid features are encoded. This approach reduces the storage size of the hash grid without compromising model performance, leading to improved rate-distortion performance. Following the Common Test Conditions (CTC) defined by the standardization committee, our method achieves an average bitrate reduction of 8% compared to the baseline approach.
Read the full abstract ▾
All authors Yangzhi Ma, Bojun Liu, Jie Li, Li Li, Dong Liu
Computer Vision
Guided Path Sampling: Steering Diffusion Models Back on Track with Principled Path Guidance
Haosen Li, Wenshuo Chen, Shaofeng Liang et al. (+3 more)
Iterative refinement methods based on a denoising-inversion cycle are powerful tools for enhancing the quality and control of
diffusion models. However, their effectiveness is critically limited when combined with standard Classifier-Free Guidance (CFG). We identify a fundamental limitation: CFG's extrapolative nature systematically pushes the sampling path off the data
manifold, causing the approximation error to diverge and undermining the refinement process. To address this, we propose Guided Path Sampling (GPS), a new paradigm for iterative refinement. GPS replaces unstable extrapolation with a principled,
manifold-constrained interpolation, ensuring the sampling path remains on the data
manifold. We theoretically prove that this correction transforms the error series from unbounded amplification to strictly bounded, guaranteeing stability. Furthermore, we devise an optimal scheduling strategy that dynamically adjusts guidance strength, aligning semantic injection with the model's natural coarse-to-fine generation process. Extensive experiments on modern backbones like SDXL and Hunyuan-DiT show that GPS outperforms existing methods in both perceptual quality and complex
prompt adherence. For instance, GPS achieves a superior ImageReward of 0.79 and HPS v2 of 0.2995 on SDXL, while improving overall semantic
alignment accuracy on GenEval to 57.45%. Our work establishes that path stability is a prerequisite for effective iterative refinement, and GPS provides a robust framework to achieve it.
Read the full abstract ▾
All authors Haosen Li, Wenshuo Chen, Shaofeng Liang, Lei Wang, Haozhe Jia, Yutao Yue
Computer Vision
SwinTF3D: A Lightweight Multimodal Fusion Approach for Text-Guided 3D Medical Image Segmentation
Hasan Faraz Khan, Noor Fatima and Muzammil Behzad
The recent integration of artificial intelligence into medical imaging has driven remarkable advances in automated organ
segmentation. However, most existing 3D
segmentation frameworks rely exclusively on visual learning from large annotated
datasets restricting their adaptability to new domains and clinical tasks. The lack of semantic understanding in these models makes them ineffective in addressing flexible, user-defined
segmentation objectives. To overcome these limitations, we propose SwinTF3D, a lightweight
multimodal fusion approach that unifies visual and linguistic representations for text-guided 3D medical image
segmentation. The model employs a
transformer-based visual encoder to extract volumetric features and integrates them with a compact text encoder via an efficient fusion mechanism. This design allows the system to understand natural-
language prompts and correctly align semantic cues with their corresponding spatial structures in medical volumes, while producing accurate, context-aware
segmentation results with low computational overhead. Extensive experiments on the BTCV
dataset demonstrate that SwinTF3D achieves competitive Dice and IoU scores across multiple organs, despite its compact architecture. The model generalizes well to unseen data and offers significant efficiency gains compared to conventional
transformer-based
segmentation networks. Bridging visual perception with linguistic understanding, SwinTF3D establishes a practical and interpretable paradigm for interactive, text-driven 3D medical image
segmentation, opening perspectives for more adaptive and resource-efficient solutions in clinical imaging.
Read the full abstract ▾
Computer Vision
M-ErasureBench: A Comprehensive Multimodal Evaluation Benchmark for Concept Erasure in Diffusion Models
Ju-Hsuan Weng, Jia-Wei Liao, Cheng-Fu Chou et al. (+1 more)
Text-to-image
diffusion models may generate harmful or copyrighted content, motivating research on concept erasure. However, existing approaches primarily focus on erasing concepts from text
prompts, overlooking other input modalities that are increasingly critical in real-world applications such as image editing and personalized generation. These modalities can become attack surfaces, where erased concepts re-emerge despite defenses. To bridge this gap, we introduce M-ErasureBench, a novel
multimodal evaluation framework that systematically
benchmarks concept erasure methods across three input modalities: text
prompts, learned
embeddings, and inverted latents. For the latter two, we evaluate both white-box and black-box access, yielding five evaluation scenarios. Our analysis shows that existing methods achieve strong erasure performance against text
prompts but largely fail under learned
embeddings and inverted latents, with Concept Reproduction Rate (CRR) exceeding 90% in the white-box setting. To address these vulnerabilities, we propose IRECE (Inference-time Robustness Enhancement for Concept Erasure), a plug-and-play module that localizes target concepts via cross-
attention and perturbs the associated latents during denoising. Experiments demonstrate that IRECE consistently restores robustness, reducing CRR by up to 40% under the most challenging white-box latent inversion scenario, while preserving visual quality. To the best of our knowledge, M-ErasureBench provides the first comprehensive
benchmark of concept erasure beyond text
prompts. Together with IRECE, our
benchmark offers practical safeguards for building more reliable protective generative models.
Read the full abstract ▾
All authors Ju-Hsuan Weng, Jia-Wei Liao, Cheng-Fu Chou, Jun-Cheng Chen
Computer Vision
Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples
Weiwei Li, Junzhuo Liu, Yuanyuan Ren et al. (+3 more)
Deep learning models are known to often learn features that spuriously correlate with the class label during training but are irrelevant to the prediction task. Existing methods typically address this issue by annotating potential spurious attributes, or filtering spurious features based on some empirical assumptions (e.g., simplicity of bias). However, these methods may yield unsatisfactory performance due to the intricate and elusive nature of spurious correlations in real-world data. In this paper, we propose a data-oriented approach to mitigate the spurious correlation in deep learning models. We observe that samples that are influenced by spurious features tend to exhibit a dispersed distribution in the learned feature space. This allows us to identify the presence of spurious features. Subsequently, we obtain a bias-invariant representation by neutralizing the spurious features based on a simple grouping strategy. Then, we learn a feature transformation to eliminate the spurious features by aligning with this bias-invariant representation. Finally, we update the classifier by incorporating the learned feature transformation and obtain an unbiased model. By integrating the aforementioned identifying, neutralizing, eliminating and updating procedures, we build an effective pipeline for mitigating spurious correlation. Experiments on image and NLP debiasing
benchmarks show an improvement in worst group accuracy of more than 20% compared to standard empirical risk minimization (ERM). Codes and checkpoints are available at https://github.com/davelee-uestc/nsf_debiasing .
Read the full abstract ▾
All authors Weiwei Li, Junzhuo Liu, Yuanyuan Ren, Yuchen Zheng, Yahao Liu, Wen Li
Computer Vision
Lamps: Learning Anatomy from Multiple Perspectives via Self-supervision in Chest Radiographs
Ziyu Zhou, Haozhe Luo, Mohammad Reza Hosseinzadeh Taher et al. (+4 more)
Foundation models have been successful in natural
language processing and computer
vision because they are capable of capturing the underlying structures (foundation) of natural
languages. However, in medical imaging, the key foundation lies in human anatomy, as these images directly represent the internal structures of the body, reflecting the consistency, coherence, and hierarchy of human anatomy. Yet, existing self-supervised learning (SSL) methods often overlook these perspectives, limiting their ability to effectively learn anatomical features. To overcome the limitation, we built Lamps (learning anatomy from multiple perspectives via self-supervision) pre-trained on large-scale chest radiographs by harmoniously utilizing the consistency, coherence, and hierarchy of human anatomy as the supervision signal. Extensive experiments across 10
datasets evaluated through fine-tuning and emergent property analysis demonstrate Lamps' superior robustness, transferability, and clinical potential when compared to 10 baseline models. By learning from multiple perspectives, Lamps presents a unique opportunity for foundation models to develop meaningful, robust representations that are aligned with the structure of human anatomy.
Read the full abstract ▾
All authors Ziyu Zhou, Haozhe Luo, Mohammad Reza Hosseinzadeh Taher, Jiaxuan Pang, Xiaowei Ding, Michael B. Gotway, Jianming Liang
Computer Vision
Zhuonan Liu, Xinyu Zhang, Zishuo Wang et al. (+8 more)
Socially compliant
navigation requires structured
reasoning about dynamic pedestrians and physical constraints to ensure safe and interpretable decisions.
Vision-
language models (VLMs) provide a promising foundation for this task because they can integrate visual observations with
language-based social knowledge. However, existing untuned VLMs still struggle to reliably understand fine-grained social norms, making task-specific fine-tuning essential. At the same time, no large-scale
egocentric dataset is available this task. To address these challenges, we introduce MUSON, a
multimodal dataset for short-horizon social
navigation containing 10,110
egocentric samples collected across diverse indoor and outdoor social scenes. MUSON adopts a structured five-step chain-of-thought annotation framework comprising perception, prediction,
reasoning, action, and explanation. It explicitly models static physical constraints and employs a standardized six-action decision space. Compared with existing social-
navigation datasets, MUSON provides consistent annotations for
reasoning, actions, and explanations. We evaluate ten representative small-to-medium VLMs on MUSON. Qwen3-VL-8B achieves the strongest decision-level performance, attaining the highest action accuracy of 0.7765 and Macro-F1 score of 0.7490, as well as the lowest collision rate of 0.0609. These results demonstrate that MUSON is an effective and reusable
benchmark for advancing socially compliant
navigation. The
dataset is publicly available at https://github.com/MUSON-
dataset/MUSON/releases/tag/v1.0.
Read the full abstract ▾
All authors Zhuonan Liu, Xinyu Zhang, Zishuo Wang, Runji Cai, Tomohito Kawabata, Qianyi Li, Xuance Peng, Tianze Yu, Zhen Xiong, Xuesu Xiao, Ling Xiao
Computer Vision · Cross-listed from physics.geo-ph
A Rapid GeoSAM-Based Workflow for Multi-Temporal Glacier Delineation: Case Study from Svalbard
Alexandru Hegyi
Consistent glacier boundary delineation is essential for monitoring glacier change, yet many existing approaches are difficult to scale across long time series and heterogeneous environments. In this report, we present a GeoSAM-based, semi-automatic workflow for rapid glacier delineation from Sentinel-2 surface reflectance imagery. The method combines late-summer image compositing, spectral-index-based identification of candidate ice areas,
prompt-guided
segmentation using GeoSAM, and physically based post-processing to derive annual glacier outlines. The workflow is demonstrated in the Ny-Alesund and Kongsfjorden region of western Svalbard across multiple years of the Sentinel-2 era. Results show that the approach produces spatially coherent and temporally consistent outlines for major glacier bodies, while most errors are associated with small features affected by water bodies, terrain shadows, or high surface variability. The reliance on derived RGB imagery makes the method flexible and transferable to other optical
datasets, with improved performance expected at higher spatial resolution. Although user inspection remains necessary to filter incorrect polygons and adjust thresholds for local conditions, the workflow provides a fast and practical alternative for multi-temporal glacier mapping and ice-loss assessment.
Read the full abstract ▾
Computer Vision
ByteLoom: Weaving Geometry-Consistent Human-Object Interactions through Progressive Curriculum Learning
Bangya Liu, Xinyu Gong, Zelin Zhao et al. (+6 more)
Human-object interaction (HOI)
video generation has garnered increasing
attention due to its promising applications in digital humans, e-commerce, advertising, and robotics imitation learning. However, existing methods face two critical limitations: (1) a lack of effective mechanisms to inject multi-view information of the object into the model, leading to poor cross-view consistency, and (2) heavy reliance on fine-grained hand mesh annotations for modeling interaction occlusions. To address these challenges, we introduce ByteLoom, a
Diffusion Transformer (DiT)-based framework that generates realistic HOI
videos with geometrically consistent object illustration, using simplified human conditioning and 3D object inputs. We first propose an RCM-cache mechanism that leverages Relative Coordinate Maps (RCM) as a universal representation to maintain object's geometry consistency and precisely control 6-DoF object transformations in the meantime. To compensate HOI
dataset scarcity and leverage existing
datasets, we further design a training curriculum that enhances model capabilities in a progressive style and relaxes the demand of hand mesh. Extensive experiments demonstrate that our method faithfully preserves human identity and the object's multi-view geometry, while maintaining smooth motion and object manipulation.
Read the full abstract ▾
All authors Bangya Liu, Xinyu Gong, Zelin Zhao, Ziyang Song, Yulei Lu, Suhui Wu, Jun Zhang, Suman Banerjee, Hao Zhang
Computer Vision
A Minimal Solver for Relative Pose Estimation with Unknown Focal Length from Two Affine Correspondences
Zhenbao Yu, Shirong Ye, Ronghe Jin et al. (+4 more)
In this paper, we aim to estimate the relative pose and focal length between two views with known intrinsic parameters except for an unknown focal length from two affine correspondences (ACs). Cameras are commonly used in combination with inertial measurement units (IMUs) in applications such as self-driving cars, smartphones, and unmanned aerial vehicles. The vertical direction of camera views can be obtained by IMU measurements. The relative pose between two cameras is reduced from 5DOF to 3DOF. We propose a new solver to estimate the 3DOF relative pose and focal length. First, we establish constraint equations from two affine correspondences when the vertical direction is known. Then, based on the properties of the equation system with nontrivial solutions, four equations can be derived. These four equations only involve two parameters: the focal length and the relative rotation angle. Finally, the
polynomial eigenvalue method is utilized to solve the problem of focal length and relative rotation angle. The proposed solver is evaluated using synthetic and real-world
datasets. The results show that our solver performs better than the existing state-of-the-art solvers.
Read the full abstract ▾
All authors Zhenbao Yu, Shirong Ye, Ronghe Jin, Shunkun Liang, Zibin Liu, Huiyun Zhang, Banglei Guan
Computer Vision
3D Scene Change Modeling With Consistent Multi-View Aggregation
Zirui Zhou, Junfeng Ni, Shujie Zhang et al. (+2 more)
Change detection plays a vital role in scene monitoring, exploration, and continual reconstruction. Existing 3D change detection methods often exhibit spatial inconsistency in the detected changes and fail to explicitly separate pre- and post-change states. To address these limitations, we propose SCaR-3D, a novel 3D scene change detection framework that identifies object-level changes from a dense-view pre-change image sequence and sparse-view post-change images. Our approach consists of a signed-distance-based 2D differencing module followed by multi-view aggregation with voting and
pruning, leveraging the consistent nature of 3DGS to robustly separate pre- and post-change states. We further develop a continual scene reconstruction strategy that selectively updates dynamic regions while preserving the unchanged areas. We also contribute CCS3D, a challenging synthetic
dataset that allows flexible combinations of 3D change types to support controlled evaluations. Extensive experiments demonstrate that our method achieves both high accuracy and efficiency, outperforming existing methods.
Read the full abstract ▾
All authors Zirui Zhou, Junfeng Ni, Shujie Zhang, Yixin Chen, Siyuan Huang
Computer Vision
KANO: Kolmogorov-Arnold Neural Operator for Image Super-Resolution
Chenyu Li, Danfeng Hong, Bing Zhang et al. (+2 more)
The highly nonlinear degradation process, complex physical interactions, and various sources of uncertainty render single-image Super-resolution (SR) a particularly challenging task. Existing interpretable SR approaches, whether based on prior learning or deep unfolding optimization frameworks, typically rely on black-box deep networks to model latent variables, which leaves the degradation process largely unknown and uncontrollable. Inspired by the Kolmogorov-Arnold
theorem (KAT), we for the first time propose a novel interpretable operator, termed Kolmogorov-Arnold Neural Operator (KANO), with the application to image SR. KANO provides a transparent and structured representation of the latent degradation fitting process. Specifically, we employ an additive structure composed of a finite number of B-spline functions to approximate continuous spectral curves in a piecewise fashion. By learning and optimizing the shape parameters of these spline functions within defined intervals, our KANO accurately captures key spectral characteristics, such as local linear trends and the peak-valley structures at nonlinear inflection points, thereby endowing SR results with physical interpretability. Furthermore, through theoretical modeling and experimental evaluations across natural images, aerial photographs, and satellite remote sensing data, we systematically compare multilayer perceptrons (MLPs) and Kolmogorov-Arnold networks (KANs) in handling complex sequence fitting tasks. This comparative study elucidates the respective advantages and limitations of these models in characterizing intricate degradation mechanisms, offering valuable insights for the development of interpretable SR techniques.
Read the full abstract ▾
All authors Chenyu Li, Danfeng Hong, Bing Zhang, Zhaojie Pan, Jocelyn Chanussot