Computer Vision
Holo360D: A Large-Scale Real-World Dataset with Continuous Trajectories for Advancing Panoramic 3D Reconstruction and Beyond
Jing Ou, Zidong Cao, Yinrui Ren et al. (+6 more)
While feed-forward 3D reconstruction models have advanced rapidly, they still exhibit degraded performance on panoramas due to spherical distortions. Moreover, existing panoramic 3D
datasets are predominantly collected with 360 cameras fixed at discrete locations, resulting in discontinuous trajectories. These limitations critically hinder the development of panoramic feed-forward 3D reconstruction, especially for the multi-view setting. In this paper, we present Holo360D, a comprehensive
dataset containing 109,495 panoramas paired with registered point clouds, meshes, and aligned camera poses. To our knowledge, Holo360D is the first large-scale
dataset that provides continuous panoramic sequences with accurately aligned high-completeness depth maps. The raw data are initially collected using a 3D
laser scanner coupled with a 360 camera. Subsequently, the raw data are processed with both online and offline SLAM systems. Furthermore, to enhance the 3D data quality, a post-processing pipeline tailored for the 360
dataset is proposed, including geometry denoising, mesh hole filling, and region-specific remeshing. Finally, we establish a new
benchmark by fine-tuning 3D reconstruction models on Holo360D, providing key insights into effective fine-tuning strategies. Our results demonstrate that Holo360D delivers superior training signals and provides a comprehensive
benchmark for advancing panoramic 3D reconstruction models.
Datasets and Code will be made publicly available.
Read the full abstract ▾
All authors Jing Ou, Zidong Cao, Yinrui Ren, Zhuoxiao Li, Jinjing Zhu, Tongyan Hua, Shuai Zhang, Hui Xiong, Wufan Zhao
Computer Vision
Improving Driver Drowsiness Detection via Personalized EAR/MAR Thresholds and CNN-Based Classification
Gökdeniz Ersoy, Mehmet Alper Tatar, Eray Tonbul et al. (+1 more)
Driver drowsiness is a major cause of traffic accidents worldwide, posing a serious threat to public safety.
Vision-based driver monitoring systems often rely on fixed Eye Aspect Ratio (EAR) and Mouth Aspect Ratio (MAR) thresholds; however, such fixed values frequently fail to generalize across individuals due to variations in facial structure, illumination, and driving conditions. This paper proposes a personalized driver drowsiness detection system that monitors eyelid movements, head position, and yawning behavior in real time and provides warnings when signs of fatigue are detected. The system employs driver-specific EAR and MAR thresholds, calibrated before driving, to improve classical metric-based detection. In addition, deep learning-based Convolutional Neural Network (CNN) models are integrated to enhance accuracy in challenging scenarios. The system is evaluated using publicly available
datasets as well as a custom
dataset collected under diverse lighting conditions, head poses, and user characteristics. Experimental results show that personalized thresholding improves detection accuracy by 2-3% compared to fixed thresholds, while CNN-based classification achieves 99.1% accuracy for eye state detection and 98.8% for yawning detection, demonstrating the effectiveness of combining classical metrics with deep learning for robust real-time driver monitoring.
Read the full abstract ▾
All authors Gökdeniz Ersoy, Mehmet Alper Tatar, Eray Tonbul, Serap K{\i}rb{\i}z
Computer Vision
Contrastive Semantic Projection: Faithful Neuron Labeling with Contrastive Examples
Oussama Bouanani, Jim Berend, Wojciech Samek et al. (+2 more)
Neuron labeling assigns textual descriptions to internal units of deep networks. Existing approaches typically rely on highly activating examples, often yielding broad or misleading labels by focusing on dominant but incidental visual factors. Prior work such as FALCON introduced contrastive examples – inputs that are semantically similar to activating examples but elicit low activations – to sharpen explanations, but it primarily addresses subspace-level interpretability rather than scalable
neuron-level labeling. We revisit contrastive explanations for
neuron-level labeling in two stages: (1) candidate label generation with
vision language models (VLMs) and (2) label assignment with CLIP-like encoders. First, we show that providing contrastive image sets to VLMs yields candidate labels that are more specific and more faithful. Second, we introduce Contrastive Semantic Projection (CSP), an extension of SemanticLens that incorporates contrastive examples directly into its CLIP-based scoring and selection pipeline. Across extensive experiments and a case study on melanoma detection, contrastive labeling improves both faithfulness and semantic granularity over state-of-the-art baselines. Our results demonstrate that contrastive examples are a simple yet powerful and currently underutilized component of
neuron labeling and analysis pipelines.
Read the full abstract ▾
All authors Oussama Bouanani, Jim Berend, Wojciech Samek, Sebastian Lapuschkin, Maximilian Dreyer
Computer Vision
All Eyes on the Workflow: Automated and Efficient Event Discovery from Video Streams
Marco Pegoraro, Jonas Seng, Dustin Heller et al. (+2 more)
Disciplines such as business process management and process mining aid organizations by discovering insights about processes on the basis of recorded event data. However, an obstacle to process analysis is data multi-modality: for instance, data in
video form are not directly interpretable as events. Existing approaches rely on a dictionary of activity label as input, cannot provide frame-by-frame labeling explanations, or rely on superseded computer
vision techniques. In this work, we present SnapLog, an approach to extract event data from
videos by converting frames to feature vectors using image
embeddings and performing temporal
segmentation through frame-wise similarity
matrices. A generalized few-shot classification is then used to assign labels to the
video segments, yielding labeled, timestamped sub-sequences of frames that are interpretable as events. Conventional process mining techniques can be used to analyze the resulting data. We show that our approach produces logs that accurately reflect the process in the
videos.
Read the full abstract ▾
All authors Marco Pegoraro, Jonas Seng, Dustin Heller, Wil M.P. van der Aalst, Kristian Kersting
Computer Vision
NRGS: Neural Regularization for Robust 3D Semantic Gaussian Splatting
Zaiyan Yang, Xinpeng Liu, Heng Guo et al. (+3 more)
We propose a neural regularization method that refines the noisy 3D semantic field produced by lifting multi-view inconsistent 2D features, in order to obtain an accurate and robust 3D semantic Gaussian Splatting. The 2D features extracted from
vision foundation models suffer from multi-view inconsistency due to a lack of cross-view constraints. Lifting these inconsistent features directly into 3D Gaussians results in a noisy semantic field, which degrades the performance of downstream tasks. Previous methods either focus on obtaining consistent multi-view features in the preprocessing stage or aim to mitigate noise through improved optimization strategies, often at the cost of increased preprocessing time or expensive computational overhead. In contrast, we introduce a variance-aware conditional MLP that operates directly on the 3D Gaussians, leveraging their
geometric and appearance attributes to correct semantic errors in 3D space. Experiments on different
datasets show that our method enhances the accuracy of lifted semantics, providing an efficient and effective approach to robust 3D semantic Gaussian Splatting.
Read the full abstract ▾
All authors Zaiyan Yang, Xinpeng Liu, Heng Guo, Jinglei Shi, Zhanyu Ma, Fumio Okura
Computer Vision
SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments
Chih-Ting Liao, Xi Xiao, Chunlei Meng et al. (+6 more)
Multimodal large
language models (MLLMs) have advanced static visual–spatial
reasoning, yet they often fail to preserve long-horizon spatial coherence in
embodied settings where beliefs must be continuously revised from
egocentric observations under environmental change. We introduce SpaMEM (Spatial
Memory from Action Sequences), a large-scale diagnostic
benchmark that isolates the mechanics of spatial belief
evolution via action-conditioned scene transformations (spawn, place, remove) over long interaction horizons. SpaMEM is built on a physically grounded
dataset with 10,601,392 high-fidelity images across four modalities (RGB, depth, instance, semantic
segmentation), collected from 25,000+ interaction sequences in 1,000 procedurally generated houses. We formalize
embodied spatial
reasoning as a three-level hierarchy with 15 diagnostic tasks: Level 1 measures atomic spatial perception from single observations; Level 2 probes temporal
reasoning with oracle textual state histories to factor out perceptual noise; and Level 3 requires end-to-end belief maintenance from raw visual streams under the same task dimensions. We further evaluate both short-term (step-wise) updates and long-term (episodic) reconstruction. Benchmarking representative open-source VLM families reveals a consistent stacked bottleneck: coordinate-consistent grounding remains a hard ceiling, and the sharp collapse from Level 2 to Level 3 exposes a pronounced symbolic scaffolding dependency, where models succeed with text-based bookkeeping but struggle to sustain robust visual
memory. SpaMEM provides a granular diagnostic standard and motivates explicit mechanisms for state representation, belief revision, and long-horizon episodic integration. A subset of SpaMEM is publicly available at https://huggingface.co/
datasets/mill-ct-liao/SpaMEM.
Read the full abstract ▾
All authors Chih-Ting Liao, Xi Xiao, Chunlei Meng, Zhangquan Chen, Yitong Qiao, Weilin Zhou, Tianyang Wang, Xu Zheng, Xin Cao
Computer Vision
Region Matters: Efficient and Reliable Region-Aware Visual Place Recognition
Shunpeng Chen, Yukun Song, Changwei Wang et al. (+6 more)
Visual Place Recognition (VPR) determines a query image's geographic location by matching it against geotagged databases. However, existing methods struggle with perceptual aliasing caused by irrelevant regions and inefficient re-ranking due to rigid candidate scheduling. To address these issues, we introduce FoL++, a method combining robust discriminative region modeling with adaptive re-ranking. Specifically, we propose a Reliability Estimation Branch to generate spatial reliability maps that explicitly model occlusion resistance. This representation is further optimized by two spatial
alignment losses (SAL and SCEL) to effectively align features and highlight salient regions. For weakly supervised learning without manual annotations, a pseudo-correspondence strategy generates dense local feature supervision directly from aggregation clusters. Our Adaptive Candidate Scheduler dynamically resizes candidate pools based on global similarity. By weighting local matches by reliability and adaptively fusing global and local evidence, FoL++ surpasses traditional independent matching systems. Extensive experiments across seven
benchmarks demonstrate that FoL++ achieves state-of-the-art performance with a lightweight
memory footprint, improving inference speed by 40% over FoL. Code and models will be released (and merged with FoL) at https://github.com/chenshunpeng/FoL.
Read the full abstract ▾
All authors Shunpeng Chen, Yukun Song, Changwei Wang, Rongtao Xu, Kexue Fu, Longxiang Gao, Li Guo, Ruisheng Wang, Shibiao Xu
Computer Vision
HFS-TriNet: A Three-Branch Collaborative Feature Learning Network for Prostate Cancer Classification from TRUS Videos
Xu Lu, Qianhong Peng, Qihao Zhou et al. (+4 more)
Transrectal ultrasound (TRUS) imaging is a cost-effective and non-invasive modality widely used in the diagnosis of prostate cancer. The computer-aided diagnosis (CAD) relying on TRUS images has been extensively investigated recently. Compared to static images, TRUS
video provides richer spatial-temporal information, which make it a promising alternative for improving the accuracy and robustness of CAD systems. However, TRUS
video analysis also introduces new challenges. These include information redundancy, which increases computational costs; high intra- and inter-class similarity, which complicates feature extraction; and a low signal-to-noise ratio, which hinders the identification of clinically relevant information. To address these problems, we propose a heuristic frame selection (HFS) and a three-branch collaborative feature learning network (HFS-TriNet) for prostate cancer classification from TRUS
videos. Specifically, selecting a clip of
video frames at intervals for training can mitigate redundancy. The HFS strategy dynamically initializes the starting point of each training clip, which ensures that the sampled clips span the entire
video sequence. For better feature extraction, besides a regular ResNet50 branch, we also utilize 1) a large model branch based a pre-trained medical segment anything model (SAM) to extract deep features of each frame and a normalization-based
attention module to explore the temporal consistency; and 2) a wavelet transform convolutional residual (WTCR) branch that extracts lesion edge information in the high-frequency domain and performs denoising in the low-frequency domain.
Read the full abstract ▾
All authors Xu Lu, Qianhong Peng, Qihao Zhou, Shaopeng Liu, Xiuqin Ye, Chuan Yang, Yuan Yuan
Computer Vision
Jincheng Ying, Yitao Chen, Li Wenlin et al. (+2 more)
Recent advances in distilling expensive
diffusion models into efficient few-step generators show significant promise. However, these methods typically demand substantial computational resources and extended training periods, limiting accessibility for resource-constrained researchers, and existing supplementary loss functions have notable limitations. Regression loss requires pre-generating large
datasets before training and limits the student model to the teacher's performance, while GAN-based losses suffer from training instability and require careful tuning. In this paper, we propose
Embedding Loss (EL), a novel supplementary loss function that complements existing
diffusion distillation methods to enhance generation quality and accelerate training with smaller batch sizes. Leveraging feature
embeddings from a diverse set of randomly initialized networks, EL effectively aligns the feature distributions between the distilled few-step generator and the original data. By computing Maximum Mean Discrepancy (MMD) in the embedded feature space, EL ensures robust distribution matching, thereby preserving sample fidelity and diversity during
distillation. Within distribution matching
distillation frameworks, EL demonstrates strong empirical performance for one-step generators. On the CIFAR-10
dataset, our approach achieves state-of-the-art FID values of 1.475 for unconditional generation and 1.380 for conditional generation. Beyond CIFAR-10, we further validate EL across multiple
benchmarks and
distillation methods, including ImageNet, AFHQ-v2, and FFHQ
datasets, using DMD, DI, and CM
distillation frameworks, demonstrating consistent improvements over existing one-step
distillation methods. Our method also reduces training iterations by up to 80%, offering a more practical and scalable solution for deploying
diffusion-based generative models in resource-constrained environments.
Read the full abstract ▾
All authors Jincheng Ying, Yitao Chen, Li Wenlin, Minghui Xu, Yinhao Xiao
Computer Vision
One Shot Learning for Edge Detection on Point Clouds
Zhikun Tu, Yuhe Zhang, Yiou Jia et al. (+2 more)
Each scanner possesses its unique characteristics and exhibits its distinct sampling error distribution. Training a network on a
dataset that includes data collected from different scanners is less effective than training it on data specific to a single scanner. Therefore, we present a novel one-shot learning method allowing for edge extraction on point clouds, by learning the specific data distribution of the target point cloud, and thus achieve superior results compared to networks that were trained on general data distributions. More specifically, we present how to train a lightweight network named OSFENet (One-Shot edge Feature Extraction Network), by designing a filtered-KNN-based surface patch representation that supports a one-shot learning framework. Additionally, we introduce an RBF_DoS module, which integrates Radial Basis Function-based Descriptor of the Surface patch, highly beneficial for the edge extraction on point clouds. The advantage of the proposed OSFENet is demonstrated through comparative analyses against 7 baselines on the ABC
dataset, and its practical utility is validated by results across diverse real-scanned
datasets, including indoor scenes like S3DIS
dataset, and outdoor scenes such as the Semantic3D
dataset and UrbanBIS
dataset.
Read the full abstract ▾
All authors Zhikun Tu, Yuhe Zhang, Yiou Jia, Kang Li, Daniel Cohen-Or
Computer Vision · Cross-listed from astro-ph.IM
Thermal background reduction for mid-infrared imaging by low-rank background and sparse point-source modelling
R.A.R. Moens, A.G.M. Pietrow, B. Brandl et al. (+1 more)
Mid-infrared astronomy from the ground faces critical challenges in accurately detecting and quantifying sources due to the dominant spatially and time-variable background noise. Moreover, chopping and nodding, the traditional methods for dealing with these background issues, will not be technically feasible on the next generation of extremely large
telescopes. This limitation requires the development of novel computational methods for a robust background reduction. We present and evaluate a novel method named LOw-RAnk Background ELimination (LORABEL) to improve the sensitivity of mid-infrared astronomical observations, without the need for classical
telescope nodding, source masking, or other overheads in observing time. We applied a low-rank background-reduction strategy to (1) data taken on the ground with the VISIR with synthetically injected sources, and (2) airborne data from SOFIA. We compared the performance of our new method to classical chopping and nodding techniques, and analysed the effect on source photometry and detection precision for different observational scenarios. In regimes with a low signal-to-noise ratio (S/N $<5$) in the ground-based VISIR data, LORABEL reduces variation in the photometric error with respect to chopping differences alone and even the classical chop-nod sequence, at the cost of introducing a bias. Secondly, we demonstrate that LORABEL increases detection precision in comparison to traditional background-reduction methods. For the SOFIA
dataset, we achieve a $20-100$ fold decrease in mean background flux with respect to the traditional chop-nod method while preserving most of the source flux. Our findings suggest that LORABEL is applicable to a wider range of instrumental observation, that is, both ground-based and airborne, and it is a suitable tool in the context of faint-source detection.
Read the full abstract ▾
All authors R.A.R. Moens, A.G.M. Pietrow, B. Brandl, R. Van de Plas
Computer Vision
PoseFM: Relative Camera Pose Estimation Through Flow Matching
Dominik Kuczkowski and Laura Ruotsalainen
Monocular visual odometry (VO) is a fundamental computer
vision problem with applications in autonomous
navigation, augmented reality and more. While deep learning-based methods have recently shown superior accuracy compared to traditional
geometric pipelines, particularly in environments where handcrafted features struggle due to poor structure or lighting conditions, most rely on deterministic regression, which lacks the uncertainty awareness required for robust applications. We propose PoseFM, the first framework to reformulate
monocular frame-to-frame VO as a generative task using Flow Matching (FM). By leveraging FM, we model camera motion as a distribution rather than a point estimate, learning to transform noise into realistic pose predictions via continuous-time ODEs. This approach provides a principled mechanism for uncertainty estimation and enables robust motion inference under challenging visual conditions. In our evaluations, PoseFM achieves strong performance on TartanAir, KITTI and TUM-RGBD
benchmarks, achieving the lowest absolute trajectory error (ATE) on some of the trajectories and overall being competitive with the best frame-to-frame
monocular VO methods. Code and model checkpoints will be made available at https://github.com/helsinki-sda-group/posefm.
Read the full abstract ▾
Computer Vision
Flow4DGS-SLAM: Optical Flow-Guided 4D Gaussian Splatting SLAM
Yunsong Wang and Gim Hee Lee
Handling the dynamic environments is a significant research challenge in Visual Simultaneous Localization and Mapping (SLAM). Recent research combines 3D Gaussian Splatting (3DGS) with SLAM to achieve both robust camera pose estimation and photorealistic renderings. However, using SLAM to efficiently reconstruct both static and dynamic regions remains challenging. In this work, we propose an efficient framework for dynamic 3DGS SLAM guided by optical flow. Using the input depth and prior optical flow, we first propose a category-agnostic motion mask generation strategy by fitting a camera ego-motion model to decompose the optical flow. This module separates dynamic and static Gaussians and simultaneously provides flow-guided camera pose initialization. We boost the training speed of dynamic 3DGS by explicitly modeling their temporal centers at keyframes. These centers are propagated using 3D scene flow priors and are dynamically initialized with an adaptive insertion strategy. Alongside this, we model the temporal opacity and rotation using a Gaussian Mixture Model (GMM) to adaptively learn the complex dynamics. The empirical results demonstrate our state-of-the-art performance in tracking, dynamic reconstruction, and training efficiency.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Selective Depthwise Separable Convolution for Lightweight Joint Source-Channel Coding in Wireless Image Transmission
Ming Ye, Kui Cai, Cunhua Pan et al. (+3 more)
Depthwise separable convolutional (DSConv) layers have been successfully applied to deep learning (DL)-based joint source-channel coding (JSCC) schemes to reduce computational complexity. However, a systematic investigation of the layerwise and ratio-wise replacement of standard convolutional (Conv) layers with DSConv layers in JSCC systems for wireless image transmission remains largely unexplored. In this letter, we propose a configurable lightweight JSCC framework that incorporates a selective replacement strategy, enabling flexible Conv-to-DSConv replacement at different replacement ratios and positions. By varying the replacement ratio, we obtain models with different computational complexities and analyze their impact on reconstruction performance. Furthermore, we investigate how replacements at different encoder and decoder depths influence reconstruction quality under a fixed replacement ratio. Our results show that Conv-to-DSConv replacement at the intermediate layers of the encoder and decoder achieves a favorable complexity-performance trade-off, revealing layer-wise redundancy in DL-based JSCC systems. Extensive experiments further demonstrate that the proposed framework achieves substantial parameter reduction with only slight performance degradation, enabling flexible complexity-performance trade-offs for resource-constrained edge devices.
Read the full abstract ▾
All authors Ming Ye, Kui Cai, Cunhua Pan, Zhen Mei, Wanting Yang, Chunguo Li
Computer Vision
FILTR: Extracting Topological Features from Pretrained 3D Models
Louis Martinez and Maks Ovsjanikov
Recent advances in pretraining 3D point cloud encoders (e.g., Point-BERT, Point-MAE) have produced powerful models, whose abilities are typically evaluated on
geometric or semantic tasks. At the same time, topological descriptors have been shown to provide informative summaries of a shape's multiscale structure. In this paper we pose the question whether topological information can be derived from features produced by 3D encoders. To address this question, we first introduce DONUT, a synthetic
benchmark with controlled topological complexity, and propose FILTR (Filtration
Transformer), a learnable framework to predict persistence diagrams directly from frozen encoders. FILTR adapts a
transformer decoder to treat diagram generation as a set prediction task. Our analysis on DONUT reveals that existing encoders retain only limited global topological signals, yet FILTR successfully leverages information produced by these encoders to approximate persistence diagrams. Our approach enables, for the first time, data-driven extraction of persistence diagrams from raw point clouds through an efficient learnable feed-forward mechanism.
Read the full abstract ▾
Computer Vision
ChangeQuery: Advancing Remote Sensing Change Analysis for Natural and Human-Induced Disasters from Visual Detection to Semantic Understanding
Dongwei Sun, Jing Yao, Kan Wei et al. (+6 more)
Rapid situational awareness is critical in post-disaster response. While remote sensing damage assessment is evolving from pixel-level change detection to high-level semantic analysis, existing
vision-
language methodologies still struggle to provide actionable intelligence for complex strategic queries. They remain severely constrained by unimodal optical dependence, a prevailing bias towards natural disasters, and a fundamental lack of grounded interactivity. To address these limitations, we present ChangeQuery, a unified
multimodal framework designed for comprehensive, all-weather disaster situation awareness. To overcome modality constraints and scenario biases, we construct the Disaster-Induced Change Query (DICQ)
dataset, a large-scale
benchmark coupling pre-event optical semantics with post-event SAR structural features across a balanced distribution of natural catastrophes and armed conflicts. Furthermore, to provide the high-quality supervision required for interactive
reasoning, we propose a novel Automated Semantic Annotation Pipeline. Adhering to a “statistics-first, generation-later” paradigm, this engine automatically transforms raw
segmentation masks into grounded, hierarchical instruction sets, effectively equipping the model with fine-grained spatial and quantitative awareness. Trained on this structured data, the ChangeQuery architecture operates as an interactive disaster analyst. It supports multi-task
reasoning driven by diverse user queries, delivering precise damage quantification, region-specific descriptions, and holistic post-disaster summaries. Extensive experiments demonstrate that ChangeQuery establishes a new state-of-the-art, providing a robust and interpretable solution for complex disaster monitoring. The code is available at \href{https://sundongwei.github.io/changequery/}{https://sundongwei.github.io/changequery/}.
Read the full abstract ▾
All authors Dongwei Sun, Jing Yao, Kan Wei, Xiangyong Cao, Chen Wu, Zhenghui Zhao, Pedram Ghamisi, Jun Zhou, Jón Atli Benediktsson
Computer Vision
Depth-Aware Rover: A Study of Edge AI and Monocular Vision for Real-World Implementation
Lomash Relia, Jai G Singla, Amitabh et al. (+1 more)
This study analyses simulated and real-world implementations of depth-aware rover
navigation, highlighting the transition from stereo
vision to
monocular depth estimation using edge AI. A Unity-based lunar terrain simulator with stereo cameras and OpenCV's StereoSGBM was used to generate disparity maps. A physical rover built on Raspberry Pi 4 employed UniDepthV2 for
monocular metric depth estimation and YOLO12n for real-time object detection. While stereo
vision yielded higher accuracy in simulation, the
monocular approach proved more robust and cost-effective in real-world deployment, achieving 0.1 FPS for depth and 10 FPS for detection.
Read the full abstract ▾
All authors Lomash Relia, Jai G Singla, Amitabh, Nitant Dube
Computer Vision
Revisiting Geometric Obfuscation with Dual Convergent Lines for Privacy-Preserving Image Queries in Visual Localization
Jeonggon Kim, Heejoon Moon and Je Hyeong Hong
Privacy-Preserving Image Queries (PPIQ) are an emerging mechanism for cloud-based visual localization, enabling pose estimation from obfuscated features instead of private images or raw keypoints. However, the main approaches for PPIQ, primarily geometry-based and
segmentation-based obfuscation, both suffer from vulnerabilities to recent
privacy attacks. In particular, a fundamental limitation of geometry-based obfuscation is that the spatial distribution of obfuscated neighboring lines still effectively surrounds the original keypoint location, providing exploitable cues for recovering the original points. We revisit this
geometric paradigm and introduce Dual Convergent Lines (DCL), a novel keypoint obfuscation method demonstrating strong resilience against such attack. DCL places two fixed anchors on a central partition line and lifts each keypoint to a line originating from one of them, with the active anchor determined by the keypoint's location. This arrangement invalidates the geometry-recovery attack by making its optimization ill-posed: Neighboring lines either misleadingly converge to one anchor, yielding a trivial solution, or become near-parallel at the partition boundary, yielding an unstable high-variance solution. Both outcomes thwart point recovery. DCL is also compatible with an existing line-based solver, enabling deployment in traditional localization pipelines. Experiments on both indoor and large-scale outdoor
datasets demonstrate DCL's robustness against
privacy attacks, efficiency, and scalability, while achieving practical localization performance.
Read the full abstract ▾
Computer Vision
Knowledge Visualization: A Benchmark and Method for Knowledge-Intensive Text-to-Image Generation
Ran Zhao, Sheng Jin, Size Wu et al. (+5 more)
Recent text-to-image (T2I) models have demonstrated impressive capabilities in photorealistic synthesis and instruction following. However, their reliability in knowledge-intensive settings remains largely unexplored. Unlike natural image generation, knowledge visualization requires not only semantic
alignment but also strict adherence to domain knowledge, structural constraints, and symbolic conventions, exposing a critical gap between visual plausibility and scientific correctness. To systematically study this problem, we introduce KVBench, a curriculum-grounded
benchmark for evaluating knowledge-intensive T2I generation. KVBench covers six senior high-school subjects: Biology, Chemistry, Geography, History, Mathematics, and Physics. The
benchmark consists of 1,800 expert-curated
prompts derived from over 30 authoritative textbooks. Using this
benchmark, we evaluate 14 state-of-the-art open- and closed-source models, revealing substantial deficiencies in logical
reasoning, symbolic precision, and
multilingual robustness, with open-source models consistently underperforming proprietary systems. To address these limitations, we further propose KE-Check, a two-stage framework that improves scientific fidelity via (1) Knowledge Elaboration for structured
prompt enrichment, and (2) Checklist-Guided Refinement for explicit constraint enforcement through violation identification and constraint-guided editing. KE-Check effectively mitigates scientific
hallucinations, narrowing the performance gap between open-source and leading closed-source models. Data and codes are publicly available at https://github.com/zhaoran66/KVBench.
Read the full abstract ▾
All authors Ran Zhao, Sheng Jin, Size Wu, Kang Liao, Zerui Gong, Zujin Guo, Yang Xiao, Wei Li
Computer Vision
Evaluation of image simulation open source solutions for simulation of synthetic images in lunar environment
Jai G Singla, Hinal B Patel and Nitant Dube
Synthetic image generation is one of the crucial input for planetary missions. It enables researchers and engineers to visualize planned planetary missions, test imaging systems and plan exploration activities in a virtual environment before actual deployment. Image simulation is essential for assessing landing sites, detecting hazards, and validating
navigation systems in a missions. This study offers a detailed evaluation of various image simulation approaches for the lunar environment, with particular emphasis on the effects of different camera models and light illumination conditions on the quality of synthetic lunar images. These images are produced using real Digital Elevation Models (DEM) and terrain data derived from instruments such as Chandrayaan-2 Orbiter High Resolution Camera (OHRC) and NASA's Wide Angle Camera (WAC), and Narrow Angle Camera (NAC) instruments. This research aims to improve the reliability of synthetic imagery in supporting autonomous
navigation and decision-making systems in lunar exploration. This work contributes to the development of more effective tools for generating important information for future lunar missions and enhances the understanding of the moon's surface environment.
Read the full abstract ▾
Computer Vision
DocPrune:Efficient Document Question Answering via Background, Question, and Comprehension-aware Token Pruning
Joonmyung Choi, Sanghyeok Lee, Jongha Kim et al. (+4 more)
Recent advances in
vision-
language models have demonstrated remarkable performance across diverse multi-modal tasks, including document question answering that leverages structured visual cues from text, tables, and figures. However, unlike natural images, document images contain large backgrounds and only sparse supporting evidence, leading to the inefficient consumption of substantial computational resources, especially for long documents. We observe that existing
token-reduction methods for natural images and
videos fall short in utilizing the structural sparsity unique to documents. To address this, we propose DocPrune, a training-free and progressive document
token pruning framework designed for efficient long-document understanding. The proposed method preserves only the essential
tokens for the task while removing unnecessary ones, such as background or question-irrelevant
tokens. Moreover, it automatically selects the appropriate layers to initiate
token pruning based on the model's level of comprehension. Our experiments on the M3DocRAG show that DocPrune improves throughput by 3.0x and 3.3x in the encoder and decoder, respectively, while boosting the F1 score by +1.0, achieving both higher accuracy and efficiency without any additional training.
Read the full abstract ▾
All authors Joonmyung Choi, Sanghyeok Lee, Jongha Kim, Sehyung Kim, Dohwan Ko, Jihyung Kil, Hyunwoo J. Kim
Computer Vision
Beyond Chain-of-Thought: Rewrite as a Universal Interface for Generative Multimodal Embeddings
Peixi Wu, Ke Mei, Feipeng Ma et al. (+15 more)
Multimodal Large
Language Models (MLLMs) have emerged as a promising foundation for universal
multimodal embeddings. Recent studies have shown that
reasoning-driven generative
multimodal embeddings can outperform discriminative
embeddings on several
embedding tasks. However, Chain-of-Thought (CoT)
reasoning tends to generate redundant thinking steps and introduce semantic ambiguity in the summarized answers in broader
retrieval scenarios. To address this limitation, we propose Rewrite-driven
Multimodal Embedding (RIME), a unified framework that jointly optimizes generation and
embedding through a
retrieval-friendly rewrite. Meanwhile, we present the Cross-Mode
Alignment (CMA) to bridge the generative and discriminative
embedding spaces, enabling flexible mutual
retrieval to trade off efficiency and accuracy. Based on this, we also introduce Refine Reinforcement Learning (Refine-RL) that treats discriminative
embeddings as stable semantic anchors to guide the rewrite optimization. Extensive experiments on MMEB-V2, MRMR and UVRB demonstrate that RIME substantially outperforms prior generative
embedding models while significantly reducing the length of thinking. Code is available at https://github.com/PeppaWu/RIME.
Read the full abstract ▾
All authors Peixi Wu, Ke Mei, Feipeng Ma, Bosong Chai, Zhibin Lan, Chenxi Zhao, Shannan Yan, Jie Chen, Zhangchi Hu, Yansong Peng, Bo Lin, Junjie Zhou, Dacheng Yin, Tianyi Wang, Fengyun Rao, Jing Lyu, Hebei Li, Xiaoyan Sun
Computer Vision
CAGE-SGG: Counterfactual Active Graph Evidence for Open-Vocabulary Scene Graph Generation
Suiyang Guang, Chenyu Liu, Ruohan Zhang et al. (+1 more)
Open-vocabulary scene graph generation (SGG) aims to describe visual scenes with flexible and fine-grained relation phrases beyond a fixed predicate vocabulary. While recent
vision-
language models greatly expand the semantic coverage of SGG, they also introduce a critical reliability issue: predicted relations may be driven by
language priors or object co-occurrence rather than grounded visual evidence. In this paper, we propose an evidence-rounded open-vocabulary SGG framework based on counterfactual relation verification. Instead of directly accepting plausible relation proposals, our method verifies whether each candidate relation is supported by relation-pecific visual,
geometric, and contextual evidence. Specifically, we first generate open-vocabulary relation candidates with a
vision-
language proposer, then decompose predicate phrases into soft evidence bases such as support, contact, containment, depth and state. A relation-conditioned evidence encoder extracts predicate-relevant cues, while a counterfactual verifier tests whether the relation score decreases when necessary vidence is removed and remains stable under irrelevant perturbations. We further introduce contradiction-aware predicate learning and graph-level preference optimization to improve fine-grained discrimination and global graph consistency. Experiments on conventional, open-vocabulary, and panoptic SGG
benchmarks show that our method consistently improves standard recall-based metrics, unseen predicate generalization, and counterfactual grounding quality. These results demonstrate that moving from relation generation to relation verification leads to more reliable, interpretable, and evidence-grounded scene graphs.
Read the full abstract ▾
All authors Suiyang Guang, Chenyu Liu, Ruohan Zhang, Siyuan Chen
Computer Vision
Towards Safe Mobility: A Unified Transportation Foundation Model enabled by Open-Ended Vision-Language Dataset
Wenhui Huang, Songyan Zhang, Collister Chua et al. (+4 more)
Urban transportation systems face growing safety challenges that require scalable intelligence for emerging smart mobility infrastructures. While recent advances in foundation models and large-scale
multimodal datasets have strengthened perception and
reasoning in intelligent transportation systems (ITS), existing research remains largely centered on microscopic autonomous driving (AD), with limited
attention to city-scale traffic analysis. In particular, open-ended safety-oriented visual question answering (VQA) and corresponding foundation models for
reasoning over heterogeneous roadside camera observations remain underexplored. To address this gap, we introduce the Land Transportation
Dataset (LTD), a large-scale open-source
vision-
language dataset for open-ended
reasoning in urban traffic environments. LTD contains 11.6K high-quality VQA pairs collected from heterogeneous roadside cameras, spanning diverse road geometries, traffic participants, illumination conditions, and adverse weather. The
dataset integrates three complementary tasks: fine-grained multi-object grounding, multi-image camera selection, and multi-image risk analysis, requiring joint
reasoning over minimally correlated views to infer hazardous objects, contributing factors, and risky road directions. To ensure annotation fidelity, we combine multi-model
vision-
language generation with cross-validation and human-in-the-loop refinement. Building upon LTD, we further propose UniVLT, a transportation foundation model trained via curriculum-based knowledge transfer to unify microscopic AD
reasoning and macroscopic traffic analysis within a single architecture. Extensive experiments on LTD and multiple AD
benchmarks demonstrate that UniVLT achieves SOTA performance on open-ended
reasoning tasks across diverse domains, while exposing limitations of existing foundation models in complex multi-view traffic scenarios.
Read the full abstract ▾
All authors Wenhui Huang, Songyan Zhang, Collister Chua, Yang Liang, Zhiqi Mao, Heng Yang, Chen Lv
Computer Vision
OccDirector: Language-Guided Behavior and Interaction Generation in 4D Occupancy Space
Zhuding Liang, Tianyi Yan, Dubing Chen et al. (+6 more)
Generative world models increasingly rely on 4D occupancy for realistic autonomous driving simulation. However, existing generation frameworks depend on rigid
geometric conditions (e.g., explicit trajectories) or simplistic attribute-level text, failing to orchestrate complex, sequential multi-
agent interactions. To address this semantic-spatiotemporal gap, we propose OccDirector, a pioneering framework that generates 4D occupancy dynamics conditioned solely on natural
language. Operating as a “scenario director”, OccDirector maps natural
language scripts into physically plausible voxel dynamics without requiring
geometric priors. Technically, it employs a VLM-driven Spatio-Temporal MMDiT equipped with a history-prefix anchoring strategy to ensure long-horizon interaction consistency. Furthermore, we introduce OccInteract-85k, a novel
dataset uniquely annotated with multi-level
language instructions: ranging from static layouts to intricate multi-
agent behaviors, alongside a novel VLM-based evaluation
benchmark. Extensive experiments demonstrate that OccDirector achieves state-of-the-art generation quality and unprecedented instruction-following capabilities, successfully shifting the paradigm from appearance synthesis to
language-driven behavior orchestration.
Read the full abstract ▾
All authors Zhuding Liang, Tianyi Yan, Dubing Chen, Jiasen Zheng, Huan Zheng, Cheng-zhong Xu, Yida Wang, Kun Zhan, Jianbing Shen
Computer Vision
Towards Temporal Compositional Reasoning in Long-Form Sports Videos
Siyu Cao, Lu Zhang, Ruizhe Zeng et al. (+1 more)
Sports
videos are a challenging domain for
multimodal understanding because they involve complex and dynamic human activities. Despite rapid progress in
Multimodal Large
Language Models (MLLMs), long-horizon
reasoning in sports
videos remains difficult, as answering questions requires both locating temporally sparse evidence and integrating it into
reasoning. We attribute this limitation to two closely coupled factors: insufficient supervision over temporally dispersed evidence, and the lack of methods that require models to identify, localize, and justify temporal evidence. To address these gaps, we introduce SportsTime, a large-scale
benchmark for long-form sports
video understanding, comprising 14K+ open-ended QA pairs and 50K+ step-wise temporal evidence annotations. Building on SportsTime, we propose Chain-of-Time
Reasoning (CoTR), which treats
reasoning as a process of temporally grounded evidence composition. Specifically, during training, CoTR introduces a temporal-reward GRPO to encourage temporally grounded
reasoning. During inference, it employs an anchor-observe-infer evidence-seeking loop to iteratively localize, verify, and compose temporal evidence before producing the final answer. Experiments demonstrate the usefulness of SportsTime as a
benchmark and the effectiveness of CoTR, which consistently improves temporal compositional
reasoning and step-wise grounding quality over strong MLLM baselines.
Read the full abstract ▾
All authors Siyu Cao, Lu Zhang, Ruizhe Zeng, Zhi-yong Liu
Computer Vision
Breaking Watermarks in the Frequency Domain: A Modulated Diffusion Attack Framework
Chunpeng Wang, Binyan Qu, Xiaoyu Wang et al. (+4 more)
Digital image watermarking has advanced rapidly for copyright protection of generative AI, yet the comparatively limited progress in watermark attack techniques has broken the attack-defense balance and hindered further advances in the field. In this paper, we propose FMDiffWA, a frequency-domain modulated
diffusion framework for watermark attacks. Specifically, we introduce a frequency-domain watermark modulation (FWM) module and incorporate it into the sampling stages both the forward and reverse
diffusion processes. This mechanism enables selective modulation of watermark-related frequency components, thereby allowing FMDiffWA to effectively neutralize the invisible watermark signals while preserving the perceptual quality of the attacked watermarked images. To achieve a better trade-off between attack efficacy and visual fidelity, we reformulate the training strategy of conventional
diffusion models by augmenting the canonical noise estimation objective with an auxiliary refinement constraint. Comprehensive experiments demonstrate that FMDiffWA achieves superior visual fidelity compared to existing watermark attacks, while exhibiting strong generalization across diverse watermarking schemes.
Read the full abstract ▾
All authors Chunpeng Wang, Binyan Qu, Xiaoyu Wang, Zhiqiu Xia, Shanshan Zhang, Yunan Liu, Qi Li
Computer Vision · Cross-listed from eess.IV
Multimodal Diffusion to Mutually Enhance Polarized Light and Low Resolution EBSD Data
Harry Dong, Timofey Efimov, Megna Shah et al. (+4 more)
In spite of the utility of 3-D electron back-scattered diffraction (EBSD) microscopy, the data collection process can be time-consuming with serial-sectioning. Hence, it is natural to look at other modalities, such as polarized light (PL) data, to accelerate EBSD data collection, supplemented with shared information. Complementarily, features in chaotic PL data could even be enriched with a handful of EBSD measurements. To inherently learn the complex dynamics between EBSD and PL to solve these inverse problems, we use an unconditional
multimodal diffusion model, motivated by progress in
diffusion models for inverse problems. Although trained solely on synthetic data once, our model has strong generalizable capabilities on real data which can be low-resolution, noisy, corrupted, and misregistered. With inference-time scaling, we show gains in performance on a variety of objectives including grain boundary prediction, super-resolution, and denoising. With our model, we demonstrate that there is little difference from full resolution performance with only 25% (1/4 the resolution) of EBSD data and corrupted PL data.
Read the full abstract ▾
All authors Harry Dong, Timofey Efimov, Megna Shah, Jeff Simmons, Sean Donegan, Marc De Graef, Yuejie Chi
Computer Vision
ArchSym: Detecting 3D-Grounded Architectural Symmetries in the Wild
Hanyu Chen, Ruojin Cai, Steve Marschner et al. (+1 more)
Symmetry detection is a fundamental problem in computer
vision, and
symmetries serve as powerful priors for downstream tasks. However, existing learning-based methods for detecting 3D
symmetries from single images have been almost exclusively trained and evaluated on object-centric or synthetic
datasets, and thus fail to generalize to real-world scenes. Furthermore, due to the inherent scale ambiguity of
monocular inputs, which makes localizing the 3D plane an ill-posed problem, many existing works only predict the plane's orientation. In this paper, we address these limitations by presenting the first framework for detecting 3D-grounded reflectional
symmetries from single, in-the-wild RGB images, focusing on architectural landmarks. We introduce two key innovations: (1) a scalable data annotation pipeline to automatically curate a large-scale
dataset of architectural
symmetries, ArchSym, from SfM reconstructions by leveraging cross-view image matching; and building on the
dataset, (2) a single-view
symmetry detector that accurately localizes
symmetries in 3D by parameterizing them as signed distance maps defined relative to predicted scene geometry. We validate our
symmetry annotation pipeline against geometry-based alternatives and demonstrate that our
symmetry detector significantly outperforms state-of-the-art baselines on our new
benchmark.
Read the full abstract ▾
All authors Hanyu Chen, Ruojin Cai, Steve Marschner, Noah Snavely
Computer Vision
CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution
Xiangxi Zheng, Kuang He, Jiayi Hu et al. (+6 more)
Chart-to-code generation demands strict visual precision and syntactic correctness from
Vision-
Language Models (VLMs). However, existing approaches are fundamentally constrained by data-centric limitations: despite the availability of growing chart-to-code
datasets, simply scaling homogeneous chart-code pairs conflates visual perception with program logic, preventing models from fully leveraging the richness of
multimodal supervision. We present CharTide, a novel data-centric framework that systematically redesigns both training and
alignment data for chart-to-code generation. First, we construct a 2M-sample
dataset via a Tri-Perspective Tuning strategy, explicitly decoupling training into visual perception, pure-text code logic, and modality fusion streams, enabling a 7B model to surpass specialized baselines using only supervised data. Second, we reformulate
alignment as a data verification problem rather than a heuristic scoring task. To this end, we introduce an Inquiry-Driven RL framework grounded in the principle of information invariance: a downstream model should yield consistent answers to identical visual queries across both original and generated charts. Moving beyond rigid rule matching or VLM scoring, we employ a frozen Inspector to objectively verify generated charts through atomic QA tasks, providing verifiable reward signals based on answer accuracy. Experiments on ChartMimic, Plot2Code, and ChartX show that CharTide-7B/8B significantly outperforms open-source baselines, surpasses GPT-4o, and is competitive with GPT-5.
Read the full abstract ▾
All authors Xiangxi Zheng, Kuang He, Jiayi Hu, Ping Yu, Rui Yan, Yuan Yao, Peng Hou, Anxiang Zeng, Alex Jinpeng Wang