Computer Vision · Cross-listed from cs.RO
Learning Vision-Based Omnidirectional Navigation: A Teacher-Student Approach Using Monocular Depth Estimation
Jan Finke, Wayne Paul Martis, Adrian Schmelter et al. (+3 more)
Reliable obstacle avoidance in industrial settings demands 3D scene understanding, but widely used 2D
LiDAR sensors perceive only a single horizontal slice of the environment, missing critical obstacles above or below the scan plane. We present a teacher-student framework for
vision-based mobile
robot navigation that eliminates the need for
LiDAR sensors. A teacher policy trained via Proximal Policy Optimization (PPO) in NVIDIA Isaac Lab leverages privileged 2D
LiDAR observations that account for the full
robot footprint to learn robust
navigation. The learned behavior is distilled into a student policy that relies solely on
monocular depth maps predicted by a fine-tuned Depth Anything V2 model from four RGB cameras. The complete inference pipeline, comprising
monocular depth estimation (MDE), policy execution, and motor control, runs entirely onboard an NVIDIA Jetson Orin AGX mounted on a DJI RoboMaster platform, requiring no external computation for inference. In simulation, the student achieves success rates of 82-96.5%, consistently outperforming the standard 2D
LiDAR teacher (50-89%). In real-world experiments, the MDE-based student outperforms the 2D
LiDAR teacher when navigating around obstacles with complex 3D geometries, such as overhanging structures and low-profile objects, that fall outside the single scan plane of a 2D
LiDAR.
Read the full abstract ▾
All authors Jan Finke, Wayne Paul Martis, Adrian Schmelter, Lars Erbach, Christian Jestel, Marvin Wiedemann
Computer Vision
Event-Only Drone Trajectory Forecasting with RPM-Modulated Kalman Filtering
Hari Prasanth S.M., Pejman Habibiroudkenar, Eerik Alamikkotervo et al. (+2 more)
Event cameras provide high-temporal-resolution visual sensing that is well suited for observing fast-moving aerial objects; however, their use for drone trajectory prediction remains limited. This work introduces an event-only drone
forecasting method that exploits propeller-induced motion cues. Propeller rotational speed are extracted directly from raw event data and fused within an RPM-aware Kalman filtering framework. Evaluations on the FRED
dataset show that the proposed method outperforms learning-based approaches and vanilla kalman filter in terms of average distance error and final distance error at 0.4s and 0.8s
forecasting horizons. The results demonstrate robust and accurate short- and medium-horizon trajectory
forecasting without reliance on RGB imagery or training data.
Read the full abstract ▾
All authors Hari Prasanth S.M., Pejman Habibiroudkenar, Eerik Alamikkotervo, Dimitrios Bouzoulas, Risto Ojala
Computer Vision
Cultivating Forensic Reasoning for Generalizable Multimodal Manipulation Detection
Yuchen Zhang, Yaxiong Wang, Kecheng Han et al. (+4 more)
Recent advances in generative AI have significantly enhanced the realism of
multimodal media manipulation, thereby posing substantial challenges to manipulation detection. Existing manipulation detection and grounding approaches predominantly focus on manipulation type classification under result-oriented supervision, which not only lacks interpretability but also tends to overfit superficial artifacts. In this paper, we argue that generalizable detection requires incorporating explicit forensic
reasoning, rather than merely classifying a limited set of manipulation types, which fails to generalize to unseen manipulation patterns. To this end, we propose REFORM, a
reasoning-driven framework that shifts learning from outcome fitting to process modeling. REFORM adopts a three-stage curriculum that first induces forensic rationales, then aligns
reasoning with final judgments, and finally refines logical consistency via reinforcement learning. To support this paradigm, we introduce ROM, a large-scale
dataset with rich
reasoning annotations. Extensive experiments show that REFORM establishes new state-of-the-art performance with superior generalization, achieving 81.52% ACC on ROM, 76.65% ACC on DGM4, and 74.9 F1 on MMFakeBench.
Read the full abstract ▾
All authors Yuchen Zhang, Yaxiong Wang, Kecheng Han, Yujiao Wu, Lianwei Wu, Li Zhu, Zhedong Zheng
Computer Vision · Cross-listed from cs.AI
According to Me: Long-Term Personalized Referential Memory QA
Jingbiao Mei, Jinghong Chen, Guangyu Yang et al. (+3 more)
Personalized AI assistants must recall and reason over long-term user
memory, which naturally spans multiple modalities and sources such as images,
videos, and emails. However, existing Long-term
Memory benchmarks focus primarily on dialogue history, failing to capture realistic personalized references grounded in lived experience. We introduce ATM-Bench, the first
benchmark for
multimodal, multi-source personalized referential
Memory QA. ATM-Bench contains approximately four years of
privacy-preserving personal
memory data and human-annotated question-answer pairs with ground-truth
memory evidence, including queries that require resolving personal references, multi-evidence
reasoning from multi-source and handling conflicting evidence. We propose Schema-Guided
Memory (SGM) to structurally represent
memory items originated from different sources. In experiments, we implement 5 state-of-the-art
memory systems along with a standard RAG baseline and evaluate variants with different
memory ingestion,
retrieval, and answer generation techniques. We find poor performance (under 20% accuracy) on the ATM-Bench-Hard set, and that SGM improves performance over Descriptive
Memory commonly adopted in prior works. Code available at: https://github.com/JingbiaoMei/ATM-Bench
Read the full abstract ▾
All authors Jingbiao Mei, Jinghong Chen, Guangyu Yang, Xinyu Hou, Margaret Li, Bill Byrne
Computer Vision
Robust White Blood Cell Classification with Stain-Normalized Decoupled Learning and Ensembling
Luu Le, Hoang-Loc Cao, Ha-Hieu Pham et al. (+2 more)
White blood
cell (WBC) classification is fundamental for hematology applications such as infection assessment, leukemia screening, and treatment monitoring. However, real-world WBC
datasets present substantial appearance variations caused by staining and scanning conditions, as well as severe class imbalance in which common
cell types dominate while rare but clinically important categories are underrepresented. To address these challenges, we propose a stain-normalized, decoupled training framework that first learns transferable representations using instance-balanced sampling, and then rebalances the classifier with class-aware sampling and a hybrid loss combining effective-number weighting and focal modulation. In inference stage, we further enhance robustness by ensembling various trained backbones with test-time augmentation. Our approach achieved the top rank on the leaderboard of the WBCBench 2026: Robust White Blood
Cell Classification Challenge at ISBI 2026.
Read the full abstract ▾
All authors Luu Le, Hoang-Loc Cao, Ha-Hieu Pham, Thanh-Huy Nguyen, Ulas Bagci
Computer Vision · Cross-listed from cs.RO
Closed-Loop Action Chunks with Dynamic Corrections for Training-Free Diffusion Policy
Pengyuan Wu, Pingrui Zhang, Zhigang Wang et al. (+3 more)
Diffusion-based policies have achieved remarkable results in robotic manipulation but often struggle to adapt rapidly in dynamic scenarios, leading to delayed responses or task failures. We present DCDP, a Dynamic Closed-Loop
Diffusion Policy framework that integrates chunk-based action generation with real-time correction. DCDP integrates a self-supervised dynamic feature encoder, cross-
attention fusion, and an asymmetric action encoder-decoder to inject environmental dynamics before action execution, achieving real-time closed-loop action correction and enhancing the system's adaptability in dynamic scenarios. In dynamic PushT simulations, DCDP improves adaptability by 19% without retraining while requiring only 5% additional computation. Its modular design enables plug-and-play integration, achieving both temporal coherence and real-time responsiveness in dynamic robotic scenarios, including real-world manipulation tasks. The project page is at: https://github.com/wupengyuan/dcdp
Read the full abstract ▾
All authors Pengyuan Wu, Pingrui Zhang, Zhigang Wang, Dong Wang, Bin Zhao, Xuelong Li
Computer Vision · Cross-listed from cs.LG
Semantic Similarity is a Spurious Measure of Comic Understanding: Lessons Learned from Hallucinations in a Benchmarking Experiment
Christopher Driggers-Ellis, Nachiketh Tibrewal, Rohit Bogulla et al. (+4 more)
A system that enables blind or visually impaired users to access comics/manga would introduce a new medium of storytelling to this community. However, no such system currently exists. Generative
vision-
language models (VLMs) have shown promise in describing images and understanding comics, but most research on comic understanding is limited to panel-level analysis. To fully support blind and visually impaired users, greater
attention must be paid to page-level understanding and interpretation. In this work, we present a preliminary
benchmark of VLM performance on comic interpretation tasks. We identify and categorize
hallucinations that emerge during this process, organizing them into generalized object-
hallucination taxonomies. We conclude with guidance on future research, emphasizing
hallucination mitigation and improved data curation for comic interpretation.
Read the full abstract ▾
All authors Christopher Driggers-Ellis, Nachiketh Tibrewal, Rohit Bogulla, Harsh Khanna, Sangpil Youm, Christan Grant, Bonnie Dorr
Computer Vision
PreSight: Preoperative Outcome Prediction for Parkinson's Disease via Region-Prior Morphometry and Patient-Specific Weighting
Yand Wang, Chen Zhang, Lanyun Zhu et al. (+6 more)
Preoperative improvement rate prediction for Parkinson's disease surgery is clinically important yet difficult because imaging signals are subtle and patients are heterogeneous. We address this setting, where only information available before surgery is used, and the goal is to predict patient-specific postoperative motor benefit. We present PreSight, a presurgical outcome model that fuses clinical priors with preoperative MRI and deformation-based morphometry (DBM) and adapts regional importance through a patient-specific weighting module. The model produces end-to-end, calibrated, decision-ready predictions with patient-level explanations. We evaluate PreSight on a real-world two-center cohort of 400 subjects with
multimodal presurgical inputs and postoperative improvement labels. PreSight outperforms strong clinical, imaging-only, and
multimodal baselines. It attains 88.89% accuracy on internal validation and 85.29% on an external-center test for responder classification and shows better probability calibration and higher decision-curve net benefit. Ablations and analyses confirm the contribution of DBM and the patient-specific weighting module and indicate that the model emphasizes disease-relevant regions in a patient-specific manner. These results demonstrate that integrating clinical prior knowledge with region-adaptive morphometry enables reliable presurgical decision support in routine practice.
Read the full abstract ▾
All authors Yand Wang, Chen Zhang, Lanyun Zhu, Yixin Chen, Qunbo Wang, Yutong Bai, Jurgen Germann, Yinghong Wen, Shuai Shao
Computer Vision
physfusion: A Transformer-based Dual-Stream Radar and Vision Fusion Framework for Open Water Surface Object Detection
Yuting Wan, Liguo Sun, Jiuwu Hao et al. (+2 more)
Detecting water-surface targets for Unmanned Surface Vehicles (USVs) is challenging due to wave clutter, specular reflections, and weak appearance cues in long-range observations. Although 4D millimeter-wave radar complements cameras under degraded illumination, maritime radar point clouds are sparse and intermittent, with reflectivity attributes exhibiting heavy-tailed variations under scattering and multipath, making conventional fusion designs struggle to exploit radar cues effectively. We propose PhysFusion, a physics-informed radar-image detection framework for water-surface perception. The framework integrates: (1) a Physics-Informed Radar Encoder (PIR Encoder) with an RCS Mapper and Quality Gate, transforming per-point radar attributes into compact scattering priors and predicting point-wise reliability for robust feature learning under clutter; (2) a Radar-guided Interactive Fusion Module (RIFM) performing query-level radar-image fusion between semantically enriched radar features and multi-scale visual features, with the radar branch modeled by a dual-stream backbone including a point-based local stream and a
transformer-based global stream using Scattering-Aware Self-
Attention (SASA); and (3) a Temporal Query Aggregation module (TQA) aggregating frame-wise fused queries over a short temporal window for temporally consistent representations. Experiments on WaterScenes and FLOW demonstrate that PhysFusion achieves 59.7% mAP50:95 and 90.3% mAP50 on WaterScenes (T=5 radar history) using 5.6M parameters and 12.5G FLOPs, and reaches 94.8% mAP50 and 46.2% mAP50:95 on FLOW under radar+camera setting. Ablation studies quantify the contributions of PIR Encoder, SASA-based global
reasoning, and RIFM.
Read the full abstract ▾
All authors Yuting Wan, Liguo Sun, Jiuwu Hao, Zao Zhang, Pin LV
Computer Vision
MobileMold: A Smartphone-Based Microscopy Dataset for Food Mold Detection
Dinh Nam Pham, Leonard Prokisch, Bennet Meyer et al. (+1 more)
Smartphone clip-on microscopes turn everyday devices into low-cost, portable imaging systems that can even reveal fungal structures at the microscopic level, enabling mold inspection beyond unaided visual checks. In this paper, we introduce MobileMold, an open smartphone-based microscopy
dataset for food mold detection and food classification. MobileMold contains 4,941 handheld microscopy images spanning 11 food types, 4 smartphones, 3 microscopes, and diverse real-world conditions. Beyond the
dataset release, we establish baselines for (i) mold detection and (ii) food-type classification, including a multi-task setting that predicts both attributes. Across multiple pretrained deep learning architectures and augmentation strategies, we obtain near-ceiling performance (accuracy = 0.9954, F1 = 0.9954, MCC = 0.9907), validating the utility of our
dataset for detecting food spoilage. To increase transparency, we complement our evaluation with saliency-based visual explanations highlighting mold regions associated with the model's predictions. MobileMold aims to contribute to research on accessible food-safety sensing, mobile imaging, and exploring the potential of smartphones enhanced with attachments.
Read the full abstract ▾
All authors Dinh Nam Pham, Leonard Prokisch, Bennet Meyer, Jonas Thumbs
Computer Vision
Haitian Wang, Xinyu Wang, Muhammad Ibrahim et al. (+2 more)
Accurate weed mapping in cereal fields requires pixel-level
segmentation from
UAV imagery that remains reliable across fields, seasons, and illumination. Existing multispectral pipelines often depend on thresholded vegetation indices, which are brittle under radiometric drift and mixed crop–weed pixels, or on single-stream CNN and
Transformer backbones that ingest stacked bands and indices, where radiance cues and normalized index cues interfere and reduce sensitivity to small weed clusters embedded in crop canopy. We propose VISA, a two-stream
segmentation network that decouples these cues and fuses them at native resolution. The radiance stream learns from calibrated five-band reflectance using local residual convolutions, channel recalibration, spatial gating, and skip-connected decoding, which preserve fine textures, row boundaries, and small weed structures that are often weakened after ratio-based index compression. The index stream operates on vegetation-index maps with windowed self-
attention to model local structure efficiently, state-space layers to propagate field-scale context without quadratic
attention cost, and Slot
Attention to form stable region descriptors that improve discrimination of sparse weeds under canopy mixing. To support supervised training and deployment-oriented evaluation, we introduce BAWSeg, a four-year
UAV multispectral
dataset collected over commercial barley paddocks in Western Australia, providing radiometrically calibrated blue, green, red, red edge, and near-infrared orthomosaics, derived vegetation indices, and dense crop, weed, and other labels with leakage-free block splits. On BAWSeg, VISA achieves 75.6% mIoU and 63.5% weed IoU with 22.8 M parameters, outperforming a multispectral SegFormer-B1 baseline by 1.2 mIoU and 1.9 weed IoU. Under cross-plot and cross-year protocols, VISA maintains 71.2% and 69.2% mIoU, respectively.
Read the full abstract ▾
All authors Haitian Wang, Xinyu Wang, Muhammad Ibrahim, Dustin Severtson, Ajmal Mian
Computer Vision
LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving
Yuechen Luo, Fang Li, Shaoqing Xu et al. (+10 more)
While
Vision-
Language-Action (VLA) models have revolutionized autonomous driving by unifying perception and planning, their reliance on explicit textual Chain-of-Thought (CoT) leads to semantic-perceptual decoupling and perceptual-symbolic conflicts. Recent shifts toward latent
reasoning attempt to bypass these bottlenecks by thinking in continuous hidden space. However, without explicit intermediate constraints, standard latent CoT often operates as a physics-agnostic representation. To address this, we propose the Latent Spatio-Temporal VLA (LaST-VLA), a framework shifting the
reasoning paradigm from discrete symbolic processing into a physically grounded Latent Spatio-Temporal CoT. By implementing a dual-feature
alignment mechanism, we distill
geometric constraints from 3D foundation models and dynamic foresight from world models directly into the latent space. Coupled with a progressive SFT training strategy that transitions from feature
alignment to trajectory generation, and refined via Reinforcement Learning with Group Relative Policy Optimization (GRPO) to ensure safety and rule compliance. \method~setting a new record on NAVSIM v1 (91.3 PDMS) and NAVSIM v2 (87.1 EPDMS), while excelling in spatial-temporal
reasoning on SURDS and NuDynamics
benchmarks.
Read the full abstract ▾
All authors Yuechen Luo, Fang Li, Shaoqing Xu, Yang Ji, Zehan Zhang, Bing Wang, Yuannan Shen, Jianwei Cui, Long Chen, Guang Chen, Hangjun Ye, Zhi-Xin Yang, Fuxi Wen
Computer Vision · Cross-listed from cs.IR
MealRec: Multi-granularity Sequential Modeling via Hierarchical Diffusion Models for Micro-Video Recommendation
Xinxin Dong, Haokai Ma, Yuze Zheng et al. (+3 more)
Micro-
video recommendation aims to capture user preferences from the collaborative and context information of the interacted micro-
videos, thereby predicting the appropriate
videos. This target is often hindered by the inherent noise within
multimodal content and unreliable implicit feedback, which weakens the correspondence between behaviors and underlying interests. While conventional works have predominantly approached such scenario through behavior-augmented modeling and content-centric
multimodal analysis, these paradigms can inadvertently give rise to two non-trivial challenges: preference-irrelative
video representation extraction and inherent modality conflicts. To address these issues, we propose a Multi-granularity sequential modeling method via hierarchical
diffusion models for micro-
video Recommendation (MealRec), which simultaneously considers temporal correlations during preference modeling from intra- and inter-
video perspectives. Specifically, we first propose Temporal-guided Content
Diffusion (TCD) to refine
video representations under intra-
video temporal guidance and personalized collaborative signals to emphasize salient content while suppressing redundancy. To achieve the semantically coherent preference modeling, we further design the Noise-unconditional Preference Denoising (NPD) to recovers informative user preferences from corrupted states under the blind denoising. Extensive experiments and analyses on four micro-
video datasets from two platforms demonstrate the effectiveness, universality, and robustness of our MealRec, further uncovering the effective mechanism of our proposed TCD and NPD. The source code and corresponding
dataset will be available upon acceptance.
Read the full abstract ▾
All authors Xinxin Dong, Haokai Ma, Yuze Zheng, Yongfu Zha, Yonghui Yang, Xiaodong Wang
Computer Vision
Zero-shot Low-Field MRI Enhancement via Diffusion-Based Adaptive Contrast Transport
Muyu Liu, Chenhe Du, Xuanyu Tian et al. (+5 more)
Low-field (LF)
magnetic resonance imaging (MRI) democratizes access to diagnostic imaging but is fundamentally limited by low signal-to-noise ratio and significant tissue contrast distortion due to field-dependent relaxation dynamics. Reconstructing high-field (HF) quality images from LF data is a blind inverse problem, severely challenged by the scarcity of paired training data and the unknown, non-linear contrast transformation operator. Existing zero-shot methods, which assume simplified linear degradation, often fail to recover authentic tissue contrast. In this paper, we propose DACT(
Diffusion-Based Adaptive Contrast Transport), a novel zero-shot framework that restores HF-quality images without paired supervision. DACT synergizes a pre-trained HF
diffusion prior to ensure anatomical fidelity with a physically-informed adaptive forward model. Specifically, we introduce a differentiable Sinkhorn optimal transport module that explicitly models and corrects the intensity distribution shift between LF and HF domains during the reverse
diffusion process. This allows the framework to dynamically learn the intractable contrast mapping while preserving topological consistency. Extensive experiments on simulated and real clinical LF
datasets demonstrate that DACT achieves state-of-the-art performance, yielding reconstructions with superior structural detail and correct tissue contrast.
Read the full abstract ▾
All authors Muyu Liu, Chenhe Du, Xuanyu Tian, Qing Wu, Xiao Wang, Haonan Zhang, Hongjiang Wei, Yuyao Zhang
Computer Vision
Generative Visual Chain-of-Thought for Image Editing
Zijin Yin, Tiankai Hang, Yiji Cheng et al. (+9 more)
Existing image editing methods struggle to perceive where to edit, especially under complex scenes and nuanced spatial instructions. To address this issue, we propose Generative Visual Chain-of-Thought (GVCoT), a unified framework that performs native visual
reasoning by first generating spatial cues to localize the target region and then executing the edit. Unlike prior text-only CoT or tool-dependent visual CoT paradigms, GVCoT jointly optimizes visual
tokens generated during the
reasoning and editing phases in an end-to-end manner. This way fosters the emergence of innate spatial
reasoning ability and enables more effective utilization of visual-domain cues. The main challenge of training GCVoT lies in the scarcity of large-scale editing data with precise edit region annotations; to this end, we construct GVCoT-Edit-Instruct, a
dataset of 1.8M high-quality samples spanning 19 tasks. We adopt a progressive training strategy: supervised fine-tuning to build foundational localization ability in
reasoning trace before final editing, followed by reinforcement learning to further improve
reasoning and editing quality. Finally, we introduce SREdit-Bench, a new
benchmark designed to comprehensively stress-test models under sophisticated scenes and fine-grained referring expressions. Experiments demonstrate that GVCoT consistently outperforms state-of-the-art models on SREdit-Bench and ImgEdit. We hope our GVCoT will inspire future research toward interpretable and precise image editing.
Read the full abstract ▾
All authors Zijin Yin, Tiankai Hang, Yiji Cheng, Shiyi Zhang, Runze He, Yu Xu, Chunyu Wang, Bing Li, Zheng Chang, Kongming Liang, Qinglin Lu, Zhanyu Ma
Computer Vision
Resolving Blind Inverse Problems under Dynamic Range Compression via Structured Forward Operator Modeling
Muyu Liu, Xuanyu Tian, Chenhe Du et al. (+3 more)
Recovering radiometric fidelity from unknown dynamic range compression (UDRC), such as low-light enhancement and HDR reconstruction, is a challenging blind inverse problem, due to the unknown forward model and irreversible information loss introduced by compression. To address this challenge, we first identify monotonicity as the fundamental physical invariant shared across UDRC tasks. Leveraging this insight, we introduce the cascaded monotonic Bernstein (CaMB) operator to parameterize the unknown forward model. CaMB enforces monotonicity as a hard architectural inductive bias, constraining optimization to physically consistent mappings and enabling robust and stable operator estimation. We further integrate CaMB with a plug-and-play
diffusion framework, proposing CaMB-Diff. Within this framework, the
diffusion model serves as a powerful
geometric prior for structural and semantic recovery, while CaMB explicitly models and corrects radiometric distortions through a physically grounded forward operator. Extensive experiments on a variety of zero-shot UDRC tasks, including low-light enhancement, low-field MRI enhancement, and HDR reconstruction, demonstrate that CaMB-Diff significantly outperforms state-of-the-art zero-shot baselines in terms of both signal fidelity and physical consistency. Moreover, we empirically validate the effectiveness of the proposed CaMB parameterization in accurately modeling the unknown forward operator.
Read the full abstract ▾
All authors Muyu Liu, Xuanyu Tian, Chenhe Du, Qing Wu, Hongjiang Wei, Yuyao Zhang
Computer Vision
CTForensics: A Comprehensive Dataset and Method for AI-Generated CT Image Detection
Yiheng Li, Zichang Tan, Guoqing Xu et al. (+3 more)
Recent advances in generative AI have made synthetic Computed
Tomography (CT) images increasingly realistic, enabling promising applications in medical data augmentation while raising serious concerns about clinical safety and data trustworthiness. Detecting AI-generated CT images remains challenging for two key reasons: existing
benchmarks cover only limited generation sources, and many detectors are adapted from natural-image forensics without explicitly modeling CT-specific imaging properties. In this paper, we introduce CTForensics, a
dataset for detecting AI-generated CT images. CTForensics contains 75,990 2D CT images, including a dedicated test
benchmark of 29,990 balanced authentic and generated samples from ten representative CT generative models spanning GAN-based and
diffusion-based paradigms. We further propose the Enhanced Spatial-Frequency CT Forgery Detector (ESF-CTFD), a CT-oriented CNN framework built around a Wavelet-Enhanced Central Stem, Multi-Scale Spatial Aggregation, and a Frequency-Aware Prediction Block. The Wavelet-Enhanced Central Stem enhances local intensity correlations and high-frequency residuals, Multi-Scale Spatial Aggregation aligns anatomical features across resolutions with lightweight residual units, and the Frequency-Aware Prediction Block models global spectral artifacts. Extensive experiments on CTForensics show that ESF-CTFD achieves 96.01% mAcc and 99.96% mAP, outperforming existing methods and maintaining strong robustness under realistic perturbations with only a 0.99% average drop. Codes will be available at https://github.com/liyih/CTForensics.
Read the full abstract ▾
All authors Yiheng Li, Zichang Tan, Guoqing Xu, Yichun Yeh, Yang Yang, Zhen Lei
Computer Vision
Streaming Real-Time Trajectory Prediction Using Endpoint-Aware Modeling
Alexander Prutsch, David Schinagl and Horst Possegger
Future trajectories of neighboring traffic
agents have a significant influence on the path planning and decision-making of autonomous vehicles. While trajectory
forecasting is a well-studied field, research mainly focuses on snapshot-based prediction, where each scenario is treated independently of its global temporal context. However, real-world autonomous driving systems need to operate in a continuous setting, requiring real-time processing of data streams with low latency and consistent predictions over successive timesteps. We leverage this continuous setting to propose a lightweight yet highly accurate streaming-based trajectory
forecasting approach. We integrate valuable information from previous predictions with a novel endpoint-aware modeling scheme. Our temporal context propagation uses the trajectory endpoints of the previous forecasts as anchors to extract targeted scenario context encodings. Our approach efficiently guides its scene encoder to extract highly relevant context information without needing refinement iterations or segment-wise decoding. Our experiments highlight that our approach effectively relays information across consecutive timesteps. Unlike methods using multi-stage refinement processing, our approach significantly reduces inference latency, making it well-suited for real-world deployment. We achieve state-of-the-art streaming trajectory prediction results on the Argoverse~2 multi-
agent and single-
agent benchmarks, while requiring substantially fewer resources.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Tiny-DroNeRF: Tiny Neural Radiance Fields aboard Federated Learning-enabled Nano-drones
Ilenia Carboni, Elia Cereda, Lorenzo Lamberti et al. (+3 more)
Sub-30g nano-sized aerial
robots can leverage their agility and form factor to autonomously explore cluttered and narrow environments, like in industrial inspection and search and rescue missions. However, the price for their tiny size is a strong limit in their resources, i.e., sub-100 mW microcontroller units (MCUs) delivering $\sim$100 GOps/s at best, and
memory budgets well below 100 MB. Despite these strict constraints, we aim to enable complex
vision-based tasks aboard nano-drones, such as dense 3D scene reconstruction: a key robotic task underlying fundamental capabilities like spatial awareness and motion planning. Top-performing 3D reconstruction methods leverage neural radiance fields (NeRF) models, which require GBs of
memory and massive computation, usually delivered by high-end GPUs consuming 100s of Watts. Our work introduces Tiny-DroNeRF, a lightweight NeRF model, based on Instant-NGP, and optimized for running on a GAP9 ultra-low-power (ULP) MCU aboard our nano-drones. Then, we further empower our Tiny-DroNeRF by leveraging a collaborative
federated learning scheme, which distributes the model training among multiple nano-drones. Our experimental results show a 96% reduction in Tiny-DroNeRF's
memory footprint compared to Instant-NGP, with only a 5.7 dB drop in reconstruction accuracy. Finally, our
federated learning scheme allows Tiny-DroNeRF to train with an amount of data otherwise impossible to keep in a single drone's
memory, increasing the overall reconstruction accuracy. Ultimately, our work combines, for the first time, NeRF training on an ULP MCU with
federated learning on nano-drones.
Read the full abstract ▾
All authors Ilenia Carboni, Elia Cereda, Lorenzo Lamberti, Daniele Malpetti, Francesco Conti, Daniele Palossi
Computer Vision
GroupEnsemble: Efficient Uncertainty Estimation for DETR-based Object Detection
Yutong Yang, Katarina Popović, Julian Wiederer et al. (+3 more)
Detection
Transformer (DETR) and its variants show strong performance on object detection, a key task for autonomous systems. However, a critical limitation of these models is that their confidence scores only reflect semantic uncertainty, failing to capture the equally important spatial uncertainty. This results in an incomplete assessment of the detection reliability. On the other hand, Deep Ensembles can tackle this by providing high-quality spatial uncertainty estimates. However, their immense
memory consumption makes them impractical for real-world applications. A cheaper alternative, Monte Carlo (MC) Dropout, suffers from high latency due to the need of multiple forward passes during inference to estimate uncertainty. To address these limitations, we introduce GroupEnsemble, an efficient and effective uncertainty estimation method for DETR-like models. GroupEnsemble simultaneously predicts multiple individual detection sets by feeding additional diverse groups of object queries to the
transformer decoder during inference. Each query group is transformed by the shared decoder in isolation and predicts a complete detection set for the same input. An
attention mask is applied to the decoder to prevent inter-group query interactions, ensuring each group detects independently to achieve reliable ensemble-based uncertainty estimation. By leveraging the decoder's inherent parallelism, GroupEnsemble efficiently estimates uncertainty in a single forward pass without sequential repetition. We validated our method under autonomous driving scenes and common daily scenes using the Cityscapes and COCO
datasets, respectively. The results show that a hybrid approach combining MC-Dropout and GroupEnsemble outperforms Deep Ensembles on several metrics at a fraction of the cost. The code is available at https://github.com/yutongy98/GroupEnsemble.
Read the full abstract ▾
All authors Yutong Yang, Katarina Popović, Julian Wiederer, Markus Braun, Vasileios Belagiannis, Bin Yang
Computer Vision
FireRed-OCR Technical Report
Hao Wu, Haoran Lou, Xinyue Li et al. (+19 more)
We present FireRed-OCR, a systematic framework to specialize general VLMs into high-performance OCR models. Large
Vision-
Language Models (VLMs) have demonstrated impressive general capabilities but frequently suffer from “structural
hallucination” when processing complex documents, limiting their utility in industrial OCR applications. In this paper, we introduce FireRed-OCR, a novel framework designed to transform general-purpose VLMs (based on Qwen3-VL) into pixel-precise structural document parsing experts. To address the scarcity of high-quality structured data, we construct a “Geometry + Semantics” Data Factory. Unlike traditional random sampling, our pipeline leverages
geometric feature clustering and multi-dimensional tagging to synthesize and curate a highly balanced
dataset, effectively handling long-tail layouts and rare document types. Furthermore, we propose a Three-Stage Progressive Training strategy that guides the model from pixel-level perception to logical structure generation. This curriculum includes: (1) Multi-task Pre-
alignment to ground the model's understanding of document structure; (2) Specialized SFT for standardizing full-image Markdown output; and (3) Format-Constrained Group Relative Policy Optimization (GRPO), which utilizes reinforcement learning to enforce strict syntactic validity and structural integrity (e.g., table closure, formula syntax). Extensive evaluations on OmniDocBench v1.5 demonstrate that FireRed-OCR achieves state-of-the-art performance with an overall score of 92.94%, significantly outperforming strong baselines such as DeepSeek-OCR 2 and OCRVerse across text, formula, table, and reading order metrics. We open-source our code and model weights to facilitate the “General VLM to Specialized Structural Expert” paradigm.
Read the full abstract ▾
All authors Hao Wu, Haoran Lou, Xinyue Li, Zuodong Zhong, Zhaojun Sun, Phellon Chen, Xuanhe Zhou, Kai Zuo, Yibo Chen, Xu Tang, Yao Hu, Boxiang Zhou, Jian Wu, Yongji Wu, Wenxin Yu, Yingmiao Liu, Yuhao Huang, Manjie Xu, Gang Liu, Yidong Ma, Zhichao Sun, Changhao Qiao
Computer Vision
LEAR: Learning Edge-Aware Representations for Event-to-LiDAR Localization
Kuangyi Chen, Jun Zhang, Yuxi Hu et al. (+2 more)
Event cameras offer high-temporal-resolution sensing that remains reliable under high-speed motion and challenging lighting, making them promising for localization from
LiDAR point clouds in GPS-denied and visually degraded environments. However, aligning sparse, asynchronous events with dense
LiDAR maps is fundamentally ill-posed, as direct correspondence estimation suffers from modality gaps. We propose LEAR, a dual-task learning framework that jointly estimates edge structures and dense event-depth flow fields to bridge the sensing-modality divide. Instead of treating edges as a post-hoc aid, LEAR couples them with flow estimation through a cross-modal fusion mechanism that injects modality-invariant
geometric cues into the motion representation, and an iterative refinement strategy that enforces mutual consistency between the two tasks over multiple update steps. This synergy produces edge-aware, depth-aligned flow fields that enable more robust and accurate pose recovery via Perspective-n-Point (PnP) solvers. On several popular and challenging
datasets, LEAR achieves superior performance over the best prior method. The source code, trained models, and demo
videos are made publicly available online.
Read the full abstract ▾
All authors Kuangyi Chen, Jun Zhang, Yuxi Hu, Yi Zhou, Friedrich Fraundorfer
Computer Vision
Affine Correspondences in Stereo Vision: Theory, Practice, and Limitations
Levente Hajder
Affine transformations have been recently used for stereo
vision. They can be exploited in various computer
vision application, e.g., when estimating surface normals, homographies, fundamental and essential
matrices. Even full 3D reconstruction can be obtained by using affine correspondences. First, this paper overviews the fundamental statements for affine transformations and epipolar geometry. Then it is investigated how the transformation accuracy influences the quality of the 3D reconstruction. Besides, we propose novel techniques for estimating the local affine transformation from corresponding image directions; moreover, the fundamental
matrix, related to the processed image pair, can also be exploited. Both synthetic and real quantitative evaluations are implemented based on the accuracy of the reconstructed surface normals. For the latter one, a special object, containing three perpendicular planes with chessboard patterns, is constructed. The quantitative evaluations are based on the accuracy of the reconstructed surface normals and it is concluded that the estimation accuracy is around a few degrees for realistic test cases. Special stereo poses and plane orientations are also evaluated in detail.
Read the full abstract ▾
Computer Vision
Neural Operator-Grounded Continuous Tensor Function Representation and Its Applications
Ruoyang Su, Xi-Le Zhao, Sheng Liu et al. (+3 more)
Recently, continuous tensor functions have attracted increasing
attention, because they can unifiedly represent data both on mesh grids and beyond mesh grids. However, since mode-$n$ product is essentially discrete and linear, the potential of current continuous tensor function representations is still locked. To break this bottleneck, we suggest neural operator-grounded mode-$n$ operators as a continuous and nonlinear alternative of discrete and linear mode-$n$ product. Instead of mapping the discrete core tensor to the discrete target tensor, proposed mode-$n$ operator directly maps the continuous core tensor function to the continuous target tensor function, which provides a genuine continuous representation of real-world data and can ameliorate discretization artifacts. Empowering with continuous and nonlinear mode-$n$ operators, we propose a neural operator-grounded continuous tensor function representation (abbreviated as NO-CTR), which can more faithfully represent complex real-world data compared with classic discrete tensor representations and continuous tensor function representations. Theoretically, we also prove that any continuous tensor function can be approximated by NO-CTR. To examine the capability of NO-CTR, we suggest an NO-CTR-based multi-dimensional data completion model. Extensive experiments across various data on regular mesh grids (multi-spectral images and color
videos), on mesh girds with different resolutions (Sentinel-2 images) and beyond mesh grids (point clouds) demonstrate the superiority of NO-CTR.
Read the full abstract ▾
All authors Ruoyang Su, Xi-Le Zhao, Sheng Liu, Wei-Hao Wu, Yisi Luo, Michael K. Ng
Computer Vision
Non-verbal Real-time Human-AI Interaction in Constrained Robotic Environments
Dragos Costea, Alina Marcu, Cristina Lazar et al. (+1 more)
We study the ongoing debate regarding the statistical fidelity of AI-generated data compared to human-generated data in the context of non-verbal communication using full body motion. Concretely, we ask if contemporary generative models move beyond surface mimicry to participate in the silent, but expressive dialogue of body
language. We tackle this question by introducing the first framework that generates a natural non-verbal interaction between Human and AI in real-time from 2D body keypoints. Our experiments utilize four lightweight architectures which run at up to 100 FPS on an NVIDIA Orin Nano, effectively closing the perception-action loop needed for natural Human-AI interaction. We trained on 437 human
video clips and demonstrated that pretraining on synthetically-generated sequences reduces motion errors significantly, without sacrificing speed. Yet, a measurable reality gap persists. When the best model is evaluated on keypoints extracted from cutting-edge text-to-
video systems, such as SORA and VEO, we observe that performance drops on SORA-generated clips. However, it degrades far less on VEO, suggesting that temporal coherence, not image fidelity, drives real-world performance. Our results demonstrate that statistically distinguishable differences persist between Human and AI motion.
Read the full abstract ▾
All authors Dragos Costea, Alina Marcu, Cristina Lazar, Marius Leordeanu
Computer Vision · Cross-listed from cs.CL
FreeAct: Freeing Activations for LLM Quantization
Xiaohao Liu, Xiaobo Xia, Manyi Zhang et al. (+6 more)
Quantization is pivotal for mitigating the significant
memory and computational overhead of Large
Language Models (LLMs). While emerging transformation-based methods have successfully enhanced quantization by projecting feature spaces onto smoother
manifolds using orthogonal
matrices, they typically enforce a rigid one-to-one transformation constraint. This static approach fails to account for the dynamic patterns inherent in input activations, particularly within
diffusion LLMs (dLLMs) and
Multimodal LLMs (MLLMs), where varying
token types exhibit distinct distributions. To advance this, we propose FreeAct, a novel quantization framework that relaxes the static one-to-one constraint to accommodate dynamic activation disparities. Theoretically, we leverage the rank-deficient nature of activations to derive a solution space that extends beyond simple inverse
matrices, enabling the decoupling of activation transformations from weights. Methodologically, FreeAct identifies
token-specific dynamics (i.e.,
vision v.s. text, or masked
tokens) and allocates distinct transformation
matrices to the activation side, while maintaining a unified, static transformation for the weights. Extensive experiments across dLLMs and MLLMs demonstrate that FreeAct significantly outperforms baselines, up to 5.3% performance improvement, with in-depth analyses. Our code will be publicly released.
Read the full abstract ▾
All authors Xiaohao Liu, Xiaobo Xia, Manyi Zhang, Ji-Fu Li, Xianzhi Yu, Fei Shen, Xiu Su, See-Kiong Ng, Tat-Seng Chua
Computer Vision
Downstream Task Inspired Underwater Image Enhancement: A Perception-Aware Study from Dataset Construction to Network Design
Bosen Lin, Feng Gao, Yanwei Yu et al. (+2 more)
In real
underwater environments, downstream image recognition tasks such as semantic
segmentation and object detection often face challenges posed by problems like blurring and color inconsistencies.
Underwater image enhancement (UIE) has emerged as a promising preprocessing approach, aiming to improve the recognizability of targets in
underwater images. However, most existing UIE methods mainly focus on enhancing images for human visual perception, frequently failing to reconstruct high-frequency details that are critical for task-specific recognition. To address this issue, we propose a Downstream Task-Inspired
Underwater Image Enhancement (DTI-UIE) framework, which leverages human visual perception model to enhance images effectively for
underwater vision tasks. Specifically, we design an efficient two-branch network with task-aware
attention module for feature mixing. The network benefits from a multi-stage training framework and a task-driven perceptual loss. Additionally, inspired by human perception, we automatically construct a Task-Inspired UIE
Dataset (TI-UIED) using various task-specific networks. Experimental results demonstrate that DTI-UIE significantly improves task performance by generating preprocessed images that are beneficial for downstream tasks such as semantic
segmentation, object detection, and instance
segmentation. The codes are publicly available at https://github.com/oucailab/DTIUIE.
Read the full abstract ▾
All authors Bosen Lin, Feng Gao, Yanwei Yu, Junyu Dong, Qian Du
Computer Vision
StreamTTO: Efficient Online Test-Time Optimization for Video Depth Completion
Minseok Seo, Wonjun Lee, Jaehyuk Jang et al. (+1 more)
Monocular depth foundation models generalize across diverse scenes, but recovering accurate metric depth consistent with a target sensor remains challenging under sensor variation and domain shift. Test-time optimization (TTO) guided by sparse depth addresses this limitation, yet independent per-frame adaptation incurs substantial computational cost. We propose StreamTTO, an online test-time optimization framework for
video depth completion that reuses visual features and adaptation state. A frozen feature extractor with causal temporal
attention incorporates past visual context into features cached for repeated decoder optimization. Causal Sliding-Window Optimization directly updates a shared depth decoder using cached features and sparse observations from recent frames, requiring only two passes over the window per incoming frame after initialization. Reusing recent observations and the adapted decoder substantially reduces optimization steps. We also introduce MS-Depth, comprising approximately 1.45M synchronized RGB–
LiDAR frames from 40 hours of recordings. Its uninterrupted sequences capture transitions among indoor, underground, and outdoor environments under daytime and nighttime conditions, enabling evaluation of adaptation to changes in illumination, scene structure, and depth range. Experiments on KITTI, Bonn, NYUv2 Raw, TUM RGB-D, DDAD, and MS-Depth demonstrate competitive depth completion accuracy. On MS-Depth, StreamTTO achieves average processing speeds of approximately 15~FPS with VGGT and 25~FPS with MoGe-2 at an input resolution of 518 X 392.
Read the full abstract ▾
All authors Minseok Seo, Wonjun Lee, Jaehyuk Jang, Changick Kim
Computer Vision
Unifying Heterogeneous Multi-Modal Remote Sensing Detection Via Language-Pivoted Pretraining
Yuxuan Li, Yuming Chen, Yunheng Li et al. (+3 more)
Heterogeneous multi-modal remote sensing object detection aims to accurately detect objects from diverse sensors (e.g., RGB, SAR, Infrared). Existing approaches largely adopt a late
alignment paradigm, in which modality
alignment and task-specific optimization are entangled during downstream fine-tuning. This tight coupling complicates optimization and often results in unstable training and suboptimal generalization. To address these limitations, we propose BabelRS, a unified
language-pivoted pretraining framework that explicitly decouples modality
alignment from downstream task learning. BabelRS comprises two key components: Concept-Shared Instruction Aligning (CSIA) and Layerwise Visual-Semantic Annealing (LVSA). CSIA aligns each sensor modality to a shared set of linguistic concepts, using
language as a semantic pivot to bridge heterogeneous visual representations. To further mitigate the granularity mismatch between high-level
language representations and dense detection objectives, LVSA progressively aggregates multi-scale visual features to provide fine-grained semantic guidance. Extensive experiments demonstrate that BabelRS stabilizes training and consistently outperforms state-of-the-art methods without bells and whistles. Code: https://github.com/zcablii/SM3Det.
Read the full abstract ▾
All authors Yuxuan Li, Yuming Chen, Yunheng Li, Ming-Ming Cheng, Xiang Li, Jian Yang
Computer Vision
StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models
Keli Liu, Zhendong Wang, Wengang Zhou et al. (+1 more)
Visual AutoRegressive (VAR) models based on next-scale prediction enable efficient hierarchical generation, yet the inference cost grows quadratically at high resolutions. We observe that the computationally intensive later scales predominantly refine high-frequency textures and exhibit substantial spatial redundancy, in contrast to earlier scales that determine the global structural layout. Existing
pruning methods primarily focus on high-frequency detection for
token selection, often overlooking structural coherence and consequently degrading global semantics. To address this limitation, we propose StepVAR, a training-free
token pruning framework that accelerates VAR inference by jointly considering structural and textural importance. Specifically, we employ a lightweight high-pass filter to capture local texture details, while leveraging Principal Component Analysis (PCA) to preserve global structural information. This dual-criterion design enables the model to retain
tokens critical for both fine-grained fidelity and overall composition. To maintain valid next-scale prediction under sparse
tokens, we further introduce a nearest neighbor feature propagation strategy to reconstruct dense feature maps from pruned representations. Extensive experiments on state-of-the-art text-to-image and text-to-
video VAR models demonstrate that StepVAR achieves substantial inference speedups while maintaining generation quality. Quantitative and qualitative evaluations consistently show that our method outperforms existing acceleration approaches, validating its effectiveness and general applicability across diverse VAR architectures.
Read the full abstract ▾
All authors Keli Liu, Zhendong Wang, Wengang Zhou, Houqiang Li