Computer Vision
Contextual Range-View Projection for 3D LiDAR Point Clouds
Seyedali Mousavi, Seyedhamidreza Mousavi and Masoud Daneshtalab
Range-view projection provides an efficient method for transforming 3D
LiDAR point clouds into 2D range image representations, enabling effective processing with 2D deep learning models. However, a major challenge in this projection is the many-to-one conflict, where multiple 3D points are mapped onto the same pixel in the range image, requiring a selection strategy. Existing approaches typically retain the point with the smallest depth (closest to the
LiDAR), disregarding semantic relevance and object structure, which leads to the loss of important contextual information. In this paper, we extend the depth-based selection rule by incorporating contextual information from both instance centers and class labels, introducing two mechanisms: Centerness-Aware Projection (CAP) and Class-Weighted-Aware Projection (CWAP). In CAP, point depths are adjusted according to their distance from the instance center, thereby prioritizing central instance points over noisy boundary and background points. In CWAP, object classes are prioritized through user-defined weights, offering flexibility in the projection strategy. Our evaluations on the SemanticKITTI
dataset show that CAP preserves more instance points during projection, achieving up to a 3.1% mIoU improvement compared to the baseline. Furthermore, CWAP enhances the performance of targeted classes while having a negligible impact on the performance of other classes
Read the full abstract ▾
Computer Vision
Revisiting Aerial Scene Classification on the AID Benchmark
Subhajeet Das, Susmita Ghosh and Abhiroop Chatterjee
Aerial images play a vital role in urban planning and environmental preservation, as they consist of various structures, representing different types of buildings, forests, mountains, and unoccupied lands. Due to its heterogeneous nature, developing robust models for scene classification remains a challenge. In this study, we conduct a literature review of various machine learning methods for aerial image classification. Our survey covers a range of approaches from handcrafted features (e.g., SIFT, LBP) to traditional CNNs (e.g., VGG, GoogLeNet), and advanced deep hybrid networks. In this connection, we have also designed Aerial-Y-Net, a spatial
attention-enhanced CNN with multi-scale feature fusion mechanism, which acts as an
attention-based model and helps us to better understand the complexities of aerial images. Evaluated on the AID
dataset, our model achieves 91.72% accuracy, outperforming several baseline architectures.
Read the full abstract ▾
Computer Vision
Depth to Anatomy: Organ Localization from Depth Images for Automated Patient Table Positioning in Radiology Workflow
Eytan Kats, Kai Geissler, Daniel Mensing et al. (+4 more)
In clinical radiology, accurate patient table positioning is essential to align specific internal organs of interest with the scanner imaging isocenter, ensuring image quality and diagnostic reliability. Automated patient positioning can streamline this process and improve radiology workflow efficiency by reducing the time required for manual table adjustments and scout-based scan planning. We propose a learning-based framework that predicts 3D organ locations and shapes for 41 anatomical structures, including both bones and soft tissues, directly from a single 2D depth image of the body surface. Leveraging $10,020$ whole-body MRI scans from the German National Cohort (NAKO)
dataset, we synthetically generate depth images paired with anatomical
segmentations to train a convolutional neural network for volumetric organ prediction. Our method achieves a mean dice similarity coefficient of $0.44\pm0.2$ and and a symmetric average surface distance of $7.69\pm5.68$ mm across all structures. Furthermore, the model derives organ bounding boxes with a mean absolute detection offset of $10.99\pm5.54$ mm. Qualitative results on real-world depth images indicate the ability of the model to generalize to practical clinical settings. These findings suggest that depth-only organ localization can support automated patient positioning reducing setup time, minimizing operator variability, and improving patient comfort.
Read the full abstract ▾
All authors Eytan Kats, Kai Geissler, Daniel Mensing, Julien Senegas, Jochen G. Hirsch, Stefan Heldman, Mattias P. Heinrich
Computer Vision
Co-PLNet: A Collaborative Point-Line Network for Prompt-Guided Wireframe Parsing
Chao Wang, Xuanying Li, Cheng Dai et al. (+4 more)
Wireframe parsing aims to recover line segments and their junctions to form a structured
geometric representation useful for downstream tasks such as Simultaneous Localization and Mapping (SLAM). Existing methods predict lines and junctions separately and reconcile them post-hoc, causing mismatches and reduced robustness. We present Co-PLNet, a point-line collaborative framework that exchanges spatial cues between the two tasks, where early detections are converted into spatial
prompts via a Point-Line
Prompt Encoder (PLP-Encoder), which encodes
geometric attributes into compact and spatially aligned maps. A Cross-Guidance Line Decoder (CGL-Decoder) then refines predictions with sparse
attention conditioned on complementary
prompts, enforcing point-line consistency and efficiency. Experiments on Wireframe and YorkUrban show consistent improvements in accuracy and robustness, together with favorable real-time efficiency, demonstrating our effectiveness for structured geometry perception. Our code is available at https://github.com/GalacticHogrider/Co-PLNet.
Read the full abstract ▾
All authors Chao Wang, Xuanying Li, Cheng Dai, Jinglei Feng, Yuxiang Luo, Hao Qin, Yuqi Ouyang
Computer Vision
A multimodal vision foundation model for generalizable knee pathology
Kang Yu, Dingyu Wang, Zimu Yuan et al. (+8 more)
Musculoskeletal disorders represent a leading cause of global disability, creating an urgent demand for precise interpretation of medical imaging. Current artificial intelligence (AI) approaches in orthopedics predominantly rely on task-specific, supervised learning paradigms. These methods are inherently fragmented, require extensive annotated
datasets, and often lack generalizability across different modalities and clinical scenarios. The development of foundation models in this field has been constrained by the scarcity of large-scale, curated, and open-source musculoskeletal
datasets. To address these challenges, we introduce OrthoFoundation, a
multimodal vision foundation model optimized for musculoskeletal pathology. We constructed a pre-training
dataset of 1.2 million unlabeled knee X-ray and MRI images from internal and public databases. Utilizing a Dinov3 backbone, the model was trained via self-supervised contrastive learning to capture robust radiological representations. OrthoFoundation achieves state-of-the-art (SOTA) performance across 14 downstream tasks. It attained superior accuracy in X-ray osteoarthritis diagnosis and ranked first in MRI structural injury detection. The model demonstrated remarkable label efficiency, matching supervised baselines using only 50% of labeled data. Furthermore, despite being pre-trained on knee images, OrthoFoundation exhibited exceptional cross-anatomy generalization to the hip, shoulder, and ankle. OrthoFoundation represents a significant advancement toward general-purpose AI for musculoskeletal imaging. By learning fundamental, joint-agnostic radiological semantics from large-scale
multimodal data, it overcomes the limitations of conventional models, which provides a robust framework for reducing annotation burdens and enhancing diagnostic accuracy in clinical practice.
Read the full abstract ▾
All authors Kang Yu, Dingyu Wang, Zimu Yuan, Nan Zhou, Jiajun Liu, Jiaxin Liu, Shanggui Liu, Yaoyan Zheng, Huishu Yuan, Di Huang, Dong Jiang
Computer Vision
Vision-Language-Model-Guided Differentiable Ray Tracing for Fast and Accurate Multi-Material RF Parameter Estimation
Zerui Kang, Yishen Lim, Zhouyou Gu et al. (+3 more)
Accurate
radio-frequency (RF) material parameters are essential for electromagnetic digital twins in 6G systems, yet gradient-based inverse ray tracing (RT) remains sensitive to initialization and costly under limited measurements. This paper proposes a
vision-
language-model (VLM) guided framework that accelerates and stabilizes multi-material parameter estimation in a differentiable RT (DRT) engine. A VLM parses scene images to infer material categories and maps them to quantitative priors via an ITU-R material table, yielding informed conductivity initializations. The VLM further selects informative transmitter/receiver placements that promote diverse, material-discriminative paths. Starting from these priors, the DRT performs gradient-based refinement using measured received signal strengths. Experiments in NVIDIA Sionna on indoor scenes show 2-4$\times$ faster convergence and 10-100$\times$ lower final parameter error compared with uniform or random initialization and random placement baselines, achieving sub-0.1% mean relative error with only a few receivers. Complexity analyses indicate per-iteration time scales near-linearly with the number of materials and measurement setups, while VLM-guided placement reduces the measurements required for accurate recovery. Ablations over RT depth and ray counts confirm further accuracy gains without significant per-iteration overhead. Results demonstrate that semantic priors from VLMs effectively guide physics-based optimization for fast and reliable RF material estimation.
Read the full abstract ▾
All authors Zerui Kang, Yishen Lim, Zhouyou Gu, Seung-Woo Ko, Tony Q.S. Quek, Jihong Park
Computer Vision
V-Loop: Visual Logical Loop Verification for Hallucination Detection in Medical Visual Question Answering
Mengyuan Jin, Zehui Liao and Yong Xia
Multimodal Large
Language Models (MLLMs) have shown remarkable capability in assisting disease diagnosis in medical visual question answering (VQA). However, their outputs remain vulnerable to
hallucinations (i.e., responses that contradict visual facts), posing significant risks in high-stakes medical scenarios. Recent introspective detection methods, particularly uncertainty-based approaches, offer computational efficiency but are fundamentally indirect, as they estimate predictive uncertainty for an image-question pair rather than verifying the factual correctness of a specific answer. To address this limitation, we propose Visual Logical Loop Verification (V-Loop), a training-free and plug-and-play framework for
hallucination detection in medical VQA. V-Loop introduces a bidirectional
reasoning process that forms a visually grounded logical loop to verify factual correctness. Given an input, the MLLM produces an answer for the primary input pair. V-Loop extracts semantic units from the primary QA pair, generates a verification question by conditioning on the answer unit to re-query the question unit, and enforces visual
attention consistency to ensure answering both primary question and verification question rely on the same image evidence. If the verification answer matches the expected semantic content, the logical loop closes, indicating factual grounding; otherwise, the primary answer is flagged as hallucinated. Extensive experiments on multiple medical VQA
benchmarks and MLLMs show that V-Loop consistently outperforms existing introspective methods, remains highly efficient, and further boosts uncertainty-based approaches when used in combination.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
TechING: Towards Real World Technical Image Understanding via VLMs
Tafazzul Nadeem, Bhavik Shangari, Manish Rai et al. (+2 more)
Professionals working in technical domain typically hand-draw (on whiteboard, paper, etc.) technical diagrams (e.g., flowcharts, block diagrams, etc.) during discussions; however, if they want to edit these later, it needs to be drawn from scratch. Modern day VLMs have made tremendous progress in image understanding but they struggle when it comes to understanding technical diagrams. One way to overcome this problem is to fine-tune on real world hand-drawn images, but it is not practically possible to generate large number of such images. In this paper, we introduce a large synthetically generated corpus (reflective of real world images) for training VLMs and subsequently evaluate VLMs on a smaller corpus of hand-drawn images (with the help of humans). We introduce several new self-supervision tasks for training and perform extensive experiments with various baseline models and fine-tune Llama 3.2 11B-instruct model on synthetic images on these tasks to obtain LLama-VL-TUG, which significantly improves the ROUGE-L performance of Llama 3.2 11B-instruct by 2.14x and achieves the best all-round performance across all baseline models. On real-world images, human evaluation reveals that we achieve minimum compilation errors across all baselines in 7 out of 8 diagram types and improve the average F1 score of Llama 3.2 11B-instruct by 6.97x.
Read the full abstract ▾
All authors Tafazzul Nadeem, Bhavik Shangari, Manish Rai, Gagan Raj Gupta, Ashutosh Modi
Computer Vision
Facial Emotion Recognition on FER-2013 using an EfficientNetB2-Based Approach
Sahil Naik, Soham Bagayatkar and Pavankumar Singh
Detection of human
emotions based on facial images in real-world scenarios is a difficult task due to low image quality, variations in lighting, pose changes, background distractions, small inter-class variations, noisy crowd-sourced labels, and severe class imbalance, as observed in the FER-2013
dataset of 48x48 grayscale images. Although recent approaches using large CNNs such as VGG and ResNet achieve reasonable accuracy, they are computationally expensive and
memory-intensive, limiting their practicality for real-time applications. We address these challenges using a lightweight and efficient facial
emotion recognition pipeline based on EfficientNetB2, trained using a two-stage warm-up and fine-tuning strategy. The model is enhanced with AdamW optimization, decoupled weight decay, label smoothing (epsilon = 0.06) to reduce annotation noise, and clipped class weights to mitigate class imbalance, along with dropout, mixed-precision training, and extensive real-time data augmentation. The model is trained using a stratified 87.5%/12.5% train-validation split while keeping the official test set intact, achieving a test accuracy of 68.78% with nearly ten times fewer parameters than VGG16-based baselines. Experimental results, including per-class metrics and learning dynamics, demonstrate stable training and strong generalization, making the proposed approach suitable for real-time and edge-based applications.
Read the full abstract ▾
Computer Vision
HomoFM: Deep Homography Estimation with Flow Matching
Mengfan He, Liangzheng Sun, Chunyu Li et al. (+1 more)
Deep homography estimation has broad applications in computer
vision and robotics. Remarkable progresses have been achieved while the existing methods typically treat it as a direct regression or iterative refinement problem and often struggling to capture complex
geometric transformations or generalize across different domains. In this work, we propose HomoFM, a new framework that introduces the flow matching technique from generative modeling into the homography estimation task for the first time. Unlike the existing methods, we formulate homography estimation problem as a velocity field learning problem. By modeling a continuous and point-wise velocity field that transforms noisy distributions into registered coordinates, the proposed network recovers high-precision transformations through a conditional flow trajectory. Furthermore, to address the challenge of domain shifts issue, e.g., the cases of
multimodal matching or varying illumination scenarios, we integrate a gradient reversal layer (GRL) into the feature extraction backbone. This domain adaptation strategy explicitly constrains the encoder to learn domain-invariant representations, significantly enhancing the network's robustness. Extensive experiments demonstrate the effectiveness of the proposed method, showing that HomoFM outperforms state-of-the-art methods in both estimation accuracy and robustness on standard
benchmarks. Code and data resource are available at https://github.com/hmf21/HomoFM.
Read the full abstract ▾
All authors Mengfan He, Liangzheng Sun, Chunyu Li, Ziyang Meng
Computer Vision · Cross-listed from physics.med-ph
Automated HER2 scoring with uncertainty quantification using lensfree holography and deep learning
Che-Yung Shen, Xilin Yang, Yuzhu Li et al. (+2 more)
Accurate assessment of human epidermal growth factor receptor 2 (HER2) expression is critical for breast cancer diagnosis, prognosis, and therapy selection; yet, most existing digital HER2 scoring methods rely on bulky and expensive optical systems. Here, we present a compact and cost-effective lensfree holography platform integrated with deep learning for automated HER2 scoring of immunohistochemically stained breast tissue sections. The system captures lensfree diffraction patterns of stained HER2 tissue sections under RGB
laser illumination and acquires complex field information over a sample area of ~1,250 mm^2 at an effective throughput of ~84 mm^2 per minute. To enhance diagnostic reliability, we incorporated an uncertainty quantification strategy based on
Bayesian Monte Carlo dropout, which provides autonomous uncertainty estimates for each prediction and supports reliable, robust HER2 scoring, with an overall correction rate of 30.4%. Using a blinded test set of 412 unique tissue samples, our approach achieved a testing accuracy of 84.9% for 4-class (0, 1+, 2+, 3+) HER2 classification and 94.8% for binary (0/1+ vs. 2+/3+) HER2 scoring with uncertainty quantification. Overall, this lensfree holography approach provides a practical pathway toward portable, high-throughput, and cost-effective HER2 scoring, particularly suited for resource-limited settings, where traditional digital pathology infrastructure is unavailable.
Read the full abstract ▾
All authors Che-Yung Shen, Xilin Yang, Yuzhu Li, Leon Lenk, Aydogan Ozcan
Computer Vision
QualiRAG: Retrieval-Augmented Generation for Visual Quality Understanding
Linhan Cao, Wei Sun, Weixia Zhang et al. (+6 more)
Visual quality assessment (VQA) is increasingly shifting from scalar score prediction toward interpretable quality understanding – a paradigm that demands fine-grained spatiotemporal perception and auxiliary contextual information. Current approaches rely on supervised fine-tuning or reinforcement learning on curated instruction
datasets, which involve labor-intensive annotation and are prone to
dataset-specific biases. To address these challenges, we propose QualiRAG, a training-free
Retrieval-Augmented Generation (RAG) framework that systematically leverages the latent perceptual knowledge of large
multimodal models (LMMs) for visual quality perception. Unlike conventional RAG that retrieves from static corpora, QualiRAG dynamically generates auxiliary knowledge by decomposing questions into structured requests and constructing four complementary knowledge sources: visual metadata, subject localization, global quality summaries, and local quality descriptions, followed by relevance-aware
retrieval for evidence-grounded
reasoning. Extensive experiments show that QualiRAG achieves substantial improvements over open-source general-purpose LMMs and VQA-finetuned LMMs on visual quality understanding tasks, and delivers competitive performance on visual quality comparison tasks, demonstrating robust quality assessment capabilities without any task-specific training. The code will be publicly available at https://github.com/clh124/QualiRAG.
Read the full abstract ▾
All authors Linhan Cao, Wei Sun, Weixia Zhang, Xiangyang Zhu, Kaiwei Zhang, Jun Jia, Dandan Zhu, Guangtao Zhai, Xiongkuo Min
Computer Vision
MindCine: Multimodal EEG-to-Video Reconstruction with Large-Scale Pretrained Models
Tian-Yi Zhou, Xuan-Hao Liu, Bao-Liang Lu et al. (+1 more)
Reconstructing human dynamic visual perception from electroencephalography (
EEG) signals is of great research significance since
EEG's non-invasiveness and high temporal resolution. However,
EEG-to-
video reconstruction remains challenging due to: 1) Single Modality: existing studies solely align
EEG signals with the text modality, which ignores other modalities and are prone to suffer from overfitting problems; 2) Data Scarcity: current methods often have difficulty training to converge with limited
EEG-
video data. To solve the above problems, we propose a novel framework MindCine to achieve high-fidelity
video reconstructions on limited data. We employ a
multimodal joint learning strategy to incorporate beyond-text modalities in the training stage and leverage a pre-trained large
EEG model to relieve the data scarcity issue for decoding semantic information, while a Seq2Seq model with causal
attention is specifically designed for decoding perceptual information. Extensive experiments demonstrate that our model outperforms state-of-the-art methods both qualitatively and quantitatively. Additionally, the results underscore the effectiveness of the complementary strengths of different modalities and demonstrate that leveraging a large-scale
EEG model can further enhance reconstruction performance by alleviating the challenges associated with limited data.
Read the full abstract ▾
All authors Tian-Yi Zhou, Xuan-Hao Liu, Bao-Liang Lu, Wei-Long Zheng
Computer Vision
Multi-Perspective Subimage CLIP with Keyword Guidance for Remote Sensing Image-Text Retrieval
Yifan Li, Shiying Wang and Jianqiang Huang
Vision-
Language Pre-training (VLP) models like CLIP have significantly advanced Remote Sensing Image-Text
Retrieval (RSITR). However, existing methods predominantly rely on coarse-grained global
alignment, which often overlooks the dense, multi-scale semantics inherent in overhead imagery. Moreover, adapting these heavy models via full fine-tuning incurs prohibitive computational costs and risks catastrophic forgetting. To address these challenges, we propose MPS-CLIP, a parameter-efficient framework designed to shift the
retrieval paradigm from global matching to keyword-guided fine-grained
alignment. Specifically, we leverage a Large
Language Model (LLM) to extract core semantic keywords, guiding the Segment Anything Model (SamGeo) to generate semantically relevant sub-perspectives. To efficiently adapt the frozen backbone, we introduce a Gated Global
Attention (G^2A) adapter, which captures global context and long-range dependencies with minimal overhead. Furthermore, a Multi-Perspective Representation (MPR) module aggregates these local cues into robust multi-perspective
embeddings. The framework is optimized via a hybrid objective combining multi-perspective contrastive and weighted triplet losses, which dynamically selects maximum-response perspectives to suppress noise and enforce precise semantic matching. Extensive experiments on the RSICD and RSITMD
benchmarks demonstrate that MPS-CLIP achieves state-of-the-art performance with 35.18% and 48.40% mean Recall (mR), respectively, significantly outperforming full fine-tuning baselines and recent competitive methods. Code is available at https://github.com/Lcrucial1f/MPS-CLIP.
Read the full abstract ▾
Computer Vision
Weiye Zhu, Zekai Zhang, Xiangchen Wang et al. (+5 more)
Vision-and-
Language Navigation (VLN) requires
agents to interpret natural
language instructions and act coherently in visually rich environments. However, most existing methods rely on reactive state-action mappings without explicitly action-grounded visual dynamics modeling. Lacking awareness of how actions transform subsequent visual observations,
agents cannot plan actions rationally, leading to unstable behaviors, weak generalization, and cumulative error along trajectory. To address these issues, we introduce NaVIDA (
Navigation with Inverse Dynamics Augmentation), a lightweight VLN framework that incorporates inverse dynamics supervision (IDS) as an explicit objective to embed action-grounded visual dynamics into policy learning. By jointly optimizing this visual dynamics with instruction-conditioned action prediction in a shared representation and action space, NaVIDA provides additional structured supervision that regularizes learning and leads to more stable and consistent
navigation. To structure this supervision and extend the effective planning range, NaVIDA employs hierarchical probabilistic action chunking (HPAC), which organizes trajectories into multi-step chunks and provides discriminative, longer-range visual-change cues. Extensive experiments show that NaVIDA achieves superior
navigation performance compared to state-of-the-art methods with fewer parameters (3B vs. 8B). Real-world
robot evaluations further validate the practical feasibility and effectiveness of our approach.
Read the full abstract ▾
All authors Weiye Zhu, Zekai Zhang, Xiangchen Wang, Hewei Pan, Teng Wang, Tiantian Geng, Rongtao Xu, Feng Zheng
Computer Vision
YOLO-DS: Fine-Grained Feature Decoupling via Dual-Statistic Synergy Operator for Object Detection
Lin Huang, Yujuan Tan, Weisheng Li et al. (+6 more)
One-stage object detection, particularly the YOLO series, strikes a favorable balance between accuracy and efficiency. However, existing YOLO detectors lack explicit modeling of heterogeneous object responses within shared feature channels, which limits further performance gains. To address this, we propose YOLO-DS, a framework built around a novel Dual-Statistic Synergy Operator (DSO). The DSO decouples object features by jointly modeling the channel-wise mean and the peak-to-mean difference. Building upon the DSO, we design two lightweight gating modules: the Dual-Statistic Synergy Gating (DSG) module for adaptive channel-wise feature selection, and the Multi-Path Segmented Gating (MSG) module for depth-wise feature weighting. On the MS-COCO
benchmark, YOLO-DS consistently outperforms YOLOv8 across five model scales (N, S, M, L, X), achieving AP gains of 1.1% to 1.7% with only a minimal increase in inference latency. Extensive visualization, ablation, and comparative studies validate the effectiveness of our approach, demonstrating its superior capability in discriminating heterogeneous objects with high efficiency.
Read the full abstract ▾
All authors Lin Huang, Yujuan Tan, Weisheng Li, Shitai Shan, Liu Liu, Bo Liu, Linlin Shen, Jing Yu, Yue Niu
Computer Vision
TempDiffReg: Temporal Diffusion Model for Non-Rigid 2D-3D Vascular Registration
Zehua Liu, Shihao Zou, Jincai Huang et al. (+3 more)
Transarterial chemoembolization (TACE) is a preferred treatment option for hepatocellular carcinoma and other liver malignancies, yet it remains a highly challenging procedure due to complex intra-operative vascular
navigation and anatomical variability. Accurate and robust 2D-3D vessel registration is essential to guide microcatheter and instruments during TACE, enabling precise localization of vascular structures and optimal therapeutic targeting. To tackle this issue, we develop a coarse-to-fine registration strategy. First, we introduce a global
alignment module, structure-aware perspective n-point (SA-PnP), to establish correspondence between 2D and 3D vessel structures. Second, we propose TempDiffReg, a temporal
diffusion model that performs vessel deformation iteratively by leveraging temporal context to capture complex anatomical variations and local structural changes. We collected data from 23 patients and constructed 626 paired multi-frame samples for comprehensive evaluation. Experimental results demonstrate that the proposed method consistently outperforms state-of-the-art (SOTA) methods in both accuracy and anatomical plausibility. Specifically, our method achieves a mean squared error (MSE) of 0.63 mm and a mean absolute error (MAE) of 0.51 mm in registration accuracy, representing 66.7% lower MSE and 17.7% lower MAE compared to the most competitive existing approaches. It has the potential to assist less-experienced clinicians in safely and efficiently performing complex TACE procedures, ultimately enhancing both
surgical outcomes and patient care. Code and data are available at: \textcolor{blue}{https://github.com/LZH970328/TempDiffReg.git}
Read the full abstract ▾
All authors Zehua Liu, Shihao Zou, Jincai Huang, Yanfang Zhang, Chao Tong, Weixin Si
Computer Vision
Agentic Very Long Video Understanding
Aniket Rege, Arka Sadhu, Yuliang Li et al. (+5 more)
The advent of always-on personal AI assistants, enabled by all-day wearable devices such as smart glasses, demands a new level of contextual understanding, one that goes beyond short, isolated events to encompass the continuous, longitudinal stream of
egocentric video. Achieving this
vision requires advances in long-horizon
video understanding, where systems must interpret and recall visual and audio information spanning days or even weeks. Existing methods, including large
language models and
retrieval-augmented generation, are constrained by limited context windows and lack the ability to perform compositional, multi-hop
reasoning over very long
video streams. In this work, we address these challenges through EGAgent, an enhanced agentic framework centered on entity scene graphs, which represent people, places, objects, and their relationships over time. Our system equips a planning
agent with tools for structured search and
reasoning over these graphs, as well as hybrid visual and audio search capabilities, enabling detailed, cross-modal, and temporally coherent
reasoning. Experiments on the EgoLifeQA and
Video-MME (Long)
datasets show that our method achieves state-of-the-art performance on EgoLifeQA (57.5%) and competitive performance on
Video-MME (Long) (74.1%) for complex longitudinal
video understanding tasks. Code is available at https://github.com/facebookresearch/egagent.
Read the full abstract ▾
All authors Aniket Rege, Arka Sadhu, Yuliang Li, Kejie Li, Ramya Korlakai Vinayak, Yuning Chai, Yong Jae Lee, Hyo Jin Kim
Computer Vision
Forward Consistency Learning with Gated Context Aggregation for Video Anomaly Detection
Jiahao Lyu, Minghua Zhao, Xuewen Huang et al. (+5 more)
As a crucial element of public security,
video anomaly detection (VAD) aims to measure deviations from normal patterns for various events in real-time surveillance systems. However, most existing VAD methods rely on large-scale models to pursue extreme accuracy, limiting their feasibility on resource-limited edge devices. Moreover, mainstream prediction-based VAD detects anomalies using only single-frame future prediction errors, overlooking the richer constraints from longer-term temporal forward information. In this paper, we introduce FoGA, a lightweight VAD model that performs Forward consistency learning with Gated context Aggregation, containing about 2M parameters and tailored for potential edge devices. Specifically, we propose a Unet-based method that performs feature extraction on consecutive frames to generate both immediate and forward predictions. Then, we introduce a gated context aggregation module into the skip connections to dynamically fuse encoder and decoder features at the same spatial scale. Finally, the model is jointly optimized with a novel forward consistency loss, and a hybrid anomaly measurement strategy is adopted to integrate errors from both immediate and forward frames for more accurate detection. Extensive experiments demonstrate the effectiveness of the proposed method, which substantially outperforms state-of-the-art competing methods, running up to 155 FPS. Hence, our FoGA achieves an excellent trade-off between performance and the efficiency metric.
Read the full abstract ▾
All authors Jiahao Lyu, Minghua Zhao, Xuewen Huang, Yifei Chen, Shuangli Du, Jing Hu, Cheng Shi, Zhiyong Lv
Computer Vision · Cross-listed from cs.RO
Grasp-and-Lift: Executable 3D Hand-Object Interaction Reconstruction via Physics-in-the-Loop Optimization
Byeonggyeol Choi, Woojin Oh and Jongwoo Lim
Dexterous hand manipulation increasingly relies on large-scale motion
datasets with precise hand-object trajectory data. However, existing resources such as DexYCB and HO3D are primarily optimized for visual
alignment but often yield physically implausible interactions when replayed in physics simulators, including penetration, missed contact, and unstable grasps. We propose a simulation-in-the-loop refinement framework that converts these visually aligned trajectories into physically executable ones. Our core contribution is to formulate this as a tractable black-box optimization problem. We parameterize the hand's motion using a low-dimensional, spline-based representation built on sparse temporal keyframes. This allows us to use a powerful gradient-free optimizer, CMA-ES, to treat the high-fidelity physics engine as a black-box objective function. Our method finds motions that simultaneously maximize physical success (e.g., stable grasp and lift) while minimizing deviation from the original human demonstration. Compared to MANIPTRANS-recent transfer pipelines, our approach achieves lower hand and object pose errors during replay and more accurately recovers hand-object physical interactions. Our approach provides a general and scalable method for converting visual demonstrations into physically valid trajectories, enabling the generation of high-fidelity data crucial for robust policy learning.
Read the full abstract ▾
Computer Vision
LungCRCT: Causal Representation based Lung CT Processing for Lung Cancer Treatment
Daeyoung Kim
Due to silence in early stages, lung cancer has been one of the most leading causes of mortality in cancer patients world-wide. Moreover, major symptoms of lung cancer are hard to differentiate with other respiratory disease symptoms such as COPD, further leading patients to overlook cancer progression in early stages. Thus, to enhance survival rates in lung cancer, early detection from consistent proactive respiratory system monitoring becomes crucial. One of the most prevalent and effective methods for lung cancer monitoring would be low-dose computed
tomography(LDCT) chest scans, which led to remarkable enhancements in lung cancer detection or tumor classification tasks under rapid advancements and applications of computer
vision based AI models such as EfficientNet or ResNet in image processing. However, though advanced CNN models under transfer learning or ViT based models led to high performing lung cancer detections, due to its intrinsic limitations in terms of correlation dependence and low interpretability due to complexity, expansions of deep learning models to lung cancer treatment analysis or causal intervention analysis simulations are still limited. Therefore, this research introduced LungCRCT: a latent causal representation learning based lung cancer analysis framework that retrieves causal representations of factors within the physical causal mechanism of lung cancer progression. With the use of advanced graph autoencoder based causal discovery
algorithms with distance Correlation disentanglement and
entropy-based image reconstruction refinement, LungCRCT not only enables causal intervention analysis for lung cancer treatments, but also leads to robust, yet extremely light downstream models in malignant tumor classification tasks with an AUC score of 93.91%.
Read the full abstract ▾
Computer Vision
James Tribble, Hao Wang, Si-En Hong et al. (+4 more)
Long-horizon
egocentric video presents significant challenges for visual
navigation due to viewpoint drift and the absence of persistent
geometric context. Although recent
vision-
language models perform well on image and short-
video reasoning, their spatial
reasoning capability in long
egocentric sequences remains limited. In this work, we study how explicit spatial signals influence VLM-based
video understanding without modifying model architectures or inference procedures. We introduce Sanpo-D, a fine-grained re-annotation of the Google Sanpo
dataset, and
benchmark multiple VLMs on
navigation-oriented spatial queries. To examine input-level inductive bias, we further fuse depth maps with RGB frames and evaluate their impact on spatial
reasoning. Our results reveal a trade-off between general-purpose accuracy and spatial specialization, showing that depth-aware and spatially grounded representations can improve performance on safety-critical tasks such as pedestrian and obstruction detection.
Read the full abstract ▾
All authors James Tribble, Hao Wang, Si-En Hong, Chaoyi Zhou, Ashish Bastola, Siyu Huang, Abolfazl Razi
Computer Vision
Computational Framework for Estimating Relative Gaussian Blur Kernels between Image Pairs
Akbar Saadat
Following the earlier verification for Gaussian model in \cite{ASaa2026}, this paper introduces a zero training forward computational framework for the model to realize it in real time applications. The framework is based on discrete calculation of the analytic expression of the defocused image from the sharper one for the application range of the standard deviation of the Gaussian kernels and selecting the best matches. The analytic expression yields multiple solutions at certain image points, but is filtered down to a single solution using similarity measures over neighboring points.The framework is structured to handle cases where two given images are partial blurred versions of each other. Experimental evaluations on real images demonstrate that the proposed framework achieves a mean absolute error (MAE) below $1.7\%$ in estimating synthetic blur values. Furthermore, the discrepancy between actual blurred image intensities and their corresponding estimates remains under $2\%$, obtained by applying the extracted defocus filters to less blurred images.
Read the full abstract ▾
Computer Vision
Text-Pass Filter: An Efficient Scene Text Detector
Chuang Yang, Haozhao Ma, Xu Han et al. (+2 more)
To pursue an efficient text assembling process, existing methods detect texts via the shrink-mask expansion strategy. However, the shrinking operation loses the visual features of text margins and confuses the foreground and background difference, which brings intrinsic limitations to recognize text features. We follow this issue and design Text-Pass Filter (TPF) for arbitrary-shaped text detection. It segments the whole text directly, which avoids the intrinsic limitations. It is noteworthy that different from previous whole text region-based methods, TPF can separate adhesive texts naturally without complex decoding or post-processing processes, which makes it possible for real-time text detection. Concretely, we find that the band-pass filter allows through components in a specified band of frequencies, called its passband but blocks components with frequencies above or below this band. It provides a natural idea for extracting whole texts separately. By simulating the band-pass filter, TPF constructs a unique feature-filter pair for each text. In the inference stage, every filter extracts the corresponding matched text by passing its pass-feature and blocking other features. Meanwhile, considering the large aspect ratio problem of ribbon-like texts makes it hard to recognize texts wholly, a Reinforcement Ensemble Unit (REU) is designed to enhance the feature consistency of the same text and to enlarge the filter's recognition field to help recognize whole texts. Furthermore, a Foreground Prior Unit (FPU) is introduced to encourage TPF to discriminate the difference between the foreground and background, which improves the feature-filter pair quality. Experiments demonstrate the effectiveness of REU and FPU while showing the TPF's superiority.
Read the full abstract ▾
All authors Chuang Yang, Haozhao Ma, Xu Han, Yuan Yuan, Qi Wang
Computer Vision
Cross-Domain Transfer with Self-Supervised Spectral-Spatial Modeling for Hyperspectral Image Classification
Jianshu Chao, Tianhua Lv, Qiqiong Ma et al. (+4 more)
Self-supervised learning has demonstrated considerable potential in hyperspectral representation, yet its application in cross-domain transfer scenarios remains under-explored. Existing methods, however, still rely on source domain annotations and are susceptible to distribution shifts, leading to degraded generalization performance in the target domain. To address this, this paper proposes a self-supervised cross-domain transfer framework that learns transferable spectral-spatial joint representations without source labels and achieves efficient adaptation under few samples in the target domain. During the self-supervised pre-training phase, a Spatial-Spectral
Transformer (S2Former) module is designed. It adopts a dual-branch spatial-spectral
transformer and introduces a bidirectional cross-
attention mechanism to achieve spectral-spatial collaborative modeling: the spatial branch enhances structural awareness through random masking, while the spectral branch captures fine-grained differences. Both branches mutually guide each other to improve semantic consistency. We further propose a Frequency Domain Constraint (FDC) to maintain frequency-domain consistency through real Fast
Fourier Transform (rFFT) and high-frequency magnitude loss, thereby enhancing the model's capability to discern fine details and boundaries. During the fine-tuning phase, we introduce a
Diffusion-Aligned Fine-tuning (DAFT)
distillation mechanism. This aligns semantic
evolution trajectories through a teacher-student structure, enabling robust transfer learning under low-label conditions. Experimental results demonstrate stable classification performance and strong cross-domain adaptability across four hyperspectral
datasets, validating the method's effectiveness under resource-constrained conditions.
Read the full abstract ▾
All authors Jianshu Chao, Tianhua Lv, Qiqiong Ma, Yunfei Qiu, Li Fang, Huifang Shen, Wei Yao
Computer Vision · Cross-listed from quant-ph
Differentiable Architecture Search for Adversarially Robust Quantum Computer Vision
Mohamed Afane, Quanjiang Long, Haoting Shen et al. (+4 more)
Current
quantum neural networks suffer from extreme sensitivity to both
adversarial perturbations and hardware noise, creating a significant barrier to real-world deployment. Existing robustness techniques typically sacrifice clean accuracy or require prohibitive computational resources. We propose a hybrid
quantum-classical Differentiable
Quantum Architecture Search (DQAS) framework that addresses these limitations by jointly optimizing
circuit structure and robustness through gradient-based methods. Our approach enhances traditional DQAS with a lightweight Classical Noise Layer applied before
quantum processing, enabling simultaneous optimization of gate selection and noise parameters. This design preserves the
quantum circuit's integrity while introducing trainable perturbations that enhance robustness without compromising standard performance. Experimental validation on MNIST, FashionMNIST, and CIFAR
datasets shows consistent improvements in both clean and
adversarial accuracy compared to existing
quantum architecture search methods. Under various attack scenarios, including Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), Basic Iterative Method (BIM), and Momentum Iterative Method (MIM), and under realistic
quantum noise conditions, our hybrid framework maintains superior performance. Testing on actual
quantum hardware confirms the practical viability of discovered architectures. These results demonstrate that strategic classical preprocessing combined with differentiable
quantum architecture optimization can significantly enhance
quantum neural network robustness while maintaining computational efficiency.
Read the full abstract ▾
All authors Mohamed Afane, Quanjiang Long, Haoting Shen, Ying Mao, Junaid Farooq, Ying Wang, Juntao Chen
Computer Vision
Semi-Supervised Hyperspectral Image Classification with Edge-Aware Superpixel Label Propagation and Adaptive Pseudo-Labeling
Yunfei Qiu, Qiqiong Ma, Tianhua Lv et al. (+3 more)
Significant progress has been made in semi-supervised hyperspectral image (HSI) classification regarding feature extraction and classification performance. However, due to high annotation costs and limited sample availability, semi-supervised learning still faces challenges such as boundary label
diffusion and pseudo-label instability. To address these issues, this paper proposes a novel semi-supervised hyperspectral classification framework integrating spatial prior information with a dynamic learning mechanism. First, we design an Edge-Aware Superpixel Label Propagation (EASLP) module. By integrating edge intensity penalty with neighborhood correction strategy, it mitigates label
diffusion from superpixel
segmentation while enhancing classification robustness in boundary regions. Second, we introduce a Dynamic History-Fused Prediction (DHP) method. By maintaining historical predictions and dynamically weighting them with current results, DHP smoothens pseudo-label fluctuations and improves temporal consistency and noise resistance. Concurrently, incorporating condifence and consistency measures, the Adaptive Tripartite Sample Categorization (ATSC) strategy implements hierarchical utilization of easy, ambiguous, and hard samples, leading to enhanced pseudo-label quality and learning efficiency. The Dynamic Reliability-Enhanced Pseudo-Label Framework (DREPL), composed of DHP and ATSC, strengthens pseudo-label stability across temporal and sample domains. Through synergizes operation with EASLP, it achieves spatio-temporal consistency optimization. Evaluations on four
benchmark datasets demonstrate its capability to maintain superior classification performance.
Read the full abstract ▾
All authors Yunfei Qiu, Qiqiong Ma, Tianhua Lv, Li Fang, Shudong Zhou, Wei Yao