Computer Vision
ReCo-KD: Region- and Context-Aware Knowledge Distillation for Efficient 3D Medical Image Segmentation
Qizhen Lan, Yu-Chun Hsu, Nida Saddaf Khan et al. (+1 more)
Accurate 3D medical image
segmentation is vital for diagnosis and treatment planning, but state-of-the-art models are often too large for clinics with limited computing resources. Lightweight architectures typically suffer significant performance loss. To address these deployment and speed constraints, we propose Region- and Context-aware Knowledge
Distillation (ReCo-KD), a training-only framework that transfers both fine-grained anatomical detail and long-range contextual information from a high-capacity teacher to a compact student network. The framework integrates Multi-Scale Structure-Aware Region
Distillation (MS-SARD), which applies class-aware masks and scale-normalized weighting to emphasize small but clinically important regions, and Multi-Scale Context
Alignment (MS-CA), which aligns teacher-student affinity patterns across feature levels. Implemented on nnU-Net in a backbone-agnostic manner, ReCo-KD requires no custom student design and is easily adapted to other architectures. Experiments on multiple public 3D medical
segmentation datasets and a challenging aggregated
dataset show that the distilled lightweight model attains accuracy close to the teacher while markedly reducing parameters and inference latency, underscoring its practicality for clinical deployment.
Read the full abstract ▾
All authors Qizhen Lan, Yu-Chun Hsu, Nida Saddaf Khan, Xiaoqian Jiang
Computer Vision
M3SR: Multi-Scale Multi-Perceptual Mamba for Efficient Spectral Reconstruction
Yuze Zhang, Lingjie Li, Qiuzhen Lin et al. (+3 more)
The Mamba architecture has been widely applied to various low-level
vision tasks due to its exceptional adaptability and strong performance. Although the Mamba architecture has been adopted for spectral reconstruction, it still faces the following two challenges: (1) Single spatial perception limits the ability to fully understand and analyze hyperspectral images; (2) Single-scale feature extraction struggles to capture the complex structures and fine details present in hyperspectral images. To address these issues, we propose a multi-scale, multi-perceptual Mamba architecture for the spectral reconstruction task, called M3SR. Specifically, we design a multi-perceptual fusion block to enhance the ability of the model to comprehensively understand and analyze the input features. By integrating the multi-perceptual fusion block into a U-Net structure, M3SR can effectively extract and fuse global, intermediate, and local features, thereby enabling accurate reconstruction of hyperspectral images at multiple scales. Extensive quantitative and qualitative experiments demonstrate that the proposed M3SR outperforms existing state-of-the-art methods while incurring a lower computational cost.
Read the full abstract ▾
All authors Yuze Zhang, Lingjie Li, Qiuzhen Lin, Zhong Ming, Fei Yu, Victor C. M. Leung
Computer Vision
KidVis: Do Multimodal Large Language Models Possess the Visual Perceptual Capabilities of a 6-Year-Old?
Xianfeng Wang, Kaiwei Zhang, Qi Jia et al. (+3 more)
While
Multimodal Large
Language Models (MLLMs) have demonstrated impressive proficiency in high-level
reasoning tasks, such as complex diagrammatic interpretation, it remains an open question whether they possess the fundamental visual primitives comparable to human intuition. To investigate this, we introduce KidVis, a novel
benchmark grounded in the theory of human visual development. KidVis deconstructs visual intelligence into six atomic capabilities - Concentration, Tracking, Discrimination,
Memory, Spatial, and Closure - already possessed by 6-7 year old children, comprising 10 categories of low-semantic-dependent visual tasks. Evaluating 20 state-of-the-art MLLMs against a human physiological baseline reveals a stark performance disparity. Results indicate that while human children achieve a near-perfect average score of 95.32, the state-of-the-art GPT-5 attains only 67.33. Crucially, we observe a "Scaling Law Paradox": simply increasing model parameters fails to yield linear improvements in these foundational visual capabilities. This study confirms that current MLLMs, despite their
reasoning prowess, lack the essential physiological perceptual primitives required for generalized visual intelligence.
Read the full abstract ▾
All authors Xianfeng Wang, Kaiwei Zhang, Qi Jia, Zijian Chen, Guangtao Zhai, Xiongkuo Min
Computer Vision
One-Shot Identification with Different Neural Network Approaches
Janis Mohr and Jörg Frochte
Convolutional neural networks (CNNs) have been widely used in the computer
vision community, significantly improving the state-of-the-art. But learning good features often is computationally expensive in machine learning settings and is especially difficult when there is a lack of data. One-shot learning is one such area where only limited data is available. In one-shot learning, predictions have to be made after seeing only one example from one class, which requires special techniques. In this paper we explore different approaches to one-shot identification tasks in different domains including an industrial application and face recognition. We use a special technique with stacked images and use siamese capsule networks. It is encouraging to see that the approach using capsule architecture achieves strong results and exceeds other techniques on a wide range of
datasets from industrial application to face recognition
benchmarks while being easy to use and optimise.
Read the full abstract ▾
Computer Vision
HIPPO: Accelerating Video Large Language Models Inference via Holistic-aware Parallel Speculative Decoding
Qitan Lv, Tianyu Liu, Wen Wu et al. (+4 more)
Speculative decoding (SD) has emerged as a promising approach to accelerate LLM inference without sacrificing output quality. Existing SD methods tailored for
video-LLMs primarily focus on
pruning redundant visual
tokens to mitigate the computational burden of massive visual inputs. However, existing methods do not achieve inference acceleration comparable to text-only LLMs. We observe from extensive experiments that this phenomenon mainly stems from two limitations: (i) their
pruning strategies inadequately preserve visual semantic
tokens, degrading draft quality and acceptance rates; (ii) even with aggressive
pruning (e.g., 90% visual
tokens removed), the draft model's remaining inference cost limits overall speedup. To address these limitations, we propose HIPPO, a general holistic-aware parallel speculative decoding framework. Specifically, HIPPO proposes (i) a semantic-aware
token preservation method, which fuses global
attention scores with local visual semantics to retain semantic information at high
pruning ratios; (ii) a
video parallel SD
algorithm that decouples and overlaps draft generation and target verification phases. Experiments on four
video-LLMs across six
benchmarks demonstrate HIPPO's effectiveness, yielding up to 3.51x speedup compared to vanilla auto-regressive decoding.
Read the full abstract ▾
All authors Qitan Lv, Tianyu Liu, Wen Wu, Xuenan Xu, Bowen Zhou, Feng Wu, Chao Zhang
Computer Vision
AIMC-Spec: A Benchmark Dataset for Automatic Intrapulse Modulation Classification under Variable Noise Conditions
Sebastian L. Cocks, Salvador Dreo, Brian Ng et al. (+1 more)
A lack of standardized
datasets has long hindered progress in automatic intrapulse modulation classification (AIMC), a critical task in radar signal analysis for electronic support systems, particularly under noisy or degraded conditions. AIMC seeks to identify the modulation type embedded within a single radar pulse from its complex in-phase and quadrature (I/Q) representation, enabling automated interpretation of intrapulse structure. This paper introduces AIMC-Spec, a comprehensive synthetic
dataset for spectrogram-based image classification, encompassing 30 modulation types across 5 signal-to-noise ratio (SNR) levels. To
benchmark AIMC-Spec, five representative deep learning
algorithms ranging from lightweight CNNs and denoising architectures to
transformer-based networks were re-implemented and evaluated under a unified input format. The results reveal significant performance variation, with frequency-modulated (FM) signals classified more reliably than phase-modulated (PM) types, particularly at low SNRs. A focused FM-only test further highlights how modulation type and network architecture influence classifier robustness. AIMC-Spec establishes a reproducible baseline and provides a foundation for future research and standardization in the AIMC domain.
Read the full abstract ▾
All authors Sebastian L. Cocks, Salvador Dreo, Brian Ng, Feras Dayoub
Computer Vision
Improving Zero-shot ADL Recognition with Large Language Models through Event-based Context and Confidence
Michele Fiori, Gabriele Civitarese, Marco Colussi et al. (+1 more)
Unobtrusive sensor-based recognition of Activities of Daily Living (ADLs) in smart homes by processing data collected from IoT sensing devices supports applications such as healthcare, safety, and energy management. Recent zero-shot methods based on Large
Language Models (LLMs) have the advantage of removing the reliance on labeled ADL sensor data. However, existing approaches rely on time-based
segmentation, which is poorly aligned with the contextual
reasoning capabilities of LLMs. Moreover, existing approaches lack methods for estimating prediction confidence. This paper proposes to improve zero-shot ADL recognition with event-based
segmentation and a novel method for estimating prediction confidence. Our experimental evaluation shows that event-based
segmentation consistently outperforms time-based LLM approaches on complex, realistic
datasets and surpasses supervised data-driven methods, even with relatively small LLMs (e.g., Gemma 3 27B). The proposed confidence measure effectively distinguishes correct from incorrect predictions.
Read the full abstract ▾
All authors Michele Fiori, Gabriele Civitarese, Marco Colussi, Claudio Bettini
Computer Vision · Cross-listed from eess.IV
Temporal-Enhanced Interpretable Multi-Modal Prognosis and Risk Stratification Framework for Diabetic Retinopathy (TIMM-ProRS)
Susmita Kar, A S M Ahsanul Sarkar Akib, Abdul Hasib et al. (+2 more)
Diabetic retinopathy (DR), affecting millions globally with projections indicating a significant rise, poses a severe blindness risk and strains healthcare systems. Diagnostic complexity arises from visual symptom overlap with conditions like age-related macular degeneration and hypertensive retinopathy, exacerbated by high misdiagnosis rates in underserved regions. This study introduces TIMM-ProRS, a novel deep learning framework integrating
Vision Transformer (ViT), Convolutional Neural Network (CNN), and Graph Neural Network (GNN) with multi-modal fusion. TIMM-ProRS uniquely leverages both retinal images and temporal biomarkers (HbA1c, retinal thickness) to capture multi-modal and temporal dynamics. Evaluated comprehensively across diverse
datasets including APTOS 2019 (trained), Messidor-2, RFMiD, EyePACS, and Messidor-1 (validated), the model achieves 97.8% accuracy and an F1-score of 0.96, demonstrating state-of-the-art performance and outperforming existing methods like RSG-Net and DeepDR. This approach enables early, precise, and interpretable diagnosis, supporting scalable telemedical management and enhancing global eye health sustainability.
Read the full abstract ▾
All authors Susmita Kar, A S M Ahsanul Sarkar Akib, Abdul Hasib, Samin Yaser, Anas Bin Azim
Computer Vision
Knowledge-based learning in Text-RAG and Image-RAG
Alexander Shim, Khalil Saieh and Samuel Clarke
This research analyzed and compared the multi-modal approach in the
Vision Transformer(EVA-ViT) based image encoder with the LlaMA or ChatGPT LLM to reduce the
hallucination problem and detect diseases in chest x-ray images. In this research, we utilized the NIH Chest X-ray image to train the model and compared it in image-based RAG, text-based RAG, and baseline. [3] [5] In a result, the text-based RAG[2] e!ectively reduces the
hallucination problem by using external knowledge information, and the image-based RAG improved the prediction con"dence and calibration by using the KNN methods. [4] Moreover, the GPT LLM showed better performance, a low
hallucination rate, and better Expected Calibration Error(ECE) than Llama Llama-based model. This research shows the challenge of data imbalance, a complex multi-stage structure, but suggests a large experience environment and a balanced example of use.
Read the full abstract ▾
Computer Vision
FUME: Fused Unified Multi-Gas Emission Network for Livestock Rumen Acidosis Detection
Taminul Islam, Toqi Tahamid Sarker, Mohamed Embaby et al. (+2 more)
Ruminal acidosis is a prevalent metabolic disorder in dairy cattle causing significant economic losses and animal welfare concerns. Current diagnostic methods rely on invasive pH measurement, limiting scalability for continuous monitoring. We present FUME (Fused Unified Multi-gas Emission Network), the first deep learning approach for rumen acidosis detection from dual-gas optical imaging under in vitro conditions. Our method leverages complementary carbon dioxide (CO2) and methane (CH4) emission patterns captured by infrared cameras to classify rumen health into Healthy, Transitional, and Acidotic states. FUME employs a lightweight dual-stream architecture with weight-shared encoders, modality-specific self-
attention, and channel
attention fusion, jointly optimizing gas plume
segmentation and classification of dairy cattle health. We introduce the first dual-gas OGI
dataset comprising 8,967 annotated frames across six pH levels with pixel-level
segmentation masks. Experiments demonstrate that FUME achieves 80.99% mIoU and 98.82% classification accuracy while using only 1.28M parameters and 1.97G MACs–outperforming state-of-the-art methods in
segmentation quality with 10x lower computational cost. Ablation studies reveal that CO2 provides the primary discriminative signal and dual-task learning is essential for optimal performance. Our work establishes the feasibility of gas emission-based livestock health monitoring, paving the way for practical, in vitro acidosis detection systems. Codes are available at https://github.com/taminulislam/fume.
Read the full abstract ▾
All authors Taminul Islam, Toqi Tahamid Sarker, Mohamed Embaby, Khaled R Ahmed, Amer AbuGhazaleh
Computer Vision
MobiDiary: Autoregressive Action Captioning with Wearable Devices and Wireless Signals
Fei Deng, Yinghui He, Chuntong Chu et al. (+4 more)
Human Activity Recognition (HAR) in smart homes is critical for health monitoring and assistive living. While
vision-based systems are common, they face
privacy concerns and environmental limitations (e.g., occlusion). In this work, we present MobiDiary, a framework that generates natural
language descriptions of daily activities directly from heterogeneous physical signals (specifically IMU and Wi-Fi). Unlike conventional approaches that restrict outputs to pre-defined labels, MobiDiary produces expressive, human-readable summaries. To bridge the semantic gap between continuous, noisy physical signals and discrete linguistic descriptions, we propose a unified sensor encoder. Instead of relying on modality-specific engineering, we exploit the shared inductive biases of motion-induced signals–where both inertial and wireless data reflect underlying kinematic dynamics. Specifically, our encoder utilizes a patch-based mechanism to capture local temporal correlations and integrates heterogeneous placement
embedding to unify spatial contexts across different sensors. These unified signal
tokens are then fed into a
Transformer-based decoder, which employs an autoregressive mechanism to generate coherent action descriptions word-by-word. We comprehensively evaluate our approach on multiple public
benchmarks (XRF V2, UWash, and WiFiTAD). Experimental results demonstrate that MobiDiary effectively generalizes across modalities, achieving state-of-the-art performance on captioning metrics (e.g., BLEU@4, CIDEr, RMC) and outperforming specialized baselines in continuous action understanding.
Read the full abstract ▾
All authors Fei Deng, Yinghui He, Chuntong Chu, Ge Wang, Han Ding, Jinsong Han, Fei Wang
Computer Vision
Unified Multi-Site Multi-Sequence Brain MRI Harmonization Enriched by Biomedical Semantic Style
Mengqi Wu, Yongheng Sun, Qianqian Wang et al. (+2 more)
Aggregating multi-site
brain MRI data can enhance deep learning model training, but also introduces non-biological heterogeneity caused by site-specific variations (e.g., differences in scanner vendors, acquisition parameters, and imaging protocols) that can undermine generalizability. Recent retrospective MRI harmonization seeks to reduce such site effects by standardizing image style (e.g., intensity, contrast, noise patterns) while preserving anatomical content. However, existing methods often rely on limited paired traveling-subject data or fail to effectively disentangle style from anatomy. Furthermore, most current approaches address only single-sequence harmonization, restricting their use in real-world settings where multi-sequence MRI is routinely acquired. To this end, we introduce MMH, a unified framework for multi-site multi-sequence
brain MRI harmonization that leverages biomedical semantic priors for sequence-aware style
alignment. MMH operates in two stages: (1) a
diffusion-based global harmonizer that maps MR images to a sequence-specific unified domain using style-agnostic gradient conditioning, and (2) a target-specific fine-tuner that adapts globally aligned images to desired target domains. A tri-planar
attention BiomedCLIP encoder aggregates multi-view
embeddings to characterize volumetric style information, allowing explicit disentanglement of image styles from anatomy without requiring paired data. Evaluations on 4,163 T1- and T2-weighted MRIs demonstrate MMH's superior harmonization over state-of-the-art methods in image feature clustering, voxel-level comparison, tissue
segmentation, and downstream age and site classification.
Read the full abstract ▾
All authors Mengqi Wu, Yongheng Sun, Qianqian Wang, Pew-Thian Yap, Mingxia Liu
Computer Vision
Route, Retrieve, Reflect, Repair: Self-Improving Agentic Framework for Visual Detection and Linguistic Reasoning in Medical Imaging
Md. Faiyaz Abdullah Sayeedi, Rashedur Rahman, Siam Tahsin Bhuiyan et al. (+4 more)
Medical image analysis increasingly relies on large
vision-
language models (VLMs), yet most systems remain single-pass black boxes that offer limited control over
reasoning, safety, and spatial grounding. We propose R^4, an agentic framework that decomposes medical imaging workflows into four coordinated
agents: a Router that configures task- and specialization-aware
prompts from the image, patient history, and metadata; a Retriever that uses exemplar
memory and pass@k sampling to jointly generate free-text reports and bounding boxes; a Reflector that critiques each draft-box pair for key clinical error modes (negation, laterality, unsupported claims, contradictions, missing findings, and localization errors); and a Repairer that iteratively revises both narrative and spatial outputs under targeted constraints while curating high-quality exemplars for future cases. Instantiated on chest X-ray analysis with multiple modern VLM backbones and evaluated on report generation and weakly supervised detection, R^4 consistently boosts LLM-as-a-Judge scores by roughly +1.7-+2.5 points and mAP50 by +2.5-+3.5 absolute points over strong single-VLM baselines, without any gradient-based fine-tuning. These results show that agentic routing, reflection, and repair can turn strong but brittle VLMs into more reliable and better grounded tools for clinical image interpretation. Our code can be found at: https://github.com/faiyazabdullah/MultimodalMedAgent
Read the full abstract ▾
All authors Md. Faiyaz Abdullah Sayeedi, Rashedur Rahman, Siam Tahsin Bhuiyan, Sefatul Wasi, Ashraful Islam, Saadia Binte Alam, AKM Mahbubur Rahman
Computer Vision
Human-inspired Global-to-Parallel Multi-scale Encoding for Lightweight Vision Models
Wei Xu
Lightweight
vision networks have witnessed remarkable progress in recent years, yet achieving a satisfactory balance among parameter scale, computational overhead, and task performance remains difficult. Although many existing lightweight models manage to reduce computation considerably, they often do so at the expense of a substantial increase in parameter count (e.g., LSNet, MobileMamba), which still poses obstacles for deployment on resource-limited devices. In parallel, some studies attempt to draw inspiration from human visual perception, but their modeling tends to oversimplify the visual process, making it hard to reflect how perception truly operates. Revisiting the cooperative mechanism of the human visual system, we propose GPM (Global-to-Parallel Multi-scale Encoding). GPM first employs a Global Insight Generator (GIG) to extract holistic cues, and subsequently processes features of different scales through parallel branches: LSAE emphasizes mid-/large-scale semantic relations, while IRB (Inverted Residual Block) preserves fine-grained texture information, jointly enabling coherent representation of global and local features. As such, GPM conforms to two characteristic behaviors of human
vision perceiving the whole before focusing on details, and maintaining broad contextual awareness even during local
attention. Built upon GPM, we further develop the lightweight H-GPE network. Experiments on image classification, object detection, and semantic
segmentation show that H-GPE achieves strong performance while maintaining a balanced footprint in both FLOPs and parameters, delivering a more favorable accuracy-efficiency trade-off compared with recent state-of-the-art lightweight models.
Read the full abstract ▾
Computer Vision
GI-Bench: A Panoramic Benchmark Revealing the Knowledge-Experience Dissociation of Multimodal Large Language Models in Gastrointestinal Endoscopy Against Clinical Standards
Yan Zhu, Te Luo, Pei-Yao Fu et al. (+12 more)
Multimodal Large
Language Models (MLLMs) show promise in gastroenterology, yet their performance against comprehensive clinical workflows and human
benchmarks remains unverified. To systematically evaluate state-of-the-art MLLMs across a panoramic gastrointestinal endoscopy workflow and determine their clinical utility compared with human endoscopists. We constructed GI-Bench, a
benchmark encompassing 20 fine-grained lesion categories. Twelve MLLMs were evaluated across a five-stage clinical workflow: anatomical localization, lesion identification, diagnosis, findings description, and management. Model performance was benchmarked against three junior endoscopists and three residency trainees using Macro-F1, mean Intersection-over-Union (mIoU), and multi-dimensional Likert scale. Gemini-3-Pro achieved state-of-the-art performance. In diagnostic
reasoning, top-tier models (Macro-F1 0.641) outperformed trainees (0.492) and rivaled junior endoscopists (0.727; p>0.05). However, a critical "spatial grounding bottleneck" persisted; human lesion localization (mIoU >0.506) significantly outperformed the best model (0.345; p<0.05). Furthermore, qualitative analysis revealed a "fluency-accuracy paradox": models generated reports with superior linguistic readability compared with humans (p<0.05) but exhibited significantly lower factual correctness (p<0.05) due to "over-interpretation" and
hallucination of visual features. GI-Bench maintains a dynamic leaderboard that tracks the evolving performance of MLLMs in clinical endoscopy. The current rankings and
benchmark results are available at https://roterdl.github.io/GIBench/.
Read the full abstract ▾
All authors Yan Zhu, Te Luo, Pei-Yao Fu, Zhen Zhang, Zi-Long Wang, Yi-Fan Qu, Zi-Han Geng, Jia-Qi Xu, Lu Yao, Li-Yun Ma, Wei Su, Wei-Feng Chen, Quan-Lin Li, Shuo Wang, Ping-Hong Zhou
Computer Vision
Second-order Gaussian directional derivative representations for image high-resolution corner detection
Jiamiao Lu, Dongbo Xie, Junjie Qiu et al. (+3 more)
Corner detection is widely used in various computer
vision tasks, such as image matching and 3D reconstruction. Our research indicates that there are theoretical flaws in Zhang et al.'s use of a simple corner model to obtain a series of corner characteristics, as the grayscale information of two adjacent corners can affect each other. In order to address the above issues, a second-order Gaussian directional derivative (SOGDD) filter is used in this work to smooth two typical high-resolution angle models (i.e. END-type and L-type models). Then, the SOGDD representations of these two corner models were derived separately, and many characteristics of high-resolution corners were discovered, which enabled us to demonstrate how to select Gaussian filtering scales to obtain intensity variation information from images, accurately depicting adjacent corners. In addition, a new high-resolution corner detection method for images has been proposed for the first time, which can accurately detect adjacent corner points. The experimental results have verified that the proposed method outperforms state-of-the-art methods in terms of localization error, robustness to image blur transformation, image matching, and 3D reconstruction.
Read the full abstract ▾
All authors Jiamiao Lu, Dongbo Xie, Junjie Qiu, Lingkun Ma, Changming Sun, Weichuan Zhang
Computer Vision
Instruction-Driven 3D Facial Expression Generation and Transition
Anh H. Vo, Tae-Seok Kim, Hulin Jin et al. (+2 more)
A 3D avatar typically has one of six cardinal facial expressions. To simulate realistic emotional variation, we should be able to render a facial transition between two arbitrary expressions. This study presents a new framework for instruction-driven facial expression generation that produces a 3D face and, starting from an image of the face, transforms the facial expression from one designated facial expression to another. The Instruction-driven Facial Expression Decomposer (IFED) module is introduced to facilitate
multimodal data learning and capture the correlation between textual descriptions and facial expression features. Subsequently, we propose the Instruction to Facial Expression Transition (I2FET) method, which leverages IFED and a vertex reconstruction loss function to refine the semantic comprehension of latent vectors, thus generating a facial expression sequence according to the given instruction. Lastly, we present the Facial Expression Transition model to generate smooth transitions between facial expressions. Extensive evaluation suggests that the proposed model outperforms state-of-the-art methods on the CK+ and CelebV-HQ
datasets. The results show that our framework can generate facial expression trajectories according to text instruction. Considering that text
prompts allow us to make diverse descriptions of human emotional states, the repertoire of facial expressions and the transitions between them can be expanded greatly. We expect our framework to find various practical applications More information about our project can be found at https://vohoanganh.github.io/tg3dfet/
Read the full abstract ▾
All authors Anh H. Vo, Tae-Seok Kim, Hulin Jin, Soo-Mi Choi, Yong-Guk Kim
Computer Vision
CogniMap3D: Cognitive 3D Mapping and Rapid Retrieval
Feiran Wang, Junyi Wu, Dawen Cai et al. (+2 more)
We present CogniMap3D, a bioinspired framework for dynamic 3D scene understanding and reconstruction that emulates human cognitive processes. Our approach maintains a persistent
memory bank of static scenes, enabling efficient spatial knowledge storage and rapid
retrieval. CogniMap3D integrates three core capabilities: a multi-stage motion cue framework for identifying dynamic objects, a cognitive mapping system for storing, recalling, and updating static scenes across multiple visits, and a factor graph optimization strategy for refining camera poses. Given an image stream, our model identifies dynamic regions through motion cues with depth and camera pose priors, then matches static elements against its
memory bank. When revisiting familiar locations, CogniMap3D retrieves stored scenes, relocates cameras, and updates
memory with new observations. Evaluations on
video depth estimation, camera pose reconstruction, and 3D mapping tasks demonstrate its state-of-the-art performance, while effectively supporting continuous scene understanding across extended sequences and multiple visits.
Read the full abstract ▾
All authors Feiran Wang, Junyi Wu, Dawen Cai, Yuan Hong, Yan Yan
Computer Vision
Towards Cross-Platform Generalization: Domain Adaptive 3D Detection with Augmentation and Pseudo-Labeling
Xiyan Feng, Wenbo Zhang, Lu Zhang et al. (+3 more)
This technical report represents the award-winning solution to the Cross-platform 3D Object Detection task in the RoboSense2025 Challenge. Our approach is built upon PVRCNN++, an efficient 3D object detection framework that effectively integrates point-based and voxel-based features. On top of this foundation, we improve cross-platform generalization by narrowing domain gaps through tailored data augmentation and a self-training strategy with pseudo-labels. These enhancements enabled our approach to secure the 3rd place in the challenge, achieving a 3D AP of 62.67% for the Car category on the phase-1 target domain, and 58.76% and 49.81% for Car and Pedestrian categories respectively on the phase-2 target domain.
Read the full abstract ▾
All authors Xiyan Feng, Wenbo Zhang, Lu Zhang, Yunzhi Zhuge, Huchuan Lu, You He
Computer Vision
Representation Learning with Semantic-aware Instance and Sparse Token Alignments
Phuoc-Nguyen Bui, Toan Duc Nguyen, Junghyun Bum et al. (+2 more)
Medical contrastive
vision-
language pre-training (VLP) has demonstrated significant potential in improving performance on downstream tasks. Traditional approaches typically employ contrastive learning, treating paired image-report samples as positives and unpaired ones as negatives. However, in medical
datasets, there can be substantial similarities between images or reports from different patients. Rigidly treating all unpaired samples as negatives, can disrupt the underlying semantic structure and negatively impact the quality of the learned representations. In this paper, we propose a multi-level
alignment framework, Representation Learning with Semantic-aware Instance and Sparse
Token Alignments (SISTA) by exploiting the semantic correspondence between medical image and radiology reports at two levels, i.e., image-report and patch-word levels. Specifically, we improve the conventional contrastive learning by incorporating inter-report similarity to eliminate the false negatives and introduce a method to effectively align image patches with relevant word
tokens. Experimental results demonstrate the effectiveness of the proposed framework in improving transfer performance across different
datasets on three downstream tasks: image classification, image
segmentation, and object detection. Notably, our framework achieves significant improvements in fine-grained tasks even with limited labeled data. Codes and pre-trained models will be made available.
Read the full abstract ▾
All authors Phuoc-Nguyen Bui, Toan Duc Nguyen, Junghyun Bum, Duc-Tai Le, Hyunseung Choo
Computer Vision
A Hardware-Algorithm Co-Designed Framework for HDR Imaging and Dehazing in Extreme Rocket Launch Environments
Jing Tao, Banglei Guan, Pengju Sun et al. (+3 more)
Quantitative optical measurement of critical mechanical parameters – such as plume flow fields, shock wave structures, and nozzle oscillations – during rocket launch faces severe challenges due to extreme imaging conditions. Intense combustion creates dense particulate haze and luminance variations exceeding 120 dB, degrading image data and undermining subsequent photogrammetric and velocimetric analyses. To address these issues, we propose a hardware-
algorithm co-design framework that combines a custom Spatially Varying Exposure (SVE) sensor with a physics-aware dehazing
algorithm. The SVE sensor acquires multi-exposure data in a single shot, enabling robust haze assessment without relying on idealized atmospheric models. Our approach dynamically estimates haze density, performs region-adaptive illumination optimization, and applies multi-scale
entropy-constrained fusion to effectively separate haze from scene radiance. Validated on real launch imagery and controlled experiments, the framework demonstrates superior performance in recovering physically accurate visual information of the plume and engine region. This offers a reliable image basis for extracting key mechanical parameters, including particle velocity, flow instability frequency, and structural vibration, thereby supporting precise quantitative analysis in extreme aerospace environments.
Read the full abstract ▾
All authors Jing Tao, Banglei Guan, Pengju Sun, Taihang Lei, Yang Shang, Qifeng Yu
Computer Vision · Cross-listed from cs.RO
Robust Subpixel Localization of Diagonal Markers in Large-Scale Navigation via Multi-Layer Screening and Adaptive Matching
Jing Tao, Banglei Guan, Yang Shang et al. (+2 more)
This paper proposes a robust, high-precision positioning methodology to address localization failures arising from complex background interference in large-scale flight
navigation and the computational inefficiency inherent in conventional sliding window matching techniques. The proposed methodology employs a three-tiered framework incorporating multi-layer corner screening and adaptive template matching. Firstly, dimensionality is reduced through illumination equalization and structural information extraction. A coarse-to-fine candidate selection strategy minimizes sliding window computational costs, enabling rapid estimation of the marker's position. Finally, adaptive templates are generated for candidate points, achieving subpixel precision through improved template matching with correlation coefficient extremum fitting. Experimental results demonstrate the method's effectiveness in extracting and localizing diagonal markers in complex, large-scale environments, making it ideal for field-of-view measurement in
navigation tasks.
Read the full abstract ▾
All authors Jing Tao, Banglei Guan, Yang Shang, Shunkun Liang, Qifeng Yu
Computer Vision
Instance-Aligned Captions for Explainable Video Anomaly Detection
Inpyo Song, Minjun Joo, Joonhyung Kwon et al. (+2 more)
Explainable
video anomaly detection (VAD) is crucial for safety-critical applications, yet even with recent progress, much of the research still lacks spatial grounding, making the explanations unverifiable. This limitation is especially pronounced in multi-entity interactions, where existing explainable VAD methods often produce incomplete or visually misaligned descriptions, reducing their trustworthiness. To address these challenges, we introduce instance-aligned captions that link each textual claim to specific object instances with appearance and motion attributes. Our framework captures who caused the anomaly, what each entity was doing, whom it affected, and where the explanationis grounded, enabling verifiable and actionable
reasoning. We annotate eight widely used VAD
benchmarks and extend the 360-degree
egocentric dataset, VIEW360, with 868 additional
videos, eight locations, and four new anomaly types, creating VIEW360+, a comprehensive testbed for explainable VAD. Experiments show that our instance-level spatially grounded captions reveal significant limitations in current LLM- and VLM-based methods while providing a robust
benchmark for future research in trustworthy and interpretable anomaly detection.
Read the full abstract ▾
All authors Inpyo Song, Minjun Joo, Joonhyung Kwon, Eunji Jeon, Jangwon Lee
Computer Vision
Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention
Shezheng Song, Shasha Li, Shan Zhao et al. (+6 more)
Multimodal Large
Language Models (MLLMs) have achieved remarkable progress in
vision-
language understanding, yet how they internally integrate visual and textual information remains poorly understood. To bridge this gap, we perform a systematic layer-wise masking analysis across multiple architectures, revealing how visual-text fusion evolves within MLLMs. The results show that fusion emerges at several specific layers rather than being uniformly distributed across the network, and certain models exhibit a late-stage "review" phenomenon where visual signals are reactivated before output generation. Besides, we further analyze layer-wise
attention evolution and observe persistent high-
attention noise on irrelevant regions, along with gradually increasing
attention on text-aligned areas. Guided by these insights, we introduce a training-free contrastive
attention framework that models the transformation between early fusion and final layers to highlight meaningful
attention shifts. Extensive experiments across various MLLMs and
benchmarks validate our analysis and demonstrate that the proposed approach improves
multimodal reasoning performance.
Read the full abstract ▾
All authors Shezheng Song, Shasha Li, Shan Zhao, Xiaopeng Li, Qian Wan, Chengyu Wang, Tianwei Yan, Jun Ma, Jie Yu
Computer Vision
Zhichen Zeng, Wenxuan Bao, Xiao Lin et al. (+8 more)
Vision-
language models (VLMs), despite their extraordinary zero-shot capabilities, are vulnerable to distribution shifts. Test-time adaptation (TTA) emerges as a predominant strategy to adapt VLMs to unlabeled test data on the fly. However, existing TTA methods heavily rely on zero-shot predictions as pseudo-labels for self-training, which can be unreliable under distribution shifts and misguide adaptation due to two fundamental limitations.First (Modality Gap), distribution shifts induce gaps between visual and textual modalities, making cross-modal relations inaccurate. Second (Visual Nuisance), visual
embeddings encode rich but task-irrelevant noise that often overwhelms task-specific semantics under distribution shifts. To address these limitations, we propose SubTTA, which aligns the semantic subspaces of both modalities to enhance zero-shot predictions to better guide the TTA process. To bridge the modality gap, SubTTA extracts the principal subspaces of both modalities and aligns the visual
manifold to the textual semantic anchor by minimizing their chordal distance. To eliminate visual nuisance, SubTTA projects the aligned visual features onto the task-specific textual subspace, which filters out task-irrelevant noise by constraining visual
embeddings within the valid semantic span, and standard TTA is further performed on the purified space to refine the decision boundaries. Extensive experiments on various
benchmarks and VLM architectures demonstrate the effectiveness of SubTTA, yielding an average improvement of 2.24% over state-of-the-art TTA methods. Our code is available at https://github.com/zhichenz98/SubTTA_EMNLP26.
Read the full abstract ▾
All authors Zhichen Zeng, Wenxuan Bao, Xiao Lin, Ruizhong Qiu, Tianxin Wei, Xuying Ning, Yuchen Yan, Chen Luo, Monica Xiao Cheng, Jingrui He, Hanghang Tong
Computer Vision
How Do Optical Flow and Textual Prompts Collaborate to Assist in Audio-Visual Semantic Segmentation?
Yujian Lee, Peng Gao, Yongqi Xu et al. (+1 more)
Audio-visual semantic
segmentation (AVSS) represents an extension of the audio-visual
segmentation (AVS) task, necessitating a semantic understanding of audio-visual scenes beyond merely identifying sound-emitting objects at the visual pixel level. Contrary to a previous methodology, by decomposing the AVSS task into two discrete subtasks by initially providing a prompted
segmentation mask to facilitate subsequent semantic analysis, our approach innovates on this foundational strategy. We introduce a novel collaborative framework, Stepping Stone Plus (SSP), which integrates optical flow and textual
prompts to assist the
segmentation process. In scenarios where sound sources frequently coexist with moving objects, our pre-mask technique leverages optical flow to capture motion dynamics, providing essential temporal context for precise
segmentation. To address the challenge posed by stationary sound-emitting objects, such as alarm clocks, SSP incorporates two specific textual
prompts: one identifies the category of the sound-emitting object, and the other provides a broader description of the scene. Additionally, we implement a visual-textual
alignment module (VTA) to facilitate cross-modal integration, delivering more coherent and contextually relevant semantic interpretations. Our training regimen involves a post-mask technique aimed at compelling the model to learn the diagram of the optical flow. Experimental results demonstrate that SSP outperforms existing AVS methods, delivering efficient and precise
segmentation results.
Read the full abstract ▾
All authors Yujian Lee, Peng Gao, Yongqi Xu, Wentao Fan
Computer Vision
Controllable Diffusion-Based Lesion Inpainting for Scalable Histopathology Data Augmentation
Mohamad Koohi-Moghadam, Mohammad-Ali Nikouei Mahani, Rex K.H. Au-Yeung et al. (+6 more)
Expert-annotated training data remains the critical bottleneck for AI in histopathology, particularly for rare pathologies where even dozens of cases may be unavailable. While data augmentation offers a solution, existing methods fail to generate sufficiently realistic lesion morphologies that preserve tissue-specific architectures. Here we present PathoGen, a
diffusion-based generative model enabling controllable, high-fidelity lesion inpainting into benign histopathology images. We validate PathoGen across four
datasets representing kidney, skin, breast, and prostate pathology. Quantitative assessment confirms PathoGen outperforms state-of-the-art baselines in image fidelity and distributional similarity. Evaluation by six expert pathologists revealed that synthetic images by PathoGen were only marginally distinguished from real tissue image slightly above chance (57.75% accuracy), demonstrating strong perceptual realism of PathoGen-generated lesions. PathoGen achieved the highest win rate (35.4%) when pathologists ranked generation quality against all baselines. Crucially, augmenting training sets with PathoGen-synthesized lesions improves
segmentation Dice scores by up to 0.18 compared to traditional augmentations, with maximum benefit in data-scarce regimes. By simultaneously generating realistic morphology and pixel-level annotations, PathoGen effectively addresses both data scarcity and annotation cost, two critical bottlenecks in computational pathology development.
Read the full abstract ▾
All authors Mohamad Koohi-Moghadam, Mohammad-Ali Nikouei Mahani, Rex K.H. Au-Yeung, Raymond Yu O, Monalyn Marabi, Piyapharom Intarawichian, Fabian Z. X. Lean, Andrew Ferguson, Kyongtae Tyler Bae
Computer Vision
From Prompts to Deployment: Auto-Curated Domain-Specific Dataset Generation via Diffusion Models
Dongsik Yoon and Jongeun Kim
In this paper, we present an automated pipeline for generating domain-specific synthetic
datasets with
diffusion models, addressing the distribution shift between pre-trained models and real-world deployment environments. Our three-stage framework first synthesizes target objects within domain-specific backgrounds through controlled inpainting. The generated outputs are then validated via a multi-modal assessment that integrates object detection, aesthetic scoring, and
vision-
language alignment. Finally, a user-preference classifier is employed to capture subjective selection criteria. This pipeline enables the efficient construction of high-quality, deployable
datasets while reducing reliance on extensive real-world data collection.
Read the full abstract ▾