Computer Vision · Cross-listed from cs.AI
Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Exploration
Sen Wang, Bangwei Liu, Zhenkun Gao et al. (+4 more)
An ideal
embodied agent should possess lifelong learning capabilities to handle long-horizon and complex tasks, enabling continuous operation in general environments. This not only requires the
agent to accurately accomplish given tasks but also to leverage long-term episodic
memory to optimize decision-making. However, existing mainstream one-shot
embodied tasks primarily focus on task completion results, neglecting the crucial process of exploration and
memory utilization. To address this, we propose Long-term
Memory Embodied Exploration (LMEE), which aims to unify the
agent's exploratory cognition and decision-making behaviors to promote lifelong learning. We further construct a corresponding
dataset and
benchmark, LMEE-Bench, incorporating multi-goal
navigation and
memory-based question answering to comprehensively evaluate both the process and outcome of
embodied exploration. To enhance the
agent's
memory recall and proactive exploration capabilities, we propose MemoryExplorer, a novel method that fine-tunes a
multimodal large
language model through reinforcement learning to encourage active
memory querying. By incorporating a multi-task reward function that includes action prediction, frontier selection, and question answering, our model achieves proactive exploration. Extensive experiments against state-of-the-art
embodied exploration models demonstrate that our approach achieves significant advantages in long-horizon
embodied tasks. Our
dataset and code will be released at https://wangsen99.github.io/papers/lmee/
Read the full abstract ▾
All authors Sen Wang, Bangwei Liu, Zhenkun Gao, Lizhuang Ma, Xuhong Wang, Yuan Xie, Xin Tan
Computer Vision
Residual Cross-Modal Fusion Networks for Audio-Visual Navigation
Yi Wang, Yinfeng Yu and Bin Ren
Audio-visual
embodied navigation aims to enable an
agent to autonomously localize and reach a sound source in unseen 3D environments by leveraging auditory cues. The key challenge of this task lies in effectively modeling the interaction between heterogeneous features during
multimodal fusion, so as to avoid single-modality dominance or information degradation, particularly in cross-domain scenarios. To address this, we propose a Cross-Modal Residual Fusion Network, which introduces bidirectional residual interactions between audio and visual streams to achieve complementary modeling and fine-grained
alignment, while maintaining the independence of their representations. Unlike conventional methods that rely on simple concatenation or
attention gating, CRFN explicitly models cross-modal interactions via residual connections and incorporates stabilization techniques to improve convergence and robustness. Experiments on the Replica and Matterport3D
datasets demonstrate that CRFN significantly outperforms state-of-the-art fusion baselines and achieves stronger cross-domain generalization. Notably, our experiments also reveal that
agents exhibit differentiated modality dependence across different
datasets. The discovery of this phenomenon provides a new perspective for understanding the cross-modal collaboration mechanism of
embodied agents.
Read the full abstract ▾
Computer Vision
R$^2$BD: A Reconstruction-Based Method for Generalizable and Efficient Detection of Fake Images
Qingyu Liu, Zhongjie Ba, Jianmin Guo et al. (+4 more)
Recently, reconstruction-based methods have gained
attention for AIGC image detection. These methods leverage pre-trained
diffusion models to reconstruct inputs and measure residuals for distinguishing real from fake images. Their key advantage lies in reducing reliance on
dataset-specific artifacts and improving generalization under distribution shifts. However, they are limited by significant inefficiency due to multi-step inversion and reconstruction, and their reliance on
diffusion backbones further limits generalization to other generative paradigms such as GANs. In this paper, we propose a novel fake image detection framework, called R$^2$BD, built upon two key designs: (1) G-LDM, a unified reconstruction model that simulates the generation behaviors of VAEs, GANs, and
diffusion models, thereby broadening the detection scope beyond prior
diffusion-only approaches; and (2) a residual bias calculation module that distinguishes real and fake images in a single inference step, which is a significant efficiency improvement over existing methods that typically require 20$+$ steps. Extensive experiments on the
benchmark from 10 public
datasets demonstrate that R$^2$BD is over 22$\times$ faster than existing reconstruction-based methods while achieving superior detection accuracy. In cross-
dataset evaluations, it outperforms state-of-the-art methods by an average of 13.87%, showing strong efficiency and generalization across diverse generative methods. The code and
dataset used for evaluation are available at https://github.com/QingyuLiu/RRBD.
Read the full abstract ▾
All authors Qingyu Liu, Zhongjie Ba, Jianmin Guo, Qiu Wang, Zhibo Wang, Jie Shi, Kui Ren
Computer Vision
3D Wavelet-Based Structural Priors for Controlled Diffusion in Whole-Body Low-Dose PET Denoising
Peiyuan Jing, Yue Yang, Chun-Wun Cheng et al. (+8 more)
Low-dose Positron Emission
Tomography (PET) imaging reduces patient radiation exposure but suffers from increased noise that degrades image quality and diagnostic reliability. Although
diffusion models have demonstrated strong denoising capability, their
stochastic nature makes it challenging to enforce anatomically consistent structures, particularly in low signal-to-noise regimes and volumetric whole-body imaging. We propose Wavelet-Conditioned ControlNet (WCC-Net), a fully 3D
diffusion-based framework that introduces explicit frequency-domain structural priors via wavelet representations to guide volumetric PET denoising. By injecting wavelet-based structural guidance into a frozen pretrained
diffusion backbone through a lightweight control branch, WCC-Net decouples anatomical structure from noise while preserving generative expressiveness and 3D structural continuity. Extensive experiments demonstrate that WCC-Net consistently outperforms CNN-, GAN-, and
diffusion-based baselines. On the internal 1/20-dose test set, WCC-Net improves PSNR by +1.21 dB and SSIM by +0.008 over a strong
diffusion baseline, while reducing structural distortion (GMSD) and intensity error (NMAE). Moreover, WCC-Net generalizes robustly to unseen dose levels (1/50 and 1/4), achieving superior quantitative performance and improved volumetric anatomical consistency.
Read the full abstract ▾
All authors Peiyuan Jing, Yue Yang, Chun-Wun Cheng, Zhenxuan Zhang, Liutao Yang, Thiago V. Lima, Klaus Strobel, Antoine Leimgruber, Angelica Aviles-Rivero, Guang Yang, Javier A. Montoya-Zegarra
Computer Vision
Efficient Visual Question Answering Pipeline for Autonomous Driving via Scene Region Compression
Yuliang Cai, Dongqiangzi Ye, Zitian Chen et al. (+1 more)
Autonomous driving increasingly relies on Visual Question Answering (VQA) to enable vehicles to understand complex surroundings by analyzing visual inputs and textual queries. Currently, a paramount concern for VQA in this domain is the stringent requirement for fast latency and real-time processing, as delays directly impact real-world safety in this safety-critical application. However, current state-of-the-art VQA models, particularly large
vision-
language models (VLMs), often prioritize performance over computational efficiency. These models typically process dense patch
tokens for every frame, leading to prohibitive computational costs (FLOPs) and significant inference latency, especially with long
video sequences. This focus limits their practical deployment in real-time autonomous driving scenarios. To tackle this issue, we propose an efficient VLM framework for autonomous driving VQA tasks, SRC-Pipeline. It learns to compress early frame
tokens into a small number of high-level
tokens while retaining full patch
tokens for recent frames. Experiments on autonomous driving
video question answering tasks show that our approach achieves 66% FLOPs reduction while maintaining comparable performance, enabling VLMs to operate more effectively in real-time, safety-critical autonomous driving settings.
Read the full abstract ▾
All authors Yuliang Cai, Dongqiangzi Ye, Zitian Chen, Chongruo Wu
Computer Vision
Billboard in Focus: Estimating Driver Gaze Duration from a Single Image
Carlos Pizarroso, Zuzana Berger Haladová, Zuzana Černeková et al. (+1 more)
Roadside billboards represent a central element of outdoor advertising, yet their presence may contribute to driver distraction and accident risk. This study introduces a fully automated pipeline for billboard detection and driver gaze duration estimation, aiming to evaluate billboard relevance without reliance on manual annotations or eye-tracking devices. Our pipeline operates in two stages: (1) a YOLO-based object detection model trained on Mapillary Vistas and fine-tuned on BillboardLamac images achieved 94% mAP@50 in the billboard detection task (2) a classifier based on the detected bounding box positions and DINOv2 features. The proposed pipeline enables estimation of billboard driver gaze duration from individual frames. We show that our method is able to achieve 68.1% accuracy on BillboardLamac when considering individual frames. These results are further validated using images collected from Google Street View.
Read the full abstract ▾
All authors Carlos Pizarroso, Zuzana Berger Haladová, Zuzana Černeková, Viktor Kocur
Computer Vision
Adversarial Attacks on Medical Hyperspectral Imaging Exploiting Spectral-Spatial Dependencies and Multiscale Features
Yunrui Gu, Zhenzhe Gao, Cong Kong et al. (+2 more)
Medical hyperspectral imaging (MHSI) has shown strong potential for disease diagnosis by capturing spectral-spatial information of tissues. While deep learning has substantially improved MHSI classification accuracy, its robustness remains limited due to the well-known trade-off between accuracy and robustness in Deep Neural Networks (DNNs). This issue is particularly critical in MHSI, where reliable prediction depends on local tissue relationships and multiscale spectral-spatial structures. A practical way to improve robustness is to identify the most unstable
adversarial examples and incorporate them into
adversarial training. However, existing attack methods do not sufficiently exploit these MHSI-specific properties, leading to suboptimal attack effectiveness and limited value for robustness enhancement. To address this gap, we propose a structured
adversarial attack framework for MHSI that progressively models its local spectral-spatial dependencies and multiscale hierarchical representations. The proposed method generates anatomically consistent perturbations by modeling neighborhood dependencies and hierarchical spectral-spatial features. Experiments on the
brain and choledoch
datasets show that our method more effectively degrades lesion-related classification performance in critical tumor regions than existing baselines while maintaining low perturbation magnitude. These results reveal a clinically relevant robustness weakness in current MHSI models and provide stronger
adversarial samples for developing targeted defense strategies.
Read the full abstract ▾
All authors Yunrui Gu, Zhenzhe Gao, Cong Kong, Jiawei Du, Zhaoxia Yin
Computer Vision · Cross-listed from cs.LG
Explainable Deep Radiogenomic Molecular Imaging for MGMT Methylation Prediction in Glioblastoma
Hasan M Jamil
Glioblastoma (GBM) is a highly aggressive primary
brain tumor with limited therapeutic options and poor prognosis. The methylation status of the O6-methylguanine-DNA methyltransferase (MGMT) gene promoter is a critical molecular biomarker that influences patient response to temozolomide chemotherapy. Traditional methods for determining MGMT status rely on invasive biopsies and are limited by intratumoral heterogeneity and procedural risks. This study presents a radiogenomic molecular imaging analysis framework for the non-invasive prediction of MGMT promoter methylation using multi-parametric
magnetic resonance imaging (mpMRI). Our approach integrates radiomics, deep learning, and explainable artificial intelligence (XAI) to analyze MRI-derived imaging phenotypes and correlate them with molecular labels. Radiomic features are extracted from FLAIR, T1-weighted, T1-contrast-enhanced, and T2-weighted MRI sequences, while a 3D convolutional neural network learns deep representations from the same modalities. These complementary features are fused using both early fusion and
attention-based strategies and classified to predict MGMT methylation status. To enhance clinical interpretability, we apply XAI methods such as Grad-CAM and SHAP to visualize and explain model decisions. The proposed framework is trained on the RSNA-MICCAI Radiogenomic Classification
dataset and externally validated on the BraTS 2021
dataset. This work advances the field of molecular imaging by demonstrating the potential of AI-driven radiogenomics for precision oncology, supporting non-invasive, accurate, and interpretable prediction of clinically actionable molecular biomarkers in GBM.
Read the full abstract ▾
Computer Vision
Spatial Multi-Task Learning for Breast Cancer Molecular Subtype Prediction from Single-Phase DCE-MRI
Sen Zeng, Hong Zhou, Zheng Zhu et al. (+1 more)
Accurate molecular subtype classification is essential for personalized breast cancer treatment, yet conventional immunohistochemical analysis relies on invasive biopsies and is prone to sampling bias. Although dynamic contrast-enhanced
magnetic resonance imaging (DCE-MRI) enables non-invasive tumor characterization, clinical workflows typically acquire only single-phase post-contrast images to reduce scan time and contrast
agent dose. In this study, we propose a spatial multi-task learning framework for breast cancer molecular subtype prediction from clinically practical single-phase DCE-MRI. The framework simultaneously predicts estrogen receptor (ER), progesterone receptor (PR), human epidermal growth factor receptor 2 (HER2) status, and the Ki-67 proliferation index – biomarkers that collectively define molecular subtypes. The architecture integrates a deep feature extraction network with multi-scale spatial
attention to capture intratumoral and peritumoral characteristics, together with a region-of-interest weighting module that emphasizes the tumor core, rim, and surrounding tissue. Multi-task learning exploits biological correlations among biomarkers through shared representations with task-specific prediction branches. Experiments on a
dataset of 960 cases (886 internal cases split 7:1:2 for training/validation/testing, and 74 external cases evaluated via five-fold cross-validation) demonstrate that the proposed method achieves an AUC of 0.893, 0.824, and 0.857 for ER, PR, and HER2 classification, respectively, and a mean absolute error of 8.2% for Ki-67 regression, significantly outperforming radiomics and single-task deep learning baselines. These results indicate the feasibility of accurate, non-invasive molecular subtype prediction using standard imaging protocols.
Read the full abstract ▾
All authors Sen Zeng, Hong Zhou, Zheng Zhu, Yang Liu
Computer Vision · Cross-listed from cs.RO
ObjSplat: Geometry-Aware Gaussian Surfels for Active Object Reconstruction
Yuetao Li, Zhizhou Jia, Yu Zhang et al. (+2 more)
Autonomous high-fidelity object reconstruction is fundamental for creating digital assets and bridging the simulation-to-reality gap in robotics. We present ObjSplat, an active reconstruction framework that leverages Gaussian surfels as a unified representation to progressively reconstruct unknown objects with both photorealistic appearance and accurate geometry. Addressing the limitations of conventional opacity or depth-based cues, we introduce a geometry-aware viewpoint evaluation pipeline that explicitly models back-face visibility and occlusion-aware multi-view covisibility, reliably identifying under-reconstructed regions even on geometrically complex objects. Furthermore, to overcome the limitations of greedy planning strategies, ObjSplat employs a next-best-path (NBP) planner that performs multi-step lookahead on a dynamically constructed spatial graph. By jointly optimizing information gain and movement cost, this planner generates globally efficient trajectories. Extensive experiments in simulation and on real-world cultural artifacts demonstrate that ObjSplat produces physically consistent models within minutes, achieving superior reconstruction fidelity and surface completeness while significantly reducing scan time and path length compared to state-of-the-art approaches. Project page: https://li-yuetao.github.io/ObjSplat-page/ .
Read the full abstract ▾
All authors Yuetao Li, Zhizhou Jia, Yu Zhang, Qun Hao, Shaohui Zhang
Computer Vision
Can Textual Reasoning Improve the Performance of MLLMs on Fine-grained Visual Classification?
Jie Zhu, Yiyang Su and Xiaoming Liu
Multi-modal large
language models (MLLMs) exhibit strong general-purpose capabilities, yet still struggle on Fine-Grained Visual Classification (FGVC), a core perception task that requires subtle visual discrimination and is crucial for many real-world applications. A widely adopted strategy for boosting performance on challenging tasks such as math and coding is Chain-of-Thought (CoT)
reasoning. However, several prior works have reported that CoT can actually harm performance on visual perception tasks. These studies, though, examine the issue from relatively narrow angles and leave open why CoT degrades perception-heavy performance. We systematically re-examine the role of CoT in FGVC through the lenses of zero-shot evaluation and multiple training paradigms. Across these settings, we uncover a central paradox: the degradation induced by CoT is largely driven by the
reasoning length, in which longer textual
reasoning consistently lowers classification accuracy. We term this phenomenon the “Cost of Thinking”. Building on this finding, we make two key contributions: (1) MRN, a simple and general plug-and-play normalization method for multi-reward optimization that balances heterogeneous reward signals, and (2) ReFine-RFT, a framework that combines ensemble rewards with MRN to constrain
reasoning length while providing dense accuracy-oriented feedback. Extensive experiments demonstrate the effectiveness of our findings and the proposed ReFine-RFT, achieving state-of-the-art performance across FGVC
benchmarks. Project page: \href{https://refine-rft.github.io/}{ReFine-RFT}.
Read the full abstract ▾
Computer Vision
Unified Personalized Understanding, Generating and Editing
Yu Zhong, Tianwei Lin, Ruike Zhu et al. (+9 more)
Unified large
multimodal models (LMMs) have achieved remarkable progress in general-purpose
multimodal understanding and generation. However, they still operate under a “one-size-fits-all” paradigm and struggle to model user-specific concepts (e.g., generate a photo of <maeve>) in a consistent and controllable manner. Existing personalization methods typically rely on external
retrieval, which is inefficient and poorly integrated into unified
multimodal pipelines. Recent personalized unified models introduce learnable soft
prompts to encode concept information, yet they either couple understanding and generation or depend on complex multi-stage training, leading to cross-task interference and ultimately to fuzzy or misaligned personalized knowledge. We present OmniPersona, an end-to-end personalization framework for unified LMMs that, for the first time, integrates personalized understanding, generation, and image editing within a single architecture. OmniPersona introduces structurally decoupled concept
tokens, allocating dedicated subspaces for different tasks to minimize interference, and incorporates an explicit knowledge replay mechanism that propagates personalized attribute knowledge across tasks, enabling consistent personalized behavior. To systematically evaluate unified personalization, we propose \textbf{OmniPBench}, extending the public UnifyBench concept set with personalized editing tasks and cross-task evaluation protocols integrating understanding, generation, and editing. Experimental results demonstrate that OmniPersona delivers competitive and robust performance across diverse personalization tasks. We hope OmniPersona will serve as a strong baseline and spur further research on controllable, unified personalization.
Read the full abstract ▾
All authors Yu Zhong, Tianwei Lin, Ruike Zhu, Yuqian Yuan, Haoyu Zheng, Liang Liang, Wenqiao Zhang, Feifei Shao, Haoyuan Li, Wanggui He, Hao Jiang, Yueting Zhuang
Computer Vision
SketchJudge: A Diagnostic Benchmark for Grading Hand-drawn Diagrams with Multimodal Large Language Models
Yuhang Su, Mei Wang, Yaoyao Zhong et al. (+4 more)
While
Multimodal Large
Language Models (MLLMs) have achieved remarkable progress in visual understanding, they often struggle when faced with the unstructured and ambiguous nature of human-generated sketches. This limitation is particularly pronounced in the underexplored task of visual grading, where models should not only solve a problem but also diagnose errors in hand-drawn diagrams. Such diagnostic capabilities depend on complex structural, semantic, and metacognitive
reasoning. To bridge this gap, we introduce SketchJudge, a novel
benchmark tailored for evaluating MLLMs as graders of hand-drawn STEM diagrams. SketchJudge encompasses 1,015 hand-drawn student responses across four domains: geometry, physics, charts, and flowcharts, featuring diverse stylistic variations and distinct error types. Evaluations on SketchJudge demonstrate that even advanced MLLMs lag significantly behind humans, validating the
benchmark's effectiveness in exposing the fragility of current
vision-
language alignment in symbolic and noisy contexts. All data, code, and evaluation scripts are publicly available at https://github.com/yuhangsu82/SketchJudge.
Read the full abstract ▾
All authors Yuhang Su, Mei Wang, Yaoyao Zhong, Guozhang Li, Shixing Li, Yihan Feng, Hua Huang
Computer Vision
Chengwen Liu, Xiaomin Yu, Zhuoyue Chang et al. (+15 more)
In real-world
video question answering scenarios,
videos often provide only localized visual cues, while verifiable answers are distributed across the open web; models therefore need to jointly perform cross-frame clue extraction, iterative
retrieval, and multi-hop
reasoning-based verification. To bridge this gap, we construct the first
video deep research
benchmark, VideoDR. VideoDR centers on
video-conditioned open-domain
video question answering, requiring cross-frame visual anchor extraction, interactive web
retrieval, and multi-hop
reasoning over joint
video-web evidence; through rigorous human annotation and quality control, we obtain high-quality
video deep research samples spanning six semantic domains. We evaluate multiple closed-source and open-source
multimodal large
language models under both the Workflow and Agentic paradigms, and the results show that Agentic is not consistently superior to Workflow: its gains depend on a model's ability to maintain the initial
video anchors over long
retrieval chains. Further analysis indicates that goal drift and long-horizon consistency are the core bottlenecks. In sum, VideoDR provides a systematic
benchmark for studying
video agents in open-web settings and reveals the key challenges for next-generation
video deep research
agents.
Read the full abstract ▾
All authors Chengwen Liu, Xiaomin Yu, Zhuoyue Chang, Zhe Huang, Shuo Zhang, Heng Lian, Jisheng Dang, Rui Xu, Sen Hu, Jianheng Hou, Chengwei Qin, Xiaobin Hu, Kunyi Wang, Zhi Yang, Hao Peng, Hong Peng, Ronghao Chen, Huacan Wang
Computer Vision
Measuring Social Bias in Vision-Language Models with Face-Only Counterfactuals from Real Photos
Haodong Chen, Qiang Huang, Jiaqi Zhao et al. (+3 more)
Vision-
Language Models (VLMs) are increasingly deployed in socially consequential settings, raising concerns about social bias driven by demographic cues. A central challenge in measuring such social bias is attribution under visual confounding: real-world images entangle race and gender with correlated factors such as background and clothing, obscuring attribution. We propose a face-only counterfactual evaluation paradigm that isolates demographic effects while preserving real-image realism. Starting from real photographs, we generate counterfactual variants by editing only facial attributes related to race and gender, keeping all other visual factors fixed. Based on this paradigm, we construct FOCUS, a
dataset of 480 scene-matched counterfactual images across six occupations and ten demographic groups, and propose REFLECT, a
benchmark comprising three decision-oriented tasks: two-alternative forced choice, multiple-choice socioeconomic inference, and numeric salary recommendation. Experiments on five state-of-the-art VLMs reveal that demographic disparities persist under strict visual control and vary substantially across task formulations. These findings underscore the necessity of controlled, counterfactual audits and highlight task design as a critical factor in evaluating social bias in
multimodal models.
Read the full abstract ▾
All authors Haodong Chen, Qiang Huang, Jiaqi Zhao, Qiuping Jiang, Xiaojun Chang, Jun Yu
Computer Vision
RenderFlow: Single-Step Neural Rendering via Flow Matching
Shenghao Zhang, Runtao Liu, Christopher Schroers et al. (+1 more)
Conventional physically based rendering (PBR) pipelines generate photorealistic images through computationally intensive light transport simulations. Although recent deep learning approaches leverage
diffusion model priors with geometry buffers (G-buffers) to produce visually compelling results without explicit scene geometry or light simulation, they remain constrained by two major limitations. First, the iterative nature of the
diffusion process introduces substantial latency. Second, the inherent stochasticity of these generative models compromises physical accuracy and temporal consistency. In response to these challenges, we propose a novel, end-to-end, deterministic, single-step neural rendering framework, RenderFlow, built upon a flow matching paradigm. To further strengthen both rendering quality and generalization, we propose an efficient and effective module for sparse keyframe guidance. Our method significantly accelerates the rendering process and, by optionally incorporating sparsely rendered keyframes as guidance, enhances both the physical plausibility and overall visual quality of the output. The resulting pipeline achieves near real-time performance with photorealistic rendering quality, effectively bridging the gap between the efficiency of modern generative models and the precision of traditional physically based rendering. Furthermore, we demonstrate the versatility of our framework by introducing a lightweight, adapter-based module that efficiently repurposes the pretrained forward model for the inverse rendering task of intrinsic decomposition.
Read the full abstract ▾
All authors Shenghao Zhang, Runtao Liu, Christopher Schroers, Yang Zhang
Computer Vision
UDPNet: Unleashing Depth-based Priors for Robust Image Dehazing
Zengyuan Zuo, Junjun Jiang, Gang Wu et al. (+1 more)
Image dehazing has witnessed significant advancements with the development of deep learning models. However, most existing methods focus solely on single-modal RGB features, neglecting the inherent correlation between scene depth and haze distribution. Even those that jointly optimize depth estimation and image dehazing often suffer from suboptimal performance due to inadequate utilization of accurate depth information. In this paper, we present UDPNet, a general framework that leverages depth-based priors from a large-scale pretrained depth estimation model DepthAnything V2 to boost existing image dehazing models. Specifically, our architecture comprises two key components: the Depth-Guided
Attention Module (DGAM) adaptively modulates features via lightweight depth-guided channel
attention, and the Depth Prior Fusion Module (DPFM) enables hierarchical fusion of multi-scale depth map features by dual sliding-window multi-head cross-
attention mechanism. These modules ensure both computational efficiency and effective integration of depth priors. Moreover, the depth priors empower the network to dynamically adapt to varying haze densities, illumination conditions, and domain gaps across synthetic and real-world data. Extensive experimental results demonstrate the effectiveness of our UDPNet, outperforming the state-of-the-art methods on popular dehazing
datasets, with PSNR improvements of 0.85 dB on SOTS-indoor, 1.19 dB on Haze4K, and 1.79 dB on NHR. Our proposed solution establishes a new
benchmark for depth-aware dehazing across various scenarios. Pretrained models and codes are released at our project https://github.com/Harbinzzy/UDPNet.
Read the full abstract ▾
All authors Zengyuan Zuo, Junjun Jiang, Gang Wu, Xianming Liu
Computer Vision
CLIMP: Contrastive Language-Image Mamba Pretraining
Nimrod Shabtay, Itamar Zimerman, Eli Schwartz et al. (+1 more)
Contrastive
Language-Image Pre-training (CLIP) relies on
Vision Transformers whose
attention mechanism is susceptible to spurious correlations, and scales quadratically with resolution. To address these limitations, We present CLIMP, the first fully Mamba-based contrastive
vision-
language model that replaces both the
vision and text encoders with Mamba. The new architecture encodes sequential structure in both
vision and
language, with VMamba capturing visual spatial inductive biases, reducing reliance on spurious correlations and producing an
embedding space favorable for cross-modal
retrieval and out-of-distribution robustness-surpassing OpenAI's CLIP-ViT-B by 7.5% on ImageNet-O. CLIMP naturally supports variable input resolutions without positional encoding interpolation or specialized training, achieving up to 6.6% higher
retrieval accuracy at 16x training resolution while using 5x less
memory and 1.8x fewer FLOPs. The autoregressive text encoder further overcomes CLIP's fixed context limitation, enabling dense captioning
retrieval. Our findings suggest that Mamba exhibits advantageous properties for
vision-
language learning, making it a compelling alternative to
Transformer-based CLIP.The code and models are publicly available at https://github.com/NimrodShabtay/CLIMP}
Read the full abstract ▾
All authors Nimrod Shabtay, Itamar Zimerman, Eli Schwartz, Raja Giryes
Computer Vision
MixRI: Mixing Features of Reference Images for Novel Object Pose Estimation
Xinhang Liu, Jiawei Shi, Zheng Dang et al. (+1 more)
We present MixRI, a lightweight network that solves the CAD-based novel object pose estimation problem in RGB images. It can be instantly applied to a novel object at test time without finetuning. We design our network to meet the demands of real-world applications, emphasizing reduced
memory requirements and fast inference time. Unlike existing works that utilize many reference images and have large network parameters, we directly match points based on the multi-view information between the query and reference images with a lightweight network. Thanks to our reference image fusion strategy, we significantly decrease the number of reference images, thus decreasing the time needed to process these images and the
memory required to store them. Furthermore, with our lightweight network, our method requires less inference time. Though with fewer reference images, experiments on seven core
datasets in the BOP challenge show that our method achieves comparable results with other methods that require more reference images and larger network parameters.
Read the full abstract ▾
All authors Xinhang Liu, Jiawei Shi, Zheng Dang, Yuchao Dai
Computer Vision
Unsupervised Domain Adaptation with SAM-RefiSeR for Enhanced Brain Tumor Segmentation
Dillan Imans, Phuoc-Nguyen Bui, Duc-Tai Le et al. (+1 more)
Unsupervised Domain Adaptation with SAM-RefiSeR for Enhanced
Brain Tumor
Segmentation
Read the full abstract ▾
All authors Dillan Imans, Phuoc-Nguyen Bui, Duc-Tai Le, Hyunseung Choo
Computer Vision
MVGGT: Multimodal Visual Geometry Grounded Transformer for Multiview 3D Referring Expression Segmentation
Changli Wu, Haodong Wang, Jiayi Ji et al. (+5 more)
Most existing 3D referring expression
segmentation (3DRES) methods rely on dense, high-quality point clouds, while real-world
agents such as
robots and mobile phones operate with only a few sparse RGB views and strict latency constraints. We introduce Multi-view 3D Referring Expression
Segmentation (MV-3DRES), where the model must recover scene structure and segment the referred object directly from sparse multi-view images. Traditional two-stage pipelines, which first reconstruct a point cloud and then perform
segmentation, often yield low-quality geometry, produce coarse or degraded target regions, and run slowly. We propose the
Multimodal Visual Geometry Grounded
Transformer (MVGGT), an efficient end-to-end framework that integrates
language information into sparse-view
geometric reasoning through a dual-branch design. Training in this setting exposes a critical optimization barrier, termed Foreground Gradient Dilution (FGD), where sparse 3D signals lead to weak supervision. To resolve this, we introduce Per-view No-target Suppression Optimization (PVSO), which provides stronger and more balanced gradients across views, enabling stable and efficient learning. To support consistent evaluation, we build MVRefer, a
benchmark that defines standardized settings and metrics for MV-3DRES. Experiments show that MVGGT establishes the first strong baseline and achieves both high accuracy and fast inference, outperforming existing alternatives. The code is available at https://mvggt.github.io/.
Read the full abstract ▾
All authors Changli Wu, Haodong Wang, Jiayi Ji, Yutian Yao, Chunsai Du, Jihua Kang, Yanwei Fu, Liujuan Cao
Computer Vision · Cross-listed from cs.CR
Learning QoE from Packet-Level Measurements in Encrypted Video Conferencing Traffic
Michael Sidorov and Ofer Hadar
The quality of the user experience has become one of the most important aspects in todays world, as it directly influences individuals willingness to continue using or abandon a product or service. In this context,
video conferencing applications (VCAs), which experienced widespread adoption following the COVID-19 pandemic, must deliver excellent performance to remain competitive in an increasingly crowded
market. Although content providers (CPs) such as Zoom, WhatsApp, Telegram, and Google Meet can assess conversation quality by comparing transmitted and received data. The widespread use of end-to-end encryption in VCAs makes quality-of-experience (QoE) evaluation by internet service providers (ISPs) far more challenging. Since ISPs do not have access to the encrypted content, they must rely on passive measurements of unencrypted traffic characteristics on the data path. In this work, we present a simple yet effective QoE prediction framework based on an almost stock convolutional neural network (CNN) architecture that uses only the packet sizes extracted from the communication between two participants in a
video conferencing (VC) call to predict two QoE metrics: BRISQUE and MOS. The proposed framework is simple, easy to implement, and does not require high-end computational resources, yet it provides superior prediction performance, as shown in our experiments on two custom
datasets collected from WhatsApp and Zoom, which achieve substantial improvements over previous models for the QoE prediction task.
Read the full abstract ▾
Computer Vision
MedGround: Bridging the Evidence Gap in Medical Vision-Language Models with Verified Grounding Data
Mengmeng Zhang, Xiaoping Wu, Hao Luo et al. (+2 more)
Vision-
Language Models (VLMs) can generate convincing clinical narratives, yet frequently struggle to visually ground their statements. We posit that this limitation arises from the scarcity of high-quality, large-scale clinical referring-localization pairs. To address this, we introduce MedGround, an automated pipeline that transforms
segmentation resources into high-quality medical referring grounding data. Leveraging expert masks as spatial anchors, MedGround precisely derives localization targets, extracts shape and spatial cues, and guides VLMs to synthesize natural, clinically grounded queries that reflect morphology and location. To ensure data rigor, a multi-stage verification system integrates strict formatting checks, geometry- and medical-prior rules, and image-based visual judging to filter out ambiguous or visually unsupported samples. Finally, we present MedGround-35K, a novel
multimodal medical
dataset. Extensive experiments demonstrate that VLMs trained with MedGround-35K consistently achieve improved referring grounding performance, enhance multi-object semantic disambiguation, and exhibit strong generalization to unseen grounding settings. This work highlights MedGround as a scalable, data-driven approach to anchor medical
language to verifiable visual evidence.
Read the full abstract ▾
All authors Mengmeng Zhang, Xiaoping Wu, Hao Luo, Fan Wang, Yisheng Lv
Computer Vision
Speak While Watching: Unleashing TRUE Real-Time Video Understanding Capability of Multimodal Large Language Models
Junyan Lin, Junlong Tong, Hao Wu et al. (+4 more)
Multimodal Large
Language Models (MLLMs) have achieved strong performance across many tasks, yet most systems remain limited to offline inference, requiring complete inputs before generating outputs. Recent streaming methods reduce latency by interleaving perception and generation, but still enforce a sequential perception-generation cycle, limiting real-time interaction. In this work, we target a fundamental bottleneck that arises when extending MLLMs to real-time
video understanding: the global positional continuity constraint imposed by standard positional encoding schemes. While natural in offline inference, this constraint tightly couples perception and generation, preventing effective input-output parallelism. To address this limitation, we propose a parallel streaming framework that relaxes positional continuity through three designs: Overlapped, Group-Decoupled, and Gap-Isolated. These designs enable simultaneous perception and generation, allowing the model to process incoming inputs while producing responses in real time. Extensive experiments reveal that Group-Decoupled achieves the best efficiency-performance balance, maintaining high fluency and accuracy while significantly reducing latency. We further show that the proposed framework yields up to 2x acceleration under balanced perception-generation workloads, establishing a principled pathway toward speak-while-watching real-time systems. We make all our code publicly available: https://github.com/EIT-NLP/Speak-While-Watching.
Read the full abstract ▾
All authors Junyan Lin, Junlong Tong, Hao Wu, Jialiang Zhang, Jinming Liu, Xin Jin, Xiaoyu Shen
Computer Vision
PRISM: Color-Stratified Point Cloud Sampling
Hansol Lim, Minhyeok Im and Jongseong Brad Choi
We present PRISM, a novel color-guided stratified sampling method for RGB-
LiDAR point clouds. Our approach is motivated by the observation that unique scene features often exhibit chromatic diversity while repetitive, redundant features are homogeneous in color. Conventional downsampling methods (Random Sampling, Voxel Grid, Normal Space Sampling) enforce spatial uniformity while ignoring this photometric content. In contrast, PRISM allocates sampling density proportional to chromatic diversity. By treating RGB color space as the stratification domain and imposing a maximum capacity k per color bin, the method preserves texture-rich regions with high color variation while substantially reducing visually homogeneous surfaces. This shifts the sampling space from spatial coverage to visual complexity to produce sparser point clouds that retain essential features for 3D reconstruction tasks.
Read the full abstract ▾
Computer Vision
OSCAR: Optical-aware Semantic Control for Aleatoric Refinement in Sar-to-Optical Translation
Hyunseo Lee, Sang Min Kim, Ho Kyung Shin et al. (+2 more)
Synthetic Aperture Radar (SAR) provides robust all-weather imaging capabilities; however, translating SAR observations into photo-realistic optical images remains a fundamentally ill-posed problem. Current approaches are often hindered by the inherent speckle noise and
geometric distortions of SAR data, which frequently result in semantic misinterpretation, ambiguous texture synthesis, and structural
hallucinations. To address these limitations, a novel SAR-to-Optical (S2O) translation framework is proposed, integrating three core technical contributions: (i) Cross-Modal Semantic
Alignment, which establishes an Optical-Aware SAR Encoder by distilling robust semantic priors from an Optical Teacher into a SAR Student (ii) Semantically-Grounded Generative Guidance, realized by a Semantically-Grounded ControlNet that integrates class-aware text
prompts for global context with hierarchical visual
prompts for local spatial guidance; and (iii) an Uncertainty-Aware Objective, which explicitly models aleatoric uncertainty to dynamically modulate the reconstruction focus, effectively mitigating artifacts caused by speckle-induced ambiguity. Extensive experiments demonstrate that the proposed method achieves superior perceptual quality and semantic consistency compared to state-of-the-art approaches.
Read the full abstract ▾
All authors Hyunseo Lee, Sang Min Kim, Ho Kyung Shin, Taeheon Kim, Woo-Jeoung Nam