Computer Vision · Cross-listed from cs.RO
EagleVision: A Multi-Task Benchmark for Cross-Domain Perception in High-Speed Autonomous Racing
Zakhar Yagudin, Murad Mebrahtu, Ren Jin et al. (+5 more)
High-speed autonomous racing presents extreme perception challenges, including large relative velocities and substantial domain shifts from conventional urban-driving
datasets. Existing
benchmarks do not adequately capture these high-dynamic conditions. We introduce EagleVision, a unified
LiDAR-based multi-task
benchmark for 3D detection and trajectory prediction in high-speed racing, providing newly annotated 3D bounding boxes for the Indy Autonomous Challenge
dataset (14,893 frames) and the A2RL Real competition
dataset (1,163 frames), together with 12,000 simulator-generated annotated frames, all standardized under a common evaluation protocol. Using a
dataset-centric transfer framework, we quantify cross-domain generalization across urban, simulator, and real racing domains. Urban pretraining improves detection over scratch training (NDS 0.72 vs. 0.69), while intermediate pretraining on real racing data achieves the best transfer to A2RL (NDS 0.726), outperforming simulator-only adaptation. For trajectory prediction, Indy-trained models surpass in-domain A2RL training on A2RL test sequences (FDE 0.947 vs. 1.250), highlighting the role of motion-distribution coverage in cross-domain
forecasting. EagleVision enables systematic study of perception generalization under extreme high-speed dynamics. The
dataset and
benchmark are publicly available at https://avlab.io/EagleVision
Read the full abstract ▾
All authors Zakhar Yagudin, Murad Mebrahtu, Ren Jin, Jiaqi Huang, Yujia Yue, Dzmitry Tsetserukou, Jorge Dias, Majid Khonji
Computer Vision
Lost in Adaptation: Layer-Selective Recovery of Temporal Reasoning in Video-Language Models
Zihang Fu, Haonan Wang, Jian Kang et al. (+2 more)
Multimodal adaptation can weaken temporal
reasoning (TR) in
video-
language models (VLMs) even when the visual evidence needed for an answer is correctly perceived. This weakened competence, however, remains partly recoverable from the model's paired pre-adaptation
language checkpoint. We introduce MERIT, a gradient-free framework that selectively restores this competence across the
language backbone while protecting temporal perception (TP). It assigns each self-
attention layer a VLM-dominant or LLM-dominant interpolation and uses the Covariance
Matrix Adaptation
Evolution Strategy (CMA-ES) to search the recipe space under a perception-constrained objective. Across three VLM families and five
video benchmarks, MERIT improves
reasoning-aligned performance in all 15 model–
benchmark evaluations while preserving TP across all three
Video-MME settings. Recipes selected using 232 diagnostic examples transfer to four unseen
benchmarks, with relative gains up to 27.8%, and outperform full-backbone, uniform-
attention, and alpha-tuned random-layer merging. Interventional masking shows that MERIT-selected layers are disproportionately important for TR, while frame-level attribution reveals increased reliance on temporally distributed, causally relevant evidence. These results show that adaptation-induced
reasoning degradation can be repaired post hoc through selective reuse of pre-adaptation competence, without learning new parameters or adding inference overhead.
Read the full abstract ▾
All authors Zihang Fu, Haonan Wang, Jian Kang, Kenji Kawaguchi, Jiaying Wu
Computer Vision
Video-based Heart Rate Estimation with Angle-guided ROI Optimization and Graph Signal Denoising
Gan Pei, Junhao Ning, Boqiu Shen et al. (+2 more)
Remote photoplethysmography (rPPG) enables non-contact heart rate measurement from facial
videos, but its performance is significantly degraded by facial motions such as speaking and head shaking. To address this issue, we propose two plug-and-play modules. The Angle-guided ROI Adaptive Optimization module quantifies ROI-Camera angles to refine motion-affected signals and capture global motion, while the Multi-region Joint Graph Signal Denoising module jointly models intra- and inter-regional ROI signals using graph signal processing to suppress motion artifacts. The modules are compatible with reflection model-based rPPG methods and validated on three public
datasets. Results show that jointly use markedly reduces MAE, with an average decrease of 20.38% over the baseline, while ablation studies confirm the effectiveness of each module. The work demonstrates the potential of angle-guided optimization and graph-based denoising to enhance rPPG performance in motion scenarios.
Read the full abstract ▾
All authors Gan Pei, Junhao Ning, Boqiu Shen, Yan Zhu, Menghan Hu
Computer Vision
Extending Reconstruction: Reconstruction-to-Vector Diffusion for Hyperspectral Anomaly Detection
Jijun Xiang, Tao Wang, Jiayi Wang et al. (+3 more)
While Hyperspectral Anomaly Detection (HAD) excels at identifying sparse targets in complex scenes, existing models remain trapped in a scalar "reconstruction-as-endpoint" paradigm. This reliance on ambiguous scalar residuals consistently triggers sub-pixel anomaly vanishing during spatial downsampling, alongside severe confirmation bias when unpurified anomalies corrupt training weights. In this paper, we propose Reconstruction-to-Vector
Diffusion (R2VD), which fundamentally redefines reconstruction as a
manifold purification origin to establish a novel residual-guided generative dynamics paradigm. Our framework introduces a four-stage pipeline: (1) a Physical Prior Extraction (PPE) stage that mitigates early confirmation bias via dual-stream statistical guidance; (2) a Guided
Manifold Purification (GMP) stage utilizing an OmniContext Autoencoder (OCA) to extract purified residual maps while preserving fragile sub-pixel
topologies; (3) a Residual Score Modeling (RSM) stage where a
Diffusion Transformer (DiT), guarded by a Physical Spectral Firewall (PSF), effectively isolates cross-spectral leakage; and (4) a Vector Dynamics Inference (VDI) stage that robustly decouples targets from backgrounds by evaluating high-dimensional vector interference patterns instead of conventional scalar errors. Comprehensive evaluations on eight
datasets confirm that R2VD establishes a new state-of-the-art, delivering exceptional target detectability and background suppression. The code is available at https://github.com/Bondojijun/R2VD.
Read the full abstract ▾
All authors Jijun Xiang, Tao Wang, Jiayi Wang, Pengxiang Wang, Cheng Chen, Nian Wang
Computer Vision
ConvFormer3D-TAP: Phase/Uncertainty-Aware Front-End Fusion for Cine CMR View Classification Pipelines
Nafiseh Ghaffar Nia, Vinesh Appadurai, Suchithra V. et al. (+4 more)
Reliable recognition of standard cine cardiac MRI views is essential because each view determines which cardiac anatomy is visualized and which quantitative analyses can be performed. Incorrect view identification, whether by a human reader or an automated deep learning system, can propagate errors into
segmentation, volumetric assessment, strain analysis, and valve evaluation. However, accurate view classification remains challenging under routine clinical variability in scanner vendor, acquisition protocol, motion artifacts, and plane prescription. We present ConvFormer3D-TAP, a cine-specific spatiotemporal architecture that integrates 3D convolutional tokenization with multiscale self-
attention. The model is trained using masked spatiotemporal reconstruction and uncertainty-weighted multi-clip fusion to enhance robustness across cardiac phases and ambiguous temporal segments. The design captures complementary cues: local anatomical structure through convolutional priors and long-range cardiac-cycle dynamics through hierarchical
attention. On a cohort of 150,974 clinically acquired cine sequences spanning six standard cine cardiac MRI views, ConvFormer3D-TAP achieved 96% validation accuracy with per-class F1-scores >= 0.94 and strong calibration (ECE = 0.025; Brier = 0.040). Error analysis shows that residual confusions are concentrated in anatomically adjacent long-axis and LVOT/AV view pairs, consistent with intrinsic prescription overlap. These results support ConvFormer3D-TAP as a scalable front-end for view routing, filtering and quality control in end-to-end cMRI workflows.
Read the full abstract ▾
All authors Nafiseh Ghaffar Nia, Vinesh Appadurai, Suchithra V., Chinmay Rane, Daniel Pittman, James Carr, Adrienne Kline
Computer Vision · Cross-listed from cs.RO
ComSim: Building Scalable Real-World Robot Data Generation via Compositional Simulation
Yiran Qin, Jiahua Ma, Li Kang et al. (+11 more)
Recent advancements in foundational models, such as large
language models and world models, have greatly enhanced the capabilities of robotics, enabling
robots to autonomously perform complex tasks. However, acquiring large-scale, high-quality training data for robotics remains a challenge, as it often requires substantial manual effort and is limited in its coverage of diverse real-world environments. To address this, we propose a novel hybrid approach called Compositional Simulation, which combines classical simulation and neural simulation to generate accurate action-
video pairs while maintaining real-world consistency. Our approach utilizes a closed-loop real-sim-real data augmentation pipeline, leveraging a small amount of real-world data to generate diverse, large-scale training
datasets that cover a broader spectrum of real-world scenarios. We train a neural simulator to transform classical simulation
videos into real-world representations, improving the accuracy of policy models trained in real-world environments. Through extensive experiments, we demonstrate that our method significantly reduces the sim2real domain gap, resulting in higher success rates in real-world policy model training. Our approach offers a scalable solution for generating robust training data and bridging the gap between simulated and real-world robotics.
Read the full abstract ▾
All authors Yiran Qin, Jiahua Ma, Li Kang, Wenzhan Li, Yihang Jiao, Xin Wen, Xiufeng Song, Heng Zhou, Jiwen Yu, Zhenfei Yin, Xihui Liu, Philip Torr, Yilun Du, Ruimao Zhang
Computer Vision
From Redaction to Restoration: Deep Learning for Medical Image Anonymization and Reconstruction
Adrienne Kline, Abhijit Gaonkar, Daniel Pittman et al. (+2 more)
Removing patient-specific information from medical images is crucial to enable sharing and open science without compromising patient identities. However, many methods currently used for deidentification have negative effects on downstream image analysis tasks because of removal of relevant but non-identifiable information. This work presents an end-to-end deep learning framework for transforming raw clinical image volumes into de-identified, analysis-ready
datasets without compromising downstream utility. The methodology developed and tested in this work first detects and redacts regions likely to contain protected health information (PHI), such as burned-in text and metadata, and then uses a generative deep learning model to inpaint the redacted areas with anatomically and imaging plausible content. The proposed pipeline leverages a lightweight hybrid architecture, combining CRNN-based redaction with a latent-
diffusion inpainting restoration module (Stable
Diffusion 2). We evaluate the approach using both
privacy-oriented metrics, which quantify residual PHI and success of redaction, and image-quality and task-based metrics, which assess the fidelity of restored volumes for representative deep learning applications. Our results suggest that the proposed method yields de-identified medical images that are visually coherent, maintaining fidelity for downstream models, while substantially reducing the risk of patient re-identification. By automating anonymization and image reconstruction within a single workflow, and dissemination of large-scale medical imaging collections, thereby lowering a key barrier to data sharing and multi-institutional collaboration in medical imaging AI.
Read the full abstract ▾
All authors Adrienne Kline, Abhijit Gaonkar, Daniel Pittman, Chris Kuehn, Nils Forkert
Computer Vision
What Do Vision-Language Models Encode for Personalized Image Aesthetics Assessment?
Koki Ryu and Hitomi Yanaka
Personalized image aesthetics assessment (PIAA) is an important research problem with practical real-world applications. While methods based on
vision-
language models (VLMs) are promising candidates for PIAA, it remains unclear whether they internally encode rich, multi-level aesthetic attributes required for effective personalization. In this paper, we first analyze the internal representations of VLMs to examine the presence and distribution of such aesthetic attributes, and then leverage them for lightweight, individual-level personalization without model fine-tuning. Our analysis reveals that VLMs encode diverse aesthetic attributes that propagate into the
language decoder layers. Building on these representations, we demonstrate that simple linear models can perform PIAA effectively. We further analyze how aesthetic information is transferred across layers in different VLM architectures and across image domains. Our findings provide insights into how VLMs can be utilized for modeling subjective, individual aesthetic preferences. Our code is available at https://github.com/ynklab/vlm-latent-piaa.
Read the full abstract ▾
Computer Vision
LEADER: Learning Reliable Local-to-Global Correspondences for LiDAR Relocalization
Jianshi Wu, Minghang Zhu, Dunqiang Liu et al. (+5 more)
LiDAR relocalization has attracted increasing
attention as it can deliver accurate 6-DoF pose estimation in complex 3D environments. Recent learning-based regression methods offer efficient solutions by directly predicting global poses without the need for explicit map storage. However, these methods often struggle in challenging scenes due to their equal treatment of all predicted points, which is vulnerable to noise and outliers. In this paper, we propose LEADER, a robust
LiDAR-based relocalization framework enhanced by a simple, yet effective
geometric encoder. Specifically, a Robust Projection-based
Geometric Encoder architecture which captures multi-scale
geometric features is first presented to enhance descriptiveness in
geometric representation. A Truncated Relative Reliability loss is then formulated to model point-wise ambiguity and mitigate the influence of unreliable predictions. Extensive experiments on the Oxford RobotCar and NCLT
datasets demonstrate that LEADER outperforms state-of-the-art methods, achieving 24.1% and 73.9% relative reductions in position error over existing techniques, respectively. The source code is released on https://github.com/JiansW/LEADER.
Read the full abstract ▾
All authors Jianshi Wu, Minghang Zhu, Dunqiang Liu, Wen Li, Sheng Ao, Siqi Shen, Chenglu Wen, Cheng Wang
Computer Vision
LoGo-MR: Screening Breast MRI for Cancer Risk Prediction by Efficient Omni-Slice Modeling
Xin Wang, Yuan Gao, George Yiasemis et al. (+13 more)
Efficient and explainable breast cancer (BC) risk prediction is critical for large-scale population-based screening. Breast MRI provides functional information for personalized risk assessment. Yet effective modeling remains challenging as fully 3D CNNs capture volumetric context at high computational cost, whereas lightweight 2D CNNs fail to model inter-slice continuity. Importantly, breast MRI modeling for shor- and long-term BC risk stratification remains underexplored. In this study, we propose LoGo-MR, a 2.5D local-global structural modeling framework for five-year BC risk prediction. Aligned with clinical interpretation, our framework first employs neighbor-slice encoding to capture subtle local cues linked to short-term risk. It then integrates
transformer-enhanced multiple-instance learning (MIL) to model distributed global patterns related to long-term risk and provide interpretable slice importance. We further apply this framework across axial, sagittal, and coronal planes as LoGo3-MR to capture complementary volumetric information. This multi-plane formulation enables voxel-level risk saliency mapping, which may assist radiologists in localizing risk-relevant regions during breast MRI interpretation. Evaluated on a large breast MRI screening cohort (~7.5K), our method outperforms 2D/3D baselines and existing SOTA MIL methods, achieving AUCs of 0.77-0.69 for 1- to 5-year prediction and improving C-index by ~6% over 3D CNNs. LoGo3-MR further improves overall performance with interpretable localization across three planes, and validation across seven backbones shows consistent gains. These results highlight the clinical potential of efficient MRI-based BC risk stratification for large-scale screening. Code will be released publicly.
Read the full abstract ▾
All authors Xin Wang, Yuan Gao, George Yiasemis, Antonio Portaluri, Zahra Aghdam, Muzhen He, Luyi Han, Yaofei Duan, Chunyao Lu, Xinglong Liang, Tianyu Zhang, Vivien van Veldhuizen, Yue Sun, Tao Tan, Ritse Mann, Jonas Teuwen
Computer Vision
A Compact and Efficient 1.251 Million Parameter Machine Learning CNN Model PD36-C for Plant Disease Detection: A Case Study
Shkelqim Sherifi
Deep learning has markedly advanced image based plant disease diagnosis as improved hardware and
dataset quality have enabled increasingly accurate neural network models. This paper presents PD36 C, a compact convolutional neural network (1,250,694 parameters and 4.77 MB) for plant disease classification. Trained with TensorFlow Keras on the New Plant Diseases
Dataset (87k images, 38 classes), PD36 C is designed for robustness and edge deployability, complemented by a Qt for Python desktop application that offers an intuitive GUI and offline inference on commodity hardware. Across experiments, training accuracy reached 0.99697 by epoch 30, and average test accuracy was 0.9953 across 38 classes. Per class performance is uniformly high; on the lower end, Corn (maize) Cercospora leaf spot achieved precision around 0.9777 and recall around 0.9634, indicating occasional confusion with visually similar categories, while on the upper end numerous classes including Apple Black rot, Cedar apple rust, Blueberry healthy, Cherry Powdery mildew, Cherry healthy, and all four grape categories achieved perfect precision 1.00 and recall of 1.00, indicating no false positives and strong coverage. These results show that with a well curated
dataset and careful architectural design, small CNNs can achieve competitive accuracy compared with recent baselines while remaining practical for edge scenarios. We also note typical constraints such as adverse weather, low quality imagery, and leaves exhibiting multiple concurrent diseases that can degrade performance and warrant future work on domain robustness. Overall, PD36 C and its application pipeline contribute a field ready, efficient solution for AI assisted plant disease detection in smart agriculture.
Read the full abstract ▾
Computer Vision
Any 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale
Dongxu Wei, Qi Xu, Zhiqi Li et al. (+6 more)
Scene-level 3D generation has long been dominated by 2D multi-view or
video diffusion models. Typically, these 2D-based approaches perform scene generation in a 2D latent space, which introduces two fundamental issues: (i) representing 3D scenes via 2D views leads to significant representation redundancy, and (ii) latent space rooted in 2D inherently limits the spatial consistency of the generated scenes. In this paper, we propose to perform 3D scene generation directly within a high-dimensional implicit 3D latent space derived from powerful 2D foundation models (e.g., DINOv2, SigLIP2, DA3). Then we tame
diffusion transformers to perform
diffusion modeling directly in the 3D latent space, enabling 3D-grounded scene generation. In our experiments, we not only demonstrate the advantages of our method over 2D-based approaches in terms of efficiency and spatial consistency, but also comprehensively analyze the impact of 3D latent representation on 3D scene generation performance. Furthermore, we validate the favorable scalability of our method with respect to representation and model capacity.
Read the full abstract ▾
All authors Dongxu Wei, Qi Xu, Zhiqi Li, Hangning Zhou, Cong Qiu, Hailong Qin, Mu Yang, Zhaopeng Cui, Peidong Liu
Computer Vision · Cross-listed from cs.CR
The Salami Slicing Threat: Exploiting Cumulative Risks in LLM Systems
Yihao Zhang, Kai Wang, Jiangrong Wu et al. (+7 more)
Large
Language Models (LLMs) face prominent security risks from jailbreaking, a practice that manipulates models to bypass built-in security constraints and generate unethical or unsafe content. Among various jailbreak techniques, multi-turn jailbreak attacks are more covert and persistent than single-turn counterparts, exposing critical vulnerabilities of LLMs. However, existing multi-turn jailbreak methods suffer from two fundamental limitations that affect the actual impact in real-world scenarios: (a) As models become more context-aware, any explicit harmful trigger is increasingly likely to be flagged and blocked; (b) Successful final-step triggers often require finely tuned, model-specific contexts, making such attacks highly context-dependent. To fill this gap, we propose Salami Slicing Risk, which operates by chaining numerous low-risk inputs that individually evade
alignment thresholds but cumulatively accumulate harmful intent to ultimately trigger high-risk behaviors, without heavy reliance on pre-designed contextual structures. Building on this risk, we develop Salami Attack, an automatic framework universally applicable to multiple model types and modalities. Rigorous experiments demonstrate its state-of-the-art performance across diverse models and modalities, achieving over 90% Attack Success Rate on GPT-4o and Gemini, as well as robustness against real-world
alignment defenses. We also proposed a defense strategy to constrain the Salami Attack by at least 44.8% while achieving a maximum blocking rate of 64.8% against other multi-turn jailbreak attacks. Our findings provide critical insights into the pervasive risks of multi-turn jailbreaking and offer actionable mitigation strategies to enhance LLM security.
Read the full abstract ▾
All authors Yihao Zhang, Kai Wang, Jiangrong Wu, Haolin Wu, Yuxuan Zhou, Zeming Wei, Dongxian Wu, Xun Chen, Jun Sun, Meng Sun
Computer Vision
Multimodal Large Language Model-Enabled Video Translation: A Role-Oriented Survey
Bingzheng Qu, Kehai Chen, Xuefeng Bai et al. (+1 more)
Recent progress in
multimodal large
language models (MLLMs) is reshaping
video translation from a cascaded pipeline of automatic
speech recognition, machine translation, text-to-
speech, and lip synchronization into a unified
multimodal reasoning and generation problem. High-quality
video translation requires not only semantic fidelity, but also temporal
alignment, speaker consistency, and emotional expressiveness across visual, acoustic, and linguistic streams. This survey provides a focused review of MLLM-enabled
video translation through a role-oriented taxonomy. We organize MLLM-enabled and MLLM-relevant studies into three functional roles: Semantic Reasoner, which grounds translation in
video understanding, temporal
reasoning, and
multimodal fusion; Expressive Performer, which supports controllable and context-aware
speech generation; and Visual Synthesizer, which enables lip synchronization and visually coherent speaker rendering. We further summarize representative
datasets,
benchmarks, and metrics for each role, and discuss how current evaluation protocols fall short of end-to-end
video translation requirements. Finally, we identify open challenges in long-form
video understanding, temporal modeling,
multimodal alignment,
multilingual robustness, and responsible deployment, outlining future directions for natural and trustworthy cross-lingual
video communication.
Read the full abstract ▾
All authors Bingzheng Qu, Kehai Chen, Xuefeng Bai, Min Zhang
Computer Vision
A Deep Equilibrium Network for Hyperspectral Unmixing
Chentong Wang, Jincheng Gao, Fei Zhu et al. (+1 more)
Hyperspectral unmixing (HU) is crucial for analyzing hyperspectral imagery, yet achieving accurate unmixing remains challenging. While traditional methods struggle to effectively model complex spectral-spatial features, deep learning approaches often lack physical interpretability. Unrolling-based methods, despite offering network interpretability, inadequately exploit spectral-spatial information and incur high
memory costs and numerical precision issues during backpropagation. To address these limitations, we propose DEQ-Unmix, which reformulates abundance estimation as a deep
equilibrium model, enabling efficient constant-
memory training via implicit differentiation. It replaces the gradient operator of the data reconstruction term with a trainable convolutional network to capture spectral-spatial information. By leveraging implicit differentiation, DEQ-Unmix enables efficient and constant-
memory backpropagation. Experiments on synthetic and two real-world
datasets demonstrate that DEQ-Unmix achieves superior unmixing performance while maintaining constant
memory cost.
Read the full abstract ▾
All authors Chentong Wang, Jincheng Gao, Fei Zhu, Jie Chen
Computer Vision
Variational Latent Entropy Estimation Disentanglement: Controlled Attribute Leakage for Face Recognition
Ünsal Öztürk, Vedrana Krivokuća Hahn, Sushil Bhattacharjee et al. (+1 more)
Face recognition
embeddings encode identity, but they also encode other factors such as gender and ethnicity. Depending on how these factors are used by a downstream system, separating them from the information needed for verification is important for both
privacy and
fairness. We propose Variational Latent
Entropy Estimation Disentanglement (VLEED), a post-hoc method that transforms pretrained
embeddings with a variational autoencoder and encourages a distilled representation where the categorical variable of interest is separated from identity-relevant information. VLEED uses a mutual information-based objective realised through the estimation of the
entropy of the categorical attribute in the latent space, and provides stable training with fine-grained control over information removal. We evaluate our method on IJB-C, RFW, and VGGFace2 for gender and ethnicity disentanglement, and compare it to various state-of-the-art methods. We report verification utility, predictability of the disentangled variable under linear and nonlinear classifiers, and group disparity metrics based on false match rates. Our results show that VLEED offers a wide range of
privacy-utility tradeoffs over existing methods and can also reduce recognition bias across demographic groups.
Read the full abstract ▾
All authors Ünsal Öztürk, Vedrana Krivokuća Hahn, Sushil Bhattacharjee, Sébastien Marcel
Computer Vision
Script-a-Video: Deep Structured Audio-visual Captions via Factorized Streams and Relational Grounding
Tencent Hunyuan Team
Advances in
Multimodal Large
Language Models (MLLMs) are transforming
video captioning from a descriptive endpoint into a semantic interface for both
video understanding and generation. However, the dominant paradigm still casts
videos as monolithic narrative paragraphs that entangle visual, auditory, and identity information. This dense coupling not only compromises representational fidelity but also limits scalability, since even local edits can trigger global rewrites. To address this structural bottleneck, we propose Multi-Stream Scene Script (MTSS), a novel paradigm that replaces monolithic text with factorized and explicitly grounded scene descriptions. MTSS is built on two core principles: Stream Factorization, which decouples a
video into complementary streams (Reference, Shot, Event, and Global), and Relational Grounding, which reconnects these isolated streams through explicit identity and temporal links to maintain holistic
video consistency. Extensive experiments demonstrate that MTSS consistently enhances
video understanding across various models, achieving an average reduction of 25% in the total error rate on
Video-SALMONN-2 and an average performance gain of 67% on the Daily-Omni
reasoning benchmark. It also narrows the performance gap between smaller and larger MLLMs, indicating a substantially more learnable caption interface. Finally, even without architectural adaptation, replacing monolithic
prompts with MTSS in multi-shot
video generation yields substantial human-rated improvements: a 45% boost in cross-shot identity consistency, a 56% boost in audio-visual
alignment, and a 71% boost in temporal controllability.
Read the full abstract ▾
Computer Vision
Decoupled Similarity for Task-Aware Token Pruning in Large Vision-Language Models
Kexin Ma, Jing Xiao, Chaofeng Chen et al. (+4 more)
Token pruning has emerged as an effective approach to reduce the substantial computational overhead of Large
Vision-
Language Models (LVLMs) by discarding less informative visual
tokens while preserving performance. However, existing methods typically rely on individual
attention sources from different LVLM components, resulting in incomplete and suboptimal
pruning decisions due to biased
attention distributions. To address this problem, we propose DeSAP, a novel Decoupled Similarity-Aware
Pruning method for precise, task-aware
token pruning within the visual encoder. Specifically, DeSAP introduces a decoupled similarity to capture fine-grained cross-modal relevance between visual features and text
tokens, providing explicit task-related guidance for
pruning. By integrating decoupled similarity with visual saliency signals derived from visual
attention, DeSAP performs
token pruning under the guidance of both task-related and visual cues, enabling robust
pruning even under aggressive
pruning ratios. Extensive experiments across diverse
benchmarks and architectures show that DeSAP consistently outperforms SOTA methods in both accuracy and efficiency. On LLaVA-1.5-7B, DeSAP achieves a 10 times FLOPs reduction and a 2.3 times prefill speedup by retaining only 11.1% of visual
tokens, while maintaining 98.1% of the original performance.
Read the full abstract ▾
All authors Kexin Ma, Jing Xiao, Chaofeng Chen, Geyong Min, Guibo Zhu, Jinqiao Wang, Liang Liao
Computer Vision
Bridging the RGB-IR Gap: Consensus and Discrepancy Modeling for Text-Guided Multispectral Detection
Jiaqi Wu, Zhen Wang, Enhao Huang et al. (+5 more)
Text-guided multispectral object detection uses text semantics to guide semantic-aware cross-modal interaction between RGB and IR for more robust perception. However, notable limitations remain: (1) existing methods often use text only as an auxiliary semantic enhancement signal, without exploiting its guiding role to bridge the inherent granularity asymmetry between RGB and IR; and (2) conventional data-driven
attention-based fusion tends to emphasize stable consensus while overlooking potentially valuable cross-modal discrepancies. To address these issues, we propose a semantic bridge fusion framework with bi-support modeling for multispectral object detection. Specifically, text is used as a shared semantic bridge to align RGB and IR responses under a unified category condition, while the recalibrated thermal semantic prior is projected onto the RGB branch for semantic-level mapping fusion. We further formulate RGB-IR interaction evidence into the regular consensus support and the complementary discrepancy support that contains potentially discriminative cues, and introduce them into fusion via dynamic recalibration as a structured inductive bias. In addition, we design a bidirectional semantic
alignment module for closed-loop
vision-text guidance enhancement. Extensive experiments demonstrate the effectiveness of the proposed fusion framework and its superior detection performance on multispectral
benchmarks. Code is available at https://github.com/zhenwang5372/Bridging-RGB-IR-Gap.
Read the full abstract ▾
All authors Jiaqi Wu, Zhen Wang, Enhao Huang, Kangqing Shen, Yulin Wang, Yang Yue, Yifan Pu, Gao Huang
Computer Vision
Seg2Change: Adapting Open-Vocabulary Semantic Segmentation Model for Remote Sensing Change Detection
You Su, Yonghong Song, Jingqi Chen et al. (+1 more)
Change detection is a fundamental task in remote sensing, aiming to quantify the impacts of human activities and ecological dynamics on land-cover changes. Existing change detection methods are limited to predefined classes in training
datasets, which constrains their scalability in real-world scenarios. In recent years, numerous advanced open-vocabulary semantic
segmentation models have emerged for remote sensing imagery. However, there is still a lack of an effective framework for directly applying these models to open-vocabulary change detection (OVCD), a novel task that integrates
vision and
language to detect changes across arbitrary categories. To address these challenges, we first construct a category-agnostic change detection
dataset, termed CA-CDD. Further, we design a category-agnostic change head to detect the transitions of arbitrary categories and index them to specific classes. Based on them, we propose Seg2Change, an adapter designed to adapt open-vocabulary semantic
segmentation models to change detection task. Without bells and whistles, this simple yet effective framework achieves state-of-the-art OVCD performance (+9.52 IoU on WHU-CD and +5.50 mIoU on SECOND). Our code is released at https://github.com/yogurts-sy/Seg2Change.
Read the full abstract ▾
All authors You Su, Yonghong Song, Jingqi Chen, Zehan Wen
Computer Vision
NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: AI Flash Portrait (Track 3)
Ya-nan Guan, Shaonan Zhang, Hang Guo et al. (+55 more)
In this paper, we present a comprehensive overview of the NTIRE 2026 3rd Restore Any Image Model (RAIM) challenge, with a specific focus on Track 3: AI Flash Portrait. Despite significant advancements in deep learning for image restoration, existing models still encounter substantial challenges in real-world low-light portrait scenarios. Specifically, they struggle to achieve an optimal balance among noise suppression, detail preservation, and faithful illumination and color reproduction. To bridge this gap, this challenge aims to establish a novel
benchmark for real-world low-light portrait restoration. We comprehensively evaluate the proposed
algorithms utilizing a hybrid evaluation system that integrates objective quantitative metrics with rigorous subjective assessment protocols. For this competition, we provide a
dataset containing 800 groups of real-captured low-light portrait data. Each group consists of a 1K-resolution low-light input image, a 1K ground truth (GT), and a 1K person mask. This challenge has garnered widespread
attention from both academia and industry, attracting over 100 participating teams and receiving more than 3,000 valid submissions. This report details the motivation behind the challenge, the
dataset construction process, the evaluation metrics, and the various phases of the competition. The released
dataset and baseline code for this track are publicly available from the same \href{https://github.com/zsn1434/AI_Flash-BaseLine/tree/main}{GitHub repository}, and the official challenge webpage is hosted on \href{https://www.codabench.org/competitions/12885/}{CodaBench}.
Read the full abstract ▾
All authors Ya-nan Guan, Shaonan Zhang, Hang Guo, Yawen Wang, Xinying Fan, Tianqu Zhuang, Jie Liang, Hui Zeng, Guanyi Qin, Lishen Qu, Tao Dai, Shu-Tao Xia, Lei Zhang, Radu Timofte, Bin Chen, Yuanbo Zhou, Hongwei Wang, Qinquan Gao, Tong Tong, Yanxin Qian, Lizhao You, Jingru Cong, Lei Xiong, Shuyuan Zhu, Zhi-Qiang Zhong, Kan Lv, Yang Yang, Kailing Tang, Minjian Zhang, Zhipei Lei, Zhe Xu, Liwen Zhang, Dingyong Gou, Yanlin Wu, Cong Li, Xiaohui Cui, Jiajia Liu, Guoyi Xu, Yaoxin Jiang, Yaokun Shi, Jiachen Tu, Liqing Wang, Shihang Li, Bo Zhang, Biao Wang, Haiming Xu, Xiang Long, Xurui Liao, Yanqiao Zhai, Haozhe Li, Shijun Shi, Jiangning Zhang, Yong Liu, Kai Hu, Jing Xu, Xianfang Zeng, Yuyang Liu, Minchen Wei
Computer Vision
Sign Language Recognition in the Age of LLMs
Vaclav Javorek, Jakub Honzik, Ivan Gruber et al. (+2 more)
Recent
Vision Language Models (VLMs) have demonstrated strong performance across a wide range of
multimodal reasoning tasks. This raises the question of whether such general-purpose models can also address specialized visual recognition problems such as isolated sign
language recognition (ISLR) without task-specific training. In this work, we investigate the capability of modern VLMs to perform ISLR in a zero-shot setting. We evaluate several open-source and proprietary VLMs on the WLASL300
benchmark. Our experiments show that, under
prompt-only zero-shot inference, current open-source VLMs remain far behind classic supervised ISLR classifiers by a wide margin. However, follow-up experiments reveal that these models capture partial visual-semantic
alignment between signs and text descriptions. Larger proprietary models achieve substantially higher accuracy, highlighting the importance of model scale and training data diversity. All our code is publicly available on GitHub.
Read the full abstract ▾
All authors Vaclav Javorek, Jakub Honzik, Ivan Gruber, Tomas Zelezny, Marek Hruz
Computer Vision
H-SPAM: Hierarchical Superpixel Anything Model
Julien Walther, Rémi Giraud and Michaël Clément
Superpixels offer a compact image representation by grouping pixels into coherent regions. Recent methods have reached a plateau in terms of
segmentation accuracy by generating noisy superpixel shapes. Moreover, most existing approaches produce a single fixed-scale partition that limits their use in
vision pipelines that would benefit multi-scale representations. In this work, we introduce H-SPAM (Hierarchical Superpixel Anything Model), a unified framework for generating accurate, regular, and perfectly nested hierarchical superpixels. Starting from a fine partition, guided by deep features and external object priors, H-SPAM constructs the hierarchy through a two-phase region merging process that first preserves object consistency and then allows controlled inter-object grouping. The hierarchy can also be modulated using visual
attention maps or user input to preserve important regions longer in the hierarchy. Experiments on standard
benchmarks show that H-SPAM strongly outperforms existing hierarchical methods in both accuracy and regularity, while performing on par with most recent state-of-the-art non-hierarchical methods. Code and pretrained models are available: https://github.com/waldo-j/hspam.
Read the full abstract ▾
Computer Vision
3DTV: A Feedforward Interpolation Network for Real-Time View Synthesis
Stefan Schulz, Fernando Edelstein, Hannah Dröge et al. (+2 more)
Real-time free-viewpoint rendering requires balancing multi-camera redundancy with the latency constraints of interactive applications. We address this challenge by combining lightweight geometry with learning and propose 3DTV, a feedforward network for real-time sparse-view interpolation. A Delaunay-based triplet selection ensures angular coverage for each target view. Building on this, we introduce a pose-aware depth module that estimates a coarse-to-fine depth pyramid, enabling efficient feature reprojection and occlusion-aware blending. Unlike methods that require scene-specific optimization, 3DTV runs feedforward without retraining, making it practical for AR/VR, telepresence, and interactive applications. Our experiments on challenging multi-view
video datasets demonstrate that 3DTV consistently achieves a strong balance of quality and efficiency, outperforming recent real-time novel-view baselines. Crucially, 3DTV avoids explicit proxies, enabling robust rendering across diverse scenes. This makes it a practical solution for low-latency multi-view streaming and interactive rendering. Project Page: https://stefanmschulz.github.io/3DTV_webpage/
Read the full abstract ▾
All authors Stefan Schulz, Fernando Edelstein, Hannah Dröge, Matthias B. Hullin, Markus Plack
Computer Vision
LoViF 2026 Challenge on Human-oriented Semantic Image Quality Assessment: Methods and Results
Xin Li, Daoli Xu, Wei Luo et al. (+27 more)
This paper reviews the LoViF 2026 Challenge on Human-oriented Semantic Image Quality Assessment. This challenge aims to raise a new direction, i.e., how to evaluate the loss of semantic information from the human perspective, intending to promote the development of some new directions, like semantic coding, processing, and semantic-oriented optimization, etc. Unlike existing
datasets of quality assessment, we form a
dataset of human-oriented semantic quality assessment, termed the SeIQA
dataset. This
dataset is divided into three parts for this competition: (i) training data: 510 pairs of degraded images and their corresponding ground truth references; (ii) validation data: 80 pairs of degraded images and their corresponding ground-truth references; (iii) testing data: 160 pairs of degraded images and their corresponding ground-truth references. The primary objective of this challenge is to establish a new and powerful
benchmark for human-oriented semantic image quality assessment. There are a total of 58 teams registered in this competition, and 6 teams submitted valid solutions and fact sheets for the final testing phase. These submissions achieved state-of-the-art (SOTA) performance on the SeIQA
dataset.
Read the full abstract ▾
All authors Xin Li, Daoli Xu, Wei Luo, Guoqiang Xiang, Haoran Li, Chengyu Zhuang, Zhibo Chen, Jian Guan, Weiping Li, Weixia Zhang, Wei Sun, Zhihua Wang, Dandan Zhu, Chengguang Zhu, Ayush Gupta, Rachit Agarwal, Shouvik Das, Biplab Ch Das, Amartya Ghosh, Kanglong Fan, Wen Wen, Shuyan Zhai, Tianwu Zhi, Aoxiang Zhang, Jianzhao Liu, Yabin Zhang, Jiajun Wang, Yipeng Sun, Kaiwei Lian, Banghao Yin
Computer Vision
MedP-CLIP: Medical CLIP with Region-Aware Prompt Integration
Jiahui Peng, He Yao, Jingwen Li et al. (+9 more)
Contrastive
Language-Image Pre-training (CLIP) has demonstrated outstanding performance in global image understanding and zero-shot transfer through large-scale text-image
alignment. However, the core of medical image analysis often lies in the fine-grained understanding of specific anatomical structures or lesion regions. Therefore, precisely comprehending region-of-interest (RoI) information provided by medical professionals or perception models becomes crucial. To address this need, we propose MedP-CLIP, a region-aware medical
vision-
language model (VLM). MedP-CLIP innovatively integrates medical prior knowledge and designs a feature-level region
prompt integration mechanism, enabling it to flexibly respond to various
prompt forms (e.g., points, bounding boxes, masks) while maintaining global contextual awareness when focusing on local regions. We pre-train the model on a meticulously constructed large-scale
dataset (containing over 6.4 million medical images and 97.3 million region-level annotations), equipping it with cross-disease and cross-modality fine-grained spatial semantic understanding capabilities. Experiments demonstrate that MedP-CLIP significantly outperforms baseline methods in various medical tasks, including zero-shot recognition, interactive
segmentation, and empowering
multimodal large
language models. This model provides a scalable, plug-and-play visual backbone for medical AI, combining holistic image understanding with precise regional analysis.
Read the full abstract ▾
All authors Jiahui Peng, He Yao, Jingwen Li, Yanzhou Su, Sibo Ju, Yujie Lu, Jin Ye, Hongchun Lu, Xue Li, Lincheng Jiang, Min Zhu, Junlong Cheng
Computer Vision
Towards Adaptive Open-Set Object Detection via Category-Level Collaboration Knowledge Mining
Yuqi Ji, Junjie Ke, Lihuo He et al. (+2 more)
Existing object detectors often struggle to generalize across domains while adapting to emerging novel categories. Adaptive open-set object detection (AOOD) addresses this challenge by training on base categories in the source domain and adapting to both base and novel categories in the target domain without target annotations. However, current AOOD methods remain limited by weak cross-domain representations, ambiguity among novel categories, and source-domain feature bias. To address these issues, we propose a category-level collaboration knowledge mining strategy that exploits both inter-class and intra-class relationships across domains. Specifically, we construct a clustering-based
memory bank to encode class prototypes, auxiliary features, and intra-class disparity information, and iteratively update it via unsupervised clustering to enhance category-level knowledge representation. We further design a base-to-novel selection metric to discover source-domain features related to novel categories and use them to initialize novel-category classifiers. In addition, an adaptive feature assignment strategy transfers the learned category-level knowledge to the target domain and asynchronously updates the
memory bank to alleviate source-domain bias. Extensive experiments on multiple
benchmarks show that our method consistently surpasses state-of-the-art AOOD methods by 1.1-5.5 mAP.
Read the full abstract ▾
All authors Yuqi Ji, Junjie Ke, Lihuo He, Lizhi Wang, Xinbo Gao
Computer Vision
Do Thought Streams Matter? Evaluating Reasoning in Gemini Vision-Language Models for Video Scene Understanding
Shivam Sharma, Sankalp Nagaonkar, Ashish Choithani et al. (+1 more)
We
benchmark how internal
reasoning traces, which we call thought streams, affect
video scene understanding in
vision-
language models. Using four configurations of Google's Gemini 2.5 Flash and Flash Lite across scenes extracted from 100 hours of
video, we ask three questions: does more thinking lead to better outputs, where do the gains stop, and what do these models actually think about? We introduce three evaluation metrics. Contentfulness measures how much of the thought stream is useful scene content versus meta-commentary. Thought-Final Coverage measures how faithfully the thought stream translates into the final output. Dominant Entity Analysis identifies which subjects, actions, and settings the model focuses on. GPT-5 serves as an independent judge. We find that quality gains from additional thinking plateau quickly, with most improvement occurring in the first few hundred
tokens. Flash Lite offers the best balance between quality and
token usage. Tight
reasoning budgets cause the model to add content in the final output that it never reasoned about, a form of compression-step
hallucination. Despite being different model tiers, Flash and Flash Lite produce similar thought streams, though they differ in style: Flash discusses its
reasoning process, while Lite focuses on describing the scene.
Read the full abstract ▾
All authors Shivam Sharma, Sankalp Nagaonkar, Ashish Choithani, Ashutosh Trivedi
Computer Vision
Precision Synthesis of Multi-Tracer PET via VLM-Modulated Rectified Flow for Stratifying Mild Cognitive Impairment
Shuijin Lin, Shaozhen Yan, Tuo Liu et al. (+5 more)
The biological definition of Alzheimer's disease (AD) relies on multi-modal neuroimaging, yet the clinical utility of positron emission
tomography (PET) is limited by cost and radiation exposure, hindering early screening at preclinical or prodromal stages. While generative models offer a promising alternative by synthesizing PET from
magnetic resonance imaging (MRI), achieving subject-specific precision remains a primary challenge. Here, we introduce DIReCT$++$, a Domain-Informed ReCTified flow model for synthesizing multi-tracer PET from MRI combined with fundamental clinical information. Our approach integrates a 3D rectified flow architecture to capture complex cross-modal and cross-tracer relationships with a domain-adapted
vision-
language model (BiomedCLIP) that provides text-guided, personalized generation using clinical scores and imaging knowledge. Extensive evaluations on multi-center
datasets demonstrate that DIReCT$++$ not only produces synthetic PET images ($^{18}$F-AV-45 and $^{18}$F-FDG) of superior fidelity and generalizability but also accurately recapitulates disease-specific patterns. Crucially, combining these synthesized PET images with MRI enables precise personalized stratification of mild cognitive impairment (MCI), advancing a scalable, data-efficient tool for the early diagnosis and prognostic prediction of AD. The source code will be released on https://github.com/ladderlab-xjtu/DIReCT-PLUS.
Read the full abstract ▾
All authors Shuijin Lin, Shaozhen Yan, Tuo Liu, Haifeng Wang, Chunfeng Lian, Jie Lu, Jianhua Ma, Lin Xie
Computer Vision · Cross-listed from cs.GR
NeuVolEx: Implicit Neural Features for Volume Exploration
Haill An, Suhyeon Kim, Donghyuk Choo et al. (+1 more)
Direct volume rendering (DVR) aims to help users identify and examine regions of interest (ROIs) within volumetric data, and feature representations that support effective ROI classification and clustering play a fundamental role in volume exploration. Existing approaches typically rely on either explicit local feature representations or implicit convolutional feature representations learned from raw volumes. However, explicit local feature representations are limited in capturing broader
geometric patterns and spatial correlations, while implicit convolutional feature representations do not necessarily ensure robust performance in practice, where user supervision is typically limited. Meanwhile, implicit neural representations (INRs) have recently shown strong promise in DVR for volume compression, owing to their ability to compactly parameterize continuous volumetric fields. In this work, we propose NeuVolEx, a neural volume exploration approach that extends the role of INRs beyond volume compression. Unlike prior compression methods that focus on INR outputs, NeuVolEx leverages feature representations learned during INR training as a robust basis for volume exploration. To better adapt these feature representations to exploration tasks, we augment a base INR with a structural encoder and a multi-task learning scheme that improve spatial coherence for ROI characterization. We validate NeuVolEx on two fundamental volume exploration tasks: image-based transfer function (TF) design and viewpoint recommendation. NeuVolEx enables accurate ROI classification under sparse user supervision for image-based TF design and supports unsupervised clustering to identify compact complementary viewpoints that reveal different ROI clusters. Experiments on diverse volume
datasets with varying modalities and ROI complexities demonstrate NeuVolEx improves both effectiveness and usability over prior methods
Read the full abstract ▾
All authors Haill An, Suhyeon Kim, Donghyuk Choo, Younhyun Jung