Computer Vision
EyeVQA: Benchmarking Ophthalmic Vision-Language Models from Recognition to Spatial Grounding
Gujie Shao, Zixun Xie, Xuechun Xing et al. (+7 more)
Vision-
language models (VLMs) have shown increasing potential for medical image understanding, yet their capabilities in ophthalmic imaging remain insufficiently characterized. Existing ophthalmic
datasets are typically designed for individual diseases or specialized tasks, making it difficult to systematically evaluate whether VLMs can move beyond disease recognition toward comparative
reasoning and fine-grained spatial grounding. We introduce EyeVQA, a unified visual question answering
benchmark for comprehensive evaluation of ophthalmic VLMs. EyeVQA is constructed from 21 available ophthalmic
datasets and contains 20,000 question-answer pairs spanning six disease groups and seven question types: Single-Choice, Multi-Select, Variable-Select, True-False, Ranking, Point Location, and Bounding Box. Gold answers are deterministically derived from source-provided diagnoses, severity grades, clinical findings, segmentation masks, bounding boxes, and anatomical landmarks, enabling reproducible evaluation without relying on model-generated annotations. Notably, 44.5% of the questions require
reasoning across multiple images, extending evaluation beyond conventional single-image medical VQA. We
benchmark fourteen representative general-purpose, scientific, and medically specialized VLMs under a unified zero-shot protocol. The best-performing model only achieves an overall score of 62.8, while substantial gaps remain in spatial grounding and cross-task generalization. These results highlight the limitations of current VLMs in comprehensive ophthalmic visual understanding and establish EyeVQA as a diagnostic
benchmark for developing more reliable and spatially grounded ophthalmic
multimodal models. The project page is available at https://github.com/PKUTHM/EyeVQA.
Read the full abstract ▾
All authors Gujie Shao, Zixun Xie, Xuechun Xing, Ruixiang Wang, Ziyun Lan, Yanlin Qi, Gangyi Zhang, Yuxin Yang, Dawei Li, Haiming Tang
Computer Vision
OpenMASC: An Open-Source Pipeline for Cross-Trajectory Metal-Aware Sampling and Correction in Accelerated MRI
Zhengyi Lu, Ming Lu, Chongyu Qu et al. (+11 more)
Metal implants corrupt MRI measurements throughout $k$-space, yet existing accelerated MRI methods assume clean data and most metal artifact reduction approaches assume fully sampled acquisitions. No public
dataset provides paired $k$-space and images with and without metal for the same anatomy, and no framework jointly addresses artifact-aware acquisition and reconstruction across sampling trajectories. We present OpenMASC, an open-source pipeline covering the full workflow from data generation to deployment. A physics-based data generation module converts public CT volumes into paired clean and metal-corrupted MRI data in both Cartesian and radial formats. MA-VarNet, an unrolled reconstruction network with a per-cascade DC Rectifier, corrects artifacts that data-consistency steps reintroduce from corrupted measurements. A reinforcement learning
agent actively selects $k$-space readouts and co-trains with the reconstruction network through a decoupled three-stage procedure. The framework is trajectory-agnostic except for the data-consistency operator, supporting both Cartesian and radial acquisition without architectural changes. Experiments on two
datasets at $4\times$ and $8\times$ acceleration demonstrate consistent improvements over conventional and learned baselines on both trajectories.
Read the full abstract ▾
All authors Zhengyi Lu, Ming Lu, Chongyu Qu, Junchao Zhu, Junlin Guo, Marilyn Lionts, Yanfan Zhu, Yuechen Yang, Tianyuan Yao, Jayasai Rajagopal, Bennett Allan Landman, Xiao Wang, Xinqiang Yan, Yuankai Huo
Computer Vision
SGA-Flow-GRPO: Spatial Gradient-Guided Credit Assignment for Flow-GRPO
Yunkai Yang, Yudong Zhang, Xinying Chen et al. (+5 more)
Reinforcement Learning (RL) has proven effective in aligning flow-based generative models with human preferences. Recently, Flow-GRPO has emerged as an efficient critic-free paradigm by calculating advantages over sampled candidate trajectories. However, standard Flow-GRPO applies a uniform scalar advantage across both temporal denoising steps and spatial latent dimensions, without explicitly accounting for the spatial structure of generated images, which may lead to sub-optimal policy updates. To address this, we propose a novel gradient-guided spatial credit assignment framework tailored for
Diffusion Transformers (DiTs). We first reformulate the transition-level log-likelihood in Flow-GRPO into a
token-wise representation natively aligned with DiT patch architectures, constructing spatially fine-grained importance sampling ratios. To allocate localized credit without rigid, boundary-sensitive segmentation heuristics, we introduce a continuous spatial credit map derived from reward gradients. Crucially, we employ an outlier-robust normalization scheme based on Median Absolute Deviation (MAD) coupled with temperature scaling, effectively eliminating gradient noise while highlighting functional
prompt-aligned regions. Extensive evaluations on GenEval show that our approach delivers SOTA
alignment quality, achieving a convergence rate comparable to top-tier methods like DiffusionNFT while substantially improving upon Flow-GRPO-based methods in
alignment performance.
Read the full abstract ▾
All authors Yunkai Yang, Yudong Zhang, Xinying Chen, Bin Luo, Jienan Lyu, Kunquan Zhang, Weitao Wan, Runmin Dong
Computer Vision
Progressive-View On-Policy Distillation for Regional-to-Global Transfer in Multimodal LLMs
Shanfeng Huang, Zhou Fang, Song Xiao et al. (+1 more)
Regional-to-global
distillation uses crop-conditioned guidance to improve full-image understanding. The challenge is to effectively transfer the teacher's crop-based advantage to the student's full-image inference. We propose progressive-view on-policy
distillation (PVD), which shifts the student's view distribution from the crop toward the full image through an intermediate aspect-preserving padded crop. The padded crop preserves regional content while matching the full image's visual-
token grid. Across stages, the view mixture assigns increasing probability to the full image. A lightweight regional-advantage weighting reallocates
token-level supervision using the crop-conditioned teacher-student log-probability gap. Evaluated under each sampled input, it applies mild reweighting when the gap is small and emphasizes higher-gap
tokens when the gap widens. A Jensen-Shannon metric decomposition interprets this schedule as a shift from matched-input imitation toward the deployment objective. Across
benchmarks spanning perception, visual mathematics and general
multimodal question answering, PVD-full reaches an average accuracy of 77.51 over three seeds, improving on the reward-free
distillation baseline by 2.01 points and on its reward-matched variant by 1.00 point. In the reward-free setting, PVD-distill still gains 1.16 points.
Read the full abstract ▾
All authors Shanfeng Huang, Zhou Fang, Song Xiao, Hai Du
Computer Vision
FoundDSR: A Generalizable Foundation Model with Guided 2D Gaussian Splatting for Depth Super-Resolution
Zhengxue Wang, Zhiqiang Yan, Yuan Wu et al. (+3 more)
We introduce FoundDSR, a generalizable foundation model for robust depth reconstruction across unseen data distributions using RGB-D pairs. FoundDSR begins with a guided 2D Gaussian Splatting strategy to model depth representations with Gaussian primitives. This strategy employs high-resolution RGB as
prompts to optimize the Gaussian parameters, thereby encouraging each Gaussian primitive to anisotropically deform along high-frequency structural directions. The resulting Gaussian-upsampled representations are then mapped to high-resolution depth through an effective depth reconstruction branch. Furthermore, to mitigate training instability and bias toward dominant sources caused by distribution gaps in large-scale heterogeneous data, we introduce heterogeneous
federated learning that allocates each data source to an independent client for local optimization and global aggregation. This design effectively endows FoundDSR with stable scalability to diverse and large-scale training data. Extensive zero-shot evaluations on synthetic, real-world, arbitrary-scale, and noisy conditions demonstrate that FoundDSR consistently outperforms existing state-of-the-art approaches, confirming its strong robustness and generalization to unknown scenes.
Read the full abstract ▾
All authors Zhengxue Wang, Zhiqiang Yan, Yuan Wu, Guangwei Gao, Xiang Li, Jian Yang
Computer Vision
One Perception, All Maneuvers: Directional Traffic Signal Understanding for Maneuver-Level Signal Intent Prediction
Ang Zou, Runzhe Zheng, Zhigang li et al. (+5 more)
Traffic lights are a key regulatory signal for autonomous driving at urban intersections, yet existing traffic signal perception is still predominantly formulated as instance-level detection or color recognition. Such formulations identify where traffic lights are and what colors they display, but leave a critical semantic gap before downstream planning: which ego maneuver is controlled by each visible signal and what dynamic permission the signal expresses for that maneuver. In this paper, we formulate Directional Traffic Signal Understanding, a decision-oriented task that predicts structured signal states for straight, left-turn, right-turn, and U-turn maneuvers from a front-view image. Each state contains the associated signal color and signal-implied passability. Based on OpenLane-V2, we provide a direction-level
benchmark with maneuver-level supervision and metrics for color recognition, passability, full-frame consistency, and safety-critical errors. A direction-aware baseline combines global context, localized traffic-light evidence, and maneuver-specific representations. Experiments show that direction-level modeling improves passability prediction over image-level classifiers and detection-oriented pipelines, particularly at complex multi-signal intersections. The resulting representation provides a direct and interpretable traffic-signal interface for downstream planning together with
topology, route, and surrounding-
agent information.
Read the full abstract ▾
All authors Ang Zou, Runzhe Zheng, Zhigang li, Zhen Yang, Han Xia, Xuewei Li, Zequn Qin, Xi Li
Computer Vision
HeroFrame-Bench: Reference-Anchored Evaluation via Rubric–Ranking Co-Evolution for Movie Hero Frame Selection
Weitai Kang, Hanieh Deilamsalehy, Yumo Xu et al. (+3 more)
Hero frames are in-film stills used as source imagery for theatrical posters, streaming cover art, film database listings, and other promotional placements. As the first visual entry point, they shape audiences' initial impressions of the movie and their subsequent willingness to watch it. Selecting these frames, a task we term hero frame selection, requires balancing content relevance with aesthetic appeal. A related task is keyframe selection, yet its
benchmarks prioritize relevance over aesthetics, using either finite annotations that exclude valid alternatives or VideoQA that entangles selection quality with downstream model capability. We therefore introduce HeroFrame-Bench, built through a scalable VLM-as-a-Judge framework. We construct
multimodal contexts from diverse metadata to ground a VLM judge that scores selected frames using our Reference-anchored Percentile. The percentile is obtained by inserting each frame into reusable, pre-ranked reference chains, enabling direct, extensible, and reliable evaluation. To reduce ambiguity and improve consistency in these subjective judgements, we further propose Rubric-Ranking Co-
Evolution, which generates movie-specific rubrics to condition the VLM judge and refines rubrics jointly with the resulting rankings. Within this process, we introduce several verifiable signals, most notably the Inverted Rubric Attack, to select robust rubrics. Finally, HeroFrame-Bench is instantiated over 204 movies with 2,031 reference chains and 1,970 learned rubrics. We build an annotation interface for human-
alignment studies which show that our construction design improves VLM agreement with human from 77.56% to 83.78%. Evaluation on multiple methods show that hero frame selection remains challenging.
Read the full abstract ▾
All authors Weitai Kang, Hanieh Deilamsalehy, Yumo Xu, Dewang Sultania, Serdar Cellat, Yan Yan
Computer Vision
FSS-UBrain: Multi-region Few-Shot Brain Tumor MRI Segmentation
Truong Viet Vu, Nguyen Phuc Nguyen, Dang Thi Thu Hang et al. (+4 more)
Accurate delineation of whole tumor (WT), tumor core (TC), and enhancing tumor (ET) from
multimodal magnetic resonance imaging remains challenging under limited annotation, cross-cohort variation, and severe target sparsity. We propose FSS-UBrain, a region-wise one-shot segmentation framework that uses a labeled positive support slice to condition binary query segmentation separately for WT, TC, and ET. Support-derived foreground and background descriptors guide query-feature adaptation, bottleneck interaction, decoder-side reconstruction, and boundary refinement. Episodic training additionally incorporates hard-negative and fully negative queries with empty-query regularization to suppress spurious foreground activation when the selected region is absent. Although inference operates on two-dimensional support–query slice pairs, checkpoint selection, threshold calibration, and final evaluation are performed after volumetric reconstruction. FSS-UBrain is evaluated on a held-out BraTS 2020 split and under target-supported cross-cohort protocols on BraTS 2023 and BraTS-Africa. Cases used as target support are excluded from the query cohorts, and no target-domain fine-tuning or test-time parameter updates are performed. On BraTS 2020, FSS-UBrain achieves volumetric Dice scores of 89.82%, 82.14%, and 77.42% for WT, TC, and ET, respectively, with corresponding 95th-percentile Hausdorff distance (HD95) values of 11.12, 9.01, and 4.46 mm. It also achieves the highest mean Dice and lowest finite-pair mean HD95 point estimates on BraTS 2023 and BraTS-Africa among the compared few-shot methods. These findings support target-conditioned few-shot segmentation while highlighting sensitivity to support selection and cohort-specific variation.
Read the full abstract ▾
All authors Truong Viet Vu, Nguyen Phuc Nguyen, Dang Thi Thu Hang, Tran Thien Thanh, Vo Nguyen Quoc Bao, Nguyen Thai Anh, Ngo Hoang Tu
Computer Vision · Cross-listed from cs.LG
Learning Through Game: Skewed Transfer of Tabular Knowledge to Strengthen Image Model
Longfei Huang, Shangdong Yang and Yang Yang
Multimodal tabular-image learning is gaining growing
attention, yet it faces challenges due to tabular data unavailable at test time. A practical solution involves transferring tabular knowledge to images during training to enhance the performance of image models at inference. However, the overlooked yet important challenges lie in the modality imbalance between images and tables, as well as their asymmetric modality relationship in cross-modal transfer, which limits the auxiliary role of tabular data. To address these issues, we propose Skewed Knowledge Transfer (SKT), which asymmetrically transfers tabular knowledge to improve the image model by adaptive integration of modality gradients in a shared parameter space. Specifically, we first introduce a
multimodal shared head, which allows the model to benefit from cross-modal structure without adding additional parameters. We then design a two-step Nash Bargaining strategy to effectively leverage tabular gradients. In the first step, SKT seeks a point of modality balance and uses preference awareness in the second step to steer combined gradients toward image-beneficial directions. Furthermore, we theoretically analyze the Pareto improvement and convergence of SKT. To this end, tabular knowledge is explicitly transferred to enhance image models. Empirical experiments on widely used tabular-image
datasets reveal that SKT consistently improves image unimodal performance by using tabular data as auxiliary information.
Read the full abstract ▾
Computer Vision
RoboSTAR: Next-Scale Autoregressive Sign Language Translation for Humanoid Robots
Yujia Zeng, Chensheng Peng, Yuxin Chen et al. (+3 more)
Sign-
language interpretation in public communication relies on qualified professional interpreters and can be difficult to scale, motivating robotic signing as a complementary accessibility interface. We present RoBoSTAR, a text-conditioned sign
language production (SLP) framework for generating human-centric sign motion that can be retargeted for robotic execution, with
speech supported optionally through an external ASR front end. Conventional autoregressive approaches flatten motion into a single full-resolution
token sequence, forcing long-range and local dependencies to be modeled at a uniform temporal granularity. RoBoSTAR instead combines part-wise Finite Scalar Quantization with next-scale autoregression, generating motion over progressively finer temporal resolutions while predicting synchronized body and hand
tokens in parallel within each step. This coarse-to-fine formulation provides compact long-range context before progressively refining motion details, while self-conditioning and context corruption improve robustness to cross-scale prediction errors. The generated motion is subsequently retargeted for physical humanoid execution. Extensive qualitative and quantitative evaluations are conducted to demonstrate the effectiveness of RoBoSTAR.
Read the full abstract ▾
All authors Yujia Zeng, Chensheng Peng, Yuxin Chen, Alex Shao, Nathan Jew, Masayoshi Tomizuka
Computer Vision
Two-Stage Multi-View Gait Recognition with a Re-Embedding Network
Long Hoang Le and Trung Thanh Ngo
Gait recognition always remains challenging due to severe overfitting and the rigid view constraints common in single-stage approaches. We propose a two-stage framework, termed Translate-First-Then-Reason (TFTR), to address these issues. In the first stage, a shallow Siamese convolutional network with triplet loss maps Gait Energy Images (GEIs) into a 128-dimensional view-specific
embedding space. In the second stage, these per-view
embeddings are treated as
tokens and processed by a 12-layer
Transformer encoder, which re-projects them into a new space with improved cosine separability. This design enables flexible fusion of an arbitrary number of views at inference, overcoming the fixed-input limitations of prior methods. Trained on the OU-MVLP
dataset (6,000 subjects) and evaluated on unseen CASIA-B across normal, bag-carrying, and coat-wearing conditions, our pipeline achieves 96.91% single-view and 99.49% three-view accuracy on OU-MVLP, and attains 100% accuracy on CASIA-B with three views.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CR
Residual Transferability in Neural Image Watermarking
Ziping Dong, Qi Li and Xinchao Wang
Neural image watermarks can be forged by extracting watermark-bearing residuals from released images and transferring them to unrelated content. While prior work has demonstrated this vulnerability, what makes these residuals transferable remains poorly understood. We formalize this vulnerability with residual transferability (RT), a metric that quantifies how well watermark evidence remains decodable after transfer across unrelated images. Through comparative analyses and controlled interventions, we find that common training-side variations do not account for the large RT differences across watermarking systems; instead, architectural design plays a central role. By contrasting high- and low-RT systems and validating their architectural differences through controlled interventions, we identify two mechanisms that strengthen the dependence of watermark evidence on the cover image, thereby suppressing the residual transferability. These findings provide concrete design guidance for developing more forgery-resistant watermarking architectures. Complementarily, for existing watermarking systems where architectural redesign is impractical, we introduce CoverLock, a plug-and-play strategy for existing watermarking systems that strengthens such image dependence without architectural redesign. Across representative watermarking systems exhibiting high residual transferability, CoverLock achieves a more favorable security–robustness trade-off than both traditional handcrafted defenses and learned classifier-based defenses.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Biprodip Pal, Kaushik Roy, Yanming Zhu et al. (+3 more)
Federated learning offers a natural way for multiple
robots to jointly improve manipulation policies without requiring centralized access to training demonstrations. However, non-IID task and environment distributions can induce representation drift and mutually incompatible
robot-policy updates, making naive parameter aggregation destructive. We present FedDRMan, a
federated subspace-guided
distillation framework for heterogeneous
robot manipulation. At each communication round, the server model provides a frozen teacher for local behavior cloning, while low-rank
multimodal subspace and action-distribution
distillation preserve globally useful representation geometry and policy behavior. To address heterogeneous aggregation, FedDRMan groups clients by update compatibility and maintains a persistent model for each cluster. The server then spectrally rebalances each compatible aggregate to mitigate attenuation of weaker task-relevant
robot-policy update directions. Extensive experiments on LIBERO across diverse non-IID settings, heterogeneity levels, client participation variation, together with ablations and aggregation analyses, show that FedDRMan substantially improves knowledge transfer and consistently outperforms strong
federated baselines achieving a peak mean success rate of 80.7%, 11.6 percentage points above the strongest evaluated
federated baseline.
Read the full abstract ▾
All authors Biprodip Pal, Kaushik Roy, Yanming Zhu, Brendan Tidd, Alan Wee-Chung Liew, Peyman Moghadam
Computer Vision
Skeletons in Flow: Graph Structured Flow Matching for Human Motion Prediction
Yixuan Wang, Brandon C. Fallin and Warren E. Dixon
Human motion prediction requires diverse future trajectories that remain consistent with observed motion and the articulated physical structure of the body. Skeletal constraints restrict individual poses, while coordinated motion depends on spatial interactions (between connected joints) and temporal interactions (between time instants). To facilitate human motion prediction in light of these constraints and interactions, we introduce Graph Structured Flow Matching (GSFM), which transports the complete future skeletal trajectory through a single conditional velocity field. The trajectory produces a spatiotemporal skeleton graph, and spatial and temporal
attention couple its
evolution according to skeletal relations and physical time offsets. Bone directions lie on unit spheres relative to a root joint, and tangent
evolution preserves input bone lengths throughout generation. We train a learned velocity field through conditional flow matching along geodesic paths connecting random trajectories centered on the last-observed pose to recorded future trajectories. Experiments on the Archive of Motion capture As Surface Shapes (AMASS)
dataset evaluate prediction accuracy, diversity calibration, and motion statistics. We demonstrate the contributions of spatial and temporal message passing in the developed architecture through an ablation study. GSFM models trained on AMASS also perform competitively on the Human3.6M skeleton without parameter updates or retraining, demonstrating applicability to an unseen skeletal structure.
Read the full abstract ▾
Computer Vision
Geometry-Preserving Blind Watermarking for Raw 3D Point Clouds
Rungui Zhou, Chuanzhi Zhou, Ruihuan Wang et al. (+1 more)
Raw 3D point clouds are a core
geometric representation. Establishing their ownership is challenging because point sets are irregular, unstructured, and frequently altered by resampling and
geometric preprocessing. We present a blind watermarking framework that operates directly on xyz coordinates and supports both object-level shapes and scene-scale scans. At verification time, the embedded message is recovered from the observed point cloud alone, without access to the original point cloud, color, normals, or mesh connectivity. The method jointly learns watermark
embedding and extraction through a feed-forward octree-based architecture, enabling efficient multi-scale
geometric reasoning on large point sets. During training, a
stochastic transformation layer exposes the decoder to common
geometric perturbations, while progressive pose
alignment improves robustness to pose changes. Experiments on object-level and scene-level
benchmarks demonstrate reliable message recovery under common
geometric processing while maintaining low
geometric distortion. Qualitative comparisons further show that the learned perturbations are less visually conspicuous and less spatially structured than those of handcrafted alternatives.
Read the full abstract ▾
All authors Rungui Zhou, Chuanzhi Zhou, Ruihuan Wang, Peng-Shuai Wang
Computer Vision
Kernel-Based Steering of CLIP with Vision-Language Model Preferences
Sajjad Ghiasvand, Haniyeh Ehsani Oskouie, Sina Mansouri et al. (+3 more)
Large
vision-
language models (VLMs) can judge visual similarity, but their judgments are not directly available as compact image
embeddings for efficient comparison. We study how to transfer these preferences into CLIP while retaining its image–text capabilities. We introduce ASK, a kernel-based steering method that learns from elicited pairwise judgments without accessing teacher
embeddings or collecting new human similarity annotations. ASK constructs positive semidefinite target kernels within small image groups and combines visual kernel matching with an image–text distributional anchor. Low-rank adapters jointly update the visual and text encoders while regularizing predictions toward frozen CLIP. After adaptation,
retrieval uses CLIP image
embeddings and cosine similarity, with no VLM calls. Experiments across five image domains, four CLIP backbones, and six judges evaluate teacher agreement,
retrieval, and recognition retention. For ViT-B/16, mean
retrieval mAP on classes excluded from adaptation increases from 53.8 to 75.0, compared with 71.7 for DINOv2 targets with KL anchoring. Mean zero-shot accuracy with jointly adapted encoders increases from 61.8% to 62.4%, averaged over 12
benchmarks and the five adaptation domains. Prompting provides an additional capability: selecting which visual distinctions the student learns. Human-annotated evaluations across four
datasets support this criterion-specific control.
Read the full abstract ▾
All authors Sajjad Ghiasvand, Haniyeh Ehsani Oskouie, Sina Mansouri, Mahnoosh Alizadeh, Farzan Farnia, Ramtin Pedarsani
Computer Vision
Devol-ONE: One Autoregressive Mixture of Transformers to Unify Vision-Language-Action and Latent World Modeling
Hongyi Cai, Yi Herng Ong, Tingshiuan C. Wu et al. (+4 more)
Vision Language Action (VLA) models condition actions directly on current visual and
language context, without an explicit account of how the scene evolves under candidate actions. World Action Models (WAM) attempt to address this limitation by predicting future states, but existing designs keep prediction and policy learning architecturally separate, connecting them only through the predicted output, whether through pixel space
video generation or a latent forecasting module trained independently of the policy. We present Devol-ONE, a Mixture of
Transformers architecture that unifies
vision language understanding, latent world dynamics prediction, and action generation within a single autoregressive framework. Instead of encoding
vision language tokens once and feeding them to the action expert, Devol-ONE runs autoregressive prediction jointly across a
vision language stream and a V-JEPA pretrained dynamics stream, attending to the
vision language key-value cache at every layer to forecast future latent states under
language guidance. The action expert is in turn shaped continuously by semantic
reasoning and predicted physical dynamics rather than by a fixed representation computed in advance. Extensive experiments are conducted on LIBERO, LIBERO-PLUS, RoboTwin2.0 along with real-world evaluation on Flexiv single-arm and dual-arm setups. Ablation studies show the effectiveness of dynamic stream prediction and layer-wise unified
attention to validate our model architectural coherency.
Read the full abstract ▾
All authors Hongyi Cai, Yi Herng Ong, Tingshiuan C. Wu, Chiew Hui Lim, Hanxia Li, Kehong Guo, Sze Yuan Cheong
Computer Vision
Evaluating Single and Multi-Omics Based Explainable Artificial Intelligence (MOXAI) for Molecular Subclass Classification of Adult-Type Diffuse Gliomas
Md Zahangir Alom, Quynh T. Tran, Breuer Alexandar et al. (+1 more)
DNA methylation (DNAM) profiling has emerged as a powerful diagnostic tool for classifying
brain and solid tumors. However, existing computational models typically analyze methylation and copy number variation (CNV) data separately, failing to capture the complementary information their integration could provide. Moreover, current classification models lack mechanisms for within-class risk assessment analogous to traditional tumor grading, and no established explainability method can attribute classification decisions to specific genomic loci. In this paper, we present MOXAI (Multi-Omics Based Explainable AI), a deep learning framework that integrates DNA methylation and copy number data from methylation arrays to classify molecular subtypes of adult-type diffuse gliomas, alongside single-modality variants for comparison. Using a cohort from The Cancer Genome Atlas (TCGA), we trained ResNet50, DINOv2, and Graph
Attention Network (GAT) models on methylation data alone, copy number data alone, and combined
multimodal data. We further developed explainable AI (XAI) methods based on class activation maps (CAMs) and gradient-weighted CAM (Grad-CAM) to identify the specific CpG sites, genes, and chromosomal regions most relevant to each classification decision. The
multimodal model achieved up to 92.98% cross-validation accuracy, outperforming models trained on CNV data alone. DINOv2 showed the strongest generalization, reaching 94.25% accuracy (confidence >0.9) on independent validation sets. XAI results aligned with established molecular features of adult-type diffuse glioma subtypes, confirming the biological interpretability of the framework.
Read the full abstract ▾
All authors Md Zahangir Alom, Quynh T. Tran, Breuer Alexandar, Brent A. Orr
Computer Vision
Presence Is Not Faithfulness: Figurative Vehicle Intrusion in Text-to-Image Generation
Xiaoyu Ma, Chen Yang and Hao Chen
Text-to-image (TTI) models increasingly generate high-quality images from natural-
language prompts, yet figurative
language exposes a failure: a vehicle that should guide the depiction of a tenor may instead be rendered as a visible object. We call this failure Figurative Vehicle Intrusion: the intruding content is textually licensed, but it is assigned the wrong visual role, showing that visual presence is not always faithfulness and that presence-oriented evaluation can miss such errors. To study it systematically, we introduce Vehicle Intrusion and Semantic Tenor Assessment (VISTA), a multilingual
benchmark of figurative
prompts organized by Figurative Form and Mapping Mechanism. We further propose V-Score, a diagnostic question-answering metric that evaluates role-aware figurative faithfulness in generated images. Evaluations on recent high-performing TTI models show that vehicle intrusion persists across
languages and figurative categories. As a lightweight mitigation, we introduce VISTA-Guard, which partially reduces vehicle intrusion and suggests a practical path toward more figuratively faithful TTI generation. All resources will be released publicly.
Read the full abstract ▾
Computer Vision
Contamination, Prior, or Evidence? Decomposing and Training Evidence Use in Whole-Slide Vision-Language Models
Wenhao Zhang, Zhongliang Zhou, Shiyuan Zhang et al. (+6 more)
Pathology
vision-
language models (VLMs) are conventionally evaluated by accuracy, but accuracy alone does not measure evidence use: it may conflate
dataset contamination, prior knowledge, and image evidence. In a motivating study of lymph-node metastasis prediction, we found that most public pathology VLMs showed minimal differences when changing from feeding the models with whole-slide images, an annotated lesion, or no image at all. To better understand the specific features leveraged by these models, this paper presents two contributions aimed at disentangling these factors. First, we present CleanSlide, a TCGA-based VQA
benchmark designed to eliminate image- and question-side contamination. It contains 149K audited multiple-choice questions over 9,985 slides, with patient- and tissue-source-disjoint splits. Every question is audited for option shortcuts, stem leakage, cross-split duplication, and blind solvability. Second, we propose Pair-DPO, a preference loss over counterfactual slide pairs from the same question and source. By controlling for shared confounding factors, Pair-DPO cancels out the question-attributable signal and leaves image evidence as the source of preference. Specifically, each pair consists of two real slides with opposite, verified findings, introducing neither editing artifacts nor unverified labels for diffuse or graded features such as invasion, necrosis, and tumor grade. Experiments show that our method gains 15.29% from image evidence on the CleanSlide, compared with 2.81% for the best published model. On the external CPTAC and BCNB cohorts, our method achieves accuracies of 57.6% and 59.0%, outperforming all other evaluated models by 9.7% and 3.4%, respectively. We will release the
benchmark and code.
Read the full abstract ▾
All authors Wenhao Zhang, Zhongliang Zhou, Shiyuan Zhang, Yiqing Yang, Pinqiao Wang, Lehan Yang, Hanyin Wang, John Kang, Sheng Li
Computer Vision
Scalable In-Domain Self-Supervised Foundation Model for Dense Representation Transfer in High-Resolution Plant Imaging
Junlin Guo, Sharmin Majumder, Isaac Lyngaas et al. (+2 more)
High-resolution plant imaging enables detailed characterization of plant morphology, but dense scientific analysis remains limited by costly pixel-level annotations, large image pixel dimensions, and substantial variation in imaging conditions. This work proposes a scalable in-domain self-supervised pretrained foundation model for high-resolution, high-pixel-dimension multi-species plant imagery. A masked autoencoder with a ViT backbone is pretrained on more than 10 million multi-view plant image tiles using distributed training. Following scalable pretraining, the learned foundation-model representations are comprehensively benchmarked across fine-grained dense prediction and coarse global feature recognition, with particular emphasis on limited supervision and realistic downstream imaging conditions. This work focuses on the domain gap of existing foundation models in dense feature representation and transfer. Through extensive experiments involving limited annotations, cross-view variation, and resolution degradation, the in-domain FM achieves a Mean Dice of 0.8686 and a Pooled Dice of 0.8959, outperforming an MAE counterpart pretrained on large-scale natural-image data by 0.0694 and 0.0613, respectively. The results further indicate that increasing pretraining scale produces consistent improvements in dense feature transfer. Overall, these findings suggest that scaling in-domain self-supervised pretraining can reduce the domain gap and improve transferable dense representations for high-pixel-dimension scientific imaging.
Read the full abstract ▾
All authors Junlin Guo, Sharmin Majumder, Isaac Lyngaas, John Lagergren, Xiao Wang
Computer Vision
KeyRec: Bounded Visual Memory for Streaming and Long-Video Understanding
Zihan Chen, Xuejian Rong, Xiaojuan Wang et al. (+4 more)
Vision-
language models are increasingly used to understand long
videos and continuous streams. However, dense visual
tokens accumulate with
video duration, making long-context inference prohibitively expensive. Existing training-free visual-
token selection methods reduce this cost by retaining informative
tokens, but may lose coherent event evidence and fail to distinguish detailed recent observations from long-range history. We propose KeyRec, a training-free framework for constructing bounded visual
memory. During query-agnostic writing, KeyRec preserves fine-grained recent observations in a visual cache and organizes historical evidence into a structured event bank. Candidate events are proposed according to their novelty relative to previously stored events and maintained through an online add–merge–evict update. When a question arrives, a text-only router adaptively allocates a fixed readout budget between recent and event
memory, without reprocessing historical frames. KeyRec operates on model-facing visual
embeddings and supports both modular encoder–projector VLMs and the encoder- and projector-free NEO-ov architecture. Across four streaming and long-
video benchmarks and three VLM backbones, KeyRec achieves the best compressed performance in 13 of 15 settings using only 10% of the dense decoder-facing visual-
token budget. It outperforms the strongest compressed baseline by 2.21–18.37 points on real-time questions, achieves the best compressed result in five of six long-
video settings, and performs best in every NEO-ov 2B setting.
Read the full abstract ▾
All authors Zihan Chen, Xuejian Rong, Xiaojuan Wang, Boqing Gong, Adi Zicher, Yael Pritch, Nikhil Karnad
Computer Vision
Binaural Audio-Visual Instance Segmentation
Saijun Wang, Guanfeng Tang, Hongbo Zhao et al. (+4 more)
Audio-visual segmentation (AVS) aims to segment sounding objects at the pixel level by integrating auditory and visual cues. However, existing methods are predominantly developed under the monaural setting and primarily rely on cross-modal semantic correspondence, which limits their ability to distinguish visually similar instances of the same semantic class. In contrast, humans naturally exploit binaural hearing, where interaural differences and direction-dependent acoustic filtering introduced by the head and pinnae provide physically grounded spatial cues for accurate sound source localization. Motivated by this observation, we introduce binaural audio-visual instance segmentation (BiAVIS), a new task that leverages synchronized binaural audio and
video frames to segment sounding instances. To advance research on this task, we establish two
benchmarks by manually annotating an existing binaural audio-visual
dataset and collecting a new real-world
dataset, BiAVIS-Bench, in more challenging and diverse scenarios. We further propose a BiAVIS model, which leverages an audio-only sound source localization network to learn spatial and semantic priors for sounding instances from binaural audio. A query-level audio-visual fusion strategy is subsequently introduced to inject these informative priors into the instance segmentation decoder. Extensive experiments conducted on the two proposed
benchmarks demonstrate the superior performance of the BiAVIS model over previous monaural AVS methods, especially in resolving instance-level intra-class ambiguity. On the more challenging BiAVIS-Bench, the proposed BiAVIS model outperforms the best-performing monaural baselines by 17.22% in mAP and 7.71% in FSLA, respectively.
Read the full abstract ▾
All authors Saijun Wang, Guanfeng Tang, Hongbo Zhao, Zhicheng Lei, Yutong Zhang, Wei Ye, Rui Fan
Computer Vision
Federated 3D Gaussian Splatting for Large-Scale Scene Reconstruction at Wireless Edge
Guanlin Wu, Chao Hu, Pu Chen et al. (+4 more)
Three-dimensional (3D) Gaussian splatting (3D-GS) has emerged as a promising technique for large-scale scene reconstruction due to its high rendering efficiency and fidelity. However, the training of large-scale 3D-GS models at wireless edge faces various technical challenges including the limited communication, computation, and graphics processing unit (GPU)
memory resources at edge devices, the structural inconsistency issue across local models hindering their effective aggregation, as well as
privacy leakage risks associated with raw visual content and camera parameters. To address these challenges, this paper proposes a novel resource-efficient
federated learning framework for efficiently training 3D-GS models of large scenes under severe resource constraints. First, we propose an on-device model lightweighting mechanism that adaptively selects and prunes Gaussian points to balance the rendering quality and training efficiency. In this mechanism, we quantitatively evaluate the importance of different Gaussian points at each device to facilitate the
pruning, and use a novel importance-to-latency ratio criterion to determine the number of pruned Gaussian points under GPU
memory and computation/communication latency constraints. Furthermore, we develop a 3D-GS model recovery mechanism that restores structural consistency across local 3D-GS models without accessing private camera parameters, enabling their effective aggregation towards a global model. Finally, extensive experiments show that our approach significantly accelerates convergence, maintains high rendering quality, and reduces training latency compared to state-of-the-art
federated 3D-GS baselines.
Read the full abstract ▾
All authors Guanlin Wu, Chao Hu, Pu Chen, Juyong Zhang, Han Hu, Shuguang Cui, Jie Xu
Computer Vision
OneFixer: High-Quality and Consistent One-Step Autoregressive 3DGS Refinement for Driving Scenes
Boseong Jeon, Junhyeop Lee, Juhan Cha et al. (+1 more)
Autoregressive
video diffusion is a promising render-time fixer for 3D Gaussian Splatting (3DGS) in autonomous-driving simulation, but deployment demands high visual quality and temporal consistency at low latency. This is especially hard for one-step causal generation, where each imperfect prediction immediately becomes context for subsequent frames. Existing approaches stabilize rollouts through staged training with multiple modules and rollout-aware regularization, yet one-step quality still falls short of what deployment requires. We introduce OneFixer, a one-step autoregressive
video-
diffusion fixer trained in a single task-specific adaptation stage. Our key idea is a deployment-matched shared rollout: the model's own one-step predictions serve as the causal context for flow matching, exposing training to deployment-time errors, while the same rollout receives direct pixel-space perceptual supervision to preserve fine detail. Because the predictions optimized for current-frame quality are exactly those reused as future context, fidelity and autoregressive robustness are learned jointly, without bidirectional-to-causal conversion or teacher-student
distillation. OneFixer further exploits cues that driving simulation readily provides, lane geometry and dynamic-
agent states, to improve
geometric fidelity. On Waymo and proprietary driving scenes with 900-frame rollouts, OneFixer achieves the lowest FVD, LPIPS, and DISTS among all baselines at one step, with temporal consistency matching or exceeding multi-stage DMD pipelines. Under identical backbone and conditioning, it matches a multi-stage DMD-with-Self-Forcing pipeline in under half the GPU-hours and keeps improving beyond its plateau. In closed-loop simulation with a driving policy, OneFixer reduces the collision rate by a third relative to raw 3DGS rendering. Project page: https://onefixer-web.vercel.app/
Read the full abstract ▾
All authors Boseong Jeon, Junhyeop Lee, Juhan Cha, Hayoung Kim
Computer Vision
PQR3D: Progressive Query Refinement over Reference-Conditioned Temporal Windows for Multi-View 3D Object Detection
Hui Ye, Yudong Liu, Yiran Chen et al. (+2 more)
Temporal context is essential for camera-only multi-view 3D object detection. Existing streaming detectors maintain and propagate query states from one frame to the next, requiring sequence-aware training and chronological inference. We propose PQR3D, which performs progressive query refinement within referenceconditioned temporal windows. This design enables random frame sampling and independent inference without persistent query
memory. Within each window, PQR3D progressively transfers motion-aligned high-confidence queries from earlier timestamps toward the target frame. We further introduce masked selfattention to regulate interactions among regular, propagated, and denoising queries while keeping denoising supervision isolated from detection queries. In addition, a stage-decoupled anchor
embedding injects position before self-
attention and size, orientation, and velocity afterward, reducing interference from temporally inconsistent attributes. With a ViT-L backbone, PQR3D sets a new state of the art on the nuScenes test set, achieving 71.6 NDS and 64.9 mAP. Source code is available at https://github.com/huiyegit/PQR3D
Read the full abstract ▾
All authors Hui Ye, Yudong Liu, Yiran Chen, Rajshekhar Sunderraman, Shihao Ji
Computer Vision
PruneForget: Joint Unlearning and Pruning of Vision Models
Yu-Shan Tai, Amber Yijia Zheng and Raymond A. Yeh
Machine unlearning and model
pruning are increasingly coupled in the real world. Models must support unlearning requests, e.g., for safety concerns, while also meeting requirements in latency and
memory budget. Until recently, existing works have studied each aspect as an independent problem, e.g., running unlearning and
pruning sequentially. In this work, we show that unlearning and
pruning are naturally aligned and should be solved jointly to be made aware of each other. Intuitively, parameters that encode information of the unlearned samples are natural
pruning targets, as unlearning and
pruning both call for the "deletion" of such parameters. We propose PruneForget, a method that uses the unlearn set as a guide for
pruning, so that unlearning and
pruning mutually benefit each other. Extensive experiments on image classifiers and generative models show that PruneForget removes the influence of the unlearned samples while producing a more compact model with reduced inference cost. It achieves a negligible performance gap relative to an oracle that retrains from scratch for unlearning and then prunes.
Read the full abstract ▾
Computer Vision
CausalDriveBench: Evaluating Causal Reasoning in Vision-Language-Action Models for Autonomous Driving
Narendiran Chembu, Navvrat Rao, Shreedhar Shreeshail Kodate et al. (+10 more)
Vision-
Language-Action (VLA) models for autonomous driving produce natural-
language reasoning alongside predicted trajectories, but whether this
reasoning reflects the causal structure of the scene remains untested. We introduce CausalDriveBench, an evaluation framework grounded in Pearl's Causal Hierarchy (PCH) that tests causal
reasoning in driving-specific VLAs through structured visual question answering (QA) and alternative-trajectory prediction. To this end, we construct causal scene graphs over nuScenes that distinguish causally active, dormant, and distractor entities, separating perceptual salience from causal relevance. The
benchmark spans all four rungs of PCH (association, intervention, and counterfactual along with causal discovery) for QA generation. For the higher rungs, we additionally provide reference trajectories under specified scene modifications, enabling action-level verification that complements
reasoning-level evaluation. In total, the
benchmark contains 7,285 verified causal QA pairs and 1,000 counterfactual trajectories derived from nuScenes. We evaluate 10 driving-specific VLAs and 3 general-purpose VLMs, and report three findings. First, the best model reaches only 70.6% QA accuracy, and 4 of 13 models score below random chance. Second, comparing each driving VLA to the general-purpose VLM that shares its
language backbone, the cost of driving fine-tuning ranges from 2 to 34 percentage points on causal QA, with post-training design explaining the spread. Third, causal QA and trajectory accuracy are statistically uncorrelated across models: under counterfactual
prompts, predicted trajectories either over-react or collapse onto the observed-scene baseline. Taken together, these results show that neither fluent rationales nor accurate observed-scene trajectories constitute evidence of causal understanding.
Read the full abstract ▾
All authors Narendiran Chembu, Navvrat Rao, Shreedhar Shreeshail Kodate, Gayatri Srujana Banda, Arko Sarkar, Abhinav Khanna, Rajarshee Das, Umesh Kanala, Siddarth Khandelwal, Kumar Aman, Aish Dubey, Kaustubh Beedkar, Arjun Jain
Computer Vision · Cross-listed from cs.RO
AquaBEV-Nav: Learned BEV Occupancy for Underwater Navigation and Exploration
Trung Tien Dong, Zhenqi Wu, Sahasra Kondapalli et al. (+3 more)
Safe underwater exploration requires a
robot to understand where surrounding structures are located and which regions are available for motion. Existing
vision-based underwater exploration systems commonly obtain this information indirectly by estimating monocular depth, unprojecting the geometry into 3D space, and accumulating it into a 2D bird's-eye-view occupancy map. This reliance on intermediate depth estimation is particularly problematic underwater, where scattering and wavelength-dependent attenuation degrade visual cues and limit the reliability of monocular depth estimates. We introduce AquaBEV-Nav, an underwater exploration framework that bypasses explicit monocular depth estimation through direct bird's-eye-view occupancy prediction. Built upon the CORAL hierarchical exploration framework, AquaBEV-Nav replaces its depth-based perception front end with AquaBEV. Given a single RGB frame, AquaBEV maps visual features into a learned polar representation, performs causal
reasoning along the range dimension, and reconstructs local Cartesian occupancy without relying on intermediate depth prediction. The resulting occupancy map is accumulated into CORAL's persistent spatial
memory, providing spatial context for VLM-based high-level planning and collision constraints for dynamics-aware local trajectory generation. Across ten simulated reef environments and six occupancy backbones evaluated under a single protocol, AquaBEV-Nav reaches 37.48 structure IoU and 53.2 target IoU, 88.95% closed-loop coverage with zero collisions.
Read the full abstract ▾
All authors Trung Tien Dong, Zhenqi Wu, Sahasra Kondapalli, Jiayi Wu, Yi Sheng, Xiaomin Lin
Computer Vision
Gaussian Image Steganography via Parameter-Domain Keyed Embeddings
Tong Wu, Runze Cheng, Xiaoyue Fan et al. (+1 more)
2D Gaussian-based image representation is becoming increasingly popular, and our work proposes a new approach to steganography by
embedding information within Gaussian parameters rather than image pixels. We first fit the parameters of this representation to the target image and employ a secret key to select a subset of these parameters for fine-tuning, allowing us to embed an 8-bit message while maintaining high visual fidelity in the reconstructed image. Thirty fitting experiments on three synthetic images show that the correct key can recover the message without error, while decoding with incorrect keys yields a Bit Error Rate (BER) of $0.543$, close to random guessing. Compared with random selection using the key, selecting the least-disturbing edits recovers the message more reliably (one-sided $p=0.031$), and the average PSNR cost is only $0.091$ dB in visual quality. The
embedding method transfers to 112 natural images at $256\times256$ using 4,096 Gaussians. The correct-key BER is $0.000$, and decoding under a wrong key stays close to random guessing at $0.520$. Our method embeds the payload through three Gaussian parameter types: log-anisotropy, opacity, and color luminance. In a separate nine-fit reduced setting, color luminance is removed, so the payload uses two instead of three parameter types, a $33.3\%$ reduction; all 256 Gaussians remain in the fitted representation. The correct key still recovers the message without error. However, wrong-key BER rises from $0.514$ to $0.571$, moving farther from random guessing ($0.5$).
Read the full abstract ▾
All authors Tong Wu, Runze Cheng, Xiaoyue Fan, Kaan Akşit