Computer Vision
Divide-and-Denoise: A Game-Theoretic Method for Fairly Composing Diffusion Models
Abhi Gupta, Polina Barabanshchikova, Vikas Garg et al. (+2 more)
The abundance of pre-trained
diffusion models provides an opportunity for composition. Combining several models, however, runs the risk of one model dominating or models disagreeing with each other. Here, we propose Divide-and-Denoise, a method for coordinating multiple pre-trained
diffusion models during sampling. Much like managing a specialized workforce, our method creates a fair but efficient division of labor across models. Central to our method is the notion of an allocation which defines the responsibility of each model to every region of the noisy sample. At every timestep, we then denoise by (i) updating the allocation by solving a fair division game, where we divide the sample into regions that maximize total utility under
fairness constraints, and (ii) aligning the models with this allocation, where we guide each model to denoise within its assigned region. This leads to a new composite denoising process that evolves in tandem with a division process. We evaluate Divide-and-Denoise on conditional image generation. Across several quality metrics, including the GenEval
benchmark, our method outperforms baselines and resolves common failures including missing objects and mismatched attributes. Experiments show that Divide-and-Denoise utilizes each model's expertise without neglecting any other model.
Read the full abstract ▾
All authors Abhi Gupta, Polina Barabanshchikova, Vikas Garg, Samuel Kaski, Tommi Jaakkola
Computer Vision · Cross-listed from eess.IV
Jonathan Schwartz, Utz Heinrich Ermel, C. Braxton Owens et al. (+6 more)
Cryo-electron
tomography (cryoET) has emerged as a powerful tool in structural and cellular biology by enabling direct visualization of macromolecular structures within intact
cells, thereby linking molecular architecture to cellular organization in a native context. Realizing the full potential of cryoET, however, increasingly depends on advances in computational analysis, particularly machine learning (ML), to interpret its complex and information-rich data. Despite rapid progress, ML development for cryoET remains bottlenecked by the lack of standardized, well-annotated
benchmarks. Existing evaluations are typically small, task-specific, and are assembled in isolation, limiting robust comparisons across methods. Here, we present POPSICLE, a
benchmark suite for cryoET
segmentation and macromolecular localization built from the CryoET Data Portal - an open, ML-ready repository of tomographic data, metadata, and annotations. POPSICLE spans eukaryotic and prokaryotic systems, both purified and fully in situ samples, and dense voxel-wise
segmentation as well as sparse localization tasks. Built on a living data resource, it can expand as new
datasets and annotations become available. Baseline experiments reveal substantial variation in model rankings across tasks, underscoring the need for
benchmarks tailored to the unique characteristics of cryoET rather than evaluation practices adapted from adjacent biomedical imaging domains. POPSICLE thus provides an open and extensible foundation for reproducible ML evaluation in cryoET.
Read the full abstract ▾
All authors Jonathan Schwartz, Utz Heinrich Ermel, C. Braxton Owens, Zhuowen Zhao, Ariana Peck, Gus L.W. Hart, Grant J. Jensen, Bridget Carragher, Dari Kimanius
Computer Vision · Cross-listed from cs.SD
Dual-Branch Gated Fusion for Open-Set Audio Deepfake Source Tracing
Awais Khan, Kutub Uddin and Khalid Malik
Attributing a synthetic utterance to its originating system remains an open challenge: closed-set models fail to reject unseen synthesizers and produce overconfident predictions. To address this, we propose a dual-branch gated fusion framework that pairs XLSR-53 with CORES, a 66-dimensional descriptor that, unlike prior Linear Filter Bank (LFB)-only work, spans cepstral, oscillatory, rhythmic, energy, and spectral dimensions to capture complementary synthesis artifacts. Our analysis shows XLSR-53 remains discriminative in-domain (ID) while CORES generalizes stably under distribution shift (OOD), yet their naive concatenation fails due to SSL representational imbalance. To resolve this, an input-conditioned gate adaptively weights each branch under joint training with cross-
entropy, an energy margin loss for ID/OOD separation, and a gate diversity term. On the MLAAD
benchmark, our system achieves 97.6% ID accuracy, 4.9% EERc, and an 83.5% relative FPR95 reduction over the Interspeech 2025 baseline.
Read the full abstract ▾
Computer Vision
An Improved Generative Adversarial Network for Micro-Resistivity Imaging Logging Restoration
Ahmed Faizul Haque, S.M. Riaz Rahman Antu, Saif Ahmed et al. (+5 more)
An improved GAN-based imaging logging image restoration method is presented in this paper for solving the problem of partially missing micro-resistivity imaging logging images. The method uses FCN as the generative network infrastructure and adds a depth-separable convolutional residual block to learn and retain more effective pixel and semantic information; an Inception module is added to increase the multi-scale perceptual field of the network and reduce the number of parameters in the network; and a multi-scale feature extraction module and a spatial
attention residual block are added to combine the channel
attention. The multi-scale module adds a multi-scale feature extraction module and a spatial
attention residual block, which combine the channel
attention mechanism and the residual block to achieve multi-scale feature extraction. The global discriminative network and the local discriminative network are designed to gradually improve the content and semantic structure coherence between the restored parts and the whole image by playing off each other and the generative network. According to the experimental results, the average structural similarity measure of the five sets of imaged logging images with different sizes of missing regions in the test set is 0.903, which is an improvement of about 0.3 compared with other similar methods. It is shown that the method in this study can be used for the restoration of micro-resistivity imaging log images with good improvement in semantic structural coherence and texture details, thus providing a new deep learning method to ensure the smooth advancement of the subsequent interpretation of micro-resistivity imaging log images.
Read the full abstract ▾
All authors Ahmed Faizul Haque, S.M. Riaz Rahman Antu, Saif Ahmed, Asadullah Hil Galib, Souvik Pramanik, Mohammad Ashrafuzzaman Khan, Mohammad Abdul Qayum, Mohsin Sajjad
Computer Vision · Cross-listed from cs.LG
Density Ridge Selective Prediction for LLM and VLM Hallucination Detection under Calibration Label Scarcity
Nina I. Shamsi
Hallucination detection in large
language and
vision-
language models is increasingly framed as selective prediction, where a detector assigns a confidence score and abstains when confidence is low. Unsupervised sampling detectors (Semantic
Entropy) avoid labels but plateau in quality, while supervised probes attain stronger in-distribution scores yet degrade sharply when calibration labels are scarce. We recover the response
manifold of an LLM as the density ridge of a kernel density estimate built on a six-dimensional kinematic feature map of hidden state generation trajectories. A test generation is scored by the negated Euclidean distance from its projected feature point to the nearest ridge vertex, yielding a low-dimensional
geometric skeleton of the
stochastic output distribution. We evaluate against Semantic
Entropy, topological methods, and log-probability on six QA
benchmarks (HaluEval-QA, TriviaQA, GSM8K, POPE, ScienceQA, A-OKVQA) using eight text and
vision LLMs in a deliberately label-scarce protocol ($n_{\text{cal}}{=}200$ queries, $N{=}5$ generations). Our ridge-based score beats on AUROC with 5-20 points gain, while demonstrating tempered degradation under calibration-label scarcity.
Read the full abstract ▾
Computer Vision
Fisher-Guided Progressive Parameter Selection for Adaptive Fine-Tuning
Ghodsiyeh Rostami, Po-Han Chen and Mahdi S. Hosseini
Parameter-efficient fine-tuning often selects trainable parameters before adaptation using architectural heuristics, without accounting for their varying importance during training. We introduce FisherAdapTune, which progressively selects parameter groups based on temporal drift in their Fisher information. Under a local Gaussian approximation, we bound the divergence between the fine-tuned posterior and pretrained prior by accumulated Fisher-weighted update costs, motivating
curvature-aware selection. FisherAdapTune measures Jensen-Shannon distance between successive Fisher-value distributions and uses an adaptive threshold to freeze stabilized groups. Across VTAB-1k classification tasks, it achieves the highest macro Top-1 accuracy among the compared methods with a smaller average trainable set than full fine-tuning. Across four
segmentation backbones, it maintains competitive in-distribution performance and improves zero-shot transfer in several settings. Its selections reveal architecture-dependent patterns where input, output, and normalization parameters can remain trainable as
attention and MLP groups freeze at different rates. The results support Fisher structural drift as a task-dependent signal for allocating updates during adaptation. We release our \href{https://github.com/AtlasAnalyticsLab/FisherAdapTune}{code} publicly to enable further application of our proposed approach.
Read the full abstract ▾
Computer Vision
Konstantin Kuklev, Viacheslav Vasilev, Alexander Kunitsyn et al. (+2 more)
Modern
Diffusion Transformers for
video generation provide limited control over the progression of time and the editing of temporal dynamics. We propose a temporal-control methodology that extends a pretrained DiT with explicit time editing, allowing control over motion speed and temporal structure without redesigning the backbone. Its core implementation augments the pretrained model with a lightweight temporal module, preserving the original generative prior while expanding its controllable dynamic range.
Read the full abstract ▾
All authors Konstantin Kuklev, Viacheslav Vasilev, Alexander Kunitsyn, Andrei Ivaniuta, Denis Dimitrov
Computer Vision
A Large Scale Open-Source Image and Video Dataset for Robust Wildfire Detection and Classification
Emadeldeen Hamdan, Yingyi Luo, B. Ugur Toreyin et al. (+4 more)
Wildfire detection and monitoring are critical for mitigating fire spread and reducing environmental and infrastructural damage. In this work, we introduce GWFP (Global Wildfire Prevention
Dataset), a large-scale, open-source
dataset of wildfire images and
videos designed to support early fire and smoke detection research. GWFP contains geographically diverse wildfire scenes, including flames, smoke, Waterdog/Fog environmental conditions, Near Infrared (NIR) imagery, Ember, and challenging negative samples collected from real-world scenarios worldwide. To evaluate
dataset robustness and cross-domain generalization, we
benchmark multiple convolutional and
transformer-based architectures across both in-domain and cross-
dataset settings. Additionally, we explore lightweight frequency–spatial feature interaction using Hadamard-enhanced residual connections (HTE-ResNet) to analyze representation robustness under domain-shift conditions. Experimental results demonstrate strong cross-
dataset generalization and practical utility for real-world wildfire monitoring applications. The
dataset is available on Kaggle: https://www.kaggle.com/
datasets/emadeldeenhamdan/wildfire.
Read the full abstract ▾
All authors Emadeldeen Hamdan, Yingyi Luo, B. Ugur Toreyin, Erdem Koyuncu, Adam J. Watts, Ugur Gudukbay, Ahmet Enis Cetin
Computer Vision
FlexPath: Adapting Learned Connectivity Guidance to Path Preferences
Taehyoung Kim, Tim Schoenbrod, David Eckel et al. (+1 more)
Recent learning-based path planners use neural networks to process occupancy representations and approximate heuristics for classical search
algorithms, yielding near-optimal paths with reduced search effort. However, these methods are tied to a fixed objective, usually the shortest-path objective, implicit in their supervision. This limits their flexibility to accommodate alternative criteria. We introduce $\textbf{FlexPath}$, a two-stage learned search-guidance framework that first learns a recall-oriented connectivity prior initialized from shortest-path planner demonstrations and then refines this prior using differentiable path-shape objectives, thereby separating demonstration-based learning of $\textbf{connectivity-biased guidance}$ from subsequent $\textbf{objective specific refinement}$. Beyond enabling adaptation to new routing preferences, the two-stage procedure improves standard shortest-path planning itself: on TMP, FlexPath improves optimal-path recovery from 75.0% to 88.6% over TransPath while reducing search expansions by 13.8%. Ablations show that neither prior learning nor objective fine-tuning alone matches the full pipeline; their combination yields the strongest path cost and search efficiency. We further demonstrate the preference adaptation by adapting guidance to non-shortest-path objectives such as obstacle clearance, class-conditioned obstacle clearance and waypoint following. For clearance with $d_{\min}=2$, FlexPath achieves 96.2% full clearance satisfaction on feasible instances while maintaining low search effort, and it reaches 98.4% waypoint-following success.
Read the full abstract ▾
All authors Taehyoung Kim, Tim Schoenbrod, David Eckel, Henri Meeß
Computer Vision
Fusing Satellite Imagery and Planimetric Maps for Cross-View Localization
Quang Long Ho Ngo, Zimin Xia and Alexandre Alahi
Current cross-view localization methods predominantly rely on satellite imagery as the aerial modality. Although recent work explores planimetric maps (e.g., OpenStreetMap tiles), these approaches often lag in performance. Yet both modalities are widely available and possess complementary properties. Satellite images are closer to ground-level camera imagery, offering finer detail, whereas planimetric maps contain annotated objects (e.g., streetlamps) and remain informative in areas where the ground is occluded, such as by foliage. Despite this, only one prior work provides an end-to-end method to fuse the two modalities, and it does not demonstrate their potential within state-of-the-art methods. To combine the strengths of both modalities, we propose a new fusion module that augments standard encoders and demonstrates that integrating satellite imagery with planimetric maps improves state-of-the-art single-modality methods. The module comprises (i) cross-modal conditioning, which processes each modality's encoding with awareness of the other, and (ii) a patch-level fusion rule that controls the granularity of information exchange. We achieve state-of-the-art results, reducing the mean localization error by 30.13%. Qualitatively, the fusion adaptively selects the more informative modality, improving overall accuracy.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
From Senses to Decisions: The Information Flow of Auditory and Visual Perception in Multimodal LLMs
Wish Suharitdamrong, Muhammad Awais, Xiatian Zhu et al. (+1 more)
Multimodal Large
Language Models (MLLMs) can listen and see, but how do audio and visual signals actually travel through the network to shape an answer? Despite their growing role in research and real-world applications, the internal pathways through which audio and visual
tokens influence the final prediction remain poorly understood. In this study, we examine audio-visual information flow inside Audio-Visual Large
Language Models (AVLLMs), tracing how AVLLMs route, utilize, and integrate audio and visual information across two input configurations, audio-visual
video and multiple interleaved audio-visual items. We find that for audio-visual
video, AVLLMs follow the sequential information flow pathway established for VLMs and VideoLLMs, with audio and visual contribution flowing along this pathway in proportion to the task's reliance on each modality. In settings with multiple interleaved audio-visual items, this routing shifts to different parallel streams. Furthermore, we demonstrate that audio-visual and other
token types can be discarded once their information is transferred to LLM, with minimal impact on the model's prediction or even slight improvement, generalizing across multiple tasks and
datasets, enabling more efficient inference. These findings hold across multiple models and scales, Qwen2.5-Omni and
Video-SALMONN2 Plus at 3B and 7B scales, leading to hypotheses on why these flow structures emerge. Together, these results deliver the first coherent picture of how AVLLMs orchestrate sound and sight inside the network and lay the groundwork for the next wave of interpretability, design, and efficiency advances in audio-visual and broader MLLMs.
Read the full abstract ▾
All authors Wish Suharitdamrong, Muhammad Awais, Xiatian Zhu, Sara Atito
Computer Vision
DB-3DME: From Dataset to Benchmark for Human-aligned Automatic 3D Mesh Evaluation
Nanshan Jia, Zhenyu Zhao, Junbo Peng et al. (+3 more)
Recent advances in 3D generation have led to substantial improvements in realism, controllability, and efficiency, yet the evaluation of 3D assets remains underexplored. Existing evaluation paradigms, including human evaluation, learned metrics, and
vision-
language models (VLMs) as judges, suffer from limitations in cost, scalability, resolution handling, or task-specific
alignment. In this work, we focus on 3D mesh evaluation and introduce DB-3DME, the
Dataset and
Benchmark for 3D Mesh Evaluation. DB-3DME contains 2,619 synthetic 3D meshes paired with human ratings on Geometry and
Prompt Adherence. Using this
dataset, we systematically
benchmark state-of-the-art VLMs and identify visual encoding of 3D representations as a key factor for human-aligned evaluation performance. Motivated by this finding, we fine-tune an open-weight VLM, Qwen-2.5-VL-7B, for 3D mesh evaluation by adapting the visual encoder while freezing the
language model. The fine-tuned model substantially outperforms existing pre-trained VLMs across multiple evaluation dimensions, establishing a new
benchmark for automatic 3D mesh evaluation. We publicly release the
benchmark dataset on GitHub and Hugging Face to facilitate future research.
Read the full abstract ▾
All authors Nanshan Jia, Zhenyu Zhao, Junbo Peng, Sui Huang, Jingshen Wang, Zeyu Zheng
Computer Vision
iSAGE: A Human-in-the-Loop Framework for Remote Sensing Semantic Segmentation via Sparse Point Supervision
Osmar Luiz Ferreira de Carvalho, Osmar Abilio de Carvalho Junior, Anesmar Olino de Albuquerque et al. (+1 more)
Semantic
segmentation in remote sensing requires costly pixel-level annotations, and nearly every problem demands a new
dataset since models rarely transfer across sensors, platforms, or geographies. Existing human-in-the-loop frameworks expand sparse clicks into dense supervision via auxiliary machinery (pseudo-labels, propagation, CRFs, foundation-model
prompts, auxiliary heads), all operating on the model's predictive distribution. A confidently wrong pixel is indistinguishable from a confidently correct one in that distribution by construction, so no rule reading it can separate the two; the distinguishing signal is external to the model. This paper hypothesizes that expert clicks targeting confident model errors, not arbitrary pixels, suffice to match dense supervision, with no expansion machinery. iSAGE (Iterative Sparse Annotation Guided by Expert) realizes this hypothesis on an integrated open-source platform, where an error-weighted loss amplifies the gradient at each click and the annotation record itself is the
dataset, extensible, correctable, and auditable. Experiments use a minimum-effort regime: at most one labeled pixel per class per frame. On BsB Aerial, iSAGE recovers 97.2% of dense supervision (74.79% mIoU on 0.040% of pixels) with contrasting class dynamics: amorphous classes (permeable areas) saturate from the seed, while small classes (cars) require late-iteration effort. On ISPRS Vaihingen (external
benchmark), iSAGE reaches 76.78% mIoU with 0.011% of pixels, matching the dense baseline (76.65%) and exceeding all published methods. Under the same pipeline, four output-reading mechanisms (oracle
entropy across budgets 1–100x, pseudo-labels across thresholds 0.90–0.99, CRF-based propagation, uniform random) plateau 7.4 to 14.5 pp below iSAGE. Across 31 surveyed methods, iSAGE is the only iterative human-in-the-loop framework operating without auxiliary machinery.
Read the full abstract ▾
All authors Osmar Luiz Ferreira de Carvalho, Osmar Abilio de Carvalho Junior, Anesmar Olino de Albuquerque, Daniel Guerreiro e Silva
Computer Vision
BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregression
Shaohao Rui, Xiaofeng Mao, Zhanyu Zhang et al. (+6 more)
Interactive
video world models commonly convert bidirectional
video generators into causal autoregressive systems through control fine-tuning, autoregressive training, causal initialization, and few-step
distillation. This pipeline is costly, while frozen causal histories accumulate errors that degrade long-horizon fidelity and controllability. We present BiWM, the first open-source full-stack training framework for bidirectional autoregressive
video world models. BiWM retains full
attention within each generated chunk and requires only two stages: camera/action-control fine-tuning and few-step Distribution Matching
Distillation (DMD). Both stages converge within a few hundred optimizer steps on 8 H200 GPUs. The framework supports Wan2.1-T2V-1.3B, Wan2.2-TI2V-5B, HunyuanVideo-1.5-TI2V-8B, and LTX-2.3-22B, together with real-world camera control, pluggable long-history compression, and optional low-bit deployment. Supervised and forward-KL anchors mitigate DMD mode collapse and preserve scene dynamics. BiWM provides a compact, reproducible path from pretrained bidirectional
video models to interactive, controllable, and efficient world models.
Read the full abstract ▾
All authors Shaohao Rui, Xiaofeng Mao, Zhanyu Zhang, Peijia Lin, Yansong Zhu, Yibo Zhang, Haibin Wan, Zhangrui Zhao, Weijie Ma
Computer Vision
Improving PET/CT-Based Whole-Body Lesion Segmentation Using Prediction Uncertainty-Augmented Models
Bashirul Azam Biswas, Biratal Raj Wagle, Zhihan Yang et al. (+4 more)
Accurate lesion
segmentation from whole-body Positron Emission
Tomography (PET)/Computed
Tomography (CT) scans is essential for cancer staging and treatment planning. PET provides functional metabolic information with different radiotracers, while CT offers anatomical localization. Lesion delineation from PET/CT imaging is clinically challenging due to subtle imaging features, confounders, and inter-reader variability. Existing deep learning approaches suffer from training-related stochasticity, inconsistent predictions, missed lesions in high tumor-burden cases, and lack uncertainty quantification, limiting their clinical reliability. Using nnU-Net as a baseline, we propose an uncertainty-aware framework for whole-body PET/CT lesion
segmentation that integrates (1)
Bayesian ensembling to reduce training stochasticity, (2) voxel-wise uncertainty quantification with epistemic and aleatoric decomposition, and (3) epistemic uncertainty-augmented training to improve lesion detection. Two public
datasets, AutoPET-III (1,611 scans) and Deep-PSMA (200 scans), comprising FDG and PSMA studies across multiple cancer types, are used for training and evaluation.
Bayesian ensembling improves robustness and performance over deterministic nnU-Net models on the unseen AutoPET-III test set. Uncertainty maps highlight regions of model disagreement and correlate with misclassifications, particularly false positives. Uncertainty-augmented training improves lesion recovery at the cost of increased FPVol, reflecting a precision-recall trade-off. A case-adaptive routing strategy further improves Dice by selecting between the base and augmented models. To our knowledge, this is the first study to systematically investigate uncertainty quantification in multi-tracer, pan-cancer PET/CT
segmentation and to combine
Bayesian ensembling with uncertainty-aware modeling for this task.
Read the full abstract ▾
All authors Bashirul Azam Biswas, Biratal Raj Wagle, Zhihan Yang, Marc A. Seltzer, Matthew E. Maeder, James B. Yu, Indrani Bhattacharya
Computer Vision
Maximum Matching Accuracy: An Instance Segmentation Evaluation Metric Utilizing Globally Optimal Matching
Kaden Stillwagon, Alexandra D. VandeLoo and Craig R. Forest
Reliable evaluation of instance
segmentation models requires metrics that accurately and consistently reflect
segmentation quality. However, the metrics most widely used in biological imaging carry fundamental mathematical weaknesses: hard Intersection-over-Union (IoU) thresholds that produce discontinuous, low sensitivity scoring; per-object normalization that distorts scores under object size variation; and greedy or one-to-many matching procedures that yield non-optimal, order-dependent correspondences. Together, these properties produce unintuitive and unreliable model rankings under common failure modes such as split
cells, merged
cells, and
cell boundary imprecision. We propose Maximum Matching Accuracy (MMA), a threshold-free continuous metric that finds a globally optimal one-to-one matching between predicted and ground truth objects and aggregates total overlap using per-pixel normalization. We evaluate MMA against AP@50, PQ, SEG, and AJI across three experiments: synthetic failure cases, progressive corruption tests, and a model ranking comparison. MMA produces scores that are more stable, more sensitive, and more interpretable than existing alternatives, providing a principled foundation for fair instance
segmentation benchmarking in biological
cell imaging.
Read the full abstract ▾
Computer Vision
Interpretable Temporal Facial-Region Motion Analysis for In-the-Wild Parkinson's Disease Video Classification
Riyadh Almushrafy
Reduced facial expressivity is a common motor manifestation of Parkinson's disease (PD), often described as hypomimia or facial bradykinesia. This paper examines whether temporal motion descriptors extracted from facial-region keypoints can support in-the-wild PD-related
video classification on the YouTubePD
benchmark. Each
video is represented using
geometric descriptors from 14 predefined facial regions. Static geometry, normalized geometry, velocity-based descriptors, relative-velocity descriptors, and a GRU sequence baseline are compared under the same binary classification protocol. To assess stability and interpretability, the study includes seed-robustness analysis, region-level ablation, and permutation importance. The best result is obtained with normalized velocity descriptors and a Random Forest classifier, reaching a balanced accuracy of 0.826 and an AUROC of 0.855 on the held-out test split. Across 10 random seeds, this representation remains stable, with balanced accuracy of 0.810 +/- 0.018 and AUROC of 0.855 +/- 0.005. Overall, the results suggest that normalized facial-region motion is a lightweight and interpretable representation for YouTubePD
video classification. The study is framed as a
benchmark-level analysis and does not claim clinical severity assessment or MDS-UPDRS facial-expression scoring.
Read the full abstract ▾
Computer Vision
Measuring Pretraining Contamination in Public Medical Vision-Language Benchmarks
Bruce Changlong Xu, Lan Wu and Alexander Ryu
Medical
vision-
language models (VLMs) are evaluated on public
benchmarks whose images and question-answer pairs have been freely available for years, yet reported accuracy assumes that these examples were not in the training data. We test this assumption for open VLMs on SLAKE-En, PathVQA, and VQA-RAD, and on a public mirror of OmniMedVQA, using four detectors: image near-neighbour search against PMC-OA-beta, canonical-order exchangeability, cohort-relative Min-K%++ tail enrichment, and cross-model top-K overlap. On the image side, 19.8% of SLAKE-En images have an unusually close neighbour in PMC-OA-beta under SigLIP-B-16 (4.2% under SigLIP-SO400M), while out-of-domain control images produce no flags. Manual inspection shows that these are images of the same modality and view from different patients, not exact copies, so we read the signal as overlap in image sources rather than memorization of individual images. On the text side, Qwen2.5-VL shows an exchangeability signal on SLAKE-En that survives an ordering ablation and two non-medical baseline models, and on the OmniMedVQA mirror exchangeability fires for five medical and general VLMs while BLIP-2 does not. The other two detectors fail a simple control: BLIP-2, which has no plausible exposure to medical VQA data, reproduces their apparent positive signals. Cohort-relative Min-K%++ and top-K overlap are therefore unreliable as standalone contamination signals for small groups of medical VLMs and should be checked against an outside baseline model.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
Continuous Neural Reparameterization as a Deep Geometric Prior for Robust Fixed-Chart UV Repair
Mohammad Sadegh Salehi
Traditional UV unwrapping relies on direct optimization of
geometric distortion energies and can fail through invalid initialization, local minima, or topological foldovers. We recast fixed-chart UV unwrapping as continuous neural reparameterization: an untrained SIREN maps per-vertex mesh features to UV coordinates, and its weights are optimized for a
geometric objective. The practical contribution is a robust chart-solver recipe, combining Laplace–Beltrami spectral inputs, Tutte residual warm-up, a $C^2$ determinant extension, an injectivity barrier, and validity-checked retry/fallback routing, rather than a claim that any single component guarantees validity or that recutting methods should be replaced. NTK–LBO diagnostics show that spectral conditioning changes update geometry, especially at initialization and mid-rank subspaces, but does not by itself predict chart success. On compact pre-cut charts and a 47-chart stratified Thingi10K/xatlas-cut
benchmark, the neural solver produces zero flips on all compact charts and 42/47 valid zero-flip stratified solves. BFF and OptCuts comparisons sharpen the scope: recutting can be faster and lower-distortion when allowed, while the neural solver targets supplied-chart validity and validation-first atlas construction. On Amara Spatial generated meshes, the full atlas construction path gives packed-atlas coverage on a 25-asset set and 1000/1000 strict locally valid atlases with zero UV flips in a large-scale Rust atlas run after fallback routing.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
GHOST: Hierarchical Sub-Goal Policies for Generalizing Robot Manipulation
Sriram Krishna, Ben Eisner, Haotian Zhan et al. (+5 more)
We present GHOST, a framework for learning visuomotor manipulation policies that generalize beyond the training distribution. GHOST factorizes control into (i) a high-level policy that predicts the next sub-goal as a distribution over 3D end-effector poses from multi-view RGB-D observations, and (ii) a low-level goal-conditioned controller that executes embodiment-specific actions. To condition image-based policies on 3D goals, we introduce a simple spatial interface that projects predicted goals into the image plane and represents them as end-effector heatmaps. Across a suite of manipulation tasks, this hierarchical factorization consistently improves performance and robustness compared to a flat
Diffusion Policy. Further, we show that this hierarchical interface also makes it easy to incorporate human demonstrations without relying on (noisy) action retargeting. As sub-goals are largely embodiment-agnostic, we train the high-level policy on human
video to specify how learned skills should be applied and composed, while keeping the low-level policy trained purely on
robot data. This hierarchy enables adaptation to novel objects and task variations using a small number of human demonstrations.
Read the full abstract ▾
All authors Sriram Krishna, Ben Eisner, Haotian Zhan, Ying Yuan, Haoyu Zhen, Chuang Gan, Shubham Tulsiani, David Held
Computer Vision
SpineReport: Automated 3D Quantification and Reporting of Lumbar Spine Degeneration on MRI
Nathan Molinier, Adrian A. Marth, Reto Sutter et al. (+6 more)
Lumbar spine conditions are a leading cause of disability worldwide, yet reliable quantification of degeneration from MRI remains challenging. In clinical practice, analysis is predominantly performed in two dimensions (2D), as manual three-dimensional (3D) assessment is time-consuming. However, 2D measurements suffer from limited reproducibility, particularly when anatomical structures are not aligned with the imaging plane. Existing automated approaches are often restricted to 2D, rely on discrete grading, or lack robustness and interpretability. We introduce SpineReport, an open-source, fully automated framework for comprehensive 3D morphometric analysis of lumbar spine MRI. Leveraging robust anatomical
segmentations, the method extracts quantitative metrics from key structures, including the spinal canal, spinal cord, vertebrae, intervertebral discs, and foramina. These include both morphological and signal-based features, enabling cross-subject and longitudinal assessment. SpineReport further generates subject-specific reports that allow comparison with cohort distributions, improving interpretability and objective characterization of spinal morphology. Clinical relevance was evaluated against radiologist-reported severity grades for central canal, lateral recess, and foraminal stenosis. Metrics showed strong associations with central canal stenosis severity, with T2-weighted CSF signal providing the highest performance (AUC = 0.95). Canal AP diameter and area ratios also demonstrated strong correlations and high discriminative ability (AUC > 0.80). For lateral recess stenosis, associations were moderate, with lateral CSF signal being the most informative (AUC = 0.73). No significant associations were observed for foraminal stenosis despite robust region-of-interest extraction. SpineReport is released as an open-access tool: https://ivadomed.github.io/SpineReport/
Read the full abstract ▾
All authors Nathan Molinier, Adrian A. Marth, Reto Sutter, Christoph Germann, Jacob A. Connolly, Mathieu Guay-Paquet, Nathan D. Schilaty, Kenneth A. Weber II, Julien Cohen-Adad
Computer Vision
Generalized-CVO: Fast and Correspondence-Free Local Point Cloud Registration with Second Order Riemannian Optimization
Ray Zhang, Marcus Greiff, Thomas Lew et al. (+1 more)
We propose a fast and correspondence-free local point cloud registration method that leverages
geometric surface structure and reproducing kernel
Hilbert space (RKHS)
embeddings. The method represents point clouds as continuous functions with point-wise anisotropic kernels that encode local geometry. This formulation improves
alignment along surface normals while relaxing
alignment along tangential directions. To solve the resulting registration problem, we propose a second-order on-
manifold optimization scheme with approximate Riemannian Hessians, achieving a speedup of up to 10x over the first-order solvers used in prior correspondence-free RKHS-based methods. We demonstrate improved frame-to-frame
LiDAR and RGB-D tracking accuracy across diverse indoor and outdoor
datasets. On a
LiDAR tracking registration task in the driving domain, we achieve a reduction of $>55\%$ in both translational and rotational drift in challenging feature-sparse environments. On object registration
benchmarks, we show improved robustness over ICP-based methods and further gains when refining global initialization, particularly under moderate misalignment.
Read the full abstract ▾
All authors Ray Zhang, Marcus Greiff, Thomas Lew, John Subosits
Computer Vision
ABot-Earth 0.5: Generative 3D Earth Model
Ming Qian, Tianjian Ouyang, Mingchao Sun et al. (+25 more)
We present ABot-Earth 0.5, a generative 3D framework designed to synthesize vast, seamless 3D environments from ubiquitous, geospatially referenced satellite imagery. To achieve this, we propose a novel generative model formulated directly with the 3D Gaussian Splatting (3DGS) representation. The model is trained on a diverse corpus of existing real-world urban reconstructions, learning to generate realistic geometry and textures. At inference, it synthesizes novel 3D scenes conditioned solely on satellite imagery at a scalable rate of under 10 minutes per square kilometer, while demonstrating exceptional realism. The framework is designed for accessibility, with integrated hierarchical level-of-detail (LOD) structures that permit real-time, interactive visualization on web-based map engines. This high-fidelity simulation sandbox effectively mitigates the sim-to-real domain gap, enabling critical downstream
Embodied AI applications like closed-loop
UAV navigation. By providing an ultra-low-cost and high-efficiency solution, ABot-Earth 0.5 significantly lowers the technical and financial barriers to large-scale 3D reconstruction and empowers the future of global digital earth visualization.
Read the full abstract ▾
All authors Ming Qian, Tianjian Ouyang, Mingchao Sun, Zijian Wang, Jincheng Xiong, Jiarong Han, Yongchang Zhang, Jiawei Zhang, Xu Wang, Yu Liu, Luyang Tang, Fei Yu, Zengye Ge, Mengmeng Du, Yuan Liu, Nianfei Fan, Song Wang, Yingliang Peng, Chunxue Jia, Yang Liu, Shiying Zeng, Haozhe Shi, Junnan Lai, Hongyu Pan, Zheng Wu, Ning Guo, Mu Xu, Hang Zhang
Computer Vision · Cross-listed from cs.AR
SPARX: Secure and Privacy-Aware Approximate CNN Acceleration with Edge RISC-V SoC
Sonu Kumar, Akash Sankhe, Mukul Lokhande et al. (+1 more)
Edge-AI systems increasingly require real-time CNN inference under strict energy, performance, security, and
privacy constraints. Approximate computing improves hardware efficiency by exploiting the error resilience of neural network workloads; however, most approximate CNN accelerators do not jointly consider secure,
privacy-aware edge deployment. This paper presents SPARX, a Secure and
Privacy-Aware Approximate CNN Acceleration framework integrated within a heterogeneous RV32IMC RISC-V System-on-Chip (SoC). SPARX combines a custom RISC-V instruction extension, an approximate logarithmic CNN acceleration unit, a lightweight differential-noise-based
privacy engine, and a challenge-response authentication mechanism. To guide arithmetic selection, an approximation-aware decision framework is introduced that uses the Approximation Severity Index (ASI), Approximation Efficiency (AE), Quality of Approximation (QoA), Approximation Figure-of-Merit (AFOM), and Hardware Acceleration Efficiency (HAE). Evaluation across 11 state-of-the-art approximate MAC architectures identifies the Iterative Logarithmic Multiplier (ILM) as the most suitable design, achieving 51.7% area reduction, 81.5% power reduction, and 2.13x throughput improvement compared with an accurate radix-4 Booth MAC, while only reducing ResNet-20/CIFAR-10 accuracy by 2.82 percentage points. FPGA implementation on a Xilinx VC707 platform achieves 58.4 GOPS/W energy efficiency at 250 MHz, while 28-nm CMOS physical implementation validates ASIC feasibility
Read the full abstract ▾
All authors Sonu Kumar, Akash Sankhe, Mukul Lokhande, Santosh Kumar Vishvakarma
Computer Vision
Latent Spatial Memory for Video World Models
Weijie Wang, Haoyu Zhao, Yifan Yang et al. (+7 more)
Video world models that maintain 3D spatial consistency across generated frames typically rely on explicit point cloud
memory constructed in RGB space. This design is both computationally expensive, requiring repeated rendering and VAE encoding, and inherently lossy, as the round trip through pixel space discards rich features of the learned latent representation. In this paper, we introduce latent spatial
memory for
video world models, a persistent 3D cache that stores scene information directly in the
diffusion latent space, avoiding pixel-space reconstruction. Building on this, we propose Mirage, a latent-space spatial
memory framework that constructs the
memory by lifting latent
tokens into 3D via depth-guided back-projection and queries it by synthesizing novel views through direct latent-space warping. This unified formulation eliminates both the information loss of pixel-space reconstruction and the computational burden of repeated encoding and rendering. Experiments show that latent spatial
memory achieves up to 10.57$\times$ faster end-to-end
video generation and 55$\times$ reduction in
memory footprint relative to explicit 3D baselines. Leveraging the
geometric prior of the
diffusion model, Mirage attains state-of-the-art performance on WorldScore and strong reconstruction quality on RealEstate10K.
Read the full abstract ▾
All authors Weijie Wang, Haoyu Zhao, Yifan Yang, Feng Chen, Zeyu Zhang, Yefei He, Zicheng Duan, Donny Y. Chen, Yuqing Yang, Bohan Zhuang
Computer Vision · Cross-listed from cs.RO
MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Models
Hao Shi, Weiye Li, Bin Xie et al. (+6 more)
Temporal modeling is essential for robotic manipulation, as effective control requires both
memory of past interactions and imagination of future states. However, most VLA models rely primarily on the current observation and therefore struggle with long-horizon, temporally dependent tasks. Cognitive science suggests that humans rely on working
memory to buffer short-lived context, the hippocampal system to preserve episodic
memory of past experience, and internal models to imagine possible future state
evolution. Inspired by these mechanisms, we propose MemoryVLA++, a full temporal modeling framework that equips VLA models with
memory and imagination for robotic manipulation. A pretrained VLM encodes the current observation into perceptual and cognitive
tokens, forming working
memory. These
tokens query a Perceptual-Cognitive
Memory Bank to retrieve relevant historical context. This bank stores low-level details and high-level semantics from past interactions, and is updated through redundancy-aware consolidation. A world model imagines future states in a denoising latent space, and the imagined latents are integrated under
memory guidance to form full temporal-aware
tokens. The resulting
tokens condition a
diffusion action expert to predict temporally consistent action sequences. We conduct extensive experiments on 5 simulation
benchmarks and 3 categories of real-
robot tasks across 3
robots, covering general manipulation, long-horizon temporal tasks, robustness, and generalization. Our method achieves strong performance across Libero, SimplerEnv, Mikasa-Robo, Calvin, Libero-Plus, and diverse real-
robot tasks, validating the effectiveness of full temporal modeling with
memory and imagination. For example, on real
robots, it achieves +9%, +26%, +28% gains on general,
memory-dependent, and imagination-dependent tasks. Project Page: https://shihao1895.github.io/MemoryVLA-PP-Web
Read the full abstract ▾
All authors Hao Shi, Weiye Li, Bin Xie, Yulin Wang, Renping Zhou, Tiancai Wang, Xiangyu Zhang, Ping Luo, Gao Huang
Computer Vision
OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics
Mingxian Lin, Shengju Qian, Yuqi Liu et al. (+9 more)
Vision-
language model (VLM)
agents are increasingly deployed in interactive game environments. Yet game
benchmarks for VLM
agents typically report a single first-attempt score per (
agent, game) pair, focus on single-
agent Solo play, and lack unified protocols for evaluating heterogeneous
agent classes (commercial VLMs, open-weight VLMs, and specialized game policies) on the same footing. We address these gaps with OmniGameArena, a real-time
benchmark of twelve newly built Unreal Engine 5 games spanning Solo (7), PvP (3), and Coop (2) with unified action interfaces, and the Improvement Dynamics Curve (IDC), an agentic-reflection harness in which a tool-using reflector LLM autonomously refines a bounded skill
prompt across multiple rounds. Beyond cold-start leaderboard scores, IDC exposes two additional observables for each (
agent, game) pair: how the score evolves across reflection rounds, and how the learned skill behaves on held-out task variants. We report these observables for twelve VLM
agents on the cold-start leaderboard and four top
agents under IDC.
Read the full abstract ▾
All authors Mingxian Lin, Shengju Qian, Yuqi Liu, Yi-Hua Huang, Yiyu Wang, Wei Huang, Yitang Li, Fan Zhang, Zeyu Hu, Lingting Zhu, Xin Wang, Xiaojuan Qi
Computer Vision
Danqi Zhuang, Jisui Huang, Xiaoyue Xi et al. (+4 more)
Standard
diffusion models typically use a single time-homogeneous Gaussian terminal distribution as the reference law for generation. While this choice is analytically convenient and empirically powerful, it provides little explicit structure for data concentrated near low-dimensional
manifolds, where different regions of the data distribution may correspond to distinct local
geometric or semantic factors. As a result, the reverse model must recover
manifold-level structure almost entirely from an unstructured terminal reference distribution. We propose PTL-
Diffusion, a proof-of-concept
diffusion framework whose forward noising process converges to a nonconstant periodic family of Gaussian terminal laws rather than to a single invariant law. Unlike a phase-conditioned DDPM, where phase information only enters the denoising network while the forward process remains unchanged, PTL-
Diffusion embeds phase structure directly into the forward noising dynamics. The proposed construction remains close to standard denoising
diffusion models: for a periodically forced Ornstein–Uhlenbeck-type forward process, we derive closed-form forward marginals, the limiting periodic Gaussian terminal family, and explicit Gaussian reverse posteriors, enabling standard noise-prediction training. We also introduce an invariant-average regularization term coupling the phase-conditioned reverse dynamics through the averaged periodic reference law. Experiments on
torus and cylinder point-cloud
benchmarks and the Olivetti face
dataset show that PTL-
Diffusion improves
manifold-level distributional matching over matched DDPM baselines, reducing phase-conditioned errors, feature-space covariance errors, and nearest-neighbour
manifold distances. These results suggest structured terminal reference laws as a promising direction, while motivating more expressive phase constructions and larger-scale evaluations.
Read the full abstract ▾
All authors Danqi Zhuang, Jisui Huang, Xiaoyue Xi, Andrew Kiggins, Xiaojie Wang, Ke Chen, Yue Wu
Computer Vision · Cross-listed from cs.RO
iMaC: Translating Actions into Motion and Contact Images for Embodied World Models
Zhenyu Wu, Xiuwei Xu, Yukun Zhou et al. (+8 more)
Embodied world models have emerged as a pivotal paradigm for visual robotic decision-making and interactive environment simulation. However, conventional
embodied frameworks rely on low-dimensional structured action vectors (e.g., joint angles and end-effector poses), which suffer from limited expressive capacity, poor generalization across diverse embodiments, and unnatural dynamic modeling for complex physical interactions. To address these limitations, this paper proposesiMac (Image as Action Control), a novel unified control paradigm that treats raw visual images as native action representations for
embodied world models. Departing from traditional explicit kinematic action encoding, iMac formulates continuous visual manipulation as image-based action
tokens, which inherently encapsulate spatial motion intentions, interactive
geometric constraints and subtle physical dynamics. We construct a dual-branch
embodied architecture consisting of an image-action encoder and a dynamic world predictor: the encoder compresses target-driven visual images into compact action
embeddings, while the predictor learns environment transition rules conditioned on image actions to achieve high-fidelity future state prediction and closed-loop
embodied control. Extensive experiments are conducted on public
embodied manipulation
benchmarks and real-world robotic scenarios. The results demonstrate that iMac outperforms vector-based action control baselines in prediction accuracy, task success rate and cross-scene generalization ability. Moreover, our image-action design eliminates the reliance on manually defined action spaces, realizing flexible and universal control for heterogeneous
embodied agents. This work provides an innovative visual-action perspective for
embodied world models, offering a simple yet effective paradigm for scalable robotic perception and manipulation.
Read the full abstract ▾
All authors Zhenyu Wu, Xiuwei Xu, Yukun Zhou, Yifan Li, Qiuping Deng, Xiaofeng Wang, Zheng Zhu, Bingyao Yu, Ziwei Wang, Jiwen Lu, Haibin Yan
Computer Vision · Cross-listed from cs.RO
AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing
Jisong Cai, Long Ling, Shiwei Chu et al. (+10 more)
World-action models have emerged as a promising paradigm for
robot manipulation, jointly modeling visual scene dynamics and actions to inject physical priors into policy learning. However, existing world-action models couple world prediction and action execution at the same temporal resolution, forcing the world branch to model near-term frame variations that are redundant and weakly informative. We posit that strictly binding world prediction and action execution to the same temporal rhythm may underutilize the potential of the
video branch for
embodied control. Therefore, we propose AHA-WAM, an Asynchronous Horizon-Adaptive World-Action Model built on a dual
Diffusion Transformer (DiT) architecture that reorganizes world-action modeling around this temporal asymmetry. AHA-WAM instantiates the
video DiT as a low-frequency world planner that maintains rolling key-value
memory over past observations and exposes reusable layerwise latent context encoding long-horizon scene
evolution, while a high-frequency action DiT executes short action chunks in closed loop by querying this context through layerwise joint
attention. To support asynchronous execution, we introduce horizon-adaptive offset training and Observation-Guided
Video-Context Routing (OVCR), which together let the action expert exploit long-horizon world context while remaining responsive to real-time execution state without rerunning the
video DiT. Experiments on RoboTwin and real-world manipulation tasks show that AHA-WAM achieves state-of-the-art performance without any
robot-data pretraining, attaining 92.80% average success on RoboTwin and 78.3% success across 4 real-world tasks, while reaching 24.17 Hz closed-loop control with a 4.59x speedup over Fast-WAM.
Read the full abstract ▾
All authors Jisong Cai, Long Ling, Shiwei Chu, Zhongshan Liu, Jiayue Kang, Zhixuan Liang, Wenjie Xu, Yinan Mao, Weinan Zhang, Xiaokang Yang, Ru Ying, Ran Zheng, Yao Mu