Computer Vision · Cross-listed from cs.AI
DECKBench: Benchmarking Multi-Agent Frameworks for Academic Slide Generation and Editing
Daesik Jang, Morgan Lindsay Heisler, Linzi Xing et al. (+5 more)
Automatically generating and iteratively editing academic slide decks requires more than document summarization. It demands faithful content selection, coherent slide organization, layout-aware rendering, and robust multi-turn instruction following. However, existing
benchmarks and evaluation protocols do not adequately measure these challenges. To address this gap, we introduce the Deck Edits and Compliance Kit
Benchmark (DECKBench), an evaluation framework for multi-
agent slide generation and editing. DECKBench is built on a curated
dataset of paper to slide pairs augmented with realistic, simulated editing instructions. Our evaluation protocol systematically assesses slide-level and deck-level fidelity, coherence, layout quality, and multi-turn instruction following. We further implement a modular multi-
agent baseline system that decomposes the slide generation and editing task into paper parsing and summarization, slide planning, HTML creation, and iterative editing. Experimental results demonstrate that the proposed
benchmark highlights strengths, exposes failure modes, and provides actionable insights for improving multi-
agent slide generation and editing systems. Overall, this work establishes a standardized foundation for reproducible and comparable evaluation of academic presentation generation and editing. Code and data are publicly available at https://github.com/morgan-heisler/DeckBench .
Read the full abstract ▾
All authors Daesik Jang, Morgan Lindsay Heisler, Linzi Xing, Yifei Li, Edward Wang, Ying Xiong, Yong Zhang, Zhenan Fan
Computer Vision
IDPruner: Harmonizing Importance and Diversity in Visual Token Pruning for MLLMs
Yifan Tan, Yifu Sun, Shirui Huang et al. (+4 more)
Multimodal Large
Language Models (MLLMs) have demonstrated impressive capabilities, yet they encounter significant computational bottlenecks due to the massive volume of visual
tokens. Consequently, visual
token pruning, which substantially reduces the
token count, has emerged as a critical technique for accelerating MLLM inference. Existing approaches focus on
token importance, diversity, or an intuitive combination of both, without a principled framework for their optimal integration. To address this issue, we first conduct a systematic analysis to characterize the trade-off between
token importance and semantic diversity. Guided by this analysis, we propose the Importance and Diversity Pruner (IDPruner), which leverages the Maximal Marginal Relevance (MMR)
algorithm to achieve a Pareto-optimal balance between these two objectives. Crucially, our method operates without requiring
attention maps, ensuring full compatibility with FlashAttention and efficient deployment via one-shot
pruning. We conduct extensive experiments across various model architectures and
multimodal benchmarks, demonstrating that IDPruner achieves state-of-the-art performance and superior generalization across diverse architectures and tasks. Notably, on Qwen2.5-VL-7B-Instruct, IDPruner retains 95.18% of baseline performance when
pruning 75% of the
tokens, and still maintains 86.40% even under an extreme 90%
pruning ratio. Our code is available at https://github.com/Tencent/AngelSlim.
Read the full abstract ▾
All authors Yifan Tan, Yifu Sun, Shirui Huang, Hong Liu, Guanghua Yu, Jianchen Zhu, Yangdong Deng
Computer Vision
Sim2Radar: Toward Bridging the Radar Sim-to-Real Gap with VLM-Guided Scene Reconstruction
Emily Bejerano, Federico Tondolo, Ayaan Qayyum et al. (+2 more)
Millimeter-wave (mmWave) radar provides reliable perception in visually degraded indoor environments (e.g., smoke,
dust, and low light), but learning-based radar perception is bottlenecked by the scarcity and cost of collecting and annotating large-scale radar
datasets. We present Sim2Radar, an end-to-end framework that synthesizes training radar data directly from single-view RGB images, enabling scalable data generation without manual scene modeling. Sim2Radar reconstructs a material-aware 3D scene by combining
monocular depth estimation,
segmentation, and
vision-
language reasoning to infer object materials, then simulates mmWave propagation with a configurable physics-based ray tracer using Fresnel reflection models parameterized by ITU-R electromagnetic properties. Evaluated on real-world indoor scenes, Sim2Radar improves downstream 3D radar perception via transfer learning: pre-training a radar point-cloud object detection model on synthetic data and fine-tuning on real radar yields up to +3.7 3D AP (IoU 0.3), with gains driven primarily by improved spatial localization. These results suggest that physics-based,
vision-driven radar simulation can provide effective
geometric priors for radar learning and measurably improve performance under limited real-data supervision.
Read the full abstract ▾
All authors Emily Bejerano, Federico Tondolo, Ayaan Qayyum, Xiaofan Yu, Xiaofan Jiang
Computer Vision
Agentic Spatio-Temporal Grounding via Collaborative Reasoning
Heng Zhao, Yew-Soon Ong and Joey Tianyi Zhou
Spatio-Temporal
Video Grounding (STVG) aims to retrieve the spatio-temporal tube of a target object or person in a
video given a text query. Most existing approaches perform frame-wise spatial localization within a predicted temporal span, resulting in redundant computation, heavy supervision requirements, and limited generalization. Weakly-supervised variants mitigate annotation costs but remain constrained by the
dataset-level train-and-fit paradigm with an inferior performance. To address these challenges, we propose the Agentic Spatio-Temporal Grounder (ASTG) framework for the task of STVG towards an open-world and training-free scenario. Specifically, two specialized
agents SRA (Spatial
Reasoning Agent) and TRA (Temporal
Reasoning Agent) constructed leveraging on modern Multimoal Large
Language Models (MLLMs) work collaboratively to retrieve the target tube in an autonomous and self-guided manner. Following a propose-and-evaluation paradigm, ASTG duly decouples spatio-temporal
reasoning and automates the tube extraction, verification and temporal localization processes. With a dedicate visual
memory and dialogue context, the
retrieval efficiency is significantly enhanced. Experiments on popular
benchmarks demonstrate the superiority of the proposed approach where it outperforms existing weakly-supervised and zero-shot approaches by a margin and is comparable to some of the fully-supervised methods.
Read the full abstract ▾
Computer Vision
Visual Para-Thinker: Divide-and-Conquer Reasoning for Visual Comprehension
Haoran Xu, Hongyu Wang, Jiaze Li et al. (+5 more)
Existing LLM test-time scaling laws emphasize the emergence of self-reflective behaviors through extended
reasoning length. Nevertheless, this vertical scaling strategy often encounters plateaus in exploration as the model becomes locked into specific thinking pattern. By shifting from depth to parallelism, parallel thinking mitigates the narrowing of exploration. However, the extension of this paradigm to visual domain remains an open research question. In this paper, we first examine the role of visual partitioning in parallelized
reasoning and subsequently propose two distinct strategies. Based on the above, we introduce Visual Para-Thinker, representing the inaugural parallel
reasoning framework for MLLMs. To maintain path independence and promote diversity in
reasoning, our approach integrates Pa-
Attention alongside LPRoPE. Leveraging the vLLM framework, we have developed a native
multimodal implementation that facilitates high-efficiency parallel processing. Empirical results on
benchmark datasets such as V*, CountBench, RefCOCO, and HallusionBench confirm that Visual Para-Thinker successfully extends the benefits of parallel
reasoning to the visual domain.
Read the full abstract ▾
All authors Haoran Xu, Hongyu Wang, Jiaze Li, Shunpeng Chen, Zizhao Tong, Jianzhong Ju, Zhenbo Luo, Jian Luan
Computer Vision · Cross-listed from eess.IV
Learning to Select Like Humans: Explainable Active Learning for Medical Imaging
Ifrat Ikhtear Uddin, Longwei Wang, Xiao Qin et al. (+2 more)
Medical image analysis requires substantial labeled data for model training, yet expert annotation is expensive and time-consuming. Active learning (AL) addresses this challenge by strategically selecting the most informative samples for the annotation purpose, but traditional methods solely rely on predictive uncertainty while ignoring whether models learn from clinically meaningful features a critical requirement for clinical deployment. We propose an explainability-guided active learning framework that integrates spatial
attention alignment into a sample acquisition process. Our approach advocates for a dual-criterion selection strategy combining: (i) classification uncertainty to identify informative examples, and (ii)
attention misalignment with radiologist-defined regions-of-interest (ROIs) to target samples where the model focuses on incorrect features. By measuring misalignment between Grad-CAM
attention maps and expert annotations using Dice similarity, our acquisition function judiciously identifies samples that enhance both predictive performance and spatial interpretability. We evaluate the framework using three expert-annotated medical imaging
datasets, namely, BraTS (MRI
brain tumors), VinDr-CXR (chest X-rays), and SIIM-COVID-19 (chest X-rays). Using only 570 strategically selected samples, our explainability-guided approach consistently outperforms random sampling across all the
datasets, achieving 77.22% accuracy on BraTS, 52.37% on VinDr-CXR, and 52.66% on SIIM-COVID. Grad-CAM visualizations confirm that the models trained by our dual-criterion selection focus on diagnostically relevant regions, demonstrating that incorporating explanation guidance into sample acquisition yields superior data efficiency while maintaining clinical interpretability.
Read the full abstract ▾
All authors Ifrat Ikhtear Uddin, Longwei Wang, Xiao Qin, Yang Zhou, KC Santosh
Computer Vision · Cross-listed from cs.LG
UltraLIF: Fully Differentiable Spiking Neural Networks via Ultradiscretization and Max-Plus Algebra
Jose Marie Antonio Miñoza
Spiking Neural Networks (SNNs) offer energy-efficient, biologically plausible computation but suffer from non-differentiable spike generation, necessitating reliance on heuristic surrogate gradients. This paper introduces UltraLIF, a principled framework that replaces surrogate gradients with ultradiscretization, a mathematical formalism from tropical geometry providing continuous relaxations of discrete dynamics. The central insight is that the max-plus semiring underlying ultradiscretization naturally models neural threshold dynamics: the log-sum-exp function serves as a differentiable soft-maximum that converges to hard thresholding as a learnable temperature parameter $\eps \to 0$. Two
neuron models are derived from distinct dynamical systems: UltraLIF from the LIF ordinary differential equation (temporal dynamics) and UltraDLIF from the
diffusion equation modeling gap junction coupling across neuronal populations (spatial dynamics). Both yield fully differentiable SNNs trainable via standard backpropagation with no forward-backward mismatch. Theoretical analysis establishes pointwise convergence to classical LIF dynamics with quantitative error bounds and bounded non-vanishing gradients. Experiments on six
benchmarks spanning static images, neuromorphic
vision, and audio demonstrate improvements over surrogate gradient baselines, with gains most pronounced in single-timestep ($T{=}1$) settings on neuromorphic and temporal
datasets. An optional sparsity penalty enables significant energy reduction while maintaining competitive accuracy.
Read the full abstract ▾
Computer Vision
Comp2Comp: Open-Source Software with FDA-Cleared Artificial Intelligence Algorithms for Computed Tomography Image Analysis
Adrit Rao, Malte Jensen, Andrea T. Fisher et al. (+28 more)
Artificial intelligence allows automatic extraction of imaging biomarkers from already-acquired radiologic images. This paradigm of opportunistic imaging adds value to medical imaging without additional imaging costs or patient radiation exposure. However, many open-source image analysis solutions lack rigorous validation while commercial solutions lack transparency, leading to unexpected failures when deployed. Here, we report development and validation for two of the first fully open-sourced, FDA-510(k)-cleared deep learning pipelines to mitigate both challenges: Abdominal Aortic Quantification (AAQ) and Bone Mineral Density (BMD) estimation are both offered within the Comp2Comp package for opportunistic analysis of computed
tomography scans. AAQ segments the abdominal aorta to assess aneurysm size; BMD segments vertebral bodies to estimate trabecular bone density and osteoporosis risk. AAQ-derived maximal aortic diameters were compared against radiologist ground-truth measurements on 258 patient scans enriched for abdominal aortic aneurysms from four external institutions. BMD binary classifications (low vs. normal bone density) were compared against concurrent DXA scan ground truths obtained on 371 patient scans from four external institutions. AAQ had an overall mean absolute error of 1.57 mm (95% CI 1.38-1.80 mm). BMD had a sensitivity of 81.0% (95% CI 74.0-86.8%) and specificity of 78.4% (95% CI 72.3-83.7%). Comp2Comp AAQ and BMD demonstrated sufficient accuracy for clinical use. Open-sourcing these
algorithms improves transparency of typically opaque FDA clearance processes, allows hospitals to test the
algorithms before cumbersome clinical pilots, and provides researchers with best-in-class methods.
Read the full abstract ▾
All authors Adrit Rao, Malte Jensen, Andrea T. Fisher, Louis Blankemeier, Pauline Berens, Arash Fereydooni, Seth Lirette, Eren Alkan, Felipe C. Kitamura, Juan M. Zambrano Chaves, Eduardo Reis, Arjun Desai, Marc H. Willis, Jason Hom, Andrew Johnston, Leon Lenchik, Robert D. Boutin, Eduardo M. J. M. Farina, Augusto S. Serpa, Marcelo S. Takahashi, Jordan Perchik, Steven A. Rothenberg, Jamie L. Schroeder, Ross Filice, Leonardo K. Bittencourt, Hari Trivedi, Marly van Assen, John Mongan, Kimberly Kallianos, Oliver Aalami, Akshay S. Chaudhari
Computer Vision · Cross-listed from q-bio.NC
ENIGMA: EEG-to-Image in 15 Minutes Using Less Than 1% of the Parameters
Reese Kneeland, Wangshu Jiang, Ugo Bruzadin Nunes et al. (+3 more)
To be practical for real-life applications, models for
brain-computer interfaces must be easily and quickly deployable on new subjects, effective on affordable scanning hardware, and small enough to run locally on accessible computing resources. To directly address these current limitations, we introduce ENIGMA, a multi-subject electroencephalography (
EEG)-to-Image decoding model that reconstructs seen images from
EEG recordings and achieves state-of-the-art (SOTA) performance on the research-grade THINGS-
EEG2 and consumer-grade AllJoined-1.6M
benchmarks, while fine-tuning effectively on new subjects with as little as 15 minutes of data. ENIGMA boasts a simpler architecture and requires less than 1% of the trainable parameters necessary for previous approaches. Our approach integrates a subject-unified spatio-temporal backbone along with a set of multi-subject latent
alignment layers and an MLP projector to map raw
EEG signals to a rich visual latent space. We evaluate our approach using a broad suite of image reconstruction metrics that have been standardized in the adjacent field of fMRI-to-Image research, and we describe the first
EEG-to-Image study to conduct extensive behavioral evaluations of our reconstructions using human raters. Our simple and robust architecture provides a significant performance boost across both research-grade and consumer-grade
EEG hardware, and a substantial improvement in fine-tuning efficiency and inference cost. Finally, we provide extensive ablations to determine the architectural choices most responsible for our performance gains in both single and multi-subject cases across multiple
benchmark datasets. Collectively, our work provides a substantial step towards the development of practical
brain-computer interface applications.
Read the full abstract ▾
All authors Reese Kneeland, Wangshu Jiang, Ugo Bruzadin Nunes, Paul Steven Scotti, Arnaud Delorme, Jonathan Xu
Computer Vision · Cross-listed from eess.IV
Beyond Calibration: Confounding Pathology Limits Foundation Model Specificity in Abdominal Trauma CT
Jineel H Raythatha, Shuchang Ye, Jeremy Hsu et al. (+1 more)
Purpose: Translating foundation models into clinical practice requires evaluating their performance under compound distribution shift, where severe class imbalance coexists with heterogeneous imaging appearances. This challenge is relevant for traumatic bowel injury, a rare but high-mortality diagnosis. We investigated whether specificity deficits in foundation models are associated with heterogeneity in the negative class. Methods: This retrospective study used the multi-institutional, RSNA Abdominal Traumatic Injury CT
dataset (2019-2023), comprising scans from 23 centres. Two foundation models (MedCLIP, zero-shot; RadDINO, linear probe) were compared against three task-specific approaches (CNN,
Transformer, Ensemble). Models were trained on 3,147 patients (2.3% bowel injury prevalence) and evaluated on an enriched 100-patient test set. To isolate negative-class effects, specificity was assessed in patients without bowel injury who had concurrent solid organ injury (n=58) versus no abdominal pathology (n=50). Results: Foundation models achieved equivalent discrimination to task-specific models (AUC, 0.64-0.68 versus 0.58-0.64) with higher sensitivity (79-91% vs 41-74%) but lower specificity (33-50% vs 50-88%). All models demonstrated high specificity in patients without abdominal pathology (84-100%). When solid organ injuries were present, specificity declined substantially for foundation models (50-51 percentage points) compared with smaller reductions of 12-41 percentage points for task-specific models. Conclusion: Foundation models matched task-specific discrimination without task-specific training, but their specificity deficits were driven primarily by confounding negative-class heterogeneity rather than prevalence alone. Susceptibility to negative-class heterogeneity decreased progressively with labelled training, suggesting adaptation is required before clinical implementation.
Read the full abstract ▾
All authors Jineel H Raythatha, Shuchang Ye, Jeremy Hsu, Jinman Kim
Computer Vision
Monte Carlo Maximum Likelihood Reconstruction for Digital Holography with Speckle
Xi Chen, Arian Maleki and Shirin Jalali
In coherent imaging, speckle is statistically modeled as multiplicative noise, posing a fundamental challenge for image reconstruction. While maximum likelihood estimation (MLE) provides a principled framework for speckle mitigation, its application to coherent imaging system such as digital holography with finite apertures is hindered by the prohibitive cost of high-dimensional
matrix inversion, especially at high resolutions. This computational burden has prevented the use of MLE-based reconstruction with physically accurate aperture modeling. In this work, we propose a randomized linear
algebra approach that enables scalable MLE optimization without explicit
matrix inversions in gradient computation. By exploiting the structural properties of sensing
matrix and using conjugate gradient for likelihood gradient evaluation, the proposed
algorithm supports accurate aperture modeling without the simplifying assumptions commonly imposed for tractability. We term the resulting method projected gradient descent with Monte Carlo estimation (PGD-MC). The proposed PGD-MC framework (i) demonstrates robustness to diverse and physically accurate aperture models, (ii) achieves substantial improvements in reconstruction quality and computational efficiency, and (iii) scales effectively to high-resolution digital holography. Extensive experiments incorporating three representative denoisers as regularization show that PGD-MC provides a flexible and effective MLE-based reconstruction framework for digital holography with finite apertures, consistently outperforming prior Plug-and-Play model-based iterative reconstruction methods in both accuracy and speed. Our code is available at: https://github.com/Computational-Imaging-RU/MC_Maximum_Likelihood_Digital_Holography_Speckle.
Read the full abstract ▾
Computer Vision
Conditional Uncertainty-Aware Political Deepfake Detection with Stochastic Convolutional Neural Networks
Rafael-Petruţ Gardoş
Recent advances in generative image models have enabled the creation of highly realistic political deepfakes, posing risks to information integrity, public trust, and democratic processes. While automated deepfake detectors are increasingly deployed in moderation and investigative pipelines, most existing systems provide only point predictions and fail to indicate when outputs are unreliable, being an operationally critical limitation in high-stakes political contexts. This work investigates conditional, uncertainty-aware political deepfake detection using
stochastic convolutional neural networks within an empirical, decision-oriented reliability framework. Rather than treating uncertainty as a purely
Bayesian construct, it is evaluated through observable criteria, including calibration quality, proper scoring rules, and its
alignment with prediction errors under both global and confidence-conditioned analyses. A politically focused binary image
dataset is constructed via deterministic metadata filtering from a large public real-synthetic corpus. Two pretrained CNN backbones (ResNet-18 and EfficientNet-B4) are fully fine-tuned for classification. Deterministic inference is compared with single-pass
stochastic prediction, Monte Carlo dropout with multiple forward passes, temperature scaling, and ensemble-based uncertainty surrogates. Evaluation reports ROC-AUC, thresholded confusion
matrices, calibration metrics, and generator-disjoint out-of-distribution performance. Results demonstrate that calibrated probabilistic outputs and uncertainty estimates enable risk-aware moderation policies. A systematic confidence-band analysis further clarifies when uncertainty provides operational value beyond predicted confidence, delineating both the benefits and limitations of uncertainty-aware deepfake detection in political settings.
Read the full abstract ▾
Computer Vision
Flow Matching with Uncertainty Quantification and Guidance
Juyeop Han, Lukas Lao Beyer and Sertac Karaman
Despite the remarkable success of sampling-based generative models such as flow matching, they can still produce samples of inconsistent or degraded quality. To assess sample reliability and improve sample fidelity, we propose uncertainty-aware flow matching (UA-Flow), a lightweight extension of flow matching that predicts the velocity field together with heteroscedastic uncertainty. UA-Flow estimates spatially localized sample uncertainty by propagating velocity uncertainty through the flow dynamics. The resulting estimates serve as reliability signals that reflect data-
manifold support for each generated sample or region. We further use them to steer generation via uncertainty-aware guidance (UG) and uncertainty-aware classifier-free guidance (UCFG). Experiments on ImageNet and CIFAR-10 show that UA-Flow's uncertainty tracks sample fidelity more consistently than element-wise baselines and that these signals also carry information about data-
manifold support. On ImageNet, UG increases precision at the cost of recall and improves FID, while UCFG maintains lower FID than CFG at large guidance scales.
Read the full abstract ▾
Computer Vision
A Low-Rank Defense Method for Adversarial Attack on Diffusion Models
Jiaxuan Zhu and Siyu Huang
Recently,
adversarial attacks for
diffusion models as well as their fine-tuning process have been developed rapidly. To prevent the abuse of these attack
algorithms from affecting the practical application of
diffusion models, it is critical to develop corresponding defensive strategies. In this work, we propose an efficient defensive strategy, named Low-Rank Defense (LoRD), to defend the
adversarial attack on Latent
Diffusion Models (LDMs). LoRD introduces the merging idea and a balance parameter, combined with the low-rank adaptation (LoRA) modules, to detect and defend the
adversarial samples. Based on LoRD, we build up a defense pipeline that applies the learned LoRD modules to help
diffusion models defend against attack
algorithms. Our method ensures that the LDM fine-tuned on both
adversarial and clean samples can still generate high-quality images. To demonstrate the effectiveness of our approach, we conduct extensive experiments on facial and landscape images, and our method shows significantly better defense performance compared to the baseline methods.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Uncertainty-Aware Ordinal Deep Learning for cross-Dataset Diabetic Retinopathy Grading
Ali El Bellaj, Aya Benradi, Salman El Youssoufi et al. (+2 more)
Diabetes mellitus is a chronic metabolic disorder characterized by persistent hyperglycemia due to insufficient insulin production or impaired insulin utilization. One of its most severe complications is diabetic retinopathy (DR), a progressive retinal disease caused by microvascular damage, leading to hemorrhages, exudates, and potential
vision loss. Early and reliable detection of DR is therefore critical for preventing irreversible blindness. In this work, we propose an uncertainty-aware deep learning framework for automated DR severity grading that explicitly models the ordinal nature of disease progression. Our approach combines a convolutional backbone with lesion-query
attention pooling and an evidential Dirichlet-based ordinal regression head, enabling both accurate severity prediction and principled estimation of predictive uncertainty. The model is trained using an ordinal evidential loss with annealed regularization to encourage calibrated confidence under domain shift. We evaluate the proposed method on a multi-domain training setup combining APTOS, Messidor-2, and a subset of EyePACS fundus
datasets. Experimental results demonstrate strong cross-
dataset generalization, achieving competitive classification accuracy and high quadratic weighted kappa on held-out test sets, while providing meaningful uncertainty estimates for low-confidence cases. These results suggest that ordinal evidential learning is a promising direction for robust and clinically reliable diabetic retinopathy grading.
Read the full abstract ▾
All authors Ali El Bellaj, Aya Benradi, Salman El Youssoufi, Taha El Marzouki, Mohammed-Amine Cheddadi
Computer Vision
ERGO: Excess-Risk-Guided Optimization for High-Fidelity Monocular 3D Gaussian Splatting
Zehua Ma, Hanhui Li, Zhenyu Xie et al. (+4 more)
Generating 3D content from a single image remains a fundamentally challenging and ill-posed problem due to the inherent absence of
geometric and textural information in occluded regions. While state-of-the-art generative models can synthesize auxiliary views to provide additional supervision, these views inevitably contain
geometric inconsistencies and textural misalignments that propagate and amplify artifacts during 3D reconstruction. To effectively harness these imperfect supervisory signals, we propose an adaptive optimization framework guided by excess risk decomposition, termed ERGO. Specifically, ERGO decomposes the optimization losses in 3D Gaussian splatting into two components, i.e., excess risk that quantifies the suboptimality gap between current and optimal parameters, and Bayes error that models the irreducible noise inherent in synthesized views. This decomposition enables ERGO to dynamically estimate the view-specific excess risk and adaptively adjust loss weights during optimization. Furthermore, we introduce geometry-aware and texture-aware objectives that complement the excess-risk-derived weighting mechanism, establishing a synergistic global-local optimization paradigm. Consequently, ERGO demonstrates robustness against supervision noise while consistently enhancing both
geometric fidelity and textural quality of the reconstructed 3D content. Extensive experiments on the Google Scanned Objects
dataset and the OmniObject3D
dataset demonstrate the superiority of ERGO over existing state-of-the-art methods.
Read the full abstract ▾
All authors Zehua Ma, Hanhui Li, Zhenyu Xie, Xiaonan Luo, Michael Kampffmeyer, Feng Gao, Xiaodan Liang
Computer Vision
Colorimeter-Supervised Skin Tone Estimation from Dermatoscopic Images for Fairness Auditing
Marin Benčević, Krešimir Romić, Ivana Hartmann Tolić et al. (+1 more)
Neural-network-based diagnosis from dermatoscopic images is increasingly used for clinical decision support, yet studies report performance disparities across skin tones.
Fairness auditing of these models is limited by the lack of reliable skin-tone annotations in public dermatoscopy
datasets. We address this gap with neural networks that predict Fitzpatrick skin type via ordinal regression and the Individual Typology Angle (ITA) via color regression, using in-person Fitzpatrick labels and colorimeter measurements as targets. We further leverage extensive pretraining on synthetic and real dermatoscopic and clinical images. The Fitzpatrick model achieves agreement comparable to human crowdsourced annotations, and ITA predictions show high concordance with colorimeter-derived ITA, substantially outperforming pixel-averaging approaches. Applying these
estimators to ISIC 2020 and MILK10k, we find that fewer than 1% of subjects belong to Fitzpatrick types V and VI. We release code and pretrained models as an open-source tool for rapid skin-tone annotation and bias
auditing. This is, to our knowledge, the first dermatoscopic skin-tone estimation neural network validated against colorimeter measurements, and it supports growing evidence of clinically relevant performance gaps across skin-tone groups.
Read the full abstract ▾
All authors Marin Benčević, Krešimir Romić, Ivana Hartmann Tolić, Irena Galić
Computer Vision
PMMA: The Polytechnique Montreal Mobility Aids Dataset
Qingwu Liu, Nicolas Saunier and Guillaume-Alexandre Bilodeau
This study introduces a new object detection
dataset of pedestrians using mobility aids, named PMMA. The
dataset was collected in an outdoor environment, where volunteers used wheelchairs, canes, and walkers, resulting in nine categories of pedestrians: pedestrians, cane users, two types of walker users, whether walking or resting, five types of wheelchair users, including wheelchair users, people pushing empty wheelchairs, and three types of users pushing occupied wheelchairs, including the entire pushing group, the pusher and the person seated on the wheelchair. To establish a
benchmark, seven object detection models (Faster R-CNN, CenterNet, YOLOX, DETR, Deformable DETR, DINO, and RT-DETR) and three tracking
algorithms (ByteTrack, BOT-SORT, and OC-SORT) were implemented under the MMDetection framework. Experimental results show that YOLOX, Deformable DETR, and Faster R-CNN achieve the best detection performance, while the differences among the three trackers are relatively small. The PMMA
dataset is publicly available at https://doi.org/10.5683/SP3/XJPQUG, and the
video processing and model training code is available at https://github.com/DatasetPMMA/PMMA.
Read the full abstract ▾
Computer Vision
XSPLAIN: XAI-enabling Splat-based Prototype Learning for Attribute-aware INterpretability
Dominik Galus, Julia Farganus, Tymoteusz Zapala et al. (+4 more)
3D Gaussian Splatting (3DGS) has rapidly become a standard for high-fidelity 3D reconstruction, yet its adoption in multiple critical domains is hindered by the lack of interpretability of the generation models as well as classification of the Splats. While explainability methods exist for other 3D representations, like point clouds, they typically rely on ambiguous saliency maps that fail to capture the volumetric coherence of Gaussian primitives. We introduce XSPLAIN, the first ante-hoc, prototype-based interpretability framework designed specifically for 3DGS classification. Our approach leverages a voxel-aggregated PointNet backbone and a novel, invertible orthogonal transformation that disentangles feature channels for interpretability while strictly preserving the original decision boundaries. Explanations are grounded in representative training examples, enabling intuitive “this looks like that”
reasoning without any degradation in classification performance. A rigorous user study (N=51) demonstrates a decisive preference for our approach: participants selected XSPLAIN explanations 48.4% of the time as the best, significantly outperforming baselines $(p<0.001)$, showing that XSPLAIN provides transparency and user trust. The source code for this work is available at: https://github.com/Solvro/ml-splat-xai
Read the full abstract ▾
All authors Dominik Galus, Julia Farganus, Tymoteusz Zapala, Mikołaj Czachorowski, Piotr Borycki, Przemysław Spurek, Piotr Syga
Computer Vision
DEGMC: Denoising Diffusion Models Based on Riemannian Equivariant Group Morphological Convolutions
El Hadji S. Diop, Thierno Fall and Mohamed Daoudi
In this work, we address two major issues in recent Denoising
Diffusion Probabilistic Models (DDPM): 1)
geometric key feature extraction and 2) network equivariance. Since the DDPM prediction network relies on the U-net architecture, which is theoretically only translation equivariant, we introduce a
geometric approach combined with an equivariance property of the more general Euclidean group, which includes rotations, reflections, and permutations. We introduce the notion of group morphological convolutions in Riemannian
manifolds, which are derived from the viscosity solutions of first-order Hamilton-Jacobi-type partial differential equations (PDEs) that act as morphological multiscale dilations and erosions. We add a convection term to the model and solve it using the method of characteristics. This helps us better capture nonlinearities, represent thin
geometric structures, and incorporate
symmetries into the learning process. Experimental results on the MNIST, RotoMNIST, and CIFAR-10
datasets show noticeable improvements compared to the baseline DDPM model.
Read the full abstract ▾
Computer Vision
When the Prompt Becomes Visual: Vision-Centric Jailbreak Attacks for Large Image Editing Models
Jiacheng Hou, Yining Sun, Ruochong Jin et al. (+4 more)
Recent advances in large image editing models have shifted the paradigm from text-driven instructions to
vision-
prompt editing, where user intent is inferred directly from visual inputs such as marks, arrows, and visual-text
prompts. While this paradigm greatly expands usability, it also introduces a critical and underexplored safety risk: the attack surface itself becomes visual. In this work, we propose
Vision-Centric Jailbreak Attack (VJA), the first visual-to-visual jailbreak attack that conveys malicious instructions purely through visual inputs. To systematically study this emerging threat, we introduce IESBench, a safety-oriented
benchmark for image editing models. Extensive experiments on IESBench demonstrate that VJA effectively compromises state-of-the-art commercial models, achieving attack success rates of up to 80.9% on Nano Banana Pro and 70.1% on GPT-Image-1.5. To mitigate this vulnerability, we propose a training-free defense based on introspective
multimodal reasoning, which substantially improves the safety of poorly aligned models to a level comparable with commercial systems, without auxiliary guard models and with negligible computational overhead. Our findings expose new vulnerabilities, provide both a
benchmark and practical defense to advance safe and trustworthy modern image editing systems. Warning: This paper contains offensive images created by large image editing models.
Read the full abstract ▾
All authors Jiacheng Hou, Yining Sun, Ruochong Jin, Haochen Han, Fangming Liu, Wai Kin Victor Chan, Alex Jinpeng Wang
Computer Vision
ArtisanGS: Interactive Tools for Gaussian Splat Selection with AI and Human in the Loop
Clement Fuji Tsang, Anita Hu, Or Perel et al. (+2 more)
Representation in the family of 3D Gaussian Splats (3DGS) are growing into a viable alternative to traditional graphics for an expanding number of application, including recent techniques that facilitate physics simulation and animation. However, extracting usable objects from in-the-wild captures remains challenging and controllable editing techniques for this representation are limited. Unlike the bulk of emerging techniques, focused on automatic solutions or high-level editing, we introduce an interactive suite of tools centered around versatile Gaussian Splat selection and
segmentation. We propose a fast AI-driven method to propagate user-guided 2D selection masks to 3DGS selections. This technique allows for user intervention in the case of errors and is further coupled with flexible manual selection and
segmentation tools. These allow a user to achieve virtually any binary
segmentation of an unstructured 3DGS scene. We evaluate our toolset against the state-of-the-art for Gaussian Splat selection and demonstrate their utility for downstream applications by developing a user-guided local editing approach, leveraging a custom
Video Diffusion Model. With flexible selection tools, users have direct control over the areas that the AI can modify. Our selection and editing tools can be used for any in-the-wild capture without additional optimization.
Read the full abstract ▾
All authors Clement Fuji Tsang, Anita Hu, Or Perel, Carsten Kolve, Maria Shugrina
Computer Vision
AD$^2$: Analysis and Detection of Adversarial Threats in Visual Perception for End-to-End Autonomous Driving Systems
Ishan Sahu, Somnath Hazra, Somak Aditya et al. (+1 more)
End-to-end autonomous driving systems have achieved significant progress, yet their
adversarial robustness remains largely underexplored. In this work, we conduct a closed-loop evaluation of state-of-the-art autonomous driving
agents under black-box
adversarial threat models in CARLA. Specifically, we consider three representative attack vectors on the visual perception pipeline: (i) a physics-based blur attack induced by acoustic waves, (ii) an electromagnetic interference attack that distorts captured images, and (iii) a digital attack that adds ghost objects as carefully crafted bounded perturbations on images. Our experiments on two advanced
agents, Transfuser and Interfuser, reveal severe vulnerabilities to such attacks, with driving scores dropping by up to 99% in the worst case, raising valid safety concerns. To help mitigate such threats, we further propose a lightweight Attack Detection model for Autonomous Driving systems (AD$^2$) based on
attention mechanisms that capture spatial-temporal consistency. Comprehensive experiments across multi-camera inputs on CARLA show that our detector achieves superior detection capability and computational efficiency compared to existing approaches.
Read the full abstract ▾
All authors Ishan Sahu, Somnath Hazra, Somak Aditya, Soumyajit Dey
Computer Vision
Beyond Closed-Pool Video Retrieval: A Benchmark and Agent Framework for Real-World Video Search and Moment Localization
Tao Yu, Yujia Yang, Haopeng Jin et al. (+17 more)
Traditional
video retrieval benchmarks focus on matching precise descriptions to closed
video pools, failing to reflect real-world searches characterized by fuzzy, multi-dimensional
memories on the open web. We present RVMS-Bench, a comprehensive system for evaluating real-world
video memory search. It consists of 1,440 samples spanning 20 diverse categories and four duration groups, sourced from real-world open-web
videos. RVMS-Bench utilizes a hierarchical description framework encompassing Global Impression, Key Moment, Temporal Context, and Auditory
Memory to mimic realistic multi-dimensional search cues, with all samples strictly verified via a human-in-the-loop protocol. We further propose RACLO, an agentic framework that employs abductive
reasoning to simulate the human “Recall-Search-Verify” cognitive process, effectively addressing the challenge of searching for
videos via fuzzy
memories in the real world. Experiments reveal that existing MLLMs still demonstrate insufficient capabilities in real-world
Video Retrieval and Moment Localization based on fuzzy
memories. We believe this work will facilitate the advancement of
video retrieval robustness in real-world unstructured scenarios.
Read the full abstract ▾
All authors Tao Yu, Yujia Yang, Haopeng Jin, Junhao Gong, Xinlong Chen, Yuxuan Zhou, Shanbin Zhang, Jiabing Yang, Xinming Wang, Hongzhu Yi, Ping Nie, Kai Zou, Zhang Zhang, Yan Huang, Liang Wang, Yeshani, Ruiwen Tao, Jin Ma, Haijin Liang, Jinwen Luo
Computer Vision
SAGE: Scalable Agentic 3D Scene Generation for Embodied AI
Hongchi Xia, Xuan Li, Zhaoshuo Li et al. (+9 more)
Real-world data collection for
embodied agents remains costly and unsafe, calling for scalable, realistic, and simulator-ready 3D environments. However, existing scene-generation systems often rely on rule-based or task-specific pipelines, yielding artifacts and physically invalid scenes. We present SAGE, an agentic framework that, given a user-specified
embodied task (e.g., "pick up a bowl and place it on the table"), understands the intent and automatically generates simulation-ready environments at scale. The
agent couples multiple generators for layout and object composition with critics that evaluate semantic plausibility, visual realism, and physical stability. Through iterative
reasoning and adaptive tool selection, it self-refines the scenes until meeting user intent and physical validity. The resulting environments are realistic, diverse, and directly deployable in modern simulators for policy training. Policies trained purely on this data exhibit clear scaling trends and generalize to unseen objects and layouts, demonstrating the promise of simulation-driven scaling for
embodied AI. Code, demos, and the SAGE-10k
dataset can be found on the project page here: https://research.nvidia.com/labs/dir/sage/.
Read the full abstract ▾
All authors Hongchi Xia, Xuan Li, Zhaoshuo Li, Qianli Ma, Jiashu Xu, Ming-Yu Liu, Yin Cui, Tsung-Yi Lin, Wei-Chiu Ma, Shenlong Wang, Shuran Song, Fangyin Wei
Computer Vision
Quantum Multiple Rotation Averaging
Shuteng Wang, Natacha Kuete Meli, Michael Möller et al. (+1 more)
Multiple rotation averaging (MRA) is a fundamental optimization problem in 3D
vision and robotics that aims to recover globally consistent absolute rotations from noisy relative measurements. Established classical methods, such as L1-IRLS and Shonan, face limitations including local minima susceptibility and reliance on convex relaxations that fail to preserve the exact
manifold geometry, leading to reduced accuracy in high-noise scenarios. We introduce IQARS (Iterative
Quantum Annealing for Rotation Synchronization), the first
algorithm that reformulates MRA as a sequence of local quadratic non-convex sub-problems executable on
quantum annealers after binarization, to leverage inherent hardware advantages. IQARS removes convex relaxation dependence and better preserves non-Euclidean rotation
manifold geometry while leveraging
quantum tunneling and parallelism for efficient solution space exploration. We evaluate IQARS's performance on synthetic and real-world
datasets. While current annealers remain in their nascent phase and only support solving problems of limited scale with constrained performance, we observed that IQARS on D-Wave annealers can already achieve ca. 12% higher accuracy than Shonan, i.e., the best-performing classical method evaluated empirically.
Read the full abstract ▾
All authors Shuteng Wang, Natacha Kuete Meli, Michael Möller, Vladislav Golyanik
Computer Vision
ConsID-Gen: View-Consistent and Identity-Preserving Image-to-Video Generation
Mingyang Wu, Ashirbad Mishra, Soumik Dey et al. (+5 more)
Image-to-
Video generation (I2V) animates a static image into a temporally coherent
video sequence following textual instructions, yet preserving fine-grained object identity under changing viewpoints remains a persistent challenge. Unlike text-to-
video models, existing I2V pipelines often suffer from appearance drift and
geometric distortion, artifacts we attribute to the sparsity of single-view 2D observations and weak cross-modal
alignment. Here we address this problem from both data and model perspectives. First, we curate ConsIDVid, a large-scale object-centric
dataset built with a scalable pipeline for high-quality, temporally aligned
videos, and establish ConsIDVid-Bench, where we present a novel benchmarking and evaluation framework for multi-view consistency using metrics sensitive to subtle
geometric and appearance deviations. We further propose ConsID-Gen, a view-assisted I2V generation framework that augments the first frame with unposed auxiliary views and fuses semantic and structural cues via a dual-stream visual-
geometric encoder as well as a text-visual connector, yielding unified conditioning for a
Diffusion Transformer backbone. Experiments across ConsIDVid-Bench demonstrate that ConsID-Gen consistently outperforms in multiple metrics, with the best overall performance surpassing leading
video generation models like Wan2.1 and HunyuanVideo, delivering superior identity fidelity and temporal coherence under challenging real-world scenarios. We will release our model and
dataset at https://myangwu.github.io/ConsID-Gen.
Read the full abstract ▾
All authors Mingyang Wu, Ashirbad Mishra, Soumik Dey, Shuo Xing, Naveen Ravipati, Hansi Wu, Binbin Li, Zhengzhong Tu
Computer Vision
Olaf-World: Orienting Latent Actions for Video World Modeling
Yuxin Jiang, Yuchao Gu, Ivor W. Tsang et al. (+1 more)
Scaling action-controllable world models is limited by the scarcity of action labels. While latent action learning promises to extract control interfaces from unlabeled
video, learned latents often fail to transfer across contexts: they entangle scene-specific cues and lack a shared coordinate system. This occurs because standard objectives operate only within each clip, providing no mechanism to align action semantics across contexts. Our key insight is that although actions are unobserved, their semantic effects are observable and can serve as a shared reference. We introduce Seq$\Delta$-REPA, a sequence-level control-effect
alignment objective that anchors integrated latent action to temporal feature differences from a frozen, self-supervised
video encoder. Building on this, we present Olaf-World, a pipeline that pretrains action-conditioned
video world models from large-scale passive
video. Extensive experiments demonstrate that our method learns a more structured latent action space, leading to stronger zero-shot action transfer and more data-efficient adaptation to new control interfaces than state-of-the-art baselines.
Read the full abstract ▾
All authors Yuxin Jiang, Yuchao Gu, Ivor W. Tsang, Mike Zheng Shou
Computer Vision
VideoWorld 2: Learning Transferable Knowledge from Real-world Videos
Zhongwei Ren, Yunchao Wei, Xiao Yu et al. (+5 more)
Learning transferable knowledge from unlabeled
video data and applying it in new environments is a fundamental capability of intelligent
agents. This work presents VideoWorld 2, which extends VideoWorld and offers the first investigation into learning transferable knowledge directly from raw real-world
videos. At its core, VideoWorld 2 introduces a dynamic-enhanced Latent Dynamics Model (dLDM) that decouples action dynamics from visual appearance: a pretrained
video diffusion model handles visual appearance modeling, enabling the dLDM to learn latent codes that focus on compact and meaningful task-related dynamics. These latent codes are then modeled autoregressively to learn task policies and support long-horizon
reasoning. We evaluate VideoWorld 2 on challenging real-world handcraft making tasks, where prior
video generation and latent-dynamics models struggle to operate reliably. Remarkably, VideoWorld 2 achieves up to 70% improvement in task success rate and produces coherent long execution
videos. In robotics, we show that VideoWorld 2 can acquire effective manipulation knowledge from the Open-X
dataset, which substantially improves task performance on CALVIN. This study reveals the potential of learning transferable world knowledge directly from raw
videos, with all code, data, and models to be open-sourced for further research.
Read the full abstract ▾
All authors Zhongwei Ren, Yunchao Wei, Xiao Yu, Guixun Luo, Yao Zhao, Bingyi Kang, Jiashi Feng, Xiaojie Jin
Computer Vision · Cross-listed from cs.LG
Learning on the Manifold: Unlocking Standard Diffusion Transformers with Representation Encoders
Amandeep Kumar and Vishal M. Patel
Leveraging representation encoders for generative modeling offers a path for efficient, high-fidelity synthesis. However, standard
diffusion transformers fail to converge on these representations directly. While recent work attributes this to a capacity bottleneck proposing computationally expensive width scaling of
diffusion transformers we demonstrate that the failure is fundamentally
geometric. We identify
Geometric Interference as the root cause: standard Euclidean flow matching forces probability paths through the low-density interior of the hyperspherical feature space of representation encoders, rather than following the
manifold surface. To resolve this, we propose Riemannian Flow Matching with Jacobi Regularization (RJF). By constraining the generative process to the
manifold geodesics and correcting for
curvature-induced error propagation, RJF enables standard
Diffusion Transformer architectures to converge without width scaling. Our method RJF enables the standard DiT-B architecture (131M parameters) to converge effectively, achieving an FID of 3.37 where prior methods fail to converge. Code: https://github.com/amandpkr/RJF
Read the full abstract ▾