Computer Vision
The Illusion of Clinical Reasoning: A Benchmark Reveals the Pervasive Gap in Vision-Language Models for Clinical Competency
Dingyu Wang, Zimu Yuan, Jiajun Liu et al. (+5 more)
Background: The rapid integration of foundation models into clinical practice and public health necessitates a rigorous evaluation of their true clinical
reasoning capabilities beyond narrow examination success. Current
benchmarks, typically based on medical licensing exams or curated vignettes, fail to capture the integrated,
multimodal reasoning essential for real-world patient care. Methods: We developed the Bones and Joints (B&J)
Benchmark, a comprehensive evaluation framework comprising 1,245 questions derived from real-world patient cases in orthopedics and sports medicine. This
benchmark assesses models across 7 tasks that mirror the clinical
reasoning pathway, including knowledge recall, text and image interpretation, diagnosis generation, treatment planning, and rationale provision. We evaluated eleven
vision-
language models (VLMs) and six large
language models (LLMs), comparing their performance against expert-derived ground truth. Results: Our results demonstrate a pronounced performance gap between task types. While state-of-the-art models achieved high accuracy, exceeding 90%, on structured multiple-choice questions, their performance markedly declined on open-ended tasks requiring
multimodal integration, with accuracy scarcely reaching 60%. VLMs demonstrated substantial limitations in interpreting medical images and frequently exhibited severe text-driven
hallucinations, often ignoring contradictory visual evidence. Notably, models specifically fine-tuned for medical applications showed no consistent advantage over general-purpose counterparts. Conclusions: Current artificial intelligence models are not yet clinically competent for complex,
multimodal reasoning. Their safe deployment should currently be limited to supportive, text-based roles. Future advancement in core clinical tasks awaits fundamental breakthroughs in
multimodal integration and visual understanding.
Read the full abstract ▾
All authors Dingyu Wang, Zimu Yuan, Jiajun Liu, Shanggui Liu, Nan Zhou, Tianxing Xu, Di Huang, Dong Jiang
Computer Vision
GeCo: Evaluating Geometric Consistency for Video Generation via Motion and Structure
Leslie Gu, Junhwa Hur, Charles Herrmann et al. (+4 more)
We introduce GeCo, a geometry-grounded metric for jointly detecting
geometric deformation and occlusion-inconsistency artifacts in static scenes. By fusing residual motion and depth priors, GeCo produces interpretable, dense consistency maps that reveal these artifacts. We use GeCo to systematically
benchmark recent
video generation models, uncovering common failure modes, and further employ it as a training-free guidance loss to reduce deformation artifacts during
video generation.
Read the full abstract ▾
All authors Leslie Gu, Junhwa Hur, Charles Herrmann, Fangneng Zhan, Todd Zickler, Deqing Sun, Hanspeter Pfister
Computer Vision
Human-Aligned Generative Perception: Bridging Psychophysics and Generative Models
Antara Titikhsha, Om Kulkarni and Dharun Muthaiah
Text-to-image
diffusion models generate highly detailed textures, yet they often rely on surface appearance and fail to follow strict
geometric constraints, particularly when those constraints conflict with the style implied by the text
prompt. This reflects a broader semantic gap between human perception and current generative models. We investigate whether
geometric understanding can be introduced without specialized training by using lightweight, off-the-shelf discriminators as external guidance signals. We propose a Human Perception
Embedding (HPE) teacher trained on the THINGS triplet
dataset, which captures human sensitivity to object shape. By injecting gradients from this teacher into the latent
diffusion process, we show that geometry and style can be separated in a controllable manner. We evaluate this approach across three architectures: Stable
Diffusion v1.5 with a U-Net backbone, the flow-matching model SiT-XL/2, and the
diffusion transformer PixArt-{\Sigma}. Our experiments reveal that flow models tend to drift back toward their default trajectories without continuous guidance, and we demonstrate zero-shot transfer of complex three-dimensional shapes, such as an Eames chair, onto conflicting materials such as pink metal. This guided generation improves semantic
alignment by about 80 percent compared to unguided baselines. Overall, our results show that small teacher models can reliably guide large generative systems, enabling stronger
geometric control and broadening the creative range of text-to-image synthesis.
Read the full abstract ▾
Computer Vision
S&P 500 Stock's Movement Prediction using CNN
Rahul Gupta
This paper is about predicting the movement of stock consist of S&P 500 index. Historically there are many approaches have been tried using various methods to predict the stock movement and being used in the
market currently for
algorithm trading and alpha generating systems using traditional mathematical approaches [1, 2]. The success of artificial neural network recently created a lot of interest and paved the way to enable prediction using cutting-edge research in the machine learning and deep learning. Some of these papers have done a great job in implementing and explaining benefits of these new technologies. Although most these papers do not go into the complexity of the financial data and mostly utilize single dimension data, still most of these papers were successful in creating the ground for future research in this comparatively new phenomenon. In this paper, I am trying to use multivariate raw data including stock split/dividend events (as-is) present in real-world
market data instead of engineered financial data. Convolution Neural Network (CNN), the best-known tool so far for image classification, is used on the multi-dimensional stock numbers taken from the
market mimicking them as a vector of historical data
matrices (read images) and the model achieves promising results. The predictions can be made stock by stock, i.e., a single stock, sector-wise or for the portfolio of stocks.
Read the full abstract ▾
Computer Vision
CellMamba: Adaptive Mamba for Accurate and Efficient Cell Detection
Ruochen Liu, Yi Tian, Jiahao Wang et al. (+3 more)
Cell detection in pathological images presents unique challenges due to densely packed objects, subtle inter-class differences, and severe background clutter. In this paper, we propose CellMamba, a lightweight and accurate one-stage detector tailored for fine-grained biomedical instance detection. Built upon a VSSD backbone, CellMamba integrates CellMamba Blocks, which couple either NC-Mamba or Multi-Head Self-
Attention (MSA) with a novel Triple-Mapping Adaptive Coupling (TMAC) module. TMAC enhances spatial discriminability by splitting channels into two parallel branches, equipped with dual idiosyncratic and one consensus
attention map, adaptively fused to preserve local sensitivity and global consistency. Furthermore, we design an Adaptive Mamba Head that fuses multi-scale features via learnable weights for robust detection under varying object sizes. Extensive experiments on two public
datasets-CoNSeP and CytoDArk0-demonstrate that CellMamba outperforms both CNN-based,
Transformer-based, and Mamba-based baselines in accuracy, while significantly reducing model size and inference latency. Our results validate CellMamba as an efficient and effective solution for high-resolution
cell detection.
Read the full abstract ▾
All authors Ruochen Liu, Yi Tian, Jiahao Wang, Hongbin Liu, Xianxu Hou, Jingxin Liu
Computer Vision
Diffusion Posterior Sampling for Super-Resolution under Gaussian Measurement Noise
Abu Hanif Muhammad Syarubany
This report studies
diffusion posterior sampling (DPS) for single-image super-resolution (SISR) under a known degradation model. We implement a likelihood-guided sampling procedure that combines an unconditional
diffusion prior with gradient-based conditioning to enforce measurement consistency for $4\times$ super-resolution with additive Gaussian noise. We evaluate posterior sampling (PS) conditioning across guidance scales and noise levels, using PSNR and SSIM as fidelity metrics and a combined selection score $(\mathrm{PSNR}/40)+\mathrm{SSIM}$. Our ablation shows that moderate guidance improves reconstruction quality, with the best configuration achieved at PS scale $0.95$ and noise standard deviation $\sigma=0.01$ (score $1.45231$). Qualitative results confirm that the selected PS setting restores sharper edges and more coherent facial details compared to the downsampled inputs, while alternative conditioning strategies (e.g., MCG and PS-annealed) exhibit different texture fidelity trade-offs. These findings highlight the importance of balancing
diffusion priors and measurement-gradient strength to obtain stable, high-quality reconstructions without retraining the
diffusion model for each operator.
Read the full abstract ▾
Computer Vision
AI for Mycetoma Diagnosis in Histopathological Images: The MICCAI 2024 Challenge
Hyam Omar Ali, Sahar Alhesseen, Lamis Elkhair et al. (+19 more)
Mycetoma is a neglected tropical disease caused by fungi or bacteria leading to severe tissue damage and disabilities. It affects poor and rural communities and presents medical challenges and socioeconomic burdens on patients and healthcare systems in endemic regions worldwide. Mycetoma diagnosis is a major challenge in mycetoma management, particularly in low-resource settings where expert pathologists are limited. To address this challenge, this paper presents an overview of the Mycetoma MicroImage: Detect and Classify Challenge (mAIcetoma) which was organized to advance mycetoma diagnosis through AI solutions. mAIcetoma focused on developing automated models for segmenting mycetoma grains and classifying mycetoma types from histopathological images. The challenge attracted the
attention of several teams worldwide to participate and five finalist teams fulfilled the challenge objectives. The teams proposed various deep learning architectures for the ultimate goal of this challenge. Mycetoma database (MyData) was provided to participants as a standardized
dataset to run the proposed models. Those models were evaluated using evaluation metrics. Results showed that all the models achieved high
segmentation accuracy, emphasizing the necessitate of grain detection as a critical step in mycetoma diagnosis. In addition, the top-performing models show a significant performance in classifying mycetoma types.
Read the full abstract ▾
All authors Hyam Omar Ali, Sahar Alhesseen, Lamis Elkhair, Adrian Galdran, Ming Feng, Zhixiang Xiong, Zengming Lin, Kele Xu, Liang Hu, Benjamin Keel, Oliver Mills, James Battye, Akshay Kumar, Asra Aslam, Prasad Dutande, Ujjwal Baid, Bhakti Baheti, Suhas Gajre, Aravind Shrenivas Murali, Eung-Joo Lee, Ahmed Fahal, Rachid Jennane
Computer Vision · Cross-listed from cs.CL
Five Years of SciCap: What We Learned and Future Directions for Scientific Figure Captioning
Ting-Hao 'Kenneth' Huang, Ryan A. Rossi, Sungchul Kim et al. (+4 more)
Between 2021 and 2025, the SciCap project grew from a small seed-funded idea at The Pennsylvania State University (Penn State) into one of the central efforts shaping the scientific figure-captioning landscape. Supported by a Penn State seed grant, Adobe, and the Alfred P. Sloan Foundation, what began as our attempt to test whether domain-specific training, which was successful in text models like SciBERT, could also work for figure captions expanded into a multi-institution collaboration. Over these five years, we curated, released, and continually updated a large collection of figure-caption pairs from arXiv papers, conducted extensive automatic and human evaluations on both generated and author-written captions, navigated the rapid rise of large
language models (LLMs), launched annual challenges, and built interactive systems that help scientists write better captions. In this piece, we look back at the first five years of SciCap and summarize the key technical and methodological lessons we learned. We then outline five major unsolved challenges and propose directions for the next phase of research in scientific figure captioning.
Read the full abstract ▾
All authors Ting-Hao 'Kenneth' Huang, Ryan A. Rossi, Sungchul Kim, Tong Yu, Ting-Yao E. Hsu, Ho Yin Ng, C. Lee Giles
Computer Vision
InstructMoLE: Instruction-Guided Mixture of Low-rank Experts for Multi-Conditional Image Generation
Jinqi Xiao, Qing Yan, Liming Jiang et al. (+8 more)
Parameter-Efficient Fine-Tuning of
Diffusion Transformers (DiTs) for diverse, multi-conditional tasks often suffers from task interference when using monolithic adapters like LoRA. The Mixture of Low-rank Experts (MoLE) architecture offers a modular solution, but its potential is usually limited by routing policies that operate at a
token level. Such local routing can conflict with the global nature of user instructions, leading to artifacts like spatial fragmentation and semantic drift in complex image generation tasks. To address these limitations, we introduce InstructMoLE, a novel framework that employs an Instruction-Guided Mixture of Low-Rank Experts. Instead of per-
token routing, InstructMoLE utilizes a global routing signal, Instruction-Guided Routing (IGR), derived from the user's comprehensive instruction. This ensures that a single, coherently chosen expert council is applied uniformly across all input
tokens, preserving the global semantics and structural integrity of the generation process. To complement this, we introduce an output-space orthogonality loss, which promotes expert functional diversity and mitigates representational collapse. Extensive experiments demonstrate that InstructMoLE significantly outperforms existing LoRA adapters and MoLE variants across challenging multi-conditional generation
benchmarks. Our work presents a robust and generalizable framework for instruction-driven fine-tuning of generative models, enabling superior compositional control and fidelity to user intent.
Read the full abstract ▾
All authors Jinqi Xiao, Qing Yan, Liming Jiang, Zichuan Liu, Hao Kang, Shen Sang, Tiancheng Zhi, Jing Liu, Cheng Yang, Xin Lu, Bo Yuan
Computer Vision
Nimrod Berman, Adam Botach, Emanuel Ben-Baruch et al. (+5 more)
Segmenting long-form
videos into semantically coherent scenes is a fundamental task in large-scale
video understanding. Existing encoder-based methods are limited by visual-centric biases, classify each shot in isolation without leveraging sequential dependencies, and lack both narrative understanding and explainability. In this paper, we present Scene-VLM, the first fine-tuned
vision-
language model (VLM) framework for
video scene
segmentation. Scene-VLM jointly processes visual and textual cues including frames, transcriptions, and optional metadata to enable
multimodal reasoning across consecutive shots. The model generates predictions sequentially with causal dependencies among shots and introduces a context-focus window mechanism to ensure sufficient temporal context for each shot-level decision. In addition, we propose a scheme to extract confidence scores from the
token-level logits of the VLM, enabling controllable precision-recall trade-offs that were previously limited to encoder-based methods. Furthermore, we demonstrate that our model can be aligned to generate coherent natural-
language rationales for its boundary decisions through minimal targeted supervision. Our approach achieves state-of-the-art performance on standard scene
segmentation benchmarks. On MovieNet, for example, Scene-VLM yields significant improvements of +6 AP and +13.7 F1 over the previous leading method.
Read the full abstract ▾
All authors Nimrod Berman, Adam Botach, Emanuel Ben-Baruch, Shunit Haviv Hakimi, Asaf Gendler, Ilan Naiman, Erez Yosef, Igor Kviatkovsky
Computer Vision
Inference-based GAN Video Generation
Jingbo Yang and Adrian G. Bors
Video generation has seen remarkable progress thanks to advancements in generative deep learning. However, generating long sequences remains a significant challenge. Generated
videos should not only display coherent and continuous movement but also meaningful movement in successions of scenes. Models such as GANs, VAEs, and
Diffusion Networks have been used for generating short
video sequences, typically up to 16 frames. In this paper, we first propose a new type of
video generator by enabling
adversarial-based unconditional
video generators with a variational encoder, akin to a VAE-GAN hybrid structure. The proposed model, as in other
video deep learning-based processing frameworks, incorporates two processing branches, one for content and another for movement. However, existing models struggle with the temporal scaling of the generated
videos. Classical approaches often result in degraded
video quality when attempting to increase the generated
video length, especially for significantly long sequences. To overcome this limitation, our research study extends the initially proposed VAE-GAN
video generation model by employing a novel,
memory-efficient approach to generate long
videos composed of hundreds or thousands of frames ensuring their temporal continuity, consistency and dynamics. Our approach leverages a Markov chain framework with a recall mechanism, where each state represents a short-length VAE-GAN
video generator. This setup enables the sequential connection of generated
video sub-sequences, maintaining temporal dependencies and resulting in meaningful long
video sequences.
Read the full abstract ▾
Computer Vision
BertsWin: Resolving Topological Sparsity in 3D Masked Autoencoders via Component-Balanced Structural Optimization
Evgeny Alves Limarenko and Anastasiia Studenikina
The application of self-supervised learning (SSL) and
Vision Transformers (ViTs) approaches demonstrates promising results in the field of 2D medical imaging, but the use of these methods on 3D volumetric images is fraught with difficulties. Standard Masked Autoencoders (MAE), which are state-of-the-art solution for 2D, have a hard time capturing three-dimensional spatial relationships, especially when 75% of
tokens are discarded during pre-training. We propose BertsWin, a hybrid architecture combining full BERT-style
token masking using Swin
Transformer windows, to enhance spatial context learning in 3D during SSL pre-training. Unlike the classic MAE, which processes only visible areas, BertsWin introduces a complete 3D grid of
tokens (masked and visible), preserving the spatial
topology. And to smooth out the quadratic complexity of ViT, single-level local Swin windows are used. We introduce a structural priority loss function and evaluate the results of cone beam computed
tomography of the temporomandibular joints. The subsequent assessment includes TMJ
segmentation on 3D CT scans. We demonstrate that the BertsWin architecture, by maintaining a complete three-dimensional spatial
topology, inherently accelerates semantic convergence by a factor of 5.8x compared to standard ViT-MAE baselines. Furthermore, when coupled with our proposed GradientConductor optimizer, the full BertsWin framework achieves a 15-fold reduction in training epochs (44 vs 660) required to reach state-of-the-art reconstruction fidelity. Analysis reveals that BertsWin achieves this acceleration without the computational penalty typically associated with dense volumetric processing. At canonical input resolutions, the architecture maintains theoretical FLOP parity with sparse ViT baselines, resulting in a significant net reduction in total computational resources due to faster convergence.
Read the full abstract ▾
Computer Vision
A-QCF-Net: An Adaptive Quaternion Cross-Fusion Network for Multimodal Liver Tumor Segmentation from Unpaired Datasets
Arunkumar V, Firos V M, Senthilkumar S et al. (+1 more)
Multimodal medical imaging provides complementary information that is crucial for accurate delineation of pathology, but the development of deep learning models is limited by the scarcity of large
datasets in which different modalities are paired and spatially aligned. This paper addresses this fundamental limitation by proposing an Adaptive Quaternion Cross-Fusion Network (A-QCF-Net) that learns a single unified
segmentation model from completely separate and unpaired CT and MRI cohorts. The architecture exploits the parameter efficiency and expressive power of Quaternion Neural Networks to construct a shared feature space. At its core is the Adaptive Quaternion Cross-Fusion (A-QCF) block, a data driven
attention module that enables bidirectional knowledge transfer between the two streams. By learning to modulate the flow of information dynamically, the A-QCF block allows the network to exchange abstract modality specific expertise, such as the sharp anatomical boundary information available in CT and the subtle soft tissue contrast provided by MRI. This mutual exchange regularizes and enriches the feature representations of both streams. We validate the framework by jointly training a single model on the unpaired LiTS (CT) and ATLAS (MRI)
datasets. The jointly trained model achieves Tumor Dice scores of 76.7% on CT and 78.3% on MRI, significantly exceeding the strong unimodal nnU-Net baseline by margins of 5.4% and 4.7% respectively. Furthermore, comprehensive explainability analysis using Grad-CAM and Grad-CAM++ confirms that the model correctly focuses on relevant pathological structures, ensuring the learned representations are clinically meaningful. This provides a robust and clinically viable paradigm for unlocking the large unpaired imaging archives that are common in healthcare.
Read the full abstract ▾
All authors Arunkumar V, Firos V M, Senthilkumar S, Gangadharan G R
Computer Vision · Cross-listed from cs.HC
Modified TSception for Analyzing Driver Drowsiness and Mental Workload from EEG
Gourav Siddhad, Anurag Singh, Rajkumar Saini et al. (+1 more)
Driver drowsiness is a leading cause of traffic accidents, necessitating real-time, reliable detection systems to ensure road safety. This study proposes a Modified TSception architecture for robust assessment of driver fatigue and mental workload using Electroencephalography (
EEG). The model introduces a five-layer hierarchical temporal refinement strategy to capture multi-scale
brain dynamics, surpassing the original TSception's three-layer approach. Key innovations include the use of Adaptive Average Pooling (ADP) for structural flexibility across varying
EEG dimensions and a two-stage fusion mechanism to optimize spatiotemporal feature integration for improved stability. Evaluated on the SEED-VIG
dataset, the Modified TSception achieves 83.46% accuracy, comparable to the original model (83.15%), but with a significantly reduced confidence interval (0.24 vs. 0.36), indicating better performance stability. The architecture's generalizability was further validated on the STEW mental workload
dataset, achieving state-of-the-art accuracies of 95.93% and 95.35% for 2-class and 3-class classification, respectively. These results show that the proposed modifications improve consistency and cross-task generalizability, making the model a reliable framework for
EEG-based safety monitoring.
Read the full abstract ▾
All authors Gourav Siddhad, Anurag Singh, Rajkumar Saini, Partha Pratim Roy
Computer Vision · Cross-listed from cs.LG
Dynamic Feedback Engines: Layer-Wise Control for Self-Regulating Continual Learning
Hengyi Wu, Zhenyi Wang and Heng Huang
Continual learning aims to acquire new tasks while preserving performance on previously learned ones, but most methods struggle with catastrophic forgetting. Existing approaches typically treat all layers uniformly, often trading stability for plasticity or vice versa. However, different layers naturally exhibit varying levels of uncertainty (
entropy) when classifying tasks. High-
entropy layers tend to underfit by failing to capture task-specific patterns, while low-
entropy layers risk overfitting by becoming overly confident and specialized. To address this imbalance, we propose an
entropy-aware continual learning method that employs a dynamic feedback mechanism to regulate each layer based on its
entropy. Specifically, our approach reduces
entropy in high-
entropy layers to mitigate underfitting and increases
entropy in overly confident layers to alleviate overfitting. This adaptive regulation encourages the model to converge to wider local minima, which have been shown to improve generalization. Our method is general and can be seamlessly integrated with both replay- and regularization-based approaches. Experiments on various
datasets demonstrate substantial performance gains over state-of-the-art continual learning baselines.
Read the full abstract ▾
Computer Vision
SyncAnyone: Implicit Disentanglement via Progressive Self-Correction for Lip-Syncing in the wild
Xindi Zhang, Dechao Meng, Steven Xiao et al. (+3 more)
High-quality AI-powered
video dubbing demands precise audio-lip synchronization, high-fidelity visual generation, and faithful preservation of identity and background. Most existing methods rely on a mask-based training strategy, where the mouth region is masked in talking-head
videos, and the model learns to synthesize lip movements from corrupted inputs and target audios. While this facilitates lip-sync accuracy, it disrupts spatiotemporal context, impairing performance on dynamic facial motions and causing instability in facial structure and background consistency. To overcome this limitation, we propose SyncAnyone, a novel two-stage learning framework that achieves accurate motion modeling and high visual fidelity simultaneously. In Stage 1, we train a
diffusion-based
video transformer for masked mouth inpainting, leveraging its strong spatiotemporal modeling to generate accurate, audio-driven lip movements. However, due to input corruption, minor artifacts may arise in the surrounding facial regions and the background. In Stage 2, we develop a mask-free tuning pipeline to address mask-induced artifacts. Specifically, on the basis of the Stage 1 model, we develop a data generation pipeline that creates pseudo-paired training samples by synthesizing lip-synced
videos from the source
video and random sampled audio. We further tune the stage 2 model on this synthetic data, achieving precise lip editing and better background consistency. Extensive experiments show that our method achieves state-of-the-art results in visual quality, temporal coherence, and identity preservation under in-the wild lip-syncing scenarios.
Read the full abstract ▾
All authors Xindi Zhang, Dechao Meng, Steven Xiao, Qi Wang, Peng Zhang, Bang Zhang
Computer Vision
Knot Forcing: Taming Autoregressive Video Diffusion Models for Real-time Infinite Interactive Portrait Animation
Steven Xiao, Xindi Zhang, Dechao Meng et al. (+3 more)
Real-time portrait animation is essential for interactive applications such as virtual assistants and live avatars, requiring high visual fidelity, temporal coherence, ultra-low latency, and responsive control from dynamic inputs like reference images and driving signals. While
diffusion-based models achieve strong quality, their non-causal nature hinders streaming deployment. Causal autoregressive
video generation approaches enable efficient frame-by-frame generation but suffer from error accumulation, motion discontinuities at chunk boundaries, and degraded long-term consistency. In this work, we present a novel streaming framework named
Knot Forcing for real-time portrait animation that addresses these challenges through three key designs: (1) a chunk-wise generation strategy with global identity preservation via cached KV states of the reference image and local temporal modeling using sliding window
attention; (2) a temporal
knot module that overlaps adjacent chunks and propagates spatio-temporal cues via image-to-
video conditioning to smooth inter-chunk motion transitions; and (3) A "running ahead" mechanism that dynamically updates the reference frame's temporal coordinate during inference, keeping its semantic context ahead of the current rollout frame to support long-term coherence.
Knot Forcing enables high-fidelity, temporally consistent, and interactive portrait animation over infinite sequences, achieving real-time performance with strong visual stability on consumer-grade GPUs.
Read the full abstract ▾
All authors Steven Xiao, Xindi Zhang, Dechao Meng, Qi Wang, Peng Zhang, Bang Zhang
Computer Vision
AstraNav-World: World Model for Foresight Control and Consistency
Jintao Chen, Junjun Hu, Haochen Bai et al. (+11 more)
Embodied navigation in open, dynamic environments demands accurate foresight of how the world will evolve and how actions will unfold over time. We propose AstraNav-World, an end-to-end world model that jointly reasons about future visual states and action sequences within a unified probabilistic framework. Our framework integrates a
diffusion-based
video generator with a
vision-
language policy, enabling synchronized rollouts where predicted scenes and planned actions are updated simultaneously. Training optimizes two complementary objectives: generating action-conditioned multi-step visual predictions and deriving trajectories conditioned on those predicted visuals. This bidirectional constraint makes visual predictions executable and keeps decisions grounded in physically consistent, task-relevant futures, mitigating cumulative errors common in decoupled "envision-then-plan" pipelines. Experiments across diverse
embodied navigation benchmarks show improved trajectory accuracy and higher success rates. Ablations confirm the necessity of tight
vision-action coupling and unified training, with either branch removal degrading both prediction quality and policy reliability. In real-world testing, AstraNav-World demonstrated exceptional zero-shot capabilities, adapting to previously unseen scenarios without any real-world fine-tuning. These results suggest that AstraNav-World captures transferable spatial understanding and planning-relevant
navigation dynamics, rather than merely overfitting to simulation-specific data distribution. Overall, by unifying foresight
vision and control within a single generative model, we move closer to reliable, interpretable, and general-purpose
embodied agents that operate robustly in open-ended real-world settings.
Read the full abstract ▾
All authors Jintao Chen, Junjun Hu, Haochen Bai, Minghua Luo, Xinda Xue, Botao Ren, Chengyu Bai, Shichao Xie, Ziyi Chen, Fei Liu, Zedong Chu, Xiaolong Wu, Mu Xu, Shanghang Zhang
Computer Vision
RAPTOR: Real-Time High-Resolution UAV Video Prediction with Efficient Video Attention
Zhan Chen, Zile Guo, Enze Zhu et al. (+4 more)
Video prediction is plagued by a fundamental trilemma: achieving high-resolution and perceptual quality typically comes at the cost of real-time speed, hindering its use in latency-critical applications. This challenge is most acute for autonomous
UAVs in dense urban environments, where foreseeing events from high-resolution imagery is non-negotiable for safety. Existing methods, reliant on iterative generation (
diffusion, autoregressive models) or quadratic-complexity
attention, fail to meet these stringent demands on edge hardware. To break this long-standing trade-off, we introduce RAPTOR, a
video prediction architecture that achieves real-time, high-resolution performance. RAPTOR's single-pass design avoids the error accumulation and latency of iterative approaches. Its core innovation is Efficient
Video Attention (EVA), a novel translator module that factorizes spatiotemporal modeling. Instead of processing flattened spacetime
tokens with $O((ST)^2)$ or $O(ST)$ complexity, EVA alternates operations along the spatial (S) and temporal (T) axes. This factorization reduces the time complexity to $O(S + T)$ and
memory complexity to $O(max(S, T))$, enabling global context modeling at $512^2$ resolution and beyond, operating directly on dense feature maps with a patch-free design. Complementing this architecture is a 3-stage training curriculum that progressively refines predictions from coarse structure to sharp, temporally coherent details. Experiments show RAPTOR is the first predictor to exceed 30 FPS on a Jetson AGX Orin for $512^2$
video, setting a new state-of-the-art on UAVid, KTH, and a custom high-resolution
dataset in PSNR, SSIM, and LPIPS. Critically, RAPTOR boosts the mission success rate in a real-world
UAV navigation task by 18%, paving the way for safer and more anticipatory
embodied agents.
Read the full abstract ▾
All authors Zhan Chen, Zile Guo, Enze Zhu, Peirong Zhang, Xiaoxuan Liu, Lei Wang, Yidan Zhang
Computer Vision
Spatiotemporal-Untrammelled Mixture of Experts for Multi-Person Motion Prediction
Zheng Yin, Chengjian Li, Xiangbo Shu et al. (+3 more)
Comprehensively and flexibly capturing the complex spatio-temporal dependencies of human motion is critical for multi-person motion prediction. Existing methods grapple with two primary limitations: i) Inflexible spatiotemporal representation due to reliance on positional encodings for capturing spatiotemporal information. ii) High computational costs stemming from the quadratic time complexity of conventional
attention mechanisms. To overcome these limitations, we propose the Spatiotemporal-Untrammelled Mixture of Experts (ST-MoE), which flexibly explores complex spatio-temporal dependencies in human motion and significantly reduces computational cost. To adaptively mine complex spatio-temporal patterns from human motion, our model incorporates four distinct types of spatiotemporal experts, each specializing in capturing different spatial or temporal dependencies. To reduce the potential computational overhead while integrating multiple experts, we introduce bidirectional spatiotemporal Mamba as experts, each sharing bidirectional temporal and spatial Mamba in distinct combinations to achieve model efficiency and parameter economy. Extensive experiments on four multi-person
benchmark datasets demonstrate that our approach not only outperforms state-of-art in accuracy but also reduces model parameter by 41.38% and achieves a 3.6x speedup in training. The code is available at https://github.com/alanyz106/ST-MoE.
Read the full abstract ▾
All authors Zheng Yin, Chengjian Li, Xiangbo Shu, Meiqi Cao, Rui Yan, Jinhui Tang
Computer Vision
FUSE: Unifying Spectral and Semantic Cues for Robust AI-Generated Image Detection
Md. Zahid Hossain, Most. Sharmin Sultana Samu, Md. Kamrozzaman Bhuiyan et al. (+2 more)
The fast
evolution of generative models has heightened the demand for reliable detection of AI-generated images. To tackle this challenge, we introduce FUSE, a hybrid system that combines spectral features extracted through Fast
Fourier Transform with semantic features obtained from the CLIP's
Vision encoder. The features are fused into a joint representation and trained progressively in two stages. Evaluations on GenImage, WildFake, DiTFake, GPT-ImgEval and Chameleon
datasets demonstrate strong generalization across multiple generators. Our FUSE (Stage 1) model demonstrates state-of-the-art results on the Chameleon
benchmark. It also attains 91.36% mean accuracy on the GenImage
dataset, 88.71% accuracy across all tested generators, and a mean Average Precision of 94.96%. Stage 2 training further improves performance for most generators. Unlike existing methods, which often perform poorly on high-fidelity images in Chameleon, our approach maintains robustness across diverse generators. These findings highlight the benefits of integrating spectral and semantic features for generalized detection of images generated by AI.
Read the full abstract ▾
All authors Md. Zahid Hossain, Most. Sharmin Sultana Samu, Md. Kamrozzaman Bhuiyan, Farhad Uz Zaman, Md. Rakibul Islam
Computer Vision
BeHGAN: Bengali Handwritten Word Generation from Plain Text Using Generative Adversarial Networks
Md. Rakibul Islam, Md. Kamrozzaman Bhuiyan, Safwan Muntasir et al. (+2 more)
Handwritten Text Recognition (HTR) is a well-established research area. In contrast, Handwritten Text Generation (HTG) is an emerging field with significant potential. This task is challenging due to the variation in individual handwriting styles. A large and diverse
dataset is required to generate realistic handwritten text. However, such
datasets are difficult to collect and are not readily available. Bengali is the fifth most spoken
language in the world. While several studies exist for
languages such as English and Arabic, Bengali handwritten text generation has received little
attention. To address this gap, we propose a method for generating Bengali handwritten words. We developed and used a self-collected
dataset of Bengali handwriting samples. The
dataset includes contributions from approximately five hundred individuals across different ages and genders. All images were pre-processed to ensure consistency and quality. Our approach demonstrates the ability to produce diverse handwritten outputs from input plain text. We believe this work contributes to the advancement of Bengali handwriting generation and can support further research in this area.
Read the full abstract ▾
All authors Md. Rakibul Islam, Md. Kamrozzaman Bhuiyan, Safwan Muntasir, Arifur Rahman Jawad, Most. Sharmin Sultana Samu
Computer Vision
Prior-AttUNet: Retinal OCT Fluid Segmentation Based on Normal Anatomical Priors and Attention Gating
Li Yang and Yuting Liu
Accurate
segmentation of macular edema, a hallmark pathological feature in
vision-threatening conditions such as age-related macular degeneration and diabetic macular edema, is essential for clinical diagnosis and management. To overcome the challenges of segmenting fluid regions in optical coherence
tomography (OCT) images-notably ambiguous boundaries and cross-device heterogeneity-this study introduces Prior-AttUNet, a
segmentation model augmented with generative anatomical priors. The framework adopts a hybrid dual-path architecture that integrates a generative prior pathway with a
segmentation network. A variational autoencoder supplies multi-scale normative anatomical priors, while the
segmentation backbone incorporates densely connected blocks and spatial pyramid pooling modules to capture richer contextual information. Additionally, a novel triple-
attention mechanism, guided by anatomical priors, dynamically modulates feature importance across decoding stages, substantially enhancing boundary delineation. Evaluated on the public RETOUCH
benchmark, Prior-AttUNet achieves excellent performance across three OCT imaging devices (Cirrus, Spectralis, and Topcon), with mean Dice similarity coefficients of 93.93%, 95.18%, and 93.47%, respectively. The model maintains a low computational cost of 0.37 TFLOPs, striking an effective balance between
segmentation precision and inference efficiency. These results demonstrate its potential as a reliable tool for automated clinical analysis.
Read the full abstract ▾
Computer Vision
ShinyNeRF: Digitizing Anisotropic Appearance in Neural Radiance Fields
Albert Barreiro, Roger Marí, Rafael Redondo et al. (+2 more)
Recent advances in digitization technologies have transformed the preservation and dissemination of cultural heritage. In this vein, Neural Radiance Fields (NeRF) have emerged as a leading technology for 3D digitization, delivering representations with exceptional realism. However, existing methods struggle to accurately model anisotropic specular surfaces, typically observed, for example, on brushed metals. In this work, we introduce ShinyNeRF, a novel framework capable of handling both isotropic and anisotropic reflections. Our method is capable of jointly estimating surface normals, tangents, specular concentration, and anisotropy magnitudes of an Anisotropic Spherical Gaussian (ASG) distribution, by learning an approximation of the outgoing radiance as an encoded mixture of isotropic von Mises-Fisher (vMF) distributions. Experimental results show that ShinyNeRF not only achieves state-of-the-art performance on digitizing anisotropic specular reflections, but also offers plausible physical interpretations and editing of material properties compared to existing methods.
Read the full abstract ▾
All authors Albert Barreiro, Roger Marí, Rafael Redondo, Gloria Haro, Carles Bosch
Computer Vision
Analyzing the Mechanism of Attention Collapse in VGGT from a Dynamics Perspective
Huan Li, Longjun Luo, Yuling Shi et al. (+1 more)
Visual Geometry Grounded
Transformer (VGGT) delivers state-of-the-art feed-forward 3D reconstruction, yet its global self-
attention layer suffers from a drastic collapse phenomenon when the input sequence exceeds a few hundred frames:
attention matrices rapidly become near rank-one,
token geometry degenerates to an almost one-dimensional subspace, and reconstruction error accumulates super-linearly.In this report,we establish a rigorous mathematical explanation of the collapse by viewing the global-
attention iteration as a degenerate
diffusion process.We prove that,in VGGT, the
token-feature flow converges toward a Dirac-type measure at a $O(1/L)$ rate, where $L$ is the layer index, yielding a closed-form mean-field partial differential equation that precisely predicts the empirically observed rank profile.The theory quantitatively matches the
attention-heat-map
evolution and a series of experiments outcomes reported in relevant works and explains why its
token-merging remedy – which periodically removes redundant
tokens – slows the effective
diffusion coefficient and thereby delays collapse without additional training.We believe the analysis provides a principled lens for interpreting future scalable 3D-
vision transformers,and we highlight its potential for multi-modal generalization.
Read the full abstract ▾
All authors Huan Li, Longjun Luo, Yuling Shi, Xiaodong Gu
Computer Vision
SlideChain: Semantic Provenance for Lecture Understanding via Blockchain Registration
Md Motaleb Hossen Manik, Md Zabirul Islam and Ge Wang
Modern
vision–
language models (VLMs) are increasingly used to interpret and generate educational content, yet their semantic outputs remain challenging to verify, reproduce, and audit over time. Inconsistencies across model families, inference settings, and computing environments undermine the reliability of AI-generated instructional material, particularly in high-stakes and quantitative STEM domains. This work introduces SlideChain, a blockchain-backed provenance framework designed to provide verifiable integrity for
multimodal semantic extraction at scale. Using the SlideChain Slides
Dataset-a curated corpus of 1,117 medical imaging lecture slides from a university course-we extract concepts and relational triples from four state-of-the-art VLMs and construct structured provenance records for every slide. SlideChain anchors cryptographic hashes of these records on a local EVM (Ethereum Virtual Machine)-compatible blockchain, providing tamper-evident auditability and persistent semantic baselines. Through the first systematic analysis of semantic disagreement, cross-model similarity, and lecture-level variability in
multimodal educational content, we reveal pronounced cross-model discrepancies, including low concept overlap and near-zero agreement in relational triples on many slides. We further evaluate gas usage, throughput, and scalability under simulated deployment conditions, and demonstrate perfect tamper detection along with deterministic reproducibility across independent extraction runs. Together, these results show that SlideChain provides a practical and scalable step toward trustworthy, verifiable
multimodal educational pipelines, supporting long-term auditability, reproducibility, and integrity for AI-assisted instructional systems.
Read the full abstract ▾
Computer Vision
Contrastive Graph Modeling for Cross-Domain Few-Shot Medical Image Segmentation
Yuntian Bo, Tao Zhou, Zechao Li et al. (+2 more)
Cross-domain few-shot medical image
segmentation (CD-FSMIS) offers a promising and data-efficient solution for medical applications where annotations are severely scarce and
multimodal analysis is required. However, existing methods typically filter out domain-specific information to improve generalization, which inadvertently limits cross-domain performance and degrades source-domain accuracy. To address this, we present Contrastive Graph Modeling (C-Graph), a framework that leverages the structural consistency of medical images as a reliable domain-transferable prior. We represent image features as graphs, with pixels as nodes and semantic affinities as edges. A Structural Prior Graph (SPG) layer is proposed to capture and transfer target-category node dependencies and enable global structure modeling through explicit node interactions. Building upon SPG layers, we introduce a Subgraph Matching Decoding (SMD) mechanism that exploits semantic relations among nodes to guide prediction. Furthermore, we design a Confusion-minimizing Node Contrast (CNC) loss to mitigate node ambiguity and subgraph heterogeneity by contrastively enhancing node discriminability in the graph space. Our method significantly outperforms prior CD-FSMIS approaches across multiple cross-domain
benchmarks, achieving state-of-the-art performance while simultaneously preserving strong
segmentation accuracy on the source domain.
Read the full abstract ▾
All authors Yuntian Bo, Tao Zhou, Zechao Li, Haofeng Zhang, Ling Shao
Computer Vision
UniPercept: Towards Unified Perceptual-Level Image Understanding across Aesthetics, Quality, Structure, and Texture
Shuo Cao, Jiayang Li, Xiaohui Li et al. (+12 more)
Multimodal large
language models (MLLMs) have achieved remarkable progress in visual understanding tasks such as visual grounding,
segmentation, and captioning. However, their ability to perceive perceptual-level image features remains limited. In this work, we present UniPercept-Bench, a unified framework for perceptual-level image understanding across three key domains: Aesthetics, Quality, Structure and Texture. We establish a hierarchical definition system and construct large-scale
datasets to evaluate perceptual-level image understanding. Based on this foundation, we develop a strong baseline UniPercept trained via Domain-Adaptive Pre-Training and Task-Aligned RL, enabling robust generalization across both Visual Rating (VR) and Visual Question Answering (VQA) tasks. UniPercept outperforms existing MLLMs on perceptual-level image understanding and can serve as a plug-and-play reward model for text-to-image generation. This work defines Perceptual-Level Image Understanding in the era of MLLMs and, through the introduction of a comprehensive
benchmark together with a strong baseline, provides a solid foundation for advancing perceptual-level
multimodal image understanding.
Read the full abstract ▾
All authors Shuo Cao, Jiayang Li, Xiaohui Li, Yuandong Pu, Kaiwen Zhu, Yuanting Gao, Siqi Luo, Yi Xin, Qi Qin, Yu Zhou, Xiangyu Chen, Wenlong Zhang, Bin Fu, Yu Qiao, Yihao Liu
Computer Vision
Comparative Analysis of Deep Learning Models for Perception in Autonomous Vehicles
Jalal Khan
Recently, a plethora of machine learning (ML) and deep learning (DL)
algorithms have been proposed to achieve the efficiency, safety, and reliability of autonomous vehicles (AVs). The AVs use a perception system to detect, localize, and identify other vehicles, pedestrians, and road signs to perform safe
navigation and decision-making. In this paper, we compare the performance of DL models, including YOLO-NAS and YOLOv8, for a detection-based perception task. We capture a custom
dataset and experiment with both DL models using our custom
dataset. Our analysis reveals that the YOLOv8s model saves 75% of training time compared to the YOLO-NAS model. In addition, the YOLOv8s model (83%) outperforms the YOLO-NAS model (81%) when the target is to achieve the highest object detection accuracy. These comparative analyses of these new emerging DL models will allow the relevant research community to understand the models' performance under real-world use case scenarios.
Read the full abstract ▾
Computer Vision
The Deepfake Detective: Interpreting Neural Forensics Through Sparse Features and Manifolds
Subramanyam Sahoo and Jared Junkin
Deepfake detection models have achieved high accuracy in identifying synthetic media, but their decision processes remain largely opaque. In this paper we present a mechanistic interpretability framework for deepfake detection applied to a
vision-
language model. Our approach combines a sparse autoencoder (SAE) analysis of internal network representations with a novel forensic
manifold analysis that probes how the model's features respond to controlled forensic artifact manipulations. We demonstrate that only a small fraction of latent features are actively used in each layer, and that the
geometric properties of the model's feature
manifold, including intrinsic dimensionality,
curvature, and feature selectivity, vary systematically with different types of deepfake artifacts. These insights provide a first step toward opening the "black box" of deepfake detectors, allowing us to identify which learned features correspond to specific forensic artifacts and to guide the development of more interpretable and robust models.
Read the full abstract ▾