Computer Vision
Strip-Fusion: Spatiotemporal Fusion for Multispectral Pedestrian Detection
Asiegbu Miracle Kanu-Asiegbu, Nitin Jotwani and Xiaoxiao Du
Pedestrian detection is a critical task in
robot perception. Multispectral modalities (visible light and thermal) can boost pedestrian detection performance by providing complementary visual information. Several gaps remain with multispectral pedestrian detection methods. First, existing approaches primarily focus on spatial fusion and often neglect temporal information. Second, RGB and thermal image pairs in multispectral
benchmarks may not always be perfectly aligned. Pedestrians are also challenging to detect due to varying lighting conditions, occlusion, etc. This work proposes Strip-Fusion, a spatial-temporal fusion network that is robust to misalignment in input images, as well as varying lighting conditions and heavy occlusions. The Strip-Fusion pipeline integrates temporally adaptive convolutions to dynamically weigh spatial-temporal features, enabling our model to better capture pedestrian motion and context over time. A novel Kullback-Leibler divergence loss was designed to mitigate modality imbalance between visible and thermal inputs, guiding feature
alignment toward the more informative modality during training. Furthermore, a novel post-processing
algorithm was developed to reduce false positives. Extensive experimental results show that our method performs competitively for both the KAIST and the CVC-14
benchmarks. We also observed significant improvements compared to previous state-of-the-art on challenging conditions such as heavy occlusion and misalignment.
Read the full abstract ▾
Computer Vision
MorphXAI: An Explainable Framework for Morphological Analysis of Parasites in Blood Smear Images
Aqsa Yousaf, Sint Sint Win, Megan Coffee et al. (+1 more)
Parasitic infections remain a pressing global health challenge, particularly in low-resource settings where diagnosis still depends on labor-intensive manual inspection of blood smears and the availability of expert domain knowledge. While deep learning models have shown strong performance in automating parasite detection, their clinical usefulness is constrained by limited interpretability. Existing explainability methods are largely restricted to visual heatmaps or
attention maps, which highlight regions of interest but fail to capture the morphological traits that clinicians rely on for diagnosis. In this work, we present MorphXAI, an explainable framework that unifies parasite detection with fine-grained morphological analysis. MorphXAI integrates morphological supervision directly into the prediction pipeline, enabling the model to localize parasites while simultaneously characterizing clinically relevant attributes such as shape,
curvature, visible dot count, flagellum presence, and developmental stage. To support this task, we curate a clinician-annotated
dataset of three parasite species (Leishmania, Trypanosoma brucei, and Trypanosoma cruzi) with detailed morphological labels, establishing a new
benchmark for interpretable parasite analysis. Experimental results show that MorphXAI not only improves detection performance over the baseline but also provides structured, biologically meaningful explanations.
Read the full abstract ▾
All authors Aqsa Yousaf, Sint Sint Win, Megan Coffee, Habeeb Olufowobi
Computer Vision · Cross-listed from cs.LG
Systematic Characterization of Minimal Deep Learning Architectures: A Unified Analysis of Convergence, Pruning, and Quantization
Ziwei Zheng, Huizhi Liang, Vaclav Snasel et al. (+4 more)
Deep learning networks excel at classification, yet identifying minimal architectures that reliably solve a task remains challenging. We present a computational methodology for systematically exploring and analyzing the relationships among convergence,
pruning, and quantization. The workflow first performs a structured design sweep across a large set of architectures, then evaluates convergence behavior,
pruning sensitivity, and quantization robustness on representative models. Focusing on well-known image classification of increasing complexity, and across Deep Neural Networks, Convolutional Neural Networks, and
Vision Transformers, our initial results show that, despite architectural diversity, performance is largely invariant and learning dynamics consistently exhibit three regimes: unstable, learning, and overfitting. We further characterize the minimal learnable parameters required for stable learning, uncover distinct convergence and
pruning phases, and quantify the effect of reduced numeric precision on trainable parameters. Aligning with intuition, the results confirm that deeper architectures are more resilient to
pruning than shallower ones, with parameter redundancy as high as 60%, and quantization impacts models with fewer learnable parameters more severely and has a larger effect on harder image
datasets. These findings provide actionable guidance for selecting compact, stable models under
pruning and low-precision constraints in image classification.
Read the full abstract ▾
All authors Ziwei Zheng, Huizhi Liang, Vaclav Snasel, Vito Latora, Panos Pardalos, Giuseppe Nicosia, Varun Ojha
Computer Vision
Domain-Expert-Guided Hybrid Mixture-of-Experts for Medical AI: Integrating Data-Driven Learning with Clinical Priors
Jinchen Gu, Nan Zhao, Lei Qiu et al. (+1 more)
Mixture-of-Experts (MoE) models increase representational capacity with modest computational cost, but their effectiveness in specialized domains such as medicine is limited by small
datasets. In contrast, clinical practice offers rich expert knowledge, such as physician gaze patterns and diagnostic heuristics, that models cannot reliably learn from limited data. Combining data-driven experts, which capture novel patterns, with domain-expert-guided experts, which encode accumulated clinical insights, provides complementary strengths for robust and clinically meaningful learning. To this end, we propose Domain-Knowledge-Guided Hybrid MoE (DKGH-MoE), a plug-and-play and interpretable module that unifies data-driven learning with domain expertise. DKGH-MoE integrates a data-driven MoE to extract novel features from raw imaging data, and a domain-expert-guided MoE incorporates clinical priors, specifically clinician eye-gaze cues, to emphasize regions of high diagnostic relevance. By integrating domain expert insights with data-driven features, DKGH-MoE improves both performance and interpretability.
Read the full abstract ▾
All authors Jinchen Gu, Nan Zhao, Lei Qiu, Lu Zhang
Computer Vision
UPLiFT: Efficient Pixel-Dense Feature Upsampling with Local Attenders
Matthew Walmer, Saksham Suri, Anirud Aggarwal et al. (+1 more)
The space of task-agnostic feature upsampling has emerged as a promising area of research to efficiently create denser features from pre-trained visual backbones. These methods act as a shortcut to achieve dense features for a fraction of the cost by learning to map low-resolution features to high-resolution versions. While early works in this space used iterative upsampling approaches, more recent works have switched to cross-
attention-based methods, which risk falling into the same efficiency scaling problems of the backbones they are upsampling. In this work, we demonstrate that iterative upsampling methods can still compete with cross-
attention-based methods; moreover, they can achieve state-of-the-art performance with lower inference costs. We propose UPLiFT, an architecture for Universal Pixel-dense Lightweight Feature Transforms. We also propose an efficient Local Attender operator to overcome the limitations of prior iterative feature upsampling methods. This operator uses an alternative attentional pooling formulation defined fully locally. We show that our Local Attender allows UPLiFT to maintain stable features throughout upsampling, enabling state-of-the-art performance with lower inference costs than existing pixel-dense feature upsamplers. In addition, we apply UPLiFT to generative downstream tasks and show that it achieves competitive performance with state-of-the-art Coupled Flow Matching models for VAE feature upsampling. Altogether, UPLiFT offers a versatile and efficient approach to creating denser features.
Read the full abstract ▾
All authors Matthew Walmer, Saksham Suri, Anirud Aggarwal, Abhinav Shrivastava
Computer Vision
FlowMorph: Physics-Consistent Self-Supervision for Label-Free Single-Cell Mechanics in Microfluidic Videos
Bora Yimenicioglu and Vishal Manikanden
Mechanical properties of red blood
cells (RBCs) are promising biomarkers for hematologic and systemic disease, motivating microfluidic assays that probe deformability at throughputs of $10^3$–$10^6$
cells per experiment. However, existing pipelines rely on supervised
segmentation or hand-crafted kymographs and rarely encode the laminar Stokes-flow physics that governs RBC shape
evolution. We introduce FlowMorph, a physics-consistent self-supervised framework that learns a label-free scalar mechanics proxy $k$ for each tracked RBC from short brightfield microfluidic
videos. FlowMorph models each
cell by a low-dimensional parametric contour, advances boundary points through a differentiable ”capsule-in-flow” combining laminar advection and
curvature-regularized elastic relaxation, and optimizes a loss coupling silhouette overlap, intra-cellular flow agreement, area conservation, wall constraints, and temporal smoothness, using only automatically derived silhouettes and optical flow. Across four public RBC microfluidic
datasets, FlowMorph achieves a mean silhouette IoU of $0.905$ on physics-rich
videos with provided velocity fields and markedly improves area conservation and wall violations over purely data-driven baselines. On $\sim 1.5\times 10^5$ centered sequences, the scalar $k$ alone separates tank-treading from flipping dynamics with an AUC of $0.863$. Using only $200$ real-time deformability cytometry (RT-DC) events for calibration, a monotone map $E=g(k)$ predicts apparent Young's modulus with a mean absolute error of $0.118$\,MPa on $600$ held-out
cells and degrades gracefully under shifts in channel geometry, optics, and frame rate.
Read the full abstract ▾
Computer Vision
DTC: A Deformable Transposed Convolution Module for Medical Image Segmentation
Chengkun Sun, Jinqian Pan, Renjie Liang et al. (+4 more)
In medical image
segmentation, particularly in UNet-like architectures, upsampling is primarily used to transform smaller feature maps into larger ones, enabling feature fusion between encoder and decoder features and supporting multi-scale prediction. Conventional upsampling methods, such as transposed convolution and linear interpolation, operate on fixed positions: transposed convolution applies kernel elements to predetermined pixel or voxel locations, while linear interpolation assigns values based on fixed coordinates in the original feature map. These fixed-position approaches may fail to capture structural information beyond predefined sampling positions and can lead to artifacts or loss of detail. Inspired by deformable convolutions, we propose a novel upsampling method, Deformable Transposed Convolution (DTC), which learns dynamic coordinates (i.e., sampling positions) to generate high-resolution feature maps for both 2D and 3D medical image
segmentation tasks. Experiments on 3D (e.g., BTCV15) and 2D
datasets (e.g., ISIC18, BUSI) demonstrate that DTC can be effectively integrated into existing medical image
segmentation models, consistently improving the decoder's feature reconstruction and detail recovery capability.
Read the full abstract ▾
All authors Chengkun Sun, Jinqian Pan, Renjie Liang, Zhengkang Fan, Xin Miao, Jiang Bian, Jie Xu
Computer Vision
From Specialist to Generalist: Unlocking SAM's Learning Potential on Unlabeled Medical Images
Vi Vu, Thanh-Huy Nguyen, Tien-Thinh Nguyen et al. (+5 more)
Foundation models like the Segment Anything Model (SAM) show strong generalization, yet adapting them to medical images remains difficult due to domain shift, scarce labels, and the inability of Parameter-Efficient Fine-Tuning (PEFT) to exploit unlabeled data. While conventional models like U-Net excel in semi-supervised medical learning, their potential to assist a PEFT SAM has been largely overlooked. We introduce SC-SAM, a specialist-generalist framework where U-Net provides point-based
prompts and pseudo-labels to guide SAM's adaptation, while SAM serves as a powerful generalist supervisor to regularize U-Net. This reciprocal guidance forms a bidirectional co-training loop that allows both models to effectively exploit the unlabeled data. Across prostate MRI and polyp
segmentation benchmarks, our method achieves state-of-the-art results, outperforming other existing semi-supervised SAM variants and even medical foundation models like MedSAM, highlighting the value of specialist-generalist cooperation for label-efficient medical image
segmentation. Our code is available at https://github.com/vnlvi2k3/SC-SAM.
Read the full abstract ▾
All authors Vi Vu, Thanh-Huy Nguyen, Tien-Thinh Nguyen, Ba-Thinh Lam, Hoang-Thien Nguyen, Tianyang Wang, Xingjian Li, Min Xu
Computer Vision · Cross-listed from cs.LG
Dissipative Learning: A Framework for Viable Adaptive Systems
Laurent Caraffa
We propose a perspective in which learning is an intrinsically dissipative process. Forgetting and regularization are not heuristic add-ons but structural requirements for adaptive systems. Drawing on information theory, thermodynamics, and information geometry, we introduce the BEDS (
Bayesian Emergent Dissipative Structures) framework, modeling learning as the
evolution of compressed belief states under dissipation constraints. A central contribution is the Conditional Optimality
Theorem, showing that Fisher-Rao regularization measuring change via information divergence rather than Euclidean distance is the unique thermodynamically optimal regularization strategy, achieving minimal dissipation. Euclidean regularization is shown to be structurally suboptimal. The framework unifies existing methods (Ridge, SIGReg, EMA, SAC) as special cases of a single governing equation. Within this view, overfitting corresponds to over-crystallization, while catastrophic forgetting reflects insufficient dissipation control. The framework distinguishes BEDS-crystallizable problems, where beliefs converge to stable
equilibria, from BEDS-maintainable problems, which require continual adaptation. It extends naturally to continual and multi-
agent systems, where viability, stability under adaptation and finite resources replaces asymptotic optimality as the primary criterion. Overall, this work reframes learning as maintaining viable belief states under dissipation constraints, providing a principled lens on forgetting, regularization, and stability.
Read the full abstract ▾
Computer Vision
RemEdit: Efficient Diffusion Editing with Riemannian Geometry
Eashan Adhikarla and Brian D. Davison
Controllable image generation is fundamental to the success of modern generative AI, yet it faces a critical trade-off between semantic fidelity and inference speed. The RemEdit
diffusion-based framework addresses this trade-off with two synergistic innovations. First, for editing fidelity, we navigate the latent space as a Riemannian
manifold. A mamba-based module efficiently learns the
manifold's structure, enabling direct and accurate geodesic path computation for smooth semantic edits. This control is further refined by a dual-SLERP blending technique and a goal-aware
prompt enrichment pass from a
Vision-
Language Model. Second, for additional acceleration, we introduce a novel task-specific
attention pruning mechanism. A lightweight
pruning head learns to retain
tokens essential to the edit, enabling effective optimization without the semantic degradation common in content-agnostic approaches. RemEdit surpasses prior state-of-the-art editing frameworks while maintaining real-time performance under 50%
pruning. Consequently, RemEdit establishes a new
benchmark for practical and powerful image editing. Source code: https://www.github.com/eashanadhikarla/RemEdit.
Read the full abstract ▾
Computer Vision
Benchmarking Direct Preference Optimization for Medical Large Vision-Language Models
Dain Kim, Jiwoo Lee, Jaehoon Yun et al. (+4 more)
Large
Vision-
Language Models (LVLMs) hold significant promise for medical applications, yet their deployment is often constrained by insufficient
alignment and reliability. While Direct Preference Optimization (DPO) has emerged as a potent framework for refining model responses, its efficacy in high-stakes medical contexts remains underexplored, lacking the rigorous empirical groundwork necessary to guide future methodological advances. To bridge this gap, we present the first comprehensive examination of diverse DPO variants within the medical domain, evaluating nine distinct formulations across two medical LVLMs: LLaVA-Med and HuatuoGPT-
Vision. Our results reveal several critical limitations: current DPO approaches often yield inconsistent gains over supervised fine-tuning, with their efficacy varying significantly across different tasks and backbones. Furthermore, they frequently fail to resolve fundamental visual misinterpretation errors. Building on these insights, we present a targeted preference construction strategy as a proof-of-concept that explicitly addresses visual misinterpretation errors frequently observed in existing DPO models. This design yields a 3.6% improvement over the strongest existing DPO baseline on visual question-answering tasks. To support future research, we release our complete framework, including all training data, model checkpoints, and our codebase at https://github.com/dmis-lab/med-vlm-dpo.
Read the full abstract ▾
All authors Dain Kim, Jiwoo Lee, Jaehoon Yun, Yong Hoe Koo, Qingyu Chen, Hyunjae Kim, Jaewoo Kang
Computer Vision · Cross-listed from cs.LG
Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding
Zhongyu Xiao, Zhiwei Hao, Jianyuan Guo et al. (+4 more)
Diffusion Large
Language Models (dLLMs) offer a compelling paradigm for natural
language generation, leveraging parallel decoding and bidirectional
attention to achieve superior global coherence compared to autoregressive models. While recent works have accelerated inference via KV cache reuse or heuristic decoding, they overlook the intrinsic inefficiencies within the block-wise
diffusion process. Specifically, they suffer from spatial redundancy by modeling informative-sparse suffix regions uniformly and temporal inefficiency by applying fixed denoising schedules across all the decoding process. To address this, we propose Streaming-dLLM, a training-free framework that streamlines inference across both spatial and temporal dimensions. Spatially, we introduce attenuation guided suffix modeling to approximate the full context by
pruning redundant mask
tokens. Temporally, we employ a dynamic confidence aware strategy with an early exit mechanism, allowing the model to skip unnecessary iterations for converged
tokens. Extensive experiments show that Streaming-dLLM achieves up to 68.2X speedup while maintaining generation quality, highlighting its effectiveness in
diffusion decoding. The code is available at https://github.com/xiaoshideta/Streaming-dLLM.
Read the full abstract ▾
All authors Zhongyu Xiao, Zhiwei Hao, Jianyuan Guo, Yong Luo, Jia Liu, Jie Xu, Han Hu
Computer Vision
Feature-Space Generative Models for One-Shot Class-Incremental Learning
Jack Foster, Kirill Paramonov, Mete Ozay et al. (+1 more)
Few-shot class-incremental learning (FSCIL) is a paradigm where a model, initially trained on a
dataset of base classes, must adapt to an expanding problem space by recognizing novel classes with limited data. We focus on the challenging FSCIL setup where a model receives only a single sample (1-shot) for each novel class and no further training or model alterations are allowed after the base training phase. This makes generalization to novel classes particularly difficult. We propose a novel approach predicated on the hypothesis that base and novel class
embeddings have structural similarity. We map the original
embedding space into a residual space by subtracting the class prototype (i.e., the average class
embedding) of input samples. Then, we leverage generative modeling with VAE or
diffusion models to learn the multi-modal distribution of residuals over the base classes, and we use this as a valuable structural prior to improve recognition of novel classes. Our approach, Gen1S, consistently improves novel class recognition over the state of the art across multiple
benchmarks and backbone architectures.
Read the full abstract ▾
All authors Jack Foster, Kirill Paramonov, Mete Ozay, Umberto Michieli
Computer Vision
Revisiting 3D Reconstruction Kernels as Low-Pass Filters
Shengjun Zhang, Min Chen, Yibo Wei et al. (+2 more)
3D reconstruction is to recover 3D signals from the sampled discrete 2D pixels, with the goal to converge continuous 3D spaces. In this paper, we revisit 3D reconstruction from the perspective of signal processing, identifying the periodic spectral extension induced by discrete sampling as the fundamental challenge. Previous 3D reconstruction kernels, such as Gaussians, Exponential functions, and Student's t distributions, serve as the low pass filters to isolate the baseband spectrum. However, their unideal low-pass property results in the overlap of high-frequency components with low-frequency components in the discrete-time signal's spectrum. To this end, we introduce Jinc kernel with an instantaneous drop to zero magnitude exactly at the cutoff frequency, which is corresponding to the ideal low pass filters. As Jinc kernel suffers from low decay speed in the spatial domain, we further propose modulated kernels to strick an effective balance, and achieves superior rendering performance by reconciling spatial efficiency and frequency-domain fidelity. Experimental results have demonstrated the effectiveness of our Jinc and modulated kernels.
Read the full abstract ▾
All authors Shengjun Zhang, Min Chen, Yibo Wei, Mingyu Dong, Yueqi Duan
Computer Vision
Masked Depth Modeling for Spatial Perception
Bin Tan, Changjiang Sun, Xiage Qin et al. (+8 more)
Spatial visual perception is a fundamental requirement in physical-world applications like autonomous driving and robotic manipulation, driven by the need to interact with 3D environments. Capturing pixel-aligned metric depth using RGB-D cameras would be the most viable way, yet it usually faces obstacles posed by hardware limitations and challenging imaging conditions, especially in the presence of specular or texture-less surfaces. In this work, we argue that the inaccuracies from depth sensors can be viewed as "masked" signals that inherently reflect underlying
geometric ambiguities. Building on this motivation, we present LingBot-Depth, a depth completion model which leverages visual context to refine depth maps through masked depth modeling and incorporates an automated data curation pipeline for scalable training. It is encouraging to see that our model outperforms top-tier RGB-D cameras in terms of both depth precision and pixel coverage. Experimental results on a range of downstream tasks further suggest that LingBot-Depth offers an aligned latent representation across RGB and depth modalities. We release the code, checkpoint, and 3M RGB-depth pairs (including 2M real data and 1M simulated data) to the community of spatial perception.
Read the full abstract ▾
All authors Bin Tan, Changjiang Sun, Xiage Qin, Hanat Adai, Zelin Fu, Tianxiang Zhou, Han Zhang, Yinghao Xu, Xing Zhu, Yujun Shen, Nan Xue
Computer Vision
PEAfowl: Perception-Enhanced Multi-View Vision-Language-Action for Bimanual Manipulation
Qingyu Fan, Zhaoxiang Li, Jinrui Hu et al. (+8 more)
Bimanual manipulation in cluttered scenes requires policies that remain stable under occlusions, viewpoint changes and scene variations. Existing
vision-
language-action models often lack such robustness because (i) multi-view features are fused via view-agnostic
token concatenation, yielding limited cross-view spatial representations, and (ii)
language is injected as global conditioning, resulting in coarse instruction grounding. In this paper, we introduce PEAfowl, a perception-enhanced multi-view VLA policy for bimanual manipulation. For spatial perception, PEAfowl predicts per-
token depth distributions, performs differentiable 3D lifting, and aggregates local cross-view neighbors to form geometrically grounded, cross-view aligned representations. For
language utilization, we propose to replace global conditioning with a Perceiver-style text-aware readout over frozen CLIP visual features, enabling iterative evidence accumulation. To better exploit commodity RGB-D sensing despite noisy and incomplete depth, PEAfowl's depth-distribution lifting naturally supports training-only depth
distillation, where a pretrained depth teacher supervises the depth-distribution head to inject refined
geometric priors without adding inference overhead. On RoboTwin 2.0 under domain-randomized setting, PEAfowl improves the strongest baseline by 23.0 pp in success rate, and physical experiments further demonstrate improved performance on the evaluated real-
robot tasks. Project website: https://peafowlvla.github.io/.
Read the full abstract ▾
All authors Qingyu Fan, Zhaoxiang Li, Jinrui Hu, Yi Lu, Wang Chen, Qiu Shen, Xiao-xiao Long, Yinghao Cai, Tao Lu, Shuo Wang, Xun Cao
Computer Vision · Cross-listed from cs.LG
EEG-FM-Compass: Progress, Benchmarking, and Future Directions for EEG Foundation Models
Dingkun Liu, Yuheng Chen, Zhu Chen et al. (+5 more)
Electroencephalography (
EEG) foundation models (FMs) have recently emerged as a promising paradigm for
brain-computer interfaces, aiming to learn transferable neural representations from large-scale heterogeneous recordings. Despite rapid progress, a fair and comprehensive comparison of existing
EEG FMs is still lacking, owing to inconsistent pre-training objectives, preprocessing choices, and downstream evaluation protocols. To fill this gap, we present
EEG-FM-Compass. We first review 55 representative models and organize their design choices into a unified taxonomic framework including data standardization, model architectures, and self-supervised pre-training strategies. We then evaluate 12 open source FMs and competitive specialist baselines across 13
EEG datasets spanning nine
brain-computer interface paradigms. Emphasizing real-world deployments, we consider both cross-subject generalization under a leave-one-subject-out protocol and rapid calibration under a within-subject few-shot setting. We further compare full-parameter fine-tuning with linear probing to assess the transferability of pre-trained representations, and examine the relationship between model scale and downstream performance. Our results indicate that: 1) linear probing is frequently insufficient; 2) specialist models trained from scratch remain competitive across many tasks; and 3) larger FMs do not necessarily yield better generalization performance under current data regimes and training practices.
Read the full abstract ▾
All authors Dingkun Liu, Yuheng Chen, Zhu Chen, Zhenyao Cui, Yaozhi Wen, Jiayu An, Jingwei Luo, Dongrui Wu
Computer Vision
Muhammad Umar Salman, Mohammad Areeb Qazi and Mohammed Talha Alam
We present Quran MD, a comprehensive
multimodal dataset of the Quran that integrates textual, linguistic, and audio dimensions at the verse and word levels. For each verse (ayah), the
dataset provides its original Arabic text, English translation, and phonetic transliteration. To capture the rich oral tradition of Quranic recitation, we include verse-level audio from 32 distinct reciters, reflecting diverse recitation styles and dialectical nuances. At the word level, each
token is paired with its corresponding Arabic script, English translation, transliteration, and an aligned audio recording, allowing fine-grained analysis of pronunciation, phonology, and semantic context. This
dataset supports various applications, including natural
language processing,
speech recognition, text-to-
speech synthesis, linguistic analysis, and digital Islamic studies. Bridging text and audio modalities across multiple reciters, this
dataset provides a unique resource to advance computational approaches to Quranic recitation and study. Beyond enabling tasks such as ASR, tajweed detection, and Quranic TTS, it lays the foundation for
multimodal embeddings, semantic
retrieval, style transfer, and personalized tutoring systems that can support both research and community applications. The
dataset is available at https://huggingface.co/
datasets/Buraaq/quran-audio-text-
dataset
Read the full abstract ▾
Computer Vision
VidLaDA: Bidirectional Diffusion Large Language Models for Efficient Video Understanding
Zhihao He, Tieyuan Chen, Kangyu Wang et al. (+6 more)
Current
Video Large
Language Models (
Video LLMs) typically encode frames via a
vision encoder and employ an autoregressive (AR) LLM for understanding and generation. However, this AR paradigm inevitably faces a dual efficiency bottleneck: strictly unidirectional
attention compromises understanding efficiency by hindering global spatiotemporal aggregation, while serial decoding restricts generation efficiency. To address this, we propose VidLaDA, a
Video LLM based on
Diffusion Language Models (DLMs) that leverages bidirectional
attention to unlock comprehensive spatiotemporal modeling and decode
tokens in parallel. To further mitigate the computational overhead of
diffusion decoding, we introduce MARS-Cache, an acceleration strategy that prunes redundancy by combining asynchronous visual cache refreshing with frame-wise chunk
attention. Experiments show VidLaDA rivals state-of-the-art AR baselines (e.g., Qwen2.5-VL and LLaVA-
Video) and outperforms DLM baselines, with MARS-Cache delivering over 12x speedup without compromising accuracy. Code and checkpoints are open-sourced at https://github.com/ziHoHe/VidLaDA.
Read the full abstract ▾
All authors Zhihao He, Tieyuan Chen, Kangyu Wang, Ziran Qin, Yang Shao, Chaofan Gan, Shijie Li, Zuxuan Wu, Weiyao Lin
Computer Vision
MV-SAM: Multi-view Promptable Segmentation using Pointmap Guidance
Yoonwoo Jeong, Cheng Sun, Yu-Chiang Frank Wang et al. (+2 more)
Promptable
segmentation has emerged as a powerful paradigm in computer
vision, enabling users to guide models in parsing complex scenes with
prompts such as clicks, boxes, or textual cues. Recent advances, exemplified by the Segment Anything Model (SAM), have extended this paradigm to
videos and multi-view images. However, the lack of 3D awareness often leads to inconsistent results, necessitating costly per-scene optimization to enforce 3D consistency. In this work, we introduce MV-SAM, a framework for multi-view
segmentation that achieves 3D consistency using pointmaps – 3D points reconstructed from unposed images by recent visual geometry models. Leveraging the pixel-point one-to-one correspondence of pointmaps, MV-SAM lifts images and
prompts into 3D space, eliminating the need for explicit 3D networks or annotated 3D data. Specifically, MV-SAM extends SAM by lifting image
embeddings from its pretrained encoder into 3D point
embeddings, which are decoded by a
transformer using cross-
attention with 3D
prompt embeddings. This design aligns 2D interactions with 3D geometry, enabling the model to implicitly learn consistent masks across views through 3D positional
embeddings. Trained on the SA-1B
dataset, our method generalizes well across domains, outperforming SAM2-
Video and achieving comparable performance with per-scene optimization baselines on NVOS,
SPIn-NeRF, ScanNet++, uCo3D, and DL3DV
benchmarks. Code will be released.
Read the full abstract ▾
All authors Yoonwoo Jeong, Cheng Sun, Yu-Chiang Frank Wang, Minsu Cho, Jaesung Choe
Computer Vision
Domain Generalization with Quantum Enhancement for Medical Image Classification: A Lightweight Approach for Cross-Center Deployment
Jingsong Xia and Siqi Wang
Medical image artificial intelligence models often achieve strong performance in single-center or single-device settings, yet their effectiveness frequently deteriorates in real-world cross-center deployment due to domain shift, limiting clinical generalizability. To address this challenge, we propose a lightweight domain generalization framework with
quantum-enhanced collaborative learning, enabling robust generalization to unseen target domains without relying on real multi-center labeled data. Specifically, a MobileNetV2-based domain-invariant encoder is constructed and optimized through three key components: (1) multi-domain imaging shift simulation using brightness, contrast, sharpening, and noise perturbations to emulate heterogeneous acquisition conditions; (2) domain-
adversarial training with gradient reversal to suppress domain-discriminative features; and (3) a lightweight
quantum feature enhancement layer that applies parameterized
quantum circuits for nonlinear feature mapping and
entanglement modeling. In addition, a test-time adaptation strategy is employed during inference to further alleviate distribution shifts. Experiments on simulated multi-center medical imaging
datasets demonstrate that the proposed method significantly outperforms baseline models without domain generalization or
quantum enhancement on unseen domains, achieving reduced domain-specific performance variance and improved AUC and sensitivity. These results highlight the clinical potential of
quantum-enhanced domain generalization under constrained computational resources and provide a feasible paradigm for hybrid
quantum–classical medical imaging systems.
Read the full abstract ▾
Computer Vision
SynMind: Reducing Semantic Hallucination in fMRI-Based Image Reconstruction
Lan Yang, Minghan Yang, Ke Li et al. (+3 more)
Recent advances in fMRI-based image reconstruction have achieved remarkable photo-realistic fidelity. Yet, a persistent limitation remains: while reconstructed images often appear naturalistic and holistically similar to the target stimuli, they frequently suffer from severe semantic misalignment – salient objects are often replaced or hallucinated despite high visual quality. In this work, we address this limitation by rethinking the role of explicit semantic interpretation in fMRI decoding. We argue that existing methods rely too heavily on entangled visual
embeddings which prioritize low-level appearance cues – such as texture and global gist – over explicit semantic identity. To overcome this, we parse fMRI signals into rich, sentence-level semantic descriptions that mirror the hierarchical and compositional nature of human visual understanding. We achieve this by leveraging grounded VLMs to generate synthetic, human-like, multi-granularity textual representations that capture object identities and spatial organization. Built upon this foundation, we propose SynMind, a framework that integrates these explicit semantic encodings with visual priors to condition a pretrained
diffusion model. Extensive experiments demonstrate that SynMind outperforms state-of-the-art methods across most quantitative metrics. Notably, by offloading semantic
reasoning to our text-
alignment module, SynMind surpasses competing methods based on SDXL while using the much smaller Stable
Diffusion 1.4 and a single consumer GPU. Large-scale human evaluations further confirm that SynMind produces reconstructions more consistent with human visual perception. Neurovisualization analyses reveal that SynMind engages broader and more semantically relevant
brain regions, mitigating the over-reliance on high-level visual areas.
Read the full abstract ▾
All authors Lan Yang, Minghan Yang, Ke Li, Honggang Zhang, Kaiyue Pang, Yi-Zhe Song
Computer Vision
Geometry-Grounded Gaussian Splatting
Baowen Zhang, Chenxing Jiang, Heng Li et al. (+2 more)
Gaussian Splatting (GS) has demonstrated impressive quality and efficiency in novel view synthesis. However, shape extraction from Gaussian primitives remains an open problem. Due to inadequate geometry parameterization and approximation, existing shape reconstruction methods suffer from poor multi-view consistency and are sensitive to floaters. In this paper, we present a rigorous theoretical derivation that establishes Gaussian primitives as a specific type of
stochastic solids. This theoretical framework provides a principled foundation for Geometry-Grounded Gaussian Splatting by enabling the direct treatment of Gaussian primitives as explicit
geometric representations. Using the volumetric nature of
stochastic solids, our method efficiently renders high-quality depth maps for fine-grained geometry extraction. Experiments show that our method achieves the best shape reconstruction results among all Gaussian Splatting-based methods on public
datasets.
Read the full abstract ▾
All authors Baowen Zhang, Chenxing Jiang, Heng Li, Shaojie Shen, Ping Tan
Computer Vision
SRA 2: Variational Autoencoder Self-Representation Alignment for Efficient Diffusion Training
Mengmeng Wang, Dengyang Jiang, Liuzhuozheng Li et al. (+6 more)
Denoising-based
diffusion transformers, despite their strong generation performance, suffer from inefficient training convergence. Existing methods addressing this issue, such as REPA (relying on external representation encoders) or SRA (requiring dual-model setups), inevitably incur heavy computational overhead during training due to external dependencies. To tackle these challenges, this paper proposes SRA 2, a lightweight intrinsic guidance framework for efficient
diffusion training. SRA 2 leverages off-the-shelf pre-trained Variational Autoencoder (VAE) features: their reconstruction property ensures inherent encoding of visual priors like rich texture details, structural patterns, and basic semantic information. Specifically, SRA 2 aligns the intermediate latent features of
diffusion transformers with VAE features via a lightweight projection layer, supervised by a feature
alignment loss. This design accelerates training without extra representation encoders or dual-model maintenance, resulting in a simple yet effective pipeline. Extensive experiments demonstrate that SRA 2 improves both generation quality and training convergence speed compared to vanilla
diffusion transformers, matches or outperforms state-of-the-art acceleration methods, and incurs merely 4% extra GFLOPs with zero additional cost for external guidance models.
Read the full abstract ▾
All authors Mengmeng Wang, Dengyang Jiang, Liuzhuozheng Li, Yucheng Lin, Guojiang Shen, Xiangjie Kong, Yong Liu, Guang Dai, Jingdong Wang
Computer Vision
ViTCoP: Accelerating Large Vision-Language Models via Visual and Textual Semantic Collaborative Pruning
Wen Luo, Peng Chen, Xiaotao Huang et al. (+1 more)
Large
Vision-
Language Models (LVLMs) incur high computational costs due to significant redundancy in their visual
tokens. To effectively reduce this cost, researchers have proposed various visual
token pruning methods. However, existing methods are generally limited, either losing critical visual information prematurely due to
pruning in the
vision encoder, or leading to information redundancy among the selected
tokens due to
pruning in the Large
Language Models (LLMs). To address these challenges, we propose a Visual and Textual Semantic Collaborative
Pruning framework (ViTCoP) that combines redundancy filtering in the
vision encoder with step-wise co-
pruning within the LLM based on its hierarchical characteristics, to efficiently preserve critical and informationally diverse visual
tokens. Meanwhile, to ensure compatibility with acceleration techniques like FlashAttention, we introduce the L2 norm of K-vectors as the
token saliency metric in the LLM. Extensive experiments on various Large
Vision-
Language Models demonstrate that ViTCoP not only achieves state-of-the-art performance surpassing existing methods on both image and
video understanding tasks, but also significantly reduces model inference latency and GPU
memory consumption. Notably, its performance advantage over other methods becomes even more pronounced under extreme
pruning rates.
Read the full abstract ▾
All authors Wen Luo, Peng Chen, Xiaotao Huang, LiQun Huang
Computer Vision
Agreement-Driven Multi-View 3D Reconstruction for Live Cattle Weight Estimation
Rabin Dulal, Wenfeng Jia, Lihong Zheng et al. (+1 more)
Accurate cattle live weight estimation is vital for livestock management, welfare, and productivity. Traditional methods, such as manual weighing using a walk-over weighing system or proximate measurements using body condition scoring, involve manual handling of stock and can impact productivity from both a stock and economic perspective. To address these issues, this study investigated a cost-effective, non-contact method for live weight calculation in cattle using 3D reconstruction. The proposed pipeline utilized multi-view RGB images with SAM 3D-based agreement-guided fusion, followed by ensemble regression. Our approach generates a single 3D point cloud per animal and compares classical ensemble models with deep learning models under low-data conditions. Results show that SAM 3D with multi-view agreement fusion outperforms other 3D generation methods, while classical ensemble models provide the most consistent performance for practical farm scenarios (R$^2$ = 0.69 $\pm$ 0.10, MAPE = 2.22 $\pm$ 0.56 %), making this practical for on-farm implementation. These findings demonstrate that improving reconstruction quality is more critical than increasing model complexity for scalable deployment on farms where producing a large volume of 3D data is challenging.
Read the full abstract ▾
All authors Rabin Dulal, Wenfeng Jia, Lihong Zheng, Jane Quinn
Computer Vision
MV-S2V: Multi-View Subject-Consistent Video Generation
Ziyang Song, Xinyu Gong, Bangya Liu et al. (+1 more)
Existing Subject-to-
Video Generation (S2V) methods have achieved high-fidelity and subject-consistent
video generation, yet remain constrained to single-view subject references. This limitation renders the S2V task reducible to an S2I + I2V pipeline, failing to exploit the full potential of
video subject control. In this work, we propose and address the challenging Multi-View S2V (MV-S2V) task, which synthesizes
videos from multiple reference views to enforce 3D-level subject consistency. Regarding the scarcity of training data, we first develop a synthetic data curation pipeline to generate highly customized synthetic data, complemented by a small-scale real-world captured
dataset to boost the training of MV-S2V. Another key issue lies in the potential confusion between cross-subject and cross-view references in conditional generation. To overcome this, we further introduce Temporally Shifted RoPE (TS-RoPE) to distinguish between different subjects and distinct views of the same subject in reference conditioning. Our framework achieves superior 3D subject consistency w.r.t. multi-view reference images and high-quality visual outputs, establishing a new meaningful direction for subject-driven
video generation. Code and data are available at: https://szy-young.github.io/mv-s2v
Read the full abstract ▾
All authors Ziyang Song, Xinyu Gong, Bangya Liu, Zelin Zhao
Computer Vision
Bridging Supervision Gaps: A Unified Framework for Remote Sensing Change Detection
Kaixuan Jiang, Chen Wu, Zhenghui Zhao et al. (+3 more)
Change detection (CD) aims to identify surface changes from multi-temporal remote sensing imagery. In real-world scenarios, Pixel-level change labels are expensive to acquire, and existing models struggle to adapt to scenarios with diverse annotation availability. To tackle this challenge, we propose a unified change detection framework (UniCD), which collaboratively handles supervised, weakly-supervised, and unsupervised tasks through a coupled architecture. UniCD eliminates architectural barriers through a shared encoder and multi-branch collaborative learning mechanism, achieving deep coupling of heterogeneous supervision signals. Specifically, UniCD consists of three supervision-specific branches. In the supervision branch, UniCD introduces the spatial-temporal awareness module (STAM), achieving efficient synergistic fusion of bi-temporal features. In the weakly-supervised branch, we construct change representation regularization (CRR), which steers model convergence from coarse-grained activations toward coherent and separable change modeling. In the unsupervised branch, we propose semantic prior-driven change inference (SPCI), which transforms unsupervised tasks into controlled weakly-supervised path optimization. Experiments on mainstream
datasets demonstrate that UniCD achieves optimal performance across three tasks. It exhibits significant accuracy improvements in weakly and unsupervised scenarios, surpassing current state-of-the-art by 12.72% and 12.37% on LEVIR-CD, respectively.
Read the full abstract ▾
All authors Kaixuan Jiang, Chen Wu, Zhenghui Zhao, Chengxi Han, Haonan Guo, Hongruixuan Chen
Computer Vision
Video Compression with Hierarchical Temporal Neural Representation
Jun Zhu, Xinfeng Zhang, Lv Tang et al. (+3 more)
Video compression has recently benefited from implicit neural representations (INRs), which model
videos as continuous functions. INRs offer compact storage and flexible reconstruction, providing a promising alternative to traditional codecs. However, most existing INR-based methods treat the temporal dimension as an independent input, limiting their ability to capture complex temporal dependencies. To address this, we propose a Hierarchical Temporal Neural Representation for
Videos, TeNeRV. TeNeRV integrates short- and long-term dependencies through two key components. First, an Inter-Frame Feature Fusion (IFF) module aggregates features from adjacent frames, enforcing local temporal coherence and capturing fine-grained motion. Second, a GoP-Adaptive Modulation (GAM) mechanism partitions
videos into Groups-of-Pictures and learns group-specific priors. The mechanism modulates network parameters, enabling adaptive representations across different GoPs. Extensive experiments demonstrate that TeNeRV consistently outperforms existing INR-based methods in rate-distortion performance, validating the effectiveness of our proposed approach.
Read the full abstract ▾
All authors Jun Zhu, Xinfeng Zhang, Lv Tang, Junhao Jiang, Gai Zhang, Jia Wang
Computer Vision
Frequency-aware Neural Representation for Videos
Jun Zhu, Xinfeng Zhang, Lv Tang et al. (+3 more)
Implicit Neural Representations (INRs) have emerged as a promising paradigm for
video compression. However, existing INR-based frameworks typically suffer from inherent spectral bias, which favors low-frequency components and leads to over-smoothed reconstructions and suboptimal rate-distortion performance. In this paper, we propose FaNeRV, a Frequency-aware Neural Representation for
videos, which explicitly decouples low- and high-frequency components to enable efficient and faithful
video reconstruction. FaNeRV introduces a multi-resolution supervision strategy that guides the network to progressively capture global structures and fine-grained textures through staged supervision . To further enhance high-frequency reconstruction, we propose a dynamic high-frequency injection mechanism that adaptively emphasizes challenging regions. In addition, we design a frequency-decomposed network module to improve feature modeling across different spectral bands. Extensive experiments on standard
benchmarks demonstrate that FaNeRV significantly outperforms state-of-the-art INR methods and achieves competitive rate-distortion performance against traditional codecs.
Read the full abstract ▾
All authors Jun Zhu, Xinfeng Zhang, Lv Tang, Junhao Jiang, Gai Zhang, Jia Wang