Computer Vision
ESICA: A Scalable Framework for Text-Guided 3D Medical Image Segmentation
Yu Xin, Gorkem Can Ates, Jun Ma et al. (+5 more)
Text guided 3D medical image
segmentation offers a flexible alternative to class based and spatial
prompt based models by allowing users to specify regions of interest directly in natural
language. This paradigm avoids reliance on predefined label sets, reduces ambiguous outputs, and aligns more naturally with clinical workflows. However, existing text guided frameworks are often computationally expensive, exhibit weak text volume feature
alignment, and fail to capture fine anatomical details. We propose ESICA, a lightweight and scalable framework that addresses these challenges through three innovations: (1) a similarity
matrix based mask prediction formulation that enhances semantic
alignment, (2) an efficient decomposed decoder with adapter modules for accurate volumetric decoding, and (3) a two pass refinement strategy that sharpens boundaries and resolves uncertain regions. To improve training stability and generalization, ESICA adopts a two stage scheme consisting of positive only pretraining followed by balanced fine tuning. On the CVPR BiomedSegFM
benchmark spanning five imaging modalities (CT, MRI, PET, ultrasound, and microscopy), ESICA achieves state of the art
segmentation accuracy, while the compact ESICA4 Lite variant attains similar
segmentation performance with substantially fewer parameters, yielding a superior efficiency accuracy trade off. Our framework advances text guided
segmentation toward efficient, scalable, and clinically deployable systems. Code will be made publicly available at https://github.com/mirthAI/ESICA.
Read the full abstract ▾
All authors Yu Xin, Gorkem Can Ates, Jun Ma, Sumin Kim, Ying Zhang, Kaleb E Smith, Kuang Gong, Wei Shao
Computer Vision
World-R1: Reinforcing 3D Constraints for Text-to-Video Generation
Weijie Wang, Xiaoxuan He, Youping Gu et al. (+9 more)
Recent
video foundation models demonstrate impressive visual synthesis but frequently suffer from
geometric inconsistencies. While existing methods attempt to inject 3D priors via architectural modifications, they often incur high computational costs and limit scalability. We propose World-R1, a framework that aligns
video generation with 3D constraints through reinforcement learning. To facilitate this
alignment, we introduce a specialized pure text
dataset tailored for world simulation. Utilizing Flow-GRPO, we optimize the model using feedback from pre-trained 3D foundation models and
vision-
language models to enforce structural coherence without altering the underlying architecture. We further employ a periodic decoupled training strategy to balance rigid
geometric consistency with dynamic scene fluidity. Extensive evaluations reveal that our approach significantly enhances 3D consistency while preserving the original visual quality of the foundation model, effectively bridging the gap between
video generation and scalable world simulation.
Read the full abstract ▾
All authors Weijie Wang, Xiaoxuan He, Youping Gu, Yifan Yang, Zeyu Zhang, Yefei He, Yanbo Ding, Xirui Hu, Donny Y. Chen, Zhiyuan He, Yuqing Yang, Bohan Zhuang
Computer Vision
Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation
Zhiheng Liu, Weiming Ren, Xiaoke Huang et al. (+12 more)
Unified
multimodal models typically rely on pretrained
vision encoders and use separate visual representations for understanding and generation, creating misalignment between the two tasks and preventing fully end-to-end optimization from raw pixels. We introduce Tuna-2, a native unified
multimodal model that performs visual understanding and generation directly based on pixel
embeddings. Tuna-2 drastically simplifies the model architecture by employing simple patch
embedding layers to encode visual input, completely discarding the modular
vision encoder designs such as the VAE or the representation encoder. Experiments show that Tuna-2 achieves state-of-the-art performance in
multimodal benchmarks, demonstrating that unified pixel-space modelling can fully compete with latent-space approaches for high-quality image generation. Moreover, while the encoder-based variant converges faster in early pretraining, Tuna-2's encoder-free design achieves stronger
multimodal understanding at scale, particularly on tasks requiring fine-grained visual perception. These results show that pretrained
vision encoders are not necessary for
multimodal modelling, and end-to-end pixel-space learning offers a scalable path toward stronger visual representations for both generation and perception.
Read the full abstract ▾
All authors Zhiheng Liu, Weiming Ren, Xiaoke Huang, Shoufa Chen, Tianhong Li, Mengzhao Chen, Yatai Ji, Sen He, Jonas Schult, Belinda Zeng, Tao Xiang, Wenhu Chen, Ping Luo, Luke Zettlemoyer, Yuren Cong
Computer Vision
OmniShotCut: Holistic Relational Shot Boundary Detection with Shot-Query Transformer
Boyang Wang, Guangyi Xu, Jiahui Zhang et al. (+2 more)
Shot Boundary Detection (SBD) aims to automatically identify shot changes and divide a
video into coherent shots. While SBD was widely studied in the literature, existing methods often produce non-interpretable boundaries on transitions, miss subtle yet harmful discontinuities, and rely on noisy, low-diversity annotations and outdated
benchmarks. To alleviate these limitations, we propose OmniShotCut to formulate SBD as structured relational prediction, jointly estimating shot ranges with intra-shot relations and inter-shot relations, by a shot query-based dense
video Transformer. To avoid imprecise manual labeling, we adopt a fully synthetic transition synthesis pipeline that automatically reproduces major transition families with precise boundaries and parameterized variants. We also introduce OmniShotCutBench, a modern wide-domain
benchmark enabling holistic and diagnostic evaluation. Experiments on the
benchmarks demonstrate the effectiveness and generality of our method.
Read the full abstract ▾
All authors Boyang Wang, Guangyi Xu, Jiahui Zhang, Zhipeng Tang, Zezhou Cheng
Computer Vision
DiffuSAM: Diffusion-Based Prompt-Free SAM2 for Few-Shot and Source-Free Medical Image Segmentation
Tal Grossman, Noa Cahan, Lev Ayzenberg et al. (+1 more)
Segmentation models such as Segment Anything Model (SAM) and SAM2 achieve strong
prompt-driven zero-shot performance. However, their training on natural images limits domain transfer to medical data. Consequently, accurate
segmentation typically requires extensive fine-tuning and expert-designed
prompts. We propose DiffuSAM, a
diffusion-based adaptation of SAM2 for
prompt-free medical image
segmentation. Our framework synthesizes SAM2-compatible
segmentation mask-like
embeddings via a lightweight
diffusion-prior from off-the-shelf frozen SAM2 image features. The generated
embeddings are integrated into SAM2's mask decoder to produce accurate
segmentations, thereby eliminating the need for user
prompts. The
diffusion prior is further conditioned on previously segmented slices, enforcing spatial consistency across volumes. Evaluated on the BTCV and CHAOS
datasets for CT and MRI under Source-Free Unsupervised Domain Adaptation (SF-UDA) and Few-Shot settings, DiffuSAM achieves competitive performance with efficient training and inference. Code is available upon request from the corresponding author.
Read the full abstract ▾
All authors Tal Grossman, Noa Cahan, Lev Ayzenberg, Hayit Greenspan
Computer Vision
WildLIFT: Lifting monocular drone video to 3D for species-agnostic wildlife monitoring
Vandita Shukla, Fabio Remondino, Blair Costelloe et al. (+1 more)
Monocular RGB cameras mounted on drones are widely used for wildlife monitoring, yet most analytical pipelines remain confined to two-dimensional image space, leaving
geometric information in
video underexploited. We present WildLIFT, a computational framework that integrates three-dimensional scene geometry from
monocular drone
video with open-vocabulary 2D instance
segmentation to enable species-agnostic 3D detection and tracking. Oriented 3D bounding box labels with semantic face information enable quantitative assessment of viewpoint coverage and inter-animal occlusion, producing structured metadata for downstream ecological analyses. We validate the framework on 2,581 manually curated frames comprising over 6,700 3D detections across four large mammal species. WildLIFT maintains high identity consistency in multi-animal scenes and substantially reduces manual 3D annotation effort through keyframe-based refinement. By transforming standard drone footage into structured 3D and viewpoint-aware representations, WildLIFT extends the analytical utility of aerial wildlife
datasets for behavioural research and population monitoring.
Read the full abstract ▾
All authors Vandita Shukla, Fabio Remondino, Blair Costelloe, Benjamin Risse
Computer Vision
NeuroClaw Technical Report
Cheng Wang, Zhibin He, Zhihao Peng et al. (+7 more)
Agentic artificial intelligence systems promise to accelerate scientific workflows, but neuroimaging poses unique challenges: heterogeneous modalities (sMRI, fMRI, dMRI,
EEG), long multi-stage pipelines, and persistent reproducibility risks. To address this gap, we present NeuroClaw, a domain-specialized multi-
agent research assistant for executable and reproducible neuroimaging research. NeuroClaw operates directly on raw neuroimaging data across formats and modalities, grounding decisions in
dataset semantics and BIDS metadata so users need not prepare curated inputs or bespoke model code. The platform combines harness engineering with end-to-end environment management, including pinned Python environments, Docker support, automated installers for common neuroimaging tools, and GPU configuration. In practice, this layer emphasizes checkpointing, post-execution verification, structured audit traces, and controlled runtime setup, making toolchains more transparent while improving reproducibility and auditability. A three-tier skill/
agent hierarchy separates user-facing interaction, high-level orchestration, and low-level tool skills to decompose complex workflows into safe, reusable units. Alongside the NeuroClaw framework, we introduce NeuroBench, a system-level
benchmark for executability, artifact validity, and reproducibility readiness. Across multiple
multimodal LLMs, NeuroClaw-enabled runs yield consistent and substantial score improvements compared with direct
agent invocation. Project homepage: https://cuhk-aim-group.github.io/NeuroClaw/index.html
Read the full abstract ▾
All authors Cheng Wang, Zhibin He, Zhihao Peng, Shengyuan Liu, Yufan Hu, Carl Yang, Lifang He, Lichao Sun, Xiang Li, Yixuan Yuan
Computer Vision
Aycromo: An Open-Source Platform for Automatic Chromosome Detection in Metaphase Images Based on Deep Learning
Jorge L. A. Lima and Filipe R. Cordeiro
Chromosome analysis is a fundamental step in the diagnosis of genetic diseases, but the manual karyotyping workflow is time-consuming and heavily dependent on expert specialists, often requiring several days per patient. Although Deep Learning models have achieved high performance in chromosome detection, most proposed solutions remain restricted to research prototypes or lack graphical interfaces suitable for clinical use. In this work, we present Aycromo, an open-source desktop platform for AI-assisted cytogenetic analysis. Built on Electron and ONNX Runtime, the tool allows cytogeneticists to load pre-trained models, compare architectures through an integrated benchmarking module, and manually correct detections via an interactive annotation interface, all without command-line interaction. Preliminary experiments on metaphase images from the CRCN-NE
dataset demonstrate that YOLOv11 achieves 99.40% mAP@50, while the platform reduces per-slide analysis to seconds
Read the full abstract ▾
Computer Vision
Benchmarking Pathology Foundation Models for Breast Cancer Survival Prediction
Fredrik K. Gustafsson, Constance Boissin, Johan Vallon-Christersson et al. (+2 more)
Pathology foundation models (PFMs) have recently emerged as powerful pretrained encoders for computational pathology, enabling transfer learning across a wide range of downstream tasks. However, systematic comparisons of these models for clinically meaningful prediction problems remain limited, especially in the context of survival prediction under external validation. In this study, we
benchmark widely used and recently proposed PFMs for breast cancer survival prediction from whole-slide histopathology images. Using a standardized pipeline based on patch-level feature extraction and a unified survival modeling framework, we evaluate model representations across three independent clinical cohorts comprising more than 5,400 patients with long-term follow-up. Models are trained on one cohort and evaluated on two independent external cohorts, enabling a rigorous assessment of cross-
dataset generalization. Overall, H-optimus-1 achieves the strongest survival prediction performance. More broadly, we observe consistent generational improvements across model families, with second-generation PFMs outperforming their first-generation counterparts. However, absolute performance differences between many recent PFMs remain modest, suggesting diminishing returns from further scaling of pretraining data or model size alone. Notably, the compact distilled model H0-mini slightly outperforms its larger teacher model H-optimus-0, despite using fewer than 8% of the parameters and enabling significantly faster feature extraction. Together, these results provide the first large-scale, externally validated
benchmark of PFMs for breast cancer survival prediction, and offer practical guidance for efficient deployment of PFMs in clinical workflows.
Read the full abstract ▾
All authors Fredrik K. Gustafsson, Constance Boissin, Johan Vallon-Christersson, David A. Clifton, Mattias Rantalainen
Computer Vision
Probing CLIP's Comprehension of 360-Degree Textual and Visual Semantics
Hai Wang, Xiaochen Yang, Mingzhi Dong et al. (+1 more)
The dream of instantly creating rich 360-degree panoramic worlds from text is rapidly becoming a reality, yet a crucial gap exists in our ability to reliably evaluate their semantic
alignment. Contrastive
Language-Image Pre-training (CLIP) models, standard AI evaluators, predominantly trained on perspective image-text pairs, face an open question regarding their understanding of the unique characteristics of 360-degree panoramic image-text pairs. This paper addresses this gap by first introducing two concepts: 360-degree textual semantics, semantic information conveyed by explicit format identifiers, and 360-degree visual semantics, invariant semantics under horizontal circular shifts. To probe CLIP's comprehension of these semantics, we then propose novel evaluation methodologies using keyword manipulation and horizontal circular shifts of varying magnitudes. Rigorous statistical analyses across popular CLIP configurations reveal that: (1) CLIP models effectively leverage explicit textual identifiers, demonstrating an understanding of 360-degree textual semantics; and (2) CLIP models fail to robustly preserve semantic
alignment under horizontal circular shifts, indicating limited comprehension of 360-degree visual semantics. To address this limitation, we propose a LoRA-based fine-tuning framework that explicitly instills invariance to circular shifts. Our fine-tuned models exhibit improved comprehension of 360-degree visual semantics, though with a slight degradation in original semantic evaluation performance, highlighting a fundamental trade-off in adapting CLIP to 360-degree panoramic images. Code is available at https://github.com/littlewhitesea/360Semantics.
Read the full abstract ▾
All authors Hai Wang, Xiaochen Yang, Mingzhi Dong, Jing-Hao Xue
Computer Vision
Meta-CoT: Enhancing Granularity and Generalization in Image Editing
Shiyi Zhang, Yiji Cheng, Tiankai Hang et al. (+8 more)
Unified multi-modal understanding/generative models have shown improved image editing performance by incorporating fine-grained understanding into their Chain-of-Thought (CoT) process. However, a critical question remains underexplored: what forms of CoT and training strategy can jointly enhance both the understanding granularity and generalization? To address this, we propose Meta-CoT, a paradigm that performs a two-level decomposition of any single-image editing operation with two key properties: (1) Decomposability. We observe that any editing intention can be represented as a triplet - (task, target, required understanding ability). Inspired by this, Meta-CoT decomposes both the editing task and the target, generating task-specific CoT and traversing editing operations on all targets. This decomposition enhances the model's understanding granularity of editing operations and guides it to learn each element of the triplet during training, substantially improving the editing capability. (2) Generalizability. In the second decomposition level, we further break down editing tasks into five fundamental meta-tasks. We find that training on these five meta-tasks, together with the other two elements of the triplet, is sufficient to achieve strong generalization across diverse, unseen editing tasks. To further align the model's editing behavior with its CoT
reasoning, we introduce the CoT-Editing Consistency Reward, which encourages more accurate and effective utilization of CoT information during editing. Experiments demonstrate that our method achieves an overall 15.8% improvement across 21 editing tasks, and generalizes effectively to unseen editing tasks when trained on only a small set of meta-tasks. Our code,
benchmark, and model are released at https://shiyi-zh0408.github.io/projectpages/Meta-CoT/
Read the full abstract ▾
All authors Shiyi Zhang, Yiji Cheng, Tiankai Hang, Zijin Yin, Runze He, Yu Xu, Wenxun Dai, Yunlong Lin, Chunyu Wang, Qinglin Lu, Yansong Tang
Computer Vision
CF-VLA: Efficient Coarse-to-Fine Action Generation for Vision-Language-Action Policies
Fan Du, Feng Yan, Jianxiong Wu et al. (+8 more)
Flow-based
vision-
language-action (VLA) policies offer strong expressivity for action generation, but suffer from a fundamental inefficiency: multi-step inference is required to recover action structure from uninformative Gaussian noise, leading to a poor efficiency-quality trade-off under real-time constraints. We address this issue by rethinking the role of the starting point in generative action modeling. Instead of shortening the sampling trajectory, we propose CF-VLA, a coarse-to-fine two-stage formulation that restructures action generation into a coarse initialization step that constructs an action-aware starting point, followed by a single-step local refinement that corrects residual errors. Concretely, the coarse stage learns a conditional posterior over endpoint velocity to transform Gaussian noise into a structured initialization, while the fine stage performs a fixed-time refinement from this initialization. To stabilize training, we introduce a stepwise strategy that first learns a controlled coarse predictor and then performs joint optimization. Experiments on CALVIN and LIBERO show that our method establishes a strong efficiency-performance frontier under low-NFE (Number of Function Evaluations) regimes: it consistently outperforms existing NFE=2 methods, matches or surpasses the NFE=10 $\pi_{0.5}$ baseline on several metrics, reduces action sampling latency by 75.4%, and achieves the best average real-
robot success rate of 83.0%, outperforming MIP by 19.5 points and $\pi_{0.5}$ by 4.0 points. These results suggest that structured, coarse-to-fine generation enables both strong performance and efficient inference. Our code is available at https://github.com/EmbodiedAI-RoboTron/CF-VLA.
Read the full abstract ▾
All authors Fan Du, Feng Yan, Jianxiong Wu, Xinrun Xu, Weiye Zhang, Weinong Wang, Yu Guo, Bin Qian, Zhihai He, Fei Wang, Heng Yang
Computer Vision
Infrastructure-Guided Connectivity-Enhanced Road Crack Detection and Estimation
Haosong Xiao, Yamini Ramesh, Rishabh Shukla et al. (+2 more)
In this paper, we report the world's first infrastructure-guided communication-enhanced road crack detection pipeline that is effective and implementable on passenger vehicles. We first design a customized communication protocol to transmit the region of interest from the infrastructure to the vehicle. With proper camera image processing (e.g., dynamic cropping and frame selection), the focused images are provided to the crack detection model. Leveraging state-of-the-art crack detection model backbones and a carefully prepared
dataset comprising a forward-facing view with a crack, we train the model to improve crack-detection performance. We demonstrate the full detection pipeline on an experimental vehicle platform, showcase the detection effectiveness, and project future research directions.
Read the full abstract ▾
All authors Haosong Xiao, Yamini Ramesh, Rishabh Shukla, Swarat Sarkar, Chaozhe R. He
Computer Vision
Majorization-Guided Test-Time Adaptation for Vision-Language Models under Modality-Specific Shift
Lixian Chen, Mingxuan Huang, Yanhui Chen et al. (+2 more)
Vision–
language models can face asymmetric visual and textual shifts at deployment. These shifts expose a
multimodal failure mode in which an unreliable branch remains overconfident, dominates fusion, and causes
entropy-based test-time adaptation to sharpen an incorrect prediction. We model this behavior as doubly
stochastic posterior mixing and cast adaptation as constrained de-mixing. Majorization-Guided
Multimodal Test-Time Adaptation (MG-MTTA) freezes both encoders and updates only a lightweight fusion module. Running-anchor consistency estimates relative branch drift, while cross-modal conflict regulates modality dominance before
entropy sharpening. The analysis gives sufficient conditions for
entropy reduction to preserve the clean decision and an explicit threshold at which a biased modality reverses the fused ranking. Across visual, textual, and joint shifts, MG-MTTA improves ImageNet top-1 accuracy from 57.97% to 66.51% under textual shift and from 21.68% to 26.27% under joint shift, while reducing wrong-more-confident failures. The largest gains occur under textual and joint shifts, where the two branches differ more in reliability. Project page: https://mg-mtta.github.io/.
Read the full abstract ▾
All authors Lixian Chen, Mingxuan Huang, Yanhui Chen, Junyi Lin, Yang Shi
Computer Vision
Point-MF: Stabilizing One-Step Mean Flows for Single-Image Point Cloud Reconstruction
Yuta Baba and Keiji Yanai
Single-image point cloud reconstruction requires recovering complete object-level geometry, including occluded regions, from a single RGB image.
Diffusion- and flow-based reconstructors can model this ambiguity, but their iterative sampling requires many network function evaluations, while naive one-step point-space updates often produce outliers and density imbalance. We propose Point-MF, a one-step conditional Mean-Flow framework for geometry-only point cloud reconstruction. Point-MF predicts an interval-averaged velocity field directly in point-cloud space and reconstructs the output with a single network function evaluation, without training a point-cloud autoencoder. To stabilize large Mean-Flow jumps, we introduce Denoised Space Anchor (DSA), a set-distance auxiliary loss that anchors the denoised point set implied by the predicted velocity to the ground-truth geometry. On all 13 ShapeNet-R2N2 categories and Pix3D, Point-MF achieves strong reconstruction quality under the aligned point-set protocol, improving average CD and EMD over the evaluated point-cloud reconstruction baselines. It runs in 40.72 ms per sample on an RTX A4000, within the same latency order as the feedforward RGB2point baseline and substantially faster than iterative
diffusion baselines such as PC$^2$ and BDM. These results show that denoised-space
geometric anchoring enables stable one-step Mean Flow directly in point-cloud space.
Read the full abstract ▾
Computer Vision
Improving Vision-language Models with Perception-centric Process Reward Models
Yingqian Min, Kun Zhou, Yifan Li et al. (+6 more)
Recent advancements in reinforcement learning with verifiable rewards (RLVR) have significantly improved the complex
reasoning ability of
vision-
language models (VLMs). However, its outcome-level supervision is too coarse to diagnose and correct errors within the
reasoning chain. To this end, we propose Perceval, a process reward model (PRM) that enables
token-level error grounding, which can extract image-related claims from the response and compare them one by one with the visual evidence in the image, ultimately returning claims that contain perceptual errors. Perceval is trained with perception-intensive supervised training data. We then integrate Perceval into the RL training process to train the policy models. Specifically, compared to traditional GRPO, which applies sequence-level advantages, we apply
token-level advantages by targeting penalties on hallucinated spans identified by Perceval, thus enabling fine-grained supervision signals. In addition to augmenting the training process, Perceval can also assist VLMs during the inference stage. Using Perceval, we can truncate the erroneous portions of the model's response, and then either have the model regenerate the response directly or induce the model to reflect on its previous output. This process can be repeated multiple times to achieve test-time scaling. Experiments show significant improvements on
benchmarks from various domains across multiple
reasoning VLMs trained with RL, highlighting the promise of perception-centric supervision as a general-purpose strategy. For test-time scaling, it also demonstrates consistent performance gains over other strategies, such as major voting. Our code and data will be publicly released at https://github.com/RUCAIBox/Perceval.
Read the full abstract ▾
All authors Yingqian Min, Kun Zhou, Yifan Li, Yuhuan Wu, Han Peng, Yifan Du, Wayne Xin Zhao, Min Yang, Ji-Rong Wen
Computer Vision
Haoxiao Wang, Antao Xiang, Haiyang Sun et al. (+8 more)
Diffusion models are primarily trained for image synthesis, yet their denoising trajectories encode rich, spatially aligned visual priors. In this paper, we demonstrate that these priors can be utilized for text-conditioned semantic and open-vocabulary
segmentation, and this approach can be generalized to various downstream tasks to make a general-purpose
diffusion segmentation framework. Concretely, we introduce DiGSeg (
Diffusion Models as a Generalist
Segmentation Learner), which repurposes a pretrained
diffusion model into a unified
segmentation framework. Our approach encodes the input image and ground-truth mask into the latent space and concatenates them as conditioning signals for the
diffusion U-Net. A parallel CLIP-aligned text pathway injects
language features across multiple scales, enabling the model to align textual queries with evolving visual representations. This design transforms an off-the-shelf
diffusion backbone into a universal interface that produces structured
segmentation masks conditioned on both appearance and arbitrary text
prompts. Extensive experiments demonstrate state-of-the-art performance on standard semantic
segmentation benchmarks, as well as strong open-vocabulary generalization and cross-domain transfer to medical, remote sensing, and agricultural scenarios-without domain-specific architectural customization. These results indicate that modern
diffusion backbones can serve as generalist
segmentation learners rather than pure generators, narrowing the gap between visual generation and visual understanding.
Read the full abstract ▾
All authors Haoxiao Wang, Antao Xiang, Haiyang Sun, Peilin Sun, Changhao Pan, Yifu Chen, Minjie Hong, Weijie Wang, Shuang Chen, Yue Chen, Zhou Zhao
Computer Vision
RACANet: Reliability-Aware Crowd Anchor Network for RGB-T Crowd Counting
Jinghao Shi, Mengqi Lei, Kunliang He et al. (+3 more)
RGB-Thermal (T) crowd counting aims to integrate visible-spectrum and thermal infrared information to improve the robustness of crowd density estimation in complex scenes. Although existing studies generally improve counting accuracy through cross-modal feature fusion, most current methods rely on implicit cross-modal fusion strategies and lack explicit modeling of local spatial discrepancies as well as fine-grained characterization of modality reliability at the positional level, thereby limiting the accuracy and interpretability of the fusion process. To address these issues, this paper proposes a two-stage fusion framework, RACANet, a Reliability-Aware Crowd Anchor Network for RGB-T crowd counting. First, we introduce a lightweight cross-modal
alignment pretraining stage, which explicitly learns cross-modal semantic correspondences through crowd-prior supervision and local bidirectional soft matching. Then, based on the priors learned during pretraining, a Local Anchor Fusion Module (LAFM) is introduced in the formal training stage. This module generates local semantic anchors by aggregating features from highly reliable regions and further enables adaptive pixel-level feature redistribution with a local
attention mechanism. In addition, we propose a discrepancy-aware consistency constraint to dynamically coordinate the reliability of regions where modal representations are consistent. Experiments conducted on two widely used
benchmark datasets, RGBT-CC and Drone-RGBT, demonstrate that RACANet outperforms existing methods. The anonymous code is available at https://anonymous.4open.science/r/RACANet-9985.
Read the full abstract ▾
All authors Jinghao Shi, Mengqi Lei, Kunliang He, Yun Li, Wei Bao, Siqi Li
Computer Vision
Point Cloud Registration for Fusion between SPECT MPI and CTA Images
Ni Yao, Xiangyu Liu, Shaojie Tang et al. (+9 more)
Clinical fusion of Single
Photon Emission Computed
Tomography Myocardial Perfusion Imaging (SPECT MPI) and Computed
Tomography Angiography (CTA) remains limited by cross-modality misregistration and reliance on manual landmarks, which can hinder accurate ischemia localization and lesion-level functional assessment. To address this issue, we propose a registration and fusion framework for SPECT MPI and CTA that integrates functional and structural information for comprehensive cardiac evaluation. The proposed pipeline performs U-Net-based
segmentation on both modalities. On SPECT MPI, only the left ventricle (LV) is extracted, and anatomical landmarks are automatically derived from characteristic LV structures. On CTA, both ventricles are segmented, and their spatial relationship is used to automatically define landmarks at the interventricular septal junction. Scale-space consistency preprocessing and landmark-driven coarse registration are applied to mitigate initial misalignment. Based on this initialization, multiple fine registration methods are evaluated on LV epicardial surface point clouds, including ICP, SICP, CPD, CluReg, FFD, and BCPD-plus-plus. The resulting transformations are then propagated to voxel-level resampling for high-precision SPECT-CTA fusion. In a retrospective cohort of 60 patients, the proposed framework preserved sub-millimeter coronary detail from CTA while accurately overlaying quantitative SPECT perfusion. Among the evaluated methods, BCPD-plus-plus achieved the highest accuracy with a mean point cloud distance of 1.7 mm. By combining robust initialization, comparative fine registration, and voxel-level fusion, the proposed approach provides a practical solution for myocardial ischemia localization and functional evaluation of coronary lesions, while remaining independent of any specific fine registration
algorithm.
Read the full abstract ▾
All authors Ni Yao, Xiangyu Liu, Shaojie Tang, Danyang Sun, Chuang Han, Yanting Li, Jiaofen Nan, Chengyang Li, Fubao Zhu, Chen Zhao, Zhihui Xu, Weihua Zhou
Computer Vision
Self-Supervised Representation Learning via Hyperspherical Density Shaping
Esteban Rodríguez-Betancourt and Edgar Casasola-Murillo
Modern self-supervised representation learning methods often relies on empirical heuristics that are not theoretically grounded. In this study we propose HyDeS, a theoretically grounded method based on multi-view mutual information maximization within an hyperspherical space using Shannon differential
entropy with a non-parametric von Mises-Fisher density
estimator. We show that HyDeS bias the trained model towards focusing on foreground features of the images and perform well on
segmentation tasks such as VOC PASCAL, while it lags in fine-grained classification. We provide a detailed analysis of the induced latent space geometry and learning dynamics, that can be used for designing other theoretically grounded self-supervised learning methods.
Read the full abstract ▾
Computer Vision
CA-IDD: Cross-Attention Guided Identity-Conditional Diffusion for Identity-Consistent Face Swapping
Md Shohel Rana and Tanoy Debnath
Face swapping aims to optimize realistic facial image generation by leveraging the identity of a source face onto a target face while preserving pose, expression, and context. However, existing methods, especially GAN-based methods, often struggle to balance identity preservation and visual realism due to limited controllability and mode collapse. In this paper, we introduce CA-IDD (Cross-
Attention Guided Identity-Conditional
Diffusion), the first
diffusion-based face swapping approach that integrates multi-modal guidance comprising gaze, identity, and facial parsing through multi-scale cross-
attention. Precomputed identity
embeddings are incorporated into the denoising process via hierarchical
attention layers, resulting in accurate and consistent identity transfer. To improve semantic coherence and visual quality, we use expert-guided supervision, with facial parsing and gaze-consistency modules. Unlike GAN-based or implicit-fusion methods, our
diffusion framework provides stable training, robust generalization, and spatially adaptive identity
alignment, allowing for fine-grained regional control across pose and expression variations. CA-IDD achieves an FID of 11.73, exceeding established baselines such as FaceShifter and MegaFS. Qualitative results also reveal improved identity retention across diverse poses, establishing CA-IDD as a strong foundation for future
diffusion-based face editing.
Read the full abstract ▾
Computer Vision
Deployment-Aligned Low-Precision Neural Architecture Search for Spaceborne Edge AI
Parampuneet Kaur Thind, Vaibhav Katturu, Giacomo Zema et al. (+1 more)
Designing deep networks that meet strict latency and accuracy constraints on edge accelerators increasingly relies on hardware-aware optimization, including neural architecture search (NAS) guided by device-level metrics. Yet most hardware-aware NAS pipelines still optimize architectures under full-precision assumptions and apply low-precision adaptation only after the search, leading to a mismatch between optimization-time behavior and deployment-time execution on low-precision hardware that can substantially degrade accuracy. We address this limitation by integrating deployment-aligned low-precision training directly into hardware-aware NAS. Candidate architectures are exposed to FP16 numerical constraints during fine-tuning and evaluation, enabling joint optimization of architectural efficiency and numerical robustness without modifying the search space or evolutionary strategy. We evaluate the proposed framework on vessel
segmentation for spaceborne maritime monitoring, targeting the Intel Movidius Myriad X Visual Processing Unit (VPU). While post-training precision conversion reduces on-device performance from 0.85 to 0.78 mIoU, deployment-aligned low-precision training achieves 0.826 mIoU on-device for the same architecture (95,791 parameters), recovering approximately two-thirds of deployment-induced accuracy gap without increasing model complexity. These results demonstrate that incorporating deployment-consistent numerical constraints into hardware-aware NAS substantially improves robustness and
alignment between optimization and deployment for resource-constrained edge Artificial Intelligence (AI).
Read the full abstract ▾
All authors Parampuneet Kaur Thind, Vaibhav Katturu, Giacomo Zema, Roberto Del Prete
Computer Vision
Zero-to-CAD: Agentic Synthesis of Interpretable CAD Programs at Million-Scale Without Real Data
Mohammadmehdi Ataei, Farzaneh Askari, Kamal Rahimi Malekshan et al. (+1 more)
Computer-Aided Design (CAD) models are defined by their construction history: a parametric recipe that encodes design intent. However, existing large-scale 3D
datasets predominantly consist of boundary representations (B-Reps) or meshes, stripping away this critical procedural information. To address this scarcity, we introduce Zero-to-CAD, a scalable framework for synthesizing executable CAD construction sequences. We frame synthesis as an agentic search problem: by
embedding a large
language model (LLM) within a feedback-driven CAD environment, our system iteratively generates, executes, and validates code using tools and documentation lookup to promote
geometric validity and operation diversity. This agentic approach enables the synthesis of approximately one million executable, readable, editable CAD sequences, covering a rich vocabulary of operations beyond sketch-and-extrude workflows. We also release a curated subset of 100,000 high-quality models selected for
geometric diversity. To demonstrate the
dataset's utility, we fine-tune a
vision-
language model on our synthetic data to reconstruct editable CAD programs from multi-view images, outperforming strong baselines, including GPT-5.2, and effectively bootstrapping sequence generation capabilities without real construction-history training data. Zero-to-CAD bridges the gap between
geometric scale and parametric interpretability, offering a vital resource for the next generation of CAD AI.
Read the full abstract ▾
All authors Mohammadmehdi Ataei, Farzaneh Askari, Kamal Rahimi Malekshan, Pradeep Kumar Jayaraman
Computer Vision · Cross-listed from cs.IR
Geometric Analysis of Self-Supervised Vision Representations for Semantic Image Retrieval
Esteban Rodríguez-Betancourt and Edgar Casasola-Murillo
Content-based image
retrieval (CBIR) systems enable users to search images based on visual content instead of relying on metadata. The text domain has benefited from vector search of representations created with unsupervised methods such as BERT. However, modern self-supervised learning methods for
vision are mostly not reported in CBIR-related literature, instead relying on supervised models or multi-modal methods that align text and
vision. We evaluate how the representations learned by modern self-supervised learning methods for
vision perform under typical
retrieval stacks that leverage vector databases and nearest neighbor search. Our evaluation reveals that the latent space geometry impacts approximate nearest neighbor (ANN) indexing. Specifically, highly anisotropic representations with high skewness produced by several modern SSL methods degrade the performance of partition-based and hashing-based search, even if their own linear probe or K-NN accuracy is not affected. In contrast, representations with higher isotropy and local purity better satisfy the distance-based assumptions of ANN indexes, leading to improved semantic
retrieval performance.
Read the full abstract ▾
Computer Vision
TextGround4M: A Prompt-Aligned Dataset for Layout-Aware Text Rendering
Dongxing Mao, Yilin Wang, Linjie Li et al. (+2 more)
Despite recent advances in text-to-image generation, models still struggle to accurately render
prompt-specified text with correct spatial layout – especially in multi-span, structured settings. This challenge is driven not only by the lack of
datasets that align
prompts with the exact text and layout expected in the image, but also by the absence of effective metrics for evaluating layout quality. To address these issues, we introduce TextGround4M, a large-scale
dataset of over 4 million
prompt-image pairs, each annotated with span-level text grounded in the
prompt and corresponding bounding boxes. This enables fine-grained supervision for layout-aware,
prompt-grounded text rendering. Building on this, we propose a lightweight training strategy for autoregressive T2I models that appends layout-aware span
tokens during training, without altering model architecture or inference behavior. We further construct a
benchmark with stratified layout complexity to evaluate both open-source and proprietary models in a zero-shot setting. In addition, we introduce two layout-aware metrics to address the long-standing lack of spatial evaluation in text rendering. Our results show that models trained on TextGround4M outperform strong baselines in text fidelity, spatial accuracy, and
prompt consistency, highlighting the importance of fine-grained layout supervision for grounded T2I generation.
Read the full abstract ▾
All authors Dongxing Mao, Yilin Wang, Linjie Li, Zhengyuan Yang, Alex Jinpeng Wang
Computer Vision
AutoGUI-v2: A Comprehensive Multi-Modal GUI Functionality Understanding Benchmark
Hongxin Li, Xiping Wang, Jingran Su et al. (+4 more)
Autonomous
agents capable of navigating Graphical User Interfaces (GUIs) hold the potential to revolutionize digital productivity. However, achieving true digital autonomy extends beyond reactive element matching; it necessitates a predictive mental model of interface dynamics and the ability to foresee the "digital world state" resulting from interactions. Despite the perceptual capabilities of modern
Vision-
Language Models (VLMs), existing
benchmarks remain bifurcated (focusing either on black-box task completion or static, shallow grounding), thereby failing to assess whether
agents truly comprehend the implicit functionality and transition logic of GUIs. To bridge this gap, we introduce AutoGUI-v2, a comprehensive
benchmark designed to evaluate deep GUI functionality understanding and interaction outcome prediction. We construct the
benchmark using a novel VLM-human collaborative pipeline that recursively parses multi-platform screenshots into hierarchical functional regions to generate diverse evaluation tasks. Providing 2,753 tasks across six operating systems, AutoGUI-v2 rigorously tests
agents on region and element-level semantics, grounding, and dynamic state prediction. Our evaluation reveals a striking dichotomy in VLMs: while open-source models fine-tuned on
agent data (e.g., Qwen3-VL) excel at functional grounding, commercial models (e.g., Gemini-2.5-Pro-Thinking) dominate in functionality captioning. Crucially, all models struggle with complex interaction logic of uncommon actions, highlighting that deep functional understanding remains a significant hurdle. By systematically measuring these foundational capabilities, AutoGUI-v2 offers a new lens for advancing the next generation of GUI
agents.
Read the full abstract ▾
All authors Hongxin Li, Xiping Wang, Jingran Su, Zheng Ju, Yuntao Chen, Qing Li, Zhaoxiang Zhang
Computer Vision
DYMAPIA: A Multi-Domain Framework for Detecting AI-based Video Manipulation
Md Shohel Rana and Andrew H. Sung
AI-generated media are advancing rapidly, raising pressing concerns for content authenticity and digital trust. We introduce DYMAPIA, a multi-domain Deepfake detection framework that fuses spatial, spectral, and temporal cues to capture subtle traces of manipulation in visual data. The system builds dynamic anomaly masks by combining evidence from
Fourier spectra, local texture descriptors, edge irregularities, and optical flow consistency, which highlight tampered regions with fine spatial accuracy. These masks guide DistXCNet, a lightweight classifier distilled from Xception and optimized with depthwise separable convolutions for fast, region-focused classification. This joint design achieves state-of-the-art results, with accuracy and F1-scores exceeding 99% on FF++, Celeb-DF, and VDFD
benchmarks, while keeping the model compact enough for real-time use. Beyond outperforming existing full-frame and multidomain detectors, DYMAPIA demonstrates deployment readiness for time-critical forensic tasks, including media verification, misinformation defense, and secure content filtering.
Read the full abstract ▾
Computer Vision
BMD-45: A Large-Scale CCTV Vehicle Detection Dataset for Urban Traffic in Developing Cities
Akash Sharma, Chinmay Mhatre, Sankalp Gawali et al. (+8 more)
Robust vehicle detection from fixed CCTV cameras is critical for Intelligent Transportation Systems. Yet existing
benchmarks predominantly feature relatively homogeneous, highly organized traffic patterns captured from ego-centric driving perspectives or controlled aerial views. This regional and sensor view bias creates a significant gap. Models trained on
datasets such as UA-DETRAC and COCO struggle to generalize to the dense, heterogeneous, disorganized traffic conditions observed in rapidly developing urban centers in emerging economies. To address this limitation, we introduce BMD-45, a large-scale
dataset comprising 480K bounding boxes annotated over 45K images captured from over 3.6K operational Safe City CCTV cameras. BMD-45 contains 14 fine-grained vehicle categories, including region-specific modes such as auto-rickshaws and tempo travellers, which are not present in existing
benchmarks. The
dataset captures real-world deployment challenges, including extreme viewpoint variation, occlusion, and vehicle density . We establish comprehensive baselines using state-of-the-art detectors and reveal a striking domain gap: models fine-tuned on UA-DETRAC achieve only 33.6% mAP@0.50:0.95, compared to 83.8% when trained in-domain on BMD-45, representing a 2.5x improvement that persists even when accounting for novel vehicle classes. This performance gap underscores the critical need for geographically diverse traffic
benchmarks and establishes BMD-45 as a baseline for developing robust perception systems in underrepresented urban environments worldwide. The
dataset is available at: https://huggingface.co/
datasets/iisc-aim/BMD-45.
Read the full abstract ▾
All authors Akash Sharma, Chinmay Mhatre, Sankalp Gawali, Ruthvik Bokkasam, Brij Sharma, Vishwajeet Pattanaik, Punit Rathore, Raghu Krishnapuram, Vijay Gopal Kovvali, Anirban Chakraborty, Yogesh Simmhan
Computer Vision · Cross-listed from cs.SI
Phase-Separated Complex Hilbert PCA on Markerless 3D Pose Estimation Data: A Global Phase Network and Its Extension to a Continuous Field on the Body Surface
Hiromitsu Goto, Tao Tao and Zheng-Lin Chia
Quantitative analysis of the kinematic chain in sports motion is essential for performance evaluation and injury prevention. Conventional methods such as the kinematic-sequence (KS) and continuous relative phase (CRP) are confined to adjacent joint pairs and lack a unified framework for whole-body coordination, while segmental power-flow analysis requires force plates and inertial parameters that restrict it to laboratory environments. We apply Complex
Hilbert Principal Component Analysis (CHPCA) separately to each motion phase (backswing and downswing) on markerless 3D pose estimation data, extracting the dominant whole-body phase pattern as a single complex eigenvector. The pipeline further includes a fully automatic signal-based phase
segmentation (no priors on strike count or rest location) and an extension to 1,079 body-surface mesh vertices, so that the kinematic chain is represented as a continuous phase field across the body. On 14 hammer-striking trials of a single subject, the framework reveals (i) a trunk-anchored global phase architecture, (ii) a functional asymmetry between preparation and execution phases quantified by Mode-1 contribution (45.5% vs. 70.5%) and inter-trial Spearman consistency (0.38 vs. 0.58), and (iii) a consistent reorganisation across both skeletal joints and mesh vertices ($p < 10^{-10}$ on 1,079 vertices). As a methodological consistency check, pairwise phase differences from the Mode-1 eigenvector are compared against CRP on all 190 joint pairs by a permutation test ($\rho = 0.473$, $p = 0.0005$). A correspondence analysis between Mode-1 amplitude and kinetic-energy mobilisation variance further shows a strong positive correlation in the downswing ($\rho \approx 0.71$ on both skeleton and mesh) and no correlation in the backswing, indicating that the proposed framework bridges kinematic and kinetic descriptions of coordination through phase structure.
Read the full abstract ▾
Computer Vision
AD-Relight: Training-Free Banner Relighting via Illumination Translation with Diffusion Priors
Rameshwar Mishra, Bishshoy Das, A V Subramanyam et al. (+1 more)
The recent surge in content consumption through streaming services has driven a growing demand for personalized content. Personalized advertisements (ads) play a crucial role in enhancing both user engagement and ad effectiveness. A key aspect of ad personalization involves replacing existing regions in a frame with custom, Photoshop-generated banners. However, existing ad-placement pipelines typically rely on simple
geometric warping, ignoring the scene's underlying lighting conditions. Similarly, state-of-the-art
diffusion-based object insertion and relighting models struggle to accurately relight these newly inserted banners, as they are not trained on ad-banner data, and training such a model for ad banners would require millions of images. This highlights the need for an effective relighting framework that enables seamless integration of custom banners into the original scene. Motivated by this, we present AD-Relight, a novel multi-stage training-free framework that adapts a
diffusion-based relighting model at test time to relight newly added Photoshop-generated ad banners. Through extensive evaluation, we demonstrate that AD-Relight outperforms both relighting baselines and existing ad-placement methods based on simple warping. User studies further show that participants consistently prefer the outputs of AD-Relight over those of prior approaches.
Read the full abstract ▾
All authors Rameshwar Mishra, Bishshoy Das, A V Subramanyam, Guan-Ming Su