Computer Vision
FIRM: Flow-based Imaging via Regularized Minimization
Shirin Shoushtari, Edward P. Chandler, Xiao Shi et al. (+1 more)
Flow matching methods for imaging inverse problems typically incorporate measurements through network conditioning or guidance during sampling. Neither approach explicitly applies the forward operator within the learned conditional velocity field. We develop a principled measurement-conditional velocity parameterization that does. For a linear interpolation path, we express the optimal velocity through the posterior mean $E[x_1|x_t, y]$ and show that this mean is the unique minimizer of a variational objective with an explicit data-consistency term. The velocity defined by this minimizer provably transports the source distribution to the measurement-conditioned posterior. This result leads to a forward operator-aware velocity field that is trained end-to-end and requires no separate guidance during sampling. Across five imaging tasks, our method achieves leading reconstruction quality with up to $50\times$ fewer network evaluations than competitive flow-based methods. Varying the number of sampling steps also controls the distortion-perception trade-off without retraining.
Read the full abstract ▾
All authors Shirin Shoushtari, Edward P. Chandler, Xiao Shi, Ulugbek S. Kamilov
Computer Vision
DementiaCare-Bench: A Modality-Validated Video Benchmark
Afrouz Sheikholeslami, Yuankai Qi, Xuyun Zhang et al. (+4 more)
Dementia affects an estimated 57 million people worldwide, and for most families the hardest part of care is not
memory loss but the behavioral and psychological symptoms of dementia (BPSD): agitation, wandering, resistance to care, sundowning. Understanding these symptoms requires more than recognizing the behavior itself; it also requires knowing what happened beforehand. The same behavior may call for a different response depending on its trigger.
Video-
language models (VLMs) could potentially support caregivers, yet no existing
benchmark evaluates this capability. To fill this gap, we present DementiaCare-Bench: 56 professionally produced caregiver training
videos segmented into 94 clips across nine BPSD categories, with 2023 questions generated by a multi-
agent pipeline that grounds every clinical claim in a verbatim transcript span. Each question is then probed under four visual conditions and labelled by the least it requires, so its visual demand is measured rather than assumed. Measurement contradicts intent: we wrote 77.7% of the questions to require ordered frames, and 34.8% do. Across 12 current VLMs the pattern is uniform. The best reach 85% overall, but that average is carried by questions a
language model can answer from clinical knowledge alone; accuracy falls by 17 points on average on questions that require the ordered clip, and a leading open model scores at chance on judging whether a caregiver's response was appropriate. A lightweight LoRA fine-tune, DemCare-VLM, moves
video dependence from -3.3 to +4.5 points, so what the
benchmark exposes can be repaired and not only measured.
Read the full abstract ▾
All authors Afrouz Sheikholeslami, Yuankai Qi, Xuyun Zhang, Luping Zhou, Amin Beheshti, Quan Z. Sheng, Ming-Hsuan Yang
Computer Vision
Input Resolution Matters: Real-Time Object Detection Latency
Qingyang Zhang, Fumio Machida and Laura Carnevali
We model total latency as the convolution of preprocessing, inference, and postprocessing distributions under a simplifying independence approximation, with selected stage parameters expressed as functions of source-image resolution. Under this assumption, the probability density of the total latency is the convolution of the stage-wise densities, and its cumulative distribution function (CDF) provides the distribution of end-to-end detection time. Each stage is modeled by a parametric distribution (e.g., Exponential, Erlang, Normal, Gamma), with parameters expressed as functions of the source-image resolution. Experiments with YOLOv11n on NVIDIA Jetson Orin NX using COCO2017 images across multiple resolutions assess the proposed models against fixed-parameter baselines using Kolmogorov Smirnov, Anderson Darling, and Cramér von Mises statistics. The results indicate that resolution-aware parameterization can improve distributional approximation in the measured setting, particularly for the more flexible Normal and Gamma models, while the quality of fit remains distribution dependent. Our contribution is a theoretically grounded and lightweight formulation for studying resolution-dependent latency distributions in a measured object detection pipeline.
Read the full abstract ▾
Computer Vision
PA-CDM: Position-Aware Character Detection Matching for Evaluating Handwritten Mathematical Expression Recognition
Shiliang Luo
Handwritten mathematical expression recognition (HMER) is conventionally scored by exact-match rates and string-similarity metrics that are blind to where an error occurs: two predictions with identical
token-error counts receive identical scores whether they misplace a subscript or swap the operands of a fraction. Render-based character detection matching (CDM) aligns glyphs robustly but remains position-blind—on controlled fraction-operand swaps it scores 0.8595 where position-aware scoring yields 0.6253. Tree-edit metrics exhibit a complementary blind spot: rewrites outside the parser's normalization coverage are penalized as structural errors (0.8552 where render-based metrics score 1.0). We propose PA-CDM, a position-aware metric that couples character detection matching with position-forest encoding and divergence-level weighting; StructPerturb v2.0, a frozen
benchmark of 1,340 controlled perturbation pairs across 15 type–intensity
cells; and a cross-metric consistency protocol combining a sensitivity
matrix, a human study, and LLM-judge calibration. In a six-annotator study, PA-CDM attains the highest correlation with human judgments among seven automatic metrics (Spearman rho=0.9535, n=990). A frontier LLM judge correlates slightly higher (rho=0.9613) but is costly, nondeterministic, and API-dependent; PA-CDM approaches it at zero marginal cost with deterministic, diagnosable behavior.
Read the full abstract ▾
Computer Vision
Parallel Training Using a CNN-DNN Architecture for Accelerated Development of Diagnostic Models
Janine Weber-Hamacher, Astha Jaiswal, Philipp Fervers et al. (+8 more)
Artificial intelligence has shown promise in assisting radiologists in imaging-based diagnosis across a wide range of diseases. Efficient training of large deep learning models is essential to cope with extremely large data sets or dynamically growing disease data, like in a pandemic like situation. In this retrospective study, we collected 300 CT scans from COVID-19 and non-COVID-19 pneumonia patients from three different centers in Germany. We investigated a hybrid CNN-DNN network model based on image decomposition and localization that naturally supports parallel and efficient training of deep learning models. In total, 156 models with three different architectures were trained to capture features at different levels resulting in 12 patient-level COVID-19 diagnosis models. Diagnostic performance as well as time saving were measured. The highest accuracy was obtained from DenseNet121 and 3D CNN models with a parallel CNN-DNN approach, resulting in $88.78\%$ training, $76.67\%$ validation and $76.03\%$ test accuracy for the DenseNet121 with $4\times4\times1$ subdomains and $87.72\%$ training, $76.82\%$ validation and $74.86\%$ test accuracy, respectively, for the 3D CNN with $4\times4\times1$ subdomains. The strongest reduction in parallel training time by a factor of $31$ was observed for the 3D CNN model and $4\times4\times2$ subdomains. Our parallel training approach improves efficiency as well as performance enabling rapid model development, among others crucial for pandemic preparedness.
Read the full abstract ▾
All authors Janine Weber-Hamacher, Astha Jaiswal, Philipp Fervers, Dorotya Móré, Athanasios Giannakis, Ricarda Fischbach, Andreas Michael Bucher, Rahil Shahzad, Jonathan Kottlors, Thorsten Persigehl, Axel Klawonn
Computer Vision
UniPart: Towards Zero-shot Language-Grounded 3D Part Segmentation for Embodied Interaction
Xinqiang Yu, Zekun qi, Jiawei He et al. (+6 more)
Fine-grained robotic manipulation depends on understanding parts, not only whole objects. Existing 3D foundation models tend to be either generalized but object-aware, or part-aware but limited to closed-set taxonomies, which weakens zero-shot transfer. We study text-conditioned 3D part
segmentation, where a free-form phrase selects a functional part on point cloud. We introduce UniPart, a feed-forward cross-modal 3D
Transformer that conditions CLIP text
embedding. To scale supervision, we build LangPart-1M with 160K+ Objaverse assets and 8M text to part pairs using multi-view consistent part generation. We further manually label a high-quality subset, LangPart-4K, for fine-tuning and evaluation. UniPart achieves strong zero-shot results on open-vocabulary part
benchmarks and transfers to
language-conditioned part grasping in real world.
Read the full abstract ▾
All authors Xinqiang Yu, Zekun qi, Jiawei He, Wenyao Zhang, Xuchuan Chen, Guaocai Yao, Li Yi, Zhaoxiang Zhang, He Wang
Computer Vision
Beyond Accuracy: Uncertainty-Guided Boundary Refinement for Reliable Biomedical Image Segmentation
Anima Kujur
Accurate biomedical image
segmentation requires not only high global overlap but also reliable delineation of clinically meaningful boundaries. In blood-smear microscopy, cytoplasm and
nucleus contours provide the structural basis for downstream morphology analysis; however, deep
segmentation models may remain uncertain or overconfident near ambiguous boundary regions even when achieving strong Dice scores. This work proposes a Reliability-Aware Boundary Refinement Network (RABR-Net), a two-stage framework for trustworthy image
segmentation. A strong UNet++ EfficientNet-B4 base segmenter first produces initial class probabilities and logits. Predictive
entropy, test-time augmentation variance, margin uncertainty, probability gradients, and soft boundary cues are then combined into a boundary-aware reliability representation. This representation guides a gated residual refiner that selectively corrects uncertain boundary pixels while preserving confident regions of the base prediction. The framework is evaluated using overlap accuracy, class-wise Dice, Boundary Dice, HD95/ASSD, calibration, risk–coverage analysis, robustness under image perturbations, qualitative correction maps, and paired statistical testing. On the held-out test set, the proposed method improves Dice from 0.9602 to 0.9614, Boundary Dice from 0.3448 to 0.3611, and HD95 from 3.0354 to 2.8274 compared with the cached base prediction. Statistical analysis confirms significant improvements in Dice, Boundary Dice, and HD95. Qualitative results show that the learned gate concentrates around uncertain cytoplasm and
nucleus boundaries, and correction maps confirm localized boundary refinement. Although calibration does not automatically improve after refinement, the proposed framework provides an interpretable and reliability-focused strategy for boundary-sensitive biomedical image
segmentation.
Read the full abstract ▾
Computer Vision
Learning Sign Language Recognition under Label Noise: A Study of Noise-Robust Losses for Isolated and Continuous Settings
Akihisa Shitara and Yoichi Ochiai
In sign
language recognition, the isolated (ISLR) classification loss treats a single label as ground truth, as does the frame-level auxiliary classifier over pseudo-labels we add to continuous (CSLR) methods, which lack one. Stylistic variation blurs ISLR annotation and the lack of temporal boundaries in CSLR forces pseudo-labels; both are noisy. We therefore apply symmetric and generalized cross
entropy (SCE, GCE), robust alternatives to cross
entropy (CE) from image classification, not to connectionist temporal classification but to the preceding single-label classifier. On ASL Citizen with injected symmetric noise on three backbones (three seeds for ST-GCN), robust losses cost at most 2.5 pt when labels are clean and beat CE by 2.9-10.0 pt in all six conditions at noise rate 0.2, one of which only after q was re-selected on dev. GCE gains more, but its optimal q does not transfer across backbones, whereas one SCE setting works in all nine conditions; both vary 2-11 times more than CE across runs, so a favorable point estimate does not establish stability. For CSLR (PHOENIX-2014) we report no gain; our frame-level targets carry a systematic assignment bias, making that study a diagnosis of a single configuration. At lambda_aux = 25 the pseudo-label CE auxiliary raises word error rate above the no-auxiliary baseline on VAC, CorrNet and SlowFastSign, and GCE/SCE improve on CE by 1.7-3.2 pt (three of six conditions return below that baseline). However, the three losses differ by more than an order of magnitude in effective gradient at a common lambda_aux: matching the initial gradient shrinks the gap to 0.4-0.9 pt, and lowering the CE weight alone already beats that baseline, so neither the degradation nor the improvement can be separated from the effect of the weight. We use only symmetric noise; multi-seed evaluation covers only ST-GCN and VAC isolated.
Read the full abstract ▾
Computer Vision
VideoTok4D: A 4D-Aware Video Tokenizer for Compact World Representation
Xinyi Chen, Hanxin Zhu, Xijun Wang et al. (+4 more)
Video tokenizers have emerged as a cornerstone of modern
video modeling, underpinning progress in compression, reconstruction and generation by mapping high-dimensional visual signals into compact latent spaces. However, despite this progress, current tokenization paradigms largely remain within the 2D visual domain, treating
videos as image sequences rather than observations of an underlying dynamic 3D world. Consequently, the learned
tokens inherit this observation-centric bias, limiting their capacity to compactly represent real-world 4D scenes. To mitigate this issue, we propose VideoTok4D, a novel 4D-aware
video tokenizer for compact world representation. Specifically, our approach comprises three key designs: 1) a spatiotemporal disentanglement strategy that factorizes
videos into static and dynamic
tokens for holistic world modeling; 2) a track-aware dynamic
attention mechanism that aggregates trajectory-aligned cues to promote cross-view motion consistency; and 3) Co4DGen, a
diffusion prior learned over the resulting VideoTok4D
token space for efficient 4D scene generation. Extensive experiments have demonstrated that our proposed method achieves state-of-the-art performance while requiring up to 4 orders of magnitude less storage than dense 4D representations. Moreover, the compact
token space substantially shortens
diffusion sequences, enabling efficient generation.
Read the full abstract ▾
All authors Xinyi Chen, Hanxin Zhu, Xijun Wang, Xingrui Wang, Sen Liang, Xin Li, Zhibo Chen
Computer Vision · Cross-listed from cs.RO
A Multi-Vehicle Dataset with Camera, LiDAR, and Radar Sensors and Scanned 3D Models for Custom Auto-Annotation using RTK-GNSS
Philipp Berthold, Bianca Forkel and Mirko Maehlisch
Datasets are a crucial element in the development of perception
algorithms. They relate sensor measurement data to annotated reference information and allow for the deduction of sensor and object characteristics. In autonomous driving, the reference data commonly consist of semantic image
segmentation, point-wise associations, or bounding box annotations. The
dataset proposed in this work, however, aims to dig deeper into the evaluation of measurement principles and provides scanned 3D models of all vehicles together with a pose and continuous kinematics reference obtained by RTK-GNSS. Combined, the state of the complete dynamic surrounding of the sensor vehicle is known for any point in time. Subsequent reference formats can be easily computed in user-defined granularity. This
dataset involves single-object and multi-object recordings with seven target vehicles. In particular, measurement effects such as occlusion, as well as reflections, can be evaluated, as the normals of the shape of the target vehicles are known. We describe the
dataset, discuss the technical background of its development, and briefly present exemplary evaluations.
Read the full abstract ▾
Computer Vision
3D CT-to-PET Translation via Latent Brownian Bridge Diffusion
Sarita Mourya, Francesco Di Feola, Pierangelo Veltri et al. (+1 more)
Computed
tomography (CT) and positron emission
tomography (PET) provide complementary anatomical and functional information for cancer diagnosis and treatment planning. However, the widespread use of PET is limited by high radiation exposure, elevated costs, and restricted availability. To address these limitations, deep learning-based CT-to-PET translation has emerged as a promising approach for synthesizing PET-like information directly from CT images, although accurately modeling the large cross-modal gap remains challenging. In this work, we propose a 3D CT-to-PET translation framework based on latent Brownian Bridge
Diffusion (BBDM). The method consists of two stages. First, a Variational Autoencoder (VAE) is trained on paired CT-PET patches, integrating contrastive learning to improve latent
alignment between anatomical and metabolic representations. Second, a BBDM is trained in the latent space to translate CT latent representations into their corresponding PET counterparts. The translated PET latents are then decoded and stitched to reconstruct the final 3D PET volume. We evaluate the proposed approach on two publicly available
datasets. Quantitative results based on image fidelity and lesion-level PET-specific metrics demonstrate improved performance compared with competing methods. In particular, the proposed approach improves PET signal fidelity, better preserves clinically relevant uptake patterns, and shows improved performance in preserving small-lesion metabolic activation, paving the way for virtual imaging applications.
Read the full abstract ▾
All authors Sarita Mourya, Francesco Di Feola, Pierangelo Veltri, Paolo Soda
Computer Vision
MGAvatar: Mesh-Bound Gaussians for Head Avatar Geometry and Appearance Modeling
Lei Shi, Sen Peng, Zhiyang Deng et al. (+4 more)
Accurate head modeling requires a stable yet expressive
geometric representation. Existing Gaussian-based head avatars commonly rely on parametric templates (e.g., FLAME) for Gaussian initialization and deformation, but these templates lack personalized priors and struggle to represent structures such as hair and clothing. To address this issue, we propose MGAvatar, a Gaussian-mesh hybrid representation that jointly models geometry and appearance through two Gaussian-mesh binding modes. Specifically, we introduce vertex-bound Gaussians and constrain their learnable parameters, enabling progressive mesh deformation to represent complex head geometry, while a pose-dependent offset module accounts for non-rigid deformations. Once geometry is stabilized, MGAvatar switches to face-bound Gaussians for appearance modeling. To improve appearance consistency across novel poses and viewpoints, we introduce a view-conditioned neural color field that alleviates artifacts caused by independently optimized Gaussian colors. In addition, we design a Gaussian offset network to predict Gaussian offset maps in the observation space, providing greater flexibility for face-bound Gaussians to capture dynamic facial textures. Extensive experiments on multi-view and
monocular videos show that MGAvatar outperforms existing methods in rendering quality, producing high-fidelity head avatars with rich texture details.
Read the full abstract ▾
All authors Lei Shi, Sen Peng, Zhiyang Deng, Zhonggui Chen, Xiaohu Guo, Baorong Yang, Xiao Dong
Computer Vision
Pre-Trained Low-Rank Tensor Decomposition for Multi-Dimensional Image Recovery
Bing-Zhang Fu, Zhi-Long Han, Ting-Zhu Huang et al. (+2 more)
Recently, tensor decompositions are prevalent for multi-dimensional image representation, which learn the instance-specific structure of each image from scratch. However, tensor decompositions neglect the common structure across different images, leading to limited semantic modeling capability, high computational cost, and a large number of learnable parameters. To address this challenge, we suggest the first pre-trained low-rank tensor decomposition (PLTD) framework, which organically integrates the pre-trained large
vision model into the classical tensor decomposition framework. Beyond the shallow and untrained deep tensor decomposition, the suggested PLTD achieves an unprecedented balance among higher recovery fidelity, fewer learnable parameters, and smaller carbon footprint. Specifically, PLTD factorizes the target tensor into a latent tensor and a learnable transform that maps the latent tensor back to the original data domain. The latent tensor consists of two indispensable and complementary terms, i.e., a fixed pre-trained latent tensor and a learnable low-rank latent tensor. The fixed pre-trained latent tensor is distilled from a pre-trained large
vision model (i.e., DINOv3) to capture the common structure of the target tensor, while the learnable low-rank latent tensor characterizes the instance-specific structure of the target tensor. To examine the potential of PLTD, we develop the corresponding multi-dimensional image recovery model and theoretically justify the advantages of this framework. Additionally, we discuss the connections between PLTD and classical tensor decomposition frameworks. Extensive experiments on multi-dimensional image recovery demonstrate that PLTD consistently achieves superior performance compared with state-of-the-art methods.
Read the full abstract ▾
All authors Bing-Zhang Fu, Zhi-Long Han, Ting-Zhu Huang, Xi-Le Zhao, Deyu Meng
Computer Vision
HemaHier: Chain-Conditioned Ordinal Hierarchies for Lineage-Aware Bone-Marrow Cytology
Afshin Bozorgpour, Peter Schüffler, Edgar Jost et al. (+1 more)
Bone-marrow cytology is inherently structured: each
cell belongs to a hematopoietic lineage, and many
cell types lie on ordered maturation trajectories. Standard flat classifiers ignore this structure, treating a mild same-lineage confusion the same as a severe cross-lineage mistake and predicting only discrete labels. We propose HemaHier, an ordinal-hierarchical prediction head for a frozen or lightly adapted cytology foundation model. Its central component is a chain-conditioned maturity score that reads a single maturity value under a per-chain query, supervised only on biologically valid healthy chains, while dysplastic and off-chain
cell types remain classes but are excluded from maturity supervision. Fine and lineage predictions are coupled through a shared posterior that guarantees hierarchical consistency, and a staged objective first stabilizes recognition, then adds lineage and maturity supervision. On three bone-marrow
datasets under a shared ontology, HemaHier achieves competitive recognition while reducing biologically severe errors and adding a within-lineage maturity ordering that flat classifiers lack. Code is available at https://github.com/xmindflow/HemaHier.
Read the full abstract ▾
All authors Afshin Bozorgpour, Peter Schüffler, Edgar Jost, Dorit Merhof
Computer Vision
Self-supervised Pre-training Helps Retinal Disease Progression Modelling Most When Data Is Scarce
Ifeoma Veronica Nwabufo, Julius Gervelmeyer, Sarah Müller et al. (+1 more)
Modelling how a disease progresses over time requires longitudinal imaging cohorts, which are scarce and small, whereas cross-sectional data – one image per participant – is abundant. Self-supervised pre-training on such data offers a way to bridge this gap, but it is unclear which strategy best supports progression modelling, or how that answer depends on the amount of labelled longitudinal data. We study this for age-related macular degeneration (AMD), pre-training encoders on the large cross-sectional NAKO cohort and predicting time to late AMD on the longitudinal AREDS
dataset. We compare in-house self-supervised encoders against a general-purpose (DINOv2) and a domain-specific (RETFound) foundation model, across contrastive, masked-autoencoding, and self-
distillation objectives, under frozen and fine-tuned protocols, and across labelled training sets from 100 to 32,250 examples. Which model performs best depends on how the encoder is used. When the encoder is frozen and labels are few – the regime typical of longitudinal cohorts – pre-trained representations reach clinically reasonable discrimination from a few hundred labelled samples, while models trained from scratch do not; this advantage fades under fine-tuning. Transfer is governed by the self-supervision objective rather than corpus scale or domain match, so that an encoder pre-trained on a modest cross-sectional cohort matches or exceeds a far larger in-domain foundation model. Together, these results offer a practical recipe for building progression models where longitudinal data is scarce: a frozen self-supervised encoder with a lightweight survival head.
Read the full abstract ▾
All authors Ifeoma Veronica Nwabufo, Julius Gervelmeyer, Sarah Müller, Philipp Berens
Computer Vision
Balancing Emotional Alignment and Semantic Consistency in Image Generation via Reinforcement Learning with Valence-Arousal Anchoring
Jisheng Dang, Zhenxuan Wang, Bin Li et al. (+3 more)
Continuous
emotion control in text-to-image generation requires a model to improve affective
alignment without changing the objects, layout, or scene described by the
prompt. Existing supervised
emotion-injection methods often optimize feature-space proxies and may therefore exhibit
emotion-semantic drift, in which stronger emotional conditioning is accompanied by unintended content changes. We address this problem with a flow-matching image-generation framework that combines continuous valence-arousal (VA) conditioning, Group Relative Policy Optimization (GRPO), and a neutral semantic anchor. The deterministic probability-flow ODE is converted into a marginal-preserving SDE, yielding non-degenerate transition densities for trajectory sampling and policy-ratio estimation. A frozen CLIP-based VA regressor supplies a terminal reward measuring the distance between the predicted and target VA coordinates, while an image generated from the same
prompt under zero VA conditioning provides a feature-space reference for semantic preservation. A reduced denoising schedule is used for online RL sampling, whereas the original schedule is retained at inference. Experiments on 3,300
prompt-
emotion combinations show substantially lower valence and arousal errors than the VA-conditioned baseline and an improved CLIPScore relative to EmotiCrafter, with a measurable trade-off in reference-free image quality. The results support anchor-regularized Flow-GRPO as a practical approach to balancing emotional
alignment and semantic consistency in continuous-affect image synthesis.
Read the full abstract ▾
All authors Jisheng Dang, Zhenxuan Wang, Bin Li, Ronghao Lin, Bin Hu, Tat-Seng Chua
Computer Vision
A Dual Cross-Attention Framework for Colposcopic CIN Grading and Swede Score Prediction Using a New Multi-Center Dataset
Dania Khan, Nuzhat Aisha Shaikh, Asfina Hassan Juicy et al. (+3 more)
Cervical cancer is a major global health challenge, with disease burden falling disproportionately on low- and middle-income countries (LMICs) due to a shortage of trained specialists and the subjective nature of colposcopy-based screening. To address this challenge, we propose a novel deep learning framework for the automated grading of Cervical Intraepithelial Neoplasia (CIN) and the prediction of clinical Swede scores. We also introduce the BUET Multi-Center Colposcopy
Dataset, a novel, multi-center cohort designed and annotated for Swede score prediction and CIN grading. Our proposed dual-stream cross-
attention architecture mimics the visual
reasoning of an expert colposcopist by explicitly fusing paired
multimodal cervigrams to evaluate comparative tissue responses. Furthermore, we introduce a custom composite loss function to address severe class imbalances and scoring inconsistencies across the five Swede score components. The proposed framework achieved 71.85% accuracy and an 86.23% AUC-ROC for three-class CIN grading, outperforming existing methods. For Swede score component prediction, the architecture achieved AUC-ROC values ranging from 75.7% to 88.4%, with the composite loss function yielding consistent F1-score improvements. Finally, the total predicted Swede Score, which ranges between 0 and 10, shows a Mean Absolute Error (MAE) of 1.489. The results show that the proposed method can pave the way towards developing AI-assisted colposcopy screening tools to support risk-based triage in resource-limited healthcare settings. The
dataset and source code are publicly available(url: https://github.com/mHealthBuet/BUET-colposcopy)
Read the full abstract ▾
All authors Dania Khan, Nuzhat Aisha Shaikh, Asfina Hassan Juicy, Raiyun Kabir, S M Shahida, Taufiq Hasan
Computer Vision
CoralscapesV2: Panoptic and Fine-Grained Visual Scene Understanding in Coral Reefs
Jonathan Sauder, Thomas Ruckli, Gabrielė Strodomskytė et al. (+17 more)
In order to design conservation and restoration strategies to counter the global decline of coral reefs, ecological monitoring of reefs needs to be scaled up dramatically. Computer
vision methods are increasingly used to tackle the vast amount of data: as the paradigm of data collection in reefs shifts from highly standardized and constrained survey images to unconstrained imagery on scalable platforms, it is necessary to design machine learning methods that help to get a fine-grained understanding of reefs from general-purpose reef imagery. This paper provides CoralscapesV2, an extension of the Coralscapes
dataset for general-purpose visual scene understanding in reefs. CoralscapesV2 increases the
dataset size, scope, label completeness and quality for semantic
segmentation, and extends the number of classes from 39 to 95 fine-grained visual categories. Furthermore, CoralscapesV2 provides 65k exhaustive fish instance mask annotations, meticulously annotated to completeness by using the
video, revealing that annotation of fish based on only static images is insufficient. CoralscapesV2 is the first
dataset for panoptic
segmentation in coral reefs, capturing a wide range of scenarios in the wild, posing a challenging
benchmark for contemporary semantic
segmentation and instance
segmentation models. CoralscapesV2 is an important step towards general-purpose panoptic
segmentation in coral reefs, which has substantial implications for scaling up coral reef monitoring, as it can be employed in a wide range of applications from benthic cover mapping from
robot or handheld
videos to designing methods for automated quantification and understanding of fish behavior and fish-reef interactions.
Read the full abstract ▾
All authors Jonathan Sauder, Thomas Ruckli, Gabrielė Strodomskytė, Ibrahim Souleiman Abdallah, Rahma Hassan Abdi, Djama Goumaneh Awaleh, Mohamed Houssein Farah, Moustapha Nour, Osama Sharhubil Saad, Mustafa Mohammed Khalafallah Altaib, Maysoon Kteifan, Farah Alsoqi, Eyad Zgool, Jafar Al-Omari, Temesgen Gebremeskel Gebreluel, Zekaria Zekeria Abdulkerim, Meron Ghirmay, Teklehaimanot Beraki, Devis Tuia, Guilhem Banc-Prandi
Computer Vision
SCDM: Spatial-Contextual Disentanglement Mamba via Differential Inference for Efficient Image Classification
Mustafa Bora Çelik, Hayriye Aktaş Dinçer and Ayse Keles
State Space Models (SSMs), particularly VMamba, have emerged as efficient alternatives for modeling long-range dependencies in medical image analysis. However, distinguishing subtle pathological features from visually similar anatomical backgrounds remains a significant challenge. Existing SSM architectures often learn entangled representations, lacking explicit mechanisms to separate disease-specific signals from normal anatomy. To address this limitation, we propose Spatial-Contextual Differential Mamba (SCDM), an asymmetric dual-branch architecture designed for selective representational disentanglement. SCDM introduces a Positive Branch for extracting discriminative features and a Negative Branch that actively models and suppresses normal anatomical context. This separation is achieved through a similarity-driven repulsion gate and a differential inference rule, which promote competitive feature learning without requiring additional branch labels or increasing model capacity. Evaluated on the RSNA Pneumonia
dataset, SCDM achieves competitive classification performance (AUC of 0.858) while requiring significantly fewer parameters (29.4M) and FLOPs (1.44G) compared to standard VMamba and
vision transformer baselines. Furthermore, activation analyses demonstrate that our differential mechanism yields highly precise localization, effectively isolating lesions by inhibiting irrelevant anatomical distractors.
Read the full abstract ▾
Computer Vision
Online Video Agent Harness for Long Video Understanding
Sen Yang, Boqiang Duan, Jing Yang et al. (+7 more)
Long
video understanding often behaves like a visual needle-in-a-haystack problem: query-relevant evidence is sparsely distributed across long temporal spans, while packing dense frames into a single VLM context incurs context rot and high cost. Existing
video agents often rely on query-agnostic offline preprocessing or ad hoc tool sets, which can miss query-specific details and waste computation. In this work, we present VideoXAgent, a purely online
video-
agent harness for long
video understanding that starts from the given
video file and user query, plans and decomposes the task, invokes specialized expert tools on demand, and aggregates
multimodal evidence to produce a final answer while resolving conflicts among observations. To support this on-demand invocation, we design a suite of heterogeneous expert tools guided by a data-driven taxonomy of atomic capabilities, spanning scripts, VLMs, and domain models (e.g., detection, OCR, ASR, face recognition). The harness further enforces objective evidence prompting and budget-aware control to curb
hallucination and non-termination. Across
Video-MME-Long, LongVideoBench-Long, LVBench, and MINERVA, VideoXAgent is competitive with frontier LMMs and
video agents under a smaller context footprint—about 50k
tokens of
agent context per sample, even on hour-long
videos. In particular, on complex
video-
reasoning benchmarks such as MINERVA, it matches this level while using only about 15% of the context of a 1,024-frame dense-packing baseline. Notably, the harness remains effective with a visually weak or even text-only orchestrator, suggesting that strong long-
video understanding can emerge from progressive agentic evidence seeking rather than from packing the full
video into a single context. Project page: https://go-
agent-x.github.io/
video_
agent_harness/
Read the full abstract ▾
All authors Sen Yang, Boqiang Duan, Jing Yang, Weihao Bo, Jie Liu, Boyuan Tong, Ze Feng, Wenkang Zhang, Jingdong Wang, Hua Wu
Computer Vision
LGFN: Lightweight Gated RGB-Polarization Fusion with Modality-Availability Conditioning for Camouflaged Object Detection
Zhuangfan Huang, Xiaosong Li, Yang Liu et al. (+2 more)
Camouflaged object detection (COD) is an important engineering task in intelligent optical perception, but it remains challenging when targets closely resemble their surroundings. Polarization imaging provides complementary physical cues, whereas existing methods typically assume fixed
multimodal input configurations and entangle intra-polarization coordination with interaction between red-green-blue (RGB) and polarization representations. We propose LGFN, a lightweight gated RGB-polarization fusion framework supporting separately optimized RGB-only and polarization-assisted configurations. A deterministic Modality Router selects the appropriate configuration according to polarization availability. In the
multimodal configuration, an availability-conditioned Modality Gate calibrates the available polarization branches; the Gated Polarization Hub coordinates learned degree of linear polarization (DoLP) and angle of polarization (AoP) representations with explicit polarization cues; and RGB-Polarization Cross Fusion introduces the coordinated representation into the RGB hierarchy through controlled residual interaction. The
multimodal configuration requires neither sample-dependent statistics nor handcrafted quality descriptors during inference. On the complete 230-image PCOD_1200 test set, the RGB-only configuration achieves a mean absolute error of 0.0090, a Dice score of 0.8806, and an intersection over union of 0.8144, obtaining the best results on all six metrics among the evaluated RGB-based methods. Under a common local reevaluation protocol, the
multimodal configuration outperforms PolarNet and IPNet on all six metrics. Relative to IPNet, it reduces the parameter count, floating-point operations, and latency by 53.1%, 73.6%, and 63.0%, respectively. The source code will be available at https://github.com/1hzf/LGFN.
Read the full abstract ▾
All authors Zhuangfan Huang, Xiaosong Li, Yang Liu, Tao Ye, Haishu Tan
Computer Vision
LG-PF: Lightweight Confidence-Guided Polarization Image Fusion
Zhuangfan Huang, Zhenyu Kuang, Gao Wang et al. (+3 more)
Polarization image fusion combines the stable luminance and structural information of the total- intensity image S0 with the material-sensitive details of the degree of linear polarization (DoLP) image. However, the reliability of DoLP varies spatially, and indiscriminate polarization transfer may amplify unstable responses or disturb the structural appearance anchored by S0. We therefore propose LG-PF, a lightweight confidence-guided framework that formulates polarization fusion as a selective residual transfer process. A Polarization Confidence Prior estimates spatially reliable polarization responses, a Mask-guided Multi-scale Fusion module regulates their transfer across three feature scales, and a Lightweight Context-aware Bounded Correction Head stabilizes local photometric and structural transitions. Confidence guidance is also incorporated into the optimization objectives to preserve reliable polarization details while suppressing unsupported responses. We also construct MSP, a multi-scene polarization fusion
dataset containing 1000 pixel-aligned image pairs from 17 indoor and outdoor scene categories. LG-PF achieves the best results across all six evaluated metrics on MSP, while subset-based evaluations on PIF and GAND show promising transferability without fine-tuning. With only 0.2936 M parameters and an inference time of 21.712 ms per image, LG-PF achieves competitive fusion quality with low computational cost. The source code will be available at https://github.com/1hzf/LG-PF.
Read the full abstract ▾
All authors Zhuangfan Huang, Zhenyu Kuang, Gao Wang, Yang Liu, Haishu Tan, Xiaosong Li
Computer Vision · Cross-listed from cs.AI
MPT: Missing Prototype Tracking via Barycentric Reconstruction in Vehicular Federated Learning
Hanju Jang, Gyeongmin Han, Sungmin Lee et al. (+3 more)
Cross-vehicle
federated learning enables vehicles to collaboratively improve perception models while keeping locally collected driving data private. However, vehicle participation is transient, and a vehicle may depart before training converges while permanently taking its local data. When this departing vehicle holds most samples of a target class, the class becomes rare in the remaining FL network, and its recognition can silently degrade as the shared backbone continues to evolve. Recovering the class is difficult since the few remaining samples provide a noisy prototype estimate, while FL
privacy constraints prevent centralized access to raw data or per-sample features. This paper presents MPT, a cross-vehicle FL framework that maintains rare-class recognition by reconstructing its prototype at every round from
privacy-preserving class-level statistics. MPT combines a barycentric decomposition that tracks drift shared with remaining-class prototypes, a covariance-based residual prediction that estimates out-of-span drift, and an adaptive calibration that weighs the remaining rare-class samples according to their reliability. We evaluate MPT on three vehicle classification tasks and four backbones against representative calibration and drift-compensation baselines. MPT outperforms all baselines in rare class F1, reaching 0.516 on the nuImages
dataset with only 1% of rare-class samples remaining, without raw data, per-sample features, or retraining.
Read the full abstract ▾
All authors Hanju Jang, Gyeongmin Han, Sungmin Lee, Kichang Lee, Chunghan Lee, JeongGil Ko
Computer Vision
Same Encoder, Different Winner: A Paired-View Framework for Cell Painting Encoder Evaluation
Tim Treis, Nikita Moshkov, Johan Fredin Haslum et al. (+2 more)
Vision encoders for
Cell Painting are typically ranked by a single evaluation, commonly replicate mean average precision (mAP). We introduce CP-BG-Bench, a paired-view evaluation framework that holds the central
cell fixed across four matched views (raw crop C, segmented S, and density-augmented variants CD and SD), ablating or augmenting surrounding pixels as a controlled intervention. Instantiating the framework on three
datasets (JUMP-CP, RxRx1, RxRx3-core) and three encoders (DINOv3 ViT-B/16, OpenPhenom, SubCell) under four community-standard protocols (replicate mAP, scIB batch integration, CellProfiler feature prediction, cross-batch perturbation recall), we find that the four protocols rank the same encoders systematically differently, with disagreements decomposing along three axes:
cell versus background, morphology versus context, and within-study versus across-batch. The largest effect: on RxRx3-core, SubCell with segmented inputs retains 94% of crop replicate mAP but only 32% of crop R@10, so the within-study signal preserved under
segmentation is largely non-transferable; density augmentation recovers 84% of the within-study C-to-S gap but only 8% of the cross-batch gap. Segmented views predict CellProfiler features as well as or better than crops on two of three
datasets, inverting the replicate-mAP ranking, and the C-to-S gap varies by an order of magnitude across
datasets while remaining similar across encoders, indicating that background-driven gain is set by experimental design rather than by the encoder. Single-metric ranking of
Cell Painting encoders is therefore sensitive to the protocol used, and protocol disagreements are interpretable as projections onto the three axes the paired-view design exposes. We will release the paired-view
datasets, reconstruction pipelines, 36 trained checkpoints, aggregated
embeddings, and the full evaluation suite.
Read the full abstract ▾
All authors Tim Treis, Nikita Moshkov, Johan Fredin Haslum, Shantanu Singh, Fabian J. Theis
Computer Vision
AquaCubeAI-Powered Monitoring Turbidity on-board {\Phi}sat-2
Pietro Di Stasio, Francesca Razzano, Elisa Liparulo et al. (+6 more)
Timely monitoring of coastal water quality is critical for environmental protection, yet conventional satellite workflows rely on downlink and ground processing, introducing latency that can limit responsiveness to rapidly evolving turbidity events. To address this limitation, we propose AquaCubeAI, a lightweight machine-learning approach for onboard estimation of coastal water turbidity from {\Phi}sat-2 multispectral imagery. By shifting inference from the ground segment to the satellite, AquaCubeAI aims to enable lower-latency, more responsive, and more operationally useful turbidity monitoring under the strict compute and bandwidth constraints of spaceborne platforms. The model is trained on simulated {\Phi}sat-2 acquisitions spatially aligned with Copernicus Marine Service (CMEMS) High-Resolution Ocean Color (HR-OC) turbidity products over selected localized coastal sites spanning four European marine macro-regions. To provide a realistic evaluation of generalization in the presence of spatial correlation, we adopt a spatial block splitting protocol that mitigates data leakage between training and evaluation subsets. The main contributions of this work are: (i) a scalable
dataset generation pipeline pairing simulated {\Phi}sat-2 multispectral patches with CMEMS HR-OC turbidity labels across selected localized European coastal sites; (ii) a compact Multi-Layer Perceptron (MLP)-based turbidity regressor trained under a leakage-aware geospatial split and tailored to embedded constraints; and (iii) a reformulation for dense spatial prediction via parameter sharing, enabling turbidity mapping and simple threshold-based anomaly masks for onboard decision logic. Embedded deployment on an Intel Myriad
Vision Processing Unit (VPU) further confirms the feasibility of low-power hardware and supports low-latency inference from multispectral inputs.
Read the full abstract ▾
All authors Pietro Di Stasio, Francesca Razzano, Elisa Liparulo, Gabriele Meoni, Nicolas Longépé, Deodato Tapete, Paolo Gamba, Gilda Schirinzi, Silvia Liberata Ullo
Computer Vision · Cross-listed from cs.LG
Physics-Guided Synthetic High-Frequency Ultrasound Generation for Skin Layer Segmentation
Junkyung ju, Kyungho Yoon and Minwoo Shin
High-frequency ultrasound (HFUS) enables noninvasive visualization of superficial skin structures, but automated skin-layer analysis is limited by the scarcity of densely annotated data. Existing real HFUS
datasets commonly provide annotations for superficial targets such as the epidermis and subepidermal low-echogenic band (SLEB), while dense labels for deeper structures such as dermis, subcutaneous tissue, fascia, and muscle are rarely available. We propose a physics-guided synthetic HFUS generation framework for skin layer
segmentation. The framework constructs multilayer acoustic skin phantoms, assigns layer dependent acoustic properties, and uses k-Wave simulation to generate paired synthetic HFUS images, dense layer masks, and simulation metadata. To evaluate whether the generated data provide transferable supervision, we use it for downstream
segmentation pretraining and fine-tune the models on real Mendeley HFUS data. Synthetic pretraining followed by real fine-tuning achieved real-domain performance comparable to real-only training and improved mean Dice/IoU in three of four evaluated trainable architectures. These results suggest that physics-guided synthetic HFUS images contain transferable anatomical and textural cues for real-domain skin layer
segmentation, although further reduction of the synthetic-real appearance gap is needed to enable greater gains. The code and data are available at: https://github.com/Finn-02/synthetic-hfus-skin-layer-
segmentation.
Read the full abstract ▾
Computer Vision
GRACE: Adaptive Concept Erasure with Geometry-Guided Retention in Diffusion Models
Qinghui Gong, Yihuai Liang, Yuanlun Xie et al. (+3 more)
Text-to-image (T2I)
diffusion models inevitably internalize sensitive or non-compliant concepts from large-scale pretraining data, necessitating post-hoc concept erasure. However, existing erasure methods often lack explicit constraints on parameter updates, leading to over-intervention and unintended semantic drift. In addition, many methods rely on manually crafted counterfactual supervision, such as surrogate
prompts, which incurs substantial data construction costs that limit scalability to new concepts. To address these limitations, we propose GRACE, a structured concept erasure framework designed to enable localized and selective intervention. Specifically, we introduce a semantically weighted sensitive subspace estimation to precisely lock intervention directions, and employ lightweight subspace-constrained adapters to prevent global semantic disturbance. To eliminate the dependency on manual
prompt engineering, we design an automatically decoupled safe-anchor mechanism. To mitigate semantic drift induced by excessive intervention, we introduce an energy-driven dynamic gating mechanism that adaptively controls the timing and strength of intervention at inference. Extensive experiments demonstrate that our method achieves a superior balance between erasure effectiveness and generation fidelity. Compared with the average performance of five state-of-the-art (SOTA) concept erasure methods, our method improves the fine-grained NSFW reduction rate by $17.86\%$, while reducing the macro-averaged target CLIP Score and preservation-oriented Fréchet Inception Distance (FID) by $4.75\%$ and $50.58\%$, respectively, indicating stronger concept suppression with substantially improved preservation of the original model's generative utility.
Read the full abstract ▾
All authors Qinghui Gong, Yihuai Liang, Yuanlun Xie, Deepak Kumar Jain, Vitomir Štruc, Zhengchun Zhou
Computer Vision
Multimodal Floorplan Encoding: Learning Dense Modality-Invariant Representations
Xavier Anadón, Rémi Pautrat and Rui Wang
Floorplans arise in many forms, from vector CAD drawings to raster renderings and sensor-derived density maps. This heterogeneity makes it difficult to build learning systems that transfer across modalities and support geometry-centric tasks such as
alignment and
retrieval. We introduce the
Multimodal Floorplan Encoder (MMFE), which maps diverse 2D indoor representations into a shared dense latent grid. MMFE combines a frozen DINOv3 backbone with a trainable Dense Prediction
Transformer (DPT) head, and is trained with a per-
cell Information Noise-Contrastive Estimation (InfoNCE) objective that aligns spatially corresponding regions across modalities while using all other
cells as negatives. To improve robustness to
geometric distortions, we incorporate controlled similarity transformations and enforce
geometric consistency through feature-grid warping. On Structured3D, a held-out out-of-domain
dataset, MMFE improves cross-modal dense matching, enables robust similarity
alignment with RANSAC, and yields strong
retrieval when paired with learned aggregation.
Read the full abstract ▾
Computer Vision
ExpertHTR: Unified Handwritten Text Recognition with Multi-Task Learning and Sparse Mixture-of-Experts
Dang Hoai Nam, Nguyen Duy Hieu, Quang Huu Hieu et al. (+1 more)
Handwritten text recognition resources are often small and distributed across collections that differ in
language, script, document structure, and annotation format, making joint page-level training difficult. We propose ExpertHTR, a unified
vision-
language framework that addresses this problem through complementary supervision and conditional model capacity. Structural annotations from heterogeneous
datasets are first organized through a common Page-Region-Line representation and used to construct four related training tasks for complete transcription, physical-line coverage, text localization, and localized recognition, without requiring additional manual labels. Building on a jointly trained dense model, ExpertHTR introduces a sparse Mixture-of-Experts architecture with an always-active shared branch and conditionally routed full-MLP experts. Sparsegen allows the number of active routed experts to vary with the hidden representation, while routing regularization reduces persistent concentration on a small subset of experts. Experiments on seven heterogeneous handwriting
benchmarks show that complementary supervision consistently improves training with page transcription alone, while joint multi-source training provides further gains on most
datasets. The proposed sparse expert model further improves the dense baseline on six of the seven sources. The final unified model also substantially outperforms the evaluated general-purpose OCR and
vision-
language systems on most
benchmarks and achieves state-of-the-art performance on the IAM paragraph-level
benchmark, while specialized HTR systems remain stronger on several challenging collections.
Read the full abstract ▾
All authors Dang Hoai Nam, Nguyen Duy Hieu, Quang Huu Hieu, Vo Nguyen Le Duy
Computer Vision
Semantically Aligned Gradient-Driven Context-Preserving Image Editing
Chiranjeev Chiranjeev, Muskan Dosi, Mayank Vatsa et al. (+1 more)
Instruction-guided image editing has a training-time blind spot. Generative editors are never required to semantically verify whether their outputs actually satisfy the instruction. Supervision stops at reconstruction and input textual-level conditioning. This produces incomplete edits, spatial spillover, and poor localization. We present IABEdit, a model-agnostic framework that embeds differentiable semantic verification into training. A frozen
vision-
language model extracts spatially-aware descriptors from the ground-truth edit. A trainable aligner then reproduces them from the generated output. The residual between the two becomes a gradient that teaches the generator both what to edit and where, with no inference-time VLM cost. IABEdit is compatible with diverse backbones, including U-Net (Stable
Diffusion) and MMDiT (FLUX), without altering their inference pipelines. On MagicBrush, it improves structural fidelity by +3.49 DINO-I over the best
diffusion baseline and +1.26 over the best overall baseline, while remaining competitive on instruction
alignment. It also achieves state-of-the-art instruction adherence performance on RealEdit and EMU Edit
benchmarks based on
embedding-based metrics. Most consequentially, on the D-LORD surveillance
benchmark, it surpasses the proprietary Gemini
agent by +5.13 DINO-P under heavy occlusion, where preserving identity is hardest. This shows that gradient-aligned VLM
distillation holds up under real-world-like surveillance and occlusion conditions. Human and GPT-4o evaluations confirm perceptually precise, well-localized edits.
Read the full abstract ▾
All authors Chiranjeev Chiranjeev, Muskan Dosi, Mayank Vatsa, Richa Singh