Computer Vision
Corentin Seutin, Mohamed Amine Ettaki, Michaël Clément et al. (+2 more)
Vision-
language segmentation models have recently achieved strong performance by leveraging high-level semantic object categories expressed in natural
language. However, this semantic dependence limits their ability to reason about intrinsic visual properties such as shape, geometry, or texture, which are essential in many real-world applications. In this work, we introduce Semantic-Agnostic aNd Shape-Aware (SANSA)
segmentation, a new paradigm that requires
segmentation models to operate solely from non-semantic textual descriptions. To this end, we propose two strategies to generate SANSA
segmentation prompts based on either dictionary constraints or example guidance, both generating semantic-agnostic textual descriptions. These
prompts are then used to finetune
segmentation models under semantic-agnostic supervision. Experiments show that finetuning on SANSA
prompts yields up to a 20% mIoU improvement on this new
segmentation task, compared to pretrained state-of-the-art models, while maintaining strong performance on standard semantic
prompts. These results highlight the importance of low- and mid-level visual
reasoning for improving the generalization and controllability of
vision-
language segmentation models.
Read the full abstract ▾
All authors Corentin Seutin, Mohamed Amine Ettaki, Michaël Clément, Pierrick Coupé, Rémi Giraud
Computer Vision
Transfer learning RGB models to hyperspectral images with trainable tensor decompositions
Mariette Schönfeld, Laurens Devos, Wannes Meert et al. (+1 more)
Transfer learning makes it possible to use large
vision networks on a variety of domains, by specializing their models' general filters to new tasks. However, these networks assume the input images to have 3 input channels, making them incompatible with multi- or hyperspectral images. Current approaches that mitigate this incompatibility sacrifice information in either the image, or the model. This work proposes a novel approach that preserves the image and spatial information present in the model by using partially trainable tensor decompositions. We create such decompositions of pretrained convolutional filters, separating the filters into spatial and spectral components. The spectral components are then replaced with trainable components of higher channel dimensionality. This creates hyperspectral filters that can specialize to new
datasets, while retaining the spatial patterns of the original filter. Experiments on a variety of hyperspectral
datasets show that our approach is more accurate and robust than other hyperspectral transfer learning methods.
Read the full abstract ▾
All authors Mariette Schönfeld, Laurens Devos, Wannes Meert, Hendrik Blockeel
Computer Vision
Inpainting-Style Conditional Diffusion for Multivariable Time Series Forecasting
Kourosh Kiani and S.M. Muyeen
In this paper, we propose a novel conditional
diffusion-based framework for multivariable time-series
solar power
forecasting. The proposed method reformulates temporal PV data as structured two-dimensional representations (images) using a sliding-window patch construction, enabling the application of Denoising
Diffusion Probabilistic Models (DDPM) within a unified spatiotemporal learning paradigm. A key contribution of this work is the formulation of
solar forecasting as an inpainting problem, where future time steps are treated as missing regions to be reconstructed. This is achieved through a mask-based conditional
diffusion mechanism, in which historical observations are preserved as conditioning context while the target (future) region is progressively corrupted and subsequently recovered via reverse
diffusion. The model learns to generate coherent future sequences conditioned on observed data, effectively performing time-series inpainting. To fully utilize all available features and ensure compatibility with U-Net architectural constraints, a zero-padding strategy is introduced to construct fixed-size inputs. The model is trained using a supervised denoising objective to predict injected noise, enabling accurate iterative reconstruction during the reverse process. Extensive experiments conducted on
benchmark PV
dataset, including GEFCom2014, demonstrate that the proposed approach achieves high
forecasting accuracy, particularly for short-term horizons. The results highlight the effectiveness of integrating
diffusion-based generative modeling with an inpainting formulation for robust, flexible, and high-fidelity
solar power
forecasting.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
EventShiftFlow: Towards Hardware-efficient FPGA-based Flow Estimation
Arianna Alonso Bizzi, Fernando Cladera and C. J. Taylor
Event-based
vision sensors offer asynchronous, high-temporal-resolution measurements that are attractive for low-latency robotic perception, but many event-based motion estimation methods are computationally intensive and difficult to map to FPGA hardware. We present a streaming velocity
estimator that discretizes asynchronous events into fixed-duration time bins, constructs a 1-bit spatial occupancy grid, and evaluates multiple velocity hypotheses in parallel using only fixed-width integer logic - shift registers, counters, comparators, and small LUT-mapped multiplies - with no dividers and no DSP blocks. It requires no frame reconstruction, no floating-point arithmetic, and no iterative optimization. The method deliberately trades dense sub-pixel optical flow for a sparse, quantized velocity estimate at each active pixel, suited to low-latency tasks such as reactive obstacle avoidance on size-, weight-, and power-constrained platforms. On noisy synthetic data with known ground-truth velocities, the method recovers both magnitude and direction, with magnitude estimates being most challenged when objects of different velocities intersect. On a real event-camera sequence, directional accuracy reaches 99.5% across all four evaluated motion segments, with performance remaining robust across occupancy densities in the 10-40% range. We characterize the
algorithm's density-dependent behavior, present a parameter sensitivity analysis, show that the proposed datapath requires less than 2 kB of storage, and implement a single-axis prototype on a low-cost Xilinx Artix-7.
Read the full abstract ▾
Computer Vision
EchoAvatar: Real-time Generative Avatar Animation from Audio Streams
Bohong Chen, Yumeng Li, Yinglin Xu et al. (+3 more)
Real-time synthesis of high-fidelity 3D character motion from audio is a pivotal component for next-generation interactive avatars and virtual assistants. However, most existing approaches are limited to offline processing of complete audio sequences or are constrained to specific domains, rarely handling both
speech and music effectively. In this paper, we introduce a novel framework designed to generate continuous, coherent full-body motion from streaming
speech and music with low latency. Central to our approach is a unified streaming architecture capable of synthesizing continuous motion from incremental audio inputs. We employ a robust training strategy that enforces strong audio dependency, allowing the model to seamlessly generalize across conversational
speech and rhythmic music without requiring explicit domain labels or mode switching. Additionally, we explored Reinforcement Learning to refine the quality of online generation. Furthermore, we bridge reactive animation with intent-driven behavior via a tool-call interface that allows upstream Large
Language Models to inject explicit semantic control. By combining this controllability with stream audio-driven synthesis, our framework serves as a plug-and-play solution for transforming voice
agents into interactive
humanoid avatars. Extensive experiments demonstrate that our method outperforms state-of-the-art realtime baselines in motion quality and synchronization while maintaining the flexibility required for live deployment. Our code, pre-trained models, and
videos are available at https://robinwitch.github.io/EchoAvatar-Page.
Read the full abstract ▾
All authors Bohong Chen, Yumeng Li, Yinglin Xu, Youyi Zheng, Yanlin Weng, Kun Zhou
Computer Vision
LV-OSD: Language-Vision-Complementary Open-Set Object Detection
Yupeng Zhang, Ruize Han, Wei Feng et al. (+2 more)
Object detection is an important task in computer
vision, which aims to detect the objects of interest. through the given category list or query images. In this work, we propose a new problem of
language-visual-complementary open-set object detection (LV-OSD), i.e., using the flexible text-based and/or image-based
prompts to specify the desired object categories. This setting is more common and practical in real-world applications. For this purpose, we design a dual-branch detection framework, LVDor, which can simultaneously accept both text and image
prompts. Specifically, we first build the Multi-modal
Prompts (MPr) containing various text descriptions and image samples for each category. Subsequently, to bridge the semantic gap among the input image, text
prompts, and image
prompts, we design a Target-guided
Prompt Dynamic Weighting (TPDW) module. Guided by the prior information of the target image, this module dynamically produces the text and image
prompts that best align with the target semantics, achieving precise
alignment and effectively reducing the discrepancy between the two modalities, thereby accommodating the LV-OSD setting. We also propose a simple
Prompt Random Masking (PRM) mechanism during training to simulate the arbitrary combination of text and/or image
prompts in testing. Extensive experimental results verify our problem formulation's reasonability and our method's effectiveness.
Prompts and code will be released publicly.
Read the full abstract ▾
All authors Yupeng Zhang, Ruize Han, Wei Feng, Song Wang, Liang Wan
Computer Vision
Every9D-22M: Large-Scale Real-World 9D Canonicalization of Everyday Objects
Leonhard Sommer, Emil Akopyan and Adam Kortylewski
Estimating the 9D pose of everyday objects from a single real-world image remains challenging. This is largely due to the lack of large-scale supervision. Most existing
datasets either rely heavily on synthetic renderings or provide limited coverage of real-world objects. We address this gap with Every9D-22M, a
dataset of 9D pose annotations for 22.8M real-world images from 114K object-centric
videos spanning 630 everyday object categories, making it nearly two orders of magnitude larger than any prior real-world 9D pose
dataset in at least one of category, object, or image count. To achieve this scale, we leverage object-centric
videos by reconstructing object-level point clouds via multi-view geometry and aligning similar instances into a shared canonical coordinate frame. Canonical poses are manually annotated for only a small set of reference objects (fewer than 0.01% of all images) and propagated to the remaining instances via cross-instance
alignment. All propagated canonical poses are then verified from multiple viewpoints. We further introduce cross-category orientation rules that induce category-level
symmetries, enabling
symmetry-aware evaluation. Beyond establishing dedicated training and evaluation splits as a
benchmark for 9D pose foundation models, we show that training on Every9D-22M generalizes better to other
datasets than training on existing
datasets, and outperforms prior foundation models in rotation accuracy on all
benchmarks except the viewpoint-biased ImageNet3D, where joint training with ImageNet3D surpasses them as well. Data and code are available at https://github.com/GenIntel/Every9D.
Read the full abstract ▾
Computer Vision
MORI-Seg: Learning Morphological Geometry for Instance Segmentation without Instance Annotations
Leiyue Zhao, Tianyu Shi, Daniel Reisenbuchler et al. (+12 more)
Instance-level quantification of kidney functional units is essential for morphometric analysis, yet most publicly available pathology
datasets provide only semantic
segmentation annotations, where adjacent structures of the same class are merged into single regions. This prevents reliable instance-level analysis and limits downstream quantitative studies. Existing heuristic post-processing methods often yield suboptimal instance separation, particularly in crowded and adherent regions, while deep learning-based instance
segmentation approaches typically require intensive instance-level annotations that are costly and labor-intensive to obtain. We propose MORI-Seg, a deep learning framework that enables instance
segmentation without requiring instance-level annotations. Instead of heuristic splitting or instance supervision, MORI-Seg learns morphology-aware
geometric representations directly from semantic masks by jointly modeling object-centric distance fields and boundary-band representations to encode interior structure and contact interfaces. A class-conditioned feature disentanglement module further promotes intra-instance coherence and inter-instance separation. Under semantic-only supervision, MORI-Seg decomposes connected semantic regions into distinct instance masks in an end-to-end manner. Experiments demonstrate improved instance separation accuracy and more reliable morphometric quantification compared with classical post-processing pipelines and representative semantic-to-instance learning approaches. The official implementation is publicly available at https://github.com/ddrrnn123/MORI-Seg.
Read the full abstract ▾
All authors Leiyue Zhao, Tianyu Shi, Daniel Reisenbuchler, Xinzi He, Junchao Zhu, Tianyuan Yao, Yuechen Yang, Yanfan Zhu, Junlin Guo, Gelei Xu, Haichun Yang, Yuankai Huo, Mert R. Sabuncu, Yihe Yang, Ruining Deng
Computer Vision · Cross-listed from cs.SE
GUI Agents for Continual Game Generation
Yixu Huang, Bo Li, Na Li et al. (+8 more)
Generating a game is not the same as making one playable. Existing code-generation approaches often translate a
prompt directly into an artifact, leaving interaction-level failures undetected. We argue that game generation requires a player and study two roles for graphical user interface (GUI)
agents. First, we introduce PlaytestArena, an evaluation environment containing 200 browser-based game-generation tasks across eight genres, each paired with rubrics of expected in-play behaviors. An independent GUI judge loads and plays each build to adjudicate these rubrics. Second, we propose Play2Code, in which a game
agent and a rubric-blind GUI playtester iteratively generate, play, and refine games through shared
memory. The playtester provides gameplay traces and actionable feedback, while a separate GPT-5.5 judge assigns final
benchmark scores. Across three frontier backbones, Play2Code achieves a 66.8% rubric pass rate, outperforming single-pass and agentic-coding baselines by 37.1 and 14.6 points, respectively. Its scores also improve monotonically across refinement rounds. Further analysis shows that GUI-
agent feedback is fully logged and traceable, while its priorities vary substantially across model backbones. These results establish GUI playtesting as an evaluation and refinement signal for interactive code generation. Our project website is available at https://continual-game-generation.vercel.app/
Read the full abstract ▾
All authors Yixu Huang, Bo Li, Na Li, Zhe Wang, Kaijie Chen, Haonan Ge, Qingyi Si, Yuanzhe Shen, Ruihan Yang, Guangjing Wang, Hongcheng Guo
Computer Vision
Category-Level 3D Correspondence in Camera Space via Morphable Object Priors
Leonhard Sommer, Artur Jesslen, Basavaraj Sunagad et al. (+1 more)
Understanding 3D objects from images is fundamental to robotics and AR/VR applications. While recent work has made progress in category-level pose estimation, current representations fail to capture the fine-grained semantics needed for
reasoning about object parts, functions, and interactions. In this work, we study category-level 3D correspondence in camera space – predicting, from a single image, 3D locations that remain consistent across instances within a category – and show that it can emerge without explicit correspondence supervision by learning a shared morphable object prior. To enable research in this direction, we introduce HouseCorr3D, the first large-scale
benchmark for
monocular category-level 3D correspondence with 178k images across 50 household object categories, 280 unique instances, and 3D keypoint annotations directly on CAD models. Crucially, HouseCorr3D provides amodal correspondence labels for occluded regions and explicit
symmetry annotations, addressing key limitations of existing
datasets. We further propose Morpheus, a method that learns morphable category-level shape priors by disentangling canonical shape, deformation, and object pose. Through this shared canonical grounding, semantically meaningful 3D correspondences in camera space emerge implicitly. These emerging 3D correspondences set a new state of the art on HouseCorr3D, demonstrating that semantic 3D object understanding can arise without direct correspondence supervision. Data and code are publicly available at https://github.com/GenIntel/HouseCorr3D.
Read the full abstract ▾
All authors Leonhard Sommer, Artur Jesslen, Basavaraj Sunagad, Adam Kortylewski
Computer Vision
PointQ-Bench: Benchmarking Diagnostic and Interpretable Point Cloud Quality Assessment
Duanchu Wang, Cheng Li, Junjie Yang et al. (+5 more)
Point cloud quality plays a critical role in 3D acquisition, reconstruction, rendering, and perception, yet existing point cloud quality assessment (PCQA) research remains largely centered on scalar score prediction. In practical inspection scenarios, quality assessment often involves identifying defects, characterizing dominant issue types, assessing downstream usability, and providing evidence-supported descriptions, which are not explicitly evaluated by current
benchmarks. We introduce PointQ-Bench, a
benchmark designed to extend PCQA from scalar scoring toward comprehensive quality understanding. PointQ-Bench consists of 3,083 point clouds spanning authentic scans, simulated distortions, and AI-generated content, covering eight major issue types. Each sample is annotated with mean opinion scores (MOS), quality levels, issue tags, expert-grounded descriptions, and 12,332 question-answer pairs. The
benchmark supports three perception-oriented tasks: anomaly sensing, defect diagnosis, and usability grading, as well as a cognition-oriented task of open-ended quality reporting. To evaluate free-form quality descriptions, we further propose SSFRQ-5D, a five-dimensional evaluation protocol validated through human-AI agreement analysis. Extensive experiments on 14
vision-
language models and traditional PCQA baselines reveal a consistent perception-diagnosis gap: while current models exhibit emerging abilities in coarse defect perception, they struggle with grounded diagnosis and quality calibration. Strong 2D MLLMs generally outperform existing 3D VLMs, and the benefit of additional views or point-level inputs is non-uniform, varying across tasks, data sources, and models, particularly under boundary-ambiguous conditions. Overall, PointQ-Bench provides a diagnostic testbed for advancing reliable and interpretable point cloud quality understanding.
Read the full abstract ▾
All authors Duanchu Wang, Cheng Li, Junjie Yang, Jing Huang, Zihang Cheng, Zhi Gao, Bohong Zhu, Di Wang
Computer Vision
Learning to Label: A Reinforced Self-Evolving Framework for Semi-supervised Referring Expression Segmentation
Runlong Cao, Ying Zang, Chuanwei Zhou et al. (+4 more)
Semi-supervised referring expression
segmentation (SS-RES) aims to achieve precise pixel-level
language grounding under limited annotation, yet suffers from limited supervision and unreliable pseudo-labels when exploiting unlabeled image-text pairs. In this work, we propose Learning to Label, a reinforced self-evolving framework (L2L) that casts pseudo-label construction as a learnable decision-making process. To build foundational understanding, we leverage a
multimodal large
language model to extract semantic-spatial priors, which are instantiated as initial soft
segmentation proposals and elevated, together with textual cues, into learnable guidance signals that condition a hierarchical
segmentation network. To ensure stable learning, reinforced pseudo-label selection is formulated as an exploratory decision process that adaptively rewards high-utility pixel-level supervision based on
multimodal priors and model predictions. This reinforced self-evolving loop enables joint optimization of the
segmentation model and pseudo-labels, progressively enhancing label reliability under sparse supervision. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg demonstrate improvements over existing methods, validating its effectiveness and generalization.
Read the full abstract ▾
All authors Runlong Cao, Ying Zang, Chuanwei Zhou, Tianrun Chen, Tong Zhang, Zhen Cui, Chunyan Xu
Computer Vision · Cross-listed from cs.RO
POINav: Benchmarking and Enhancing Final-Meters Arrival in Real-World Vision-Language Navigation
Ruiyan Gong, Meisheng Zhang, Yuxiang Zhao et al. (+12 more)
Real-world
navigation is fundamentally driven by Points of Interest (POIs), yet reaching a precise POI remains a critical "final-meters" challenge. Existing
Vision-
Language Navigation (VLN)
benchmarks of POI-goal
navigation often suffer from coarse granularity or significant sim-to-real gaps due to generated scene. To bridge this gap, we present POINav-Bench, the first
benchmark designed for closed-loop evaluation of real-world POI-goal
navigation. It comprises 11 commercial areas reconstructed from real-world captures using 3D Gaussian Splatting (3DGS), covering 126,398 $m^{2}$ in total and spanning 163 distinct POIs. With traversability-aware annotations and reference trajectories, POINav-Bench enables high-fidelity evaluation of
navigation agents in realistic, POI-rich real-world environments. Building on this, we propose the POINav
Brain-Action Framework where a
Brain module performs POI-grounded
reasoning to guide an Action module in predicting continuous waypoints for real-world execution. We further curate the POINav-
Dataset, containing 70K real-world signage-entrance pairs. Experiments show that our framework provides a viable path toward refining real-world POI-goal
navigation.
Read the full abstract ▾
All authors Ruiyan Gong, Meisheng Zhang, Yuxiang Zhao, Mingchao Sun, Yanfen Shen, Zedong Chu, Zhining Gu, Wei Guo, Xiaolong Cheng, Qiming Li, Kangning Niu, Yanqing Zhu, Xiaolong Wu, Tianlun Li, Mu Xu
Computer Vision
Bridging the Sampling Distribution Shift in Radio Map Estimation: A Trajectory-Aware Paradigm
Feng Qiu, Zheng Fang, Shuhang Zhang et al. (+4 more)
Learning-based
radio map estimation (RME) plays a critical role in
UAV-assisted wireless sensing, enabling tasks such as coverage prediction and network optimization. Most current methods assume an independently and identically distributed (i.i.d.) training and testing setting based on random sampling. However, practical
UAV measurements are collected sequentially along feasible trajectories, resulting in highly structured and spatially correlated patterns. This mismatch introduces a sampling distribution shift that increases the intrinsic difficulty of spatial field recovery and compromises the generalization of models trained under i.i.d. assumptions. To mitigate this issue, we propose a trajectory-aware training paradigm based on
Stochastic-Triggered Trajectory-Based Sampling (ST-TBS), which preserves trajectory continuity while introducing sampling variability. Moreover, from a statistical perspective, we show that trajectory-based sampling reduces spatial diversity and increases information redundancy compared to random sampling. Extensive experiments on the RadioMapSeer and SpectrumNet
datasets demonstrate that models trained with random sampling suffer significant performance degradation under trajectory-based observations, with RMSE increasing from 0.0391 to 0.2632 on SpectrumNet. Conversely, our proposed ST-TBS method effectively reduces the RMSE to 0.0571. These results highlight the necessity of aligning training and deployment sampling distributions for reliable RME.
Read the full abstract ▾
All authors Feng Qiu, Zheng Fang, Shuhang Zhang, Kangjun Liu, Longkun Zou, Jing Liu, Ke Chen
Computer Vision
Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation
Mariam Hassan, Kaouther Messaoud, Wuyang Li et al. (+1 more)
Modern
video generative models produce visually impressive results, yet frequently violate basic physical principles. We propose Proprio, a training-free framework that enables a frozen
video generator to assess and improve the physical plausibility of its own outputs. Inspired by proprioception, the biological sense of one's own movement, Proprio treats the model's flow residual under controlled latent perturbations as a self-scoring signal. Samples that are better explained by the generator's learned dynamics induce smaller and more stable residuals. We aggregate this signal across timesteps and perturbations, focus it on motion-relevant regions with a dynamic spatiotemporal mask, and use it for best-of-N search, gradient-based self-refinement, or both. Across text-to-
video and image-to-
video benchmarks, Proprio consistently improves physical plausibility, outperforming VLM-based scoring, and external world-model baselines in several settings. With TurboWan2.2, Proprio improves Physics-IQ from 32.2 to 37.5 (+16.5%) and VideoPhy2-hard physical commonsense from 45.6 to 55.0 (+20.6%). Human evaluation further shows that raters prefer Proprio-selected or refined
videos for physical plausibility in roughly two-thirds of comparisons. These results suggest that frozen
video generators contain actionable internal signals for evaluating and improving the physical plausibility of their own outputs.
Read the full abstract ▾
All authors Mariam Hassan, Kaouther Messaoud, Wuyang Li, Alexandre Alahi
Computer Vision
VidPrism: Heterogeneous Mixture of Experts for Image-to-Video Transfer
Rui Lin, Chuanming Wang and Huadong Ma
With the rapid development of pre-training technologies, adapting large-scale
Vision-
Language Models (VLMs) for
video understanding \ie image-to-
video transfer learning has become a dominant paradigm. To achieve superior performance, it raises as an effective strategy among recent advances to employ Mixture-of-Experts (MoE) to enhance VLMs' temporal modeling capabilities. However, conventional MoE designs suffer from expert homogenization, where all experts act as identical generalists, inefficiently learning spatio-temporal features from undifferentiated
video streams. To overcome this problem, we propose VidPrism, a novel heterogeneous temporal Mixture-of-Experts framework. VidPrism pioneers a division of labor by deploying functionally specialized experts, each assuming a role ranging from spatial understanding to temporal modeling. To feed these specialists appropriately, we introduce a content-aware, multi-rate sampling module that dynamically generates streams ranging from semantically rich to motion-focused representations, providing specialized inputs for experts. Furthermore, a dynamic, bidirectional fusion mechanism enables synergistic information exchange between these pathways, leading to a comprehensive
video representation. Extensive experiments on various
video recognition
benchmarks demonstrate that VidPrism achieves state-of-the-art performance and effectively fosters expert specialization. Our source code is available at \href{https://github.com/Lrrrr549/VidPrism.git}{https://github.com/Lrrrr549/VidPrism.git}.
Read the full abstract ▾
Computer Vision
A Patient-Specific Pulmonary Arterial Tree Digital Twin to Extract Pulmonary Embolism Biomarkers
Morgane des Ligneris, Nathan Painchaud, Allan Serva et al. (+4 more)
Pulmonary embolism, the obstruction of a pulmonary artery by a blood clot, is one of the leading causes of acute cardiovascular syndrome. In clinical practice, therapeutic decisions after diagnosis via computed
tomography pulmonary angiography rely on risk stratification, which categorizes 30-day mortality risk into three categories. This stratification depends on the right-to-left ventricular diameter ratio and blood levels of two cardiac enzymes. However, blood biomarkers are not always available in emergency settings, and manual calculation of established severity scores - such as Qanadli and Mastora - is time-consuming and rarely performed in clinical routine practice. This study introduces an automated pipeline that models a directed graph representation of the pulmonary arterial tree, labeling its hierarchical structure and characterizing pulmonary embolism. The pipeline derives image-based biomarkers, including local artery-level features (morphological information, hierarchical position, clot volume, and resulting obstruction) and global patient-level biomarkers such as automatically calculated severity scores (Qanadli and Mastora) and the total embolic volume distribution by lobes and hierarchical levels. Using artificial-intelligence-generated binary masks of arteries, emboli, lungs, and lobes, it creates a patient digital twin of the arterial structure. Validation of the pipeline through comparison to an existing pipeline, anatomical expectations, and manual severity score calculations demonstrates the pipeline's ability to automatically generate anatomically accurate digital twins and severity scores with strong agreement. This supports the potential of these image-derived biomarkers to automatically provide rapid, precise information on thrombotic burden and spatial clot distribution.
Read the full abstract ▾
All authors Morgane des Ligneris, Nathan Painchaud, Allan Serva, Laurent Bertoletti, Pierre Croisille, Carole Frindel, Odyssée Merveille
Computer Vision
From Kellgren-Lawrence to Calcium Pyrophosphate Crystal Deposition: A Soft-Labelling Framework for Knee Osteoarthritis Assessmen
Francisco Bérchez-Moreno, Riccardo Rosati, Maria Chiara Fiorentino et al. (+6 more)
Background and objective. Conventional Deep Learning (DL) approaches for Knee Osteoarthritis (KOA) grading rely on one-hot labels, which fail to capture both the ordinal uncertainty of Kellgren–Lawrence (KL) and Calcium Pyrophosphate Deposition Disease (CPPD) severity scores and the asymmetric relationship between the two scales observed in clinical practice. Methods. We retrospectively collected 2172 knee X-ray images, including 968 radiographs jointly annotated for KL and CPPD severity. An ordinal DL framework based on soft-labelling was developed for both tasks, replacing one-hot targets with unimodal probability distributions centred on the annotated grade. Four formulations were investigated: binomial, beta, triangular, and exponential. Results. All soft-labelling strategies consistently outperformed the nominal baseline. For CPPD grading, the triangular formulation achieved the highest Quadratic Weighted Kappa (QWK) and the lowest Mean Absolute Error (MAE) (QWK = 0.796; MAE = 0.438), while the beta formulation yielded the most balanced class-wise performance considering Average MAE (AMAE) and Maximum MAE (MMAE) across classes (AMAE = 0.458; MMAE = 0.573). For KL grading, the beta-based approach provided the best overall performance, achieving the highest QWK together with the lowest MAE and class-wise errors (QWK = 0.777; MAE = 0.529; AMAE = 0.523; MMAE = 0.775). Statistical analysis demonstrated significant improvements over conventional one-hot supervision (p < 0.001).
Read the full abstract ▾
All authors Francisco Bérchez-Moreno, Riccardo Rosati, Maria Chiara Fiorentino, Víctor M. Vargas, Edoardo Cipolletta, Emilio Filippucci, Luca Romeo, Pedro A. Gutiérrez, César Hervás-Martínez
Computer Vision
FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales
Jorge L. Rodriguez, Victor Angulo Morales, Areej Alwahas et al. (+6 more)
Foundation models offer a promising route to transferable remote sensing representations, but many current approaches depend on very large pretraining
datasets and fixed sensor configurations, limiting their suitability for ecological and environmental applications, where observations often vary across platforms, spatial and spectral resolutions, and available modalities. We introduce FLORO, a
multimodal geospatial foundation model designed to learn transferable representations from a small but highly diverse remote sensing corpus. FLORO is pretrained using masked autoencoding on a heterogeneous combination of Sentinel-1, Sentinel-2, SkySAT imagery, elevation, and
UAV-derived data. To accommodate sensor variability, FLORO incorporates availability-aware inputs that indicate which spectral bands and auxiliary modalities are present in each sample, enabling a unified input space across heterogeneous sensor configurations. We evaluated FLORO on the PANGAEA
benchmark under a frozen-encoder protocol across scene classification,
segmentation, and regression tasks. Despite being pretrained on a smaller corpus than competing foundation models, FLORO achieved strong and stable transfer across optical, optical-SAR, and optical-elevation
benchmarks spanning medium-resolution satellite, airborne, and ultra-high-resolution
UAV imagery. FLORO obtained the second-best average
segmentation performance across six PANGAEA
benchmarks, the third-best average rank across three scene-classification
benchmarks, and excelled in regression tasks, while qualitative results showed improved preservation of spatial structure in flood, urban, biomass, and canopy-height prediction settings. Overall, these results position FLORO as a practical framework for ecological and environmental monitoring under heterogeneous sensing conditions.
Read the full abstract ▾
All authors Jorge L. Rodriguez, Victor Angulo Morales, Areej Alwahas, Mariana Elias Lara, Fida Mohammad Thoker, Kasper Johansen, Bernard Ghanem, Fernando T. Maestre, Matthew F. McCabe
Computer Vision
MangaFlow: An End-to-End Agentic Framework for Controllable Story to Manga Generation
Muyao Wang, Zeke Xie, Yanhao Chen et al. (+2 more)
End-to-end manga generation is a structured visual storytelling task that requires story decomposition, recurring character and scene grounding, page layout design, panel rendering, page composition, and lettering. However, existing generative models often perform direct page synthesis, entangling these factors in a single visual output and limiting precise control over layout geometry, visual references, and cross-panel consistency. To address these limitations, we propose MangaFlow, an agentic framework for controllable long-form manga generation that decomposes manga creation into planning, grounding, layout construction, reference-conditioned rendering, composition, and text placement. By treating layout and visual references as explicit intermediate variables, MangaFlow enables both simple text-to-manga generation and more precise user-controlled manga creation. This design exposes layout, visual assets, and lettering as editable intermediate controls for refining panel geometry, references, and text placement. To support long-form consistency, MangaFlow introduces a story section
memory that links section descriptions with corresponding character, scene, and object references for reuse across panels. We further present a meta-
benchmark for evaluating layout controllability, visual consistency, and generation quality. Experiments show that MangaFlow improves layout adherence and cross-panel consistency over direct generation baselines while supporting flexible human control.
Read the full abstract ▾
All authors Muyao Wang, Zeke Xie, Yanhao Chen, Lixin Xiu, Hideki Nakayama
Computer Vision
DebFilter: Eradicating Biases Stashed in Value
Seung Hyuk Lee and Songkuk Kim
Text-to-image
diffusion models, which are theoretically equivalent to score-based generative models, generate images through a multi-step denoising process guided by text
embeddings extracted from pretrained
vision-
language models such as CLIP. However, these text
embeddings inherently encode social and semantic biases – such as those related to gender and age – that are subsequently propagated and amplified through the guidance mechanism, along with the model's training on large-scale
datasets that are imbalanced with respect to these bias-related concepts, often leading to skewed outputs in text-to-image generation. We propose DebFilter, a lightweight and training-free framework for mitigating such biases in text-to-image
diffusion models. Observing that the model's error prediction at each denoising step is primarily influenced by cross-
attention dynamics, we introduce a bias-correction strategy that adjusts the value components within cross-
attention. Specifically, we apply a fixed offset to the slice of guidance
embedding, effectively steering the semantic direction of cross-
attention values toward unbiased representations. This adjustment reconfigures the score landscape to produce balanced outputs while maintaining
alignment with the intended text semantics. Unlike prior approaches that rely on fine-tuning or retraining, DebFilter operates entirely at inference time, requiring no additional data or model updates. Our results demonstrate that this method effectively mitigates social biases in generated images, offering an efficient and scalable pathway toward fairer and more inclusive text-to-image generation.
Read the full abstract ▾
Computer Vision
MeniOmni: A Structured Multimodal Benchmark for Holistic Meniscus Injury Assessment
Shurui Xu, Siqi Yang, Weiping Ding et al. (+4 more)
Clinical diagnosis of meniscus injuries requires radiologists to integrate volumetric MRI evidence with patient context (e.g., sex, age, BMI) and to produce structured diagnostic reports. Existing knee MRI
benchmarks are typically unimodal and rely on coarse labels, limiting their ability to evaluate holistic clinical
reasoning. We introduce MeniOmni, a structured
multimodal benchmark for meniscus injury assessment, consisting of 746 multi-center MRI studies with tri-planar volumetric inputs, Clinical Priors, and expert-annotated clinical text. MeniOmni supports two tasks: (1) fine-grained Stoller severity grading and (2) diagnostic report generation. We further propose risk-aware ordinal evaluation and a semantic consistency metric (Meni-Score) to better reflect clinical relevance. Baseline experiments show that incorporating Clinical Priors improves grading performance and reduces severe errors, highlighting the value of
multimodal context for safer assessment. Code and data are available at https://github.com/ShuruiXu/MeniOmni.
Read the full abstract ▾
All authors Shurui Xu, Siqi Yang, Weiping Ding, Hui Wang, Mengzhen Fan, Yuyu Sun, Shuyan Li
Computer Vision
Intra-YOLO: A Small Object Detection Model for Caries and Molar-Incisor Hypomineralization in Intraoral Photography Based on Transfer Learning with Reinforcement Learning
Po-Lun Chwang, Po-Yu Chang, Wen-Liang Lin et al. (+3 more)
This study developed a computer-aided diagnosis (CAD) system for detecting caries and molar-incisor hypomineralization (MIH) in intraoral photographs. These lesions share similar appearances, making clinical differentiation challenging, especially given their small size and variability in imaging conditions.
Read the full abstract ▾
All authors Po-Lun Chwang, Po-Yu Chang, Wen-Liang Lin, Tung-Sheng Wu, Min-Ching Wang, Yun-Chien Cheng
Computer Vision
A novel ordinal multi-view aggregation scheme for oak defoliation
Francisco Bérchez-Moreno, Ricardo Enrique Hernández-Lambraño, David Guijo-Rubio et al. (+4 more)
Forest decline driven by
climate and biotic stressors threatens ecosystem functioning, making accurate monitoring of tree health essential. In this work, we address tree defoliation estimation as an ordinal classification problem using ground-level imagery. We propose a novel multi-view ensemble framework that aggregates predictions from Convolutional Neural Networks (CNNs) trained on different perspectives of individual trees (north, south, and crown). This approach leverages complementary visual information while preserving modelling consistency through a homogeneous ensemble design. A comprehensive evaluation is conducted by comparing multiple ordinal classification methods and analysing the contribution of each view and their combinations. Results show that modelling the ordinal structure of defoliation levels improves performance over nominal approaches, while the proposed multi-view ensemble consistently outperforms single-view and pairwise configurations. In particular, the three-view ensemble achieves the most robust and accurate predictions across all evaluation metrics. These findings highlight the potential of combining Deep Learning (DL), Ordinal Classification (OC), and multi-view aggregation for scalable, consistent, and objective forest health assessment in complex ecosystems such as Mediterranean dehesas.
Read the full abstract ▾
All authors Francisco Bérchez-Moreno, Ricardo Enrique Hernández-Lambraño, David Guijo-Rubio, Víctor Manuel Vargas, Francisco José Ruiz-Gómez, Juan Carlos Fernández, Pablo González-Moreno
Computer Vision
No Safe Dose: How Training Data Drives Unsafe Image Generation
Felix Friedrich, Lukas Helff, Niharika Hegde et al. (+2 more)
Text-to-image models trained on large-scale data often inevitably ingest unsafe content. While some people observe input-output amplifications, it remains unclear whether and how training data composition directly drives model output safety or by other factors. We shed light on this question by isolating this variable: we train the same text-to-image model on
datasets that differ only in their fraction of unsafe images (0% to 9.6%), across several
dataset scales (100K to 8M). Then we generate images with the resulting models, and evaluate them with four independent safety classifiers. Output unsafety rises monotonically from 16.6% at 0% contamination to 25.5% at 5%. A factorial design reveals that the proportion, not the absolute count, of unsafe training images is the operative variable. The 16.6% irreducible baseline at zero contamination implicates the other components, e.g. frozen text encoder, as a residual safety risk – confirmed by a text encoder ablation showing that SafeCLIP reduces this floor to 9.6%, while the dose-response effect persists across all three encoders tested. Critically, no quality degradation in terms of FID, CLIPscore and ImageReward accompanies safety filtering. These results establish that data curation and text encoder safety are complementary and independently effective interventions. At the same time, the remaining level of unsafety poses questions for future research about emerging capabilities and compositionality.
Read the full abstract ▾
All authors Felix Friedrich, Lukas Helff, Niharika Hegde, Patrick Schramowski, Kristian Kersting
Computer Vision
SAM-Enhanced Segmentation on Road Datasets: Balancing Critical Classes in Autonomous Driving
Toomas Tahves, Mauro Bellone, Junyi Gu et al. (+1 more)
Dense semantic
segmentation is essential for autonomous driving, yet many multi-modal
datasets lack pixel-level annotations. The Zenseact Open
Dataset (ZOD) provides rich multi-sensor data but only bounding-box labels, limiting its use for
segmentation research. Our primary contribution is a Segment Anything Model (SAM)-based annotation pipeline that produces dense, pixel-level annotations for ZOD by converting bounding boxes into semantic masks. In this pilot study, we process over 100,000 frames and manually curate a 2,300-frame subset (36% acceptance rate) to establish a reliable baseline. Using these annotations, we evaluate
transformer-based CLFT and CNN-based DeepLabV3+ architectures across diverse weather conditions, achieving up to 48.1% mIoU with CLFT-Hybrid. To address extreme class imbalance, where pedestrians, cyclists, and signs constitute less than 1% of pixels, we explore specialized models targeting rare classes. We further validate the pipeline on the Iseauto autonomous-vehicle platform, achieving 77.5% mIoU, and show that SAM-derived representations transfer effectively across sensor configurations via bidirectional transfer learning. All code and annotations are released to support reproducible research.
Read the full abstract ▾
All authors Toomas Tahves, Mauro Bellone, Junyi Gu, Raivo Sell
Computer Vision
Which Pretraining Paradigm Better Serves Spatial Intelligence? An Empirical Comparison of Vision-Language and Video Generation Models
Haozhan Shen, Tiancheng Zhao, Kangjia Zhao et al. (+1 more)
Spatial intelligence requires visual representations that capture both semantic objects and
geometric structure in the physical world. To support this, two major pre-training schemes are now widely used as foundation backbones:
Vision-
Language Models (VLMs), which use
language supervision to align visual observations with semantic concepts, and
Video Generation Models (VGMs), which learn from temporally evolving visual worlds. However, it still remains unclear which pre-training scheme provides a better representation substrate for spatial intelligence. In this paper, we present the first systematic frozen-feature probing study of VLMs and VGMs across three representative axes of spatial intelligence: semantic tagging, instance grouping, and 3D geometry prediction. Using the lightweight probe, our framework enables a controlled comparison of what information is already encoded in frozen representations from two model families. Experimental results reveal a clear complementarity: VLMs are stronger at semantic tagging and instance grouping, while VGMs provide more accessible signals for dense geometry and camera motion. Moreover, a naive fusion of the two already yields a representation that excels at both geometry and semantics, suggesting a promising direction for building stronger spatial-intelligence backbones by effectively integrating features from both model families. Our code is available at \href{https://github.com/om-ai-lab/Probing-VLM-VGM}{https://github.com/om-ai-lab/Probing-VLM-VGM}.
Read the full abstract ▾
All authors Haozhan Shen, Tiancheng Zhao, Kangjia Zhao, Jianwei Yin
Computer Vision
CLEAR-NeRF: Collinearity and Local-region Enhanced Accurate 3D Reconstruction in Unbounded Scenes
Vladislav Polianskii, Elijs Dima, Isabel Salmerón Marazuela et al. (+3 more)
Many real-world 3D reconstruction applications demand photorealism and metric accuracy across unbounded, complex scenes with challenging lighting and imperfect captures that current Neural Radiance Field (NeRF) pipelines only partly satisfy. This study adapts NeRF-based 3D reconstruction to multi-region of interest unbounded scenes to improve robustness to lighting and pose variation while enforcing metric accuracy suitable for digital-twin applications. Our approach introduces (i) automated local region localization/detection and reconstruction to seamlessly prioritize areas of interest without proliferating submodules, (ii) collinearity-enforcing ray sampling to learn smooth planar and curved surfaces, (iii) depth-localized neighborhood point extraction to suppress surface artifacts, and (iv) geometry-relevant color aggregation to mitigate lighting- and pose-caused variations. Results indicate superior performance of the proposed pipeline over the baseline NeRF models and established Structure from Motion (SfM) - Multi-View Stereo (MVS) solutions.
Read the full abstract ▾
All authors Vladislav Polianskii, Elijs Dima, Isabel Salmerón Marazuela, Gergő László Nagy, Sigurdur Sverrisson, Volodya Grancharov
Computer Vision
ST-ColoNet: Spatio-Temporal Colon Segment Recognition via Hybrid Attention and Edge-Guided Feature Learning
Crystal Cai, Ziyi Wang, Zhengjie Zhang et al. (+3 more)
Colo-segment recognition in colonoscopy
videos is a key requirement for many downstream tasks, but existing automatic recognition methods only use colonoscopy images without fully exploiting the use of temporal information, leading to poor performance. Additionally, relevant public
video-based
datasets are in scarcity. To tackle this problem, we curate and release a labeled
dataset specifically for the task of colo-segment recognition. In addition, we propose a two-stage deep learning-based framework, Colo-Segment Recognition via SpatioTemporal Network (ST-ColoNet), for the task of colo-segment recognition from colonoscopy
videos which includes the Colorlaus module that uses metric learning to optimize edge-mediated spatial feature extraction, as well as the Full-Temp module which combines three self-
attention patterns to better approximate full self-
attention on long colonoscopy sequences and optimize temporal feature aggregation. Through extensive ablation experiments, we show that our framework is capable of achieving state-of-the-art performance on the task of colo-segment recognition, achieving an accuracy of 81.0% and F1-score of 70.7%, which is a tremendous improvement over state-of-the-art methods.
Read the full abstract ▾
All authors Crystal Cai, Ziyi Wang, Zhengjie Zhang, Jingsheng Gao, Dahong Qian, Suncheng Xiang
Computer Vision
Revisiting Change Detection Methods for their Application to Serac Fall Time-Lapse Monitoring
Arthur Dérédel, Carlos Crispim-Junior, Pierre Lemaire et al. (+2 more)
In an era where
climate change aggravates environmental uncertainties, the identification and detection of event precursors are becoming crucial to mitigate the impacts of disastrous natural hazards. While classical sensors such as interferometric
lasers or seismometers are reliable, their widespread deployment is often hindered by logistical and economic barriers, leaving numerous blind spots. Time-lapse cameras, which already provide cost-effective, high-resolution visual context to such sensors, present a promising alternative. However, processing their output automatically faces significant challenges, notably linked to extreme shape and lighting variations. Overcoming those issues is essential to deploy them at large-scale as a monitoring tool. This paper introduces a novel sub-task of change detection, namely volumetric change detection, applied to time-lapse cameras and slope instabilities. We conduct a comprehensive review of state-of-the-art change detection methods and related tasks, analyze their core components and assess their applicability to this context. To that end, we introduce the new
dataset SeracFallDet, which contains serac fall annotations and has been thoroughly annotated to meet the latter demand. Through generalization experiments, we demonstrate that dense and semi-dense feature matching, although not trained specifically for this task, exhibit robust performance. Alternatively, supervised approaches struggle with data scarcity and annotation imbalance. This suggests that hybrid methods may offer a path forward by leveraging the strengths of both tasks. These findings highlight the potential of feature matching techniques and the need for further innovation to overcome the challenges of real-world deployment in environmental monitoring.
Read the full abstract ▾
All authors Arthur Dérédel, Carlos Crispim-Junior, Pierre Lemaire, Johan Berthet, Laure Tougne Rodet