Computer Vision
Match4Annotate: Cross-Video Annotation Transfer in Ultrasound via Implicit Feature Flow-Guided Matching
Zhuorui Zhang, Roger Pallarès-López, Praneeth Namburi et al. (+1 more)
Acquiring per-frame annotations for ultrasound
videos is costly and requires clinical expertise, limiting learning-based analysis. We study cross-
video annotation transfer: propagating user-specified annotations from a labeled ultrasound
video to an independently acquired target
video with no target-side labels or manual initialization.
Video trackers and
segmentation propagators rely on temporal continuity and require a
prompt in every new sequence, whereas cross-image feature matching and one-shot
segmentation estimate correspondences independently, without enforcing coherent deformations or supporting both point and mask annotations. We present Match4Annotate, a test-time framework with three stages. A spatiotemporal implicit feature representation lifts frozen
vision foundation-model features into a continuous field over space and time, enabling queries beyond the backbone resolution. A continuous implicit feature flow then aligns the source and target fields under a smooth-deformation prior, estimating correspondence in feature space rather than relying on intensity consistency, which is often violated in ultrasound by speckle and acquisition-dependent appearance. Finally, flow-guided annotation transfer uses the estimated flow as a spatial prior over feature similarity. This formulation unifies sparse point and dense mask transfer and includes unconstrained feature matching and direct flow warping as limiting cases. On four clinical ultrasound
datasets spanning echocardiography and musculoskeletal imaging, Match4Annotate achieves state-of-the-art annotation transfer, outperforming dense feature-matching baselines across PCK thresholds and one-shot
segmentation methods in Dice score. It also demonstrates bidirectional transfer of left-ventricular annotations across
datasets. It requires no task-specific training and adapts to each
video in minutes on a single consumer GPU.
Read the full abstract ▾
All authors Zhuorui Zhang, Roger Pallarès-López, Praneeth Namburi, Brian W. Anthony
Computer Vision
GreenRFM: Learning a resource-efficient radiology vision-language foundation model via supervision-centric pre-training
Yingtai Li, Shuai Ming, Qiuli Wang et al. (+13 more)
Radiology foundation models (RFMs) have largely inherited the scale-first recipe of natural-image
vision–
language pre-training. This recipe is difficult to deploy in 3D radiology, where training corpora are smaller, reports vary across institutions, and receiving hospitals often need local adaptation under
privacy and compute constraints. We ask whether routine radiology reports can instead be converted into auditable diagnostic supervision that shapes the image encoder, text encoder, aligned space, and local-adaptation procedure. We develop GreenRFM, a supervision-centric pre-training framework organized around four empirical principles: More distilled, Ubiquitous, Semantic-enforcing, and Task-aligning (MUST) supervision. These principles convert noisy reports into structured diagnostic signals and use them to learn discriminative unimodal encoders plus an aligned image–text space for diagnosis-centered
multimodal use. GreenRFM requires 24 GPU-hours on a single 24GB GPU (lightweight variant: 6GB VRAM, 4~hours) and reaches a zero-shot CT-RATE AUC of 84.8. Evaluations using more than 200,000 volumes from six institutions and two modalities show transfer to private clinical cohorts and to musculoskeletal MRI. On a local institutional cohort, computationally feasible retraining raises macro-AUC from 70.5 to 82.1. The aligned space also improves hepatocellular-carcinoma microvascular-invasion prediction and trans-arterial chemoembolization response analysis over established clinical scores. These results support supervision-centric pre-training as a practical route to resource-efficient, locally adaptable, diagnosis-centered radiology
vision–
language representations.
Read the full abstract ▾
All authors Yingtai Li, Shuai Ming, Qiuli Wang, Mingyue Zhao, Hongchun Zhang, Yuhe Tian, Haoran Lai, Rongsheng Wang, Rui Zhou, Rundong Wang, Yujia Li, Zhiyang He, Xiaodong Tao, Wei Chen, Wei Wei, Shaohua Kevin Zhou
Computer Vision
Do Foundation Models Know Geometry? Probing Frozen Features for Continuous Physical Measurement
Yakov Pyotr Shkolnikov
Vision-
language models encode continuous geometry that their text pathway fails to express: a 6,000-parameter linear probe extracts hand joint angles at 6.1 degrees MAE from frozen features, while the best text output achieves only 20.0 degrees – a 3.3x bottleneck. LoRA fine-tuning (r=16, 2,000 images) narrows this gap to 6.5 degrees, providing evidence for a pathway-training deficit rather than a representational one. Training objective determines accuracy more than architecture: five encoders spanning self-supervised, contrastive, and hybrid paradigms converge to statistically equivalent accuracy (R^2 approximately 0.55, TOST-equivalent at delta=0.03) despite sharing as little as CKA=0.41 representational similarity – functional convergence without representational convergence. Autoregressive generation damages
geometric fidelity, but the damage originates in the generation process, not in
language alignment: Qwen2.5-VL's LLM layers actually improve probe accuracy over its raw
vision encoder. Layer-wise analysis reveals a universal mid-network accuracy peak across all architectures, with
attention heads in layers 18-22 carrying disproportionate
geometric signal. These findings enable a single frozen backbone to function as a multi-task
geometric sensor through lightweight probes, without fine-tuning or text generation.
Read the full abstract ▾
Computer Vision
Training Flow Matching: The Role of Weighting and Parameterization
Anne Gagneux, Ségolène Martin, Rémi Gribonval et al. (+1 more)
We study the training objectives of denoising-based generative models, with a particular focus on loss weighting and output parameterization, including noise-, clean image-, and velocity-based formulations. Through a systematic numerical study, we analyze how these training choices interact with the intrinsic dimensionality of the data
manifold, model architecture, and
dataset size. Our experiments span synthetic
datasets with controlled geometry as well as image data, and compare training objectives using quantitative metrics for denoising accuracy (PSNR across noise levels) and generative quality (FID). Rather than proposing a new method, our goal is to disentangle the various factors that matter when training a flow matching model, in order to provide practical insights on design choices.
Read the full abstract ▾
All authors Anne Gagneux, Ségolène Martin, Rémi Gribonval, Mathurin Massias
Computer Vision
Pinterest Canvas: Large-Scale Image Generation at Pinterest
Yu Wang, Eric Tzeng, Raymond Shiau et al. (+3 more)
While recent image generation models demonstrate a remarkable ability to handle a wide variety of image generation tasks, this flexibility makes them hard to control via prompting or simple inference adaptation alone, rendering them unsuitable for use cases with strict product requirements. In this paper, we introduce Pinterest Canvas, our large-scale image generation system built to support image editing and enhancement use cases at Pinterest. Canvas is first trained on a diverse,
multimodal dataset to produce a foundational
diffusion model with broad image-editing capabilities. However, rather than relying on one generic model to handle every downstream task, we instead rapidly fine-tune variants of this base model on task-specific
datasets, producing specialized models for individual use cases. We describe key components of Canvas and summarize our best practices for
dataset curation, training, and inference. We also showcase task-specific variants through case studies on background enhancement and aspect-ratio outpainting, highlighting how we tackle their specific product requirements. Online A/B experiments demonstrate that our enhanced images receive a significant 18.0% and 12.5% engagement lift, respectively, and comparisons with human raters further validate that our models outperform third-party models on these tasks. Finally, we showcase other Canvas variants, including multi-image scene synthesis and image-to-
video generation, demonstrating that our approach can generalize to a wide variety of potential downstream tasks.
Read the full abstract ▾
All authors Yu Wang, Eric Tzeng, Raymond Shiau, Jie Yang, Dmitry Kislyuk, Charles Rosenberg
Computer Vision
CaTok: Taming Mean Flows for One-Dimensional Causal Image Tokenization
Yitong Chen, Zuxuan Wu, Xipeng Qiu et al. (+1 more)
Autoregressive (AR)
language models rely on causal tokenization, but extending this paradigm to
vision remains non-trivial. Current visual tokenizers either flatten 2D patches into non-causal sequences or enforce heuristic orderings that misalign with the "next-
token prediction" pattern. Recent
diffusion autoencoders similarly fall short: conditioning the decoder on all
tokens lacks causality, while applying nested dropout mechanism introduces imbalance. To address these challenges, we present CaTok, a 1D causal image tokenizer with a MeanFlow decoder. By selecting
tokens over time intervals and binding them to the MeanFlow objective, as illustrated in Fig. 1, CaTok learns causal 1D representations that support both fast one-step generation and high-fidelity multi-step sampling, while naturally capturing diverse visual concepts across
token intervals. To further stabilize and accelerate training, we propose a straightforward regularization REPA-A, which aligns encoder features with
Vision Foundation Models (VFMs). Experiments demonstrate that CaTok achieves state-of-the-art results on ImageNet reconstruction, reaching 0.75 FID, 22.53 PSNR and 0.674 SSIM with fewer training epochs, and the AR model attains performance comparable to leading approaches.
Read the full abstract ▾
All authors Yitong Chen, Zuxuan Wu, Xipeng Qiu, Yu-Gang Jiang
Computer Vision
What if? Emulative Simulation with World Models for Situated Reasoning
Ruiping Liu, Yufan Chen, Yuheng Zhang et al. (+8 more)
Situated
reasoning often relies on active exploration, yet in many real-world scenarios such exploration is infeasible due to physical constraints of
robots or safety concerns of visually impaired users. Given only a limited observation, can an
agent mentally simulate a future trajectory toward a target situation and answer spatial what-if questions? We introduce WanderDream, the first large-scale
dataset designed for the emulative simulation of mental exploration, enabling models to reason without active exploration. WanderDream-Gen comprises 15.8K panoramic
videos across 1,088 real scenes from HM3D, ScanNet++, and real-world captures, depicting imagined trajectories from current viewpoints to target situations. WanderDream-QA contains 158K question-answer pairs, covering starting states, paths, and end states along each trajectory to comprehensively evaluate exploration-based
reasoning. Extensive experiments with world models and MLLMs demonstrate (1) that mental exploration is essential for situated
reasoning, (2) that world models achieve compelling performance on WanderDream-Gen, (3) that imagination substantially facilitates
reasoning on WanderDream-QA, and (4) that WanderDream data exhibit remarkable transferability to real-world scenarios.
Read the full abstract ▾
All authors Ruiping Liu, Yufan Chen, Yuheng Zhang, Junwei Zheng, Kunyu Peng, Chengzhi Wu, Chenguang Huang, Di Wen, Jiaming Zhang, Kailun Yang, Rainer Stiefelhagen
Computer Vision
CLoPA: Continual Low Parameter Adaptation of Interactive Segmentation for Medical Image Annotation
Parhom Esmaeili, Chayanin Tangwiriyasakul, Eli Gibson et al. (+2 more)
Interactive
segmentation enables clinicians to guide annotation, but existing zero-shot models like nnInteractive fail to consistently reach expert-level performance across diverse medical imaging tasks. Because annotation campaigns produce a growing stream of task-specific labelled data, online adaptation of the
segmentation model is a natural complement to zero-shot inference. We propose CLoPA, a continual adaptation strategy that tunes a small fraction of nnInteractive's parameters on the annotation cache, triggered by lightweight episode scheduling. CLoPA requires no new parameters or changes to the inference pipeline, and operates entirely within the existing annotation workflow. Across eight Medical
Segmentation Decathlon tasks spanning diverse anatomical targets and imaging characteristics, CLoPA rapidly elevates performance to expert-level, even for tasks where nnInteractive previously failed, with the majority of gains realised after a single training episode. We show that the benefits of tuning different parameter groups depends on task characteristics and data regimes. Also, that for targets with complex geometries (e.g., hepatic vessels), instance normalisation and low-level feature tuning saturates, suggesting a need for deeper feature-representation
alignment in the most challenging scenarios.
Read the full abstract ▾
All authors Parhom Esmaeili, Chayanin Tangwiriyasakul, Eli Gibson, Sebastien Ourselin, M. Jorge Cardoso
Computer Vision
Non-invasive Growth Monitoring of Small Freshwater Fish in Home Aquariums via Stereo Vision
Clemens Seibold, Anna Hilsmann and Peter Eisert
Monitoring fish growth behavior provides relevant information about fish health in aquaculture and home aquariums. Yet, monitoring fish sizes poses different challenges, as fish are small and subject to strong refractive distortions in aquarium environments. Image-based measurement offers a practical, non-invasive alternative that allows frequent monitoring without disturbing the fish. In this paper, we propose a non-invasive refraction-aware stereo
vision method to estimate fish length in aquariums. Our approach uses a YOLOv11-Pose network to detect fish and predict anatomical keypoints on the fish in each stereo image. A refraction-aware epipolar constraint accounting for the air-glass-water interfaces enables robust matching, and unreliable detections are removed using a learned quality score. A subsequent refraction-aware 3D triangulation recovers 3D keypoints, from which fish length is measured. We validate our approach on a new stereo
dataset of endangered Sulawesi ricefish captured under aquarium-like conditions and demonstrate that filtering low-quality detections is essential for accurate length estimation. The proposed system offers a simple and practical solution for non-invasive growth monitoring and can be easily applied in home aquariums.
Read the full abstract ▾
Computer Vision
Physical Simulator In-the-Loop Video Generation
Lin Geng Foo, Mark He Huang, Alexandros Lattas et al. (+3 more)
Recent advances in
diffusion-based
video generation have achieved remarkable visual realism but still struggle to obey basic physical laws such as
gravity, inertia, and collision. Generated objects often move inconsistently across frames, exhibit implausible dynamics, or violate physical constraints, limiting the realism and reliability of AI-generated
videos. We address this gap by introducing Physical Simulator In-the-loop
Video Generation (PSIVG), a novel framework that integrates a physical simulator into the
video diffusion process. Starting from a template
video generated by a pre-trained
diffusion model, PSIVG reconstructs the 4D scene and foreground object meshes, initializes them within a physical simulator, and generates physically consistent trajectories. These simulated trajectories are then used to guide the
video generator toward spatio-temporally physically coherent motion. To further improve texture consistency during object movement, we propose a Test-Time Texture Consistency Optimization (TTCO) technique that adapts text and feature
embeddings based on pixel correspondences from the simulator. Comprehensive experiments demonstrate that PSIVG produces
videos that better adhere to real-world physics while preserving visual quality and diversity. Project Page: https://vcai.mpi-inf.mpg.de/projects/PSIVG/
Read the full abstract ▾
All authors Lin Geng Foo, Mark He Huang, Alexandros Lattas, Stylianos Moschoglou, Thabo Beeler, Christian Theobalt
Computer Vision
Locating and Editing Figure-Ground Organization in Vision Transformers
Stefan Arnold and René Gröbner
Vision Transformers must resolve figure-ground organization by choosing between completions driven by local
geometric evidence and those favored by global organizational priors, giving rise to a characteristic perceptual ambiguity. We aim to locate where the canonical Gestalt prior convexity is realized within the internal components of BEiT. Using a controlled perceptual conflict based on synthetic shapes of darts, we systematically mask regions that equally admit either a concave completion or a convex completion. We show that BEiT reliably favors convex completion under this competition. Projecting internal activations into the model's discrete visual codebook space via logit attribution reveals that this preference is governed by identifiable functional units within
transformer substructures. Specifically, we find that figure-ground organization is ambiguous through early and intermediate layers and resolves abruptly in later layers. By decomposing the direct effect of
attention heads, we identify head L0H9 acting as an early seed, introducing a weak bias toward convexity. Downscaling this single
attention head shifts the distributional mass of the perceptual conflict across a continuous decision boundary, allowing concave evidence to guide completion.
Read the full abstract ▾
Computer Vision
DiffInf: Influence-Guided Diffusion for Supervision Alignment in Facial Attribute Learning
Basudha Pal, Zhaoyang Wang and Rama Chellappa
Facial attribute classification relies on large-scale annotated
datasets in which many traits, such as age and expression, are inherently ambiguous and continuous but are discretized into categorical labels. Annotation inconsistencies arise from subjectivity and visual confounders such as pose, illumination, expression, and demographic variation, creating mismatch between images and assigned labels. These inconsistencies introduce supervision errors that impair representation learning and degrade downstream prediction. We introduce DiffInf, a self-influence–guided
diffusion framework for mitigating annotation inconsistencies in facial attribute learning. We first train a baseline classifier and compute sample-wise self-influence scores using a practical first-order approximation to identify training instances that disproportionately destabilize optimization. Instead of discarding these influential samples, we apply targeted generative correction via a latent
diffusion autoencoder to better align visual content with assigned labels while preserving identity and realism. To enable differentiable guidance during correction, we train a lightweight predictor of high-influence membership and use it as a surrogate influence regularizer. The edited samples replace the originals, yielding an influence-refined
dataset of unchanged size. Across multi-class facial attribute classification, DiffInf consistently improves generalization compared with standard noisy-label training, robust optimization baselines, and influence-based filtering. Our results demonstrate that repairing influential annotation inconsistencies at the image level enhances downstream facial attribute classification without sacrificing distributional coverage.
Read the full abstract ▾
Computer Vision
Solving Jigsaw Puzzles in the Wild: Human-Guided Reconstruction of Cultural Heritage Fragments
Omidreza Safaei, Sinem Aslan, Sebastiano Vascon et al. (+3 more)
Reassembling real-world archaeological artifacts from fragmented pieces poses significant challenges due to erosion, missing regions, irregular shapes, and large-scale ambiguity. Traditional jigsaw puzzle solvers, often designed for clean synthetic scenarios, struggle under these conditions, especially when the number of fragments grows into the thousands, as in the RePAIR
benchmark. In this paper, we propose a human-in-the-loop (HIL) puzzle solving framework designed to address the complexity and scale of real-world cultural heritage reconstruction. Our approach integrates an automatic relaxation-labeling solver with interactive human guidance, allowing users to iteratively lock verified placements, correct errors, and guide the system toward semantically and geometrically coherent assemblies. We introduce two complementary interaction strategies, Iterative Anchoring and Continuous Interactive Refinement, which support scalable reconstruction across varying levels of ambiguity and puzzle size. Experiments on several RePAIR groups demonstrate that our hybrid approach substantially outperforms both fully automatic and manual baselines in accuracy and efficiency, offering a practical solution for large-scale expert-in-the-loop artifact reassembly.
Read the full abstract ▾
All authors Omidreza Safaei, Sinem Aslan, Sebastiano Vascon, Luca Palmieri, Marina Khoroshiltseva, Marcello Pelillo
Computer Vision
REACT++: Efficient Cross-Attention for Real-Time Scene Graph Generation
Maëlic Neau and Zoe Falomir
Scene Graph Generation (SGG) is a task that encodes visual relationships between objects in images as graph structures. SGG shows significant promise as a foundational component for downstream tasks, such as
reasoning for
embodied agents. To enable real-time applications, SGG must address the trade-off between performance and inference speed. However, current methods tend to focus on one of the following: (1) improving relation prediction accuracy, (2) enhancing object detection accuracy, or (3) reducing latency, without aiming to balance all three objectives simultaneously. To address this limitation, we build on the powerful Real-time Efficiency and Accuracy Compromise for Tradeoffs in Scene Graph Generation (REACT) architecture and propose REACT++, a new state-of-the-art model for real-time SGG. By leveraging efficient feature extraction and subject-to-object cross-
attention within the prototype space, REACT++ balances latency and representational power. REACT++ achieves the highest inference speed among existing SGG models, improving relation prediction accuracy without sacrificing object detection performance. Compared to the previous REACT version, REACT++ is 20% faster with a gain of 10% in relation prediction accuracy on average. The code is available at https://github.com/Maelic/SGG-
Benchmark.
Read the full abstract ▾
Computer Vision
Yonghuang Wu, Zhenyang Liang, Wenwen Zeng et al. (+2 more)
Foundation models such as Segment Anything Model 3 (SAM3) enable flexible text-guided medical image
segmentation, yet their predictions remain highly sensitive to
prompt formulation. Even semantically equivalent descriptions can yield inconsistent masks, limiting reliability in clinical and pathology workflows. We reformulate
prompt sensitivity as a group-wise consistency problem. Semantically related
prompts are organized into
prompt groups sharing the same ground-truth mask, and a
prompt group-aware training framework is introduced for robust text-guided
nuclei segmentation. The approach combines (i) a quality-guided group regularization that leverages
segmentation loss as an implicit ranking signal, and (ii) a logit-level consistency constraint with a stop-gradient strategy to align predictions within each group. The method requires no architectural modification and leaves inference unchanged. Extensive experiments on multi-
dataset nuclei benchmarks show consistent gains under textual prompting and markedly reduced performance variance across
prompt quality levels. On six zero-shot cross-
dataset tasks, our method improves Dice by an average of 2.16 points. These results demonstrate improved robustness and generalization for
vision-
language segmentation in computational pathology.
Read the full abstract ▾
All authors Yonghuang Wu, Zhenyang Liang, Wenwen Zeng, Xuan Xie, Jinhua Yu
Computer Vision
CHMv2: Improvements in Global Canopy Height Mapping using DINOv3
John Brandt, Seungeun Yi, Jamie Tolan et al. (+9 more)
Accurate canopy height information is essential for quantifying forest carbon, monitoring restoration and degradation, and assessing habitat structure, yet high-fidelity measurements from airborne
laser scanning (ALS) remain unevenly available globally. Here we present CHMv2, a global, meter-resolution canopy height map derived from high-resolution optical satellite imagery using a depth-estimation model built on DINOv3 and trained against ALS canopy height models. Compared to existing products, CHMv2 substantially improves accuracy, reduces bias in tall forests, and better preserves fine-scale structure such as canopy edges and gaps. These gains are enabled by a large expansion of geographically diverse training data, automated data curation and registration, and a loss formulation and data sampling strategy tailored to canopy height distributions. We validate CHMv2 against independent ALS test sets and against tens of millions of GEDI and ICESat-2 observations, demonstrating consistent performance across major forest biomes.
Read the full abstract ▾
All authors John Brandt, Seungeun Yi, Jamie Tolan, Xinyuan Li, Peter Potapov, Jessica Ertel, Justine Spore, Huy V. Vo, Michaël Ramamonjisoa, Patrick Labatut, Piotr Bojanowski, Camille Couprie
Computer Vision
MoEMambaMIL: Structure-Aware Selective State Space Modeling for Whole-Slide Image Analysis
Dongqing Xie and Yonghuang Wu
Whole-slide image (WSI) analysis is challenging due to the gigapixel scale of slides and their inherent hierarchical multi-resolution structure. Existing multiple instance learning (MIL) approaches often model WSIs as unordered collections of patches, which limits their ability to capture structured dependencies between global tissue organization and local cellular patterns. Although recent State Space Models (SSMs) enable efficient modeling of long sequences, how to structure WSI
tokens to fully exploit their spatial hierarchy remains an open problem.We propose MoEMambaMIL, a structure-aware SSM framework for WSI analysis that integrates region-nested selective scanning with mixture-of-experts (MoE) modeling. Leveraging multi-resolution preprocessing, MoEMambaMIL organizes patch
tokens into region-aware sequences that preserve spatial containment across resolutions. On top of this structured sequence, we decouple resolution-aware encoding and region-adaptive contextual modeling via a combination of static, resolution-specific experts and dynamic sparse experts with learned routing. This design enables efficient long-sequence modeling while promoting expert specialization across heterogeneous diagnostic patterns. Experiments demonstrate that MoEMambaMIL achieves the best performance across 9 downstream tasks.
Read the full abstract ▾
Computer Vision
Rewis3d: Reconstruction Improves Weakly-Supervised Semantic Segmentation
Jonas Ernst, Wolfgang Boettcher, Lukas Hoyer et al. (+2 more)
We present Rewis3d, a framework that leverages recent advances in feed-forward 3D reconstruction to significantly improve weakly supervised semantic
segmentation on 2D images. Obtaining dense, pixel-level annotations remains a costly bottleneck for training
segmentation models. Alleviating this issue, sparse annotations offer an efficient weakly-supervised alternative. However, they still incur a performance gap. To address this, we introduce a novel approach that leverages 3D scene reconstruction as an auxiliary supervisory signal. Our key insight is that 3D
geometric structure recovered from 2D
videos provides strong cues that can propagate sparse annotations across entire scenes. Specifically, a dual student-teacher architecture enforces semantic consistency between 2D images and reconstructed 3D point clouds, using state-of-the-art feed-forward reconstruction to generate reliable
geometric supervision. Extensive experiments demonstrate that Rewis3d achieves state-of-the-art performance in sparse supervision, outperforming existing approaches by 2-7% without requiring additional labels or inference overhead.
Read the full abstract ▾
All authors Jonas Ernst, Wolfgang Boettcher, Lukas Hoyer, Jan Eric Lenssen, Bernt Schiele
Computer Vision
OralGPT-Plus: Learning to Use Visual Tools via Reinforcement Learning for Panoramic X-ray Analysis
Yuxuan Fan, Jing Hao, Hong Chen et al. (+5 more)
Panoramic dental radiographs require fine-grained spatial
reasoning, bilateral
symmetry understanding, and multi-step diagnostic verification, yet existing
vision-
language models operate under a static single-pass paradigm that limits their clinical reliability. In this paper, we introduce OralGPT-Plus, an agentic
vision-
language model designed to perform iterative and
symmetry-aware diagnostic
reasoning for panoramic dental radiograph analysis. To support this paradigm, we construct DentalProbe, a five-thousand-image
dataset with expert-curated diagnostic trajectories that provide structured supervision for localized inspection and contralateral comparison. We further develop a Reinspection-driven reinforcement learning framework that encourages clinically meaningful re-examination and stabilizes long-horizon
reasoning with rubric-based reward and conditioned diagnostic-driven reward. In parallel, we present MMOral-X, the first
benchmark for holistic panoramic diagnosis, containing 300 open-ended questions and region-level annotations across multiple difficulty levels. OralGPT-Plus demonstrates consistent and reliable improvements over strong baselines on MMOral-X and established panoramic
benchmarks, indicating the effectiveness of interactive and
symmetry-informed
reasoning. Our work highlights the value of agentic modeling for dental imaging and provides a foundation for future research in clinically aligned panoramic radiograph analysis.
Read the full abstract ▾
All authors Yuxuan Fan, Jing Hao, Hong Chen, Jiahao Bao, Yihua Shao, Yuci Liang, Kuo Feng Hung, Hao Tang
Computer Vision
Computer vision-based estimation of small arthropod biomass
Mikko Impiö, Philipp M. Rehsen, Jarrett Blair et al. (+5 more)
The ability to estimate invertebrate biomass using only images could help scaling up quantitative biodiversity monitoring efforts. Computer
vision-based methods have the potential to omit the manual, time-consuming, and destructive process of dry weighing specimens. We present two approaches for dry mass estimation that do not require additional manual effort apart from imaging the specimens: fitting a linear model with novel predictors, automatically calculated by an imaging device, and training a family of end-to-end deep neural networks for the task, using single-view, multi-view, and metadata-aware architectures. We propose using area and sinking speed as predictors. These can be calculated with BIODISCOVER, which is a dual-camera system that captures image sequences of specimens sinking in an ethanol column. For this study, we collected a large
dataset of dry mass measurement and image sequence pairs to train and evaluate models. We show that our methods can estimate specimen dry mass even with complex and visually diverse specimen morphologies. Combined with automatic taxonomic classification, our approach achieves a median individual error of 10%-20% and produces group-level dry mass distributions that do not differ significantly from observed distributions. We highlight the importance of choosing appropriate evaluation metrics, and encourage using both percentage errors and absolute errors as metrics, because they measure different properties. We also explore different optimization losses, data augmentation methods, and model architectures for training deep-learning models.
Read the full abstract ▾
All authors Mikko Impiö, Philipp M. Rehsen, Jarrett Blair, Cecilie Mielec, Arne J. Beermann, Florian Leese, Toke T. Høye, Jenni Raitoharju
Computer Vision
LATO: 3D Mesh Flow Matching with Structured TOpology Preserving LAtents
Tianhao Zhao, Youjia Zhang, Hang Long et al. (+7 more)
In this paper, we introduce LATO, a novel
topology-preserving latent representation that enables scalable, flow matching-based synthesis of explicit 3D meshes. LATO represents a mesh as a Vertex Displacement Field (VDF) anchored on surface, incorporating a sparse voxel Variational Autoencoder (VAE) to compress this explicit signal into a structured,
topology-aware voxel latent. To decapsulate the mesh, the VAE decoder progressively subdivides and prunes latent voxels to instantiate precise vertex locations. In the end, a dedicated connection head queries the voxel latent to predict edge connectivity between vertex pairs directly, allowing mesh
topology to be recovered without isosurface extraction or heuristic meshing. For generative modeling, LATO adopts a two-stage flow matching process, first synthesizing the structure voxels and subsequently refining the voxel-wise
topology features. Compared to prior isosurface/triangle-based
diffusion models and autoregressive generation approaches, LATO generates meshes with complex geometry, well-formed
topology while being highly efficient in inference.
Read the full abstract ▾
All authors Tianhao Zhao, Youjia Zhang, Hang Long, Jinshen Zhang, Wenbing Li, Yang Yang, Gongbo Zhang, Jozef Hladký, Matthias Nießner, Wei Yang
Computer Vision
DC-DiT: Adaptive Compute and Elastic Inference for Visual Generation via Dynamic Chunking
Akash Haridas, Utkarsh Saxena, Parsa Ashrafi Fashi et al. (+3 more)
Diffusion Transformers rely on static patchify tokenization, assigning the same
token budget to smooth backgrounds, detailed object regions, noisy early timesteps, and late-stage refinements. We introduce the Dynamic Chunking
Diffusion Transformer (DC-DiT), which replaces fixed patchification with a learned encoder-router-decoder scaffold that adaptively compresses the 2D input into a shorter
token sequence through a chunking mechanism learned end-to-end with
diffusion training. DC-DiT allocates fewer
tokens to predictable regions and noisy timesteps, and more
tokens to detailed regions and later refinement stages, yielding meaningful spatial
segmentations and timestep-adaptive compression schedules without supervision. Furthermore, the router provides an importance ordering over retained
tokens, enabling elastic inference: a single checkpoint can be evaluated at flexible compute budgets with a smooth quality-compute tradeoff. Additionally, DC-DiT can be upcycled from pretrained DiT checkpoints and is also compatible with orthogonal dynamic computation approaches. On class-conditional ImageNet generation, DC-DiT reduces inference FLOPs by up to 36.8% and improves FID by up to 37.8% over DiT baselines, yielding a stronger quality–compute Pareto frontier across model scales, resolutions, and guidance settings. More broadly, these results suggest that adaptive tokenization is a general mechanism for making visual generation both more efficient and more flexible at inference time.
Read the full abstract ▾
All authors Akash Haridas, Utkarsh Saxena, Parsa Ashrafi Fashi, Mehdi Rezagholizadeh, Vikram Appia, Emad Barsoum
Computer Vision
K-MaT: Knowledge-Anchored Manifold Transport for Cross-Modal Prompt Learning in Medical Imaging
Jiajun Zeng and Shadi Albarqouni
Large-scale biomedical
vision-
language models (VLMs) adapted on high-end imaging (e.g., CT) often fail to transfer to frontline low-end modalities (e.g., radiography), collapsing into modality-specific shortcuts. We propose K-MaT (Knowledge-Anchored
Manifold Transport), a
prompt-learning framework that transfers decision structures to low-end modalities without requiring low-end training images. K-MaT factorizes
prompts, anchors them to clinical text descriptions, and aligns the low-end
prompt manifold to the visually-grounded high-end space using Fused Gromov-Wasserstein optimal transport. We evaluate K-MaT on four cross-modal
benchmarks, including dermoscopy, mammography to ultrasound, and CT to chest X-ray. K-MaT achieves state-of-the-art results, improving the average harmonic mean of accuracy to 44.1% (from BiomedCoOp's 42.0%) and macro-F1 to 36.2%. Notably, on the challenging breast imaging task, it mitigates the catastrophic forgetting seen in standard methods like CoOp (which drops to 27.0% accuracy on the low-end), preserving robust performance across modalities. Aligning
prompt manifolds via optimal transport provides a highly effective route for the zero-shot cross-modal deployment of medical VLMs.
Read the full abstract ▾
Computer Vision
WorldCache: Accelerating World Models for Free via Heterogeneous Token Caching
Weilun Feng, Guoxin Fan, Haotong Qin et al. (+10 more)
Diffusion-based world models have shown strong potential for unified world simulation, but the iterative denoising remains too costly for interactive use and long-horizon rollouts. While feature caching can accelerate inference without training, we find that policies designed for single-modal
diffusion transfer poorly to world models due to two world-model-specific obstacles:
token heterogeneity from multi-modal coupling and spatial variation, and non-uniform temporal dynamics where a small set of hard
tokens drives error growth, making uniform skipping either unstable or overly conservative. We propose WorldCache, a caching framework tailored to
diffusion world models. We introduce
Curvature-guided Heterogeneous
Token Prediction, which uses a physics-grounded
curvature score to estimate
token predictability and applies a Hermite-guided damped predictor for chaotic
tokens with abrupt direction changes. We also design Chaotic-prioritized Adaptive Skipping, which accumulates a
curvature-normalized, dimensionless drift signal and recomputes only when bottleneck
tokens begin to drift. Experiments on
diffusion world models show that WorldCache delivers up to \textbf{3.7$\times$} end-to-end speedups while maintaining 98% rollout quality, demonstrating the vast advantages and practicality of WorldCache in resource-constrained scenarios. Our code is released in https://github.com/FofGofx/WorldCache.
Read the full abstract ▾
All authors Weilun Feng, Guoxin Fan, Haotong Qin, Mingqiang Wu, Yuqi Li, Xiangqi Li, Zhulin An, Libo Huang, Dingrui Wang, Longlong Liao, Michele Magno, Yongjun Xu, Chuanguang Yang
Computer Vision · Cross-listed from cs.CL
The Art That Poses Back: Assessing AI Pastiches after Contemporary Artworks
Anca Dinu, Andreiana Mihail, Andra-Maria Florescu et al. (+1 more)
This study explores artificial visual creativity, focusing on ChatGPT's ability to generate new images intentionally pastiching original artworks such as paintings, drawings, sculptures and installations. The process involved twelve artists from Romania, Bulgaria, France, Austria, and the United Kingdom, each invited to contribute with three of their artworks and to grade and comment on the AI-generated versions. The analysis combines human evaluation with computational methods aimed at detecting visual and stylistic similarities or divergences between the original works and their AI-produced renditions. The results point to a significant gap between color and texture-based similarity and compositional, conceptual, and perceptual one. Consequently, we advocate for the use of a "style transfer dashboard" of complementary metrics to evaluate the similarity between pastiches and originals, rather than using a single style metric. The artists' comments revealed limitations of ChatGPT's pastiches after contemporary artworks, which were perceived by the authors of the originals as lacking dimensionality, context, and intentional sense, and seeming more of a paraphrase or an approximate quotation rather than as a valuable,
emotion-evoking artwork.
Read the full abstract ▾
All authors Anca Dinu, Andreiana Mihail, Andra-Maria Florescu, Claudiu Creanga
Computer Vision
P-SLCR: Unsupervised Point Cloud Semantic Segmentation via Prototypes Structure Learning and Consistent Reasoning
Lixin Zhan, Jie Jiang, Tianjian Zhou et al. (+3 more)
Current semantic
segmentation approaches for point cloud scenes heavily rely on manual labeling, while research on unsupervised semantic
segmentation methods specifically for raw point clouds is still in its early stages. Unsupervised point cloud learning poses significant challenges due to the absence of annotation information and the lack of pre-training. The development of effective strategies is crucial in this context. In this paper, we propose a novel prototype library-driven unsupervised point cloud semantic
segmentation strategy that utilizes Structure Learning and Consistent
Reasoning (P-SLCR). First, we propose a Consistent Structure Learning to establish structural feature learning between consistent points and the library of consistent prototypes by selecting high-quality features. Second, we propose a Semantic Relation Consistent
Reasoning that constructs a prototype inter-relation
matrix between consistent and ambiguous prototype libraries separately. This process ensures the preservation of semantic consistency by imposing constraints on consistent and ambiguous prototype libraries through the prototype inter-relation
matrix. Finally, our method was extensively evaluated on the S3DIS, SemanticKITTI, and Scannet
datasets, achieving the best performance compared to unsupervised methods. Specifically, the mIoU of 47.1% is achieved for Area-5 of the S3DIS
dataset, surpassing the classical fully supervised method PointNet by 2.5%.
Read the full abstract ▾
All authors Lixin Zhan, Jie Jiang, Tianjian Zhou, Yukun Du, Yan Zheng, Xuehu Duan
Computer Vision
WMoE-CLIP: Wavelet-Enhanced Mixture-of-Experts Prompt Learning for Zero-Shot Anomaly Detection
Peng Chen and Chao Huang
Vision-
language models have recently shown strong generalization in zero-shot anomaly detection (ZSAD), enabling the detection of unseen anomalies without task-specific supervision. However, existing approaches typically rely on fixed textual
prompts, which struggle to capture complex semantics, and focus solely on spatial-domain features, limiting their ability to detect subtle anomalies. To address these challenges, we propose a wavelet-enhanced mixture-of-experts
prompt learning method for ZSAD. Specifically, a variational autoencoder is employed to model global semantic representations and integrate them into
prompts to enhance adaptability to diverse anomaly patterns. Wavelet decomposition extracts multi-frequency image features that dynamically refine textual
embeddings through cross-modal interactions. Furthermore, a semantic-aware mixture-of-experts module is introduced to aggregate contextual information. Extensive experiments on 14 industrial and medical
datasets demonstrate the effectiveness of the proposed method.
Read the full abstract ▾
Computer Vision
Latent Transfer Attack: Adversarial Examples via Generative Latent Spaces
Eitan Shaar, Ariel Shaulov, Yalcin Tur et al. (+2 more)
Adversarial attacks are a central tool for probing the robustness of modern
vision models, yet most methods optimize perturbations directly in pixel space under $\ell_\infty$ or $\ell_2$ constraints. While effective in white-box settings, pixel-space optimization often produces high-frequency, texture-like noise that is brittle to common preprocessing (e.g., resizing and cropping) and transfers poorly across architectures. We propose $\textbf{LTA}$ ($\textbf{L}$atent $\textbf{T}$ransfer $\textbf{A}$ttack), a transfer-based attack that instead optimizes perturbations in the latent space of a pretrained Stable
Diffusion VAE. Given a clean image, we encode it into a latent code and optimize the latent representation to maximize a surrogate classifier loss, while softly enforcing a pixel-space $\ell_\infty$ budget after decoding. To improve robustness to resolution mismatch and standard input pipelines, we incorporate Expectation Over Transformations (EOT) via randomized resizing, interpolation, and cropping, and apply periodic latent Gaussian smoothing to suppress emerging artifacts and stabilize optimization. Across a suite of CNN and
vision-
transformer targets, LTA achieves strong transfer attack success while producing spatially coherent, predominantly low-frequency perturbations that differ qualitatively from pixel-space baselines and occupy a distinct point in the transfer-quality trade-off. Our results highlight pretrained generative latent spaces as an effective and structured domain for
adversarial optimization, bridging robustness evaluation with modern generative priors.
Read the full abstract ▾
All authors Eitan Shaar, Ariel Shaulov, Yalcin Tur, Gal Chechik, Ravid Shwartz-Ziv
Computer Vision
DEX-AR: A Dynamic Explainability Method for Autoregressive Vision-Language Models
Walid Bousselham, Angie Boggust, Hendrik Strobelt et al. (+1 more)
As
Vision-
Language Models (VLMs) become increasingly sophisticated and widely used, it becomes more and more crucial to understand their decision-making process. Traditional explainability methods, designed for classification tasks, struggle with modern autoregressive VLMs due to their complex
token-by-
token generation process and intricate interactions between visual and textual modalities. We present DEX-AR (Dynamic Explainability for AutoRegressive models), a novel explainability method designed to address these challenges by generating both per-
token and sequence-level 2D heatmaps highlighting image regions crucial for the model's textual responses. The proposed method offers to interpret autoregressive VLMs-including varying importance of layers and generated
tokens-by computing layer-wise gradients with respect to
attention maps during the
token-by-
token generation process. DEX-AR introduces two key innovations: a dynamic head filtering mechanism that identifies
attention heads focused on visual information, and a sequence-level filtering approach that aggregates per-
token explanations while distinguishing between visually-grounded and purely linguistic
tokens. Our evaluation on ImageNet, VQAv2, and PascalVOC, shows a consistent improvement in both perturbation-based metrics, using a novel normalized perplexity measure, as well as
segmentation-based metrics.
Read the full abstract ▾
All authors Walid Bousselham, Angie Boggust, Hendrik Strobelt, Hilde Kuehne
Computer Vision
3D CBCT Artefact Removal Using Perpendicular Score-Based Diffusion Models
Susanne Schaub, Florentin Bieder, Matheus L. Oliveira et al. (+4 more)
Cone-beam computed
tomography (CBCT) is a widely used 3D imaging technique in dentistry, offering high-resolution images while minimising radiation exposure for patients. However, CBCT is highly susceptible to artefacts arising from high-density objects such as dental implants, which can compromise image quality and diagnostic accuracy. To reduce artefacts, implant inpainting in the sequence of projections plays a crucial role in many artefact reduction approaches. Recently,
diffusion models have achieved state-of-the-art results in image generation and have widely been applied to image inpainting tasks. However, to our knowledge, existing
diffusion-based methods for implant inpainting operate on independent 2D projections. This approach neglects the correlations among individual projections, resulting in inconsistencies in the reconstructed images. To address this, we propose a 3D dental implant inpainting approach based on perpendicular score-based
diffusion models, each trained in two different planes and operating in the projection domain. The 3D distribution of the projection series is modelled by combining the two 2D score-based
diffusion models in the sampling scheme. Our results demonstrate the method's effectiveness in producing high-quality, artefact-reduced 3D CBCT images, making it a promising solution for improving clinical imaging.
Read the full abstract ▾
All authors Susanne Schaub, Florentin Bieder, Matheus L. Oliveira, Yulan Wang, Dorothea Dagassan-Berndt, Michael M. Bornstein, Philippe C. Cattin