Computer Vision · Cross-listed from cs.CR
Robust Watermarks Meet Backdoored Models: Evading Diffusion Semantic Watermarks via Stealthy Backdoor
Jinyuan Liu, Tianshuo Cong, Pei Li et al. (+4 more)
Although semantic watermarking is considered a promising safeguard for images generated by Latent
Diffusion Models (LDMs), the reliance of the watermark detection pipeline on neural networks introduces a critical yet underexplored
backdoor attack surface. To systematically study this vulnerability, we propose GhostVAE to plant a stealthy
backdoor into the encoder of Variational Autoencoder (VAE), enabling reliable evasion of watermark detection. GhostVAE operates in two stages: it first constructs a universal trigger via power spectrum regularization to improve the trigger robustness, and then trains a backdoored VAE encoder with a parameter-aligned objective. Through extensive evaluations across three state-of-the-art semantic watermarking schemes and three widely adopted LDMs, we show that GhostVAE preserves watermark detection performance on benign images (achieving an average true positive rate of 94.4%), while simultaneously enabling highly effective evasion under trigger activation (achieving an average attack success rate of 94.6%). Moreover, we comprehensively analyze seventeen representative defenses and demonstrate that GhostVAE remains stealthy across the input space, parameter space, and latent space. Our work fundamentally undermines the trustworthiness of semantic watermarking systems and highlights that secure deployment of semantic watermarks requires end-to-end security considerations, particularly for neural network components.
Read the full abstract ▾
All authors Jinyuan Liu, Tianshuo Cong, Pei Li, Tianrui Wang, Xinlei He, Anyu Wang, Xiaoyun Wang
Computer Vision
DiffuseAgent-MI: Distributionally-Grounded,Tool-Integrated Self-Evolving Agents for Faithful Visual Reasoning
An Lanji, Dawei Liu, Jin Li et al. (+3 more)
Tool-integrated
vision-
language agents have made remarkable progress on compositional and multi-step visual
reasoning. Yet their outputs frequently exhibit unfaithfulness: the stated
reasoning path diverges from the computation that actually produced the answer, undermining reliability in safety-critical applications. We present DiffuseAgent-MI, a self-evolving
agent whose perceptual grounding is governed by a KL-minimal energy model over feature units, providing a distributional view of visual mechanistic interpretability. The
agent learns an energy landscape that softly constrains generated samples to lie near the native prior conditioned on the chosen interpretable unit, closing the gap between the explanation and the internal representation. A verifier then supplies trajectory-level faithfulness rewards, and a repair branch re-conditions the energy when the verifier flags an unfaithful step. On GeoQA, SciVis, VQA-v2 and an in-house
multimodal reasoning set, DiffuseAgent-MI improves accuracy by up to 5.1 points over prior self-evolving
agents while more than doubling mutual-information faithfulness and human-interpretability agreement. Our analysis shows the energy term and the verifier are complementary: the former guarantees distributional faithfulness, the latter trajectory-level faithfulness, and only their combination closes both gaps.
Read the full abstract ▾
All authors An Lanji, Dawei Liu, Jin Li, Haoran Xu, Mei Chen, Yu Tian
Computer Vision
Hybrid-Domain Posterior Sampling for Inverse Problems via Latent Flow Matching
Hongjie Wu, Yiping Xie and Jiancheng Lv
Latent Flow Models have revolutionized compressed-space image synthesis, yet their application to high-fidelity inverse problems remains bottlenecked. In this paper, we trace this dilemma to a fundamental
geometric limitation of pre-trained autoencoders, which we term First-Order
Manifold Blindness. Severe decoder compression (e.g., retaining only $\sim\!2\%$ of the original degrees of freedom) produces a rank-deficient Jacobian, rendering high-frequency measurement residuals in its orthogonal complement invisible to latent gradients even when the decoder can represent the target image. To overcome this bottleneck, we propose Hybrid-Domain Posterior Sampling (HDPS), a decoupled inference framework that disentangles physical measurement consistency from semantic prior modeling. HDPS diverges into the pixel space, leveraging Langevin dynamics to absorb precise orthogonal measurement gradients, and subsequently projects these structural corrections back onto the generative
manifold. An optimization-based latent
alignment is introduced to filter pixel-space artifacts while avoiding the semantic drift of direct encoding. Extensive experiments on diverse inverse problems demonstrate that HDPS establishes a new state-of-the-art, successfully recovering the high-frequency structural precision that latent-only solvers inherently discard. The code is available at \href{https://github.com/74587887/HDPS}{https://github.com/74587887/HDPS}.
Read the full abstract ▾
Computer Vision
DocPO: Advancing Document Policy Optimization via Tailored Step-Aware Rewards
Yunhao Wang, Binghong Wu, Zhenyu Huang et al. (+4 more)
Reinforcement learning (RL) for document parsing often relies on reference-based rewards rooted in edit distance (e.g., tree edit distance), yet it remains hard to optimize in the high-accuracy regime because such rewards become weakly discriminative: near-correct outputs receive very similar scores, providing limited learning signal for hard cases. We propose Step-Aware Annealing (SAA), a plug-and-play reward sharpening mechanism that progressively increases reward
curvature during training, amplifying subtle quality differences among high-scoring samples while preserving stability in early learning. Built on SAA, we introduce DocPO, a document policy optimization framework with element-specific, reference-based rewards anchored by edit-distance signals: normalized string edit distance (NED) for text, tree edit distance similarity (TEDS) for tables, and a hybrid Rubric+edit reward for formulas. Experiments on OmniDocBench and DocElemHard show that SAA consistently improves GRPO-style RL across document elements over non-annealed rewards, without requiring additional human supervision for reward construction.
Read the full abstract ▾
All authors Yunhao Wang, Binghong Wu, Zhenyu Huang, Jiacheng Shi, Shuo Huang, Tinghao Yu, Feng Zhang
Computer Vision
Unleashing the Power of Text: Text-Guided Flow Matching for Image Fusion under Complex Degradations
Axi Niu, Jieheng Li, Kang Zhang et al. (+3 more)
Infrared-visible image fusion under realistic degradation scenarios is a challenging task, as degradations not only cause a loss of reliable modality-specific information in observed images but also hinder the fusion process. Recent studies indicate that text can provide prior information about degradation characteristics, complementing the limited evidence available from corrupted input images and facilitating fusion. However, existing methods typically inject fixed global text representations into visual features, making it difficult for textual guidance to adapt to spatially varying degradations, local structures, and thermal saliency. To this end, we propose TGFusion, a text-guided latent-space flow matching framework that unifies degradation suppression and cross-modal fusion. TGFusion encodes task, degradation, and generation cues into structured
prompts. To fully exploit these priors, we design a
Prompt-conditioned Multi-stream Joint Flow
Transformer that represents text as an independent semantic stream alongside fusion, visible, and infrared streams. Joint
attention enables
token-level bidirectional interaction and layer-wise updating among semantic and visual representations, allowing degradation semantics to dynamically guide reliable information selection and fusion latent generation. Extensive experiments on public
benchmarks and complex degradation scenarios demonstrate that TGFusion achieves superior or competitive performance in perceptual quality, image naturalness, structural-detail preservation, and infrared-saliency retention, while remaining robust across diverse single and compound degradations.
Read the full abstract ▾
All authors Axi Niu, Jieheng Li, Kang Zhang, Qingsen Yan, Jinqiu Sun, Yanning Zhang
Computer Vision
GuideGround: VLM-guided Semantic Understanding and Viewpoint-aware Reasoning for 3D Visual Grounding
Yiwen Wang, Yuyang Deng, Yihao Long et al. (+1 more)
3D visual grounding aims to localize the target object in a 3D scene from a natural
language query, requiring both fine-grained semantic understanding and viewpoint-dependent spatial
reasoning. Existing methods typically formulate semantic understanding as an auxiliary closed-set object classification task and rely on multi-view feature aggregation for viewpoint
reasoning, limiting semantic generalization and weakening viewpoint-specific evidence. We observe that
vision-
language models naturally provide complementary capabilities through open-vocabulary semantic understanding and global scene perception. Based on this insight, we propose GuideGround, a VLM-guided framework that complements rather than replaces task-specific grounding models by leveraging VLMs for semantic enhancement and viewpoint-specific hypothesis verification. Specifically, we replace auxiliary closed-set object classification with VLM-generated object semantic descriptions to enhance semantic understanding. Meanwhile, instead of directly aggregating multi-view representations, we preserve viewpoint-specific grounding hypotheses through per-view grounding and explicitly verify them using VLMs across candidate viewpoints. Extensive experiments on the ReferIt3D
benchmark demonstrate that GuideGround consistently outperforms previous state-of-the-art methods. Comprehensive ablation studies further confirm the effectiveness of both the proposed semantic understanding and viewpoint
reasoning strategies.
Read the full abstract ▾
All authors Yiwen Wang, Yuyang Deng, Yihao Long, Xi Zhao
Computer Vision
Test-time Adaptation of Pelvic Bone Segmentation Models via Dynamic Reliability-Guided
Ling Ren, Chao Deng, Ziming Wang et al. (+2 more)
Reliable pelvic bone
segmentation (PBS) from CT is essential for
robot-assisted pelvic trauma surgery, yet deploying a source-trained model to a new hospital suffers from severe performance degradation due to cross-center domain shifts. While test-time adaptation (TTA) enables online model adaptation without accessing source data, existing methods show limited effectiveness for PBS, facing challenges including boundary degradation, anatomical inconsistency under domain shifts, and voxel-level class imbalance. To address these challenges, we propose a novel closed-loop dynamic Reliability-Guided TTA framework (ReGA) for PBS. Specifically, we introduce a pseudo-label reliability criterion termed
Segmentation Inference Consistency Evaluation (SICE), which jointly measures region overlap and boundary deviation via dropout-based ensemble predictions. Based on SICE, a trust-weighted refinement module adaptively updates features to mitigate boundary errors in pseudo-labels. Furthermore, a confidence-weighted region-level contrastive learning strategy is proposed to enforce anatomical consistency. Finally, ReGA follows the teacher-student (TS) scheme to alleviate voxel-level class imbalance. Experiments on three heterogeneous 3D pelvic CT
datasets demonstrate that ReGA consistently outperforms state-of-the-art TTA methods, enabling effective adaptation of the source-trained PBS model to unseen clinical domains. The code is available at https://github.com/Ren-ling/ReGA.
Read the full abstract ▾
All authors Ling Ren, Chao Deng, Ziming Wang, Yuecong Xu, Kai Zheng
Computer Vision
RadYOLO: Computationally Efficient 3D Object Detection and Segmentation in CT and MRI
Kai Geissler, Laurens Müller-Groh and Hans Meine
Object detection and
segmentation in three-dimensional medical images is a very active area of research. However, most proposed deep learning models carry a high computational cost, and only few aim to be broadly applicable, achieve high detection performance, and remain fast to execute on resource-constrained hardware. To address this gap, we present RadYOLO, a 3D extension of YOLO11 tailored to medical images. We compare it with nnU-Net and nnDetection on five
datasets comprising CT and MRI data with varying object sizes and prevalence. RadYOLO's detection performance surpasses that of nnDetection on four of five
datasets and is comparable on one. Compared to nnU-Net, RadYOLO performs better on lesion detection tasks, while nnU-Net excels at detecting large organs when precise localization is required. When rough object localization is sufficient, RadYOLO matches or outperforms nnU-Net on all five
datasets. Regarding inference time, RadYOLO is 8-46x faster than nnU-Net on a GPU. Compared to nnDetection the speedup is even higher. When executed on a CPU, RadYOLO's inference runs within seconds (still faster than nnU-Net on a GPU) offering a significant advantage for clinical and edge-device deployment. RadYOLO repository: https://github.com/FraunhoferMEVIS/RadYOLO
Read the full abstract ▾
Computer Vision
SpatialAfford: Teaching Compact VLMs Where to Look and Where to Ground for Affordance
Yufei Zhang, Chenlu Zhan, Donghui Sun et al. (+2 more)
Affordance grounding aims to localize the functional region for interaction, such as the handle to grasp or the button to press, rather than the whole object. This makes it more challenging than generic visual grounding because the target region is smaller, more ambiguous, and more dependent on task context, especially for compact
vision-
language models (VLMs) used in
embodied settings. Recent sequence-level supervision and reinforcement learning improve coordinate prediction quality, yet compact autoregressive VLMs still lack reliable affordance-aware visual focus before coordinate generation: the model can produce better coordinate
tokens while its cross-modal
attention remains diffuse and weakly anchored to the true affordance evidence. To address it, we propose SpatialAfford, a two-stage framework that first aligns
attention to the ground-truth affordance region through Spatial
Attention Alignment (SAA), then refines coordinate prediction with Spatial-Aware GRPO. By explicitly teaching the model where to look before optimizing where to ground, SpatialAfford turns affordance grounding from a purely output-constrained objective into
attention-grounded spatial
reasoning. Across ShareRobot-Bench, ReasonAff, and PartAfford, SpatialAfford consistently improves affordance grounding, with a compact 4B model outperforming stronger 7B+ baselines.
Read the full abstract ▾
All authors Yufei Zhang, Chenlu Zhan, Donghui Sun, Xiaoxin Chen, Hongwei Wang
Computer Vision
Optical Flow from Photons
Wendi Liu, Weichao Zeng, Weihang Ran et al. (+2 more)
Optical flow remains challenging in high-speed and low-light scenes, where the limited frame rate and sensitivity of conventional cameras lead to motion blur and underexposure. Single-
photon avalanche diode (SPAD) cameras offer single-
photon sensitivity and extremely fine temporal sampling. However, individual slices in these high FPS binary
photon streams are too sparse for dense correspondence. Temporal aggregation can provide the spatial cues required by optical flow, but accumulating
photons at fixed coordinates blurs moving structures. Motion-aware aggregation can reduce this blur, yet it depends on the flow being estimated. To address this dependency, we propose QuantaFlow, the first method for dense optical flow directly from SPAD streams. Instead of constructing a fixed input representation, QuantaFlow embeds SPAD representation construction into iterative flow refinement. At each iteration, the current flow coarsely aligns the slices within the source and target sub-streams. A
photon-flux transformation then constructs multi-scale representations containing intensity and structural cues, while adaptive multi-scale fusion balances
photon noise and residual motion blur at each pixel. The fused representations drive a feature-warping flow update, and the refined flow guides representation construction in the next iteration. We further construct a synthetic
dataset for SPAD optical-flow training and evaluation. Experiments on the synthetic
dataset and real-world SPAD data demonstrate the effectiveness and generalization of QuantaFlow.
Read the full abstract ▾
All authors Wendi Liu, Weichao Zeng, Weihang Ran, Yujie Lu, Yinqiang Zheng
Computer Vision
Image-Space Rule Discovery
Misora Sugiyama, Toya Oyama and Hirokatsu Kataoka
Can image editors solve worksheets while preserving unrelated content? WISP (Worksheet Image-Space Problem Solving) defines 11 tasks and eight information conditions. On 2,200 shared items per editor, Nano Banana Pro achieves a 48.7% strict automatic proxy pass rate (Auto-Strict), versus 0.0-13.4% for four other editors. Analysis of 13,200 outputs, including a separate diagnostic, distinguishes target, preservation, and format failures. Auto-Strict agrees with human-majority judgments on 95.5% of 440 outputs (inter-rater Fleiss' $\kappa = 0.894$). Excluding the two no-edit controls, Nano Banana Pro achieves 45.1% Auto-Strict, versus at most 3.2% for the other editors. Further probes show high symbol-tracking proxy scores but only 49% exact Wason selection. These results measure problem solving through edits, without establishing open-ended rule induction or transfer.
Read the full abstract ▾
Computer Vision
Practical Noise Modeling for SPAD Intensity Imaging
Wendi Liu, Yujie Lu, Zengxi Zhang et al. (+3 more)
Single-
photon avalanche diode (SPAD) cameras are promising for low-light and high-dynamic-range intensity imaging, but their practical use is limited by complex sensor-specific noise. Unlike time-correlated single-
photon counting (TCSPC) systems, SPAD cameras record whether at least one detection occurred in each gate without
photon timestamps in intensity imaging mode, making explicit noise decomposition difficult. We present a practical noise modeling and calibration framework for SPAD intensity denoising. Our forward model describes binary-frame accumulation with a Binomial observation process, models signal-independent dark noise as an exposure-dependent pure dark count term plus an exposure-independent dark-frame bias term, and incorporates pixel-wise response non-uniformity. We design a dedicated calibration procedure for the proposed model and use it to build a count-domain noise-synthesis pipeline for network training. For denoising, we further design a SPAD-specific dark-shading correction (SPAD-DSC) to remove most systematic noise before network training. We construct a real-world SPAD intensity
dataset for testing. Experimental results demonstrate the superiority of the proposed noise model.
Read the full abstract ▾
All authors Wendi Liu, Yujie Lu, Zengxi Zhang, Haiyang Jiang, Weihang Ran, Yinqiang Zheng
Computer Vision
DreamTraj: Generating 6-DoF Object Trajectories by Reading Unrendered Video Diffusion Latents
Tongsheng Ding, Zhen Luo, Yixuan Yang et al. (+4 more)
Accurate prediction of object trajectories during manipulation is essential for closing the perception-action loop. Progress is limited on two fronts: available
datasets lack fine-grained
language-to-motion annotations, and existing predictors either rely on privileged inputs such as
video, depth, or CAD models, or recover motion from fully generated
videos through costly, error-prone perception pipelines. We close the supervision gap with the MOVE
dataset, 5,038 object-centric
egocentric trajectories, each paired with a fine-grained natural-
language instruction rather than a coarse verb-noun label. We further propose DreamTraj, which predicts a 6-DoF object trajectory from a single RGB image and a task instruction, requiring no
video, depth, or CAD model at inference: rather than generating a
video, it reads motion from the internal representations of a frozen image-to-
video diffusion model at an early denoising step. A lightweight flow-matching Reader decodes query-key
attention tracks and pooled hidden states into relative 6-DoF poses. To our knowledge, this is the first approach to directly decode object 6-DoF trajectories from intermediate
video diffusion representations rather than generated pixels. DreamTraj sets a new state of the art on both translation and rotation against forecasters that consume multi-frame or privileged inputs, and runs 4.6x faster than generate-then-extract pipelines.
Read the full abstract ▾
All authors Tongsheng Ding, Zhen Luo, Yixuan Yang, Boyu Wang, Luyang Xie, Jinyu Yang, Feng Zheng
Computer Vision
CrossProjection: Geometric Grounding Beyond Viewpoint Change in Architectural Drawings
Kaho Li, Pengyu Zeng, Yuqin Dai et al. (+3 more)
Architectural drawings violate the usual assumption behind multi-view
reasoning: plans and sections are cuts, while elevations are facade projections, so corresponding components change appearance in ways camera motion cannot explain. We introduce CrossProjection, an anchor-grounded diagnostic of whether
vision-
language models preserve component identity and externalize geometry across heterogeneous architectural views. It evaluates Matching, Registration, and
Geometric Grounding through categorical judgments, candidate selection, and free point, line, and region localization. Across 23 real drawing sets and 1,954 categorical conditions per model, GPT-5.5 scores 82.4%, Qwen3-VL-32B-Instruct 62.2%, and GLM-4.5V 57.2%. A matched 200-target study crosses natural and vector-text-suppressed drawings with closed-candidate and free-geometry outputs. Candidate-supported performance is often higher, but free localization remains fragile: on natural drawings, point/region PCK@.05 is 54-76% for GPT, 8-10% for Qwen, and 14-36% for GLM; line endpoint PCK@.05 is 22%, 4%, and 0%. A coordinate grid recovers some GPT point/region precision but not lines. Three architecture-trained participants reach 87.3-93.3% categorical accuracy and 76-92% GT-region hit, supporting task feasibility rather than a population-level human ceiling. Because the categorical families do not form a same-item Matching-Registration contrast and interface controls alter multiple burdens, we avoid mechanistic claims. The supported conclusion is narrower: closed-choice or marked-element success does not entail reliable explicit
geometric grounding. For drawing-guided CAD/BIM systems, categorical correctness should not be treated as evidence of candidate-free spatial reliability. Reusable on-sheet anchors, fixed-denominator scoring, and hash-locked artifacts establish an audit trail for this gap.
Read the full abstract ▾
All authors Kaho Li, Pengyu Zeng, Yuqin Dai, Jun Yin, Tianjing Feng, Shuai Lu
Computer Vision
Scene2Sound: Auditory-Grounded Soundscape Generation for 3D Gaussian Worlds
Masaki Yoshida, Ren Togo, Takahiro Ogawa et al. (+1 more)
3D Gaussian Splatting (3DGS) turns captured or generated imagery into photorealistic 3D world simulations that users can freely explore, yet these worlds remain silent. Because existing audio generation methods condition on a single image or viewpoint, their sound is tied to that observation and cannot stay consistent while a listener moves. We introduce the task of generating a spatially consistent soundscape for a given 3DGS world through auditory grounding, identifying which objects in the world should emit sound and anchoring each to a persistent 3D position, and present Scene2Sound, a training-free framework built on this grounding. From the input world alone, our pipeline selects viewpoints that jointly cover the scene, identifies sound-emitting objects with a
vision-
language model, and associates the multi-view detections into 3D instances through Gaussian set matching, which measures the overlap between the Gaussian sets that render each detection. Each source then receives generated audio that a standard object-based audio engine spatializes in real time at arbitrary listener poses. We further propose two spatial-consistency metrics, one testing whether rendered audio responds consistently to listener motion and one testing whether the claimed sources are supported by views held out from their placement. On a curated set of generated 3DGS worlds and on 3DGS scenes generated from real-world 360-degree captures, Scene2Sound preserves the audio quality of strong per-viewpoint baselines while remaining spatially consistent where per-viewpoint and single-panorama pipelines do not, and a user study confirms the perceptual benefit. Project page: https://masaki-lmd.github.io/scene2sound/.
Read the full abstract ▾
All authors Masaki Yoshida, Ren Togo, Takahiro Ogawa, Miki Haseyama
Computer Vision
Structured Proxy Features for Multimodal NSCLC Survival Prediction from Pretreatment CT
Huu Phong Nguyen, Delower Hossain, Ehsan Saghapour et al. (+2 more)
Lung cancer results in roughly 1.8 million fatalities annually worldwide, with non-small
cell lung cancer (NSCLC) comprising the majority of cases. Despite advancements in treatment, survival stratification remains challenging due to intratumoral heterogeneity inadequately captured by conventional descriptors. Standard radiomic and deep learning techniques regard imaging features as independent quantities, overlooking structured interactions between tumor characteristics. We evaluate whether structured proxy features can enhance
multimodal NSCLC survival prediction by augmenting pretreatment computed
tomography (CT) representations, radiomics, and clinical variables with six simulation-derived features designed to capture interactions between heterogeneity and morphology. A radiomic-parameterized cellular automaton generates growth-rate and necrosis-ratio proxy features from baseline CT by using
entropy and sphericity to compute low-dimensional proxy parameters. The imaging backbone is a
Transformer-based Masked Autoencoder (TMAE), which was chosen after a systematic evaluation with alternative encoders within the same pipeline and provides
attention-based visualizations that highlight tumor regions receiving higher model
attention. On the public Lung1 cohort (n = 390), the primary four-modality fusion attained a C-index of 0.641 (iAUC 0.731, log-rank p < 0.001). The primary result compares favorably with prior
multimodal results on Lung1 (C-index 0.631; iAUC 0.592 [15]) under a comparable evaluation protocol, while a separate exploratory coefficient-optimization analysis achieved a best observed C-index of 0.662 (iAUC 0.748). These results indicate that, in addition to conventional radiomic, deep, and clinical representations within the Lung1
benchmark, simulation-derived proxy features may provide complementary predictive information within this fixed Lung1
benchmark.
Read the full abstract ▾
All authors Huu Phong Nguyen, Delower Hossain, Ehsan Saghapour, Zhandos Sembay, Jake Y. Chen
Computer Vision
Reconstruction-Shift Discrimination via Mask-Guided Latent Diffusion for Medical Anomaly Detection
Yibo Wan, Jinyu Cai, Yunhe Zhang et al. (+2 more)
Unsupervised medical anomaly detection learns normal anatomical patterns from healthy training images and identifies deviations at test time. Reconstruction-based and
diffusion-based methods commonly use the difference between an input image and its reconstruction as anomaly evidence. However, this residual can be ambiguous. Expressive models may preserve pathological structures, while benign anatomical variation, imaging noise, and acquisition differences may also produce large reconstruction errors. We propose discriminative mask-guided
diffusion (DMD), a medical anomaly detection framework that complements residual-based localization with reconstruction-shift discrimination. DMD first learns a compact quantized latent representation of normal images. Localized masks then perturb selected latent regions, and a latent
diffusion model reconstructs the perturbed representations. The resulting reconstructions are paired with their original normal images to define a self-supervised classification task. At inference, the classifier provides a learned image-level anomaly score, while the residual between the input and its
diffusion-based reconstruction yields a pixel-level anomaly map. Experiments on five
datasets spanning
brain MRI, breast ultrasound, and chest radiography show that DMD achieves the best overall performance among the state-of-the-art baseline methods.
Read the full abstract ▾
All authors Yibo Wan, Jinyu Cai, Yunhe Zhang, Yi Bin, See-kiong Ng
Computer Vision
Beyond Static Anchors: Bounded Prototype Conditioning for Language-Free Medical Anomaly Detection
Yibo Wan, Jinyu Cai and See-kiong Ng
Medical anomaly detection identifies abnormal images and localizes lesions under scarce supervision while generalizing across organs and modalities. Existing CLIP-based methods reduce annotation requirements through
vision–
language alignment, but their normal and abnormal references, whether text
prompts or learned visual
tokens, remain fixed across test images. Such static references may not transfer reliably to unseen targets in a cross-domain medical imaging scenario. To address this, we propose ReCAP, a
language-free framework that replaces static anchors with input-conditioned visual prototypes. ReCAP re-centers separated normal and abnormal prototypes for each image through a bounded gated modulation, enabling query-adaptive anomaly scoring while constraining context-induced prototype drift. For the few-shot setting, we introduce a non-parametric normal-reference
memory to preserve instance-level target-domain variation and complement the conditional prototype branch. Across six medical
benchmarks, ReCAP achieves the best image-level AUROC on all zero-shot and 23 of 24 few-shot settings, and the best zero-shot pixel-level AUROC on all three
segmentation datasets. Particularly, it reduces inference latency by over 70% compared to the fastest baseline, without text
prompts or test-time gradient updates.
Read the full abstract ▾
Computer Vision
Poplar: A Scalable Pipeline for Human-Centric Image Dataset Synthesis
Zhishan Zou
Recent image generators can synthesize convincing human-centric images, yet producing a useful collection remains different from producing a single successful image. A human-centric
dataset must cover varied people and contexts, avoid implausible attribute combinations, preserve an everyday photographic character, and expose quality-control decisions at scale. We present Poplar, a reproducible Specify–Render–Inspect pipeline for human-centric image
dataset synthesis. Specify samples structured attributes under commonsense constraints and verbalizes them as photography-oriented
prompts. Render uses a realism-adapted image generator across composition-aware aspect ratios and retries obvious technical failures. Inspect applies a single structured
vision–
language review to each candidate, preserving the original
prompt while rejecting intrinsic image defects or material
prompt mismatches. Using Poplar, we construct Poplar-9K: 9,401 curated human-centric image–text pairs retained from 11,765 reviewed candidates (79.9% acceptance). We release the
dataset together with the pipeline, configurations, immutable generation
prompts, and auditable inspection records as a compact resource for building customizable human-centric collections.
Read the full abstract ▾
Computer Vision
Boosting Generalizable Depth Estimation in Endoscopy by Mixture of Lightweight Experts and Intrinsic Image Alignment
Liangjing Shao, Beilei Cui, Yiming Huang et al. (+2 more)
Depth estimation is a significant task for 3D perception in endoscopic surgeries. However, illumination interference and feature diversity in various endoscopic scenes are still challenges for generalizable depth estimation and ego-motion estimation. Based on this, a novel self-supervised framework, EndoMINI, is proposed for depth estimation in endoscopic scenes. Specifically, mixture of low-rank experts (MiLoRE) is proposed to perform parameter-efficient fine-tuning, which can also boost the model adaptation to scenes with different characteristics. Meanwhile, an intrinsic image
alignment (IIA) is introduced into the training loss to alleviate the influence of light reflectance in endoscopy with a novel intrinsic image decomposition network. The proposed method is evaluated on SCARED
datasets for supervised depth estimation, and two endoscopic
datasets, Hamlyn and SERV-CT, for zero-shot depth estimation, compared with state-of-the-art works as well. The experimental results demonstrate outstanding performance of the proposed model and the effects of the main contributions.
Read the full abstract ▾
All authors Liangjing Shao, Beilei Cui, Yiming Huang, Changjing Liu, Hongliang Ren
Computer Vision · Cross-listed from cs.AI
Where did the ambiguity go? Examining how multimodal models interpret polysemous words
Jasin Cekinmez, Addison J. Wu, Raja Marjieh et al. (+1 more)
Human
language is highly polysemous. Many common words (e.g., "bank" or "palm") carry several distinct meanings that shape what humans communicate and imagine. Large
language models (LLMs) have been shown to understand this multiplicity of meaning, but much less is known about how polysemy surfaces in other modalities such as images. We study this across 17 text-to-image and 15 text-generation models by giving each a polysemous word with no context to fix its meaning and measuring which senses are produced over many samples. We find a clear
multimodal gap, where within every model family, generated images settle on far fewer senses than generated sentences (normalized
entropy 0.10 vs. 0.25), and both are far less varied than what people imagine for the same words (normalized
entropy 0.47). However, when we instead ask a model to list how often it would generate outputs corresponding to each possible meaning of a word, it predicts distributions that are more diverse than the actual space of outputs. These results reveal a
multimodal gap in how foundation models express meaning, and how their understanding may not transfer faithfully nor equally across modalities.
Read the full abstract ▾
All authors Jasin Cekinmez, Addison J. Wu, Raja Marjieh, Thomas L. Griffiths
Computer Vision
Decoding Children's Gait Behavior
Yifan Shen, Boyi Li, Meihuan Huang et al. (+12 more)
We introduce a new problem domain for human action recognition: the fine-grained analysis of children's gait behaviors from standard RGB
video. We specifically target the ambulatory patterns of children aged 3-17 years. Such behaviors arise naturally in the diagnosis and treatment of several critical developmental and neuromuscular disorders, such as cerebral palsy and hemiplegia. Despite their clinical value, current 3D sensor-based gait analysis systems are expensive, intrusive, and often impractical for young subjects. To address this, we introduce a new
dataset comprising over 1,100 high-frame-rate (60 FPS)
video sequences from 110 subjects, accompanied by synchronized, anonymized pose sequences. In each session, the child performs a 5-second "walk-around" task, capturing the gait cycle from multiple viewpoints. Crucially, we demonstrate that current state-of-the-art approaches, including gait foundation models and
Multimodal Large
Language Models (MLLMs), fail to effectively resolve these clinical nuances. We identify the key technical challenges in analyzing these erratic and subtle motor patterns and describe a unified end-to-end framework for decoding fundamental components of pediatric gait. Through comprehensive experimental results, we demonstrate the potential of this
dataset to drive novel research questions and establish a rigorous baseline for automated child gait assessment.
Read the full abstract ▾
All authors Yifan Shen, Boyi Li, Meihuan Huang, Yuanzhe Liu, Xu Cao, Jinyang Jin, Zhengyuan Li, Anglin Liu, Junho Kim, Jingyuan Zhu, Lan Fangzhou, Jianguo Cao, Jintai Chen, Ismini Lourentzou, James Matthew Rehg
Computer Vision
Artificial Intelligence for the Characterization of Particles and Fibers by Optical Microscopy
Simiao Sun, Kenneth Ng, Lynn Lee et al. (+6 more)
Optical microscopy of particle and fiber dispersions involves interpreting subtle visual cues influenced by specimen morphology, chemical composition, magnification, and illumination conditions. We introduce an artificial intelligence (AI)
distillation framework that extracts semantically rich image
embeddings from microscopy images using semantic anchors. A
multimodal teacher combines each image's visual
embedding with three text
embeddings representing illumination modality, magnification, and specimen identity and morphology. Generated by LongCLIP's extended-context text encoder, this yields a 2304-dimensional block-structured teacher vector whose component blocks remain physically interpretable throughout training and inference. A student
vision transformer (ViT) with a multi-layer perceptron (MLP) decoder is trained to reconstruct this teacher vector from the image alone, minimizing a mean absolute error (L1) loss that enforces coordinate-level fidelity to the teacher's block structure. A cross-
entropy term over pseudo-classes derived from HDBSCAN clustering of the teacher
embedding space acts as a collapse-prevention regularizer, enforcing inter-cluster separation without requiring contrastive negative mining. At inference, the student operates on image input alone, producing compact
embeddings that recover the full semantic content of the teacher vector. The framework achieves approximately 80% pseudo-class validation accuracy and 75% Recall@1 on fine-grained specimen description labels under leave-one-out nearest-neighbor
retrieval. These results demonstrate that semantic anchoring enables a
vision-only student to acquire richer and more interpretable representations than image-only training, with direct applicability to
retrieval, classification, and exploratory analysis of heterogeneous particle and fiber dispersions.
Read the full abstract ▾
All authors Simiao Sun, Kenneth Ng, Lynn Lee, Astrid Harth, Asami Odate, Aggelos Katsaggelos, Manuel Ballester Matito, Nicholas Eastaugh, Marc Walton
Computer Vision
The 1st AI Children Challenge
Boyi Li, Yifan Shen, Houze Yang et al. (+7 more)
The First AI Children Challenge aims to advance real-world applications of computer
vision and AI in child healthcare, child education, and pediatrics. The 2026 CV4CHL edition featured the first track in this domain: Children Gait Visual Analysis. The main goal of Children Gait Visual Analysis is the fine-grained analysis of children's gait behaviors from keypoint sequences. This is still a big challenge for human action recognition. Experienced medical doctors can distinguish these subtle nuances, but none of the people test AI models in this domain. To bridge this gap, we introduce thousands of 2D children keypoint sequences walking around
videos across various age groups of children (3-16 years old). There is a significant opportunity for batch analysis of these
videos to provide clinically relevant insights into medical diagnosis. The Challenge will be launched with two problem tracks: Edinburgh Visual Gait Score (EVGS) Scoring and Classification of Gait Patterns in Bilateral Spastic Cerebral Palsy. Each track is chosen in consultation with board-certified pediatricians based on the value of potential solutions. With the first available
dataset for such tasks and ground truth for each track, the challenge enabled participants to evaluate their solutions. Final rankings will be revealed after the competition concludes, fostering reproducibility and mitigating overfitting.
Read the full abstract ▾
All authors Boyi Li, Yifan Shen, Houze Yang, Xu Cao, Guojun Yun, Li Gao, Turong Chen, Long Xu, Jianguo Cao, Meihuan Huang