Computer Vision
FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control
Songchun Zhang, Sitong Guo, Xianghao Kong et al. (+4 more)
Generative
video compositing, which involves inserting external assets seamlessly into existing
video sequences, is essential for content creation and visual effects. However, existing approaches suffer from a control-fidelity trade-off: they either hallucinate motion from static images, failing to preserve the dynamics of pre-animated assets, or lack fine-grained spatial control for precise asset placement along user-defined trajectories. We propose FlexComposer, a unified framework that standardizes
video compositing as a trajectory-guided conditional generation task, enabling the seamless integration of both static images and dynamic footage. Our approach introduces three key designs: (1) a Unified Canonical Foreground Representation that decouples an object's intrinsic motion from its global displacement, standardizing heterogeneous inputs into a stabilized, centered latent space; (2) a Spatial-Aware Latent Injection strategy that exploits the translation equivariance of VAE latent spaces to transport canonical features onto target trajectories via a parameter-free mechanism; and (3) a Hybrid
Dataset and Synthetic-to-Real Curriculum that synergizes procedural simulation, real-world cinematic footage, and generative data to implicitly learn physically plausible illumination and shadow harmonization. This unified design handles diverse inputs from product photos to dynamic subjects achieving high-fidelity motion control and environmental integration without the need for explicit 3D reconstruction or auxiliary learnable adapters. Extensive experiments demonstrate that FlexComposer outperforms state-of-the-art methods in visual quality, temporal consistency, and trajectory adherence.
Read the full abstract ▾
All authors Songchun Zhang, Sitong Guo, Xianghao Kong, Pengwei Liu, Yuwei Guo, Lvmin Zhang, Anyi Rao
Computer Vision · Cross-listed from cs.RO
RayViT: Ray-Conditioned Visual Representations for Viewpoint-Robust Imitation Learning
Qian Wang, Longrui Chen, Peiran Sun et al. (+8 more)
Visual imitation learning enables
robots to acquire visuomotor skills directly from images, yet RGB observations lack explicit
geometric cues, making learned policies brittle to camera perturbations. To address this, we propose Ray-conditioned
Vision Transformer Encoder (RayViT), a lightweight architecture that injects camera geometry into pretrained ViT backbones. RayViT represents camera geometry as a Plücker ray map, patchifies it into ray features, and uses gated cross-
attention to produce a ray-conditioned class
token. These ray features are added as dense positional
embeddings, while the ray class
token replaces the original ViT class
token to provide a geometry-aware summary representation. We combine this approach with an auxiliary cosine similarity loss to consistently improve the performance and robustness for geometry-aware
tokens. Experiments on sim- and real-
robot tasks demonstrate that RayViT improves robustness by approximately 13 percentage points under camera perturbations in multi-task RoboCasa
benchmark and by 1.78 average completed stages in real-world multi-task success rate compared to baselines.
Read the full abstract ▾
All authors Qian Wang, Longrui Chen, Peiran Sun, Aleksandar Taranovic, Niklas Freymuth, Ge Li, Weiran Liao, C. F. Maximilian Nagy, Yucheng Tan, Tao Chen, Gerhard Neumann
Computer Vision · Cross-listed from cs.LG
A Human-Centered Validation of the Explainability-Performance Coefficient
Christian Oliva and Luis F. Lago-Fernández
The rapid adoption of deep learning models in high-risk domains has intensified the need for trustworthy Explainable Artificial Intelligence (XAI). However, objectively evaluating explanation fidelity and aligning XAI metrics with human-centered understanding remain critical open challenges. In this work, we propose a model-agnostic metric, the EPC score, which is an extension of the Explainability-Performance Coefficient (EPC), that quantifies explanation quality by explicitly balancing the trade-off between feature selection sparsity and preserved model performance. Through an empirical validation across tabular, text, and image modalities, we show that the EPC score effectively uncovers operational dependencies among network activations, data dimensionality, and explainer performance. Furthermore, we validate the EPC score against independent human-based explanations, proving that higher EPC scores strongly align with human lexical sentiment judgments and spatial visual annotations.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
WCM: A World Critic Model for Vision-Language-Action Reinforcement Learning
Senyu Fei, Xiaopeng Yu, Siyin Wang et al. (+3 more)
Reinforcement learning (RL) post-training of
Vision-
Language-Action (VLA) models has shown strong promise for robotic manipulation. Among RL methods, critic-based approaches rely on a value
estimator that predominantly operates on single-frame observations or single-frame VLM backbone latents, which is a fundamental mismatch with the partially observable nature of
robot control. A naive approach to incorporate observation history into the critic incurs exponential complexity with high-dimensional visual space, and still fails because pure scalar-return regression provides insufficient supervision for learning cross-temporal dynamics. We identify the root cause as a state approximation problem: without an explicit world modeling objective, the critic's representation cannot capture the temporal structure needed for accurate value estimation. To address this, we propose the World Critic Model (WCM), built on a lightweight LeJEPA architecture; WCM jointly predicts future latent state and estimates values, such that the critic's representation is explicitly trained to capture temporal dynamics rather than merely regress scalar returns. WCM integrates seamlessly into both on-policy and off-policy training pipelines and is compatible with state-of-the-art VLA backbones including Pi0, Pi0.5, and OpenVLA-OFT. Extensive experiments on 149 tasks across four
benchmarks demonstrate that WCM consistently achieves state-of-the-art performance in both in-distribution and out-of-distribution settings, with particularly strong generalization gains. We further validate WCM on seven real-world manipulation tasks using OpenVLA-OFT and Pi0.5 with off-policy RL, confirming stable deployment across diverse settings.
Read the full abstract ▾
All authors Senyu Fei, Xiaopeng Yu, Siyin Wang, Xianzhong Zhao, Jingjing Gong, Xipeng Qiu
Computer Vision · Cross-listed from cs.CL
FriendBench: Benchmarking Dyadic Familiarity Inference in Humans and Multimodal Large Language Models
Jeffrey M. Girard, Jason Z. Zheng, Jacqueline R. Vertino et al. (+2 more)
Reading a social situation often depends on behavior, not words alone. We introduce FriendBench, a
benchmark for inferring whether two people are already familiar or are meeting as strangers, from a 20-second clip of a dyadic ice-breaker conversation. Every pair answers the same type of
prompt, so only the manner of interaction can reveal the answer. Across text, audio, and
video, we compare 26 models from seven companies against matched human panels over 96 balanced dyads. The best model and the human crowd are statistically indistinguishable on accuracy in every modality, but reach it differently: humans stay balanced across the two answers, while the strongest models favor “stranger.” This is a difference in effective prior, not in discrimination. Richer channels help both unequally, and only humans gain from visible behavior on top of
speech. We release the stimuli, human ratings, and model predictions.
Read the full abstract ▾
All authors Jeffrey M. Girard, Jason Z. Zheng, Jacqueline R. Vertino, Antony D'Avirro, Benjamin Peloquin
Computer Vision · Cross-listed from cs.RO
FibVLA: An Efficient Temporal Vision-Language-Action Model with Fibonacci Sampling
Li Lin, Wujun Xu, Weiwei Meng et al. (+3 more)
Vision-
language-action models (VLAs), which leverage the cognition of
multimodal information to infer physical-world actions, provide a generalized solution for
embodied AI applications. Conventional VLAs usually concentrate on current digital cognition. While some efforts are made to enhance VLAs'
reasoning capabilities by capturing temporal information, encoding the long-context history causes an efficiency-decreasing issue. To reconcile the conflict between capturing temporal information and maintaining inference efficiency in VLAs, this paper introduces FibVLA, an efficient framework featuring temporal perception of long-context history. Specifically, we leverage logarithmic hindsight sampling to both proprioceptive states and visual frames to capture long-term temporal dependencies with minimal redundancy. For the action expert, we introduce the flow matching to produce action distributions, and the Fibonacci recurrent inference strategy to generate long-range planning steps based on real-time closed-loop feedback. Experiments demonstrate that FibVLA significantly improves action smoothness and success rates without retraining large-scale visual encoders. Efficiency analysis demonstrates superior real-time responsiveness compared to
video-based baselines in real-world evaluations.
Read the full abstract ▾
All authors Li Lin, Wujun Xu, Weiwei Meng, Kaiwen Xia, Kang Hao Cheong, Shuai Wang
Computer Vision
CoDe-SSM: Context-Detail Decoupled State Space Model for Efficient UHD Image Restoration
Jiaxu Su, Zhijian Wu, Jun Li et al. (+2 more)
Ultra-high-definition (UHD) image restoration must balance the aggregation of spatially recurring degradation cues with the preservation of localized image structures. Compact aggregation can reduce redundant processing but may attenuate edges, textures, and other fine structures. Existing approaches manage UHD restoration cost through downsampling, window partitioning, or cluster-based
token reduction; yet many of them do not explicitly retain information that is poorly represented by shared aggregation. In this study, we propose a Context-Detail Decoupled State Space Model (CoDe-SSM) for UHD restoration, which processes aggregated context and clustering residuals in separate pathways. The context modeling pathway, implemented by the Global Cluster Scan Module (GCSM), aggregates features into $K$ input-dependent cluster centers and applies selective SSM
reasoning over the resulting fixed-order sequence, enabling cross-region context sharing while decoupling computational cost from spatial resolution. The detail recovery pathway, implemented by the Local High-Frequency Module (LHFM), processes the clustering residual with an input-derived high-frequency mask and a sparse mixture of convolutional experts. Extensive experiments on five UHD
benchmarks and five degradation types demonstrate that our explicit context-detail decoupling strategy yields substantial gains in restoration quality while maintaining desirable efficiency.
Read the full abstract ▾
All authors Jiaxu Su, Zhijian Wu, Jun Li, Bo Zhang, Yefeng Zheng
Computer Vision
TOOD: Task-Aware Out-of-Distribution Score Calibration for Continual Learners
Mostafa ElAraby, Samer B. Nashed and Liam Paull
The primary challenge of continual learning (CL) systems is to learn new tasks while remaining performant on previously learned tasks. A similarly important though less well-studied aspect of CL systems is their ability to distinguish inputs that are unlikely to come from within the set of tasks the system has already encountered, often called out-of-distribution (OOD) detection. This paper presents several findings related to the dynamics of OOD detection in CL systems, causes of performance degradation over time which we call OOD forgetting (OODF), and proposed mitigation strategies for this degradation. Chiefly, we find the unintuitive result that OODF is only weakly anti-correlated with classification performance on previous tasks, suggesting that the underlying mechanisms producing OODF are distinct. Moreover, this effect is observed for both energy-based and feature-based OOD detection methods. Energy-based detectors suffer a drop in logit scale as additional tasks are learned, which we term the Confidence Gap, while feature-based detectors also degrade under a complementary effect we call
Manifold Crowding. Motivated by these observations, we propose TOOD, a training-free post-hoc method that decomposes logits into per-task energy scores and re-calibrates them using replay-buffer statistics. Experiments on CIFAR-10, CIFAR-100, and a 100-task ImageNet-1K stream show that TOOD improves OOD detection performance over uncalibrated energy in most settings and ranks first or second in nine of ten CIFAR configurations, with the largest gains when the confidence gap is most severe. These results suggest that a substantial portion of OOD deterioration in continual learning arises from score miscalibration rather than from a complete loss of discriminative structure.
Read the full abstract ▾
Computer Vision
TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning
Binnan Liu, Yechi Ma, Tian Xie et al. (+1 more)
The Abstraction and
Reasoning Corpus (ARC) tests whether a model can infer an unseen transformation from a few input-output examples and apply it to a new grid. Looped visual reasoners refine predictions over multiple iterations, but conventional training constrains only the final output, leaving intermediate refinements unconstrained. We propose that these refinements should instead follow the transformation step by step. We introduce TraceViT, a looped visual reasoner trained with semantically monotonic transformation chains. We obtain these chains by rewriting and verifying programmatic task implementations, decomposing each solution into intermediate grid states. Each iteration is grounded by a task reference derived from the few-shot demonstrations and an object workspace representing the current grid state. Because these chains may differ in length from the loop, soft trace
alignment enforces only their ordering, letting the model allocate iterations freely. TraceViT achieves 67.8% pass@2 on ARC-AGI-1 and 24.3% on ARC-AGI-2. Controlled ablations on ARC-AGI-1 show that trace supervision becomes beneficial only when paired with grounding. Code and data will be available at https://github.com/LiuBinnan/TraceViT.
Read the full abstract ▾
All authors Binnan Liu, Yechi Ma, Tian Xie, Wei Hua
Computer Vision
Explaining AI-Image Detection: What the Heatmap Actually Shows
Leonid Kuturin, Ilya Sotnikov, Mark Khusnutdinov et al. (+4 more)
A marketplace review photograph is a document: platforms approve refunds on it, and generative models drove the cost of forging one to zero. We study that detection problem, so we build a detector and attach an attribution map as its evidence, then measure what that pair delivers on 186,527 images under controls designed to change our conclusions when something is wrong. Compression history, not synthesis, drives naive evaluation: our strongest model reaches 0.9999 PR-AUC (area under the precision-recall curve) on a product-disjoint split, yet falls to 0.7254 once we re-encode synthetics into the real class's format, while five public detectors move by at most 0.07. Aligning one class relocates the cue rather than removing it, and the repaired model then assigns native files a median probability of synthesis of 0.0004. One identical final encode for both classes repairs that, and a three-seed factorial credits the encoding change with the whole gain (+0.176 +- 0.009 PR-AUC). That encode equalises the last stage only: forensic features alone still separate the classes at 0.7145 against a base rate of 0.254. For evidence we test maps causally, against controls that never consult the detector. Whether an attribution ranking exists at all depends on whether the detector reacts to the image. On our first-fix detector, which calls 96 of 100 edited frames real, no map beats a random one. On the detector we selected, twelve of seventeen maps clear that control on edited images and eight on generated ones; perturbation leads both axes and no gradient-CAM variant shows a positive advantage. The trivial controls never clear it, and on generated images the centre prior is worse than random. Our ensembled regional map clears both axes and takes the top pixel AP at 12.4 s per map against 44.9 for occlusion. Clearing a detector-blind control is not yet a faithful explanation, and we demonstrate none.
Read the full abstract ▾
All authors Leonid Kuturin, Ilya Sotnikov, Mark Khusnutdinov, Mikhail Potemkin, Pavel Baranas, Aleksandra Korepanova, Alexander Kalashnikov
Computer Vision
DynoDINO: Harnessing Dynamic Latent Information from DINO Features for Multi-Phase Medical Image Segmentation
Yu-Pu Hsu, Jen-Jee Chen and Yu-Chee Tseng
Multi-phase Contrast-Enhanced Computed
Tomography (CECT) plays a central role in the diagnosis and characterization of focal lesions by capturing temporal enhancement patterns across multiple acquisition phases. Accurate lesion
segmentation from such data remains challenging because clinically relevant contrast kinetics are distributed across phases, while anatomical inconsistencies, respiratory motion, and incomplete acquisitions often lead to inter-phase misalignment and interrupted temporal information. Conventional
segmentation frameworks typically process each phase independently or rely on simple fusion strategies, limiting their temporal
reasoning capability. To address these challenges, we propose DynoDINO, a unified framework tailored to address the core challenges of multi-phase medical image
segmentation. DynoDINO first performs slice-level
alignment to establish inter-phase anatomical correspondence and then employs a Multi-phase Fusion Model to jointly enhance temporal correlations across phases. Our fusion model incorporates a Mix-
attention (MA) mechanism for efficient multi-phase feature calibration and an Adaptive Gating Mechanism with difference-based residual learning to selectively preserve diagnostically relevant contrast variations while suppressing artifacts caused by residual misalignment. In addition, the adaptive gating mechanism improves training stability by preventing feature degradation caused by unguided subtraction operations. Experiments on three large-scale
datasets, including LiTS, PLC-CECT, and WAW-TACE, demonstrate that DynoDINO consistently improves boundary delineation and structural fidelity under standard, shifted, and missing-phase conditions.
Read the full abstract ▾
Computer Vision
MoRoute: Dynamic Routing for In-Context Multimodal Video Generation
Chong Gao, Jie Ma, Zhan Peng et al. (+5 more)
Multimodal video generation aims to generate and edit
videos conditioned on arbitrary combinations of text, images, and
videos within a single model, allowing diverse tasks to share complementary data and generative priors. Unifying these tasks requires
multimodal understanding of diverse conditions, which is typically provided by a pretrained
vision-
language model (VLM). A key challenge is how to connect the VLM's hierarchical
multimodal representations with a pretrained
video diffusion transformer (DiT). Existing methods either inject features from only the final or a few manually selected VLM layers, or jointly train architecture-matched understanding and generation streams, making it difficult to reuse heterogeneous pretrained backbones. We introduce MoRoute, a unified
multimodal video generation framework that formulates a frozen VLM and a pretrained
video DiT with different architectures as heterogeneous experts connected through dynamic layer routing. For each input, a lightweight block-wise router enables every DiT block to select the VLM layer most relevant to its generation stage, thereby learning an adaptive correspondence between
multimodal understanding and
video synthesis. MoRoute further incorporates reference images and source
videos directly into the DiT
token sequence through unified in-context conditioning, preserving fine-grained visual details across diverse generation and editing tasks. Experiments on IntelligentVBench, OpenVE-Bench, and RefVIE-Bench show that MoRoute consistently surpasses the best competing method on each
benchmark, improving the average score by 0.15, 0.18, and 0.34 on a 1-5 scale, respectively.
Read the full abstract ▾
All authors Chong Gao, Jie Ma, Zhan Peng, Chongxiao Wang, Haoxue Wu, Jun Liang, Guanbin Li, Jing Li
Computer Vision
The K-Space Signature: Frequency-Domain Representation Learning for Medical Deepfake Detection
Riccardo Raciti, Francesco Guarnera, Francesco Rundo et al. (+2 more)
In medical imaging, generative models are increasingly deployed to synthesize realistic data and augment limited
datasets. Unfortunately, while beneficial for
privacy-preserving data sharing, these synthesized images can be repurposed for malicious intents, threatening public health through the creation of Medical Deepfakes. To address this threat, we introduce the K-Space Signature (KSS), a novel forensic framework that isolates hardware and generative traces within the spectral domain. By shifting analysis to the frequency domain, the KSS suppresses macroscopic anatomical variance by subtracting an empirical global anatomical prior computed in the Logarithmic Power Spectral Density (Log-PSD) space. To effectively process these globally distributed spectral artifacts without the local spatial bias inherent to Convolutional Neural Networks, we pair the KSS representation with a novel 3D MLP-Mixer architecture equipped with an ArcFace metric-learning head. Extensive experiments on multi-center 3D MRI
datasets demonstrate that this combined approach achieves exceptional detection performance, exceeding 0.99 Accuracy and ROC-AUC on multi-generator synthetic
datasets. Furthermore, the framework exhibits robust zero-shot generalization, maintaining strong discriminative power (up to 0.93 Accuracy) on independent
datasets acquired from entirely unseen scanners. To ensure full reproducibility, the complete source code and pre-trained models will be made publicly available upon acceptance.
Read the full abstract ▾
All authors Riccardo Raciti, Francesco Guarnera, Francesco Rundo, Luca Guarnera, Sebastiano Battiato
Computer Vision
OSAGEN: Object-Aware Mask Priors and Multistage Decoupled Diffusion for Industrial Anomaly Generation
Jinyi Xu, Peng Chen, Yunkang Cao et al. (+3 more)
Industrial anomaly detection and localization are limited by scarce real anomalies and pixel-level annotations, a bottleneck that synthetic image-mask pairs can alleviate. However, existing few-shot mask-guided generation may over-follow mask geometry, produce weak anomalies, or use condition masks incompatible with the current object instance. We propose OSAGEN, which combines object-aware mask priors with multistage decoupled
diffusion. Its three-stage adaptation sequentially learns normal appearance, defect appearance under coarse conditions, and fine-grained mask calibration, improving defect realization and local control. QBG injects object structure from a matched normal image into mask
diffusion to produce object-aware priors, while ISC restricts anomaly propagation and preserves normal content during sampling. A lightweight materialization step recovers pixel-level labels aligned with the realized defects. On MVTec AD and VisA, OSAGEN achieves AP-P/F1-P scores of 88.1/82.2 and 68.5/66.1, respectively, under a unified downstream localization protocol. The code will be released upon acceptance.
Read the full abstract ▾
All authors Jinyi Xu, Peng Chen, Yunkang Cao, Chengliang Liu, Xinghui Dong, Chao Huang
Computer Vision
Multi-Source Multi-View Graph Domain Adaptation with Hyperbolic Residual Encoding for Cross-Site MDD Identification from rs-fMRI
Zhanpeng Zheng, Xiran Chen, Haiteng Jiang et al. (+6 more)
Cross-site identification of major depressive disorder (MDD) from resting-state functional
magnetic resonance imaging (rs-fMRI) is hindered by inter-site distribution shifts and heterogeneous functional connectivity (FC) views. These views capture complementary neural relationships but exhibit distinct site biases and graph
topologies, complicating
alignment without sacrificing disease-relevant information or cross-view consistency. Existing studies largely treat multi-view connectome learning and cross-site adaptation separately. To the best of our knowledge, few studies have jointly modeled multiple FC views under multi-source unsupervised domain adaptation for cross-site rs-fMRI-based MDD classification. We construct Pearson correlation, sparse representation, and Granger causality graphs, each encoded by a view-specific graph
attention network. Dual-stream adaptive fusion explicitly integrates pairwise cross-view interactions, followed by lightweight
hyperbolic residual encoding for
curvature-aware representation refinement. Class-wise Cauchy–Schwarz
alignment reduces inter-source and source-target discrepancies, complemented by
adversarial learning, information maximization, and confidence-aware pseudo-labeling. Across seven unlabeled target domains, our framework achieves 73.60% mean accuracy and 71.90% AUC, demonstrating effective generalization under heterogeneous acquisition conditions. These results highlight the effectiveness of unified heterogeneous-view modeling,
curvature-aware refinement, and multi-source domain adaptation for cross-site MDD identification.The source code is at https://github.com/OPUS-Lightphenexx/MM-HyperGDA
Read the full abstract ▾
All authors Zhanpeng Zheng, Xiran Chen, Haiteng Jiang, Renjie Tian, Qinyu Cai, Jiexi Liu, Xiaofeng Chen, Weikai Li, Yansu Wang
Computer Vision
Leveraging Transfer Learning with Class-Specific Decoders for Laparoscopic Segmentation
Priya Tomar, Aditya Parikh, Christian Bauckhage et al. (+1 more)
Effective multi-organ
segmentation in
surgical data requires learning the intricate anatomical features and alleviating the challenge of class imbalance, which results from relatively lower proportions of small and limitedly exposed structures. Recent works on laparoscopic multi-organ
segmentation focus on learning structure-specific features through class-specific decoder architectures and report favorable results. This work extends the decoder-focused architectures to investigate knowledge sharing in the cross-
surgical domain. We utilize two
datasets representing different
surgical domains, rectal and cholecystectomy surgeries, to explore how
surgical conceptual knowledge transfers under partially common anatomical representations. Additionally, we compare the feature adaptation for the encoder and decoder at different training stages to analyse the knowledge adaptation and retention in the network. Our results corroborate previous findings on decoder-specific architectures and demonstrate that the organ-specific decoder model (CEMD), fully fine-tuned after cross-domain pre-training, achieves the highest
segmentation performance (62.4% dice) while converging substantially faster than training from scratch. However, we also find that class imbalance in
surgical data remains a persistent challenge that transfer learning does not fully resolve for underrepresented anatomical structures.
Read the full abstract ▾
All authors Priya Tomar, Aditya Parikh, Christian Bauckhage, Rafet Sifa
Computer Vision
Lightweight Neural Networks for Affordance Segmentation: Enhancement of the Decoder Module
Simone Lugani, Edoardo Ragusa, Rodolfo Zunino et al. (+1 more)
The deployment of deep neural networks for visual affordance
segmentation on wearable
robots poses may prove critical, due to some conflicting aspects of the problem. On one hand, affordance
segmentation requires high-level abstraction capabilities, that typically involve large-size models. On the other hand, computing resources hosted on wearable
robots prevent to run large-size models in real-time. The paper presents an analysis of the role of the
segmentation head in the trade-off between generalization performance and compute cost. The obtained models outperform modern baseline solutions in well-known, real-world
datasets while meeting low computing requirements.
Read the full abstract ▾
All authors Simone Lugani, Edoardo Ragusa, Rodolfo Zunino, Paolo Gastaldo
Computer Vision
Weight-Space Mixture-of-Experts for Implicit Neural Representation Classification
Stanislaw Janik and Michal Byra
Implicit Neural Representations (INRs) encode signals as the weights of a coordinate-based neural network and have recently been proposed as an alternative domain for downstream learning. While promising, classification directly in weight space remains challenging due to the high dimensionality and complex structure of INR parameters. Furthermore, the way discriminative information is distributed across INR weights remains poorly understood. We propose a hierarchical Mixture-of-Experts (HMoE)
Transformer that processes INR weights using conditional computation aligned with the structure of the underlying implicit network. Coupled with a meta-learning framework that shapes INR parameters for downstream tasks, our model achieves state-of-the-art accuracy across standard
benchmarks, ranging from low-resolution
datasets to high-resolution ImageNet-1K. To gain insight into how INRs encode discriminative information, we develop weight-space attribution and
pruning methods that identify parameters most relevant for classification. These analyses reveal how class-specific structure emerges within INR layers and support the suitability of MoE architectures for weight-space learning. Our approach advances both the performance and interpretability of weight-space classifiers.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
MoPET: Parameter-Efficient Mixture-of-Experts for Unified Medical Image Classification
Sebastian Doerrich, Daniel Würtinger, Francesco Di Salvo et al. (+2 more)
Adapting deep learning models to profound clinical heterogeneity typically relies on parameter-efficient fine-tuning (PEFT) to avoid the severe overfitting associated with full end-to-end network updates. Although PEFT successfully navigates limited data scenarios, it inherently forces the training of a separate, isolated adapter for every specific diagnostic task. Consolidating these isolated adapters into a single generalist network risks negative transfer, as optimization gradients from conflicting visual domains interfere. To address this, we propose MoPET, a mixture-of-experts (MoE) method that uses a learned sparse router to direct each input through a small subset of low-rank PEFT experts injected into a frozen foundation model, sharing capacity across
datasets while limiting cross-domain gradient conflict. Through selected evaluations on the MedMNIST
benchmark, we first establish that PEFT outperforms full network updates, improving average accuracy from 86.50% to 88.97%. We then show that a single MoPET model consolidates four heterogeneous
datasets into one network, improving average accuracy over the best isolated PEFT adapters (93.46% versus 92.83%). Finally, we show that co-training with auxiliary
datasets improves accuracy on data-constrained clinical targets, raising average target accuracy over the strongest isolated adapter from 81.58% to 83.58%. Our source code is publicly available at https://github.com/sdoerrich97/mopet.
Read the full abstract ▾
All authors Sebastian Doerrich, Daniel Würtinger, Francesco Di Salvo, Shyam Nandan Rai, Christian Ledig
Computer Vision
QR-Structured Thermal Triggers for Targeted Semantic Attacks on Infrared Vision-Language Models
Xiang Chen, Yingying Zhao, Chao Li et al. (+7 more)
Infrared
vision-
language models (IR-VLMs) extend thermal perception to open-vocabulary classification, image captioning, and visual question answering. However, their robustness to structured thermal perturbations and the stability of cross-modal semantic
alignment remain insufficiently studied. We propose QR-Structured Thermal Triggers (QR-STT), a stealthy, training-free, black-box framework for targeted semantic steering of IR-VLMs. QR-STT preserves the functional regions of a QR pattern while optimizing its internal modules, each of which is assigned a cold, neutral, or hot thermal state. The framework jointly searches module
topology and rendering parameters, including position, scale, rotation, intensity, blur, and roundness. A three-stage gradient-free procedure with greedy module-flip refinement efficiently handles the mixed discrete and continuous search space. The objective promotes
alignment with an attacker-selected target, suppresses source-class evidence, and regularizes QR structure and visual similarity. Experiments on multiple CLIP-style encoders show that QR-STT consistently redirects image-text
alignment toward chosen concepts while maintaining visual stealth. Perturbations optimized for classification also transfer to image captioning and VQA, causing target-consistent semantic drift in generated outputs. These results identify QR-structured thermal patterns as an interpretable attack surface for
language-driven infrared perception and highlight the need for robustness evaluation against structured cross-task semantic attacks.
Read the full abstract ▾
All authors Xiang Chen, Yingying Zhao, Chao Li, Jiaju Han, Ben Zhang, Ang Li, Jiahuan Long, Yiwei Wei, Jiujiang Guo, Chengyin Hu
Computer Vision
Role-Break in Attention Heads: Understanding and Detecting Hallucinations in VLMs
Mingyu Wang, Weilin Jin, Wenbo Li et al. (+5 more)
Despite remarkable progress in
vision-
language generation,
Vision-
Language Models (VLMs) remain prone to
hallucinations, producing content that is inconsistent with or unsupported by the input image. Existing works largely design detection or mitigation methods around one specific
hallucination pattern, such as visual-textual imbalance, but real VLM
hallucinations arise from a mixture of multiple patterns, so signals bound to a single pattern struggle to remain stable across models and tasks. Under a unified head-level view, we find that
hallucination-induced changes manifest as localized deviations from each head's faithful contextual behavior, a phenomenon we term Role-Break. Detailed analysis reveals that these deviations are systematically organized across
attention heads, contextual sources, and deviation directions, and that the resulting signal is linearly readable once head identity is preserved. Based on these findings, we build a lightweight linear detector on top of Role-Break that requires no fine-tuning of the VLM, whose feature dimension stays below 5,000 and reaches an average AUROC of 93.23 across six VLMs and four
benchmarks. A small-scale intervention experiment further shows that the detected
tokens can be directly acted upon in the discriminative setting.
Read the full abstract ▾
All authors Mingyu Wang, Weilin Jin, Wenbo Li, Haoyang Huang, Nan Duan, Tong Jia, Chaoran Luo, Ying Li
Computer Vision
OSEF: One-Step Evidence Fusion for Cross-Video Scene Procedure Planning
Zhentong Ye, Lei Zhang, Sijia Zhou et al. (+7 more)
Video Scene Procedure Planning (VSPP) supplies the target start-goal observations in advance, leaving open how a planner should act when the evidence must itself be retrieved. We introduce Cross-
Video Scene Procedure Planning (CVSPP): given an answer-redacted start-goal query and K candidate
videos, a model must retrieve the supporting
video, localize the relevant window, and predict the action sequence. Two obstacles couple here. Same-task demonstrations share stages and windows, and an early hard selection passes the wrong scene chain to the planner. We build an eleven-source
benchmark with typed negative roles, a fail-closed answer-leakage gate, and separate Evidence- and Plan-axis metrics. On its 14 source-horizon
cells we adapt nine planner families against a majority-sequence floor. We then present One-Step Evidence Fusion (OSEF), which scores a query-conditioned
cell-and-span
lattice over all candidates and feeds the full soft
lattice to the planner through a
token-global adapter, cropping no window beforehand. OSEF ranks first on all six
cells the
benchmark certifies as method-rankable. On four matched same-task COIN and CrossTask
cells it improves exact-
video-and-plan success by 2.9-10.7 points over an enhanced hard-selection SOTA, and a component study assigns the largest single increment to the
token-global interface. Five converted-source
cells sit at or near the majority-sequence floor, the
benchmark's remaining headroom. The supplementary package includes model constructors and evaluation code.
Read the full abstract ▾
All authors Zhentong Ye, Lei Zhang, Sijia Zhou, Yingda Yu, Yuehan Shi, Jiaqi Xuan, Shuaiwu Dong, Guanchao Tong, Meimei Zhang, Bin Li
Computer Vision
Dense Temporal Contrast Synthesis via Conditioned Latent Transport
Smriti Joshi, Apostolia Tsirikoglou, Daniel M. Lang et al. (+15 more)
Dynamic contrast-enhanced
magnetic resonance imaging (DCE-MRI) is essential for breast cancer management, but reliance on gadolinium-based contrast
agents (GBCAs) restricts use in contraindicated populations, prolongs scan protocols, and presents environmental toxicity concerns. Contrast synthesis offers a non-invasive alternative; however, existing approaches struggle to balance spatial realism with temporal continuity, suffer from slow iterative sampling, underutilize structural priors, and lack clinical validation. We propose a novel conditioned latent transport framework that predicts contrast enhancement in a single forward pass. By anchoring the latent trajectory to the pre-contrast anatomy and applying continuous time conditioning, the model synthesizes patient-specific contrast
evolution at any acquisition time. The proposed approach outperforms baseline and the state-of-the-art models across spatial, perceptual, temporal, and distributional metrics. Evaluated on an independent external cohort, the method demonstrates robustness to domain shifts induced by scanner noise as well as differing acquisition protocol. Furthermore, our synthetic contrast enhancement significantly improved downstream tumor
segmentation performance, yielding a 22.4% relative increase in Dice coefficient (0.60 vs. 0.49 baseline pre-contrast, p < 0.01), reducing boundary
segmentation error by over 39%, while outperforming all other generative model baselines. Finally, a reader study involving four breast radiologists evaluated the image quality, kinetic fidelity, and diagnostic viability of our synthesized sequences across 40 randomly selected cases. The results demonstrated that in 70% of cases, synthesized images provided sufficient clinical information to support the same management decisions as real DCE-MRI, suggesting a path toward safer and faster contrast-free or contrast-reduced imaging workflows.
Read the full abstract ▾
All authors Smriti Joshi, Apostolia Tsirikoglou, Daniel M. Lang, Richard Osuala, Noah Márquez Varaa, Alejandro Guzman, Grzegorz Skorupko, Sebastian Ibarra Arregui, Lidia Garrucho, Akane Ohashi, Dimitra Ntoula, Eugen Divjak, Oğuz Lafc{\i}, Jan C. Peeken, Julia A. Schnabel, Fredrik Strand, Oliver Diaz, Karim Lekadir
Computer Vision
VFAD: Variational Semantic Prompting Meets Frequency-Adaptive Representation Learning for Zero-Shot Anomaly Detection
Peng Chen, Kaige Li, Wei Wang et al. (+5 more)
Zero-shot anomaly detection (ZSAD) aims to detect and localize anomalies in unseen categories without access to target-specific training data. Although recent CLIP-based methods have demonstrated promising generalization through
vision-
language alignment, they remain limited in capturing diverse anomaly semantics and subtle local variations. To address these limitations, we propose VFAD, a unified framework that combines variational semantic prompting with frequency-adaptive representation learning. Specifically, we introduce a Variational Semantic
Prompt Extractor (VSPE), which adaptively aggregates anomaly-relevant local semantics from dense patch
tokens and regularizes them through a variational information bottleneck, thereby incorporating fine-grained visual cues and enabling more precise cross-modal
alignment. Furthermore, we develop a Frequency-Adaptive Representation Aggregation (FARA) module that leverages wavelet-based frequency decomposition and frequency-specific expert aggregation to enhance anomaly-discriminative visual representations. By jointly strengthening semantic guidance and visual representation learning, VFAD improves both anomaly discrimination and fine-grained localization. Extensive experiments on 13 industrial and medical
benchmarks demonstrate that VFAD consistently outperforms existing state-of-the-art ZSAD methods across diverse anomaly scenarios. The code will be publicly available upon publication.
Read the full abstract ▾
All authors Peng Chen, Kaige Li, Wei Wang, Mingbo Yang, Wenqiang Wang, Li Shen, Fangjun Huang, Chao Huang
Computer Vision
SatEdit: Mask-Conditioned Image Editing via VLM-Guided Segment Annotation
Muhammad Talha and Muhammad Ahmed Amer
Satellite image editing requires spatially precise object-level control, but supervised editing
datasets for overhead imagery are costly to build because object masks, semantic labels, and paired edits are rarely available at scale. We introduce SatEdit, a mask-conditioned satellite image editing framework that constructs training supervision from unlabeled imagery. SatEdit proposes object masks with a seg- mentation foundation model, assigns semantic la- bels to sampled segments with a
Vision-
Language Model, and applies lightweight human verification before generating paired addition and removal exam- ples through mask-guided inpainting. We fine-tune a high-resolution image editing backbone with LoRA on a SODA-A-derived
dataset containing 1,014 im- ages and 852 verified object annotations across 91 classes. In controlled comparisons with open- source and proprietary image editing models, SatE- dit achieves the highest aggregate masked-region se- mantic
alignment, with a CLIP score of 0.6322 and CLIP delta of 0.0726, while preserving the surround- ing scene qualitatively. These results suggest that VLM-assisted segment annotation is a practical route to data-efficient, spatially controllable satellite image editing.
Read the full abstract ▾
Computer Vision
DualDiT: A Conditional Dual-Output Diffusion Transformer for Joint OCT Image and Segmentation Mask Generation
Fernando García-Torres, Rocío del Amor, Sandra Morales et al. (+4 more)
Background and Objective: Generating realistic medical images with anatomically accurate
segmentation masks helps address the shortage of annotated data in medical imaging, particularly in optical coherence
tomography (OCT) of mouse eyes, where manual retinal layer delineation is labour-intensive due to tiny structures and required expertise, resulting in scarce
datasets. While
diffusion models perform well in medical image synthesis, joint image-mask generation has relied mainly on U-Net-based denoisers, leaving
diffusion transformers largely unexplored. Methods: We propose a conditional dual-output
Diffusion Transformer (DualDiT) for joint synthesis of OCT B-scans and
segmentation masks of the upper retinal
cell layers in ex vivo mouse retina. DualDiT encodes both modalities into a shared latent space via a pretrained VAE, concatenates their latent representations, and performs conditional
diffusion over the joint tensor. We compared DualDiT against two adapted
diffusion baselines: DDPM and LDM. Generative quality was assessed via Fréchet Inception Distance (FID) and spatial FID (sFID); practical utility via synthetic data augmentation for downstream U-Net
segmentation; and perceptual realism via evaluation by three domain experts. Results: DualDiT achieved the best generative quality (FID 56.14, sFID 114.35), outperforming DDPM and LDM. Expert panels misclassified 46% of synthetic samples as real and 42% of real samples as synthetic. Adding DualDiT-generated images and masks improved Dice and IoU scores on a held-out
segmentation test set. Conclusions: DualDiT shows that
transformer-based
diffusion models can effectively learn the joint distribution of OCT images and
segmentation masks, surpassing DDPM- and LDM-based baselines in generative fidelity, downstream utility, and perceptual realism, highlighting its potential for data augmentation in annotation-scarce medical imaging.
Read the full abstract ▾
All authors Fernando García-Torres, Rocío del Amor, Sandra Morales, Álvaro Barroso, Peter Heiduschka, Björn Kemper, Valery Naranjo
Computer Vision
CALM-AH: An ABAW11-Calibrated Multimodal Ensemble with Reliability-Gated Multi-Expert Consensus for Video-Level Ambivalence and Hesitancy Recognition
Wenzhuo Sun, Mingjian Liang, Richard Attfield et al. (+3 more)
Ambivalence and hesitancy (A/H) are subtle behavioural states that may be expressed through
language, voice, facial activity, and other non-verbal cues. The ABAW11 A/H
Video Recognition Challenge asks systems to assign a binary A/H label to each naturalistic interview
video. Performance is measured using Macro-F1 so that recognition of both A/H and No-A/H samples receives equal importance. We present CALM-AH, a
multimodal ensemble that combines textual, acoustic, visual, and derived behavioural-statistical features. We construct 15 non-empty combinations of these feature branches. For each combination, we select the best of three classifier families using validation binary cross-
entropy and optimise its decision threshold for validation Macro-F1. The resulting binary decisions are combined using fixed hard-voting weights transferred from BROTHER. We further introduce Reliability-Gated Multi-Expert Consensus(RG-MEC), an anchor-preserving decision-level ensemble that combines an initial prediction with three complementary correction experts: CALM-AH, AffectGPT, and a GPT-based semantic verifier. The initial system provides the default prediction. Its label is overridden only when all three correction experts unanimously support the same alternative class; otherwise, the anchor prediction is retained. This unanimity-gated design limits the influence of isolated expert errors while permitting bidirectional correction when task-specific,
multimodal-affective, and semantic-pragmatic evidence are fully consistent. On the participant-disjoint ABAW11
dataset, CALM-AH achieves a Macro-F1 of 0.7525, and the complete RG-MEC system achieves 0.7771.
Read the full abstract ▾
All authors Wenzhuo Sun, Mingjian Liang, Richard Attfield, Zongyuan Ge, Xuelian Cheng, Pamela Carreno-Medrano
Computer Vision · Cross-listed from cs.RO
BWM: A Low-Cost High-Fidelity World Simulator for Robot Learning
BWM Team
Reliable
robot learning requires a world simulator that can predict action consequences before execution on physical hardware, including risky and failure-prone outcomes. Existing physics simulators require substantial asset construction and calibration and still face a sim-to-real gap, while
video generators often lack precise control over their responses to fine-grained
robot actions. In this paper, we present the Boundless World Model (BWM), an open-source, low-cost, high-fidelity world simulator for
robot manipulation. BWM is an action-conditioned world model that combines initial-environment guidance, dynamic visual history, and temporally aligned
robot-action conditioning for stateful autoregressive prediction of future observations. We construct action-aligned training clips through trajectory replay, overlapping clip sampling, and initial-observation enhancement. BWM serves as a data engine that augments imitation-learning data with action-aligned rollouts, and as a policy evaluator for closed-loop assessment, risk anticipation, and policy ranking. Experiments on the WorldArena
benchmark and physical
robots demonstrate improved simulator fidelity and functional utility across the data-engine and policy-evaluator settings. BWM ranks first overall in the WorldArena Challenge across Track 1 and its two Track 2 applications. We release the BWM open-source ecosystem, including model checkpoints, training and inference code, and interfaces for data generation and policy evaluation.
Read the full abstract ▾
Computer Vision
FillGS: Filling Observation Gaps in 4D Gaussian Splatting via Viewpoint-Time Selection and Generative Refinement
Takashi Otonari and Toshihiko Yamasaki
4D Gaussian Splatting (4DGS) can render dynamic scenes photorealistically. However, with limited viewpoint coverage, some spatiotemporal regions remain sparsely observed, leading to artifacts, particularly in scenes with large motion. Existing approaches leveraging generative models rely on heuristic virtual-viewpoint selection before refining rendered views. As a result, they cannot actively explore such sparsely observed regions. To address this issue, we propose a pipeline that actively selects spatiotemporal virtual viewpoints to improve 4DGS reconstruction. Our method selects virtual viewpoints for generative enhancement based on the rendering sensitivity and motion-aware observation density of 4D Gaussians, prioritizing views that alleviate observation sparsity. In the refined images, we filter out regions that conflict with captured observations or are likely to contain generative artifacts and then fine-tune 4DGS using only the reliable regions. We evaluate our method on multi-view
video benchmarks using new train/test splits designed to induce observation gaps. Results show consistent improvements over prior viewpoint selection strategies and fine-tuning methods in both qualitative and quantitative evaluations, while reducing artifacts.
Read the full abstract ▾
Computer Vision
Training-Free Entity-Level Few-Shot Segmentation of Remote Sensing Images with Advection Refinement
Xueting Bai and Huan Ni
Existing cross-domain few-shot
segmentation approaches suffer from high training costs due to source-domain episodic training and pixel-wise dense prediction, while often producing fragmented and noisy predictions. To overcome these issues, we propose a training-free entity-level few-shot
segmentation framework for remote sensing images with advection refinement. Specifically, we first leverage SAM3's generic
geometric priors to generate category-agnostic entity primitives. By reformulating few-shot inference from pixel-level prediction to entity-level
reasoning, foreground and background prototypes are constructed and combined with dense textual semantic responses from SAM3 to build a multi-modal semantic potential field. Furthermore, an advection equation-based semantic refinement mechanism is introduced to propagate category-aware information across both feature and similarity spaces, enhancing semantic continuity and suppressing local texture noise. Extensive experiments on multiple remote sensing
datasets demonstrate that the proposed framework effectively mitigates domain shift and local noise, substantially improving SAM3's adaptation capability for remote sensing few-shot
segmentation without additional training. Our code will be publicly available at https://github.com/yu-ni1989/ELFSS-AR.
Read the full abstract ▾