Computer Vision
Hierarchical Prototype-Memory Adaptation of SAM for Surgical Instrument Segmentation
Xinning Yao, Jingjing Wang, Jinghua Yue et al. (+3 more)
Surgical instrument
segmentation (SIS) is fundamental for computer-assisted surgery, where reliable instrument masks enable precise scene understanding and clinical assistance. Recently, adapting foundation models like the Segment Anything Model (SAM) to the
surgical domain via
prompt-learning has shown encouraging results. However, the performance of these adapted models under challenging
surgical conditions is constrained by suboptimal adaptation mechanisms. Specifically, optimizing
prompts or prototypes purely via downstream
segmentation loss tends to cause them to degenerate into task-specific parameters rather than serving as persistent, stable category
memory, thereby degrading their robustness against complex intraoperative variations. Moreover, routing multi-scale visual cues through a single
prompt pathway creates a bottleneck that hinders effective scale-matched coupling. To address these limitations, we propose HPMA, a Hierarchical Prototype-
Memory Adaptation framework for SAM. Specifically, HPMA constructs a frozen, multi-scale visual prototype
memory bank from annotated
surgical scenes and integrates it into SAM's feature space using lightweight adapters to preserve stable category evidence. To maximize the utility of multi-scale cues, we introduce a scale-matched coupling mechanism where global prototypes calibrate class-level
prompt features, structural prototypes guide decoder object queries, and local prototypes align high-resolution feature maps through a local
alignment objective. Extensive experiments on the public EndoVis2017 and EndoVis2018
datasets demonstrate that our approach achieves state-of-the-art performance, outperforming existing foundation model adaptation methods.
Read the full abstract ▾
All authors Xinning Yao, Jingjing Wang, Jinghua Yue, Xiaoyan Luo, Fugen Zhou, Bo Liu
Computer Vision
VizAnchor: Decoding Manipulation Intent from Tampering Visualizations via Dual-Anchor Reasoning
Xiaotian Zhang, Huayuan Ye, Haiyang Zhang et al. (+3 more)
Data visualizations are widely used for communicating information, but they are also vulnerable to intentional manipulations that induce misleading interpretations. Existing methods focus on locating tampered regions or recovering hidden information, without explaining how the visualization has been manipulated or why the resulting changes may mislead viewers. We propose VizAnchor, a framework for visualization manipulation understanding through dual-anchor evidence construction and VLM-based
reasoning. In the first stage, VizAnchor constructs a semantic anchor to recover authentic chart information and a spatial anchor to localize tampered regions. In the second stage, three specialized
agents decode the manipulation. The misleader grounding
agent analyzes a four-panel visual
prompt to predict the misleader information. The chart narrative reconstruction
agent takes the original and tampered charts as inputs and reconstructs their respective visual narratives. Finally, the intent inferring
agent integrates the visual evidence and misleader information to infer the misleading intent. We further construct a
dataset for tampering localization and a
dataset for misleading intent inferring. Evaluation shows that VizAnchor accurately localizes manipulations and produces faithful explanations of their manipulation, misleaders, and misleading intents.
Read the full abstract ▾
All authors Xiaotian Zhang, Huayuan Ye, Haiyang Zhang, Chenhui Li, Changbo Wang, Sicheng Song
Computer Vision · Cross-listed from cs.LG
It depends: Incorporating correlations for joint aleatoric and epistemic uncertainties of high-dimensional output spaces
Leonhard F. Feiner, Manuel Nickel, Martin Menten et al. (+6 more)
Uncertainty Quantification (UQ) plays a vital role in enhancing the reliability of deep learning model predictions, especially in scenarios with high-dimensional output spaces. This paper addresses the dual nature of uncertainty – aleatoric and epistemic – focusing on their joint integration in high-dimensional regression tasks. For example, in applications like medical image
segmentation or restoration, aleatoric uncertainty captures inherent data noise, while epistemic uncertainty quantifies the model's confidence in unfamiliar conditions. Modeling both jointly enables more reliable predictions by reflecting both unavoidable variability and knowledge gaps, whereas modeling only one limits transparency and robustness. We propose a novel approach that approximates the resulting joint uncertainty using a low-rank plus diagonal covariance structure, capturing essential output correlations while avoiding the computational burdens of full covariance
matrices. Unlike prior work, our method explicitly combines aleatoric and epistemic uncertainties into a unified second-order distribution that supports robust downstream analyses like sampling and log-likelihood evaluation. We further introduce stabilization strategies for efficient training and inference, achieving superior UQ in the tasks of image inpainting, colorization, optical flow, and depth estimation.
Read the full abstract ▾
All authors Leonhard F. Feiner, Manuel Nickel, Martin Menten, Laurin Lux, Rickmer Braren, Daniel Rueckert, Georgios Kaissis, Raphael Rehms, Johannes Paetzold
Computer Vision · Cross-listed from eess.IV
Unified-protocol voxel-level pulmonary embolism annotations for three public CT angiography datasets
Qihang Sun, Zhongxiao Liu, Bailiang Jian et al. (+6 more)
Reliable clot-volume quantification and subsequent risk assessment in pulmonary embolism depend on precise
segmentation of emboli on computed
tomography pulmonary angiography. Deep learning models for this task must be trained on accurate voxel-level labels. The three public
datasets that provide such labels were annotated under different protocols, and some of their studies contain unlabeled emboli or labels that are discontinuous across slices. This Data Descriptor presents voxel-level pulmonary embolism annotations for 149 of the 166 studies in these
datasets. A primary rater drew all annotations under a single protocol. A thoracic radiologist with more than 20 years of experience reviewed and revised them. Three raters at three different centers independently annotated a subset of 15 studies. The subset was selected by source
dataset and embolus location. Technical validation quantifies volumetric agreement with the source annotations, changes in within-mask attenuation, and inter-rater agreement on the subset. The
dataset is intended to allow
segmentation models to be developed and compared under a common reference standard.
Read the full abstract ▾
All authors Qihang Sun, Zhongxiao Liu, Bailiang Jian, Shenman Qiu, Jingyuan Wang, Lei Zhang, Lixiang Xie, Jiazhen Pan, Christian Wachinger
Computer Vision
Low-Rank Ternary Adaptation for Fine-Tuning Transformers
Alexandru-Dragos Manolache, Yunqiang Li and Jan van Gemert
Ternary
transformers offer extreme
memory and compute efficiency, but existing low-bit LoRA-based methods cannot directly fine-tune ternary weights. Current approaches either require dequantization, restoring low-bit base weights to higher precision to merge with adaptation weight, or update only quantization parameters, preventing a merged model that remains ternary. We propose ternary multiplicative adaptation, which represents discrete updates of ternary weights such as sign flips or zeroing through a low-rank Kronecker factorization into two small ternary
matrices applied element-wise to ternary weights. This design is parameter-efficient and expressive, preserves the ternary domain, and supports direct merging without dequantization. Experiments on six models across
language and
vision, including ternarized LLaMA-3 1B and 3B and a ternary ViT-B/16, demonstrate that our method recovers much of the performance lost to quantization and outperforms strong low-bit and ternary baselines. Code is available at https://github.com/alexmanoo/ternary_adaptation.
Read the full abstract ▾
Computer Vision
DoublesEval: Diagnosing Multi-Agent Tactical Reasoning in Vision-Language Models via Professional Doubles Badminton
Jintao Cheng and Weibin Li
Visual
Language Models (VLMs) excel at describing visible scene content but struggle to reason about dynamic multi-
agent interactions, where action semantics depend on coordinated roles and spatial-temporal dependencies. We formalize this capability as multi-
agent tactical
reasoning and introduce DoublesEval, a diagnostic evaluation framework that leverages professional doubles badminton as a structurally tractable testbed. DoublesEval employs a key-moment-based protocol that decomposes rallies into tactically salient instants and probes models across four interpretable dimensions: atomic recognition, intra-segment composite understanding, cross-segment causal
reasoning, and high-level tactical abstraction. This design isolates where
reasoning fails, rather than merely measuring answer correctness. To address observed failure modes, we propose TacticCheck, a lightweight constraint-guided test-time consistency checker that reranks candidate answers using the model's own lower-level tactical predictions, requiring no parameter updates or ground-truth labels at inference time. Evaluating four representative open-source VLMs on 60 curated rallies (yielding $\sim$9.6K structured instances) via a zero-shot protocol, we find that models remain weak across all diagnostic levels, with especially clear bottlenecks in spatial state, interaction binding, and terminal evidence. TacticCheck delivers consistent gains across all evaluated models, while still leaving a substantial gap to robust tactical
reasoning. These results highlight the need for structured, interaction-aware evaluation paradigms for next-generation VLMs. The source code is available in \href{https://github.com/Chengjt1999/DoublesEval}{\textcolor{blue}{our GitHub repository}}.
Read the full abstract ▾
Computer Vision
Vision Language Model Fusion for Explainable Face Recognition
Ana Estrada-Real, Lydia Alapatt, Christoph Busch et al. (+1 more)
Responsible deployment of face verification systems requires more than accurate decisions: systems should also provide interpretable and auditable evidence that enables users to understand, assess, and challenge their decisions.
Vision-
language models (VLMs) provide a promising foundation for explainable face recognition by combining visual analysis with natural-
language reasoning. However, relying on a single model may further limit the decision accuracy as well as provided explanations. This work therefore investigates whether multiple VLMs can be combined to improve recognition accuracy, and to enrich the explanations associated with those decisions. This work evaluates four VLMs as standalone face verification systems and subsequently proposes a fusion framework, where two source models provide similarity scores and textual justifications and a third VLM acts as a decider model. Four different fusion scenarios are considered, progressively providing the decider model with scores, justifications, face images, and combinations of these modalities. Overall, the findings suggest that the value of multi-VLM fusion extends beyond recognition performance. VLMs can provide complementary justifications and perspectives that enable richer explanations of face recognition decisions, supporting greater transparency, auditability, and error analysis. This is relevant to the development of responsible explainable face verification systems, where users and operators should be able to understand not only the final decision but also the evidence and potential sources underlying it. The proposed
multimodal VLM, which combines decision scores, explanations, and face images, achieves higher recognition accuracy than state-of-the-art VLMs and domain-specific face recognition models, while also providing fused explanations that are expected to be more robust than those generated by individual VLMs.
Read the full abstract ▾
All authors Ana Estrada-Real, Lydia Alapatt, Christoph Busch, Christian Rathgeb
Computer Vision · Cross-listed from cs.LG
Joint Distribution Alignment for Universal Domain Adaptation
Shizhe Li, Hongshan Pu, Mengying Xie et al. (+2 more)
Unsupervised domain adaptation (UDA) has been widely concerned in the fields of machine learning, pattern recognition, and computer
vision. Traditional UDA learning usually assumes that the label spaces of the source and target domains are exactly the same and only needs to solve the problem of sample distribution drift existing between two domains. However, in real world applications, the label spaces between two domains may be different. In this case, there are both sample distribution drift and class spatial difference between domains, namely Universal Domain Adaptation (UniDA) learning scenario. At present, existing works rarely offer theoretical analysis for universal domain adaptation. In this paper, we provide an upper bound of the generalization error for universal domain adaptation. According to the proposed generalization error bound, we propose a novel UniDA
algorithm called Joint Distribution
Alignment for Universal Domain Adaptation (JAUA), which aligns the joint distributions by minimizing the distribution discrepancy calculated by Chi-Square divergence. Furthermore, we propose a progressive pseudo-labeling method to assign the pseudo labels to unlabeled target samples. The experiment results on six public image
datasets demonstrate the superiority of JAUA in handling the UniDA problem.
Read the full abstract ▾
All authors Shizhe Li, Hongshan Pu, Mengying Xie, Yi Xiang, Xiaowei Yang
Computer Vision
ZODIAC: Zero-shot Octree-based Diffusion for Anatomical Completion
Miruna-Alexandra Gafencu, Vlad Bratulescu, Yordanka Velikova et al. (+2 more)
Recovering the full 3D spine anatomy from intraoperative ultrasound is an ill-posed inverse problem, as the complete structure must be inferred from incomplete and noisy observations. Acoustic occlusions and limited field of view create large unobserved regions, while view-dependent artifacts lead to variability in expert annotations of the visible anatomy. Current supervised ultrasound shape completion methods rely on synthetically generated incomplete-complete paired data to learn conditional mappings under a predefined distribution of simulated occlusions. However, real intraoperative occlusions do not necessarily follow this distribution, which can limit generalization to patient data. As a result, accurate and robust completion from noisy partial observations remains an unsolved problem. We propose a zero-shot shape completion framework that reconstructs the entire lumbar spine from partial ultrasound observations without relying on simulated training data. To accommodate unseen and irregular patterns of missing structures, we introduce blended completion, a mechanism that integrates the learned anatomical prior with incoming partial geometry at inference time. The method learns a generative
diffusion prior over full anatomical shapes represented in an adaptive octree structure, enabling efficient modeling of the complete spine in a single forward pass. Validation on phantom and volunteer data shows that decoupling completion from a predefined corruption distribution improves generalisation under real occlusions, outperforming a fully supervised variant by 22% on HD95 completion error. Code and data are available at https://github.com/miruna20/ZODIAC.
Read the full abstract ▾
All authors Miruna-Alexandra Gafencu, Vlad Bratulescu, Yordanka Velikova, Mohammad Farid Azampour, Nassir Navab
Computer Vision
MRI-based Deep Radiomic Phenotyping of Neuromuscular Disorders: A Topology-driven Characterization
Martyna Żur, Łukasz Piórecki, Marek Socha et al. (+11 more)
Quantitative assessment of muscle MRI is crucial for monitoring neuromuscular disorders (NMD). This study introduces an automated radiomic phenotyping framework based on original features engineered across five main architectural domains: quantitative morphometry, spatial distribution,
geometric shape, interactions between progressive fat replacement stages, and graph-based
topology. Utilizing 1184 MRI scans from the CoMPaSS-NMD project, we map the complex 3D architecture of heterogeneous intramuscular lipodegeneration into objective, morphologically interpretable biomarkers. We introduce a graph-based skeletonization of fat infiltrates to quantify muscle architectural changes, establishing a multi-dimensional extension of traditional, spatially-agnostic volume metrics by mapping topological networks across the entire 3D muscle volume. Statistical screening via non-parametric Kruskal-Wallis analysis confirmed the discriminative power of these novel descriptors across the genetic hierarchy. Notably, topological network metrics (e.g., SF1_Skel_Nodes, $\epsilon^2$ = 0.2656) and interface dynamics metrics (e.g., SF2_To_SF1_Dist_Min, $\epsilon^2$ = 0.2092) demonstrated substantial effect sizes, providing deeper structural insights than classical volumetric assessments. Post-hoc pairwise evaluations and UMAP projections further indicated the capability of these topological and 3D
geometric invariants to capture disease-specific macroscopic infiltration patterns. These results demonstrate that global architectural features represent a highly promising class of biomarkers for differential diagnosis, offering new avenues for tracking longitudinal disease dynamics in neuromuscular diagnostics. The developed automated feature extraction pipeline is integrated and available within the MUSCAT (MUSCle fAt
Topology) library.
Read the full abstract ▾
All authors Martyna Żur, Łukasz Piórecki, Marek Socha, Jose Verdu Diaz, Ana Topf, Michael Obach, Marco Savarese, Jocelyn Laporte, Benedikt Schoeser, Filippo Santorelli, Jordi Diaz-Manera, Volker Straub, Rossella Tupler, Joanna Polańska
Computer Vision
Markerless Pose Estimation for Resistance Training Technique Assessment
Joseph Turner, Jeff Clark and Nawid Keshtmand
Resistance training can be a high risk activity, and safe form is essential to avoiding injury. Laboratory-based movement analysis provides quantitive technique assessment, yet is not easily accessible. Markerless pose estimation infers body landmarks from images or
video without physical markers and could offer a feasible alternative for technique assessment. We present a pose estimation framework to evaluate resistance-training technique from ordinary
video footage. Using BlazePose, anatomical landmarks were extracted from squat, bench press, and deadlift
videos and converted into joint-angle trajectories, with the squat serving as the primary case study. Trajectories were assessed against a defined reference repetition using root mean square error (RMSE). Results show that the framework recovers meaningful kinematic patterns for the squat and deadlift, enabling quantitative comparison between repetitions and identification of technique variability within a set. Performance depended strongly on camera orientation and visual occlusion, with non-sagittal views distorting 2D joint-angle estimates. The findings demonstrate that markerless pose estimation can support accessible biomechanical assessment outside laboratory environments.
Read the full abstract ▾
Computer Vision
Bridging Adversarial and Collaborative Learning for AI-Generated Image Quality Assessment
Baoliang Chen, Qing Lin and Sijie Mai
AI-generated image quality assessment (AIGIQA) requires jointly
reasoning about perceptual fidelity and
prompt alignment, two quality dimensions that are often treated as independent in existing AIGIQA models. However, by re-examining human ratings, we uncover a previously overlooked phenomenon: the two dimensions are interdependent and exhibit both competitive and cooperative interactions during human rating. This observation suggests that a unified model should neither collapse the two dimensions nor rigidly separate them, but rather adaptively negotiate their interplay. Motivated by this insight, we introduce an interaction-aware learning framework that models perception-
alignment relations through
adversarial and collaborative inference pathways. Instead of designing a rigid dual-branch architecture, our method employs a gated interaction module that dynamically routes features according to the inferred relationship between the two dimensions. Task-aware
prompts further modulate the gating behaviour, enabling the model to switch between competition and cooperation when necessary. Experiments across multiple AIGIQA
benchmarks demonstrate that our approach not only achieves state-of-the-art accuracy but also yields interpretable interaction patterns, offering a more faithful approximation of human judgment. The codes are available at https://github.com/LQAMEI/ACL-IQA.
Read the full abstract ▾
Computer Vision
Variance-Guided Spatial Attention Fusion for Robust End-to-End Driving under Asymmetric Sensor Degradation
Weizhi Tao, Zengwang Jin, Xiao Wang et al. (+1 more)
End-to-end
multimodal driving has progressed rapidly by fusing camera and
LiDAR streams. Existing pipelines remain fragile under asymmetric sensor degradation, where either an entire modality or only a localized region is corrupted while other regions remain useful. The key difficulty is not simply to add an uncertainty head, but to obtain dense reliability supervision, calibrate this reliability against physical fault severity, and use it before unreliable features bias the planner. We propose Variance-Guided Spatial
Attention Fusion (VG-SAF), in which dense heteroscedastic reliability estimates act as interpretable spatial gates. The framework couples three components. First, a physically grounded augmentor simulates representative camera and
LiDAR failures and emits a continuous spatial mask, providing dense supervision without additional annotation. Second, modality-specific experts predict per-pixel reliability scales through cross-branch dense
distillation in log space, enforcing a monotone severity-to-scale response. Third, calibrated reliability maps drive a hybrid
attention mechanism that suppresses unreliable
cells with a local spatial gate and arbitrates between modalities through a cross-modal trust softmax. A Laplace uncertainty head emits a systemic waypoint uncertainty scale that signals severe or combined sensor degradation, including severities outside the training ranges. On the CARLA Longest6
benchmark, VG-SAF consistently improves closed-loop robustness over the baselines across camera-only,
LiDAR-only, and joint degradation regimes, as measured by driving score, route completion, and infraction score.
Read the full abstract ▾
All authors Weizhi Tao, Zengwang Jin, Xiao Wang, Hailong Huang
Computer Vision
MaST: Motion-aware Sparse Pipeline for Lightweight Object Tracking
Qingmao Wei, Fagui Liu, Dengke Zhang et al. (+2 more)
Transformer-based object trackers are renowned for their strong performance, yet dense
token processing often leads to prohibitive computational cost, limiting real-time deployment on edge devices. While recent works explore
token pruning to reduce computation, they often stop short of an end-to-end sparse pipeline, as early-layer
token scores can be noisy without a motion prior, and many trackers ultimately fall back to dense reshaping to feed the dense prediction head that partially negates the savings. We introduce Motion-aware Sparse Tracker (MaST), a sparse tracking framework that makes sparsity effective from
tokens to boxes. First, MaST injects a lightweight motion prior to refine cross-
attention-based importance scores, enabling earlier and more stable
token reduction in the search region. Second, we introduce a natively sparse prediction head that operates directly on the retained unstructured
tokens with a score-first, regress-once design, eliminating dense padding/reshaping and reducing redundant computation. Extensive experiments on multiple
benchmarks demonstrate that MaST establishes new state of the art among lightweight trackers, where MaST-tiny attains 63.8 AUC on LaSOT and 80.1 SUC on TrackingNet, surpassing the prior best AsymTrack-S by +1.0 AUC and +2.2 SUC while running at 152 FPS on Jetson Nano, nearly twice as fast as AsymTrack-S at 88 FPS. Code is available at https://github.com/TsingWei/MaST.
Read the full abstract ▾
All authors Qingmao Wei, Fagui Liu, Dengke Zhang, Qingze He, Quan Tang
Computer Vision
B-MIM: Biased Masked Image Modeling for Generalizable Segmentation of Fine-Grained Anatomical Structures
Sebastián González, Karen Sanchez, José M. Saavedra et al. (+2 more)
Self-supervised pretraining enables transferable representations for medical imaging, yet most CT encoders remain biased toward coarse semantic understanding, limiting their sensitivity to fine-grained anatomical structures such as vessels or small tumors. In this paper, we introduce Biased Masked Image Modeling (B-MIM), a modification of the iBOT objective that stochastically reduces global semantic
alignment to prioritize local patch reconstruction. This bias encourages the encoder to capture high-frequency morphological details and structural continuity. We curate a multi-institutional CT abdominal
dataset of 9,955 filtered studies from 17 public sources and pretrain a 3D Swin
Transformer backbone using B-MIM. Across inter-
dataset experiments on liver vessel
segmentation, the proposed encoder improves topological fidelity (clDice) and achieves competitive Dice scores in tumor
segmentation, compared to fully fine-tuned baselines, despite updating only a fraction of the parameters. Our results suggest that reducing global semantic pressure during pretraining enhances generalization to intricate anatomical structures.
Read the full abstract ▾
All authors Sebastián González, Karen Sanchez, José M. Saavedra, Marcelo Pizarro, Bernard Ghanem
Computer Vision
Metadata-Aware Adaptation of a Generative Foundation Model for Conditional CMR Synthesis
Marc Rodríguez, Grzegorz Skorupko, Nay Aung et al. (+3 more)
Synthetic image generation is a promising strategy to address data scarcity and the underrepresentation of clinically important phenotypes in medical imaging, yet generating images that faithfully reflect meaningful patient characteristics remains challenging. In this work, we investigate metadata-conditioned cardiac
magnetic resonance (CMR) synthesis using a pretrained latent
diffusion model, encoding structured clinical metadata and slice position as textual
prompts to guide CMR generation. To improve metadata adherence and address the imbalance of clinical attributes, we integrate three strategies: Metadata-Free Classifier-Free Guidance (CFG), Contrastive Batching, and Inverse-Frequency Sampling. The framework was fine-tuned and evaluated on 59,058 short-axis CMR from the UK Biobank using paired image similarity, distributional fidelity, and subgroup-level analyses. The combined approach achieved a Fréchet Inception Distance (FID) of 37.47, improving by 57.04% over the same model fine-tuned without these strategies and by 28.68% over a previous text-conditioned CMR
diffusion baseline requiring cardiac geometry as additional input, while relying solely on patient metadata. This distributional gain, driven mainly by Metadata-Free CFG, came with a modest reduction in paired similarity, suggesting that the model prioritizes population-level realism over exact image reproduction. Subgroup analyses demonstrated improved
alignment across demographic and acquisition-related metadata, with disease-specific conditioning being the most challenging task. These findings demonstrate the potential of generative foundation models for clinically meaningful CMR synthesis while highlighting the need for more effective metadata-aware conditioning strategies. Our code is available at https://github.com/rodriguezmarc/conditional-cmr.
Read the full abstract ▾
All authors Marc Rodríguez, Grzegorz Skorupko, Nay Aung, Steffen E Petersen, Karim Lekadir, Polyxeni Gkontra
Computer Vision
Mind the Student: Behavioral and Contextual Cues for Automated Engagement Prediction in Online Learning
Alperen Kantarci, Visvanathan Ramesh and Gemma Roig
The prediction of student engagement from the online tutoring
videos is difficult because engagement is a multidimensional construct comprising distinct behavioral, emotional, and cognitive states. A reliable prediction requires bringing together different types of behavioral signals as well as expressive cues. Through our analysis of the CASED
dataset, it is clear that engagement prediction gets even harder due to the high inter-person variability as well as the subjectivity of the engagement annotation. To tackle these challenges, we develop a
multimodal framework that integrates the implicit spatiotemporal features extracted from pretrained
video, audio, and image encoders along with structured behavioral modalities like head pose, gaze, facial action units,
emotion, and wavelet-based audio features. We integrate these modalities via a Perceiver IO latent bottleneck. Moreover, student and instructor personalities are modeled as variational posteriors over learnable
embeddings to enable partial pooling across participants. We employ evidential regression and spectral-normalized Gaussian process classification heads for uncertainty-aware prediction to further improve robustness and calibration.
Benchmark on the CASED challenge test set shows that all participating methods converge near random-chance performance, revealing the difficulty of the
dataset. In this highly ambiguous regime, our framework achieves competitive performance while uniquely offering well-calibrated uncertainty metrics, demonstrating that reliable risk-quantification is an essential prerequisite for deploying engagement models in real-world educational tools.
Read the full abstract ▾
Computer Vision
SeMoCo: A Semantic-First Motion Codec for Motion Language Modeling
Tianlv Huang, Hetian Guo, Ziyi Cai et al. (+6 more)
Discrete motion representations have substantially advanced autoregressive text-to-motion generation. However, most motion tokenizers are optimized for reconstruction and do not explicitly allocate capacity according to semantic role. Action-level meaning and fine-grained kinematic detail must therefore be encoded through the same reconstruction-driven hierarchy. We introduce SeMoCo, a semantic-first motion codec, together with a dual-axis motion generator for
language-conditioned motion generation. Each motion
token contains one semantic
token and a residual sequence of kinematic
tokens. The generator models semantic progression across time and autoregressively refines the residual entries. We also construct $\Omega$-MotionVerse, a large-scale, multi-source human-motion
dataset unified under the SOMA representation. Across the reported comparisons, SeMoCo achieves the best reconstruction accuracy among the compared codecs, while strong text-to-motion results demonstrate the effectiveness of its motion
tokens for downstream generation.
Read the full abstract ▾
All authors Tianlv Huang, Hetian Guo, Ziyi Cai, Song Wang, Yanping Zhang, Zipei Fan, Xuan Song, Guangming Wu, Xin Zheng
Computer Vision
Keep-or-Drop? Adaptive Tokenizer for Compact Video Representation
Yeonkyeong Lee, Hyunsung Go, Jongmin Kim et al. (+2 more)
Latent
diffusion models have emerged as a dominant framework for high-fidelity image and
video synthesis, operating in compact latent spaces with variational autoencoders (VAEs) to enhance computational efficiency without compromising visual quality. However, conventional VAEs are suboptimal for
video data as they employ fixed compression ratios that cannot adapt to the varying complexity of spatio-temporal content. We present KATok (Keep-or-Drop? Adaptive Tokenizer for Compact
Video Representation), a
transformer-based VAE that incorporates an adaptive
token selector which is jointly learned with latent
tokens. By evaluating each
token's content-richness as keep-or-drop probability, the
token selector effectively discards uninformative
tokens, naturally allowing data-dependent compression. Applying adaptive tokenization to
diffusion models may cause spatial misalignment, as
token dropping can disturb the original spatio-temporal structure. To alleviate this issue, we propose two position-prediction strategies: cascaded and joint generation, to ensure spatial consistency. We empirically show that our model achieves strong reconstruction and generation quality at a state-of-the-art compression ratio. Further analysis on
video data reveals that this improvement is primarily achieved by reducing spatio-temporal redundancy and removing uninformative
tokens, as supported by both quantitative and qualitative results.
Read the full abstract ▾
All authors Yeonkyeong Lee, Hyunsung Go, Jongmin Kim, Sewoong Lim, Donghoon Lee
Computer Vision
CARE: Camera-Residual Reserves for First Sightings in Adaptive LiDAR Sensing
Jiachen Gong, Yun Li, Ehsan Javanmardi et al. (+2 more)
Adaptive
LiDAR scanning concentrates a limited sensing budget on regions of interest predicted from past object tracks, lowering data volume in autonomous driving while maintaining detection accuracy. However, existing scanning policies face three challenges. First, history-driven approaches depend on past tracks, so unseen objects are detected late or missed. Second, random or uniform sampling outside the predicted regions has no awareness of where new objects appear. Third, camera-guided alternatives spend budget on all camera detections, resampling objects already covered, costing recall in crowded scenes and range when budgets are scarce. This paper introduces the CAmera-REsidual reserve (CARE), a training-free allocation rule that reserves part of a fixed ray budget for the directions of current camera detections that the track forecasts cannot explain; the rest follows the base history policy, and unused reserve returns to a random floor. The paper makes three contributions. First, a leakage-free ray-budget evaluation on nuScenes (150 scenes, 4,148 events) measuring the first-sighting loss of history-driven scanning, with a strict-causal variant using the preceding keyframe. Second, CARE raises first-sighting recall by 5.2, 5.2, and 4.3 points at 10%, 20%, and 35% budgets over the history policy, with paired intervals excluding zero; the camera cue drives this gain, and the first-sighting versus overall trade-off is a budget-dependent Pareto choice. Third, a safety-bounded forgetting module that releases budget from receding or static tracks beyond a speed-dependent guard distance; at tight budgets, forgetting without the guard significantly harms near-field recall, so the guard is what keeps it safe. The pipeline runs end to end on a real vehicle and, in closed-loop simulation, detects an occluded pedestrian earlier and brakes more reliably than history-driven scanning.
Read the full abstract ▾
All authors Jiachen Gong, Yun Li, Ehsan Javanmardi, Wencan Mao, Manabu Tsukada
Computer Vision
Example-based Robust Abnormality Detection with Minimal Annotations using Exemplar Med-DETR
Sheethal Bhat, Bogdan Georgescu, Awais Mansoor et al. (+5 more)
Reducing annotation requirements remains a key challenge in developing robust medical object detectors. To address this,
Vision-
Language (VL) object detection methods leverage grounding text information to enable powerful zero-shot and few-shot object detectors in the natural image domain [1, 2, 3, 4]. However, transferring these methods to the medical domain is challenging due to the absence of comparable quality and quantity of the grounding data. Regardless, significant contextual and non-imaging information exists in medical images that remains underutilized. Few-shot learning (FSL) techniques partially address this limitation but struggle to general ize to unseen medical findings and require extensive retraining when new findings are introduced [5, 6]. To overcome these challenges, we extend our prior EM-DETR framework [7] and introduce a scalable FS detection approach designed for efficient abnormality detection in Chest X-Ray (CXR) images under minimal supervision. The proposed architecture incorporates exemplar-based feature generation and domain-aware contrastive optimization, enabling effective adaptation to novel disease findings without exhaustive retraining. Our method achieves near state-of-the-art (SOTA) detection performance using less than 10% of the annotated data, demonstrating its potential for practical, annotation-efficient clinical deployment across both proprietary and public CXR
datasets.
Read the full abstract ▾
All authors Sheethal Bhat, Bogdan Georgescu, Awais Mansoor, Mathias Zinnen, Pranjal Sahu, Florin C. Ghesu, Sasa Grbic, Andreas Maier
Computer Vision · Cross-listed from cs.AI
Real-World Knowledge-Guided Change Data Synthesis for Remote Sensing
Yaoyi Qi, Xingxing Weng, Chao Pang et al. (+5 more)
Change data synthesis provides a cost-effective solution for expanding training data and improving the performance of change detection models. However, existing synthesis methods typically rely on handcrafted rules to simulate changes, where limited coverage of class transitions restricts the diversity of synthesized data, while predefined transition designs limit their flexibility in accommodating varied change types. In this work, we introduce KnowChange, a knowledge-guided change data synthesis framework that leverages pretrained
vision-
language models as knowledge sources to reason about plausible change locations and class transitions from pre-change scenes and desired change types. By integrating knowledge-guided change simulation with generalizable synthesis models, KnowChange enables flexible synthesis of diverse change types within a unified framework. Extensive experiments demonstrate that KnowChange-generated data consistently outperforms existing synthetic
datasets in both synthetic-to-real transfer and synthetic data augmentation, despite being generated at a compact scale. Further analyses show that the knowledge-guided change simulation can be seamlessly integrated into existing synthesis pipelines and enhance the downstream utility of synthesized data.
Read the full abstract ▾
All authors Yaoyi Qi, Xingxing Weng, Chao Pang, Yongkang Cui, Xiangyu Hao, Xiaokang Zhang, Guibo Zhu, Gui-Song Xia
Computer Vision
Event-Based Motion Estimation via Oriented Distance Fields
Lei Sun, Yuqin Ma, Weilun Li et al. (+5 more)
Event-based motion estimation is central to tasks that demand high temporal resolution and robustness to fast motion. Existing methods typically rely on iterative optimization or repeated hypothesis comparison, offsetting the sensor's low-latency advantage. We propose Oriented Distance Field Motion Estimation (ODF Motion Estimation), which replaces this optimization with a single averaging step over a precomputed field of event distance vectors, combined with an adaptive event-count selection strategy and a parameter-free trail filter. On public and self-collected
datasets, ODF motion estimation reaches sub-pixel accuracy at the lowest latency among compared methods. We validate its generality on two downstream applications rather than treating them as separate contributions. First, the estimated trajectory is converted into a blur kernel and paired with a compact iterative-unfolding network, trained on simulated motion-estimation noise, for real-time non-blind image deblurring, attaining competitive or superior PSNR/SSIM with under 1M parameters. Second, the same precomputed field is repurposed for directional event filtering in a low-power asynchronous pupil and glint tracker, sustaining stable tracking for tens of seconds while lowering a near-eye module's power draw.
Read the full abstract ▾
All authors Lei Sun, Yuqin Ma, Weilun Li, Haoran Liang, Runyi Yang, Kaiwei Wang, Danda Pani Paudel, Luc Van Gool
Computer Vision
Beauty is in the ELBO of the Beholder: A Variational Account of Processing Fluency in Face Perception
Francisco M. López and Jochen Triesch
Facial attractiveness has been linked to statistical regularities such as
symmetry and averageness, suggesting that beauty may depend on the ease with which a face is perceived. We empirically test this hypothesis by training variational autoencoders on four face
datasets without attractiveness supervision and evaluating their representations on the 597 faces from the Chicago Face Database. We report three main novel findings. First, we identify that human attractiveness ratings closely aligns with the direction defined by the VAE evidence lower bound (ELBO) in rate-distortion space. Second, we find that independently learned latent spaces contain an attractiveness direction that transfers strongly across random initializations and training data. Finally, we observe that attractive faces are more prototypical in both shape and latent space. Our work provides the first explicit connection between classic accounts of aesthetics with learned generative models and empirical support for a variational interpretation of the processing fluency theory of aesthetic pleasure.
Read the full abstract ▾
Computer Vision
NeoWorld-Pro: Programming Interactive Scenes from Monocular Images for Embodied Simulation
Yumeng He, Eachen Soong, Xiaotian Yang et al. (+5 more)
The advancement of
Embodied AI necessitates high-quality simulation assets that faithfully mirror the real world. However, transforming raw visual observations into simulation-ready scenes remains challenging due to the lack of physical grounding and scene-level interactivity in current image-to-URDF methods. We propose NeoWorld-Pro, a framework that reformulates
monocular scene reconstruction as procedural programming for interactive 3D environments. Leveraging the zero-shot
reasoning and code synthesis capabilities of MLLMs, NeoWorld-Pro converts a single RGB image into executable programs specifying object geometry, articulation, and physical properties. A physics-in-the-loop mechanism then iteratively refines the generated programs by validating their execution in a physics engine, enforcing physically plausible articulations, valid object compositions and interactions, and accurate spatial relationships. Experiments show that NeoWorld-Pro outperforms open-loop and prior
monocular reconstruction methods, while enabling complex downstream tasks such as stable stacking and fine-grained manipulation.
Read the full abstract ▾
All authors Yumeng He, Eachen Soong, Xiaotian Yang, Weijia Zhang, Zanwei Zhou, Junru Gong, Xiaokang Yang, Yunbo Wang
Computer Vision
Amortized Set Prediction for Inverse IFS Reconstruction from Density Maps
Yutaka Yamaguti
Iterated Function Systems (IFS) generate self-similar fractals from a few contractive affine maps. The forward map from parameters to images is computationally inexpensive and well understood, whereas the inverse problem of estimating maps from an image is difficult and is typically handled by per-image optimization. We replace this loop with a single forward pass of a learned
estimator that predicts the affine-map set directly from a visit-frequency density map, thereby amortizing the inverse problem. The design follows two constraints. First, density maps do not uniquely identify IFS parameters, so evaluation is based on reconstruction rather than parameter recovery; unordered map sets are handled by Hungarian matching, and ground-truth parameters provide a stable training surrogate. Second, the fully known forward model lets us generate exact synthetic training pairs and also supports image-only test-time refinement. On in-distribution tests, amortized initialization plus a few refinement steps lies on a better quality–speed frontier than equal-budget random-initialized per-image optimization, and a 30-step refinement (about $0.56$ s per sample) remains better than a doubled-budget baseline. Extending optimization to 1000 steps shows that the benefit is not only speed: amortized initialization reaches high-quality reconstructions more frequently than random starts. On real images (MNIST and Fashion-MNIST), it improves density metrics on average over a published per-image optimizer while being roughly 12 to 2600 times faster.
Read the full abstract ▾
Computer Vision
SandwichQuant: Which Parameters Matter Before and After Quantization?
Peng Xia and Junbiao Pang
Quantization correction methods usually optimize weights, quantization parameters, or reconstruction objectives, while the underlying parameter subspaces responsible for effective correction remain unclear. In this work, we study quantization correction from a parameter subspace perspective and reveal that correction capability is highly non-uniform across parameter groups. By decomposing trainable parameters into backbone weights, normalization-affine parameters, and quantization parameters, we show that the low-dimensional normalization-affine subspace provides a highly efficient correction direction under matched budgets. Based on this finding, we propose SandwichQuant, a two-stage normalization-affine correction framework that performs adaptation before and after quantization. The pre-stage improves quantization robustness, while the post-stage compensates residual errors after the quantized graph is fixed. Extensive experiments on
vision models and large
language models demonstrate consistent improvements under various low-bit quantization settings, validating the effectiveness of subspace-aligned correction.
Read the full abstract ▾
Computer Vision
ViSculpt: Visual-Centric Agentic Geometry Editing
Bo Pang, Jiaqi Pan, Xiaocheng Zhang et al. (+3 more)
3D geometry editing is a critical yet labor-intensive part of the graphics pipeline, requiring artists to translate creative intent into precise operations in complex professional software. Large
language models (LLMs) have shown promise for script-based 3D creation, but script generation is less suited to perception-driven editing of arbitrary existing meshes, where execution must remain visually grounded and untouched regions should be preserved. We present a visual-centric, training-free multi-
agent system that edits existing 3D meshes directly in Blender by emulating the iterative workflow of human artists. Rather than generating scripts or regenerating geometry, our system operates through the Blender GUI:
multimodal LLM
agents observe the viewport, reason about the current mesh state, and execute localized edits through simulated user interactions. Experiments on a curated
benchmark provide initial evidence that this agentic approach can follow natural
language instructions, perform representative localized mesh edits, and preserve the overall identity of the input asset. Our results highlight a complementary regime for
language-driven 3D editing: direct in-place modification of existing meshes within the native 3D editing workflow. We view this work as an exploratory step toward visual-centric agentic geometry editing in professional graphics software.
Read the full abstract ▾
All authors Bo Pang, Jiaqi Pan, Xiaocheng Zhang, Jiacheng Xu, Guoping Wang, Peng-Shuai Wang
Computer Vision · Cross-listed from astro-ph.GA
Decoupling candidate dual AGN from chance superpositions in the GOTHIC survey via a deep-learning framework
Bhavesh Mukheja, Snehanshu Saha, Anwesh Bhattacharya et al. (+3 more)
Dual active
galactic nuclei (DAGN) mark a critical phase in the
evolution of merging
galaxies and the pairing of supermassive black holes, yet they remain difficult to identify in large imaging surveys because of projection effects and limited spatial resolution. Compact foreground
stars and unresolved substructure can mimic dual
nuclei through chance superposition, complicating automated detection. We revisit the 46,061
galaxies flagged but rejected as DAGN candidates by the GOTHIC pipeline, primarily because the two
nuclei fell within the SDSS fibre aperture or exceeded its separation threshold. We train a supervised deep-learning framework based on the YOLOv11 oriented-bounding-box architecture on annotated SDSS imaging to separate genuine dual
nuclei from foreground
stellar contaminants and other spurious
alignments. The final model attains a validation precision of 0.919, recall of 0.905, and $F_1$ of 0.912, and yields 29,605 dual-
nucleus candidates after removing
star-dominated and blended detections. Structured visual inspection indicates that $54.5$–$62\%$ are consistent with genuine dual
nuclei, implying $\sim(1.4$–$1.8)\times10^{4}$ plausible systems. Cross-calibrating the YOLO separation against the deterministic GOTHIC centroid measurement and restricting to the compact regime ($d \le 6.87''$) gives a conservative subset of $\sim 13{,}672$ candidates, reaching calibrated separations of $\sim 0.56''$. Spectroscopy of the most compact ($\le 1$~kpc) systems shows spectra dominated by
stellar absorption, with no resolved double-peaked emission, so confirmation requires higher-resolution follow-up. The catalogue is a statistically refined list of candidates, not confirmed DAGN. Nonetheless, deep-learning detection substantially reduces contamination and expands the plausible DAGN census.
Read the full abstract ▾
All authors Bhavesh Mukheja, Snehanshu Saha, Anwesh Bhattacharya, Mousumi Das, Françoise Combes, Sudhanshu Barway
Computer Vision
Rethinking Pre-Training and Augmentation for Zero-Shot Cross-City Object Detection
Long Hoang Pham, Quoc Pham-Nam Ho, Huy-Hung Nguyen et al. (+10 more)
Real-world deployment of traffic surveillance systems is bottlenecked by geographic domain shift, in which models trained in one city underperform when applied to an unseen target city. Conventional domain adaptation relies on hyperparameter-sensitive architectures or direct profiling of target data. Both are fundamentally precluded in
privacy-conscious ecosystems that require completely blind training and evaluation loops. In this setting, we explore the effects of pre-training and augmentation in addressing the domain shift problem. Specifically, we propose a new modular training pipeline for object detection structured around two core orthogonal pillars: (1) a multi-
dataset pre-training strategy featuring a class-agnostic objectness
distillation to decouple structural vehicle geometry from semantic taxonomies, and (2) a domain-resilient augmentation stream featuring a novel Grayworld transformation that forces global
attention heads to strip volatile chromatic shortcuts in favor of robust shape priors. When evaluated with the real-time
transformer-based detector RF-DETR, our framework bridges cross-city distribution gaps while using limited GPU
memory (16GB). Our optimized variants, RF-DETR-HR and RF-DETR-Grayworld, deliver a substantial empirical gain of +24.29 over the baseline, achieving 1st place (47.53 mAP) on the AI City Challenge Track 6 leaderboard. Code and data are available at: \href{https://github.com/SKKUAutoLab/aic26_cross_city}{SKKUAutoLab/aic26\_cross\_city}.
Read the full abstract ▾
All authors Long Hoang Pham, Quoc Pham-Nam Ho, Huy-Hung Nguyen, Duong Nguyen-Ngoc Tran, Ngoc Doan-Minh Huynh, Cu Quoc Le, Hoang-Khang Nguyen, Hyung-Min Jeon, Chi Dai Tran, Son Hong Phan, Duong Khac Vu, Trinh Le Ba Khanh, Jae Wook Jeon