Computer Vision
Exchange Is All You Need for Remote Sensing Change Detection
Sijun Dong, Siming Fu, Kaiyu Li et al. (+3 more)
Remote sensing change detection fundamentally relies on the effective fusion and discrimination of bi-temporal features. Prevailing paradigms typically utilize Siamese encoders bridged by explicit difference computation modules, such as subtraction or concatenation, to identify changes. In this work, we challenge this complexity with SEED (Siamese Encoder-Exchange-Decoder), a streamlined paradigm that replaces explicit differencing with parameter-free feature exchange. By sharing weights across both Siamese encoders and decoders, SEED effectively operates as a single parameter set model. Theoretically, we formalize feature exchange as an orthogonal permutation operator and prove that, under pixel consistency, this mechanism preserves mutual information and Bayes optimal risk, whereas common arithmetic fusion methods often introduce information loss. Extensive experiments across five
benchmarks, including SYSU-CD, LEVIR-CD, PX-CLCD, WaterCD, and CDD, and three backbones, namely SwinT, EfficientNet, and ResNet, demonstrate that SEED matches or surpasses state of the art methods despite its simplicity. Furthermore, we reveal that standard semantic
segmentation models can be transformed into competitive change detectors solely by inserting this exchange mechanism, referred to as SEG2CD. The proposed paradigm offers a robust, unified, and interpretable framework for change detection, demonstrating that simple feature exchange is sufficient for high performance information fusion. Code and full training and evaluation protocols will be released at https://github.com/dyzy41/open-rscd.
Read the full abstract ▾
All authors Sijun Dong, Siming Fu, Kaiyu Li, Xiangyong Cao, Xiaoliang Meng, Bo Du
Computer Vision
Vision-Language Model for Accurate Crater Detection
Patrick Bauer, Marius Schwinning, Florian Renk et al. (+2 more)
The European Space Agency (ESA), driven by its ambitions on planned lunar missions with the Argonaut lander, has a profound interest in reliable crater detection, since craters pose a risk to safe lunar landings. This task is usually addressed with automated crater detection
algorithms (CDA) based on deep learning techniques. It is non-trivial due to the vast amount of craters of various sizes and shapes, as well as challenging conditions such as varying illumination and rugged terrain. Therefore, we propose a deep-learning CDA based on the OWLv2 model, which is built on a
Vision Transformer, that has proven highly effective in various computer
vision tasks. For fine-tuning, we utilize a manually labeled
dataset fom the IMPACT project, that provides crater annotations on high-resolution Lunar Reconnaissance Orbiter Camera Calibrated Data Record images. We insert trainable parameters using a parameter-efficient fine-tuning strategy with Low-Rank Adaptation, and optimize a combined loss function consisting of Complete Intersection over Union (CIoU) for localization and a contrastive loss for classification. We achieve satisfactory visual results, along with a maximum recall of 92.6% and a maximum precision of 71.4% on a test
dataset from IMPACT. Our method achieves reliable crater detection across challenging lunar imaging conditions, paving the way for robust crater analysis in future lunar exploration.
Read the full abstract ▾
All authors Patrick Bauer, Marius Schwinning, Florian Renk, Andreas Weinmann, Hichem Snoussi
Computer Vision · Cross-listed from cs.MA
OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent
Bowen Yang, Kaiming Jin, Zhenyu Wu et al. (+12 more)
While
Vision-
Language Models (VLMs) have significantly advanced Computer-Using
Agents (CUAs), current frameworks struggle with robustness in long-horizon workflows and generalization in novel domains. These limitations stem from a lack of granular control over historical visual context curation and the absence of visual-aware tutorial
retrieval. To bridge these gaps, we introduce OS-Symphony, a holistic framework that comprises an Orchestrator coordinating two key innovations for robust automation: (1) a Reflection-
Memory Agent that utilizes milestone-driven long-term
memory to enable trajectory-level self-correction, effectively mitigating visual context loss in long-horizon tasks; (2) Versatile Tool
Agents featuring a
Multimodal Searcher that adopts a SeeAct paradigm to navigate a browser-based sandbox to synthesize live, visually aligned tutorials, thereby resolving fidelity issues in unseen scenarios. Experimental results demonstrate that OS-Symphony delivers substantial performance gains across varying model scales, establishing new state-of-the-art results on three online
benchmarks, notably achieving 65.84% on OSWorld.
Read the full abstract ▾
All authors Bowen Yang, Kaiming Jin, Zhenyu Wu, Zhaoyang Liu, Qiushi Sun, Zehao Li, JingJing Xie, Zhoumianze Liu, Fangzhi Xu, Kanzhi Cheng, Qingyun Li, Yian Wang, Yu Qiao, Zun Wang, Zichen Ding
Computer Vision
Self-transcendence: Is External Feature Guidance Indispensable for Accelerating Diffusion Transformer Training?
Lingchen Sun, Rongyuan Wu, Zhengqiang Zhang et al. (+4 more)
Recent works such as REPA have shown that guiding
diffusion models with external semantic features (e.g., DINO) can significantly accelerate the training of
diffusion transformers (DiTs). However, the use of pretrained external features as guidance signals introduces additional dependencies. We argue that DiTs actually have the power to guide the training of themselves, and propose SelfTranscendence, an effective method that achieves fast convergence using internal feature supervision only. The desired internal guidance features should meet two requirements: structurally clean to help shallow blocks separate noise from signal, and semantically discriminative to help shallow layers learn effective representations. With this consideration, we first align the DiT features with the clean VAE latent features, a native component of latent
diffusion, for a short training phase (e.g., 40 epochs) to improve their structural representations, then apply the classifier-free guidance to the intermediate features, enhancing their discriminative capability and semantic expressiveness. These enriched internal features, learned entirely within the model, are used as supervision signals to guide a new DiT training from scratch. Compared to existing self-contained methods, our approach achieves a significant performance boost. It can even surpass REPA, which uses the external DINO features as guidance, in both generation quality and convergence speed for both class-to-image and text-to-image generation tasks. The source code of our method can be found at https://github.com/csslc/Self-Transcendence.
Read the full abstract ▾
All authors Lingchen Sun, Rongyuan Wu, Zhengqiang Zhang, Ruibin Li, Yujing Sun, Shuaizheng Liu, Lei Zhang
Computer Vision
Video Evidence to Reasoning Efficient Video Understanding via Explicit Evidence Grounding
Yanxiang Huang, Guohua Gao and Zhaoyang Wei
Large
Vision-
Language Models (LVLMs) face a fundamental dilemma in
video reasoning: they are caught between the prohibitive computational costs of verbose
reasoning and the
hallucination risks of efficient, ungrounded approaches. To resolve this, we introduce the Chain of Evidence (CoE), a novel framework that architecturally decouples and co-optimizes perceptual grounding and
reasoning efficiency. CoE incorporates two core innovations: (1) A lightweight Evidence Grounding Module (EGM) that acts as a query-guided filter, dynamically identifying and extracting a compact set of high-fidelity visual evidence; and (2) An Evidence-Anchoring Protocol optimized via Reinforcement Learning. Crucially, we design a composite reward mechanism that enforces process
alignment, compelling the model to strictly reference identified temporal anchors during deduction, thereby mitigating
hallucinations. To enable this, we construct CoE-Instruct, a large-scale
dataset (164k samples) featuring a novel dual-annotation schema for separate perception and
reasoning supervision. Extensive experiments on five
benchmarks, including
Video-MME, MVBench, and VSI-Bench, demonstrate that CoE-enhanced models establish a new state-of-the-art. They significantly outperform existing methods in accuracy, proving CoE to be a powerful and practical paradigm for reliable
video understanding.
Read the full abstract ▾
Computer Vision
On the application of the Wasserstein metric to 2D curves classification
Agnieszka Kaliszewska and Monika Syga
In this work we analyse a number of variants of the Wasserstein distance which allow to focus the classification on the prescribed parts (fragments) of classified 2D curves. These variants are based on the use of a number of discrete probability measures which reflect the importance of given fragments of curves. The performance of this approach is tested through a series of experiments related to the clustering analysis of 2D curves performed on data coming from the field of archaeology.
Read the full abstract ▾
Computer Vision
Seeing vs. Believing: Evaluating the Language Bias of Open-Source MLLMs in Counter-Intuitive Scenes
Chen Ling, Tongwei Zhang, Hanqian Li et al. (+1 more)
Multimodal Large
Language Models (MLLMs) have demonstrated remarkable performance in mainstream visual understanding tasks, but their ability to process action scenes that contradict everyday common sense remains undertested. To address this gap, we introduce CAIT, a
benchmark comprising 400 high-fidelity synthetic scenes focused on counter-intuitive visual actions, such as “a rabbit is chasing a tiger”, where visual evidence explicitly contradicts common-sense expectations. We evaluate human, leading proprietary models (e.g., Claude and Gemini), and 14 representative open-source MLLMs. Humans achieve near-perfect performance (around 0.95 accuracy) and proprietary models demonstrate robust understanding (achieving up to 0.88 accuracy), standard open-source instruction-tuned models perform at the chance level. Further analysis demonstrates that this failure is driven by a strong
language prior: rather than trusting the visual input, they automatically override the anomalous visual signals with statistically common text descriptions. Although introducing Chain-of-Thought
reasoning mechanisms can improve accuracy, it significantly slows down the response and generates a new failure mode: models overthink the scenario and refuse to accept the actual visual content simply because it violates real-world physical laws. Finally, we demonstrate that targeted fine-tuning and structured prompting can effectively mitigate this reliance on
language priors, enabling open-source models to accurately ground their
reasoning in actual visual evidence.
Read the full abstract ▾
All authors Chen Ling, Tongwei Zhang, Hanqian Li, Nai Ding
Computer Vision
FMAC: a Fair Fiducial Marker Accuracy Comparison Software
Guillaume J. Laurent and Patrick Sandoz
This paper presents a method for carrying fair comparisons of the accuracy of pose estimation using fiducial markers. These comparisons rely on large sets of high-fidelity synthetic images enabling deep exploration of the 6 degrees of freedom. A low-discrepancy sampling of the space allows to check the correlations between each degree of freedom and the pose errors by plotting the 36 pairs of combinations. The images are rendered using a physically based ray tracing code that has been specifically developed to use the standard calibration coefficients of any camera directly. The software reproduces image distortions, defocus and diffraction blur. Furthermore, sub-pixel sampling is applied to sharp edges to enhance the fidelity of the rendered image. After introducing the rendering
algorithm and its experimental validation, the paper proposes a method for evaluating the pose accuracy. This method is applied to well-known markers, revealing their strengths and weaknesses for pose estimation. The code is open source and available on GitHub.
Read the full abstract ▾
Computer Vision
Hidden Monotonicity: Explaining Deep Neural Networks via their DC Decomposition
Jakob Paul Zimmermann and Georg Loho
It has been demonstrated in various contexts that monotonicity leads to better explainability in neural networks. However, not every function can be well approximated by a monotone neural network. We demonstrate that monotonicity can still be used in two ways to boost explainability. First, we use an adaptation of the decomposition of a trained ReLU network into two monotone and convex parts, thereby overcoming numerical obstacles from an inherent blowup of the weights in this procedure. Our proposed saliency methods - SplitCAM and SplitLRP - improve on state of the art results on both VGG16 and Resnet18 networks on ImageNet-S across all Quantus saliency metric categories. Second, we exhibit that training a model as the difference between two monotone neural networks results in a system with strong self-explainability properties.
Read the full abstract ▾
Computer Vision
Smooth Operator: Smooth Verifiable Reward Activates Spatial Reasoning Ability of Vision-Language Model
Siwen Jiao, Tianxiong Lv, Kangan Qian et al. (+7 more)
Vision-
Language Models (VLMs) face a critical bottleneck in achieving precise numerical prediction for 3D scene understanding. Traditional reinforcement learning (RL) approaches, primarily based on relative ranking, often suffer from severe reward sparsity and gradient instability, failing to effectively exploit the verifiable signals provided by 3D physical constraints. Notably, in standard GRPO frameworks, relative normalization causes "near-miss" samples (characterized by small but non-zero errors) to suffer from advantage collapse. This leads to a severe data utilization bottleneck where valuable boundary samples are discarded during optimization. To address this, we introduce the Smooth Numerical Reward Activation (SNRA) operator and the Absolute-Preserving GRPO (AP-GRPO) framework. SNRA employs a dynamically parameterized Sigmoid function to transform raw feedback into a dense, continuous reward continuum. Concurrently, AP-GRPO integrates absolute scalar gradients to mitigate the numerical information loss inherent in conventional relative-ranking mechanisms. By leveraging this approach, we constructed Numerical3D-50k, a
dataset comprising 50,000 verifiable 3D subtasks. Empirical results indicate that AP-GRPO achieves performance parity with large-scale supervised methods while maintaining higher data efficiency, effectively activating latent 3D
reasoning in VLMs without requiring architectural modifications.
Read the full abstract ▾
All authors Siwen Jiao, Tianxiong Lv, Kangan Qian, Chenxu Zhao, Xiuyuan Zhu, Tianlun Li, Xiaolong Cheng, Jinyu Li, Zhihao Liao, Yang Cai
Computer Vision
R3DPA: Leveraging 3D Representation Alignment and RGB Pretrained Priors for LiDAR Scene Generation
Nicolas Sereyjol-Garros, Ellington Kirby, Victor Besnier et al. (+1 more)
LiDAR scene synthesis is an emerging solution to scarcity in 3D data for robotic tasks such as autonomous driving. Recent approaches employ
diffusion or flow matching models to generate realistic scenes, but 3D data remains limited compared to RGB
datasets with millions of samples. We introduce R3DPA, the first
LiDAR scene generation method to unlock image-pretrained priors for
LiDAR point clouds, and leverage self-supervised 3D representations for state-of-the-art results. Specifically, we (i) align intermediate features of our generative model with self-supervised 3D features, which substantially improves generation quality; (ii) transfer knowledge from large-scale image-pretrained generative models to
LiDAR generation, mitigating limited
LiDAR datasets; and (iii) enable point cloud control at inference for object inpainting and scene mixing with solely an unconditional model. On the KITTI-360
benchmark R3DPA achieves state of the art performance. Code and pretrained models are available at https://github.com/valeoai/R3DPA.
Read the full abstract ▾
All authors Nicolas Sereyjol-Garros, Ellington Kirby, Victor Besnier, Nermin Samet
Computer Vision
Advancing Multinational License Plate Recognition Through Synthetic and Real Data Fusion: A Comprehensive Evaluation
Rayson Laroca, Valter Estevam, Gladston J. P. Moreira et al. (+2 more)
Automatic License Plate Recognition is a frequent research topic due to its wide-ranging practical applications. While recent studies use synthetic images to improve License Plate Recognition (LPR) results, there remain several limitations in these efforts. This work addresses these constraints by comprehensively exploring the integration of real and synthetic data to enhance LPR performance. We subject 16 Optical Character Recognition (OCR) models to a benchmarking process involving 12 public
datasets acquired from various regions. Several key findings emerge from our investigation. Primarily, the massive incorporation of synthetic data substantially boosts model performance in both intra- and cross-
dataset scenarios. We examine three distinct methodologies for generating synthetic data: template-based generation, character permutation, and utilizing a Generative
Adversarial Network (GAN) model, each contributing significantly to performance enhancement. The combined use of these methodologies demonstrates a notable synergistic effect, leading to end-to-end results that surpass those reached by state-of-the-art methods and established commercial systems. Our experiments also underscore the efficacy of synthetic data in mitigating challenges posed by limited training data, enabling remarkable results to be achieved even with small fractions of the original training data. Finally, we investigate the trade-off between accuracy and speed among different models, identifying those that strike the optimal balance in each intra-
dataset and cross-
dataset settings.
Read the full abstract ▾
All authors Rayson Laroca, Valter Estevam, Gladston J. P. Moreira, Rodrigo Minetto, David Menotti
Computer Vision
Variational Contrastive Learning for Skeleton-based Action Recognition
Dang Dinh Nguyen, Decky Aspandi Latif and Titus Zaharia
In recent years, self-supervised representation learning for skeleton-based action recognition has advanced with the development of contrastive learning methods. However, most of contrastive paradigms are inherently discriminative and often struggle to capture the variability and uncertainty intrinsic to human motion. To address this issue, we propose a variational contrastive learning framework that integrates probabilistic latent modeling with contrastive self-supervised learning. This formulation enables the learning of structured and semantically meaningful representations that generalize across different
datasets and supervision levels. Extensive experiments on three widely used skeleton-based action recognition
benchmarks show that our proposed method consistently outperforms existing approaches, particularly in low-label regimes. Moreover, qualitative analyses show that the features provided by our method are more relevant given the motion and sample characteristics, with more focus on important skeleton joints, when compared to the other methods.
Read the full abstract ▾
Computer Vision
StdGEN++: A Comprehensive System for Semantic-Decomposed 3D Character Generation
Yuze He, Yanning Zhou, Wang Zhao et al. (+4 more)
We present StdGEN++, a novel and comprehensive system for generating high-fidelity, semantically decomposed 3D characters from diverse inputs. Existing 3D generative methods often produce monolithic meshes that lack the structural flexibility required by industrial pipelines in gaming and animation. Addressing this gap, StdGEN++ is built upon a Dual-branch Semantic-aware Large Reconstruction Model (Dual-Branch S-LRM), which jointly reconstructs geometry, color, and per-component semantics in a feed-forward manner. To achieve production-level fidelity, we introduce a novel semantic surface extraction formalism compatible with hybrid implicit fields. This mechanism is accelerated by a coarse-to-fine proposal scheme, which significantly reduces
memory footprint and enables high-resolution mesh generation. Furthermore, we propose a
video-
diffusion-based texture decomposition module that disentangles appearance into editable layers (e.g., separated iris and skin), resolving semantic confusion in facial regions. Experiments demonstrate that StdGEN++ achieves state-of-the-art performance, significantly outperforming existing methods in
geometric accuracy and semantic disentanglement. Crucially, the resulting structural independence unlocks advanced downstream capabilities, including non-destructive editing, physics-compliant animation, and gaze tracking, making it a robust solution for automated character asset production.
Read the full abstract ▾
All authors Yuze He, Yanning Zhou, Wang Zhao, Jingwen Ye, Zhongkai Wu, Ran Yi, Yong-Jin Liu
Computer Vision
GeoMotionGPT: Geometry-Aligned Motion Understanding with Large Language Models
Zhankai Ye, Bofan Li, Yukai Jin et al. (+5 more)
Discrete motion tokenization has recently enabled Large
Language Models (LLMs) to serve as versatile backbones for motion understanding and motion-
language reasoning. However, existing pipelines typically decouple motion quantization from semantic
embedding learning, linking them solely via
token IDs. This approach fails to effectively align the intrinsic geometry of the motion space with the
embedding space, thereby hindering the LLM's capacity for nuanced motion
reasoning. We argue that
alignment is most effective when both modalities share a unified
geometric basis. Therefore, instead of forcing the LLM to reconstruct the complex geometry among motion
tokens from scratch, we present a novel framework that explicitly enforces orthogonality on both the motion codebook and the LLM
embedding space, ensuring that their relational structures naturally mirror each other. Specifically, we employ a decoder-only quantizer with Gumbel-Softmax for differentiable training and balanced codebook usage. To bridge the modalities, we use a sparse projection that maps motion codes into the LLM
embedding space while preserving orthogonality. Finally, a two-stage orthonormal regularization schedule enforces soft constraints during tokenizer training and LLM fine-tuning to maintain
geometric alignment without hindering semantic adaptation. Extensive experiments show that our framework improves the aggregated Average by 22.4% over the strongest baseline on HumanML3D and by 14.4% on KIT-ML, while ablations confirm the effectiveness of the tokenizer, projection, and regularization designs.
Read the full abstract ▾
All authors Zhankai Ye, Bofan Li, Yukai Jin, Shuoqiu Li, Wei Wang, Yanfu Zhang, Shangqian Gao, Xin Liu
Computer Vision
PARL: Position-Aware Relation Learning Network for Document Layout Analysis
Fuyuan Liu, Dianyu Yu, He Ren et al. (+10 more)
Document layout analysis aims to detect and categorize structural elements (e.g., titles, tables, figures) in scanned or digital documents. Popular methods often rely on high-quality Optical Character Recognition (OCR) to merge visual features with extracted text. This dependency introduces two major drawbacks: propagation of text recognition errors and substantial computational overhead, limiting the robustness and practical applicability of
multimodal approaches. In contrast to the prevailing
multimodal trend, we argue that effective layout analysis depends not on text-visual fusion, but on a deep understanding of documents' intrinsic visual structure. To this end, we propose PARL (Position-Aware Relation Learning Network), a novel OCR-free,
vision-only framework that models layout through positional sensitivity and relational structure. Specifically, we first introduce a Bidirectional Spatial Position-Guided Deformable
Attention module to embed explicit positional dependencies among layout elements directly into visual features. Second, we design a Graph Refinement Classifier (GRC) to refine predictions by modeling contextual relationships through a dynamically constructed layout graph. Extensive experiments show PARL achieves state-of-the-art results. It establishes a new
benchmark for
vision-only methods on DocLayNet and, notably, surpasses even strong
multimodal models on M6Doc. Crucially, PARL (65M) is highly efficient, using roughly four times fewer parameters than large
multimodal models (256M), demonstrating that sophisticated visual structure modeling can be both more efficient and robust than
multimodal fusion.
Read the full abstract ▾
All authors Fuyuan Liu, Dianyu Yu, He Ren, Nayu Liu, Xiaomian Kang, Delai Qiu, Fa Zhang, Genpeng Zhen, Shengping Liu, Jiaen Liang, Wei Huang, Yining Wang, Junnan Zhu
Computer Vision
UIKA: Fast Universal Head Avatar from Pose-Free Images
Zijian Wu, Boyao Zhou, Liangxiao Hu et al. (+6 more)
We present UIKA, a feed-forward animatable Gaussian head model from an arbitrary number of pose-free inputs, including a single image, multi-view captures, and smartphone-captured
videos. Unlike the traditional avatar method, which requires a studio-level multi-view capture system and reconstructs a human-specific model through a long-time optimization process, we rethink the task through the lenses of model representation, network design, and data preparation. First, we introduce a UV-guided avatar modeling strategy, in which each input image is associated with a pixel-wise facial correspondence estimation. Such correspondence estimation allows us to reproject each valid pixel color from screen space to UV space, which is independent of camera pose and character expression. Furthermore, we design learnable UV
tokens on which the
attention mechanism can be applied at both the screen and UV levels. The learned UV
tokens can be decoded into canonical Gaussian attributes using aggregated UV information from all input views. To train our large avatar model, we additionally prepare a large-scale, identity-rich synthetic training
dataset. Our method significantly outperforms existing approaches in both
monocular and multi-view settings.
Read the full abstract ▾
All authors Zijian Wu, Boyao Zhou, Liangxiao Hu, Hongyu Liu, Yuan Sun, Xuan Wang, Xun Cao, Yujun Shen, Hao Zhu
Computer Vision
Fundamental Recovery Bounds for SPAD Signals under Stationary Flux
Lior Dvir, Nadav Torem, Mohit Gupta et al. (+1 more)
Single-
photon avalanche diodes (SPADs) record light as a discrete stream of individual detections. The signal is
stochastic. Its statistical structure depends on the sensor's operation mode: binary detection in fixed bins, timestamped detection in fixed bins, or free-running timestamped detection. We derive the likelihood score function for each of these three passive modes. From this single object, stem both fundamental limits of recovery (Cramer-Rao bounds) and practical recovery
algorithms based on
diffusion posterior sampling. The paper further generalizes fundamental limits to
Bayesian Cramer-Rao lower bounds. This generalization makes use of a learned approximation of the score function of signal priors. In prior art, analyses and
diffusion-based reconstruction for SPAD data have treated individual modes in isolation. Our unified treatment shows a qualitative high-flux gap between modes: binary counts saturate exponentially, while timestamped modes degrade only linearly. We further extend
diffusion posterior sampling, previously restricted to binary SPAD data, to a full timestamped case using the suitable score function. We demonstrate experimentally that matching the score to the operation mode is beneficial for high-fidelity reconstruction. By tying the recovery bounds and
diffusion to the score function, this work aims to establish a common foundation for both asking what is recoverable in single-
photon sensing, and building methods that approach the bound.
Read the full abstract ▾
All authors Lior Dvir, Nadav Torem, Mohit Gupta, Yoav Y. Schechner
Computer Vision
Robust Multicentre Detection and Classification of Colorectal Liver Metastases on CT: Application of Foundation Models
Shruti Atul Mali, Zohaib Salahuddin, Yumeng Zhang et al. (+10 more)
Colorectal liver metastases (CRLM) are a major cause of cancer-related mortality, and reliable detection on CT remains challenging in multi-centre settings. We developed a foundation model-based AI pipeline for patient-level classification and lesion-level detection of CRLM on contrast-enhanced CT, integrating uncertainty quantification and explainability. CT data from the EuCanImage consortium (n=2437) and an external TCIA cohort (n=197) were used. Among several pretrained models, UMedPT achieved the best performance and was fine-tuned with an MLP head for classification and an FCOS-based head for lesion detection. The classification model achieved an AUC of 0.90 and a sensitivity of 0.82 on the combined test set, with a sensitivity of 0.85 on the external cohort. Excluding the most uncertain 20 percent of cases improved AUC to 0.91 and balanced accuracy to 0.86. Decision curve analysis showed clinical benefit for threshold probabilities between 0.30 and 0.40. The detection model identified 69.1 percent of lesions overall, increasing from 30 percent to 98 percent across lesion size quartiles. Grad-CAM highlighted lesion-corresponding regions in high-confidence cases. These results demonstrate that foundation model-based pipelines can support robust and interpretable CRLM detection and classification across heterogeneous CT data.
Read the full abstract ▾
All authors Shruti Atul Mali, Zohaib Salahuddin, Yumeng Zhang, Andre Aichert, Xian Zhong, Henry C. Woodruff, Maciej Bobowicz, Katrine Riklund, Juozas Kupčinskas, Lorenzo Faggioni, Roberto Francischello, Razvan L Miclea, Philippe Lambin
Computer Vision
Ahmad AlMughrabi, Guillermo Rivo, Carlos Jiménez-Farfán et al. (+6 more)
Food image
segmentation is a critical task for dietary analysis, enabling accurate estimation of food volume and nutrients. However, current methods suffer from limited multi-view data and poor generalization to new viewpoints. We introduce BenchSeg, a novel multi-view food
video segmentation dataset and
benchmark. BenchSeg aggregates 55 dish scenes (from Nutrition5k, Vegetables & Fruits, MetaFood3D, and FoodKit) with 25,284 meticulously annotated frames, capturing each dish under free 360{\deg} camera motion. We evaluate a diverse set of 20 state-of-the-art
segmentation models (e.g., SAM-based,
transformer, CNN, and large
multimodal) on the existing FoodSeg103
dataset and evaluate them (alone and combined with
video-
memory modules) on BenchSeg. Quantitative and qualitative results demonstrate that while standard image segmenters degrade sharply under novel viewpoints,
memory-augmented methods maintain temporal consistency across frames. Our best model based on a combination of SeTR-MLA+XMem2 outperforms prior work (e.g., improving over FoodMem by ~2.63% mAP), offering new insights into food
segmentation and tracking for dietary analysis. In addition to frame-wise spatial accuracy, we introduce a dedicated temporal evaluation protocol that explicitly quantifies
segmentation stability over time through continuity, flicker rate, and IoU drift metrics. This allows us to reveal failure modes that remain invisible under standard per-frame evaluations. We release BenchSeg to foster future research. The project page including the
dataset annotations and the food
segmentation models can be found at https://amughrabi.github.io/benchseg.
Read the full abstract ▾
All authors Ahmad AlMughrabi, Guillermo Rivo, Carlos Jiménez-Farfán, Umair Haroon, Farid Al-Areqi, Hyunjun Jung, Benjamin Busam, Ricardo Marques, Petia Radeva
Computer Vision · Cross-listed from cs.HC
A Multimodal Dataset of Student Oral Presentations with Sensors and Evaluation Data
Alvaro Becerra, Ruth Cobos and Roberto Daza
Oral presentation skills are a critical component of higher education, yet comprehensive
datasets capturing real-world student performance across multiple modalities remain scarce. To address this gap, we present SOPHIAS (Student Oral Presentation monitoring for Holistic Insights & Analytics using Sensors), a 12-hour
multimodal dataset containing recordings of 50 oral presentations delivered by 65 undergraduate and master's students at the Universidad Autonoma de Madrid, comprising 46 individual presentations with a mean presentation duration of 9 min 48 s (SD = 33 s) followed by a mean Q&A duration of 6 min 28 s (SD = 3 min 02 s), and 4 group presentations with a mean presentation duration of 14 min 11 s (SD = 1 min 40 s) followed by a mean Q&A duration of 8 min 22 s (SD = 1 min 21 s). SOPHIAS integrates eight timestamped sensor streams from high-definition webcams, ambient and webcam audio, eye-tracking glasses, smartwatch physiological sensors, and clicker, keyboard and mouse interactions. In addition, the
dataset includes slides and rubric-based evaluations from teachers, peers, and self-assessments, along with timestamped contextual annotations. The
dataset captures presentations conducted in real classroom settings, preserving authentic student behaviors, interactions, and physiological responses. SOPHIAS enables the exploration of relationships between
multimodal behavioral and physiological signals and presentation performance, supports the study of peer-assessment and provides a
benchmark for developing automated feedback and
Multimodal Learning Analytics tools. The
dataset is available for research, including both academic and legitimate commercial research and development, under controlled access through Science Data Bank, subject to approval of a Data Usage Agreement (DUA), with code provided through GitHub.
Read the full abstract ▾
Computer Vision
Enhancing Novel View Synthesis via Geometry Grounded Set Diffusion
Farhad G. Zanjani, Hong Cai and Amirhossein Habibian
We present SetDiff, a geometry-grounded multi-view
diffusion framework that enhances novel-view renderings produced by 3D Gaussian Splatting. Our method integrates explicit 3D priors, pixel-aligned coordinate maps and pose-aware Plucker ray
embeddings, into a set-based
diffusion model capable of jointly processing variable numbers of reference and target views. This formulation enables robust occlusion handling, reduces
hallucinations under low-signal conditions, and improves photometric fidelity in visual content restoration. A unified set mixer performs global
token-level
attention across all input views, supporting scalable multi-camera enhancement while maintaining computational efficiency through latent-space supervision and selective decoding. Extensive experiments on EUVS, Para-Lane, nuScenes, and DL3DV demonstrate significant gains in perceptual fidelity, structural similarity, and robustness under severe extrapolation. SetDiff establishes a state-of-the-art
diffusion-based solution for realistic and reliable novel-view synthesis in autonomous driving scenarios.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
cSVR: Convolutional Slice-to-Volume Reconstruction
Margherita Firenze, Sean I. Young, Clinton J. Wang et al. (+5 more)
Unpredictable fetal motion during MRI scans can result in oblique 2D slices that are difficult to interpret and often leads to prolonged scan times. Slice-to-volume reconstruction (SVR) methods align 2D oblique slices from multiple slice stacks into a coherent 3D volume. We propose a convolutional network that predicts slice poses by performing multiscale, unrolled optimization and further refines them and super-resolves the volume using model-based optimization. Our convolutional feed-forward SVR network achieves accuracy on par with state-of-the-art results while offering significant speedup, paving the way for scanner-side use of SVR. The code is available https://github.com/MedicalVisionGroup/cSVR.
Read the full abstract ▾
All authors Margherita Firenze, Sean I. Young, Clinton J. Wang, Hyuk Jin Yun, Elfar Adalsteinsson, Kiho Im, P. Ellen Grant, Polina Golland
Computer Vision
Mon3tr: Monocular 3D Telepresence with Pre-built Gaussian Avatars as Amortization
Fangyu Lin, Yingdong Hu, Zhening Liu et al. (+3 more)
Immersive telepresence aims to transform human interaction in AR/VR applications by enabling lifelike full-body
holographic representations for enhanced remote collaboration. However, existing systems rely on hardware-intensive multi-camera setups and demand high bandwidth for volumetric streaming, limiting their real-time performance on mobile devices. To overcome these challenges, we propose Mon3tr, a novel
Monocular 3D telepresence framework that integrates 3D Gaussian splatting (3DGS) based parametric human modeling into telepresence for the first time. Mon3tr adopts an amortized computation strategy, dividing the process into a one-time offline multi-view reconstruction phase to build a user-specific avatar and a
monocular online inference phase during live telepresence sessions. A single
monocular RGB camera is used to capture body motions and facial expressions in real time to drive the 3DGS-based parametric human model, significantly reducing system complexity and cost. The extracted motion and appearance features are transmitted at < 0.2 Mbps over WebRTC's data channel, allowing robust adaptation to network fluctuations. On the receiver side, e.g., Meta Quest 3, we develop a lightweight 3DGS attribute deformation network to dynamically generate corrective 3DGS attribute adjustments on the pre-built avatar, synthesizing photorealistic motion and appearance at ~ 60 FPS. Extensive experiments demonstrate the state-of-the-art performance of our method, achieving a PSNR of > 28 dB for novel poses, an end-to-end latency of ~ 80 ms, and > 1000x bandwidth reduction compared to point-cloud streaming, while supporting real-time operation from
monocular inputs across diverse scenarios. Our demos can be found at https://mon3tr3d.github.io.
Read the full abstract ▾
All authors Fangyu Lin, Yingdong Hu, Zhening Liu, Yufan Zhuang, Zehong Lin, Jun Zhang
Computer Vision
Anatomy Aware Cascade Network: Bridging Epistemic Uncertainty and Geometric Manifold for 3D Tooth Segmentation
Bing Yu, Liu Shi, Haitao Wang et al. (+4 more)
Accurate three-dimensional (3D) tooth
segmentation from Cone-Beam Computed
Tomography (CBCT) is a prerequisite for digital dental workflows. However, achieving high-fidelity
segmentation remains challenging due to adhesion artifacts in naturally occluded scans, which are caused by low contrast and indistinct inter-arch boundaries. To address these limitations, we propose the Anatomy Aware Cascade Network (AACNet), a coarse-to-fine framework designed to resolve boundary ambiguity while maintaining global structural consistency. Specifically, we introduce two mechanisms: the Ambiguity Gated Boundary Refiner (AGBR) and the Signed Distance Map guided Anatomical
Attention (SDMAA). The AGBR employs an
entropy based gating mechanism to perform targeted feature rectification in high uncertainty transition zones. Meanwhile, the SDMAA integrates implicit
geometric constraints via signed distance map to enforce topological consistency, preventing the loss of spatial details associated with standard pooling. Experimental results on a
dataset of 125 CBCT volumes demonstrate that AACNet achieves a Dice Similarity Coefficient of 90.17 % and a 95% Hausdorff Distance of 3.63 mm, significantly outperforming state-of-the-art methods. Furthermore, the model exhibits strong generalization on an external
dataset with an HD95 of 2.19 mm, validating its reliability for downstream clinical applications such as
surgical planning. Code for AACNet is available at https://github.com/shiliu0114/AACNet.
Read the full abstract ▾
All authors Bing Yu, Liu Shi, Haitao Wang, Deran Qi, Xiang Cai, Wei Zhong, Qiegen Liu
Computer Vision
FocalOrder: Focal Preference Optimization for Reading Order Detection
Fuyuan Liu, Dianyu Yu, He Ren et al. (+10 more)
Reading order detection is the foundation of document understanding. Most existing methods rely on uniform supervision, implicitly assuming a constant difficulty distribution across layout regions. In this work, we challenge this assumption by revealing a critical flaw: Positional Disparity, a phenomenon where models demonstrate mastery over the deterministic start and end regions but suffer a performance collapse in the complex intermediate sections. This degradation arises because standard training allows the massive volume of easy patterns to drown out the learning signals from difficult layouts. To address this, we propose FocalOrder, a framework driven by Focal Preference Optimization (FPO). Specifically, FocalOrder employs adaptive difficulty discovery with exponential moving average mechanism to dynamically pinpoint hard-to-learn transitions, while introducing a difficulty-calibrated pairwise ranking objective to enforce global logical consistency. Extensive experiments demonstrate that FocalOrder establishes new state-of-the-art results on OmniDocBench v1.0 and Comp-HRDoc. Our compact model not only outperforms competitive specialized baselines but also significantly surpasses large-scale general VLMs. These results demonstrate that aligning the optimization with intrinsic structural ambiguity of documents is critical for mastering complex document structures.
Read the full abstract ▾
All authors Fuyuan Liu, Dianyu Yu, He Ren, Nayu Liu, Xiaomian Kang, Delai Qiu, Fa Zhang, Genpeng Zhen, Shengping Liu, Jiaen Liang, Wei Huang, Yining Wang, Junnan Zhu
Computer Vision · Cross-listed from cs.LG
Task Prototype-Based Knowledge Retrieval for Multi-Task Learning from Partially Annotated Data
Youngmin Oh, Hyung-Il Kim and Jung Uk Kim
Multi-task learning (MTL) is critical in real-world applications such as autonomous driving and robotics, enabling simultaneous handling of diverse tasks. However, obtaining fully annotated data for all tasks is impractical due to labeling costs. Existing methods for partially labeled MTL typically rely on predictions from unlabeled tasks, making it difficult to establish reliable task associations and potentially leading to negative transfer and suboptimal performance. To address these issues, we propose a prototype-based knowledge
retrieval framework that achieves robust MTL instead of relying on predictions from unlabeled tasks. Our framework consists of two key components: (1) a task prototype
embedding task-specific characteristics and quantifying task associations, and (2) a knowledge
retrieval transformer that adaptively refines feature representations based on these associations. To achieve this, we introduce an association knowledge generating (AKG) loss to ensure the task prototype consistently captures task-specific characteristics. Extensive experiments demonstrate the effectiveness of our framework, highlighting its potential for robust multi-task learning, even when only a subset of tasks is annotated.
Read the full abstract ▾
Computer Vision
From Sketch to Fresco: Efficient Diffusion Transformer with Progressive Resolution
Shikang Zheng, Guantao Chen, Lixuan He et al. (+4 more)
Diffusion Transformers achieve impressive generative quality but remain computationally expensive due to iterative sampling. Recently, dynamic resolution sampling has emerged as a promising acceleration technique by reducing the resolution of early sampling steps. However, existing methods rely on heuristic re-noising at every resolution transition, injecting noise that breaks cross-stage consistency and forces the model to relearn global structure. In addition, these methods indiscriminately upsample the entire latent space at once without checking which regions have actually converged, causing accumulated errors, and visible artifacts. Therefore, we propose Fresco, a dynamic resolution framework that unifies re-noise and global structure across stages with progressive upsampling, preserving both the efficiency of low-resolution drafting and the fidelity of high-resolution refinement, with all stages aligned toward the same final target. Fresco achieves near-lossless acceleration across diverse domains and models, including 10$\times$ speedup on FLUX, and 5$\times$ on HunyuanVideo, while remaining orthogonal to
distillation, quantization and feature caching, reaching 22$\times$ speedup when combined with distilled models. Our code is in supplementary material and will be released on Github.
Read the full abstract ▾
All authors Shikang Zheng, Guantao Chen, Lixuan He, Jiacheng Liu, Yuqi Lin, Chang Zou, Linfeng Zhang
Computer Vision
Improving Video Question Answering through query-based frame selection
Himanshu Patil, Geo Jolly, Ramana Raja Buddala et al. (+2 more)
Video Question Answering (VideoQA) models enhance understanding and interaction with audiovisual content, making it more accessible, searchable, and useful for a wide range of fields such as education, surveillance, entertainment, and content creation. Due to heavy compute requirements, most large visual
language models (VLMs) for VideoQA rely on a fixed number of frames by uniformly sampling the
video. However, this process does not pick important frames or capture the context of the
video. We present a novel query-based selection of frames relevant to the questions based on the submodular mutual Information (SMI) functions. By replacing uniform frame sampling with query-based selection, our method ensures that the chosen frames provide complementary and essential visual information for accurate VideoQA. We evaluate our approach on the MVBench
dataset, which spans a diverse set of multi-action
video tasks. VideoQA accuracy on this
dataset was assessed using two VLMs, namely
Video-LLaVA and LLaVA-NeXT, both of which originally employed uniform frame sampling. Experiments were conducted using both uniform and query-based sampling strategies. An accuracy improvement of up to 4% was observed when using query-based frame selection over uniform sampling. Qualitative analysis further highlights that query-based selection, using SMI functions, consistently picks frames better aligned with the question. We opine that such query-based frame selection can enhance accuracy in a wide range of tasks that rely on only a subset of
video frames.
Read the full abstract ▾
All authors Himanshu Patil, Geo Jolly, Ramana Raja Buddala, Ganesh Ramakrishnan, Rohit Saluja
Computer Vision
PanoSAMic: Panoramic Image Segmentation from SAM Feature Encoding and Dual View Fusion
Mahdi Chamseddine, Didier Stricker and Jason Rambach
Existing image foundation models are not optimized for spherical images having been trained primarily on perspective images. PanoSAMic integrates the pre-trained Segment Anything (SAM) encoder to make use of its extensive training and integrate it into a semantic
segmentation model for panoramic images using multiple modalities. We modify the SAM encoder to output multi-stage features and introduce a novel spatio-modal fusion module that allows the model to select the relevant modalities and best features from each modality for different areas of the input. Furthermore, our semantic decoder uses spherical
attention and dual view fusion to overcome the distortions and edge discontinuity often associated with panoramic images. PanoSAMic achieves state-of-the-art (SotA) results on Stanford2D3DS for RGB, RGB-D, and RGB-D-N modalities and on Matterport3D for RGB and RGB-D modalities. https://github.com/dfki-av/PanoSAMic
Read the full abstract ▾