Computer Vision · Cross-listed from cs.CL
LightMem-Ego: Your AI Memory for Everyday Life
Yijun Chen, Boyi Xiao, Yixian Zhao et al. (+10 more)
Personal AI assistants on mobile and wearable devices continuously perceive users' daily lives through visual and audio streams. However, answering queries about past experiences requires lightweight
multimodal memory that can continuously accumulate, organize, and retrieve long-term experiences, which remains challenging. To address this challenge, we present LightMem-Ego, a lightweight streaming
multimodal memory system for everyday-life assistance. The system continuously captures
egocentric visual and audio streams, aligns them on a shared timeline, and organizes them into a hierarchical
memory consisting of current, short-term, and long-term
memory. Given a user query, LightMem-Ego dynamically routes
retrieval to the appropriate
memory level and generates answers grounded in
multimodal evidence. The demonstration can be deployed on smartphones and AI glasses, supporting object finding, conversation recall, life summarization, routine discovery, and personalized assistance. Code is available at https://github.com/zjunlp/LightMem-Ego.
Read the full abstract ▾
All authors Yijun Chen, Boyi Xiao, Yixian Zhao, Haoting Xia, Buqiang Xu, Jizhan Fang, Yanya Li, Yaqi Zheng, Xuehai Wang, Zirui Xue, Liuxin Zhang, Hui Li, Ningyu Zhang
Computer Vision
Towards Efficient Convolutional Neural Network for Embedded Hardware via Multi-Dimensional Pruning
Hao Kong, Di Liu, Xiangzhong Luo et al. (+5 more)
In this paper, we propose TECO, a multi-dimensional
pruning framework to collaboratively prune the three dimensions (depth, width, and resolution) of convolutional neural networks (CNNs) for better execution efficiency on embedded hardware. In TECO, we first introduce a two-stage importance evaluation framework, which efficiently and comprehensively evaluates each
pruning unit according to both the local importance inside each dimension and the global importance across different dimensions. Based on the evaluation framework, we present a heuristic
pruning algorithm to progressively prune the three dimensions of CNNs towards the optimal trade-off between accuracy and efficiency. Experiments on multiple
benchmarks validate the advantages of TECO over existing state-of-the-art (SOTA) approaches. The code and pre-trained models are available at https://github.com/ntuliuteam/Teco.
Read the full abstract ▾
All authors Hao Kong, Di Liu, Xiangzhong Luo, Shuo Huai, Ravi Subramaniam, Christian Makaya, Qian Lin, Weichen Liu
Computer Vision · Cross-listed from cs.CL
Direct Image-to-Modern Vietnamese Translation of Han-Nom Manuscripts via Multimodal RLHF Preference Alignment
Thi Kim Trang Vo, Nghia Hieu Nguyen and Ha Minh Tan
Translating Han-Nom manuscripts into modern Vietnamese is challenging because historical pages are often degraded, the script contains rare logographic characters, and parallel supervision is limited. We propose a
multimodal RLHF preference-
alignment framework that conditions Vietnamese generation on manuscript images and aligned Han-Nom source text. The model combines four streams: CLIP ViT-L/14@336 for visual features, bert-base-chinese for Han-Nom representations, vinai/phobert-base for Vietnamese representations, and T5-small encoder states. Modality-specific projections and a fusion block compress the resulting 2,048-dimensional concatenation into a shared 512-dimensional representation. Starting from the same supervised fine-tuned policy, we compare PPO, DPO, and KTO under matched work-level macro-averaged evaluation. DPO achieves the best BLEU-4, ROUGE-L, BERTScore, semantic similarity, CER, WER, and
token accuracy, whereas PPO obtains the highest precision, recall, and F1. KTO remains competitive through its desirable-undesirable utility objective. All preference-aligned policies improve the BLEU-4 and semantic-similarity scores available for the SFT baseline. These results indicate that
multimodal preference optimization complements supervised learning by improving lexical and semantic quality in low-resource historical translation.
Read the full abstract ▾
Computer Vision
Video Transformer for Remote Identity Document Hologram Detection
Joris Voerman, Nicolas Sidere and Jean-Christophe Burie
Remote identity authentification using Identification Documents has been a major challenge for several years. DeepFakes advent and the development of AI-guided tools helps fraudsters creating counterfeit ID Documents. Ensuring the authenticity of ID Documents has become a real clue in the seurization of remote authentification. This need is all the more pressing given the increasing digitization of administrative and transactional processes. To ensure widespread accessibility, the system should rely solely on
video captured via mobile devices. In this specific context, confirming the authenticity of ID is a real challenge as many security features needs specific device like infrared sensor for instance. Among underutilized but promising security features,
holographic printings hold a special place. Difficult to counterfeit, they produce distinctive visual effects according enlightment, making them both detectable in a
video captured by a smartphone camera and difficult to imitate. In this paper, we propose a Remote Identity Document Verification System (RIDVS) and an approach based on a
video transformer for detecting holograms in simple
videos captured by smartphones. Our system is designed for a smartphone-based capture process, followed by a server-side verification. The hologram detection method builds on a robust model previously validated in a related research domain. We demonstrate that it outperforms existing SotA methods, achieving near-perfect accuracy even when trained on medium- to small-sized
datasets. In particular, we report improvements of +26.86% in Recall and +17.93% in accuracy over the best MIDV-Holo baseline. This study includes several experiments that evaluate the model adaptation to frugality, both for training samples and computational resources.
Read the full abstract ▾
Computer Vision
Uncertainty Quantification for EO Regression Tasks: Building Height, Tree Canopy Height and Above-ground Biomass Estimation
Ritu Yadav, Andrea Nascetti and Yifang Ban
Earth Observation regression tasks such as building height, canopy height, and above-ground biomass estimation underpin critical applications in urban planning, forest monitoring, and
climate policy, where both accuracy and reliability are critical. Yet most deep learning models yield only deterministic predictions, providing no indication of per-pixel reliability. These regression tasks are inherently challenging due to heterogeneous land surfaces, skewed target distributions, sensor noise, and signal saturation at high target values, making uncertainty (UC) estimation essential for reliable inference. We address this gap by modeling aleatoric uncertainty using year-long Sentinel-1 SAR and Sentinel-2 MSI time series, proposing two complementary approaches: (i) Gaussian UC, which jointly predicts mean and standard deviation under a Gaussian assumption, and (ii)
Quantile UC, which estimates the 10th, 50th, and 90th
quantiles to capture asymmetric and heteroscedastic error distributions. Both models are evaluated on three representative EO regression tasks at 10 m spatial resolution. Results show that both approaches match or surpass deterministic
benchmarks and existing global products, while delivering well-calibrated, interpretable, and operationally useful confidence estimates. Notably, both models outperform the current 10 m state-of-the-art uncertainty-aware model for canopy height estimation. Our implementation will be available at: https://github.com/RituYadav92/EO-Regression-Uncertainty-Estimation
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Diffusion MRI preprocessing affects ADC estimation and automatic PI-RADS v2.1 classification in bi-parametric prostate MRI
Christos Kanakis, Mathias Perslev, Tim Schakel et al. (+4 more)
Diffusion-weighted imaging (DWI) is acquired as part of bi-parametric prostate MRI, but suffers from artifacts that degrade downstream quantitative and diagnostic performance. While DWI preprocessing is standard in
brain imaging, its adoption in prostate imaging remains limited and lacks standardized pipelines. This study investigated the effect of different DWI preprocessing strategies on apparent
diffusion coefficient (ADC) estimation and automatic Prostate Imaging Reporting and Data System (PI-RADS) classification. 268 cases were derived from the fastMRI prostate cohort by sequentially applying denoising, Gibbs-ringing correction, and diffeomorphic registration for susceptibility distortion correction. ADC maps were compared using linear least squares (LLS) and iteratively-weighted LLS (IWLLS). A 3-class DenseNet classifier was trained to predict PI-RADS scores from multi-channel MRI inputs. ADC analysis revealed statistically significant differences across preprocessing pipelines, with LLS and IWLLS producing numerically equivalent maps. Linear relationships between ADC values were preserved across most
datasets (PCC ~0.99), while distortion correction realigned DWI to T2w anatomy and altered ADC values accordingly (PCC ~0.90). Classification showed the best AUROC and sensitivity for high-risk PI-RADS classes in the fully processed
dataset. False-negative analysis revealed this
dataset produced the least overconfident incorrect predictions on high-risk classes, which is a desirable property for clinical triage. DWI preprocessing, particularly distortion correction, enhances both ADC map quality and the predictive power of deep learning models for PI-RADS classification, supporting the need for optimized preprocessing pipelines in prostate MRI.
Read the full abstract ▾
All authors Christos Kanakis, Mathias Perslev, Tim Schakel, Silvia Ingala, Akshay Pai, Dennis Klomp, Chantal M.W. Tax
Computer Vision
Self-supervised training for high-resolution close-range multispectral remote sensing imagery
Leon-Friedrich Thomas, Mikael Änäkkälä and Antti Lajunen
Although self-supervised learning (SSL) offers a promising way to reduce annotation effort in close-range remote sensing, its effectiveness for high-resolution multispectral unmanned aerial vehicle (
UAV) imagery remains underexplored due to limited data. This study evaluated SSL pretraining for precision agriculture using cm-scale multispectral drone imagery collected across multiple sensors, years, and regions.
Transformer-based encoders were pretrained with Momentum Contrast v3 (MoCo-v3) and Masked Autoencoders on a harmonized
dataset combining msuav500K with newly collected multi-year
UAV imagery from agricultural fields in Finland. Pretraining used four spectral bands (Green, Red, Red-Edge, Near-Infrared) for cross-sensor compatibility. The models were evaluated on crop-weed semantic
segmentation using the WeedMap
dataset with 5–100% training data. The following two subsets served as downstream tasks: Task A (Germany, RedEdge-M), where all pretrained models were compared under partial and full fine-tuning, and Task B (Switzerland, Sequoia), where the best encoder from Task A was assessed. Our Swin
Transformer pretrained with MoCo-v3 achieved the strongest performance on both tasks, surpassing the Swin
Transformer model of Doornbos et al. pretrained on a pre-release of msuav500K. Our pretrained Swin
Transformer further demonstrated cross-sensor and cross-region generalization. We additionally provide a public multi-year multispectral
UAV dataset from Finland to support future research.
Read the full abstract ▾
Computer Vision
Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models
Peng Xia, Junbiao Pang and Muhammad Ayub Sabir
Post-training quantization (PTQ) compresses deep neural networks for deployment under limited
memory and computational budgets. However, low-bit (i.e., 2-bit or 4-bit) PTQ often suffers from substantial performance degradation. Most existing PTQ methods operate on an unconstrained full-precision (FP) model and primarily address quantization errors through post-hoc reconstruction. We argue that low-bit PTQ accuracy is limited not only by post-quantization error minimization, but also by the quantization-error tolerance of a FP model itself. In this paper, we propose Efficient Tuning Before Quantization (ETBQ), a pre-conditioning tuning stage for
Stochastic Gradient Descent (SGD)-optimized models before PTQ. During tuning, the FP model is optimized under perturbations sampled from the error distributions of weight and activation quantization, guiding the model toward a loss-landscape region that is less sensitive to the subsequent PTQ. Unlike QAT, ETBQ does not train a fake-quantized deployment model, which is computationally and
memory intensive. Instead, ETBQ outputs a FP model that can be used by any PTQ backend. Experiments on CIFAR-100, Tiny-ImageNet, ImageNet, and Cityscapes provide consistent evidence that ETBQ improves low-bit PTQ across diverse tasks. Under W2A4 settings, e.g., ETBQ improves over naive PTQ by 2.14% top-1 accuracy on Tiny-ImageNet and by 5.80% mIoU on Cityscapes. Code is available at https://github.com/xpxpxp2001xpxpxp/ETBQ.
Read the full abstract ▾
Computer Vision
Benchmarking Edge Inference Strategies for Deep Learning Models in Industrial Machine Vision
Miguel Gomez Fernandez, David Castro Boga, Roi Mendez-Rial et al. (+1 more)
Edge deployment is often the preferred solution for industrial machine
vision systems when low latency, data security, or limited connectivity are critical requirements. Several frameworks are available to optimise inference on edge devices; however, relatively few studies have systematically compared their inference-time performance under industrial deployment conditions. In this work, we present a comparative study of four widely used approaches for machine
vision inference in industrial settings: plain PyTorch, ONNX Runtime, OpenVINO, and TensorRT. The evaluation focuses on inference time, covers several CPU- and GPU-based hardware platforms, and includes both conventional convolutional neural networks and a
transformer-based
vision model. For the hardware platforms and models evaluated, the results show that OpenVINO achieves the lowest inference time on CPUs, while TensorRT achieves the lowest inference time on GPUs. However, TensorRT does not outperform plain PyTorch for the
transformer-based model considered in this study.
Read the full abstract ▾
All authors Miguel Gomez Fernandez, David Castro Boga, Roi Mendez-Rial, Eric Lopez-Lopez
Computer Vision
Longitudinal Multi-View Breast Cancer Risk Prediction
Solveig Thrun, Zijun Sun, Suaiba A. Salahuddin et al. (+5 more)
Accurate breast cancer risk prediction from screening mammography is critical for enabling personalized screening intervals and early detection. Recent deep learning methods have shown the value of longitudinal data and explicit temporal
alignment. However, existing approaches either perform explicit
alignment using a single mammographic view or model multiple views without explicit longitudinal
alignment, limiting their ability to exploit the complementary spatial-temporal information used in clinical practice. To address this gap, we propose LMV-Net, a longitudinal multi-view breast cancer risk prediction model that jointly analyzes anatomically complementary CC and MLO views within an explicitly aligned longitudinal framework. We evaluate our approach on the public EMBED and CSAW-CC
datasets, comparing it to state-of-the-art breast cancer risk prediction methods. Our model consistently outperforms existing approaches in overall risk prediction performance and across different breast density and cancer subgroups. Importantly, these improvements highlight the potential of longitudinal multi-view modeling to enhance risk stratification, paving the way for future work on personalized screening, earlier identification of high-risk patients, and more efficient screening resource allocation. The code is available at https://github.com/sot176/LMV-Net.
Read the full abstract ▾
All authors Solveig Thrun, Zijun Sun, Suaiba A. Salahuddin, Kristoffer Wickstrøm, Elisabeth Wetzer, Stine Hansen, Robert Jenssen, Michael Kampffmeyer
Computer Vision · Cross-listed from cs.CL
The In-Car Sign Language Corpus (ICSL): A Multi-Modal Resource for Constrained-Space Sign Language Recognition
Raviteja Boddu, Guilherme Vieira Leite, Joed Lopes da Silva et al. (+9 more)
This paper addresses the challenges of using sign
language within shared mobility services, such as taxis, carpools, or ride-sharing platforms. The use of sign
language recognition (SLR) in real-world, confined environments, specifically vehicle interiors remains largely unexplored. To motivate research in this area, we present the In-Car Sign
Language (ICSL)
dataset for Brazilian Sign
Language (Libras), with the long-term goal of improving public transport accessibility for the Deaf and Hard-of-Hearing community. The
dataset consists of: (1) high-precision laboratory motion capture (MoCap) data to establish an idealized linguistic baseline and (2) real-world multi-modal in-car recordings captured using a 2D camera and 3D Time-of-Flight sensors. The
dataset provides a basis for comparative analyses between synthesized signing avatar animations and recorded real signing interpreter
videos, which enable future research into robust "in-the-wild" SLR models and domain adaptation. We describe in detail the use cases, the setup, the data collection protocol, and the metadata structure of the corpus. In total, we recorded a
multimodal dataset exceeding 1.5 million frames, comprising the synchronized
multimodal streams described above featuring Libras users across various in-car scenarios. The corpus is provided with gloss annotation of lexical signs and non-lexical sign
language elements specially designed to support the training and evaluation of deep neural networks for constrained space recognition. In-vehicle signing offers a technically significant example of a constrained, occluded, and non-frontal environment. While recognizing the diverse communication strategies already employed by the Deaf community, identifying automotive-specific limitations provides a useful stepping stone for research into enhancing in-car accessibility and passenger quality of life.
Read the full abstract ▾
All authors Raviteja Boddu, Guilherme Vieira Leite, Joed Lopes da Silva, Ângelo Benetti, Isabela Barbieri, Natália de Melo Afonso, Thyago Santos, Helio Pedrini, Felipe Venâncio Barbosa, José Mario De Martino, Munir Georges, Alessandro Zimmer
Computer Vision
HierCAD: Hierarchical Text-to-CAD Design via Structure Alignment and Parameter Grounding
Jimin Xu, Tianbao Wang, Tao Jin et al. (+1 more)
Recent text-to-CAD approaches have shown promising results by leveraging large
language models, but they often struggle with maintaining structural consistency in complex designs and accurately grounding
geometric parameters. To address these issues, we propose HierCAD, a hierarchical text-to-CAD framework that improves both structural
reasoning and parameter prediction. HierCAD reformulates CAD generation as progressive
reasoning by decomposing CAD construction trees into object-level procedural
reasoning and part-level
topology reasoning trajectories. To further improve generation fidelity, we introduce a unified Structure
Alignment and Parameter Grounding (SAPG) learning strategy. Structure
alignment aligns
topology reasoning trajectories with their corresponding parametric CAD spans, while parameter grounding mitigates shortcut learning through structure-preserving parameter perturbations and ranking-based supervision. Experiments demonstrate that HierCAD outperforms prior state-of-the-art methods on both CAD sequence generation and reconstructed CAD model evaluation. Our code is available at https://github.com/Collab-Gen/HierCAD.
Read the full abstract ▾
All authors Jimin Xu, Tianbao Wang, Tao Jin, Zhou Zhao
Computer Vision
SLVMBench: Skill Learning from Video Memory
Yudong Yang, Guangzhi Sun, Yixuan Li et al. (+1 more)
We introduce Skill Learning from
Video Memory (SLVMBench), the first
benchmark that jointly evaluates whether
video large
language models (
video-LLMs) can learn skills from long
video memory and apply them to real-time tasks. SLVMBench presents models with 2-3 hour
video streams that contain a tutorial
video embedded in a stream of arbitrary irrelevant
videos, resembling real-world human learning practices.
Video-LLMs are asked to apply the acquired skill to answer real-time questions about an ongoing
video. Unlike long-
video understanding
benchmarks that emphasize passive comprehension and skill-learning
benchmarks that rely on short, immediate demonstrations, SLVMBench tests the full pipeline of memorizing and extracting procedural knowledge, as well as transferring it to real-time tasks. Moreover, rigorous human annotations feature sub-second-level temporal calibration, manually engineered questions eliminating common-sense guessing, and collated tutorials to ensure coverage of the required skills. Evaluations on state-of-the-art proprietary and open-source
video LLMs show that
video-LLMs struggle substantially with learning and applying skill knowledge from
videos. Moreover, performance degrades markedly when the skill knowledge is placed within a long
video memory. These results reveal a key limitation of existing
video LLMs and position SLVMBench as the first
benchmark for studying real-time skill acquisition and application from long-context
video memory.
Read the full abstract ▾
All authors Yudong Yang, Guangzhi Sun, Yixuan Li, Chao Zhang
Computer Vision
ASUMOT: Motion-Consistency-Based Asynchronous UAV Detection and Tracking with Event Cameras
Baofeng Jia, Xiaoyu Chen, Jingyuan Zhang et al. (+4 more)
Event cameras offer microsecond-level temporal resolution and high dynamic range for low-altitude
UAV perception. However, long-range
UAVs often produce sparse, fragmented, and noise-contaminated event responses, where one semantic target may appear as multiple spatially separated blobs. Direct blob-level asynchronous tracking therefore suffers from duplicate trajectories and unstable identities. We propose ASUMOT, a motion-consistency-based asynchronous
UAV detection and tracking framework operating directly on raw events. ASUMOT models each
UAV as a set of motion-consistent event blobs. A local motion-consistency
estimator triggers reliable candidates, a lightweight multi-task verifier provides
UAV confidence and motion-direction cues, and motion-consistency clustering aggregates fragmented blobs into identity-consistent
UAV tracks. We also introduce ES-
UAV, a high-definition event-level
UAV benchmark with dense semantic annotations. Experiments on public
UAV tracking data and ES-
UAV show that ASUMOT improves the accuracy–efficiency trade-off while preserving asynchronous event processing. Code and
Dataset will be released.
Read the full abstract ▾
All authors Baofeng Jia, Xiaoyu Chen, Jingyuan Zhang, Zongze Wu, Haochen li, Jing Han, Lianfa Bai
Computer Vision
Metadata Supervised Imaging Representations for Modelling and Controlling Acquisition Variability
Mehmet Yigit Avci, Pedro Borges, Virginia Fernandez et al. (+5 more)
Biomedical imaging data exhibit substantial acquisition variability, where identical biological structures can appear markedly different due to differences in imaging devices, acquisition protocols, sites, and reconstruction settings. Consequently, learned representations often entangle underlying biological information with acquisition-dependent appearance, limiting interpretability, generalisation, and clinical deployment. We show that these sources of variation can be disentangled by jointly modelling medical images and acquisition metadata. Using large-scale clinical
brain MRI data as a case study, we learn representations that disentangle anatomical structure from contrast-dependent appearance. The resulting framework enables the organisation of heterogeneous imaging protocols, sequence understanding, the detection of image-metadata inconsistencies and imaging artifacts, while preserving biologically relevant anatomical features across diverse acquisitions. Building on these disentangled representations, it further supports generative and translational capabilities, performing both metadata-conditioned synthesis of realistic 3D
brain MRIs and anatomy-preserving harmonisation for cross-modality and cross-site adaptation. Our findings demonstrate that acquisition variability is a structured component of the imaging process that can be modeled, audited, synthesised, and controlled, establishing a foundation for acquisition-aware representation learning in large-scale biomedical imaging.
Read the full abstract ▾
All authors Mehmet Yigit Avci, Pedro Borges, Virginia Fernandez, Natalia Glazman, Paul Wright, Mehmet Yigitsoy, Sebastien Ourselin, Jorge Cardoso
Computer Vision
A Unified Framework for Comprehensive Cardiac CT Segmentation and Phenotyping: Human-in-the-Loop Data Annotation, Vision Foundation Model Development, Multicenter Evaluation and Clinical Validation
Pooya Mohammadi Kazaj, Leo Fridolin Weber, Wen Xie et al. (+17 more)
Comprehensive quantification of cardiac structures from computed
tomography (CT) remains limited not by data availability but by the scalability of measurements, which makes routine use impractical. Here we present a unified framework for comprehensive cardiac CT
segmentation and phenotyping that combines a human-in-the-loop annotation pipeline, a cardiac CT augmentation technique, and a self-supervised foundation model pre-trained on 60,000 unlabeled cardiac CT scans. Using this approach, we assembled the largest and most comprehensive expert-annotated cardiac CT
segmentation dataset to date, comprising 1598 cases and 14 distinct cardiac structures (1000 for training, 598 for the external test set). Across five external
datasets, the framework segmented all structures more accurately and comprehensively than existing open-source tools. Self-supervised pre-training improved labeling efficiency, with the most significant gains observed during external evaluation in the low-data regime. Benchmarking across convolutional,
transformer, and state-space architectures showed comparable performance, indicating that data quality and pre-training, rather than architecture, drove accuracy. The framework was scaled to population-level phenotyping, with segmented anatomy that carries functionally relevant information about ventricular function and disease severity beyond demographic variables. By openly releasing the largest
dataset with human labels, code, model weights, a CT augmentation library, and software, this work provides a reproducible foundation for opportunistic cardiac phenotyping from routinely acquired CT scans.
Read the full abstract ▾
All authors Pooya Mohammadi Kazaj, Leo Fridolin Weber, Wen Xie, Seyed Amir Ahmad Safavi-Naini, Anselm Stark, Giovanni Baj, Ali Mokhtari, Toshiya Yoshida, Christoph Ryffel, Taishi Okuno, Yoshihiro Akashi, Ronny R. Buechel, Thomas Pilgrim, Waldo Valenzuela, George C. M. Siontis, Xiaowei Xu, Moritz Hundertmark, Stephan Windecker, Christoph Grani, Isaac Shiri
Computer Vision
SalientGS: Unified SfM-to-3DGS with Importance-Guided MCMC Gaussian Allocation
Tianyu Xiong, Rui Li, Suning Ge et al. (+1 more)
Reconstructing 3D scenes from unordered images remains bottlenecked by expensive Structure-from-Motion (SfM) preprocessing and frozen pose interfaces. We present SalientGS, a unified SfM-to-3D Gaussian Splatting (3DGS) pipeline. Its central contribution is importance-guided Markov Chain Monte Carlo (MCMC) Gaussian allocation, which aggregates multi-view residuals into per-Gaussian underfit and redundancy signals. These signals define a smooth importance-weighted sampling distribution that biases both birth and relocation toward underfit regions. This reallocates capacity from well-fit areas without altering the underlying
stochastic gradient Langevin dynamics (SGLD). SalientGS achieves end-to-end reconstruction in 15 minutes with state-of-the-art perceptual quality. The supplementary material provides dedicated sections for Per-Scene Qualitative Comparisons and Per-Image Learned Perceptual Image Patch Similarity (LPIPS) Analysis, including failure cases. Code and evaluation scripts are available at https://github.com/Six-Bit-TX/SalientGS.
Read the full abstract ▾
All authors Tianyu Xiong, Rui Li, Suning Ge, Jiaqi Yang
Computer Vision
The Devil Is in the Leakage: A Disentangled Dual-Purification Framework for High-Fidelity Hairstyle Transfer
Jijie Li, Jiankuo Zhao, Xiangyu Zhu et al. (+1 more)
Hairstyle transfer aims to synthesize a photorealistic portrait by transplanting the hairstyle from a reference image onto a source subject while preserving the source identity. Recent foundation models show strong generative capability, but they struggle with the zero-shot disentanglement required for precise local editing, often entangling the reference hairstyle with its original identity and pose. Existing
diffusion-based pipelines typically decompose the task by first generating a "bald" image from the source and then injecting hairstyle features from the reference. However, we show that this paradigm suffers from a fundamental leakage problem. Identity Leakage in Hairstyle occurs when hairstyle features retain reference identity or pose information, while Flaw Leakage in Bald arises when residual artifacts in the bald image are propagated into the final synthesis. To address both issues, we propose the Dual-Purification Framework (DPF), which introduces two complementary training-time regularizers.
Adversarial Hairstyle Purification (AHP) purifies hairstyle features by suppressing identity predictability under a mutual-information-inspired
adversarial objective. Contrastive
Geometric Purification (CGP) regularizes the ControlNet pathway with a contrastive objective, reducing the model's reliance on
geometric artifacts in the bald condition. By jointly purifying the hairstyle representation and
geometric pathway, DPF achieves high-fidelity, identity-preserving hairstyle transfer and state-of-the-art performance on diverse
benchmarks.
Read the full abstract ▾
All authors Jijie Li, Jiankuo Zhao, Xiangyu Zhu, Zhen Lei
Computer Vision
LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Models
Gangsu Kim and Won-Ki Jeong
Pathology Foundation Models (PFMs) offer powerful Whole Slide Image (WSI) representations but suffer from massive computational costs. While Knowledge
Distillation (KD) can create efficient student models, existing multi-teacher methods often use suboptimal uniform weighting that ignores tissue heterogeneity. We propose LaGuadia (
Language-Guided Adaptive
DistillAtion), a framework that develops a compact pathology image encoder by dynamically integrating expertise from multiple PFMs under clinical linguistic guidance. Our approach utilizes a multi-stage pipeline: first, extracting visually observable clinical keywords from pathology reports; second, aligning visual features with these keywords via a
Vision-
Language meta-teacher (MedSigLIP) to provide dense semantic guidance; and finally, performing adaptive KD where teacher contributions are weighted based on their semantic
alignment with the clinical narrative. Experiments on WSI captioning, visual question answering, and slide-level classification tasks demonstrate that an 87M parameter LaGuadia student model matches or exceeds foundation-scale models such as GigaPath and UNI, achieving strong factual consistency and robust generalization. These results highlight clinical
language as an effective semantic anchor for building efficient and reliable digital pathology systems. Code is available at https://github.com/hvcl/LaGuadia.
Read the full abstract ▾
Computer Vision
A Nearable Soft Mat Based on Distributed Optical Fiber Sensing for Physiological Monitoring
Vincenzo Lavorgna, Martina Pulcinelli, Andrea Polimadei et al. (+5 more)
Distributed optical fiber sensing (DOFS) combines the advantages of fiber optic sensors, including flexibility, small size, immunity to electromagnetic interference, and high metrological performance, with the capability to transform a single optical fiber into a continuous sensing element for spatially resolved mechanical measurements. Optical frequency domain reflectometry (OFDR), based on Rayleigh backscattering, enables high spatial resolution DOFS measurements, broadening the range of potential sensing applications. However, OFDR based DOFS remains largely unexplored for biomedical applications, despite the need for sensitive, spatially resolved, and conformable sensing interfaces. This study presents a soft DOFS based mat as a large-area interface for physiological monitoring. A single-mode optical fiber was embedded in a flexible silicone
matrix and arranged in a serpentine layout to distribute sensing over the mat surface. With a gage pitch of 2.6 mm, the system provided 2250 sensing sites across the active area at a sampling frequency of 50 Hz. The mat was assessed on six healthy volunteers in a seated nearable configuration on the backrest of a standard office chair. The distributed output enabled two dimensional mapping of the mat response, reflecting back mat mechanical coupling and cardiorespiratory induced perturbations. Respiratory rate and heart rate were therefore estimated and compared with a reference wearable system. The maps revealed physiologically coherent spatial and temporal patterns, while the estimated rates showed good agreement with the reference measurements. These results demonstrate the feasibility of combining large area distributed sensing, spatial mapping, and quantitative cardiorespiratory monitoring within a DOFS based soft nearable interface.
Read the full abstract ▾
All authors Vincenzo Lavorgna, Martina Pulcinelli, Andrea Polimadei, Rosaria D Amato, Carlo Massaroni, Michele Arturo Caponero, Emiliano Schena, Daniela Lo Presti
Computer Vision
Structure-Detail Decoupled Autoregressive Generation for Fast and High-Fidelity Virtual Try-On
Lu Yang, Xiaonan Hu, Yanan Li et al. (+3 more)
Virtual try-on (VTON) is a bi-conditional image generation problem that requires not only accurate person preservation but also faithful garment deformation and detail synthesis.
Diffusion-based VTON methods can jointly model these factors in a compressed latent space, but suffer from high-frequency detail loss due to inherent latent compression, even with costly multi-step denoising. Recent visual autoregressive (VAR) models offer a promising alternative for high-quality generation with faster inference, yet remain unexplored for VTON due to the lack of effective bi-conditioning mechanisms. To bridge this gap, we first introduce VAR-VTON, a VAR-based VTON model that incorporates garment conditioning and structural guidance for efficient latent-space VTON. Despite its efficacy, latent-space generation still struggles to preserve fine-grained garment details. We argue that different VTON sub-tasks should be addressed in different representation spaces: structural synthesis such as garment warping and person layout is suited to the latent space, whereas fine-grained detail recovery should be tackled in the pixel space. Motivated by this insight, we further propose
STAR-VTON, a Two-Stage AutoRegressive framework that builds upon VAR-VTON by decoupling latent-space structural synthesis from pixel-space detail recovery. Our idea is to resort to a matching-informed refiner to establish dense correspondences between the stage-one generation and the source garment to directly map fine-grained pixel-space details. Extensive experiments show that
STAR-VTON achieves an impressive efficiency-fidelity trade-off: VAR-VTON runs at least $4\times$ faster than
diffusion-based counterparts without degrading quality, and the pixel-space refiner effectively restores fine details and acts as a plug-and-play module that can benefit existing VTON approaches.
Read the full abstract ▾
All authors Lu Yang, Xiaonan Hu, Yanan Li, Daqi Liu, Hao Lu, Xiang Bai
Computer Vision
HandFlow: Fully Generative 4D Hand Recovery with Flow Matching
Mingxi Xu, Bowen Duan, Yi Gu et al. (+3 more)
Accurate
monocular 4D hand reconstruction remains challenging. Per-frame discriminative regressors lack temporal context and often produce jittery predictions. Temporal models improve consistency by aggregating information across frames, but they are typically deterministic regressors, making them vulnerable to ambiguous observations caused by occlusion and motion blur. Generative modeling offers a natural alternative by learning a prior over plausible hand motion sequences, enabling coherent hand-state recovery when visual evidence is incomplete or unreliable. Motivated by this observation, we present HandFlow, a fully generative flow-matching framework for temporally coherent 3D hand pose and shape estimation from
monocular video. Given visual and skeletal observations, HandFlow denoises an entire temporal window of MANO parameters through a single ODE integration. To support this, we use a Flux-style dual-stream
transformer that attends across the full sequence to capture long-range dependencies without autoregressive decoding, and a confidence-aware continuous masking mechanism that blends observed features with learnable mask
tokens to handle noisy or missing observations. Experiments on DexYCB and HOT3D show that HandFlow achieves state-of-the-art performance, with particularly large gains in world-space accuracy and temporal smoothness. It reduces world-space pose error by over 30% compared with the strongest baseline and achieves the lowest acceleration error among all evaluated methods, while remaining competitive in per-frame pose accuracy. Moreover, on a single GPU HandFlow reconstructs a 150-frame sequence at 47 fps, about 12x faster than the fastest prior
video-based method, with reconstruction itself accounting for only a small fraction of the end-to-end latency.
Read the full abstract ▾
All authors Mingxi Xu, Bowen Duan, Yi Gu, Zhengyang Shen, Renjing Xu, Yutao Yue
Computer Vision
A Novel Method to Evaluate Models on Unreliable, Noisy and Inconsistent Labels: Adaptive Resolution Label Aggregation (ARLA)
Natasha Randall and Gernot Heisenberg
Labels are critical for both training and evaluating deep learning
segmentation models, but are often inconsistent, noisy, or ambiguous at class boundaries. Many approaches have been developed to support training models on weak labels, but few to none currently exist to facilitate evaluating models on unreliable labels. We therefore introduce a method called "Adaptive Resolution Label Aggregation", or "ARLA", which dynamically adapts the resolution of both the label and the model prediction at inference time before the evaluation metrics are computed. We demonstrate how ARLA can be used to better analyse model behaviour with a practical application to a real flood prediction model, where ARLA was able to overcome issues with inconsistent labelling of forested areas and errors in labels within regions of heavy cloud cover. Our work presents a new approach to evaluating
segmentation models, with adjustable parameters to adapt the aggregated resolution to the precision of the label or the level of label noise. Fundamentally, ARLA exploits the information encapsulated by a label but minimises the label error, extracting from the noise a clearer signal of a model's true performance.
Read the full abstract ▾
Computer Vision
Parallax Portrait Matting
Xin Cai, Jiawen Chen, Lars Jebe et al. (+2 more)
Image matting is highly ill-posed, especially when both the foreground and background are richly textured. While single-image matting methods learn strong priors from data, they often struggle on these challenging cases. Existing approaches improve results by requiring additional signals such as green screens, polarized lighting, or clean background images, but these typically rely on specialized capture setups. We present Parallax Portrait Matting, a practical two-frame matting method that uses a second image captured with slight viewpoint change. Such a setting arises naturally in burst photography, where small camera motion induces foreground-background parallax and provides complementary observations for matting. Our pipeline estimates trimaps and foreground/background motion, then constructs aligned views for prediction. To handle imperfect motion estimation, the network uses the background-aligned pair for direct fusion and the foreground-aligned cue through cross-
attention for error compensation. Experiments show that our method recovers finer details and more accurate foreground colors than strong single-image matting baselines on challenging portrait cases.
Read the full abstract ▾
All authors Xin Cai, Jiawen Chen, Lars Jebe, Tianfan Xue, Zhoutong Zhang
Computer Vision
DynEval: Holistic Evaluations of T2I Generative Models in the Wild
Shyam Marjit, Dheeraj Baiju, Anuj Shikarkhane et al. (+3 more)
Recent advances in text-to-image (T2I) generation have led to models capable of producing highly realistic images. Yet, reliably evaluating their outputs remains challenging, especially at scale. Existing automatic evaluators, often relying on a static
prompt set, struggle to capture subtle failure modes such as partial
prompt misalignment, compositional errors, or visually plausible but semantically incorrect generations. In this work, we introduce DynEval, a Dynamic Evaluation framework designed to jointly assess text-to-image
alignment and image quality of T2I models. To support scalable training beyond limited human-annotated data, we construct two large
datasets. First, we build GenDB, a collection of 500K
prompt-image pairs generated from human-written
prompts drawn from DiffusionDB using a tiered
prompt-model generation strategy. Second, building upon GenDB, we construct DynEvalInstruct, a 250K instruction
dataset comprising
prompt-image-response triplets distilled from a structured evaluation pipeline that decomposes evaluation into text-image
alignment and visual quality
reasoning. Using this
dataset, we perform full fine-tuning of a compact evaluator through a curriculum learning strategy to effectively distill the superior evaluation capabilities of a larger teacher
vision-
language model, resulting in DynEval-2B and DynEval-4B. In extensive comparisons against existing evaluators across 11
benchmarks, our evaluator achieves a higher overall correlation with human judgments. Furthermore, it provides fine-grained analysis of the capabilities and failure modes of 36 T2I models across 42 subcategories and 9 semantic dimensions.
Read the full abstract ▾
All authors Shyam Marjit, Dheeraj Baiju, Anuj Shikarkhane, Akhil Sakthieswaran, Sayak Paul, Anirban Chakraborty
Computer Vision
Slot-RAE: Streamlining Object-Centric Learning via Direct Representation Auto-Encoders
Alexandre Chapin, Emmanuel Dellandrea and Liming Chen
Deploying object-centric models for real-world scene understanding typically requires complex pipelines to achieve both robust scene decomposition and high-fidelity generation. Recent
diffusion-based approaches have improved visual quality, but they almost universally rely on heavy, pretrained generative priors (e.g., Stable
Diffusion) and external VAE latent spaces. In this paper, we propose Slot-RAE, a much simpler, fully integrated framework that operates directly within the continuous semantic feature space of visual foundation models (e.g., DINOv3). Slot-RAE employs a feature-space
diffusion process using a
Diffusion Transformer (DiT) decoder and a Representation
Alignment (REPA) head. Unlike existing
diffusion-based objectcentric methods that rely heavily on subsidized text-toimage priors, the generative core of Slot-RAE (Slot
Attention and the DiT) is trained from scratch within the frozen VFM feature space. This eliminates the need for VAE bottlenecks and task-agnostic generative pre-training. Experiments on the COCO
dataset demonstrate that despite its architectural simplicity, Slot-RAE achieves state-of-the-art results. It delivers comparable unsupervised object discovery, higher-fidelity image reconstruction, and robust zero-shot compositionality, all while being significantly faster and more computationally efficient than existing object-centric latent
diffusion models.
Read the full abstract ▾
Computer Vision
GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents
Suhaas Garre, Emily Ritchie, Sushant Mehta et al. (+1 more)
A large share of day-to-day work in professional domains happens inside PDF files: benefits packets, leases, datasheets, clinical guidelines, construction plans.
Benchmarks for document AI have generally measured the required capabilities in isolation: OCR, layout analysis, chart
reasoning, table QA, document VQA. A high score on any one of them does not necessarily reveal whether a model can answer a realistic question that someone in the field would actually ask about a specific PDF. GDP_pdf is a
benchmark built to measure this directly. It consists of question-document pairs authored by working professionals in ten fields, and a candidate question was kept only when at least two frontier
multimodal models failed it in a way that mattered: a wrong answer, missed decisive evidence, or a fabricated claim, rather than a superficial difference such as style. Each item comes with a rubric of atomic criteria, so we can report a graded rubric score as well as a strict task-level pass rate, and each item is tagged against a taxonomy of eleven capabilities in three tiers, spanning text extraction and grounding, table and chart comprehension, cross-referencing, spatial
reasoning, and abstention on unsupported queries. We report results for seventeen frontier models on the 100-item
benchmark: the best model passes only 30.7% of the items and the worst passes 2%. Most errors trace back to a small set of recurring loss patterns: misaligned tables, misread charts, skipped footnotes and exclusions, miscounted floor-plan symbols, scan noise, and amendments that supersede earlier text.
Read the full abstract ▾
All authors Suhaas Garre, Emily Ritchie, Sushant Mehta, Edwin Chen
Computer Vision
When Depth Is Better Told Than Shown: Depth-Ordinal Prompting for Vision-Language Spatial Reasoning
Quynh Vo, Phuc Dao, Cong-Duy Nguyen et al. (+1 more)
Vision-
language models (VLMs) are expected to reason about physical space – which object is closer, what lies behind what, and how objects are arranged in 3D – yet they still struggle with such spatial judgments. A natural remedy is to show the model a depth map, but we find that this can make performance worse. We show that depth is not absent: it reaches the
language model, but becomes difficult to access for downstream
reasoning, while rendered pseudo-depth maps act as noisy auxiliary images that frozen VLMs cannot easily regulate. We propose Depth-Ordinal Prompting (DOP), a training-free method that converts
monocular depth into a single question-targeted ordinal text cue at the queried objects, without adding a depth image, training a module, injecting features, or using labels. Our key finding is form dependence: the same depth signal can hurt when shown as an image but help when told as text.Across
benchmarks, models, and depth
estimators, DOP improves spatial
reasoning when pseudo-depth provides reliable object-level ordering and remains largely neutral in strong original-image regimes. It is also competitive with the strongest training-free depth-prompting alternative while being simpler and more targeted.
Read the full abstract ▾
All authors Quynh Vo, Phuc Dao, Cong-Duy Nguyen, Thong Nguyen
Computer Vision
TC-MAF: Train-Calibrated Bounded Multi-Evidence Fusion for Multimodal Industrial Anomaly Detection
Ming Deng, Sijin Sun, Xiaochuan Hu et al. (+1 more)
Multimodal anomaly detection benefits from complementary RGB and 3D evidence, yet auxiliary RGB reconstruction is not equally reliable across product categories and class-wise test-time policy selection is usually unavailable. We propose TC-MAF, a base-anchored multi-evidence fusion design that combines a
multimodal detector, complementary Dinomaly evidence, and a small cross-modal consistency cue under one fixed pixel-level fusion formula. A lightweight training-dispersion confidence (TDC) term scales auxiliary participation using only normal training statistics. On MVTec-3D, TC-MAF reaches 0.979 image-level AUROC and 0.990 pixel-level AUPRO, achieving the best mean results on both detection and localization among the compared
multimodal methods. Systematic ablations show that the fusion structure itself is the dominant factor, while TDC provides a smaller but reproducible calibration gain over no calibration or arbitrary calibration. Additional experiments show that the same design remains effective under a pooled-statistics variant, auxiliary-branch and backbone substitutions, few-shot settings, a missing-3D setting, and cross-
dataset evaluation on Eyecandies. Code is available at https://anonymous.4open.science/r/TC_MAF-C3BB.
Read the full abstract ▾
All authors Ming Deng, Sijin Sun, Xiaochuan Hu, Xing Wu
Computer Vision
SISA-Rec: A Semantically Integrated Sequential Recommender with Contrastive Alignment
Soohan Abbasi, Shahid Munir Shah, Rafia Shaikh et al. (+1 more)
Recommendation systems help users recommend relevant items from a large collection of choices. Present work on
transformer-based sequential recommendation learns user preferences from interaction logs, but it mostly focuses on item identifiers and doesn't fully use the semantic meaning of items. This limitation becomes a major challenge in sparse and cold-start scenarios where historical interaction data is limited. To solve this problem, we introduce SISA-Rec (Semantically Integrated Sequential Recommendation), a
transformer-based framework that embeds semantic context directly into sequential modeling. Our approach fuses item ID
embeddings with BERT-based text
embeddings via a gated fusion module, injects semantic similarity into the self-
attention mechanism, and leverages an
attention-based aggregation module to construct comprehensive user representations. Finally, a joint learning objective which combines
Bayesian Personalized Ranking (BPR) and contrastive
alignment loss, aligns the underlying behavioral and semantic spaces. Experiments were conducted on the two highly sparse Amazon Beauty and Amazon Toys & Games
datasets, both having 99.93% sparsity. The results show that SISA-Rec outperforms state-of-the-art baseline models across all evaluation metrics. Compared with the BERT4Rec \cite{petrov2022systematic}, SISA-Rec improves HR@10 by 16.6% and NDCG@10 by 10.3% on Amazon Beauty, and HR@10 by 23.1% and NDCG@10 by 17.9% on Amazon Toys & Games. Cold-start analysis further shows that the proposed model achieves the largest improvements for users with limited interaction historical records. This showcases the value of semantic information when user behavior data is scarce. Overall, the results demonstrate that integrating semantic information into the
attention mechanism leads to more accurate and reliable recommendations.
Read the full abstract ▾
All authors Soohan Abbasi, Shahid Munir Shah, Rafia Shaikh, Mahmoud Aljawarneh