Computer Vision · Cross-listed from eess.IV
Non-contact, Real-time, Heart-rate Measurement using Image Processing with Commodity Cameras and AI Agents
Kelly Li and Fulu Li
Heart rate measurement is one of the key requirements for real-time health monitoring, in particular for health caring of elderly people. Traditional heart rate measurement relies on contact sensing mechanisms such as some heart rate measurement devices at medical hospitals or some wearable devices with embedded sensors such as Apple Watch, etc. In this paper, we develop a system for non-contact, real-time, heart rate measurement using image processing with commodity cameras such as an embedded camera on a laptop, where we use an innovative
algorithm to capture the relevant signals for the computation of heart rate in a time series in real life environments. The presented heart rate computation (HRC) process is composed with four major steps: (a) identify frames per second of the camera in use, i.e., 30 frames per second for a given camera, (b) face detection (FD) with shape predictor of 68 face landmarks using deep learning (DL) method, (c) time sliding window (TSW)
algorithm to de-noise the signal by smoothing out the noise, and (d) compute heart rate based on identified signal periodicity. We test and analyze the developed prototypes against heart rate results by Apple Watch and check the difference range in multiple rounds and compute the mean of the difference for the measurement values of the heart rate of the same person at the same time. We will do further tuning and optimization of the present methods and deploy the system as a personal AI
agent [6] for health monitoring as our future directions.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Reconfigurable Radiology Labels Without Relabeling
Jean-Benoit Delbrouck, Dave Van Veen, Akash Pattnaik et al. (+4 more)
Public chest-radiograph (CXR)
datasets are typically released with small, fixed label schemas such as CheXpert-14. However, the underlying free-text reports describe far more findings – and which findings matter depends on the task, site, and reader. We release a pipeline that converts free-text reports into multi-label
matrices and then reconfigures the label schema through dictionary edits rather than new inference passes, i.e., without relabeling the corpus. After this one-time pass, reconfiguring MIMIC-CXR (223K reports) from cached annotations takes 196 seconds with no API cost, compared to \$6.6K for an equivalent relabeling pass with Claude Opus 4.7. Using a 58-label taxonomy, we show that 43% of CXR studies contain at least one finding outside CheXpert-14. Image probes trained on these labels match CheXpert-14 probes on shared targets while also reaching 0.78 AUROC on expert-reviewed long-tail labels that CheXpert-14 cannot represent. These results suggest a different unit of work for radiology labeling: once reports are structured, the label schema becomes a configuration to edit, not a corpus to relabel.
Read the full abstract ▾
All authors Jean-Benoit Delbrouck, Dave Van Veen, Akash Pattnaik, Kalina Slavkova, Javid Abderezaei, Harris Bergman, Khan Siddiqui
Computer Vision
LipSSD: Lipschitz-Constrained Single-Shot Detection for Adversarially Robust Object Detection
Vincent Lébé, Yannick Prudent, Corentin Friedrich et al. (+3 more)
Object detectors have many applications in safety-critical systems, but they are known to be sensitive to worst-case perturbations such as
adversarial attacks, which limits their applicability in real-world scenarios. Compared with classification,
adversarial robustness for object detection has received less
attention, and existing methods are often tied to
adversarial training, whose performance may not transfer across attacks, perturbation budgets, or architectures. In this work, we introduce Lipschitz-constrained variants of object detection architectures as robust-by-design alternatives to standard detectors. We validate this approach with LipSSD, a Lipschitz-constrained Single Shot MultiBox Detector (SSD), and provide a comprehensive study of its
adversarial robustness using multiple white-box
adversarial attacks and
datasets. We first analyze the accuracyrobustness trade-off induced by Lipschitz constraints and show that it can be controlled through a single training hyperparameter. We then demonstrate that Lipschitzconstrained detectors are complementary to
adversarial training: under the same training setup on the Pascal VOC
dataset, adversarially trained LipSSD improves mAP@50 on unseen attacks by up to 15 points over classical adversarially trained SSD. Finally, we use more specific safety-critical
datasets such as LARD and KITTI, and show that Lipschitz-constrained detectors can improve robustness while largely preserving clean performance. These results suggest that architectural Lipschitz control is a practical and attack-agnostic direction for improving the robustness of object detectors.
Read the full abstract ▾
All authors Vincent Lébé, Yannick Prudent, Corentin Friedrich, Thomas Massena, Ronan Sicre, Franck Mamalet
Computer Vision
AI for Cultural Heritage Textiles: Fine-Tuned Latent Diffusion for Novel Ulos Motif Synthesis
Humasak Tommy Argo Simanjuntak, Jesika Purba, Sitogab Girsang et al. (+4 more)
Preserving and revitalising traditional textiles such as Ulos, a cultural heritage of the Batak ethnic group in North Sumatra, Indonesia, requires balancing fidelity to tradition with innovative approaches that meet contemporary design demands. Traditional Ulos weaving faces two key limitations: a narrow range of motifs and a time-intensive design process. This study presents a generative AI framework that fine-tunes two pretrained latent
diffusion models: Protogen v3.4 and Stable
Diffusion v1.4, on a curated, annotated
dataset of high-resolution Ulos motifs to generate culturally consistent yet novel designs. Model performance is evaluated quantitatively using Frechet Inception Distance (FID), Inception Score (IS), and qualitatively through assessments by traditional weavers and members of the public. Protogen v3.4 consistently outperforms Stable
Diffusion v1.4, achieving substantially lower FID (~10.5x) and higher IS (2.0x), indicating superior visual fidelity, diversity, and closer
alignment with the real Ulos motif distribution. We further examine the effects of strength and guidance scale on generation quality across both models. Lower strength values consistently yield higher fidelity (lower FID), while higher strength values increase generative diversity at the cost of realism, revealing a clear fidelity-diversity tradeoff for both models. Across all tested configurations, a guidance scale of 5-9 provides the most effective balance between fidelity and diversity, stabilising FID, KID, and IS, and is recommended as the operating range for high-quality, diverse Ulos motif generation. These findings demonstrate that carefully fine-tuned generative AI can support the creative renewal of intangible cultural heritage while preserving its stylistic and symbolic integrity.
Read the full abstract ▾
All authors Humasak Tommy Argo Simanjuntak, Jesika Purba, Sitogab Girsang, Widya Manurung, Samuel Situmeang, Arlinta Barus, Daniel Oranova Siahaan
Computer Vision · Cross-listed from cs.RO
IMR: Iterative Mode-World Weighted Regression for Multi-Agent Trajectory Prediction
Honglin Wang, Shiyao Pan and Yun-Fu Liu
Multi-
agent motion prediction is essential for automated vehicles to understand the intentions of surrounding vehicles. However, previous prediction-based and anchor-based methods have limitations in mode diversity and prediction accuracy, respectively. These limitations may cause inadequate safety assessments and behavioral deviations in automated vehicles. To address this issue, a mode-world weighted regression loss is proposed to bridge the gap between these features. Specifically, this approach mitigates mode collapse while simultaneously improving world ranking and top-1 confidence. Furthermore, the proposed iterative decoder improves prediction accuracy by recurrently and segmentally generating trajectories. Experimental results show the proposed method ranks first in the Argoverse 2 multi-
agent motion
forecasting benchmark against other methods.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Curating Same-Family Neural Networks for LLM-Guided Model Improvement: A Controlled Case Study
Kabir Dev Paul Baghel, Radu Timofte and Dmitry Ignatov
Neural-network repositories contain executable models, recipes, input transformations, and measured accuracies. We study whether one same-family experiment can be curated as
prompt guidance for LLM-based improvement of a low-performing target under equal generation and evaluation budgets. TuneNNGen extends NNGPT with a source-guided route and compares it with target-only generation on one CIFAR-10 target, two fixed source-selection rules, three code LLMs, and an additional SVHN target. Under the historical one-epoch search protocol, best-of-budget accuracy on the available evaluation split rises from 23.98% to 50.49% on CIFAR-10 and from 22.54% to 78.80% on SVHN. Selected five-epoch, three-seed means on train-derived validation splits retain gains of 40.94 points on CIFAR-10, 18.83 on Imagenette, and 7.27 on CIFAR-100. Direct-copy and negative-control analyses show that gains depend on source-target compatibility and LLM adaptation; stored source accuracy alone does not predict transferability.
Read the full abstract ▾
Computer Vision
Robust Face Super-Resolution and Recognition Through Multi-Feature Aggregation in Diffusion Models
Marcelo dos Santos, Rayson Laroca, João Carlos Raposo Neves et al. (+1 more)
Images acquired in surveillance environments often suffer from conditions such as low resolution, variations in pose, irregular illumination, and occlusions. Due to the low quality of these images, face recognition
algorithms often struggle. This major limitation can be addressed by employing super-resolution techniques that enhance the details of the image. However, due to the high degree of difficulty of the problem, most super-resolution
algorithms tend to cause distortions in the image and in the individual's identity. Thus, additional information must be incorporated into the processing to improve recognition robustness. In this regard, surveillance cameras can capture multiple images, even at low quality, and the data extracted from these images, such as consecutive
video frames, can significantly enhance both super-resolution and facial recognition. In this work, we introduce FASR++, a
diffusion-model-based super-resolution
algorithm. It leverages a reference low-resolution image and features extracted from multiple auxiliary low-quality images to generate a super-resolved output, minimizing distortions in the individual's identity. Our approach recovers facial features without explicitly providing soft attributes or computing a function gradient to guide the reconstruction process. FASR++ generates high-quality images that can considerably improve performance in face recognition tasks when used as a pre-processing step. We validate our approach on two standard face recognition
datasets and attain state-of-the-art results for verification, face recognition, and image quality metrics such as PSNR, SSIM, and LPIPS.
Read the full abstract ▾
All authors Marcelo dos Santos, Rayson Laroca, João Carlos Raposo Neves, David Menotti
Computer Vision
Clustered Codebook Quantization for 2D Gaussian-based Image Compression
Runze Cheng, Yicheng Zhan, Josef Spjut et al. (+1 more)
Gaussian-based image representations effectively model image content using compact parametric primitives while preserving high visual fidelity, yet storing a large number of floating-point parameters per primitive degrades rate-distortion efficiency at higher fidelity targets. To improve the rate-distortion performance in Gaussian representation, we present our Cluster-Guided Vector Quantization (CGVQ), a Gaussian primitive based image compression method. Our key idea is to partition Gaussian parameters further into homogeneous groups prior to quantization, enabling higher compression efficiency and accurate parameter reconstruction. In practice, our extensive experiments show that CGVQ decreases the bpp by 20% with respect to our baseline, while maintaining on-par visual quality
Read the full abstract ▾
All authors Runze Cheng, Yicheng Zhan, Josef Spjut, Kaan Akşit
Computer Vision
REVIVE: A Multi-Modal Framework for Vandalism Detection and Recovery in Autonomous Vehicles
Abdullah Tariq Choudhry and Tapadhir Das
Autonomous vehicles (AVs) face increasing threats from vandalism-induced occlusion attacks (VOAs) that compromise camera-based perception. While detection frameworks can identify vandalized images, restoring camera-stream utility after physical occlusion remains underexplored. This paper presents present the Recovery and Enhancement of Vandalized Images for
Vision Excellence (REVIVE) framework, a vandalism recovery pipeline integrating: (1) binary VOA detection, (2) multi-class VOA pattern identification, (3) EfficientNet-based U-Net
segmentation, and (4) type-aware recovery using Bootstrapping
Language-Image Pre-training (BLIP)-guided Stable
Diffusion inpainting, direct pixel replacement, or adaptive median filtering. Stable
Diffusion shows variable reconstruction performance (per-pattern SSIM 0.667-0.867, PSNR 15.4-26.7dB) across VOA patterns, while aligned direct pixel replacement achieves near-identical reconstruction under the aligned-reference condition. On 500 tracked clean/vandalized image pairs, unrecovered VOAs reduce YOLOv8l object-detection recall to 0.588, while direct pixel replacement restores recall to 0.967 and F1-score to 0.970 under that aligned-reference condition. LaMa, Telea, and Navier-Stokes baselines improve image similarity but provide more limited downstream detection recovery, and Stable
Diffusion is treated as an asynchronous recovery branch subject to a quality gate rather than a blocking real-time perception step. We evaluate a reference-available quality gate that filters recovered candidates before downstream use: without it, type-aware routing degrades per-image recall to 0.304, whereas with it, recall returns to 0.608, at or above the unrecovered baseline, ensuring the forwarded stream is never worse than the unrecovered frame. REVIVE therefore, provides a structured recovery framework from VOAs in AVs.
Read the full abstract ▾
Computer Vision
VEIL: How Visual Encoding Hijacking Induces Bias In Vision Models
Suranjana Sooraj, Xuyang Chen, Madhumitha Venkatesan et al. (+1 more)
Rendering time series as chart images for CNN-based classification has become increasingly common in time-series classification (TSC). However, it remains unclear whether models learn underlying temporal patterns or rely on encoding-specific visual cues introduced by chart design. We present VEIL: a systematic study examining how chart encodings influence learned representations through complementary analyses of similarity, transferability, and attribution.
Attention-guided training appears to mitigate this effect when encoding sensitivity is consistently identified across diagnostics, but provides limited or negative benefit when such signals are absent. These findings position VEIL within the broader question of how machines perceive visualizations – extending graphical perception from human readers to
vision models – and show that visualization design choices shape learned representations in ways that warrant treating chart-based TSC as a representation and measurement problem rather than a simple modeling decision.
Read the full abstract ▾
All authors Suranjana Sooraj, Xuyang Chen, Madhumitha Venkatesan, Dongyu Liu
Computer Vision
Recovering Cloud Microstructures with Cascaded Diffusion Inversion
Hanan Gani, Guy Pulik, Daniel Rosenfeld et al. (+2 more)
High-resolution satellite imagery is critical for observing fine-scale cloud structures that inform weather modification strategies like cloud seeding for rain-enhancement. However, the spatial resolution of current geostationary and polar-orbiting satellites is often insufficient for capturing small cloud features. Current super-resolution methodologies are suited for natural images and, therefore, struggle to generalize to satellite-captured spectral images of cloud cover. To address this, we propose a two-stage
diffusion-based super-resolution framework to enhance the resolution of multi-spectral cloud microstructures by a factor of $4\times$. Specifically, we use inverse
diffusion to recover the high resolution properties from low resolution. Stage 1 utilizes real-world paired data to learn robust degradation handling and inter-sensor
alignment, while Stage 2 employs a self-supervised internal downgrading of high resolution data to refine structural learning and texture synthesis. Our approach outperforms the state-of-the-art
transformer and
diffusion-based baselines in both reconstruction accuracy and visual quality. We demonstrate that the two-stage method better captures fine cloud microstructures (e.g. convective turrets and cloud gaps) that are crucial for effective cloud seeding decisions. Ablation studies confirm the complementary benefits of the two stages: Stage 1 excels in coarse structural fidelity, while Stage 2 contributes enhanced detail and realism. These results highlight a practical path toward improving cloud microphysics analysis and as a step towards utilizing AI for
climate and sustainability. Our code and models are publicly available at: https://github.com/hananshafi/superresolution-cloud-microphysics.
Read the full abstract ▾
All authors Hanan Gani, Guy Pulik, Daniel Rosenfeld, Duncan Watson-Parris, Salman Khan
Computer Vision
Taxlifier: Leveraging Disease Taxonomy for Enhanced Multi-Label Classification in Chest Radiography
Mohammad S. Majdi and Jeffrey J. Rodriguez
Accurate and efficient classification of thoracic diseases in chest X-ray (CXR) images is crucial for timely diagnosis and treatment. However, the presence of multiple pathologies with overlapping visual characteristics poses significant challenges for automated classification systems. In this study, we propose two novel hierarchical multi-label classification techniques, namely the loss-based and logit-based methods, to address these challenges by leveraging the hierarchical relationships among different thoracic pathologies. The loss-based technique integrates hierarchical information directly into the optimization process, while the logit-based method adjusts the predicted probabilities of each class based on its parent class in the disease taxonomy. We evaluate the performance of both techniques using three large-scale CXR
datasets: CheXpert (224,316 CXRs), PADCHEST (160,000 CXRs), and NIH (112,120 CXRs). The experimental results demonstrate significant improvements in accuracy, AUC, and F1 scores compared to the baseline method across various pathologies. The logit-based and loss-based methods improve accuracy by 12% and 11%, AUC by 13% and 10%, and F1 scores by 24% and 12%, respectively compared to the baseline. These results represent a substantial improvement over the baseline method. Furthermore, we conduct a comprehensive statistical analysis to validate the robustness and reliability of the proposed techniques. The integration of domain-specific hierarchical knowledge not only enhances the classification performance but also provides a more interpretable output for clinical decision support. Our findings highlight the potential of hierarchical multi-label classification in advancing computer-aided diagnosis systems for chest radiography.
Read the full abstract ▾
Computer Vision
Cross-Contextual Vision-Language Adaptation with LoRA for Personalized Severe Adverse Event Detection in Clinical Wound Monitoring
Aditi Naiknaware, Jian Sun, Aminreza Khandan et al. (+4 more)
Wound monitoring is a critical yet underserved clinical challenge, where timely identification of severe adverse events (SAEs) such as infection, tissue deterioration, and delayed healing can significantly impact patient outcomes. While
vision-
language models (VLMs) show strong
multimodal reasoning, they often lack domain-specific grounding to integrate wound imagery with heterogeneous clinical information, and provide limited mechanisms for detecting cases that diverge from the training distribution. We present a
multimodal framework for automated wound monitoring and SAE detection. Our approach leverages paired clinical notes and wound descriptions capturing visual characteristics such as appearance, surrounding skin condition, color changes, and signs of inflammation or healing progression, encoded through a dual-stream Low-Rank Adaptation (LoRA) framework built on a frozen BiomedCLIP backbone. We introduce a cross-contextual LoRA fusion mechanism enabling information exchange between clinical semantics and visual wound descriptors, producing context-aware
multimodal representations without full model fine-tuning. To identify personalized SAEs, we propose a wound-specific out-of-distribution (OOD) detection framework combining semantic matching, visual typicality, caption-text
alignment, and caption-visual
alignment into a unified SAE (OOD) score. To capture healing dynamics, we incorporate
covariate consistency and temporal drift penalties that leverage changes in wound characteristics across visits. Experiments on a longitudinal wound
dataset collected through clinical visits show promising performance on both wound healing assessment and SAE detection, highlighting the potential of semantically enriched, temporally aware
vision-
language systems for clinical wound monitoring and early risk identification.
Read the full abstract ▾
All authors Aditi Naiknaware, Jian Sun, Aminreza Khandan, Shengyang Huang, Sean Dow, Bijan Najafi, Salimeh Sekeh
Computer Vision · Cross-listed from cs.CL
BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension
Abu Tyeb Azad, Ishita Sur Apan, Fahim Ahmed et al. (+8 more)
Document comprehension is a challenging yet impactful task for
Multimodal Large
Language Models, especially as these systems see growing adoption in real-world, human-centric applications. However, this adoption is limited for low-resource
languages such as Bangla due to the scarcity of high-quality annotated data. To address this gap, we introduce BaFCo, a
benchmark dataset for Bangla form comprehension with a focus on Document Layout Analysis (DLA) and Key Information Extraction (KIE). BaFCo curates 200 multi-page complex Bangladeshi government forms, sourced from across diverse sectors including agriculture, education, banking, and land management. To accurately capture the structural and contextual complexity of these forms, we define a fine-grained annotation schema comprising 26 types of form entities, along with a separate coarse form entity set consisting of 5 types. We evaluate the latest MLLMs from the ChatGPT, Gemini, Claude, Qwen, and Kimi series using zero-shot and chain-of-thought
prompts under both low and high
reasoning setups. Our results reveal limitations in current MLLMs' ability in comprehending Bangla forms, particularly in accurately localizing highly granular form entities. Our
dataset and code is available at: https://huggingface.co/
datasets/Mausul/bafco
Read the full abstract ▾
All authors Abu Tyeb Azad, Ishita Sur Apan, Fahim Ahmed, Sumaiya Karim Katha, Ezharuddin Jubaer, Armun Alam, Pranjal Kumar Nandi, Amin Ahsan Ali, Aman Chadha, Md Mofijul Islam, AKM Mahbubur Rahman
Computer Vision
Patch Knowledge Transfer for Efficient AI-Generated Image Quality Assessment
Jiquan Yuan
With the rapid advancement of image generation technologies, perceptual quality assessment of AI-generated images has emerged as a crucial research direction in computer
vision. The core challenge of this task lies in achieving efficient quality assessment for massive generated images. Current mainstream approaches exhibit two key limitations: 1) Methods employing complex feature extraction strategies, while improving performance, incur prohibitive computational costs that hinder real-time inference; 2) Simple image scaling-based solutions, despite their computational efficiency, demonstrate significantly inferior assessment accuracy. To address this critical issue, we propose Patch Knowledge Transfer (PKT), a knowledge
distillation-based optimization framework that achieves synergistic optimization of visual representation capability and inference efficiency through an innovative multi-level knowledge transfer mechanism. Specifically, we design a dual-model architecture: a teacher model with local-global hybrid processing provides high-quality supervision signals, while a student model relying solely on global processing efficiently inherits the teacher's representation capacity through multi-level supervision. Extensive experiments conducted on 4 AIGIQA databases demonstrate that the PKT framework enables the student model to maintain performance comparable to the teacher while reducing computational costs by 67.7%. Furthermore, compared to existing methods, our approach achieves a superior balance between model efficiency and assessment accuracy.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
SSA-3DGS: Unsupervised Removal of Screen-Space Artifacts for 3D Gaussian Splatting
Kristof Overdulve, Lode Jorissen and Nick Michiels
Novel View Synthesis (NVS) methods, such as 3D Gaussian Splatting (3DGS), rely on the assumption of clean, multi-view consistent, posed input images. Real-world captures can violate this assumption due to screen-space artifacts—static occlusions fixed to the 2D image plane rather than to the 3D world. Common examples include physical sensor defects, environmental obstructions (such as rain or mud on the lens enclosure), capture obstructions (such as a thumb over the camera sensor or a dashboard visible in dashcam footage), and digital overlays (such as watermarks or UI elements). When present, they are erroneously baked into the 3D geometry as “floaters” or near-camera artifacts, degrading the quality of novel-view rendering. In this work, we propose SSA-3DGS, an unsupervised framework that jointly optimizes a 3D scene and a learnable 2D overlay to recover a clean 3D scene and the corrupting artifacts. By exploiting
geometric consensus across views, our method effectively disentangles static artifacts from the 3D scene geometry without supervision or manual input. Across diverse synthetic corruptions and a self-captured real-world
dataset, SSA-3DGS improves reconstruction fidelity by up to ${\sim}8$~dB PSNR over 3DGS trained on the same corrupted inputs, while faithfully preserving the corrupting artifact.
Read the full abstract ▾
Computer Vision
Hierarchical Classification via Cascading Feature Elimination: Application to Human Phenotype Ontology-Aligned Facial Phenotyping (FaceMesh2HPO)
Fabio Hellmann, Alexander Hustinx, Benjamin D. Solomon et al. (+4 more)
FaceMesh2HPO is a framework for classifying facial phenotypic descriptors aligned with the Human Phenotype Ontology (HPO) to support clinical diagnosis. Using annotations from 124 clinicians across 10 disorders (107 HPO terms) combined with non-syndromic controls, we generated 3D facial meshes (478 landmarks) from 2D images and trained a hierarchical PointNet-based pipeline with cascading classification and feature elimination. The best models, incorporating 3D meshes, facial outline, and demographic metadata, achieved AUROCs between ~0.55 and ~0.89, with higher performance at parent nodes than leaf terms. External validation showed variable generalizability across disorders. Results demonstrate that hierarchical modeling of 3D facial geometry enables interpretable, ontology-linked phenotype classification, though performance on rare leaf terms remains limited. Improved data diversity and feature selection strategies are needed to enhance robustness and clinical utility.
Read the full abstract ▾
All authors Fabio Hellmann, Alexander Hustinx, Benjamin D. Solomon, GestaltMatcher Database Consortium, Tzung-Chien Hsieh, Peter Krawitz, Elisabeth André
Computer Vision
Prompting Image Generators for Training-free Primitive Shape Abstraction
Gregor Kobsik, Tim Elsner and Leif Kobbelt
Compact primitive abstractions represent 3D shapes with a few
geometric primitives while preserving recognizable components. Learned methods depend on their training classes, and optimization-based methods split shapes geometrically rather than into parts. We instead reuse the visual part knowledge of pretrained models without task-specific training or fine-tuning. A
vision-
language model names parts in multi-view renders, and an unmodified image generator paints color-coded part masks. Reprojection and spatial clustering recover 3D instances, and a classical optimizer fits one tapered and bent superquadric per part. With five to eight primitives per object, the abstractions match the Chamfer distance of the strongest learned baseline on HumanPrim, improve on it by 10% on Toys4K, and have the lowest overlap among compact methods, while chair legs, backrest bars and wheels remain separate primitives. Our accuracy also transfers better than theirs to objects outside the learned methods' ShapeNet training classes. Replacing the generated masks with part labels from the 3D
segmentation methods P3-SAM or PartField lowers IoU by 7 to 17 points under the same fitter. Further studies relate the remaining volumetric error to part granularity and to parts that the rendered views observe from one side only.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory
Hu Zhu, Bohan Li, Xianda Guo et al. (+8 more)
Embodied agents exploring indoor environments require reliable semantic occupancy
memory that persists across observations and revisits. Building such
memory is challenging because each observation provides incomplete and uncertain
geometric and semantic evidence. We introduce GEM-Occ, a Gaussian Evidence
Memory framework that consolidates evidence accumulated over time into persistent semantic occupancy
memory. Local predictions are converted into occupied semantic Gaussians and free-space ray evidence. Confidence- and visibility-aware causal updates integrate supporting observations, suppress occupancy contradicted by observed free space, and preserve previously observed structures through occlusion. A hierarchical
memory organization supports continued mapping and efficient queries across connected indoor spaces. To evaluate this capability, we introduce HIOcc, a unified
benchmark for
embodied semantic occupancy
memory. HIOcc establishes a shared semantic label space and evaluation framework spanning local prediction, room-level online mapping, and building-level mapping, while accommodating perspective and panoramic observations. Experiments on HIOcc demonstrate that GEM-Occ outperforms existing methods, enabling accurate semantic occupancy prediction and consistent online mapping across spatial scales with efficient
memory usage and fast occupancy queries.
Read the full abstract ▾
All authors Hu Zhu, Bohan Li, Xianda Guo, Yanlun Peng, Hongsi Liu, Baorui Peng, Xiaofeng Wang, Mingqi Yuan, Xin Jin, Wenjun Zeng, Chang Wen Chen
Computer Vision
Multi-Teacher Contrastive Distillation for Edge-Efficient Pathology Foundation Models
Tim Lenz, Maurice Heide, Marco Gustav et al. (+2 more)
Computational pathology foundation models (PFMs) have advanced whole-slide image analysis. However, their size and inference cost hinder local deployment in pathology departments. We propose MuCoDi, a pretraining framework that distills frozen tile
embeddings from multiple PFMs into compact edge-oriented encoders. Instead of regressing individual teacher features, MuCoDi trains lightweight MobileOne and RepViT students with a contrastive
distillation objective adapted from MoCo v3, where cached Virchow2, UNI2, and H-Optimus-1
embeddings replace momentum-encoder keys. We pretrain students on 14.3M TCGA tiles from only 11.8K WSIs and evaluate frozen encoders on 23 clinically curated downstream classification tasks. RepViT-based MuCoEdge students retain near-teacher performance while reducing model size by orders of magnitude: MuCoEdge-R2.3 and MuCoEdge-R1.5 reach 71.0% external AUROC, within 0.8 percentage points of the best teacher (Virchow2, 71.8%), while MuCoEdge-R2.3 obtains the best external F1 and the second-best AUPRC (51.8% and 53.3%). MuCoEdge-R1.0 reaches 70.9% AUROC with only 6.4M parameters and 1.12 GFLOPs. On a Raspberry Pi 5, sub-million-parameter MobileOne students achieve up to 605-fold single-tile speedup over Virchow2 while retaining 66.5% to 66.9% external AUROC, demonstrating that PFM-quality pathology representations can be moved toward practical edge deployment. Code is available at https://anonymous.4open.science/r/mucodi-6243.
Read the full abstract ▾
All authors Tim Lenz, Maurice Heide, Marco Gustav, Nic G. Reitsam, Jakob Nikolas Kather
Computer Vision · Cross-listed from cs.LG
$\mathbf{\lambda}$-VAE: Variance Equalization for Posterior Collapse
Girum Demisse
Variational Autoencoders (VAEs) frequently suffer from posterior collapse, a failure mode in which the approximate posterior converges to the prior, rendering the latent code uninformative. Despite extensive research, a unified account of why collapse occurs has remained an open question. We identify and formalize two logically independent but coupled causes. Gradient imbalance occurs when the decoder's reconstruction signal vanishes faster than the $\mathbb{KL}$ regularization pressure as the posterior widens. Information gap occurs when the
stochastic sampling step discards a substantial fraction of the encoder's computed representation, attenuating decoder sensitivity and making collapse inexpensive. Both causes share the same collapse trajectory, and we show that the information gap is algebraically equivalent to mismatch between the aggregate posterior and the prior, unifying two pathologies. Subsequently, we introduce $\lambda$-VAE, which resolves both causes through a single modification to the reparameterization step: the sampling noise is scaled by per-dimension exponent, while the $\mathbb{KL}$ penalty retains the original posterior variance. This asymmetry shifts the stable training attractor away from the degenerate collapsed state, driving all latent dimensions toward the same
equilibrium – a mechanism we term variance equalization. A closed-form optimal exponent per dimension follows from a net information gain objective, with a single hyperparameter controlling the reconstruction-generation tradeoff. We validate on standard
benchmarks (Binary MNIST, Binary Omniglot, CIFAR-10, CelebA-64), showing consistent reductions in collapsed dimensions, information capacity gains of up to $2.8\times$ nats, and reconstruction quality improvements of up to $+0.33$ BPD.
Read the full abstract ▾
Computer Vision
Rendering-Aware Bayesian 3D Gaussian Splatting with Native Uncertainty and Adaptive Complexity Control
Gaoxiang Jia, Vikram Appia, Junzhou Huang et al. (+1 more)
3D Gaussian splatting (3DGS) is a strong representation for real-time novel-view synthesis, but its standard training pipeline relies on point estimates and hand-tuned heuristics, providing no native uncertainty or principled complexity control. This is most limiting under sparse views or fixed acquisition budgets, where a model must identify weakly supported geometry and select informative views. We introduce a rendering-aware
Bayesian 3DGS framework that tracks Gaussian geometry with a Normal-Inverse-Wishart posterior over means and covariances using renderer-derived surrogate summaries. An optional Dirichlet-process extension adds a probabilistic component-usage signal, and the training schedule makes the closed-form versus approximate inference boundary explicit. Re-rendering posterior geometry samples yields native predictive uncertainty for interval calibration and active view selection. In a fixed-budget 16-to-32 active-view task, native NIW acquisition improves PSNR by +0.453 dB and LPIPS by -0.0146 over a scoring-only 3-member standard-ensemble baseline, winning 29/39 scene-seed pairs and 10/13 scene means; it also improves over PPU-style (+0.355 dB) and NIW-proxy (+0.401 dB) acquisition. NIW native intervals reduce 95% coverage error by about 17x relative to a shared proxy (0.046 vs. 0.796) and are about 10x closer to nominal coverage than a 3-member deep ensemble (0.047 vs. 0.454) at roughly one-third the training cost. As a reconstruction compatibility check, paired NIW-vs-standard analysis over 39 scene-seed runs yields +0.030 dB PSNR with 1.6% additional training time. These results position
Bayesian 3DGS as a practical probabilistic scene representation for decision-facing tasks such as active view selection.
Read the full abstract ▾
All authors Gaoxiang Jia, Vikram Appia, Junzhou Huang, Xinlei Wang
Computer Vision
Paul K. Mandal, Pavan Reddy and Tristan Malatynski
Model-specific
adversarial attacks have been extensively studied. We study a different failure mode: naturally occurring statistical signals in
vision data that can behave as
backdoor-like triggers without being maliciously inserted. We call these signals statistical adversaries. We analyse ImageNet to find patterns that are strongly linked to certain labels. We then use statistical controls to remove random correlations from our candidate signals. Finally, we demonstrate that these signals directly and predictably alter model predictions. These statistical adversaries are more targeted than generic corruptions and transfer across different model architectures. This suggests that some vulnerabilities are driven by
dataset structure and distribution rather than a single model's idiosyncrasies. We conclude that ordinary
datasets can contain exploitable
adversarial surfaces even in the absence of poisoning, and suggest that
dataset audits should treat spurious structure not only as a source of bias or interpretability failure, but also as a latent attack surface for
vision models.
Read the full abstract ▾
Computer Vision
Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory
Chang Nie, Jiaju Wei, Junlan Feng et al. (+2 more)
Agentic
video understanding equips models with long-term
memory to autonomously process and respond to continuous, long-horizon
multimodal streams. However, advanced
video agents often rely on “detective-style” iterative
reasoning for action control (e.g., $\mathtt{search}$) and evidence aggregation, incurring prohibitive costs and latency. We argue that such heavy
reasoning primarily compensates for the lack of global context and semantic misalignment in
retrieval. This paper introduces Light-Omni, a
multimodal agent framework for reflexive and lightweight
video understanding. It achieves this through dual contextual states that instantly build the required context in a single forward pass. First, we maintain a global state, a finite-sized
multimodal script continuously consolidated from episodic
memory, serving as the global context for Light-Omni. Through hierarchical merging, it preserves recent details while summarizing past events. Second, conditioned on this global context, we generate a parametric latent state that directly drives autonomous actions and produces
retrieval embeddings, with minimal latency. Benefiting from this coupled design, Light-Omni achieves semantically aligned
retrieval and reflexive responses while avoiding iterative
reasoning. Extensive experiments validate the effectiveness of Light-Omni across multiple
video benchmarks. Notably, it outperforms M3-
Agent with an average 2.4% accuracy gain, a 12.1$\times$ speedup, and a 2.6$\times$ improvement in GPU
memory efficiency. Furthermore, it serves as a
memory system to enhance both the performance and efficiency of existing MLLMs. Project page: https://clare-nie.github.io/Light-Omni.
Read the full abstract ▾
All authors Chang Nie, Jiaju Wei, Junlan Feng, Chaoyou Fu, Caifeng Shan
Computer Vision
Ground3D-LMM: Fine-Grained 3D Point Grounding and Spatial Reasoning with LMM
Amol Harsh, Zongyan Han, Jean Lahoud et al. (+5 more)
Natural-
language queries about 3D environments become actionable when responses are verifiable and metric. Verifiability requires explicit grounding to the referred 3D region, while metric answers report physical measurements in real-world units (e.g., size, thickness, clearance, and distance). Existing 3D large
multimodal models (LMMs) approaches remain limited: conversational systems typically respond without explicit 3D grounding, while 3D grounding models are not designed for interactive, metric-aware dialogue. In this paper, we present Ground3D-LMM, a unified model that takes a point cloud and an optional RGB image as input and supports 3D spatial conversation with (i) point-grounded responses and (ii) metric numeric outputs at both object and part granularity, including multi-object queries. To evaluate this intersection of grounding and measurement, we define the 3D Grounded Measurement task, which requires predicting the referred 3D region and the corresponding metric quantities in real-world units. We introduce a large-scale
dataset built on ScanNet and ScanNet++
datasets with dense object and part annotations and roughly 2.5M question-answer pairs spanning eight tasks, along with a manually verified test set. Extensive experiments on multiple
datasets and tasks show that our proposed Ground3D-LMM model provides a strong baseline for grounded, metric-aware 3D conversational understanding. Our
dataset and model are publicly available.
Read the full abstract ▾
All authors Amol Harsh, Zongyan Han, Jean Lahoud, Ye Liu, Rao Muhammad Anwer, Hisham Cholakkal, Salman Khan, Fahad Khan
Computer Vision
Binocular Gaze Estimation with Single Camera and Single Light Source
Tongbing Huang, Yang Fu, Yunfei Wang et al. (+1 more)
According to commonly consented theories, the minimum hardware requirement for gaze tracker is one camera and two light sources to realize gaze estimation with free head movements. However, in some scenarios such as eye tracking on mobile devices, it is preferable to use less components, especially light sources. We propose a gaze estimation method with one camera and one light source. A "virtual light source" is introduced, which is geometrically placed symmetrically to the real light source with respect to the camera, and generates a "virtual glint" in the acquired image. We estimate the "virtual glint" by exploiting the relationship between the distance between two pupils and two glints in the captured image, and estimate the gaze with
polynomial regression assuming two light sources are available. A new normalization factor for regression method is verified, which turns out to be practical for one-glint system. The performance is proved to be acceptable, while degradation is noticed compared to system with two actual light sources.
Read the full abstract ▾
All authors Tongbing Huang, Yang Fu, Yunfei Wang, Zhaocan Wang
Computer Vision
A Task-Driven Evaluation of UAV Detection and Tracking under Synthetic Fog
Amir Pouladi, Vesal Ahsani, Haijun Li et al. (+2 more)
Fog severely degrades the visibility of small unmanned aerial vehicles (
UAVs) in skydominant, long-range imagery, reducing the reliability of downstream detection and tracking. This paper presents a task-driven evaluation framework that links depth-aware synthetic fog generation, image restoration, object detection, and tracking within a unified pipeline. Given the practical difficulty of collecting and annotating foggy
UAV scenes, synthetic fog is generated from real clear-weather outdoor images containing
UAV targets using
monocular depth estimation and the atmospheric scattering model. Representative restoration methods from classical, convolutional neural network (CNN)-based, and
transformer-based families are first compared, after which the selected restoration model is integrated into the downstream perception pipeline. Detection is evaluated under both clean-only and fog-inclusive training regimes using multiple detector variants, while tracking-by-detection is assessed on clean, foggy, and restored
video sequences. Beyond image-level restoration metrics, the study evaluates how fog and restoration affect detection robustness and tracking performance. The results show that fog substantially degrades both detection and tracking, primarily through increased missed detections. Fog-inclusive training provides the most consistent improvement in robustness, whereas test-time restoration is most beneficial when the detector has been trained only on clean imagery. These findings show that restoration quality does not necessarily translate into proportional gains in downstream perception and therefore should be evaluated jointly with detection and tracking performance.
Read the full abstract ▾
All authors Amir Pouladi, Vesal Ahsani, Haijun Li, Homayoun Najjaran, Afzal Suleman
Computer Vision
CanvasAgent: Enabling Complex Image Creation and Editing via Visual Tool Orchestration
Hairui Zhu, Yiying Yang, Tengjin Weng et al. (+5 more)
Complex image creation and editing often require more than a single generation or editing model. A user request may involve synthesizing images, localizing objects, segmenting regions, editing selected content, compositing intermediate assets, reading text, and enhancing the final result. Such tasks shift
multimodal agents from perception-augmented
reasoning to manipulation-centered visual creation, where tools must actively transform visual states rather than merely inspect them. However, existing
multimodal tool-use
agents are mostly optimized for perception, search, or domain-specific editing, and lack large-scale supervision for executable image-creation trajectories. In this paper, we introduce CanvasCraft, a large-scale
multimodal tool-use
dataset for complex image creation and editing, and CanvasAgent, a tool-augmented
multimodal agent that learns to orchestrate heterogeneous visual tools through multi-turn interaction. CanvasCraft contains 140K fully annotated executable trajectories and 10K RL task specifications. CanvasAgent is first trained with SFT to learn executable
reasoning-action trajectories, and is then optimized with GRPO using a hybrid reward that combines outcome- and process-level signals. During rollout, CanvasAgent inspects intermediate results, tracks visual assets, and adapts tool decisions to the evolving visual state. Experiments evaluate both final image quality and trajectory behavior, demonstrating the effectiveness of CanvasAgent and the proposed
dataset for complex multi-tool image creation workflows.
Read the full abstract ▾
All authors Hairui Zhu, Yiying Yang, Tengjin Weng, Ziyu Lu, Xiao Yao, Xiaoyang Ye, Lin Ma, Wenhao Jiang
Computer Vision
From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model
Wenhao Li, Xueying Jiang, Quanhao Qian et al. (+4 more)
Real-world
robot deployment rarely maintains the training-stage camera setup, where cameras often experience repositioning or remounting depending on actual scenarios. Existing view-robust
Vision-
Language-Action (VLA) policies tolerate such camera variations only when the camera extrinsics are explicitly provided, making them fragile and hard to use especially when view robustness is critical. We argue that the policy should not be told where the camera is, but rather figure it out by itself. To this end, we introduce Camera-Centric VLA (CamVLA), a new VLA model that decouples manipulation controls from camera geometry by predicting (i) a camera-centric end-effector action expressed in the local camera frame, and (ii) a 6-DoF hand-eye
matrix relating cameras to the
robot base. A deterministic
geometric transformation composes the two predictions into a
robot base-frame action. This disentangles how I should move in pose-independent camera-centric action generation from where I am looking from in camera-perspective
geometric grounding. The resulting policy is calibration-free, depth-free, and single-view, requiring only a single
monocular RGB image as the visual observation and task instruction at deployment. Evaluations in both simulation and real-world
robot data show that CamVLA consistently improves success rates across diverse unseen viewpoints. Project page: https://alibaba-damo-academy.github.io/CamVLA/.
Read the full abstract ▾
All authors Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao, Shijian Lu, Gongjie Zhang, Ran Xu
Computer Vision
SynCity 3000: Bootstrapping Scene-Scale 3D Diffusion
Paul Engstler, Iro Laina, Christian Rupprecht et al. (+1 more)
We present SynCity 3000, a framework for generating 3D scenes that are globally coherent while enabling fine-grained layout control. Building on the ability of current image-to-3D generators to produce complex 3D assets from a single image, we extend this capability to the scale of entire scenes by adapting the generator to be applicable as a convolutional operator. We achieve this by fine-tuning the model on scene-like data generated by a new synthetic data engine, which we propose to address the scarcity of 3D scene data for training. The convolutional generator is then applied to a dimetric image of the entire scene, generated from the user
prompt, resulting in 3D scenes of arbitrary size and complexity. Across diverse
prompts and layouts, SynCity 3000 produces large, coherent, and detailed scenes, addressing the shortcomings of prior approaches to 3D scene generation.
Read the full abstract ▾
All authors Paul Engstler, Iro Laina, Christian Rupprecht, Andrea Vedaldi