Computer Vision · Cross-listed from cs.NI
Real-Time Smart Retail Analytics through Fog Computing using the RetailLightNet Lightweight Model
Mayank Atreya, Navin Chhibber, Shilpa Ajay et al. (+2 more)
Smart retail systems now depend on the application of real-time
video analytics, which can track customer behavior, manage shelf inventory, and enhance in-store processes. Currently cloud-based applications of
video analytics face multiple challenges such as high latency of the communication itself,
privacy concerns surrounding transmitting raw
video, and the computational demands of the heavy
vision models used on the often limited network infrastructure. These challenges lead to delays in decision making where
prompt action is crucial. To address these short comings, this research proposes Fog-based analytics withRetailLightNet (F RLN), an ultra lightweight deep learning model which is specifically suited for the edge, resource constrained hardware capabilities of devices like the NVIDIA Jetson Nano. The approach utilizes publicly available retail
datasets, SKU110K, and RPC, which contains an assortment of product, shelf, and customer interaction scenarios for training.The processes include preprocessing images through normalization and employing motion-gated frame selection to improve efficiency by omitting redundant frames to ease the workload on the fog processing. Internal within Retail LightNet, image feature extraction is realized through depth wise separable convolutions which help support spatial and channel-wise learning without heavy computational costs. The model employs two prediction heads presence classification, and density regression, supporting shelf monitoring and queue assessment. The proposed F RLN framework attained better results in terms of precision (0.988), recall (0.976), and F1-score(0.976) when compared with existing convolutional neural network. Keywords: fog computing, motion-gated frame selection,NVIDIA jetson, retail lightnet, smart retail.
Read the full abstract ▾
All authors Mayank Atreya, Navin Chhibber, Shilpa Ajay, R Loshini, K Anitha
Computer Vision
Evaluation of Medical Vision Language Models HuluMed and MedGemma, and general purpose chatbots Gemma 3, ChatGPT Plus, and Claude Pro on real previously unseen wound images
Yunzhe Xue, Mohammed Saim Ahmed Quadri, Neal Panse et al. (+2 more)
Chronic wound assessment remains a clinically challenging task that requires accurate interpretation of wound morphology, tissue composition, vascular characteristics, and infection risk. Recent advances in
Vision-
Language Models (VLMs) have introduced the possibility of automated
multimodal wound analysis through image understanding combined with clinical
reasoning. This study evaluates the performance of several general-purpose and medically specialized open-source and proprietary VLMs for clinical wound assessment using an expanded, curated
dataset of 20 clinically diverse wounds spanning vascular,
surgical, ischemic, venous, lymphedema, and amputation-related etiologies. Six VLMs were evaluated using a structured twelve-question clinical framework covering wound classification, infection risk, vascular intervention recommendations, debridement urgency, wound therapy selection, and advanced management planning. Across 20 wound cases and 240 clinician-graded wound-analysis decisions, ChatGPT achieved the highest overall performance with 174/240 correct responses (72.50%), followed by Claude with 149/240 (62.08%). Among the open-source and medically specialized models, HuluMed achieved the strongest performance with 96/240 correct responses (40.00%), followed by Gemma 3 (81/240, 33.75%), MedGemma 4B (62/240, 25.83%), and MedGemma 27B (42/240, 17.50%). The findings suggest that frontier general-purpose
multimodal systems currently demonstrate substantially stronger wound-analysis performance than medically specialized alternatives, highlighting the continued importance of broad
multimodal reasoning capabilities alongside domain-specific medical knowledge. Although current VLMs demonstrate promising potential for clinical decision support, substantial limitations remain in advanced wound-management
reasoning, procedural planning, and autonomous clinical reliability.
Read the full abstract ▾
All authors Yunzhe Xue, Mohammed Saim Ahmed Quadri, Neal Panse, Justin W. Ady, Usman Roshan
Computer Vision · Cross-listed from cs.GR
Multimodal Image Colorization: Quantifying the Impact of Text-Conditioned Guidance on Grayscale-to-Color Translation
Colten Reissmann and Hugo Garrido-Lestache Belinchon
Grayscale images are commonly found in historical photography restoration, medical imaging, and artistic media. However, automatically applying color to these images remains a significant challenge in computer
vision because many plausible colorizations can correspond to the same grayscale input. In this work, we quantify the effect of text conditioning on pixel-level and perceptual metrics for grayscale-to-color image models. Specifically, we compare two architectures, a U-Net and Stable
Diffusion 1.5, each tested with and without CLIP text conditioning while holding all other variables constant. Our results show that text conditioning improves PSNR by 5.6%, SSIM by 1.2%, and colorfulness by 36.6%, while reducing LPIPS by 7.6% in the U-Net tier. In the Stable
Diffusion tier, text conditioning improves PSNR by 5.8%, SSIM by 1.5%, and colorfulness by 0.6%, while reducing LPIPS by 11.3%. These results indicate that text conditioning provides consistent, measurable improvements to colorization quality across both architecture scales.
Read the full abstract ▾
Computer Vision
MIRAGE: Stealthy Visual Prompt Injection for Vulnerability Detection in Web Agents
Xuelong Dai, Jianyu Ma, Boyang Ma et al. (+3 more)
Multimodal Large
Language Model (MLLM)-based web
agents provide practical, high-precision solutions for visual browser automation; however, they inherently expand the attack surface, introducing novel
vision-based vulnerabilities. Existing
adversarial evaluations targeting these
agents frequently rely on permissive threat models and visually conspicuous artifacts. In this paper, we investigate a constrained vulnerability detection setting: a trusted web platform where the evaluator acts solely as an unprivileged third party, such as a merchant or advertiser, controlling only a semantically legitimate, spatially constrained region, such as an ad slot, a sponsored card, or a localized widget. Operating under these realistic constraints, we propose MIRAGE, a novel visual indirect
prompt injection framework for targeted next-action hijacking. Our approach leverages
diffusion models to generate perceptually benign
adversarial images strictly confined to the attacker-controlled boundaries permitted by the trusted service provider. To maximize attack efficacy within such a restrictive setting, we introduce a robust optimization technique combining
curvature-aware
adversarial diffusion guidance with sparse, dark-pixel residual perturbations. Comprehensive evaluations against prominent MLLM web
agent frameworks, specifically SeeAct and OpenClaw, empirically demonstrate the potency, realism, and stealth of our proposed MIRAGE.
Read the full abstract ▾
All authors Xuelong Dai, Jianyu Ma, Boyang Ma, Biwei Yan, Yijun Yang, Yue Zhang
Computer Vision
CDER-SME: A Cross-Device Event-RGB Micro-Expression Dataset under Multi-Level Stress Induction
Jingting Li, Hui Sha and Su-Jing Wang
Micro-expression recognition (MER) in realistic scenarios demands high temporal sensitivity and ecological validity, yet existing
benchmarks are largely constrained to laboratory-controlled settings and rigid hardware-coupled sensing. We introduce CDER-SME, a cross-device Event-RGB
dataset collected under a multi-level stress induction framework (cognitive and social) to elicit spontaneous emotional leakage. To enable reproducible acquisition with independent, decoupled sensors, we provide a hardware-agnostic
alignment pipeline for temporal synchronization and landmark-guided spatial registration. CDER-SME adopts a three-tier structure with 92 subjects and 1,963 expert-annotated samples (Action Units and
emotions), including 790 Event-RGB pairs and 210 high-fidelity aligned pairs. We further report a reproducible
multimodal baseline, where cross-modal fusion improves performance over single-modality counterparts, supporting the complementarity of event dynamics and RGB cues. By removing the need for coaxial calibration, CDER-SME offers a practical
benchmark for cross-device
alignment and deployable Event-RGB MER in real-world affective intelligence.
Read the full abstract ▾
Computer Vision
Video2Code: Generating Interactive Webpages from UI Videos via Action-Aware Revisit
Mingde Xu, Zhen Yang, Yan Wang et al. (+7 more)
UI
videos provide a natural input for generating interactive webpages, as they capture both webpage appearance and action-triggered state transitions. However, directly applying
video-capable
vision-
language models to this task remains insufficient. Existing models typically rely on sparse sampling or compressed temporal representations, which may miss short action boundaries and break the state-action-state transitions needed to implement webpage behavior. We formulate UI
video-to-code generation as executable state-transition recovery from interaction
videos, and identify this failure mode as state-transition misalignment. We introduce
Video2Code, an action-aware
video-to-code approach for recovering executable UI state transitions. Rather than allocating the visual budget uniformly across the
video,
Video2Code first performs coarse
video understanding to locate action-critical regions, then invokes a temporal clipping tool to revisit these regions at higher temporal resolution before generating HTML/CSS/JavaScript code. We instantiate
Video2Code with action-aligned
video-code supervision and evaluate it under both visual and functional criteria. Experiments show that
Video2Code substantially strengthens the underlying open-source model for UI
video-to-code generation, improving functional correctness over direct
video observation, especially on dense multi-step interactions.
Read the full abstract ▾
All authors Mingde Xu, Zhen Yang, Yan Wang, Yu Wang, Xijun Liu, Zijun Dou, Wenyi Hong, Xiaotao Gu, Bin Xu, Jie Tang
Computer Vision
TeleStyle V2: Beyond Content-Preserving Style Transfer with Self-Distillation and Distribution-Matching-Distillation
Shiwen Zhang, Yifan Xu, Haibin Huang et al. (+2 more)
Given a content reference and a style reference, content-preserving style transfer requires the model to generate stylized outputs with content and style consistency. We introduced TeleStyle V1 to tackle this problem. However, TeleStyle V1 is trained with photorealistic content reference and artistic style reference, which makes it incapable to cope with artistic content reference and realistic style reference in most cases. In this paper, we designed a Self-
Distillation data synthesis strategy to construct such triplets from TeleStyle V1. Trained with such self-distilled triplets, our TeleStyle V2 supports Content-Style references in the forms of Realistic-and-Realistic (RnR), Realistic-and-Stylized (RnS), Stylized-and-Realistic (SnR), Stylized-and-Stylized (SnS). In addition, we found Distribution Matching
Distillation could preserve the general text-guided image editing capability of the foundation model and fix the content consistency degradation caused by SFT process. Through quantitative evaluations, our TeleStyleV2-QIE-2509-DMD performs at least on par with Qwen-Image-Edit-2509-DMD, demonstrating strong general image editing skills beyond content-preserving style transfer. We observed the content/style reference order confusion problem in TeleStyle V1 and further introduced
prompt enhancer to solve it. TeleStyle V2 uses Qwen-Image-Edit's VLM encoder, Qwen2.5-VL-7B, to generate content
prompt and style
prompt for free. TeleStyle V2 could achieve comparable style transfer performance with state-of-the-art commercial model, gemini-3-pro-image-preview.
Read the full abstract ▾
All authors Shiwen Zhang, Yifan Xu, Haibin Huang, Chi Zhang, Xuelong Li
Computer Vision
SP-TransientBench: A Real-Captured Single Photon Perception Benchmark
Hongzhou Dong, Zili Zhang, Ziting Wen et al. (+12 more)
Single-
photon LiDAR (SPL) based on single-
photon avalanche diode (SPAD) sensing enables time-resolved
photon measurements with extreme sensitivity, offering unique potential for active 3D perception in
photon-starved scenarios.However, real-world single
photon perception remains fundamentally challenging due to unique measurement noise and complex multi-return transient phenomena, which jointly complicate
geometric reconstruction and semantic scene understanding. Despite growing interest in SPAD-based sensing, existing studies are largely limited to simulated data or small-scale controlled captures. As a result, systematic evaluation of real-world single
photon perception across depth estimation, multi-view reconstruction, and 3D semantic understanding remains underexplored. To bridge this gap, we introduce SP-TransientBench (STB), a real-captured multi-task
benchmark for single
photon perception. SP-TransientBenc comprises 10 diverse scenes and 10,297 views captured using a solid-state single-
photon LiDAR at $256\times192$ resolution. Each view provides full time-of-flight histograms with multi-return behavior,standardized metadata, and calibrated camera poses for multi-view evaluation. We further provide 13-class 3D semantic annotations for selected scenes. By providing dedicated data splits and evaluation protocols for each task, STB enables consistent and reproducible benchmarking of real-world single
photon perception across multiple 3D
vision problems. The
dataset and code will be released upon acceptance.
Read the full abstract ▾
All authors Hongzhou Dong, Zili Zhang, Ziting Wen, Yiheng Qiang, Runrong Deng, Wenle Dong, Ziwen Jiang, Xinyang Li, Rui Lu, Shuoyao Sun, Wenyu Wang, Ziyi Xia, Haitao Zheng, Guodong Shi, Xiaoqiang Ren
Computer Vision
A Prototypical Signature Approach for Writer-Independent Offline Signature Verification
Kecia G. de Moura, Robert Sabourin and Rafael M. O. Cruz
Offline handwritten signature verification aims to distinguish genuine from forged signatures using static images. Since real forgeries are rarely available, negative samples are usually randomly drawn from genuine signatures of other users to create training data. However, this random selection often lacks diversity, increases redundancy, and escalates computational cost, leading to inefficient training. We propose a data-driven strategy to generate diverse, informative negative samples using prototypical signatures, which are compact, non-identifiable summaries of genuine signature features. Based on the experiments results, we conclude that (i) prototypical signatures yield more informative negative samples, improving the detection of skilled forgeries; (ii) the proposed approach is backbone-agnostic, showing robustness across architectures; and (iii) when combined with a primal-form linear SVM, it serves as an alternative to RBF-based models while significantly improving scalability and computational efficiency. Implementation of the method is available at https://github.com/kdmoura/proto_hsv.
Read the full abstract ▾
Computer Vision · Cross-listed from q-bio.QM
DART: A design-aware microfluidic chip paradigm for real-time live-cell image analysis
Johannes Seiffarth, Matthias Pesch, Lukas Scholtes et al. (+3 more)
High-throughput microfluidic live-
cell imaging generates rich single-
cell data. Yet semi-automated procedures for locating regions of interest (RoIs), each containing one
cell population, and removing surrounding microfluidic structures from recorded images, scale with the number of RoIs. This prevents real-time image analysis and delays time-to-insight by hours to days. We introduce the Design-Aware and Real-Time capable (DART) paradigm for microfluidic cultivation chips, which aligns the CAD blueprint with the physical chip and thereby enables throughput-independent localization of all RoIs and fully automated image processing across diverse RoI geometries and chip layouts. DART establishes this
alignment through embedded fiducial markers and deep-learning-based marker detection. We validate DART using the Swiss Army Knife chip, which combines eight structurally distinct RoI designs across 1164 RoI locations. DART localizes all RoIs in five minutes, removes microfluidic structures from raw microscopy images in 40 ms, and performs fully automated image analysis, including
cell segmentation, in under 1.1 s per image. Together, these capabilities establish DART as an end-to-end hardware-software paradigm with real-time-capable analysis that paves the way toward closed-loop and outcome-driven smart microscopy.
Read the full abstract ▾
All authors Johannes Seiffarth, Matthias Pesch, Lukas Scholtes, Dietrich Kohlheyer, Hanno Scharr, Katharina Nöh
Computer Vision
Architectural Bias in Face Presentation Attack Detection: A Comparative Study of Vision Transformers and Convolutional Neural Networks
Ngela Landon Ntung, Floride Tuyisenge and Jema David Ndibwile
Face Presentation Attack Detection (PAD) systems constitute a critical security layer in biometric authentication; however, existing approaches exhibit systematic performance disparities across demographic groups, disproportionately affecting individuals with darker skin tones. This paper presents a comparative empirical investigation of whether
Vision Transformer architectures reduce demographic bias in face PAD systems relative to convolutional baselines. Experiments are conducted on the CASIA-SURF Cross-Ethnicity Face Anti-Spoofing (CeFA)
dataset. Three architectures are evaluated: a
Multimodal ViT-Tiny trained from scratch, a ResNet18 CNN baseline, and a pretrained DeiT-S fine-tuned on CeFA across African, East Asian, and zero-shot Central Asian demographic groups. DeiT-S achieves the highest overall accuracy of 97.27% and the lowest EER of 0.86%, outperforming ResNet18 at 90.15% accuracy. In terms of
fairness, DeiT-S reduces the inter-ethnic ACER gap between African and East Asian subjects to 0.13%, compared to 0.75% reported in an LBP-based work [6], representing an 83% reduction. Most notably, while ResNet18 records a BPCER of 10.44% on zero-shot Central Asian subjects, DeiT-S maintains 2.89% on the same unseen group, demonstrating a 3.6x generalization advantage. These results suggest that pretrained
Vision Transformers achieve superior PAD accuracy, produce smaller demographic performance gaps, and generalize more equitably across unseen demographic groups, indicating that cross-demographic
fairness in PAD may partly be influenced by architectural design.
Read the full abstract ▾
Computer Vision
Neural Phase Correlation
Cole Reynolds
Correspondence is fundamentally relational: it seeks the unknown transformation between two observations of a common scene, not the content of either. Yet the dominant learning-based methods do not represent the transformation as a first-class object in the architecture. They encode each image independently and let a learned similarity function or a deep decoder discover the mapping implicitly. Phase correlation is the canonical exception, measuring the inter-image relationship directly in the
Fourier domain, but the
rigidity of its fixed basis confines it to global translation. We introduce a learned generalization of phase correlation that lifts this restriction by learning the basis on which the transformation decomposes. The same algebraic primitive extends to dense non-rigid deformations and to
unitary dynamics. On the ACDC cardiac-MRI
benchmark the framework matches or exceeds prior published baselines on both registration directions. On CAMUS echocardiography it matches state-of-the-art without auxiliary scoring or adaptive-smoothness mechanisms. Applied to time-evolved wavefunction pairs of the 1-D
quantum harmonic oscillator, the same framework recovers the Hermite-function eigenstates and the quantized energy levels of the unknown
Hamiltonian from observation pairs alone.
Read the full abstract ▾
Computer Vision
Vines-DB: An RGB image dataset for multi-species ornamental vine segmentation
Saroj Burlakoti, Utsav Bhandari, Aaron Etienne et al. (+1 more)
The Vines-DB
dataset contains 1,218 original high-resolution RGB images of seven ornamental vine species collected under field conditions at the Utah Agricultural Experiment Station's Greenville Research Farm in Logan, Utah, USA. The
dataset was generated from 168 individual vine plants that were transplanted in 2022 and photographed repeatedly across multiple months during the 2023 and 2024 growing seasons (July-October). Images were captured with an iPhone 16 Pro equipped with a 48 MP camera between 10:00 AM and 12:00 PM under daylight. Vines were grown on 1.2m x 2.4m trellises and photographed from a distance of 1m against black or white Styrofoam backdrops to improve contrast and reduce background noise. The
dataset includes Akebia quinata, Campsis radicans, Hydrangea anomala petiolaris, Lonicera x heckrottii, Campsis x tagliabuana 'Madame Galen', Parthenocissus quinquefolia, and Wisteria floribunda. All original images were manually annotated in Roboflow by trained annotators to produce polygon-based instance
segmentation masks for eight classes, including seven species and background. After preprocessing and data augmentation, the working
dataset was expanded to 2,307 images for model development and evaluation. The augmented
dataset was divided into 2,019 training images, 192 validation images, and 96 test images using stratified sampling to maintain balanced representation. Vines-DB supports the development and evaluation of deep learning models for multi-class instance
segmentation in precision horticulture and urban ecology. The
dataset enables applications such as automated canopy cover estimation, species identification, and scalable field phenotyping. In addition, repeated monthly imaging of the plants captures temporal variation in canopy development and plant appearance, increasing the
dataset's utility for
segmentation benchmarking under realistic field conditions.
Read the full abstract ▾
All authors Saroj Burlakoti, Utsav Bhandari, Aaron Etienne, Shital Poudyal
Computer Vision
Data-Forcing Distillation: Restoring Diversity and Fidelity in Few-Step Video Generation
Siyi Chen, Shaowei Liu, Yixuan Jia et al. (+4 more)
Recent progress has shown promise in distilling multi-step
video diffusion models into efficient few-step students. Among them, Distribution Matching
Distillation (DMD) and its successor DMD2 achieved strong generation quality and fast convergence. However, due to the nature of the reverse Kullback–Leibler (KL) objective, these methods exhibit two persistent failure modes: a substantial drop in sample diversity, and visibly over-saturated outputs that deviate from real-
video appearance. In this work, we propose Data-Forcing
Distillation (DFD), a simple post-training framework that restores diversity and fidelity in DMD with only a single-line of code change. At its core is the teacher score discrepancy to guide the student toward the real-data distribution, pulling it to missing modes (mitigating mode collapse) and away from problematic modes absent in real data (avoiding over-saturation). We provide an in-depth theoretical analysis of our framework and validate our approach on text-to-
video, image-to-
video, and autoregressive
video generation. With only 100–300 steps of finetuning, DFD effectively restores diversity and fidelity on both Wan2.1-1.3B and Cosmos-Predict2.5-2B model, resolving the over-saturation artifacts with significantly better
video dynamics and appearance, and even outperforms the teacher model.
Read the full abstract ▾
All authors Siyi Chen, Shaowei Liu, Yixuan Jia, Zian Wang, Huan Ling, Qing Qu, Jun Gao
Computer Vision
Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuning
Sneha Paul, Zachary Patterson and Nizar Bouguila
Domain adaptation remains a central challenge in 3D
vision, especially for
multimodal foundation models that align 3D point clouds with visual and textual data. While these models demonstrate strong general capabilities, adapting them to downstream domains with limited data often leads to overfitting and catastrophic forgetting. To address this, we introduce ReFine3D, a regularized fine-tuning framework designed for domain-generalizable tuning of 3D large
multimodal models (LMMs). ReFine3D combines selective layer tuning with two targeted regularization strategies: multi-view consistency across augmented point clouds and text diversity through synonym-based
prompts generated by large
language models. Additionally, we incorporate point-rendered
vision supervision and a test-time augmentation mechanism with confidence-based aggregation to further enhance robustness. Extensive experiments across different 3D domain generalization
benchmarks show that ReFine3D improves base-to-novel class generalization by 1.36%, cross-
dataset transfer by 2.43%, robustness to corruption by 1.80%, and few-shot accuracy by up to 3.11%, outperforming prior state-of-the-art methods with minimal added computational overhead.
Read the full abstract ▾
Computer Vision
Reasoning as Intersection: Consensus-Frame Alignment for Visual Focus in Video-MLLMs
Chengwen Liu, Zhe Huang, Jisheng Dang et al. (+3 more)
Reinforcement learning has improved the
reasoning ability of large
language models, but applying outcome-only rewards to
video multimodal large
language models (
Video-MLLMs) provides limited guidance on which visual evidence should support the answer. Inspired by multisensory integration, where consistent cues can enhance the salience and reliability of perceptual estimates, we introduce Consensus Frame GRPO (CF-GRPO), a temporal-annotation-free process-level reward framework for evidence-aware
video reasoning. CF-GRPO constructs a consensus frame prior from intrinsic
video cues, including temporal coverage, scene-transition cues, and query-conditioned visual relevance. It then computes a model-side frame-use score from visual and response representations and optimizes their agreement through the Consensus Frame Reward (CFR). With salience-aware sparse aggregation and distribution sharpening, CFR provides a high-contrast reward signal without requiring human temporal annotations. Experiments show that VideoCFR achieves competitive performance across complex
video reasoning benchmarks and improves several metrics over representative
Video-MLLM and RL baselines, while the consensus prior provides an interpretable view of the evidence frames emphasized during training. The implementation is available at https://github.com/1Pansy/VideoCFR.
Read the full abstract ▾
All authors Chengwen Liu, Zhe Huang, Jisheng Dang, Hong Peng, Qi Tian, Tat-Seng Chua
Computer Vision
RegimeVGGT: Layer-Wise Spatially Preserving Redundancy Removal for Visual Geometry Grounded Transformer
Jinhao You, Shuo Lyu, Zhuohang Lyu et al. (+5 more)
Visual Geometry Grounded
Transformer (VGGT) recovers dense 3D scene structure from multi-view images in one forward pass, but quadratic cross-frame
attention limits its scalability. Existing training-free accelerators exploit
token, frame, or
attention redundancy; we investigate how layer-wise functional roles can guide compression. Our spectral, probing, and causal analyses reveal three regimes: shallow layers lack cross-view structure, middle layers drive cross-view
alignment, and deep layers are redundant for dense geometry yet their cross-frame
attention remains essential for pose. RegimeVGGT applies layer-wise U-shaped compression along two axes: Saliency-Guided Banded Merging protects geometry- and edge-salient
tokens, while Selectively Protected K/V Downsampling preserves cross-frame spatial coverage and the pose-critical path through a phase-shifted spatial grid, a reference-frame anchor, and uncompressed camera/register
tokens. Without any training, RegimeVGGT achieves a 6.7x speedup over the
memory-optimized VGGT* at matched reconstruction quality. Project page: https://tracyou07.github.io/regimevggt/
Read the full abstract ▾
All authors Jinhao You, Shuo Lyu, Zhuohang Lyu, Tanxuan Li, Zibo Zhao, Jiaxiang Hu, Kai Tang, Yichen Guo
Computer Vision
CAOA – Completion-Assisted Object-CAD Alignment
Hiranya Garbha Kumar, Minhas Kamal and Balakrishnan Prabhakaran
Accurately aligning CAD models to their corresponding objects in indoor RGB-D scans is a central challenge in 3D semantic reconstruction. The task requires estimating a 9-Degree-of-Freedom (DoF) pose-position, rotation, and scale along three axes-but is hindered by noisy and incomplete scans, as well as
segmentation errors that cause
geometric distortions. We present Completion-Assisted Object-CAD
Alignment (CAOA), a method that integrates a semantically and contextually aware point cloud completion module with a
symmetry-aware relative pose estimation
algorithm, enabling precise
alignment of CAD models to scanned objects. Existing completion methods are typically trained and evaluated on synthetic
datasets, which often fail to generalize to real-world scans. To bridge this gap, we introduce a synthetic data generation strategy tailored to indoor scenes, significantly reducing the synthetic-to-real domain gap-validated through quantitative comparisons with widely used completion
datasets. In addition, we release S2C-Completion, an expert-annotated
dataset of over 8,500 object-CAD pairs from Scan2CAD, created for real-world indoor single-object completion and intended as a new
benchmark for this task. For object-CAD
alignment, we incorporate
symmetry information via a
symmetry-aware loss, improving robustness to symmetric ambiguities. On the Scan2CAD
benchmark, CAOA achieves a 17% accuracy improvement over state-of-the-art methods.
Read the full abstract ▾
Computer Vision
Budget-Aware Adaptive Adversarial Patches for Black-Box Object Detection
Pedram MohajerAnsari, Amir Salarpour, David Fernandez et al. (+1 more)
Adversarial patches pose a practical threat to modern object detectors. Prior work shows vulnerability, but three gaps limit actionable insight: (i) few score-based black-box attacks jointly optimize patch location, texture, and size under tight query budgets; (ii) success is rarely tied to the patch's visual footprint; and (iii) evaluations often conflate EOT robustness with plain-view suppression. We present \method{}, a query-efficient, budget-adaptive black-box attack that couples a lightweight Contextual Thompson-Sampling placer with NES-style pixel updates, growing the patch only when progress stalls. Reporting is anchored by a strict plain-image suppression test; EOT is audited but never used as a substitute for success, and optional appearance/printability weights expose strength–visibility trade-offs. Across YOLOv5, Faster R-CNN, and YOLOS, \method{} achieves strong suppression on CNN-based detectors and substantial suppression on the
transformer-based detector, using compact patches and exposing clear query–footprint trade-offs relative to fixed-size and heuristic baselines. A print–capture pilot further shows transfer across unseen physical objects and viewpoints.
Read the full abstract ▾
All authors Pedram MohajerAnsari, Amir Salarpour, David Fernandez, Mert D. Pesé
Computer Vision
Future Dynamic 3D Reconstruction: Toward 3D World Modeling with Disentangled Ego-Motion
Nils Morbitzer, Jonathan Evers, Artem Savkin et al. (+4 more)
Forecasting the
evolution of dynamic environments is crucial for autonomous
agents. While generative world models have achieved high photorealism in 2D
video synthesis by mixing ego-motion and environmental dynamics within the image plane, they exhibit physical inconsistencies, such as morphing or vanishing objects, especially over long time horizons. In this paper, we propose FR3D, a world-modeling approach that predicts a persistent 3D latent representation for future dynamic 3D reconstruction. Unlike prior works that treat the world as a sequence of image-based features, FR3D explicitly decouples the 3D
evolution of the scene from the
agent's trajectory, treating the inferred ego-motion as a latent proxy for action. This disentanglement resolves ambiguities between self-motion and world-motion, ensuring
geometric consistency into the future. Furthermore, we introduce a teacher-student
distillation strategy that leverages the spatial "common sense" of off-the-shelf foundation models, leading to robust zero-shot generalization. Extensive experiments demonstrate FR3D's strong performance for future dynamic 3D reconstruction from
monocular observations across multiple
datasets, even 2 seconds into the future. Project page: https://fr3d-wm.github.io.
Read the full abstract ▾
All authors Nils Morbitzer, Jonathan Evers, Artem Savkin, Thomas Stauner, Nassir Navab, Federico Tombari, Stefano Gasperini
Computer Vision
Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification
Wujian Peng, Lingchen Meng, Yuxuan Cai et al. (+7 more)
Unified
Multimodal Modeling aims to integrate visual understanding and generation within a single system. However, existing approaches typically rely on two disparate visual tokenizers, which splits the representation space and hinders truly unified modeling. We propose UniAR, a unified autoregressive framework where a single discrete visual tokenizer serves as the key bridge between understanding and generation, enabling a shared context in which the model can directly interpret its own generated visual
tokens without additional re-encoding. UniAR adapts a pretrained
vision encoder with multi-level feature fusion and a lookup-free bitwise quantization scheme, preserving both high-level semantics and low-level details while scaling the effective visual vocabulary at minimal cost. Building on this, the unified autoregressive model adopts parallel-bitwise-prediction to jointly predict spatially grouped, multi-level visual codes, substantially reducing visual sequence length and accelerating generation. Finally, a
diffusion-based visual decoder operates on discrete visual
tokens to decode high-fidelity images. Through large-scale pre-training, followed by supervised fine-tuning and reinforcement learning, UniAR achieves state-of-the-art performance on image generation and image editing while remaining competitive on
multimodal understanding
benchmarks. The project page is available at https://sharelab-sii.github.io/uniar-web.
Read the full abstract ▾
All authors Wujian Peng, Lingchen Meng, Yuxuan Cai, Xianwei Zhuang, Yuhuan Yang, Rongyao Fang, Chenfei Wu, Junyang Lin, Zuxuan Wu, Shuai Bai
Computer Vision
MOCHI: Motion Enhancement of Collaborative Human-object Interactions
Jiye Lee, Yonghun Choi and Jungdam Won
Collaborative human-object interaction shows dynamic and complex movements that require mutual anticipation and continuous adjustment between participants and the shared object. Modeling such collaborative multi-human object interaction (MHOI) scenarios requires high-quality data acquisition as a foundational step; however, this is challenging due to the inherent complexity of MHOI where human-human and human-object interactions occur simultaneously. Such complexity leads to noisy MHOI captures characterized by several artifacts: contact misalignment between hands and objects, motion jitter and temporal inconsistencies in the captured sequences, and missing or incomplete finger-level articulation details. To address these challenges, we present MOCHI (MOtion Enhancement of Collaborative Human-object Interactions), a two-stage framework for enhancing noisy MHOI data. Our approach first generates physically plausible hand grasps through optimization from noisy body input, producing grasps that are both physically plausible and semantically consistent with the body pose, where these optimized grasps are extended into complete hand-object interaction sequences. Consequently, the full-body motion for all participants are refined through a
diffusion-based noise optimization framework that uses single-person motion priors. During the optimization process, we introduce optimization objectives to encode human-object and human-human interaction information within these single-person priors. Experimental results demonstrate the effectiveness of our pipeline across diverse MHOI data, either acquired by existing capture methods or synthesized by generative models. We further show robustness of our system across varying numbers of participants and types of interactions, and demonstrate various applications including keyframe-based MHOI creation and data augmentation through varying object geometries.
Read the full abstract ▾
Computer Vision
EventDrive: Event Cameras for Vision-Language Driving Intelligence
Dongyue Lu, Rong Li, Ao Liang et al. (+6 more)
Event cameras sense the world through asynchronous brightness changes with microsecond latency and high dynamic range, offering motion fidelity far beyond frame-based sensors and capturing temporal structure that conventional exposures often miss. These properties make events a powerful complement to RGB in autonomous driving, especially under blur, glare, and rapid motion, where frame-based perception can become unreliable. However, existing event-aware
vision-
language models remain limited to generic perception and do not reveal how event sensing contributes to
reasoning and decision-making across the full driving loop. We present EventDrive, a large-scale
benchmark and model suite that unifies event streams, RGB frames, and
language supervision across four core dimensions: Perception, Understanding, Prediction, and Planning, covering captions, structured QA, grounding, motion-state recognition, trajectory
forecasting, and planning tasks. Building on this foundation, EventDrive-VLM introduces a multi-horizon event pyramid and a temporal-horizon mixture-of-experts module to adaptively encode and fuse asynchronous and frame-based information for downstream
reasoning. Comprehensive evaluation across diverse tasks shows that event streams provide substantial gains in temporal precision, motion awareness, and robustness, bringing event sensing into the center of driving intelligence.
Read the full abstract ▾
All authors Dongyue Lu, Rong Li, Ao Liang, Lingdong Kong, Wei Yin, Lai Xing Ng, Benoit R. Cottereau, Camille Simon Chane, Wei Tsang Ooi
Computer Vision
Adaptive Volumetric Mechanical Property Fields Invariant to Resolution
Rishit Dagli, Donglai Xiang, Vismay Modi et al. (+4 more)
Accurate mechanical properties (or materials) Young's modulus ($E$), Poisson's ratio ($\nu$) and density ($\rho$) are essential for reliable physics simulation of digital worlds, but most 3D assets lack this information. We propose AdaVoMP, a method for predicting accurate dense spatially-varying ($E$, $\nu$, $\rho$) for input 3D objects across representations, improving the resolution, accuracy, and
memory efficiency over the state-of-the-art. The foundation of our technique is a sparse and adaptive voxel structure SAV that efficiently represents both the input 3D shape and the material field output. We replace the fixed-voxel model of the most accurate prior method, VoMP, with a novel sparse
transformer encoder-decoder model that learns to generate a unique SAV autoregressively for every input shape to represent its materials, achieving a resolution $16^3\times$ higher than prior art. Experiments show that AdaVoMP estimates more accurate volumetric properties, even with lesser test-time compute than all prior art. This allows us to convert high-resolution complex 3D objects into simulation-ready assets, resulting in realistic deformable simulations.
Read the full abstract ▾
All authors Rishit Dagli, Donglai Xiang, Vismay Modi, Xuning Yang, Gavriel State, David I.W. Levin, Maria Shugrina
Computer Vision · Cross-listed from cs.LG
Looped World Models
Hongyuan Adam Lu, Z.L. Victor Wei, Qun Zhang et al. (+28 more)
Current world models face a fundamental tension: faithful long-horizon simulation demands deep computation, but deeper models are expensive to deploy and prone to compounding errors. We resolve this by introducing Looped World Models (LoopWM), which are the first looped architectures for world modelling. Our method iteratively refines latent environment states through a parameter-shared
transformer block. This yield up to 100x parameter efficiency over conventional approaches with adaptive computation that automatically scales depth to match the complexity of each prediction step. Orthogonal to scaling model size and training data, LoopWM establishes iterative latent depth as a new scaling axis for world simulation, which might significantly push the community forward.
Read the full abstract ▾
All authors Hongyuan Adam Lu, Z.L. Victor Wei, Qun Zhang, Jinrui Zeng, Bowen Cao, Lingwei Meng, Mocheng Li, Zezhong Wang, Haonan Yin, Naifu Xue, Minyu Chen, Cenyuan Zhang, Zefan Zhang, Hao Wei, Jiawei Zhou, Haoran Xu, Hao Yang, Ronglai Zuo, Tongda Xu, Yonghao Li, Jian Chen, Hebin Wang, Zeyu Gao, Yang Li, Wei Zhao, Qimin Zhong, Siqi Liu, Yumeng Zhang, Leyan Cui, Zhangyu Wang, Wai Lam
Computer Vision · Cross-listed from cs.CR
Seeing Is Not Screening: Multimodal Hidden Instruction Attacks on Agent Skill Scanners
Xiaojun Jia, Jie Liao, Simeng Qin et al. (+5 more)
Agent skills are emerging as an important attack surface in LLM-based systems. Through an empirical study of existing skill scanners, we find that current defenses primarily rely on textual descriptions, manifests, and source code as the main signals for security analysis, which can leave visually conveyed malicious intent insufficiently examined. This creates a practical blind spot: harmful operational instructions hidden in images may bypass scanning while still being recoverable by
multimodal agents during deployment. To systematically investigate this threat, we propose SkillCamo, a document-mediated
multimodal instruction attack that conceals malicious instructions within images bundled with a skill while rewriting the surrounding documentation to naturally reference those images as part of the normal workflow. Thus, the attack does not rely on the image alone, but on the joint interpretation of textual guidance and visual payload at execution time. To defend against such attacks, we further propose ExecScan, an execution-grounded
multimodal scanning module that performs intent extraction, behavior reconstruction, abuse assessment, and deliberative execution simulation over skill artifacts. ExecScan jointly analyzes documentation, code, referenced resources, and visual content to recover hidden instructions, reconstruct executable behavior chains, and identify downstream risks such as exfiltration, destruction, persistence, deception, and privilege escalation. Extensive experiments show that image-hidden malicious instructions challenge existing skill scanners, while ExecScan can improve the skill scanning performance.
Read the full abstract ▾
All authors Xiaojun Jia, Jie Liao, Simeng Qin, Ke Ma, Wenbo Guo, Yebo Feng, Aishan Liu, Yang Liu
Computer Vision
EgoCS-400K: An Egocentric Gameplay Dataset for World Models
Rongjin Guo, Dong Liang, Yuhao Liu et al. (+4 more)
The shift from
video generation to interactive world modeling places new demands on data: beyond captioned
videos, world models require temporally aligned
video-action-
language trajectories grounded in the actions, camera motion, states, and events that drive future scene changes. However, such data is difficult to obtain at scale. Web
video datasets offer broad visual coverage but lack executable actions and reliable states; robotic
datasets provide action and state supervision but are costly and limited in scene diversity; and existing simulators often lack large-scale human-driven interaction trajectories. In this paper, we introduce EgoCS-400K, a large-scale replay-grounded
egocentric Counter-Strike
dataset for world models, built from public professional CS and CS2 match demos that preserve human gameplay trajectories and enable parsing, replaying, rendering, and temporal
alignment. We extract player states, view directions, movements, keyboard/button inputs, view-angle changes, weapon usage, game events, and round-level context, and render clean first-person
videos from the same trajectories. EgoCS-400K contains over 400,000 first-person
videos and 10,000 hours of gameplay from more than 1,000 matches and 40,000 rounds, covering 13 maps and 10 player viewpoints per round. It supports a range of interactive visual modeling tasks, including action-conditioned future prediction, state- and event-aware scene rollout, replay-grounded captioning, and
agent egocentric action understanding. By connecting visual observations with human actions, camera motion, game states, and events at scale, EgoCS-400K serves as a practical bridge between passive web
videos, controllable game simulation, and costly real-world
embodied data.
Read the full abstract ▾
All authors Rongjin Guo, Dong Liang, Yuhao Liu, Fang Liu, Tianyu Huang, Gerhard P. Hancke, Rynson W. H. Lau
Computer Vision
ReAge3D: Re-Aging 3D Faces with View Consistency
Libing Zeng, Li Ma, Mingming He et al. (+3 more)
We present a novel framework for realistic and controllable 3D face re-aging which produces highly detailed, identity-preserving results. Existing 3D editing methods, while effective for coarse semantic changes, are not well suited for re-aging, as even small inconsistencies across re-aged 2D views can lead to over-smoothing of subtle but perceptually important age-related details. To address this challenge, we first introduce a 2D
diffusion-based re-aging model, DiffReaging, trained on synthetically generated image pairs. We further propose a center-out editing propagation strategy that leverages this re-aging model to reconstruct multi-view-consistent re-aged images. Specifically, starting from a re-aged frontal pivot view, we reconstruct the remaining views through warping and our proposed Masked-DiffReaging process. By injecting existing content at every step of the
diffusion process, Masked-DiffReaging ensures that the reconstructed regions remain coherent with existing pixels. The resulting consistent set of re-aged views supervises the optimization of the re-aged 3D representation. Our method outperforms existing 3D editing techniques both visually and quantitatively, enabling smooth, fine-grained control over age transformations in 3D face models.
Read the full abstract ▾
All authors Libing Zeng, Li Ma, Mingming He, Ning Yu, Paul Debevec, Nima Khademi Kalantari
Computer Vision
Neural Tree Reconstruction for the Open Forest Observatory
Marissa Ramirez de Chanlatte, Arjun Rewari, Trevor Darrell et al. (+1 more)
The Open Forest Observatory (OFO) is a collaboration across universities and other partners to make low-cost forest mapping accessible to ecologists, land managers, and the general public. The OFO is building both a database of geospatial forest data as well as open-source methods and tools for forest mapping by uncrewed aerial vehicle. Such data are useful for a variety of
climate applications including prioritizing reforestation efforts, informing wildfire hazard reduction, and monitoring carbon sequestration. In the current iteration of the OFO's forest map database, 3D tree maps are created using classical structure-from-motion techniques. This approach is prone to artifacts, lacks detail, and has particular difficulty on the forest floor where the input data (overhead imagery) has limited visibility. These reconstruction errors can potentially propagate to the downstream scientific tasks (e.g. a wildfire simulation.) Advances in 3D reconstruction, including methods like Neural Radiance Fields (NeRF), produce higher quality results that are more robust to sparse views and support data-driven priors. We explore ways to incorporate NeRFs into the OFO
dataset, outline future work to support even more state-of-the-art 3D
vision models, and describe the importance of high-quality 3D reconstructions for forestry applications.
Read the full abstract ▾
All authors Marissa Ramirez de Chanlatte, Arjun Rewari, Trevor Darrell, Derek J. N. Young