Computer Vision
What Breaks Local Watermarks? A Robustness Benchmark for Local Invisible Image Watermarking
Kai Yao, Bence Szilágyi, Sebestyén Kamp et al. (+4 more)
Local image watermarking embeds an invisible signal into selected image regions rather than spreading it across the entire image, enabling payload recovery from specific objects or regions without perceptibly altering the image. Existing studies evaluate the robustness of payload recovery and localization under image transformations, but they often focus on their own proposed method, resulting in narrow evaluations with inconsistent choices of transformations,
datasets, and metrics. These inconsistencies across studies limit direct comparisons across methods and muddle the overall picture of local watermark robustness. To address this gap, we present the first systematic robustness
benchmark for local watermarks across 55 image transformations, including (i) signal distortions, (ii) changes in image coordinate
alignment, (iii) indirect local edits, and (iv) direct watermark edits. The
benchmark evaluates MaskWM, WAM, OmniGuard, TrustMark, and PixelSeal, all methods that either provide native localization or require minimal adaptation to support it. Our results show that all evaluated methods are vulnerable to some transformation, with MaskWM standing out as offering the strongest payload recovery and localization, although it has the lowest image quality in the clean setting. Synchronization further improves MaskWM's payload recovery under several
geometric transformations, albeit at an additional cost to image quality. A key finding is that local watermark robustness depends strongly on the nature of the transformation: signal distortions are often tolerated by the strongest methods, while
geometric misalignment and generative local edits, such as inpainting and outpainting, can completely impair payload recovery. We observe that payload recovery and localization are related but not interchangeable, and both strongly depend on the transformation's impact on the watermark region.
Read the full abstract ▾
All authors Kai Yao, Bence Szilágyi, Sebestyén Kamp, Máté Poór, Máté Szilveszter, Matyas K. Zsoldos, Marc Juarez
Computer Vision
Hyper-RED: Scalable Event Pre-training via Semantic Hypergraph Distillation
Meisen Wang, Zhiqiang Tian, Wei Bao et al. (+3 more)
Event cameras have shown great potential for robust visual perception, yet scaling event representation learning remains challenging due to the scarcity of large-scale annotated event data. Pretrained image models provide scalable semantic supervision, but existing image-to-event methods rely on rigid pixel-wise or
token-wise
alignment that overlooks modality discrepancies in texture, density, and appearance, potentially causing semantic collapse and limiting transferability. To address this issue, we propose Hyper-RED, a simple, painless, and scalable image-to-event pretraining framework that transfers high-order semantic structures from images to events. Hyper-RED uses hypergraphs to model and align high-order semantic associations among multiple image and event
tokens, enabling cross-modal knowledge transfer while accommodating modality-specific differences rather than enforcing rigid one-to-one correspondence. Specifically, given a paired event–image sample, Hyper-RED leverages DINOv3 to extract spatial
token representations and constructs image, event, and cross-modal semantic hypergraphs, where each hyperedge connects multiple semantically correlated
tokens. We further introduce a hypergraph relational
distillation loss that imposes complementary intra- and cross-modal constraints, enabling the event encoder to inherit image-derived semantic organization while preserving local relational consistency and event-specific characteristics. Experiments on three tasks across five event
datasets demonstrate consistent scaling from ViT-S to ViT-L and state-of-the-art performance (Fig.1). The code is available at: https://github.com/meisenwang/Hyper–RED.
Read the full abstract ▾
All authors Meisen Wang, Zhiqiang Tian, Wei Bao, Chengjie Wang, Shaoyi Du, Siqi Li
Computer Vision
Jiahong Yuan, Weiming Mi, Tao Zhang et al. (+1 more)
Query-based
segmentation methods have shown promising potential for
surgical instrument
segmentation and recognition, which is essential for scene understanding and downstream tasks in computer assisted surgery. However, most existing approaches predominantly rely on per-frame predictions and overlook cross-frame temporal priors as well as temporal-consistency constraints. This limitation often leads to unstable query representations and suboptimal category recognition. In this paper, we propose TEDi, a Temporal
memory-Enhanced and Denoising
transformer for
surgical instrument
segmentation that addresses these is sues through
Memory Search Enhancement and Temporal Consistency Denoising. The former introduces a query-level
memory bank and a
memory search enhancement encoder to retrieve discriminative representations from historical frames, enriching current-frame features. The latter constructs a temporally consistent reference as a cross-frame semantic anchor to suppress temporally unstable predictions and promote semantic coherence across frames. Extensive experiments on two
benchmark datasets, EndoVis 2017 and EndoVis 2018, demonstrate that TEDi consistently outperforms state-of-the-art methods, highlighting its potential to further advance computer-assisted surgery. Our code is available at github.com/argon-xixi/TEDi.
Read the full abstract ▾
All authors Jiahong Yuan, Weiming Mi, Tao Zhang, Haoyin Zhou
Computer Vision · Cross-listed from cs.LG
Noise2Noise Revisited: Training Pair Distributions Dominate Loss Choice in Self-Supervised Denoising
Dingyan Shang, Zhenyu Xu, Youting Wang et al. (+2 more)
Noise2Noise (N2N) trains denoisers on pairs of independently corrupted observations, eliminating clean references. We stress-test two natural
conjectures about why the L1 loss outperforms L2 here. First, the hypothesis that the L1 loss confers robustness via parameter sparsity confuses the loss with Lasso regularization: an explicit Lasso penalty produces the predicted sparsity yet fails to reproduce L1's cross-noise behavior, while L1- and L2-trained weight distributions are indistinguishable. Second, the population optima of the two losses coincide exactly for symmetric signal posteriors and nearly so for concentrated ones. Measured differences are therefore dominated by optimization dynamics (bounded-influence gradients), which we probe with gradient statistics and contaminated-target training. On Kodak24 with five synthetic noise families, the L1 loss holds a statistically significant edge over L2, below 1 dB PSNR, holding across three seeds on 13 of the 14 noise columns. On real camera noise the loss is not the decisive variable in distribution: on official SIDD validation blocks, synthetic-Gaussian-trained N2N models gain only 0.8 to 3.7 dB over the noisy input regardless of loss, while retraining on SIDD's own noisy pairs, never reading ground truth, gains 9.4 to 11.0 dB, far ahead of BM3D. All metrics are on raw network outputs, and the study makes no leaderboard claim. The training pair distribution, not the loss, carries the inductive bias. That design rule applies wherever clean references are unobtainable, from microscopy to industrial inspection sensors.
Read the full abstract ▾
All authors Dingyan Shang, Zhenyu Xu, Youting Wang, Bonan Shen, Bowen Liu
Computer Vision
PSMP-CLIP: Patch-Prompt SAM and Multi-Semantic Prompting for CLIP-Based Zero-Shot Anomaly Detection
Xuezhi Xiang, Guanghao Wu, Heqi Xiang et al. (+4 more)
Zero-shot anomaly detection aims to localize anomalies without target-domain samples. Existing CLIP-based methods suffer from coarse anomaly maps and limited semantic
prompts. We propose PSMP-CLIP, integrating patch-
prompt SAM2
segmentation (PPSS) and multi-semantic guided
prompt regularization (MSGPR). PPSS samples
prompts directly from intermediate patch features, avoiding threshold drift and guiding SAM2 to produce precise masks. MSGPR uses multiple learnable
prompts constrained by semantic anchors to preserve generalization. Experiments on 14
datasets show highly competitive performance, achieving the best pixel-level AUROC on MVTec AD, BTAD, DTD-Synthetic, CVC-ClinicDB, TN3K, Endo, and Kvasir.
Read the full abstract ▾
All authors Xuezhi Xiang, Guanghao Wu, Heqi Xiang, Jiayao Liu, Xiaoheng Li, Yiming Chen, Shanjun Zhang
Computer Vision
Unifying Semantic Priors and High-Frequency Traces: Enhancing V-JEPA with Mixture-of-Experts for Robust Synthetic Image Forensics
Simone Teglia and Irene Amerini
The unchecked proliferation of manipulated images on social media platforms has increased the spread of misinformation, posing a severe threat to public trust and information integrity. Modern deepfake detectors typically rely on
Vision Transformers (ViTs) to capture the low-level inconsistencies that characterize fully synthetic or locally tampered images. However, the global understanding of such foundation models is not enough to discriminate alone between real and fake multimedia content, especially in challenging scenarios where images are compressed or transmitted through social media. In this paper we pioneer the application of Joint-
Embedding Predictive Architecture (JEPA) models to deepfake detection, taking advantage of the generalized representation of visual reality that such World Models have exhibited. We hypothesize, and empirically demonstrate, that the intrinsic world understanding of JEPA models can be used as a strong prior for a deepfake detector. To fully exploit JEPA capabilities, we propose MoE-JEPA, a dual-stream architecture for deepfake detection. By enhancing a V-JEPA 2 backbone with a Residual Mixture-of-Experts (MoE) mechanism, along with a noise stream branch, our model dynamically internalizes forensic knowledge. Furthermore, a Gated
Attention Multiple Instance Learning (MIL) module is employed to ensure precise spatial semantic understanding. Evaluated on the SID-Set
benchmark, comprising 300K AI-generated, tampered and authentic images, MoE-JEPA establishes a new state-of-the-art with an accuracy of 95.54%, successfully outperforming vastly larger models.
Read the full abstract ▾
Computer Vision
IMVS: Interactive Medical Volume Segmentation with Test-Time Adaptation - A New Method for Annotating Radiology Datasets
Abhilaksh Singh Reen, Kushal Borkar and Ritvik Mahapatra
Annotating large radiology
datasets is bottlenecked by the manual effort of delineating structures slice-by-slice in 3D volumes. Interactive methods reduce this effort but stay interaction-inefficient: slice-wise methods (including many foundation models) ignore inter-slice continuity, while 3D and
video-based methods propagate a
prompt with a fixed propagator that never adapts to the target volume, so it drifts on low-contrast or pathological structures and must be re-prompted. We present IMVS, a human-in-the-loop annotation framework that composes three components into a closed loop rather than a new
segmentation primitive: a lightweight 2D Slice Mask Adapter (SMA) fine-tuned online from user scribbles, a frozen Volume Mask Tracker (VMT) that propagates corrected masks across adjacent slices, and a soft teacher–student
alignment that limits forgetting. The SMA is backbone-agnostic (UNet++, DeepLabV3, TransUNet). Across 8 public CT/MRI
datasets, IMVS matches strong interactive baselines in quality while sharply cutting annotation effort: $14.4\times$ faster than a proficient copy-based manual workflow ($22.3\times$ over naive manual), $4.6\times$ over slice-wise and $1.9\times$ over 3D interactive methods. MedSAM2 and ScribblePrompt stay competitive or stronger on well-delineated organs; IMVS's advantage is largest on challenging targets and on interaction efficiency. Source code and Demo
Video: https://github.com/AbhilakshSinghReen/imvs.
Read the full abstract ▾
Computer Vision
FSANet: Frequency-Spatial Aware Network for Image Segmentation
Ruibo Wang, Ziyi Shen, Huaming Wu et al. (+2 more)
Image
segmentation remains challenging due to occlusions, poor lighting, and irregular structures. Although
transformer-based methods achieve high accuracy, they rely heavily on long-range spatial features, leading to high computational costs and neglecting prior knowledge or noise patterns, resulting in missing details and unclear boundaries. To address these issues, we propose Frequency Spatial Aware Network (FSANet), which integrates prior knowledge with a dual-domain solver to sequentially adapt to diverse
segmentation tasks. Specifically, we design three key modules: (1) Structure Prior Module, which recovers overlooked details; (2) Dual-Domain Awareness Module, which captures salient features while disentangling noise; and (3) Edge Estimation Module, which enhances edge awareness for more precise
segmentation. In addition, the limited availability of comprehensive
segmentation datasets covering various real-world scenarios hinders the performance of existing methods. To address this, we introduce SceneX, a novel open-source
dataset featuring 10 challenging non-ideal scenarios, establishing a new
benchmark for evaluating and improving the robustness and real-world applicability of the
segmentation models. Extensive experiments demonstrate the efficiency and effectiveness of FSANet.
Read the full abstract ▾
All authors Ruibo Wang, Ziyi Shen, Huaming Wu, Dong Liang, Kun Shang
Computer Vision
Robust 3D Reconstruction from Multi-View Optical Satellite Imagery via Reliability-Aware Height-Evidence Fusion in Gaussian Splatting
Jie Yang, Yingdong Pi, Qiyan Luo et al. (+3 more)
Robust 3D reconstruction from multi-view optical satellite imagery requires fusing complementary but sometimes conflicting
geometric evidence. Digital surface models (DSMs) are the primary elevation representations for satellite-based 3D reconstruction, making reliable height estimation essential. However, in a Gaussian scene representation jointly optimized from multiple views, Gaussian responses at different elevations can support competing height hypotheses at the same rendered location, while conventional alpha-weighted elevation aggregation may produce intermediate elevations that do not correspond to physical surfaces. To address this challenge, we formulate DSM reconstruction as a reliability-aware height-hypothesis fusion problem and propose HLC-GS, a reliability-aware Height-Layer Consistency Gaussian Splatting framework for multi-view satellite 3D reconstruction. HLC-GS organizes projected Gaussian responses into candidate height hypotheses and evaluates their relative support using layer competition and Gaussian footprint support. A continuous height-layer risk map guides dominant-layer reliability correction and secondary-layer suppression during optimization. The proposed training strategy regulates conflicting Gaussian responses within the shared representation to improve the reliability of reconstructed surface elevations. Experiments on seven scenes from the DFC2019 and IARPA2016
datasets demonstrate improved DSM reconstruction accuracy. Compared with EOGS, HLC-GS reduces the average DSM MAE from 1.46~m to 1.18~m and RMSE from 2.78~m to 2.58~m, while increasing PAG$_{2.5}$ from 86.09% to 88.61%, with comparable computational cost.
Read the full abstract ▾
All authors Jie Yang, Yingdong Pi, Qiyan Luo, Xiaoyu Wang, Lekang Wen, Mi Wang
Computer Vision
De-GAN - Dynamic Parameter Tuned GAN for 3D Medical Image Segmentation: A Step Towards Generalisation
Zoha Usama and Azadeh Alavi
Brain tumor
segmentation remains difficult because enhancing tumor (ET) has low contrast and overlaps surrounding tissue, while scanner and site variation causes domain shift. We propose DE-GAN, a contrast-enhancing conditional GAN that combines input-adaptive dynamic convolutions, style-aware feature mixing, and coordinate encoding to synthesize slice-adaptive FLAIR images. A label-guided, class-conditional target separates tumor-core (TC) and ET intensities while preserving anatomy. The generated FLAIR is concatenated with the original MR modalities and used to train a 3D U-Net. Across BraTS 2015, 2018, and 2019, DE-GAN improves
segmentation over the baseline and static EnhGAN replacement on most reported TC/ET metrics, with the largest gains from retaining both original and enhanced FLAIR. Code and pretrained models are available at https://github.com/zkhansuri-ui/DE-GAN.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Visual Cue Guided Video Planning for Generalizable Robot Navigation
Hojin Lee, Sizhe Lester Li, Maximilian Hilger et al. (+4 more)
Generative
video models can serve as a promising backbone for
robot navigation by predicting future observations as
video plans. Recent approaches often condition
video planning on short-horizon guidance and recover
geometric waypoints through scene reconstruction, leaving longer-horizon planning and precise
video-to-action translation less explored. We present CueNav, a
video model-based
navigation framework combining visual cue guided
video planning with an embodiment-specific Inverse-Dynamics Model (IDM). As visual cues, we use a Bird's-Eye View (BEV) map to convey global task context and retain part of the
robot body in the
egocentric observation to expose embodiment context. These cues guide the
video planner, while the IDM translates dense flow fields extracted from the
video plan into
robot actions. With the visual cue encoding global task context, CueNav achieves nearly 2x higher success in maze
navigation than planning without the cue. The body-aware view with the IDM enables precise
navigation with 70% success in a narrow passage where comparison methods largely fail to complete the task. We further demonstrate zero-shot semantic-conditioned
navigation and deployment of the same
video planner across different
robot platforms. Our results show that visual cue-guided
video planning with embodiment-specific action grounding paves the way toward a generalizable
navigation framework for longer-horizon planning and embodiment-aware control. Additional results and code are available on our project website: https://cuenav.github.io.
Read the full abstract ▾
All authors Hojin Lee, Sizhe Lester Li, Maximilian Hilger, Susie Lu, Achim J. Lilienthal, Vincent Sitzmann, Daniel A. Duecker
Computer Vision
PriorPose: Reference-Guided Joint Deformation and Alignment for Category-Level Object Pose Estimation
Yihan Chen, Huan Ren, Wenfei Yang et al. (+3 more)
Category-level object pose estimation seeks to recover a similarity transform $(R,t,s)$ for unseen instances without instance-specific CAD models. Most competitive methods are correspondence-based: prior-free variants regress canonical (NOCS) coordinates directly from local observations and implicitly memorize the canonical frame in the weights, which ties the parameters to category-typical orientations and hurts generalization under distribution shift; prior-based variants introduce a category prior but typically follow a serial deform-then-align pipeline, where underconstrained canonical completion can corrupt correspondences and induce error cascades in pose. We propose PriorPose, a reference-guided correspondence framework that keeps the category prior explicit and solves canonicalization and
alignment jointly in a shared feature space. A reference-guided seeded
transformer embeds the partial observation and the category prior as
token sets and fuses them via geometry-aware seeds, from which the network jointly predicts a per-point NOCS field for visible points and a canonical deformation of the prior that reconstructs a full canonical instance, while a deep pose head regresses $(R,t,s)$ from the induced correspondences. A two-part shape consistency objective, with canonical-space and camera-space consistency losses, couples correspondence, deformation, and pose, reducing reliance on memorized canonical orientations and avoiding deform-then-align error cascades. Experiments on standard and larger-category
benchmarks demonstrate that PriorPose sets new state-of-the-art results on most evaluated metrics, especially under strict pose thresholds, while remaining competitive on relaxed pose and IoU metrics and showing improved robustness under shape variation and domain shift.
Read the full abstract ▾
All authors Yihan Chen, Huan Ren, Wenfei Yang, Hang Du, Tianzhu Zhang, Feng Wu
Computer Vision · Cross-listed from cs.AI
VideoMM: Adaptive Macro-Micro Inference for Efficient Video MLLMs
Haoyu Guo, Yuan Feng, Junlin Lv et al. (+3 more)
Scaling
Multimodal Large
Language Models (MLLMs) to long-form
video understanding is bottlenecked by the explosion of visual
tokens, which saturates context windows and incurs prohibitive costs. Current solutions predominantly rely on auxiliary models for
token reduction but face a fundamental dilemma: lightweight encoder-driven approaches often overlook critical semantic information, whereas heavyweight MLLM-driven reduction negates the efficiency gains. {In this work, we identify a more fundamental inefficiency underlying this dilemma: while fine-grained visual details are essential for detailed understanding, they are largely redundant for the preliminary task of selecting semantically relevant regions. } Motivated by this, we introduce VideoMM, which marks a paradigm shift from model-centric downsizing to adaptive perceptual granularity. Specifically, our framework {decouples selection from
reasoning} by executing semantic filtering on a cost-effective Macro Proxy (derived from downscaled frames), and projecting the selected regions onto high-fidelity Micro
Tokens for detailed understanding only when necessary. Extensive evaluations show that VideoMM significantly outperforms existing solutions. It achieves a 6.13$\times$ speedup and a 7.4% accuracy gain over full-context baselines on LongVideoBench, and further accelerates inference by 2.73$\times$ over current leading methods, establishing a highly scalable paradigm for long-
video understanding. Our code is available at: https://github.com/adfh917k/VideoMM.
Read the full abstract ▾
All authors Haoyu Guo, Yuan Feng, Junlin Lv, Mingjun Xiao, S Kevin Zhou, Xike Xie
Computer Vision
MAETrack: Unleashing the Potential of Pretrained Geometric Priors for 3D Single Object Tracking
Sifan Zhou, Qiwei Wang, Linyue Tan et al. (+3 more)
Large-scale pre-training has transformed representation learning in 2D
vision, yet its transferability to 3D single object tracking (SOT) remains insufficiently understood. Directly fine-tuning self-supervised 3D encoders, such as masked autoencoders (MAE), often leads to sub-optimal adaptation because the reconstruction objective is not fully aligned with the spatial-temporal matching requirements of tracking. In this paper, we observe that this difficulty can be interpreted as a layer-wise transfer mismatch: shallow layers tend to preserve transferable
geometric cues, while deeper layers become increasingly specialized to the reconstruction pretext task and are less suitable for downstream tracking. Based on this observation, we propose MAETrack, a lightweight adaptation framework for transferring pre-training MAE representations to 3D SOT. MAETrack includes Layer-Selective Initialization (LSI), which initializes only the shallow stages of the tracking backbone from pre-trained weights while re-initializing deeper stages, and
Geometric Residual Gating (GRG), which reinforces structurally salient regions in the search BEV features before template-search fusion through residual spatial modulation. Extensive experiments on standard 3D SOT
benchmarks show that MAETrack consistently improves upon vanilla fine-tuning baselines with limited computational overhead. More broadly, our results suggest that effective transfer from 3D reconstruction pre-training to 3D tracking is not merely a matter of partial fine-tuning, but depends on a tracking-oriented transfer principle that preserves shallow geometry while adapting deeper representations to the downstream objective.
Read the full abstract ▾
All authors Sifan Zhou, Qiwei Wang, Linyue Tan, Ziyu Liu, Ziyu Zhao, Xiaobo Lu
Computer Vision
Efficient 3D Whole-Body PET Image Denoising via Conditional Rectified Flow With Optimized Sampling Strategy
Jiale Shen, Guolin Wang, Chenhao Wang et al. (+3 more)
Reducing radiation exposure in Positron Emission
Tomography (PET) is important for patient safety; however, ultra-low-dose imaging suffers from severe noise, which may affect diagnostic interpretation without appropriate image enhancement. While current 3D deep generative models, particularly
diffusion models, have shown strong reconstruction fidelity, their practical use can be limited by long inference times. In contrast, faster 2D-based alternatives may have difficulty maintaining volumetric consistency, an important consideration for whole-body PET imaging analysis. To bridge this gap, we propose a one-pass conditional 3D rectified flow (3D Flow) framework for whole-body PET image denoising that incorporates a novel optimized non-uniform sampling strategy. The model is trained with a one-pass linear-interpolant velocity-matching objective. This approach reconstructs a full 3D volume in approximately 30 seconds in our implementation, compared with multi-hour inference for the evaluated 3D DDPM baseline. Evaluations including zero-shot transfer to an independent clinical
dataset show that our model achieves favorable global image quality and lesion conspicuity compared with the evaluated 3D DDPM and DDIM baselines, including on challenging short-acquisition data. Furthermore, the proposed method shows promising zero-shot transfer performance across the evaluated
datasets and unseen dose levels (down to 1/100 of the standard dose), with artifact-focused visual comparisons supporting the need for further lesion-level validation. By balancing reconstruction fidelity and computational efficiency, this work presents a candidate approach for ultra-low-dose whole-body PET image denoising.
Read the full abstract ▾
All authors Jiale Shen, Guolin Wang, Chenhao Wang, Xinhui Su, Wei Luo, Feng Yu
Computer Vision
Jinwoo Jeon, GyuYeop Do, Yubin Lim et al. (+4 more)
Large-scale
vision-
language models (VLM) such as CLIP enable strong open-vocabulary
reasoning, yet deploying these capabilities on resource-constrained edge devices remains challenging. EdgeVL addresses this problem by distilling CLIP representations into lightweight multi-modal encoders and applying quantization-aware training (QAT) for efficient Open-Vocabulary Classification (OVC) on edge hardware. However, its two-stage optimization applies different objectives for
distillation and QAT, and contrastive learning is performed within the quantized student space, which can result in inconsistent optimization and reduced training efficiency. Moreover, identical supervision across RGB and non-RGB modalities may lead to modality imbalance. We propose a unified framework for quantized semantic
distillation tailored to edge deployment. By jointly optimizing
distillation and quantization within a unified teacher-anchored framework, our method ensures consistent training under quantization, suppressing hard negatives and enlarging decision margins. Additionally, we design a lightweight cross-
attention adapter that enhances non-RGB representations through RGB-guided semantic transfer, narrowing the modality gap. Extensive experiments demonstrate consistent improvements on non-RGB modalities while maintaining deployment efficiency.
Read the full abstract ▾
All authors Jinwoo Jeon, GyuYeop Do, Yubin Lim, Nam-Joon Kim, Hyun Gon Ryu, Hyuk-Jae Lee, Byung-Jun Lee
Computer Vision · Cross-listed from cs.RO
Differentiable Mesh State Estimation via Factor Graph Inference for Deformable Object Reconstruction
Lidia Al-Zogbi, Fangjie Li, Samuel Tobin et al. (+13 more)
Estimating deformable object states remains a fundamental challenge in robotics and simulation. We propose a novel factor graph-based framework for probabilistic mesh state estimation of deformable objects. The method directly updates a tetrahedral mesh, a rich and physically-grounded representation of an environment, by combining physics priors, noisy sensor measurements, and temporal smoothness constraints within a unified probabilistic formulation. The estimation problem is posed as a nonlinear least-squares optimization and solved using Levenberg-Marquardt. Ex vivo central-airway obstruction experiments and simulations on deforming cube models demonstrate reliable and accurate reconstruction under both rigid motion and deformation, highlighting the potential of this probabilistic approach for principled, measurement-driven mesh state estimation in deformable object reconstruction.
Read the full abstract ▾
All authors Lidia Al-Zogbi, Fangjie Li, Samuel Tobin, James Ferguson, Nithesh Kumar, Alejandro Chara, Kuan-I Chung, Mingxing Rao, Ayberk Acar, Susheela Sharma Stern, Robert Webster, Daniel Moyer, Alan Kuntz, Caleb Rucker, Tucker Hermans, Jie Ying Wu
Computer Vision
MEgoVista: Multi-view Ego-aware Motion Estimation for Metric 4D Hands and Head in the Wild
Jiangong Xiao, Zhihao Zhang, Yifei Dong et al. (+7 more)
Learning manipulation from human
video requires high-fidelity hand-motion reconstruction in metric units. Today's metric hand labels come from studio rigs and instrumented headsets, and both are confined in the same two ways: neither leaves a prepared setting, and neither is checked against an independent reference. Unconstrained head-worn recording promises the opposite trade-off, scaling with the number of people wearing a device. We therefore introduce MEgoVista, an offline pipeline that turns a single unprepared MEgo View recording into metric two-hand and head motion in one
gravity-aligned world frame. Three properties set it apart from existing
egocentric reconstruction systems: first, it reconstructs in settings studio volumes and tabletop rigs cannot reach, settling hand ownership at detection so bystander hands stay out of the wearer's trajectory; second, it takes its metric gauge from calibrated stereo rather than a
monocular prior, installing scale at initialisation so policies receive physical units, not arbitrary coordinates; third, both outputs are scored inside a motion-capture volume against independent Chingmu optical capture, under a protocol that audits its own reference and charges what a method declines to predict. MEgoVista is offered as a measured route from
egocentric video to metric hand supervision, one that widens where such labels can be gathered.
Read the full abstract ▾
All authors Jiangong Xiao, Zhihao Zhang, Yifei Dong, Chao Ma, Zhouyi Jin, Zhiwen Hou, Li Liu, Weihuang Chen, Hongbin Sun, Maoqing Yao
Computer Vision
Lesion-centered 3D mapping of colonoscopy procedures: validation of a hierarchical ensemble pipeline on public benchmark videos
Hyunjun Kim, Hyeonwoo Na and Jaewoo Lee
Background and Objective: Colonoscopy recording practice preserves text reports and still photographs, while the spatial information already present in the recorded
video - where the scope traveled, where a lesion was observed, and whether the same lesion was seen again - is discarded when the procedure ends. This study determines whether a lesion-centered spatial record can be assembled and validated without full-colon 3D reconstruction. Methods: A four-layer hierarchical pipeline was assembled - (1) a global topological map, (2) lesion-level spatio-temporal tracks, (3) on-demand local 3D reconstruction, and (4) persistent lesion identity across repeated observations - and ran end to end on four public
videos (two C3VDv2 sequences with ground-truth depth and two full REAL-Colon procedures; 40,245 frames). All components are published, individually validated methods; the contribution is their lesion-centered assembly, linking rules, and evaluation. Results: Revisits, impossible under forward-only mapping by construction, were detected by entry-map
Bayesian localization: 5,614 and 4,043 revisit events (56 and 68 distinct nodes) in the two full procedures. Lesion-identity merging at the adopted threshold 0.5 maintained ground-truth purity 1.0 while auto-merging 20 of 231 candidate pairs. The endoscopy-specific geometry engine outperformed a general-purpose foundation model on all metrics (overall absolute relative error (AbsRel) 0.2276 vs. 0.3523). Conclusions: The results are partial but establish a concrete near-term path: revisit detection, lesion identity, and local 3D each returned quantitative, reproducible output without waiting for complete
geometric reconstruction; validating the record on clinical data is the next step.
Read the full abstract ▾
Computer Vision
Bridging the Perceptual Gap: Residual-Enhanced Downscaling and Manifold-Aware Perception Alignment Adaptation for NR-IQA
Yu Li, Zhengran Shen, Yachun Mi et al. (+2 more)
Leveraging Large
Vision-
Language Models like CLIP has recently set new
benchmarks for No-Reference Image Quality Assessment (NR-IQA). However, the contrastive pretraining of CLIP inherently prioritizes semantic invariance, which often suppresses subtle perceptual signals, a phenomenon we term perceptual submergence. Furthermore, standard preprocessing techniques (e.g., cropping and interpolation) further exacerbate the loss of critical high-frequency quality cues. In this paper, we propose the Cross-modal Perception
Alignment Adapter (CMPA), a
manifold-aware framework designed to disentangle perceptual distortions from dominant semantics. CMPA introduces a Perception-Sensitive Feature Extractor (PFE) that projects CLIP features into a compact, low-dimensional subspace, explicitly magnifying distortion-induced off-
manifold deviations. Subsequently, a Cross-Modal Perception
Alignment Injector (PAI) aligns these features with quality-aware text anchors and re-injects them into the backbone. To ensure input fidelity, we also devise a Residual-enhanced Perceptual Downscaling strategy that adaptively compensates for resolution-induced information loss using Just Noticeable Difference (JND) guided frequency re-injection. Extensive evaluations on several
benchmark datasets demonstrate that our approach significantly outperforms state-of-the-art methods, effectively recovering the perceptual signals submerged in semantic-dense representations.
Read the full abstract ▾
All authors Yu Li, Zhengran Shen, Yachun Mi, Puchao Zhou, Shaohui Liu
Computer Vision
SAVTrack: Selective Vote Aggregation for Reliability-Aware Point Cloud Tracking
Sifan Zhou, Linyue Tan, Qiwei Wang et al. (+2 more)
3D single object tracking (SOT) in
LiDAR point clouds is essential for autonomous systems, but remains challenging under sparse and incomplete observations. In such cases, different target points provide highly uneven constraints on the object center, causing some point-to-center votes to be substantially less reliable than others. Existing point-based trackers typically aggregate these hypotheses without explicitly modeling their reliability, allowing inaccurate votes to contaminate proposal clustering and degrade localization accuracy. To address this issue, we propose SAVTrack, a motion-aware tracking framework with Selective Vote Aggregation (SAV). SAVTrack estimates the reliability of each candidate vote from both local seed features and inter-frame motion context, and removes low-confidence hypotheses before proposal clustering. This pre-aggregation gating prevents unreliable hypotheses from affecting cluster formation while introducing only modest computational overhead. SAVTrack achieves competitive performance on KITTI and nuScenes, reaching 68.4/87.4 and 58.44/69.82 Success/Precision, respectively, while running at 82 FPS. It retains fewer than one-sixth of the candidate votes used by dense aggregation and remains particularly effective under sparse target observations.
Read the full abstract ▾
All authors Sifan Zhou, Linyue Tan, Qiwei Wang, Ziyu Zhao, Xiaobo Lu
Computer Vision
Channel-Wise and Token-Aware Post-Training Quantization for Visual State Space Duality
Jonghyeon Lim and Changhoon Yim
State space models (SSMs), particularly Mamba, have emerged as efficient alternatives to
attention-based architectures and have been extended to
vision through ViM, VMamba, and Visual State Space Duality (VSSD). Yet the low-bit post-training quantization (PTQ) behavior of VSSD remains insufficiently understood. A weight-activation split on VSSD-Tiny identifies activation quantization as the dominant low-bit bottleneck, while representative inputs to selected VSSD-backbone linear layers exhibit strong channel-wise magnitude variation and
token-localized extremes. We propose the Channel-wise
Token-balanced Output-Aware Clipping (CTOAC) method, which learns per-input-channel clipping bounds by minimizing a
token-balanced reconstruction loss on the corresponding linear outputs. Only the selected linear layers and their input activations are quantized; other backbone operations retain their original precision. Across VSSD-Tiny, VSSD-Small, and VSSD-Base, the proposed CTOAC method retains ImageNet-1K accuracy and remains substantially more robust than the evaluated baselines at more aggressive precision settings. Applying the same quantization scope to VSSD backbones on COCO and ADE20K preserves strong object detection, instance
segmentation, and semantic
segmentation performance. An optimized RTX 4090 deployment configuration achieves up to 1.42x end-to-end speedup over FP32.
Read the full abstract ▾
Computer Vision
ViD: Vision-Dominant Gender Bias Mitigation for Large Vision-Language Models
Zhipeng Zhao, Zhaoqiang Wei, Peishun Liu et al. (+2 more)
Gender bias in large
vision-
language models (LVLMs) undermines their
fairness and reliability, compromising output trustworthiness. Current mitigation methods rely on training-phase adjustments or post-hoc calibration, but face limitations in dynamic visual bias mitigation. These include inability to capture real-time visual-textual incongruence, dependence on predefined gender bias taxonomies, and degraded cross-modal
alignment with emergent bias patterns. To address these challenges, we propose ViD, a causally-inspired framework that analyzes
attention mechanisms across five distinct patterns, revealing confounding effects from strong
language priors. ViD demonstrates that visual-to-
language cross-
attention effectively suppresses bias while preserving general
reasoning capabilities and text generation quality. ViD incorporates dual mechanisms:
backdoor adjustment counters strong
language priors, while refined
token selection in decoding layers optimizes processing. This enhances model robustness and inference efficiency. Our integrated approach significantly mitigates gender bias across multidimensional social attributes in LVLMs, improving visual grounding and output
fairness. Cross-
benchmark validation shows ViD reduces gender bias by 14.7% on single-attribute evaluations (FACET) and achieves significant improvements on image captioning tasks (MS COCO), with gender bias score improving from 0.6708 to 0.9978 for LLaVA. Crucially, these improvements require no additional training overhead, making ViD a scalable and practical solution for bias mitigation in LVLMs.
Read the full abstract ▾
All authors Zhipeng Zhao, Zhaoqiang Wei, Peishun Liu, Youwei Zhao, Ruichun Tang
Computer Vision
What Do Hallucinations Reveal About Multimodal Reasoning? Diagnosing Visual Grounding Failures via Contrastive Decoding Probes
Zhipeng Zhao, Wenxu Wang, Peishun Liu et al. (+1 more)
When strong
multimodal models are widely available, progress requires new scientific methodologies beyond
benchmark scores—using models as instruments for understanding behavior. We address this by asking: can we use large
vision-
language models (LVLMs) as experimental instruments for studying their own failure dynamics? Focusing on visual
hallucination, we introduce SAFE, a training-free decoding framework that contrasts visually-grounded and
vision-ablated generation paths to produce a
token-level contrastive grounding score that identifies when the model favors linguistic priors over visual evidence. This signal serves dual roles: as a practical proxy for detecting visually-ungrounded
tokens, and as the basis for decoding-time penalties. Our analysis yields three empirical observations: visual dependency decays over generation,
hallucinations co-occur in temporal clusters, and early intervention reduces clustering without substantially degrading fluency. On MMHalBench, SAFE substantially outperforms all compared baselines; results elsewhere are more mixed. We argue that designing contrastive probes exemplifies a broader mission: using models as instruments for scientific understanding. Code: https://github.com/zhaozhipeng1997/SAFE_public.
Read the full abstract ▾
All authors Zhipeng Zhao, Wenxu Wang, Peishun Liu, Ruichun Tang
Computer Vision
Can Knowledge Transfer Parameters Be Learned? LePoKet for Efficient Robotic Vision
Yanick C. Tchenko, Felix Mohr, Hicham Hadj-Abdelkader et al. (+1 more)
Efficient perception is central to robotic systems operating under constrained computation,
memory, and latency budgets. Knowledge transfer from larger pretrained models offers a practical route to stronger compact perception networks, but existing approaches commonly rely on fixed
distillation objectives or manually designed interaction mechanisms. Building on Hereditary Knowledge Transfer (HKT), we propose LePoKet (Learnable Parameter Optimization for Knowledge Transfer), a structural transfer framework that embeds knowledge inheritance directly into the forward computation. LePoKet introduces a block-wise Extract-Transform-Mix interface whose interaction parameters are optimized jointly with the child network through a Learnable Genetic
Attention (LGA) operator, without auxiliary
distillation losses or temperature scaling. We first characterize the mechanism on CIFAR-10 and CIFAR-100 using ResNet parent-child pairs, obtaining relative error reductions of 24.57% and 25.1%, respectively, over standard child training. We then evaluate LePoKet for dense motion estimation by integrating it into a compact RAFT-based optical-flow model trained only on FlyingChairs and FlyingThings3D. LePoKet improves the compact RAFT baseline from 2.21 to 1.92 EPE on Sintel Clean, from 3.35 to 3.01 on Sintel Final, and from 7.51 to 6.39 on KITTI. A direct comparison with HKT further shows that LePoKet improves CIFAR-10 accuracy from 92.40% to 93.40% while achieving the best Sintel Final and KITTI errors among the evaluated compact transfer variants, with comparable performance on Sintel Clean. These results demonstrate that learnable structural transfer generalizes across recognition and motion perception tasks and provides a promising approach for efficient robotic
vision.
Read the full abstract ▾
All authors Yanick C. Tchenko, Felix Mohr, Hicham Hadj-Abdelkader, Hedi Tabia
Computer Vision
JewelTry: Mask-Free Scale Aware Jewelry Virtual Try-On
Xinlei Niu, Peixia Li, Jun Wang et al. (+5 more)
Virtual try-on (VTON) enables customers to visualize how fashion products appear when worn and has become an important technology for online shopping. While recent advances have substantially improved garment VTON, jewelry remains a challenging and underexplored category due to its small size, rigid structure, and sensitivity to fine-grained visual details. Realistic jewelry VTON requires not only faithful appearance transfer but also accurate scale and placement relative to the wearer. Existing jewelry VTON methods typically rely on mask guidance, whereas mask-free approaches lack explicit guidance for modeling the product scale. To bridge this gap, we introduce JVTO-Bench, a
benchmark dataset for scale-faithful jewelry VTON, providing reference source target triplets with real-world product-scale annotations across four major jewelry categories. Building upon this
benchmark, we propose JewelTry, a mask-free
diffusion framework for scale-aware jewelry VTON. JewelTry incorporates a scale adapter that encodes product dimensions into a scale
token, enabling the model to learn scale relationships between jewelry items and surrounding human anatomy in-context. To further improve jewelry consistency, we introduce a single-directional condition
attention mechanism and an
attention refinement loss that preserve both coarse geometry and fine-grained structural details of the reference jewelry. Extensive experiments show that JewelTry achieves a balance among visual fidelity, background preservation, object consistency and scale accuracy, establishing a strong baseline for mask-free, scale-aware jewelry virtual try-on.
Read the full abstract ▾
All authors Xinlei Niu, Peixia Li, Jun Wang, Chenchen Xu, Jiayu Yang, Jing Zhang, Pulak Purkait, Hongdong Li
Computer Vision
EgoPathBench: Evaluating Zero-Shot Egocentric Waypoint Decision-Making in Vision-Language Models
Yang Zhao, Zhuo Chen and Xubo Yang
Zero-shot waypoint
navigation requires
vision-
language models to select, from the current first-person observation, a sequence of spatial actions that is feasible for the
agent and reaches the goal, placing joint demands on the integrated spatial intelligence of today's foundation VLMs. Existing spatial-intelligence
benchmarks primarily evaluate isolated judgments of relations, directions, or targets and therefore do not directly measure the integrated
navigation ability required to combine target recognition, action-consequence assessment, distance estimation, and path planning. To fill this evaluation gap, we introduce EgoPathBench, a
dataset and five-task
benchmark for first-person waypoint decision-making. Each question presents an
egocentric RGB image, a natural-
language goal, and numbered visible waypoints; a model returns traversable candidates or an ordered route. Predictions are evaluated for candidate feasibility, adjacent-edge legality, and goal arrival under point-
agent or
embodied geometry. EgoPathBench contains 31,852 training, 1,345 validation, and 1,111
benchmark questions and retains at least one geometrically verified reference route for every route question. Across nine VLMs, the highest EgoPath Score is only 28.3. The top-ranked model reaches 35.9% success on Point Path, but only 2.9% and 4.0% on
Embodied Path and Intent Path, respectively, showing that current models remain limited in forming complete, goal-consistent routes under embodiment constraints. Beyond the evaluation data, we release the corresponding training resource. Fine-tuning Qwen 3.5 4B on the released training split raises its EgoPath Score from 3.9 to 38.9 and improves all four reported evaluations across three external spatial
benchmarks, with gains of 1.4–9.6 points.
Read the full abstract ▾
Computer Vision
G3AR: Graph-Guided Neural Visual Geometry for Scalable Multi-Sequence Aerial Registration
Jeng Wen Joshua Lean, Ting-Yu Yen, Wei-Fang Sun et al. (+3 more)
Full-context neural visual geometry is impractical for thousands of images, while sequence-based chunking poorly captures irregular non-local overlap in multi-sequence aerial collections. We present Graph-Guided Neural Visual Geometry for Aerial Registration (G3AR), a graph-guided framework for scalable dense neural geometry. Before local inference, G3AR builds a geometrically verified image-proximity graph that guides bounded overlapping chunks and induces a chunk graph whose maximum spanning tree defines
alignment topology. Compatible backbones process chunks independently; shared-image predictions then estimate three-dimensional similarity (Sim(3)) transforms that register local cameras and geometry in a common frame. Across four real aerial scenes, G3AR improves pose error and runtime in matched VGGT- and Pi3-backed comparisons, while its DA3 variant achieves the lowest pose error among evaluated neural-geometry methods.
Read the full abstract ▾
All authors Jeng Wen Joshua Lean, Ting-Yu Yen, Wei-Fang Sun, Simon See, Hung-Kuo Chu, Shih-Hsuan Hung
Computer Vision
SAVOR: Self-Aware Visual Grounding via Confidence-Calibrated Reinforcement Learning for Multimodal Hallucination Mitigation
Zixiu Ding, Zilin Zhao, Yingjie He et al. (+3 more)
Multimodal large
language models (MLLMs) have made strong progress on visual question answering and image captioning, yet they still produce fluent claims about objects, attributes, or relations that are not grounded in the image. Many remedies either modify decoding at test time, which adds latency, or fine tune with preferences such as DPO variants, which teach which answer is preferred but not when the model's own answer is unreliable. We argue that calibrated self assessment is the missing signal. We introduce Savor, a training framework that (i) augments the output schema with
token and answer confidence, (ii) optimises the policy with a Group Relative Policy Optimisation (GRPO) objective that penalises calibration error and poor abstention decisions, and (iii) uses the learned confidence at inference time to revisit visual evidence only when the model is uncertain. Experiments on POPE, HallusionBench, AMBER and MMHal-Bench across two recent backbones (InternVL3-8B and Qwen3-VL-8B) show that Savor reduces
hallucination while preserving general capability on MME and MMBench, with lower Expected Calibration Error than DPO and decoding baselines.
Read the full abstract ▾
All authors Zixiu Ding, Zilin Zhao, Yingjie He, Xinlang Kang, Guansu Wang, Wei Zhang
Computer Vision
A Vision-Language Foundation Model for Precise and Comprehensive Brain Tumor Diagnosis from Preoperative Multimodal Data
Yinong Wang, Jianwen Chen, Zhou Chen et al. (+27 more)
We developed BrainVLM to classify all 12 World Health Organization (WHO) 2021
brain tumor types. BrainVLM integrates an uncertainty quantification strategy to indicate prediction reliability and a module for generating radiology reports to elucidate the clinical rationale. BrainVLM was trained on multi-modal data (MRI scans, demographics, and radiology reports) from 40,043 individuals. It was validated on 5,211 patients with pathologically confirmed
brain tumors, including 3,877 held-out patients from the primary hospital and 1,334 patients from 11 independent hospitals. We further conducted two proof-of-concept studies to validate its clinical utility in AI-clinician workflows: 1) a blinded multireader study where 12 neuroradiologists across varying experience levels interpreted 248 retrospective cases with or without AI assistance, and 2) a real-world prospective study in which 1,009 patients were independently and blindly assessed by BrainVLM and radiologists before surgery. Additionally, we demonstrated BrainVLM's utility in preoperative molecular subgroup prediction for adult-type diffuse gliomas, using a multi-center cohort of 632 patients. In primary evaluation, BrainVLM achieved an area under the curve (macro-AUC) of 0.85 (95% CI: 0.84-0.86), and an F1 score of 0.82 (95% CI: 0.81-0.83), surpassing neuroradiologists (F1 = 0.80 (95% CI: 0.79-0.81)). In external validation across 11 centers, BrainVLM achieved an AUC = 0.80 (95% CI: 0.79-0.82) and F1 = 0.75 (95% CI: 0.73-0.78), compared with F1 = 0.71 (95% CI: 0.69-0.73) for neuroradiologists. In prospective real-world evaluation, BrainVLM maintained performance comparable to neuroradiologists. The BrainVLM project page is available at https://hku-healthai.github.io/brainvlm_project.github.io/.
Read the full abstract ▾
All authors Yinong Wang, Jianwen Chen, Zhou Chen, Shuwen Kuang, Haoning Jiang, Yanzhao Shi, Huichun Yuan, Yan-ran Wang, Bing Wang, Lei Wu, Bin Tang, Li Meng, Baihua Luo, Bin Zhou, Wei Ding, Weiming Zhong, Wei Hou, Yuanbing Chen, Zhiping Wan, Wei Wang, Zhenkun Xiao, Wenwu Wan, Allen He, Yuyin Zhou, Longbo Zhang, Feifei Wang, Zhixiong Liu, Michael Iv, Xuan Gong, Liangqiong Qu