Computer Vision
NewtonGS: Physics-Structured Object-Level Neural Newtonian Dynamics for Gaussian Scene Animation
Lianlei Shan, Feiyang Ye, Yan Chen et al. (+1 more)
Animating objects in a static 3D Gaussian scene requires an explicit object-level dynamic state and a controllable model of object motion. Existing dynamic Gaussian methods primarily reconstruct time-varying scenes or simulate deformation, rather than provide compact object states for direct control. To address this gap, we present NewtonGS, a physics-structured framework for object-level state rollout and Gaussian scene animation. NewtonGS represents each object with a 22-dimensional state covering pose, linear and angular velocity, anisotropic scale and its rate, mass, and contact properties. Its Gaussian Neural Newtonian Dynamics (Gaussian-NND) model combines analytic translation, quaternion kinematics,
gravity, damping, and scale-restoration dynamics with learned continuous and contact residuals. A discrete event map handles floor contact. Predicted poses and scales define a shared affine transformation that updates the means and covariances of all Gaussians associated with each object. We construct two procedurally generated
datasets: State-32 for state-rollout evaluation and Gaussian-32 for state-to-Gaussian transformation. On both the in-distribution and velocity-range-shift splits of State-32, NewtonGS achieves lower trajectory RMSE, final displacement error, and velocity RMSE than five analytic baselines. Experiments on Gaussian-32 further demonstrate effective conversion from predicted states to animated Gaussian objects.
Read the full abstract ▾
All authors Lianlei Shan, Feiyang Ye, Yan Chen, Yong Wu
Computer Vision · Cross-listed from cs.RO
LIRA: Local Cross-Layer Information Routing for Vision-Language-Action Decoding
Zhewei Zhang, Puyue Wang, Guanren Qiao et al. (+10 more)
Vision-
Language-Action (VLA) models transform representations from pretrained
vision-
language models (VLMs) into
robot actions, yet the interface that routes intermediate VLM features into action decoders remains underexplored. Existing designs either expose only a narrow part of the representation hierarchy or rigidly match each decoder block to one VLM layer, restricting access to complementary task evidence across depths. We introduce LIRA, a local cross-layer action-conditioning mechanism that formulates VLM-to-action conditioning as depth-aware information routing. LIRA operates on task-
token features and LIRA Query features derived from intermediate VLM states, then assigns each Parallel Fusion Block a depth-aligned local window centered on its corresponding VLM layer. Parallel Fusion Blocks aggregate neighboring LIRA Query features and integrate them with task-
token features and proprioceptive inputs before action prediction. This routing interface leaves the backbone architecture, action decoder, and supervised training recipe unchanged. Across LIBERO, LIBERO-Plus, CALVIN ABC$\rightarrow$D, and real-world manipulation, LIRA improves the principal aggregate metrics over the VLA-Adapter baseline under the same 0.5B-parameter configuration. In zero-shot transfer to LIBERO-Plus, LIRA increases average success from 59.1% to 78.0%, an 18.9-point gain indicating improved robustness under controlled distribution shifts.
Read the full abstract ▾
All authors Zhewei Zhang, Puyue Wang, Guanren Qiao, Yijie Weng, Jiawei Hu, Guo Li, Lujia Wang, Junyan Wang, Tao Gu, Hongliang Lu, Guiliang Liu, Hong Jia, Xinhu Zheng
Computer Vision
Do 3D Medical Foundation Models See Through MRI Artifacts? A Controlled Study of Representation Robustness
Julia Anna Mielcarz, Daniel Klaaby and Mostafa Mehdipour Ghazi
Self-supervised 3D medical foundation models are increasingly used as general-purpose feature extractors, yet their sensitivity to MRI artifacts remains poorly understood. We present a controlled evaluation of representation robustness across five pretrained 3D encoders spanning different architectures, objectives, pretraining domains, and
dataset scales. Using BraTS-Africa cases with four MRI sequences, we generate seven frequency- and image-domain artifacts at five predefined corruption settings. Robustness is assessed using linear centered kernel
alignment (CKA), RankMe, and UMAP, complemented by an independent
segmentation-consistency analysis. We find that robustness is strongly model- and artifact-dependent. 3DINO exhibits the most consistently stable representations, while BrainIAC is highly sensitive to several corruptions; NeuroVFM, BrainFM, and Neuro-SimCLR show intermediate but distinct artifact-specific profiles. Across many conditions, CKA decreases substantially while RankMe remains comparatively stable, indicating that artifacts often distort representation geometry without causing dimensional collapse.
Segmentation consistency also degrades under corruption, particularly for ghosting and Rician noise, but aligns only partially with representation-level robustness. These findings show that larger-scale or domain-specific pretraining alone does not guarantee artifact invariance and motivate explicit robustness evaluation before deploying 3D foundation models in heterogeneous MRI settings.
Read the full abstract ▾
Computer Vision
SLED: Scalable Location Encoding via Distillation
Kevin Lane, Zhongying Wang, Esther Rolf et al. (+1 more)
The plethora of readily available geospatial data offers exciting opportunities to learn high quality representations of the
planet, but the sheer size of the Earth Observations (EO), differing modalities, and different sensor types pose significant challenges in doing so. Location encoders have emerged as an efficient way of compressing EOs into location-specific
embeddings. However, current state-of-the-art location encoders rely on computationally expensive CLIP-style frameworks that require large batch sizes in the 16K–32K range, suffer from false negative samples, and scale poorly with additional modalities. We introduce the Scalable Location Encoder via
Distillation (SLED), a
distillation-based location encoder that uses geospatial location as a binding modality to pretrain location encoders with any modality of geospatial data. The resulting location encoder framework is lightweight, modular, and can flexibly incorporate multiple modes, while eliminating the need for spatiotemporal coregistration of samples. SLED is performant with batch sizes as small as 128, enabling pretraining at a fraction of the runtime and compute costs of current state-of-the-art models. We demonstrate our approach by pretraining unimodal and
multimodal SLED models on Sentinel-1, Sentinel-2, and Landsat imagery. We show that both unimodal and
multimodal SLED models keep pace with or outperform existing approaches on a diverse set of 19 human-centric
benchmark tasks and explore the benefits of using additional modes in pretraining.
Read the full abstract ▾
All authors Kevin Lane, Zhongying Wang, Esther Rolf, Morteza Karimzadeh
Computer Vision
Toward surface-based registration of a virtual preoperative cutting guide onto the mandible for reconstruction surgery
Yue Yang and Jie Ying Wu
Mandibular reconstruction restores facial continuity and oral function after segmental resection. Patient-specific cutting guides transfer a computed
tomography (CT)-based plan to the operating room with three-dimensional information, but printed guides add cost and lead time, cannot adapt after fabrication, and may interrupt surgery if sterility is lost. We investigate a markerless augmented reality (AR) alternative that registers a virtual cutting guide to the exposed mandible from surface geometry. The method extends surface-based registration for the transoral setting, where teeth form the most distinctive visible surface. After camera calibration, a HoloLens 2 time-of-flight camera captures a partial intraoperative point cloud. The user supplies a rough head-based
alignment only to crop the region of interest. A teeth-weighted global stage computes correspondences and solves a truncated least-squares rigid
alignment. An asymmetric point-to-plane iterative closest point (ICP) stage refines the complete CT mandible against the partial depth cloud target. The guide-to-mandible transform places the guide in the HoloLens world frame, while pose updates and interpolation follow target motion. We define a blinded phantom protocol with 30 target registration error (TRE) points under full, intermediate, and teeth-only exposure, plus a motion-to-display latency test. Our median TRE is 4.05, 6.10, and 7.10 mm respectively, and median latency is 0.805 s. These values support the feasibility of using AR to replace physical prints. The workflow removes mounted fiducials and manual landmark selection and provides a testable path toward transoral cutting guidance.
Read the full abstract ▾
Computer Vision
Improving Low-Resolution Face Recognition under Limited Data: How Synthetic Data Generation Can Close the Domain Gap
Luis S. Luevano, Ünsal Öztürk, Hatef Otroshi Shahreza et al. (+2 more)
Face Recognition (FR) systems in surveillance settings often encounter Low Resolution (LR) faces, those whose face region falls below the standard 112 $\times$ 112 input size. While labelled High Resolution (HR) training data is abundant, labelled native-LR data, and above all paired native LR/HR data, is scarce. One workaround is to synthesize LR data from the available HR faces, but how much synthesis effort is repaid in recognition accuracy remains unclear. We present a study of simple synthetic generation strategies for a compact, edge device-oriented face recognition system, spanning interpolation-based degradation, knowledge
distillation, a Prepended Domain
Transformer (PDT), Real ESRGAN-style degradation, and a learned Super Resolution (SR) front-end with an identity-aware loss. We evaluate these strategies on synthetic cross-resolution face
benchmarks (LFW, CFP-FP, AgeDB-30) and on TinyFace, a real-world native LR
dataset, and expose a synthetic-real gap: the degradation setting that is optimal on synthetic
benchmarks is not the one that is optimal on real LR. We find that more synthesis effort does not help monotonically: the learned SR front-end does not surpass a direct feed of the aligned LR image into a strong backbone, while simple interpolation augmentation of a compact backbone is the only synthesis that improves over its own baseline. We conclude that generative methods for LR face recognition must be validated on real LR and against a direct-feed baseline, and release our pipeline at https://idiap.ch/paper/synth-lrfr
Read the full abstract ▾
All authors Luis S. Luevano, Ünsal Öztürk, Hatef Otroshi Shahreza, Anjith George, Sébastien Marcel
Computer Vision · Cross-listed from eess.IV
Implicit Neural Speckle Denoising
Matthew R. Ziemann, Casey J. Pellizzari, Tyler J. Hardy et al. (+1 more)
Speckle fundamentally limits coherent imaging by introducing multiplicative, spatially correlated noise that obscures scene structure. Removing speckle noise from dynamic scenes–that do not benefit from conventional speckle averaging–is particularly challenging. We introduce a training-free framework that combines a spatiotemporal implicit neural representation with an aperture-aware maximum-likelihood formulation to recover dynamic, speckle-free imagery directly from noisy observations. The coherent likelihood explicitly models the aperture-dependent spatial covariance of speckle, enabling adaptation to arbitrary pupil geometries without retraining. A
matrix-free implementation based on FFT-accelerated operators,
stochastic approximations, and conjugate gradients makes optimization practical for realistic image sizes. Meanwhile, a blind holdout criterion provides automatic early stopping without clean reference data. Simulated and laboratory results demonstrate improved spatial fidelity, temporal consistency, and robustness to varying speckle statistics relative to classical, unsupervised, and supervised baselines.
Read the full abstract ▾
All authors Matthew R. Ziemann, Casey J. Pellizzari, Tyler J. Hardy, Christopher A. Metzler
Computer Vision · Cross-listed from cs.AI
TaskSense: Focusing on What Matters in World Models
SM Mazharul Islam and Manfred Huber
World models for visual control typically learn compact latent states by reconstructing observations, implicitly encouraging representations to preserve information across the entire visual input. However, task-relevant content often occupies only a small fraction of the observation, while background clutter and distractors consume valuable representational capacity. This mismatch between visual reconstruction and control objectives biases latent representations to model task-irrelevant visual content, diluting learning signals for control-relevant features and severely degrading downstream performance under visual distractions. We introduce TaskSense, a task-centric world modeling framework that enforces task relevance before latent encoding through a differentiable
stochastic spatial
attention mechanism conditioned on the previous latent state. To steer
attention toward control-relevant regions, we augment training with an auxiliary inverse-dynamics objective. Rather than reconstructing the full observation, the world model reconstructs only the attended regions, encouraging latent representations to preserve task-relevant information while discarding irrelevant visual content. The decoder is further conditioned on the sampled
attention map, enabling consistent reconstruction despite
stochastic attention. Compared with the DreamerV3 baseline, TaskSense maintains competitive performance on the DeepMind Control Suite while consistently outperforming DreamerV3 on the Distracting Control Suite, demonstrating substantially improved robustness to visual distractions. Qualitative analysis further confirms that the learned
attention, guided by inverse-dynamics supervision, consistently localizes control-relevant regions while suppressing irrelevant visual content.
Read the full abstract ▾
Computer Vision
InsertFuse: A Unified Framework for Multi-Category Reference-Guided Image Insertion
Guangzhao Li, Qingyan Wei, Huayu Zheng et al. (+7 more)
We present InsertFuse, a unified framework for multi-category reference-guided image insertion. Its key idea is to decouple category-specific expertise learning from cross-category capability consolidation. InsertFuse first trains specialized experts for different insertion categories and then introduces Insertion On-Policy
Distillation (IOPD) to consolidate their capabilities into a single student. By querying the matched expert at states visited by the student, IOPD preserves category-specific insertion behavior while mitigating the cross-category interference caused by direct joint training. To improve spatial control, we propose
Token-Aligned Geometry Conditioning (TAGC), which maps mask-derived
geometric cues to the visual
token grid, and Region-Balanced Flow Matching, which separately normalizes prediction errors inside and outside the insertion region to prevent background-dominated and scale-dependent supervision. We further introduce Reference CFG to isolate and strengthen the guidance induced by the visual reference under fixed scene and geometry conditions, with IOPD transferring this enhanced supervision into the unified student. Extensive experiments on the public AnyInsertion
benchmark and our multi-category test set demonstrate state-of-the-art performance on most metrics, showing strong reference fidelity and generation quality across diverse insertion categories.
Read the full abstract ▾
All authors Guangzhao Li, Qingyan Wei, Huayu Zheng, Yige Zheng, Chaoyang Zhang, Jie Yang, Yunan Ding, Yan Tai, Siqi Luo, Xiaohong Liu
Computer Vision
Test-Time Adaptation with Online Personalized Energy-Based Cache for Fine-Grained Video Expression Recognition
Masoumeh Sharafi, Muhammad Osama Zeeshan, Soufiane Belharbi et al. (+3 more)
Facial expression recognition (FER) in
videos is challenging because models must identify subtle, temporally evolving affective states that vary across individuals. Although
vision-
language models provide transferable visual-semantic representations, models trained on subject-independent data often degrade under subject-specific distribution shifts at inference time. Existing test-time adaptation (TTA) methods commonly update model parameters during inference, increasing computational cost and latency. Cache-based methods avoid parameter updates, but they usually require enough target samples to form reliable class prototypes, which is difficult early in adaptation and for rarely observed classes. We introduce Energy-Based Cache Personalization (EB-CaP), a subject-based online TTA method for
video FER that generates class-specific prototypes personalized to each target
video. EB-CaP uses a lightweight energy-based model to sample prototypes from the current unlabeled
video and populate a personalized cache online, without accumulating large amounts of target data or storing diverse source prototypes. Its energy function relies only on pretrained CLIP: similarities between the target
video embedding and class text
embeddings guide prototype sampling. In parallel, positive and negative caches store reliable and uncertain target
embeddings. An adaptive
entropy gate controls cache updates according to the evolving confidence distribution, while a diversity gate limits redundant samples. Final predictions combine cache-derived scores with the current CLIP scores. Experiments on BioVid, StressID, and BAH show that EB-CaP outperforms state-of-the-art TTA methods while maintaining low computational and
memory overhead. Code is available at https://github.com/MasoumehSharafi/EB-CaP.
Read the full abstract ▾
All authors Masoumeh Sharafi, Muhammad Osama Zeeshan, Soufiane Belharbi, Alessandro Lameiras Koerich, Marco Pedersoli, Eric Granger
Computer Vision
Does FLAIR super-resolution erase or hallucinate small white-matter lesions?
Zahra Khodakarami, Yue Li, Pulkit Khandelwal et al. (+5 more)
White matter hyperintensities (WMH), bright regions on Fluid-attenuated Inversion Recovery (FLAIR) scans are associated with cerebrovascular pathology and neurodegeneration. FLAIR is usually acquired with thick slices in clinical settings, giving it poor through-plane resolution. Super-resolution (SR) is a widely used method for recovering an isotropic volume from an anisotropic scan. Yet whether applying it prior to WMH
segmentation preserves lesion content remains unknown: a model may erase small real lesions or hallucinate absent ones. We used 1-mm isotropic high-resolution (HR) FLAIR scans from 29 individuals in the ADNI cohort, each manually segmented for WMH by an expert. Then, we degraded each to simulated 3 and 5 mm through-plane acquisitions. Multi-contrast implicit neural representation (INR), a single-contrast self-supervised model (ECLARE), and cubic interpolation were used to upsample them onto the HR grid. WMH
segmentation from a simulated thick slice and the original HR FLAIR set the floor and ceiling, respectively, for the per-lesion analysis. Of four WMH
segmentation methods (WMH-SynthSeg, segcsvd, MARS-WMH, TrUE-Net), we ran the analysis under the most sensitive one to small lesions on HR (MARS-WMH) with the evaluation metrics of detection sensitivity, erasure rate (HR-detected lesions lost after reconstruction), and
hallucination rate (predicted components absent from both the manual and HR
segmentation). The dominant effect of SR was erasure of small real lesions, not
hallucination, and it increased with slice thickness, though every reconstruction still improved lesion detection over the raw thick slice. ECLARE recovered small lesion signal best at both thicknesses, while the INR was no better than cubic interpolation.
Read the full abstract ▾
All authors Zahra Khodakarami, Yue Li, Pulkit Khandelwal, John Detre, Sandhitsu Das, Christopher Brown, David Wolk, Paul Yushkevich
Computer Vision
UQ-Loc: Uncertainty-Aware LiDAR Scene Coordinate Regression
Jacek Komorowski
LiDAR-based Scene Coordinate Regression (SCR) maps point clouds directly to 3D scene coordinates, enabling precise 6-DoF localisation without explicit map
retrieval. However, existing methods produce deterministic predictions, discarding aleatoric uncertainty that could improve robustness and downstream decision-making. We present UQ-Loc, which extends the LightLoc architecture with an anisotropic Gaussian covariance head that predicts a full 3x3 positive-definite covariance
matrix per voxel. Training uses a Negative Log-Likelihood (NLL) loss augmented with a kNN-based spatial smoothness regulariser, while inference employs a modified SC2-PCR solver with uncertainty-weighted seed scoring and a Mahalanobis-distance inlier test. We adopt Expected Calibration Error (ECE) as a principled metric for evaluating the quality of the predicted uncertainty. Experiments demonstrate that UQ-Loc achieves consistent improvement in 6-DoF localization accuracy while producing well-calibrated covariances.
Read the full abstract ▾
Computer Vision
TLNM: Externally Validated Tooth Detection, Numbering and Segmentation from Smartphone Photographs Using Mask R-CNN
Arash Nedaei, Henna Tiensuu, Elina Väyrynen et al. (+2 more)
Oral health issues affect billions of people globally, but the cost and limited access to professional dental care hinder preventive oral healthcare. Research relies on clinical-grade sensors, unavailable for public self-screening. This study introduces a tooth localisation and numbering model for smartphone photographs. We developed a customised Mask Region-based Convolutional Neural Network pipeline trained on 1,272 annotated smartphone images. To address variability in patient-generated health data, the pipeline incorporates two domain-informed mechanisms: a masked gray-world white-balancing
algorithm to mitigate artificial colour casts and an anatomically constrained detection layer to enforce structural validity and suppress false positives. The system was evaluated using internal and external testing, descriptive ablation study and training stability analysis. On the internal test set, the model achieved an instance-mask AP50 of 0.818, class-aware PQ of 0.780, and operational F1 of 0.884. On the external
dataset, the model achieved an instance-mask AP50 of 0.901, class-aware PQ of 0.832, and operational F1 of 0.928 despite differences in population, sensors, and acquisition protocols. The inference pipeline is available as an open-source, containerised API. These results demonstrate that consumer-grade smartphone imagery can support automated tooth-level anatomical mapping, offering a scalable, potentially low-cost foundation for remote screening and tele-dentistry in resource-constrained environments.
Read the full abstract ▾
All authors Arash Nedaei, Henna Tiensuu, Elina Väyrynen, Saujanya Karki, Jaakko Suutala
Computer Vision
OTLesMix: Wasserstein Barycenter and Optimal Transport Map for Synthetic Lesion Generation with Diverse Shapes and Locations
Robin Trombetta and Carole Lartizien
The development of deep learning over the past decade has revolutionized medical imaging
segmentation, allowing the extraction of precise descriptors from large volumes to characterize pathologies. Data augmentation is a technique widely regarded as a way to improve model training. It includes simple transformations like spatial operations or intensity modifications, but also more advanced synthesis techniques. Their goal is to generate new realistic samples from an existing
dataset to diversify the images used during training. Among them, several propose different mixing strategies to combine real samples. However, one of their major shortcomings is to yield limited variability in terms of generated lesion shapes and locations. In this work, we introduce a novel image synthesis method, called OTLesMix, that leverages Wasserstein barycenter and optimal transport plan to generate realistic and diverse samples. We evaluated our method on three
brain lesion
segmentation tasks, on which it improves the Dice score compared to a model trained without synthetic data by 2.9 to 6.6 points, and outperforms state-of-the-art mix-based methods.
Read the full abstract ▾
Computer Vision
MASS: Multiplayer World Models with Authoritative Shared State
Ziqi Cai, Siqi Yang, Yimu Wang et al. (+6 more)
Current
video world models struggle in multiplayer environments because they entangle world state with view-dependent visual latents, leading to redundant compute, view inconsistencies, and poor scalability. We propose MASS (Multiplayer world models with Authoritative Shared State) to resolve this limitation. Inspired by multiplayer game architectures, MASS disentangles world dynamics and view rendering. A learned Logic Engine advances a global, authoritative typed state from joint actions without any hand-written transition function, acting as the sole recurrent
memory and synchronization reference. From this shared state, a learned Rendering Engine generates independent and consistent views for any requested camera on demand. This explicit disentangling allows MASS to achieve superior state accuracy and lower cross-view inconsistency compared to state-of-the-art multi-view baselines on a matched multiplayer Snake
benchmark. It advances predicted worlds with 1,024 concurrent players for 10,000 recurrent steps. Our results show that explicit, authoritative state modeling provides a practical foundation for scalable and consistent multi-
agent world simulation.
Read the full abstract ▾
All authors Ziqi Cai, Siqi Yang, Yimu Wang, Zixian Gao, Yunheng Liu, Shuchen Weng, Erwin Wu, Kaipeng Zhang, Boxin Shi
Computer Vision
Toward Deployable Bangla Sign Language Recognition with Expert-Validated Data and a Lightweight Attention-Based Model
Saad Ahmed and Md Khalid Syfullah
Deaf and hard-of-hearing people in Bangladesh communicate mainly through Bangla Sign
Language (BdSL). Automatic BdSL recognition on personal devices could widen access to education and services. Existing systems use controlled-setting
datasets without expert verification and heavyweight pretrained backbones unsuited to on-device use. We introduce RSBdSL38, 10,874 expert-validated images spanning all 38 BdSL hand signs, representing the 51 letters of the Bangla alphabet, recorded from real signers at three special-needs schools across Bangladesh. We propose a lightweight
attention based convolutional network of 298,470 parameters, built from grouped bottleneck residual blocks, channel and spatial
attention, a multi-scale depthwise hand-feature block, dual pooling, and Swish activations. Trained from scratch, it attains 96.37% accuracy (95.72% +- 0.54% over five seeds), within 1.08 percentage points of the best of nine ImageNet-pretrained efficient architectures under an identical protocol, using 8.5 to 68x fewer parameters and 1.3 to 21.7x fewer MACs. Retrained, it reaches 92.95 to 98.33% on six public BdSL
benchmarks, 97.04% on a merged corpus, and 76.25% zero-shot on BdSL-38. Removing any architectural stage costs 7.61 to 89.30 points, against at most 3.17 for the training recipe. Grad-CAM with deletion-insertion and weight-randomization checks confirms that predictions follow the signing hand. A signer-independent split holding out 6 of 36 signers yields 85.18%. Quantized to 0.48 MB, it runs at 3.98 ms per image within a 15.5 MB footprint on a commodity smartphone. Together, RSBdSL38 and our from-scratch model turn
benchmark accuracy into deployable accessibility at a fraction of pretrained-backbone cost;
dataset, code, and models are released.
Read the full abstract ▾
Computer Vision
PRISM: Distribution-Gated Flow Matching for Controllable Unpaired Image Translation
Elad Yoshai and Natan T. Shaked
Unpaired image-to-image translation must decide, per image, what to change and what to preserve without paired supervision. Many
diffusion-based unpaired translators control preservation through a single global noise or guidance value applied across the image, which cannot separate content to keep from appearance to change. We present PRISM, a GAN-free flow-matching framework that replaces this global control with a learned per-feature gate. The gate's spatial prior is derived from each source feature's standardized distance to the target feature distribution, so features far from the target are freed while target-consistent features are preserved. The same gate controls both the initialization, which mixes the real source latent with a task-matched corruption, and the transport timing during Ordinary Differential Equation (ODE) integration. The corruption is matched to the task, content-anchored (AdaIN) for structure-preserving translation and partially anchored for structure-changing translation, and the gate can be overridden locally at inference from text or a detector without retraining, preserving important structures of the original image while still generating realistic results. We evaluate PRISM on five natural and biomedical
benchmarks (AFHQ cat->dog, CelebA-HQ appearance translation, day->night relighting, virtual staining, and breast frozen->permanent histopathology). Among the evaluated methods under a shared same-split protocol, PRISM attains the best Inception FID and KID on four
benchmarks and a competitive result on the fifth, and on histopathology yields the
nuclei-count ratio closest to the ideal, supporting a favorable balance between target realism and structural preservation.
Read the full abstract ▾
Computer Vision
Depth-Guided Video Object Counting in Crowded Scenes
Yuanjing Xu, Xinyan Liu, Weidong Chen et al. (+5 more)
Our primary objective is to advance
video object counting in crowded scenes, aiming to robustly count all instances of a target category based on given text or visual
prompts. Existing methods rely on RGB information, limiting their discriminative ability in crowded and occluded conditions. To address this, we propose a Depth-Guided Detector (DG-Det) along with a general post-processing pipeline. By integrating depth cues with multi-scale RGB-D cross-
attention and explicit occlusion prediction, our method enhances spatial understanding and achieves robust detection in crowded and occluded scenes. Furthermore, we introduce a unified de-duplication framework to eliminate cross-frame redundant counting. To facilitate future research, we also release a new RGB-D
Video Object Counting
dataset featuring depth information and multiple object categories persequence. Extensive experiments demonstrate that our method achieves a 62.01% reduction in MAE compared to existing baselines, and also produces consistent improvements in RMSE. We provide the source code at https://github.com/streamer-AP/DG-Net and the
dataset at https://huggingface.co/
datasets/aerospace123/RGBD-VideoCount.
Read the full abstract ▾
All authors Yuanjing Xu, Xinyan Liu, Weidong Chen, Zixuan Zou, Linhao Zhang, Zhuangzhe Meng, Antoni B. Chan, Weigang Zhang
Computer Vision
EmoWorld: A Decoupled Affective Field for Controllable Emotional Video Generation
Bingyuan Wang, Baistan Zhyldyzbekov, Kunyu Feng et al. (+1 more)
Emotion shapes how viewers interpret a scene, yet existing
video generators entangle global atmosphere, affect-bearing semantic cues, and temporal progression within a single text condition. We present EmoWorld, a framework that decouples these factors within a frozen flow-matching
video diffusion transformer (
Video DiT). A one-time preparation stage extracts layer-specific affect directions and a reusable cue library from geometry-preserving neutral and
emotion-edited panoramas. At inference, Visual Atmosphere Steering (VAS) injects atmosphere directions into hidden states, Semantic Affective Steering (SAS) isolates a separately scalable
prompt residual for semantic cues, and Temporal Affective Steering (TAS) interpolates endpoint residual fields across denoising and
video time. On Wan2.2, VAS improves target-
emotion alignment by 19% while reducing a temporal-fluctuation proxy by 48%; SAS improves target-
emotion alignment by 37% and increases detected affect-bearing cues by 36%; and TAS improves transition monotonicity by 15% over the strongest baseline. EmoWorld is evaluated across 27
emotion categories in text-to-
video and image-to-
video settings, demonstrates portability across multiple
Video-DiT backbones, and supports camera-conditioned composition without updating generator parameters.
Read the full abstract ▾
All authors Bingyuan Wang, Baistan Zhyldyzbekov, Kunyu Feng, Zeyu Wang
Computer Vision · Cross-listed from cs.CR
Reversible Unlearnable Examples: Towards the Copyright Protection in Deep Learning Era
Binze Wang, Jinyu Tian, Xingrun Wang et al. (+2 more)
Significant advancements in deep learning have been made possible by the utilization of large
datasets, underscoring the critical importance of copyright protection. Adding meticulously designed perturbations to examples, making them unlearnable has become a crucial approach for safeguarding data copyright. Existing methods for creating unlearnable examples overlook the risk of data leakage, which can threaten data ownership. Thus, copyright protection in deep learning faces two main threats: illegal model training and malicious data leakage. We investigate that these two threats cannot be solved by straightforwardly combining existing availability attacks and watermarking techniques as their negative interaction effects. Therefore, in this paper, we propose a novel copyright protection mechanism for the aforementioned security concerns. Considering that the prevention of unauthorized model training requires powerful generalizability of unlearnable perturbations, we generate perturbations to induce the model to learn uncorrelated features of input images. It works by minimizing the mutual information of the input and output of the model. On the other hand, to eliminate the side impact of unlearnable perturbations on the watermark extraction, we design a dual extraction strategy by using two distinct watermark extractors. Extensive experiments on the image
datasets {ImageNet, CIFAR10, and Pets} show that our proposed method could provide comprehensive copyright protection to images. The code is available at {https://github.com/Yeah21/ReversibleUnlearnableExamples}.
Read the full abstract ▾
All authors Binze Wang, Jinyu Tian, Xingrun Wang, Xiaochen Yuan, Jianqing Li
Computer Vision
CFGPNet: Cross-Attention-Based Fused Gradient Programmed Network Framework for Multispectral Object Detection
Nima Hatami, Karim Faez, Saeed Sharifian et al. (+1 more)
Multispectral object detection combines visible and thermal imagery to improve perception under challenging illumination and environmental conditions. However, differences in modality appearance and reliability can introduce redundant or conflicting responses, limiting the use of complementary information. Complex fusion mechanisms further increase computational cost, creating a persistent trade-off between detection accuracy and efficiency. To address these challenges, CFGPNet is proposed, a cross-
attention-based fused gradient programmed network. The framework incorporates re-parameterized RepViT blocks into the YOLOv9 architecture to strengthen spatial and channel representations while maintaining efficient feature extraction. Cross Computation Efficient
Attention (CrossCEA) exchanges spatial
attention maps between modalities at multiple detection scales, allowing each stream to emphasize regions supported by the other while preserving modality-specific information.
Attention Selection and Aggregation Fusion (ASAF) combines dense feature aggregation with selection of the strongest responses from multiple
attention branches to form compact, discriminative fused representations. A programmable gradient information pathway provides auxiliary supervision during training to improve feature learning. This pathway is removed after training, adding no parameters or operations at inference. Experiments on FLIR, M3FD, LLVIP, VEDAI, and MFAD demonstrate favorable accuracy-efficiency trade-offs across three model scales, with the smallest variant requiring 15.3 million parameters and 56.9 GFLOPs. The code is available at https://github.com/NimaHatami99/CFGPNet.
Read the full abstract ▾
All authors Nima Hatami, Karim Faez, Saeed Sharifian, Hamidreza Amindavar
Computer Vision
HOPE: Hand-Object Pressure Estimation from Monocular Videos
Subin Jeon, Byungjun Kim and Hanbyul Joo
Estimating physical pressure from
vision is essential for understanding contact-rich hand-object interaction. However, prior
vision-based pressure estimation methods are largely limited to planar surfaces and single image input, making them difficult to apply to dynamic hand-object interaction with diverse objects. We instead formulate pressure estimation as a hand-centric
video prediction problem with
monocular video as input. This formulation predicts temporally evolving per-vertex normal pressure and contact directly on the hand mesh, yielding a unified output space independent of object shape and sensor layout. Building on this formulation, we propose HOPE, a framework with two key components. First, we lift tactile-glove pressure, planar-sensor pressure, and distance-based hand-object contact annotations into a shared hand vertex space, allowing bare-hand contact data to regularize pressure learning where metric labels are unavailable. Second, we introduce a vertex-anchored
video transformer that treats each vertex as a persistent
token, aggregates visual features and hand pose over time, and uses a contact-gated pressure head to enforce that pressure vanishes without contact. Experiments on OpenTouch, PressureVisionDB, and hand-object contact
benchmarks validate HOPE across object-pressure, surface-pressure, and contact-supervised HOI settings. Despite using metric pressure supervision primarily from gloved-hand
videos, HOPE generalizes to bare-hand
egocentric and in-the-wild
videos, producing joint contact and pressure predictions beyond the scope of contact-only or planar-pressure baselines.
Read the full abstract ▾
Computer Vision
EvReflection: Event-Driven Micro-Dynamics for Reflection Removal
Jiaxiao Wang, Dachun Kai, Huyue Zhu et al. (+3 more)
Despite remarkable progress in reflection removal, current methods primarily exploit static image priors from a single frame and still suffer from severe residual artifacts due to the inherent ambiguity between the reflection and transmission layers. In this paper, we propose leveraging event signals to break this ambiguity. By employing event cameras to capture micro-dynamics, we reveal the differential motion between these two layers. We thereby present a novel event-driven reflection removal network, EvReflection, that utilizes these dynamic cues for layer separation. Specifically, we design a Micro-Dynamics Decoupler to disentangle layer-specific motions from event streams as priors, which then guide a Parallax-
Attention Rectifier to cleanly remove artifacts from the RGB image. Furthermore, to address data scarcity, we develop a parallax-aware simulation pipeline and construct the EVR$^2$
benchmark dataset, the first real-world
dataset for this task. Extensive experiments demonstrate that EvReflection achieves state-of-the-art performance on both synthetic and real-world
benchmarks, surpassing the best competing method by more than 1.6 dB and 1.2 dB in PSNR, respectively. The code,
dataset, and pre-trained models are available at https://github.com/JiaxiaoWang/EvReflection.
Read the full abstract ▾
All authors Jiaxiao Wang, Dachun Kai, Huyue Zhu, Quanquan Hu, Zhenyang Xu, Xiaoyan Sun
Computer Vision
Support Operation Factorization: Compositional Readout of Frozen Vision Encoders under Controlled Interventions
Zhongyao Wang, Wanli Ouyang, Taoyong Cui et al. (+1 more)
Compositional analysis of frozen
vision encoders should determine both what changed and where it changed. Standard factor probes score these axes separately, however, and can reward multiple operations that reuse the same predicted slot. We call this failure operation laundering. We introduce an injectively aligned leave-one-
cell-out protocol over support x operation grids and SO-OPF, a readout that factors
cell energy into support salience and a competitive operation posterior. This formulation separates two questions that aggregate scores conflate: whether the carrier composes held-out bindings when the grid is known, and whether that grid can be recovered from flat
cell labels. With frozen DINOv3 features, known factorial assignment reaches 0.874 injective accuracy on Shapes3D-Extended and 0.799 on globally image-disjoint COCO; learning the assignment from flat labels reaches 0.769 and 0.762, respectively. Under matched-axis-aware supervision on Shapes3D, the factored carrier improves learned-assignment accuracy from 0.653 to 0.841 over a dense carrier and eliminates its laundering gap. SigLIP2 replicates the COCO separation. A rebuilt MuJoCo substrate exposes a boundary: learned-assignment accuracy is 0.569 with DINOv3 and 0.484 with SigLIP2, with substantial slot collapse. Thus factored readout and injective evaluation recover held-out bindings on two substrates while exposing, rather than hiding, a renderer-specific failure boundary; they do not establish universal recovery from flat labels.
Read the full abstract ▾
All authors Zhongyao Wang, Wanli Ouyang, Taoyong Cui, Pheng Ann Heng
Computer Vision · Cross-listed from cs.RO
Prior-SG: Task and Prior Driven Region Segmentation for Scene Graphs in Arbitrarily-Structured Environments
Giorgio Tonetti, Laurent Kneip, Abel Gawel et al. (+1 more)
Hierarchical 3D scene graphs are a promising representation for high-level spatial
reasoning in autonomous mobile platforms. However, existing extraction frameworks typically rely on purely local visual clustering or strict
geometric heuristics, such as wall-separated rooms, which fail in open-plan or arbitrarily-structured environments. We propose Prior-SG, a task- and prior-driven framework that casts scene graph generation fundamentally as a probabilistic
alignment problem. As the
robot explores, it continuously aggregates an incoming RGB-D sensor stream into a physically grounded Instance Graph utilizing a multi-scale, open-vocabulary feature fusion strategy. The system then infers the high-level functional semantics of this map through a Maximum A Posteriori (MAP) estimate, guided by a Prior Graph-a logical expectation of the environment's structure and task-relevant vocabulary synthesized dynamically by a Large
Language Model. By optimizing a Markov Random Field that fuses heterogeneous experts (visual,
geometric, and discrete objects) with these topological priors, the system resolves local perceptual ambiguities. We validate this approach across diverse simulated residential
datasets and large, open-plan real-world environments. Prior-SG achieves state-of-the-art semantic region
segmentation accuracy compared to recent baselines, robustly delineates distant functional boundaries in the absence of physical walls, and uniquely provides zero-shot ontological flexibility, enabling the
robot to entirely restructure its spatial partitioning based on a given high-level task.
Read the full abstract ▾
All authors Giorgio Tonetti, Laurent Kneip, Abel Gawel, Marco Hutter
Computer Vision
BendTwin: Robust Dense-to-Sparse Physical Reconstruction with Bending-Aware Differentiable Spring-Mass Models
Yixiong Jing, Qi Wang, Lin Chen et al. (+6 more)
Reconstructing objects with mechanical properties from
video observations enables physically consistent dynamic prediction, benefiting robotics planning and interaction. Existing spring–mass based physical driven reconstruction approaches offer efficient and differentiable physical reconstruction, but they typically rely on axial springs alone. Such formulations oversimplify the underlying structural mechanics and can become mechanically under-constrained when the physical graph is coarsened, limiting their ability to preserve stable local deformation. We present BendTwin, a bending-aware differentiable spring–mass framework for
video-based reconstruction and future prediction of deformable objects. BendTwin introduces bending stiffness and damping over local surface triplets, penalizing deviations from rest angles and regularizing higher-order deformation. These bending constraints improve mechanical stability while preserving the simplicity of spring–mass system. Experiments show that BendTwin consistently outperforms the axial-only PhysTwin baseline. Ablation studies further demonstrate that the bending constraints maintain system stability across different downsampling ratios and consistently improve upon the original PhysTwin formulation. Overall, BendTwin provides an effective approach for constructing mechanically faithful digital twins from sparse-view RGB-D
videos.
Read the full abstract ▾
All authors Yixiong Jing, Qi Wang, Lin Chen, Junwei Jiang, Guangming Wang, Haibing Wu, Olaf Wysocki, Wanli Ma, Brian Sheil
Computer Vision · Cross-listed from cs.RO
Visual Grounding in Zero-Shot Vision-Language Control
J. de Curtò, Dayani Plasencia, Diego Sánchez et al. (+1 more)
Vision-
language models (VLMs) are increasingly used as zero-shot controllers, but successful trajectories do not necessarily show that decisions are grounded in visual input: simulator dynamics and conservative action priors can produce favourable scores without meaningful perception. We investigate this with an input-ablation battery: blind-image controls, repeated identical inputs, lane-axis reflection, non-visual baselines, and pipeline-integrity checks. Across nine direct-action models, six structured local VLMs, and an exploratory VLM-MPC hierarchy, we analyse 32,874 scored calls over two embodiments and three simulators. The direct-control results are largely negative: a constant-SLOW policy outperforms a scripted
geometric controller, several models are image-invariant or nearly constant, and models that recognize longitudinal hazards still fail to transform LEFT and RIGHT under reflection. No local VLM meets the joint longitudinal and lateral grounding criteria. However, an image-only deterministic positive control estimates the lead gap with 0.090 m MAE and exact mirror equivariance, confirming the stimuli carry sufficient visual information; the failures are modular, not universal. A post-hoc, leakage-controlled
symmetry-consensus guardian selects two models from 16 calibration frames and freezes a 2-of-4 hazard vote across original and reflected views. On 272 held-out frames it reaches 0.954 balanced accuracy (episode-cluster bootstrap 95% CI [0.895,0.990]); nested leave-one-episode-out recovers the same pair and threshold in all 12 folds. Abstaining on ties raises committed balanced accuracy to 0.973 at 0.824 coverage. With deterministic perception retaining lateral authority, offline modular replay achieves 0.934 action agreement and exact mirror equivariance. These results support current VLMs as bounded, selective hazard assistants, not monolithic zero-shot controllers.
Read the full abstract ▾
All authors J. de Curtò, Dayani Plasencia, Diego Sánchez, I. de Zarzà
Computer Vision · Cross-listed from cs.AI
CogVis: Must Open-Vocabulary Change Detection Perceive the Scene Anew for Every Query?
Zijie Wang, Chen Zhong and Wei He
Earth-surface monitoring requires change detection models capable of recognizing arbitrary semantic categories. Open-Vocabulary Change Detection (OVCD) addresses this need. However, existing methods often entangle temporal perception, semantic discrimination, and region verification, causing unstable results and redundant computation. Inspired by human visual change perception, we propose CogVis, a cognitive
memory-guided framework that reformulates OVCD as a perception-
memory-verification paradigm. CogVis first employs a Scene Change Perceptron (SCP) to extract a reusable, category-agnostic change prior from frozen bi-temporal features, thereby decoupling temporal evidence from semantic category decisions. A Semantic
Memory Calibrator (SMC) then compensates for category-dependent score shifts by dynamically estimating an image-query-specific decision threshold. Finally, an Adaptive Region Filter (ARF) filters connected candidates using learned semantic, temporal, and structural reliability. Experiments on seven
benchmarks spanning semantic change detection, binary change localization, and building-damage assessment show that CogVis achieves state-of-the-art performance across all evaluated
datasets. By sharing scene-level change perception, CogVis further avoids repeating category-agnostic temporal perception across queries and improves inference throughput by 28.50%.
Read the full abstract ▾
Computer Vision
Learning visual representations for compositional analysis of artworks and photographs
Fatemeh Behrad, Tinne Tuytelaars and Johan Wagemans
Composition, the deliberate arrangement of visual elements, is central to how meaning,
emotion, and aesthetic quality are conveyed in artwork, yet it remains among the least formalized dimensions of visual understanding. Prior work highlights a persistent gap in learning meaningful compositional representations, attributing it to semantic bias and suggesting that human-inspired approaches may be key. We compare two parallel paradigms for composition analysis: a human-inspired method grounded in perceptual grouping, and fine-tuned foundation models enabled by recent large-scale compositional
datasets. The human-inspired approach uses object-centric models for region-level decomposition and a graph
attention network to capture spatial relationships between elements. Both paradigms are evaluated on composition score/category prediction, compositional image
retrieval, and visual saliency detection. With frozen encoders, the human-inspired method achieves competitive performance while remaining interpretable. When sufficient data enables fine-tuning, large self-supervised models outperform significantly, but at the cost of interpretability and cross-domain generalization. Code and pre-trained models are available on GitHub.
Read the full abstract ▾
Computer Vision
Patient Pose Assessment Using a CT-Based Framework for Synthetic Data Generation
Manuel Laufer, Dominik Mairhöfer, Malte Sieren et al. (+7 more)
An adequate diagnostic quality of radiographs is essential for reliable diagnoses and treatment planning. The patient's pose during radiography is one of the most important factors determining the diagnostic quality. Since patient positioning is difficult and not standardized, an automated AI-based approach using depth images to automatically assess the patient's pose before the radiograph has been taken would be helpful. Due to regulatory hurdles, however, it is difficult in practice to acquire the required depth images and corresponding radiographs. In this paper, we present a framework that can generate such training data synthetically from Computed
Tomography scans. We further show that by pretraining on our generated synthetic
dataset consisting of 3077 image pairs of upper ankle joints, the pose assessment of real upper ankle joints can be improved by up to 11 percentage points.
Read the full abstract ▾
All authors Manuel Laufer, Dominik Mairhöfer, Malte Sieren, Hauke Gerdes, Fabio Leal dos Reis, Arpad Bischof, Thomas Käster, Erhardt Barth, Jörg Barkhausen, Thomas Martinetz