Computer Vision
CIPHER: Culvert Inspection through Pairwise Frame Selection and High-Efficiency Reconstruction
Seoyoung Lee and Zhangyang Wang
Automated culvert inspection systems can help increase the safety and efficiency of flood management operations. As a key step to this system, we present an efficient RGB-based 3D reconstruction pipeline for culvert-like structures in visually repetitive environments. Our approach first selects informative frame pairs to maximize viewpoint diversity while ensuring valid correspondence matching using a plug-and-play module, followed by a reconstruction model that simultaneously estimates RGB appearance, geometry, and semantics in real-time. Experiments demonstrate that our method effectively generates accurate 3D reconstructions and depth maps, enhancing culvert inspection efficiency with minimal human intervention.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos
Arushi Goel, Sreyan Ghosh, Vatsal Agarwal et al. (+16 more)
Multimodal Large
Language Models (MLLMs) have shown strong performance in visual and audio understanding when evaluated in isolation. However, their ability to jointly reason over omni-modal (visual, audio, and textual) signals in long and complex
videos remains largely unexplored. We introduce MMOU, a new
benchmark designed to systematically evaluate
multimodal understanding and
reasoning under these challenging, real-world conditions. MMOU consists of 20,000 carefully curated questions paired with 11877 web-collected
videos of varying length, spanning diverse domains and exhibiting rich, tightly coupled audio-visual content. The
benchmark covers 13 fundamental skill categories, all of which require integrating evidence across modalities and time. All questions are manually annotated across multiple turns by professional annotators, ensuring high quality and
reasoning fidelity. We evaluate 20+ state-of-the-art open-source and proprietary
multimodal models on MMOU. The results expose substantial performance gaps: the best closed-source model achieves only 64.2% accuracy, while the strongest open-source model reaches just 46.8%. Our results highlight the challenges of long-form omni-modal understanding, revealing that current models frequently fail to apply even fundamental skills in long
videos. Through detailed analysis, we further identify systematic failure modes and provide insights into where and why current models break.
Read the full abstract ▾
All authors Arushi Goel, Sreyan Ghosh, Vatsal Agarwal, Nishit Anand, Kaousheik Jayakumar, Lasha Koroshinadze, Yao Xu, Katie Lyons, James Case, Karan Sapra, Kevin J. Shih, Siddharth Gururani, Abhinav Shrivastava, Ramani Duraiswami, Dinesh Manocha, Andrew Tao, Bryan Catanzaro, Mohammad Shoeybi, Wei Ping
Computer Vision
DualSwinFusionSeg: Multimodal Martian Landslide Segmentation via Dual Swin Transformer with Multi-Scale Fusion and UNet++
Shahriar Kabir, Abdullah Muhammed Amimul Ehsan, Istiak Ahmmed Rifti et al. (+1 more)
Automated
segmentation of Martian landslides, particularly in tectonically active regions such as Valles Marineris,is important for planetary geology, hazard assessment, and future robotic exploration. However, detecting landslides from planetary imagery is challenging due to the heterogeneous nature of available sensing modalities and the limited number of labeled samples. Each observation combines RGB imagery with geophysical measurements such as digital elevation models, slope maps, thermal inertia, and contextual grayscale imagery, which differ significantly in resolution and statistical properties. To address these challenges, we propose DualSwinFusionSeg, a
multimodal segmentation architecture that separates modality-specific feature extraction and performs multi-scale cross-modal fusion. The model employs two parallel Swin
Transformer V2 encoders to independently process RGB and auxiliary geophysical inputs, producing hierarchical feature representations. Corresponding features from the two streams are fused at multiple scales and decoded using a UNet++ decoder with dense nested skip connections to preserve fine boundary details. Extensive ablation studies evaluate modality contributions, loss functions, decoder architectures, and fusion strategies. Experiments on the MMLSv2
dataset from the PBVS 2026 Mars-LS Challenge show that modality-specific encoders and simple concatenation-based fusion improve
segmentation accuracy under limited training data. The final model achieves 0.867 mIoU and 0.905 F1 on the development
benchmark and 0.783 mIoU on the held-out test set, demonstrating strong performance for
multimodal planetary surface
segmentation.
Read the full abstract ▾
All authors Shahriar Kabir, Abdullah Muhammed Amimul Ehsan, Istiak Ahmmed Rifti, Md Kaykobad Reza
Computer Vision
Diffusion Reinforcement Learning via Centered Reward Distillation
Yuanzhi Zhu, Xi Wang, Stéphane Lathuilière et al. (+1 more)
Diffusion and flow models achieve State-Of-The-Art (SOTA) generative performance, yet many practically important behaviors such as fine-grained
prompt fidelity, compositional correctness, and text rendering are weakly specified by score or flow matching pretraining objectives. Reinforcement Learning (RL) fine-tuning with external, black-box rewards is a natural remedy, but
diffusion RL is often brittle. Trajectory-based methods incur high
memory cost and high-variance gradient estimates; forward-process approaches converge faster but can suffer from distribution drift, and hence reward hacking. In this work, we present Centered Reward
Distillation (CRD), a
diffusion RL framework derived from KL-regularized reward maximization built on forward-process-based fine-tuning. The key insight is that the intractable normalizing constant cancels under within-
prompt centering, yielding a well-posed reward-matching objective. To enable reliable text-to-image fine-tuning, we introduce techniques that explicitly control distribution drift: (i) decoupling the sampler from the moving reference to prevent ratio-signal collapse, (ii) KL anchoring to a CFG-guided pretrained model to control long-run drift and align with the inference-time semantics of the pre-trained model, and (iii) reward-adaptive KL strength to accelerate early learning under large KL regularization while reducing late-stage exploitation of reward-model loopholes. Experiments on text-to-image post-training with GenEval and OCR rewards show that CRD achieves competitive SOTA reward optimization results with fast convergence and reduced reward hacking, as validated on unseen preference metrics.
Read the full abstract ▾
All authors Yuanzhi Zhu, Xi Wang, Stéphane Lathuilière, Vicky Kalogeiton
Computer Vision
Implementation and discussion of the Pith Estimation on Rough Log End Images using Local Fourier Spectrum Analysis method
Henry Marichal, Diego Passarella and Gregory Randall
In this article, we analyze and propose a Python implementation of the method "Pith Estimation on Rough Log End images using Local
Fourier Spectrum Analysis", by Rudolf Schraml and Andreas Uhl. The
algorithm is tested over two
datasets.
Computer Vision
Low-Field Magnetic Resonance Image Enhancement using Undersampled k-Space
Daniel Tweneboah Anyimadu, Mohammed Abdalla, Mohammed M. Abdelsamea et al. (+1 more)
Low-field
magnetic resonance imaging (MRI) offers a cost-effective alternative for medical imaging in resource-limited settings. However, its widespread adoption is hindered by two key challenges: prolonged scan times and reduced image quality. Accelerated acquisition can be achieved using k-space undersampling, while image enhancement traditionally relies on spatial-domain postprocessing. In this work, we propose a novel deep learning framework based on a U-Net variant that operates directly in k-space to super-resolve low-field MR images directly using undersampled data while quantifying the impact of reduced k-space sampling. Unlike conventional approaches that treat image super-resolution as a postprocessing step following image reconstruction from undersampled k-space, our unified model integrates both processes, leveraging k-space information to achieve superior image fidelity. Extensive experiments on synthetic and real low-field
brain MRI
datasets demonstrate that k-space-driven image super-resolution outperforms conventional spatial-domain counterparts. Furthermore, our results show that undersampled k-space reconstructions achieve comparable quality to full k-space acquisitions, enabling substantial scan-time acceleration without compromising diagnostic utility.
Read the full abstract ▾
All authors Daniel Tweneboah Anyimadu, Mohammed Abdalla, Mohammed M. Abdelsamea, Ahmed Karam Eldaly
Computer Vision
Low-Field Magnetic Resonance Image Quality Enhancement using Undersampled k-Space and Out-of-Distribution Generalisation
Daniel Tweneboah Anyimadu, Mohammed M. Abdelsamea and Ahmed Karam Eldaly
Low-field
magnetic resonance imaging (MRI) offers affordable access to diagnostic imaging but faces challenges such as prolonged acquisition times and reduced image quality. Although accelerated imaging via k-space undersampling helps reduce scan time, image quality enhancement methods often rely on spatial-domain postprocessing. Deep learning achieved state-of-the-art results in both domains. However, most models are trained and evaluated using in-distribution (InD) data, creating a significant gap in understanding model performance when tested using out-of-distribution (OOD) data. To address these issues, we propose a novel framework that reconstructs high-field-like MR images directly from undersampled low-field MRI k-space, quantifies the impact of reduced sampling, and evaluates the generalisability of the model using OOD. Our approach utilises a k-space dual channel U-Net to jointly process the real and imaginary components of undersampled k-space, restoring missing frequency content, and incorporates an ensemble strategy to generate uncertainty maps. Experiments on low-field
brain MRI demonstrate that our k-space-driven image quality enhancement outperforms the counterpart spatial-domain and other state-of-the-art baselines, achieving image quality comparable to full high-field k-space acquisitions using OOD data. To the best of our knowledge, this work is among the first to combine low-field MR image reconstruction, quality enhancement using undersampled k-space, and uncertainty quantification within a unified framework.
Read the full abstract ▾
Computer Vision
Improving Visual Reasoning with Iterative Evidence Refinement
Zeru Shi, Kai Mei, Yihao Quan et al. (+2 more)
Vision language models (VLMs) are increasingly capable of
reasoning over images, but robust visual
reasoning often requires re-grounding intermediate steps in the underlying visual evidence. Recent approaches typically rely on external image operations such as zooming or cropping to re-access fine-grained details during inference, which requires additional image re-encoding and can disrupt the
reasoning trajectory. We argue that VLMs already provide strong internal signals for identifying and reusing visual evidence, and that these signals can be directly leveraged to support image-grounded
reasoning. Motivated by this insight, we propose an end-to-end self-revisit framework, SIEVE, that trains models to re-engage image evidence through internal representations. SIEVE automatically extracts
embeddings of salient image regions and injects them into the
reasoning chain when additional grounding is needed, enabling later steps to condition on relevant visual cues without external tool calls or re-encoding. We use reinforcement learning to teach the model when to trigger visual revisiting and which region
embeddings to retrieve and insert during the
reasoning process. Experiments on multiple visual
reasoning benchmarks, together with perception,
reasoning, and
hallucination evaluations, show that SIEVE yields consistent gains, improving performance by 8 percent on average across several
benchmarks.
Read the full abstract ▾
All authors Zeru Shi, Kai Mei, Yihao Quan, Dimitris N.Metaxas, Ruixiang Tang
Computer Vision
Revisiting the Perception-Distortion Trade-off with Spatial-Semantic Guided Super-Resolution
Dan Wang, Haiyan Sun, Shan Du et al. (+4 more)
Image super-resolution (SR) aims to reconstruct high resolution images with both high perceptual quality and low distortion, but is fundamentally limited by the perception-distortion trade-off. GAN-based SR methods reduce distortion but still struggle with realistic fine-grained textures, whereas
diffusion-based approaches synthesize rich details but often deviate from the input, hallucinating structures and degrading fidelity. This tension raises a key challenge: how to exploit the powerful generative priors of
diffusion models without sacrificing fidelity. To address this, we propose SpaSemSR, a spatial-semantic guided
diffusion framework with two complementary guidances. First, spatial-grounded textual guidance integrates object-level spatial cues with semantic
prompts, aligning textual and visual structures to reduce distortion. Second, semantic-enhanced visual guidance with a multi-encoder design and semantic degradation constraints unifies
multimodal semantic priors, improving perceptual realism under severe degradations. These complementary guidances are adaptively fused into the
diffusion process via spatial-semantic
attention, suppressing distortion and
hallucination while retaining the strengths of
diffusion models. Extensive experiments on multiple
benchmarks show that SpaSemSR achieves a superior perception-distortion balance, producing both realistic and faithful restorations.
Read the full abstract ▾
All authors Dan Wang, Haiyan Sun, Shan Du, Z. Jane Wang, Zhaochong An, Serge Belongie, Xinrui Cui
Computer Vision
Effective Feature Learning for 3D Medical Registration via Domain-Specialized DINO Pretraining
Eytan Kats and Mattias P. Heinrich
Medical image registration is a critical component of clinical imaging workflows, enabling accurate longitudinal assessment, multi-modal data fusion, and image-guided interventions. Intensity-based approaches often struggle with interscanner variability and complex anatomical deformations, whereas feature-based methods offer improved robustness by leveraging semantically informed representations. In this work, we investigate DINO-style self-supervised pretraining directly on 3D medical imaging data, aiming to learn dense volumetric features well suited for deformable registration. We assess the resulting representations on challenging interpatient abdominal registration task across both MRI and CT modalities. Our domain-specialized pretraining outperforms the DINOv2 model trained on a large-scale collection of natural images, while requiring substantially lower computational resources at inference time. Moreover, it surpasses established registration models under out-of-domain evaluation, demonstrating the value of task-agnostic yet medical imaging-focused pretraining for robust and efficient 3D image registration.
Read the full abstract ▾
Computer Vision
Enhancing Eye Feature Estimation from Event Data Streams through Adaptive Inference State Space Modeling
Viet Dung Nguyen, Mobina Ghorbaninejad, Chengyi Ma et al. (+5 more)
Eye feature extraction from event-based data streams can be performed efficiently and with low energy consumption, offering great utility to real-world eye tracking pipelines. However, few eye feature extractors are designed to handle sudden changes in event density caused by the changes between gaze behaviors that vary in their kinematics, leading to degraded prediction performance. In this work, we address this problem by introducing the adaptive inference state space model (AISSM), a novel architecture for feature extraction that is capable of dynamically adjusting the relative weight placed on current versus recent information. This relative weighting is determined via estimates of the signal-to-noise ratio and event density produced by a complementary dynamic confidence network. Lastly, we craft and evaluate a novel learning technique that improves training efficiency. Experimental results demonstrate that the AISSM system outperforms state-of-the-art models for event-based eye feature extraction.
Read the full abstract ▾
All authors Viet Dung Nguyen, Mobina Ghorbaninejad, Chengyi Ma, Reynold Bailey, Gabriel J. Diaz, Alexander Fix, Ryan J. Suess, Alexander Ororbia
Computer Vision
SGR-OCC: Evolving Monocular Priors for Embodied 3D Occupancy Prediction via Soft-Gating Lifting and Semantic-Adaptive Geometric Refinement
Yiran Guo, Simone Mentasti, Xiaofeng Jin et al. (+2 more)
3D semantic occupancy prediction is a cornerstone for
embodied AI, enabling
agents to perceive dense scene geometry and semantics incrementally from
monocular video streams. However, current online frameworks face two critical bottlenecks: the inherent depth ambiguity of
monocular estimation that causes "feature bleeding" at object boundaries , and the "cold start" instability where uninitialized temporal fusion layers distort high-quality spatial priors during early training stages. In this paper, we propose SGR-OCC (Soft-Gating and Ray-refinement Occupancy), a unified framework driven by the philosophy of "Inheritance and
Evolution". To perfectly inherit
monocular spatial expertise, we introduce a Soft-Gating Feature Lifter that explicitly models depth uncertainty via a Gaussian gate to probabilistically suppress background noise. Furthermore, a Dynamic Ray-Constrained Anchor Refinement module simplifies complex 3D displacement searches into efficient 1D depth corrections along camera rays, ensuring sub-voxel adherence to physical surfaces. To ensure stable
evolution toward temporal consistency, we employ a Two-Phase Progressive Training Strategy equipped with identity-initialized fusion, effectively resolving the cold start problem and shielding spatial priors from noisy early gradients. Extensive experiments on the EmbodiedOcc-ScanNet and Occ-ScanNet
benchmarks demonstrate that SGR-OCC achieves state-of-the-art performance. In local prediction tasks, SGR-OCC achieves a completion IoU of 58.55$\%$ and a semantic mIoU of 49.89$\%$, surpassing the previous best method, EmbodiedOcc++, by 3.65$\%$ and 3.69$\%$ respectively. In challenging
embodied prediction tasks, our model reaches 55.72$\%$ SC-IoU and 46.22$\%$ mIoU. Qualitative results further confirm our model's superior capability in preserving structural integrity and boundary sharpness in complex indoor environments.
Read the full abstract ▾
All authors Yiran Guo, Simone Mentasti, Xiaofeng Jin, Matteo Frosi, Matteo Matteucci
Computer Vision
Self-Supervised Uncertainty Estimation For Super-Resolution of Satellite Images
Zhe Zheng, Valéry Dewil and Pablo Arias
Super-resolution (SR) of satellite imagery is challenging due to the lack of paired low-/high-resolution data. Recent self-supervised SR methods overcome this limitation by exploiting the temporal redundancy in burst observations, but they lack a mechanism to quantify uncertainty in the reconstruction. In this work, we introduce a novel self-supervised loss that allows to estimate uncertainty in image super-resolution without ever accessing the ground-truth high-resolution data. We adopt a decision-theoretic perspective and show that minimizing the corresponding
Bayesian risk yields the posterior mean and variance as optimal
estimators. We validate our approach on a synthetic SkySat L1B
dataset and demonstrate that it produces calibrated uncertainty estimates comparable to supervised methods. Our work bridges self-supervised restoration with uncertainty quantification, making a practical framework for uncertainty-aware image reconstruction.
Read the full abstract ▾
Computer Vision
MotionCFG: Boosting Motion Dynamics via Semantic Motion Sharpening
Byungjun Kim, Soobin Um and Jong Chul Ye
Despite recent advancements in Text-to-
Video (T2V) synthesis, generating high-fidelity and dynamic motion remains a significant challenge. Existing methods primarily rely on Classifier-Free Guidance (CFG), but standard CFG treats all semantic dimensions uniformly and provides no mechanism to selectively resolve ambiguity along the motion axis. To address this, we propose MotionCFG, a training-free approach that introduces selective guidance along the motion subspace of the condition
embedding. Specifically, we construct a motion-perturbed negative condition by injecting Gaussian noise into motion-related
embeddings, and steer generation away from it to selectively amplify motion intent. We show theoretically that this procedure implicitly approximates Semantic Laplacian Sharpening, a second-order correction that suppresses motion-ambiguous regions of the score landscape and amplifies well-resolved dynamic peaks. Combined with a piecewise guidance schedule that confines intervention to the early denoising steps, MotionCFG consistently improves motion dynamics across state-of-the-art T2V frameworks with negligible overhead. We further demonstrate that this Laplacian sharpening principle generalizes beyond motion, effectively steering complex, non-linear concepts such as precise object numerosity that are typically difficult to modulate via standard text-based guidance. Our project page is available at https://motion-cfg.vercel.app/.
Read the full abstract ▾
Computer Vision
TMPDiff: Temporal Mixed-Precision for Diffusion Models
Basile Lewandowski, Simon Kurz, Aditya Shankar et al. (+3 more)
Diffusion models are the go-to method for Text-to-Image generation, but their iterative denoising processes has high inference latency. Quantization reduces compute time by using lower bitwidths, but applies a fixed precision across all denoising timesteps, leaving an entire optimization axis unexplored. We propose TMPDiff, a temporal mixed-precision framework for
diffusion models that assigns different numeric precision to different denoising timesteps. We hypothesize that quantization errors accumulate additively across timesteps, which we then validate experimentally. Based on our observations, we develop an adaptive bisectioning-based
algorithm, which assigns per-step precisions with linear evaluation complexity, reducing an otherwise exponential search problem. Across four state-of-the-art
diffusion models and three
datasets, TMPDiff consistently outperforms uniform-precision baselines at matched speedup, achieving 10 to 20% improvement in perceptual quality. On FLUX.1-dev, TMPDiff achieves 90% SSIM relative to the full-precision model at a speedup of 2.5x over 16-bit inference.
Read the full abstract ▾
All authors Basile Lewandowski, Simon Kurz, Aditya Shankar, Robert Birke, Jian-Jia Chen, Lydia Y. Chen
Computer Vision
A Multi-Agent Perception-Action Alliance for Efficient Long Video Reasoning
Yichang Xu, Gaowen Liu, Ramana Rao Kompella et al. (+6 more)
This paper presents a multi-
agent perception-action exploration alliance, dubbed A4VL, for efficient long-
video reasoning. A4VL operates in a multi-round perception-action exploration loop with a selection of VLM
agents. In each round, the team of
agents performs
video question-answer (VideoQA) via perception exploration followed by action exploration. During perception exploration, each
agent learns to extract query-specific perception clue(s) from a few sampled frames and performs clue-based
alignment to find the
video block(s) that are most relevant to the query-specific event. During action exploration, A4VL performs
video reasoning in three steps: (1) each
agent produces its initial answer with rational, (2) all
agents collaboratively scores one another through cross-reviews and relevance ranking, and (3) based on whether a satisfactory consensus is reached, the decision is made either to start a new round of perception-action deliberation by
pruning (e.g., filtering out the lowest performing
agent) and re-staging (e.g., new-clue and matching block based perception-action exploration), or to conclude by producing its final answer. The integration of the multi-
agent alliance through multi-round perception-action exploration, coupled with event-driven partitioning and cue-guided block
alignment, enables A4VL to effectively scale to real world long
videos while preserving high quality
video reasoning. Evaluation Results on five popular VideoQA
benchmarks show that A4VL outperforms 18 existing representative VLMs and 11 recent methods optimized for long-
video reasoning, while achieving significantly lower inference latency. Our code is released at https://github.com/git-disl/A4VL.
Read the full abstract ▾
All authors Yichang Xu, Gaowen Liu, Ramana Rao Kompella, Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, Zachary Yahn, Ling Liu
Computer Vision
EyeWorld: A Generative World Model of Ocular State and Dynamics
Ziyu Gao, Xinyuan Wu, Xiaolan Chen et al. (+10 more)
Ophthalmic decision-making depends on subtle lesion-scale cues interpreted across
multimodal imaging and over time, yet most medical foundation models remain static and degrade under modality and acquisition shifts. Here we introduce EyeWorld, a generative world model that conceptualizes the eye as a partially observed dynamical system grounded in clinical imaging. EyeWorld learns an observation-stable latent ocular state shared across modalities, unifying fine-grained parsing, structure-preserving cross-modality translation and quality-robust enhancement within a single framework. Longitudinal supervision further enables time-conditioned state transitions, supporting
forecasting of clinically meaningful progression while preserving stable anatomy. By moving from static representation learning to explicit dynamical modeling, EyeWorld provides a unified approach to robust
multimodal interpretation and prognosis-oriented simulation in medicine.
Read the full abstract ▾
All authors Ziyu Gao, Xinyuan Wu, Xiaolan Chen, Zhuoran Liu, Ruoyu Chen, Bowen Liu, Bingjie Yan, Zhenhan Wang, Kai Jin, Jiancheng Yang, Yih Chung Tham, Mingguang He, Danli Shi
Computer Vision
Intrinsic Tolerance in C-Arm Imaging: How Extrinsic Re-optimization Preserves 3D Reconstruction Accuracy
Lin Li, Benjamin Aubert, Paul Kemper et al. (+1 more)
Purpose: C-arm fluoroscopy's 3D reconstruction relies on accurate intrinsic calibration, which is often challenging in clinical practice. This study ensures high-precision reconstruction accuracy by re-optimizing the extrinsic parameters to compensate for intrinsic calibration errors. \noindentMethods: We conducted both simulation and real-world experiments using five commercial C-arm systems. Intrinsic parameters were perturbed in controlled increments. Focal length was increased by 100 to 700 pixels ($\approx$20 mm to 140 mm) and principal point by 20 to 200 pixels. For each perturbation, we (1) reconstructed 3D points from known phantom geometries, (2) re-estimated extrinsic poses using standard optimization, and (3) measured reconstruction and reprojection errors relative to ground truth. \noindentResults: Even with focal length errors up to 500 pixels ($\approx$100 mm, assuming a nominal focal length of $\sim$1000 mm), mean 3D reconstruction error remained under 0.2 mm. Larger focal length deviations (700 pixels) elevated error to only $\approx$0.3 mm. Principal point shifts up to 200 pixels introduced negligible reconstruction error once extrinsic parameters were re-optimized, with reprojection error increases below 0.5 pixels. \noindentConclusion: Moderate errors in intrinsic calibration can be effectively mitigated by extrinsic re-optimization, preserving submillimeter 3D reconstruction accuracy. This intrinsic tolerance suggests a practical pathway to relax calibration precision requirements, thereby simplifying C-arm system setup and reducing clinical workflow burden without compromising performance.
Read the full abstract ▾
All authors Lin Li, Benjamin Aubert, Paul Kemper, Aric Plumley
Computer Vision
High-speed Imaging through Turbulence with Event-based Light Fields
Yu-Hsiang Huang, Levi Burner, Sachin Shah et al. (+3 more)
This work introduces and demonstrates the first system capable of imaging fast-moving extended non-rigid objects through strong atmospheric
turbulence at high frame rate. Event cameras are a novel sensing architecture capable of estimating high-speed imagery at thousands of frames per second. However, on their own event cameras are unable to disambiguate scene motion from
turbulence. In this work, we overcome this limitation using event-based light field cameras: By simultaneously capturing multiple views of a scene, event-based light field cameras and machine learning-based reconstruction
algorithms are able to disambiguate motion-induced dynamics, which produce events that are strongly correlated across views, from
turbulence-induced dynamics, which produce events that are weakly correlated across view. Tabletop experiments demonstrate event-based light field can overcome strong
turbulence while imaging high-speed objects traveling at up to 16,000 pixels per second.
Read the full abstract ▾
All authors Yu-Hsiang Huang, Levi Burner, Sachin Shah, Ziyuan Qu, Adithya Pediredla, Christopher A. Metzler
Computer Vision
A Hyperbolic Perspective on Hierarchical Structure in Object-Centric Scene Representations
Neelu Madan, Àlex Pujol, Andreas Møgelmose et al. (+4 more)
Slot
attention has emerged as a powerful framework for unsupervised object-centric learning, decomposing visual scenes into a small set of compact vector representations called slots, each capturing a distinct region or object. However, these slots are learned in Euclidean space, which provides no
geometric inductive bias for the hierarchical relationships that naturally structure visual scenes. In this work, we propose a simple post-hoc pipeline to project Euclidean slot
embeddings onto the Lorentz hyperboloid of
hyperbolic space, without modifying the underlying training pipeline. We construct five-level visual hierarchies directly from slot
attention masks and analyse whether
hyperbolic geometry reveals latent hierarchical structure that remains invisible in Euclidean space. Integrating our pipeline with SPOT (images), VideoSAUR (
video), and SlotContrast (
video), We find that
hyperbolic projection exposes a consistent scene-level to object-level organisation, where coarse slots occupy greater
manifold depth than fine slots, which is absent in Euclidean space. We further identify a "
curvature–task tradeoff": low
curvature ($c{=}0.2$) matches or outperforms Euclidean on parent slot
retrieval, while moderate
curvature ($c{=}0.5$) achieves better inter-level separation. Together, these findings suggest that slot representations already encode latent hierarchy that
hyperbolic geometry reveals, motivating end-to-end
hyperbolic training as a natural next step. Code and models are available at \href{https://github.com/NeeluMadan/HHS}{github.com/NeeluMadan/HHS}.
Read the full abstract ▾
All authors Neelu Madan, Àlex Pujol, Andreas Møgelmose, Sergio Escalera, Kamal Nasrollahi, Graham W. Taylor, Thomas B. Moeslund
Computer Vision
EI-Part: Explode for Completion and Implode for Refinement
Wanhu Sun, Zhongjin Luo, Heliang Zheng et al. (+6 more)
Part-level 3D generation is crucial for various downstream applications, including gaming, film production, and industrial design. However, decomposing a 3D shape into geometrically plausible and meaningful components remains a significant challenge. Previous part-based generation methods often struggle to produce well-constructed parts, exhibiting poor structural coherence,
geometric implausibility, inaccuracy, or inefficiency. To address these challenges, we introduce EI-Part, a novel framework specifically designed to generate high-quality 3D shapes with components, characterized by strong structural coherence,
geometric plausibility,
geometric fidelity, and generation efficiency. We propose utilizing distinct representations at different stages: an Explode state for part completion and an Implode state for geometry refinement. This strategy fully leverages spatial resolution, enabling flexible part completion and fine
geometric detail generation. To maintain structural coherence between parts, a self-
attention mechanism is incorporated in both exploded and imploded states, facilitating effective information perception and feature fusion among components during generation. Extensive experiments on multiple
benchmarks demonstrate that EI-Part efficiently produces semantically meaningful and structurally coherent parts with fine-grained
geometric details, achieving state-of-the-art performance in part-level 3D generation. Project page: https://cvhadessun.github.io/EI-Part/
Read the full abstract ▾
All authors Wanhu Sun, Zhongjin Luo, Heliang Zheng, Jiahao Chang, Chongjie Ye, Huiang He, Shengchu Zhao, Rongfei Jia, Xiaoguang Han
Computer Vision
Multi-Grained Vision-Language Alignment for Domain Generalized Person Re-Identification
Jiachen Li, Xiaojin Gong and Dongping Zhang
Domain Generalized person Re-identification (DG Re-ID) is a challenging task, where models are trained on source domains but tested on unseen target domains. Although previous pure
vision-based models have achieved significant progress, the performance remains further improved. Recently,
Vision-
Language Models (VLMs) present outstanding generalization capabilities in various visual applications. However, directly adapting a VLM to Re-ID shows limited generalization improvement. This is because the VLM only produces with global features that are insensitive to ID nuances. To tacle this problem, we propose a CLIP-based multi-grained
vision-
language alignment framework in this work. Specifically, several multi-grained
prompts are introduced in
language modality to describe different body parts and align with their counterparts in
vision modality. To obtain fine-grained visual information, an adaptively masked multi-head self-
attention module is employed to precisely extract specific part features. To train the proposed module, an MLLM-based visual grounding expert is employed to automatically generate pseudo labels of body parts for supervision. Extensive experiments conducted on both single- and multi-source generalization protocols demonstrate the superior performance of our approach. The implementation code will be released at https://github.com/RikoLi/MUVA.
Read the full abstract ▾
Computer Vision
Towards Generalizable Deepfake Detection via Real Distribution Bias Correction
Ming-Hui Liu, Harry Cheng, Xin Luo et al. (+2 more)
To generalize deepfake detectors to future unseen forgeries, most existing methods attempt to simulate the dynamically evolving forgery types using available source domain data. However, predicting an unbounded set of future manipulations from limited prior examples is infeasible. To overcome this limitation, we propose to exploit the invariance of real data from two complementary perspectives: the fixed population distribution of the entire real class and the inherent Gaussianity of individual real images. Building on these properties, we introduce the Real Distribution Bias Correction (RDBC) framework, which consists of two key components: the Real Population Distribution Estimation module and the Distribution-Sampled Feature Whitening module. The former utilizes the independent and identically distributed (\iid) property of real samples to derive the normal distribution form of their statistics, from which the distribution parameters can be estimated using limited source domain data. Based on the learned population distribution, the latter utilizes the inherent Gaussianity of real data as a discriminative prior and performs a sampling-based whitening operation to amplify the Gaussianity gap between real and fake samples. Through synergistic coupling of the two modules, our model captures the real-world properties of real samples, thereby enhancing its generalizability to unseen target domains. Extensive experiments demonstrate that RDBC achieves state-of-the-art performance in both in-domain and cross-domain deepfake detection.
Read the full abstract ▾
All authors Ming-Hui Liu, Harry Cheng, Xin Luo, Xin-Shun Xu, Mohan S. Kankanhalli
Computer Vision
U-Face: An Efficient and Generalizable Framework for Unsupervised Facial Attribute Editing via Subspace Learning
Bo Liu, Xuan Cui, Run Zeng et al. (+5 more)
Latent space-based facial attribute editing methods have gained popularity in applications such as digital entertainment, virtual avatar creation, and human-computer interaction systems due to their potential for efficient and flexible attribute manipulation, particularly for continuous edits. Among these, unsupervised latent space-based methods, which discover effective semantic vectors without relying on labeled data, have attracted considerable
attention in the research community. However, existing methods still encounter difficulties in disentanglement, as manipulating a specific facial attribute may unintentionally affect other attributes, complicating fine-grained controllability. To address these challenges, we propose a novel framework designed to offer an effective and adaptable solution for unsupervised facial attribute editing, called Unsupervised Facial Attribute Controllable Editing (U-Face). The proposed method frames semantic vector learning as a subspace learning problem, where latent vectors are approximated within a lower-dimensional semantic subspace spanned by a semantic vector
matrix. This formulation can also be equivalently interpreted from a projection-reconstruction perspective and further generalized into an autoencoder framework, providing a foundation that can support disentangled representation learning in a flexible manner. To improve disentanglement and controllability, we impose orthogonal non-negative constraints on the semantic vectors and incorporate attribute boundary vectors to reduce
entanglement in the learned directions. Although these constraints make the optimization problem challenging, we design an alternating iterative
algorithm, called Alternating Iterative Disentanglement and Controllability (AIDC), with closed-form updates and provable convergence under specific conditions.
Read the full abstract ▾
All authors Bo Liu, Xuan Cui, Run Zeng, Wei Duan, Chongwen Liu, Jinrui Qian, Lianggui Tang, Hongping Gan
Computer Vision
PhyGaP: Physically-Grounded Gaussians with Polarization Cues
Jiale Wu, Xiaoyang Bai, Zongqi He et al. (+2 more)
Recent advances in 3D Gaussian Splatting (3DGS) have demonstrated great success in modeling reflective 3D objects and their interaction with the environment via deferred rendering (DR). However, existing methods often struggle with correctly reconstructing physical attributes such as albedo and reflectance, and therefore they do not support high-fidelity relighting. Observing that this limitation stems from the lack of shape and material information in RGB images, we present PhyGaP, a physically-grounded 3DGS method that leverages polarization cues to facilitate precise reflection decomposition and visually consistent relighting of reconstructed objects. Specifically, we design a polarimetric deferred rendering (PolarDR) process to model polarization by reflection, and a self-occlusion-aware environment map building technique (GridMap) to resolve indirect lighting of non-convex objects. We validate on multiple synthetic and real-world scenes, including those featuring only partial polarization cues, that PhyGaP not only excels in reconstructing the appearance and surface normal of reflective 3D objects (~2 dB in PSNR and 45.7% in Cosine Distance better than existing RGB-based methods on average), but also achieves state-of-the-art inverse rendering and relighting capability. Our code will be released soon.
Read the full abstract ▾
All authors Jiale Wu, Xiaoyang Bai, Zongqi He, Weiwei Xu, Yifan Peng
Computer Vision
Human-like Object Grouping in Self-supervised Vision Transformers
Hossein Adeli, Seoyoung Ahn, Andrew Luo et al. (+3 more)
Vision foundation models trained with self-supervised objectives achieve strong performance across diverse tasks and exhibit emergent object
segmentation properties. However, their
alignment with human object perception remains poorly understood. Here, we introduce a behavioral
benchmark in which participants make same/different object judgments for dot pairs on naturalistic scenes, scaling up a classical psychophysics paradigm to over 1000 trials. We test a diverse set of
vision models using a simple readout from their representations to predict subjects' reaction times. We observe a steady improvement across model generations, with both architecture and training objective contributing to
alignment, and
transformer-based models trained with the DINO self-supervised objective showing the strongest performance. To investigate the source of this improvement, we propose a novel metric to quantify the object-centric component of representations by measuring patch similarity within and between objects. Across models, stronger object-centric structure predicts human
segmentation behavior more accurately. We further show that matching the Gram
matrix of supervised
transformer models, capturing similarity structure across image patches, with that of a self-supervised model through
distillation improves their
alignment with human behavior, converging with the prior finding that Gram anchoring improves DINOv3's feature quality. Together, these results demonstrate that self-supervised
vision models capture object structure in a behaviorally human-like manner, and that Gram
matrix structure plays a role in driving perceptual
alignment.
Read the full abstract ▾
All authors Hossein Adeli, Seoyoung Ahn, Andrew Luo, Mengmi Zhang, Nikolaus Kriegeskorte, Gregory Zelinsky
Computer Vision
VAD4Space: Visual Anomaly Detection for Planetary Surface Imagery
Fabrizio Genilotti, Arianna Stropeni, Francesco Borsatti et al. (+3 more)
Space missions generate massive volumes of high-resolution orbital and surface imagery that far exceed the capacity for manual inspection. Detecting rare phenomena is scientifically critical, yet traditional supervised learning struggles due to scarce labeled examples and closed-world assumptions that prevent discovery of genuinely novel observations. In this work, we investigate Visual Anomaly Detection (VAD) as a framework for automated discovery in planetary exploration. We present the first empirical evaluation of state-of-the-art feature-based VAD methods on real planetary imagery, encompassing both orbital lunar data and Mars rover surface imagery. To support this evaluation, we introduce two
benchmarks: (i) a lunar
dataset derived from Lunar Reconnaissance Orbiter Camera Narrow Angle imagery, comprising of fresh and degraded craters as anomalies alongside normal terrain; and (ii) a Mars surface
dataset designed to reflect the characteristics of rover-acquired imagery. We evaluate multiple VAD approaches with a focus on computationally efficient, edge-oriented solutions suitable for onboard deployment, applicable to both orbital platforms surveying the lunar surface and surface rovers operating on Mars. Our results demonstrate that feature-based VAD methods can effectively identify rare planetary surface phenomena while remaining feasible for resource-constrained environments. By grounding anomaly detection in planetary science, this work establishes practical
benchmarks and highlights the potential of open-world perception systems to support a range of mission-critical applications, including tactical planning, landing site selection, hazard detection, bandwidth-aware data prioritization, and the discovery of unanticipated geological processes.
Read the full abstract ▾
All authors Fabrizio Genilotti, Arianna Stropeni, Francesco Borsatti, Manuel Barusco, Davide Dalle Pezze, Gian Antonio Susto
Computer Vision
Xiaofei Hui, Qian Wu, Haoxuan Qu et al. (+3 more)
The emergence of
Multimodal Large
Language Models (MLLMs) and the widespread usage of MLLM cloud services such as GPT-4V raised great concerns about
privacy leakage in visual data. As these models are typically deployed in cloud services, users are required to submit their images and
videos, posing serious
privacy risks. However, how to tackle such
privacy concerns is an under-explored problem. Thus, in this paper, we aim to conduct a new investigation to protect visual
privacy when enjoying the convenience brought by MLLM services. We address the practical case where the MLLM is a "black box", i.e., we only have access to its input and output without knowing its internal model information. To tackle such a challenging yet demanding problem, we propose a novel framework, in which we carefully design the learning objective with Pareto optimality to seek a better trade-off between visual
privacy and MLLM's performance, and propose critical-history enhanced optimization to effectively optimize the framework with the black-box MLLM. Our experiments show that our method is effective on different
benchmarks.
Read the full abstract ▾
All authors Xiaofei Hui, Qian Wu, Haoxuan Qu, Majid Mirmehdi, Hossein Rahmani, Jun Liu
Computer Vision
Leveraging a Statistical Shape Model for Efficient Generation of Annotated Training Data: A Case Study on Liver Landmarks Segmentation
Denis Krnjaca, Lorena Krames and Werner Nahm
Anatomical landmark
segmentation serves as a critical initial step for robust
multimodal registration during computer-assisted interventions. Current approaches predominantly rely on deep learning, which often necessitates the extensive manual generation of annotated
datasets. In this paper, we present a novel strategy for creating large annotated
datasets using a statistical shape model (SSM) based on a mean shape that is manually labeled only once. We demonstrate the method's efficacy through its application to deep-learning-based anatomical landmark
segmentation, specifically targeting the detection of the anterior ridge and the falciform ligament in 3D liver shapes. A specialized deep learning network was trained with 8,800 annotated liver shapes generated by the SSM. The network's performance was evaluated on 500 unseen synthetic SSM shapes, yielding a mean Intersection over Union of 91.4% (87.4% for the anterior ridge and 87.6% for the falciform ligament). Subsequently, the network was applied to clinical patient liver shapes, with qualitative evaluation indicating promising results and highlighting the generalizability of the proposed approach. Our findings suggest that the SSM-based data generation approach alleviates the labor-intensive process of manual labeling while enabling the creation of large annotated training
datasets for machine learning. Although our study focuses on liver anatomy, the proposed methodology holds potential for a broad range of applications where annotated training
datasets play a pivotal role in developing accurate deep-learning models.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
EchoLVFM: One-Step Video Generation via Latent Flow Matching for Echocardiogram Synthesis
Emmanuel Oladokun, Sarina Thomas, Jurica Šprem et al. (+1 more)
Echocardiography is widely used for assessing cardiac function, where clinically meaningful parameters such as left-ventricular ejection fraction (EF) play a central role in diagnosis and management. Generative models capable of synthesising realistic echocardiogram
videos with explicit control over such parameters are valuable for data augmentation, counterfactual analysis, and specialist training. However, existing approaches typically rely on computationally expensive multi-step sampling and aggressive temporal normalisation, limiting efficiency and applicability to heterogeneous real-world data. We introduce EchoLVFM, a one-step latent
video flow-matching framework for controllable echocardiogram generation. Operating in the latent space, EchoLVFM synthesises temporally coherent
videos in a single inference step, achieving a $\mathbf{\sim 50\times}$ improvement in sampling efficiency compared to multi-step flow baselines while maintaining visual fidelity. The model supports global conditioning on clinical variables, demonstrated through precise control of EF, and enables reconstruction and counterfactual generation from partially observed sequences. A masked conditioning strategy further removes fixed-length constraints, allowing shorter sequences to be retained rather than discarded. We evaluate EchoLVFM on the CAMUS
dataset under challenging single-frame conditioning. Quantitative and qualitative results demonstrate competitive
video quality, strong EF adherence, and 57.9% discrimination accuracy by expert clinicians which is close to chance. These findings indicate that efficient, one-step flow matching can enable practical, controllable echocardiogram
video synthesis without sacrificing fidelity. Code available at: https://github.com/EngEmmanuel/EchoLVFM
Read the full abstract ▾
All authors Emmanuel Oladokun, Sarina Thomas, Jurica Šprem, Vicente Grau