Computer Vision
OuroReward: Sequential Reward Scheduling for Reinforcement Learning in Text-to-3D Generation
Bingyang Cui, Yujie Zhang, Yiling Xu et al. (+1 more)
Reinforcement learning (RL) for Text-to-3D (T23D) generation requires optimization across multiple quality dimensions such as semantic
alignment and texture clarity. Existing methods typically optimize these dimensions simultaneously through multiple reward aggregation, without explicitly modeling inter-dimension dependencies. This can cause imbalanced optimization and persistent interference among conflicting dimensions. To address this limitation, we propose OuroReward, an interference-aware sequential reward scheduling strategy for T23D RL. OuroReward first estimates pairwise dependencies among dimensions and constructs a cyclic optimization path that minimizes cumulative interference. By incorporating the tail-to-head dependency, the cycle captures global compatibility across the entire schedule. Then, OuroReward converts the cycle into a one-pass sequence, and starts optimization from the dimension with the lowest aggregate interference. Rather than assigning a fixed optimization budget to each dimension-wise reward, training adaptively determines when to advance to the next reward according to the remaining optimization headroom of the current one. We further introduce AdaSelect, an adaptive
prompt selection strategy that identifies reliable and informative
prompts aligned with the model's current capability. By focusing policy updates on these
prompts, AdaSelect effectively improves training stability. Extensive experiments across different T23D models and RL
algorithms demonstrate that our framework consistently improves generation quality across multiple dimensions.
Read the full abstract ▾
All authors Bingyang Cui, Yujie Zhang, Yiling Xu, Yunfeng Guan
Computer Vision · Cross-listed from stat.ML
Iterating Consistency Models: Stability, Error Bounds and Noise Schedules
Alessio Spagnoletti, Abdul-Lateef Haji-Ali, Andrés Almansa et al. (+3 more)
Consistency models (CMs) have become a leading approach for generating high-quality samples in few steps. However, adding steps can improve or degrade sample quality in ways that are highly sensitive to the schedule and that existing theory does not fully explain. To provide accuracy guarantees and guide CM sampler design, we analyze multistep CM sampling as a composition of noising and approximate denoising operators. Under explicit, verifiable stability assumptions, we derive a non-asymptotic error bound that separates contraction of the initialization error from accumulation of approximation error. The bound assigns distinct roles to the schedule: large early noise levels drive contraction, while small late noise levels control the residual bias. As a corollary, we obtain explicit constants for strongly log-concave and semi-log-concave targets. We further establish a complementary guarantee whose assumptions, one-step accuracy and stability, can be estimated for a given trained model. Experiments show that the contraction and approximation profiles entering our bounds can be reliably measured and closely match the predicted functional forms. Together, these results provide a meaningful convergence theory for multi-step CMs and a practical route to sampler design.
Read the full abstract ▾
All authors Alessio Spagnoletti, Abdul-Lateef Haji-Ali, Andrés Almansa, Alain Oliviero Durmus, Eric Moulines, Marcelo Pereyra
Computer Vision
ForestQuery: Boundary-Aware and Spatially Anchored Query Learning for Unified Forest Point Cloud Segmentation
Zhihao Zhan, Le Tao, Yifei Tian et al. (+2 more)
Forest point cloud segmentation is fundamental for fine-grained 3D forest scene understanding, yet remains challenging due to irregular tree structures, severe occlusions, density variations, and ambiguous instance boundaries. Recent query-based forest segmentation methods have shown promise for unified semantic and instance prediction, but they still insufficiently exploit forest-specific spatial structure and account for boundary uncertainty. In this paper, we propose ForestQuery, a boundary-aware and spatially anchored query learning framework for unified forest point cloud segmentation. ForestQuery enhances instance and semantic query learning through two complementary designs. Specifically, boundary uncertainty is explicitly modeled to guide reliable instance query construction and modulate query optimization through adaptive loss reweighting. Meanwhile, spatially anchored semantic query enhancement (SA-SQE) introduces learnable 3D anchors encoding forest vertical stratification priors to enrich semantic queries with explicit spatial references. We evaluate ForestQuery on multiple public forest point cloud
benchmarks and a self-collected annotated real-world
dataset. Extensive experiments demonstrate consistent improvements in both individual-tree segmentation and semantic segmentation across diverse forest scenes. Code and data are publicly available at https://zhan994.github.io/ForestQuery
Read the full abstract ▾
All authors Zhihao Zhan, Le Tao, Yifei Tian, Xin Liu, Jie Yuan
Computer Vision
Beyond Entropy: Self-Diagnostic Multi-Role Token Optimization for Video Reasoning
Yudong Han, Yong Wang, Zaiquan Yang et al. (+4 more)
Reinforcement learning with verifiable rewards has substantially advanced
multimodal reasoning, yet it remains fundamentally limited by ambiguous
token-level credit assignment. While high-
entropy token heuristics encourage possibility exploration, naively extending them to
video reasoning tends to induce lengthy
reasoning, as the model becomes overly reliant on high-
entropy visual activations. Alternative approaches that rely on counterfactual-based visual
token localization for credit assignment also tend to over-prioritize visual exploration at the expense of decisive
reasoning cues for answer derivation, thereby exacerbating the interference from spurious visual nuances. Moreover, these methods employ static counterfactual strategies that fail to co-evolve with the policy during training. In this paper, we introduce DyCPO, a co-evolutionary framework that jointly optimizes reliable
token selection and adaptive counterfactual intervention. It constructs a multi-role dependence metric to balance visual exploration and answer-relevance mining in
token-wise contrastive learning, while suppressing exploration-only filler
tokens and spurious visual noise. Rather than relying on static counterfactual priors, DyCPO dynamically derives counterfactual signals from the model's own successful and failed rollouts, enabling self-diagnostic analysis and co-
evolution of the optimization objective with the policy. Extensive experiments on complex
video reasoning and general
video understanding
benchmarks demonstrate consistent performance improvements, establishing DyCPO as a robust
token-level credit assignment paradigm for
multimodal reinforcement learning.
Read the full abstract ▾
All authors Yudong Han, Yong Wang, Zaiquan Yang, Liang Lin, Chongyang Tao, Xiangxiang Chu, Liyuan Pan
Computer Vision
Native Action-Prior Learning from Videos for World Action Models
Zhaochong An, Fei Zhang, Menglin Jia et al. (+10 more)
World action models integrate future visual dynamics with
robot action prediction, but their scalability remains limited by the need for action-annotated
robot trajectories. Observation-only
videos contain rich evidence about interaction dynamics, but existing approaches typically use them either to pretrain visual representations that must later be adapted for control, or to infer latent actions that are subsequently grounded to
robot commands. We present NAVA-WAM, which introduces native action-prior learning by directly pretraining the action policy from observation-only
videos, avoiding indirect representation-to-control transfer or a separate latent-action model. Our training consists of two stages. First, we pretrain on observation-only
videos, where future-
video flow-matching supervision over visual transitions is propagated through transition-structured joint
attention to optimize the Action-DiT and learn action-relevant priors. Second, we use action-labeled demonstrations to post-train the Action-DiT for
robot control through joint
video–action flow matching, while asymmetric
attention decouples the visual branch from iterative action denoising and enables efficient action-only inference. Extensive experiments show that NAVA-WAM consistently outperforms prior approaches under both in-distribution and out-of-distribution settings, while demonstrating strong action-label efficiency and effective real-
robot generalization. These results establish native action-prior learning as an effective approach to directly pretrain action policies from observation-only
videos, providing a scalable path beyond action-labeled
robot data.
Read the full abstract ▾
All authors Zhaochong An, Fei Zhang, Menglin Jia, Duncan Frost, Zijian Zhou, Yikai Wang, Xudong Wang, Aditya Patel, Belinda Zeng, Tao Xiang, Serge Belongie, Amir Bar, Sen He
Computer Vision
From Patching to Pruning Visual Computation in Vision Language Models
Rahul Chowdhury, Timothy A Rupprecht, Xuan Shen et al. (+3 more)
Vision language models (VLMs) incur substantial inference cost because every visual
token is processed by the
attention and MLP projections of every decoder layer, even when
token-specific visual computation is unnecessary at many depths. We introduce Patch-to-Prune (P2P), inspired by Mechanistic Interpretability, a training-free framework that converts activation patching from a diagnostic tool into an inference-time computation bypass. P2P performs validation-guided forward and backward layer sweeps to identify decoder regions whose visual-
token projection outputs can be replaced by fixed neutral proxy activation vectors within a user-specified accuracy tolerance. Unlike conventional
token-
pruning methods, P2P preserves the sequence length,
token order, positional information,
attention mask, and residual pathways, thereby
pruning computation without removing
tokens or modifying the pretrained model weights. We evaluate P2P on four VLMs from the Qwen2.5-VL and LLaVA families across seven multi-modal
benchmarks using mutually disjoint calibration, validation, and test partitions. P2P at a 3% tolerance retains around 94% of dense accuracy while reducing FLOPs by 55%. Beyond these efficiency gains, our layer-wise analysis suggests that visual processing in VLMs is non-uniformly distributed across decoder depth: early and late layers often require little
token-specific visual computation, whereas intermediate layers appear to perform most task-relevant visual integration, enabling later
reasoning to rely largely on visual information already embedded in shared residual and textual representations. This makes P2P both an efficient inference framework and a causal lens into visual information processing in VLMs.
Read the full abstract ▾
All authors Rahul Chowdhury, Timothy A Rupprecht, Xuan Shen, Shaoyi Huang, Pu Zhao, Yanzhi Wang
Computer Vision
Interpretable Deepfake Detection in Videos via Explicit Forensic Features and Temporal Modeling
Chahira Benhama, Mohand Saïd Allili and Assia Hamadene
Deepfake detection in
videos remains challenging, as manipulated content may appear visually consistent at the frame level while exhibiting subtle temporal inconsistencies. This paper introduces an interpretable deepfake detection framework that models spatially and temporally coherent facial features in
video sequences. Unlike end-to-end deep models relying on implicit representations, the proposed approach explicitly encodes physically grounded forensic cues, enabling transparent analysis and improved multi-
dataset generalization. The pipeline transforms
videos into identity-consistent facial trajectories, segments them into fixed-length temporal windows, and represents each frame using 68 structured descriptors spanning four complementary domains: photometric, textural,
geometric, and compression-based features. These descriptors provide a compact multi-domain representation of manipulation artifacts and are processed by a Long Short-Term
Memory (LSTM) network to capture temporal dependencies and subtle irregularities. Evaluation on four
benchmark datasets, FaceForensics++, Celeb-DF v2, a curated subset of the DeepFake Detection Challenge (DFDC), and DeeperForensics, yields strong and consistent F1-scores of 98.0%, 91.0%, 97.6%, and 96.2%, respectively. The approach also demonstrated a good cross-
dataset generalization, providing a robust and interpretable solution for
video deepfake detection.
Read the full abstract ▾
Computer Vision
EVEWorld: Physical Evolution Supervision for Embodied World Models
Kaiqi Wang, Songxin Zhang, Zejian Xie et al. (+7 more)
Embodied world models enable scalable simulation of embodied interactions for
robot learning. However, existing models are prone to Model Laziness, as they focus on visual fidelity at the expense of physical
reasoning and lack process-level supervision over the temporal dynamics of manipulated objects. In this work, we propose EVEWorld, a physical
evolution-supervision framework for physically consistent target
evolution. EVEWorld consists of two components: Instance-Guided Restoration (IGR) and Temporal Instance
Alignment (TIA). First, IGR promotes instance consistency through restoration supervision. Second, TIA promotes cross-frame consistency by aligning target instances across adjacent frames. We further introduce the Model Laziness Rate (MLR), a metric that measures persistent violations of instance consistency in generated trajectories. Extensive experiments on DreamGenBench, EWMBench, and PBench demonstrate the effectiveness of EVEWorld, notably achieving an 87.5% reduction in MLR compared with GigaWorld-0. On the WorldArena 2.0 Track 1 leaderboard, our model ranks 6th in JEPA Similarity and 17th overall, which further validates the performance of our
evolution supervision strategy.
Read the full abstract ▾
All authors Kaiqi Wang, Songxin Zhang, Zejian Xie, Xiao Xiong, Zhuoyang Song, Ziwei Wu, Jun Yu Lu, Yitan Teng, Ziying Song, Jiaxing Zhang
Computer Vision
LAS-CLIP: A Lightweight Adapter Steering Approach for CLIP's Visual Encoder
Anh-Khoa Dinh-Duc, Duc-Tai Dinh, Tam V. Nguyen et al. (+1 more)
CLIP's visual encoder produces only global image representations, limiting its use in region-level tasks. Existing adaptations rely on visual prompting, input masking, or encoder fine-tuning, each compromising pre-trained representations. We propose LAS-CLIP, a Lightweight Adapter Steering approach that keeps every CLIP parameter frozen. A compact MaskAdapter generates per-head, per-layer
attention biases from an input mask and injects them into the frozen self-
attention layers, steering
attention toward the target region. Crucially, because the backbone remains strictly untouched, LAS-CLIP seamlessly reverts to vanilla CLIP when no mask is provided, preserving its foundational zero-shot capabilities. With approximately 116K to 145K trainable parameters and 100K training samples on two T4 GPUs, LAS-CLIP achieves competitive or superior results compared to Alpha-CLIP on ImageNet-S zero-shot classification and RefCOCO referring expression comprehension, despite the latter fine-tuning its entire encoder on millions of samples. Qualitative analysis further confirms stronger representational fidelity under incorrect masks and in downstream generation.
Read the full abstract ▾
All authors Anh-Khoa Dinh-Duc, Duc-Tai Dinh, Tam V. Nguyen, Minh-Triet Tran
Computer Vision
A Fully Automatic Pipeline for 3D Dendrite Instance Segmentation in SBF-SEM
Zewen Zhuo, Ilya Belevich, Eija Jokitalo et al. (+2 more)
Accurate three-dimensional (3D) reconstruction of individual dendrites in serial block-face scanning electron microscopy (SBF-SEM) is essential for quantifying structural plasticity in the
brain, yet manual annotation at scale is infeasible. We present a fully automatic pipeline for 3D dendrite instance segmentation that unifies YOLOv6-guided Segment Anything Model (SAM) prompting on downsampled slices, iterative two-dimensional mask refinement, random forest 3D instance linking, and instance-aware high-resolution refinement using nnU-Net at native resolution into a single system requiring no manual prompting at inference. Applied to hippocampal CA1 SBF-SEM
datasets from a control rat and a pilocarpine- induced epileptic rat, our pipeline reconstructs coherent, well- separated dendrites with high semantic accuracy (Dice 0.93 and 0.91) and strong instance-level performance on control tissue, while analysis of the more challenging epileptic tissue identifies instance recognition in dense regions as the principal remaining limitation. The high-resolution refinement stage recovers thin dendritic protrusions, providing a basis for downstream spine- level analysis. Code is available at https://github.com/ ZE-WEN/dendrite-3d-instance-seg.
Read the full abstract ▾
All authors Zewen Zhuo, Ilya Belevich, Eija Jokitalo, Alejandra Sierra, Jussi Tohka
Computer Vision
T3lescope: Arbitrary-Resolution High-Fidelity Generative Surface Reconstruction from Images
Atsuhiro Noguchi, Tianhan Xu, Yiming Liang et al. (+5 more)
We reconstruct high-fidelity 3D scene meshes from posed multi-view images without per-scene optimization, across scales ranging from single objects to large outdoor scenes. Per-scene optimization methods lack the learned 3D prior needed when observations are sparse or surfaces are glossy or transparent. Existing generative methods leverage such priors to complete geometry in sparsely observed regions, but typically operate at a fixed resolution over a limited spatial extent, trading spatial coverage against detail. Reconstructing a large scene therefore often requires partitioning it into independently processed overlapping local regions, making it difficult to maintain global
geometric consistency. To address these issues, we propose T3lescope, which applies a single fixed-resolution generator across scene scales in an inference-time coarse-to-fine cascade. A coarse level establishes the scene layout, and finer levels perturb and denoise geometry inherited from the coarser level within progressively finer spatial
cells to recover surface detail. The model is trained on individual
cells at multiple scales and shares its weights across all levels, so no hierarchy is fixed during training, and the number of levels,
cell scales, and
cell locations are determined at inference time. On indoor, outdoor, and city-scale scenes, T3lescope outperforms feed-forward and generative baselines, matches or surpasses per-scene optimization, and recovers fine structures as well as glossy and transparent surfaces. These results show that our method generalizes across diverse scenes, view counts, and image resolutions. Project page: https://pfnet-research.github.io/t3lescope/
Read the full abstract ▾
All authors Atsuhiro Noguchi, Tianhan Xu, Yiming Liang, Yuta Kikuchi, Masahiro Ishiyama, Shintaro Takagi, Hitoshi Murai, Eiichi Matsumoto
Computer Vision · Cross-listed from eess.IV
Wrong Organ, Right Physics: Transferring Echocardiography Pretraining to Lung Ultrasound for Tuberculosis Screening
Christiaan M. Geldenhuys, Joshua M. Jansen van Vüren, Véronique Suttels et al. (+6 more)
Lung ultrasound (LUS) is attractive for tuberculosis (TB) screening at primary-care level, but labelled cohorts are small. Echocardiography carries no such constraint, while sharing the same underlying ultrasound imaging physics, signal processing and B-mode appearance as LUS. We ask whether an encoder pretrained on that high-resource ultrasound domain carries representations that remain usable in the low-resource one. Only the encoder varies, across seventeen encoders spanning three architecture families. Among them, a latent-predictive
video encoder pretrained on generic
video (V-JEPA2-L) and its echocardiography counterpart (EchoJEPA-L) differ in pretraining corpus alone. The choice among these encoders does not resolve the classification, the whole family spanning 2.50 percentage points against a measurement resolution of 2.71. What moves the task instead is feature conditioning. Standardising the features between the encoder and the classifier improves all seventeen encoders by a mean of +1.23 percentage points at $p=1.5\times10^{-5}$. On the held-out test set every encoder selected on the development folds stands above the baseline system by up to +2.57 percentage points of area under the receiver operating characteristic curve (AUROC), and specificity at 90% sensitivity reaches 79.3% against 60.3%. The contrast specified in advance, EchoJEPA-L against V-JEPA2-L, measures -0.16 percentage points at $p=0.926$. We therefore find no evidence that shared ultrasonic physics alone makes echocardiography a more productive pretraining corpus than generic
video, and any advantage, if present, is smaller than this cohort can resolve. The
video encoders receive replicated still images, however, so whether this absence of an effect reflects the pretraining domain or a
video encoder applied to static frames cannot be separated. The limiting factor is the labelled cohort rather than the encoder.
Read the full abstract ▾
All authors Christiaan M. Geldenhuys, Joshua M. Jansen van Vüren, Véronique Suttels, Trevor Brokowski, Ablo P. Wachinou, Mary-Anne Hartley, Rensu P. Theart, Grant Theron, Thomas R. Niesler
Computer Vision · Cross-listed from cs.RO
HexVIO: Towards All-Day Stereo-Inertial Tracking Through Commodity DSPs
Patrick Wolf, Mateo de Mayo and Daniel Cremers
The ability of a device to localize itself within its surroundings is a fundamental prerequisite for spatial computing. Visual-inertial odometry (VIO) has proven to be a cost-effective and accurate solution for this task.
Robots, wearables, XR devices, and drones can benefit significantly from efficient implementations of VIO since they allow for cooler, lighter, and cheaper devices with longer battery life and a better user experience. In this work, we propose to enhance the efficiency of a VIO system by leveraging the Hexagon DSP, a commodity co-processor present in many modern smartphones and XR devices. Our approach offloads the visual frontend of a stereo-inertial odometry system to the DSP while keeping the backend on the main CPU. By optimizing the implementation for the DSP architecture, we achieve significant reductions in power consumption and latency compared to CPU-only execution. Our system, HexVIO, demonstrates a 67% reduction in power consumption or an 86% increase in throughput on a commodity smartphone, with the ability to sustain long-term real-time 30 fps tracking for 0.83 W, corresponding to ~18 hours of tracking on the testing device. These results highlight the potential of commodity DSPs for enabling all-day visual-inertial tracking in robotics and mobile devices.
Read the full abstract ▾
Computer Vision
Moving Forward with Video Saliency: A New Dataset and Benchmark where Motion Matters
Susmit Agrawal, Rebecca Wanner, Juliane Verwiebe et al. (+3 more)
Video saliency prediction is inherently harder to model than static image saliency due to the additional temporal dimension.
Video saliency
benchmarks rest on the premise that predicting gaze on
video requires utilizing temporal activity distributed across frames. Prior work has challenged this, showing that static baselines recover a significant fraction of the explainable gaze information on LEDOV, a popular
video saliency
dataset, and that
video saliency models fail in the same places as this static baseline. We verify that this diagnosis still stands: under a more capable gold standard than the original analysis, and an updated panel of recent architectures, the strongest temporal architecture in the panel still does not substantially improve over a fine-tuned static baseline. However, it remains unclear whether the marginal gain reflects limitations of current temporal architectures or a lack of temporal patterns in the
benchmark itself. We introduce SalTempto, a
video saliency
benchmark with greater dynamism: 224 clips of highly dynamic content, sourced from the HACS-Segments
dataset so that each clip contains an event together with its lead-up and aftermath, with gaze recordings from up to 16 subjects and a training split for adapting pretrained models. On SalTempto, the static baseline recovers only about 13% of the headroom above the centerbias, against more than half on LEDOV. A fine-tuned temporal architecture shows a substantial gain in performance over the static baseline, indicating that it does capture meaningfully more temporal information, which LEDOV fails to measure. Yet, even this SoTA model still leaves nearly half of SalTempto's headroom unexplained, indicating room for improvement in
video saliency modelling. Examination of SalTempto also lets us describe human tendencies that models miss. SalTempto link: https://huggingface.co/
datasets/bethgelab/
video_saliency.
Read the full abstract ▾
All authors Susmit Agrawal, Rebecca Wanner, Juliane Verwiebe, Matthias Tangemann, Matthias Bethge, Matthias Kümmerer
Computer Vision
Consecutive Posterior Fusion for Diffusive Recovery of Unobservable Image Structures
Elena Morotti, Davide Evangelista and Elena Loli Piccolomini
Solving severely ill-posed imaging inverse problems requires recovering image structures that are unobservable or weakly constrained by the measurements.
Diffusion models provide expressive learned priors for inferring such missing information, while posterior sampling incorporates measurement consistency along the reverse process. Standard
diffusion posterior samplers, however, rely on instantaneous measurement-aware estimates, without explicitly exploiting information carried by previous posterior corrections. We introduce Consecutive Posterior Fusion Denoising
Diffusion Null-Space Models (CPF-DDNM), an inference-time strategy that fuses consecutive measurement-aware estimates to improve the diffusive recovery of unobservable image structures, without requiring retraining or additional denoiser evaluations. We instantiate this principle within DDNM, whose range/null-space decomposition reveals that consecutive fusion preserves the measurement-determined component while acting exclusively on the prior-driven null-space estimate. We thus provide a
geometric interpretation of CPF-DDNM and a local error analysis that characterizes the optimal time-dependent fusion coefficient, including the extrapolative regime. Experiments on sparse-view and simulated low-dose computed
tomography, as well as medical image super-resolution, show consistent improvements over DDNM and competitive performance against
diffusion-based inverse solvers.
Read the full abstract ▾
Computer Vision
COSMI: COmpositional Synthesis of Multi-object Interactions
Daniel Eskandar, Ilya A. Petrov and Gerard Pons-Moll
Generative models of human-object interaction are bounded by the data that exists: everyday activities involve several objects, but most captured
datasets record one at a time, as multi-object capture is combinatorially expensive. Our observation is that interactions are local, so single-object captures already contain the parts of multi-object activities. We compose them: contact-consistent clips of single interactions, mirrored to balance the hands, transfer between bodies, and a
language model and
geometric checks admit only the pairings that are plausible, semantically and physically. Therefore, the
dataset grows combinatorially with the clips rather than recording time. The COSMI
dataset holds 222k sequences and 275 hours with up to five objects, nearly thirty times the largest multi-object capture, and can be extended by adding
datasets or even hand-object recordings. On this data we train the COSMI method, a text-to-interaction
diffusion transformer that follows how the data is built: weight-shared object slots generate a variable number of objects, predicted relative to the body parts that move them. On a
benchmark with an unseen object and unseen interaction combinations, models trained on the
dataset generalize to the unseen combinations. COSMI outperforms baselines in text
alignment and contact accuracy, where its margin is largest on the unseen object. Code, models, and the
dataset pipeline will be released on the project page: https://ptrvilya.github.io/cosmi.
Read the full abstract ▾
Computer Vision
EmbPASS: Towards Cross-Embodiment Open Panoramic Segmentation
Pujun Guo, Yuanfan Zheng, Fei Teng et al. (+5 more)
Panoramic images provide a complete 360-degree field of view, enabling comprehensive scene understanding for embodied perception. However, heterogeneous embodied platforms exhibit substantial differences in observation viewpoints and spatial layouts, giving rise to cross-embodiment observation shifts that pose additional challenges to consistent and reliable panoramic perception, while systematic studies of this problem remain limited. To bridge this gap, we introduce a new task, termed Cross-Embodiment Open Panoramic Segmentation. Meanwhile, we establish EmbPASS, a multi-platform panoramic semantic segmentation
benchmark spanning Vehicle, Drone, Wearable, and Quadruped platforms under a unified semantic taxonomy, providing a testbed for systematically studying cross-embodiment panoramic perception. We further propose EPONet, an open-vocabulary panoramic semantic segmentation network that integrates Relation-Aware Metric Adapter (RAMA) and Content-Adaptive Semantic Transfer (CAST) to enhance spatial modeling and semantic transfer under heterogeneous embodied observations. Extensive experiments show that EPONet achieves the best platform-balanced performance on EmbPASS with 35.82% mIoU, outperforming the strongest baseline by 1.10%, while remaining competitive on existing panoramic segmentation
benchmarks. The source code and EmbPASS
benchmark will be made publicly available at https://github.com/guopj1/EmbPASS.
Read the full abstract ▾
All authors Pujun Guo, Yuanfan Zheng, Fei Teng, Mengfei Duan, Guoqiang Zhao, Yuheng Zhang, Kai Luo, Kailun Yang
Computer Vision
Uncertainty as a Proxy for Semantic Correctness in Diffusion-Based Medical Image Synthesis
Yuxuan Ou, Konstantinos Kamnitsas, OxAAA Study et al. (+3 more)
Diffusion models can synthesise contrast-enhanced CT (CECT) from non-contrast CT (NCCT), avoiding contrast administration and its environmental and patient-access costs. However, visually realistic images are not necessarily anatomically correct, and the pixel-intensity and feature-space similarity metrics used to assess generation quality do not directly measure anatomical correctness. In this work, we investigate whether uncertainty can serve as a proxy for semantic correctness in
diffusion-based medical image synthesis. We study NCCT-to-CECT synthesis using AortaDiff, a multitask
diffusion framework that jointly generates CECT images and lumen segmentations. The segmentation output provides an explicit representation of the generated vascular anatomy, enabling segmentation-derived errors to be used as a quantitative measure of generation correctness. Six methods spanning weight (Ensemble, HyperDiff, BayesDiff), architecture-perturbation (MCDropout), generative-stochasticity (RDS) and input-perturbation (TTA) uncertainty are compared at the pixel, region and image levels, and for detection of clinically relevant out-of-distribution (OOD) cases. Uncertainty proves informative at all three spatial scales, remains informative on an external multi-centre
dataset under distribution shift, and supports OOD detection. MCDropout stands out among the six: it ranks among the leading methods at every scale, generalizes well on the external
dataset, and can be enabled at inference on any model already trained with dropout, so reliable uncertainty comes at no extra training cost. Uncertainty reliably flags severe failures but discriminates poorly among already high-quality images. These findings support uncertainty as a practical and computationally economical signal for quality filtering, reliability assessment and OOD detection in NCCT-to CECT synthesis.
Read the full abstract ▾
All authors Yuxuan Ou, Konstantinos Kamnitsas, OxAAA Study, AICT Consortium, Regent Lee, Vicente Grau
Computer Vision
VDOT++: Unified Few-Step Video Generation via Unbalanced Optimal Transport Distillation
Yutong Wang, Xingtong Ge, Enhuai Liu et al. (+6 more)
Video creation spans text-to-
video (T2V), image-to-
video (I2V), and condition-based generation, yet
video diffusion models remain costly because they repeatedly evaluate large backbones during sampling. Distribution matching
distillation (DMD) reduces this cost, but its reverse Kullback–Leibler (KL) objective can provide unstable or incomplete guidance when the student and teacher distributions have limited overlap. VDOT addressed this issue by adding optimal transport
distillation (OTD), whose explicit coupling supplies
geometric directions for condition-based generation. Balanced OTD, however, performs full-mass matching between the spatial
tokens of each corresponding student–teacher frame pair. This assumption weakens for T2V and I2V, where one condition admits many valid outputs and spatial content need not align across different realizations. We present VDOT++, a unified
distillation framework that applies the same training recipe separately to generators for the three task families. It makes OTD robust to output diversity through an asymmetric unbalanced formulation that allows unreliable student
tokens to carry less mass while maintaining coverage of the teacher
tokens. An $\ell_1$ ground cost further replaces mean-based aggregation with a more mode-preserving weighted median that limits the influence of distant transport targets. The two changes respectively determine whom to match and how the selected targets should be aggregated. We additionally combine distribution matching and
adversarial refinement through sequential backward passes, and exploit the decoupled score networks for cross-scale
distillation, where larger score networks improve a compact generator. Experiments on UVCBench, VBench, VBench-I2V, and the VACE
benchmark show that the resulting four-step generators are competitive with many-step teachers and strong few-step baselines across all three task families.
Read the full abstract ▾
All authors Yutong Wang, Xingtong Ge, Enhuai Liu, Yunke Wang, Tianfan Xue, Yu Qiao, Yaohui Wang, Xinyuan Chen, Chang Xu
Computer Vision · Cross-listed from cs.NE
Evolving Hybrid Quantum-Classical Architectures for Image Classification
Devroop Kar, Daniel Krutz and Travis Desell
Hybrid
quantum classical neural networks integrate parameterized
quantum circuits (PQCs) with established deep learning architectures, but their performance depends strongly on the choice of
quantum circuit architecture, a choice that remains largely manual. Most existing approaches rely on hand-designed or fixed
circuit ansätze, requiring
circuit structure, gate composition, and
qubit connectivity to be specified in advance with no guarantee that they suit the task. This limitation is especially acute in image classification, where
quantum circuits must transform features extracted by classical networks while remaining compact enough for practical training, requirements that generic, task-agnostic ansätze are unlikely to satisfy simultaneously. We extend EXAQC, an evolutionary framework for automated
quantum circuit discovery, to image classification. EXAQC evolves PQCs as intermediate processing modules while retaining classical feature-extraction and prediction layers. On MNIST, Fashion-MNIST, and CIFAR-10, EXAQC achieves 98.42%, 90.62%, and 85.47% accuracy, respectively, while using comparable gate counts to other
quantum architecture-search methods. Against classical networks, evolved hybrid models maintain comparable accuracy with substantially fewer trainable parameters, reaching 85.68% on CIFAR-10 with over 25$\times$ fewer parameters than a 10-layer CNN. Encoding choice also matters: rotation-based encodings (RX, RY, U3) outperform amplitude encoding by 22-25 points on CIFAR-10. These results demonstrate that automated
circuit discovery yields compact
quantum modules that can replace larger classical components in
vision architectures while retaining competitive accuracy.
Read the full abstract ▾
Computer Vision
VisionMX: Unlocking Microscaling Post-Training Quantization for Vision Models
Elad Dror Cohen, Ofir Gordon, Lior Dikstein et al. (+2 more)
Microscaling (MX) formats are emerging as a hardware-supported approach to efficient training and inference. They combine low-precision elements with shared block scales, but their impact on
vision models remains underexplored. We systematically investigate post-training MX quantization across
vision models and tasks. An analysis of direct conversion identifies three sources of error: block-scale representation, the poor
alignment of some small convolutional weight tensors with nonuniform element grids, and the underuse of signed codes by nonnegative activations. These findings motivate VisionMX, a post-training MX quantization method that optimizes bounded weight rounding and applies a foldable affine correction to activations. We evaluate VisionMX across image classification, object detection, semantic segmentation, and low-light image enhancement using several MX-style formats. It improves on direct conversion and the evaluated post-training quantization baselines, with the largest performance recoveries in architectures most sensitive to MX conversion
Read the full abstract ▾
All authors Elad Dror Cohen, Ofir Gordon, Lior Dikstein, Idan Achituve, Hai Victor Habi
Computer Vision
Contextual Flow Matching: Adaptive Step Selection in Flow Models for Efficient Visual Generation
Divya Jyoti Bajpai, Arun Verma and Manjesh Kumar Hanawal
Flow Matching enables high-quality visual generation via continuous-time dynamics, but inference remains costly due to multiple sequential function evaluations. Existing acceleration methods reduce the number of function evaluations but often introduce additional training overhead, degrade quality, or fail to account for input-dependent variability. We propose COFLOW, an inference-time method that adaptively selects the step counts each generation based on the
prompt features. Our context-aware COFLOW is trained online with an unsupervised reward that balances inference efficiency and generation fidelity. Our method is plug-and-play, requiring no retraining of the underlying generative model. It generalizes to image and
video generation, achieving over 2.5x speedup while preserving perceptual and semantic quality. We further provide a theoretical analysis establishing an O(1/K) forward-Euler discretization error bound under standard regularity conditions.
Read the full abstract ▾
Computer Vision
Bridging Research and Practice: A Systematic Evaluation of Generalist and Dermatology-Specific Models in Clinical Skin Lesion Classification
Emanoel dos Santos, Kelvin Cunha, Rodrigo Mota et al. (+8 more)
The application of machine learning to dermatology has grown substantially in recent years, moving beyond proof-of-concept studies toward potential applications. However, clinical dermatology remains a challenging and still open problem. Diagnostic assessment is often ambiguous, and skin lesions exhibit high variability, compounded by differences in acquisition modality, device quality, and patient demographics. These factors hinder the development of robust models suitable for safe and equitable clinical use. To support translation into practice, it is essential to systematically evaluate how contemporary models generalize across heterogeneous data sources. In this work, we
benchmark a diverse set of architectures on recent dermatology
datasets, spanning dermoscopic images and smartphone-based clinical photographs. We assess the robustness of recent general-purpose and medical
vision-
language models, as well as foundation models, and compare them against task-specific dermatology classifiers, including
embedding-based approaches and convolutional neural networks. Our study provides an evaluation of model performance under distribution shifts, modality changes, and demographic variability. By quantifying the gap between current state-of-the-art models and the requirements of clinical deployment, we aim to contribute to the development of reliable, accessible, and clinically applicable AI systems for dermatology.
Read the full abstract ▾
All authors Emanoel dos Santos, Kelvin Cunha, Rodrigo Mota, Fabio Papais, Thales Bezerra, Natalia Lopes, Erico Medeiros, Shirley Cruz, Jessica Araujo, Paulo Borba, Tsang Ing Ren
Computer Vision
PocketSplat: Mobile Gaussian Reconstruction via World-Space Latent Allocatio
Wenzhi Guo, Xianda Chen, Dongxuan Chen et al. (+2 more)
Mobile Gaussian reconstruction must satisfy two requirements: the reconstruction model must execute within a device resource envelope, and the resulting Gaussian asset must expose a representation size suited to downstream mobile use. Existing feed-forward Gaussian reconstructors commonly decode dense, image-aligned candidates whose final cardinality is implicitly determined by the input resolution and number of views. We present PocketSplat, a feed-forward framework for budgeted mobile Gaussian asset construction. Given a prescribed output budget, PocketSplat organizes dense geometry-aware latent candidates in predicted world space, allocates exact integer capacity across local latent
cells, and decodes complete Gaussian attributes only for retained candidates.
Cell-conditioned latent fusion aggregates repeated multi-view evidence before decoding, while spatial responsibility decoding adapts Gaussian support after local sparsification. Experiments on DL3DV and out-of-distribution
benchmarks establish a strong quality–budget trade-off against feed-forward Gaussian reconstruction baselines. On Mip-NeRF 360, PocketSplat executes directly on a target iPhone and constructs compact, higher-quality Gaussian assets substantially faster than a deployable streamed MVSplat variant; native MVSplat and DepthSplat exceed the device
memory budget.
Read the full abstract ▾
All authors Wenzhi Guo, Xianda Chen, Dongxuan Chen, Guangchi Fang, Bing Wang
Computer Vision · Cross-listed from cs.DC
Lightweight and Resource-Efficient Perception for Robotic Guide Dogs
Jinse Kwon, Yoojin Lim, Choonghan Lee et al. (+3 more)
Robotic guide dogs should understand their surroundings, objects, and potential risks. Prior research has focused on raw sensor data from cameras and 2D or 3D
LiDAR, which precisely measure distance points rather than provide a semantic understanding of the scene. While these physical measurements are effective for
robot-centric collision avoidance and
robot safety, they are not suitable for human-centric guidance. The system should recognize the type and relevance of obstacles and explain them, clearly and actionably, in terms of their spatial relation to the user. We present complete on-device perception modules that fuse a 360 camera and a 2D
LiDAR for reliable collision avoidance, with moving-object detection and tracking for human-centric guidance. Finally, in walking-impossible situations, a
vision–
language model delivers pathway explanations as a safety mechanism to reduce user anxiety. In experiments, verification of fused 360 camera–
LiDAR depth shows reliable near-range perception but inherent mid-range bias, while the system as a whole sustained real-time performance under 55 W. On the real-world egocentric GuideDogQA
benchmark, our system achieved 83.8% accuracy, compared with 67.1% for GPT-4o. These results demonstrate that practical human-centric guidance with real-time on-device inference is feasible even on quadrupeds.
Read the full abstract ▾
All authors Jinse Kwon, Yoojin Lim, Choonghan Lee, Yongseung Yu, Yongin Kwon, Jemin Lee
Computer Vision
Geometry-Aligned Semantic Matching for Cross-Modal Planar Image Registration
Zhiwei Wang, Defeng He, Yuxing Li et al. (+2 more)
Cross-modal image matching establishes stable and accurate
geometric correspondences across modalities for planar registration. Existing semantic representations provide cross-modal consistency, but semantic similarity does not necessarily imply
geometric correspondence. Meanwhile, fine-grained CNN features provide accurate local details but lack global cross-modal semantic guidance for stable refinement. To address these issues, we propose CDPM, which first establishes geometrically consistent semantic representations and then preserves their dominant role in correspondence estimation during fine-grained localization. Specifically, we progressively adapt DINOv3 using geometrically consistent cross-modal patch pairs, enabling feature similarity to better reflect true cross-modal spatial correspondences. We then construct a DINO-Centric Feature Pyramid, where multi-scale DINO representations maintain stable cross-modal correspondences, while a lightweight CNN branch provides auxiliary structural details for precise local refinement. Extensive experiments on three cross-modal
datasets demonstrate the superior performance of CDPM. On VIS-IR, compared with the dense matcher RoMa, CDPM improves AUC@3/5/10/20 by 7.36, 13.40, 13.75, and 10.42 percentage points, respectively, and reduces mACE from 5.83 to 2.78 pixels. It also outperforms RoMa v2 across all metrics while requiring 45.6% fewer FLOPs. The online demo and
dataset are available, and the code will be released on our project page at https://warren-wzw.github.io/CDPM/.
Read the full abstract ▾
All authors Zhiwei Wang, Defeng He, Yuxing Li, Meilu Zhu, Edmund Y. Lam
Computer Vision · Cross-listed from cs.GR
Budgeted-GS: Real-Time Large-Scale Gaussian Splatting via Factoring LOD
Haipeng Wang
3D Gaussian Splatting achieves excellent visual quality with real-time rendering, but at the scale of entire cities it does not fit: a trained model carries millions of primitives and gigabytes of
memory, and real-time rendering at high quality on a consumer GPU remains out of reach. We introduce Budgeted-GS, a post-hoc method that turns any trained 3DGS model into a factoring tree, a multi-resolution hierarchy of moment-matched aggregates. After a construction pass of a few seconds, a single quality parameter selects, for each view, the level of detail that fits the
memory of the target device, so the same city-scale model serves GPUs with widely different
memory capacities. When a new scene is to be trained, the same theory applies: instead of growing a full-sized model and compressing it afterwards, budget-centered training first measures how many primitives the scene needs and then trains the model directly at that size, avoiding the wasted effort of optimizing primitives that are later discarded. Both methods are grounded in a measurable capacity floor, a budget-error law derived from optimal transport in phase space; selection rules certified by recent covering
theorems decide which primitives are redundant. The floor answers how many primitives a scene actually needs and how many can safely be given up. We validate the floor on 13 public scenes under a preregistered protocol, and exercise both methods from object scenes to an official city capture, rendering it at native 1920x1080, full SH, in real time on one consumer GPU.
Read the full abstract ▾
Computer Vision
Does Physics Live in the Activations? Localizing Physical Quantities in Video Diffusion Models
Jonas Kneifl, Jakub Skalski, Bartłomiej Twardowski et al. (+1 more)
Video generation models produce strikingly realistic sequences and are increasingly proposed as world models, yet recent
benchmarks reveal pronounced deficits in their physical
reasoning. This raises the question of whether these models internalize physical principles or merely reproduce familiar motion patterns. We address this by probing internal representations of
video Diffusion Transformers (DiTs) for simulator-derived ground-truth physical quantities spanning kinematic motion and rigid-body dynamics under
gravity and contact. We find that these quantities are linearly decodable with high accuracy early in the denoising process, substantially outperforming a baseline decoded directly from the model's own noised latents, indicating that the relevant physical information is actively constructed during denoising rather than already present in the input. Additionally, we show that activations at on-object
tokens carry the relevant physical information and that quantities defined over multiple frames are readable from single latent frames. Hence, information is sharply localized within the
token sequence and is computed globally but stored locally. The probes further show partial extrapolation, transferring to scene variations and object configurations outside their training regime, so what they read is not simply a correlate of the scenes they were fit on. When fitted directly in the full-resolution activation space, the probing directions can serve as steering vectors to change the model's output.
Read the full abstract ▾
All authors Jonas Kneifl, Jakub Skalski, Bartłomiej Twardowski, Kamil Deja
Computer Vision
CalCErt: Bin-wise Certification of Confidence Calibration in Medical Image Classification
Leo Fillioux, Stergios Christodoulidis, Stergios Christodoulidis et al. (+2 more)
Deep neural networks remain vulnerable to
adversarial perturbations, which can distort not only predictions but also confidence scores, undermining uncertainty calibration. While existing certification methods focus on preserving the predicted category, providing guarantees on how calibration behaves under
adversarial attacks remains overlooked. In this work, we introduce CalCErt, a simple and efficient post-hoc strategy that certifies bin-wise confidence calibration for any pretrained differentiable classifier. Our approach combines empirical calibration estimates, statistical concentration bounds, and local Lipschitz estimates of the confidence function to derive data-dependent upper bounds on worst-case miscalibration within an $ell_2$-ball of radius R. We evaluate CalCErt across 11 medical image classification tasks and multiple
adversarial perturbations, demonstrating substantially higher certified coverage than baseline strategies while maintaining competitive tightness. Our code is available at https://github.com/leofillioux/calcert.
Read the full abstract ▾
All authors Leo Fillioux, Stergios Christodoulidis, Stergios Christodoulidis, Maria Vakalopoulou, Jose Dolz
Computer Vision
Behavior Pack Optimization for Video MLLM Post-Training
Zhaolu Kang, Shiyu Liu, Tailong Luo et al. (+12 more)
Video multimodal large
language models (MLLMs) keep climbing
video question answering
benchmarks, yet shuffling the frames, masking the segment that supports the answer, or occluding the target object barely changes their predictions. The accuracy rests on appearance and
language priors, not on the temporal evidence the question asks for. We trace this to the unit of post-training: rewards are computed on a single response to the original clip, so the model is never asked to behave consistently across views. We propose Behavior Pack Optimization (BPO), which replaces the single response with a behavior pack of outputs across counterfactual views chosen by question type, scored jointly. The pack reward asks for stability when the intervention is irrelevant, sensitivity when key evidence is removed, and abstention when no evidence remains. To keep this objective stable at small pack sizes, BPO uses an anchor-relative advantage: the response on the original view serves as a per-
prompt reference instead of a group mean over mixed views. On TempCompass, MVBench, and NExT-QA, BPO improves the macro accuracy of Qwen2.5-VL-7B-Instruct by 4.7 pp, the temporal-hard subset by 7.8 pp, and abstention F1 by 20.0 pp over a budget-matched vanilla GRPO baseline from the same SFT checkpoint. The gains transfer to
Video-MME, LongVideoBench, and to LLaVA-
Video-7B; ablations confirm they follow the view sets, not the rollout count. We hope this pack-level perspective offers a useful starting point for the
video MLLM and
multimodal post-training community as the field moves toward evidence-grounded
video reasoning.
Read the full abstract ▾
All authors Zhaolu Kang, Shiyu Liu, Tailong Luo, Wei Zhang, Yingjie He, Lei Wei, Guansu Wang, Liang He, Siheng Wang, Guangyuan Dong, Jiaqi Su, Shuang Chen, Haoyu Ji, Qishi Zhan, Kaiyue Zhou