Computer Vision
End-to-end reconstruction of OCT optical properties and speckle-reduced structural intensity via physics-based learning
Jinglun Yu, Yaning Wang, Wenhan Guo et al. (+3 more)
Inverse scattering in optical coherence
tomography (OCT) seeks to recover both structural images and intrinsic tissue optical properties, including refractive index, scattering coefficient, and anisotropy. This inverse problem is challenging due to attenuation, speckle noise, and strong coupling among parameters. We propose a regularized end-to-end deep learning framework that jointly reconstructs optical parameter maps and speckle-reduced OCT structural intensity for layer visualization. Trained with Monte Carlo-simulated ground truth, our network incorporates a physics-based OCT forward model that generates predicted signals from the estimated parameters, providing physics-consistent supervision for parameter recovery and artifact suppression. Experiments on the synthetic corneal OCT
dataset demonstrate robust optical map recovery under noise, improved resolution, and enhanced structural fidelity. This approach enables quantitative multi-parameter tissue characterization and highlights the benefit of combining physics-informed modeling with deep learning for computational OCT.
Read the full abstract ▾
All authors Jinglun Yu, Yaning Wang, Wenhan Guo, Yuan Gao, Yu Sun, Jin U. Kang
Computer Vision · Cross-listed from physics.med-ph
Perfusion Imaging and Single Material Reconstruction in Polychromatic Photon Counting CT
Namhoon Kim, Ashwin Pananjady, Amir Pourmorteza et al. (+1 more)
Background: Perfusion computed
tomography (CT) images the dynamics of a contrast
agent through the body over time, and is one of the highest X-ray dose scans in medical imaging. Recently, a theoretically justified reconstruction
algorithm based on a monotone variational inequality (VI) was proposed for single material polychromatic
photon-counting CT, and showed promising early results at low-dose imaging. Purpose: We adapt this reconstruction
algorithm for perfusion CT, to reconstruct the concentration map of the contrast
agent while the static background tissue is assumed known; we call our method VI-PRISM (VI-based PeRfusion Imaging and Single Material reconstruction). We evaluate its potential for dose-reduced perfusion CT, using a digital phantom with water and iodine of varying concentration. Methods: Simulated iodine concentrations range from 0.05 to 2.5 mg/ml. The simulated X-ray source emits
photons up to 100 keV, with average intensity ranging from $10^5$ down to $10^2$
photons per detector element. The number of tomographic projections was varied from 984 down to 8 to characterize the tradeoff in
photon allocation between views and intensity. Results: We compare VI-PRISM against filtered back-projection (FBP), and find that VI-PRISM recovers iodine concentration with error below 0.4 mg/ml at all source intensity levels tested. Even with a dose reduction between 10x and 100x compared to FBP, VI-PRISM exhibits reconstruction quality on par with FBP. Conclusion: Across all
photon budgets and angular sampling densities tested, VI-PRISM achieved consistently lower RMSE, reduced noise, and higher SNR compared to filtered back-projection. Even in extremely
photon-limited and sparsely sampled regimes, VI-PRISM recovered iodine concentrations with errors below 0.4 mg/ml, showing that VI-PRISM can support accurate and dose-efficient perfusion imaging in
photon-counting CT.
Read the full abstract ▾
All authors Namhoon Kim, Ashwin Pananjady, Amir Pourmorteza, Sara Fridovich-Keil
Computer Vision
AdaptMMBench: Benchmarking Adaptive Multimodal Reasoning for Mode Selection and Reasoning Process
Xintong Zhang, Xiaowen Zhang, Jingrong Wu et al. (+8 more)
Adaptive
multimodal reasoning has emerged as a promising frontier in
Vision-
Language Models (VLMs), aiming to dynamically modulate between tool-augmented visual
reasoning and text
reasoning to enhance both effectiveness and efficiency. However, existing evaluations rely on static difficulty labels and simplistic metrics, which fail to capture the dynamic nature of difficulty relative to varying model capacities. Consequently, they obscure the distinction between adaptive mode selection and general performance while neglecting fine-grained process analyses. In this paper, we propose AdaptMMBench, a comprehensive
benchmark for adaptive
multimodal reasoning across five domains: real-world, OCR, GUI, knowledge, and math, encompassing both direct perception and complex
reasoning tasks. AdaptMMBench utilizes a Matthews Correlation Coefficient (MCC) metric to evaluate the selection rationality of different
reasoning modes, isolating this meta-cognition ability by dynamically identifying task difficulties based on models' capability boundaries. Moreover, AdaptMMBench facilitates multi-dimensional process evaluation across key step coverage, tool effectiveness, and computational efficiency. Our evaluation reveals that while adaptive mode selection scales with model capacity, it notably decouples from final accuracy. Conversely, key step coverage aligns with performance, though tool effectiveness remains highly inconsistent across model architectures.
Read the full abstract ▾
All authors Xintong Zhang, Xiaowen Zhang, Jingrong Wu, Zhi Gao, Shilin Yan, Zhenxin Diao, Kunpeng Gao, Xuanyan Chen, Yuwei Wu, Yunde Jia, Qing Li
Computer Vision · Cross-listed from eess.IV
EchoJEPA: A Latent Predictive Foundation Model for Echocardiography
Alif Munim, Adibvafa Fallahpour, Teodora Szasz et al. (+9 more)
Foundation models for echocardiography often struggle to disentangle anatomical signal from the
stochastic speckle and acquisition artifacts inherent to ultrasound. We present EchoJEPA, a foundation model trained on 18 million echocardiograms across 300K patients, representing the largest pretraining corpus for this modality to date. By leveraging a latent predictive objective, EchoJEPA learns robust anatomical representations that ignore speckle noise. We validate this using a novel multi-view probing framework with frozen backbones, where EchoJEPA outperforms leading baselines by approximately 20% in left ventricular ejection fraction (LVEF) estimation and 17% in right ventricular systolic pressure (RVSP) estimation. The model also exhibits remarkable sample efficiency, reaching 79% view classification accuracy with only 1% of labeled data versus 42% for the best baseline trained on 100%. Crucially, EchoJEPA demonstrates superior generalization, degrading by only 2% under physics-informed acoustic perturbations compared to 17% for competitors. Most remarkably, its zero-shot performance on pediatric patients surpasses fully fine-tuned baselines, establishing latent prediction as a superior paradigm for robust, generalizable medical AI.
Read the full abstract ▾
All authors Alif Munim, Adibvafa Fallahpour, Teodora Szasz, Ahmadreza Attarpour, River Jiang, Brana Sooriyakanthan, Maala Sooriyakanthan, Heather Whitney, Jeremy Slivnick, Barry Rubin, Wendy Tsang, Bo Wang
Computer Vision
PixelGen: Improving Pixel Diffusion with Perceptual Supervision
Zehong Ma, Ruihan Xu and Shiliang Zhang
Pixel
diffusion generates images directly in pixel space, avoiding the VAE artifacts and representational bottlenecks of two-stage latent
diffusion. Recent JiT further simplifies pixel
diffusion with x-prediction, where the model predicts clean images rather than velocity. However, the standard pixel-wise
diffusion loss treats all pixels equally, spending model capacity to perceptually insignificant signals and often leading to blurry samples. We propose PixelGen, an end-to-end pixel
diffusion framework that augments x-prediction with perceptual supervision. Specifically, PixelGen introduces two complementary perceptual losses on top of x-prediction: an LPIPS loss for local textures and a P-DINO loss for global semantics. To preserve sample coverage, PixelGen further proposes a noise-gating strategy that applies these losses only at lower-noise timesteps. On ImageNet-256 without classifier-free guidance, PixelGen achieves an FID of 5.11 in 80 training epochs, surpassing the latent
diffusion baselines. Moreover, PixelGen scales efficiently to text-to-image generation, reaching a GenEval score of 0.79 with only 6 days of training on 8xH800 GPUs. These results show that perceptual supervision substantially narrows the gap between pixel and latent
diffusion while preserving a simple one-stage pipeline. Codes are available at https://github.com/Zehong-Ma/PixelGen.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL System
Yinjie Wang, Tianbao Xie, Ke Shen et al. (+2 more)
We propose RLAnything, a reinforcement learning framework that dynamically forges environment, policy, and reward models through closed-loop optimization, amplifying learning signals and strengthening the overall RL system for any LLM or agentic scenarios. Specifically, the policy is trained with integrated feedback from step-wise and outcome signals, while the reward model is jointly optimized via consistency feedback, which in turn further improves policy training. Moreover, our theory-motivated automatic environment adaptation improves training for both the reward and policy models by leveraging critic feedback from each, enabling learning from experience. Empirically, each added component consistently improves the overall system, and RLAnything yields substantial gains across various representative LLM and agentic tasks, boosting Qwen3-VL-8B-Thinking by 9.1% on OSWorld and Qwen2.5-7B-Instruct by 18.7% and 11.9% on AlfWorld and LiveBench, respectively. We also that optimized reward-model signals outperform outcomes that rely on human labels. Code: https://github.com/Gen-Verse/Open-AgentRL
Read the full abstract ▾
All authors Yinjie Wang, Tianbao Xie, Ke Shen, Mengdi Wang, Ling Yang
Computer Vision
Multi-head automated segmentation by incorporating detection head into the contextual layer neural network
Edwin Kys and Febian Febian
Deep learning based auto
segmentation is increasingly used in radiotherapy, but conventional models often produce anatomically implausible false positives, or
hallucinations, in slices lacking target structures. We propose a gated multi-head
Transformer architecture based on Swin U-Net, augmented with inter-slice context integration and a parallel detection head, which jointly performs slice-level structure detection via a multi-layer perceptron and pixel-level
segmentation through a context-enhanced stream. Detection outputs gate the
segmentation predictions to suppress false positives in anatomically invalid slices, and training uses slice-wise Tversky loss to address class imbalance. Experiments on the Prostate-Anatomical-Edge-Cases
dataset from The Cancer Imaging Archive demonstrate that the gated model substantially outperforms a non-gated
segmentation-only baseline, achieving a mean Dice loss of $0.013 \pm 0.036$ versus $0.732 \pm 0.314$, with detection probabilities strongly correlated with anatomical presence, effectively eliminating spurious
segmentations. In contrast, the non-gated model exhibited higher variability and persistent false positives across all slices. These results indicate that detection-based gating enhances robustness and anatomical plausibility in automated
segmentation applications, reducing hallucinated predictions without compromising
segmentation quality in valid slices, and offers a promising approach for improving the reliability of clinical radiotherapy auto-contouring workflows.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
MentisOculi: Revealing the Limits of Reasoning with Mental Imagery
Jana Zeller, Thaddäus Wiedemer, Fanfei Li et al. (+6 more)
Frontier models are transitioning from
multimodal large
language models (MLLMs) that merely ingest visual information to unified
multimodal models (UMMs) capable of native interleaved generation. This shift has sparked interest in using intermediate visualizations as a
reasoning aid, akin to human mental imagery. Central to this idea is the ability to form, maintain, and manipulate visual representations in a goal-oriented manner. To evaluate and probe this capability, we develop MentisOculi, a procedural, stratified suite of multi-step
reasoning problems amenable to visual solution, tuned to challenge frontier models. Evaluating visual strategies ranging from latent
tokens to explicit generated imagery, we find they generally fail to improve performance. Analysis of UMMs specifically exposes a critical limitation: While they possess the textual
reasoning capacity to solve a task and can sometimes generate correct visuals, they suffer from compounding generation errors and fail to leverage even ground-truth visualizations. Our findings suggest that despite their inherent appeal, visual thoughts do not yet benefit model
reasoning. MentisOculi establishes the necessary foundation to analyze and close this gap across diverse model families.
Read the full abstract ▾
All authors Jana Zeller, Thaddäus Wiedemer, Fanfei Li, Thomas Klein, Prasanna Mayilvahanan, Matthias Bethge, Felix Wichmann, Ryan Cotterell, Wieland Brendel
Computer Vision · Cross-listed from cs.IR
Tyler Skow, Alexander Martin, Benjamin Van Durme et al. (+2 more)
Reranking is a critical component of modern
retrieval systems, which typically pair an efficient first-stage retriever with a more expressive model to refine results. While large
reasoning models have driven rapid progress in text-centric reranking,
reasoning-based reranking for
video retrieval remains underexplored. To address this gap, we introduce RANKVIDEO, a
reasoning-based reranker for
video retrieval that explicitly reasons over query-
video pairs using
video content to assess relevance. RANKVIDEO is trained using a two-stage curriculum consisting of perception-grounded supervised fine-tuning followed by reranking training that combines pointwise, pairwise, and teacher confidence
distillation objectives, and is supported by a data synthesis pipeline for constructing
reasoning-intensive query-
video pairs. Experiments on the large-scale MultiVENT 2.0
benchmark demonstrate that RANKVIDEO consistently improves
retrieval performance within a two-stage framework, yielding an average improvement of 31% on nDCG@10 and outperforming text-only and
vision-
language reranking alternatives, while more efficient.
Read the full abstract ▾
All authors Tyler Skow, Alexander Martin, Benjamin Van Durme, Rama Chellappa, Reno Kriz
Computer Vision
UniReason 1.0: A Unified Reasoning Framework for World Knowledge Aligned Image Generation and Editing
Dianyi Wang, Chaofan Ma, Feng Han et al. (+8 more)
Unified
multimodal models often struggle with complex synthesis tasks that demand deep
reasoning, and typically treat text-to-image generation and image editing as isolated capabilities rather than interconnected
reasoning steps. To address this, we propose UniReason, a unified framework that harmonizes these two tasks through two complementary
reasoning paradigms. We incorporate world knowledge-enhanced textual
reasoning into generation to infer implicit knowledge, and leverage editing capabilities for fine-grained editing-like visual refinement to further correct visual errors via self-reflection. This approach unifies generation and editing within a shared architecture, mirroring the human cognitive process of planning followed by refinement. We support this framework by systematically constructing a large-scale
reasoning-centric
dataset (~300k samples) covering five major knowledge domains (e.g., cultural commonsense, physics, etc.) for textual
reasoning, alongside an
agent-generated corpus for visual refinement. Extensive experiments demonstrate that UniReason achieves advanced performance on
reasoning-intensive
benchmarks such as WISE, KrisBench and UniREditBench, while maintaining superior general synthesis capabilities.
Read the full abstract ▾
All authors Dianyi Wang, Chaofan Ma, Feng Han, Size Wu, Wei Song, Yibin Wang, Zhixiong Zhang, Tianhang Wang, Siyuan Wang, Zhongyu Wei, Jiaqi Wang
Computer Vision
SelvaMask: Segmenting Trees in Tropical Forests and Beyond
Simon-Olivier Duguay, Hugo Baudchon, Etienne Laliberté et al. (+3 more)
Tropical forests harbor most of the
planet's tree biodiversity and are critical to global ecological balance. Canopy trees in particular play a disproportionate role in carbon storage and functioning of these ecosystems. Studying canopy trees at scale requires accurate delineation of individual tree crowns, typically performed using high-resolution aerial imagery. Despite advances in
transformer-based models for individual tree crown
segmentation, performance remains low in most forests, especially tropical ones. To this end, we introduce SelvaMask, a new tropical
dataset containing over 8,800 manually delineated tree crowns across three Neotropical forest sites in Panama, Brazil, and Ecuador. SelvaMask features comprehensive annotations, including an inter-annotator agreement evaluation, capturing the dense structure of tropical forests and highlighting the difficulty of the task. Leveraging this
benchmark, we propose a modular detection-
segmentation pipeline that adapts
vision foundation models (VFMs), using domain-specific detection-prompter. Our approach reaches state-of-the-art performance, outperforming both zero-shot generalist models and fully supervised end-to-end methods in dense tropical forests. We validate these gains on external tropical and temperate
datasets, demonstrating that SelvaMask serves as both a challenging
benchmark and a key enabler for generalized forest monitoring. Our code and
dataset will be released publicly.
Read the full abstract ▾
All authors Simon-Olivier Duguay, Hugo Baudchon, Etienne Laliberté, Helene Muller-Landau, Gonzalo Rivas-Torres, Arthur Ouaknine
Computer Vision
Catalyst: Out-of-Distribution Detection via Elastic Scaling
Abid Hassan, Tuan Ngo, Saad Shafiq et al. (+1 more)
Out-of-distribution (OOD) detection is critical for the safe deployment of deep neural networks. State-of-the-art post-hoc methods typically derive OOD scores from the output logits or penultimate feature vector obtained via global average pooling (GAP). We contend that this exclusive reliance on the logit or feature vector discards a rich, complementary signal: the raw channel-wise statistics of the pre-pooling feature map lost in GAP. In this paper, we introduce Catalyst, a post-hoc framework that exploits these under-explored signals. Catalyst computes an input-dependent scaling factor ($\gamma$) on-the-fly from these raw statistics (e.g., mean, standard deviation, and maximum activation). This $\gamma$ is then fused with the existing baseline score, multiplicatively modulating it – an $\textit{elastic scaling}$ – to push the ID and OOD distributions further apart. We demonstrate Catalyst is a generalizable framework: it seamlessly integrates with logit-based methods (e.g., Energy, ReAct, SCALE) and also provides a significant boost to distance-based detectors like KNN. As a result, Catalyst achieves substantial and consistent performance gains, reducing the average False Positive Rate by 32.87 on CIFAR-10 (ResNet-18), 27.94% on CIFAR-100 (ResNet-18), and 22.25% on ImageNet (ResNet-50). Our results highlight the untapped potential of pre-pooling statistics and demonstrate that Catalyst is complementary to existing OOD detection approaches. Our code is available here: https://github.com/bingabid/Catalyst
Read the full abstract ▾
All authors Abid Hassan, Tuan Ngo, Saad Shafiq, Nenad Medvidovic
Computer Vision
Jiaxing Qiu, Kaihua Hou, Roxana Daneshjou et al. (+2 more)
Model editing aims to correct errors in large, pretrained models without altering unrelated behaviors. While some recent works have edited
vision-
language models (VLMs), no existing editors tackle
reasoning-heavy tasks, which typically require humans and models to reason about images. We therefore propose ReasonEdit, the first VLM editor to let users explain their
reasoning during editing, introducing a new, practical model editing setup. ReasonEdit continuously stores human
reasoning in a codebook, and retrieves only relevant facts during inference using a novel
topology-balanced
multimodal embedding method inspired by network science. Across four VLMs on multiple rationale-based visual question answering
datasets, ReasonEdit achieves state-of-the-art editing performance, ultimately showing that using human
reasoning during editing greatly improves edit generalization.
Read the full abstract ▾
All authors Jiaxing Qiu, Kaihua Hou, Roxana Daneshjou, Ahmed Alaa, Thomas Hartvigsen
Computer Vision · Cross-listed from cs.RO
SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-body Manipulation
Mu Huang, Hui Wang, Kerui Ren et al. (+5 more)
Simulating deformable objects under rich interactions remains a fundamental challenge for real-to-sim
robot manipulation, with dynamics jointly driven by environmental effects and
robot actions. Existing simulators rely on predefined physics or data-driven dynamics without
robot-conditioned control, limiting accuracy, stability, and generalization. This paper presents SoMA, a 3D Gaussian Splat simulator for soft-body manipulation. SoMA couples deformable dynamics, environmental forces, and
robot joint actions in a unified latent neural space for end-to-end real-to-sim simulation. Modeling interactions over learned Gaussian splats enables controllable, stable long-horizon manipulation and generalization beyond observed trajectories without predefined physical models. SoMA improves resimulation accuracy and generalization on real-world
robot manipulation by 20%, enabling stable simulation of complex tasks such as long-horizon cloth folding.
Read the full abstract ▾
All authors Mu Huang, Hui Wang, Kerui Ren, Linning Xu, Yunsong Zhou, Mulin Yu, Bo Dai, Jiangmiao Pang
Computer Vision
Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation
Xinshun Wang, Peiming Li, Ziyi Wang et al. (+5 more)
Human motion analysis tasks, such as temporal 3D pose estimation, motion prediction, and motion in-betweening, play an essential role in computer
vision. However, current paradigms suffer from severe fragmentation. First, the field is split between “perception” models that understand motion from
video but only output text, and “generation” models that cannot perceive from raw visual input. Second, generative MLLMs are often limited to single-frame, static poses using dense, parametric SMPL models, failing to handle temporal motion. Third, existing motion vocabularies are built from skeleton data alone, severing the link to the visual domain. To address these challenges, we introduce Superman, a unified framework that bridges visual perception with temporal, skeleton-based motion generation. Our solution is twofold. First, to overcome the modality disconnect, we propose a
Vision-Guided Motion Tokenizer. Leveraging the natural
geometric alignment between 3D skeletons and visual data, this module pioneers robust joint learning from both modalities, creating a unified, cross-modal motion vocabulary. Second, grounded in this motion
language, a single, unified MLLM architecture is trained to handle all tasks. This module flexibly processes diverse, temporal inputs, unifying 3D skeleton pose estimation from
video (perception) with skeleton-based motion prediction and in-betweening (generation). Extensive experiments on standard
benchmarks, including Human3.6M, demonstrate that our unified method achieves state-of-the-art or competitive performance across all motion tasks. This showcases a more efficient and scalable path for generative motion analysis using skeletons.
Read the full abstract ▾
All authors Xinshun Wang, Peiming Li, Ziyi Wang, Zhongbin Fang, Zhichao Deng, Songtao Wu, Jason Li, Mengyuan Liu
Computer Vision
Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory
Ruiqi Wu, Xuanhua He, Meng Cheng et al. (+8 more)
We propose Infinite-World, a robust interactive world model capable of maintaining coherent visual
memory over 1000+ frames in complex real-world environments. While existing world models can be efficiently optimized on synthetic data with perfect ground-truth, they lack an effective training paradigm for real-world
videos due to noisy pose estimations and the scarcity of viewpoint revisits. To bridge this gap, we first introduce a Hierarchical Pose-free
Memory Compressor (HPMC) that recursively distills historical latents into a fixed-budget representation. By jointly optimizing the compressor with the generative backbone, HPMC enables the model to autonomously anchor generations in the distant past with bounded computational cost, eliminating the need for explicit
geometric priors. Second, we propose an Uncertainty-aware Action Labeling module that discretizes continuous motion into a tri-state logic. This strategy maximizes the utilization of raw
video data while shielding the deterministic action space from being corrupted by noisy trajectories, ensuring robust action-response learning. Furthermore, guided by insights from a pilot toy study, we employ a Revisit-Dense Finetuning Strategy using a compact, 30-minute
dataset to efficiently activate the model's long-range loop-closure capabilities. Extensive experiments, including objective metrics and user studies, demonstrate that Infinite-World achieves superior performance in visual quality, action controllability, and spatial consistency.
Read the full abstract ▾
All authors Ruiqi Wu, Xuanhua He, Meng Cheng, Tianyu Yang, Yong Zhang, Zhuoliang Kang, Xunliang Cai, Xiaoming Wei, Chunle Guo, Chongyi Li, Ming-Ming Cheng
Computer Vision
Personalized Image Generation via Human-in-the-loop Bayesian Optimization
Rajalaxmi Rajagopalan, Debottam Dutta, Yu-Lin Wei et al. (+1 more)
Imagine Alice has a specific image $x^\ast$ in her mind, say, the view of the street in which she grew up during her childhood. To generate that exact image, she guides a generative model with multiple rounds of prompting and arrives at an image $x^{p*}$. Although $x^{p*}$ is reasonably close to $x^\ast$, Alice finds it difficult to close that gap using
language prompts. This paper aims to narrow this gap by observing that even after
language has reached its limits, humans can still tell when a new image $x^+$ is closer to $x^\ast$ than $x^{p*}$. Leveraging this observation, we develop MultiBO (Multi-Choice Preferential
Bayesian Optimization) that carefully generates $K$ new images as a function of $x^{p*}$, gets preferential feedback from the user, uses the feedback to guide the
diffusion model, and ultimately generates a new set of $K$ images. We show that within $B$ rounds of user feedback, it is possible to arrive much closer to $x^\ast$, even though the generative model has no information about $x^\ast$. Qualitative scores from $30$ users, combined with quantitative metrics compared across $5$ baselines, show promising results, suggesting that multi-choice feedback from humans can be effectively harnessed for personalized image generation.
Read the full abstract ▾
All authors Rajalaxmi Rajagopalan, Debottam Dutta, Yu-Lin Wei, Romit Roy Choudhury
Computer Vision
Unified Personalized Reward Model for Vision Generation
Yibin Wang, Yuhang Zang, Feng Han et al. (+4 more)
Recent advancements in
multimodal reward models (RMs) have significantly propelled the development of visual generation. Existing frameworks typically adopt Bradley-Terry-style preference modeling or leverage generative VLMs as judges, and subsequently optimize visual generation models via reinforcement learning. However, current RMs suffer from inherent limitations: they often follow a one-size-fits-all paradigm that assumes a monolithic preference distribution or relies on fixed evaluation rubrics. As a result, they are insensitive to content-specific visual cues, leading to systematic misalignment with subjective and context-dependent human preferences. To this end, inspired by human assessment, we propose UnifiedReward-Flex, a unified personalized reward model for
vision generation that couples reward modeling with flexible and context-adaptive
reasoning. Specifically, given a
prompt and the generated visual content, it first interprets the semantic intent and grounds on visual evidence, then dynamically constructs a hierarchical assessment by instantiating fine-grained criteria under both predefined and self-generated high-level dimensions. Our training pipeline follows a two-stage process: (1) we first distill structured, high-quality
reasoning traces from advanced closed-source VLMs to bootstrap SFT, equipping the model with flexible and context-adaptive
reasoning behaviors; (2) we then perform direct preference optimization (DPO) on carefully curated preference pairs to further strengthen
reasoning fidelity and discriminative
alignment. To validate the effectiveness, we integrate UnifiedReward-Flex into the GRPO framework for image and
video synthesis, and extensive results demonstrate its superiority.
Read the full abstract ▾
All authors Yibin Wang, Yuhang Zang, Feng Han, Jiazi Bu, Yujie Zhou, Cheng Jin, Jiaqi Wang
Computer Vision
Uncertainty-Aware Image Classification In Biomedical Imaging Using Spectral-normalized Neural Gaussian Processes
Uma Meleti and Jeffrey J. Nirschl
Accurate histopathologic interpretation is key for clinical decision-making; however, current deep learning models for digital pathology are often overconfident and poorly calibrated in out-of-distribution (OOD) settings, which limit trust and clinical adoption. Safety-critical medical imaging workflows benefit from intrinsic uncertainty-aware properties that can accurately reject OOD input. We implement the Spectral-normalized Neural Gaussian Process (SNGP), a set of lightweight modifications that apply spectral normalization and replace the final dense layer with a Gaussian process layer to improve single-model uncertainty estimation and OOD detection. We evaluate SNGP vs. deterministic and MonteCarlo dropout on six
datasets across three biomedical classification tasks: white blood
cells, amyloid plaques, and colorectal histopathology. SNGP has comparable in-distribution performance while significantly improving uncertainty estimation and OOD detection. Thus, SNGP or related models offer a useful framework for uncertainty-aware classification in digital pathology, supporting safe deployment and building trust with pathologists.
Read the full abstract ▾
Computer Vision
NAB: Neural Adaptive Binning for Sparse-View CT reconstruction
Wangduo Xie and Matthew B. Blaschko
Computed
Tomography (CT) plays a vital role in inspecting the internal structures of industrial objects. Furthermore, achieving high-quality CT reconstruction from sparse views is essential for reducing production costs. While classic implicit neural networks have shown promising results for sparse reconstruction, they are unable to leverage shape priors of objects. Motivated by the observation that numerous industrial objects exhibit rectangular structures, we propose a novel Neural Adaptive Binning (NAB) method that effectively integrates rectangular priors into the reconstruction process. Specifically, our approach first maps coordinate space into a binned vector space. This mapping relies on an innovative binning mechanism based on differences between shifted
hyperbolic tangent functions, with our extension enabling rotations around the input-plane normal vector. The resulting representations are then processed by a neural network to predict CT attenuation coefficients. This design enables end-to-end optimization of the encoding parameters – including position, size, steepness, and rotation – via gradient flow from the projection data, thus enhancing reconstruction accuracy. By adjusting the smoothness of the binning function, NAB can generalize to objects with more complex geometries. This research provides a new perspective on integrating shape priors into neural network-based reconstruction. Extensive experiments demonstrate that NAB achieves superior performance on two industrial
datasets. It also maintains robust on medical
datasets when the binning function is extended to more general expression. The code is available at https://github.com/Wangduo-Xie/NAB_CT_reconstruction.
Read the full abstract ▾
Computer Vision
Implicit neural representation of textures
Albert Kwok, Zheyuan Hu and Dounia Hammou
Implicit neural representation (INR) has proven to be accurate and efficient in various domains. In this work, we explore how different neural networks can be designed as a new texture INR, which operates in a continuous manner rather than a discrete one over the input UV coordinate space. Through thorough experiments, we demonstrate that these INRs perform well in terms of image quality, with considerable
memory usage and rendering inference time. We analyze the balance between these objectives. In addition, we investigate various related applications in real-time rendering and down-stream tasks, e.g. mipmap fitting and INR-space generation.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics
Ziwen Xu, Chenyan Wu, Hengyu Sun et al. (+9 more)
Methods for controlling large
language models (LLMs), including local weight fine-tuning, LoRA-based adaptation, and activation-based interventions, are often studied in isolation, obscuring their connections and making comparison difficult. In this work, we present a unified view that frames these interventions as dynamic weight updates induced by a control signal, placing them within a single conceptual framework. Building on this view, we propose a unified preference-utility analysis that separates control effects into preference, defined as the tendency toward a target concept, and utility, defined as coherent and task-valid generation, and measures both on a shared log-odds scale using polarity-paired contrastive examples. Across methods, we observe a consistent trade-off between preference and utility: stronger control increases preference while predictably reducing utility. We further explain this behavior through an activation
manifold perspective, in which control shifts representations along target-concept directions to enhance preference, while utility declines primarily when interventions push representations off the model's valid-generation
manifold. Finally, we introduce a new steering approach SPLIT guided by this analysis that improves preference while better preserving utility. Code is available at https://github.com/zjunlp/EasyEdit/blob/main/examples/SPLIT.md.
Read the full abstract ▾
All authors Ziwen Xu, Chenyan Wu, Hengyu Sun, Haiwen Hong, Mengru Wang, Yunzhi Yao, Longtao Huang, Hui Xue, Shumin Deng, Zhixuan Chu, Huajun Chen, Ningyu Zhang
Computer Vision
LongVPO: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization
Zhenpeng Huang, Jiaqi Li, Zihan Jia et al. (+6 more)
We present LongVPO, a novel two-stage Direct Preference Optimization framework that enables short-context
vision-
language models to robustly understand ultra-long
videos without any long-
video annotations. In Stage 1, we synthesize preference triples by anchoring questions to individual short clips, interleaving them with distractors, and applying visual-similarity and question-specificity filtering to mitigate positional bias and ensure unambiguous supervision. We also approximate the reference model's scoring over long contexts by evaluating only the anchor clip, reducing computational overhead. In Stage 2, we employ a recursive captioning pipeline on long
videos to generate scene-level metadata, then use a large
language model to craft multi-segment
reasoning queries and dispreferred responses, aligning the model's preferences through multi-segment
reasoning tasks. With only 16K synthetic examples and no costly human labels, LongVPO outperforms the state-of-the-art open-source models on multiple long-
video benchmarks, while maintaining strong short-
video performance (e.g., on MVBench), offering a scalable paradigm for efficient long-form
video understanding.
Read the full abstract ▾
All authors Zhenpeng Huang, Jiaqi Li, Zihan Jia, Xinhao Li, Desen Meng, Lingxue Song, Xi Chen, Liang Li, Limin Wang
Computer Vision
VQ-Style: Disentangling Style and Content in Motion with Residual Quantized Representations
Fatemeh Zargarbashi, Dhruv Agrawal, Jakob Buhmann et al. (+3 more)
Human motion data is inherently rich and complex, containing both semantic content and subtle stylistic features that are challenging to model. We propose a novel method for effective disentanglement of the style and content in human motion data to facilitate style transfer. Our approach is guided by the insight that content corresponds to coarse motion attributes while style captures the finer, expressive details. To model this hierarchy, we employ Residual Vector Quantized Variational Autoencoders (RVQ-VAEs) to learn a coarse-to-fine representation of motion. We further enhance the disentanglement by integrating codebook learning with contrastive learning and a novel information leakage loss to organize the content and the style across different codebooks. We harness this disentangled representation using our simple and effective inference-time technique Quantized Code Swapping, which enables motion style transfer without requiring any fine-tuning for unseen styles. Our framework demonstrates strong versatility across multiple inference applications, including style transfer, style removal, and motion blending.
Read the full abstract ▾
All authors Fatemeh Zargarbashi, Dhruv Agrawal, Jakob Buhmann, Martin Guay, Stelian Coros, Robert W. Sumner
Computer Vision
Enhancing Indoor Occupancy Prediction via Sparse Query-Based Multi-Level Consistent Knowledge Distillation
Xiang Li, Yupeng Zheng, Pengfei Li et al. (+3 more)
Occupancy prediction provides critical
geometric and semantic understanding for robotics but faces efficiency-accuracy trade-offs. Current dense methods suffer computational waste on empty voxels, while sparse query-based approaches lack robustness in diverse and complex indoor scenes. In this paper, we propose DiScene, a novel sparse query-based framework that leverages multi-level
distillation to achieve efficient and robust occupancy prediction. In particular, our method incorporates two key innovations: (1) a Multi-level Consistent Knowledge
Distillation strategy, which transfers hierarchical representations from large teacher models to lightweight students through coordinated
alignment across four levels, including encoder-level feature
alignment, query-level feature matching, prior-level spatial guidance, and anchor-level high-confidence knowledge transfer and (2) a Teacher-Guided Initialization policy, employing optimized parameter warm-up to accelerate model convergence. Validated on the Occ-Scannet
benchmark, DiScene achieves 23.2 FPS without depth priors while outperforming our baseline method, OPUS, by 36.1% and even better than the depth-enhanced version, OPUS{\dag}. With depth integration, DiScene{\dag} attains new SOTA performance, surpassing EmbodiedOcc by 3.7% with 1.62$\times$ faster inference speed. Furthermore, experiments on the Occ3D-nuScenes
benchmark and in-the-wild scenarios demonstrate the versatility of our approach in various environments. Code and models can be accessed at https://github.com/getterupper/DiScene.
Read the full abstract ▾
All authors Xiang Li, Yupeng Zheng, Pengfei Li, Yilun Chen, Ya-Qin Zhang, Wenchao Ding
Computer Vision · Cross-listed from cs.LG
Segment to Focus: Guiding Latent Action Models in the Presence of Distractors
Marcus Fechner, Hamza Adnan, Constantin C. Lüth et al. (+3 more)
Latent action models (LAMs) offer a promising path to pre-training
embodied agents on large amounts of action-free
video. They infer latent actions between consecutive observations that can later be decoded to ground-truth actions using a small number of labels. However, recent work has shown that this recipe fails in the presence of action-correlated visual distractors common in real-world
video, such as dynamic backgrounds, camera shake, or other moving objects. In these scenarios, the standard reconstruction objective drives latent actions to encode exogenous motion instead of
agent-controlled dynamics, resulting in policies that underperform when fine-tuned. We observe, however, that endogenous and exogenous factors are typically spatially separated in pixel space: control-relevant change is concentrated on the
agent, while distractor motion occurs elsewhere. We exploit this observation by restricting the reconstruction objective to
agent pixels, forcing latent actions to explain
agent-controlled dynamics rather than exogenous ones. We call this method MaskLAM; it obtains the
agent mask zero-shot from off-the-shelf
segmentation foundation models (e.g., SAM) and requires no architectural changes, auxiliary losses, or action labels during pre-training. Across two continuous-control
benchmarks (Distracting Control Suite, Distracting Meta-World), MaskLAM reduces normalized linear-probe MSE by up to $3.51\times$ and improves normalized return by up to $4.97\times$ over LAPO, while narrowing the gap to LAOM-Labels, which relies on ground-truth action supervision.
Read the full abstract ▾
All authors Marcus Fechner, Hamza Adnan, Constantin C. Lüth, Matthew T. Jackson, Alexey Zakharov, J. Marius Zöllner
Computer Vision
LiFlow: Flow Matching for 3D LiDAR Scene Completion
Andrea Matteazzi and Dietmar Tutsch
In autonomous driving scenarios, the collected
LiDAR point clouds can be challenged by occlusion and long-range sparsity, limiting the perception of autonomous driving systems. Scene completion methods can infer the missing parts of incomplete 3D
LiDAR scenes. Recent methods adopt local point-level denoising
diffusion probabilistic models, which require predicting Gaussian noise, leading to a mismatch between training and inference initial distributions. This paper introduces the first flow matching framework for 3D
LiDAR scene completion, improving upon
diffusion-based methods by ensuring consistent initial distributions between training and inference. The model employs a nearest neighbor flow matching loss and a Chamfer distance loss to enhance both local structure and global coverage in the
alignment of point clouds. LiFlow achieves state-of-the-art performance across multiple metrics. Code: https://github.com/matteandre/LiFlow.
Read the full abstract ▾
Computer Vision
Show, Don't Tell: Morphing Latent Reasoning into Image Generation
Harold Haodong Chen, Xinxiang Yin, Wen-Jie Shu et al. (+6 more)
Text-to-image (T2I) generation has achieved remarkable progress, yet existing methods often lack the ability to dynamically reason and refine during generation–a hallmark of human creativity. Current
reasoning-augmented paradigms most rely on explicit thought processes, where intermediate
reasoning is decoded into discrete text at fixed steps with frequent image decoding and re-encoding, leading to inefficiencies, information loss, and cognitive mismatches. To bridge this gap, we introduce LatentMorph, a novel framework that seamlessly integrates implicit latent
reasoning into the T2I generation process. At its core, LatentMorph introduces four lightweight components: (i) a condenser for summarizing intermediate generation states into compact visual
memory, (ii) a translator for converting latent thoughts into actionable guidance, (iii) a shaper for dynamically steering next image
token predictions, and (iv) an RL-trained invoker for adaptively determining when to invoke
reasoning. By performing
reasoning entirely in continuous latent spaces, LatentMorph avoids the bottlenecks of explicit
reasoning and enables more adaptive self-refinement. Extensive experiments demonstrate that LatentMorph (I) enhances the base model Janus-Pro by $16\%$ on GenEval and $25\%$ on T2I-CompBench; (II) outperforms explicit paradigms (e.g., TwiG) by $15\%$ and $11\%$ on abstract
reasoning tasks like WISE and IPV-Txt, (III) while reducing inference time by $44\%$ and
token consumption by $51\%$; and (IV) exhibits $71\%$ cognitive
alignment with human intuition on
reasoning invocation.
Read the full abstract ▾
All authors Harold Haodong Chen, Xinxiang Yin, Wen-Jie Shu, Hongfei Zhang, Zixin Zhang, Chenfei Liao, Litao Guo, Qifeng Chen, Ying-Cong Chen
Computer Vision
Evaluating OCR Performance for Assistive Technology: Effects of Walking Speed, Camera Placement, and Camera Type
Junchi Feng, Nikhil Ballem, Mahya Beheshti et al. (+5 more)
Optical character recognition (OCR), a process that converts printed or handwritten text into machine-readable form, is widely used in assistive technology for people with blindness and low
vision. Yet most evaluations rely on static
datasets that do not reflect the challenges of mobile use. In this study, we systematically evaluated OCR performance under both static and dynamic conditions. Static tests measured detection range across distances of 1-7 meters and viewing angles of 0-75 degrees horizontally. Dynamic tests examined the impact of motion by varying walking speed from slow (0.8 m/s) to very fast (1.8 m/s) and compared three camera mounting positions: head-mounted, shoulder-mounted, and handheld. We evaluated both a smartphone and smart glasses, using the phone's main and ultra-wide cameras. Four OCR engines were benchmarked to assess accuracy at different distances and viewing angles: Google
Vision, PaddleOCR 3.0, EasyOCR, and Tesseract. PaddleOCR 3.0 was then used to evaluate OCR performance under dynamic walking conditions. Accuracy was computed at the character-level using the Levenshtein ratio against manually defined ground truth. Results showed that recognition accuracy declined with increased walking speed and wider viewing angles. Google
Vision achieved the highest overall accuracy, with PaddleOCR close behind as the strongest open-source alternative. Across devices, the phone's main camera achieved the highest accuracy, and a shoulder-mounted placement yielded the highest average among body positions; however, differences among shoulder, head, and hand were not statistically significant.
Read the full abstract ▾
All authors Junchi Feng, Nikhil Ballem, Mahya Beheshti, Giles Hamilton-Fletcher, Todd Hudson, Maurizio Porfiri, William H. Seiple, John-Ross Rizzo
Computer Vision
MIRROR: Manifold Ideal Reference ReconstructOR for Generalizable AI-Generated Image Detection
Ruiqi Liu, Manni Cui, Ziheng Qin et al. (+12 more)
High-fidelity generative models have narrowed the perceptual gap between synthetic and real images, posing serious threats to media security. Most existing AI-generated image (AIGI) detectors rely on artifact-based classification and struggle to generalize to evolving generative traces. In contrast, human judgment relies on stable real-world regularities, with deviations from the human cognitive
manifold serving as a more generalizable signal of forgery. Motivated by this insight, we reformulate AIGI detection as a Reference-Comparison problem that verifies consistency with the real-image
manifold rather than fitting specific forgery cues. We propose MIRROR (
Manifold Ideal Reference ReconstructOR), a framework that explicitly encodes reality priors using a learnable discrete
memory bank. MIRROR projects an input into a
manifold-consistent ideal reference via sparse linear combination, and uses the resulting residuals as robust detection signals. To evaluate whether detectors reach the "superhuman crossover" required to replace human experts, we introduce the Human-AIGI
benchmark, featuring a psychophysically curated human-imperceptible subset. Across 14
benchmarks, MIRROR consistently outperforms prior methods, achieving gains of 2.1% on six standard
benchmarks and 8.1% on seven in-the-wild
benchmarks. On Human-AIGI, MIRROR reaches 89.6% accuracy across 27 generators, surpassing both lay users and visual experts, and further approaching the human perceptual limit as pretrained backbones scale. The code is publicly available at: https://github.com/349793927/MIRROR
Read the full abstract ▾
All authors Ruiqi Liu, Manni Cui, Ziheng Qin, Zhiyuan Yan, Ruoxin Chen, Yi Han, Zhiheng Li, Junkai Chen, ZhiJin Chen, Kaiqing Lin, Jialiang Shen, Lubin Weng, Jing Dong, Yan Wang, Shu Wu