Computer Vision · Cross-listed from cs.MM
Omni-C: Compressing Heterogeneous Modalities into a Single Dense Encoder
Kin Wai Lau, Yasar Abbas Ur Rehman, Lai-Man Po et al. (+1 more)
Recent
multimodal systems often rely on separate expert modality encoders which cause linearly scaling complexity and computational overhead with added modalities. While unified Omni-models address this via Mixture-of-Expert (MoE) architectures with specialized experts and routing, they still inflate parameter counts and introduce routing overhead. In this paper, we propose Omni-C (Omni-Compress), a single dense
Transformer-based encoder that learns competitive shared representations across heterogeneous modalities–images, audio, and text–through unimodal contrastive pretraining on large-scale unaligned data. By maximizing parameter sharing in the backbone and using lightweight modality-specific projection heads, Omni-C effectively mitigates inter-modality conflicts without requiring MoE, paired supervision, or routing. This design supports efficient deployment on
memory-constrained systems via sequential modality processing and low-
memory inference, eliminating the need for parallel expert loading or specialized hardware. Experiments show Omni-C achieves performance comparable to expert models in unimodal and cross-model tasks, with modest zero-shot degradation on audio and text that is largely recovered through lightweight linear probing or parameter efficient fine-tuning. The unified architecture substantially reduces inference
memory usage compared to multi-encoder baselines, advancing efficient and scalable
multimodal learning.
Read the full abstract ▾
All authors Kin Wai Lau, Yasar Abbas Ur Rehman, Lai-Man Po, Pedro Porto Buarque de Gusmão
Computer Vision
CamDirector: Towards Long-Term Coherent Video Trajectory Editing
Zhihao Shi, Kejia Yin, Weilin Wan et al. (+5 more)
Video (camera) trajectory editing aims to synthesize new
videos that follow user-defined camera paths while preserving scene content and plausibly inpainting previously unseen regions, upgrading amateur footage into professionally styled
videos. Existing VTE methods struggle with precise camera control and long-range consistency because they either inject target poses through a limited-capacity
embedding or rely on single-frame warping with only implicit cross-frame aggregation in
video diffusion models. To address these issues, we introduce a new VTE framework that 1) explicitly aggregates information across the entire source
video via a hybrid warping scheme. Specifically, static regions are progressively fused into a world cache then rendered to target camera poses, while dynamic regions are directly warped; their fusion yields globally consistent coarse frames that guide refinement. 2) processes
video segments jointly with their history via a history-guided autoregressive
diffusion model, while the world cache is incrementally updated to reinforce already inpainted content, enabling long-term temporal coherence. Finally, we present iPhone-PTZ, a new VTE
benchmark with diverse camera motions and large trajectory variations, and achieve state-of-the-art performance with fewer parameters.
Read the full abstract ▾
All authors Zhihao Shi, Kejia Yin, Weilin Wan, Yuhongze Zhou, Yuanhao Yu, Xinxin Zuo, Qiang Sun, Juwei Lu
Computer Vision
DiffSOS: Acoustic Conditional Diffusion Model for Speed-of-Sound Reconstruction in Ultrasound Computed Tomography
Yujia Wu, Shuoqi Chen, Shiru Wang et al. (+3 more)
Accurate Speed-of-Sound (SoS) reconstruction from acoustic waveforms is a cornerstone of ultrasound computed
tomography (USCT), enabling quantitative velocity mapping that reveals subtle anatomical details and pathological variations often invisible in conventional imaging. However, practical utility is hindered by the limitations of existing
algorithms; traditional Full Waveform Inversion (FWI) is computationally intensive, while current deep learning approaches tend to produce oversmoothed results lacking fine details. We propose DiffSOS, a conditional
diffusion model that directly maps acoustic waveforms to SoS maps. Our framework employs a specialized acoustic ControlNet to strictly ground the denoising process in physical wave measurements. To ensure structural consistency, we optimize a hybrid loss function that integrates noise prediction, spatial reconstruction, and noise frequency content. To accelerate inference, we employ
stochastic Denoising
Diffusion Implicit Model (DDIM) sampling, achieving near real-time reconstruction with only 10 steps. Crucially, we exploit the
stochastic generative nature of our framework to estimate pixel-wise uncertainty, providing a measure of reliability that is often absent in deterministic approaches. Evaluated on the OpenPros USCT
benchmark, DiffSOS significantly outperforms state-of-the-art networks, achieving an average Multi-scale Structural Similarity of 0.957. Our approach provides high-fidelity SoS maps with a principled measure of confidence, facilitating safer and faster clinical interpretation.
Read the full abstract ▾
All authors Yujia Wu, Shuoqi Chen, Shiru Wang, Yucheng Tang, Petr Bruza, Geoffrey P. Luke
Computer Vision
Unsupervised Semantic Segmentation in Synchrotron Computed Tomography with Self-Correcting Pseudo Labels
Austin Yunker, Peter Kenesei, Hemant Sharma et al. (+3 more)
X-ray computed
tomography (CT) is a widely used imaging technique that provides detailed examinations into the internal structure of an object with synchrotron CT (SR-CT) enabling improved data quality by using higher energy, monochromatic X-rays. While SR-CT allows for improved resolution, time-resolved experimentation, and reduced imaging artifacts, it also produces significantly larger
datasets than conventional CT. Accurate and efficient evaluation of these
datasets is a critical component of these workflows; yet is often done manually representing a major bottleneck in the analysis phase. While deep learning has emerged as a powerful tool capable of providing a wide range of purely data-driven solutions, it requires a substantial amount of labeled data for training and manual annotation of SR-CT
datasets is impractical in practice. In this paper, we introduce a novel framework that enables automatic
segmentation of large, high-resolution SR-CT
datasets by eliminating the need to hand label images for deep learning training. First, we generate pseudo labels by clustering on the voxel values identifying regions in the volume with similar attenuation coefficients producing an initial semantic map. Afterwards, we train a
segmentation model on the pseudo labels before utilizing the Unbiased Teacher approach to self-correct them ensuring accurate final
segmentations. We find our approach improves pixel-wise accuracy and mIoU by 13.31% and 15.94%, respectively, over the baseline pseudo labels when using a magnesium
crystal SR-CT sample. Additionally, we extensively evaluate the different components of our workflow including
segmentation model, loss function, pseudo labeling strategy, and input type. Finally, we evaluate our approach on to two additional samples highlighting our frameworks ability to produce
segmentations that are considerably better than the original pseudo labels.
Read the full abstract ▾
All authors Austin Yunker, Peter Kenesei, Hemant Sharma, Jun-Sang Park, Antonino Miceli, Rajkumar Kettimuthu
Computer Vision · Cross-listed from cs.LG
Deep Learning-Based Meat Freshness Detection with Segmentation and OOD-Aware Classification
Hutama Arif Bramantyo, Mukarram Ali Faridi, Rui Chen et al. (+2 more)
In this study, we present a meat freshness classification framework from Red-Green-Blue (RGB) images that supports both packaged and unpackaged meat
datasets. The system classifies four in-distribution (ID) meat classes and uses an out-of-distribution (OOD)-aware abstention mechanism that flags low-confidence samples as No Result. The pipeline combines U-Net-based
segmentation with deep feature classifiers.
Segmentation is used as a preprocessing step to isolate the meat region and reduce background, producing more consistent inputs for classification. The
segmentation module achieved an Intersection over Union (IoU) of 75% and a Dice coefficient of 82%, producing standardized inputs for the classification stage. For classification, we
benchmark five backbones: Residual Network-50 (ResNet-50),
Vision Transformer-Base/16 (ViT-B/16), Swin
Transformer-Tiny (Swin-T), EfficientNet-B0, and MobileNetV3-Small. We use nested 5x3 cross-validation (CV) for model selection and hyperparameter tuning. On the held-out ID test set, EfficientNet-B0 achieves the highest accuracy (98.10%), followed by ResNet-50 and MobileNetV3-Small (both 97.63%) and Swin-T (97.51%), while ViT-B/16 is lower (94.42%). We additionally evaluate OOD scoring and thresholding using standard OOD metrics and sensitivity analysis over the abstention threshold. Finally, we report on-device latency using TensorFlow Lite (TFLite) on a smartphone, highlighting practical accuracy-latency trade-offs for future deployment.
Read the full abstract ▾
All authors Hutama Arif Bramantyo, Mukarram Ali Faridi, Rui Chen, Clarissa Harris, Yin Sun
Computer Vision
Percept-Aware Surgical Planning for Visual Cortical Prostheses with Vascular Avoidance
Galen Pogoncheff, Alvin Wang, Jacob Granley et al. (+1 more)
Cortical visual prostheses aim to restore sight by electrically stimulating
neurons in early visual cortex (V1). With the emergence of high-density and flexible neural interfaces, electrode placement within three-dimensional cortex has become a critical
surgical planning problem. Existing strategies emphasize visual field coverage and anatomical heuristics but do not directly optimize predicted perceptual outcomes under safety constraints. We present a percept-aware framework for
surgical planning of cortical visual prostheses that formulates electrode placement as a constrained optimization problem in anatomical space. Electrode coordinates are treated as learnable parameters and optimized end-to-end using a differentiable forward model of prosthetic
vision. The objective minimizes task-level perceptual error while incorporating vascular avoidance and gray matter feasibility constraints. Evaluated on simulated reading and natural image tasks using realistic folded cortical geometry (FreeSurfer fsaverage), percept-aware optimization consistently improves reconstruction fidelity relative to coverage-based placement strategies. Importantly, vascular safety constraints eliminate margin violations while preserving perceptual performance. The framework further enables co-optimization of multi-electrode thread configurations under fixed insertion budgets. These results demonstrate how differentiable percept models can inform anatomically grounded, safety-aware computer-assisted planning for cortical neural interfaces and provide a foundation for optimizing next-generation visual prostheses.
Read the full abstract ▾
All authors Galen Pogoncheff, Alvin Wang, Jacob Granley, Michael Beyeler
Computer Vision
Diffusion-Based Low-Light Image Enhancement with Color and Luminance Priors
Xuanshuo Fu, Lei Kang and Javier Vazquez-Corral
Low-light images often suffer from low contrast, noise, and color distortion, degrading visual quality and impairing downstream
vision tasks. We propose a novel conditional
diffusion framework for low-light image enhancement that incorporates a Structured Control
Embedding Module (SCEM). SCEM decomposes a low-light image into four informative components including illumination, illumination-invariant features, shadow priors, and color-invariant cues. These components serve as control signals that condition a U-Net-based
diffusion model trained with a simplified noise-prediction loss. Thus, the proposed SCEM equipped
Diffusion method enforces structured enhancement guided by physical priors. In experiments, our model is trained only on the LOLv1
dataset and evaluated without fine-tuning on LOLv2-real, LSRW, DICM, MEF, and LIME. The method achieves state-of-the-art performance in quantitative and perceptual metrics, demonstrating strong generalization across
benchmarks. https://casted.github.io/scem/.
Read the full abstract ▾
Computer Vision
Proof-of-Perception: Certified Tool-Using Multimodal Reasoning with Compositional Conformal Guarantees
Arya Fayyazi and Haleh Akrami
We present Proof-of-Perception (PoP), a tool-using framework that casts
multimodal reasoning as an executable graph with explicit reliability guarantees. Each perception or logic node outputs a conformal set, yielding calibrated, stepwise uncertainty; a lightweight controller uses these certificates to allocate compute under a budget, expanding with extra tool calls only when needed and stopping early otherwise. This grounds answers in verifiable evidence, reduces error compounding and
hallucinations, and enables principled accuracy-compute trade-offs. Across document, chart, and multi-image QA
benchmarks, PoP improves performance and reliability over strong chain-of-thought, ReAct-style, and program-of-thought baselines while using computation more efficiently.
Read the full abstract ▾
Computer Vision
Seeking Necessary and Sufficient Information from Multimodal Medical Data
Boyu Chen, Weiye Bao, Junjie Liu et al. (+5 more)
Learning
multimodal representations from medical images and other data sources can provide richer information for decision-making. While various
multimodal models have been developed for this, they overlook learning features that are both necessary (must be present for the outcome to occur) and sufficient (enough to determine the outcome). We argue learning such features is crucial as they can improve model performance by capturing essential predictive information, and enhance model robustness to missing modalities as each modality can provide adequate predictive signals. Such features can be learned by leveraging the Probability of Necessity and Sufficiency (PNS) as a learning objective, an approach that has proven effective in unimodal settings. However, extending PNS to
multimodal scenarios remains underexplored and is non-trivial as key conditions of PNS estimation are violated. We address this by decomposing
multimodal representations into modality-invariant and modality-specific components, then deriving tractable PNS objectives for each. Experiments on synthetic and real-world medical
datasets demonstrate our method's effectiveness. Code will be available on GitHub.
Read the full abstract ▾
All authors Boyu Chen, Weiye Bao, Junjie Liu, Michael Shen, Bo Peng, Paul Taylor, Zhu Li, Mengyue Yang
Computer Vision
Ozone Cues Mitigate Reflected Downwelling Radiance in LWIR Absorption-Based Ranging
Unay Dorken Gallastegi, Wentao Shangguan, Vaibhav Choudhary et al. (+4 more)
Passive long-wave infrared (LWIR) absorption-based ranging relies on atmospheric absorption to estimate distances to objects from their emitted thermal radiation. First demonstrated decades ago for objects much hotter than the air and recently extended to scenes with low temperature variations, this ranging has depended on reflected radiance being negligible. Downwelling radiance is especially problematic, sometimes causing large inaccuracies. In two new ranging methods, we use characteristic features from ozone absorption to estimate the contribution of reflected downwelling radiance. The quadspectral method gives a simple closed-form range estimate from four narrowband measurements, two at a water vapor absorption line and two at an ozone absorption line. The hyperspectral method uses a broader spectral range to improve accuracy while also providing estimates of temperature, emissivity profiles, and contributions of downwelling from a collection of zenith angles. Experimental results demonstrate improved ranging accuracy, in one case reducing error from over 100 m when reflected light is not modeled to 6.8 m with the quadspectral method and 1.2 m with the hyperspectral method.
Read the full abstract ▾
All authors Unay Dorken Gallastegi, Wentao Shangguan, Vaibhav Choudhary, Akshay Agarwal, Hoover Rueda-Chacón, Martin J. Stevens, Vivek K Goyal
Computer Vision
Adversarial Patch Generation for Visual-Infrared Dense Prediction Tasks via Joint Position-Color Optimization
He Li, Wenyue He, Weihang Kong et al. (+1 more)
Multimodal adversarial attacks for dense prediction remain largely underexplored. In particular, visual-infrared (VI) perception systems introduce unique challenges due to heterogeneous spectral characteristics and modality-specific intensity distributions. Existing
adversarial patch methods are primarily designed for single-modal inputs and fail to account for crossspectral inconsistencies, leading to reduced attack effectiveness and poor stealthiness when applied to VI dense prediction models. To address these challenges, we propose a joint position-color optimization framework (AP-PCO) for generating
adversarial patches in visual-infrared settings. The proposed method optimizes patch placement and color composition simultaneously using a fitness function derived from model outputs, enabling a single patch to perturb both visible and infrared modalities. To further bridge spectral discrepancies, we introduce a crossmodal color adaptation strategy that constrains patch appearance according to infrared grayscale characteristics while maintaining strong perturbations in the visible domain, thereby reducing cross-spectral saliency. The optimization procedure operates without requiring internal model information, supporting flexible black-box attacks. Extensive experiments on visual-infrared dense prediction tasks demonstrate that the proposed AP-PCO achieves consistently strong attack performance across multiple architectures, providing a practical
benchmark for robustness evaluation in VI perception systems.
Read the full abstract ▾
All authors He Li, Wenyue He, Weihang Kong, Xingchen Zhang
Computer Vision · Cross-listed from quant-ph
Scaling Quantum Machine Learning without Tricks: Full-Resolution and Diverse Image Generation
Jonas Jäger, Florian J. Kiwit and Carlos A. Riofrío
Quantum generative modeling is a rapidly evolving discipline at the intersection of
quantum computing and machine learning. Contemporary
quantum machine learning is generally limited to toy examples or heavily restricted
datasets with few elements. This is not only due to the current limitations of available
quantum hardware but also due to the absence of inductive biases arising from application-agnostic designs. Current
quantum solutions must resort to tricks to scale down high-resolution images, such as relying heavily on dimensionality reduction or utilizing multiple
quantum models for low-resolution image patches. Building on recent developments in classical image loading to
quantum computers, we circumvent these limitations and train
quantum Wasserstein GANs on the established classical MNIST and Fashion-MNIST
datasets. Using the complete
datasets, our system generates full-resolution images across all ten classes and establishes a new state-of-the-art performance with a single end-to-end
quantum generator without tricks. As a proof-of-principle, we also demonstrate that our approach can be extended to color images, exemplified on the Street View House Numbers
dataset. We analyze how the choice of variational
circuit architecture introduces inductive biases, which crucially unlock this performance. Furthermore, enhanced noise input techniques enable highly diverse image generation while maintaining quality. Finally, we show promising results even under
quantum shot noise conditions.
Read the full abstract ▾
Computer Vision
Pretty Good Measurement for Radiomics: A Quantum-Inspired Multi-Class Classifier for Lung Cancer Subtyping and Prostate Cancer Risk Stratification
Giuseppe Sergioli, Carlo Cuccu, Giovanni Pasini et al. (+4 more)
We investigate a
quantum-inspired approach to supervised multi-class classification based on the Pretty Good Measurement (PGM), viewed as an operator-valued decision rule derived from
quantum state discrimination. The method associates each class with an encoded mixed state and performs classification through a single POVM construction, thus providing a genuinely multi-class strategy without reduction to pairwise or one-vs-rest schemes. In this perspective, classification is reformulated as the discrimination of a finite ensemble of class-dependent density operators, with performance governed by the geometry induced by the encoding map and by the overlap structure among classes. To assess the practical scope of this framework, we apply the PGM-based classifier to two biomedical radiomics case studies: histopathological subtyping of non-small-
cell lung carcinoma (NSCLC) and prostate cancer (PCa) risk stratification. The evaluation is conducted under protocols aligned with previously reported radiomics studies, enabling direct comparison with established classical baselines. The results show that the PGM-based classifier is consistently competitive and, in several settings, improves upon standard methods. In particular, the method performs especially well in the NSCLC binary and three-class tasks, while remaining competitive in the four-class case, where increased class overlap yields a more demanding discrimination geometry. In the PCa study, the PGM classifier remains close to the strongest ensemble baseline and exhibits clinically relevant sensitivity–specificity trade-offs across feature-selection scenarios.
Read the full abstract ▾
All authors Giuseppe Sergioli, Carlo Cuccu, Giovanni Pasini, Alessandro Stefano, Giorgio Russo, Andrés Camilo Granda Arango, Roberto Giuntini
Computer Vision · Cross-listed from eess.IV
GLIDE-Reg: Global-to-Local Deformable Registration Using Co-Optimized Foundation and Handcrafted Features
Yunzheng Zhu, Aichi Chien, Kimaya kulkarni et al. (+5 more)
Deformable registration is crucial in medical imaging. Several existing applications include lesion tracking, probabilistic atlas generation, and treatment response evaluation. However, current methods often lack robustness and generalizability across two key factors: spatial resolution and differences in anatomical coverage. We jointly optimize a registration field and a learnable dimensionality reduction module so that compressed VFM
embeddings remain registration-relevant, and fuse these global semantic cues with MIND local descriptors. GLIDE-Reg achieves average dice similarity coefficients (DSC) across 6 anatomical structures of 0.859, 0.862, and 0.901 in two public cohorts (Lung250M and NLST) and one institution cohort (UCLA5DCT), and outperforms the state-of-the-art DEEDS (0.834, 0.858, 0.900) with relative improvements of 3.0%, 0.5%, and 0.1%. For target registration errors, GLIDE-Reg achieves 1.58 mm on Lung250M landmarks (compared to 1.25 mm on corrField and 1.91 mm on DEEDS) and 1.11 mm on NLST nodule centers (compared to 1.11 mm on DEEDS). The substantiated performance on the nodule centers also demonstrates its robustness across challenging downstream tasks, such as nodule tracking, which is an essential prior step for early-stage lung cancer diagnosis.
Read the full abstract ▾
All authors Yunzheng Zhu, Aichi Chien, Kimaya kulkarni, Luoting Zhuang, Stephen Park, Ricky Savjani, Daniel Low, William Hsu
Computer Vision
Physical Evaluation of Naturalistic Adversarial Patches for Camera-Based Traffic-Sign Detection
Brianna D'Urso, Tahmid Hasan Sakib, Syed Rafay Hasan et al. (+1 more)
This paper studies how well Naturalistic
Adversarial Patches (NAPs) transfer to a physical traffic sign setting when the detector is trained on a customized
dataset for an autonomous vehicle (AV) environment. We construct a composite
dataset, CompGTSRB (which is customized
dataset for AV environment), by pasting traffic sign instances from the German Traffic Sign Recognition
Benchmark (GTSRB) onto undistorted backgrounds captured from the target platform. CompGTSRB is used to train a YOLOv5 model and generate patches using a Generative
Adversarial Network (GAN) with latent space optimization, following existing NAP methods. We carried out a series of experiments on our Quanser QCar testbed utilizing the front CSI camera provided in QCar. Across configurations, NAPs reduce the detector's STOP class confidence. Different configurations include distance, patch sizes, and patch placement. These results along with a detailed step-by-step methodology indicate the utility of CompGTSRB
dataset and the proposed systematic physical protocols for credible patch evaluation. The research further motivate researching the defenses that address localized patch corruption in embedded perception pipelines.
Read the full abstract ▾
All authors Brianna D'Urso, Tahmid Hasan Sakib, Syed Rafay Hasan, Terry N. Guo
Computer Vision
VisRef: Visual Refocusing while Thinking Improves Test-Time Scaling in Multi-Modal Large Reasoning Models
Soumya Suvra Ghosal, Youngeun Kim, Zhuowei Li et al. (+6 more)
Advances in large
reasoning models have shown strong performance on complex
reasoning tasks by scaling test-time compute through extended
reasoning. However, recent studies observe that in
vision-dependent tasks, extended textual
reasoning at inference time can degrade performance as models progressively lose
attention to visual
tokens and increasingly rely on textual priors alone. To address this, prior works use reinforcement learning (RL)-based fine-tuning to route visual
tokens or employ refocusing mechanisms during
reasoning. While effective, these methods are computationally expensive, requiring large-scale data generation and policy optimization. To leverage the benefits of test-time compute without additional RL fine-tuning, we propose VisRef, a visually grounded test-time scaling framework. Our key idea is to actively guide the
reasoning process by re-injecting a coreset of visual
tokens that are semantically relevant to the
reasoning context while remaining diverse and globally representative of the image, enabling more grounded multi-modal
reasoning. Experiments on three visual
reasoning benchmarks with state-of-the-art multi-modal large
reasoning models demonstrate that, under fixed test-time compute budgets, VisRef consistently outperforms existing test-time scaling approaches by up to 6.4%.
Read the full abstract ▾
All authors Soumya Suvra Ghosal, Youngeun Kim, Zhuowei Li, Ritwick Chaudhry, Linghan Xu, Hongjing Zhang, Jakub Zablocki, Yifan Xing, Qin Zhang
Computer Vision
TACIT Benchmark: A Programmatic Visual Reasoning Benchmark for Generative and Discriminative Models
Daniel Nobrega Medeiros
Existing visual
reasoning benchmarks predominantly rely on natural
language prompts, evaluate narrow
reasoning modalities, or depend on subjective scoring procedures such as LLM-as-judge. We introduce the TACIT
Benchmark, a programmatic visual
reasoning benchmark comprising 10 tasks across 6
reasoning domains: spatial
navigation, abstract pattern completion, causal simulation, logical constraint satisfaction, graph theory, and
topology. The
benchmark provides dual-track evaluation: a generative track in which models must produce solution images verified through deterministic computer-
vision pipelines, and a discriminative track offering five-way multiple choice with structurally plausible near-miss distractors. Each distractor violates exactly one structural constraint, requiring models to reason about fine-grained visual differences rather than exploit superficial cues. Version 0.1.0 distributes 6,000 puzzles (108,000 PNG images across three resolutions) with fully deterministic seeded generation and reproducible verification. The
dataset, generation code, and evaluation harness are released under the Apache 2.0 license on HuggingFace (DOI: 10.57967/hf/7904).
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Efficient Flow Matching for Sparse-View CT Reconstruction
Jiayang Shi, Lincen Yang, Zhong Li et al. (+3 more)
Generative models, particularly
Diffusion Models (DM), have shown strong potential for Computed
Tomography (CT) reconstruction serving as expressive priors for solving ill-posed inverse problems. However,
diffusion-based reconstruction relies on
Stochastic Differential Equations (SDEs) for forward
diffusion and reverse denoising, where such stochasticity can interfere with repeated data consistency corrections in CT reconstruction. Since CT reconstruction is often time-critical in clinical and interventional scenarios, improving reconstruction efficiency is essential. In contrast, Flow Matching (FM) models sampling as a deterministic Ordinary Differential Equation (ODE), yielding smooth trajectories without
stochastic noise injection. This deterministic formulation is naturally compatible with repeated data consistency operations. Furthermore, we observe that FM-predicted velocity fields exhibit strong correlations across adjacent steps. Motivated by this, we propose an FM-based CT reconstruction framework (FMCT) and an efficient variant (EFMCT) that reuses previously predicted velocity fields over consecutive steps to substantially reduce the number of Neural network Function Evaluations (NFEs), thereby improving inference efficiency. We provide theoretical analysis showing that the error introduced by velocity reuse is bounded when combined with data consistency operations. Extensive experiments demonstrate that FMCT/EFMCT achieve competitive reconstruction quality while significantly improving computational efficiency compared with
diffusion-based methods. The codebase is open-sourced at https://github.com/EFMCT/EFMCT.
Read the full abstract ▾
All authors Jiayang Shi, Lincen Yang, Zhong Li, Tristan van Leeuwen, Daniel M. Pelt, K. Joost Batenburg
Computer Vision · Cross-listed from eess.IV
Optimisation of SOUP-GAN and CSR-GAN for High Resolution MR Images Reconstruction
Muneeba Rashid, Hina Shakir, Humaira Mehwish et al. (+2 more)
Magnetic Resonance (MR) imaging is a diagnostic tool used in modern medicine; however, its output can be affected by motion artefacts and may be limited by equipment. This research focuses on MRI image quality enhancement using two efficient Generative
Adversarial Networks (GANs) models: SOUP-GAN and CSR-GAN. In both models, meaningful architectural modifications were introduced. The generator and discriminator of each were further deepened by adding convolutional layers and were enhanced in filter sizes as well. The LeakyReLU activation function was used to improve gradient flow, and hyperparameter tuning strategies were applied, including a reduced learning rate and an optimal batch size. Moreover, spectral normalisation was proposed to address mode collapse and improve training stability. The experiment shows that CSR-GAN has better performance in reconstructing the image with higher frequency details and reducing noise compared to other methods, with an optimised PSNR of 34.6 and SSIM of 0.89. However, SOUP-GAN performed the best in terms of delivering less noisy images with good structures, achieving a PSNR of 34.4 and SSIM of 0.83. The obtained results indicate that the proposed enhanced GAN model can be a useful tool for MR image quality improvement for subsequent better disease diagnostics.
Read the full abstract ▾
All authors Muneeba Rashid, Hina Shakir, Humaira Mehwish, Asarim Amir, Reema Qaiser Khan
Computer Vision
AdURA-Net: Adaptive Uncertainty and Region-Aware Network
Antik Aich Roy and Ujjwal Bhattacharya
One of the common issues in clinical decision-making is the presence of uncertainty, which often arises due to ambiguity in radiology reports, which often reflect genuine diagnostic uncertainty or limitations of automated label extraction in various complex cases. Especially the case of multilabel
datasets such as CheXpert, MIMIC-CXR, etc., which contain labels such as positive, negative, and uncertain. In clinical decision-making, the uncertain label plays a tricky role as the model should not be forced to provide a confident prediction in the absence of sufficient evidence. The ability of the model to say it does not understand whenever it is not confident is crucial, especially in the cases of clinical decision-making involving high risks. Here, we propose AdURA-Net, a geometry-driven adaptive uncertainty-aware framework for reliable thoracic disease classification. The key highlights of the proposed model are: a) Adaptive dilated convolution and multiscale deformable
alignment coupled with the backbone Densenet architecture capturing the anatomical complexities of the medical images, and b) Dual Head Loss, which combines masked binary cross
entropy with logit and a Dirichlet evidential learning objective.
Read the full abstract ▾
Computer Vision
Stateful Token Reduction for Long-Video Hybrid VLMs
Jindong Jiang, Amala Sanjay Deshmukh, Kateryna Chumachenko et al. (+7 more)
Token reduction accelerates long-
video vision–
language models (VLMs), but existing methods target
Transformers, where reduction is treated as
token pruning. We study
token reduction in hybrid Mamba–
Transformer VLMs and find that it is stateful: Mamba layers maintain a recurrent state that accumulates information from earlier
tokens, allowing discarded
tokens to persist, so reduction behaves more like compression than dropping.We support this view with a representation-based probing method measuring how much information from discarded
tokens is retained, and analyze layer-wise sparsity and cross-layer importance stability. Our findings show importance is sparse within layers but unstable across layers, making aggressive early
pruning unreliable while hybrids remain robust to later reduction.Motivated by this, we propose a hybrid-aware
token reduction framework with a low-to-high progressive schedule and a unified query-conditioned importance score for
attention and Mamba layers. For Mamba, excluding the position-dependent decay from the recurrence produces a stronger selection signal. Across long-
video benchmarks, our method achieves $3.8{\times}$–$4.2{\times}$ prefilling speedups at a 25%
token budget while maintaining near-baseline accuracy and improving with light finetuning. Hybrid models benefit from aggressive reduction, improving both efficiency and accuracy, whereas
Transformers exhibit the standard trade-off. Our method also outperforms prior baselines on the same hybrid backbone and combines effectively with visual redundancy reduction methods.
Read the full abstract ▾
All authors Jindong Jiang, Amala Sanjay Deshmukh, Kateryna Chumachenko, Karan Sapra, Zhiding Yu, Guilin Liu, Andrew Tao, Pavlo Molchanov, Jan Kautz, Wonmin Byeon
Computer Vision
A Case Study on Concept Induction for Neuron-Level Interpretability in CNN
Moumita Sen Sarma, Samatha Ereshi Akkamahadevi and Pascal Hitzler
Deep Neural Networks (DNNs) have advanced applications in domains such as healthcare, autonomous systems, and scene understanding, yet the internal semantics of their hidden
neurons remain poorly understood. Prior work introduced a Concept Induction-based framework for hidden
neuron analysis and demonstrated its effectiveness on the ADE20K
dataset. In this case study, we investigate whether the approach generalizes by applying it to the SUN2012
dataset, a large-scale scene recognition
benchmark. Using the same workflow, we assign interpretable semantic labels to
neurons and validate them through web-sourced images and statistical testing. Our findings confirm that the method transfers to SUN2012, showing its broader applicability.
Read the full abstract ▾
Computer Vision
SKeDA: A Generative Watermarking Framework for Text-to-video Diffusion Models
Yang Yang, Xinze Zou, Zehua Ma et al. (+2 more)
The rise of text-to-
video generation models has raised growing concerns over content authenticity, copyright protection, and malicious misuse. Watermarking serves as an effective mechanism for regulating such AI-generated content, where high fidelity and strong robustness are particularly critical. Recent generative image watermarking methods provide a promising foundation by leveraging watermark information and pseudo-random keys to control the initial sampling noise, enabling lossless
embedding. However, directly extending these techniques to
videos introduces two key limitations: Existing designs implicitly rely on strict
alignment between
video frames and frame-dependent pseudo-random binary sequences used for watermark encryption. Once this
alignment is disrupted, subsequent watermark extraction becomes unreliable; and
Video-specific distortions, such as inter-frame compression, significantly degrade watermark reliability. To address these issues, we propose SKeDA, a generative watermarking framework tailored for text-to-
video diffusion models. SKeDA consists of two components: (1) Shuffle-Key-based Distribution-preserving Sampling (SKe) employs a single base pseudo-random binary sequence for watermark encryption and derives frame-level encryption sequences through permutation. This design transforms watermark extraction from synchronization-sensitive sequence decoding into permutation-tolerant set-level aggregation, substantially improving robustness against frame reordering and loss; and (2) Differential
Attention (DA), which computes inter-frame differences and dynamically adjusts
attention weights during extraction, enhancing robustness against temporal distortions. Extensive experiments demonstrate that SKeDA preserves high
video generation quality and watermark robustness.
Read the full abstract ▾
All authors Yang Yang, Xinze Zou, Zehua Ma, Han Fang, Weiming Zhang
Computer Vision · Cross-listed from cs.LG
Task-Driven Subspace Decomposition for Knowledge Sharing and Isolation in LoRA-based Continual Learning
Lingfeng He, De Cheng, Huaijie Wang et al. (+3 more)
Continual Learning (CL) requires models to sequentially adapt to new tasks without forgetting old knowledge. Recently, Low-Rank Adaptation (LoRA), a representative Parameter-Efficient Fine-Tuning (PEFT) method, has gained increasing
attention in CL. Several LoRA-based CL methods reduce interference across tasks by separating their update spaces, typically building the new space from the estimated null space of past tasks. However, they (i) overlook task-shared directions, which suppresses knowledge transfer, and (ii) fail to capture truly effective task-specific directions since these “null bases" of old tasks can remain nearly inactive for new task under correlated tasks. To address this, we study LoRA learning capability from a projection energy perspective, and propose Low-rank Decomposition and Adaptation (LoDA). It performs a task-driven decomposition to build general and truly task-specific LoRA subspaces by solving two energy-based objectives, decoupling directions for knowledge sharing and isolation. LoDA fixes LoRA down-projections on two subspaces and learns robust up-projections via a Gradient-Aligned Optimization (GAO) approach. After each task, before integrating the LoRA updates into the backbone, LoDA derives a closed-form recalibration for the general update, approximating a feature-level joint optimum along this task-shared direction. Experiments indicate that LoDA outperforms existing CL methods. Our code is available at https://github.com/HHHLF/LoDA_ICML2026.
Read the full abstract ▾
All authors Lingfeng He, De Cheng, Huaijie Wang, Xi Yang, Nannan Wang, Xinbo Gao
Computer Vision
ST-Lite: Training-Free KV Cache Compression with Spatio-Trajectory Guidance for Long-Horizon GUI Agents
Bowen Zhou, Zhou Xu, Wanli Li et al. (+3 more)
Training-free KV cache compression is essential for deploying
vision-
language GUI
agents under
memory and latency constraints, yet existing methods are designed for generic
language workloads and ignore the distinctive structure of GUI interaction traces. We characterize three GUI-specific workload properties–high inter-frame visual redundancy, extremely small UI-element spatial footprints, and near-uniform cross-layer
attention sparsity–that cause existing schemes to retain as few as 39% of oracle-important KV pairs at the standard 20% budget. To address this, we propose ST-Lite, a training-free compression scheme whose three components each target one property: Trajectory-aware Semantic Gating (TSG) filters redundant historical frames, Component-centric Spatial Saliency (CSS) preserves fine-grained element boundaries, and a flat per-layer budget avoids hierarchical misallocation. Across seven GUI
benchmarks and two backbones in the deployment-relevant 10%-40% window, ST-Lite consistently outperforms all existing compression baselines, matching or exceeding Full Cache task accuracy on the primary backbone at the 20% budget while delivering up to 2.35x decoding speedup at fivefold compression. The implementation is available at https://github.com/94wen94/ST-Lite.
Read the full abstract ▾
All authors Bowen Zhou, Zhou Xu, Wanli Li, Jingyu Xiao, Pingan Gan, Haoqian Wang
Computer Vision
UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images
Junhwa Hur, Charles Herrmann, Songyou Peng et al. (+4 more)
Dense 4D reconstruction from unposed images remains a critical challenge, with current methods relying on slow test-time optimization or fragmented, task-specific feedforward models. We introduce UFO-4D, a unified feedforward framework to reconstruct a dense, explicit 4D representation from just a pair of unposed images. UFO-4D directly estimates dynamic 3D Gaussian Splats, enabling the joint and consistent estimation of 3D geometry, 3D motion, and camera pose in a feedforward manner. Our core insight is that differentiably rendering multiple signals from a single Dynamic 3D Gaussian representation offers major training advantages. This approach enables a self-supervised image synthesis loss while tightly coupling appearance, depth, and motion. Since all modalities share the same
geometric primitives, supervising one inherently regularizes and improves the others. This synergy overcomes data scarcity, allowing UFO-4D to outperform prior work by up to 3 times in joint geometry, motion, and camera pose estimation. Our representation also enables high-fidelity 4D interpolation across novel views and time. Please visit our project page for visual results: https://ufo-4d.github.io/
Read the full abstract ▾
All authors Junhwa Hur, Charles Herrmann, Songyou Peng, Philipp Henzler, Zeyu Ma, Todd Zickler, Deqing Sun
Computer Vision
Mode Seeking meets Mean Seeking for Fast Long Video Generation
Shengqu Cai, Weili Nie, Chao Liu et al. (+8 more)
Scaling
video generation from seconds to minutes faces a critical bottleneck: while short-
video data is abundant and high-fidelity, coherent long-form data is scarce and limited to narrow domains. To address this, we propose a training paradigm where Mode Seeking meets Mean Seeking, decoupling local fidelity from long-term coherence based on a unified representation via a Decoupled
Diffusion Transformer. Our approach utilizes a global Flow Matching head trained via supervised learning on long
videos to capture narrative structure, while simultaneously employing a local Distribution Matching head that aligns sliding windows to a frozen short-
video teacher via a mode-seeking reverse-KL divergence. This strategy enables the synthesis of minute-scale
videos that learns long-range coherence and motions from limited long
videos via supervised flow matching, while inheriting local realism by aligning every sliding-window segment of the student to a frozen short-
video teacher, resulting in a few-step fast long
video generator. Evaluations show that our method effectively closes the fidelity-horizon gap by jointly improving local sharpness, motion and long-range consistency. Project website: https://primecai.github.io/mmm/.
Read the full abstract ▾
All authors Shengqu Cai, Weili Nie, Chao Liu, Julius Berner, Lvmin Zhang, Nanye Ma, Hansheng Chen, Maneesh Agrawala, Leonidas Guibas, Gordon Wetzstein, Arash Vahdat
Computer Vision
Hierarchical Action Learning for Weakly-Supervised Action Segmentation
Junxian Huang, Ruichu Cai, Hao Zhu et al. (+5 more)
Humans perceive actions through key transitions that structure actions across multiple abstraction levels, whereas machines, relying on visual features, tend to over-segment. This highlights the difficulty of enabling hierarchical
reasoning in
video understanding. Interestingly, we observe that lower-level visual and high-level action latent variables evolve at different rates, with low-level visual variables changing rapidly, while high-level action variables evolve more slowly, making them easier to identify. Building on this insight, we propose the Hierarchical Action Learning (HAL) model for weakly-supervised action
segmentation. Our approach introduces a hierarchical causal data generation process, where high-level latent action governs the dynamics of low-level visual features. To model these varying timescales effectively, we introduce deterministic processes to align these latent variables over time. The HAL model employs a hierarchical pyramid
transformer to capture both visual features and latent variables, and a sparse transition constraint is applied to enforce the slower dynamics of high-level action variables. This mechanism enhances the identification of these latent variables over time. Under mild assumptions, we prove that these latent action variables are strictly identifiable. Experimental results on several
benchmarks show that the HAL model significantly outperforms existing methods for weakly-supervised action
segmentation, confirming its practical effectiveness in real-world applications.
Read the full abstract ▾
All authors Junxian Huang, Ruichu Cai, Hao Zhu, Juntao Fang, Boyan Xu, Weilin Chen, Zijian Li, Shenghua Gao
Computer Vision
Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models
Arnas Uselis, Andrea Dittadi and Seong Joon Oh
Compositional generalization, the ability to recognize familiar parts in novel contexts, is a defining property of intelligent systems. Although modern models are trained on massive
datasets, they still cover only a tiny fraction of the combinatorial space of possible inputs, raising the question of what structure representations must have to support generalization to unseen combinations. We formalize three desiderata for compositional generalization under standard training (divisibility, transferability, stability) and show they impose necessary
geometric constraints: representations must decompose linearly into per-concept components, and these components must be orthogonal across concepts. This provides theoretical grounding for the Linear Representation Hypothesis: the linear structure widely observed in neural representations is a necessary consequence of compositional generalization. We further derive dimension bounds linking the number of composable concepts to the
embedding geometry. Empirically, we evaluate these predictions across modern
vision models (CLIP, SigLIP, DINO) and find that representations exhibit partial linear factorization with low-rank, near-orthogonal per-concept factors, and that the degree of this structure correlates with compositional generalization on unseen combinations. As models continue to scale, these conditions predict the representational geometry they may converge to. Code is available at https://github.com/oshapio/necessary-compositionality.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Histopathology Image Normalization via Latent Manifold Compaction
Xiaolong Zhang, Jianwei Zhang, Selim Sevim et al. (+3 more)
Batch effects arising from technical variations in histopathology staining protocols, scanners, and acquisition pipelines pose a persistent challenge for computational pathology, hindering cross-batch generalization and limiting reliable deployment of models across clinical sites. In this work, we introduce Latent
Manifold Compaction (LMC), an unsupervised representation learning framework that performs image harmonization by learning batch-invariant
embeddings from a single source
dataset through explicit compaction of stain-induced latent
manifolds. This allows LMC to generalize to target domain data unseen during training. Evaluated on three challenging public and in-house
benchmarks, LMC substantially reduces batch-induced separations across multiple
datasets and consistently outperforms state-of-the-art normalization methods in downstream cross-batch classification and detection tasks, enabling superior generalization.
Read the full abstract ▾
All authors Xiaolong Zhang, Jianwei Zhang, Selim Sevim, Emek Demir, Ece Eksi, Xubo Song