Computer Vision
Namritha Lasyapriya Maddali, Rajini Makam, Suresh Sundaram et al. (+1 more)
This paper presents $\pi$-SUB, a physics-informed framework for generating synthetic
underwater benchmark datasets that bridges the synthetic-to-real gap for
Underwater Image Enhancement (UIE). The proposed framework extends the classical
underwater image formation model by incorporating depth-dependent downwelling irradiance, biologically resolved absorption, and environmental scattering across all ten Jerlov water types, together with independently controllable residual phenomena. Using this framework, the $\pi$-SUB
dataset consists of paired synthetic
underwater-reference images spanning shallow-to-deep and coastal-to-oceanic environments. Extensive simulation studies have been carried out to evaluate $\pi$-SUB along two criteria namely hyper-realism and generalizability. For hyper-realism, $\pi$-SUB attains a global Frechet Inception Distance (FID) that is 46% lower than Syrea. For generalizability, four state-of-the-art UIE architectures (FUnIE-GAN, Pix2Pix, PUIE-Net, and Phaseformer) are used for comparative evaluation of $\pi$-SUB. These models were independently trained on six
datasets including one real and five synthetic
datasets and tested on six real-world
benchmarks datasets. Across four UIE architectures and six real
benchmark datasets, $\pi$-SUB improves UIQM by 4.18% over PHISWID (next best) and 9.46% over Syrea (next best), while reducing NIQE by 48.78% and 23.98%, respectively. These results establish $\pi$-SUB as a hyper-realistic and generalizable
benchmark for developing the next generation of
underwater image enhancement methods. The code and
dataset are available at https://github.com/airl-iisc/pi-SUB
Read the full abstract ▾
All authors Namritha Lasyapriya Maddali, Rajini Makam, Suresh Sundaram, Narasimhan Sundararajan
Computer Vision
A HamNoSys-Guided Dataset and Baselines for Fine-Grained Isolated Handshape Recognition in Sign Language
Ushnish Sarkar, Suvajit Patra, Bhaswar Chattopadhyay et al. (+2 more)
Purpose: Fine-grained handshape recognition supports computational sign-
language transcription, recognition, and translation, but broad, phonetically defined visual inventories with signer-aware evaluation remain limited. This work introduces a
benchmark grounded in the
language-independent Hamburg Notation System (HamNoSys). Methods: A balanced
dataset of 144,000 RGB images was collected from 15 participants for 160 handshape classes defined by the official HamNoSys 4 Handshapes Chart. ResNet-18 and ViT-B/16 were evaluated as appearance-based models, while a graph convolutional network and XGBoost were evaluated from hand landmarks. Both a class-stratified subject-dependent split and a 15-fold leave-one-subject-out (LOSO) protocol were used. The same model families were additionally assessed on LSWH100 and ASL Fingerspelling
Dataset A for external context. Results: The subject-dependent
benchmarks established reproducible reference performance across all four model families, whereas LOSO evaluation exposed a substantial reduction when recognition was required to generalise to unseen participants. On ASL Fingerspelling
Dataset A, mean LOSO top-1 accuracy ranged from 82.20% to 87.40%. Conclusion: The documented acquisition, curation, and complementary evaluation protocols pro-vide a reproducible resource for fine-grained isolated-handshape research and for developing more accessible sign-
language technologies.
Read the full abstract ▾
All authors Ushnish Sarkar, Suvajit Patra, Bhaswar Chattopadhyay, Pranab Singha Roy, Tapas Samanta
Computer Vision · Cross-listed from stat.ML
Iterative Erasure Count Is Not an Affine-Invariant Concept Dimension
Tingan Jin, Shuhang Dong, Haosong Li et al. (+1 more)
How many directions does a neural representation use to encode a concept? A common answer repeatedly erases probe directions and reports the stopping count or cumulative removed rank. We show that both quantities can change under an information-preserving invertible reparameterization, so neither is intrinsically a concept dimension. We distinguish model-defined population quantities (generating dimension, sufficient linear dimension, and minimum guarding rank) from procedure-defined quantities such as stopping count and cumulative edit rank. In a population Gaussian construction, an invertible shear preserves the prediction problem and all three quantities, yet changes the cumulative Euclidean erasure count from one to two. The separation holds for Moore–Penrose ordinary least squares and every finite nonnegative ridge weight. For a two-output full-QR procedure matching our motivating
video analysis, cumulative edit rank similarly changes from two to the ambient dimension four. Conversely, the complete cumulative metric-QR trajectory is affine-equivariant when its positive-definite metric, probe, regularizer, and tie-breaking are transported consistently; exact covariance is one corollary, not a canonical semantic metric. In a known-rank finite-sample Adam/QR calibration, identity mixing stops after one accepted update in all 20 large-sample runs, whereas each tested shear $a\in\{.5,.75,1,1.25,2\}$ accepts at least two updates in all 20 runs. Controlled reparameterizations of frozen V-JEPA2 features preserve rank-zero predictions yet alter later Euclidean trajectories under practical optimization. These visual contact experiments are stress tests, not estimates of contact dimension. Iterative erasure therefore returns a procedure-relative estimand jointly determined by representation geometry and the full measurement procedure, not a semantic dimension by itself.
Read the full abstract ▾
All authors Tingan Jin, Shuhang Dong, Haosong Li, Chung-Hsien Chou
Computer Vision
Flow Straight to Reality: Perceptually Consistent Flow Matching for Efficient Image Restoration
Sangwoo Jo, Donggeun Ko, Jayeon Kang et al. (+3 more)
Image restoration is fundamentally constrained by the tradeoff between distortion and perception: minimizing pixel-wise error yields over-smoothed results, whereas optimizing for perceptual realism often introduces structural deviations. Recent approaches attempt to balance this tradeoff via posterior sampling or multi-stage generative pipelines, yet remain computationally expensive and architecturally complex. To overcome these limitations, we propose PCFlow (Perceptually Consistent Flow Matching), a unified framework that directly parameterizes a continuous transport from degraded observations to clean targets, jointly optimizing distortion and perceptual quality. While its latent consistency flow objective drives stable and efficient few-step inference, a Latent Consistency Perceptual Loss (LCPL) imposes semantic constraints directly on the guiding velocity field, steering the dynamics toward visually sharp data
manifolds. Furthermore, recognizing the inherent conflict between structural and perceptual consistencies, we integrate a conflict-free gradient projection strategy to stabilize the multi-objective optimization landscape. Combined with lightweight, convolution-only backbone, PCFlow achieves competitive performance across diverse restoration tasks at a fraction of traditional computational costs.
Read the full abstract ▾
All authors Sangwoo Jo, Donggeun Ko, Jayeon Kang, Youngsang Kwak, Jaehwa Kwak, Sungjoon Choi
Computer Vision
Dynamic Context Adapters: Efficiently Infusing History into Vision-and-Language Models
Yuhang Song, Bor-Jiun Lin, Jiaxu Liu et al. (+3 more)
Historical context integration presents a fundamental challenge for
Vision-
Language Models (VLMs) in sequential decision-making tasks. Current VLMs process visual inputs independently, which creates critical limitations for downstream applications that require temporal understanding. Direct incorporation of historical frames into
Transformer inputs produces quadratic
attention complexity and excessive
memory consumption. Existing approaches suffer from significant drawbacks: computational inflation or substantial information loss through temporal compression. To address these challenges, we introduce Dynamic Context Adapter (DCA), a novel context injection approach for pretrained VLMs. Our method employs fixed-size, dynamically compressed
memory to preserve historical semantics without frame concatenation. DCA bridges static VLMs and recurrent policies and enables
memory capabilities in pretrained models while maintaining computational efficiency. DCA achieves over $25\%$ reduction in
attention FLOPs and $13\%$
memory savings while improving performance on long-horizon tasks.
Read the full abstract ▾
All authors Yuhang Song, Bor-Jiun Lin, Jiaxu Liu, Te-Chuan Chiu, Anh Nguyen, Chun-Yi Lee
Computer Vision
Rethinking Text-Based Image Retrieval in Specific Domain
Jingyang Tan, Sheng Yang, Yuanpeng Chen et al. (+4 more)
Driven by the rapid advancement of
vision-
language representation learning, Text-based Image
Retrieval (TBIR) has made notable progress. However, existing
benchmarks are predominantly constructed on an exclusive single-match assumption between query and images. While effective in general scenarios, this assumption fails to reflect practical system performance in specific domains (e.g., surveillance), where a single query often corresponds to multiple relevant candidate images. To address this limitation, we design a Domain-Specific Multi-Match Text-based Image
Retrieval (DSMM-TBIR) data engine. Leveraging this engine, we construct Security Multi-Match TBIR (SecMM-TBIR), a
benchmark comprising 50k surveillance images with 200 comprehensive queries. Furthermore, we observe that vanilla contrastive learning in specific domains suffers from severe false negatives, forcing the model to push apart semantically similar pairs and thus degrading
retrieval performance. We propose the Semantic-Aware Fine-Tuning (SAFT) framework to address semantic compression in specific domains, which incorporates Semantic-Aware Soft-Label Supervision (SASS) and Intra-modal Structural
Distillation (ISD) to establish a promising paradigm for domain-specific TBIR tasks. Experiments across diverse CLIP-like models demonstrate that SAFT yields an average mAP@20 gain of 7.8 points on SecMM-TBIR over standard image-text contrastive (ITC) fine-tuning, while also improving general-domain performance. The entire
benchmark will be released to facilitate further research.
Read the full abstract ▾
All authors Jingyang Tan, Sheng Yang, Yuanpeng Chen, Jian Wang, Nianjin Ye, Chen Xing, Lanpeng Jia
Computer Vision
Unlocking the Power of Medical Tabular Data via Semantic-Aware Multimodal Pre-training
Yingsheng Liu, Haiming Li, Jingmin Zhu et al. (+6 more)
While
vision-
language models dominate medical representation learning, unstructured text lacks the dense, quantitative diagnostic phenotypes inherent in structured clinical tables. However, existing
multimodal pre-training methods underutilize this potential due to semantic-agnostic designs that treat tabular inputs as flat vectors and employ unstable continuous regression objectives. To overcome this, we propose a novel semantic-aware framework explicitly modeling the intrinsic two-dimensional structure of tabular data. First, addressing the inter-feature hierarchy of varying diagnostic importance, we introduce Importance-Aware Adaptive Masking to construct a label-free curriculum prioritizing salient features. Second, addressing the intra-feature continuity-discreteness duality, we propose a Soft-Label Discretized Module that replaces unstable numerical regression with stable distribution matching, thereby mathematically preserving ordinal relationships. Extensive experiments across large-scale dermatology (SLICE-3D, HOP) and ophthalmology (EyePACS)
datasets establish a new state-of-the-art (SOTA), demonstrating exceptional robustness and cross-domain generalizability.
Read the full abstract ▾
All authors Yingsheng Liu, Haiming Li, Jingmin Zhu, Jiajun Sun, Victoria Mar, Monika Janda, H. Peter Soyer, Zongyuan Ge, Zhen Yu
Computer Vision
SparSTAR: Sparse Attention for SpaceTime AutoRegressive Video Synthesis
Jongbeom Lee, Hyunwoo Yu, Jincheol Yang et al. (+2 more)
InfinityStar extends visual autoregressive generation to
video through a sequence of image and clip pyramids. Its changing scale and cross-clip context, however, leave late-scale
attention costly and make sparse patterns reused from
diffusion or image VAR models unreliable. We introduce SparSTAR, a training-free block-sparse
attention method tailored to this setting. At each expensive scale and
attention head, SparSTAR scores contiguous key blocks from the current query and key activations, retains required conditioning context, and executes the selected blocks through a forward-only sparse path. We analyze cross-scale consistency within a clip, pattern persistence across clip boundaries, and quality degradation as reuse spans increasingly distant scales. Across these analyses, important key blocks shift, showing that recomputing block selection at each target scale is more reliable than reusing a transferred mask. On 720p text-to-
video and image-to-
video generation, SparSTAR preserves every
token and refinement scale while providing about a 1.6x end-to-end speedup and maintaining VBench and paired-output reconstruction fidelity close to dense InfinityStar.
Read the full abstract ▾
All authors Jongbeom Lee, Hyunwoo Yu, Jincheol Yang, Jaemin Choi, Suk-Ju Kang
Computer Vision
SafeCap: Improving LVLM Safety with Image Captioning Reinforcement Learning
Caoyuan Ma, Wenpu Liu, Weichu Xie et al. (+12 more)
Large
vision-
language models (LVLMs) remain vulnerable to jailbreak attacks that exploit visual inputs to bypass safety
alignment inherited from their
language backbones. We propose SafeCap, a reinforcement-learning framework that aligns LVLMs through learned self-captioning. SafeCap trains a policy model to first generate a safety-relevant image caption and then produce a final answer; the caption is further optimized by whether it enables a frozen LLM to reach a safety-aligned decision. This caption-mediated objective encourages the policy to expose visual cues relevant to safe response generation rather than relying solely on direct refusal supervision. Across five
multimodal safety
benchmarks and six
vision-utility
benchmarks, SafeCap substantially improves aggregate safety performance under its intended DirectCap protocol, with gains of 3.7-19.0 points in safety average across four model settings while maintaining comparable or improved
vision utility. Under controlled comparisons on matched backbones and data, SafeCap outperforms safety SFT, DPO, and SafeGRPO, demonstrating the effectiveness of caption-mediated reinforcement learning for
multimodal safety
alignment.
Read the full abstract ▾
All authors Caoyuan Ma, Wenpu Liu, Weichu Xie, Tian Gu, Shilei Zhao, Lingxi Min, Shuai Dong, Yuqi Xu, Ji Zhao, Ziyue Wang, Wenzheng Chang, Taiqiang Wu, Yongfu Zhu, Wenqi Shao, Yinqiang Zheng
Computer Vision
Towards Color-Faithful Low-Light Image Enhancement via Adaptive Color Debiasing and Saturation Rectification
Zhichen Yang, Rui Xu, Yuzhen Niu et al. (+3 more)
Low-light imaging often introduces color bias caused by the low signal-to-noise ratio and the image formation process. Although recent low-light image enhancement methods have achieved strong brightness recovery, faithful color restoration remains challenging, manifesting as overall color bias together with local under- and over-saturation. To address this issue, we propose CAGE, a cylindrical color correction framework with adaptive color debiasing and gamut-harmonized saturation rectification for color-faithful low-light image enhancement. We first introduce AdaLAB, a cylindrical adaptive LAB color space that provides a decoupled and image-specific basis for uniform color correction. Building on this color space, we further develop AdaCCT, an adaptive cylindrical color transform with forward and inverse transforms for the conversion between RGB and AdaLAB color space, as well as necessary color debiasing and saturation rectification. The forward transform suppresses embedded color bias before backbone enhancement by reorganizing the chromatic distribution through chromatic-plane shifting and scaling, while the inverse transform achieves faithful saturation rectification through out-of-gamut lightness compensation. Extensive experiments on multiple
benchmarks show that CAGE achieves more faithful color restoration, specifically reduces color bias and saturation abnormality, and delivers better overall visual quality across different low-light enhancement backbones. The code is available at https://yangzhichen763.github.io/CAGE/.
Read the full abstract ▾
All authors Zhichen Yang, Rui Xu, Yuzhen Niu, Fusheng Li, Hui Da, Ri Cheng
Computer Vision · Cross-listed from cs.AI
RadFusion: Towards Threshold-Controllable Radiology Report Generation
Ying Jin, Noel C. F. Codella, John Corring et al. (+3 more)
Automated radiology report generation is advancing rapidly in response to the shortage of radiologists, yet unlike a perception model, existing generation models offer no control over the sensitivity-specificity trade-off of their diagnostic content. Such control is essential because clinical scenarios diverge: emergency triage prioritizes sensitivity to reduce missed findings, whereas confirmatory interpretation emphasizes specificity to limit unnecessary interventions. A single fixed report can neither adapt to these scenarios nor support the ROC-based validation widely expected for regulatory clearance. We introduce RadFusion, a framework that equips report generation with threshold controllability. Our method fuses a multi-label classifier, which provides per-disease confidence scores, with a VQA-based report generator, which describes medical findings in detail; an LLM then rewrites the report so that its stated diagnoses follow the classifier's decisions at the selected threshold while staying grounded in the generator's descriptions. On MIMIC-CXR, the performance of RadFusion conforms to the classifier's ROC curve: sweeping the threshold and mapping the reports back to class labels reproduces the classifier's validated ROC performance. This conformance makes generated reports quantitatively evaluable through ROC analysis, strengthening the case for regulatory clearance, and enables operating-point selection that matches report behavior to clinical context. Moreover, combining the two model types improves diagnostic accuracy over uncontrolled generation: sensitivity increases by 6.9% at matched specificity, and specificity by 20.7% at matched sensitivity. These results show that RadFusion makes report generation clinically adaptable, quantitatively verifiable, and diagnostically more reliable.
Read the full abstract ▾
All authors Ying Jin, Noel C. F. Codella, John Corring, Mu Wei, Dinei Florencio, Eric Horvitz
Computer Vision
DSAR: Dual-Stream Autoregressive Modeling of Temporal Cloth Dynamics for Photorealistic Animatable Avatars
Haozhong Xiong, Yao Yu, Yu Zhou et al. (+1 more)
Creating photorealistic and temporally coherent animatable human avatars from RGB
videos remains challenging. Current methods struggle to capture realistic cloth dynamics, producing over-smoothed appearance or severe artifacts on out-of-distribution poses. This limitation stems from a fundamental oversight: existing approaches neglect the temporal causality inherent in cloth physics, where current states emerge from previous states through temporal
evolution rather than instantaneous skeletal configurations alone. Without explicit modeling of this causal structure, networks learn pose-appearance correlations instead of motion
evolution, leading to poor generalization. We introduce a dual-stream autoregressive framework that explicitly models both observable
geometric information and implicit internal state. The
geometric stream propagates surface displacement from the previous frame, while the state stream fuses current features with historical states retrieved from a
memory bank. Motion-adaptive aggregation handles spatially-varying dynamics, and adaptive regularization balances smoothness with flexibility. Experiments on challenging
datasets demonstrate significant improvements in rendering quality, temporal consistency, and generalization to motion patterns beyond training distributions, validating that dual-stream temporal modeling enables realistic cloth dynamics.
Read the full abstract ▾
All authors Haozhong Xiong, Yao Yu, Yu Zhou, Sidan Du
Computer Vision
SapiensID 2.0: Aligning Human Recognition Foundation Models with Human Perception
Yiyang Su, Jie Zhu, Feng Liu et al. (+2 more)
While foundation models have significantly advanced human recognition across diverse modalities, they predominantly rely on static,
geometric feature extraction. This approach fundamentally diverges from human perception. Consequently, current models often suffer from "semantic blindness," overfitting to transient noise while failing to leverage invariant soft biometrics, and struggle to capture temporal motion signatures. To bridge this gap, we propose SapiensID 2.0, a human recognition framework enriched with both semantic and temporal awareness. To overcome the lack of soft-biometric annotations, we transfer zero-shot semantic knowledge from
Multimodal Large
Language Models (MLLMs) into a discriminative
embedding space. We resolve the dimensional mismatch between these spaces using Invariant Trait
Alignment (ITA) to distill core persistent traits, and Transient Noise Disentanglement (TND) to decouple artifacts like clothing. Furthermore, we design a Kinematic Semantic
Attention Head (K-SAH) that extends spatial
attention across temporal windows. By tracking semantic patches over time, K-SAH captures rich kinematic signatures without requiring large-scale
video datasets. Extensive experiments demonstrate that SapiensID 2.0 achieves state-of-the-art performance across image- and
video-based person re-identification and gait recognition, while maintaining robust face recognition capabilities.
Read the full abstract ▾
All authors Yiyang Su, Jie Zhu, Feng Liu, Anil K. Jain, Xiaoming Liu
Computer Vision
When Vision Becomes Text: Visual Token Pruning via Cross-Modal Residual Guidance in VLMs
Congyang Ou, Ruike Song, Yang Zhou et al. (+3 more)
Abundant visual information strengthens
vision-
language model (VLM) perception, yet massive visual
tokens raise inference costs. Existing visual
token pruning methods rely on similarity-based guidance, which exploits pairwise text-
vision and
vision-
vision token correlations for compression. However, such methods only capture local layer-level signals and overlook the whole inference process in VLM. In this paper, we revisit VLM inference and present a new efficient guidance scheme that complements similarity-based guidance. In particular, we identify a key observation: as LLM layers deepen, text
tokens continuously aggregate visual information via self-
attention and progressively absorb partial visual content into textual representations. To quantify this phenomenon, we propose Cross Modal Absorption (CMA) from a
geometric representation perspective to measure how much visual information is absorbed by text, revealing that more visual
tokens in deeper layers can be approximately explained by the text subspace. We accordingly propose Cross Modal Residual (CMR). It projects visual
tokens onto the text subspace via Tikhonov regularized least squares and exploits reconstruction residuals to quantify visual information that cannot be explained by text. Finally, based on CMR, we present SIEVE, a training-free visual
token compression method that combines CMR, text-
attention relevance, and residual-space diversity to retain task-relevant and complementary
tokens. Experiments on diverse VLM architectures verify the effectiveness of SIEVE. For instance, on LLaVA-NeXT-7B, SIEVE keeps only $11.1\%$ of visual
tokens while preserving $97.5\%$ of the original average performance, achieving $3.62\times$ prefill speedup, $2.49\times$ end-to-end speedup, and a $6.02\times$ KV-cache reduction.
Read the full abstract ▾
All authors Congyang Ou, Ruike Song, Yang Zhou, Libo Sun, Haokui Zhang, Zhenbo Luo
Computer Vision
Bridging Event Streams and DiT: Event-Guided Video Frame Interpolation
Guixu Lin, Yuyang Yu, Xiang Ji et al. (+6 more)
Latent
diffusion models have recently advanced
video frame interpolation by synthesizing intermediate frames between input images. However, handling large temporal gaps and complex motion remains challenging, often resulting in motion blur, structural distortions, and temporal inconsistencies. Event cameras provide high-temporal-resolution motion cues that are well suited for bridging these gaps and improving interpolation quality. To exploit this advantage without training an event-assisted model from scratch, we propose an adapter-based framework that incorporates event-derived cues into a pre-trained image-to-
video diffusion model with minimal architectural changes. Specifically, our method leverages Image Warped Events (IWEs) and bidirectional sparse optical flow to provide spatially and temporally aligned guidance during generation. By injecting these event-guided structural and motion cues into the
diffusion process, our approach reduces interpolation artifacts and improves both reconstruction fidelity and temporal coherence. Experimental results on real and synthetic
benchmarks show that our method consistently outperforms existing state-of-the-art approaches. The project page is at https://joseph-lin-tech.github.io/BridgeEventDiT-VFI/.
Read the full abstract ▾
All authors Guixu Lin, Yuyang Yu, Xiang Ji, Linyao Chen, Zhengwei Yin, Mengshun Hu, Mingdeng Cao, Shengfeng He, Yinqiang Zheng
Computer Vision
FUSE: Frame-Unified Stress Estimation from Facial Video
Stefanos Gkikas, Thomas Kassiotis, Yang Guo et al. (+2 more)
Automatic stress detection from facial
video offers a practical path to non-intrusive affect monitoring, yet existing
video-based approaches commonly decompose full recordings into short temporal windows before classification. This design introduces additional choices regarding window length, overlap, and aggregation, while limiting direct analysis of temporal information across the entire recording. In this study, we present FUSE (Frame-Unified Stress Estimation), a facial-
video stress detection framework that processes complete recordings as a single input without temporal windowing or external
segmentation. The name reflects the defining operation of the method: rather than dividing a recording into short clips, all frames are fused into one unified two-dimensional representation from which the stress state is estimated. This unification is realized by folding the temporal dimension into the channel dimension of the spatial representation, and the resulting high-dimensional input is processed using a unified asymmetric-
attention architecture. At a temporal stride of t = 1, FUSE retains the full 120-second recording as one input, corresponding to 3,600 frames at 30 fps. Experiments on a 58-subject stress
dataset using a stratified subject-level protocol evaluate seven temporal-stride configurations, ranging from full-frame input to sparse subsampling. FUSE achieves the highest test accuracy of 69.44% at t = 15, while the full-frame configuration remains competitive at 69.03%. Across the stride range, computational cost varies from 12.48 to 348.78 GFLOPs, showing the trade-off between temporal density and efficiency. These results demonstrate that temporal windowing is not required for effective facial-
video stress detection in this setting, and that complete-recording inference can be achieved within a single unified architecture.
Read the full abstract ▾
All authors Stefanos Gkikas, Thomas Kassiotis, Yang Guo, Guangliang Li, Giorgos Giannakakis
Computer Vision
Stream Forcing: Constructing Unified Training Trajectory for Robust Streaming Video Generation
Yueting Zhu, Yuehao Song, Kaicheng Zhang et al. (+5 more)
Streaming
video generation holds strong potential for world modeling, where future frames must be inferred online sequentially to form a continuous
video stream. However, streaming
video diffusion models introduce a fundamental train-inference mismatch: inference follows a specialized denoising order, whereas advanced training strategies typically require diverse noise-level configurations. To address this trade-off between train-inference consistency and training coverage, we reformulate the
video diffusion sampling as a frame-indexed
stochastic process over noise levels. Within this
stochastic process space, we construct a continuous training trajectory along which the sampling schedule progressively evolves from independent sampling to inference-consistent sampling. We further introduce a joint calibration
algorithm and a temporal correlative sampling
algorithm to ensure trajectory smoothness and cross-frame correlation. Building on these designs, we propose Stream Forcing, a unified training framework for streaming
video generation that balances training sufficiency and inference efficiency. Extensive experiments demonstrate that Stream Forcing significantly improves generation quality with a 36.6% FVD improvement on the UCF-101
benchmark. Furthermore, our method facilitates robust zero-shot extrapolation to long-horizon
video generation with a 27.9% FVD improvement on the UCF-101
benchmark.
Read the full abstract ▾
All authors Yueting Zhu, Yuehao Song, Kaicheng Zhang, Bao Tang, Shaoyu Chen, Qian Zhang, Wenyu Liu, Xinggang Wang
Computer Vision
MammoMix: Leveraging Mixture of Experts for Robust Mammogram Breast Detection
Dinh Tan Nguyen, Hoang Quan Dang, Chen Zhang et al. (+1 more)
Breast lesion detection in mammography remains a challenging task due to variations in image quality, lesion appearance, and population demographics across
datasets. While current object detectors such as YOLO and DETR achieve strong results on individual
datasets, their performance often degrades when trained on or applied across heterogeneous sources. To address this, we propose MammoMix, a novel framework based on Mixture-of-Experts (MoE) paradigm for robust and generalizable lesion detection. In MammoMix, each expert model is trained on a specific domain, allowing it to specialize in distinct characteristics of its source data. A gating mechanism adaptively weighs contributions from each expert based on input image, combining their outputs to enable domain-adaptive inference. To improve reliability, we further incorporate a calibration module, MoCAE, which adjusts confidence scores to reflect true predictive uncertainty. We evaluate MammoMix on 3 public mammography
datasets: CSAW, DDSM, and DMID, covering diverse clinical settings. Results show that MammoMix outperforms baseline detectors in both average precision and reliability, particularly on
datasets with greater variability. Our findings demonstrate that expert specialization and calibrated ensemble fusion significantly enhance model generalization and robustness. MammoMix offers a promising step toward dependable AI-assisted breast cancer screening across real-world clinical domains.
Read the full abstract ▾
All authors Dinh Tan Nguyen, Hoang Quan Dang, Chen Zhang, Sai Ho Ling
Computer Vision
DynaPPI: A Large-scale Dynamic Protein Dataset for AI-driven Advances in Protein Interactomics
Jiabao Wei, Zilong Geng, Yuze Wang et al. (+5 more)
Diffusion models have been widely explored in
protein backbone generation due to their powerful generation capabilities.However, in today's AI-driven biological research, predicting the structure of unknown multi-chain
protein aggregates (called "complexes" in biology) remains an unsolved challenge.This is because existing static or dynamic
protein datasets focus solely on static snapshots or single-entity trajectories, neglecting the dynamic process of multiple monomers forming complexes.To alleviate this dilemma, we present DynaPPI, a dynamic
protein dataset comprising molecular dynamics (MD) trajectories of
protein complex formation from dissociated chains to the bound state, as a pivotal resource to bridge the gap between static structural biology and the inherently temporal nature of dynamic molecular interactions.Benefiting from this
dataset,
diffusion models can explicitly learn the dynamic binding trajectories of known complexes and accurately predict the structures of unknown complexes based on their diverse generative properties, thereby further catalyzing AI-driven structural biology and
protein interactomics.
Read the full abstract ▾
All authors Jiabao Wei, Zilong Geng, Yuze Wang, Jianjun Li, Ning Ding, Bowen Zhou, Bing Zhang, Zhiyuan Ma
Computer Vision
Lesion-Aware Adaptive Fourier Neural Operator for CT-to-PSMA PET Synthesis in Prostate Cancer
Rashmi Bhaskara, Waleed M. Almutairi, Matthew Gopaulchan et al. (+5 more)
Deep learning models that synthesize PET from CT or MRI can reduce patient dose and scanner demand, but are typically optimized with global losses such as L1 or mean squared error (MSE) that treat all voxels similarly. In whole-body PSMA-PET, tumor voxels occupy only a small fraction of the volume, yet carry the clinically relevant activity signal; as a result, models can achieve high structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR) while still underestimating lesion activity or failing to preserve tumor-specific structure. Radiomics provides biologically meaningful descriptors of tumor intensity and texture, but direct radiomics conditioning is time-consuming because it requires feature extraction from delineated lesion regions. We propose LAFNO, a Lesion-Aware Adaptive
Fourier Neural Operator for CT-to-PSMA-PET synthesis that replaces high-dimensional radiomics conditioning with two efficient CT-derived proxy channels. Motivated by radiomics analysis of PSMA-avid tumor core and peritumoral regions, LAFNO uses a contrast proxy for local density variation and a disorder proxy for local texture heterogeneity, both injected into the model bottleneck. LAFNO combines whole-volume reconstruction with lesion-level total lesion activity (TLA), tumor-core contrast, and peritumoral supervision. We evaluated LAFNO against four baseline architectures on the TCIA PSMA-PET-CT-Lesions
dataset. LAFNO remained competitive on whole-volume image quality, achieving SSIM of 0.960 and 0.938 for 18F- and 68Ga-PSMA, respectively, while reducing per-patient TLA error to 48.3% and 64.0% for 18F- and 68Ga-PSMA, respectively, and achieving the highest tumor-core radiomics reproducibility across all feature classes for both tracers. Peritumoral reproducibility remained tracer-dependent, indicating that biological fidelity in synthetic PSMA-PET remains challenging.
Read the full abstract ▾
All authors Rashmi Bhaskara, Waleed M. Almutairi, Matthew Gopaulchan, Maram Musaad Alqurashi, Francis Asamoah, Alex Ocana, Clinton D. Bahler, Oluwaseyi M. Oderinde
Computer Vision
GeoSeg-OV: Bridging Geospatial Gaps with Structural Guidance for Open-Vocabulary Remote Sensing Segmentation
Ruizhong Liu, Tingzhang Luo, Zaiyan Zhang et al. (+4 more)
Open-vocabulary remote sensing
segmentation has recently emerged as a promising paradigm that enables pixel-level recognition of arbitrary categories specified by natural
language, including classes unseen during training. However, geospatial domain shifts caused by heterogeneous regions, spatial resolutions, and acquisition platforms weaken visual-text matching and limit cross-
dataset generalization. Recent attempts have begun to incorporate auxiliary
vision foundation models (VFMs), typically coupling their features with text
embeddings as additional matching evidence. However, this strategy may introduce inconsistent matching signals while leaving the structure-sensitive representations of VFMs insufficiently exploited. We therefore propose GeoSeg-OV, which decouples auxiliary VFM features from visual-text matching and repurposes them as structural guidance for cost aggregation and decoding. GeoSeg-OV constructs an orientation-robust cost volume from multi-rotation CLIP features, while a frozen VFM extracts multi-scale structure-sensitive features in parallel. We propose Structure-Guided Aggregation (SGA), which integrates cost
tokens and CLIP semantic guidance with VFM-derived pairwise structural biases for coherent spatial propagation, followed by text-conditioned class-wise
reasoning. We further introduce Cost-Aware Decoding (CAD) to adaptively refine and fuse multi-scale semantic and structural guidance based on the current decoder context. On the global High-Resolution Land Cover (HRLC)
benchmark spanning seven
datasets across six continents, GeoSeg-OV outperforms the state-of-the-art by +2.5 and +2.7 average mIoU under two training settings. A large-scale zero-shot case study further demonstrates its generalization across geographic domains and category systems without target-domain annotations or retraining.
Read the full abstract ▾
All authors Ruizhong Liu, Tingzhang Luo, Zaiyan Zhang, Jundong Chen, Hongruixuan Chen, Shaoguang Huang, Hongyan Zhang
Computer Vision
DriveVLA-M0: Failure-Aware Memory Augmentation for Autonomous Driving
Zebin Xing, Yupeng Zheng, Qiang Chen et al. (+10 more)
Vision-
Language-Action (VLA) models have recently emerged as a promising paradigm for end-to-end autonomous driving by enabling unified
reasoning across perception,
language, and planning. However, existing approaches lack mechanisms to exploit past failures or adapt to distribution shifts, causing the model to persistently underperform on similar scenarios where it has previously failed. In this paper, we propose DriveVLA-M0, a
retrieval-augmented VLA with failure-aware latent
memory. We construct a latent
memory pool that stores failure cases along with their structure scene representations and expert trajectory labels, and design a dedicated Retrieve Model that decouples static road structure and dynamic
agent interactions to enable structurally grounded
retrieval. At inference time, retrieved cases are injected into the model via a lightweight decoupled LoRA-based test-time training (TTT) mechanism, allowing targeted and scenario-specific correction without modifying the backbone. Extensive experiments on NAVSIMv1 and NAVSIMv2
benchmark demonstrate that our approach consistently outperforms prior methods, achieving 94.1 PDMS on Navtest and 47.0 EPDMS on Navhard with only 26.44 ms TTT backward latency overhead. Furthermore, we show that DriveVLA-M0 scales effectively with additional
memory, enabling training-free performance gains through
memory expansion. The code is available at https://github.com/ZebinX/DriveVLA-M0.
Read the full abstract ▾
All authors Zebin Xing, Yupeng Zheng, Qiang Chen, Linbo Wang, Yichen Zhang, Pengxuan Yang, Junli Wang, Deheng Qian, Xiaoqing Ye, Junyu Han, Yifeng Pan, Qichao Zhang, Dongbin Zhao
Computer Vision
A second-order theory of texture for depth from focus
Sreekar Ranganathan and Ioannis Gkioulekas
We present a theory of textured appearance of optically rough surfaces based on wave optics, emphasizing the role of texture for passive depth from focus. Our theory shows that even surfaces that traditional computer
vision would consider textureless can produce textured appearance, due to subjective speckle from surface microgeometry. We analyze the properties of this second-order texture, and show that we can enhance its contrast under natural ambient lighting by simply using a narrowband spectral filter. Doing so results in dramatic improvements in passive depth reconstruction of seemingly textureless scenes, as we demonstrate through extensive theory, simulations, and real-world experiments.
Read the full abstract ▾
Computer Vision
FormStruct-Bench:A Hierarchical and Diagnostic Benchmark for Table-Form Document Structure Recognition
Lujie Ban, Jiangtao Zhu, Yuanheng Yu et al. (+2 more)
Transforming table-form documents into machine-processable records requires recovering not only their visible content but also the multilevel structure that organizes it. However, existing
benchmarks evaluate either holistic document outputs or conventional table grids, and their aggregate scores provide little insight into where structural failures occur. We introduce FormStruct-Bench, a hierarchical and diagnostic
benchmark that evaluates table-form document structure recognition at both the document level and progressively finer component levels, allowing aggregate performance to be traced back to specific structural failure modes. To construct auditable ground truth at scale, we annotate 70 reusable templates and expand them into 7,000 verified instances through a provenance-preserving Director–Artist–Verifier pipeline; all 1,100 instances in the template-disjoint test set additionally receive human review. Our evaluation protocol uses five primary metrics and three structure-specific diagnostics across page, schema, and component levels, together with slices over difficulty, structural constraints, and visual degradation. Across 14 API-hosted and locally deployable systems plus two SFT variants, the best document-level score reaches 83.85%, whereas the best reported fine-grained structural score remains below 18%. These results reveal a pronounced gap between reading document content and recovering the hierarchy and regional organization required for reliable table-form understanding.
Read the full abstract ▾
All authors Lujie Ban, Jiangtao Zhu, Yuanheng Yu, Jiasheng Shi, Chenhao Ma
Computer Vision · Cross-listed from cs.LG
Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation
Gongli Zhang, Zhulin Liu and C. L. Philip Chen
Mixture-of-experts (MoE) models have recently moved beyond routing a fixed number of complete experts. Shared-expert designs preserve reusable knowledge, fine-grained methods vary computation within experts, and dynamic routers adapt the number of active experts. Yet these decisions are usually made independently, overlooking a basic dependency: extracting reusable computation changes both what remains and how much expert capacity the remainder needs. We study this dependency by decomposing sparsely upcycled feed-forward experts into key-value channels. Co-activated experts align at a subset of value positions; removing these positions changes expert preference; and greater shared coverage is associated with lower residual expert demand. These observations lead to one principle: share first, then route what remains. We instantiate it in UniF-MoE, a unified framework for
token-adaptive MoE computation. Each expert is partitioned into aligned blocks. A shared-demand score sets the shared block count and pathway weight, key prototypes select the shared content, and the complementary demand determines the residual expert count through cumulative routing mass. A Gram regularizer separates and normalizes router
embeddings, promoting diverse routing directions, sparse expert overlap, and a simple routing geometry. Experiments on DomainBed and GLUE show that this unified design improves predictive performance over representative static and dynamic MoEs while reducing activated computation, inference latency, and
memory. Code is available at https://github.com/existence0420/UniF-MoE.
Read the full abstract ▾
Computer Vision
Towards Unified Dynamic Face Landmark Detection
Sebastian Regalado, Varshanth R. Rao, Ruowei Jiang et al. (+2 more)
Although advancements in face landmark detection (FLD) methods continue to push performance boundaries, they overlook two major functional limitations: (1) different network parameters need to be trained independently for each “$N$-point”
benchmark dataset, and (2) a model trained on an “$N$-point”
dataset reliably outputs only the $N$ landmarks. In our work, we first conceptualize Face Part-Anchored Landmark Positions (FPALPs), wherein each landmark is treated as a progression value between zero (start) and one (end) along a face part's contour. Every landmark can be expressed in the FPALP format, irrespective of its source
dataset, hence unlocking the ability to unify all “$N$-point”
datasets into a single
dataset. Secondly, we represent each landmark with an FPALP-based query, refine it progressively with a cross-modality decoder, and predict its coordinates based on the final representation. Our approach, called Unified Dynamic FLD, embodies these two design choices and streamlines the landmark detection pipeline by enabling (1) a single model to learn on any number of “$N$-point”
datasets, and (2) yield any number of specific landmark predictions by loading the designated landmark queries at runtime. Extensive experiments on multiple
benchmark datasets show that our method delivers these benefits while remaining competitive with, and in several cases outperforming existing state-of-the-art methods.
Read the full abstract ▾
All authors Sebastian Regalado, Varshanth R. Rao, Ruowei Jiang, Parham Aarabi, Igor Gilitschenski
Computer Vision
CasDeblurGS: Cascaded 2D-to-3D Multi-View Consistency for 3D Gaussian Splatting from Two Blurry Images
Haeyun Choi, Minhyuk Jang and I-Gil Kim
Free-viewpoint 3D scene media is increasingly important for immersive applications, yet practical capture often suffers from severe view sparsity and motion blur. Although neural rendering has advanced sparse-view synthesis, existing blur-aware methods typically require substantial multi-view redundancy, accurate camera poses, or costly per-scene optimization. We address a stringent yet practical setting: reconstructing a coherent 3D scene from only two motion-blurred images with known intrinsics, without input-view poses, auxiliary sharp images, or per-scene test-time optimization. To this end, we propose CasDeblurGS, a cascaded framework that progressively recovers reliable cross-view information from local 2D correspondences to global 3D guidance. Stage 1 constructs locally reliable guidance through occlusion-aware correspondence filtering, while Stage 2 aggregates the intermediate restorations into a provisional pose-free 3D Gaussian representation whose input-view re-renders provide dense global guidance for final restoration. The resulting views enable a more coherent 3D representation and higher-quality novel-view synthesis. Experiments on real-world and synthetic Deblur-NeRF scenes show consistent gains over strong baselines, improving PSNR by 1.19 dB and 2.11 dB, respectively. Progressive ablations, cross-view correspondence visualization, and camera reprojection analysis further demonstrate improvements in both rendering quality and multi-view
geometric consistency.
Read the full abstract ▾
Computer Vision
Efficient Noise Context-Aware Representation for Low-Dose CT Denoising
Minwoo Yu, N. Robert Bennett, Jongduk Baek et al. (+1 more)
While deep learning-based denoising has become widely adopted in low-dose CT, conventional models use generic architectures designed for natural images, failing to account for non-stationary and spatially correlated CT noise characteristics. To address this, we propose an Efficient Noise COntext-aware REpresentation (ENCORE) framework that explicitly leverages CT noise characteristics and anatomical features. First, we reformulate the noise synthesis procedure based on a realistic noise distribution beyond the conventional Gaussian approximation, establishing a rigorous foundation for training pair generation. Next, we extract local noise power and correlation contexts to guide the denoising process. To fully leverage the potential of noise context, we propose a FlyingConv module, which adaptively changes convolution weights for each local image region. Notably, our approach demonstrates substantial gains in both denoising quality and computational efficiency. Furthermore, manipulating the intensity of the noise context maps at inference time enables zero-shot conditional denoising, allowing for dynamic control over the output image texture. The entire pipeline is available at https://github.com/minwoo-yu/ENCORE.git
Read the full abstract ▾
All authors Minwoo Yu, N. Robert Bennett, Jongduk Baek, Adam S. Wang