Computer Vision
MeInTime: Bridging Age Gap in Identity-Preserving Face Restoration
Teer Song, Yue Zhang, Yu Tian et al. (+6 more)
To better preserve an individual's identity, face restoration has evolved from reference-free to reference-based approaches, which leverage high-quality reference images of the same identity to enhance identity fidelity in the restored outputs. However, most existing methods implicitly assume that the reference and degraded input are age-aligned, limiting their effectiveness in real-world scenarios where only cross-age references are available, such as historical photo restoration. This paper proposes MeInTime, a
diffusion-based face restoration method that extends reference-based restoration from same-age to cross-age settings. Given one or few reference images along with an age
prompt corresponding to the degraded input, MeInTime achieves faithful restoration with both identity fidelity and age consistency. Specifically, we decouple the modeling of identity and age conditions. During training, we focus solely on effectively injecting identity features through a newly introduced
attention mechanism and introduce Gated Residual Fusion modules to facilitate the integration between degraded features and identity representations. At inference, we propose Age-Aware Gradient Guidance, a training-free sampling strategy, using an age-driven direction to iteratively nudge the identity-aware denoising latent toward the desired age semantic
manifold. Extensive experiments demonstrate that MeInTime outperforms existing face restoration methods in both identity preservation and age consistency. Our code is available at: https://github.com/teer4/MeInTime
Read the full abstract ▾
All authors Teer Song, Yue Zhang, Yu Tian, Ziyang Wang, Xianlin Zhang, Guixuan Zhang, Xuan Liu, Xueming Li, Yasen Zhang
Computer Vision
OrthoPhys: Physically Plausible Video Generation with Orthogonal-View Geometry Guidance
Cong Wang, Hanxin Zhu, Xiao Tang et al. (+4 more)
Recent progress in
video generation has led to substantial improvements in visual fidelity, yet ensuring physically consistent motion remains a fundamental challenge. Intuitively, this limitation can be attributed to the fact that real-world object motion unfolds in three-dimensional space, while
video observations provide only partial, view-dependent projections of such dynamics. To address these issues, we propose OrthoPhys, a two-stage framework that leverages orthogonal-view geometry guidance to enforce physical plausibility. Instead of directly generating unstructured 2D
videos, our first stage generates synchronized, four-view orthogonal
videos of the foreground dynamics. By incorporating a geometry-enhanced
attention mechanism across these orthogonal views, this stage effectively enforces 3D spatial coherence and implicitly grounds the motion in physical attributes. In the second stage, these physically consistent orthogonal foregrounds serve as rigid guidance to synthesize the final complete
video, seamlessly learning the interaction between foreground dynamics and the background context. To support this orthogonal-view training paradigm, we construct PhysMV, a
dataset containing 40K scenes, each consisting of four orthogonal viewpoints, resulting in a total of 160K
video sequences. Extensive experiments demonstrate that OrthoPhys significantly improves physical realism and spatial-temporal coherence over existing
video generation methods. Project page: https://anonymous.4open.science/w/Phys4D/.
Read the full abstract ▾
All authors Cong Wang, Hanxin Zhu, Xiao Tang, Jiayi Luo, Xin Jin, Long Chen, Zhibo Chen
Computer Vision
Attention Sparsity is Input-Stable: Training-Free Sparse Attention for Video Generation via Offline Sparsity Profiling and Online QK Co-Clustering
Jiayi Luo, Jiayu Chen, Jiankun Wang et al. (+6 more)
Diffusion Transformers (DiTs) achieve strong
video generation quality but suffer from high inference cost due to dense 3D
attention, motivating sparse
attention techniques for improving efficiency. However, existing training-free sparse
attention methods for
video generation still face two unresolved limitations: ignoring layer heterogeneity in
attention pruning and ignoring query-key coupling in block partitioning, which hinder a better quality-speedup trade-off. In this work, we uncover a critical insight:
attention sparsity is an intrinsic layer-wise property, with only minor variation across different inputs. Motivated by this observation, we propose SVOO, a training-free sparse
attention framework for fast
video generation via offline layer-wise sparsity profiling and online bidirectional co-clustering. Specifically, SVOO adopts a two-stage paradigm: (i) offline layer-wise sensitivity profiling to derive intrinsic per-layer
pruning levels, and (ii) online block-wise sparse
attention via a bidirectional co-clustering
algorithm. Extensive experiments on seven widely used
video generation models demonstrate that SVOO achieves a superior quality-speedup trade-off over state-of-the-art methods, delivering up to 1.93x speedup while maintaining a PSNR of up to 29 dB on Wan2.1. Code is available at: https://github.com/Mutual-Luo/SVOO.
Read the full abstract ▾
All authors Jiayi Luo, Jiayu Chen, Jiankun Wang, Cong Wang, Hanxin Zhu, Qingyun Sun, Chen Gao, Zhibo Chen, Jianxin Li
Computer Vision
SwiftGS: Episodic Priors for Immediate Satellite Surface Recovery
Rong Fu, Jiekai Wu, Xiaowen Ma et al. (+4 more)
Rapid, large-scale 3D reconstruction from multi-date satellite imagery is vital for environmental monitoring, urban planning, and disaster response, yet remains difficult due to illumination changes, sensor heterogeneity, and the cost of per-scene optimization. We introduce SwiftGS, a meta-learned system that reconstructs 3D surfaces in a single forward pass by predicting geometry-radiation-decoupled Gaussian primitives together with a lightweight SDF, replacing expensive per-scene fitting with episodic training that captures transferable priors. The model couples a differentiable physics graph for projection, illumination, and sensor response with spatial gating that blends sparse Gaussian detail and global SDF structure, and incorporates semantic-
geometric fusion, conditional lightweight task heads, and multi-view supervision from a frozen
geometric teacher under an uncertainty-aware multi-task loss. At inference, SwiftGS operates zero-shot with optional compact calibration and achieves accurate DSM reconstruction and view-consistent rendering at significantly reduced computational cost, with ablations highlighting the benefits of the hybrid representation, physics-aware rendering, and episodic meta-training.
Read the full abstract ▾
All authors Rong Fu, Jiekai Wu, Xiaowen Ma, Shiyin Lin, Kangan Qian, Chuang Liu, Simon James Fong
Computer Vision
GEAR: Geography-knowledge Enhanced Analog Recognition Framework in Extreme Environments
Zelin Liu, Bocheng Li, Yuling Zhou et al. (+5 more)
The Mariana Trench and the Qinghai-Tibet Plateau exhibit significant similarities in geological origins and microbial metabolic functions. Given that deep-sea biological sampling faces prohibitive costs, recognizing structurally homologous terrestrial analogs of the Mariana Trench on the Qinghai-Tibet Plateau is of great significance. Yet, no existing model adequately addresses cross-domain topographic similarity
retrieval, either neglecting geographical knowledge or sacrificing computational efficiency. To address these challenges, we present \underline{G}eography-knowledge \underline{E}nhanced \underline{A}nalog \underline{R}ecognition (GEAR) Framework, a three-stage pipeline designed to efficiently retrieve analogs from 2.5 million square kilometers of the Qinghai-Tibet Plateau: (1) Skeleton guided Screening and Clipping: Recognition of candidate valleys and initial screening based on size and linear morphological criteria. (2) Physics aware Filtering: The Topographic Waveform Comparator (TWC) and Morphological Texture Module (MTM) evaluate the waveform and texture and filter out inconsistent candidate valleys. (3) Graph based Fine Recognition: We design a \underline{M}orphology-integrated \underline{S}iamese \underline{G}raph \underline{N}etwork (MSG-Net) based on geomorphological metrics. Correspondingly, we release an expert-annotated topographic similarity
dataset targeting tectonic collision zones. Experiments demonstrate the effectiveness of every stage. Besides, MSG-Net achieved an F1-Score 1.38 percentage points higher than the SOTA baseline. Using features extracted by MSG-Net, we discovered a significant correlation with biological data, providing evidence for future biological analysis.
Read the full abstract ▾
All authors Zelin Liu, Bocheng Li, Yuling Zhou, Xuanting Li, Yixuan Yang, Jing Wang, Weishu Zhao, Xiaofeng Gao
Computer Vision
GenVideoLens: Where LVLMs Fall Short in AI-Generated Video Detection?
Yueying Zou, Pei Pei Li, Zekun Li et al. (+4 more)
In recent years, AI-generated
videos have become increasingly realistic and sophisticated. Meanwhile, Large
Vision-
Language Models (LVLMs) have shown strong potential for detecting such content. However, existing evaluation protocols largely treat the task as a binary classification problem and rely on coarse-grained metrics such as overall accuracy, providing limited insight into where LVLMs succeed or fail. To address this limitation, we introduce GenVideoLens, a fine-grained
benchmark that enables dimension-wise evaluation of LVLM capabilities in AI-generated
video detection. The
benchmark contains 400 highly deceptive AI-generated
videos and 100 real
videos, annotated by experts across 15 authenticity dimensions covering perceptual, optical, physical, and temporal cues. We evaluate eleven representative LVLMs on this
benchmark. Our analysis reveals a pronounced dimensional imbalance. While LVLMs perform relatively well on perceptual cues, they struggle with optical consistency, physical interactions, and temporal-causal
reasoning. Model performance also varies substantially across dimensions, with smaller open-source models sometimes outperforming stronger proprietary models on specific authenticity cues. Temporal perturbation experiments further show that current LVLMs make limited use of temporal information. Overall, GenVideoLens provides diagnostic insights into LVLM behavior, revealing key capability gaps and offering guidance for improving future AI-generated
video detection systems.
Read the full abstract ▾
All authors Yueying Zou, Pei Pei Li, Zekun Li, Xinyu Guo, Xing Cui, Huaibo Huang, Ran He
Computer Vision · Cross-listed from cs.RO
REST: Receding Horizon Explorative Steiner Tree for Zero-Shot Object-Goal Navigation
Shuqi Xiao, Maani Ghaffari, Chengzhong Xu et al. (+1 more)
Zero-shot object-goal
navigation (ZSON) requires navigating unknown environments to find a target object without task-specific training. Prior hierarchical solutions mainly focus on either scene understanding and representations (belief) or high-level decision-making and planning (policy), yet treat the option, i.e., the subgoal candidate that belief proposes and policy selects, as an interface inherited from adjacent modules rather than a design axis in its own right. In practice, options are predominantly single waypoints scored by destination utility: a lone destination hides the value gathered en route, and a flat list obscures the relationships among candidates. Our insight is that the option space should be a tree of paths. Full paths expose en-route information gain that destination-only scoring systematically neglects; a tree of shared segments enables coarse-to-fine LLM
reasoning that dismisses or pursues entire branches before examining individual leaves, compressing the combinatorial path space into an efficient hierarchy. We instantiate this insight in REST (Receding Horizon Explorative Steiner Tree), a training-free framework that (1) builds an explicit open-vocabulary 3D map from online RGB-D streams; (2) grows an
agent-centric tree of safe and informative paths as the option space via sampling-based planning; and (3) textualizes each branch into a spatial narrative and selects the next-best path through chain-of-thought LLM
reasoning. Across the Gibson, HM3D, and HSSD
benchmarks, REST consistently ranks among the top methods in success rate and path efficiency.
Read the full abstract ▾
All authors Shuqi Xiao, Maani Ghaffari, Chengzhong Xu, Hui Kong
Computer Vision
OpenT2M: No-frill Motion Generation with Open-source,Large-scale, High-quality Data
Bin Cao, Sipeng Zheng, Hao Luo et al. (+3 more)
Text-to-motion (T2M) generation aims to create realistic human movements from text descriptions, with promising applications in animation and robotics. Despite recent progress, current T2M models perform poorly on unseen text descriptions due to the small scale and limited diversity of existing motion
datasets. To address this problem, we introduce OpenT2M, a million-level, high-quality, and open-source motion
dataset containing over 2800 hours of human motion. Each sequence undergoes rigorous quality control through physical feasibility validation and multi-granularity filtering, with detailed second-wise text annotations. We also develop an automated pipeline for creating long-horizon sequences, enabling complex motion generation. Building upon OpenT2M, we introduce MonoFrill, a pretrained motion model that achieves compelling T2M results without complicated designs or technique tricks as "frills". Its core component is 2D-PRQ, a novel motion tokenizer that captures spatiotemporal dependencies by dividing the human body into biology parts. Experiments show that OpenT2M significantly improves generalization of existing T2M models, while 2D-PRQ achieves superior reconstruction and strong zero-shot performance. We expect OpenT2M and MonoFrill will advance the T2M field by addressing longstanding data quality and benchmarking challenges.
Read the full abstract ▾
All authors Bin Cao, Sipeng Zheng, Hao Luo, Boyuan Li, Jing Liu, Zongqing Lu
Computer Vision
Benchmarking CNN-based Models against Transformer-based Models for Abdominal Multi-Organ Segmentation on the RATIC Dataset
Lukas Bayer, Sheethal Bhat and Andreas Maier
Accurate multi-organ
segmentation in abdominal CT scans is essential for computer-aided diagnosis and treatment. While convolutional neural networks (CNNs) have long been the standard approach in medical image
segmentation,
transformer-based architectures have recently gained
attention due to their ability to model long-range dependencies. In this study, we systematically
benchmark the three hybrid
transformer-based models UNETR, SwinUNETR, and UNETR++ against a strong CNN baseline, SegResNet, for volumetric multi-organ
segmentation on the heterogeneous RATIC
dataset. The
dataset comprises 206 annotated CT scans from 23 institutions worldwide, covering five abdominal organs. All models were trained and evaluated under identical preprocessing and training conditions using the Dice Similarity Coefficient (DSC) as the primary metric. The results show that the CNN-based SegResNet achieves the highest overall performance, outperforming all hybrid
transformer-based models across all organs. Among the
transformer-based approaches, UNETR++ delivers the most competitive results, while UNETR demonstrates notably faster convergence with fewer training iterations. These findings suggest that, for small- to medium-sized heterogeneous
datasets, well-optimized CNN architectures remain highly competitive and may outperform hybrid
transformer-based designs.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Cross-Modal Rationale Transfer for Explainable Humanitarian Classification on Social Media
Thi Huyen Nguyen, Koustav Rudra and Wolfgang Nejdl
Advances in social media data dissemination enable the provision of real-time information during a crisis. The information comes from different classes, such as infrastructure damages, persons missing or stranded in the affected zone, etc. Existing methods attempted to classify text and images into various humanitarian categories, but their decision-making process remains largely opaque, which affects their deployment in real-life applications. Recent work has sought to improve transparency by extracting textual rationales from tweets to explain predicted classes. However, such explainable classification methods have mostly focused on text, rather than crisis-related images. In this paper, we propose an interpretable-by-design
multimodal classification framework. Our method first learns the joint representation of text and image using a visual
language transformer model and extracts text rationales. Next, it extracts the image rationales via the mapping with text rationales. Our approach demonstrates how to learn rationales in one modality from another through cross-modal rationale transfer, which saves annotation effort. Finally, tweets are classified based on extracted rationales. Experiments are conducted over CrisisMMD
benchmark dataset, and results show that our proposed method boosts the classification Macro-F1 by 2-35% while extracting accurate text
tokens and image patches as rationales. Human evaluation also supports the claim that our proposed method is able to retrieve better image rationale patches (12%) that help to identify humanitarian classes. Our method adapts well to new, unseen
datasets in zero-shot mode, achieving an accuracy of 80%.
Read the full abstract ▾
Computer Vision
Improving Joint Audio-Video Generation with Cross-Modal Context Learning
Bingqi Ma, Linlong Lang, Ming Zhang et al. (+5 more)
The dual-stream
transformer architecture-based joint audio-
video generation method has become the dominant paradigm in current research. By incorporating pre-trained
video diffusion models and audio
diffusion models, along with a cross-modal interaction
attention module, high-quality, temporally synchronized audio-
video content can be generated with minimal training data. In this paper, we first revisit the dual-stream
transformer paradigm and further analyze its limitations, including model
manifold variations caused by the gating mechanism controlling cross-modal interactions, biases in multi-modal background regions introduced by cross-modal
attention, and the inconsistencies in multi-modal classifier-free guidance (CFG) during training and inference, as well as conflicts between multiple conditions. To alleviate these issues, we propose Cross-Modal Context Learning (CCL), equipped with several carefully designed modules. Temporally Aligned RoPE and Partitioning (TARP) effectively enhances the temporal
alignment between audio latent and
video latent representations. The Learnable Context
Tokens (LCT) and Dynamic Context Routing (DCR) in the Cross-Modal Context
Attention (CCA) module provide stable unconditional anchors for cross-modal information, while dynamically routing based on different training tasks, further enhancing the model's convergence speed and generation quality. During inference, Unconditional Context Guidance (UCG) leverages the unconditional support provided by LCT to facilitate different forms of CFG, improving train-inference consistency and further alleviating conflicts. Through comprehensive evaluations, CCL achieves state-of-the-art performance compared with recent academic methods while requiring substantially fewer resources.
Read the full abstract ▾
All authors Bingqi Ma, Linlong Lang, Ming Zhang, Dailan He, Xingtong Ge, Yi Zhang, Guanglu Song, Yu Liu
Computer Vision
SJD-PAC: Accelerating Speculative Jacobi Decoding via Proactive Drafting and Adaptive Continuation
Jialiang Kang, Han Shu, Wenshuo Li et al. (+2 more)
Speculative Jacobi Decoding (SJD) offers a draft-model-free approach to accelerate autoregressive text-to-image synthesis. However, the high-
entropy nature of visual generation yields low draft-
token acceptance rates in complex regions, creating a bottleneck that severely limits overall throughput. To overcome this, we introduce SJD-PAC, an enhanced SJD framework. First, SJD-PAC employs a proactive drafting strategy to improve local acceptance rates in these challenging high-
entropy regions. Second, we introduce an adaptive continuation mechanism that sustains sequence validation after an initial rejection, bypassing the need for full resampling. Working in tandem, these optimizations significantly increase the average acceptance length per step, boosting inference speed while strictly preserving the target distribution. Experiments on standard text-to-image
benchmarks demonstrate that SJD-PAC achieves a $3.8\times$ speedup with lossless image quality. Code is available at https://github.com/KangJialiang/SJD-PAC.
Read the full abstract ▾
All authors Jialiang Kang, Han Shu, Wenshuo Li, Yingjie Zhai, Xinghao Chen
Computer Vision
Complementary Text-Guided Attention for Zero-Shot Adversarial Robustness
Lu Yu, Haiyang Zhang and Changsheng Xu
Due to the impressive zero-shot capabilities, pre-trained
vision-
language models (e.g., CLIP), have attracted widespread
attention and adoption across various domains. Nonetheless, CLIP has been observed to be susceptible to
adversarial examples. Through experimental analysis, we have observed a phenomenon wherein
adversarial perturbations induce shifts in text-guided
attention. Building upon this observation, we propose a simple yet effective strategy: Text-Guided
Attention for Zero-Shot Robustness (TGA-ZSR). This framework incorporates two components: Local
Attention Refinement Module and Global
Attention Constraint Module. Our goal is to maintain the generalization of the CLIP model and enhance its
adversarial robustness. Additionally, the Global
Attention Constraint Module acquires text-guided
attention from both the target and original models using clean examples. Its objective is to maintain model performance on clean samples while enhancing overall robustness. However, we observe that the method occasionally focuses on irrelevant or spurious features, which can lead to suboptimal performance and undermine its robustness in certain scenarios. To overcome this limitation, we further propose a novel approach called Complementary Text-Guided
Attention (Comp-TGA). This method integrates two types of foreground
attention:
attention guided by the class
prompt and reversed
attention driven by the non-class
prompt. These complementary
attention mechanisms allow the model to capture a more comprehensive and accurate representation of the foreground. The experiments validate that TGA-ZSR and Comp-TGA yield 9.58% and 11.95% improvements respectively, in zero-shot robust accuracy over the current state-of-the-art techniques across 16
datasets.
Read the full abstract ▾
Computer Vision
myMNIST: Benchmark of PETNN, KAN, and Classical Deep Learning Models for Burmese Handwritten Digit Recognition
Ye Kyaw Thu, Thazin Myint Oo and Thepchai Supnithi
We present the first systematic
benchmark on a standardized iteration of the publicly available Burmese Handwritten Digit
Dataset (BHDD), which we have designated as myMNIST Benchmarking. While BHDD serves as a foundational resource for Myanmar NLP/AI, it lacks a comprehensive, reproducible performance baseline across modern architectures. We evaluate eleven architectures spanning classical deep learning models (Multi-Layer Perceptron, Convolutional Neural Network, Long Short-Term
Memory, Gated Recurrent Unit,
Transformer), recent alternatives (FastKAN, EfficientKAN), an energy-based model (JEM), and physics-inspired PETNN variants (Sigmoid, GELU, SiLU). Using Precision, Recall, F1-Score, and Accuracy as evaluation metrics, our results show that the CNN remains a strong baseline, achieving the best overall scores (F1 = 0.9959, Accuracy = 0.9970). The PETNN (GELU) model closely follows (F1 = 0.9955, Accuracy = 0.9966), outperforming LSTM, GRU,
Transformer, and KAN variants. JEM, representing energy-based modeling, performs competitively (F1 = 0.9944, Accuracy = 0.9958). KAN-based models (FastKAN, EfficientKAN) trail the top performers but provide a meaningful alternative baseline (Accuracy ~0.992). These findings (i) establish reproducible baselines for BHDD across diverse modeling paradigms, (ii) highlight PETNN's strong performance relative to classical and
Transformer-based models, and (iii) quantify the gap between energy-inspired PETNNs and a true energy-based model (JEM). We release this
benchmark to facilitate future research on Myanmar digit recognition and to encourage broader evaluation of emerging architectures on regional scripts.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Elastic Weight Consolidation Done Right for Continual Learning
Xuan Liu and Xiaobin Chang
Weight regularization methods in continual learning (CL) alleviate catastrophic forgetting by assessing and penalizing changes to important model weights. Elastic Weight Consolidation (EWC) is a foundational and widely used approach within this framework that estimates weight importance based on gradients. However, it has consistently shown suboptimal performance. In this paper, we conduct a systematic analysis of importance estimation in EWC from a gradient-based perspective. For the first time, we find that EWC's reliance on the Fisher Information
Matrix (FIM) results in gradient vanishing and inaccurate importance estimation in certain scenarios. Our analysis also reveals that
Memory Aware Synapses (MAS), a variant of EWC, imposes unnecessary constraints on parameters irrelevant to prior tasks, termed the redundant protection. Consequently, both EWC and its variants exhibit fundamental misalignments in estimating weight importance, leading to inferior performance. To tackle these issues, we propose the Logits Reversal (LR) operation, a simple yet effective modification that rectifies EWC's importance estimation. Specifically, reversing the logit values during the calculation of FIM can effectively prevent both gradient vanishing and redundant protection. Extensive experiments across various CL tasks and
datasets show that the proposed method significantly outperforms existing EWC and its variants. Therefore, we refer to it as EWC Done Right (EWC-DR). Code is available at https://github.com/scarlet0703/EWC-DR.
Read the full abstract ▾
Computer Vision
A Novel FACS-Aligned Anatomical Text Description Paradigm for Fine-Grained Facial Behavior Synthesis
Jiahe Wang, Cong Liang, Xuandong Huang et al. (+5 more)
Facial behavior constitutes the primary medium of human nonverbal communication. Existing synthesis methods predominantly follow two paradigms: coarse
emotion category labels or one-hot Action Unit (AU) vectors from the Facial Action Coding System (FACS). Neither paradigm reliably renders fine-grained facial behaviors nor resolves anatomically implausible artifacts caused by conflicting AUs. Therefore, we propose a novel task paradigm: anatomically grounded facial behavior synthesis from FACS-based AU descriptions. This paradigm explicitly encodes FACS-defined muscle movement rules, inter-AU interactions, and conflict resolution mechanisms into natural
language control signals. To enable systematic research, we develop a dynamic AU text processor, a FACS rule-based module that converts raw AU annotations into anatomically consistent natural
language descriptions. Using this processor, we construct BP4D-AUText, the first large-scale text-image paired
dataset for fine-grained facial behavior synthesis, comprising over 302K high-quality samples. Given that existing general semantic consistency metrics cannot capture the
alignment between anatomical facial descriptions and synthesized muscle movements, we propose the
Alignment Accuracy of AU Probability Distributions (AAAD), a task-specific metric that quantifies semantic consistency. Finally, we design VQ-AUFace, a robust baseline framework incorporating anatomical priors and progressive cross-modal
alignment, to validate the paradigm. Extensive quantitative experiments and user studies demonstrate the paradigm significantly outperforms state-of-the-art methods, particularly in challenging conflicting AU scenarios, achieving superior anatomical fidelity, semantic consistency, and visual quality.
Read the full abstract ▾
All authors Jiahe Wang, Cong Liang, Xuandong Huang, Yuxin Wang, Xin Yun, Yi Wu, Yanan Chang, Shangfei Wang
Computer Vision
Color image restoration based on nonlocal saturation-value similarity
Wei Wang and Yakun Li
In this paper, we propose and develop a novel nonlocal variational technique based on saturation-value similarity for color image restoration. In traditional nonlocal methods, image patches are extracted from red, green and blue channels of a color image directly, and the color information can not be described finely because the patch similarity is mainly based on the grayscale value of independent channel. The main aim of this paper is to propose and develop a novel nonlocal regularization method by considering the similarity of image patches in saturation-value channel of a color image. In particular, we first establish saturation-value similarity based nonlocal total variation by incorporating saturation-value similarity of color image patches into the proposed nonlocal gradients, which can describe the saturation and value similarity of two adjacent color image patches. The proposed nonlocal variational models are then formulated based on saturation-value similarity based nonlocal total variation. Moreover, we design an effective and efficient
algorithm to solve the proposed optimization problem numerically by employing bregmanized operator splitting method, and we also study the convergence of the proposed
algorithms. Numerical examples are presented to demonstrate that the performance of the proposed models is better than that of other testing methods in terms of visual quality and some quantitative metrics including peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), quaternion structural similarity index (QSSIM) and S-CIELAB color error.
Read the full abstract ▾
Computer Vision
Swarnendu Banik, Manish Das, Shiv Ram Dubey et al. (+1 more)
Vision Transformers have excelled in computer
vision but their
attention mechanisms operate independently across layers, limiting information flow and feature learning. We propose an effective cross-layer
attention propagation method that preserves and integrates historical
attention matrices across encoder layers, offering a principled refinement of inter-layer information flow in
Vision Transformers. This approach enables progressive refinement of
attention patterns throughout the
transformer hierarchy, enhancing feature acquisition and optimization dynamics. The method requires minimal architectural changes, adding only
attention matrix storage and blending operations. Comprehensive experiments on CIFAR-100 and TinyImageNet demonstrate consistent accuracy improvements, with ViT performance increasing from 75.74% to 77.07% on CIFAR-100 (+1.33%) and from 57.82% to 59.07% on TinyImageNet (+1.25%). Cross-architecture validation shows similar gains across
transformer variants, with CaiT showing 1.01% enhancement. Systematic analysis identifies the blending hyperparameter of historical
attention (alpha = 0.45) as optimal across all configurations, providing the ideal balance between current and historical
attention information. Random initialization consistently outperforms zero initialization, indicating that diverse initial
attention patterns accelerate convergence and improve final performance. Our code is publicly available at https://github.com/banik-s/HAViT.
Read the full abstract ▾
All authors Swarnendu Banik, Manish Das, Shiv Ram Dubey, Satish Kumar Singh
Computer Vision · Cross-listed from eess.IV
UEPS: Robust and Efficient MRI Reconstruction
Xiang Zhou, Hong Shang, Zijian Zhan et al. (+3 more)
Deep unrolled models (DUM) have become the state-of-the-art for accelerated MRI reconstruction, yet their robustness under domain shift remains a critical barrier to clinical adoption. In this work, we identify coil sensitivity map (CSM) estimation as the primary bottleneck limiting generalization. To address this, we propose UEPS, a novel DUM architecture featuring three key designs: (i) an Unrolled Expanded (UE) design that eliminates CSM dependency by reconstructing each coil independently; (ii) progressive resolution, which leverages k-space-to-image mapping for efficient coarse-to-fine refinement; and (iii) sparse
attention tailored to MRI's 1D undersampling nature. These physics-grounded designs enable gains in robustness and computational efficiency. We construct a large-scale zero-shot transfer
benchmark comprising 10 out-of-distribution (OOD) test sets spanning diverse clinical shifts, anatomy, view, contrast, vendor, field strength, and coil configurations. Extensive experiments demonstrate that UEPS consistently and substantially outperforms existing DUM, end-to-end,
diffusion, and untrained methods across all OOD tests, achieving state-of-the-art robustness with low-latency inference suitable for real-time deployment.
Read the full abstract ▾
All authors Xiang Zhou, Hong Shang, Zijian Zhan, Tianyu He, Jintao Meng, Dong Liang
Computer Vision
CausalVAD: De-confounding End-to-End Autonomous Driving via Causal Intervention
Jiacheng Tang, Zhiyuan Zhou, Zhuolin He et al. (+3 more)
Planning-oriented end-to-end driving models show great promise, yet they fundamentally learn statistical correlations instead of true causal relationships. This vulnerability leads to causal confusion, where models exploit
dataset biases as shortcuts, critically harming their reliability and safety in complex scenarios. To address this, we introduce CausalVAD, a de-confounding training framework that leverages causal intervention. At its core, we design the sparse causal intervention scheme (SCIS), a lightweight, plug-and-play module to instantiate the
backdoor adjustment theory in neural networks. SCIS constructs a dictionary of prototypes representing latent driving contexts. It then uses this dictionary to intervene on the model's sparse vectorized queries. This step actively eliminates spurious associations induced by confounders, thereby eliminating spurious factors from the representations for downstream tasks. Extensive experiments on
benchmarks like nuScenes show CausalVAD achieves state-of-the-art planning accuracy and safety. Furthermore, our method demonstrates superior robustness against both data bias and noisy scenarios configured to induce causal confusion.
Read the full abstract ▾
All authors Jiacheng Tang, Zhiyuan Zhou, Zhuolin He, Jia Zhang, Kai Zhang, Jian Pu
Computer Vision
HiMu: Hierarchical Multimodal Frame Selection for Long Video Question Answering
Dan Ben-Ami, Gabriele Serussi, Kobi Cohen et al. (+1 more)
Long-form
video question answering requires
reasoning over extended temporal contexts, making frame selection a critical bottleneck for multi-modal large
language models (MLLMs) bound by finite context windows. Within the controlled frame-budget regime that governs practical deployment, prior selectors score frames against a single global query
embedding; as a result, compositional
multimodal questions that involve temporal ordering or cross-modal cues such as "what happens on screen right after the narrator mentions the reaction?" are flattened into a representation that loses sub-event ordering and modality bindings. We introduce HiMu, a training-free framework for compositional
multimodal frame selection. A single text-only LLM call decomposes the query into a hierarchical logic tree whose leaves are atomic predicates, each routed to a lightweight expert spanning
vision (CLIP, open-vocabulary detection, OCR) and audio (
speech recognition and non-
speech sound matching). Expert signals are normalized, smoothed to align across modalities, and composed bottom-up through fuzzy-logic operators that enforce temporal sequencing and adjacency, yielding a continuous per-frame satisfaction curve. Under the standard 16-frame budget on
Video-MME, LongVideoBench, and HERBench-Lite, HiMu achieves state-of-the-art accuracy among frame selection methods and improves over uniform sampling across seven diverse MLLMs as a drop-in module, matching the accuracy of uniform sampling at 4x its frame budget, without retraining and without multiple iterative MLLM calls during selection.
Read the full abstract ▾
All authors Dan Ben-Ami, Gabriele Serussi, Kobi Cohen, Chaim Baskin
Computer Vision · Cross-listed from quant-ph
End-to-End QGAN-Based Image Synthesis via Neural Noise Encoding and Intensity Calibration
Xue Yang, Rigui Zhou, Shizheng Jia et al. (+5 more)
Quantum Generative
Adversarial Networks (QGANs) offer a promising path for learning data distributions on near-term
quantum devices. However, existing QGANs for image synthesis avoid direct full-image generation, relying on classical post-processing or patch-based methods. These approaches dilute the
quantum generator's role and struggle to capture global image semantics. To address this, we propose ReQGAN, an end-to-end framework that synthesizes an entire N=2^D-pixel image using a single D-
qubit quantum circuit. ReQGAN overcomes two fundamental bottlenecks hindering direct pixel generation: (1) the rigid classical-to-
quantum noise interface and (2) the output mismatch between normalized
quantum statistics and the desired pixel-intensity space. We introduce a learnable Neural Noise Encoder for adaptive state preparation and a differentiable Intensity Calibration module to map measurements to a stable, visually meaningful pixel domain. Experiments on MNIST and Fashion-MNIST demonstrate that ReQGAN achieves stable training and effective image synthesis under stringent
qubit budgets, with ablation studies verifying the contribution of each component.
Read the full abstract ▾
All authors Xue Yang, Rigui Zhou, Shizheng Jia, Dax Enshan Koh, Siong Thye Goh, Yaochong Li, Hongyu Chen, Fuhui Xiong
Computer Vision
CoDA: Exploring Chain-of-Distribution Attacks and Post-Hoc Token-Space Repair for Medical Vision-Language Models
Xiang Chen, Fangfang Yang, Chunlei Meng et al. (+6 more)
Medical
vision–
language models (MVLMs) are increasingly used as perceptual backbones in radiology pipelines and as the visual front end of
multimodal assistants, yet their reliability under real clinical workflows remains underexplored. Prior robustness evaluations often assume clean, curated inputs or study isolated corruptions, overlooking routine acquisition, reconstruction, display, and delivery operations that preserve clinical readability while shifting image statistics. To address this gap, we propose CoDA, a chain-of-distribution framework that constructs clinically plausible pipeline shifts by composing acquisition-like shading, reconstruction and display remapping, and delivery and export degradations. Under masked structural-similarity constraints, CoDA jointly optimizes stage compositions and parameters to induce failures while preserving visual plausibility. Across
brain MRI, chest X-ray, and abdominal CT, CoDA substantially degrades the zero-shot performance of CLIP-style MVLMs, with chained compositions consistently more damaging than any single stage. We also evaluate
multimodal large
language models (MLLMs) as technical-authenticity auditors of imaging realism and quality rather than pathology. Proprietary
multimodal models show degraded
auditing reliability and persistent high-confidence errors on CoDA-shifted samples, while the medical-specific MLLMs we test exhibit clear deficiencies in medical image quality
auditing. Finally, we introduce a post-hoc repair strategy based on teacher-guided
token-space adaptation with patch-level
alignment, which improves accuracy on archived CoDA outputs. Overall, our findings characterize a clinically grounded threat surface for MVLM deployment and show that lightweight
alignment improves robustness in deployment.
Read the full abstract ▾
All authors Xiang Chen, Fangfang Yang, Chunlei Meng, Yuxian Dong, Ang Li, Yiwei Wei, Jiahuan Long, Jiujiang Guo, Chengyin Hu
Computer Vision · Cross-listed from eess.IV
SCISSR: Scribble-Conditioned Interactive Surgical Segmentation and Refinement
Haonan Ping, Jian Jiang, Cheng Yuan et al. (+3 more)
Accurate
segmentation of tissues and instruments in
surgical scenes is annotation-intensive due to irregular shapes, thin structures, specularities, and frequent occlusions. While SAM models support point, box, and mask
prompts, points are often too sparse and boxes too coarse to localize such challenging targets. We present SCISSR, a scribble-promptable framework for interactive
surgical scene
segmentation. It introduces a lightweight Scribble Encoder that converts freehand scribbles into dense
prompt embeddings compatible with the mask decoder, enabling iterative refinement for a target object by drawing corrective strokes on error regions. Because all added modules (the Scribble Encoder, Spatial Gated Fusion, and LoRA adapters) interact with the backbone only through its standard
embedding interfaces, the framework is not tied to a single model: we build on SAM 2 in this work, yet the same components transfer to other
prompt-driven
segmentation architectures such as SAM 3 without structural modification. To preserve pre-trained capabilities, we train only these lightweight additions while keeping the remaining backbone frozen. Experiments on EndoVis 2018 demonstrate strong in-domain performance, while evaluation on the out-of-distribution CholecSeg8k further confirms robustness across
surgical domains. SCISSR achieves 95.41% Dice on EndoVis 2018 with five interaction rounds and 96.30% Dice on CholecSeg8k with three interaction rounds, outperforming iterative point prompting on both
benchmarks.
Read the full abstract ▾
All authors Haonan Ping, Jian Jiang, Cheng Yuan, Qizhen Sun, Lv Wu, Yutong Ban
Computer Vision
Remedying Target-Domain Astigmatism for Cross-Domain Few-Shot Object Detection
Yongwei Jiang, Yixiong Zou, Yuhua Li et al. (+1 more)
Cross-domain few-shot object detection (CD-FSOD) aims to adapt pretrained detectors from a source domain to target domains with limited annotations, suffering from severe domain shifts and data scarcity problems. In this work, we find a previously overlooked phenomenon: models exhibit dispersed and unfocused
attention in target domains, leading to imprecise localization and redundant predictions, just like a human cannot focus on visual objects. Therefore, we call it the target-domain Astigmatism problem. Analysis on
attention distances across
transformer layers reveals that regular fine-tuning inherently shows a trend to remedy this problem, but results are still far from satisfactory, which we aim to enhance in this paper. Biologically inspired by the human fovea-style visual system, we enhance the fine-tuning's inherent trend through a center-periphery
attention refinement framework, which contains (1) a Positive Pattern Refinement module to reshape
attention toward semantic objects using class-specific prototypes, simulating the visual center region; (2) a Negative Context Modulation module to enhance boundary discrimination by modeling background context, simulating the visual periphery region; and (3) a Textual Semantic
Alignment module to strengthen center-periphery distinction through cross-modal cues. Our bio-inspired approach transforms astigmatic
attention into focused patterns, substantially improving adaptation to target domains. Experiments on six challenging CD-FSOD
benchmarks consistently demonstrate improved detection accuracy and establish new state-of-the-art results.
Read the full abstract ▾
All authors Yongwei Jiang, Yixiong Zou, Yuhua Li, Ruixuan Li
Computer Vision
3DreamBooth: High-Fidelity 3D Subject-Driven Video Generation Model
Hyun-kyu Ko, Jihyeon Park, Younghyun Kim et al. (+2 more)
Creating dynamic, view-consistent
videos of customized subjects is highly sought after for a wide range of emerging applications, including immersive VR/AR, virtual production, and next-generation e-commerce. However, despite rapid progress in subject-driven
video generation, existing methods predominantly treat subjects as 2D entities, focusing on transferring identity through single-view visual features or textual
prompts. Because real-world subjects are inherently 3D, applying these 2D-centric approaches to 3D object customization reveals a fundamental limitation: they lack the comprehensive spatial priors necessary to reconstruct the 3D geometry. Consequently, when synthesizing novel views, they must rely on generating plausible but arbitrary details for unseen regions, rather than preserving the true 3D identity. Achieving genuine 3D-aware customization remains challenging due to the scarcity of multi-view
video datasets. While one might attempt to fine-tune models on limited
video sequences, this often leads to temporal overfitting. To resolve these issues, we introduce a novel framework for 3D-aware
video customization, comprising 3DreamBooth and 3Dapter. 3DreamBooth decouples spatial geometry from temporal motion through a 1-frame optimization paradigm. By restricting updates to spatial representations, it effectively bakes a robust 3D prior into the model without the need for exhaustive
video-based training. To enhance fine-grained textures and accelerate convergence, we incorporate 3Dapter, a visual conditioning module. Following single-view pre-training, 3Dapter undergoes multi-view joint optimization with the main generation branch via an asymmetrical conditioning strategy. This design allows the module to act as a dynamic selective router, querying view-specific
geometric hints from a minimal reference set. Project page: https://ko-lani.github.io/3DreamBooth/
Read the full abstract ▾
All authors Hyun-kyu Ko, Jihyeon Park, Younghyun Kim, Dongheok Park, Eunbyung Park
Computer Vision
Counting Circuits: Mechanistic Interpretability of Visual Reasoning in Large Vision-Language Models
Liwei Che, Zhiyu Xue, Yihao Quan et al. (+7 more)
Counting serves as a simple but powerful test of a Large
Vision-
Language Model's (LVLM's)
reasoning; it forces the model to identify each individual object and then add them all up. In this study, we investigate how LVLMs implement counting using controlled synthetic and real-world
benchmarks, combined with mechanistic analyses. Our results show that LVLMs display a human-like counting behavior, with precise performance on small numerosities and noisy estimation for larger quantities. We introduce two novel interpretability methods, Visual Activation Patching and HeadLens, and use them to uncover a structured "counting
circuit" that is largely shared across a variety of visual
reasoning tasks. Building on these insights, we propose a lightweight intervention strategy that exploits simple and abundantly available synthetic images to fine-tune arbitrary pretrained LVLMs exclusively on counting. Despite the narrow scope of this fine-tuning, the intervention not only enhances counting accuracy on in-distribution synthetic data, but also yields an average improvement of +8.36% on out-of-distribution counting
benchmarks and an average gain of +1.54% on complex, general visual
reasoning tasks for Qwen2.5-VL. These findings highlight the central, influential role of counting in visual
reasoning and suggest a potential pathway for improving overall visual
reasoning capabilities through targeted enhancement of counting mechanisms.
Read the full abstract ▾
All authors Liwei Che, Zhiyu Xue, Yihao Quan, Benlin Liu, Zeru Shi, Michelle Hurst, Jacob Feldman, Ruixiang Tang, Ranjay Krishna, Vladimir Pavlovic
Computer Vision
CAFlow: Adaptive-Depth Single-Step Flow Matching for Efficient Histopathology Super-Resolution
Elad Yoshai, Ariel D. Yoshai and Natan T. Shaked
In digital pathology, whole-slide images routinely exceed gigapixel resolution, making computationally intensive generative super-resolution (SR) impractical for routine deployment. We introduce CAFlow, an adaptive-depth single-step flow-matching framework that routes each image tile to the shallowest network exit that preserves reconstruction quality. CAFlow performs flow matching in pixel-unshuffled rearranged space, reducing spatial computation by 16x while enabling direct inference. We show that dedicating half of training to exact t=0 samples is essential for single-step quality (-1.5 dB without it). The backbone, FlowResNet (1.90M parameters), mixes convolution and window self-
attention blocks across four early exits spanning 3.1 to 13.3 GFLOPs. A lightweight exit classifier (~6K parameters) achieves 33% compute savings at only 0.12 dB cost. On multi-organ histopathology x4 SR, adaptive routing achieves 31.72 dB PSNR versus 31.84 dB at full depth, while the shallowest exit exceeds bicubic by +1.9 dB at 2.8x less compute than SwinIR-light. The method generalizes to held-out colon tissue with minimal quality loss (-0.02 dB), and at x8 upscaling it outperforms all comparable-compute baselines while remaining competitive with the much larger SwinIR-Medium model. Downstream
nuclei segmentation confirms preservation of clinically relevant structure. The model trains in under 5 hours on a single GPU, and adaptive routing can reduce whole-slide inference from minutes to seconds.
Read the full abstract ▾
Computer Vision
OnlinePG: Online Open-Vocabulary Panoptic Mapping with 3D Gaussian Splatting
Hongjia Zhai, Qi Zhang, Xiaokun Pan et al. (+5 more)
Open-vocabulary scene understanding with online panoptic mapping is essential for
embodied applications to perceive and interact with environments. However, existing methods are predominantly offline or lack instance-level understanding, limiting their applicability to real-world robotic tasks. In this paper, we propose OnlinePG, a novel and effective system that integrates
geometric reconstruction and open-vocabulary perception using 3D Gaussian Splatting in an online setting. Technically, to achieve online panoptic mapping, we employ an efficient local-to-global paradigm with a sliding window. To build local consistency map, we construct a 3D segment clustering graph that jointly leverages
geometric and semantic cues, fusing inconsistent segments within sliding window into complete instances. Subsequently, to update the global map, we construct explicit grids with spatial attributes for the local 3D Gaussian map and fuse them into the global map via robust bidirectional bipartite 3D Gaussian instance matching. Finally, we utilize the fused VLM features inside the 3D spatial attribute grids to achieve open-vocabulary scene understanding. Extensive experiments on widely used
datasets demonstrate that our method achieves better performance among online approaches, while maintaining real-time efficiency.
Read the full abstract ▾
All authors Hongjia Zhai, Qi Zhang, Xiaokun Pan, Xiyu Zhang, Yitong Dong, Huaqi Zhang, Dan Xu, Guofeng Zhang
Computer Vision
Foundations and Architectures of Artificial Intelligence for Motor Insurance
Teerapong Panboonyuen
This handbook presents a systematic treatment of the foundations and architectures of artificial intelligence for motor insurance, grounded in large-scale real-world deployment. It formalizes a vertically integrated AI paradigm that unifies perception,
multimodal reasoning, and production infrastructure into a cohesive intelligence stack for automotive risk assessment and claims processing. At its core, the handbook develops domain-adapted
transformer architectures for structured visual understanding, relational vehicle representation learning, and
multimodal document intelligence, enabling end-to-end automation of vehicle damage analysis, claims evaluation, and underwriting workflows. These components are composed into a scalable pipeline operating under practical constraints observed in nationwide motor insurance systems in Thailand. Beyond model design, the handbook emphasizes the co-
evolution of learning
algorithms and MLOps practices, establishing a principled framework for translating modern artificial intelligence into reliable, production-grade systems in high-stakes industrial environments.
Read the full abstract ▾