Computer Vision · Cross-listed from eess.IV
HistDiT: A Structure-Aware Latent Conditional Diffusion Model for High-Fidelity Virtual Staining in Histopathology
Aasim Bin Saleem, Amr Ahmed, Ardhendu Behera et al. (+5 more)
Immunohistochemistry (IHC) is essential for assessing specific immune biomarkers like Human Epidermal growth-factor Receptor 2 (HER2) in breast cancer. However, the traditional protocols of obtaining IHC stains are resource-intensive, time-consuming, and prone to structural damages. Virtual staining has emerged as a scalable alternative, but it faces significant challenges in preserving fine-grained cellular structures while accurately translating biochemical expressions. Current state-of-the-art methods still rely on Generative
Adversarial Networks (GANs) or standard convolutional U-Net
diffusion models that often struggle with "structure and staining trade-offs". The generated samples are either structurally relevant but blurry, or texturally realistic but have artifacts that compromise their diagnostic use. In this paper, we introduce HistDiT, a novel latent conditional
Diffusion Transformer (DiT) architecture that establishes a new
benchmark for visual fidelity in virtual histological staining. The novelty introduced in this work is, a) the Dual-Stream Conditioning strategy that explicitly maintains a balance between spatial constraints via VAE-encoded latents and semantic phenotype guidance via UNI
embeddings; b) the multi-objective loss function that contributes to sharper images with clear morphological structure; and c) the use of the Structural Correlation Metric (SCM) to focus on the core morphological structure for precise assessment of sample quality. Consequently, our model outperforms existing baselines, as demonstrated through rigorous quantitative and qualitative evaluations.
Read the full abstract ▾
All authors Aasim Bin Saleem, Amr Ahmed, Ardhendu Behera, Hafeezullah Amin, Iman Yi Liao, Mahmoud Khattab, Pan Jia Wern, Haslina Makmur
Computer Vision
GroundingAnomaly: Spatially-Grounded Diffusion for Few-Shot Anomaly Synthesis
Yishen Liu, Hongcang Chen, Pengcheng Zhao et al. (+8 more)
The performance of visual anomaly inspection in industrial quality control is often constrained by the scarcity of real anomalous samples. Consequently, anomaly synthesis techniques have been developed to enlarge training sets and enhance downstream inspection. However, existing methods either suffer from poor integration caused by inpainting or fail to provide accurate masks. To address these limitations, we propose GroundingAnomaly, a novel few-shot anomaly image generation framework. Our framework introduces a Spatial Conditioning Module that leverages per-pixel semantic maps to enable precise spatial control over the synthesized anomalies. Furthermore, a Gated Self-
Attention Module is designed to inject conditioning
tokens into a frozen U-Net via gated
attention layers. This carefully preserves pretrained priors while ensuring stable few-shot adaptation. Extensive evaluations on the MVTec AD and VisA
datasets demonstrate that GroundingAnomaly generates high-quality anomalies and achieves state-of-the-art performance across multiple downstream tasks, including anomaly detection,
segmentation, and instance-level detection.
Read the full abstract ▾
All authors Yishen Liu, Hongcang Chen, Pengcheng Zhao, Yunfan Bao, Yuxi Tian, Jieming Zhang, Hao Chen, Zheng Zhi, Yongchun Liu, Ying Li, Dongpu Cao
Computer Vision · Cross-listed from cs.AI
U-CECE: A Universal Multi-Resolution Framework for Conceptual Counterfactual Explanations
Angeliki Dimitriou, Nikolaos Chaidos, Maria Lymperaiou et al. (+2 more)
As AI models grow more complex, explainability is essential for building trust, yet concept-based counterfactual methods still face a trade-off between expressivity and efficiency. Representing underlying concepts as atomic sets is fast but misses relational context, whereas full graph representations are more faithful but require solving the NP-hard Graph Edit Distance (GED) problem. We propose U-CECE, a unified, model-agnostic multi-resolution framework for conceptual counterfactual explanations that adapts to data regime and compute budget. U-CECE spans three levels of expressivity: atomic concepts for broad explanations, relational sets-of-sets for simple interactions, and structural graphs for full semantic structure. At the structural level, both a precision-{\omicron}riented transductive mode based on supervised Graph Neural Networks (GNNs) and a scalable inductive mode based on unsupervised graph autoencoders (GAEs) are supported. Experiments on the structurally divergent CUB and Visual Genome
datasets characterize the efficiency-expressivity trade-off across levels, while human surveys and LVLM-based evaluation show that, on CUB, the retrieved structural counterfactuals are frequently judged semantically equivalent to, and often preferred over, reference deterministic GED-based explanations.
Read the full abstract ▾
All authors Angeliki Dimitriou, Nikolaos Chaidos, Maria Lymperaiou, Giorgos Filandrianos, Giorgos Stamou
Computer Vision
Can Vision Language Models Judge Action Quality? An Empirical Evaluation
Miguel Monte e Freitas, Rui Henriques, Ricardo Rei et al. (+1 more)
Action Quality Assessment (AQA) has broad applications in physical therapy, sports coaching, and competitive judging. Although
Vision Language Models (VLMs) hold considerable promise for AQA, their actual performance in this domain remains largely uncharacterised. We present a comprehensive evaluation of state-of-the-art VLMs across activity domains (e.g. fitness, figure skating, diving), tasks, representations, and prompting strategies. Baseline results reveal that Gemini 3.1 Pro, Qwen3-VL and InternVL3.5 models perform only marginally above random chance, and although strategies such as incorporation of skeleton information, grounding instructions,
reasoning structures and in-context learning lead to isolated gains, none is consistently effective. Analysis of prediction distributions uncovers two systematic biases: a tendency to predict correct execution regardless of visual evidence, and a sensitivity to superficial linguistic framing. Reformulating tasks contrastively to mitigate these biases yields minimal improvement, suggesting that the models' limitations go beyond these biases, pointing to a fundamental difficulty with fine-grained movement quality assessment. Our findings establish a rigorous baseline for future VLM-based AQA research and provide an actionable outline for failure modes requiring mitigation prior to reliable real-world deployment.
Read the full abstract ▾
All authors Miguel Monte e Freitas, Rui Henriques, Ricardo Rei, Pedro Henrique Martins
Computer Vision
CAMotion: A High-Quality Benchmark for Camouflaged Moving Object Detection in the Wild
Siyuan Yao, Hao Sun, Ruiqi Yu et al. (+3 more)
Discovering camouflaged objects is a challenging task in computer
vision due to the high similarity between camouflaged objects and their surroundings. While the problem of camouflaged object detection over sequential
video frames has received increasing
attention, the scale and diversity of existing
video camouflaged object detection (VCOD)
datasets are greatly limited, which hinders the deeper analysis and broader evaluation of recent deep learning-based
algorithms with data-hungry training strategy. To break this bottleneck, in this paper, we construct CAMotion, a high-quality
benchmark covers a wide range of species for camouflaged moving object detection in the wild. CAMotion comprises various sequences with multiple challenging attributes such as uncertain edge, occlusion, motion blur, and shape complexity, etc. The sequence annotation details and statistical distribution are presented from various perspectives, allowing CAMotion to provide in-depth analyses on the camouflaged object's motion characteristics in different challenging scenarios. Additionally, we conduct a comprehensive evaluation of existing SOTA models on CAMotion, and discuss the major challenges in VCOD task. The
benchmark is available at https://www.camotion.focuslab.net.cn, we hope that our CAMotion can lead to further advancements in the research community.
Read the full abstract ▾
All authors Siyuan Yao, Hao Sun, Ruiqi Yu, Xiwei Jiang, Wenqi Ren, Xiaochun Cao
Computer Vision
Revisiting Radar Perception With Spectral Point Clouds
Hamza Alsharif, Jing Gu, Pavol Jancura et al. (+2 more)
Radar perception models are trained with different inputs, from range-Doppler spectra to sparse point clouds. Dense spectra are assumed to outperform sparse point clouds, yet they can vary considerably across sensors and configurations, which hinders transfer. In this paper, we provide alternatives for incorporating spectral information into radar point clouds and show that, point clouds need not underperform compared to spectra. We introduce the spectral point cloud paradigm, where point clouds are treated as sparse, compressed representations of the radar spectra, and argue that, when enriched with spectral information, they serve as strong candidates for a unified input representation that is more robust against sensor-specific differences. We develop an experimental framework that compares spectral point cloud (PC) models at varying densities against a dense range-Doppler (RD)
benchmark, and report the density levels where the PC configurations meet the performance of the RD
benchmark. Furthermore, we experiment with two basic spectral enrichment approaches, that inject additional target-relevant information into the point clouds. Contrary to the common belief that the dense RD approach is superior, we show that point clouds can do just as well, and can surpass the RD
benchmark when enrichment is applied. Spectral point clouds can therefore serve as strong candidates for unified radar perception, paving the way for future radar foundation models.
Read the full abstract ▾
All authors Hamza Alsharif, Jing Gu, Pavol Jancura, Satish Ravindran, Gijs Dubbelman
Computer Vision
Preventing Overfitting in Deep Image Prior for Hyperspectral Image Denoising
Panagiotis Gkotsis and Athanasios A. Rontogiannis
Deep image prior (DIP) is an unsupervised deep learning framework that has been successfully applied to a variety of inverse imaging problems. However, DIP-based methods are inherently prone to overfitting, which leads to performance degradation and necessitates early stopping. In this paper, we propose a method to mitigate overfitting in DIP-based hyperspectral image (HSI) denoising by jointly combining robust data fidelity and explicit sensitivity regularization. The proposed approach employs a Smooth $\ell_1$ data term together with a divergence-based regularization and input optimization during training. Experimental results on real HSIs corrupted by Gaussian, sparse, and stripe noise demonstrate that the proposed method effectively prevents overfitting and achieves superior denoising performance compared to state-of-the-art DIP-based HSI denoising methods.
Read the full abstract ▾
Computer Vision
Orion-Lite: Distilling LLM Reasoning into Efficient Vision-Only Driving Models
Jing Gu, Niccolò Cavagnero and Gijs Dubbelman
Leveraging the general world knowledge of Large
Language Models (LLMs) holds significant promise for improving the ability of autonomous driving systems to handle rare and complex scenarios. While integrating LLMs into
Vision-
Language-Action (VLA) models has yielded state-of-the-art performance, their massive parameter counts pose severe challenges for latency-sensitive and energy-efficient deployment. Distilling LLM knowledge into a compact driving model offers a compelling solution to retain these
reasoning capabilities while maintaining a manageable computational footprint. Although previous works have demonstrated the efficacy of
distillation, these efforts have primarily focused on relatively simple scenarios and open-loop evaluations. Therefore, in this work, we investigate LLM
distillation in more complex, interactive scenarios under closed-loop evaluation. We demonstrate that through a combination of latent feature
distillation and ground-truth trajectory supervision, an efficient
vision-only student model Orion-Lite can even surpass the performance of its massive VLA teacher, ORION. Setting a new state-of-the-art on the rigorous Bench2Drive
benchmark, with a Driving Score of 80.6. Ultimately, this reveals that
vision-only architectures still possess significant, untapped potential for high-performance reactive planning.
Read the full abstract ▾
Computer Vision
DBMF: A Dual-Branch Multimodal Framework for Out-of-Distribution Detection
Jiangbei Yue, Darren Treanor, Venkataraman Subramanian et al. (+1 more)
The complex and dynamic real-world clinical environment demands reliable deep learning (DL) systems. Out-of-distribution (OOD) detection plays a critical role in enhancing the reliability and generalizability of DL models when encountering data that deviate from the training distribution, such as unseen disease cases. However, existing OOD detection methods typically rely either on a single visual modality or solely on image-text matching, failing to fully leverage
multimodal information. To overcome the challenge, we propose a novel dual-branch
multimodal framework by introducing a text-image branch and a
vision branch. Our framework fully exploits
multimodal representations to identify OOD samples through these two complementary branches. After training, we compute scores from the text-image branch ($S_t$) and
vision branch ($S_v$), and integrate them to obtain the final OOD score $S$ that is compared with a threshold for OOD detection. Comprehensive experiments on publicly available endoscopic image
datasets demonstrate that our proposed framework is robust across diverse backbones and improves state-of-the-art performance in OOD detection by up to 24.84%
Read the full abstract ▾
All authors Jiangbei Yue, Darren Treanor, Venkataraman Subramanian, Sharib Ali
Computer Vision
$\oslash$ Source Models Leak What They Shouldn't $\nrightarrow$: Unlearning Zero-Shot Transfer in Domain Adaptation Through Adversarial Optimization
Arnav Devalapally, Poornima Jain, Kartik Srinivas et al. (+1 more)
The increasing adaptation of
vision models across domains, such as satellite imagery and medical scans, has raised an emerging
privacy risk: models may inadvertently retain and leak sensitive source-domain specific information in the target domain. This creates a compelling use case for machine unlearning to protect the
privacy of sensitive source-domain data. Among adaptation techniques, source-free domain adaptation (SFDA) calls for an urgent need for machine unlearning (MU), where the source data itself is protected, yet the source model exposed during adaptation encodes its influence. Our experiments reveal that existing SFDA methods exhibit strong zero-shot performance on source-exclusive classes in the target domain, indicating they inadvertently leak knowledge of these classes into the target domain, even when they are not represented in the target data. We identify and address this risk by proposing an MU setting called SCADA-UL: Unlearning Source-exclusive ClAsses in Domain Adaptation. Existing MU methods do not address this setting as they are not designed to handle data distribution shifts. We propose a new unlearning method, where an adversarially generated forget class sample is unlearned by the model during the domain adaptation process using a novel rescaled labeling strategy and
adversarial optimization. We also extend our study to two variants: a continual version of this problem setting and to one where the specific source classes to be forgotten may be unknown. Alongside theoretical interpretations, our comprehensive empirical results show that our method consistently outperforms baselines in the proposed setting while achieving retraining-level unlearning performance on
benchmark datasets. Our code is available at https://github.com/D-Arnav/SCADA
Read the full abstract ▾
All authors Arnav Devalapally, Poornima Jain, Kartik Srinivas, Vineeth N. Balasubramanian
Computer Vision
Generalization Under Scrutiny: Cross-Domain Detection Progresses, Pitfalls, and Persistent Challenges
Saniya M.Deshmukh, Kailash A. Hambarde and Hugo Proença
Object detection models trained on a source domain often exhibit significant performance degradation when deployed in unseen target domains, due to various kinds of variations, such as sensing conditions, environments and data distributions. Hence, regardless the recent breakthrough advances in deep learning-based detection technology, cross-domain object detection (CDOD) remains a critical research area. Moreover, the existing literature remains fragmented, lacking a unified perspective on the structural challenges underlying domain shift and the effectiveness of adaptation strategies. This survey provides a comprehensive and systematic analysis of CDOD. We start upon a problem formulation that highlights the multi-stage nature of object detection under domain shift. Then, we organize the existing methods through a conceptual taxonomy that categorizes approaches based on adaptation paradigms, modeling assumptions, and pipeline components. Furthermore, we analyze how domain shift propagates across detection stages and discuss why adaptation in object detection is inherently more complex than in classification. In addition, we review commonly used
datasets, evaluation protocols, and benchmarking practices. Finally, we identify the key challenges and outline promising future research directions. Cohesively, this survey aims to provide a unified framework for understanding CDOD and to guide the development of more robust detection systems.
Read the full abstract ▾
Computer Vision
EditCaption: Human-Refined SFT and HAE-DPO for Image Editing Instruction Synthesis
Xiangyuan Wang, Honghao Cai, Yunhao Bai et al. (+9 more)
High-quality source-target image pairs with precise editing instructions are essential for instruction-guided image editing, yet constructing such training triplets at scale remains costly. Recent pipelines often rely on
vision-
language models to synthesize editing instructions automatically, but we find that strong VLMs still struggle to describe visual transformations between image pairs. In particular, they exhibit three recurring failure modes: orientation inconsistency, viewpoint ambiguity, and missing fine-grained attributes. In a human evaluation on 400 image pairs, several open-source VLM baselines produce critical-error rates above 47%, making many synthesized instructions unsuitable for downstream training. To address this, we propose EditCaption, a two-stage post-training pipeline for image editing instruction synthesis. First, we construct a 100K supervised fine-tuning
dataset through GLM-based auto-captioning, EditScore filtering, and human refinement. Second, we collect 10K human-annotated preference pairs, where each rejected instruction is labeled with its primary error type and severity. Based on this
dataset, we propose Hardness-Adaptive Error-Aware DPO (HAE-DPO), a task-adapted DPO objective that introduces an adaptive margin based on human-labeled severity, failure-mode type, and reference-model hardness. Experiments across three
benchmarks demonstrate that our 235B model with SFT+HAE-DPO achieves state-of-the-art performance among open-source and closed models, scoring 4.720 on Eval-400, 4.672 on HQ-Edit, and 4.651 on ByteMorph-Bench – surpassing Gemini-3-Pro on all three. Human evaluation confirms critical error rates drop from 47.75% to 17.50%, with correct rates improving from 41.75% to 70.25%, surpassing Gemini-3-Pro (66.00%).
Read the full abstract ▾
All authors Xiangyuan Wang, Honghao Cai, Yunhao Bai, Chao Hui, Tianze Zhou, Haohua Chen, Hao Shi, Yuling Wu, Yao Hu, Xu Tang, Yibo Chen, Wei Zhu
Computer Vision
Vision-Language Foundation Models for Comprehensive Automated Pavement Condition Assessment
Blessing Agyei Kyem, Joshua Kofi Asamoah, Anthony Dontoh et al. (+1 more)
General-purpose
vision-
language models demonstrate strong performance in everyday domains but struggle with specialized technical fields requiring precise terminology, structured
reasoning, and adherence to engineering standards. This work addresses whether domain-specific instruction tuning can enable comprehensive pavement condition assessment through
vision-
language models. PaveInstruct, a
dataset containing 278,889 image-instruction-response pairs spanning 32 task types, was created by unifying annotations from nine heterogeneous pavement
datasets. PaveGPT, a pavement foundation model trained on this
dataset, was evaluated against state-of-the-art
vision-
language models across perception, understanding, and
reasoning tasks. Instruction tuning transformed model capabilities, achieving improvements exceeding 20% in spatial grounding,
reasoning, and generation tasks while producing ASTM D6433-compliant outputs. These results enable transportation agencies to deploy unified conversational assessment tools that replace multiple specialized systems, simplifying workflows and reducing technical expertise requirements. The approach establishes a pathway for developing instruction-driven AI systems across infrastructure domains including bridge inspection, railway maintenance, and building condition assessment.
Read the full abstract ▾
All authors Blessing Agyei Kyem, Joshua Kofi Asamoah, Anthony Dontoh, Armstrong Aboah
Computer Vision
SciFigDetect: A Benchmark for AI-Generated Scientific Figure Detection
You Hu, Chenzhuo Zhao, Changfa Mo et al. (+2 more)
Modern
multimodal generators can now produce scientific figures at near-publishable quality, creating a new challenge for visual forensics and research integrity. Unlike conventional AI-generated natural images, scientific figures are structured, text-dense, and tightly aligned with scholarly semantics, making them a distinct and difficult detection target. However, existing AI-generated image detection
benchmarks and methods are almost entirely developed for open-domain imagery, leaving this setting largely unexplored. We present the first
benchmark for AI-generated scientific figure detection. To construct it, we develop an
agent-based data pipeline that retrieves licensed source papers, performs
multimodal understanding of paper text and figures, builds structured
prompts, synthesizes candidate figures, and filters them through a review-driven refinement loop. The resulting
benchmark covers multiple figure categories, multiple generation sources and aligned real–synthetic pairs. We
benchmark representative detectors under zero-shot, cross-generator, and degraded-image settings. Results show that current methods fail dramatically in zero-shot transfer, exhibit strong generator-specific overfitting, and remain fragile under common post-processing corruptions. These findings reveal a substantial gap between existing AIGI detection capabilities and the emerging distribution of high-quality scientific figures. We hope this
benchmark can serve as a foundation for future research on robust and generalizable scientific-figure forensics. The
dataset is available at https://github.com/Joyce-yoyo/SciFigDetect.
Read the full abstract ▾
All authors You Hu, Chenzhuo Zhao, Changfa Mo, Haotian Liu, Xiaobai Li
Computer Vision
OmniJigsaw: Enhancing Omni-Modal Reasoning via Modality-Orchestrated Reordering
Yiduo Jia, Muzhi Zhu, Hao Zhong et al. (+7 more)
To extend the reinforcement learning post-training paradigm to omni-modal models for concurrently bolstering
video-audio understanding and collaborative
reasoning, we propose OmniJigsaw, a generic self-supervised framework built upon a temporal reordering proxy task. Centered on the chronological reconstruction of shuffled audio-visual clips, this paradigm strategically orchestrates visual and auditory signals to compel cross-modal integration through three distinct strategies: Joint Modality Integration, Sample-level Modality Selection, and Clip-level Modality Masking. Recognizing that the efficacy of such proxy tasks is fundamentally tied to puzzle quality, we design a two-stage coarse-to-fine data filtering pipeline, which facilitates the efficient adaptation of OmniJigsaw to massive unannotated omni-modal data. Our analysis reveals a “bi-modal shortcut phenomenon” in joint modality integration and demonstrates that fine-grained clip-level modality masking mitigates this issue while outperforming sample-level modality selection. Extensive evaluations on 15
benchmarks show substantial gains in
video, audio, and collaborative
reasoning, validating OmniJigsaw as a scalable paradigm for self-supervised omni-modal learning.
Read the full abstract ▾
All authors Yiduo Jia, Muzhi Zhu, Hao Zhong, Mingyu Liu, Yuling Xi, Hao Chen, Bin Qin, Yongjie Yang, Zhenbo Luo, Chunhua Shen
Computer Vision
MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning
Zheng Jiang, Heng Guo, Chengyu Fang et al. (+4 more)
Medical
Vision-
Language Models (VLMs) hold immense promise for complex clinical tasks, but their
reasoning capabilities are often constrained by text-only paradigms that fail to ground inferences in visual evidence. This limitation not only curtails performance on tasks requiring fine-grained visual analysis but also introduces risks of visual
hallucination in safety-critical applications. Thus, we introduce MedVR, a novel reinforcement learning framework that enables annotation-free visual
reasoning for medical VLMs. Its core innovation lies in two synergistic mechanisms:
Entropy-guided Visual Regrounding (EVR) uses model uncertainty to direct exploration, while Consensus-based Credit Assignment (CCA) distills pseudo-supervision from rollout agreement. Without any human annotations for intermediate steps, MedVR achieves state-of-the-art performance on diverse public medical VQA
benchmarks, significantly outperforming existing models. By learning to reason directly with visual evidence, MedVR promotes the robustness and transparency essential for accelerating the clinical deployment of medical AI.
Read the full abstract ▾
All authors Zheng Jiang, Heng Guo, Chengyu Fang, Changchen Xiao, Xinyang Hu, Lifeng Sun, Minfeng Xu
Computer Vision · Cross-listed from cs.LG
Inside-Out: Measuring Generalization in Vision Transformers Through Inner Workings
Yunxiang Peng, Mengmeng Ma, Ziyu Yao et al. (+1 more)
Reliable generalization metrics are fundamental to the evaluation of machine learning models. Especially in high-stakes applications where labeled target data are scarce, evaluation of models' generalization performance under distribution shift is a pressing need. We focus on two practical scenarios: (1) Before deployment, how to select the best model for unlabeled target data? (2) After deployment, how to monitor model performance under distribution shift? The central need in both cases is a reliable and label-free proxy metric. Yet existing proxy metrics, such as model confidence or accuracy-on-the-line, are often unreliable as they only assess model output while ignoring the internal mechanisms that produce them. We address this limitation by introducing a new perspective: using the inner workings of a model, i.e.,
circuits, as a predictive metric of generalization performance. Leveraging
circuit discovery, we extract the causal interactions between internal representations as a
circuit, from which we derive two metrics tailored to the two practical scenarios. (1) Before deployment, we introduce Dependency Depth Bias, which measures different models' generalization capability on target data. (2) After deployment, we propose
Circuit Shift Score, which predicts a model's generalization under different distribution shifts. Across various tasks, both metrics demonstrate significantly improved correlation with generalization performance, outperforming existing proxies by an average of 13.4% and 34.1%, respectively. Our code is available at https://github.com/deep-real/GenCircuit.
Read the full abstract ▾
All authors Yunxiang Peng, Mengmeng Ma, Ziyu Yao, Xi Peng
Computer Vision
On the Global Photometric Alignment for Low-Level Vision
Mingjia Li, Tianle Du, Hainuo Wang et al. (+2 more)
Supervised low-level
vision models rely on pixel-wise losses against paired references, yet paired training sets exhibit per-pair photometric inconsistency, say, different image pairs demand different global brightness, color, or white-balance mappings. This inconsistency enters through task-intrinsic photometric transfer (e.g., low-light enhancement) or unintended acquisition shifts (e.g., de-raining), and in either case causes an optimization pathology. Standard reconstruction losses allocate disproportionate gradient budget to conflicting per-pair photometric targets, crowding out content restoration. In this paper, we investigate this issue and prove that, under least-squares decomposition, the photometric and structural components of the prediction-target residual are orthogonal, and that the spatially dense photometric component dominates the gradient energy. Motivated by this analysis, we propose Photometric
Alignment Loss (PAL). This flexible supervision objective discounts nuisance photometric discrepancy via closed-form affine color
alignment while preserving restoration-relevant supervision, requiring only covariance statistics and tiny
matrix inversion with negligible overhead. Across 6 tasks, 16
datasets, and 16 architectures, PAL consistently improves metrics and generalization. The implementation is in the appendix.
Read the full abstract ▾
All authors Mingjia Li, Tianle Du, Hainuo Wang, Qiming Hu, Xiaojie Guo
Computer Vision
OceanMAE: A Foundation Model for Ocean Remote Sensing
Viola-Joanna Stamer, Panagiotis Agrafiotis, Behnood Rasti et al. (+1 more)
Accurate ocean mapping is essential for applications such as bathymetry estimation, seabed characterization, marine litter detection, and ecosystem monitoring. However, ocean remote sensing (RS) remains constrained by limited labeled data and by the reduced transferability of models pre-trained mainly on land-dominated Earth observation imagery. In this paper, we propose OceanMAE, an ocean-specific masked autoencoder that extends standard MAE pre-training by integrating multispectral Sentinel-2 observations with physically meaningful ocean descriptors during self-supervised learning. By incorporating these auxiliary ocean features, OceanMAE is designed to learn more informative and ocean-aware latent representations from large- scale unlabeled data. To transfer these representations to downstream applications, we further employ a modified UNet-based framework for marine
segmentation and bathymetry estimation. Pre-trained on the Hydro
dataset, OceanMAE is evaluated on MADOS and MARIDA for marine pollutant and debris
segmentation, and on MagicBathyNet for bathymetry regression. The experiments show that OceanMAE yields the strongest gains on marine
segmentation, while bathymetry benefits are competitive and task-dependent. In addition, an ablation against a standard MAE on MARIDA indicates that incorporating auxiliary ocean descriptors during pre-training improves downstream
segmentation quality. These findings highlight the value of physically informed and domain-aligned self-supervised pre- training for ocean RS. Code and weights are publicly available at https://git.tu-berlin.de/joanna.stamer/SSLORS2.
Read the full abstract ▾
All authors Viola-Joanna Stamer, Panagiotis Agrafiotis, Behnood Rasti, Begüm Demir
Computer Vision
T-Gated Adapter: A Lightweight Temporal Adapter for Vision-Language Medical Segmentation
Pranjal Khadka
Medical image
segmentation traditionally relies on fully supervised 3D architectures that demand a large amount of dense, voxel-level annotations from clinical experts which is a prohibitively expensive process.
Vision Language Models (VLMs) offer a powerful alternative by leveraging broad visual semantic representations learned from billions of images. However, when applied independently to 2D slices of a 3D scan, these models often produce noisy and anatomically implausible
segmentations that violate the inherent continuity of anatomical structures. We propose a temporal adapter that addresses this by injecting adjacent-slice context directly into the model's visual
token representations. The adapter comprises a temporal
transformer attending across a fixed context window at the
token level, a spatial context block refining within-slice representations, and an adaptive gate balancing temporal and single-slice features. Training on 30 labeled volumes from the
FLARE22
dataset, our method achieves a mean Dice of 0.704 across 13 abdominal organs with a gain of +0.206 over the baseline VLM trained with no temporal context. Zero-shot evaluation on BTCV and AMOS22
datasets yields consistent improvements of +0.210 and +0.230, with the average cross-domain performance drop reducing from 38.0% to 24.9%. Furthermore, in a cross-modality evaluation on AMOS22 MRI with neither model receiving any MRI supervision, our method achieves a mean Dice of 0.366, outperforming a fully supervised 3D baseline (DynUNet, 0.224) trained exclusively on CT, suggesting that CLIP's visual semantic representations generalize more gracefully across imaging modalities than convolutional features.
Read the full abstract ▾
Computer Vision
Face-D(^2)CL: Multi-Domain Synergistic Representation with Dual Continual Learning for Facial DeepFake Detection
Yushuo Zhang, Yu Cheng, Yongkang Hu et al. (+3 more)
Facial forgery techniques are advancing rapidly, posing severe threats to public trust and information security while imposing higher demands on the continual adaptation of DeepFake detection models. Although continual learning enables models to adapt to emerging forgery methods, existing approaches still face two key bottlenecks. On the one hand, they lack sufficient feature representation capacity to handle increasingly diverse and complex forgery traces. On the other hand, continual adaptation to new forgery distributions leads to severe catastrophic forgetting of prior knowledge, which substantially degrades detection performance. To address these issues, we propose Face-D(^2)CL, a framework for facial DeepFake detection. It leverages multi-domain synergistic representation to fuse spatial and frequency-domain features, enabling comprehensive capture of diverse forgery traces. Additionally, it employs a dual continual learning mechanism that combines Real/Fake-aware Elastic Weight Consolidation (RF-EWC) and Domain-wise Orthogonal Gradient Constraint (D-OGC). RF-EWC distinguishes the parameter importance for real versus fake samples, while D-OGC ensures that updates to task-specific expert modules do not interfere with previously learned knowledge. This synergy allows the model to achieve a dynamic balance between robust anti-forgetting capabilities and agile adaptability to emerging facial forgery paradigms, all without relying on historical data replay. Extensive experiments demonstrate that our method surpasses current state-of-the-art (SOTA) approaches in both stability and plasticity, achieving a 60.7% relative reduction in the average detection error rate. On unseen forgery domains, it further improves the average detection AUC by 7.9% compared to the current SOTA method.
Read the full abstract ▾
All authors Yushuo Zhang, Yu Cheng, Yongkang Hu, Jiuan Zhou, Jiawei Chen, Zhaoxia Yin
Computer Vision · Cross-listed from cs.SD
Semantic Noise Reduction via Teacher-Guided Dual-Path Audio-Visual Representation Learning
Linge Wang, Yingying Chen, Bingke Zhu et al. (+2 more)
Recent advances in audio-visual representation learning have shown the value of combining contrastive
alignment with masked reconstruction. However, jointly optimizing these objectives in a single forward pass forces the contrastive branch to rely on randomly visible patches designed for reconstruction rather than cross-modal
alignment, introducing semantic noise and optimization interference. We propose TG-DP, a Teacher-Guided Dual-Path framework that decouples reconstruction and
alignment into separate optimization paths. By disentangling the masking regimes of the two branches, TG-DP enables the contrastive pathway to use a visibility pattern better suited to cross-modal
alignment. A teacher model further provides auxiliary guidance for organizing visible
tokens in this branch, helping reduce interference and stabilize cross-modal representation learning. TG-DP achieves state-of-the-art performance in zero-shot
retrieval. On AudioSet, it improves R@1 from 35.2% to 37.4% for
video-to-audio
retrieval and from 27.9% to 37.1% for audio-to-
video retrieval. The learned representations also remain semantically robust, achieving state-of-the-art linear-probe performance on AS20K and VGGSound. Taken together, our results suggest that decoupling
multimodal objectives and introducing teacher-guided structure into the contrastive pathway provide an effective framework for improving large-scale audio-visual pretraining. Code is available at https://github.com/wanglg20/TG-DP.
Read the full abstract ▾
All authors Linge Wang, Yingying Chen, Bingke Zhu, Lu Zhou, Jinqiao Wang
Computer Vision
Bag of Bags: Adaptive Visual Vocabularies for Genizah Join Image Retrieval
Sharva Gogawale, Gal Grudka, Daria Vasyutinsky-Shapira et al. (+3 more)
A join is a set of manuscript fragments identified as originally emanating from the same manuscript. We study manuscript join
retrieval: Given a query image of a fragment, retrieve other fragments originating from the same physical manuscript. We propose Bag of Bags (BoB), an image-level representation that replaces the global-level visual codebook of classical Bag of Words (BoW) with a fragment-specific vocabulary of local visual words. Our pipeline trains a sparse convolutional autoencoder on binarized fragment patches, encodes connected components from each page, clusters the resulting
embeddings with per-image k-means, and compares images using set-to-set distances between their local vocabularies. Evaluated on fragments from the Cairo Genizah, the best BoB variant (viz. Chamfer) achieves Hit@1 of 0.78 and MRR of 0.84, compared to 0.74 and 0.80, respectively, for the strongest BoW baseline (BoW-RawPatches-$\chi^2$), a 6.1% relative improvement in top-1 accuracy. We furthermore study a mass-weighted BoB-OT variant that incorporates cluster population into prototype matching and present a formal approximation guarantee bounding its deviation from full component-level optimal transport. A two-stage pipeline using a BoW shortlist followed by BoB-OT reranking provides a practical compromise between
retrieval strength and computational cost, supporting applicability to larger manuscript collections. The code and
dataset are available at https://github.com/TAU-CH/midrash_bob.
Read the full abstract ▾
All authors Sharva Gogawale, Gal Grudka, Daria Vasyutinsky-Shapira, Omer Ventura, Berat Kurar-Barakat, Nachum Dershowitz
Computer Vision
PolySLGen: Online Multimodal Speaking-Listening Reaction Generation in Polyadic Interaction
Zhi-Yi Lin, Thomas Markhorst, Jouh Yeong Chew et al. (+1 more)
Human-like
multimodal reaction generation is essential for natural group interactions between humans and
embodied AI. However, existing approaches are limited to single-modality or speaking-only responses in dyadic interactions, making them unsuitable for realistic social scenarios. Many also overlook nonverbal cues and complex dynamics of polyadic interactions, both critical for engagement and conversational coherence. In this work, we present PolySLGen, an online framework for Polyadic
multimodal Speaking and Listening reaction Generation. Given past conversation and motion from all participants, PolySLGen generates a future speaking or listening reaction for a target participant, including
speech, body motion, and speaking state score. To model group interactions effectively, we propose a pose fusion module and a social cue encoder that jointly aggregate motion and social signals from the group. Extensive experiments, along with quantitative and qualitative evaluations, show that PolySLGen produces contextually appropriate and temporally coherent multi-modal reactions, outperforming several adapted and state-of-the-art baselines in motion quality, motion-
speech alignment, speaking state prediction, and human-perceived realism.
Read the full abstract ▾
All authors Zhi-Yi Lin, Thomas Markhorst, Jouh Yeong Chew, Xucong Zhang
Computer Vision
Uni-ViGU: Towards Unified Video Generation and Understanding via A Diffusion-Based Video Generator
Luozheng Qin, Jia Gong, Qian Qiao et al. (+6 more)
Unified
multimodal models integrating visual understanding and generation face a fundamental challenge: visual generation incurs substantially higher computational costs than understanding, particularly for
video. This imbalance motivates us to invert the conventional paradigm: rather than extending understanding-centric MLLMs to support generation, we propose Uni-ViGU, a framework that unifies
video generation and understanding by extending a
video generator as the foundation. We introduce a unified flow method that performs continuous flow matching for
video and discrete flow matching for text within a single process, enabling coherent
multimodal generation. We further propose a modality-driven MoE-based framework that augments
Transformer blocks with lightweight layers for text generation while preserving generative priors. To repurpose generation knowledge for understanding, we design a bidirectional training mechanism with two stages: Knowledge Recall reconstructs input
prompts to leverage learned text-
video correspondences, while Capability Refinement fine-tunes on detailed captions to establish discriminative shared representations. Experiments demonstrate that Uni-ViGU achieves competitive performance on both
video generation and understanding, validating generation-centric architectures as a scalable path toward unified
multimodal intelligence. Project Page and Code: https://fr0zencrane.github.io/uni-vigu-page/.
Read the full abstract ▾
All authors Luozheng Qin, Jia Gong, Qian Qiao, Tianjiao Li, Li Xu, Haoyu Pan, Chao Qu, Zhiyu Tan, Hao Li
Computer Vision
Small Vision-Language Models are Smart Compressors for Long Video Understanding
Junjie Fei, Jun Chen, Zechun Liu et al. (+13 more)
Adapting
Multimodal Large
Language Models (MLLMs) for hour-long
videos is bottlenecked by context limits. Dense visual streams saturate
token budgets and exacerbate the lost-in-the-middle phenomenon. Existing heuristics, like sparse sampling or uniform pooling, blindly sacrifice fidelity by discarding decisive moments and wasting bandwidth on irrelevant backgrounds. We propose Tempo, an efficient query-aware framework compressing long
videos for downstream understanding. Tempo leverages a Small
Vision-
Language Model (SVLM) as a local temporal compressor, casting
token reduction as an early cross-modal
distillation process to generate compact, intent-aligned representations in a single forward pass. To enforce strict budgets without breaking causality, we introduce Adaptive
Token Allocation (ATA). Exploiting the SVLM's zero-shot relevance prior and semantic front-loading, ATA acts as a training-free $O(1)$ dynamic router. It allocates dense bandwidth to query-critical segments while compressing redundancies into minimal temporal anchors to maintain the global storyline. Extensive experiments show our 6B architecture achieves state-of-the-art performance with aggressive dynamic compression (0.5-16
tokens/frame). On the extreme-long LVBench (4101s), Tempo scores 52.3 under a strict 8K visual budget, outperforming GPT-4o and Gemini 1.5 Pro. Scaling to 2048 frames reaches 53.7. Crucially, Tempo compresses hour-long
videos substantially below theoretical limits, proving true long-form
video understanding relies on intent-driven efficiency rather than greedily padded context windows.
Read the full abstract ▾
All authors Junjie Fei, Jun Chen, Zechun Liu, Yunyang Xiong, Chong Zhou, Wei Wen, Junlin Han, Mingchen Zhuge, Saksham Suri, Qi Qian, Shuming Liu, Lemeng Wu, Raghuraman Krishnamoorthi, Vikas Chandra, Mohamed Elhoseiny, Chenchen Zhu
Computer Vision · Cross-listed from cs.LG
Bias Redistribution in Visual Machine Unlearning: Does Forgetting One Group Harm Another?
Yunusa Haruna, Adamu Lawan, Ibrahim Haruna Abdulhamid et al. (+4 more)
Machine unlearning enables models to selectively forget training data, driven by
privacy regulations such as GDPR and CCPA. However, its
fairness implications remain underexplored: when a model forgets a demographic group, does it neutralize that concept or redistribute it to correlated groups, potentially amplifying bias? We investigate this bias redistribution phenomenon on CelebA using CLIP models (ViT/B-32, ViT-L/14, ViT-B/16) under a zero-shot classification setting across intersectional groups defined by age and gender. We evaluate three unlearning methods,
Prompt Erasure,
Prompt Reweighting, and Refusal Vector using per-group accuracy shifts, demographic parity gaps, and a redistribution score. Our results show that unlearning does not eliminate bias but redistributes it primarily along gender rather than age boundaries. In particular, removing the dominant Young Female group consistently transfers performance to Old Female across all model scales, revealing a gender-dominant structure in CLIP's
embedding space. While the Refusal Vector method reduces redistribution, it fails to achieve complete forgetting and significantly degrades retained performance. These findings highlight a fundamental limitation of current unlearning methods: without accounting for
embedding geometry, they risk amplifying bias in retained groups.
Read the full abstract ▾
All authors Yunusa Haruna, Adamu Lawan, Ibrahim Haruna Abdulhamid, Hamza Mohammed Dauda, Jiaquan Zhang, Chaoning Zhang, Shamsuddeen Hassan Muhammad
Computer Vision
OV-Stitcher: A Global Context-Aware Framework for Training-Free Open-Vocabulary Semantic Segmentation
Seungjae Moon, Seunghyun Oh and Youngmin Ro
Training-free open-vocabulary semantic
segmentation(TF-OVSS) has recently attracted
attention for its ability to perform dense prediction by leveraging the pretrained knowledge of large
vision and
vision-
language models, without requiring additional training. However, due to the limited input resolution of these pretrained encoders, existing TF-OVSS methods commonly adopt a sliding-window strategy that processes cropped sub-images independently. While effective for managing high-resolution inputs, this approach prevents global
attention over the full image, leading to fragmented feature representations and limited contextual
reasoning. We propose OV-Stitcher, a training-free framework that addresses this limitation by stitching fragmented sub-image features directly within the final encoder block. By reconstructing
attention representations from fragmented sub-image features, OV-Stitcher enables global
attention within the final encoder block, producing coherent context aggregation and spatially consistent, semantically aligned
segmentation maps. Extensive evaluations across eight
benchmarks demonstrate that OV-Stitcher establishes a scalable and effective solution for open-vocabulary
segmentation, achieving a notable improvement in mean Intersection over Union(mIoU) from 48.7 to 50.7 compared with prior training-free baselines.
Read the full abstract ▾
Computer Vision
EPIR: An Efficient Patch Tokenization, Integration and Representation Framework for Micro-expression Recognition
Junbo Wang, Liangyu Fu, Yuke Li et al. (+3 more)
Micro-expression recognition can obtain the real
emotion of the individual at the current moment. Although deep learning-based methods, especially
Transformer-based methods, have achieved impressive results, these methods have high computational complexity due to the large number of
tokens in the multi-head self-
attention. In addition, the existing micro-expression
datasets are small-scale, which makes it difficult for
Transformer-based models to learn effective micro-expression representations. Therefore, we propose a novel Efficient Patch tokenization, Integration and Representation framework (EPIR), which can balance high recognition performance and low computational complexity. Specifically, we first propose a dual norm shifted tokenization (DNSPT) module to learn the spatial relationship between neighboring pixels in the face region, which is implemented by a refined spatial transformation and dual norm projection. Then, we propose a
token integration module to integrate partial
tokens among multiple cascaded
Transformer blocks, thereby reducing the number of
tokens without information loss. Furthermore, we design a discriminative
token extractor, which first improves the
attention in the
Transformer block to reduce the unnecessary focus of the
attention calculation on self-
tokens, and uses the dynamic
token selection module (DTSM) to select key
tokens, thereby capturing more discriminative micro-expression representations. We conduct extensive experiments on four popular public
datasets (i.e., CASME II, SAMM, SMIC, and CAS(ME)3. The experimental results show that our method achieves significant performance gains over the state-of-the-art methods, such as 9.6% improvement on the CAS(ME)$^3$
dataset in terms of UF1 and 4.58% improvement on the SMIC
dataset in terms of UAR metric.
Read the full abstract ▾
All authors Junbo Wang, Liangyu Fu, Yuke Li, Yining Zhu, Xuecheng Wu, Kun Hu
Computer Vision
Coordinate-Based Dual-Constrained Autoregressive Motion Generation
Kang Ding, Hongsong Wang, Jie Gui et al. (+1 more)
Text-to-motion generation has attracted increasing
attention in the research community recently, with potential applications in animation, virtual reality, robotics, and human-computer interaction.
Diffusion and autoregressive models are two popular and parallel research directions for text-to-motion generation. However,
diffusion models often suffer from error amplification during noise prediction, while autoregressive models exhibit mode collapse due to motion discretization. To address these limitations, we propose a flexible, high-fidelity, and semantically faithful text-to-motion framework, named Coordinate-based Dual-constrained Autoregressive Motion Generation (CDAMD). With motion coordinates as input, CDAMD follows the autoregressive paradigm and leverages
diffusion-inspired multi-layer perceptrons to enhance the fidelity of predicted motions. Furthermore, a Dual-Constrained Causal Mask is introduced to guide autoregressive generation, where motion
tokens act as priors and are concatenated with textual encodings. Since there is limited work on coordinate-based motion synthesis, we establish new
benchmarks for both text-to-motion generation and motion editing. Experimental results demonstrate that our approach achieves state-of-the-art performance in terms of both fidelity and semantic consistency on these
benchmarks.
Read the full abstract ▾
All authors Kang Ding, Hongsong Wang, Jie Gui, Liang Wang