Computer Vision · Cross-listed from cs.CY
Happy Young Women, Grumpy Old Men? Emotion-Driven Demographic Biases in Synthetic Face Generation
Mengting Wei, Aditya Gulati, Guoying Zhao et al. (+1 more)
Synthetic faces from text-to-image (T2I) models pervade digital media, yet their demographic biases under emotionally conditioned
prompts remain poorly understood. We aim to systematically audit how emotionally conditioned
prompts affect demographic and perceived-attractiveness biases in synthetic faces generated by T2I models, with particular
attention to intersectional patterns and cross-ecosystem differences across model families. We audited eight (4 Western and 4 Chinese) T2I models and generated 56,000 faces under seven
prompt conditions: a neutral baseline and six
emotion conditions. We quantified biases in gender, race, age, and perceived attractiveness using information-theoretic divergence metrics. We further conducted intersectional analyses across combined demographic attributes and compared patterns between the Western and Chinese model groups to assess cross-ecosystem consistency and divergence in bias behavior. All models show strong overrepresentation of young faces, and most also overrepresent White-coded individuals. Intersectional analysis reveals compound underrepresentation or near-erasure of specific demographic combinations, such as young x female x Black faces, which are largely absent across models and are not captured by single-attribute audits.
Emotion prompts act as additional demographic selectors: negatively valenced
emotions (including sadness and fear) consistently shift outputs toward White, middle-aged, male-coded faces. This produces a valence-driven mapping that is also associated with lower perceived attractiveness in generated faces. These findings indicate that demographic bias in T2I face generation is both pervasive and shaped by emotional conditioning. They underscore the need for intersectional,
emotion-conditioned, and
multilingual demographic audits as part of standard pre-deployment evaluation practices.
Read the full abstract ▾
All authors Mengting Wei, Aditya Gulati, Guoying Zhao, Nuria Oliver
Computer Vision
Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models
Raphi Kang, Hongqiao Chen, Georgia Gkioxari et al. (+1 more)
Spatio-temporal
reasoning is a remarkable capability of
Vision Language Models (VLMs), but the underlying mechanisms of such abilities remain largely opaque. We postulate that visual/geometrical and textual representations of spatial structure must be combined at some point in VLM computations. We search for such confluence, and ask whether the identified representation can causally explain aspects of input-output model behavior through a linear model. We show empirically that VLMs encode object locations by linearly binding spatial IDs to textual activations, then perform
reasoning via
language tokens. Through rigorous causal interventions we demonstrate that these IDs, which are ubiquitous across the model, can systematically mediate model beliefs at intermediate VLM layers. Additionally, we find that spatial IDs serve as a diagnostic tool for identifying limitations in existing VLMs, and as a valuable learning signal. We extend our analysis to
video VLMs and identify an analogous linear temporal ID mechanism. By characterizing our proposed spatiotemporal ID mechanism, we elucidate a previously underexplored internal
reasoning process in VLMs, toward improved interpretability and the principled design of more aligned and capable models. We release our code for reproducibility: https://github.com/Raphoo/linear-mech-vlms.
Read the full abstract ▾
All authors Raphi Kang, Hongqiao Chen, Georgia Gkioxari, Pietro Perona
Computer Vision · Cross-listed from cs.LG
Towards Robust Universal Perturbation Attacks: A Float-Coded, Penalty-Driven Evolutionary Approach
Shiqi Wang, Mahdi Khosravy, Neeraj Gupta et al. (+1 more)
Universal
adversarial perturbations (UAPs) have garnered significant
attention due to their ability to undermine deep neural networks across multiple inputs using a single noise pattern. Evolutionary
algorithms offer a promising approach to generating such perturbations due to their ability to navigate non-convex, gradient-free landscapes. In this work, we introduce a float-coded, penalty-driven single-objective evolutionary framework for UAP generation that achieves lower visibility perturbations while enhancing attack success rates. Our approach leverages continuous gene representations aligned with contemporary deep learning scales, incorporates dynamic evolutionary operators with adaptive scheduling, and utilizes a modular PyTorch implementation for seamless integration with modern architectures. Additionally, we ensure the universality of the generated perturbations by testing across diverse models and by periodically switching batches to prevent overfitting. Experimental results on the ImageNet
dataset demonstrate that our framework consistently produces perturbations with smaller norms, higher misclassification effectiveness, and faster convergence compared to existing evolutionary-based methods. These findings highlight the robustness and scalability of our approach for universal
adversarial attacks across various deep learning architectures.
Read the full abstract ▾
All authors Shiqi Wang, Mahdi Khosravy, Neeraj Gupta, Olaf Witkowski
Computer Vision
Camera Pose Revisited
Władysław Skarbek, Michał Salomonowicz and Michał Król
Estimating the position and orientation of a camera with respect to an observed scene is one of the central problems in computer
vision, particularly in the context of camera calibration and multi-sensor systems. This paper addresses the planar Perspective–$n$–Point problem, with special emphasis on the initial estimation of the pose of a calibration object. As a solution, we propose the PnP-ProCay78
algorithm, which combines the classical quadratic formulation of the reconstruction error with a Cayley parameterization of rotations and least-squares optimization. The key component of the method is a deterministic selection of starting points based on an analysis of the reconstruction error for two canonical vectors, allowing costly solution-space search procedures to be avoided. Experimental validation is performed using data acquired also from high-resolution RGB cameras and very low-resolution thermal cameras in an integrated RGB–IR setup. The results demonstrate that the proposed
algorithm achieves practically the same projection accuracy as optimal SQPnP and slightly higher than IPPE, both prominent PnP-OpenCV procedures. However, PnP-ProCay78 maintains a significantly simpler algorithmic structure. Moreover, the analysis of optimization trajectories in Cayley space provides an intuitive insight into the convergence process, making the method attractive also from a didactic perspective. Unlike existing PnP solvers, the proposed PnP-ProCay78
algorithm combines projection error minimization with an analytically eliminated reconstruction-error surrogate for translation, yielding a hybrid cost formulation that is both geometrically transparent and computationally efficient.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Life, Machine Learning, and the Search for Habitability: Predicting Biosignature Fluxes for the Habitable Worlds Observatory
Mark Moussa, Amber V. Young, Brianna Isola et al. (+4 more)
Future direct-imaging flagship missions, such as NASA's Habitable Worlds Observatory (HWO), face critical decisions in prioritizing observations due to extremely stringent time and resource constraints. In this paper, we introduce two advanced machine-learning architectures tailored for predicting biosignature species fluxes from exoplanetary reflected-light spectra: a
Bayesian Convolutional Neural Network (BCNN) and our novel model architecture, the Spectral Query Adaptive
Transformer (SQuAT). The BCNN robustly quantifies both epistemic and aleatoric uncertainties, offering reliable predictions under diverse observational conditions, whereas SQuAT employs query-driven
attention mechanisms to enhance interpretability by explicitly associating spectral features with specific biosignature species. We demonstrate that both models achieve comparably high predictive accuracy on an augmented
dataset spanning a wide range of exoplanetary conditions, while highlighting their distinct advantages in uncertainty quantification and spectral interpretability. These capabilities position our methods as promising tools for accelerating target triage, optimizing observation schedules, and maximizing scientific return for upcoming flagship missions such as HWO.
Read the full abstract ▾
All authors Mark Moussa, Amber V. Young, Brianna Isola, Vasuda Trehan, Michael D. Himes, Nicholas Wogan, Giada Arney
Computer Vision
PISE: Physics-Anchored Semantically-Enhanced Deep Computational Ghost Imaging for Robust Low-Bandwidth Machine Perception
Tong Wu
We propose PISE, a physics-informed deep ghost imaging framework for low-bandwidth edge perception. By combining adjoint operator initialization with semantic guidance, PISE improves classification accuracy by 2.57% and reduces variance by 9x at 5% sampling.
Computer Vision
Encoding Emotion Through Self-Supervised Eye Movement Reconstruction
Marcus Ma, Jordan Prescott, Emily Zhou et al. (+4 more)
The relationship between emotional expression and eye movement is well-documented, with literature establishing gaze patterns are reliable indicators of
emotion. However, most studies utilize specialized, high-resolution eye-tracking equipment, limiting the potential reach of findings. We investigate how eye movement can be used to predict
multimodal markers of emotional expression from naturalistic, low-resolution
videos. We utilize a collection of
video interviews from the USC Shoah Foundation's Visual History Archive with Holocaust survivors as they recount their experiences in the Auschwitz concentration camp. Inspired by pretraining methods on
language models, we develop a novel gaze detection model that uses self-supervised eye movement reconstruction that can effectively leverage unlabeled
video. We use this model's encoder
embeddings to fine-tune models on two downstream tasks related to emotional expression. The first is aligning eye movement with directional
emotion estimates from
speech. The second task is using eye gaze as a predictor of three momentary manifestations of emotional behaviors: laughing, crying/sobbing, and sighing. We find our new model is predictive of
emotion outcomes and observe a positive correlation between pretraining performance and
emotion processing performance for both experiments. We conclude self-supervised eye movement reconstruction is an effective method for encoding the affective signal they carry.
Read the full abstract ▾
All authors Marcus Ma, Jordan Prescott, Emily Zhou, Tiantian Feng, Kleanthis Avramidis, Gabor Mihaly Toth, Shrikanth Narayanan
Computer Vision
BirdsEye-RU: A Dataset For Detecting Faces from Overhead Images
Md. Ahanaf Arif Khan, Ariful Islam, Sangeeta Biswas et al. (+4 more)
Detecting faces in overhead images remains a significant challenge due to extreme scale variations and environmental clutter. To address this, we created the BirdsEye-RU
dataset, a comprehensive collection of 2,978 images containing over eight thousand annotated faces. This
dataset is specifically designed to capture small and distant faces across diverse environments, containing both drone images and smartphone-captured images from high altitude. We present a detailed description of the BirdsEye-RU
dataset in this paper. We made our
dataset freely available to the public, and it can be accessed at https://www.kaggle.com/
datasets/mdahanafarifkhan/birdseye-ru.
Read the full abstract ▾
All authors Md. Ahanaf Arif Khan, Ariful Islam, Sangeeta Biswas, Md. Iqbal Aziz Khan, Subrata Pramanik, Sanjoy Kumar Chakravarty, Bimal Kumar Pramanik
Computer Vision
XRefine: Attention-Guided Keypoint Match Refinement
Jan Fabian Schmid and Annika Hagemann
Sparse keypoint matching is crucial for 3D
vision tasks, yet current keypoint detectors often produce spatially inaccurate matches. Existing refinement methods mitigate this issue through
alignment of matched keypoint locations, but they are typically detector-specific, requiring retraining for each keypoint detector. We introduce XRefine, a novel, detector-agnostic approach for sub-pixel keypoint refinement that operates solely on image patches centered at matched keypoints. Our cross-
attention-based architecture learns to predict refined keypoint coordinates without relying on internal detector representations, enabling generalization across detectors. Furthermore, XRefine can be extended to handle multi-view feature tracks. Experiments on MegaDepth, KITTI, and ScanNet demonstrate that the approach consistently improves
geometric estimation accuracy, achieving superior performance compared to existing refinement methods while maintaining runtime efficiency. Our code and trained models can be found at https://github.com/boschresearch/xrefine.
Read the full abstract ▾
Computer Vision
Deep Feature Deformation Weights
Richard Liu, Itai Lang and Rana Hanocka
Handle-based mesh deformation is a classic paradigm in computer graphics which enables intuitive edits from sparse controls. Classical techniques are fast and precise, but require users to know ideal handle placement apriori, which can be unintuitive and inconsistent. Handle sets cannot be adjusted easily, as weights are typically optimized through energies defined by the handles. Modern data-driven methods, on the other hand, provide semantic edits but sacrifice fine-grained control and speed. We propose a technique that achieves the best of both worlds: deep feature proximity yields smooth, visual-aware deformation weights with no additional regularization. Importantly, these weights are computed in real-time for any surface point, unlike prior methods which require expensive optimization. We introduce barycentric feature
distillation, an improved feature
distillation pipeline which leverages the full visual signal from shape renders to make
distillation complexity robust to mesh resolution. This enables high resolution meshes to be processed in minutes versus potentially hours for prior methods. We preserve and extend classical properties through feature space constraints and locality weighting. Our field representation enables automatic visual
symmetry detection, which we use to produce
symmetry-preserving deformations. We show a proof-of-concept application which can produce deformations for meshes up to 1 million faces in real-time on a consumer-grade machine. Project page at https://threedle.github.io/dfd.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Deep Lightweight Unrolled Network for High Dynamic Range Modulo Imaging
Brayan Monroy and Jorge Bacca
Modulo-Imaging (MI) offers a promising alternative for expanding the dynamic range of images by resetting the signal intensity when it reaches the saturation level. Subsequently, high-dynamic range (HDR) modulo imaging requires a recovery process to obtain the HDR image. MI is a non-convex and ill-posed problem where recent recovery networks suffer in high-noise scenarios. In this work, we formulate the HDR reconstruction task as an optimization problem that incorporates a deep prior and subsequently unrolls it into an optimization-inspired deep neural network. The network employs a lightweight convolutional denoiser for fast inference with minimal computational overhead, effectively recovering intensity values while mitigating noise. Moreover, we introduce the Scaling Equivariance term that facilitates self-supervised fine-tuning, thereby enabling the model to adapt to new modulo images that fall outside the original training distribution. Extensive evaluations demonstrate the superiority of our method compared to state-of-the-art recovery
algorithms in terms of performance and quality.
Read the full abstract ▾
Computer Vision
Fine-Tuning Cycle-GAN for Domain Adaptation of MRI Images
Mohd Usama, Belal Ahmad and Faleh Menawer R Althiyabi
Magnetic Resonance Imaging (MRI) scans acquired from different scanners or institutions often suffer from domain shifts owing to variations in hardware, protocols, and acquisition parameters. This discrepancy degrades the performance of deep learning models trained on source domain data when applied to target domain images. In this study, we propose a Cycle-GAN-based model for unsupervised medical-image domain adaptation. Leveraging CycleGANs, our model learns bidirectional mappings between the source and target domains without paired training data, preserving the anatomical content of the images. By leveraging Cycle-GAN capabilities with content and disparity loss for adaptation tasks, we ensured image-domain adaptation while maintaining image integrity. Several experiments on MRI
datasets demonstrated the efficacy of our model in bidirectional domain adaptation without labelled data. Furthermore, research offers promising avenues for improving the diagnostic accuracy of healthcare. The statistical results confirm that our approach improves model performance and reduces domain-related variability, thus contributing to more precise and consistent medical image analysis.
Read the full abstract ▾
Computer Vision
CoLR-Det: Collaborative Latent Restoration for Small Object Detection in Low-Resolution Remote Sensing Images
Ruo Qi, Linhui Dai, Yusong Qin et al. (+2 more)
Low-resolution remote sensing small object detection is limited by both missing visual details and the ambiguity of how details serve detection. Existing super-resolution-assisted detectors generally follow a restoration-first paradigm to explicitly enhance inputs before detection, which implicitly assumes visual fidelity benefits recognition. Yet super-resolution favors dense texture and edge recovery, while object detection relies on sparse instance-level semantics, making restoration amplify visually plausible but semantically irrelevant background textures. To tackle this issue, we propose CoLR-Det, a Collaborative Latent-Restoration-Assisted Small Object Detection framework that treats super-resolution supervision as detection-oriented latent regularization rather than explicit image-level enhancement. Instead of reconstructing high-resolution images for inference, CoLR-Det uses a training-only restoration branch to impose auxiliary reconstruction constraints on shared multiscale representations, and the inference pathway remains purely detection-driven. We further design a saliency-guided object-preserving
token routing mechanism, which prioritizes high-saliency
tokens for
attention-based refinement while retaining information of bypassed
tokens. Besides, a detection-prioritized two-stage optimization strategy is developed: it first builds stable object-level semantics before introducing restoration supervision, and assigns a smaller learning rate to the SR decoder to keep its updates conservative and reduce perturbations in collaborative training. With this design, CoLR-Det transforms restoration from an explicit visual enhancement operator into an implicit semantic regularizer. Experiments on resolution-degraded NWPU VHR-10-Split, DOTAv1.5-Split and HRSSD-Split show that CoLR-Det outperforms state-of-the-art methods, with code available at https://github.com/qiruo-ya/CoLR-Det.
Read the full abstract ▾
All authors Ruo Qi, Linhui Dai, Yusong Qin, Chaolei Yang, Yanshan Li
Computer Vision
Video Individual Counting and Tracking from Moving Drones: A Benchmark and Methods
Yaowu Fan, Jia Wan, Tao Han et al. (+3 more)
Counting and tracking dense crowds in large-scale scenes is valuable yet challenging, while existing methods and
datasets are largely limited to fixed cameras with small scene coverage. We introduce MovingDroneCrowd++, a large-scale
video-level
dataset dedicated to dense crowd counting and tracking from moving drones, captured under diverse flight altitudes, camera angles, and illumination conditions. Existing methods, however, still fail to achieve satisfactory
Video Individual Counting (VIC) or Multi-Object Tracking (MOT) performance under these challenging aerial conditions. To this end, we propose GD3A (Global Density Map Decomposition via Group-wise Density Assignment) for VIC and GIA-Track (Group-wise Identity Association Tracker) for MOT. Both methods are unified under a framework that aggregates cross-frame pixel-level matches among multiple descriptors per pedestrian through group-wise density assignment (GDA) and group-wise identity association (GIA), respectively. This group-wise aggregation tolerates intra-group spatial mismatches and limits the propagation of inter-group errors. To establish reliable pixel-level descriptor correspondences across frames, we design a frame-pair-conditioned dustbin score inferred from the pedestrian descriptors of each frame pair, enabling Optimal Transport to better distinguish identity differences. Based on these matches, GD3A decomposes global density maps into shared, inflow, and
outflow components, while GIA-Track establishes robust pedestrian trajectories. Experiments show that our methods achieve substantial gains in both VIC and MOT on moving-drone
videos with dense crowds and complex motions, reducing counting error by 47.4% and improving tracking accuracy by 64.6%. Code,
dataset, and pretrained models are available at https://github.com/fyw1999/MovingDroneCrowd.
Read the full abstract ▾
All authors Yaowu Fan, Jia Wan, Tao Han, Andy J. Ma, Wanli Ouyang, Antoni B. Chan
Computer Vision
Histopath-C: Towards Realistic Domain Shifts for Histopathology Vision-Language Adaptation
Mehrdad Noori, Gustavo Adolfo Vargas Hakim, David Osowiechi et al. (+6 more)
Medical
Vision-
language models (VLMs) have shown remarkable performances in various medical imaging domains such as histo\-pathology by leveraging pre-trained, contrastive models that exploit visual and textual information. However, histopathology images may exhibit severe domain shifts, such as staining, contamination, blurring, and noise, which may severely degrade the VLM's downstream performance. In this work, we introduce Histopath-C, a new
benchmark with realistic synthetic corruptions designed to mimic real-world distribution shifts observed in digital histopathology. Our framework dynamically applies corruptions to any available
dataset and evaluates Test-Time Adaptation (TTA) mechanisms on the fly. We then propose LATTE, a transductive, low-rank adaptation strategy that exploits multiple text templates, mitigating the sensitivity of histopathology VLMs to diverse text inputs. Our approach outperforms state-of-the-art TTA methods originally designed for natural images across a breadth of histopathology
datasets, demonstrating the effectiveness of our proposed design for robust adaptation in histopathology images. Code and data are available at https://github.com/Mehrdad-Noori/Histopath-C.
Read the full abstract ▾
All authors Mehrdad Noori, Gustavo Adolfo Vargas Hakim, David Osowiechi, Fereshteh Shakeri, Ali Bahri, Moslem Yazdanpanah, Sahar Dastani, Ismail Ben Ayed, Christian Desrosiers
Computer Vision
NeuralFur: Animal Fur Reconstruction From Multi-View Images
Vanessa Sklyarova, Berna Kabadayi, Anastasios Yiannakidis et al. (+3 more)
Reconstructing realistic animal fur geometry from images is a challenging task due to the fine-scale details, self-occlusion, and view-dependent appearance of fur. In contrast to human hairstyle reconstruction, there are also no
datasets that can be leveraged to learn a fur prior for different animals. In this work, we present a first multi-view-based method for high-fidelity 3D fur modeling of animals using a strand-based representation, leveraging the general knowledge of a
vision language model. Given multi-view RGB images, we first reconstruct a coarse surface geometry using traditional multi-view stereo techniques. We then use a
vision language model (VLM) system to retrieve information about the realistic length structure of the fur for each part of the body. We use this knowledge to construct the animal's furless geometry and grow strands atop it. The fur reconstruction is supervised with both
geometric and photometric losses computed from multi-view images. To mitigate orientation ambiguities stemming from the Gabor filters that are applied to the input images, we additionally utilize the VLM to guide the strands' growth direction and their relation to the
gravity vector that we incorporate as a loss. With this new schema of using a VLM to guide 3D reconstruction from multi-view inputs, we show generalization across a variety of animals with different fur types. For additional results and code, please refer to https://neuralfur.is.tue.mpg.de.
Read the full abstract ▾
All authors Vanessa Sklyarova, Berna Kabadayi, Anastasios Yiannakidis, Giorgio Becherini, Michael J. Black, Justus Thies
Computer Vision
DCAC: Dynamic Class-Aware Cache Creates Stronger Out-of-Distribution Detectors
Yanqi Wu, Qichao Chen, Runhe Lai et al. (+5 more)
Out-of-distribution (OOD) detection remains a fundamental challenge for deep neural networks, particularly due to overconfident predictions on unseen OOD samples during testing. We reveal a key insight: OOD samples predicted as the same class, or given high probabilities for it, are visually more similar to each other than to the true in-distribution (ID) samples. Motivated by this class-specific observation, we propose DCAC (Dynamic Class-Aware Cache), a training-free, test-time calibration module that maintains separate caches for each ID class to collect high-
entropy samples and calibrate the raw predictions of input samples. DCAC leverages cached visual features and predicted probabilities through a lightweight two-layer module to mitigate overconfident predictions on OOD samples. This module can be seamlessly integrated with various existing OOD detection methods across both unimodal and
vision-
language models while introducing minimal computational overhead. Extensive experiments on multiple OOD
benchmarks demonstrate that DCAC significantly enhances existing methods, achieving substantial improvements, i.e., reducing FPR95 by 6.55% when integrated with ASH-S on ImageNet OOD
benchmark.
Read the full abstract ▾
All authors Yanqi Wu, Qichao Chen, Runhe Lai, Xinhua Lu, Jia-Xin Zhuang, Zhilin Zhao, Wei-Shi Zheng, Ruixuan Wang
Computer Vision
Large-scale EM Benchmark for Multi-Organelle Instance Segmentation in the Wild
Yanrui Lu, Danyang Chen, Haowen Xiao et al. (+12 more)
Accurate instance-level
segmentation of organelles in electron microscopy (EM) is critical for quantitative analysis of subcellular morphology and inter-organelle interactions. However, current
benchmarks, based on small, curated
datasets, fail to capture the inherent heterogeneity and large spatial context of in-the-wild EM data, imposing fundamental limitations on current patch-based methods. To address these limitations, we developed a large-scale, multi-source
benchmark for multi-organelle instance
segmentation, comprising over 100,000 2D EM images across variety
cell types and five organelle classes that capture real-world variability.
Dataset annotations were generated by our designed connectivity-aware Label Propagation
Algorithm (3D LPA) with expert refinement. We further benchmarked several state-of-the-art models, including U-Net, SAM variants, and Mask2Former. Our results show several limitations: current models struggle to generalize across heterogeneous EM data and perform poorly on organelles with global, distributed morphologies (e.g., Endoplasmic Reticulum). These findings underscore the fundamental mismatch between local-context models and the challenge of modeling long-range structural continuity in the presence of real-world variability. The
benchmark dataset and labeling tool will be publicly released soon.
Read the full abstract ▾
All authors Yanrui Lu, Danyang Chen, Haowen Xiao, Jiarui Zhu, Fukang Ge, Binqian Zou, Jiali Guan, Jiayin Liang, Yuting Wang, Ziqian Guan, Xiangcheng Bao, Jinhao Bi, Lin Gu, Jun He, Yingying Zhu
Computer Vision · Cross-listed from cs.CR
Privacy-Preserving Federated Learning with Verifiable Fairness Guarantees
Mohammed Himayath Ali, Mohammed Aqib Abdullah, Syed Muneer Hussain et al. (+2 more)
Federated learning enables collaborative model training across distributed institutions without centralizing sensitive data; however, ensuring algorithmic
fairness across heterogeneous data distributions while preserving
privacy remains fundamentally unresolved. This paper introduces CryptoFair-FL, a novel cryptographic framework providing the first verifiable
fairness guarantees for
federated learning systems under formal security definitions. The proposed approach combines additively homomorphic encryption with secure multi-party computation to enable
privacy-preserving verification of demographic parity and equalized odds metrics without revealing protected attribute distributions or individual predictions. A novel batched verification protocol reduces computational complexity from BigO(n^2) to BigO(n \log n) while maintaining (\dparam, \deltap)-differential
privacy with dparam = 0.5 and deltap = 10^{-6}. Theoretical analysis establishes information-theoretic lower bounds on the
privacy cost of
fairness verification, demonstrating that the proposed protocol achieves near-optimal
privacy-
fairness tradeoffs. Comprehensive experiments across four
benchmark datasets (MIMIC-IV healthcare records, Adult Income, CelebA, and a novel FedFair-100
benchmark) demonstrate that CryptoFair-FL reduces
fairness violations from 0.231 to 0.031 demographic parity difference while incurring only 2.3 times computational overhead compared to standard
federated averaging. The framework successfully defends against attribute inference attacks, maintaining
adversarial success probability below 0.05 across all tested configurations. These results establish a practical pathway for deploying
fairness-aware
federated learning in regulated industries requiring both
privacy protection and algorithmic accountability.
Read the full abstract ▾
All authors Mohammed Himayath Ali, Mohammed Aqib Abdullah, Syed Muneer Hussain, Mohammed Mudassir Uddin, Shahnawaz Alam
Computer Vision
Xiaowei Fu and Lei Zhang
The widespread use of
Vision Language Models (VLMs, e.g. CLIP) has raised concerns about their vulnerability to sophisticated and imperceptible
adversarial attacks. These attacks could compromise model performance and system security in cross-modal tasks. To address this challenge, three main defense paradigms have been proposed: Training-time Defense, Test-time Adaptation Defense, and Training-free Defense. Training-time Defense involves modifying the training process, typically through
adversarial fine-tuning to improve the robustness to
adversarial examples. While effective, this approach requires substantial computational resources and may not generalize across all
adversarial attacks. Test-time Adaptation Defense focuses on adapting the model at inference time by updating its parameters to handle unlabeled
adversarial examples, offering flexibility but often at the cost of increased complexity and computational overhead. Training-free Defense avoids modifying the model itself, instead focusing on altering the
adversarial inputs or their feature
embeddings, which enforces input perturbations to mitigate the impact of attacks without additional training. This survey reviews the latest advancements in
adversarial defense strategies for VLMs, highlighting the strengths and limitations of such approaches and discussing ongoing challenges in enhancing the robustness of VLMs.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Constraint-Aware Neurosymbolic Uncertainty Quantification with Bayesian Deep Learning for Scientific Discovery
Shahnawaz Alam, Mohammed Mudassir Uddin and Mohammed Kaif Pasha
Scientific Artificial Intelligence (AI) applications require models that deliver trustworthy uncertainty estimates while respecting domain constraints. Existing uncertainty quantification methods lack mechanisms to incorporate symbolic scientific knowledge, while neurosymbolic approaches operate deterministically without principled uncertainty modeling. We introduce the Constraint-Aware Neurosymbolic Uncertainty Framework (CANUF), unifying
Bayesian deep learning with differentiable symbolic
reasoning. The architecture comprises three components: automated constraint extraction from scientific literature, probabilistic neural backbone with variational inference, and differentiable constraint satisfaction layer ensuring physical consistency. Experiments on Materials Project (140,000+ materials), QM9 molecular properties, and
climate benchmarks show CANUF reduces Expected Calibration Error by 34.7% versus
Bayesian neural networks while maintaining 99.2% constraint satisfaction. Ablations reveal constraint-guided recalibration contributes 18.3% performance gain, with constraint extraction achieving 91.4% precision. CANUF provides the first end-to-end differentiable pipeline simultaneously addressing uncertainty quantification, constraint satisfaction, and interpretable explanations for scientific predictions.
Read the full abstract ▾
Computer Vision
SkeFi: Cross-Modal Knowledge Transfer for Wireless Skeleton-Based Action Recognition
Shunyu Huang, Yunjiao Zhou and Jianfei Yang
Skeleton-based action recognition leverages human pose keypoints to categorize human actions, which shows superior generalization and interoperability compared to regular end-to-end action recognition. Existing solutions use RGB cameras to annotate skeletal keypoints, but their performance declines in dark environments and raises
privacy concerns, limiting their use in smart homes and hospitals. This paper explores non-invasive wireless sensors, i.e.,
LiDAR and mmWave, to mitigate these challenges as a feasible alternative. Two problems are addressed: (1) insufficient data on wireless sensor modality to train an accurate skeleton estimation model, and (2) skeletal keypoints derived from wireless sensors are noisier than RGB, causing great difficulties for subsequent action recognition models. Our work, SkeFi, overcomes these gaps through a novel cross-modal knowledge transfer method acquired from the data-rich RGB modality. We propose the enhanced Temporal Correlation Adaptive Graph Convolution (TC-AGC) with frame interactive enhancement to overcome the noise from missing or inconsecutive frames. Additionally, our research underscores the effectiveness of enhancing multiscale temporal modeling through dual temporal convolution. By integrating TC-AGC with temporal modeling for cross-modal transfer, our framework can extract accurate poses and actions from noisy wireless sensors. Experiments demonstrate that SkeFi realizes state-of-the-art performances on mmWave and
LiDAR. The code is available at https://github.com/Huang0035/Skefi.
Read the full abstract ▾