Papers

Interference–in which a unit's outcome is affected by the treatment of other units–poses significant challenges for the identification and estimation of causal effects. Most existing methods for estimating interference effects assume that the interference networks are known. In many practical settings, this assumption is unrealistic as such networks are typically latent. To address this challenge, we propose a novel framework for identifying and estimating heterogeneous group-level interference effects without requiring a known interference network. Specifically, we assume a shared latent community structure between the observed network and the unknown interference network. We demonstrate that interference effects are identifiable if and only if group-level interference effects are heterogeneous, and we establish the consistency and asymptotic normality of the maximum likelihood estimator (MLE). To handle the intractable likelihood function and facilitate the computation, we propose a Bayesian implementation and show that the posterior concentrates around the MLE. A series of simulation studies demonstrate the effectiveness of the proposed method and its superior performance compared with competitors. We apply our proposed framework to the encounter data of stroke patients from the California Department of Healthcare Access and Information (HCAI) and evaluate the causal interference effects of certain intervention in one hospital on the outcomes of other hospitals.

Zero-shot image anomaly classification (AC) and segmentation (AS) are vital for industrial quality control, detecting defects without prior training data. Existing representation-based methods compare patch features with nearest neighbors in unlabeled test images but struggle with consistent anomalies – similar defects recurring across multiple images – resulting in poor AC/AS performance. We introduce Consistent-Anomaly Detection Graph (CoDeGraph), a novel algorithm that identifies and filters consistent anomalies from similarity computations. Our key insight is that normal patches in industrial images show stable, gradually increasing similarity to other test images, while consistent-anomaly patches exhibit abrupt similarity spikes after exhausting a limited set of similar matches, a phenomenon we term “neighbor-burnout.” CoDeGraph constructs an image-level graph, with images as nodes and edges connecting those with shared consistent-anomaly patterns, using community detection to filter these anomalies. We provide a theoretical foundation using Extreme Value Theory to explain the effectiveness of our approach. Experiments on MVTec AD with the ViT-L-14-336 backbone achieve 98.3% AUROC for AC and AS performance of 66.8% (+4.2%) F1 and 68.1% (+5.4%) AP over state-of-the-art zero-shot methods. Using the DINOv2 backbone further improves segmentation, yielding 69.1% (+6.5%) F1 and 71.9% (+9.2%) AP, demonstrating robustness across architectures.

Matrix factorization is a widely used approach for top-N recommendation and collaborative filtering. When implemented on implicit feedback data (such as clicks), a common heuristic is to upweight the observed interactions. This strategy has been shown to improve performance for certain algorithms. In this paper, we conduct a systematic study of various weighting schemes and matrix factorization algorithms. Somewhat surprisingly, we find that training with unweighted data can perform comparably to, and sometimes outperform, training with weighted data, especially for large models. This observation challenges the conventional wisdom. Nevertheless, we identify cases where weighting can be beneficial, particularly for models with lower capacity and specific regularization schemes. We also derive efficient algorithms for exactly minimizing several weighted objectives that were previously considered computationally intractable. Our work provides a comprehensive analysis of the interplay between weighting, regularization, and model capacity in matrix factorization for recommender systems.

The recent advancements in mathematical modeling of biochemical systems have generated increased interest in sensitivity analysis methodologies. There are two primary approaches for analyzing these mathematical models: the stochastic approach, which employs chemical master equations (CME), and the deterministic approach, which utilizes ordinary differential equations (ODEs). The intractable discrete states present in most biochemical processes render the direct simulation of the CME infeasible. Moment closure approximations are recognized for their numerical efficiency in estimating the statistics of the CME solution. Since classical sensitivity analysis is not directly applicable to stochastic modeling, this work conducts sensitivity analysis using moment-based ordinary differential equations (ODEs) to identify key parameters that significantly influence the dynamics of the model. We conduct numerical tests to evaluate the effectiveness of both local and global sensitivity analyses of the moment-based ODEs. These tests enable us to examine how variations in input parameters influence the model's output.

The tidyverse is a popular meta-package comprising several core R packages to aid in various data science tasks, including data import, manipulation and visualisation. Although functionalities offered by the tidyverse can generally be replicated using other packages, its widespread adoption in both teaching and practice indicates there are factors contributing to its preference, despite some debate over its usage. This suggests that particular aspects, such as interface design, may play a significant role in its selection. Examining the interface design can potentially reveal aspects that aid the design process for developers. While Tidyverse has been lauded for adopting a user-centered design, arguably some elements of the design focus on the work domain instead of the end-user. We examine the Tidyverse interface design via the lens of human computer interaction, with an emphasis on data visualisation and data wrangling, to identify factors that might serve as a model for developers designing their packages. We recommend that developers adopt an iterative design that is informed by user feedback, analysis and complete coverage of the work domain, and ensure perceptual visibility of system constraints and relationships.

Non-negative matrix factorization (NMF) is widely used for dimensionality reduction and interpretable analysis, but standard formulations are unsupervised and cannot directly exploit class labels. Existing supervised or semi-supervised extensions usually incorporate labels only via penalties or graph constraints, still requiring an external classifier. We propose NMF-LAB (Non-negative Matrix Factorization for Label Matrix), which redefines classification as the inverse problem of non-negative matrix tri-factorization (tri-NMF). Unlike joint NMF methods, which reconstruct both features and labels, NMF-LAB directly factorizes the label matrix $Y$ as the observation, while covariates $A$ are treated as given explanatory variables. This yields a direct probabilistic mapping from covariates to labels, distinguishing our method from label-matrix factorization approaches that mainly model label correlations or impute missing labels. Our inversion offers two key advantages: (i) class-membership probabilities are obtained directly from the factorization without a separate classifier, and (ii) covariates, including kernel-based similarities, can be seamlessly integrated to generalize predictions to unseen samples. In addition, unlabeled data can be encoded as uniform distributions, supporting semi-supervised learning. Experiments on diverse datasets, from small-scale benchmarks to the large-scale MNIST dataset, demonstrate that NMF-LAB achieves competitive predictive accuracy, robustness to noisy or incomplete labels, and scalability to high-dimensional problems, while preserving interpretability. By unifying regression and classification within the tri-NMF framework, NMF-LAB provides a novel, probabilistic, and scalable approach to modern classification tasks.

We address the problem of multi-group mean estimation, which seeks to allocate a finite sampling budget across multiple groups to obtain uniformly accurate estimates of their means. Unlike classical multi-armed bandits, whose objective is to minimize regret by identifying and exploiting the best arm, the optimal allocation in this setting requires sampling every group on the order of $\Theta(T)$ times. This fundamental distinction makes exploration-free algorithms both natural and effective. Our work makes three contributions. First, we strengthen the existing results on subgaussian variance concentration using the Hanson-Wright inequality and identify a class of strictly subgaussian distributions that yield sharper guarantees. Second, we design exploration-free non-adaptive and adaptive algorithms, and we establish tighter regret bounds than the existing results. Third, we extend the framework to contextual bandit settings, an underexplored direction, and propose algorithms that leverage side information with provable guarantees. Overall, these results position exploration-free allocation as a principled and efficient approach to multi-group mean estimation, with potential applications in experimental design, personalization, and other domains requiring accurate multi-group inference.

It is often of interest to study the association between covariates and the cumulative incidence of a right-censored time-to-event outcome. When time-varying covariates are measured on a fixed discrete time scale, it is desirable to account for more up-to-date covariates that may inform future censoring. Existing methods rely on stringent parametric assumptions, do not account for informative censoring due to time-varying covariates for continuous time, estimate a marginal survival probability, or do not fully use the discrete-time structure of time-varying covariates. We propose a nonparametric and sequentially doubly robust estimator of the continuous-time survival probability conditional on covariates in this setting. We also show that, in the special case of estimating the marginal survival probability, our estimator is asymptotically efficient. We demonstrate the superior performance of our estimator in a simulation experiment, and apply the method to a COVID-19 vaccine efficacy trial.

Stochastic processes with renewal properties are powerful tools for modeling systems where memory effects and long-time correlations play a significant role. In this work, we study a broad class of renewal processes where a variable's value changes according to a prescribed Probability Density Function (PDF), $p(\xi)$, after random waiting times $\theta$. This model is relevant across many fields, including classical chaos, nonlinear hydrodynamics, quantum dots, cold atom dynamics, biological motion, foraging, and finance. We derive a general analytical expression for the $n$-time correlation function by averaging over process realizations. Our analysis identifies the conditions for stationarity, aging, and long-range correlations based on the waiting time and jump distributions. Among the many consequences of our analysis, two new key results emerge. First, for Poissonian waiting times, the correlation function quickly approaches that of telegraphic noise. Second, for power-law waiting times with $\mu>2$, , \emph{any $n$-time correlation function asymptotically reduces to the two-time correlation evaluated at the earliest and latest time points}. This second result reveals a universal long-time behavior where the system's full statistical structure becomes effectively two-time reducible. Furthermore, if the jump PDF $p(\xi)$ has fat tails, this convergence becomes independent of the waiting time PDF and is significantly accelerated, requiring only modest increases in either the number of realizations or the trajectory lengths. Building upon earlier work that established the universality of the two-point correlation function (i.e., a unique formal expression depending solely on the variance of $\xi$ and on the waiting-time PDF), the present study extends that universality to the full statistical description of a broad class of renewal-type stochastic processes.

Concerns about the misuse and misinterpretation of p-values and statistical significance have motivated alternatives for quantifying evidence. We define a generalized form of Jeffreys's approximate objective Bayes factor (eJAB), a one-line calculation that is a function of the p-value, sample size, and parameter dimension. We establish conditions under which eJAB is model-selection consistent and verify them for ten statistical tests. We assess finite-sample accuracy by comparing eJAB with Markov chain Monte Carlo computed Bayes factors in 12 simulation studies. We then apply eJAB to 71,126 results from ClinicalTrials.gov (CTG) and find that the proportion of findings with $\text{p-value} \le \alpha$ yet $eJAB_{01}>1$ (favoring the null) closely tracks the significance level $\alpha$, suggesting that such contradictions are pointing to the type I errors. We catalog 4,088 such candidate type I errors and provide details for 131 with reported $\text{p-value} \le 0.01$. We also identify 487 instances of the Jeffreys-Lindley paradox. Finally, we estimate that 75% (6%) of clinical trial plans from CTG set $\alpha \ge 0.05$ as the target evidence threshold, and that 35.5% (0.22%) of results significant at $\alpha =0.05$ correspond to evidence that is no stronger than anecdotal under eJAB.

Conformal prediction is a model-free machine learning method for constructing prediction regions at a guaranteed coverage probability level. However, a data scientist often faces three challenges in practice: (i) the determination of a conformal prediction region is only approximate, jeopardizing the finite-sample validity of prediction, (ii) the computation required could be prohibitively expensive, and (iii) the shape of a conformal prediction region is hard to control. This article offers new insights into the relationship among the monotonicity of the non-conformity measure, the monotonicity of the plausibility function, and the exact determination of a conformal prediction region. Based on these new insights, we propose a quadratic-polynomial non-conformity measure that allows a data scientist to circumvent the three challenges simultaneously within the full conformal prediction framework.

Spatio temporal data consist of measurement for one or more raster fields such as weather, traffic volume, crime rate, or disease incidents. Advances in modern technology have increased the number of available information for this type of data hence the rise of multidimensional data. In this paper we take advantage of the multidimensional structure of the data but also its temporal and spatial structure. In fact, we will be using the NCAR Climate Data Gateway website which provides data discovery and access services for global and regional climate model data. The daily values of total precipitation (prec), maximum (tmax), and minimum (tmin) temperature are combined to create a multidimensional data called tensor (a multidimensional array). In this paper, we propose a spatio temporal principal component analysis to initialize CP decomposition component. We take full advantage of the spatial and temporal structure of the data in the initialization step for cp component analysis. The performance of our method is tested via comparison with most popular initialization method. We also run a clustering analysis to further show the performance of our analysis.

Since being proposed in 2006, differential privacy has become a standard method for quantifying certain risks in publishing or sharing analyses of sensitive data. At its heart, differential privacy measures risk in terms of the differences between probability distributions, which is a central topic in information theory. A differentially private algorithm is a channel between the underlying data and the output of the analysis. Seen in this way, the guarantees made by differential privacy can be understood in terms of properties of this channel. In this article we examine a few of the key connections between information theory and the formulation/application of differential privacy, giving an “operational significance” for relevant information measures.

In many applications, one must propagate parameter uncertainty from an earlier (upstream) analysis, available as samples, to subsequent (downstream) analyses without feedback. This problem is called cutting feedback or cut-Bayes, and the cut-posterior, the optimal posterior preserving information-flow constraints, is well characterized. However, sampling from it (e.g., via nested MCMC) is computationally intensive, while existing variational inference methods for cut-Bayes require access to upstream data and model, often unavailable. We propose a modular and provably accurate cut-Bayes approach requiring no access to upstream data or model. We leverage the characterization of the cut-posterior as the minimizer of the expected downstream conditional Kullback-Leibler divergence over the upstream posterior, replacing the expectation with the sample average over upstream draws. Our method, NeVI-Cut (neural variational inference for cut-Bayes), employs conditional normalizing flows as the variational family for downstream parameters. We provide fixed-data convergence rates of NeVI-Cut in terms of the richness of neural architecture and complexity of the cut-posterior. We establish, to our knowledge, first results on uniform Kullback-Leibler approximation rates of conditional distributions by common flow classes, yielding widely applicable fixed-data error rates for variational flows. A stochastic algorithm implements NeVI-Cut efficiently, and we demonstrate its speed and accuracy on multiple applications.

We propose and analyze a continuous-time robust reinforcement learning framework for optimal stopping under ambiguity. In this framework, an agent chooses a robust exploratory stopping time motivated by two objectives: robust decision-making under ambiguity and learning about the unknown environment. Here, ambiguity refers to considering multiple probability measures dominated by a reference measure, reflecting the agent's awareness that the reference measure representing her learned belief about the environment would be erroneous. Using the $g$-expectation framework, we reformulate the optimal stopping problem under ambiguity as a robust exploratory control problem with Bernoulli distributed controls. We then characterize the optimal Bernoulli distributed control via backward stochastic differential equations and, based on this, construct the robust exploratory stopping time that approximates the optimal stopping time under ambiguity. Last, we establish a policy iteration theorem and implement it as a reinforcement learning algorithm. Numerical experiments demonstrate the convergence, robustness, and scalability of our reinforcement learning algorithm across different levels of ambiguity and exploration.

Adaptive experiments improve efficiency by adjusting treatment assignments based on past outcomes, but this adaptivity breaks the i.i.d.\ assumptions that underpin classical asymptotics. At the same time, many questions of interest are distributional, extending beyond average effects. Kernel treatment effects (KTE) provide a flexible framework by representing interventional outcome distributions in an RKHS and comparing them via kernel distances. We present the first kernel-based framework for distributional inference under adaptive data collection. Our method combines doubly robust RKHS scores with a witness function learned on one fold, and performs inference on a second fold using a projected, sequentially normalized scalar statistic with valid type-I error. Experiments show that the resulting procedure is well calibrated and effective for both mean shifts and higher-moment differences, outperforming adaptive baselines limited to scalar effects.

Deep learning-based weather forecasting (DLWF) models leverage past weather observations to generate future forecasts, supporting a wide range of downstream applications, including tropical cyclone (TC) prediction. In this paper, we investigate their vulnerability to adversarial attacks, where subtle perturbations to the upstream forecasts can alter the downstream TC trajectory predictions. Although research into adversarial attacks on DLWF models has grown recently, it remains challenging to craft perturbed upstream forecasts that steer the downstream outputs toward attacker-specified trajectories. First, conventional TC detection systems are opaque, non-differentiable black boxes, making standard gradient-based attacks infeasible. Second, the extreme rarity of TC events leads to severe class imbalance problem, making it difficult to develop attack methods for perturbing upstream forecasts that produce realistic-looking cyclone paths aligned with attacker's target trajectories. To overcome these limitations, we propose Cyc-Attack, a novel method for perturbing the upstream forecasts of DLWF models to generate adversarial trajectories. The proposed method uses a differentiable surrogate model to approximate the TC detector's output, enabling the application of gradient-based attacks. Cyc-Attack also employs a skewness-aware loss function with kernel dilation strategy to address the imbalance problem. Finally, a distance-based gradient weighting scheme and regularization are used to constrain the perturbations and eliminate unrealistic-looking trajectories, thereby making the adversarial upstream forecasts less easily detectable. Our experiments show that Cyc-Attack achieves a higher true positive rate in matching the attacker's target trajectories, along with lower false alarm rates and stealthier perturbations than conventional attack methods.

While looped transformers (termed as Looped-Attn) often outperform standard transformers (termed as Single-Attn) on complex reasoning tasks, the mechanism for this advantage remains underexplored. In this paper, we explain this phenomenon through the lens of loss landscape geometry, inspired by empirical observations of their distinct dynamics at both sample and Hessian levels. To formalize this, we extend the River-Valley landscape model by distinguishing between U-shaped valleys (flat) and V-shaped valleys (steep). Based on empirical observations, we conjecture that the recursive architecture of Looped-Attn induces a landscape-level inductive bias towards River-V-Valley. This inductive bias suggest a better loss convergence along the river due to valley hopping, and further encourage learning about complex patterns compared to the River-U-Valley induced by Single-Attn. Building on this insight, we propose SHIFT (Staged HIerarchical Framework for Progressive Training), a principled training strategy that accelerates the training process of Looped-Attn while achieving comparable performances.

We consider the problem of calibrating a posterior estimator of a categorical latent variable given a fixed-length ternary string using only unlabelled observations and without a specified likelihood. We primarily consider a setting in which the estimator is parametrised by estimated conditional probabilities of elements of the string given the latent variable, with calibration if the estimates are correct. Our motivating application is the `Verbal Autopsy' procedure, whereby a cause of death is probabilistically inferred following a structured interview with associates of the deceased. More generally, our setting applies to circumstances where experts can more readily describe posterior beliefs than likelihoods, due to similarity with diagnostic practices. We argue combinatorially that in general the problem is intractable without a simplifying assumption on the data distribution, though some posterior estimators can be ruled out as incompatible. We propose an assumption of block-conditional independence on substrings, allowing calibration procedures based on substring frequency, imputation, and pairwise and three-way distributions of string elements. We give theoretical results on identifiability, on consistency for distributions of either fixed support or fixed entropy, and on robustness to assumptions, finding essentially that three conditional independence blocks of size at least the number of latent categories are necessary and sufficient for calibration. We empirically evaluate methods on data simulated to resemble realistic verbal autopsy questionnaires, and find substantial promise for the approach in the practical problem of calibrating posterior estimates for causes of death.

Bayesian methods have received increasing attention in medical research, where sensitivity analysis of prior distributions is essential. Such analyses typically require the evaluation of the posterior distribution of a parameter under multiple alternative prior settings. When the posterior distribution of the parameter of interest cannot be derived analytically, the standard approach is to re-fit the model using Markov chain Monte Carlo (MCMC) for each setting, which incurs substantial computational costs. This issue is particularly relevant in tipping-point analysis, in which the posterior must be evaluated across gradually changing degrees of borrowing. Sampling-importance resampling (SIR) provides an efficient alternative by approximating posterior samples under new settings without MCMC re-fitting. Despite its potential computational advantages, the practical performance of SIR in repeated prior-sensitivity analyses, including tipping-point analysis, and in complex Bayesian models used in medical research has not been sufficiently illustrated. In this study, we illustrate the practical utility of SIR through two case studies: one involving tipping-point analysis under external data borrowing and another involving sensitivity analysis for a Bayesian nonparametric model in meta-analysis. In both examples, SIR substantially reduced computation time and produced posterior summaries similar to those obtained by MCMC re-fitting.

We introduce PPOPT - Proximal Policy Optimization using Pretraining, a novel, model-free deep-reinforcement-learning algorithm that leverages pretraining to achieve high training efficiency and stability on very small training samples in physics-based environments. Reinforcement learning agents typically rely on large samples of environment interactions to learn a policy. However, frequent interactions with a (computer-simulated) environment may incur high computational costs, especially when the environment is complex. Our main innovation is a new policy neural network architecture that consists of a pretrained neural network middle section sandwiched between two fully-connected networks. Pretraining part of the network on a different environment with similar physics will help the agent learn the target environment with high efficiency because it will leverage a general understanding of the transferrable physics characteristics from the pretraining environment. We demonstrate that PPOPT outperforms baseline classic PPO on small training samples both in terms of rewards gained and general training stability. While PPOPT underperforms against classic model-based methods such as DYNA DDPG, the model-free nature of PPOPT allows it to train in significantly less time than its model-based counterparts. Finally, we present our implementation of PPOPT as open-source software, available at github.com/Davidrxyang/PPOPT.

Generative models frequently suffer miscalibration, wherein statistics of the sampling distribution, such as the fraction of generations in a given class, deviate from desired values. We frame calibration as a constrained optimization problem and seek the closest model in Kullback-Leibler divergence satisfying a calibration constraint. To address the intractability of imposing these constraints exactly, we introduce two surrogate objectives for fine-tuning: (1) the relax loss, which replaces the constraint with a miscalibration penalty, and (2) the reward loss, which converts calibration into a reward fine-tuning problem. We demonstrate that these approaches substantially reduce calibration error across hundreds of simultaneous constraints and models with up to nine billion parameters, spanning applications in protein design, image generation, and language modeling.

Wasserstein distributionally robust optimization (WDRO) provides a framework for evaluating model robustness through the worst-case expectation in the Wasserstein ball around a reference distribution. We prove that ordering hypotheses by increasing WDRO risk is equivalent to ordering them by decreasing robust accuracy if and only if the loss function is a positive multiple of the zero-one loss plus a label-dependent offset. For a class of loss functions including cross-entropy, we show that a hypothesis with perfect robust accuracy can conversely have a higher WDRO risk than another hypothesis with near zero robust accuracy. To evaluate robust accuracy directly, we develop Wasserstein distributional attacks (WDA), which add a weight generation stage after candidate support search to optimize the allocation of empirical mass under the Wasserstein constraint, rather than assigning each data point's full mass to one returned attack point as existing methods do. We further introduce a support-search algorithm for WDA that finds nearby misclassified inputs. On CIFAR-10, CIFAR-100, and ImageNet, WDA using our search algorithm achieves the lowest adversarial accuracy in most tested settings. Our code is available at https://github.com/OLab-Repo/WDA.

Mendelian randomization (MR) is a pivotal tool in genetics, genomics, and epidemiology, leveraging genetic variants as instrumental variables to infer causal relationships between exposures and outcomes. Traditional MR methods, while powerful, often rely on stringent assumptions such as the absence of feedback loops, which are frequently violated in complex biological networks. In addition, many popular MR approaches focus on only two variables (i.e., one exposure and one outcome), whereas our motivating applications of gene regulatory networks have many variables. In this article, we introduce a novel Bayesian framework for multivariable MR that concurrently addresses unmeasured confounding and feedback loops. Central to our approach is a sparse conditional cyclic graphical model with a sparse error variance-covariance matrix. Two structural priors are employed to enable the modeling and inference of causal relationships as well as latent confounding structures. Our method is designed to operate effectively with summary-level data, facilitating its application in contexts where individual-level data are inaccessible, e.g., due to privacy concerns. It can also account for horizontal pleiotropy, under which we establish the sufficient identifiability conditions. Through extensive simulations and applications to the GTEx and OneK1K data, we demonstrate the superior performance of our approach in recovering biologically plausible causal relationships in the presence of possible feedback loops and unmeasured confounding. Using posterior samples, we further quantify uncertainty in inferred network motifs by computing their posterior probabilities. The R package MR.RGM that implements the proposed method is available on CRAN (https://cran.r-project.org/package=MR.RGM).

State aggregation aims to reduce the computational complexity of solving Markov Decision Processes (MDPs) while preserving the performance of the original system. A fundamental challenge lies in optimizing policies within the aggregated, or abstract, space such that the performance remains optimal in the ground MDP-a property referred to as {"}optimal policy equivalence {"}. This paper presents an abstraction framework based on the notion of homomorphism, in which two Markov chains are deemed homomorphic if their value functions exhibit a linear relationship. Within this theoretical framework, we establish a sufficient condition for the equivalence of optimal policy. We further examine scenarios where the sufficient condition is not met and derive an upper bound on the approximation error and a performance lower bound for the objective function under the ground MDP. We propose Homomorphic Policy Gradient (HPG), which guarantees optimal policy equivalence under sufficient conditions, and its extension, Error-Bounded HPG (EBHPG), which balances computational efficiency and the performance loss induced by aggregation. In the experiments, we validated the theoretical results and conducted comparative evaluations against seven algorithms.

Incomplete multi-view spectral clustering generalizes spectral clustering to multi-view data and simultaneously realizes the partition of multi-view data with missing views. For this category of method, K-means algorithm needs to be performed to generate the clustering result after the procedure of feature extraction. More importantly, the connectivity of samples reflected by the clustering result is not utilized effectively. To overcome these defects, we propose Clustering Result re-Guided Incomplete Multi-view Spectral Clustering (CRG_IMSC). CRG_IMSC obtains the clustering result directly by imposing nonnegative constraint to the extracted feature. Furthermore, it constructs the connectivity matrix according to the result of spectral clustering, and minimizes the residual of self-representation based on the connectivity matrix. A novel iterative algorithm using multiplicative update is developed to solve the optimization problem of CRG_IMSC, and its convergence is proved rigorously. On benchmark datasets, for multi-view data, CRG_IMSC performs better than state-of-the-art clustering methods, and the experimental results also demonstrate the convergence of CRG_IMSC algorithm.

The empirical success of multi-agent reinforcement learning (MARL) has motivated the search for more efficient and scalable algorithms for large scale multi-agent systems. However, existing state-of-the-art algorithms do not fully exploit inter-agent coupling information to develop MARL algorithms. In this paper, we propose a systematic approach to leverage structures in the inter-agent couplings for efficient model-free reinforcement learning. We model the cooperative MARL problem via a Bayesian network and characterize the subset of agents, termed as the value dependency set, whose information is required by each agent to estimate its local action value function exactly. Moreover, we propose a partially decentralized training decentralized execution (P-DTDE) paradigm based on the value dependency set. We theoretically establish that the total variance of our P-DTDE policy gradient estimator is less than the centralized training decentralized execution (CTDE) policy gradient estimator. We derive a multi-agent policy gradient theorem based on the P-DTDE scheme and develop a scalable actor-critic algorithm. We demonstrate the efficiency and scalability of the proposed algorithm on multi-warehouse resource allocation and multi-zone temperature control examples. For dense value dependency sets, we propose an approximation scheme based on truncation of the Bayesian network and empirically show that it achieves a faster convergence than the exact value dependence set for applications with a large number of agents.

When creating choropleth maps, mapmakers often bin (i.e., group, classify) quantitative data values into groups to help show that certain areas fall within a similar range of values. For instance, a mapmaker may divide counties into groups of high, middle, and low life expectancy (measured in years). It is well known that different binning methods (e.g., natural breaks, quantile) yield different groupings, meaning the same data can be presented differently depending on how it is divided into bins. To help guide a wide variety of users, we present a new, open source, web-based, geospatial visualization tool, Exploropleth, that lets users interact with a catalog of established data binning methods, and subsequently compare, customize, and export custom maps. This tool advances the state of the art by providing multiple binning methods in one view and supporting administrative unit reclassification on-the-fly. We interviewed 16 cartographers and geographic information systems (GIS) experts from 13 government organizations, non-government organizations (NGOs), and federal agencies who identified opportunities to integrate Exploropleth into their existing mapmaking workflow, and found that the tool has potential to educate students as well as mapmakers with varying levels of experience. Exploropleth is open-source and publicly available at https://exploropleth.github.io.

As wildfires become increasingly destructive and expensive to control, effective management of active wildfires requires accurate, real-time fire spread predictions. To enhance the forecasting accuracy of active fires, data assimilation plays a vital role by integrating observations (such as remote-sensing data) and fire predictions generated from numerical models. This paper provides a comprehensive investigation on the application of a recently proposed diffusion-model-based filtering algorithm – the Ensemble Score Filter (EnSF) – to the data assimilation problem for real-time active wildfire spread predictions. Leveraging a score-based generative diffusion model, EnSF has been shown to have superior accuracy for high-dimensional nonlinear filtering problems, making it an ideal candidate for the filtering problems of wildfire spread models. Technical details are provided, and our numerical investigations demonstrate that EnSF provides superior accuracy, stability, and computational efficiency, establishing it as a robust and practical method for wildfire data assimilation. Our code has been made publicly available.

We present a new method for the statistical process control of lattice structures using tools from Topological Data Analysis. Motivated by applications in additive manufacturing, such as aerospace components and biomedical implants, where hollow lattice geometries are critical, the proposed framework is based on monitoring the persistent homology properties of parts. Specifically, we focus on homological features of dimensions zero and one, corresponding to connected components and one-dimensional loops, to characterize and detect changes in the topology of lattice structures. A nonparametric hypothesis testing procedure and a control charting scheme are introduced to monitor these features during production. Furthermore, we conduct extensive run-length analysis via various simulated but real-life lattice-structured parts. Our results demonstrate that persistent homology is well-suited for detecting topological anomalies in complex geometries and offers a robust, intrinsically geometrical alternative to other SPC methods for mesh and point data.