Statistics Applications · Cross-listed from q-fin.GN
Exploring Drivers of Extreme Housing Prices in Australia
Grace Burtenshaw, Ashley Burtenshaw and Meagan Carney
In recent years Australia has observed a growing, unexplained resilience of increasing house price trends. Here, we seek to understand what is driving Australia's indestructible asset using insights from
market experts. We construct a differential equation model of house price to develop intuition for its historical behaviour and responsiveness to changes in mortgage rates. Using this model, we identify a point of 'decoupling' between house price and mortgage rate in the system with supply limitations found to be the main driver for this change. From there, modern extreme value techniques are implemented on real-world data to investigate how the effectiveness of mortgage rate in moderating extreme house price has changed before and after this historical decoupling. We find that without an increase in the housing supply chain, through either deregulation or reduced competition with government building, an 11% increase in mortgage rate will be needed to slow extreme housing costs.
Read the full abstract ▾
Statistics Theory
Bayesian bivariate survival estimation
J.K. Ghosh, Nils Lid Hjort, C. Messan et al. (+1 more)
There is no easy extension of Kaplan-Meier and Nelson-Aalen
estimators to the bivariate case, and estimating bivariate survival distributions nonparametrically is associated with various non-trivial problems. The Dabrowska
estimator will for example associate negative mass to some subsets.
Bayesian methods hold some promise as they will avoid the negative mass problem, butare also prone to difficulties. We simplify and extend an example by Pruitt to show that the posterior distribution from a Dirichlet process prior is inconsistent. We construct a different nonparametric prior via Beta processes and provide an updating scheme that utilizes only the most relevant parts of the likelihood, and show that this leads to a consistent
estimator.
Read the full abstract ▾
All authors J.K. Ghosh, Nils Lid Hjort, C. Messan, R.V. Ramamoorthi
Statistics Theory
Confidence envelopes for the false discoveries with heterogeneous data
Romain Périer, Gilles Blanchard, Sebastian Döhler et al. (+2 more)
In the context of selective inference, confidence envelopes for the false discoveries allow the user to select any subset of null hypotheses while having a statistical guarantee on the number of false discoveries in the selected set. Many constructions of such envelopes have been proposed recently, using local test families (Genovese and Wasserman, 2006; Goeman and Solari, 2011), paths (Katsevich and Ramdas, 2020) or interpolation (Blanchard et al., 2020a). All those methods have in common that they have been well-studied for the homogeneous case where all p-values under the null have a uniform distribution over [0, 1]. However, in many applications the data are heterogeneous and discrete, hence the p-values have heterogeneous, discrete distributions, and the previous constructions may incur a loss of power, in the sense that they over-estimate the number of false discoveries. In this paper, we bridge the previous constructions under the homogeneous case with new tools. We also apply these tools to propose several confidence envelopes based on tools tailored for heterogeneous data, like the Bretagnolle inequality, or a new variant of the Simes inequality. We compare these new envelopes to their homogeneous counterparts on simulated data.
Read the full abstract ▾
All authors Romain Périer, Gilles Blanchard, Sebastian Döhler, Guillermo Durand, Etienne Roquain
Statistical Methodology
Partial correlation networks of Gaussian processes
Michele Peruzzi
In Gaussian graphical models, conditional independence and partial correlations are natural inferential targets for understanding direct relationships in multivariate data. No comparable framework exists for spatial processes, where multivariate analysis defaults to modeling unconditional cross-covariance structure, even when direct relationships remain of scientific interest. We address this gap by establishing a novel characterization of process-level partial correlation for multivariate Gaussian processes that recovers a direct link with Gaussian graphical models. Our analysis proceeds through a class of stationary multivariate processes, termed spectrally inside-out, in which a precision
matrix modulates the strength of conditional dependence and yields necessary and sufficient conditions for conditional independence. Within this class, partial cross-correlation functions factorize into a process-level partial correlation coefficient and an attenuation term independent of cross-process parameters. The spectrally inside-out class includes the separable coregionalization model, a process convolution construction, and the parsimonious multivariate Matérn, for which such a characterization was previously thought unavailable. We further show that a nonstationary inside-out model satisfies the same factorization and admits the same necessary and sufficient conditions. Our results clarify the limitations of existing approaches: linear coregionalization models encode conditional independence through the zero pattern of the inverse factor loading
matrix and do not result in interpretable partial cross-correlation functions. Low-rank spatial factor models lack a meaningful graphical characterization. Methods that enforce network structure through auxiliary graphical layers only characterize presence or absence of graph edges. We illustrate our results through synthetic and real data.
Read the full abstract ▾
Statistics Theory
On a copula product characterizing independence and perfect functional dependence
Jonathan Ansari
Recently studied dependence measures take values in [0,1], where 0 characterizes independence of X and Y, and 1 characterizes perfect functional (not necessarily monotone) dependence of Y on X. The most prominent example is Chatterjee's rank correlation, which is based on the concept of conditional independence. In contrast, we show that dependence measures such as Wasserstein correlations and rearranged dependence measures are naturally linked to the concept of conditional comonotonicity. This connection is captured by a new copula product T(C)=C\vee\Pi which models conditional comonotonicity and characterizes independence and perfect functional dependence. As a main contribution, we prove that the mapping T acts as a reflection on the class of stochastically increasing copulas. Further, T^2=T \circ T projects a copula onto its increasing rearranged copula. To better understand the behavior of such dependence measures, we also study fixed points, ordering results, and continuity properties of T. Our results demonstrate that conditional comonotonicity is a rather intrinsic feature of dependence measures, whereas conditional independence underlying Chatterjee's rank correlation is a quite exceptional property.
Read the full abstract ▾
Statistical Methodology
Performance of weakly-supervised electronic health record-based phenotyping methods in rare-outcome settings
Yunjing Hong, Jennifer C. Nelson and Brian D. Williamson
Accurately identifying patients with specific medical conditions is a key challenge when using clinical data from electronic health records. Our objective was to comprehensively assess when weakly-supervised prediction methods, which use silver-standard labels (proxy measures of the true outcome) rather than gold-standard true labels, perform well in rare-outcome settings like vaccine safety studies. We compared three methods (PheNorm, MAP, and sureLDA) that combine structured features and features derived from clinical text using natural
language processing, through an extensive simulation study with data-generating mechanisms ranging from simple to complex, varying outcome rates, and varying degrees of informative silver labels. We also considered using predicted probabilities to design a chart review validation study. No single method dominated the other across all prediction performance metrics. Probability-guided sampling selected a cohort enriched for patients with more mentions of important concepts in chart notes. SureLDA, the most complex of the three
algorithms we considered, often performed well in simulations. Performance depended greatly on selected tuning parameters. Care should be taken when using weakly-supervised prediction methods in rare-outcome settings, particularly if the probabilities will be used in downstream analysis, but these methods can work well when silver labels are strong predictors of true outcomes.
Read the full abstract ▾
Statistical Methodology
Mixed Membership Models for Multilevel Functional Data
Donatello Telesca, Nicholas Marco and Emma Landry
Mixed membership models extend classical clustering by substituting the notion of uncertain membership with the notion of mixed membership. In particular, these models allow each observation to partially belong to multiple pure membership classes. We discuss mixed membership models for functional data by extending the framework to multilevel functional observations. We show how the classical multivariate Karhunen-Loeve decomposition can be translated into a simple hierarchical model for scalable and flexible expressivity of the underlying
stochastic processes. The identifiability of partial membership structures is aided by the definition of a hierarchical repulsive prior on the
unitary simplex. Our work is motivated and illustrated by applications to a study on functional
brain imaging through electroencephalography (
EEG) of children with autism spectrum disorder (ASD).
Read the full abstract ▾
Machine Learning (Statistics) · Cross-listed from cs.LG
Last-Iterate Convergence of Randomized Kaczmarz and SGD with Greedy Step Size
Michał Dereziński and Xiaoyu Dong
We study last-iterate convergence of SGD with greedy step size over smooth quadratics in the interpolation regime, a setting which captures the classical Randomized Kaczmarz
algorithm as well as other popular iterative linear system solvers. For these methods, we show that the $t$-th iterate attains an $O(1/t^{3/4})$ convergence rate, addressing a question posed by Attia, Schliserman, Sherman, and Koren, who gave an $O(1/t^{1/2})$ guarantee for this setting. In the proof, we introduce the family of
stochastic contraction processes, whose behavior can be described by the
evolution of a certain deterministic
eigenvalue equation, which we analyze via a careful discrete-to-continuous reduction.
Read the full abstract ▾
Statistical Methodology
crumble: A comprehensive framework for modern causal mediation analysis with intermediate confounding
Richard Liu, Nicholas T. Williams, Kara E. Rudolph et al. (+1 more)
Causal mediation analysis is widely used to investigate how causal effects operate through specific pathways linking treatments or exposures to outcomes. Recently, crumble was developed to enable nonparametric estimation of several mediation parameters, even when mediators are continuous and/or multi-dimensional or when treatments are non-binary. But a practical and accessible guide to using crumble – one that does not require deep familiarity with mediation analysis or semiparametric theory – is currently lacking. This tutorial aims to an accessible introduction to crumble while minimizing technical complexity. We first review the mediation parameters implemented in crumble – natural direct and indirect effects, randomized interventional effects, and recanting-twin effects. For each, we give the definition, interpretation, identification assumptions, and suitability in the presence or absence of intermediate confounding. Then, we demonstrate the usage of crumble by examining an example configuration. Next, we describe how crumble accommodates non-binary treatments through modified treatment policies. Finally, we illustrate the practical use of crumble through two case studies – one with a binary treatment and one with a non-binary treatment – based on the Job Search Intervention Study data.
Read the full abstract ▾
All authors Richard Liu, Nicholas T. Williams, Kara E. Rudolph, Ivan Diaz
Statistics Applications
Evaluating the impact of longitudinal treatment strategies in the presence of informative monitoring and time-dependent confounding
Leah Pirondini, Karla Diaz-Ordaz, Edward Palmer et al. (+1 more)
Routinely collected data from electronic health records (EHR) provide opportunities to study effects of longitudinal treatment strategies in real-world clinical settings. A challenge presented by EHR data is that frequency of
covariate monitoring differs by patient,
covariate type and over time, and may be informative about a patient's health status. Many causal inference methods assume measurements of
covariates are observed at a common set of regular time points. In this paper we describe and evaluate methods for estimating causal effects of longitudinal treatments on time-to-event outcomes in the presence of informative monitoring of time-dependent confounders. We show how methods based on inverse probability weighting, G-computation and longitudinal targeted maximum likelihood estimation (TMLE) can be adapted to allow for informative monitoring by incorporating monitoring indicator variables as additional time-dependent confounders. We evaluate these methods using a simulation study, comparing against more simple approaches that ignore monitoring variables. We demonstrate that ignoring monitoring can result in biased estimates of treatment effects. The methods are illustrated through an investigation into the effect of early versus delayed initiation of invasive mechanical ventilation on mortality of intensive care patients using routinely-collected data from an intensive care unit. We consider static treatment strategies such as `always treat' and `never treat' but also generalise to treatment strategies that allow for flexibility in the exact initiation time and duration of treatment.
Read the full abstract ▾
All authors Leah Pirondini, Karla Diaz-Ordaz, Edward Palmer, Ruth H. Keogh
Statistics Applications
Blume-Capel model: Estimation of a three stable state network for $-\bf 1$, $\bf 0$ and $\bf +1$ data
Lourens Waldorp, Jonas Dalege, Maarten Marsman et al. (+3 more)
An extension of the Ising model is proposed as a viable alternative for data with values $-1$, $0$ and $+1$ in the inverse problem, i.e., estimation of the parameters. This model is called the Blume-Capel (BC) model, adapted from physics for small networks. The advantage of the BC model is not only the fact that it is possible to have a neutral (centrist) position on the response scale, but also that this model allows for three stable states. We illustrate magnetisation properties of the BC model using simulations and mean field results. For estimation of the BC parameters, we show that the BC model is part of the exponential family of distributions and show that the model is identified, except for the (inverse) temperature. We then show that combining pseudo-likelihood with lasso yields accurate parameter recovery for the BC model, even in small networks. Moreover, confidence intervals with good coverage properties can be obtained using the desparsified lasso together with sandwich and shrinkage techniques. We apply the methods to data obtained from the online platform Stemwijzer, intended to aid people in deciding for whom to vote.
Read the full abstract ▾
All authors Lourens Waldorp, Jonas Dalege, Maarten Marsman, Adam Finnemann, Irene Ferri, Han L. J. van der Maas