arXiv subjects

Nancy Reid

Publications and source records attributed to Nancy Reid.

17 recordsLinked to original sources

arXiv2026

Induced replication and the assessment of models

We study the assessment of semiparametric and other highly-parametrised models from the perspective of foundational principles of parametric statistical inference. In doing so, we highlight the possibility of avoiding the usual semiparametric considerations, which typically require estimation of nuisance components through kernel smoothing or basis expansion, with the associated difficulties of tuning-parameter choice that blur the distinction between estimation and model assessment. A key aspect is the inducement of replication under the postulated model. This can be cast in terms of some non-standard inferential separations, in the vein of Fisherian ancillarity/co-ancillarity and sufficiency/co-sufficiency separations, allowing the replacement of out-of-sample prediction error as a criterion for semiparametric model assessment by a type of within-sample prediction error. Framed in this light are new methodological contributions in multiple example settings, including model assessment for the proportional hazards model, for a time-dependent Poisson process with semiparametric intensity function, and for matched-pair and two-group examples. Also subsumed within the framework is a post-reduction inference approach to the construction of confidence sets of sparse regression models. Numerical work confirms recovery of nominal error rates under the postulated model and high sensitivity to departures in the direction of semiparametric alternatives. We conclude by emphasising open challenges and unifying perspectives.

stat.ME↗

arXiv2026

Robust Regression with Student's T: The Role of Degrees of Freedom

Linear regression estimators are known to be sensitive to outliers, and one alternative to obtain a robust and efficient estimator of the regression parameter is to model the error with Student's $t$ distribution. In this article, we compare estimators of the degrees of freedom parameter in the $t$ distribution using frequentist and Bayesian methods, and then study properties of the corresponding estimated regression coefficient. We also include the comparison with some recommended approaches in the literature, including fixing the degrees of freedom and robust regression using the Huber loss. Our extensive simulations on both synthetic and real data demonstrate that estimating the degrees of freedom via the adjusted profile log-likelihood approach yields regression coefficient estimators with high accuracy, performing comparably to the maximum likelihood estimators where the degrees of freedom are fixed at their true values. These findings provide a detailed synthesis of $t$-based robust regression and underscore a key insight: the proper calibration of the degrees of freedom is as crucial as the choice of the robust distribution itself for achieving optimal performance. The {\tt R} package that implements our method is available at https://github.com/amanda-ng518/RobustTRegression.

stat.ME↗

arXiv2024

Inference for overparametrized hierarchical Archimedean copulas

Hierarchical Archimedean copulas (HACs) are multivariate uniform distributions constructed by nesting Archimedean copulas into one another, and provide a flexible approach to modeling non-exchangeable data. However, this flexibility in the model structure may lead to over-fitting when the model estimation procedure is not performed properly. In this paper, we examine the problem of structure estimation and more generally on the selection of a parsimonious model from the hypothesis testing perspective. Formal tests for structural hypotheses concerning HACs have been lacking so far, most likely due to the restrictions on their associated parameter space which hinders the use of standard inference methodology. Building on previously developed asymptotic methods for these non-standard parameter spaces, we provide an asymptotic stochastic representation for the maximum likelihood estimators of (potentially) overparametrized HACs, which we then use to formulate a likelihood ratio test for certain common structural hypotheses. Additionally, we also derive analytical expressions for the first- and second-order partial derivatives of two-level HACs based on Clayton and Gumbel generators, as well as general numerical approximation schemes for the Fisher information matrix.

stat.ME↗

arXiv2024

A Semiparametric Approach to Causal Inference

In causal inference, an important problem is to quantify the effects of interventions or treatments. Many studies focus on estimating the mean causal effects; however, these estimands may offer limited insight since two distributions can share the same mean yet exhibit significant differences. Examining the causal effects from a distributional perspective provides a more thorough understanding. In this paper, we employ a semiparametric density ratio model (DRM) to characterize the counterfactual distributions, introducing a framework that assumes a latent structure shared by these distributions. Our model offers flexibility by avoiding strict parametric assumptions on the counterfactual distributions. Specifically, the DRM incorporates a nonparametric component that can be estimated through the method of empirical likelihood (EL), using the data from all the groups stemming from multiple interventions. Consequently, the EL-DRM framework enables inference of the counterfactual distribution functions and their functionals, facilitating direct and transparent causal inference from a distributional perspective. Numerical studies on both synthetic and real-world data validate the effectiveness of our approach.

stat.ME↗

arXiv2024

On the role of parametrization in models with a misspecified nuisance component

The paper is concerned with inference for a parameter of interest in models that share a common interpretation for that parameter but that may differ appreciably in other respects. We study the general structure of models under which the maximum likelihood estimator of the parameter of interest is consistent under arbitrary misspecification of the nuisance part of the model. A specialization of the general results to matched-comparison and two-groups problems gives a more explicit and easily checkable condition in terms of a new notion of symmetric parametrization, leading to a broadening and unification of existing results in those problems. The role of a generalized definition of parameter orthogonality is highlighted, as well as connections to Neyman orthogonality. The issues involved in obtaining inferential guarantees beyond consistency are briefly discussed.

math.ST↗

arXiv2023

Gaussian variational approximation with composite likelihood for crossed random effect models

Composite likelihood usually ignores dependencies among response components, while variational approximation to likelihood ignores dependencies among parameter components. We derive a Gaussian variational approximation to the composite log-likelihood function for Poisson and Gamma regression models with crossed random effects. We show consistency and asymptotic normality of the estimates derived from this approximation and support this theory with some simulation studies. The approach is computationally much faster than a Gaussian variational approximation to the full log-likelihood function.

math.ST↗

arXiv2023

Variability of extragalactic X-ray jets on kiloparsec scales

Unexpectedly strong X-ray emission from extragalactic radio jets on kiloparsec scales has been one of the major discoveries of Chandra, the only X-ray observatory capable of sub-arcsecond-scale imaging. The origin of this X-ray emission, which appears as a second spectral component from that of the radio emission, has been debated for over two decades. The most commonly assumed mechanism is inverse Compton upscattering of the Cosmic Microwave Background (IC-CMB) by very low-energy electrons in a still highly relativistic jet. Under this mechanism, no variability in the X-ray emission is expected. Here we report the detection of X-ray variability in the large-scale jet population, using a novel statistical analysis of 53 jets with multiple Chandra observations. Taken as a population, we find that the distribution of p-values from a Poisson model is strongly inconsistent with steady emission, with a global p-value of 1.96e-4 under a Kolmogorov-Smirnov test against the expected Uniform (0,1) distribution. These results strongly imply that the dominant mechanism of X-ray production in kpc-scale jets is synchrotron emission by a second population of electrons reaching multi-TeV energies. X-ray variability on the time-scale of months to a few years implies extremely small emitting volumes much smaller than the cross-section of the jet.

astro-ph.HE↗

arXiv2022

A universal robustification procedure

We develop a procedure that transforms any asymptotically normal estimator into an asymptotically normal estimator whose distribution is robust to arbitrary data contamination. More generally, our procedure transforms any estimator whose asymptotic distribution has positive and continuous density at the origin into an asymptotically normal estimator whose distribution is robust to arbitrary contamination. In developing such a procedure we prove new general properties of componentwise and geometric quantiles in both finite and infinite dimensions.

math.ST↗

arXiv2022

Asymptotic Behaviour of the Modified Likelihood Root

We examine the normal approximation of the modified likelihood root, an inferential tool from higher-order asymptotic theory, for the linear exponential and location-scale family. We show that the $r^\star$ statistic can be thought of as a location and scale adjustment to the likelihood root $r$ up to $O_p(n^{-3/2})$, and more generally $r^\star$ can be expressed as a polynomial in $r$. We also show the linearity of the modified likelihood root in the likelihood root for these two families.

stat.ME↗

arXiv2021

Laplace and Saddlepoint Approximations in High Dimensions

We examine the behaviour of the Laplace and saddlepoint approximations in the high-dimensional setting, where the dimension of the model is allowed to increase with the number of observations. Approximations to the joint density, the marginal posterior density and the conditional density are considered. Our results show that under the mildest assumptions on the model, the error of the joint density approximation is $O(p^4/n)$ if $p = o(n^{1/4})$ for the Laplace approximation and saddlepoint approximation, and $O(p^3/n)$ if $p = o(n^{1/3})$ under additional assumptions on the second derivative of the log-likelihood. Stronger results are obtained for the approximation to the marginal posterior density.

math.ST↗

arXiv2021

Inference in High-dimensional Linear Regression

This paper develops an approach to inference in a linear regression model when the number of potential explanatory variables is larger than the sample size. The approach treats each regression coefficient in turn as the interest parameter, the remaining coefficients being nuisance parameters, and seeks an optimal interest-respecting transformation, inducing sparsity on the relevant blocks of the notional Fisher information matrix. The induced sparsity is exploited through a marginal least squares analysis for each variable, as in a factorial experiment, thereby avoiding penalization. One parameterization of the problem is found to be particularly convenient, both computationally and mathematically. In particular, it permits an analytic solution to the optimal transformation problem, facilitating theoretical analysis and comparison to other work. In contrast to regularized regression such as the lasso and its extensions, neither adjustment for selection nor rescaling of the explanatory variables is needed, ensuring the physical interpretation of regression coefficients is retained. Recommended usage is within a broader set of inferential statements, so as to reflect uncertainty over the model as well as over the parameters. The considerations involved in extending the work to other regression models are briefly discussed.

stat.ME↗

arXiv2021

The Tangent Exponential Model

The likelihood function is central to both frequentist and Bayesian formulations of parametric statistical inference, and large-sample approximations to the sampling distributions of estimators and test statistics, and to posterior densities, are widely used in practice. Improved approximations have been widely studied and can provide highly accurate inferences when samples are small or there are many nuisance parameters. This article reviews improved approximations based on the tangent exponential model developed in a series of articles by D.~A.~S.~Fraser and co-workers, attempting to explain the theoretical basis of this model and to provide a guide to the associated literature, including a partially-annotated bibliography.

stat.ME↗

arXiv2021

Directional tests in Gaussian graphical models

Directional tests to compare incomplete undirected graphs are developed in the general context of covariance selection for Gaussian graphical models. The exactness of the underlying saddlepoint approximation is proved for chordal graphs and leads to exact control of the size of the tests, given that the only approximation error involved is due to the numerical calculation of two scalar integrals. Although exactness is not guaranteed for non-chordal graphs, the ability of the saddlepoint approximation to control the relative error leads the directional test to overperform its competitors even in these cases. The accuracy of our proposal is verified by simulation experiments under challenging scenarios, where inference via standard asymptotic approximations to the likelihood ratio test and some of its higher-order modifications fails. The directional approach is used to illustrate the assessment of Markovian dependencies in a dataset from a veterinary trial on cattle. A second example with microarray data shows how to select the graph structure related to genetic anomalies due to acute lymphocytic leukemia.

stat.ME↗

arXiv2018

A New Look at $F$-Tests

Directional inference for vector parameters based on higher order approximations in likelihood inference has recently been developed in the literature. Here we explore examples of directional inference where the calculations can be simplified. It is found that in several classical situations the directional test is equivalent to the usual $F$-test. These situations include testing the ratio of group variances and rates from normal and exponential distributions, linear regression with a linear constraint and Hotelling's $T^2$-test.

math.ST↗

arXiv2016

Note on information bias and efficiency of composite likelihood

Does the asymptotic variance of the maximum composite likelihood estimator of a parameter of interest always decrease when the nuisance parameters are known? Will a composite likelihood necessarily become more efficient by incorporating addi- tional independent component likelihoods, or by using component likelihoods with higher dimension? In this note we show through illustrative examples that the an- swer to both questions is no, and indeed the opposite direction might be observed. The role of information bias is highlighted to understand the occurrence of these paradoxical phenomenon.

math.ST↗

arXiv2015

Statistical Inference, Learning and Models in Big Data

The need for new methods to deal with big data is a common theme in most scientific fields, although its definition tends to vary with the context. Statistical ideas are an essential part of this, and as a partial response, a thematic program on statistical inference, learning, and models in big data was held in 2015 in Canada, under the general direction of the Canadian Statistical Sciences Institute, with major funding from, and most activities located at, the Fields Institute for Research in Mathematical Sciences. This paper gives an overview of the topics covered, describing challenges and strategies that seem common to many different areas of application, and including some examples of applications to make these challenges and strategies more concrete.

stat.ML↗

arXiv2013

Aspects of likelihood inference

I review the classical theory of likelihood based inference and consider how it is being extended and developed for use in complex models and sampling schemes.

math.ST↗