arXiv subjects

Juha Karvanen

Publications and source records attributed to Juha Karvanen.

At least 19 recordsLinked to original sources

arXiv2026

Graph-based causal variance decompositions: When "variance explained" means causation

Recursive application of the law of total variance decomposes the marginal variance of an outcome into components attributed to explanatory variables and a residual component. The resulting decomposition depends on the chosen conditioning order, and its components do not in general have causal interpretations. We develop a graph-based framework for defining causal counterparts of ordered variance components and establishing their identification from observed data. Under topological orderings, identification can be assessed component by component against the full causal graph, without requiring the variables included in the decomposition to form a causally sufficient system. We also consider scientifically motivated departures from topological orderings, in which selected intermediate variables are conditioned on to obtain controlled-effect interpretations, motivated by the context of disparities in healthcare delivery. We relate the resulting estimands to causal attribution and variable-importance approaches in machine learning, and propose model-based plug-in estimators together with an approximate Bayesian procedure for uncertainty quantification. A simulation study examines finite-sample performance and sensitivity to outcome-model misspecification and flexible machine-learning estimation.

stat.ME↗

arXiv2026

A context-specific causal model for estimating the effect of extended length of overnight stay on traveller's total expenditure

Tourism significantly affects the economies of many countries. Understanding the causal relationship between the length of overnight stay and traveller's expenditure is crucial for stakeholders to characterize spending profiles and to design marketing strategies. Causal mechanisms differ between personal and work-related travel because the decision-making processes have different drivers and constraints. We apply context-specific independence relations to model causal mechanisms in contexts specified by trip purpose and identify the causal effect of the length of stay on expenditure. Using the international visitor survey data on foreign travellers to Finland, we fit a hierarchical Bayesian model to estimate the posterior distribution of the counterfactual expenditure due to extending the length of stay by one night. We also perform a Bayesian sensitivity analysis of the estimated causal effect with respect to omitted variable bias.

stat.AP↗

arXiv2025

Complementary strengths of the Neyman-Rubin and graphical causal frameworks

This article contributes to the discussion on the relationship between the Neyman-Rubin and the graphical frameworks for causal inference. We present specific examples of data-generating mechanisms - such as those involving undirected or deterministic relationships and cycles - where analyses using a directed acyclic graph are challenging, but where the tools from the Neyman-Rubin causal framework are readily applicable. We also provide examples of data-generating mechanisms with M-bias, trapdoor variables, and complex front-door structures, where the application of the Neyman-Rubin approach is complicated, but the graphical approach is directly usable. The examples offer insights into commonly used causal inference frameworks and aim to improve comprehension of the languages for causal reasoning among a broad audience.

stat.ME↗

arXiv2025

Clustering and Pruning in Causal Data Fusion

Data fusion, the process of combining observational and experimental data, can enable the identification of causal effects that would otherwise remain non-identifiable. Although identification algorithms have been developed for specific scenarios, do-calculus remains the only general-purpose tool for causal data fusion, particularly when variables are present in some data sources but not others. However, approaches based on do-calculus may encounter computational challenges as the number of variables increases and the causal graph grows in complexity. Consequently, there exists a need to reduce the size of such models while preserving the essential features. For this purpose, we propose pruning (removing unnecessary variables) and clustering (combining variables) as preprocessing operations for causal data fusion. We generalize earlier results on a single data source and derive conditions for applying pruning and clustering in the case of multiple data sources. We give sufficient conditions for inferring the identifiability or non-identifiability of a causal effect in a larger graph based on a smaller graph and show how to obtain the corresponding identifying functional for identifiable causal effects. Examples from epidemiology and social science demonstrate the use of the results.

stat.ML↗

arXiv2024

Monotone Missing Data: A Blessing and a Curse

Monotone missingness is commonly encountered in practice when a missing measurement compels another measurement to be missing. Because of the simpler missing data pattern, monotone missing data is often viewed as beneficial from the perspective of practical data analysis. However, in graphical missing data models, monotonicity has implications for the identifiability of the full law, i.e., the joint distribution of actual variables and response indicators. In the general nonmonotone case, the full law is known to be nonparametrically identifiable if and only if specific graphical structures are not present. We show that while monotonicity may enable the identification of the full law despite some of these structures, it also prevents the identification in certain cases that are identifiable without monotonicity. The results emphasize the importance of proper treatment of monotone missingness in the analysis of incomplete data.

stat.ME↗

arXiv2024

Multiple Imputation Guided by Full Law and Target Law Identifiability under the Missing Not At Random Assumption

The central challenges in missing data models concern the identifiability of two distributions: the target law and the full law. The target law refers to the joint distribution of the data variables, whereas the full law refers to the joint distribution of the data variables and their corresponding response indicators. However, the relationship between the identifiability of these two distributions and the feasibility of multiple imputation has not been clearly established when the data are missing not at random (MNAR). We present a procedure in which the choice of imputation method is guided by identifiability considerations. The identifiability of the full law implies the applicability of conditionally complete imputation methods that draw imputations for all missing data patterns. By contrast, the non-identifiability of the full law implies that any multiple imputation method aiming to implement conditionally complete imputation will produce biased estimates, thereby also restricting the options for estimating the target law. We demonstrate that alternative imputation strategies can sometimes enable the estimation of the target law in such cases. Specifically, we introduce factorizable imputation where certain observed values are also imputed and the imputed data are weighted in the analysis.

math.ST↗

arXiv2024

Dynamic programming principle in cost-efficient sequential design: optimal update scheduling under cost constraints

We study sequential cost-efficient design in a situation where each update of covariates involves a fixed time cost typically considerable compared to a single measurement time. The problem arises from parameter estimation in switching measurements on superconducting Josephson junctions which are components needed in quantum computers and other superconducting electronics. In switching measurements, a sequence of current pulses is applied to the junction and a binary voltage response is observed. The measurement requires a very low temperature that can be kept stable only for a relatively short time, and therefore it is essential to use an efficient design. We use the dynamic programming principle from the mathematical theory of optimal control to solve the optimal update times. Specifically, we give a formulation for $D$-optimal experimental design based on the dynamic programming principle with a binary response model. Our simulations demonstrate the cost-efficiency compared to the previously used methods.

stat.ME↗

arXiv2024

Full Law Identification under Missing Data with Categorical Variables

Missing data may be disastrous for the identifiability of causal and statistical estimands. In graphical missing data models, colluders are dependence structures that have a special importance for identification considerations. It has been shown that the presence of a colluder makes the full law, i.e., the joint distribution of variables and response indicators, non-parametrically non-identifiable. However, when the variables related to the colluder structure are categorical, it is sometimes possible to regain the identifiability of the full law. We present a necessary and sufficient condition for the identification of the full law in the presence of colluder structures with arbitrary categorical variables. Maximum likelihood estimation of the full law in identifiable models with categorical variables is demonstrated with simulated and real data.

stat.ME↗

arXiv2023

Simulating counterfactuals

Counterfactual inference considers a hypothetical intervention in a parallel world that shares some evidence with the factual world. If the evidence specifies a conditional distribution on a manifold, counterfactuals may be analytically intractable. We present an algorithm for simulating values from a counterfactual distribution where conditions can be set on both discrete and continuous variables. We show that the proposed algorithm can be presented as a particle filter leading to asymptotically valid inference. The algorithm is applied to fairness analysis in credit-scoring.

stat.ML↗

arXiv2023

Price Optimization Combining Conjoint Data and Purchase History: A Causal Modeling Approach

Pricing decisions of companies require an understanding of the causal effect of a price change on the demand. When real-life pricing experiments are infeasible, data-driven decision-making must be based on alternative data sources such as purchase history (sales data) and conjoint studies where a group of customers is asked to make imaginary purchases in an artificial setup. We present an approach for price optimization that combines population statistics, purchase history and conjoint data in a systematic way. We build on the recent advances in causal inference to identify and quantify the effect of price on the purchase probability at the customer level. The identification task is a transportability problem whose solution requires a parametric assumption on the differences between the conjoint study and real purchases. The causal effect is estimated using Bayesian methods that take into account the uncertainty of the data sources. The pricing decision is made by comparing the estimated posterior distributions of gross profit for different prices. The approach is demonstrated with simulated data resembling the features of real-world data.

stat.AP↗

arXiv2022

Generalizing experimental findings: identification beyond adjustments

We aim to generalize the results of a randomized controlled trial (RCT) to a target population with the help of some observational data. This is a problem of causal effect identification with multiple data sources. Challenges arise when the RCT is conducted in a context that differs from the target population. Earlier research has focused on cases where the estimates from the RCT can be adjusted by observational data in order to remove the selection bias and other domain specific differences. We consider examples where the experimental findings cannot be generalized by an adjustment and show that the generalization may still be possible by other identification strategies that can be derived by applying do-calculus. The obtained identifying functionals for these examples contain trapdoor variables of a new type. The value of a trapdoor variable needs to be fixed in the estimation and the choice of the value may have a major effect on the bias and accuracy of estimates, which is also seen in simulations. The presented results expand the scope of settings where the generalization of experimental findings is doable

stat.ME↗

arXiv2021

Contrasting Identifying Assumptions of Average Causal Effects: Robustness and Semiparametric Efficiency

Semiparametric inference on average causal effects from observational data is based on assumptions yielding identification of the effects. In practice, several distinct identifying assumptions may be plausible; an analyst has to make a delicate choice between these models. In this paper, we study three identifying assumptions based on the potential outcome framework: the back-door assumption, which uses pre-treatment covariates, the front-door assumption, which uses mediators, and the two-door assumption using pre-treatment covariates and mediators simultaneously. We provide the efficient influence functions and the corresponding semiparametric efficiency bounds that hold under these assumptions, and their combinations. We demonstrate that neither of the identification models provides uniformly the most efficient estimation and give conditions under which some bounds are lower than others. We show when semiparametric estimating equation estimators based on influence functions attain the bounds, and study the robustness of the estimators to misspecification of the nuisance models. The theory is complemented with simulation experiments on the finite sample behavior of the estimators. The results obtained are relevant for an analyst facing a choice between several plausible identifying assumptions and corresponding estimators. Our results show that this choice implies a trade-off between efficiency and robustness to misspecification of the nuisance models.

stat.ME↗

arXiv2021

Clustering and Structural Robustness in Causal Diagrams

Graphs are commonly used to represent and visualize causal relations. For a small number of variables, this approach provides a succinct and clear view of the scenario at hand. As the number of variables under study increases, the graphical approach may become impractical, and the clarity of the representation is lost. Clustering of variables is a natural way to reduce the size of the causal diagram, but it may erroneously change the essential properties of the causal relations if implemented arbitrarily. We define a specific type of cluster, called transit cluster, that is guaranteed to preserve the identifiability properties of causal effects under certain conditions. We provide a sound and complete algorithm for finding all transit clusters in a given graph and demonstrate how clustering can simplify the identification of causal effects. We also study the inverse problem, where one starts with a clustered graph and looks for extended graphs where the identifiability properties of causal effects remain unchanged. We show that this kind of structural robustness is closely related to transit clusters.

stat.ML↗

arXiv2020

Identifying Causal Effects via Context-specific Independence Relations

Causal effect identification considers whether an interventional probability distribution can be uniquely determined from a passively observed distribution in a given causal structure. If the generating system induces context-specific independence (CSI) relations, the existing identification procedures and criteria based on do-calculus are inherently incomplete. We show that deciding causal effect non-identifiability is NP-hard in the presence of CSIs. Motivated by this, we design a calculus and an automated search procedure for identifying causal effects in the presence of CSIs. The approach is provably sound and it includes standard do-calculus as a special case. With the approach we can obtain identifying formulas that were unobtainable previously, and demonstrate that a small number of CSI-relations may be sufficient to turn a previously non-identifiable instance to identifiable.

cs.AI↗

arXiv2020

Simulation Framework for Realistic Large-scale Individual-level Data Generation with an Application in the Health Domain

We propose a framework for realistic data generation and simulation of complex systems and demonstrate its capabilities in the health domain. The main use cases of the framework are predicting the development of risk factors and disease occurrence, evaluating the impact of interventions and policy decisions, and statistical method development. We present the fundamentals of the framework using rigorous mathematical definitions. The framework supports calibration to a real population as well as various manipulations and data collection processes. The freely available open-source implementation in R embraces efficient data structures, parallel computing and fast random number generation which ensure reproducibility and scalability. With the framework it is possible to run daily-level simulations for populations of millions of individuals for decades of simulated time. An example on the occurrence of stroke, type 2 diabetes and mortality illustrates the usage of the framework in the Finnish context. In the example, we demonstrate the data-collection functionality by studying the impact of non-participation on the estimated risk models and interventions related to controlling the additional salt intake.

stat.ME↗

arXiv2020

Do-search -- a tool for causal inference and study design with multiple data sources

Epidemiological evidence is based on multiple data sources including clinical trials, cohort studies, surveys, registries and expert opinions. Merging information from different sources opens up new possibilities for the estimation of causal effects. We show how causal effects can be identified and estimated by combining experiments and observations in real and realistic scenarios. As a new tool, we present do-search, a recently developed algorithmic approach that can determine the identifiability of a causal effect. The approach is based on do-calculus, and it can utilize data with non-trivial missing data and selection bias mechanisms. When the effect is identifiable, do-search outputs an identifying formula on which numerical estimation can be based. When the effect is not identifiable, we can use do-search to recognize additional data sources and assumptions that would make the effect identifiable. Throughout the paper, we consider the effect of salt-adding behavior on blood pressure mediated by the salt intake as an example. The identifiability of this effect is resolved in various scenarios with different assumptions on confounding. There are scenarios where the causal effect is identifiable from a chain of experiments but not from survey data, as well as scenarios where the opposite is true. As an illustration, we use survey data from NHANES 2013--2016 and the results from a meta-analysis of randomized controlled trials and estimate the reduction in average systolic blood pressure under an intervention where the use of table salt is discontinued.

stat.AP↗

arXiv2020

Estimation of causal effects with small data in the presence of trapdoor variables

We consider the problem of estimating causal effects of interventions from observational data when well-known back-door and front-door adjustments are not applicable. We show that when an identifiable causal effect is subject to an implicit functional constraint that is not deducible from conditional independence relations, the estimator of the causal effect can exhibit bias in small samples. This bias is related to variables that we call trapdoor variables. We use simulated data to study different strategies to account for trapdoor variables and suggest how the related trapdoor bias might be minimized. The importance of trapdoor variables in causal effect estimation is illustrated with real data from the Life Course 1971-2002 study. Using this dataset, we estimate the causal effect of education on income in the Finnish context. Bayesian modelling allows us to take the parameter uncertainty into account and to present the estimated causal effects as posterior distributions.

stat.ME↗

arXiv2019

Causal Effect Identification from Multiple Incomplete Data Sources: A General Search-based Approach

Causal effect identification considers whether an interventional probability distribution can be uniquely determined without parametric assumptions from measured source distributions and structural knowledge on the generating system. While complete graphical criteria and procedures exist for many identification problems, there are still challenging but important extensions that have not been considered in the literature. To tackle these new settings, we present a search algorithm directly over the rules of do-calculus. Due to generality of do-calculus, the search is capable of taking more advanced data-generating mechanisms into account along with an arbitrary type of both observational and experimental source distributions. The search is enhanced via a heuristic and search space reduction techniques. The approach, called do-search, is provably sound, and it is complete with respect to identifiability problems that have been shown to be completely characterized by do-calculus. When extended with additional rules, the search is capable of handling missing data problems as well. With the versatile search, we are able to approach new problems such as combined transportability and selection bias, or multiple sources of selection bias. We perform a systematic analysis of bivariate missing data problems and study causal inference under case-control design. We also present the R package dosearch that provides an interface for a C++ implementation of the search.

stat.ML↗