Sensitivity Analyses of Clinical Trial Designs: Selecting Scenarios and Summarizing Operating Characteristics

Abstract

The use of simulation-based sensitivity analyses is fundamental to evaluate and compare candidate designs for future clinical trials. In this context, sensitivity analyses are especially useful to assess the dependence of important design operating characteristics (OCs) with respect to various unknown parameters (UPs). Typical examples of OCs include the likelihood of detecting treatment effects and the average study duration, which depend on UPs that are not known until after the onset of the clinical study, such as the distributions of the primary outcomes and patient profiles. Two crucial components of sensitivity analyses are (i) the choice of a set of plausible simulation scenarios $\{\boldsymbolθ_1,...,\boldsymbolθ_K\}$ and (ii) the list of OCs of interest. We propose a new approach to choose the set of scenarios for inclusion in design sensitivity analyses. Our approach balances the need for simplicity and interpretability of OCs computed across several scenarios with the need to faithfully summarize -- through simulations -- how the OCs vary across all plausible values of the UPs. Our proposal also supports the selection of the number of simulation scenarios to be included in the final sensitivity analysis report. To achieve these goals, we minimize a loss function $\mathcal{L}(\boldsymbolθ_1,...,\boldsymbolθ_K)$ that formalizes whether a specific set of $K$ sensitivity scenarios $\{\boldsymbolθ_1,...,\boldsymbolθ_K\}$ is adequate to summarize how the OCs of the trial design vary across all plausible values of the UPs. Then, we use optimization techniques to select the best set of simulation scenarios to exemplify the OCs of the trial design.