Uncertainty, Sample Size and the 95/95 Tolerance Limit

Abstract

Monte Carlo simulation is often used to derive the distribution of model output as a result of random or uncertain model input variables. Due to model complexity and heavy computational demand, it is of interest to know how many simulations should be performed to ensure sufficient confidence in the results? The objective of this paper is to illustrate, using very simple examples and the concept of order statistics, how an upper bound value of the model output variable can be estimated with high confidence (e.g., the 95/95 value) using a very small number (e.g., 59) of simulation trials. The results show that, while the methodology is statistically consistent and independent of the underlying probability distribution, it may yield an upper bound value that is too high or unrealistic. Therefore, rather than estimation, the approach is most valuable for demonstrating compliance with a specified acceptance (i.e., regulatory, safety, etc.) limit.