Evaluation and Demonstration of Predictive Capability Maturity Quantification (PCMQ) Method for Assessing Software Validation and Applicability

Abstract

The use of any modeling and simulation (M&S) tool, such as GOTHIC, MAAP, RETRAN or VIPRE, requires the tool be qualified for use for a given application. Generally, this process requires the tool to adequately address the important phenomena with a sufficient level of accuracy; however, these determinations tend to be largely qualitative and rely heavily on expert knowledge and engineering judgement. That approach may be sufficient for well-established applications, but not first-of-a-kind (FOAK) applications, such as advanced reactor design, where expertise and operational experience may not exist. Predictive Capability Maturity Quantification (PCMQ) type methods have the potential to provide a quantitative metric for evaluating adequacy considering the applicability, scaling, relevance, and coverage of the available evidence as well as the accuracy of the M&S tool. PCMQ can potentially also identify which experiments/benchmarks are most applicable for a given application, whether the available benchmarking is sufficient, and what improvements or additional benchmarks would provide the most value. The current work presents a feasibility study with a proof-of-concept to evaluate the process, identify potential constraints or limitations, and assess the credibility of applying PCMQ for an industry tool like GOTHIC∗. Two use cases are presented – a simpler Separate Effects Test (SET) and a more complex Integrated Effect Tests (IET) – that GOTHIC has already been benchmarked against. This approach provides a controlled environment for evaluating PCMQ since the applicability of GOTHIC for those scenarios is well-established and can be confirmed with comparisons to the experimental data.