Figure 4. Boxplots for 100 values of \(\text{SS}D_{\theta}\) for Case I, when designing three sequential A-optimal experiment all at once using Bayes-Bayes, LO-LO, Bayes-LO, and LO-Bayes approaches
Computation times for both Bayesian and LO approaches to MBDOE were compared using MC simulations for the Bayes-Bayes and LO-Bayes approaches. These two approaches use the same approach for parameter estimation, but different methods for MBDOE, making it possible to isolate the effects of Bayesian and LO approaches for MBDoE. Using a core i5 laptop with 8 GB RAM, the average computations time for each Bayes-Bayes run was 51.2 s, which is faster than 89.9 s on average for a LO-Bayes run. Although this difference is relatively small for the current case study, we anticipate that larger differences could occur for larger models with more parameters and decision variables.

Case II: Results when moderately-informative initial guesses are available

Figure 5 compares boxplots for the 100 values of \(\text{SS}D_{\theta}\)for four approaches when designing three A-optimal experiments one at a time using the prior parameter guesses in Case II. Similar patterns are observed compared to Case I: as more experiments are designed and more data become available, the mean and median values of boxplots for\(\text{SS}D_{\theta}\) becomes smaller. However, since the parameter initial guesses are not as good as in Case I, the parameter estimates in Case II are less accurate than in Case I. As in Case I, the Bayes-LO approach provides the best parameter values on average. A key difference between the results in Case I and Case II is that the LO approach tended to estimate more parameters in Case II, due to higher initial parameter uncertainties. Details concerning the frequency with which different parameters were estimated are provided in the Supplementary Information.

Case III: Results when misinformed initial guesses are used

Figure 6 shows boxplots for 100 values of \(\text{SS}D_{\theta}\)obtained for Case III, with the misinformed parameter initial guesses described in the third row of Table 8. As expected, because parameter initial guesses were worse than in Case I and II, the mean and median for \(\text{SS}D_{\theta}\) are larger than Case I and II for all four approaches. The Bayes-LO approach to designing experiments and estimating parameters was the best approach, even though the modeler believed he or she had better prior knowledge about the parameters than was justifiable. These results suggest that the Bayes-LO approach is somewhat robust to specification of misinformed prior information. Note that the settings in Case II and Case III were also used to design three experiments all-at-once instead of one at a time. As in Case I, the Bayes-LO approach was the best and the parameter estimations resulting from the three-at-once experiments were not as good as those obtained using the one-at-a-time approach. Details are provided in the Supplementary Information.