Introduction

Mathematical models are used in chemical and pharmaceutical industries for analysis, design and control of chemical processes and for maximizing product quality and profit.1,2 Especially in pharmaceutical industries, models are important for Quality by Design and development of continuous manufacturing processes, which are becoming more widespread.3–5 Mathematical models for pharmaceutical product development can be either empirical or mechanistic.5–7 Although empirical models are commonly used for pharmaceutical processes, they cannot reliably predict the system behavior outside the range of operating conditions used for model development.8 Therefore, fundamental models, based on underlying chemistry and physics, are preferred.9 These models usually contain unknown parameters that require estimation using experimental data.10 To obtain informative data, it is advantageous to carefully plan the experiments aimed at parameter estimation using design of experiment (DoE) techniques.11 As shown in Table 1, optimal model-based design-of-experiments (MBDoE) techniques select experiments to minimize uncertainties in parameters estimates or model predictions.12–14 MBDoE techniques are effective because they account for the structure of the model as well as parameter and measurement uncertainties when selecting new run conditions.13,15 Other benefits of MBDoE techniques, compared to traditional factorial designs, are that they can be readily used to design any number of experiments, e.g., one, three or seven experiments, depending on available resources for experimentation.15,16 MBDoE techniques have been developed to satisfy a variety of objectives including minimizing total variances of parameter estimates, minimizing the average variance of model predictions, and designing experiments for model discrimination.16
Table 1 shows several MBDoE objective functions that have been used for development of chemical and pharmaceutical production models.15–17 If modelers are interested in obtaining accurate parameter estimates for their model, A-, D- or E- optimal designs can be selected.15–17 Alternatively, G- and V-optimal designs focus on obtaining accurate model predictions at specified operating conditions of interest to the modeler.18–21 All of these MBDoE techniques in Table 1 require computation of the inverse of the Fisher Information Matrix (FIM ) when selecting experimental settings.21–23 The FIM carries information about how changes in parameter values can affect the model predictions and is therefore crucial for both MBDoE calculations and parameter inference.24 For nonlinear models, which are common in chemical and pharmaceutical applications, computation of theFIM requires linearizing the model around some nominal parameter values.17,25 If these nominal parameter values are significantly different from the corresponding true values, the selected MBDoE settings may lead to experimental data that are not very informative.17,25 Sequential design approaches are appealing because they enable updating of the parameter values, as well as the experimental strategy, as more data become available.26 Using sequential experimental designs, valuable information from old experimental data can be used, which might have been collected for other objectives than model development.27,28
Computation of the objective functions for sequential MBDoE is problematic if the FIM is noninvertible or ill-conditioned. Typical causes are limited experimental data, strongly correlated influences of different parameters, and parameters with little or no influence on the model predictions.29 In chemical, biochemical and pharmacological systems, models often contain a large number of kinetic and transport parameters (e.g., 10-80 parameters) which may result in noninvertible/ill-conditionedFIM s.30–34 To avoid this problem, several approaches have been considered during sequential MBDoE calculations including parameter subset selection,14,29,35pseudoinverse methods,21,36 Tikhonov regularization,37–40 and Bayesian approaches.13,41,42
The parameter-subset-selection approach uses a model-reduction perspective.35,43,44 In one methodology, parameters are ranked from most-estimable to least-estimable so that problematic (low-ranked) parameters can be recognized and fixed at their nominal values.35,45 In this way, experiments can be designed using a well-conditioned reduced FIM that ignores problematic parameters. Alternatively, pseudoinverse methods approximate the inverse of the FIM (e.g., using the Moore-Penrose pseudoinverse) during MBDoE calculations.21,36,46 In Tikhonov regularization, a penalty is added to diagonal elements of theFIM to make it invertible.29,38–40 Bayesian MBDoE using linear models results in Tikhonov penalties that account for prior knowledge about parameters. However, for nonlinear models, the situation can be considerably more complex, depending on how the nonlinearity is treated.13,41,42 There is little information in the literature regarding which approach is most effective. In two previous articles, we considered pharmaceutical case studies involving noninvertible FIM s. Two different approaches were compared: i) a subset-selection-based approach that leaves out problematic parameters (LO approach) and ii) a simpler approach that uses a Moore-Penrose pseudoinverse in place of\(\mathbf{\text{FI}}\mathbf{M}^{\mathbf{-1}}\)(PI approach).21,46 These case studies suggest that the LO approach is often superior to the PI approach for designing both A- and V-optimal experiments.21,46 A shortcoming of the LO approach is that it can be complicated and computationally expensive due to changes in the subset of parameters that is left out during MBDoE calculations. This complication motivates us to find a more convenient approach to deal with singular FIMs during MBDoE.
The focus of the current study is on a simplified Bayesian approach for dealing with singular/ill-conditioned FIMs during MBDoE. Bayesian approaches have been used in several past MBDoE studies for chemical and biochemical systems.47–49 The main benefit of the Bayesian MBDoE framework is that it accounts for prior knowledge about plausible values of the model parameters.13 However, many researchers raise concerns about the use of Bayesian approaches in practical engineering systems.13,50 Disadvantages of the Bayesian approach include uncertainty about the reliability of assumptions made when specifying prior information.49–51 Undesirable computational complexity can also arise, depending on the assumptions that are made. As a result, Bayesian MBDoE has not enjoyed widespread applications in chemical process modeling.
Table 1. Optimality criteria for model-based design of experiments 21,46