Background Information
Fisher Information Matrix for Nonlinear
Models
Consider the nonlinear model:
\(\mathbf{Y=g}\left(\mathbf{d,\theta}\right)\mathbf{+\varepsilon}\)(1)
where \(\mathbf{Y\ \in\ }\mathbf{R}^{N}\) is a vector of stacked
measured responses, g is the solution of equations that
describe the system, \(\mathbf{d}\in\mathbf{R}^{r\times D}\) is a
matrix of experimental settings (for \(r\) runs with \(D\) decision
variables specified for each), \(\mathbf{\theta}\in\mathbf{R}^{p}\) is
the vector of model parameters and\(\mathbf{\varepsilon}\mathbf{\in}\mathbf{R}^{N}\) is a vector of a
measurement noise with diagonal covariance matrix\(\mathbf{\Sigma}_{\mathbf{y}}\in\mathbf{R}^{N\times N}\). For
dynamic multi-response models with \(n\) sample times per run and \(v\)response variables, the total number of data values is \(N=nvr\). TheFIM is computed using a parametric sensitivity matrixS \({\in\mathbf{R}}^{N\times p}\) with elements:
\(S_{\text{ij}}=\left.\ \frac{\partial g\left(\mathbf{d},\mathbf{\theta}\right)}{\partial\theta_{j}}\right|_{{\hat{\theta}}_{k\neq j}}\)(2)
computed by linearizing the model around the best currently-available
parameter values:53
The elements of \(\mathbf{S}\) should be scaled using parameter
uncertainties \(s_{\theta_{j}}\) and measurement uncertainties\(s_{y_{i}}\) to reflect the modeler’s prior
knowledge:54
\(Z_{\text{ij}}=S_{\text{ij}}\frac{s_{\theta_{j}}}{s_{y_{i}}}\) (3)
resulting in a scaled sensitivity matrix Z . The FIM is
related to Z by:
\(\mathbf{FIM=}\mathbf{Z}^{T}\mathbf{Z}\) (4)
When performing sequential MBDoE calculations, Z contains two
parts:21,46
\(\mathbf{Z=}\par
\begin{bmatrix}\mathbf{Z}_{\mathbf{\text{old}}}\\
\mathbf{Z}_{\mathbf{\text{new}}}\\
\end{bmatrix}\) (5)
where \(\mathbf{Z}_{\mathbf{\text{old}}}\) corresponds to experimental
settings and data from old experiments. The elements of\(\mathbf{Z}_{\mathbf{\text{old}}}\) are fixed during sequential MBDoE
and elements of \(\mathbf{Z}_{\mathbf{\text{new}}}\) are determined by
the optimizer. After each sequential design, elements of\(\mathbf{Z}_{\mathbf{\text{old}}}\) are updated based on the new
parameter values and the number of rows in\(\mathbf{Z}_{\mathbf{\text{old}}}\) increases due to the recent
experiments.
Parameter estimation with a noninvertible
FIM
When estimating parameters, the FIM should be invertible,
otherwise unique estimates for the parameters cannot be
obtained.22,29 Several regularization approaches have
been used to overcome this problem.38,39,55 One
popular approach is to estimate a subset of the model parameters that
are estimable, with the remaining parameters fixed at nominal
values.23,45,56 Table 2 shows computational steps for
a commonly used orthogonalization-based approach that ranks parameters
from the most-estimable so problematic (unranked) parameters that lead
to a noninvertible FIM can be
determined.45,54 The ranking starts by computing the
magnitude of each column of the scaled sensitivity matrix Z(Step 1). The parameter corresponding to the column with the highest
magnitude is selected as the most-estimable parameter (Step 2). The
columns of Z are then regressed onto columns of\(\mathbf{X}_{k}\), a matrix that contains columns from Z that
correspond to the ranked parameters (Step 3). Residual matrix\(\mathbf{R}_{k}\) is then computed to remove correlation between
columns for the unranked parameters and columns for the parameters that
have already been ranked (Step 4). The next-most-estimable parameter is
the one with the largest magnitude among columns of\(\mathbf{R}_{\mathbf{k}}\). In Step 5, the column corresponding to the
next-most-estimable parameter is selected from the original\(\mathbf{Z}\) matrix and included in \(\mathbf{X}_{k}\), resulting in
matrix \(\mathbf{X}_{k+1}\). Steps two to five are repeated to produce
a ranked list with up to \(p\) parameters. The ranking stops when all of
the parameters are ranked or at the iteration where\(\mathbf{X}_{k}^{T}\mathbf{X}_{k}\) (the reduced FIM ) becomes
noninvertible. The remaining unranked parameters are categorized as
problematic. They either have very little influence on the predicted
responses or highly correlated effects with parameters on the ranked
list.45,57 Using this orthogonalization-based ranking
approach prior to parameter estimation helps to avoid numerical problems
that would arise due to a noninvertible FIM .
Table
2. Orthogonalization algorithm 45,54