Step 4: joining domains into larger evaluation units based on the
performance.
Once domains were defined, models were trimmed accordingly and evaluated
against the domains and their pair-wise combinations. GDT_TS scores
from LGA’s sequence-dependent superpositions served as the numerical
basis for deciding whether domains should be kept separate or combined
into larger Evaluation Units (EUs) for the final evaluation.
A rationale and numeric procedure for combining domains /splitting
targets into evaluation units were suggested by Nick Grishin and
coworkers in CASP9 6.
They argued that targets should be split into domains only if this can
help reveal interesting predicted features in models. Rephrasing this
postulate for the bottom-to-top approach (split first, then consider
re-joining), domains should be merged if their separate evaluation does
not provide additional benefits for the assessment. A good indicator of
this scenario is the similarity of model accuracy scores on the combined
and individual domains. To facilitate the decision-making, Kinch et al6 plotted GDT_TS scores
for combined domains versus the weighted sum of scores for individual
domains. Such a graph became later known as the Grishin plot and was
adopted for defining EUs in subsequent CASPs6-10,20.
If the points in such a graph line up close to the diagonal line, then
joining a pair of domains into a larger evaluation unit is advised.
In CASP15, domains were joined if the slope of the zero-intercept best
fit line in a Grishin plot was <1.2. Three or more domains
were joined into one EU when the plots for all pairwise domain
combinations supported the merger.
The process was repeated iteratively until no further combining of EUs
was needed.