AI improved the doctors’ performance on CNS malformations
identification
When facilitated with the AI diagnosis, the overall diagnostic efficacy
of three subgroups of doctors got significantly improved (Table 3,
Figure 5a, b, c) in terms of accuracy, sensitivity, and AUC. For the
experts, the accuracy, sensitivity and AUC were improved from 78.9% to
84.7% (p = 0.002), from 77.5% to 83.4% (p = 0.003), and from 0.853 to
0.910 (p = 0.019), respectively. For the competent doctors, the
improvements for accuracy was from 69.6% to 85.1% (p = 0.005),
sensitivity was from 67.5% to 84.0% (p = 0.006), and AUC from 0.793 to
0.905(p = 0.002). For trainee doctors, the progress was shown in
accuracy (51.5% vs. 80.2%, p = 0.001), sensitivity (48.6% vs.78.7%,
p = 0.001), and AUC (0.654 vs. 0.872, p = 0.006), respectively. Whereas,
no significant difference was noted in specificity with and without AI
assistance. Among the three groups of doctors, the trainee group
received maximum improvement with AI assistance, whose diagnostic
performance advanced to the level of expert group in terms of
accuracy[ (80.2% (95% CI 75.0-85.3%) vs. 78.9 %(95% CI
75.2-85.2%), P = 0.593] and AUC [0.872 (95% CI 0.861-0.882) vs.
0.853(95% CI 0.809-0.905), p = 0.238]. (Table 4).
The average time for diagnosis required by 13 doctors reduced
significantly (7040s vs. 11571s, p < 0.001) with AI
assistance, compared to that without AI assistance. Compared the time in
subgroup, the time required by trainee doctors (7383s vs. 12663s, p =
0.008) and competent doctors (7729s vs. 12801s, p = 0.018) also
decreased. However, for experts, no significant time-saving was observed
(5923s vs. 8864s, p = 0.114).