Results
Descriptive statistics are presented by CF program cohort in Table 2 and represent aspects of the Inner Setting. There were no statistically significant differences in characteristics of CF programs across cohorts and thus, analyses were conducted across programs.
The first aim was to analyze the top-ranked barriers and successes of this implementation effort. Barriers were rank-ordered in terms of their significance, and successes were evaluated by their frequency of endorsement.
Barriers to implementation. Respondents were asked to rank order a list of 9 potential barriers each year from 1 (most significant) to 10 (least significant). Rank ordering of barriers for Cohort 1 across the 3 years was highly consistent, with staff time, dedicated space (e.g., to screen adolescents privately from parents), and perceived patient burden (time and availability) ranked as the top 3 barriers to implementation. Rank ordering of barriers across all cohorts and years yielded similar results, in a slightly different order: #1) patient burden, #2) space limitations, and #3) limited staff time. These barriers represent characteristics of the Inner Setting that are difficult to ameliorate.
Successes of implementation. Respondents were asked to report the top 3 successes of implementation. The 4 most frequently endorsed successes in 2018, representing the 3rd year of implementation were (representing Cohort 1 year 3, Cohort 2 year 2, and Cohort 3 year 1) were: #1) early identification of depression/anxiety (mental health issues); tied –#2) improved access to psychological services and interventions and #2) increased awareness/education in the CF team about depression/anxiety. The 4th most frequently endorsed success was reduced stigma about mental health/normalization.
The second objective was to evaluate the success of implementation in the first year and whether it improved over subsequent years, using the survey scoring criteria outlined above. As can be seen in Table E.1 (supplement), implementation scores were very similar across cohorts in Year 1, ranging from 21.12 to 26.17. Implementation scores increased significantly over time (see 95% confidence intervals), nearly doubling with each year of implementation. The longitudinal trajectories of these implementation scores were then modeled over time for individual programs. The estimated implementation scores and 95% confidence intervals for each cohort across the 3 years of the study are depicted in Figure 2, along with the trajectories for individual centers. As expected, there was a pronounced increase in implementation scores over time, with improved implementation reported each year. In addition, there was marked variability among programs both in terms of initial implementation and progression across time.
The third aim was to identify predictors of implementation success, considering Characteristics of the Inner Setting and of Individuals. Predictors for the first year of implementation are listed in Table 3 and indicate that the mental health providers’ years of experience on the CF team was significantly related to higher implementation scores. Predictors for changes across time for Cohort 1 are presented in Table 4. Similar to the first-year model, years of experience on the CF team was significantly related to both implementation scores in Year 1 and changes in implementation over time.
Convergent validity using registry data. An external source of data from the Outer Setting, the Cystic Fibrosis Foundation Patient Registry (CFFPR),19 was used to evaluate convergent validity with the survey results. Annual data from the CFFPR on national rates of mental health screening were obtained for the year priorto the initiation of MHC grants in 2015 and for the 3 years of the implementation study (2016-2018). Responses to the following 2 questions were recorded: “Was the patient screened for symptoms of classic depression using the Patient Health Questionnaire (PHQ-9) or other valid depression screening tools?” [yes, no, unknown]; “Was the patient screened for an anxiety disorder using the Generalized Anxiety Disorder Tool (GAD-7 or similar?)?” [yes, no, unknown]. As expected, rates of screening in 2015 for depression were quite low: 21.4% for those 12-17 years (adolescents) and 24.1% for those 18 and older (adults). Similarly low rates of screening for anxiety were documented: 17.6% for adolescents and 19.5% for adults. In 2016, when the MHC grants were launched, depression screening rates more than doubled to 57% for adolescents and 61.2% for adults; anxiety screening rates increased to 53.2% for adolescents and 59.2% for adults. Screening rates for depression continued to increase in 2017 and 2018 to 69% and 73.2% for adolescents and 75.1% and 80.1% for adults, respectively. Anxiety screening demonstrated similar increases across those two years: 67.2% and 72.5% for adolescents and 73.9% and 79.5% for adults.19