Discussion

Through formal assessment of the Macrosystems EDDIE ecological forecasting curriculum for undergraduates, we found that modules were successful in increasing student confidence and knowledge of ecological forecasting and data science (Fig. 2) and lowered the barrier of entry to these fields for instructors (Fig. 3). In an era when data science and ecological forecasting skills are increasingly needed to tackle pressing biological and environmental science problems (Hampton et al. 2017, National Academies of Sciences 2018, Feng et al. 2020, Emery et al. 2021), the Macrosystems EDDIE curriculum provides one pathway to introducing these skills to both students and instructors.
Our results indicate that flexible, short, and easy-to-use modules increase student confidence in data science and ecological forecasting skills. In particular, students showed the greatest gains in confidence in ecological forecasting skills (Fig. 2a), likely because they had lower initial confidence in ecological forecasting skills (e.g., generating forecasts, for which students reported a median pre-module Likert score of 2, or ‘slightly confident’). In comparison, student confidence in their data science skills was relatively higher prior to completing the module (e.g., graphing data, with a median pre-module score of 4, or ‘very confident’; Fig. S1). The Dunning-Kruger effect (Kruger and Dunning 1999) may explain the few students that exhibited decreases in confidence (ranging from n = 24 students for the skill of generating a forecast to n = 77 students for the skill of graphing data), in which novice students overestimate their abilities, and as they progress, are much better able to estimate their abilities, which are less than they previously thought (Fig. S1). Ultimately, increased student confidence and knowledge of data science and forecasting are relevant beyond the life sciences, as workers with data science and predictive modeling skills are sought across multiple sectors (Stanton and Stanton 2019).
Instructor feedback after teaching a module indicates that the Macrosystems EDDIE approach of “just-in-time” background skills training (sensu Novak et al. 1999) and robust instructional supporting material may be successful strategies for instructor professional development in data science. We received positive feedback regarding the effect of Macrosystems EDDIE modules on both the growth of instructor pedagogical (e.g., active learning) and disciplinary (e.g., data science and ecological forecasting) knowledge (Auerbach and Andrews 2018, Andrews et al. 2019). Most instructors said that Macrosystems EDDIE modules were easy to use and very to extremely effective in teaching ecological forecasting and data science concepts (Fig. 3). Qualitative responses to our instructor survey indicated that a comprehensive introduction to the structure, development, and interpretation of the forecasting models used in each module (e.g., reviewing the structure of a simple ecosystem primary productivity model in the Intro to Forecasting module) was helpful to both students and instructors (Text S2). In addition, instructors reported that the accompanying instructor manual with detailed talking points for each slide in the introductory presentation and suggested timing for each activity within the module were helpful for classroom implementation. Finally, most instructors reported that they were better equipped to use long-term and high-frequency data and more likely to use sensor network data after teaching a module (Fig. 3b), indicating that the modules build skills and data science familiarity with instructors as well as students. Overall, an important achievement of this adaptable, accessible curriculum is “training the trainers,” in which an instructor gains skills and knowledge in a new area, which are then transferred to students (Beyer et al. 2009, Emery et al. 2021).
Modules were iteratively revised in response to student and faculty feedback. For example, we revised early versions of the modules to provide a more in-depth introduction in Activity A to the modeling approaches used for forecasting as a method of “just-in-time” training for both students and instructors. In addition, RMarkdown versions of the Forecasts & Uncertainty and Forecasts & Data modules were developed based on requests from instructors. The RMarkdown files provide scaffolding for both students and instructors, who can start by working through materials in the point-and-click R Shiny interface and then move to the code “under the hood” of the Shiny application if they wish. Importantly, this scaffolding may enable students and instructors to transfer skills learned from teaching the module to their own research projects, as they can modify the code for their own datasets and research questions.
Macrosystems EDDIE ecological forecasting modules may facilitate the use and analysis of large datasets, including NEON data, by instructors who have not had extensive data science training. While interdisciplinary collaborations with, e.g., computer scientists can facilitate analyses with large computational demands, ecologists must still possess basic data science skills, such as coding and data wrangling, modeling, and visualization, to make these collaborations a success (Cheruvelil et al. 2014, Cheruvelil and Soranno 2018, Carey et al. 2019). In sum, we found that the development of comprehensive supporting materials aimed to provide background skills and pedagogical training for instructors is critical for the effective implementation of new data science material into existing undergraduate curricula and may also facilitate new research efforts for instructors. Up-to-date versions of the modules are available on GitHub (https://github.com/MacrosystemsEDDIE) and feedback on module content and ease of use is welcome and can be submitted at MacrosystemsEDDIE.org.
To train ecological and environmental scientists in data science and ecological forecasting concepts and skills, these topics need to be presented in a relevant, approachable way for both students and instructors. Our data indicate that the Macrosystems EDDIE approach is effective in engaging both instructors and students in data science and ecological forecasting, and our observed increases in student confidence may foster greater student “science identity” and retention in STEM (Stets et al. 2017, Vincent-Ruz and Schunn 2018, O’Brien et al. 2020, Bowser and Cid 2021). Ultimately, increased data science confidence and proficiency by both undergraduate students and instructors unleashes tremendous potential to leverage large datasets for addressing environmental challenges.

Acknowledgements

We thank the students and instructors who tested Macrosystems EDDIE ecological forecasting modules, especially Kait Farrell, Matt Hipsey, Leah Johnson, Nick Record, and Kiyoko Yokota. We also thank the Virginia Tech Reservoir Group and our undergraduate focus groups for their feedback, especially Caroline Bryant, Arpita Das, George Haynie, Ryan Keverline, Michael Kricheldorf, Rose Thai, Evelyn Tipper, and Jacob Wynne. We thank Monica Bruckner, Ashley Carlson, Kristin O’Connell, and Cailin Huyck Orr at the Science Education Resource Center at Carleton College for administrative assistance with module testing. All module testing and assessment was conducted following approved Institutional Review Board (IRB) protocols (Virginia Tech IRB 19-669 and Carleton College IRB 19-20 065). This work was funded by NSF DEB-1926050, DBI-1933016, and EF-2318861.