3.1 Achievement of major science objectives
A primary and fundamental accomplishment in the past 3 years has been the successful achievement of all major science objectives of the instrument. These objectives include the detection of lightning during day and night, with storm-scale spatial resolution, millisecond timing, and high flash detection efficiency without a land/ocean bias. ISS LIS was designed to measure radiant energy and provide background images/intensity, and its deployment on the ISS enables the delivery of realtime lightning data. These objectives were achieved by successfully meeting several instrument/platform requirements. Most of these are discussed in Appendix A. Below we focus on validation of timing and geolocation accuracy, detection efficiency, and false alarm rate. Note that, as detailed in Appendix A, we expect an improvement of 2-5% in many of the instrument performance parameters discussed below based on planned future improvements to ISS LIS data processing.
Like TRMM LIS, ISS LIS has a frame rate of 500 s-1. This implies a native 2-ms timing precision. The actual on-orbit timingaccuracy of ISS LIS was determined by comparison against multiple ground-based and spaceborne reference datasets. The ground-based reference datasets were the EarthNetworks Global Lightning Network (ENGLN) [e.g., Marchand et al. , 2019], which operates wideband sensors (1 Hz to 12 MHz), and the Vaisala Global Lightning Dataset 360 (GLD360) [e.g., Rudlosky et al. , 2017], which detects waveforms in the very low frequency range (VLF; ~500 Hz to ~50 kHz). Both of these are global datasets. The spaceborne reference datasets were the Geostationary Operational Environmental Satellite (GOES) 16 and 17 Geostationary Lightning Mappers (GLM-16 and GLM-17) [Goodman et al. , 2013, Rudlosky et al. , 2018]. GLM is built on the LIS/OTD optical detection heritage and is sensitive to both intracloud (IC) and cloud-to-ground (CG) lightning.
The timing data as initially received from ISS exhibited offsets up to ± 1 s with respect to the reference data, with an alternating drift pattern that cycled approximately every 9 days. This drift was accurately characterized based on careful analysis. On the basis of this, timing correction variables, and an additional constant offset, were applied to produce the current timing accuracy (Fig. 3 left).
After correction, ISS LIS has modal temporal offset of +1 ms, or approximately one-half the LIS frame duration, when compared with GLM-16/17 (Fig. 3 left). The standard deviation in the offset is less than 2 ms in either direction, compared to GLM-16/17. This comparison was based on group timings between ISS LIS and the GLM datasets. Relative to the ground-based reference datasets, there is zero modal offset, and the standard deviation is also below 2 ms. Thus, after correction for the ISS timing errors, the ISS LIS timing accuracy is less than the native timing precision of the instrument itself (2 ms). This is similar to the independent analysis performed by Erdmann et al. [2020], using a different reference dataset, and is consistent with (though not as precise as) the timing analysis for TRMM LIS by Bitzer and Christian [2015].
ISS LIS geolocation accuracy was analyzed through a coordinate system transform technique that allowed LIS location errors with respect to the reference data to be displayed in the native LIS field-of-view. This analysis revealed that the ISS navigation variables used for LIS geolocation vary systematically during each orbit, creating location errors of up to 25 km. This particular issue was unrelated to the TRMM LIS geolocation issue discussed by Zhang et al. [2019]. The ISS LIS team worked diligently to troubleshoot the initial issues with geolocation (and timing), as these posed complex interconnected problems that required focused analysis and skilled interpretations to resolve.
An iterative tuning process resolved these initial problems and produced corrected geolocation data and the current analysis of ISS LIS spatial accuracy (Fig. 3 right). Relative to GLM-16/17, corrected ISS LIS spatial offsets are almost entirely less than 10 km, with the vast majority below 5 km. Offsets relative to ENGLN and GLD360 are distributed more broadly, but for each of the spaceborne and ground-based reference datasets the modal offset is approximately 2-3 km. This means that ISS LIS has achieved sub-pixel (< 4 km) location accuracy. The independent analysis by Erdmann et al.[2020] supports this assessment.
Timing, location, flash DE, and false alarm rate (FAR) have been stable during most of the mission to date. Figure 4 shows the time series of these parameters through early 2020. ISS LIS temporal accuracy offset shifted by about 1 ms on 16 December 2018, such that LIS now slightly leads the reference data (Fig. 4 top). A few larger deviations occurred during two periods in early 2019, related to atypical ISS maneuvers. However, in most circumstances the absolute magnitude of the offsets remain less than 2 ms. LIS geolocation accuracy also has been stable throughout the mission (Fig. 4 middle), with the modal peak of the offset normally less than 5 km. The detection performance of ISS LIS is also stable over time (Fig. 4 bottom), aside from the known deviations in early 2019 mentioned above. The flash DE (calculated using a 50-km, 200-ms matching window) is stable with respect to each of the reference datasets (64% relative to GLM, and 56-57% relative to ENGLN/GLD360), and the FAR, calculated over the Americas domain where the greatest quantity and best quality reference data are available, is under 5% on average. This is a higher FAR than that published for TRMM LIS [Boccippio et al. , 2002], but that study did not include the impact of specular reflections (e.g., cloud and ocean glint) like this analysis did. The flash DE is comparable to the values computed independently for ISS LIS by Erdmann et al. [2020].
Detection efficiency was also examined as a function of time of day (Fig. 5). As expected for optical instruments like ISS LIS, flash DE is maximized during local nighttime (64-75% near local midnight, depending on reference dataset) and minimized during local daytime (51-65% around 1700 LT, just before sunset/dusk). There is another apparent minimum in flash DE against GLM-16, around sunrise (~0600 LT); however, the analysis includes a period of time before the GLM-16 blooming filter was implemented to reduce the impact of solar glint. Since a similar sunrise decrease is not observed in the glint-filtered GLM-17 data, we infer that this reduction in DE against GLM-16 is primarily caused by increased GLM-16 false alarms during local sunrise.
Analysis of TRMM LIS DE versus the ground-based reference datasets indicates that ISS LIS DE is approximately 4-7% lower overall compared to TRMM LIS (using 2014-2015 ENGLN and GLD360 data; not shown). As mentioned previously (and explained in Appendix A), we expect increases of ~2-5% in flash DE after planned ISS LIS dataset processing improvements. In addition, the ground networks often improve each year [e.g., Rudlosky et al. , 2017], and this may also explain some of the DE discrepancy between the ISS LIS and TRMM LIS eras (i.e., the ground networks are likely detecting more lightning now versus 5+ years ago). Finally, there is a possibility that ISS LIS is slightly less sensitive than TRMM LIS. Future work will attempt to quantify any LIS instrument sensitivity differences, if they exist.
Note that none of the above flash comparisons follows the Bayesian approach of Bitzer et al. [2016]. That study found a similarly low TRMM LIS detection efficiency against ground networks (~53%), but when corrected within a Bayesian probability framework the estimated detection efficiency increased to 80%. Bayesian analysis of ISS LIS detection efficiency is planned in the future.