To assess how well the OHL can manage occlusion of a stationary object, a proof-of-concept 2 x 10 pixel sensor was fabricated (image in Figure 1c), containing the (FK209)2+ only electrolyte. The sensor is exposed to the aforementioned situation illustrated in movie 1, involving a simple white 2 x 2 pixel square temporarily blocked by a crossing 2 x 2 pixel red square (Figure 3a), simulating a typical occlusion scenario. Ideally, for the sensor to correctly manage occlusion, the OHL pixels’ VOC responses corresponding to the stationary white square should not deviate much while the blocking red square passes by. Figure 3b shows these four active pixels’ temporal responses while detecting this scene. The white square’s pixels’ maximum VOC drop is less than 10% during the whole occlusion event (indicated by the blue and green dashed frames in Figure 3b), implying that the sensor is indeed capable of reconstructing the masked object. Note that without the pixels’ learning effect, this VOC decay would immediately decay close to zero during the intersection by the dimmer red square foreground, as evidenced by the much lower VOC in response to red light (red line in Figure 3b). thus failing to recognize the covered white square.
To further demonstrate the occlusion handling capabilities of our AI accelerating camera, we expanded the case study to the detection of a more complex object, using a proof-of-concept sensor consisting of 4 x 6 OHL pixels. The letter ‘E’ was chosen as the stationary image being briefly occluded for 1s by a black 2 x 3 pixel rectangle such that ‘E’ temporarily appears as the letter ‘L’ (Figure 3c). The VOC outputs of the OHL pixels seeing the letter ‘E’ were recorded simultaneously and are depicted in Figure 3c. The four blue curves reflect the responses of those four pixels that are directly exposed to the blocking black rectangle. During the occlusion event, marked by the two dashed vertical red lines, these four pixels experience the largest deviation in VOC (the inset illustrates the absolute percent changes in VOC for each sensor pixel, whereas Figure 3f summarizes the corresponding absolute VOC values, with the four occluded pixels outlined by the blue frames). However, these deviations are small as compared to any standard ODL pixel when transitioning from light to dark. Therefore, our sensor is able to correctly recognize the original object ‘E’ by reconstructing the blocked key features that may have tricked the computer to see ‘L’ instead. More precisely, the ODL detects the letter ‘L’, while the OHL allows the computer to interpret the situation to be actually the original letter ‘E’ being obstructed by a black rectangle. To handle a new appearing object, the OHL pixels’ memory has to be erased via a reset to zero VOC. This can be achieved by simply short circuiting the cells, in this way depleting all the stored electrons in the TiO2 film.
So far, we have considered only cases where the occlusion occurs by a darker foreground. Generally, feature masking can also take place by a brighter object. The OHL pixels’ responses to a sudden increase in light intensities are summarized in Figures 3d and 3e, highlighting that the pixels exhibit VOC retention of more than 76% for the probed region of light exposure. Within the limit of these light intensity ranges, occlusion can be handled through a variation of pixel value of less than 24%.