We develop a new approach based on the variance of noise cross-correlation to characterize the noise source and wave propagation under the influence of a non-diffuse noise field. Based on the random errors for the Fourier spectra of stacked noise cross-correlation (Liu et al. 2016) and the assumption of diffuse field, we derive an analytical expression for the variance of every time point in the stacked cross-correlation and validate the theory using synthetic diffuse noise data. The ambient seismic noise field in Southern California is, however, not fully diffuse. The observed correlated neighboring frequencies in the noise data – a definitive character of the non-diffuse field (Liu & Ben-Zion 2016), map into the noise cross-correlation, biasing its variances from theoretical predictions under fully diffuse field assumption. For the secondary ocean microseism, we find strong positive correlation between the correlated neighboring frequencies and the deviations from diffuse field theory predicted variances. The ballistic arrivals on average contain more significant bias than the coda segments of the noise cross-correlation, which agrees with previous time-lapse monitoring studies based on ambient seismic noise. In addition, the station pairs having significant bias between actual and diffuse field theory predicted variances are aligned with the strongest source direction and exhibit significant beamforming source fluctuation.