For more discussion about those two metrics, see my previous paper.
Variance vs. entropy
This internal diversity formula is essentially a variance (without the square). However, variance can be a poor measure of diversity, when data is clustered in few distant regions.