Federal agencies in the US must evaluate the environmental justice implications of regulatory actions. Environmental justice analyses frequently use demographic tests to determine whether regulated projects will disproportionately impact vulnerable communities, including American Indian communities. Demographic tests typically yield negative results, which are often cited as evidence of no environmental justice implications. However, susceptibility of demographic tests to false negative errors is unknown. In these cases, false negative errors occur when a test cannot identify a vulnerable population concentrated disproportionately within a project study area. We developed a technique to evaluate the susceptibility of demographic tests to false negative errors. We used the technique to assess a test commonly used by regulators to permit fossil fuel pipelines. The Atlantic Coast Pipeline served as a case study. The demographic test did not identify disproportionately large American Indian populations under any realistic scenario, a false negative error rate of 100%. In our case study, the test did not detect a disproportionately large American Indian population until the study area contained a four times greater fraction of American Indians than the reference area. We extend the results to study the test’s performance throughout the US. The test’s inability to detect disproportionately large American Indian populations calls into question the validity of negative results and the general ability of the test to inform conclusions about environmental justice or sustainability. We recommend abandoning the test in favor of more rigorous methods.

In the United States, federal policies exist to ensure environmental justice and to protect Indigenous rights. However, the effectiveness of these policies can be influenced by analytical tools chosen by decision-makers to study disproportionate impacts of federal actions, including environmental permitting, on Indigenous peoples in particular and marginalized communities in general. Strong analytics can help identify, early on, communities likely to be impacted by federal permitting and decision-making, providing opportunities to consult meaningfully with communities and address potential injustices or inequities prior to key planning and permitting decisions. In contrast, weak analytics can create blind spots to potential inequities and injustices that may not be revealed until late in planning and permitting processes if at all. Here we evaluate environmental justice analytics used in federal decision-making with particular attention to recent fossil fuel pipeline permitting. Using the Atlantic Coast Pipeline - a proposed shale gas project in West Virginia, Virginia, and North Carolina - as a case study, we identify methodological weaknesses that contribute to decision-making blind spots surrounding environmental justice, focusing especially on implications for American Indian tribes. We discuss findings in the broader contexts of public policies surrounding environmental justice and Indigenous rights. We offer recommendations for policy-makers, regulators, pipeline developers, and members of affected communities.