Habitat fragmentation is a major driver of genetic differentiation and genetic diversity loss in endangered species due to restricted gene flow, genetic drift, and inbreeding. However, the impact on ectomycorrhizal (ECM) fungi remains unexplored, despite their critical roles in forest ecosystems. In this study, we investigated the population genetic structure of the ECM fungus Rhizopogon togasawarius, which specifically colonizes Pseudotsuga japonica across its entire distribution range. Both symbionts are found only in small, fragmented forests in Japan; they are designated as endangered species on the IUCN Red List. We assessed the population genetic structure of 236 R. togasawarius specimens collected from five remaining populations, distributed in two regions separated by a sea channel. Microsatellite analyses using 20 loci revealed strong genetic differentiation among populations (FST = 0.255), even in the nearest population pair separated by a distance of only 8 km (FST = 0.075), indicating extremely limited gene flow between populations. Population history analyses implied that population divergence occurred approximately 2200 generations ago between the two regions, and nearly 1000 generations ago between the nearest populations within the Shikoku region. Because of prolonged genetic isolation, significant inbreeding was confirmed in four of five populations, where effective population sizes became very small (Ne = 9.0–58.0). Although evaluation of extinction risks for microorganisms is challenging, our conservation genetic results indicated that habitat fragmentation increases extinction risk through population genetic mechanisms, as demonstrated in plants and animals, and therefore should not be overlooked in biodiversity conservation efforts.