Effective conservation and utilization of farm animals are fundamental for realizing sustainable increases in food production. In situ and ex situ conservation are the two main strategies currently used to protect domestic chicken in China. However, genomic diversity and population structure have not been compared in these conserved populations. One potential risk is that the use of genome-wide SNPs to optimize genomic diversity might not preserve particular alleles that are associated with breed-specific characteristics. Here, 361 individuals from three Chinese domestic chicken breeds were collected from populations conserved in situ and ex situ, and genotyped using GBS (genotyping-by-sequencing). We estimated the genomic diversity, analyzed population structures, and found that the small ex situ conserved populations that have been maintained in controlled environments retained less genetic diversity than the in situ’s. In addition, genetic differentiation was detected between in situ and ex situ conserved populations within a single breed. We next analyzed selective signatures (FST, Pi, and XPEHH) to examine the genetic mechanisms underlying differentiation between in situ and ex situ conserved populations. We concluded that differentiation might be caused by genetic drift, or the differences were due to variants from the original populations. Finally, based on sequencing data obtained from the ex situ conserved populations, we used Di and Pi to identify “genomic conservation units” for breed-specific characteristics. Loci associated with the “genomic conservation unit” could be used to preserve breed-specific characteristics in the conservation program.