An allele-sharing, moment-based estimator of global, population-specific and population-pair < i > F < /i > < sub > ST < /sub > under a general model of population structure

by Jerome Goudet, Bruce S. Weir

Being able to properly quantify genetic differentiation is key to understanding the evolutionary potential of a species. One central parameter in this context isFST, the mean coancestry within populations relative to the mean coancestry between populations. Researchers have been estimatingFST globally or between pairs of populations for a long time. More recently, it has been proposed to estimate population-specificFST values, and population-pair mean relative coancestry. Here, we review the several definitions and estimation methods ofFST, and stress that they provide values relative to a reference population. We show the good statistical properties of an allele-sharing, method of moments based estimator ofFST (global, population-specific and population-pair) under a very general model of population structure. We point to the limitation of existing likelihood and Bayesian estimators when the populations are not independent. Last, we show that recent attempts to estimate absolute, rather than relative, mean coancestry fail to do so.

Source: PLoS Genetics - November 27, 2023 Category: Genetics & Stem Cells Authors: Jerome Goudet Source Type: research