Effects of Social Structure on Effective Population Size Change Estimates

Abstract

Most methods currently used to infer the “demographic history of species” interpret this expression as a history of population sizechanges. The detection, quantification, and dating of demographic changes often rely on the assumption that population struc-ture can be neglected. However, most vertebrates are typically organized in populations subdivided into social groups that areusually ignored in the interpretation of genetic data. This could be problematic since an increasing number of studies have shownthat population structure can generate spurious signatures of population size change. Here, we simulate microsatellite data froma species subdivided into social groups where reproduction occurs according to different mating systems (monogamy, polygy-nandry, and polygyny). We estimate the effective population size (Ne) and quantify the effect of social structure on estimatesof changes in Ne . We analyze the simulated data with two widely used methods for demographic inference. The first approach,BOTTLENECK, tests whether the samples are at mutation–drift equilibrium and thus whether a single Ne can be estimated.The second approach, msvar, aims at quantifying and dating changes in Ne . We find that social structure may lead to signals ofdeparture from mutation–drift equilibrium including signals of expansion and bottlenecks. We also find that expansion signalsmay be observed under simple stationary Wright–Fisher models with low diversity. Since small populations tend to characterizemany endangered species, we stress that methods trying to infer Ne should be interpreted with care and validated with simulateddata incorporating information about structure. Spurious expansion signals due to social structure can mask critical populationsize changes. These can obscure true bottleneck events and be particularly problematic in endangered species