Skip to content
TheChatArchiveThe Chat Archive
GLMGLM-5 / Pony Alpha· unknownHugging Face Frontier Reasoning Traces

Explain coalescent‑based demographic inference: model specification, site frequency…

  1. User

    Explain coalescent‑based demographic inference: model specification, site frequency spectrum fitting, goodness‑of‑fit tests, and uncertainty communication.

  2. GLM Assistant

    1. Analyze the Request: * Topic: Coalescent-based demographic inference. * Key Components: * Model specification. * Site Frequency Spectrum (SFS) fitting (Likelihood approximation). * Goodness-of-fit tests. * Uncertainty communication. * Goal: Explain these concepts clearly, covering the theoretical basis and practical application.

    1. Deconstruct the Components:

      • Introduction:

        • What is the Coalescent? (A brief recap: looking backward in time to find the Most Recent Common Ancestor - MRCA).
        • What is demographic inference? (Estimating population size changes, migration, divergence times).
        • Why coalescent-based? It provides a mathematical framework connecting population history to genealogical trees.
      • Component 1: Model Specification:

        • Concept: We need to define a hypothesis about history.
        • Parameters (Θ\Theta): NeN_e (effective population size), mm (migration rates), tt (time of events), rr (growth rates).
        • Types of Models:
          • Simple: Constant size, Exponential growth.
          • Complex: Bottlenecks, Multi-population divergence (Isolation-with-Migration - IM), Piecewise-constant (skyline/stairway plots).
        • The Role of the Genealogy: The coalescent model defines the probability distribution of genealogies given the parameters.
      • Component 2: Site Frequency Spectrum (SFS) Fitting:

        • Concept: How do we compare the model to data? We usually don't know the true genealogy. We use the SFS.
        • What is the SFS? Distribution of allele frequencies in a sample. (Folded vs. Unfolded).
        • The Likelihood: L(Θ∣Data)≈L(Θ∣SFS)L(\Theta | Data) \approx L(\Theta | SFS).
        • The Theory (Coalescent + Mutation):
          • E[SFS]=Mutation Rate×Length of Branches contributing to specific frequency bins\mathbb{E}[SFS] = \text{Mutation Rate} \times \text{Length of Branches contributing to specific frequency bins}.
          • Under the Infinite Sites Model, the expected SFS is a deterministic function of the expected branch lengths of the genealogy (which depends on demography).
        • Diffusion Approximation (∂a∂i\partial a \partial i, fastsimcoal2): Usually, we don't simulate trees one by one. We solve diffusion equations to get the expected SFS.
        • Fitting: Maximize the composite likelihood (usually Poisson likelihood) to minimize the distance between the observed SFS and the model-expected SFS.
      • Component 3: Goodness-of-Fit (GoF) Tests:

        • Concept: Just because we found the "best" parameters doesn't mean the model is right. (All models are wrong, some are useful).
        • Visual Inspection: Plotting Model SFS vs. Observed SFS (residuals).
        • Statistical Tests:
          • Chi-square test (χ2\chi^2): Sum of squared residuals normalized by variance.
          • Likelihood Ratio Test (LRT): Comparing nested models (e.g., constant size vs. growth).
          • AIC/BIC: For non-nested or complex models.
        • Parametric Bootstrapping: Simulating data under the fitted model to see if the observed statistic is an outlier. (This is crucial because SFS entries are not independent, violating assumptions of simple chi-sq).
      • Component 4: Uncertainty Communication:

        • Concept: Point estimates are dangerous. We need confidence intervals.
        • Profile Likelihoods: Changing one parameter, maximizing others, seeing where likelihood drops (e.g., by 2 units).
        • Bootstrapping:
          • Non-parametric: Resampling loci (snps) with replacement. Recalculate SFS, refit model.
          • Block Bootstrapping: Important to handle linkage disequilibrium (LD).
        • Jackknifing: Leave-one-out (usually chromosome) approach.
        • Visualization: Credibility intervals, densities.
    2. Structure and Refine the Output:

      • Use clear headings.
      • Use LaTeX formatting for math where appropriate.
      • Keep the tone academic but accessible.
      • Self-Correction during drafting: Make sure to distinguish between the genealogy (hidden variable) and the SFS (observable data). Emphasize that SFS-based inference is "composite likelihood" because SNPs aren't fully independent.
      • Check specifically regarding "Goodness-of-fit": Ensure the distinction between absolute fit (does the model fit the data?) and relative fit (is model A better than model B?) is clear. Mention residuals (quantile-quantile plots or residual plots).
    3. Drafting the Content (Iterative refinement):

      • Introduction: Define Kingman's Coalescent.
      • Model Spec: Piecewise constant is the standard in tools like stairway plot or fastsimcoal2. Mention NeN_e and μ\mu.
      • SFS Fitting: Explain the expected SFS (ψ(Θ)\psi(\Theta)) vs Observed (S\mathbf{S}). The objective function is usually Composite Log-Likelihood.
      • GoF: Residuals are key here. Mention the "hockey stick" or specific patterns in residuals indicating model misspecification.
      • Uncertainty: Block bootstrap is the gold standard for genomic data due to linkage.
    4. Final Review against User Prompt:

      • Model specification? Yes.
      • SFS fitting? Yes.
      • Goodness-of-fit? Yes.
      • Uncertainty communication? Yes.
    5. Final Polish: (Produce the structured response below). Coalescent-based demographic inference is a statistical framework used in population genetics to estimate historical population parameters (size changes, divergence times, migration rates) from genetic sequence data.

    The core logic works backward in time. Instead of simulating how a population evolves forward, the Coalescent Theory models the process of lineages merging ("coalescing") as we trace the ancestry of a sample of genes back to a common ancestor. The rate at which these lineages merge is determined by the effective population size (NeN_e).

    Here is a detailed explanation of the four key components you requested.


    1. Model Specification

    Model specification is the process of defining the demographic history we wish to test. In a coalescent framework, a "model" is a mathematical description of how population size and structure have changed over time.

    • Parameters (Θ\Theta): The user defines a set of parameters to be estimated. Common parameters include:
      • Ne\mathbf{N_e}: Effective population size.
      • T\mathbf{T}: Time of demographic events (in generations or coalescent units).
      • m\mathbf{m}: Migration rates between populations.
      • r\mathbf{r}: Population growth rates.
    • Model Topologies:
      • Simple: A single population with constant size.
      • Parametric Growth: Exponential or logistic growth models (e.g., bottleneck followed by expansion).
      • Complex/Structured: Multiple populations diverging at specific times with continuous or pulse migration (Isolation-with-Migration models).
      • Non-parametric: "Skyline" or "stairway" plots where time is divided into many segments, and NeN_e is estimated for each segment freely, without assuming a specific functional form (like exponential growth).

    The Goal: We define a model MM with parameters Θ\Theta. Our objective is to find the values of Θ\Theta that maximize the probability of observing our genetic data.


    2. Site Frequency Spectrum (SFS) Fitting

    While the coalescent defines the probability of genealogies, calculating the likelihood of the full sequence data (every base pair) and every possible tree topology is computationally intractable for large datasets. To solve this, researchers summarize the data into the Site Frequency Spectrum (SFS).

    The Data Summary (SFS)

    The SFS is a histogram of allele frequencies.

    • Unfolded SFS: For a sample of nn chromosomes, we count how many mutations are found in 11 copy, 22 copies, ... n−1n-1 copies. (The 00 and nn classes are usually fixed for ancestral/derived states).
    • Folded SFS: If the ancestral state is unknown, we fold the spectrum symmetrically (e.g., a mutation at frequency 1 is indistinguishable from frequency n−1n-1).
    • Joint SFS: For multiple populations, this becomes a multidimensional array counting the number of alleles found at specific frequencies in Pop A and Pop B simultaneously.

    The Fitting Process

    Most inference tools (e.g., ∂\partiala∂\partiali, fastsimcoal2, stairway plot) use the Diffusion Approximation to fit the model to the observed SFS.

    1. Calculate Expected SFS (E[S]\mathbb{E}[S]): Given a demographic model (e.g., "Population A had a bottleneck 5,000 years ago"), the diffusion equation predicts the expected shape of the SFS. For example, a bottleneck creates an excess of rare variants (left side of the histogram), while population expansion creates an excess of intermediate frequency variants.
    2. Composite Likelihood: The software compares the Observed SFS (SobsS_{obs}) with the Expected SFS (SexpS_{exp}). Because SNPs are not fully independent (due to linkage), we usually calculate a Composite Likelihood (assuming independence) rather than a full likelihood.
      • This is typically modeled as a Poisson likelihood across the frequency bins.
    3. Optimization: The algorithm searches the parameter space to find the parameters Θ^\hat{\Theta} that minimize the distance (or maximize the log-likelihood) between the expected and observed spectra.

    3. Goodness‑of‑Fit Tests

    Once the parameters are estimated, a critical step remains: Does the model actually describe the data well? This is Goodness-of-Fit (GoF).

    Visual Inspection

    The first test is almost always visual. We plot the Model SFS (the theoretical expectation based on estimated parameters) against the Observed SFS.

    • If the model fits well, the points should lie on the diagonal (y=x).
    • Systematic deviations (e.g., the model predicts too many rare variants compared to reality) suggest the model is misspecified.

    Statistical Tests

    Because linkage disequilibrium makes standard statistical tests (like χ2\chi^2) difficult to interpret accurately on SFS data, specialized approaches are used:

    1. Likelihood Ratio Test (LRT): Used to compare two nested models (e.g., Model A: Constant size vs. Model B: Exponential growth).
      • D=2×(LogLikelihoodComplex−LogLikelihoodSimple)D = 2 \times (\text{LogLikelihood}_{Complex} - \text{LogLikelihood}_{Simple})
      • We check if the improvement in likelihood is statistically significant.
    2. Parametric Bootstrapping:
      • Simulate genomic data using the estimated "best-fit" parameters.
      • Calculate the likelihood score (or a GoF statistic) for these simulations.
      • Compare the observed data's score to the distribution of simulated scores. If the real data falls in the extreme tails of the simulated distribution, the model is a poor fit.

    Common Failure Modes:

    • Mis-specification: The SFS often captures the "average" coalescent time, but has low power to distinguish between a long bottleneck vs. a recent expansion, as both can skew the SFS in similar ways.

    4. Uncertainty Communication

    A point estimate (e.g., "Population size was 10,230") is useless without a confidence interval. In coalescent inference, uncertainty arises from both the randomness of the evolutionary process and the sampling of individuals.

    Profile Likelihoods

    We examine the likelihood surface around the maximum estimate.

    • We plot the likelihood as one parameter changes while others are optimized.
    • The "sharpness" of this peak determines the certainty. A flat likelihood surface implies high uncertainty; a sharp peak implies high certainty.

    Bootstrapping

    To generate Confidence Intervals (CIs), we use resampling methods:

    1. Non-parametric Block Bootstrap: Because SNPs are linked, we cannot simply resample individual sites. Instead, we divide the genome into blocks (e.g., 100kb chunks) or whole chromosomes.
    2. Resampling: We sample these blocks with replacement to create a "pseudo-genome."
    3. Re-inference: We run the demographic inference on this pseudo-genome.
    4. Repetition: Repeating this 100+ times creates a distribution of estimates for each parameter.
    5. Result: We report the 95% Confidence Interval (the range containing 95% of the bootstrap estimates).

    Uncertainty in NeN_e vs. Time

    It is crucial to communicate that uncertainty is not uniform over time.

    • Recent history: Inferred from many lineages (high information), usually low uncertainty.
    • Ancient history: Inferred from deep coalescent events (fewer lineages surviving), resulting in much wider confidence intervals. This is often visualized as a "trumpet shape" in skyline plots (wide intervals at the bottom of the plot/ancient times).