How to determine whether changes of an allele's frequency are due to genetic drift or selection?

Question

Given that both natural/artifical selection and genetic drift are evolutionary mechanisms that influence how allele frequencies shift in a population:

Are there ways to determine whether a frequency shift for a specific gene/allele is due to genetic drift or selection? In other words: when we see an allele take over a population, is there a way to know whether it was selected or just 'got lucky'?

UPDATE: since apparently this issue is quite complex in experimental biology I want to clarify why I'm asking this and what I hope to learn.

I'm looking for a 'theoretical way' of determining whether the changes in allele frequencies in subsequent generations are the product of selection or mere drift.

I'm applying a genetic algorithm to the optimisation of websites, where each gene determines an aspect of the user experience of that website. The question is whether the alleles I see emerging in the fittest solutions (= the best user experience, rated by visitors) got there by chance or actually contribute to the fitness of the candidate solution (= make a difference in shaping the user experience).

Answer 1

Is there a way to disentangle selection from drift

Yes, there are actually many ways...so many so different ways that it is impossible to correctly answer this question. Hopefully, below answer will give you some appreciation of why there is such a diversity of methods and wil give you sources to further your knowledge on the question.

It depends upon the data you have

The answer very much depends on the data you have in mind. Are you looking at a

• panmictic population
• a structured population
• are you comparing a large number of lineages along a phylogeny
• do you have information about a proxy for fitness in this population
• can you perform an evolution experiment (depends a lot on the generation time)
• do you have genetic data that are temporally separated
• ...

Also, what type of selection are you thinking of?

• balancing selection
• Are you trying to find epistasis for selection
• Are you looking at some form of balancing selection
• Local selection
• ...

Also depends upon whether you have data about potentially correlated variables such as

• phenotypic data
• ecological data

There are tens (or even hundreds of methods esp. when having data about related lineages) under each bullet point above, so your question is definitely quite broad.

Source of information

Statistical tests that test to the null that there is no selection are often called "test of selective neutrality". You might want to read Nielsen 2001 and Nielsen 2005 to get an overview of tests of selective neutrality in panmictic populations.

You might want to have a look at this answer, this answer and especially this answer.

What does it take to understand the methods for disentangling selection from drift?

Understanding the tests of selective neutrality can be a bit demanding as it requires having good knowledge in classical statistics, bayesian statistics, MCMC, Approximate Bayesian Computation and even machine learning algorithms. It also requires good knowledge in population genetics (typically incl. coalescent theory) and statistical phylogenetic.

A specific example

Unfortunately, I can't think of any specific example of a test which is really easy to grasp without a priori knowledge. A classical, not too hard and historically example test for neutrality in panmictic population is the used of Tajima's D (which derivation require basic knowledge in branching processes). You might want to learn about it to have a specific example. I recommend Gillespie's book Population Genetics: A Concise Guide to learn about this test. More book recommendation can be found here.