How statistically significant was the “sweaty T-shirt” experiment?

Question

An Overview of the Experiment.

In 1995 there was an experiment that involves young people smelling T-shirts worn by another gender and rating their preference. The researchers found that people preferred shirts belonging to people with a dissimilar Major Histocompatibility Complex (MHC). Another interesting finding was that this preference reversed for the women 'sniffers' that were using oral contraceptives.

Here is a link to the wikipedia entry.

These findings were reiterated by a study in 2005.

My Question.

I'm very ignorant in the subject of pheromones, particularly in humans. For those of us not 'in the know' a result like this seems like magic! I want to get a better grasp on how set in stone pheromone preferences like this are.

Unfortunately I do not have access to these articles at the moment. The actual statistical difference in populations isn't mentioned in the abstracts (the cynical sceptic alarm bells are ringing!) other than the 2005 one stating that the total sample size was 58, which sounds low given that the total population is split into two.

Generally I'm looking for any supporting or opposing critiques of the study, specifically I want to know more about the statistics of the studies.

My question is:

• What statistical validations and scores appear in these studies?

I also am willing to accept answers discussing validation in similar studies. I'm after a number or quantification for how consistent pheromone preferences are.

Answer 1

First, let's consider the original Wedekind et al (1995) study.

Sample sizes seem reasonable, if the effect is not very noisy:

[...] 49 female students (average age: 25.2 years, s.d. = ­4.0) and 44 male students (average age: 24.7 years, s.d. = 2.6) [...]

... and the statistical design accounts for individual differences in a fairly robust way:

[...] women were asked to rate the odours of six T-shirts each, three of them worn by men who were dissimilar to the rating woman's MHC (average number of dissimilar HLA-antigens = 5.9, s.d. = 0.26), and three worn by men who were more similar to it (average number of dissimilar HLA-antigens = 2.7, s.d. = 0.74). We tried to present every T-shirt as often to MHC-dissimilar women as to MHC-similar women (average difference of presentations to thc two groups: -0.02, s.d. = 0.73). The presentation was random in every other respect [...]

Rather irritatingly, although the study design involved asking women to rate at least three attributes of the smells (sexiness, pleasantness, and intensity) the study performs a bait-and-switch in its results:

The scores for sexiness are not shown in the figures as they were highly correlated with pleasantness (all scorings: r = 0.85, n = 294; for women who do not take the pill: r = 0.87, n = 186; for women who take the pill : r = 0.83, n = 108, p always << 0.001)

This is a bait-and-switch because even if A correlates with C, and B correlates with C, we do not know that A and B correlate with each other. Of the three measured attributes, 'sexiness' arguably has the most direct bearing on mate-choice. Because analysis for 'sexiness' is not presented, the paper ends up telling us much less than it could have about the relationship between MHC relatedness and mate choice.

The results for 'pleasantness' and 'intensity' look like this:

One further piece of analysis is presented: some number of the female subjects seem to have reported that particular smells reminded them of their partner or ex-partner; or reminded them of family members. Mentioning that a shirt smelled like a partner or ex-partner happened more frequently for MHC-dissimilar shirt smells (~17% vs ~9%; p = 0.038, two-tailed Fisher exact test). Oddly, no written response-collection is described in the methods, so I can only assume that these assessments were ad-hoc.

If this was all of the analysis that had been done on the topic, my assessment would be: this is amongst the many published research findings that are probably false. My main concerns would be that

1. Not all of the collected data are presented;
2. The most directly-informative contrast is not shown;
3. No adjustments for multiple comparisons are made on the statistical tests;
4. The data that are shown are very noisy, and sample sizes are not huge, raising concerns that the tests are underpowered;
5. The authors conduct tests based on data-collection which is not detailed in their methods, raising concerns about p-hacking.

And now, the 2005 paper!

Right off the bat, this paper cannot possibly replicate the findings of the Wedekind (1995) paper, because Wedekind et al found significant preferences for dissimilar MHC odours in women not taking the pill, and significant preferences for similar MHC odours in women taking the pill. In this study:

[...] we did not distinguish between, in our statistical analysis, women who were taking oral contraceptives from women who were not.

[...] Nine female participants (31.03% of female subjects) were taking oral contraceptives.

Sample sizes are smaller than Wedekind 1995 in terms of number of participants, but larger in terms of number of contrasts, because this experiment had two different sample types (sweat odour and urine), and had men smelling women as well as women smelling men. Within-individual effects are not modelled, so there is a potential for (e.g.) a small number of generically good-smelling individuals with rare MHC alleles at the population level to bias the analyses - whereas this potential bias was accounted for in Wedekind by performing within-individual correlations. The sample-sizes look like this:

The analysis was a series of Chi-squared tests, for data collected in four different sessions: one with men smelling female urine, one with men smelling female body odour, one with women smelling male urine, and one with women smelling male body odour. All chi-squared tests returned non-significant results with the exception of women smelling male body odour, for which the Chi-squared table looks like this:

So, this study didn't find the same result as the Wedekind study, but then, it couldn't find the same thing as the Wedekind study because of the way the analysis was performed, so that is no surprise. The authors claim that they found a significant effect of MHC relatedness on odour perception, but only in the proportions in the hard-to-interpret 'indifferent' response category. I am inclined to disagree. The authors presumably conducted 16 contrasts (a test of independence and three goodness-of-fit tests in each of four treatment types), and found one instance of p < 0.05. I would argue that the one 'significant' result is best explained as a failure to account for multiple comparisons.

Ending notes

If these two papers were the only things that had been published on MHC-directed mate preferences in humans, I would be far from convinced that the claimed effects were real.

however...

These are far from the only papers on the subject of MHC-driven mate-choice in humans. Wedekind and Füri (1997) conducted an analysis with very similar experimental design to Wedekind et al (1995), but arguably better statistical analysis, and found the same direction of association as Wedekind et al (1995), albeit still with a fairly high noise-to-signal ratio. Subjectively, this goes a reasonable distance towards making me think that the findings of Wedekind et al (1995) were not just a statistical fluke. I'm still not utterly convinced, but certainly more convinced than I would be if Wedekind's final paper on the topic had been the 1995 publication.

MHC-driven mate-choice has also been investigated using facial preferences and mate choice surveys. The combined evidence (across odour preference, facial preference, and mate choice surveys) is reviewed in Havlicek and Roberts (2009).

Answer 2

@bshane has already provided an excellent answer to the question. However, I just found a recent review published in Molecular Ecology (Winternitz et al., 2016) in which the authors performed a phylogenetically controlled meta-analysis on MHC-dependent patterns of mate selection in humans and non-human primates. They used studies from seven primate species and looked at 30 studies (both observational and experimental) with a total of 58 effect sizes.

Short answer | Conclusion:

Winternitz et al. (2016) conclude that their data suggest a significant preference for mates with higher MHC diversity in humans that seems to be conserved across primates. Diversity here means heterozygosity in MHC loci.

---

An overview of their major findings:

First, a few words on the methodology. In principle, one needs to disentangle actual differences in the outcome of mating from error/random noise. To do that, they estimated heterogeneity in effect sizes, accounted for study ID and phylogeny effects on heterogeneity, and - for the residual unexplained heterogeneity - constructed meta-regression models with which they identified mediators of the respective effects. Effect sizes are given as Fisher's normalized correlation coefficients $Z_r$ that is loosely similar to the correlation coefficient $r$ (they refer to ecological literature and claim that $r \approx 0.1 \rightarrow Z_r \approx 0.1$ is a small effect, $r \approx 0.3 \rightarrow Z_r \approx 0.31$ is a medium effect and $r \approx 0.5 \rightarrow Z_r \approx 0.55$ is a large effect). They then used mean posteriors and highest posterior densities (HDP, as 95% confidence intervals) to estimate $Z_r$, i.e. an assessment of how strong the mediators influence the effect. If highest posterior densities do not overlap zero, they consider the finding as a (significant) real effect.

Here is a list of what they found (HDP always in parentheses):

• There is a statistically significant preference of MHC-diverse (i.e. a preference for heterozygosity in MHC loci) mates in humans ($Z_r$ = 0.153 (0.020 to 0.283), ten effect sizes). This preference also is a (non-significant) trend in primates (humans and non-humans combined) for preferring MHC-diverse mates ($Z_r$ = 0.128 (-0.064 to 0.373), 17 effect sizes). The preference is stronger in females than in males which is known from several vertebrates.

• Across primates there is no significant tendency for favouring mates with MHC-dissimilarity (0.044 (-0.178 to 0.289), 37 effect sizes), even though the direction of effect sizes hints at MHC-dissimilarity rather than MHC-similarity.

• For humans, there is larger variation to that point. They found effect sizes that indicate choice of both MHC-dissimilar (see above point, also true for humans) and MHC-similar mates ($Z_r$ = -0.022 (-0.107 to 0.073), 31 effect sizes). The latter may rather be an artifact of assortative mating with respect to ethnicity: effect sizes of MHC-similarity were only significant in ethnically heterogeneous populations, not in homogeneous populations. Further evidence for this comes from the fact that this is only true for observational studies, but not in experimental studies that tend to control for socio-cultural biases.

One needs to note that, in general, their reported effect sizes of MHC-related mating preferences in primates are weak (MHC-dissimilarity $Z_r$ = 0.044, MHC-diversity $Z_r$ = 0.153). But nevertheless interestingly, they report a significant effect for choice of mates with heterozygous MHC loci in humans. And - this kind of blew my mind just a little bit - they cite some literature that suggests that heterozygosity in general and of MHC loci in particular is not only reflected in body odour but also in other features like facial morphology.

Answer 3

I have seen many studies on that matter, but I am far from convinced that MHC really effects mate choice. Reasons for this are the following: - iayork mentioned it already, the premise for the studies about mice mate choice has been disproved. - In population level, almost every one of the studies find no evidence to support MHC based mate choices in general population. - Perception studies done in the area have what I consider serious problems in replicating the results. For example, wedekind's both studies feature same smellers and t-shirt wearers, but their answers are not the same as in the first study. - The latest perception study cannot replicate wedekinds results and the methods are almost the same.

So we have a hypotese, which has lost its original premise (mice mhc based mate choices), first studies doesnt replicate the results with same individuals, there are only couple of population level studies that suggest disassortative mhc mate choices and the latest study cannot replicate the first two perception studies.

On an individual level, these studies are kind of silly. If the effect is true, how can you discriminate between partners who are dissimilar or similar, if you would score them 5/10 or 4/10. The same applies to the brazilian study: your probability of percepting your partner as indifferent or unpleasant changes based on your HLA-similarity from 50% to 60% in different scenarios. Your overall chance of being 2-3 HLA-similar with this sample is about 10%.

I am just worried about these studies as media picks them up. Some people may come up with anxiety over their partners smell, which is quite silly because with over 50% probability you may be completely dissimilar with your partner regarding HLA, but one still perceives his/her odour as indifferent or unpleasant.

bshane and iayork, do you have any comments of the subject at hand?