One of the problems that occur during (artificial) neural network training is called overfitting. The error on the training set is driven to a very small value, but when new data is presented to the network the error is large. The network has memorized the training examples, but it has not learned to generalize to new situations.

The human brain does not overfit when training at some task, but becomes better and better instead. Why? Or conversely, why do artificial neural networks overfit when are trained too much, in contrast with the human brain?

  • 2
    $\begingroup$ What do you mean with "over fit"? $\endgroup$
    – AliceD
    Jun 4, 2015 at 23:56
  • 4
    $\begingroup$ @AliceD Overfitting is when an artificial neural network captures too many extraneous features of the data during training so that generalization becomes nearly impossible. It's an interesting question, but it's unclear, I agree with both of you. $\endgroup$
    – jonsca
    Jun 5, 2015 at 0:30
  • 6
    $\begingroup$ (parenthetically, I'm not convinced that we don't "overfit" at times, especially in the developing brain, but analogies between ANNs and BNNs are often stretched to the point where such shared features become muddled) $\endgroup$
    – jonsca
    Jun 5, 2015 at 0:32
  • $\begingroup$ @jonsca - aha, interesting stuff, I never heard of this! $\endgroup$
    – AliceD
    Jun 5, 2015 at 0:43
  • $\begingroup$ @jonsca - hopefully OP clarifies the question, I am getting curious :) $\endgroup$
    – AliceD
    Jun 5, 2015 at 11:30

4 Answers 4


I would say the human brain overfits all the time! Gambling addiction, superstition and anxiety disorders are all examples of overfitting. We are optimized for seeking patterns and avoiding threats. Our brains mess this up all the time! But having said that, one of the major differences between the human brain and a neural network is the amount of information that's being supplied. Neural networks overfit in part because of limited data sets. Our brains have nearly unlimited data sets available, and this provides us at least the opportunity for some self correction.

  • $\begingroup$ Do you think that since human brain works on a reinforcement learning basis, this is another factor that outperforms supervised learning in ANN regarding issue of overfitting? $\endgroup$
    – Fraïssé
    Jun 7, 2015 at 6:25
  • $\begingroup$ @Shelly DeForte Well, forget for a moment such complex behaviors. At simple classifications tasks, human do not over fit. Or I am wrong? $\endgroup$
    – emanuele
    Jun 7, 2015 at 14:21
  • $\begingroup$ I'm having trouble thinking of a good side by side comparison. The problem is that we're actually pretty terrible at most things you might use an ANN for because we can't really hold more than a couple of dimensions in our minds at a time. Can you give me an example of a simple classification task that a human outperforms on because the ANN is overfitting and the human is not? $\endgroup$ Jun 7, 2015 at 16:42
  • $\begingroup$ characters recognitions... $\endgroup$
    – emanuele
    Jun 8, 2015 at 12:23
  • 1
    $\begingroup$ I don't think we're better at character recognition because we don't overfit. I think we're better at it because our architecture is immensely better for shape recognition and differentiation. But I admit I don't know how ANNs do character recognition, so I'm not sure what the differences are. $\endgroup$ Jun 8, 2015 at 15:35

In this fascinating talk by Geoff Hinton, there is the strong implication that the human brain doesn't overfit, because it implements a version of dropout, i.e. of randomly leaving out signals to prevent coadaptation between neurons.

Dropout has been found to be extremely effective in combatting overfitting.

Basically the idea is that the human brain has a stochastic activation of neurons, which is pretty similar to the dropout technique (except that dropout is usually done with the same probability for each neuron).


Like any statistical learner, the human brain surely experiences overfitting. You can convince people of lots of things that aren't true by showing them biased examples.

That said, don't forget that the tasks by which we judge artificial intelligence are largely defined by what human brains happen to be good at. That's part of what makes us call it "intelligence" instead of just high-performance computing.

In other words, in your character recognition example, don't forget that the character system was developed by humans. Presumably we converged on systems of characters that our neural architectures are good at recognizing and discriminating. And in general, the computer vision tasks that we consider "intelligent" often match tasks that our brains and visual systems have evolved to do well.

So, in a sense, the concepts and tasks that serve as our tests are also inherently "overfit" to our brains.


I'm confident we now have a good hypothesis on this topic: dreams are constantly working to reduce the overfitting and improving the generalization. https://www.sciencedirect.com/science/article/pii/S2666389921000945

If this is true, dreaming should be present in much more animals than we currently know.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .