How does a neuron change as you learn?

Question

I am currently taking a course called "Introduction to Machine Learning with ENCOG 3", and I have a question about how well the Artificial Intelligence (AI) algorithm for a "neural network" corresponds with how an actual neuron works.

In the course, they model a neuron like this:

x1, x2, etc. are voltage inputs, the wij are weights. The inputs are multiplied by these weights and summed up by the neuron. The neuron then has an "activation function" which then takes the sum of weighted inputs and calculates an output, typically a value between 0 and 1 (or between -1 and 1). You can think of the wij as representing dendrites (a higher weight means a more dense and thus conductive dendrite), and the output of the activation function as the voltage that gets sent down an axon.

The AI neural network algorithm creates a kind of intelligence by modifying the weights (wij shown in the picture).

My first questions is: Is this a good approximation as to how neurons actually work? That is, do our neurons "learn" by changing the weights (dendrite density, conductivity)? Or, is there some other mechanism that is more important (e.g. do neurons learn by changing their activation or summation functions?)

My second question is: If neurons really do learn by changing the density of dendrites, then how fast does this happen? Is this a fast process like DNA replication? Does the neuron quickly generate (or decrease) dendrite density when it receives some kind of biochemical signal that it needs to learn now?

I understand that much of this might not yet be known, but would like to know how well the AI algorithm corresponds with current theories on biological neural networks.

Answer 1

With respect to your first question, that model isn't intended to take time into account, but is based on Hebbian learning with the goal of computability. It's generally used in simple pattern recognition situations where each case has no bearing on the next. The learning portion is performed ahead of time during the training phase. For example, a deterministic perceptron isn't permitted to change after training. In contrast, the Biological Neuron Model is much more complex, and uses a variety of ordinary differential equations to integrate the cumulative behavior of neurons over time. As such, those models are non-deterministic and don't see as much practical use outside of experimentation.

To address your second question, neurons themselves don't "learn." A single neuron is essentially meaningless. Learning is an emergent process created by the interaction of several systems at once. While one influencing factor is connectivity (zero for no connection, non-zero for the inhibitory or excitatory connection which emulates both synaptic and non-synaptic), what you might call short term learning can also be performed by existing clusters of neurons, without any change in connectivity. Biologically, this is what must occur before any of the comparably slow process of tissue remodelling can take place, and the computationally equivalent process is only possible in time-aware models like the aforementioned Biological Neuron Model.

Take, for example, someone who wishes to learn to play guitar. When they begin playing, existing clusters emulate the desired behavior as best as they can. These neurons act as the functional scaffold that initiates and drives the neuroplastic process. Playing becomes easier because this scaffold becomes more efficient as new connections (shortcuts) are created, existing connections are strengthened, and irrelevant connections are inhibited. The improved scaffold in turn allows further improvements. Newborn neurons may also migrate to the area, though the how, why, and when of that process is unclear to me. This "behavior emulation" or "short term learning" process used in practicing the guitar, or whenever a novel situation is encountered, must be primarily governed by excitatory and inhibitory neurons' influence. Otherwise the whole process cannot even begin.

Further reading

• Parameter estimation of a spiking silicon neuron.
• Synaptic plasticity, memory and the hippocampus: a neural network approach to causality
• Signaling mechanisms in cortical axon growth, guidance, and branching

Answer 2

This model was a reasonable approximation when it was formulated. However, as we learn more about synaptic integration, it appears that neurons are best approximated by two-layer neural networks (1). In this way the active properties of individual dendrites are better taken into account. Each dendrite, or at least a group of dendrites, is viewed as capable of performing input summation from individual synapses, independently of the integration at the level of the soma.

Regarding your first specific question, mechanisms of plasticity other than modifying the individual weights have also been described, for example as you suggested, by modifying the intrinsic biophysical properties of neurons that lead to firing (2). These would be equivalent to altering the sigmoid function or all the weights collectively.

Regarding your second question, we do not think that changes in the weights happen through modification of the dendrite as a whole. Rather, they represent changes at the level of single synapses, what are termed 'synaptic weights'. The biological phenomenon underlying synaptic weight change is referred to as synaptic plasticity. It is thought to manifest through a variety of biophysical mechanisms, including incorporation of new receptors at the post-synapse, changes in those receptors' conductivity, increase in the number of pre-synaptic vesicles etc. These events can be quite fast, in the order of seconds to minutes (3). More long-term changes of the weights, referred to as long-term potentiation are thought to depend on protein synthesis which is a slower process.

---

References

1) Poirazi, P., Brannon, T., & Mel, B. W. (2003). Pyramidal neuron as two-layer neural network. Neuron, 37(6), 989–99. http://www.cell.com/neuron/fulltext/S0896-6273(03)00149-1

2) http://www.scholarpedia.org/article/Intrinsic_plasticity

3) Harvey, C. D., & Svoboda, K. (2007). Locally dynamic synaptic learning rules in pyramidal neuron dendrites. Nature, 450(7173), 1195–1200. https://doi.org/10.1038/nature06416