3
$\begingroup$

In computer science, 'convolutional neural networks' are used, that are meant to be inspired by biological network structures like found in the human brain visual cortex.

In the computer implementation, the 'convolution' means that any neuron of a certain 'layer' is linked with several spatial adjacent neurons in the previous layer. When those networks are trained, the 'weights', which are analog to synaptic strength, of all neurons of one layer have an equivalent local topology, and thus can be mapped between the neurons, and finally unified. This way, only one 'weight' is trained for all equivalent synapses on all neurons of an layer.

Is there any biological mechanism found that may sport this feature? My guess would be that the vision cortex is partly organised by self structuring not depending on neural activation to achieve low level vision processing. Otherwise, all neurons in one 'layer' would need a way to exchange information to synchronize their synaptic structure.

However, I guess that the human brain would have some mechanism of this kind, as it would provide for example the ability to learn the detection of certain visual patterns or events independent of their retinal position, without learning the detection individually for every image location.

Is any such mechanism known or how could it work?

Improve this question
$\endgroup$
3
  • $\begingroup$ As you already said, there are several parts of the brain that have this kind of mechanism. I am not clear about what you are asking. $\endgroup$
    – WYSIWYG
    Commented Jul 27, 2015 at 5:06
  • $\begingroup$ The usual model of neurons and plastic synapses does not explain this feature, as large numbers of physically unconnected synapses would need to have some magic remote communication to equalize their transmission strength. So my question is if there are any mechanisms known in neural tissue which may implement this feature. $\endgroup$
    – dronus
    Commented Aug 10, 2015 at 10:32
  • $\begingroup$ I believe CNNs were inspired on the fact that the visual cortex has neurons that have a fairly small "visual field" and they overlap each other. Not necessarily that they are tied together. $\endgroup$
    – Felipe
    Commented Dec 9, 2015 at 14:27

1 Answer 1

Reset to default
1
$\begingroup$

The eye might provide alone an example of a convolution neural network (as made popular for image classification tasks).

The cell types and synaptic connections in the retina are even organized in layers (laminae) to make the parallel clearer, as seen here:

enter image description here

source: http://webvision.med.utah.edu/book/part-vi-development-of-cell-types-and-synaptic-connections-in-the-retina/development-of-cell-types-and-synaptic-connections-in-the-retina/

The topology of the network is in a sense self-structured but a relatively slow process as the blueprint is a product of Evolution and the practical implementation (actual cells and connections) takes place during the development of the organ. "Weight tying" can then be seen as a result of having different cells types in the network.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I would agree that this is a data processing convulutional structure. But shared wheights (maybe encoded by cell types) would be a genetical feature then, and they can not be trained in a shared fashion in the living organism. Simply said, if the circuit learns by weight adaption how to detect an eye projectet to part of the retina, that doesn't mean other parts of the visual field would detect this eye as well. This feature however holds for CNNs by sharing the weights on all convolutional subunits in one layer and speeds up learning by orders of magnitude. So can brain do this? $\endgroup$
    – dronus
    Commented Mar 9, 2016 at 11:36
  • $\begingroup$ I believe it is important to create a distinction in terms of what level of visual system you are referring to; i.e. retina vs higher level cortical visual areas. Also, if I remember correctly, the weights are represented by the strength of synaptic connections, which are modified adaptively during the process of learning and not hard-coded by genetics. You might find it useful to take a look at goo.gl/ygP6nF $\endgroup$
    – Noushin
    Commented Mar 9, 2016 at 13:44
  • $\begingroup$ One could hypothesize that "learning" takes place at two levels: one is evolutionary and one is within an organism's lifespan. Cell (neuron) types are "trained through evolution (may be not the idea you had of speed though but the same advantage, presumably). Regarding the amount of visual "processing" performed down by the retina (not everything is down in the cortex), there is a relatively short read on wikipedia: en.wikipedia.org/wiki/Receptive_field $\endgroup$
    – lgautier
    Commented Mar 13, 2016 at 2:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .