2
$\begingroup$

The brain is the most complex thing that mankind discovered thus far in the universe. Super computers aren't able yet to outperform the human brain (running only on ~25 watt or so) on the domain of general intelligence. With a computer it is fairly straightforward to capture an image stream and split it into 2 streams. So in a videoclip depth data could be contained by alternating the images from 2 streams from two different angles.

What if an organism would loose sight in 1 eye. And the other eye would be fed with data that contains actually 2 image streams, with these alternating angled images, at an always exact framerate. Would the organism (the brain basically) be able to comprehend the data and experience depth? For that it would kind of need to buffer every other image and put them through simultaneously, mimicking the feed from two eyes basically.

Would this be possible? Is there any research done on a topic similar to the example above? Where data is buffered by a neural network to be send in combination with other data?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

For this answer, I'll use a very broad definition of "buffer" as a mechanism to integrate information at different delays. I have no interest in getting into specific computer science definitions of what a buffer is and I don't think it makes any sense to use them in biology.

Parallel processing and predictive coding

The brain is a massively parallel system, so of course there are different "streams" of information flowing at all times. Certainly the brain as a whole does some level of buffering, though the mechanisms available to individual neurons are unlikely to do what you suggest. The "buffering" in individual neurons has a lot more to do with learning and memory, for example in spike-timing-dependent plasticity which incorporates a physical record of past inputs in a learning rule.

One model of cognitive function is predictive coding, in which the brain holds a model of the world at different hierarchical scales and updates this model with new information. This model is effectively a buffer against which new incoming information is compared to draw attention to discrepancies that require further attention (such as the sudden appearance of a predator).

Audiovisual integration

One simple example of buffering is in audiovisual integration: the brain's processing of visual information is much slower than auditory information, and yet the perception of, say, a human speaker's voice vs. lips or the sound and sight of a bouncing ball are perceived in synchrony (see for example Recanzone 2009; note this has absolutely nothing to do with speed of sound vs speed of light).

It's important to understand that the visual system doesn't process visual information anything like a computer. There are no frames in biological vision. That said, humans can deal with a lot of unnatural sensory input, and they also use a lot of other visual cues besides stereo vision, the so-called "monocular cues".

Motion parallax and wiggle stereoscopy

One of these monocular cues is motion parallax (binocular vision can also utilize parallax without motion). Wiggle stereoscopy simulates motion parallax, as if your head were oscillating back and forth, to create an illusion of 3D, and seems much like what you are proposing. It doesn't matter whether you view the image with one eye or two.

Recanzone, G. H. (2009). Interactions of auditory and visual stimuli in space and time. Hearing research, 258(1-2), 89-99.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .