Spaun performing recognition. Spaun must categorize the presented visual input. The images are taken from the publically available MNIST database. Overall the model has 94% accuracy (people have about 98% accuracy on this data set).
Spaun performs this simple recognition task three times in a row. The thought bubble at the back of the brain shows the decoding of the neural activity at the highest level of the visual hierarchy. As you can see, this is quite accurate, and only very briefly delayed from the input stimulus.
The activity of this top-level, which we can think of as inferotemporal cortex, is at the end of a four layer hierarchy that includes visual areas V1, V2, and V4. The neurons in the earliest visual area (V1, also called primary visual cortex) have receptive fields and neural responses like those of primates. However, these areas are not shown in the video.
Although the images Spaun is shown are quite variable, being examples of human hand-writing, the model is about 94% accurate in recognizing digits. This is only slightly below the 98% accuracy of humans on the same data set. It is almost 100% accurate on the images of type-written digits used to specify tasks.
The neural activity in the motor area is at the top of the motor hierarchy. This can be thought of as a low dimensional representation of a motor plan, that is made progressively higher-dimensional as it proceeds down the hierarchy. The plan needs to become higher dimensional in order to control the many muscles that ultimately drive the arm.
This task demonstrates that Spaun's neural representations not only allow successful categorization of naturally varying stimuli, but also allow that categorization to drive appropriate behaviour.