Spaun performing several tasks. Only neural activity is shown (no decoding of the activity). The model is running at about 1/2 real time. The tasks it performs are: A1 (recognition), A3 (serial working memory), A7 (syntactic pattern induction). The tasks are described in the videos of each separate task.
The purpose of this video is to allow you to watch just the input, output, and neural activity of Spaun while it performs several tasks. These tasks are demonstrated and described in more detail in a previous video.
Here we can see that the input is a 28x28 pixel image on the right, which is processed by the spiking neural networks of the model. Activity of some of these networks is shown by the coloured plots on the brain. Red indicates high activity, and blue indicates low activity. As the model has 2.5 million neurons, only a small proportion are shown.
However, these are mapped the their corresponding anatomical areas. For example, inferotemporal cortex (IT), the highest level of the visual hierarchy is a the back of the brain, while motor areas are in the vertical stripe in the middle. Executive control areas are at the front, with working memory areas just behind them in prefrontal cortex (PFC).
In addition, two parts of the basal ganglia (BG), which in fact lies underneath cortex, are shown in the horizontal stripes in the middle. The top stripe is the striatum (Str), the input to BG, the bottom stripe is globus pallidus internus (GPi), output from BG. The BG as a whole is monitoring cortical states to determine the next appropriate cognitive or physical action.
Physical actions themselves are evident in the movement of the arm. Though simplified, the arm is a dynamic, physical model of a limb, having mass, length, inertia, and so on.
The simulation is likely too fast and unfamiliar to demonstrate the subtleties of the model. In the next video, I show similar tasks, but include graphs that allow us to see dynamic interpretations of this neural activity at a slower speed -- essentially, we can read Spaun's mind to get a sense of how it thinks.