We propose a method for training convolutional neural networks to learn similar systems of transformations as exist in the human brain. We recorded brain data from exposing a participant to plenty of visual data (a TV series) and use the same visual data as input to a network. We train the network by attaching brain activity observation models for different visual system regions to the layer activity tensors (through low-rank tensor decomposition).

​

The sole training signal is the error between the measured brain activity and the activity predicted by these observation models. We could verify that the model learns several known properties of the visual system.

https://www.biorxiv.org/content/10.1101/553255v2