Unraveling neural coding in the visual system by combining human and non-human primate Neurophysiology and Deep Convolutional Neural Networks.

The complexity of our visual environment is represented in the brain through hierarchically organized patterns of activity across the ventral visual cortex. Studies focusing on how exactly information is represented by these neurons and on the specific features being encoded, have been constrained by the choice of stimuli to probe neural activity. Selection of images to discover neuronal feature preferences has traditionally been guided by a combination of natural stimulus statistics or intuitions about the underlying representations. Despite enormous and satisfactory progress, whether specific image features can be simply described just using conventional stimulus categories is still unclear. What is the structure of the visual world that neurons really care about? Recent work by Kreiman and colleagues used a generative network and a genetic algorithm to let neurons dictate their feature preferences and thus generate their own preferred stimuli guided by neural activity. Here I aim to expand this approach and combine it with neurophysiological recordings along the ventral visual cortex in humans and monkeys, together with stateof-the-art deep convolutional neural networks. Thus, we will be in a unique position to investigate and compare coding properties of neurons across species and computational models, and to assess how these feature preferences evolve along the cortical visual hierarchy.