< Back to previous page
Representations of regular and irregular shapes by deep Convolutional Neural Networks, monkey inferotemporal neurons and human judgments
Journal Contribution - Journal Article
Recent studies suggest that deep Convolutional Neural Network (CNN) models show higherrepresentational similarity, compared to any other existing object recognition models, withmacaque inferior temporal (IT) cortical responses, human ventral stream fMRI activationsand human object recognition. These studies employed natural images of objects. A longresearch tradition employed abstract shapes to probe the selectivity of IT neurons. If CNNmodels provide a realistic model of IT responses, then they should capture the IT selectivityfor such shapes. Here, we compare the activations of CNN units to a stimulus set of 2D regularand irregular shapes with the response selectivity of macaque IT neurons and withhuman similarity judgements. The shape set consisted of regular shapes that differed innonaccidental properties, and irregular, asymmetrical shapes with curved or straight boundaries.We found that deep CNNs (Alexnet, VGG-16 and VGG-19) that were trained to classifynatural images show response modulations to these shapes that were similar to thoseof IT neurons. Untrained CNNs with the same architecture than trained CNNs, but with randomweights, demonstrated a poorer similarity than CNNs trained in classification. The differencebetween the trained and untrained CNNs emerged at the deep convolutional layers,where the similarity between the shape-related response modulations of IT neurons and thetrained CNNs was high. Unlike IT neurons, human similarity judgements of the same shapescorrelated best with the last layers of the trained CNNs. In particular, these deepest layersshowed an enhanced sensitivity for straight versus curved irregular shapes, similar to thatshown in human shape judgments. In conclusion, the representations of abstract shape similarityare highly comparable between macaque IT neurons and deep convolutional layers ofCNNs that were trained to classify natural images, while human shape similarity judgmentscorrelate better with the deepest layers.
Journal: PLoS Computational Biology
Number of pages: 26