Toward a better understanding of image memorability through behavioral and computational approaches: A memorable inquiry

How many images have you encountered today (e.g., on your smartphone, on a billboard, in a magazine)? Probably a lot. Which ones would you still recognize if shown to you again now? This depends on numerous factors, including the image itself. Indeed, images consistently differ in their memorability. In this doctoral project, we sought to contribute, sometimes indirectly, to our understanding of what makes an image memorable. We thereby focused on an image's visual attributes, rather than only on the content that it depicts. In a first study (see Chapter 2), we were able to show that an image's position in a memorability ranking does not depend on whether its memorability is assessed through a repeat-detection memory game or a more traditional long-term recognition memory task. Neither does it depend on how long the images involved in the task have to be held in memory. These findings offer further support for the concept of image memorability as an image property. Adding further to that support as well as to the ecological validity of image memorability, were the results presented in Chapter 3. They demonstrated that there is still a large degree of agreement among individuals with respect to which images they remember and which ones they forget even when they are not anticipating a memory test. More specifically, our participants were shown a series of images under the guise of a study about eye movements and were later surprised with a recognition memory test. In addition, we found that the resulting image memorability ranking correlates well to one that is obtained by having participants intentionally study the images. In Chapter 4, we hypothesized that images with a good organization (i.e., the pictorial elements form a coherent whole) would tend to be more memorable. Because this goodness is hard to quantify directly, we focused on two of its characteristics: fast, efficient processing and robustness against a shrinking transformation. The former was operationalized through a rapid-scene categorization task in which participants were very briefly presented with an image, followed by a mask and then a label, and had to indicate whether the label matched the image. The latter was operationalized through a thumbnail search task, in which participants had to find a thumbnail version of a regular-sized image among eight distractor thumbnails. We found that, after controlling for distinctiveness, memorable images were significantly more likely to be categorized accurately and were located significantly faster in the thumbnail search task. These results constitute a first indication that perceptual goodness contributes to image memorability. Chapter 5 presents MemCat, a large new dataset to help researchers study what makes an image memorable beyond its semantic category. To build MemCat, we carefully selected 10,000 images belonging to five broader memorability-relevant categories (animal, food, landscape, sports, and vehicle), further divided into subcategories. Using a repeat-detection memory game, we quantified each image on memorability based on the responses of, on average, 99 participants. As expected, we found that images still show consistent differences in memorability, even within semantic categories. In Chapter 6, finally, we developed GANalyze: a new framework, based on generative adversarial networks (GANs), to help understand cognitive image properties such as memorability. GANs allow us to generate a manifold of natural-looking images with fine-grained differences in their visual attributes. By navigating this manifold in directions that increase memorability, we can visualize what it looks like for a particular generated image to become more or less memorable. The resulting "visual definitions" surfaced image features (like "object size") that may underlie memorability. Finally, we also demonstrated that GANalyze can be used to manipulate such a candidate feature in generated images and test its effect on memorability. Together, the findings presented throughout this dissertation constitute an important step toward a better understanding of image memorability.

Jaar van publicatie:2019

Toegankelijkheid:Open