MOON RABBIT – A case of AI pareidolia?

The night sky has always been a mystery, since the beginning of time – or at least, since our species has developed the intellect to wonder. (Rector, 2015)

Vision is after all, the richest of the senses and we are used to interpret the endless information of the world around us through our eyes. Since ancient times man has appreciated the motions of planets and the changing of seasons and has also identified constellations.

Constellations have been identified in the sky, with Europeans generally agreeing on images of animals, hunters, and other mythical creatures, with the most popular having entered culture as astrological signs as part of Zodiacal systems. Such astrological signs were purported to dictate the nature and character of a person according to the placement of the stars and the planets at the time of their birth. Other cultures, such as the Incas and Mayans, used the concept of negative shapes and shadows in the Milky Way to define further objects, such as Llamas, using the shape of interstellar gas together with the stars to give rise to wildly different constellations from the classical ones.

People tend to see patterns in everyday phenomena other than constellations in the sky. The most common are faces and shapes in the clouds, in landscapes such as hills, plant and tree forms (and even in food!) Well known optical illusions take advantage of these tendencies in the human visual system.

On May 7, 2015, NASA’s Mars Curiosity rover took a snapshot of an atypical stone on the Red Planet. A raw image from the mission shows a rock that appears to be pyramid-shaped, leading some to speculate that it may be the result of intelligent sculptors. NASA, for its part, says it’s just an ordinary rock. (Mars pyramid: Alien structure or everyday pareidolia? – ProQuest, no date)

So one is prompted to ask: but was it a real pyramid or just another case of pareidolia?

Pareidolia is a phenomenon that has evolved in humans as a survival tool that allows us to recognize shapes and faces from a distance or in the dark, enabling one to identify a friend or foe. For example, being able to see a predator crouching in the savannah before it sees you gives an unquestionable evolutionary advantage. But as in many cases of perceptual specialisation, there is a downside to fast reaction time, leading us to misinterpret patterns.

Pareidolia also brings to light the power of expectations in us humans. “Being able to see Jesus’s face in toast is telling you more about what’s happening with your expectations, and how you’re interpreting the world based on your expectations, rather than anything that’s necessarily in the toast,” according to Dr. Sophie Scott, professor of neuroscience at the University College London, in a BBC interview.

Pareidolia is also actively encouraged in art education as a way of generating novel images. In AI, generative processes such as Deep Dream, Generative Adversarial Networks (GANs) are similar to the AI version of pareidolia, where the system is forced on purpose to generate images based on initially random patterns that are then subsequently refined into visually appealing images. The way that this is done is via so-called latent spaces, spaces that constrict and bottleneck visual information on purpose to distil its most pertinent features – the AI equivalent of a human squinting at an image on purpose just to see its main outlines and shapes and then continuing by imagining or hallucinating similar shapes. There is also a beneficial related aspect to this by using a technique called image perturbation, where training images used to train AI are deliberately modified with random noise and in many cases rotated or distorted purposefully, to enable AI to recognise shapes that are not exactly the same as the images they were trained with. This is something that human beings and animals deal with naturally, even at a very young age, but that current AI struggles with. This struggle is taken advantage of by Adversarial Image attacks, that often modify an image imperceptibly or by adding labels or presenting unusual combinations that are immediately processed correctly by humans but that completely mislead AI.

In the research and development of the Project ‘Moon Rabbit’, artists Sarah Pektus and Mark J Kosch teach an AI to recognize familiar shapes and objects in images of star clusters. In their sketches, they present how the AI has identified rabbits, crabs and even lamps in star clusters. In other words, they are attempting to teach pareidolia to an AI. The exhibit is visually appealing with three large panels projecting the universe. In the room there are also three beautiful lamps. Each time the AI recognizes a shape in the stars, a highlighted border appears around the clusters with the identified shape label, and one of the lamp lights up. The artwork was created as part of the AI lab residency issued under the AI and astronomy theme by the Leiden Observatory.

Pektus and Kosch explore how AI can be taught to find known patterns in the night sky. Their AI was taught over a period of research and development throughout the spring and summer of 2021. Custom datasets have been used to teach several AI programs to recognise familiar shapes within astronomical images. The artists hope that through this process, they will understand how the individual AI programs learn to develop their own personality via the data that has been used for training, the training process itself and the way that the AI reacts to new images.

Quotations

Mars pyramid: Alien structure or everyday pareidolia? – ProQuest (no date). Available at: https://www.proquest.com/docview/1691127363?pq-origsite=primo (Accessed: 22 September 2021).

Rector, T.A. (2015) Coloring the universe: an insider’s look at making spectacular images of space. Fairbanks, Alaska: University of Alaska Press.