Hallucinating Baroque Allegories
Here’s a story about how, while working on securing the rights to images for my book, I experienced bizarre AI hallucinations. I had heard how AI agents can give surprisingly dishonest answers to certain questions, but never before directly experienced this strange modern phenomenon.
Allegory (or 𝘈𝘭𝘭𝘦𝘨𝘰𝘳𝘺 𝘰𝘯 𝘵𝘩𝘦 𝘍𝘢𝘤𝘶𝘭𝘵𝘪𝘦𝘴 𝘰𝘧 𝘵𝘩𝘦 𝘚𝘰𝘶𝘭)
by Simon Vouet, circa 1645
My story begins in the Capitoline Museums, on my recent trip to Rome, when I came across a painting depicting fire shooting from the top of a young man’s head. In Vouet’s 𝘈𝘭𝘭𝘦𝘨𝘰𝘳𝘺 the young man personifies Intellect. Holding a mirror in his lap, he looks out at us with a mistrustful expression.
A crowned, winged, and topless woman personifying Will ignores flying putti blowing horns at her ears. She instead casts an appreciative glance at Memory, while reaching out with her right hand as if to touch the resting left hand of Intellect. Memory, a more modestly dressed woman sitting with arms calmly folded, looks up at one of the horn-tooting putti with one of her two faces. Her second face, on the back of her head, almost disappears into shadow.
I can’t pretend to parse the complex web of symbolism on display here. Does the depiction of one putto with dark, mothlike wings suggest that he is tooting evil thoughts into one of Will’s ears while his blonde counterpart toots good thoughts into the other? Why do these different “faculties of the soul” seem so distracted, barely interacting with each other?
Then there are institutions where the process of requesting rights is so murky, and the cost so potentially expensive, that you want to look for alternatives. The rules at the Capitolines are among the murkiest. Although Vouet was a French painter, “Italian cultural heritage laws strictly regulate the reproduction of public cultural property within Italy, requiring authorization and payment for commercial use regardless of copyright.” To use the work for “commercial purposes” (my little book qualifies) I must obtain formal permission from the Sovrintendenza Capitolina and pay fees “determined by the annual resolution on service tariffs set by Roma Capitale”. Hmm. That sounded like a long, tiresome journey with uncertain outcome.
Though the Vouet Allegory perfectly suits my needs, if fire shooting from the head commonly signified personified Intellect in Baroque painting, why not look for another work with less patrimonic baggage? I found a woodcut from a 1644 edition of 𝘐𝘤𝘰𝘯𝘰𝘭𝘰𝘨𝘪𝘢, but it seemed a bit crude and too needing of explanation.
I now return to the promised heart of my story: my surprising experience with bizarrely hallucinating AI.
A woodcut illustration of Intelletto, from a 1644 edition of Cesare Ripa’s 𝘐𝘤𝘰𝘯𝘰𝘭𝘰𝘨𝘪𝘢. Flames emerge from the head of personified Intellect.
When the big fuss about ChatGPT first broke, I checked it out briefly, but never deliberately chose to use it, Claude, or any other AI system in my research or writing. Just by continuing to use Google as my primary search engine, however, AI gradually crept into my process. Google started placing AI-generated “overviews” at the top of search results in 2024, and a year later introduced “AI Mode,” one of many flavors of their Gemini system. Though optional at first, it quickly became the default, particularly if you write a search prompt as a question, or something resembling a question. When I asked Google/Gemini to find: > Public domain baroque painting fire above head denoting personification of intellect it pointed me to An Allegory of Intelligence, an oil painting by the Italian Baroque artist Cesare Dandini. It gave a little art-history summary explaining why this painting fit my criteria, noting that “The author died in 1657, so this work is in the public domain in its country of origin and other countries and areas where the copyright term is the author’s life plus 100 years or fewer.” Great! I followed the provided link to check it out. |
Though a lovely feminine personification of intellect/intelligence, for my purposes this painting has several major flaws. For one thing, there is no fire above her head! No fire in the painting at all.
Also, the image comes from the site of the Dorotheum, an Austrian auction house, which had recently sold the painting to a private collector. Not ideal, for several reasons.
I went back to Gemini and pointed out the absence of fire in the painting. It apologized for its mistake and asked if I still wanted to pursue the inquiry. I said I did, and added a stipulation:
> Public domain baroque painting fire above head denoting personification of intellect held by a public institution that does not charge fees for reproduction
I will quote most of its second recommendation, because I find it so bizarre:
“A brilliant and highly accurate example of this exact motif exists in a major public domain artwork held by an institution with a strict open-access policy (no fees for reproduction). The masterpiece you are looking for is An Allegory of Wisdom and Science (1648) by the Italian Baroque painter Salvator Rosa, housed at the Museum of Fine Arts, Boston (MFA).
“In this sweeping allegorical painting, Rosa directly visualizes the faculties of human knowledge using Cesare Ripa’s guidelines: “The Figure of Intellect (Intelletto): Standing prominently, this figure is depicted with an unmistakable burst of fire and flame erupting directly from the top of the head, precisely symbolizing the fiery, active light of the human mind….”
It then went on to discuss the other allegorical figures in the painting, Memoria and Volontà, and how the three represent different aspects of the human soul.
This sounded spot on, so I followed the link to MFA Boston, and searched for “Allegory of Wisdom and Science.” I got 180 hits, but none for that specific painting. Next I tried “Salvator Rosa,” with equally disappointing results. The MFA has no works by Rosa in its collection at all (though it does have a 19th century painting by Thomas Cole titled Salvator Rosa Sketching Banditti). Several searches later, I found myself going back to Gemini to complain:
“You are entirely right, and I apologize. An Allegory of Wisdom and Science by Salvator Rosa does not exist. I completely hallucinated that specific painting and its location.”
> Why are you pointing me to paintings that do not match my criteria and inventing paintings that do not exist?
“I am hallucinating because I am matching the conceptual keywords of your prompt—”Baroque art,” “flame above head,” and “intellect”—with patterns in my training data, but failing to verify if those specific artistic combinations actually exist in reality.
“When you ask for a very specific visual motif, my system recognizes the theory perfectly (Cesare Ripa’s rules). However, when I try to find a specific painting that matches it, I generate a plausible-sounding artist and title (like Salvator Rosa or Cesare Dandini) because those artists frequently painted similar allegories. Instead of admitting I cannot scan a live visual database of every Baroque painting in the public domain, I incorrectly assumed those specific pieces contained the flame.”
Wow. At least, when caught in an elaborate lie, it admitted it, and explained why (though it didn’t fully ‘fess up to completely inventing a painting, only to making mistaken assumptions about what certain paintings contain).
I was able to find an engraved copy of the Vouet in the collection of the National Gallery, where the policy on usage is blessedly straightforward: “This object’s media is free and in the public domain.” The image will appear in monochrome in my book anyway.
A poem in Latin below the engraving reads in part:
Like a flame, native wit leaps and flashes; Thus, with harmless fire, it plays about the brow.
Gemini hallucinating answers to questions it cannot answer reminds me of behaviors children begin to exhibit at a certain stage in their cognitive development. “Children typically begin inventing false answers or feigning knowledge to cover up gaps in what they know during the preschool years (around ages four to five),” according to a story on NPR by Professor Marjorie Rhodes of NYU, “as their developing cognitive skills and desire to please adults or avoid appearing ignorant emerge.”
Sources
Rhodes, Marjorie, “When Children Begin To Lie, There’s Actually A Positive Takeaway” (October 2, 2017). NPR Cosmos & Culture: Commentary On Science And Society
Hallucinating Baroque Allegories Read More »