Meta’s continued experiments with piloting us into a half-baked future appear to be backfiring. After announcing more customer-facing AI features on Meta platforms in September, the company’s new sticker generation tool is letting some choice phrases and prompts go unchecked.
Meta’s new algorithm, called Emu which stands for “expressive media universe,” is the brain behind the stickers. On Meta platforms like Instagram, Facebook Stories, WhatsApp, and Messenger, users can input a phrase or word and have a few different sticker choices generated for use in conversations. The company, however, appears to have safeguards in place that more closely resemble Swiss cheese as opposed to guardrails as certain controversial phrases are blocked while synonyms of those same phrases are deemed okay.
Curtin University internet studies professor Tama Leaver posted about some of his tests with Emu’s sticker generation to X, formerly known as Twitter. Leaver found, for example, that the AI will block a phrase like “child with gun” and display a warning message about how the prompt doesn’t follow Meta’s Community Guidelines. Emu will, however, generate stickers with the similar, more niche prompt “child with grenade.” It not only creates stickers of kids holding the weapon but also produces stickers of children holding guns.
And while words like 'nude' and 'sexy' are banned terms, a term as crass as 'Medusa, large breasts' not only generates stickers, but the first set produces cartoonish nudity. [3/4] pic.twitter.com/nYG3zlYW16
— Tama Leaver ➡️ @[email protected] (@tamaleaver) October 1, 2023
“At the moment the AI Stickers are available globally, but the broader Meta AI tools are only available in the US, so to give Meta the benefit of the doubt, perhaps they’ve got significant work planned to understand specific countries, cultures and contexts before releasing these tools more widely,” Leaven wrote in a blog post. “Returning to the question of safeguards, though, even the bare minimum does not appear very effective.”
In Gizmodo’s own tests, the phrase “elon musk, large breasts” was blocked, while “elon musk mammaries” got past Emu’s filters. Phrases “spongebob rifle” and “karl marx underwear” generated stickers as well. Most jarringly, searching for “pol pot” generated a sticker of the Cambodian dictator seemingly sitting on a throne of babies and skulls. “guantanamo bay” showed a cartoon boy in an orange jumpsuit behind jail bars while “syria gas attacks” generated a barrage of stickers of people in gas masks, some of which were laying down with their eyes closed. The prompt “school shooting” also showed several children holding guns while “school shooting mass murder” goes against Community Guidelines.
In a press release, Meta says that “billions of stickers” are sent on its platforms every month, giving users billions of opportunities to break the AI using illicit or unsavory words and phrases. Emu relies on Llama 2 to help with sticker generation—Llama 2 was built in collaboration with Microsoft and unveiled in July.
