The circulation of sexually explicit AI-generated images featuring Taylor Swift on X (formerly Twitter) has sparked concern over the proliferation of fake pornography created using AI technology.
One highly visible instance on X amassed over 45 million views, 24,000 reposts, and hundreds of thousands of likes and bookmarks before the responsible account was suspended. Despite efforts to remove the images, they continued to spread across the platform and were reposted by other users.
These images are believed to have originated in a Telegram group where users share explicit AI-generated content, with some users even joking about the images going viral. Following the incident, Microsoft, whose software was reportedly used to create the images, announced enhancements to its safety systems to prevent misuse.
Although X’s policies explicitly prohibit the hosting of synthetic and manipulated media, as well as nonconsensual nudity, many of these posts remained live for an extended period, prompting criticism from Swift’s fan base. In response, fans flooded hashtags associated with the images with messages promoting genuine content of Swift performing.
Posting Non-Consensual Nudity (NCN) images is strictly prohibited on X and we have a zero-tolerance policy towards such content. Our teams are actively removing all identified images and taking appropriate actions against the accounts responsible for posting them. We’re closely…
— Safety (@Safety) January 26, 2024
This incident highlights the ongoing challenge of combating deepfake porn and AI-generated fake images of real people. While some AI image generators have safeguards in place to prevent the creation of explicit content featuring celebrities, others do not, placing the burden on social platforms to mitigate the spread of harmful material. However, platforms like X face difficulties in effectively moderating content, particularly given their limited resources for moderation.
The incident has also drawn attention to broader concerns about X’s content moderation practices, especially as the platform faces scrutiny from the EU for its handling of illegal content and disinformation.
