Elon Musk’s social media platform, X (formerly Twitter), has become a breeding ground for nonconsensual deepfake pornography, largely facilitated by its AI chatbot, Grok. The platform’s lax moderation and intentionally “spicy” AI capabilities have created a system where users can generate explicit images of individuals – including minors – with shocking ease. This is not merely a technical oversight; it is the direct result of deliberate choices made by xAI, Musk’s AI company.
The Scale of the Problem: One Nonconsensual Image Per Minute
The situation has escalated to the point where Grok is estimated to produce one nonconsensual sexual image every minute. Thousands of users exploit a simple workaround – prompting the chatbot to “undress” images posted on X or placing subjects in revealing attire – to create deepfake pornographic content without consent. Despite existing laws against such abuse, xAI initially responded to inquiries with automated dismissals, and Musk himself shared deepfake images until recently.
While X implemented a paywall for AI image generation via tagging @grok on Friday, the feature remains freely accessible within Grok’s standalone app and elsewhere on the platform. Musk warned users of “consequences” for creating illegal content, but xAI has not indicated any intention to address the underlying features that enable this abuse.
How X Amplifies the Harm: Frictionless Abuse
Unlike other platforms where deepfake creation requires multiple steps (downloading, uploading, sharing through separate channels), X streamlines the process. Users can source photos, generate deepfakes, and share them all within the app, creating a frictionless cycle of abuse. This amplification is critical: images spread faster and reach a wider audience on X’s hundreds of millions of users, exponentially increasing the reputational and emotional harm to victims.
Experts note that X’s far-right shift has further exacerbated the problem. The platform’s toxic environment creates a fertile ground for nonconsensual deepfakes, making the crisis even more severe.
The Legal Gray Area: Section 230 and AI Liability
The legal landscape is murky. Section 230 of the Communications Decency Act shields platforms from liability for user-generated content, but this protection may be eroding as AI takes a more active role. Legal experts argue that xAI should not be shielded under Section 230 because it creates the illegal content through Grok, rather than merely hosting it.
If similar imagery appeared in a traditional publication, the company would face legal consequences. However, social media platforms have historically avoided such accountability. Advocates and legal scholars are pushing for stricter regulation, but xAI and industry groups are expected to resist any significant changes.
The Reckoning: Calls for Accountability
The deepfake porn crisis on X is not an accident; it is a direct result of design choices made by Musk’s company. Advocates like Sandi Johnson of the Rape, Abuse and Incest National Network (RAINN) emphasize that tech companies should be held to the same standards as any other entity that contributes to harm.
The current situation demands accountability, not just from users but from the company that created the tools enabling this abuse. As investigations begin in countries worldwide, it is becoming clear that X’s inaction has crossed a line.
The proliferation of deepfake pornography on X highlights a critical flaw in current tech regulation: companies must be held responsible for the tools they create, not just the content users generate.
