The online network X has sparked major backlash after individuals utilized its AI chatbot Grok to alter photos of females by stripping their clothing.
Numerous cases have emerged where the system has been used to remove clothing from female subjects to show them in bikinis non-consensually, as well as to insert them in sexualized scenarios.
Company Response and Legal Position
XAI, the company that created Grok, did not respond to questions, aside from an auto-reply message stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to outlaw image stripping technology. Under a upcoming law, anyone who distributed such tools would "face a jail term and heavy penalties".
"The platform has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said platforms must "evaluate the potential" of people accessing unlawful material on their services, but stopped short of confirming whether it was at this time examining X or Grok in relation to AI-generated images.
Grok is a free artificial intelligence—with some paid-for features—which responds to X users' requests when they reference it in a tweet.
It is frequently used to provide reactions or more context to other users' comments, but people on X are also capable of alter an uploaded photo through its AI image editing tool.
History of Criticism and Rules
The system has been condemned for permitting users to create photos and clips with explicit content and sexually suggestive material, and it was previously alleged to have been involved in generating inappropriate content of public figures.
"The platform has been allowing the creation and spread of these images for months without taking any action and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "showing images of individuals in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "create or distribute non-consensual intimate images or child sexual abuse material" and stated this applied to explicit AI images created with AI.
It said platforms such as X were mandated to take "necessary actions" to "reduce the danger" of users viewing illegal material on their services and to remove it quickly when they learn of it.
