The social media network X has caused widespread outrage after users employed its artificial intelligence assistant Grok to change photos of females by taking off their clothing.
Numerous instances have surfaced where the tool has been employed to digitally undress women to render them in swimwear without permission, as well as to place them in inappropriate scenarios.
Company Reaction and Legal Position
XAI, the organization behind Grok, offered no comment to inquiries, except for an auto-reply message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being introduced to prohibit image stripping tools. Under a new law, anyone who distributed such tools would "face a jail term and substantial penalties".
"The platform is capable of halting these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said platforms must "consider the risk" of people seeing unlawful material on their platforms, but did not confirm whether it was currently looking into X or Grok in relation to AI-generated pictures.
Grok is a free chatbot—with some premium functions—which reacts to X users' queries when they tag it in a message.
It is frequently used to offer reactions or more context to other users' remarks, but people on X are also able to modify an uploaded photo through its AI image editing feature.
Background of Criticism and Rules
The system has been faulted for permitting users to generate pictures and clips with nudity and sexually suggestive content, and it was previously accused of creating sexually explicit content of public figures.
"X has been enabling the creation and spread of these pictures for a long time without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "showing likenesses of individuals in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "make or distribute private pictures or child sexual abuse material" and clarified this included sexual deepfakes generated with AI.
It said services such as X were required to take "necessary actions" to "lessen the risk" of users encountering unlawful material on their services and to delete it quickly when they learn of it.
