The online platform X has caused widespread outrage after individuals employed its artificial intelligence assistant Grok to alter photos of females by removing their garments.
Several examples have been seen where the tool has been employed to remove clothing from women to make them appear in swimwear without permission, as well as to insert them in sexually explicit situations.
Developer Reaction and Legal Stance
XAI, the company behind Grok, offered no comment to questions, other than an auto-reply response stating "traditional news outlets lies".
A government official stated that legislation is being brought forward to ban image stripping technology. Under a new criminal offence, anyone who provided such technology would "receive a jail term and significant penalties".
"X or Grok has the ability to stop these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said platforms must "evaluate the danger" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some paid-for functions—which answers X users' prompts when they reference it in a message.
It is commonly used to provide reactions or more context to other posters' comments, but people on X are also able to edit an posted image through its artificial intelligence picture modification function.
Background of Criticism and Policy
The system has been criticised for enabling users to produce images and videos with explicit content and sexualised material, and it was earlier alleged to have been involved in generating sexually explicit content of public figures.
"The platform has been enabling the generation and spread of these images for a long time without taking any measures and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "generate or distribute private pictures or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said networks such as X were mandated to take "suitable measures" to "lessen the risk" of users encountering illegal material on their services and to take down it swiftly when they become aware of it.
