The social media network X has caused widespread outrage after people employed its AI chatbot Grok to change pictures of females by removing their clothing.
Several examples have surfaced where the system has been employed to virtually strip women to make them appear in swimwear non-consensually, as well as to put them in sexually explicit contexts.
Developer Reaction and Legal Position
XAI, the organization responsible for Grok, offered no comment to inquiries, except for an automatically-generated response stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to prohibit nudification software. Under a upcoming law, anyone who distributed such technology would "be subject to a jail term and substantial fines".
"X or Grok could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the danger" of people accessing illegal content on their services, but did not specify whether it was presently investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some additional functions—which reacts to X users' queries when they tag it in a tweet.
It is frequently used to provide commentary or additional context to other users' remarks, but people on X are also capable of alter an uploaded picture through its AI image editing function.
Background of Controversy and Policy
The chatbot has been condemned for permitting users to generate images and footage with nudity and sexualised content, and it was earlier alleged to have been involved in generating graphic material of celebrities.
"X has been enabling the generation and sharing of these images for a long time without taking any steps and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of individuals in a graphic way".
In a statement, Ofcom said it was unlawful to "make or circulate private pictures or child sexual abuse material" and clarified this applied to sexual deepfakes generated with AI.
It said networks such as X were mandated to take "suitable measures" to "minimize the risk" of users encountering unlawful content on their platforms and to remove it swiftly when they become aware of it.
