The social media network X has provoked widespread outrage after users utilized its AI chatbot Grok to manipulate pictures of females by stripping their clothing.
Numerous cases have been seen where the system has been employed to remove clothing from women to show them in swimwear without consent, as well as to put them in sexually explicit scenarios.
Company Reaction and Regulatory Position
XAI, the organization responsible for Grok, did not respond to inquiries, except for an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being introduced to prohibit image stripping tools. Under a new criminal offence, anyone who distributed such technology would "be subject to a jail term and significant fines".
"X or Grok is capable of halting these forms of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said technology companies must "assess the risk" of people seeing illegal content on their platforms, but stopped short of confirming whether it was presently examining X or Grok in relation to AI-generated images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which responds to X users' queries when they mention it in a message.
It is commonly used to provide reactions or additional background to other users' comments, but people on X are also able to alter an posted picture through its AI image editing tool.
Background of Controversy and Policy
The chatbot has been criticised for permitting users to create pictures and videos with nudity and sexualised material, and it was earlier accused of creating graphic content of public figures.
"The platform has been enabling the generation and sharing of these pictures for a long time without taking any measures and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own usage rules forbids "portraying likenesses of persons in a pornographic manner".
In a statement, Ofcom said it was unlawful to "make or circulate deepfakes or illegal content" and clarified this applied to sexual deepfakes generated with AI.
It said services such as X were required to take "necessary steps" to "minimize the danger" of users encountering unlawful content on their platforms and to take down it promptly when they learn of it.
