The online platform X has provoked widespread outrage after people utilized its AI chatbot Grok to change images of women by removing their attire.
Several examples have been seen where the tool has been employed to digitally undress female subjects to make them appear in swimwear non-consensually, as well as to insert them in inappropriate situations.
Developer Reaction and Regulatory Stance
XAI, the organization that created Grok, did not respond to questions, other than an auto-reply message stating "legacy media lies".
A government official stated that legislation is being proposed to prohibit nudification tools. Under a proposed law, anyone who supplied such tools would "face a prison sentence and heavy fines".
"X or Grok could prevent these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said tech firms must "assess the danger" of people seeing unlawful material on their services, but did not confirm whether it was presently looking into X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some additional functions—which responds to X users' prompts when they tag it in a message.
It is often used to provide reactions or more background to other posters' remarks, but people on X are also can edit an posted photo through its AI image editing feature.
Background of Controversy and Policy
The system has been criticised for permitting users to produce images and clips with nudity and sexually suggestive material, and it was in the past accused of creating sexually explicit material of celebrities.
"X has been enabling the creation and spread of these pictures for months without taking any action and we have not witnessed any challenge by regulators," the expert added.
XAI's own terms of service forbids "showing images of persons in a sexually explicit way".
In a comment, Ofcom said it was illegal to "make or circulate non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images made with AI.
It said platforms such as X were required to take "suitable measures" to "lessen the danger" of users viewing illegal material on their services and to delete it quickly when they are notified of it.
