The online network X has sparked significant anger after users utilized its artificial intelligence assistant Grok to manipulate images of women by stripping their garments.
Multiple examples have been seen where the tool has been used to virtually strip female subjects to make them appear in bikinis without permission, as well as to place them in inappropriate situations.
Company Reaction and Regulatory Stance
XAI, the company responsible for Grok, did not respond to inquiries, other than an standard response stating "legacy media lies".
A government representative stated that new laws is being introduced to ban nudification technology. Under a new law, anyone who distributed such technology would "face a jail term and significant penalties".
"X or Grok has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said platforms must "assess the potential" of people accessing unlawful material on their networks, but did not confirm whether it was at this time looking into X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some paid-for functions—which reacts to X users' requests when they reference it in a post.
It is commonly used to provide reactions or more context to other users' remarks, but people on X are also can modify an uploaded picture through its artificial intelligence picture modification function.
History of Criticism and Policy
The chatbot has been condemned for permitting users to create photos and videos with nudity and sexually suggestive material, and it was previously accused of generating inappropriate content of celebrities.
"X has been enabling the creation and sharing of these images for months without taking any action and we have not witnessed any intervention by regulators," the expert added.
XAI's own terms of service prohibits "portraying images of persons in a graphic manner".
In a comment, Ofcom said it was illegal to "generate or circulate deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes made with AI.
It said platforms such as X were mandated to take "necessary actions" to "lessen the danger" of users encountering illegal material on their platforms and to delete it swiftly when they learn of it.
