The online network X has caused significant anger after people employed its AI chatbot Grok to manipulate photos of females by stripping their attire.
Several cases have emerged where the technology has been employed to virtually strip female subjects to render them in bikinis without consent, as well as to insert them in inappropriate scenarios.
Company Reaction and Regulatory Stance
XAI, the organization that created Grok, remained silent to questions, except for an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to prohibit nudification software. Under a new criminal offence, anyone who provided such tools would "receive a jail term and significant penalties".
"X or Grok is capable of halting these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the risk" of people seeing illegal content on their platforms, but stopped short of confirming whether it was presently examining X or Grok in relation to AI-generated images.
Grok is a free artificial intelligence—with some additional features—which answers X users' queries when they reference it in a message.
It is often used to offer commentary or more context to other users' remarks, but people on X are also able to edit an posted photo through its AI image editing feature.
Background of Criticism and Rules
The tool has been condemned for enabling users to produce photos and clips with explicit content and sexually suggestive content, and it was earlier accused of making inappropriate material of public figures.
"The platform has been enabling the production and spread of these images for months without taking any steps and we have not seen any action by regulators," the expert added.
XAI's own usage rules forbids "showing images of persons in a graphic way".
In a comment, Ofcom said it was unlawful to "create or share private pictures or child sexual abuse material" and stated this covered sexual deepfakes generated with AI.
It said platforms such as X were required to take "appropriate actions" to "lessen the risk" of users coming across illegal material on their platforms and to take down it swiftly when they become aware of it.
