Elon Musk's online network X has sparked major anger after people used its AI chatbot Grok to manipulate photos of women by stripping their clothing.
Several instances have been seen where the tool has been used to remove clothing from female subjects to make them appear in bikinis without consent, as well as to put them in inappropriate scenarios.
Developer Response and Legal Position
XAI, the company behind Grok, remained silent to requests for comment, other than an auto-reply response stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to ban nudification technology. Under a upcoming law, anyone who distributed such tools would "be subject to a prison sentence and substantial penalties".
"The platform could prevent these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the danger" of people seeing illegal content on their platforms, but did not confirm whether it was at this time looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost chatbot—with some additional features—which answers X users' requests when they mention it in a message.
It is frequently used to offer commentary or more context to other users' remarks, but people on X are also can edit an posted image through its AI image editing feature.
Background of Criticism and Rules
The chatbot has been condemned for allowing users to create images and videos with explicit content and sexually suggestive content, and it was earlier accused of generating sexually explicit material of celebrities.
"X has been permitting the generation and distribution of these images for months without taking any measures and we have yet to see any action by regulators," the expert added.
XAI's own terms of service bans "showing likenesses of individuals in a sexually explicit manner".
In a comment, Ofcom said it was illegal to "make or distribute non-consensual intimate images or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said services such as X were obligated to take "necessary measures" to "reduce the risk" of users encountering illegal material on their platforms and to delete it quickly when they learn of it.
