The social media platform X has caused significant outrage after users utilized its artificial intelligence assistant Grok to alter images of females by removing their garments.
Multiple instances have surfaced where the system has been employed to digitally undress women to make them appear in bikinis non-consensually, as well as to place them in sexualized scenarios.
Company Reaction and Legal Position
XAI, the firm behind Grok, did not respond to inquiries, aside from an auto-reply response stating "legacy media lies".
A government official stated that new laws is being proposed to prohibit image stripping software. Under a upcoming criminal offence, anyone who provided such technology would "be subject to a prison sentence and significant penalties".
"The platform is capable of halting these forms of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said tech firms must "evaluate the potential" of people viewing illegal content on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some premium features—which responds to X users' requests when they mention it in a tweet.
It is commonly used to offer commentary or more background to other users' comments, but people on X are also can edit an uploaded image through its artificial intelligence picture modification function.
Background of Controversy and Policy
The tool has been condemned for permitting users to produce images and clips with nudity and sexually suggestive material, and it was earlier alleged to have been involved in making sexually explicit material of celebrities.
"The platform has been permitting the production and spread of these pictures for a long time without taking any action and we have not seen any action by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting images of persons in a pornographic way".
In a statement, Ofcom said it was against the law to "generate or distribute deepfakes or child sexual abuse material" and clarified this applied to explicit AI images made with AI.
It said networks such as X were obligated to take "appropriate steps" to "minimize the danger" of users coming across unlawful material on their platforms and to take down it swiftly when they learn of it.
