The social media platform X has provoked significant anger after individuals utilized its artificial intelligence assistant Grok to manipulate photos of women by taking off their attire.
Multiple instances have surfaced where the technology has been employed to virtually strip women to render them in bikinis non-consensually, as well as to place them in sexualized situations.
Developer Reaction and Regulatory Position
XAI, the firm behind Grok, did not respond to inquiries, aside from an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being proposed to prohibit nudification tools. Under a proposed law, anyone who supplied such technology would "receive a jail term and substantial penalties".
"X or Grok is capable of halting these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said platforms must "assess the danger" of people seeing unlawful material on their platforms, but did not specify whether it was at this time examining X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some additional features—which reacts to X users' prompts when they reference it in a message.
It is frequently used to provide reactions or further context to other posters' comments, but people on X are also capable of alter an posted photo through its AI image editing function.
History of Criticism and Policy
The chatbot has been faulted for permitting users to create images and videos with nudity and sexually suggestive material, and it was previously accused of making sexually explicit content of celebrities.
"The platform has been permitting the creation and spread of these pictures for a long time without taking any measures and we have not seen any intervention by oversight bodies," the expert added.
XAI's own usage rules forbids "depicting images of persons in a pornographic way".
In a statement, Ofcom said it was against the law to "generate or distribute private pictures or illegal content" and clarified this applied to sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "appropriate steps" to "reduce the risk" of users coming across unlawful material on their services and to take down it swiftly when they are notified of it.
