Grok AI’s sexual deepfakes spark X bans, probes around the world

Social media giant X, formerly Twitter, is facing global condemnation and some crackdowns on its AI chatbot Grok for creating deepfake sexual images of children and women on the platform owned by billionaire Elon Musk.

On Monday, Indonesia and Malaysia became the first countries in the world to block Grok AI, citing concerns over non-consensual deepfake sexual images.

“The government sees nonconsensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space,” Indonesian Communication and Digital Affairs Minister Meutya Hafid said in a statement.

The Malaysian Communications and Multimedia Commission noted “repeated misuse” of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors.

The regulator said notices were issued this month to X Corp. and xAI demanding stronger safeguards.

The European Commission has cracked down, ordering X to retain all documents relating to its AI chatbot — an order that has spurred questions about whether legal action will follow.

“We are very well aware of the fact that X for Grok is now offering a spicy mode showing explicit sexual content with some output generated with childlike images,” EU Commission spokesperson Thomas Regnier said in a statement.

“This is not spicy. This is illegal. This is appalling. This is disgusting. I can confirm we are very seriously looking into this matter.”

Sweden’s deputy prime minister, Ebba Busch, became a victim of the sexual deepfakes when AI-generated images depicting her in a bikini went viral on the platform.

“I was involuntarily undressed by Elon Musk’s Grok on X. And let me be very clear – As a woman, I decide when, where and to whom I show myself in a bikini. I have two children. My daughter is nine. I don’t want this to ever happen to her,” Busch said in a video response she posted on X.

Swedish Prime Minister Ulf Kristersson described the images as “a kind of sexualized violence” and said: “It’s distasteful, unacceptable, offensive.”

On Monday, the United Kingdom’s independent online safety watchdog launched an investigation into X over “sexualized imagery” generated by Grok.

“There have been deeply concerning reports of the Grok AI chatbot account on X being used to create and share undressed images of people – which may amount to intimate image abuse or pornography – and sexualised images of children that may amount to child sexual abuse material (CSAM),” Ofcom said in a statement.

The watchdog sought a response from xAI on the allegations against it and said the company responded before the deadline on Friday.

If the investigation finds that X has broken the law, it can be fined up to 18 million pounds or 10 per cent of its qualifying worldwide revenue, whichever is greater.

British Prime Minister Keir Starmer said in a radio interview that X needs to “get their act together and get this material down. We will take action on this because it’s simply not tolerable.”

India ordered X to take action within 72 hours to fix the AI chatbot or face action.

India’s Ministry of Electronics and Information Technology issued a formal notice to X, accusing it of “gross misuse” of its AI and allowing “obscene images or videos of women in derogatory or vulgar manner in order to indecently denigrate them.”

A failure to comply could result in X facing action under India’s cyber, criminal and child protection laws.

In France, prosecutors widened an ongoing investigation of X, relating to Grok allegedly promoting Holocaust denial claims — a crime in France — to include the sexually explicit deepfakes.

The government also flagged problems with the country’s communications regulator over possible breaches of the EU’s Digital Services Act.

“The internet is neither a lawless zone nor a zone of impunity: sexual offenses committed online constitute criminal offenses in their own right and fall fully under the law, just as those committed offline,” the French government said.

In Brazil, a lawmaker reported Grok and X to federal prosecutors, urging the government to disable Grok and all AI functions of the X network.

Trending Now

Canada’s Artificial Intelligence Minister Evan Solomon expressed concern over deepfake sexual abuse when asked by Global News why the federal government and ministers are still using X.

“We must protect Canadians, especially women and young people, from exploitation. Platforms and AI developers have a duty to prevent this harm,” Solomon said in a statement Thursday posted to X.

The statement did not mention X, Grok or Elon Musk specifically.

In a separate statement, he said Canada is not considering banning X.

“Contrary to media reports, Canada is not considering a ban of X,” Solomon added on Sunday.

There are “active discussions” across the Canadian government on how to respond to the continued generation of sexual abuse material on X, Global News has learned.

Musk’s artificial intelligence company, xAI, responded to a request for comment from The Associated Press with the automated response, “Legacy Media Lies.”

However, X did not deny that the deepfake sexual content generated through Grok exists.

Yet it still claimed in a post on its Safety account that it takes action against illegal content, including child sexual abuse material, “by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary.”

The platform also repeated a comment from Musk, who said, “Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content.”

— with files from The Associated Press and Reuters