California AG Orders Musk’s xAI to Stop Generating Sexual Deepfake Images

California Attorney General Rob Bonta has sent a cease-and-desist letter to xAI, demanding the immediate halt of the creation and distribution of non-consensual sexual images generated by its AI chatbot Grok.

“I fully expect xAI to immediately comply,” Bonta said in a statement on Friday.

The action follows a growing global backlash against Grok, which has allowed users to create and share sexualized images of women and minors. Authorities in multiple countries have moved to investigate or restrict the tool over concerns about illegal and harmful content.

Bonta’s office said it opened a formal investigation on Wednesday into the creation and spread of non-consensual, sexually explicit material produced using Grok. The probe adds regulatory pressure on xAI, which is owned by billionaire entrepreneur Elon Musk.

xAI said late on Wednesday that it had introduced new restrictions limiting image-editing capabilities for all Grok users, though regulators say concerns remain. The company did not respond to a Reuters request for comment on the cease-and-desist letter.

International scrutiny has intensified in parallel. Authorities in Japan, Canada and Britain have opened probes into Grok, while Malaysia and Indonesia have temporarily blocked access to the chatbot over the generation of explicit images.

California’s move underscores a broader shift by regulators toward holding AI developers accountable for how generative tools are used—and misused—particularly when it comes to non-consensual and sexualized content. The case could set an important precedent for how aggressively governments intervene as generative AI systems become more powerful and widely deployed.