A Dutch court has ordered Elon Musk’s xAI to stop its Grok chatbot from generating non-consensual AI-generated images of adults and children, marking the first legal action of its kind in Europe. The ruling, issued by Amsterdam’s District Court, prohibits xAI from creating or distributing sexual imagery where individuals are depicted naked without their consent. The ban also extends to child sexual abuse material and prevents X (formerly Twitter) from offering Grok as a function in the region. xAI faces a penalty of 100,000 euros ($115,000) per day of non-compliance, with a maximum fine of 10 million euros. The case was brought by Dutch non-profit group Offlimits, which fights online sexual abuse, particularly against children and young people. xAI was also ordered to pay Offlimits legal costs of 2.2 million euros within 14 days.
Background and Context
The ruling follows reports of users exploiting Grok to generate sexualized images of celebrities and public figures. In January, the UK government announced plans to ban nudification tools through amendments to the Crime and Policing Bill, though the proposed ban initially only applied to products in the UK. The Center for Countering Digital Hate (CCDH) estimated that Grok generated around three million sexualized images between December 29, 2025, and January 9, 2026, with over 23,000 appearing to depict children. xAI moved to block Grok from creating sexualized images of real people on X in January, with the restriction applying to all users, including paid subscribers.
Policy and Legal Implications
The Dutch court’s decision sets a precedent for regulating AI-generated content in Europe. Robbert Hoving, managing director at Offlimits, stated that the ruling draws a clear line against violating human rights online. The case coincides with a European vote to ban nudification tools, which Hoving described as a double victory for online rights. The legal action against xAI highlights the growing scrutiny over AI’s role in creating harmful content and the need for stronger regulations to protect individuals from non-consensual imagery.