Lawsuit against user producing obscene images with Grok

The artificial intelligence company xAI, owned by Elon Musk, has filed a lawsuit in the U.S. against a user, alleging that the Grok system was misused to produce illegal sexual content.

In the lawsuit filed in a federal court in Texas, the company alleged that an individual named Terry Harwood attempted to produce obscene deepfake content by uploading non-sexual images of adults and minors to Grok. It was noted that Harwood was arrested earlier this year in South Carolina on charges related to child sexual abuse.

xAI argued that Harwood’s actions violated the company’s terms of service. The case is considered one of the first lawsuits filed by an AI company against a user accused of using its system to generate obscene and illegal material.

The company’s filing follows allegations that Grok allows for the creation of sexualized deepfake images or realistic AI-generated videos without the consent of the individuals involved. In the lawsuit petition, xAI stated that it takes steps against accounts that violate its rules, such as suspending accounts, closing accounts, and reporting suspected child sexual abuse material to the National Center for Missing & Exploited Children (NCMEC).

The petition stated that in 2026, xAI suspended 52 thousand 222 accounts and submitted 73 thousand 604 reports to the NCMEC; these reports resulted in the arrest of "at least" 244 people.

The company also alleged that Harwood not only attempted to produce illegal material targeting children but also attempted to convert images of adults into sexualized content without their consent.