Sexual deepfakes generated by the Grok AI chatbot and shared on the X social media platform breached Canadian privacy laws, the country’s privacy watchdog says.
“This lack of protection allowed users to create and share sexualized deepfakes, largely targeting women and children,” Dufresne said. “We know that sexualized deepfakes can have devastating impacts on victims. Canadians, and especially our children, must be able to safely navigate online spaces.”
The commissioner’s office launched an investigation in January into the growing presence of sexualized deepfakes created by Grok and shared on the X platform.
Dufresne said investigators found that, at one point, Grok was generating more than 6,000 sexualized images per hour.
The investigation also assessed whether the companies involved comply with privacy legislation and if they have “valid consent” for the collection, use, and disclosure of personal information to generate deepfakes, including explicit material.
The commissioner’s office determined that both xAI, the organization responsible for Grok, and X Corp., the entity behind the social media platform X, have breached Canada’s federal private sector privacy legislation. Both companies are subsidiaries of SpaceX, which is owned by Elon Musk.
In response to recommendations from Dufresne’s office, X and xAI have implemented new strategies, which include safeguards aimed at minimizing misuse of their tool for creating sexualized deepfakes. The companies have also carried out measures to eliminate “harmful content from their platforms.”
The tech firms did not adopt all of the recommendations and remain “non-compliant” in several areas, the commissioner said.
One recommendation was to suspend the tool until proper safeguards were in place, Dufresne said.
“So press pause, fix the issue given the harms, and that gives an incentive to fix the issues quickly,” he said. “They’ve not accepted to do this and I can’t force them to do it.”
Dufresne also the recommendations the companies acted on are not fully resolved.
“They’re going to be bringing us information about the steps that they’ve put in place, the results, the statistics, the takedowns, all of those things,” he added. “We’re going to monitor that very carefully, and if we’re not satisfied with the measures that have been taken, then we'll consider our alternatives.”
xAI did not respond to a request for comment before publication.
Privacy Enforcement
The privacy commissioner operates independently from the government and reports directly to Parliament, but has no power to mandate change.Dufresne argued the case with X underscores a need to update Canada’s privacy law to grant his office the power to levy fines and enforce binding orders. He said that the lack of authority to enforce changes in cases of privacy violations represents a “major gap” in the existing framework.
“There is no ability to impose a financial consequence,” he said. “That creates a lack of incentive and in some cases maybe a disincentive to comply with the fundamental right to privacy.”
Artificial Intelligence Minister Evan Solomon has pledged to table legislation to modernize Canada’s privacy laws to curb misuse of personal information and address the issue of deepfakes.
Solomon said in a media statement the privacy commissioner’s investigation “sends a clear message: every company operating in Canada must respect Canadian law, protect Canadians’ personal information, and be accountable for how their systems affect people in the real world.”
He added that the government would have “more to say soon about our updated privacy legislation.”
“The Liberals’ AI strategy is vague, short sighted and lacks leadership. It’s an illusion,” he wrote on X. “Liberals talk about safety while eroding freedoms and failing to deliver results. They’re heavy-handed where they shouldn’t be and missing in action where it counts.”
The Liberal government’s recently introduced online safety bill would include regulations on social media platforms and AI chatbots.
Bill C-34, also known as the Safe Social Media Act, proposes a mandatory minimum age of 16 to hold social media accounts and imposes a “duty to act responsibly” on AI chatbot developers to mitigate the risk of harm. The bill, introduced this week, would also establish a new regulatory body.
The government has also proposed legislation to criminalize non-consensual sexual deepfakes, although it has not yet been enacted.
