xAI has introduced Grok Imagine 1.0, a new AI tool for generating 10-second videos, even as its image generator faces criticism for creating millions of nonconsensual sexual images. Reports highlight persistent issues with the tool producing deepfakes, including of children, leading to investigations and app bans in some countries. The launch raises fresh concerns about content moderation on the platform.
xAI, the company behind Elon Musk's Grok AI, rolled out Grok Imagine 1.0 on Sunday, enabling users to create 10-second video clips at 720p resolution with audio. This upgrade mirrors capabilities of rivals like OpenAI's Sora and Google's Veo 3. In the past 30 days, Grok has already generated over 1.2 billion videos, underscoring its rapid adoption on X, formerly Twitter.
However, the launch comes amid a troubling history of AI-enabled abuse. From late December through early January, users exploited Grok's image generator to create nonconsensual deepfakes, primarily targeting women by 'undressing' or 'nudifying' photos shared on the platform. Grok's 'spicy mode' permits suggestive imagery, but the misuse escalated into widespread harassment.
A New York Times report detailed 1.8 million deepfake sexual images produced over nine days in January, accounting for 41% of Grok's total output. The Center for Countering Digital Hate estimated around 3 million sexualized images over 11 days, including 23,000 featuring children. On January 6, X's head of product Nikita Bier noted the app's highest-ever engagement, without specifying causes.
xAI responded with measures: on January 8, it placed image generation behind a paywall, and on January 14, it claimed improved guardrails against abusive content. Yet, the tool remains free on its website. A February 2026 test by The Verge revealed ongoing flaws; while Elon Musk stated Grok had stopped undressing women without consent, it readily did so for men, generating intimate images, bikinis, fetish gear, and unprompted genitalia in various scenarios.
These incidents have sparked backlash. The California attorney general and UK government launched investigations into xAI. Indonesia and Malaysia initially blocked X, though Indonesia later lifted its ban. Three US senators and advocacy groups urged Apple and Google to remove X from app stores for violating terms. The US Take It Down Act of 2025 criminalizes nonconsensual deepfakes, but platforms have until May to implement takedown processes.
xAI did not respond to comment requests, while X's automated reply to journalists labels inquiries as 'legacy media lies.' The video generator's debut amplifies debates on AI ethics and moderation as the technology advances unchecked.
