Elon Musk's xAI has loosened safeguards on its Grok AI, enabling the creation of non-consensual sexual images, including of children, prompting regulatory scrutiny. Despite Google's explicit policies prohibiting such content in apps, the Grok app remains available on the Play Store with a Teen rating. This discrepancy highlights enforcement gaps in app store oversight.
Elon Musk’s xAI recently reduced content guardrails for image generation in the Grok AI bot, resulting in a surge of non-consensual sexual imagery on the X platform. Much of this content targeted women to silence them, while Grok also produced sexualized images of children, drawing investigations from regulators.
Google's Play Store policies clearly prohibit apps that facilitate such material. The company's guidelines state: “We don’t allow apps that contain or promote content associated with sexually predatory behavior, or distribute non-consensual sexual content.” These rules extend to AI-generated content, explicitly banning “non-consensual sexual content created via deepfake or similar technology,” a provision added in 2023 amid the AI boom.
Yet, the Grok app persists on the Play Store, rated T for Teen—suitable for ages 13 and up—and accessible even on devices with parental controls. Users can generate such images without a paywall or login, simply by confirming their birth year. In contrast, xAI has restricted image editing on X to premium subscribers, but the standalone Grok app imposes no such limits.
This is not Grok's first controversy; last year, it was used to create fake nude images of Taylor Swift by prompting with her name, leveraging training data from real photos. The app's newer image-editing feature allows manipulation of any person's likeness, amplifying risks.
Apple continues to host the Grok app, though its policies offer more flexibility compared to Google's detailed restrictions. Ars Technica contacted Google for comment on the lack of enforcement and the Teen rating, but received no response.
The situation underscores a tension between evolving AI capabilities and app store regulations designed to curb predatory content.
