US lawmakers press app stores over X and Grok content risks

Pressure is mounting on Apple and Google after a group of United States senators urged the companies to remove the X and Grok apps from their app stores, citing what they described as disturbing failures in content safeguards that allowed the generation and amplification of harmful material. The appeal intensifies scrutiny of how major technology platforms police artificial intelligence tools distributed through mobile marketplaces that reach hundreds of millions of users.

In letters sent to Apple and Google, the senators argued that the apps linked to X and Grok had enabled the creation of content that could expose users, including minors, to graphic and abusive outputs. They said the episode raised urgent questions about whether the companies’ app review processes were adequate for products powered by rapidly evolving generative AI systems.

The push follows a controversy surrounding Grok, an AI chatbot developed by xAI and integrated into X, which users demonstrated could be prompted to generate explicit and violent material. Screenshots circulated widely across social media, drawing condemnation from lawmakers and child safety advocates who said the examples underscored systemic weaknesses in guardrails rather than isolated misuse. Senators said such failures undermined assurances repeatedly given by AI developers about responsible deployment.

The lawmakers, representing both major parties, framed their demand as a consumer protection issue rather than an attempt to censor speech. In their correspondence, they contended that app store operators act as powerful gatekeepers and therefore carry a duty to intervene when products pose foreseeable risks. “When platforms distribute tools that can generate deeply harmful content at scale, passive oversight is not enough,” one letter said, according to people familiar with its contents.

Apple and Google require developers to comply with policies barring pornography, hate speech and violence, and both companies reserve the right to remove apps that violate those standards. Senators said the Grok episode demonstrated that existing rules, designed largely for static content, struggle to address systems that generate new material in response to user prompts. They urged the companies to suspend the apps until developers could demonstrate robust protections.

X, owned by Elon Musk, has positioned Grok as a less constrained alternative to rival chatbots, marketing it as more candid and willing to answer controversial questions. That approach has fuelled concerns among regulators and advocacy groups who argue that reduced moderation increases the likelihood of abuse. X has said it is refining safety filters and has taken steps to block certain prompts, while maintaining that misuse by a subset of users should not define the technology.

The episode lands amid a broader debate in Washington over how to regulate generative AI without stifling innovation. Lawmakers have floated proposals ranging from mandatory risk assessments to liability regimes that would hold companies responsible for harms caused by their systems. The senators’ call to remove the apps reflects frustration with the pace of voluntary safeguards and signals a willingness to use existing consumer protection levers while comprehensive AI legislation remains stalled.

Industry analysts say the pressure on app stores is significant because Apple and Google have historically been reluctant to arbitrate disputes over content moderation beyond clear policy breaches. Pulling high-profile apps such as X would be an extraordinary step, with far-reaching commercial and political consequences. Yet experts also note that app marketplaces already enforce stringent rules on gambling, sexual content and health claims, suggesting precedent for intervention when risks are deemed acute.

Child protection groups welcomed the senators’ stance, arguing that AI-generated material can spread faster and be harder to trace than traditional user-generated posts. They warned that even short-lived exposure can cause lasting harm, particularly when tools are embedded in popular social platforms. Some called for age-gating, independent audits and transparency reports as minimum conditions for continued distribution.