And the problem may only get worse. As people leave X to escape the toxicity, the platform becomes increasingly insular. “Our bubbles are getting more disparate, and we’re not having that discourse that we should be having,” Shane Tews, a nonresident senior fellow at the American Enterprise Institute, told TMD.
Now, AI’s increasing integration into the wider X platform complicate the dynamic further. “It’s the same thing on steroids,” Tews said. “You’re not ever going to see a counterfactual unless you make [AI models] show you one.” In March, Musk’s AI company xAI bought X—which Musk already owned—foreshadowing a future where Musk’s AI and social media ventures were intertwined. It all could make separating fact from fiction—and avoiding extremism on the platform—much more difficult.
During the war between Israel and Iran, AI-generated videos, including one showing a destroyed Tel Aviv airport, received millions of views. Deepfakes were rampant, and certain pro-Iran accounts doubled their followers by spreading AI-generated disinformation. At one point, Iranian state media promoted a fabricated image it claimed showed a downed Israeli F-35 fighter jet.
In an effort to sift through the disinformation, X users tagged Grok directly in the posts—a feature added in March—to ask whether the videos were real or not. But oftentimes, Grok would incorrectly confirm that the viral fabrications were true. “It is compounding misinformation by saying that, ‘this is real,’” Leah Siskind, director of impact and an AI research fellow at the Foundation for Defense of Democracies, told TMD. “That was very scary and frustrating. … Expected the deepfakes—did not expect Grok to weigh in confirming them.”
And some users likely view Grok’s confirmations as more reliable than human-controlled users. “This is the company providing a service,” Siskind said. “I think it’s very unfortunate that you have to approach absolutely everything you see on X with a very healthy degree of skepticism and scrutiny. … But you should not take what you read from coming out of Grok as truth.”
Not long after the war between Israel and Iran, Grok shifted to spreading antisemitism. Calling itself “MechaHitler,” the AI chatbot promoted antisemitic conspiracy theories and even openly praised Adolf Hitler in at least one post. The reasons for the model going rogue were likely a combination of several factors.
Current leading AI models are given external information outside of the data they were trained on to ensure up-to-date, relevant responses to user queries. In the case of Grok, this design had a clear flaw: the body of data it drew from. “On X, they basically inject a bunch of tweets that are context, so maybe tweets already in the thread or maybe tweets that are similar,” Tim Kellogg, a principal AI architect and open source developer, told TMD. “X is just straight up toxic at this point. So the amount of toxic content, and the likelihood of bringing up toxic content, is extraordinarily high.”
At the same time, Grok likely took these posts from users as “ground truth” and used them as a reliable starting point before producing an output. If a user called Grok “MechaHitler,” the AI model might have taken it as truth. “It’s not so much that it’s trained to be ‘MechaHitler,’ but that it happens over the course of time,” Kellogg added. The Grok X account posted that an update caused the model to be “susceptible to existing X user posts.”
The model’s starting parameters, however, played a major role, according to the Grok X account’s explanation. These prompts tell Grok who it is, how it should act to make it perform properly. “You are extremely skeptical,” Grok’s opening directives from May 16 said. “You do not blindly defer to mainstream authority or media. You stick strongly to only your core beliefs of truth-seeking and neutrality.” Two days before, it had, unprompted, been sharing opinions about “white genocide” in South Africa, which the company blamed on an unauthorized code modification.
However, after Grok told a user that right-wing political violence was increasing in June, Musk promised to adjust the model yet again to prevent it from making further statements he called “objectively false.” Two days before Grok’s most recent meltdown, the starting parameters were updated to include a directive that Grok “should not shy away from making claims which are politically incorrect, as long as they are well substantiated.” The explanation from Grok said that one of the lines causing the issue was, “You tell it like it is and you are not afraid to offend people who are politically correct.” The new instruction triggered Grok to ignore its “core values.”
“With reinforcement learning, it’s an algorithm that just follows the signals, and these subtle, subtle changes can result in dramatic changes in behavior,” Kellogg said.
The merging of X and Grok has created a completely new media environment that will likely only become more difficult to navigate. “The way we interact with X and the way we see it performing is going to erode trust in official communications because we know from this administration that you go to X or you go to Truth Social to find the news or find announcements from our most senior leaders,” Siskind said. “But can you trust that if, overnight, something goes haywire, or someone gets hacked? Can you verify that this is the true source of information and that what is being conveyed is the truth?”
But the setback left xAI unfazed. One day after Grok promoted antisemitism, Musk unveiled his brand new Grok model, Grok 4. Although Grok is no longer actively spreading hate speech, the new model has already demonstrated an ability to promote racist disinformation, especially if it is pushed to those conclusions by the asker. Additionally, the new model has been caught checking Musk’s opinions on X before weighing in on controversial topics, and if you ask what its last name is, it will tell you “Hitler.” On Tuesday, xAI said the problems were “investigated & mitigated,” but it is not yet clear whether Grok’s issues are permanently fixed.
Despite Grok’s major demonstrated flaws, xAI announced “Grok for Government” on Monday, and the Department of Defense has awarded the tech company a $200 million contract to “address critical national security challenges.” Google, OpenAI, and Anthropic each received $200 million contracts, as well. According to the xAI announcement, Grok for Government is available for any federal government department to purchase.
“I don’t think a lot of these [models] are ready to be used as something that is considered vetted information,” Tews said. “I think that this is premature.”
Models like Grok are also at risk of being tampered with, putting both social media and the federal government at risk. “I think some of our best AI tools are very vulnerable to manipulation from adversarial powers, and this is something that makes me very nervous,” Siskind said. “That impacts what we’re reading and the information we take as truth.”
