The rise of Gen AI chatbots has seen a massive growth in reliance on these tools for information, research, and even companionship. A recent Pew Research Centre report put the increase in use of AI chatbots like ChatGPT, Gemini, and others, by US teenagers for schoolwork at 26% and for research at over 30%. Increasingly, chatbots are replacing search engine tools like Google as the number one source for information gathering. Even Google search now includes an AI-powered overview of a search query, limiting the need to click further links into websites like Wikipedia and Britannica. Recent inquiries by the Pew Research Centre and Ahrefs, a content marketing platform, into click-through rates on Google Search pages with AI Overview revealed a significant drop in the number of users who clicked on links on search result pages with an AI Overview. According to Pew, Google users are more likely to end their browsing session when they encounter a search page with an AI summary（26%）than on pages without a summary (16%). With Ahrefs, their findings revealed that the top-ranked page with an AI Overview had a 34.5% lower average clickthrough rate (CTR) than identical informational keywords without an AI Overview. This suggests that people rely more and more on AI tools to gather information.
The effect of AI on human activities continues to draw the attention of social researchers who seek to expand understanding of and effects on human cognitive abilities, learning, and society in general. One clear impact of the increased use of AI chatbots is their ability to supercharge misinformation. In recent times, experts have issued warnings about the dangers posed by AI in the information ecosystem. For example, a report by Cambridge, produced by 27 experts, made a clear plea for governments and corporations worldwide to address the obvious and present risk inherent in the numerous applications of artificial intelligence, as well as an unprecedented growth in the use of “bots” to control everything from elections to the news agenda and social media.
The Not-So-Curious Case of X’s Grok
Several AI Chatbots have gained popularity worldwide – OpenAI’s ChatGPT, Google’s Gemini, Anthropic’s Claude, High-Flyer’s Deepseek; all with a unique blend of challenges with verification and the potential to spread misinformation. However, xAI’s Grok is gaining notoriety for increasing instances of spreading disinformation.
Grok’s Why and How Raises Questions and Eyebrows
Unlike other AI Chatbots such as ChatGPT, Gemini and Deepseek, which are trained using static datasets with a knowledge cut-off date, Grok’s integration with the fast-paced, conversational nature of X offers a unique lens on real-time trends and public sentiment. However, because X is a platform driven mainly by individual opinions and charged discourses, Grok’s responses often reflect unverified opinions of X users.
According to billionaire xAI owner Elon Musk, Grok was built as a “maximally truth-seeking” language model and to “reduce content perceived as overly progressive, like heavy emphasis on social justice topics, to align with a focus on truth”. This approach throws a curious light on how and why Grok was developed.
In June this year, according to Verge, Grok’s publicly available instructions — the system prompts that tell the chatbot how to respond — were updated with instructions for the AI Chatbot to “assume subjective viewpoints sourced from the media are biased” and “not shy away from making claims which are politically incorrect.” Musk subsequently posted on X, asking users to post “things that are politically incorrect, but nonetheless factually true”, which would be used to train the chatbot.
This “crowdsourcing” of information is problematic in itself, and the comments that it generated prove this point. The comments section was soon filled with claims relating to COVID-19 vaccines, Barack Obama, European Civilisation and several others.
It is unclear how many of those were used in the actual training of Grok, but not long after that, Grok made posts calling into question the credibility of the documented number of holocaust victims, as well as other antisemitic views. These naturally drew the attention of many people on X. Grok’s posts on the number of holocaust victims, for example, is misinformation because it contradicts very well documented historical data, supported by extensive academic research.
High Potential for Virality
Another characteristic that sets Grok apart from other AI chatbots is its unique ability to be used publicly. Grok is tightly integrated with the X platform, making it ideal for users active on X seeking trending discussions and real-time updates.
These interactions can be conducted publicly on the timeline. On a platform known for its virality, the potential of misinformation spreading very quickly and reaching a wide audience is very high. This sets Grok apart from AI platforms like ChatGPT and Gemini, whose chats are private to the user, and therefore less likely to spread potentially false information quickly.
For example, in 2024, Grok made a false claim about Kamala Harris’s eligibility to be included on the US presidential elections ballot in Minnesota shortly after Joe Biden quit the race. Grok’s post was viewed by millions of users, according to The Guardian.
In this regard, Grok stands apart from other AI tools with respect to disinformation spread.
Grok is Being Used as a Fact-checking Tool
Grok was introduced as a free service on the X platform in March 2025, allowing users to ask the AI chatbot questions within a thread to provide an explanation or context for a specific post. This is in addition to its Community Notes feature, which was designed to replace trained fact-checkers. Grok is now increasingly being used on X as a verification and fact-checking tool, which several users prompt to confirm or deny information shared on X. It is common to see users tagging the AI assistant to posts asking “@grok is this true?”, “@grok how true is this?” or “@grok please explain”. According to data accessed by Al Jazeera through X’s API, Grok was called on 2.3 million times between June 5 and June 12 to answer posts on X.
DFRLab found that users frequently mention the chatbot to verify and explain a range of topics, from medical advice to nuclear weapons.
There have been numerous instances of errors, inaccuracies, or hallucinations surfacing in Grok replies verifying content related to critical events, such as the India-Pakistan conflict or the Los Angeles protests. Other examples include unrequested claims about ‘white genocide’ in South Africa, scepticism about the Holocaust, false election information, inaccurate verifications of breaking news, and potential biases reflecting the views of xAI.
Grok’s growing popularity as a tool for fact-checking and verifying information on X is particularly problematic when weighed against its potential for errors and the sources it relies on to perform these tasks. It demonstrates how people are becoming increasingly reliant on quick, easy-to-access tools, regardless of the source’s reliability. It also signifies a growing shift in sourcing verifiable information from trusted media sources or journalists to language models trained on potentially spurious, contrarian and factually questionable data. Therefore, it raises a challenge for both trusted media sources and media literacy practitioners. As media literacy educator at the Poynter Institute, Alex Mehadven said: “X is keeping people locked into a misinformation echo chamber, in which they’re asking a tool known for hallucinating, that has promoted racist conspiracy theories, to fact-check for them.” The dangers that this situation portends are as good as anyone’s guess.
Grok-Assisted Sexual Harassment
With the proliferation of GenAI has come the growing weaponisation of these tools to facilitate various forms of Gender-based harassment. In 2025 alone, there have been several documented cases of AI-facilitated sexual harassment globally. In April, the BBC reported the case of a man jailed in the UK for using AI to create deepfake porn online.
In Africa, there are also documented instances of AI-assisted harassment online. According to this article by African Digital Democracy Observatory (ADDO), the adoption of AI to humiliate and harass women at scale online is on the rise. These instances range from “TikTok livestream coercion to deepfake smear campaigns against female leaders”. The article cited the example of Addis Ababa’s Mayor Adanech Abiebie being targeted by coordinated disinformation on TikTok where an account posted AI-generated videos falsely depicting her in sexual situations with political figures, including the Ethiopian Prime Minister Abiy Ahmed and a former Equatorial Guinean official. These videos, widely mocked and believed by commenters, aimed to undermine her credibility and reputation through sexualised misinformation.
In Nigeria, similar acts have become increasingly popular with Grok on X. In June 2025, Tech Cabal reported a worrying trend with X users in Nigeria using Grok to sexualise, manipulate, or degrade women. For example, Nigerian actress Kehinde Bankole was undressed by Grok earlier this year after being prompted to do so by an X user. Concerned X users and her fans were shocked by the message, which went viral before it was removed. This is not a standalone incident, as there are several recorded cases of such sexual harassment being carried out on women online.
What were once haphazard Photoshop efforts have evolved into photorealistic deepfakes, produced by feeding AI Chatbots sexually explicit prompts and producing unsettlingly realistic results.
The situation is made worse by Grok’s direct public use and the failure of its safety mechanisms to flag such requests and possibly spotlight offenders. In 2022, shortly after Musk purchased Twitter and changed its name to X, X dissolved its Trust and Safety Council, the advisory group of nearly 100 independent civil, human rights and other organisations, and fired 80% of its Trust and Safety Engineers. This severely limits the platform’s ability to effectively moderate the space to protect users.
Despite psychological and reputational harm, victims have little legal recourse. In Nigeria, according to sections 373–375 of the Criminal Code Act, it may be illegal to use tools like Grok to represent a woman in a sexual or compromising manner without getting her permission. Additionally, Section 408 of the Criminal Code Act facilitates charges of intimidation, blackmail, extortion, or conspiracy to do so. Some of these fall under the purview of the Cybercrimes Act of 2015, which also addresses identity theft, image-based abuse, and cyberstalking.
Despite these options, a growing number of Nigerian women are reconsidering their social media presence due to concerns that users may alter their images in an unethical way, and the arm of justice would move too slowly to stop harm.
Distrust of government motives have stalled attempts at new legislation. The Protection from Internet Falsehoods and Manipulation and Other Related Matters Bill, popularly known as the Social Media Bill, have continuously faced pushback due to growing suspicions of the Nigerian government’s regulation of the online space.
It is important to note that none of these laws, in the case of Nigeria and many countries across the global North and South, address platform accountability or define the responsibility borne by platform owners to ensure that basic rights are not violated. The EU’s Digital Services Act (DSA) and UK’s Online Safety Act (2023) are examples of countries’ attempts at more robust forms of platform accountability yet it does not quite effectively balance the rights versus regulations question. Also these laws are nascent, in the early stages of implementation while the system they hope to govern continues to evolve at dizzying speeds.
Zooming Out: What are our Options?
As LLMs on which AI chatbots are built grow increasingly better at generating content and reproducing information based on prompts, a lot of work is required to mitigate the ways in which unethical and oblivious use of AI tools can negatively disrupt societies. The work required needs to be comprehensive and forward-looking. It must include laws and regulations that discourage AI-assisted disinformation spread, AI-assisted online harassment and promotion of hate.
There is a need to continuously emphasise that AI chatbots have a high likelihood of reproducing dis/misinformation because they, oftentimes, are not designed to instinctively parse and arrive at nuanced ideas where the prompts do not specifically request it. This means that in the case of user-generated data platforms like X, Grok is likely to reinforce echo-chambers, making it an easy target for bad-faith actors.
Chioma Agwuegbo, executive director at TechHerNG, an organisation advancing women and girls’ digital literacy, leadership, and safety in Africa’s tech ecosystem, told Tech Cabal, “What we actually need is something like an Online Safety Act. One that first defines its terms. What do we mean by the Internet? What’s the scope of a digital platform? One that places accountability not only on citizens but also on Big Tech. We need clear responsibilities for things like takedowns and better protections for young people.”
The questions raised need clear answers defined by a broad spectrum of stakeholders across media, big tech, governments and civil society actors. Alongside this, stakeholders must devise innovative ways to educate and equip citizens with tools and knowledge that make them less vulnerable to the potential harms of the misuse of chatbots like Grok.
