The Gist
- Search shift. The search engines that originally set out to organize the world’s information and make it universally accessible and useful may be drifting into the information interpretation business.
- Bias overcompensation. Google’s Gemini platform’s recent highly publicized breakdown illustrates how these generative AI platforms are beginning to overcompensate for the inherited biases in the data and algorithms.
- AI challenge. AI bias will continue to be a challenge for generative AI platform providers and may require rethinking of the model to provide citations and parametrization to enable users to have more transparency.
- Truth ambassadors. There’s a tremendous opportunity for generative AI builders to become ambassadors of the truth by ensuring high-quality data sources and increasing transparency by providing citations to source information.
There’s a concerning shift occurring in the information platforms we use every day. The search engines that originally set out to organize the world’s information and make it universally accessible and useful may be drifting into the information interpretation business. This trend is particularly evident in generative AI platforms, which are increasingly influencing how information is presented and understood.
What Has Happened to Search?
This is becoming more evident in the AI age where generative AI platforms are seemingly creating an interpretation layer between our prompts and the data sources to filter the responses. The motivation is supposedly to “protect” users from biases, controversial content and inaccuracies.
However, these corrections may in fact be introducing some new biases and inaccuracies that may be reflective of the organizational biases. Recently there have been highly publicized examples of responses from generative AI platforms that are “over-correcting” their responses to compensate for anticipated biases in training data.
This is a very dangerous line to cross.
The original search service provided the ability to index massive amounts of online content and provide an intuitive and easy interface for information retrieval. It basically provided direct access to others’ information based on relevance without any interpretation.
What has emerged is an information interpretation service. This is a very different product and paradigm from the original mission. These generative AI platforms are now acting as an aggregation, synthesis, filtering and interpretation layer. While this may be useful on some levels, it has obvious implications.
The fundamental question is how do we get the truth?
Related Article: Overcoming AI Bias in CX With Latimer
Continued Challenges for the Google AI Team
When OpenAI launched ChatGPT back in November 2022, Google was highly criticized for being sidelined in the AI race. The expectation was that Google has access to much of the world’s information and search data enabling them to be a prime leader in the AI race.
The justification was that Google had superior technology but was overly cautious about its business, reputation and customer relationships. Google actively considers how these potential backlashes could damage its reputation.
But then Google appeared to be on the AI path again, first releasing Bard and then more recently rebranding their entire AI suite to Gemini. However, just as things seemed to be moving along, disaster struck.
Gemini began to exhibit some controversial answers to basic questions. For instance, generating pictures of America’s founding fathers and the Pope in genders and ethnicities that are historically inconsistent. Other concerns such as calling India’s Prime Minister Modi a “fascist” and comparing Elon Musk to Hitler. This started a string of accusations calling Gemini “ultra-woke.”
Google’s CEO Sundar Pichai responded in an internal email calling the Gemini responses “completely unacceptable” and promised that Gemini would roll out after a clear set of actions. The company had to take the platform down to deal with the concerns. It is expected that the changes will likely involve structural changes, updated product guidelines, improved launch procedures, better testing and red-teaming.
The Gemini controversy quickly hit Alphabet’s bottom line, losing over $90 billion in market value as shares fell by 4%.
Related Article: 4 Tips for Taming the Bias in Artificial Intelligence
Generative AI Platforms: Information Filtering Was Never a Goal
The concept of organizing complex information and providing access through a centralized retrieval system is not new.
In 1945, Vannevar Bush, an American engineer, described an information retrieval system that would allow a user to access a great expanse of information, all at a single desk. The system was called memex and initially described in an Atlantic Monthly article titled “As We May Think“. The memex was intended to give users the capability to overcome the ever-increasing difficulty of locating information in ever-growing centralized indices of scientific work.
The first web search engine was actually not created by Google. It was called Archie, and was created in 1990 by Alan Emtage, a student at McGill University in Montreal. The web crawler initially focused on indexing FTP sites but was the first tool to index content.
Additionally, early Internet location services like WHOIS were also leveraged as search and retrieval platforms for users, servers and even information. Some of these platforms even predated the debut of the Web in December 1990, with WHOIS dating back to 1982.
It’s amazing to note that prior to September 1993, the World Wide Web was entirely indexed by hand. Tim Berners-Lee would actually manually update the list of servers.
Google arrived on the scene in 1996 to revolutionize how search algorithms worked. In the late 1990s and early 2000s a proliferation of search engines followed such as Yahoo!, AltaVista, Excite, Lycos, Microsoft MSN and others. Many of these search platforms have since been acquired and vanished but Google still maintains 80%+ of the overall search marketshare.
Search bias has certainly crept in over time motivated by commercial, political and social influences. For instance, the ability to purchase certain search keywords to improve search relevance biases the native search results. Or the decision by a search engine not to index certain politically motivated sites also restricts access to information.
The important point to note is that none of these platforms were initially designed with the intent of filtering or interpreting information. Access to “raw” information and the ability to search was paramount. The goal was to present the most relevant information to a user based on the search ranking algorithms. The user was then empowered to navigate the search list and select which links appeared most relevant to their search intent.
Related Article: Addressing AI Bias: A Proposed 8th Principle for ‘Privacy by Design’
Let’s Talk About Truth and Bias
Ethics and the avoidance of bias have long been a cornerstone of journalism and the creation of knowledge platforms. Ethical journalism strives to ensure the free exchange of information that is accurate, fair and thorough. An ethical journalist acts with integrity and avoids stereotypes and other biases. These same principles must guide the builders of AI systems.
Providing the truth is an important objective for any knowledge platform. If the information being provided is not accurate, it can create many downstream issues. The base principle for any AI product should be that it is accurate and correct.
Accuracy is largely dependent on the training data and the algorithms used to tune the AI model. Both can contain inherent biases — the datasets may not adequately represent all cases fairly and the algorithms are developed by humans who have their own built-in biases. So removing biases completely is difficult.
However, creating an interpretation layer that makes decisions on the final output is equally problematic. It’s nearly impossible for a general-purpose AI platform to remove biases and stereotypes through an interpretation layer at a scale. In fact, it’s likely to create new anomalies by programmatically overcompensating for biases. It will end up giving inaccurate and unpredictable results.
The other issue that arises is what might be “stereotypical” for one person, might be “typical” for another. Users should be able to look at the raw data directly and make their own interpretations as to whether the data contains stereotypes or bias. Having the platform do that for you will, by definition, get it wrong based on a person’s context. AI platforms frequently inherit the biases of their creators.
There are two major observations in how generative AI platforms respond to prompts. One is the determination that they won’t provide an answer to a specific prompt, which is typically pre-determined by some filtering. The other is when the answer is inaccurate or wrong, which is even more concerning.
In either case, the determination of what questions should be allowed to be answered or the determination of what the “truth” is should not sit with the developers of the platform. Generative AI platforms can’t put their own filter on what the truth is. Generative AI platform provider’s mission should at all times be to present the truth.
The question for generative AI platform providers is do they have the ability to change the function and ideology of their platform teams? It might be that these generative AI platform providers have inadvertently built too many biases into their own culture and internal teams, which is resulting in biased foundational models.
The bottom line is, filtering information will never help people find the truth.
Eliminating AI Bias Is Not Easy
AI bias may be unavoidable as training data frequently contains inadvertent data anomalies that manifest in inaccurate responses or, even worse, “hallucinations.” Algorithm-based technology will always be susceptible to this. At the core, these generative AI platforms are prediction engines and provide probabilistic, not deterministic, answers. The ultimate goal of these generative AI platforms is to get them to be as deterministic as possible.
There are absolutely consequences of this. We’ve seen racial and ethnic biases in healthcare algorithms, police surveillance, applicant screening speech recognition and many other high-profile applications.
The quality of data is a big factor in determining how generative AI platforms can report the truth. Many of these generative AI platforms use the Internet as a primary source of data. The popular Common Crawl has become pivotal to the development of generative AI as the largest freely available source of web crawl data. The Common Crawl’s massive dataset is more than 9.5 petabytes in size and makes up a significant portion of the training data for many Large Language Models (LLMs). A recent study analyzed 47 LLMs published between 2019 and October 2023 and found at least 64% of them were trained on Common Crawl.
The issue with using Internet-based data sources is that the quality and accuracy of the data are problematic. While generative AI builders do typically filter the Common Crawl dataset before the training, it’s not always easy to identify which sources to filter out. Identifying which data sources are providing inaccuracies or, even harder, which data sources contain bias is not easy.
Let’s consider an obvious example. Empirical data supports that the world is roughly a sphere and not flat. Not since the 17th century, when this myth was created, has this been a popular belief. Despite this, the Internet is littered with conspiracy theorists who proclaim the world is flat. These “Flat Earthers” firmly believe this and gather facts and twist historical data to make a case. Given the swell of data this has created on the Internet, it might be hard to completely filter out. While a low overall percentage of weight would be given to this content in an LLM, it wouldn’t be zero. This in turn makes it possible for generative aI platforms to respond that the world is indeed flat, which is not factual.
Many such examples make generating the truth a difficult task for generative AI platforms.
Citations May Be the Answer
Developing the principles, processes and systems to determine which sources of data are the most accurate for inclusion in model training is very difficult.
Generative AI platform creators should focus on acquiring data sources that improve the quality of training data. In some cases, this may mean entering into license agreements with content creators. Recently, Google entered into an annual $60 million licensing agreement for Reddit’s massive repository of posts and comments. Reddit indicated in its S1 filing that they expect significant growth to come from data licensing strategies from both API access and model training.
Enabling LLMs to provide citations on the source of the data is also an important strategy. Similar to how search engines function, this would enable users to access the source content directly and make their own assessment of the validity of the content. Most of the current LLMs do not provide citation information today. Since they don’t have direct access to sources or databases like a search engine does, it becomes difficult to have traceability back to source training data. This is not easy as citations in an LLM are not a 1:1 correlation as the output text could be derived from millions of examples.
There has been significant research on this problem. Retrieval augmented generation (RAG) emerged as a popular LLM-based architecture to enhance the accuracy and reliability of generative AI models with data from external sources. Retrieval-augmented generation gives models sources they can cite, like footnotes in a research paper, so users can check any claims.
RAG is a framework that bridges the gap between pure generative AI models and data from a predefined dataset. It combines these two approaches, enhancing the capability to produce coherent and contextually appropriate responses by augmenting data in the model from real-time sources. RAG does have limitations such as being susceptible to low-quality data and performance at scale.
While there may not be one single magic solution here, there is an opportunity to re-think how the current architecture enables more end user transparency.
Truth and Transparency Are a Choice
Generative AI platform providers have a choice. There’s a tremendous opportunity to become ambassadors of the truth by ensuring high-quality data sources and increasing transparency by providing citations to source information.
Failure to do this in the short term will erode end-user confidence and delay the widespread adoption of these potentially powerful platforms.
Learn how you can join our contributor community.
