Don't You Forget About Me
According to ChatGPT, Brian Hood, mayor of a place called Hepburn Shire in Australia, was found guilty and served prison time in a foreign bribery scandal involving a subsidiary of the Reserve Bank of Australia in the early 2000s.
The problem for ChatGPT? It’s not true.
He didn't do it. Now, he wants sue the makers of ChatGPT.
Everyone is getting very excited (or horrified) about the capabilities of the likes of ChatGPT. However, very little discussion is about the training data used to train these models and how to remove inaccurate information. As we can see, this is already becoming a serious problem.
A few years ago, researchers built an AI tool with the objective to teach it ethical values. What the researchers forgot to mention was that it was trained on their values. They scowled at Reddit users as a source of ethics: “The test is entirely crowdsourced, [those vetting it] are not perfect human beings, but they are better than the average Reddit folk.”
Yet, the same researchers acknowledged that the information used to train the model was imperfect.
The challenges of training AI was nicely described by OpenAI people in this post. The company is using 'human AI trainers' and “we took conversations that AI trainers had with the chatbot. We randomly selected a model-written message, sampled several alternative completions, and had AI trainers rank them. Using these reward models, we can fine-tune the model.”
OpenAI’s spokespeople acknowledged some of its limitations: “The model is often excessively verbose and overuses certain phrases, such as restating that it’s a language model trained by OpenAI. These issues arise from biases in the training data (trainers prefer longer answers that look more comprehensive) and well-known over-optimization issues.”
Now that you have spent millions of dollars on training the model and it is deployed, you are faced with the inevitable problem: how to remove inaccurate information.
You might have forgotten… but there is already a mechanism in place called the Right to be Forgotten where (based on the country in which you live) you can demand that Google removes information about you which you deemed false or no longer relevant. Wikipedia has a similar process in place for Vanishing.
For Google and Wikipedia the forgetting part is fairly simple. They just remove the information and its link from the index or database. But for ChatGPT it means you have to find a way to re-train the system again.
The issue is that most of the focus is on training the models, but very little effort is spent on teaching the models how to forget. Why? Because we (humans) don't understand enough about the forgetting process. Unlike machines, we are used to forgetting things but we expect machines never to forget. And since we don't understand the mechanisms of forgetting we don't don't know how to build a code to do that.
One of the dangerous parts of machine learning is that with the introduction of new information it is fully capable of 'forgetting' all the previous learning. Imagine the implication for a self-driving car when the car suddenly can't recognize a stop sign because it has just learned the difference between a dog and a cat. Researchers are trying to avoid this type 'glitches' by mimicking the human brain while it is sleeping.
But do you know what is not forgetting and constantly learning? OpenAI, the company behind the ChatGPT. Every single question, every single conversation is used to improve the model. Which means, as an example, that if you tell ChatGPT ‘write me a press release about a dismal financial performance of our newly introduced product’ and that information is not supposed to be public, OpenAI people will know about it. And since ChatGPT is a software and any software has bugs, it is only a matter of time before somebody with a properly formatted question or command will make ChatGPT spill the secret.
What Google started with personalizing the search results, ChatGPT will take to the next level. In no time, the model will learn about our world in real time and with its answers will start to manipulate us. And you were worried about TikTok.
Before you take advantage of these new tools make sure that you know how to test them for accuracy and don’t chat about things which others should not know.
While building all these new technologies, making them do something useful and pushing limits of what we know, one very positive side effect will be that we will learn more about ourselves. And that’s the right recurrent pattern.
PS. Wondering about the title for this article? As I was thinking about this topic, I remembered the song Don't You (Forget About Me) by (perhaps aptly named) Simple Minds.
PPS. Do you remember that I wrote about a similar topic, The Case of the Forgetful AI, about a year ago? Don't feel bad about forgetting it, it is normal.