There is rightly a lot of news about the capabilities of chatgpt. And when discussing I notice there is a misunderstanding what it is good at and what are it’s limitations. There is a tendency to be in either one of two camps: it’s the best thing since sliced bread or it is totally rubbish. To answer this I will first look a bit at the history of chatgpt, which will instruct us how we can play around to get some really impressive and some utter garbage responses.
Chatgpt is part of a class of machine learning models called large language models. Large language models are models that try to mimic human speech, they are a part of an area of AI called natural language processing, which tries to teach computers to understand human language. This is actually one of the oldest branches of AI and large language models date back all the way to the 1950’s. A major breakthrough was achieved in the 1990’s when it became possible to use unsupervised learning or learning without answers. Most models in AI require supervised learning, which means that someone needs to go and tell for billions of pieces of text if they are correct for the model or not. This labeling of inputs with answers is a very laborious tasks, which anyone that ever wrote an exam will be able to testify and being able to train on raw data without answers allows for the construction of much more sophisticated models.
The next big breakthrough came in 2017 with transformers, in particular Bert developed by google which rightly became quite famous. Language has a context, and transformers are really well able to deal with this. Chatgpt is a very advanced transformer that was based on the Bert model.
What this means in very brief is that these models are very good to find similar content, but they are terrible to extract or understand facts.
Categories of AI
There are several classes of problems in AI with specific models (figure 2). Large language models are very good at detecting similarities and finding similarities within a context. What they are terrible at are classification and understanding.
There is actually a ladder of difficulty in cognitive tasks. The easiest is to find similarities, think of looking for similar cases. The next is to actually understand the concept. Third is to reason and be self aware about the concept and finally one can reason about how others would reason about the concept and doubt ones own accuracy. The latter is actually a very difficult task, not only for AI but also for most people. If you ask a doctor how confident they are their diagnosis is correct they will answer close to 100%, even though in practice doctors only diagnose with maybe a 60% accuracy. Worse giving anyone more information, generally doesn’t improve accuracy, but does make experts more confident their diagnosis is correct. And of course I am myself 100% confident I am correct.
What it does and doesn’t do
Chatgpt is very good to find similarities, or text in context. For example if you ask it to give a definition it will usually answer perfectly, if we ask it to rewrite an article in a certain style (context) it will do very well, if we ask it to classify something according to the definition just given it will give completely ridiculous answers. We asked chatgpt some questions through our interface: Zygon.ai
We all know what is a prime number. If we ask chatgpt what is a prime number we will get a perfect answer. It knows how to retrieve information really well and can take into account context (figure 3). Retrieving the information, does not however mean that it understands the concept. This is a classification problem and is something large language models are very bad at, as can be seen in figure 3.
Now something fun
With this knowledge in hand we can design questions to specifically trigger chatgpt to give either very impressive answers or answer something completely ridiculous. Very impressive are Q&A type questions with a clear context that are hard to remember for people or definitions on a certain topic (figure 4).
Figure 4: Chatgpt at it’s best. Answering very sophisticated Q&A questions.
However if we press for understanding, especially by asking something that does not make sense chatgpt goes off the rails. Even more so when we press it to explain it’s reasoning (figure 5). It is clear that chatgpt is very good to answer Q&A questions and repeat definitions, but it does not understand the concepts behind those definitions.
This also means that it does not understand biases in text or training data. In this way we can easily press chatgpt to become racist by putting a word with a negative connotation by itself in a context where it should be neutral (figure 6). Since dark is generally negative a dark person is therefore erroneously classified as negative and viola we turned our AI racist. Though I must give credit to openai in that they are trying their very best to avoid this and when trying the more obvious persons they explicitly caught this adding that in the case of complexion the expression is neutral. However this is an inherent weakness in the system.
Conclusion
Though the focus of this article was on how to make fun of chatgpt I think it should be remembered that like all models chatgpt does very well at some specific tasks, but is terrible at others. Actually I believe this is both true for people and the future of AI. As people we are all specialized, we do not ask an accountant to play soccer or a soccer player to solve math problems. If we keep in mind that a certain model is trained to do a specific task it can be very powerful. Chatgpt’s strength lies in rewriting existing text or answering Q & A type questions. This makes it very powerful for chatbots and tools aiding in rewriting text, but we can easily trigger it to give disastrous results by asking it to perform other tasks.