Part 2: Welcome to Terra Incognita
The sizes and complexity of deep learning neural networks have snowballed over the last decade. ChatGPT feels different. A threshold was reached, as if a space alien suddenly appeared that could communicate with us eerily humanly, talking with us in perfectly formed English sentences and better grammar than most native speakers.
LLMs have made considerable leaps in size and capability . The latest results have stunned experts, some of whom have difficulty accepting that talking humans have been joined by talking neural networks created from our verbiage. The rate at which ChatGPT and other LLMs are improving is even more remarkable. We have stepped through the looking glass and are on an adventure that is taking us to terra incognita.
This visitation from another world has evoked a wide range of views on whether LLMs understand what they are saying. The debate has polarized the linguistic and computational communities, touching emotional nerves in experts. On July 10, 2023, Geoffrey Hinton, who received the ACM Turing Award for “conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing,” gave a talk at the Association for Computational Linguistics conference. The association’s vice president, Emily Bender, asked the first question and asserted loudly that GPT-4 does not understand what it is saying.
How do we know whether or not LLMs understand anything? Do we know how humans understand? It is difficult even to know how to test an LLM for understanding, and no consensus exists for which criteria to use to evaluate their intelligence genuinely. Some aspects of their behavior appear to be intelligent, but if it’s not human intelligence, what is the nature of their intelligence?
Only one thing is clear—ChatGPT is not human, even though LLMs are already superhuman in their ability to extract information from the world’s vast database of text. In some ways, this is even more impressive than Arnold Schwarzenegger in the science fiction/action movie The Terminator, who claimed that he had learned human behavior from a neural network but was not as omniscient as an LLM.
The Talking Dog
This story about a talking dog begins with a chance encounter on the backroads of rural America when a curious driver came upon a sign: “TALKING DOG FOR SALE.” The owner took him to the backyard and left him with an old Border Collie. The dog looked up and said: “Woof. Woof. Hi, I’m Carl. Pleased to meet you.” The driver was stunned. “Where did you learn how to talk?” “Language school,” said Carl, “I was in a top-secret language program with the CIA. They taught me three languages: How can I help you? как я могу вам помочь? 我怎么帮你?”
“That’s incredible,” said the driver, “What was your job with the CIA?”
“I was a field operative. The CIA flew me around the world. I sat in a corner and eavesdropped on conversations between foreign agents and diplomats, who never suspected I could understand what they were saying. I reported back to the CIA what I overheard.”
“You were a spy for the CIA?” asked the driver, increasingly astonished.
“When I retired, I received the Distinguished Intelligence Cross, the highest honor awarded by the CIA, and honorary citizenship for extraordinary services rendered to my country.”
The driver was shaken by this encounter and asked the owner how much he wanted for the dog.
“You can have the dog for ten bucks.”
“I can’t believe you are asking so little for such an amazing dog.”
The farmer chuckled and said: “Did you really believe all that bullshit about the CIA?”
Have We Created a Talking Dog? LLMs can converse with us and spin a good story, not unlike Carl. The AI taught itself solely from unlabeled text—it is blind, deaf, and numb, but far from dumb—an achievement even more impressive than learning a new language by watching subtitled TV shows.
Self-supervised LLMs are foundation models that are surprisingly versatile and can perform many different language tasks, exhibiting new language skills with just a few examples. LLMs are already being used as personal muses by journalists to help them write news articles faster, by ad writers to help them sell more products, by authors to help them write novels, by lawyers to help them search court cases and write briefs, and even by programmers to help them write computer programs.
The output from LLMs is not a final copy but a good first draft, often with new insights, which speeds up and improves the final product. There are concerns that AI will replace us, but so far, LLMs are making us smarter and more productive.
Part 3 explores the controversy about whether ChatGPT understands language.
I love the story of The Talking Dog, and I value how it pokes holes in the idea that AI has made us more productive. In my observation, AI is talked about as if it is an automatically useful tool that people can just pick up and use. But in my experience, AI has caused more setbacks than improvements, mainly caused by people assuming the output is correct, then acting accordingly. In other words, because it isn’t built to find truth, but is instead built to assist, respond, and create, it (sometimes hilariously) points unsuspecting real-life individuals (as opposed to the generalized “people”) in the wrong direction, creating the need for a lot of “re-work” and corporate “circling back.” The general use models I’ve encountered seem fundamentally flawed because their aim is to chat and entertain, but users presume they’re getting Answers (capital A intentional) that are precise and correct and modify their behavior accordingly.