Chatbots -- how do they work?
The arrival of artificial intelligence (AI) has brought us programs that respond to questions in a way that makes them seem like real people – and they are called “Chatbots!”
Remember chatting with SmarterChild back on AOL Instant Messenger back in the day? The way these bots function sounds pretty sophisticated, right? But how do these so-called chatbots work—and what’s the best way to use them to our advantage? Now when it comes down to it, they’re doing one thing really well: predicting one word after another.
But it’s important to remember that while they might appear pretty human-like, they are most definitely not—they’re only imitating us. They don’t have common sense, and they aren’t taught the rules of grammar like you or I were in school. They are also only as good as what they were schooled on—and they can also produce a lot of nonsense.
Chatbots might appear to be complex conversationalists that respond like real people. But if you take a closer look, they are essentially an advanced version of a program that finishes your sentences by predicting which words will come next. Bard, ChatGPT, and other AI technologies are large language models—a kind of algorithm trained on exercises similar to the Mad Libs-style questions found on elementary school quizzes. More simply put, they are human-written instructions that tell computers how to solve a problem or make a calculation. In this case, the algorithm uses your prompt and any sentences it comes across to auto-complete the answer.
Systems like ChatGPT can use only what they’ve gleaned from the web. “All it’s doing is taking the internet it has access to and then filling in what would come next,” says Rayid Ghani, a professor in the machine learning department at Carnegie Mellon University.
The way these programs process information and arrive at a decision sort of resembles how the human brain behaves. “As simple as this task [predicting the most likely response] is, it actually requires an incredibly sophisticated knowledge of both how language works and how the world works,” says Yoon Kim, a researcher at MIT’s Computer Science and Artificial Intelligence Laboratory. “You can think of [chatbots] as algorithms with little knobs on them. These knobs basically learn on data that you see out in the wild,” allowing the software to create “probabilities over the entire English vocab.”
-
The beauty of language models is that researchers don’t have to rigidly define any rules or grammar for them to follow. An AI chatbot implicitly learns how to form sentences that make sense by consuming tokens, which are common sequences of characters grouped together taken from the raw text of books, articles, and websites. All it needs are the patterns and associations it finds among certain words or phrases.
But these tools often spit out answers that are imprecise or incorrect—and that’s partly because of how they were schooled. “Language models are trained on both fiction and nonfiction. They’re trained on every text that’s out on the internet,” says Kim. If MoonPie tweets that its cookies really come from the moon, ChatGPT might incorporate that in a write-up on the product. And if Bard concludes that a cat sat on the democracy after scanning this article, well, you might have to get more used to the idea.
