LLMs Hallucinate

Подробнее о видео

Why Do Large Language Models Hallucinate?
A Bit of Security for July 31, 2024
LLMs always hallucinate. Imagine a language with only 800 words – for example, basic English. What an LLM does is look at a sample of some sentences written in that language and pick the statistical likelihood that this word will follow that word. Then it starts spewing out strings of words that are nothing more than the most probable strings of words. Is there anything like thinking going on here? Is there any knowledge at all in this process? Cleary not. If we happen to like one sentence better than another, that’s nothing more than an aesthetic choice.
Now consider what happens when these LLM-generated word strings get dumped into the training set. The frequency distribution shifts, amplifying the most likely word choices a bit and de-emphasizing others, till over time the model produces the one true sentence. Is it meaningful in any practical sense? No. It’s merely the most likely string of letters.
Let’s take this down one level. Instead of looking at the frequency of words in a sentence, let’s look at the frequency of letters in a word. A statistical analysis of the most likely letters already exists – cryptographers have been using them for decades. By stringing together the most popular letters, augmented by an algorithm that scores the frequency of letter pairs, we eventually would develop the “perfect” word. This definition of perfection means simply the most likely finite string of letters. Every language will have its own, of course. Is there anything in this model that suggests the most likely string of letters is somehow more meaningful than all other strings of letters? Is there anything in this process that resembles thinking, or is it more like babbling? Listening to infants playing with language is wonderful for a parent, and the first time the childe says a recognizable word the parents delight in it. Not because the baby has said anything profound, but because the baby has made the first step towards conversing, exchanging ideas.
The philosopher Ludwig Wittgenstein suggested that language is a game we play with each other, agreeing by convention that this sound means that idea. Since we can only observe sounds but not ideas, there is always room for interpretation, for new meanings, for language to drift. My kids say “It’s been a minute” meaning it has been a long time. Once, “In a minute’ meant “very soon.” Once, “hot” meant really engrossing and emotionally engaging, which was supplanted by “cool” in the 1950s.
So before you go too far over the top into LLMs looking for meaning, take a minute and see if you’re getting all hot about something that’s too cool for words.
A Bit of Security for July 31, 2024
Why do LLMs Hallucinate?
LLMs always hallucinate. We aren’t bothered by their outputs until they violate our expectations too much. We need to stop anthropomorphizing word salad, for safety’s sake. Listen to this -
Let me know what you think in the comments below or at wjmalik@noc.social
#cybersecuritytips #LLM #AI #hallucinations #BitofSec