Technology

I don't think I would have made too much of a difference because the state-of-the-art models still aren't a database.

Maybe more recent models could store more information in a smaller number of parameters, but it's probably going to come down to the size of the model.

The Only exception there is if there is indeed some pattern in modern history that the model is able to learn, but I really doubt that.

What this article really calls to light is that people tend to use these models for things that they're not good at because it's being marketed contrary to what it is.

Humans and LLMs learn in fundamentally different ways, though.

The nuance bit is really interesting, since I feel that nuance arises from these fundamental differences.

Specifically they are completely incapable of unifying information into a self consistent model.

To use an analogy you see a shadow and know its being cast by some object with a definite shape, even if you can't be sure what that shape is. An LLM sees a shadow and its idea of what's casting it is as fuzzy and mutable as the shadow itself.

Funnily enough old school AI from the 70s, like logic engines, possessed a super-human ability for logical self consistancy. A human can hold contradictory beliefs without realizing it, a logic engine is incapable of self-contradiction once all of the facts in its database have been collated. (This is where the SciFi idea of robots like HAL-9000 and Data from Star Trek come from.) However this perfect reasoning ability left logic engines completely unable to deal with contradictory or ambiguous information, as well as logical paradoxes. They were also severely limited by the fact that practically everything they knew had to be explicitly programmed into them. So if you wanted one to be able to hold a conversion in plain English you would have to enter all kinds of information that we know implicitly, like the fact that water makes things wet or that most, but not all, people have two legs. A basically impossible task.

With the rise of machine learning and large artificial neural networks we solved the problem of dealing with implicit, ambiguous, and paradoxical information but in the process completely removed the ability to logically reason.

They’re really just basic input output mechanisms.

I mean, I'd argue they're highly complex I/O mechanisms, which is how you get weird hallucinations that developers can't easily explain.

But expecting cognition out of a graph is like demanding novelty out of a plinko machine. Not only do you get out what you get in, but you get a very statistically well-determined output. That's the whole point. The LLM isn't supposed to be doing high level cognitive extrapolations. It's supposed to be doing statistical aggregates on word association using a natural language schema.

That is accurate, but people who design and distribute the LLMs refer to the process as machine learning and use terms like hallucinations which is the primary cause of the confusion.

Bullshit, they are prediction engines biased from their input. Saying that no one understands what they do is a gross simplification. Do you understand how encoders work? Great! Then there is a common vocabulay! Now we move on to a probability matrix given those inputs, and we get the out put token witch circles through the input layer.

How do you define “understand”?

I just like the analogy of a dashboard with knobs. Input text on one wide output text on the other. "Training" AI is simply letting the knobs adjust themselves based on feedback of the output. AI never "learns" it only produces output based on how the knobs are dialed in. Its not a magic box, its just a lot of settings converting data to new data.

I think they all would have performed significantly better with a degree of context.

Trying to use a large language model like a database is simply A misapplication of the technology.

The real question is if you gave a human an entire library of history. Would they be able to identify relevant paragraphs based on a paragraph that only contains semantic information? The answer is probably not. This is the way that we need to be using these things.

Unfortunately companies like openai really want this to be the next Google because there's so much money to be hired by selling this is a product to businesses who don't care to roll more efficient solutions.

Aren't you glad education is cheap or even free in some places?

Came here to say this. The headline should read "LLMs fail to generate convincing output output on certain topics."

Try asking one to write a sentence that ends with the letter "r", or a poem that rhymes.

They know words as black boxen with weights attached for how likely they are to appear in certain contexts. Prediction happens by comparing the chain of these boxes leading up to the current cursor and using weights and statistics to fill in the next box.

They don't understand that those words are made of letters unless they have been programmed to break each word down into its component letters/syllables. None of them have been programmed to do this because that increases the already astronomical compute and training costs.

About a decade ago I played with an LLM whose markov chain did predictions based on what letter came next instead of what word came next (pretty easy modification of the base code). It was surprisingly comparably good at putting sentences and grammar together when working at the letter-scale. It also was horribly less efficient to train (which is saying something in comparison to word-level prediction LLMs) because it needs to consider many more units (letters vs words) leading up to the current one to maintain the same coherence. If the markov chain was looking at the past 10 words, a word-level prediction has 10 boxes to factor into its calculations and trainings. If those words have an average of 5 letters, then letter-level prediction needs to consider at least 50 boxes to maintain the same awareness of context within a sentence/paragraph. This is a five-fold increase in memory footprint, and an even greater increase in compute time (since most operations are at least of linear order and sometimes more).

That efficiency hit would allow for LLMs to understand sub-word concepts like alphabetization, rhyming, root words, etc. The expense and energy requirements aren't worth this modest expansion of understanding.

Adding a General Purpose Transformer just adds some plasticity to those weights and statistics beyond the markov chain example I use above.

50% is decent, if it had any idea of when it actually was correct or not. But 50% is not very good, when the 50% that’s faulty, results in it going of on a long tangent spewing lies. Lies that are incredibly real looking, takes immense knowledge or huge amounts of time to check.

If you’re well versed enough in the subject to spot the lies, you likely wont get much help from AI. And if you aren’t, well, you’re going to be learning a lot of incorrect information. Or spend ridiculous amount of times fact checking.

Works a bit like that for software developing at the moment. AI is incredibly at spewing out code quickly. But the time won by copying it, is lost looking for errors that are extremely well hidden.

For it to be a totally fair test you'd be testing the worlds population in an open book exam since the model likely has every history book they could find in its training data.

Make sure to read a bit further, they usually get about 10000 attempts. And unfortunately most tests are just about recalling stuff, understanding is not something a text predictor does. It can't actually think.

But that's exactly the problem. Humans with degrees in history can figure out what is an insult and what is a statement of fact a hell of a lot better than an LLM.

LLMs are general purpose models trained on text. The thougbt is they should be able to address anything that can be represented in a textual format.

While you could focus the model by only providing specific types of text, the general notion is they should be able to handle tasks ranging across different domains/disciplines.

Well it has read all we know about it.