I mean sure, an imaginary LLM that exceeds the fundamental limitations of the technology could be convincing, but then that’s not an LLM. LLMs are statistical models, they don’t know anything. They use stats calculated from training data to guess what token should follow another. Hallucinations cannot be eliminated because that would require it to be capable of knowing things and then it would have to be able to error check itself rationally. In other words, it would have to be intelligent.

An AGI wouldn’t need to read every book because it can build on the knowledge it already has to draw new conclusions it wasn’t “taught.”

Also, an AGI would be able to keep a consistent narrative regardless of the amount of data or context it has, because it would be able to create an internal model of what is happening and selectively remember the most important things more so than things that are inconsequential (not to mention assess what’s important and what can be forgotten to shed processing overhead), all things a human does instinctively when given more information than your brain can immediately handle. Meanwhile, an LLM is totally dependent on how much context it actually has bufferered, and giving it too much information will literally push all the old information out of its context, never to be recalled again. It has no ability to determine what’s worth keeping and that’s not, only what’s more or less recent. I’ve personally noticed this especially with smaller locally run LLMs with very limited context windows. If I begin troubleshooting some Linux issue using it, I have to be careful with how much of a log I paste into the prompt, because if I paste too much, it will literally forget why I pasted the log in the first place. This is most obvious with Deepseek and other reasoning models because it will actually start trying to figure out why it was given that input when “thinking,” but it’s a problem with any context based model because that’s its only active memory. I think the reason this happens so obviously when you paste too much in a single prompt and less so when having a conversation with smaller prompts is because it also has its previous outputs in its context, so while it might have forgotten the very first prompt and response, it repeats the information enough times in subsequent prompts to keep it in its more recent context (ever notice how verbose AI tends to be? That could potentially be a mitigation strategy). Meanwhile, when you give it a very large prompt as big or bigger than its context window, it completely overwrites the previous responses, leaving no hints to what was there before.

That’s like asking what’s the difference between a chef who has memorized every recipe in the world and a chef who can actually cook. One is a database and the other has understanding.

The LLM you’re describing is just a highly sophisticated autocomplete. It has read every book, so it can perfectly mimic the syntax of human thought including the words, the emotional descriptions, and the moral arguments. It can put on a flawless textual facade. But it has no internal experience. It has never burned its hand on a stove, felt betrayal, or tried to build a chair and had it collapse underneath it.

AGI implies a world model which is an internal, causal understanding of how reality works, which we build through continous interaction with it. If we get AGI, then it’s likely going to come from robotics. A robot learns that gravity is a real, it learns that “heavy” isn’t an abstract concept but a physical property that changes how you move. It has to interact with its environment, and develop a predictive model that allows it to accomplish its tasks effectively.

This embodiment creates a feedback loop LLMs completely lack: action -> consequence -> learning -> updated model. An LLM can infer from the past, but an AGI would reason about the future because it operates with the same fundamental rules we do. Your super-LLM is just a library of human ghosts. A real AGI would be another entity in the world.

Such an LLM would have the “knowledge” of almost every

Most human knowledge is not written down. Your premise is flawed to the core.

Uh, simple.

Clear your chat history, and see if it remembers anything.

LLMs are, by current defitions, static. They’re like clones you take out of cryostasis every time you hit enter; nothing you say has an impact on them. Meanwhile, the ‘memory’ and thinking of a true AGI are not seperable; it has a state that changes with time, and everything it experiences impacts its output.

…There are a ton of other differences. Transformers models trained with glorified linear regression are about a million miles away from AGI, but this one thing is an easy one to test right now. It’d work as an LLM vs human test too.

You’re sitting on the Chinese Translator problem, and to some extent the basis of the Turing Test (mostly the translator problem).

https://en.wikipedia.org/wiki/Chinese_room

Knowledge != Intelligence

Regurgitating things you’ve read is only interpolative. You’re only able to reply with things you’ve seen before, never new things. Intelligence is extrapolative. You’re able to generate new ideas or material beyond what has been done before.

So far, the LLM world remains interpolative, even if it reads everything created by others before it.

The “AGI” in this example would have way less knowledge than the LLM in this example. Just ask a few trivia questions, and you’ll be able to tell them apart.

LLMs do not inferr.

An LLM trained on all books ever written would probably take romance novels, books by flat earthers, or even “Atlas Shrugged” as truth as much as current AIs consider all stack overflow comments to contain useful and accurate information.

Thinking about it, your questions comes back to the very first and original instance of a computer and the question interested people asked about it:

If you put into the machine wrong figures, will the right answer come out?

Now if we allow ourselves the illusion of assuming that an AGI could exist, and that it can actually learn by itself in a similar way as humans, than just that quote above leads us to these two truths:

• LLMs cannot help being stupid, they just do not know any better.
• AGIs will probably be idiots, just like the humans asking the above question, but there is at least a chance that they will not.

An agi won’t need large knowledge banks to function.

If it has intelligence it should be able to “create” or “invent” in the absence of data knowledge.

Quality may vary. It would depend on some different factors.

But it should be able to create novel solutions. Just like a lot of animals can.

“Play” is often considered a sign of “sentience” in animals.

An agi that participated in behaviors for fun with no other advantage could possibly be a marker.

This assumes emotional components to agi. Which I personally believe is 100% necessary for sentience.

I also agree with you tho. That it’s no where near to existing and I personally think it’s not possible.

An AGI is just a software that can do anything a human can (more or less). Something that could use blender for example or do proper research.

Llms can already already understand images and other data formats, they might be a pathway to agi but most think they have too many constraints for it to reach that far.