The idea that LLMs are just like how the brain works, except limited by running in a CPU, comes from software engineers - not neuroscientists.
Although there are many analogies that could be made between how CPUs do work and how the brain integrates information, they're actually fundamentally different and use completely different logic.
You could, theoretically, create a computing language to work using neurons. And therefore you could also train machine learning algorithms. But that's like using calculators to sum 2+2 by buying 4 calculators and putting them all together, rather than actually using what a calculator does to get the result, if you get what I mean.