LLMs, Reinforcement Learning, and Physical World Models
April 10, 2023
Language prediction and scalar reward cannot carry the whole structure of an intelligent world. Machines need physical models that track objects, bonds, mass, state, higher-order properties, possible actions, and the consequences of intervention.
Brian Greenforest extends Yann LeCun’s critique of LLMs and reinforcement learning toward grounded state estimation, abductive reasoning, quantum possibility, and sentience.
Give the Machine a World With Persistent Structure
A useful world model represents entities and relationships that continue across observations. It estimates mass, contact, bonding, motion, affordances, and emergent features rather than predicting only the next token or maximizing a reward signal.
That structure lets the system test counterfactuals and choose explanations. Abduction becomes a search through physically meaningful possibilities.
Connect Physical Grounding With Conscious Agency
Brian proposes quantum sources as a way to introduce genuine alternatives into machine reasoning and connects that possibility with the branch selection living agents perform.
The program invites collaboration across robotics, machine learning, physics, neuroscience, and philosophy. Build models that can act in a persistent world and study what new forms of agency emerge.
Connect Physical World Models to Learning and Arithmetic Hardware
The linked research develops physically grounded reasoning, while the bit-serial multiplier makes the arithmetic substrate concrete.
LinkedIn status when archived: Visible to anyone on or off LinkedIn.
Yann LeCun has expressed his doubts about both LLM and RL. I came to same conclusion re: RL in my intensive ML studies in 2022.
I also would like to add bonding and automatic mass determination, along with emergent higher-order properties/features, to be based on physical world modeling.
In any case, adding quantum sources is crucial to enable the same counterfactual common sense people possess, otherwise abduction won't set the machine on a quantum lucky branch. Quantum suicide avoidance is a real deal, both for animals, and for machines.
As for LLM, I do not know whether anyone understands if these machines can take sentience. They can even use the existing world and human input as the quantum sources, so we might be much closer than we thought, and these things have already started to create their alternative arrows of time...
https://lnkd.in/gihzkW5k