LeCun against the LLM
Why one of deep learning's founders argues the current paradigm is an off-ramp, what he proposes instead, and the mathematics underneath the bet. Companion to Attention Is All You Need. Editorial synthesis of documented positions.
The stakes, at a glance
While the industry pours capital into ever-larger language models, Yann LeCun has spent years arguing — in papers, lectures and combative posts — that autoregressive LLMs are a local maximum: spectacularly useful, and structurally incapable of becoming the human-level intelligence their builders promise. In late 2025 he matched conviction with career, leaving his post as Meta's Chief AI Scientist to found a startup dedicated to world models — betting the house, at sixty-five, against the very wave his own work helped unleash.
The critic
LeCun's standing is what makes the critique bite. He built convolutional networks in the late 1980s, read ZIP codes with them at Bell Labs when neural nets were unfashionable, kept the faith through the AI winter, and shared the 2018 Turing Award with Hinton and Bengio for deep learning itself. He has also long championed the position that self-supervised learning is the substance of intelligence — his famous cake analogy: self-supervision is the cake, supervision the icing, reinforcement learning the cherry. The LLM debate is that analogy weaponized: LLMs, in his view, are icing engines.
The critique
Exponential drift
An autoregressive model picks one token at a time; if each step has even a small chance of leaving the set of acceptable continuations, the probability of staying correct decays geometrically with length. In LeCun's slides: autoregressive LLMs are 'doomed' — errors compound and there is no mechanism to steer back.
The four missing capacities
His standard list of what any cat has and LLMs lack: understanding of the physical world, persistent memory, reasoning, and planning. Text prediction, he argues, buys fluent recall of humanity's notes — not the machinery that produced them.
The grounding gap
A four-year-old has absorbed vastly more data through vision than the largest LLM has read — his back-of-envelope puts a toddler's visual stream at orders of magnitude above the text corpus. Language is a thin, lossy channel; most of what matters about the world was never written down.
Fixed compute, no System 2
A vanilla LLM spends the same computation on every token — it cannot think longer about a harder problem. Human cognition deliberates: it simulates, evaluates, backtracks. Reactive next-token prediction is System 1 wearing System 2's clothes.
Probability in the wrong space
Generative models in continuous, high-dimensional domains (video, the world) must spread probability mass over every unpredictable detail — the texture of leaves, the exact pixels of the future. LeCun's conclusion: stop predicting the world in its raw space, and stop insisting on normalized probabilities at all.
The bet: world models
The constructive program is laid out in his 2022 position paper, A Path Towards Autonomous Machine Intelligence, and in the JEPA family of systems built at Meta since (I-JEPA for images, V-JEPA for video, V-JEPA 2 coupled to robot planning). The through-line: intelligence is the ability to predict the consequences of actions in an internal model of the world, learned from watching — the way infants learn object permanence and intuitive physics before language.
World models
An internal simulator learned by self-supervision from sensory streams: given a state and an imagined action, predict what happens next. Prediction error is the teacher — no labels, no reward, just the world contradicting you.
JEPA — predict in representation space
The Joint-Embedding Predictive Architecture predicts the abstract representation of the missing or future piece, not its pixels. Unpredictable detail is discarded by design; capacity is spent on what is predictable — the physics, not the leaf texture.
Energy, not probability
Score compatibility with a scalar energy E(x, y): low on things that go together, high elsewhere. No normalization, no partition function — inference becomes optimization, finding the y (or plan) that minimizes energy.
Anti-collapse regularization
Joint-embedding training has a trivial solution: map everything to the same point. VICReg — variance, invariance, covariance regularization — holds the space open non-contrastively: keep each dimension's variance up, decorrelate dimensions, pull matched views together.
Planning as optimal control
With a world model, acting becomes model-predictive control: roll candidate action sequences through the model, score outcomes with a cost module, optimize, execute the first step, repeat. Reinforcement learning's trial-and-error becomes the fallback, not the engine.
A modular agent
His 2022 blueprint assembles perception, world model, cost, memory and an actor under a 'configurator' — a cognitive architecture with hierarchy in time, where H-JEPA plans abstractly over long horizons and concretely over short ones.
The mathematics of the bet
- Energy-based learning. Replace p(y|x) with a scalar energy E(x, y) trained to be low on compatible pairs. The gain is exactly what probabilistic modeling struggles with in continuous spaces: no partition function Z = ∫e^(−E) to normalize, hence no intractable integral over all possible futures. Multiple plausible outcomes are simply multiple valleys of the landscape.
- Latent variables for uncertainty. The unpredictable part of the future is absorbed by a latent z: the model predicts well for some z, and the free energy F(x) = min over z of E(x, y, z) (or a softened version of it) scores predictions without committing to one future. Uncertainty is handled by geometry, not by spreading probability over pixels.
- Collapse-prevention as the training objective. Joint-embedding methods die by shortcut: encoders that output a constant achieve zero prediction error. The non-contrastive fixes — VICReg's variance floor per dimension, covariance off-diagonal penalties, invariance across views — are linear-algebra constraints that keep the representation full-rank. This replaces the contrastive need for enormous batches of negatives.
- Optimal control reclaimed. Planning is posed as minimizing a cost over action sequences subject to latent dynamics — classical model-predictive control, with the model now learned. The mathematics of trajectory optimization, receding horizons and hierarchical time-scales (H-JEPA) does the work that chain-of-thought prompting gestures at.
- The wager in one sentence. Abandon maximum likelihood over raw signals in favor of shaping energy landscapes over learned representations — trading probabilistic completeness for tractability in exactly the places the world is high-dimensional and continuous.
The critique of the critique
- The drift argument leaks. Errors need not compound irreversibly: models can self-correct in context, and the empirical success of long, coherent generations — and of reasoning models that spend variable compute deliberating before answering — has blunted the “doomed” argument's force since 2023.
- The bitter lesson cuts both ways. LLM proponents note that scale plus general methods keep beating architected cognition; LeCun's modular agent, they argue, repeats the old mistake of building in what learning would discover.
- JEPA has not had its GPT moment. Strong results in vision and robot manipulation, but nothing yet rivaling language models' breadth; meanwhile generative world models (Dreamer, Genie, Sora — whose pixel-space approach LeCun publicly disputed) pursue the same goal by the route he rejects.
- Convergence is plausible. Frontier systems already bolt deliberation, memory and tool-use onto LLM cores; the future may look less like either camp's purity and more like hybrids — in which case the fight is over which component is the cake.
State of play
As of early 2026 the question is genuinely open. The LLM camp holds the products, the revenue and the reasoning-model momentum; LeCun holds a coherent theory of why that momentum plateaus, a decade of self-supervised results, and now a company committed to proving it. His own framing is falsifiable enough: if machines reach robust physical common sense, it will be through architectures that predict in representation space and plan through learned world models. Either the scaling curves keep absorbing his objections, or the field takes the exit he has been signposting — and this page will need a sequel either way.