The thesis

Generative prediction in observation space is the wrong objective, because most of the bits in a future observation are unpredictable and irrelevant. A joint embedding predictive architecture instead encodes both \(x\) and \(y\) and predicts in that latent space:

\[ \min_{\theta}\; D\big(s_y(y),\, \mathrm{Pred}_\theta(s_x(x), z)\big) \quad \text{subject to a capacity constraint on } s_y \]

(1)

The capacity constraint is doing the real work: without it the encoder collapses to a constant and prediction becomes trivial.

What would make it right

• Hierarchical JEPAs learn representations at multiple timescales, enabling planning that is coarse far ahead and fine nearby.
• Energy-based inference at deployment substitutes for the explicit search that model-based reinforcement learning performs badly.
• Intrinsic cost modules make objectives inspectable rather than emergent from a scalar reward.

What would make it wrong

If a system trained purely on next-token prediction over a sufficiently broad corpus develops usable predictive models of physical and social dynamics — with planning behaviour that holds up under distribution shift — then the architectural separation this proposal insists on is not necessary, only convenient. That is a live empirical question, not a settled one, which is why the document is filed here as active rather than superseded.

Open questions this leaves

• No account of how the configurator is trained, which is where task generality would have to come from.
• The capacity constraint is specified as a desideratum rather than a mechanism.
• Long-horizon credit assignment is deferred to the hierarchy without an argument that hierarchy is sufficient — see OP-001.