LeWorldModel vs V-JEPA: What Actually Changed
LeWorldModel removes masking, EMA, and stop-gradient from V-JEPA, trains end-to-end from pixels in hours on one GPU, and plans 48x faster. Here’s the full architecture and what it learns differently.
Self-Supervised Learning
Posts
SIGReg: The Anti-Collapse Trick That Actually Has a Proof
V-JEPA prevents collapse with heuristics. SIGReg prevents it with a theorem. Here’s what isotropic Gaussian means, why variance isn’t enough, and how the Cramér-Wold trick makes it cheap.
V-JEPA Internals: Building My Understanding
Going back through V-JEPA’s encoder, predictor, and target encoder in detail — the mechanics of masking, EMA, stop-gradient, and why collapse prevention is harder than it looks.