Speculative Speculative Decoding
Speculative decoding has a hidden serial step of its own. SSD (Saguaro) parallelizes it — the draft model pre-guesses verification outcomes — for 30% over speculative baselines and up to 5× over plain decoding.
Three editions of research talks, rebuilt as approachable technical stories. Start with the question that interests you; every note explains the intuition, system design, evidence, limits, and original sources.
From faster decoding to world models and infinite-compute pre-training: five ways to rethink what a model learns and how it spends compute.
Speculative decoding has a hidden serial step of its own. SSD (Saguaro) parallelizes it — the draft model pre-guesses verification outcomes — for 30% over speculative baselines and up to 5× over plain decoding.
D-MPC learns both a multi-step action proposal and a multi-step dynamics model as diffusion models, then plans online. Matches SOTA offline RL on D4RL and adapts to new rewards and dynamics at run time.
JEPAs collapse without elaborate tricks. LeWM trains end-to-end from raw pixels with just two loss terms and one hyperparameter — 15M params, single GPU, 48× faster planning than foundation-model baselines.
Benign overfitting, double descent, overparametrization — Andrew Gordon Wilson argues none are unique to neural nets. Soft inductive biases, PAC-Bayes, and what actually does make deep learning special.
Compute grows faster than text. When data is fixed and compute is free, how do you pre-train? Regularization 30× standard, parameter and ensemble scaling laws, distillation. Written deeper for PhD readers.
Biology, self-play, voice tools, and formal proof meet at one question: how can an AI learn from feedback that is useful, grounded, and difficult to fake?
Train a language model on 2.8B protein sequences and a world model of biology materializes — predicting structure and designing binders that work in the lab. ESM, from Biohub/EvolutionaryScale. The bitter lesson, arriving for proteins.
Self-play plateaus because the problem-generator reward-hacks into hard-but-useless problems. SGS adds a Guide that scores problems for relevance and cleanliness — and a 7B model beats a 671B baseline.
Voice assistants are fast-but-hallucinating or grounded-but-laggy. Stream RAG fires the tool call while you're still talking — +200% relative QA accuracy, 20% less tool latency. Speculative decoding's instinct, for tool use.
Machine-checked formal proofs reshape math, AI, and scientific computing. The key for AI: a proof checker is an incorruptible reward — you can't hallucinate a proof past it. The trust substrate under self-play and AlphaProof.
Begin with why chips and data centers specialize, then follow intelligence through GPU fabric, local devices, generated kernels, heterogeneous serving, and thousands of reinforcement-learning worlds.
Why today’s token demand can finally repay the cost of specialized algorithms, kernels, chips, and data centers.
Three trade-offs—transfer path, overlap schedule, and abstraction overhead—make multi-GPU kernels easier to reason about.
Capability, power, and total energy turn local-versus-cloud inference into a measurable routing decision.
Competitive AI-written kernels expose both startling optimizations and reward hacks that harden the evaluation.
Arithmetic intensity explains why prefill, decode, attention, MoE, and speculative drafting may want different systems.
A data-oriented ECS batches complete dynamic game worlds into one sustained GPU workload.