Yasa Baig opened Part 2 with the paper that reframes a whole science. The claim in the title — Language Modeling Materializes a World Model of Protein Biology — is bold, and the work behind it, from Biohub's team (the group formerly known as EvolutionaryScale), is the largest application of AI to proteins so far. It is the protein field's version of the moment language models stopped being autocomplete and started looking like they understood the world.
Train a big enough model to predict protein sequences, and it doesn't just memorize biology — it builds an internal model of how biology works. Structure, function, and evolution fall out as side effects of next-token-style prediction over the language of life.
The setup: proteins are a language
A protein is a chain of amino acids — 20 letters, strung in sequences from dozens to thousands long. That chain folds into a 3D shape, and the shape determines what the protein does: catalyze a reaction, carry oxygen, fight a pathogen. For decades, going from sequence to structure to function was the central, brutally hard problem of molecular biology.
The reframing is simple and powerful: treat the amino-acid sequence as text, and train a transformer on it the way you'd train an LLM. Evolution has already written billions of these sentences and filtered them by what survives. A model that learns to predict them well must, somewhere in its weights, capture the grammar of biology — which substitutions are tolerated, which residues pair up, how a fold holds together.
The three pieces
ESM is not one model but a stack, and the talk's clarity came from separating them:
| Component | What it is | What it gives you |
|---|---|---|
| ESMC | Evolutionary Scale Modeling Cambrian — a protein language model trained on ~2.8B sequences spanning the tree of life, including extremophiles and 20,000+ human protein types. | Rich learned representations of any protein sequence — the substrate everything else uses. |
| ESMFold2 | A design and structure engine that turns ESMC representations into atomically-resolved 3D structures of biomolecular complexes. | Sequence → structure, fast, and the ability to design new sequences toward a target shape. |
| ESM Atlas | A navigable map of ~6.8B sequences and ~1.1B predicted structures. | A browsable cartography of protein space — the largest AI-built map of biology to date. |
Why "world model" is the right word
This is the part worth slowing down on. A world model is an internal representation of how a domain behaves, learned implicitly rather than programmed. When an LLM trained only on text can reason about physics or hold a coherent model of a story's characters, we say it built a world model of language's referents. The protein claim is the same shape: ESMC is trained only to model sequences, yet from that single objective it recovers folding rules, functional sites, evolutionary relationships, and structural constraints — none of which it was explicitly taught.
That is the deep point. You don't hand the model the laws of biochemistry. You hand it evolution's output and a prediction objective, and the laws condense in the weights because they are what makes the sequences predictable. Structure prediction, function annotation, and protein design then become queries against that learned world model rather than separate hand-built pipelines.
Alex Rives' framing, channeled in the talk: "the bitter lesson is coming for proteins." General models plus scale plus evolution's data will outrun decades of carefully hand-engineered biophysical method — the same story that played out in vision and language.
What it can actually do
The capabilities the team demonstrated push past prediction into creation:
- Map proteins across the tree of life — place any sequence in the context of all known protein space via the Atlas.
- Predict 3D structure — fold sequences into atomically-resolved complexes, including multi-protein assemblies, not just single chains.
- Design novel binders — generate brand-new protein sequences that bind a chosen target, and have them actually work in wet-lab experiments — the hard, real test that separates a demo from a tool.
- Reason about function and disease — use the representations to connect proteins to biological roles, with an eye toward therapeutics.
Designing a binder that folds and functions in a test tube is the protein equivalent of a generative model producing something genuinely new and correct, not a remix. It's the line between a model that describes biology and one that can do biology.
Scale, and why it matters here
The numbers are the argument. ~2.8B training sequences; an Atlas of ~6.8B sequences and ~1.1B predicted structures. In proteins, scale isn't just bragging rights — coverage across the tree of life is what lets the model generalize to sequences and folds no single organism's biology would teach. Extremophiles, obscure microbial proteins, and the long tail of human proteins all widen the world the model has internalized.
Why this was a YC Paper Club pick
Because it's a template, not just a result. The pattern — take a domain with messy, expensive, hand-built methods; find the domain's "sequences"; train a large model to predict them; discover the domain's rules materialize for free — is the most reliable recipe in modern AI, and it's now jumping disciplines. Proteins today; the talk's subtext is that genomics, chemistry, and materials are next.
For founders the read is direct: the moat is no longer the hand-crafted biophysics simulator. It's the data breadth, the scale, and the engineering to make the world model queryable for design. A model that can propose working binders compresses the early, slow, expensive part of drug discovery into a search over a learned map.
The honest caveats
A world model of protein biology is still a model. Predicted structures are predictions; designed binders must be validated in the lab, and many won't fold or function. The Atlas's billion structures are computed, not crystallized. The value is in massively narrowing the search — proposing the few candidates worth the expensive experiment — not in replacing the experiment. As with LLMs, the failure modes are confident and plausible-looking, which is exactly why wet-lab validation stays in the loop.
How it differs from AlphaFold
The obvious comparison is DeepMind's AlphaFold, which won a Nobel Prize for predicting protein structure from sequence. The difference is the framing, and it matters. AlphaFold is a brilliant structure predictor — a specialized model aimed at one task, leaning on evolutionary alignments (multiple-sequence alignments) as input. ESM's bet is broader: train a general language model on raw sequences, and structure prediction becomes one capability that drops out of a single learned representation, alongside function, design, and navigation of protein space.
That's the "world model" distinction in practice. AlphaFold answers "what shape is this?" ESM aims to be the substrate you query for many questions — including the generative one AlphaFold wasn't built for: "design me a sequence that does X." It's the difference between a great task model and a foundation model, and it's why the talk frames this as the bitter lesson arriving: the general, scaled approach starting to subsume the specialized one.
The data flywheel, and what's next
The reason to care about the Atlas being navigable is that it turns the model into infrastructure. Once 6.8B sequences and 1.1B structures are mapped into one representation space, you can search biology by similarity, cluster unknown proteins next to characterized ones, and generate candidates in regions of protein space evolution never explored. Each designed-and-validated binder is also new labeled data, feeding back into a flywheel.
The honest frontier: the same recipe is being eyed for the rest of molecular biology — DNA and RNA, gene regulation, whole-cell models. Proteins were the first domain with enough clean "sequences" and a sharp validation test (does the binder work?). The talk's real argument is that this is a method, and proteins are just where it landed first.
The takeaway
If language models materialize world models of text, and protein language models materialize world models of biology, the lesson generalizes: scale a predictive model on a domain's native sequences and the domain's laws will condense inside it. Proteins are the proof of concept that the recipe leaves the world of text behind. That's why this was the paper to open Part 2 — it's the one that makes you re-examine which science is next.