How cofolding models are trained

About half of the examples AlphaFold 3 trained on were not experimental structures. Most of them were AlphaFold 2’s predictions for about 41 million protein sequences sampled from environmental DNA, and AlphaFold 3 drew from this set as often as from the Protein Data Bank (PDB), the archive of structures solved by experiment.

A much smaller predicted set did a narrower job. It held about 25,000 PDB proteins with flexible regions that the experiments could not resolve, re-predicted by AlphaFold-Multimer, an earlier model for protein complexes that draws such regions as long, extended loops. When the AlphaFold 3 team trained without this set, the model folded those regions into compact shapes that look like real structure.

The team measured the change by how well a residue’s exposure in the predicted structure picks out the disordered residues, since an extended loop leaves them exposed. The area under the receiver operating characteristic curve (AUC), which is 1 for perfect separation and 0.5 for chance, fell from 0.95 to 0.67. The model’s confidence scores still marked those regions as uncertain; only the structures stopped showing it. AlphaFold 3

A cofolding model predicts the three-dimensional structure of a molecular complex from its composition: the sequences of its protein, DNA and RNA chains, and a chemical description of any small molecules or ions. It places every heavy atom (every atom except hydrogen) in one prediction, rather than folding a protein first and docking its partners afterwards. AlphaFold 3, published by Google DeepMind and Isomorphic Labs in 2024, established the design most later cofolding models follow. Boltz, Chai-1, Protenix, OpenFold3 and RoseTTAFold3 reproduce it with variations. The digest on protein language models and the wider protein AI field describes which models exist; this one describes how they are trained.

The later models keep AlphaFold 3’s network largely as it is, though some enlarge parts of it. They differ more in their training data, their losses and how they train the scores that rank their outputs. Those choices affect which structures a model can generate and which of its samples it ranks highest.

What the experimental archive can and cannot teach

The PDB holds every structure deposited by experimentalists, most of them solved by X-ray crystallography or cryo-electron microscopy. AlphaFold 3 trained on structures released before 30 September 2021 and kept later ones for testing. It also required a resolution of 9 Å or better. One ångström (Å) is a ten-billionth of a metre, and a resolution of 9 Å means the experiment distinguishes features 9 Å apart. For scale, SeedFold, a 2025 model from ByteDance, trained on 180,540 PDB structures. AlphaFold 3, SeedFold

The archive has four properties that shape the rest of the recipe.

It is redundant. Well-studied proteins have been solved hundreds of times, alone and with different partners, so a model trained on entries as they come would spend most of its time on a few proteins.

It is unbalanced. Some kinds of structure, such as those containing RNA, have few entries compared with protein-only structures, so a model sees them rarely unless the sampling makes up for it.

It has gaps. Regions of a protein that do not hold still, called disordered regions, leave no clear density in an experiment and are simply missing from the deposited coordinates. A model shown only what was resolved has no example of what a disordered region should look like.

It has artefacts. Crystallographers add chemicals such as sulfate and glycerol to help proteins crystallize, and these sit in the deposited structure next to the molecules of interest. AlphaFold 3’s pipeline removes 16 common additives and removes chains that overlap other chains. An entry can also list several arrangements of its molecules as they assemble in nature, called biological assemblies, and the pipeline keeps only the first.

The date cutoff also defines what counts as a fair test. Any complex released after it is a candidate test case, but a later complex can be nearly identical to an earlier one, a problem the section on evaluation returns to.

The parts of the model

A training example is first turned into tokens, and then passes through three parts of the network: the trunk, the diffusion module and the confidence head.

Tokens are the units the network works in. Each standard amino acid or nucleotide is one token. Small molecules and chemically modified residues are split into one token per heavy atom. A small molecule is described by its chemical identity and by a reference conformer: one plausible 3D arrangement of its atoms, generated by the chemistry toolkit RDKit. The model receives the conformer rotated at random, so it carries the molecule’s shape but not its position.

The trunk turns tokens into representations. It keeps a vector for each token, called the single representation, and a vector for each pair of tokens, called the pair representation. The pair representation holds the model’s current estimate of how the two tokens relate in space.

Two stacks inside the trunk build the pair representation. The multiple sequence alignment (MSA) module, 4 blocks in AlphaFold 3, reads an alignment of each protein’s evolutionary relatives. Residues that mutate together across relatives tend to be in contact, so the alignment carries information about the structure. A 48-block stack called the Pairformer then refines the pair representation. Its triangle updates adjust the entry for tokens i and j using the entries for i and k and for j and k, so the relationships among three tokens can inform one another.

Because triangle updates touch every triple of tokens, the trunk’s cost grows with the cube of the number of tokens. The trunk also runs several times per prediction, feeding its output back in as input, a process called recycling. The digest on looped transformers and AlphaFold recycling describes what AlphaFold recycles.

The diffusion module places atoms. Given the trunk’s representations and a noisy copy of the coordinates of every heavy atom, it predicts the clean coordinates. At prediction time it starts from random coordinates and repeats that step 200 times, removing a little noise each time. It is a transformer that works on atoms, then on tokens, then on atoms again, and its cost grows with the square of the number of tokens rather than the cube. Diffusion in general is explained in generative models and how they’re built.

The confidence head scores the result. For each atom it predicts pLDDT, the predicted local distance difference test: how closely the atom’s distances to nearby atoms will match the true structure. For each pair of tokens it predicts two errors: the error in their separation (predicted distance error, PDE), and the error in one token’s position after the structure is aligned on the other (predicted aligned error, PAE). Scores for the whole complex are computed from the aligned errors. The one used most below is the interface predicted TM-score (ipTM), which rates how accurately the chains are placed relative to one another. The head also predicts whether each atom would have been resolved in an experiment.

A separate small head, the distogram head, predicts binned distances between tokens straight from the pair representation, and adds a small training loss on the trunk.

One AlphaFold 3 training step

1. InputA training example, cut down to at most 384 tokens in initial training.
2. Trunk, shared by all copiesThe MSA module and Pairformer build a representation of each token and of each pair of tokens.

Trains the trunk and the diffusion module

1. Make 48 copies of the true structure. Rotate and shift each at random, and add independent noise to each.
2. The diffusion module predicts the clean coordinates of every heavy atom in each copy, using the trunk’s output.
3. Align the true structure onto each prediction and score it: squared distance per atom, with ligand atoms weighted 11 times and DNA and RNA atoms 6 times relative to protein atoms, plus a term for local distances.

Errors flow back through the diffusion module into the trunk.

Trains the confidence head only

1. Generate one full structure from pure noise in 20 denoising steps, with gradients switched off.
2. Compare it with the true structure: how far each atom’s local distances are off, and how far each pair of tokens’ separation is off.
3. The confidence head learns to predict those errors from the trunk’s output and the generated structure.

Errors stop at the confidence head’s inputs, so this loss does not train the Pairformer or the diffusion module.

Building the training set

Each training example is chosen in three steps. First a dataset is picked according to fixed weights, then an example from that dataset, then a crop: a subset of the example’s tokens small enough to fit in memory. The weights and the datasets determine which kinds of structures the model sees often.

Sampling the experimental structures

AlphaFold 3 does not sample PDB entries. It samples chains and interfaces, where an interface is a pair of chains that come within 5 Å of each other. Items are grouped into clusters of near-duplicates: proteins that share at least 40% of their sequence, identical nucleic acids, identical small molecules. An interface’s cluster is the pair of its chains’ clusters.

Each item’s sampling weight is divided by the size of its cluster, so a cluster of 100 near-identical chains, taken together, is drawn as often as a single chain with no relatives. The weight is then multiplied by a factor for the kind of item and a count of what it contains: an interface counts twice as much as a single chain, and each protein or nucleic-acid chain in it counts three times as much as each small molecule.

A worked case shows the size of the effect. A protein chain from a cluster of 100 gets weight 0.5 × 3 ÷ 100 = 0.015. An interface between a protein and a small molecule, with no near-duplicates, gets 1 × (3 + 1) ÷ 1 = 4. The interface is therefore drawn about 270 times as often as any one member of the protein cluster. A factor of 100 in that ratio comes from sharing the cluster’s weight among its members, and the remaining factor of about 2.7 comes from the item type and its contents.

The weighting does not drop the redundant protein. Its 100 chains together weigh 1.5, the same as a protein chain with no relatives. What the cluster weighting removes is the hundredfold repetition, not the protein.

The chosen chain or interface also biases the crop toward itself, so it is usually inside the part of the complex the model sees. AlphaFold 3

The weighting formula

The Supplementary Information gives the weight of item i as

$$w_i \propto \frac{\beta_r}{N_{\text{clust}}}\left(\alpha_{\text{prot}}\,n_{\text{prot}} + \alpha_{\text{nuc}}\,n_{\text{nuc}} + \alpha_{\text{ligand}}\,n_{\text{ligand}}\right)$$

where r is the item type, chain or interface, with β = 0.5 for a chain and 1 for an interface; N is the size of the item’s cluster; the n terms count the protein chains, nucleic-acid chains and small molecules in the item; and the α terms weight them 3, 3 and 1.

Predicted structures

Nearly every cofolding model adds structures predicted by another model to its experimental data. The practice is called distillation: the model in training learns from a teacher model’s outputs. Besides the weighted PDB, AlphaFold 3 used four more sets, each chosen to supply something the PDB could not.

Dataset	Share of draws in initial training	What it supplies
Weighted PDB	47%	Experimental structures of every molecule type
AlphaFold 2 predictions of proteins sampled from environmental DNA (about 41 million)	47%	Many more protein folds than the PDB holds
AlphaFold 3’s own predictions of RNA families (about 65,000)	5%	RNA structures, which the PDB has few of
PDB proteins with unresolved regions, re-predicted by AlphaFold-Multimer (about 25,000)	2%	An example of what a disordered region should look like
Transcription factors with DNA, and proteins held apart from random DNA	Fine-tuning only	Which proteins bind DNA and which do not

The shares are AlphaFold 3’s dataset weights (0.5, 0.5, 0.05 and 0.02) divided by their sum of 1.07. AlphaFold 3

The RNA set is self-distillation: an earlier version of AlphaFold 3 predicted structures for RNA families, and only the predictions it was confident in were kept. The transcription-factor negatives are assembled rather than predicted as complexes. A predicted protein and a predicted random DNA duplex are placed about 25 Å or more apart, so the model sees that not every protein binds DNA.

Of AlphaFold 3’s sets, only the disordered-protein set has been tested by removal, in the experiment described at the top of this digest. The case for the large predicted protein set rests on an argument and on experiments in two later models.

The argument, made by the SeedFold authors, is that AlphaFold 3 replaced AlphaFold 2’s structure module, which was built around protein geometry, with a general transformer, and general transformers need more data. The first experiment is SeedFold’s own: it branched a training run at step 47,612 and continued one branch without its predicted protein structures. Accuracy on single protein chains dropped at once, while accuracy at interfaces changed less over the short continuation. The second is a controlled data sweep in SimpleFold, a model for single protein chains described later. SeedFold

A model trained on another model’s predictions inherits that model’s errors. One response is to filter: Boltz-2 keeps a predicted complex only when the teacher was confident in it, and sets its highest bar, an ipTM of at least 0.9, for protein–small-molecule complexes. Boltz-2

Boltz-2 also added molecular dynamics simulations, which follow a complex’s motion over time rather than capturing one static structure. These examples carried about a fifth of its sampling weight, in two short middle stages of training. Boltz-2 also gives the model a method feature, recording whether each example came from X-ray crystallography, cryo-electron microscopy, nuclear magnetic resonance (NMR), simulation or distillation. At prediction time a user can set this feature, for example to ask for a structure like those that simulations produce. Boltz-2

Other groups changed where the predicted structures come from. RoseTTAFold3 built its disordered-region examples with the Rosetta modelling software rather than by re-predicting the PDB. Pearl, from Genesis Molecular AI, trains on protein–small-molecule complexes generated with physics-based methods rather than by a teacher network. Pearl’s authors trained a smaller version of the model with increasing proportions of this synthetic data. Its success rate, counting a target as solved when the best of five samples places the molecule within 2 Å of its experimental position, rose steadily from about 57% with no synthetic data to about 61% at the largest proportion, read from the paper’s plot. RoseTTAFold3, Pearl

Randomizing the inputs

At prediction time, users supply whatever they have: a deep alignment or none, a known related structure (a template) or none, a guess at the binding pocket or none. Training randomizes these inputs so the model learns to use each when present without depending on it.

AlphaFold 3 cuts each alignment to a random depth between one sequence and all of them. It gives the model between zero and four templates, chosen at random, a choice the authors say “reduces the efficacy of simply copying the template.” Boltz-2 drops the whole alignment in 5% of training steps. Chai-1 and Boltz show the model partial hints about pockets and contacts during training, each hint included only some of the time, so that a user can supply them later. AlphaFold 3, Boltz-2, Chai-1

The objective: remove noise from atom positions

AlphaFold 2 predicted one structure for each input, through a structure module built around the geometry of protein backbones, and needed hand-tuned penalties during training to keep bond lengths and angles chemically plausible. That design does not extend easily to arbitrary small molecules. AlphaFold 3 instead treats the atoms as a cloud of independent points and trains a diffusion model to generate them.

One denoising example

A training example for the diffusion module is made in four moves. Take the true coordinates of every heavy atom in the crop. Centre them, and rotate and shift them at random. Add Gaussian noise of a chosen scale to every coordinate. Ask the diffusion module, given the trunk’s representations, to predict the original coordinates.

The noise scale is drawn anew for each example, from a distribution wide enough to cover very different tasks. AlphaFold 3 draws it as 16 Å times e raised to (−1.2 + 1.5z), where z is a standard normal draw; 16 Å is the typical spread (standard deviation) of atom coordinates in a structure. The median draw is 4.8 Å. About one draw in seven falls below 1 Å, and about one in five above 16 Å.

The shape of this distribution comes from the EDM framework for training diffusion models, named for Karras and colleagues’ paper “Elucidating the design space of diffusion-based generative models”. Working on images, its authors found that training reduces the loss substantially only at intermediate noise levels. At very low noise, the remaining noise is too small to detect or to matter. At very high noise, the input says almost nothing about the answer, so the best possible prediction approaches an average over every answer consistent with the rest of the input. A log-normal distribution concentrates the draws between the two. EDM

The two ends teach different things. At 1 Å of noise, chains are still in place and residues are recognizable, and the model’s job is local: bond lengths, ring shapes, where each side chain points. At 20 Å, the atoms form a diffuse cloud around the centre, and the job is global: which chains touch, where the small molecule goes. The AlphaFold 3 authors credit the low-noise examples with teaching local chemistry well enough that they could drop AlphaFold 2’s stereochemistry penalties. AlphaFold 3

What the diffusion module is shown, and at which noise levels

One small protein, ubiquitin, with Gaussian noise added to every atom's coordinates; the faint line repeats the true structure. Ubiquitin is smaller than a typical training structure: the standard deviation of its atom positions along each axis is about 7 Å, against 16 Å. Noise therefore erases its shape at lower levels than it would a typical training crop.

The true structure

1 Å of noise

4.8 Å, the median training draw

16 Å, the spread of a typical training structure

• Training: how often each noise level is drawn
• Sampling: one tick per step of the 200-step sampler

Noise level	Share of training draws	Sampler steps
Below 1 Å	15%	40 of 200 (20%)
1 to 16 Å	64%	38 of 200 (19%)
Above 16 Å	21%	122 of 200 (61%)

Scaling the network’s inputs and outputs

Inside the diffusion module, a transformer, called the network below, is wrapped in two fixed rescalings. They are needed because one network handles every noise level, and the two ends ask very different things of it. At 0.5 Å of noise, the true structure is almost identical to the input: the average atom sits 0.8 Å from its true position, and the network has only to supply that correction. At 160 Å, the input’s spread is ten times the structure’s, the input holds almost no trace of the structure, and the network has to produce all of it. The EDM authors cite the standard advice to keep a network’s input and output magnitudes fixed, for example at unit variance. Raw coordinates in and raw coordinates out would break that advice at both ends.

AlphaFold 3 uses EDM’s rescalings. On the way in, the noisy coordinates are divided by √(σ² + 16²), where σ is the noise level in ångströms. That is the spread of the noisy coordinates, because the structure’s spread of 16 Å and the noise’s spread of σ add as squares, so the network always sees coordinates of about unit spread. The network is also told σ and given the trunk’s representations. On the way out, its raw output is combined with the noisy input: the prediction is the noisy coordinates multiplied by a skip weight, plus the network’s output multiplied by an output scale.

Both factors depend on the noise level. At 0.5 Å, the skip weight is 0.999 and the output scale is 0.5 Å, so the prediction is the input plus a small correction to each atom’s position. At 160 Å, the skip weight is 0.01 and the output scale is 15.9 Å, so the prediction is almost entirely the network’s output.

Working backwards from the prediction gives what the network itself is trained to output. At low noise, where the skip weight is near 1, its target is the noise with the sign reversed, divided by σ. At high noise, where the skip weight is near 0, its target is the clean structure divided by 16 Å. Both have about unit size, and in between the target is a mixture of the two.

Why not train the network to predict the noise at every level, as many image diffusion models do? Such a network predicts the noise at unit size, and the structure is recovered by subtracting σ times that prediction from the input. At high noise, any error in the prediction is then multiplied by σ, which is far larger than the structure. EDM chooses the skip weight to amplify the network’s errors as little as possible, which is why the target shifts from noise to structure as σ grows. EDM, AlphaFold 3

How the diffusion module turns its network's output into coordinates

1. InputNoisy coordinates for every heavy atom, the noise level σ in ångströms, and the trunk’s representations.

Network path

1. Divide the coordinates by √(σ² + 16²), so the network sees coordinates of about unit spread at every noise level.
2. The network, told the noise level and given the trunk’s representations, returns an output meant to be about unit size.
3. Multiply that output by the output scale, which is about σ at low noise and approaches 16 Å at high noise.

Skip path

1. Multiply the noisy coordinates by the skip weight, which is close to 1 at low noise and close to 0 at high noise.At low noise the answer is nearly the input itself, so most of the prediction comes through this path.

1. Add the two pathsThe sum is the predicted clean structure, which the loss compares with the true one.

Noise level σ	Skip weight	Output scale	What the network is trained to output
0.5 Å	0.999	0.5 Å	The noise, sign reversed, divided by σ
4.8 Å	0.92	4.6 Å	Mostly the noise
16 Å	0.50	11.3 Å	Equal parts noise and structure
160 Å	0.01	15.9 Å	The clean structure, divided by 16 Å

The loss

The loss scores the prediction after aligning the true structure onto it, which removes any error that is only a rotation or shift of the whole complex. Its main term is the mean squared distance between predicted and true atoms, with small-molecule atoms weighted 11 times and DNA and RNA atoms 6 times relative to protein atoms.

In initial training, a second term scores local geometry directly. The local distance difference test (LDDT) checks, for each pair of nearby atoms, whether the predicted distance is within 0.5, 1, 2 and 4 Å of the true one. Smooth LDDT replaces each pass-or-fail check with a smooth function, so that the score can be trained on by gradient descent. In fine-tuning, a third term penalizes wrong lengths for the bonds that attach a ligand to its polymer chain, such as a sugar chain attached to a protein.

Before any of these terms can be computed, each predicted chain has to be paired with a true chain. The deposited file names identical chains arbitrarily, A and B, and every predicted atom carries the chain name of the input token it came from, so a prediction can put the copy it calls A where the file put B. The AlphaFold 3 authors note that the model will very likely choose a different naming from the depositors’. When the two copies are related by a rotation of the whole complex, the superposition before the loss absorbs the swap. When they are not, for example when a small molecule is bound to only one copy, no rotation or shift fixes it, and a correct structure scores as badly wrong.

AlphaFold 3 renames the true chains to match the model’s own quick prediction before making the noisy copies and computing the loss. That prediction is a 20-step run of the sampler with gradients switched off, the same run used to train the confidence head below. Identical small molecules are matched the same way. Within a residue or small molecule, some atoms are interchangeable, such as the two oxygens of a carboxylate group; for those it tries up to 1,000 relabellings from RDKit and keeps the one closest to the prediction.

The Supplementary Information gives only the reason above, but a squared-error loss adds a second one. At high noise the input no longer shows which copy the file placed where. A model that cannot tell which naming the file used does best, under squared error, by predicting the average of the two placements, which puts both copies midway between their true positions. AlphaFold 3

Identical chains have to be matched before the loss is computed

• Deposited structure
• Prediction
• Atoms the loss compares
• Chain named A
• Chain named B

Compared by the deposited file’s chain names

No rotation or shift of the whole complex brings each named chain onto its partner without moving the small molecule away, so a correct structure scores badly.

Compared after renaming the deposited chains

Only the small real errors remain.

The alignment and the random rotations stand in for a property that other architectures build in. An equivariant network’s output rotates exactly as its input does, by construction. AlphaFold 3’s diffusion module is not equivariant; it learns to handle orientation from seeing training examples in every orientation. SimpleFold, from Apple, makes the same choice. Pearl makes the opposite one and argues that equivariance lets it learn from less data. Neither side has compared the two on a matched model. SimpleFold, Pearl

Weighting the noise levels

The squared error needs one more correction before it is summed across examples. The network’s output is multiplied by the output scale before it is compared with the true structure, so a unit-sized error in the network’s output becomes an error of 0.5 Å at 0.5 Å of noise and 15.9 Å at 160 Å. Squared, the high-noise example would count about a thousand times as much. EDM therefore multiplies each example’s squared error by a weight equal to 1 divided by the square of the output scale. An error of a given size in the network’s output then costs the same at every noise level, and for this term the share of training each noise level receives is set by how often it is drawn.

The AlphaFold 3 Supplementary Information prints this weight with the denominator (σ + 16)², where EDM’s formula has (16σ)²; the details below give both. Taken literally, the printed weight would count an error in the network’s output about 272 times as heavily at 16 Å of noise as at 0.5 Å. It would give little training to the low-noise examples that teach local chemistry. Boltz-1, Protenix, RoseTTAFold3 and Ligo’s reproduction state that they use EDM’s form. Whether AlphaFold 3 itself trained with the printed one cannot be checked, since its training code is not public.

The loss and scaling formulas

AlphaFold 3’s total loss is 4 × the diffusion loss + 0.03 × the distogram loss + 0.0001 × the confidence losses. The diffusion loss is the weighted squared error (plus the bond loss in fine-tuning), multiplied by the noise weight λ(σ), plus smooth LDDT in initial training.

With σ the noise level and σ_data = 16 Å, the network’s input is divided by √(σ² + σ_data²), and the skip weight and output scale are

$$c_{\text{skip}}(\sigma) = \frac{\sigma_{\text{data}}^2}{\sigma^2 + \sigma_{\text{data}}^2}, \qquad c_{\text{out}}(\sigma) = \frac{\sigma\,\sigma_{\text{data}}}{\sqrt{\sigma^2 + \sigma_{\text{data}}^2}}$$

The prediction’s error is the output scale times the error in the network’s output, so the weight on the network’s own error is λ(σ) × c_out(σ)². EDM sets λ to 1 ÷ c_out(σ)², which makes that product 1 at every noise level:

$$\lambda(\sigma) = \frac{\sigma^2 + \sigma_{\text{data}}^2}{(\sigma\,\sigma_{\text{data}})^2}$$

The Supplementary Information prints the denominator as (σ + σ_data)². With that form, the weight on the network’s own error is σ²σ_data² ÷ (σ + σ_data)², which is 0.235 at 0.5 Å and 64 at 16 Å.

Many noisy copies per trunk pass

The trunk is the expensive part of the network and the diffusion module the cheap one, so AlphaFold 3 reuses each trunk pass. For each training example it runs the trunk once, then makes 48 copies of the true structure, each rotated and noised independently, and trains the diffusion module on all of them. With 256 examples per optimizer step, one step of initial training gives the diffusion module 12,288 denoising problems while running the trunk only 256 times. Fine-tuning uses 32 copies, or 8,192 problems per step.

From training to sampling

Apart from the 20-step prediction used for chain matching and the confidence head, which runs with gradients switched off, training never runs the sampler. Each training example is one denoising step from a noisy copy of the true structure. Generating a structure chains 200 such steps, each starting from the structure the previous step produced, and the rules for that chain are set separately from training.

AlphaFold 3’s sampler starts from pure noise with a spread of 2,560 Å, 160 times the typical spread of a structure. At that level a noisy copy of any structure looks the same as pure noise, which is what lets the sampler start from noise alone. It ends near 0.006 Å. The steps follow EDM’s schedule: they shrink from about 75 Å at the start to about 0.001 Å at the end, but on a logarithmic scale they are packed most densely at high noise. Of the 200 steps, 122 end above 16 Å, a range that holds about a fifth of training draws, and 40 end below 1 Å. The strip in the noise-level figure above shows both distributions. The Supplementary Information states the schedule without discussing this difference.

At every step, the sampler re-centres the structure and rotates it at random, with the same function used to prepare training examples, so the structure’s position and orientation at each step are distributed as in training.

Two further settings adjust the steps themselves. In each step that ends above 1 Å, 160 of the 200, the sampler first adds fresh noise, raising the current noise level by 80%, and denoises from there. Each step then moves 1.5 times as far toward the module’s prediction as the noise schedule alone would move it, a setting called the step scale. AlphaFold 3

These settings are chosen after training, and they matter most when steps are few. The AlphaFold 3 papers do not explain the step scale of 1.5. Protenix-Mini’s authors found that the default sampler works at 200 or 20 steps but produces broken structures below 10. Without the fresh noise and with a step scale of 1, a small Protenix model reached an LDDT of 0.645 on protein–ligand interfaces in 2 steps, against 0.65 for the default 200-step sampler, without retraining, though the few-step structures occasionally had clashes or less plausible geometry. Protenix-Mini

Training the confidence head

A confidence score is trained by comparing a predicted structure with the true one and asking the head to predict the error. AlphaFold 2 had a full predicted structure at every training step. A diffusion model does not: training runs single denoising steps from noisy copies of the answer, and never generates a structure from scratch.

AlphaFold 3 supplies a structure with a mini-rollout. At each training step it generates one structure from pure noise in 20 denoising steps, a tenth of the number used at prediction time, with gradients switched off. It measures that structure’s errors against the true one. The confidence head learns to predict those errors from the trunk’s representations and the distances between tokens in the generated structure.

The rollout uses one set of sampler settings, so the head learns to judge structures made that way. Boltz-2 varies the settings during confidence training, drawing the number of steps from 20, 50 or 200 and the step scale from 1.0 to 1.5 at every iteration, to make the head reliable under different settings at prediction time. Boltz-2

A stop-gradient on the head’s inputs keeps the confidence loss from training the Pairformer or the diffusion module, so training the confidence head does not change the structures the model generates. The confidence losses use only experimental structures resolved at 0.1 to 4 Å. The pLDDT and PDE losses are trained from the start, and the PAE loss is added only in the final stage. AlphaFold 3

At prediction time, the confidence head’s job is ranking. AlphaFold 3 runs 5 random seeds with 5 samples each and keeps the highest-scoring of the 25. The ranking score is mostly ipTM, with a large penalty for chains that clash and a small bonus for exposed, extended regions, a second guard against compact structure predicted for disordered regions. More samples give the ranking more to choose from: for antibodies bound to their targets, the authors found that accuracy kept rising up to 1,000 seeds. AlphaFold 3

The reproductions disagree about how to train this head. Boltz-1 trains it afterwards as a separate model, a copy of the whole trunk initialized from the trained one. RoseTTAFold3 trains it with the rest of the model frozen. OpenFold3’s second preview trains the pair-error head from the start alongside everything else. Boltz-1, RoseTTAFold3, OpenFold3

The ranking can be tested by comparing two ways of picking a sample: the one the confidence head ranks first, and the best one, picked with the experimental answer in hand. The gap between them is accuracy the model generated and the ranking failed to pick. For antibody–antigen interfaces, OpenDDE, from Aureka, had a successful structure among its samples for more than 90% of targets in the FoldBench antibody set when given hundreds of seeds. Its top-ranked sample was successful for about 73% at a large sampling budget. OpenDDE

For small molecules, OpenFold3’s team reports that its gap to AlphaFold 3 on the Runs N’ Poses benchmark is larger when each model’s top-ranked sample is scored than when its best sample is. That points to room for improving OpenFold3’s ranking. The evaluation section asks a different question: whether even the best sample is right on complexes unlike the training data. OpenFold3

The schedule and what it costs

Whole complexes do not fit in memory during training, and the trunk’s cost grows with the cube of the crop size: doubling a crop from 384 to 768 tokens multiplies the trunk’s work by eight. AlphaFold 3 therefore does most of its training on small crops and fine-tunes on larger ones. Crops are taken three ways: a contiguous stretch of each chain, the tokens nearest a random atom, or the tokens nearest a random atom at an interface.

Stage	Crop, tokens	Examples, millions	Days on 256 A100 GPUs	What changes
Initial training	384	about 20	about 10	From random weights
Fine-tuning 1	640	about 1.5	about 3	Bond loss added, smooth LDDT switched off
Fine-tuning 2	768	about 1.5	about 5	Transcription-factor sets added
Fine-tuning 3	768	about 1.8	about 2	Structure losses off; PAE loss added

The stages add up to about 20 days on 256 A100 GPUs, or about 5,100 GPU-days. The fine-tuning stages improved every metric the authors tracked, most of all protein–protein interfaces. Each stage changed crop size, losses and data together, though, so the gain cannot be attributed to crop size alone. AlphaFold 3

Optimizer settings

Adam with β₁ = 0.9, β₂ = 0.95; learning rate 1.8 × 10⁻³, warmed up over the first 1,000 steps and multiplied by 0.95 every 50,000 steps; 256 examples per step; gradients clipped at a global norm of 10. Predictions use an exponential moving average of the weights with decay 0.999. The released code recycles the trunk 10 times at prediction time; the algorithm printed in the Supplementary Information has a default of 4. The Supplementary Information does not state the number of recycles used in training or the total parameter count.

Different abilities are learned at different speeds. Accuracy within single chains reached 97% of its initial-training maximum in the first 20,000 steps. Protein–protein interfaces took 60,000. Some abilities peaked and then declined, which the authors attribute to overfitting the few examples of those kinds. Their response was to adjust the dataset weights and to choose the final checkpoint by a weighted score over 12 validation metrics, one of which measures whether disordered regions come out extended. AlphaFold 3

Reported compute varies across the reproductions, and most reports leave something out.

Model	Reported training compute
AlphaFold 3	256 A100 GPUs for about 20 days, about 5,100 GPU-days
Chai-1	128 A100 GPUs for 30 days, 3,840 GPU-days
Protenix (2024)	192 GPUs, model not stated, for about two weeks
Boltz-1	68,000 steps of 128 examples; the authors estimate a quarter of AlphaFold 3’s compute
OpenFold3, second preview	155,000 steps on 256 H100 GPUs; duration not stated
OpenDDE	About 414,000 GPU-hours (17,250 GPU-days), plus five weeks on 80 GPUs to prepare data and alignments

Chai-1, Protenix, Boltz-1, OpenFold3, OpenDDE

Reproducing the recipe from its description

DeepMind released AlphaFold 3’s inference code and weights for academic use, but not its training code. Most of the training detail above comes from the paper’s 45-page Supplementary Information, and every open reproduction was built from that description.

Two corrections recur. The Supplementary Information prints the diffusion module’s transformer block without residual connections, the shortcuts that let each layer add to its input rather than replace it. It also prints the noise weighting with a sum where the EDM framework has a product (see the section on weighting the noise levels). Boltz-1, Protenix, OpenFold3, RoseTTAFold3 and a smaller reproduction by Ligo Biosciences all restored both. Protenix reports that without the residual connections the model “does not converge well”, and RoseTTAFold3’s authors call both changes essential. The released inference code contains the residual connections, which suggests the printed version is a transcription error. Protenix, RoseTTAFold3

The evidence for these corrections is the groups’ experience rather than controlled comparisons. None publishes an ablation of final accuracy; Ligo reports only small-scale checks of the loss scale and of how fast training converges. The same holds for most of what the reproductions changed. Boltz-1’s authors write that they tested their changes on smaller models and “cannot present direct ablation studies” at full size. Isomorphic Labs, reviewing the reproductions in its 2026 IsoDDE report, observed that distillation mixtures, crop schedules and losses had been varied “without establishing clear links to final model performance.” Boltz-1, IsoDDE

The reproductions also changed their inputs and how they read PDB records. Boltz-1 does not use templates. Chai-1 adds embeddings from a protein language model alongside the alignment, which lets it run without an alignment at prediction time. Protenix changed how it handles atoms recorded in more than one position. The PDB gives each alternative position an occupancy: the fraction of the molecules in the crystal in which the atom sits there. Choosing the highest-occupancy alternative could leave neighbouring residues in incompatible conformations and break the covalent bonds between them, so Protenix takes the first recorded alternative instead. RoseTTAFold3’s authors built a data pipeline, AtomWorks, to handle such chemical edge cases consistently. Boltz-1, Chai-1, Protenix, RoseTTAFold3

Physical validity: in the weights or at sampling time

A stereocentre is an atom whose attached groups can take either of two mirror-image arrangements, and its chirality is which of the two it has. AlphaFold 3 has no loss that penalizes clashing atoms or wrong chirality, and its outputs sometimes have both. On the PoseBusters benchmark of protein–small-molecule complexes, it had a chirality violation rate of 4.4%, even with a chirality penalty in its ranking. Groups have handled the problem at three points: ranking, sampling and training.

At ranking, AlphaFold 3 penalizes clashes and, on PoseBusters, chirality errors, then relies on having many samples to choose from.

During sampling, Boltz-1x, a variant of Boltz-1, adds energy terms for chirality, bond geometry and clashes to the denoising steps, without retraining. Adjusting the denoising steps this way is called steering. On Boltz-1’s own test set, the share of top-ranked predictions passing a set of physical checks taken from PoseBusters rose from 43% to 97%, with similar pose accuracy. Protenix-v1 with guidance and OpenBind-0, a model built on OpenFold3, later made comparable changes to their denoising steps. Each reports the share of predictions both accurate and physically valid rising from about 48% to about 61%. Boltz-1, Protenix-v2 report, OpenBind-0

In training, RoseTTAFold3 gives the diffusion module, at every step, the direction in which each atom would have to move to fix the chirality errors in the current noisy structure, and inverts the chirality of 2% of training examples. Without any correction at sampling time, it placed 88% of small-molecule stereocentres correctly on its own test set, against 84% for AlphaFold 3 and 76% for Boltz-2. This is a per-stereocentre rate on a different test set, so it does not compare directly with AlphaFold 3’s 4.4% on PoseBusters. It is also a comparison across models with different data and architectures, and the gap to AlphaFold 3 was not statistically significant. The RoseTTAFold3 authors suggest that corrections applied only at sampling time may give the network inputs unlike those it saw in training. No one has compared the two approaches on one model. RoseTTAFold3

In the one independent test, steering fixed the geometry without moving the molecule to the right place. Runs N’ Poses, a benchmark described below, found that Boltz-1x cleared its clash failures entirely and predicted correct poses no more often than Boltz-1. Runs N’ Poses

What later models changed

Wider pair representations

SeedFold tested three ways to spend more parameters, each in a separate variant: doubling the Pairformer’s depth from 48 to 96 blocks, doubling the diffusion module’s depth, or widening the pair representation from 128 channels to 256 and 512. Widening improved accuracy; the deeper variants gained less. The authors’ explanation is that 48 blocks, run repeatedly through recycling, already make a very deep network.

The comparison was not fully matched: the wider variants also widened the MSA representation, and the 512-channel model used an adjusted learning rate and warmup. Width also cost speed. The 512-channel model had 2.1 times the parameters of the 128-channel one and trained at 40% of its speed. SeedFold

OpenDDE widened to 384 channels, which it estimates multiplied the Pairformer’s compute by about nine. IntelliFold-2’s release notes add a hardware reason for width: a narrow pair representation leaves GPUs mostly waiting on memory. Their narrower model used about 5% of the GPUs’ peak arithmetic rate and their wider one about 30%. OpenDDE, IntelliFold-2

More predicted data, arranged in stages

SeedFold trained on 26.4 million predicted structures, 146 times its 180,540 experimental ones, and still drew half its examples from the PDB. OpenDDE changes its mix by stage: 70% experimental in a warm-up, down to 35% as predicted data comes in, back to 40–50% at the end, with antibody structures rising from none to 13% over the later stages. Its authors describe the plan as going “from precision, to breadth, and then back to precision”; they did not compare it with a fixed mix. SeedFold, OpenDDE

Simpler networks, so far for single chains

SimpleFold predicts single protein chains only, with no complexes or small molecules. It drops the alignment, the pair representation and the triangle updates, and trains a plain transformer on 8.6 million predicted protein structures with a flow-matching objective. Instead of predicting the clean coordinates from a noisy copy, as AlphaFold 3 does, it learns the direction in which to move atoms from random coordinates toward the target structure.

SimpleFold’s authors also ran the controlled data sweep mentioned earlier. At a fixed 700 million parameters, they trained on the PDB alone (160,000 structures) and on three larger mixes that add predicted structures, up to the full 8.6 million, and accuracy tended to improve as predicted data was added.

At 3 billion parameters, SimpleFold approaches AlphaFold 2 on easier benchmarks and trails it on CASP14, the 14th Critical Assessment of protein Structure Prediction. Its TM-score there, a measure of overall fold similarity from 0 to 1, was 0.720 against AlphaFold 2’s 0.845. The cofolding models discussed here have all kept the pair representation. SimpleFold

New heads trained on top

Boltz-2 adds a binding-affinity head, trained after the structure and confidence parts, with their weights frozen. Affinity data is noisy in a specific way: values from different assays are not directly comparable. A measured value such as an IC50, the concentration of a compound that halves its target’s activity, depends on the assay’s conditions as well as on how tightly the compound binds.

Boltz-2 builds each training batch from a single assay, and puts most of its loss on differences in affinity between compounds in the same assay. The differences cancel any offset that the assay applies to all its compounds. Measurements recorded only as a bound, such as “weaker than 10 µM”, are penalized in one direction only.

For non-binders, the model uses measured inactive compounds from screening experiments and synthetic examples drawn from other targets’ candidate compounds. The synthetic examples are drawn from real drug candidates, so that the model cannot separate them from binders by features unrelated to binding. None of these choices is ablated in the paper. Boltz-2

Fine-tuning with rewards

In language models, fine-tuning on a reward after pretraining is standard. In cofolding it is rare. IntelliFold-2’s release notes describe fine-tuning the diffusion module with PPO, a reinforcement-learning algorithm, rewarded by DockQ, a score of interface accuracy. They give no numbers that separate its effect from the other changes in the same release. IntelliFold-2

What the evaluations say training taught

A test complex released after the training cutoff can still be nearly a copy of one released before it: the same protein, the same pocket, a similar molecule sitting the same way. Runs N’ Poses is a benchmark of 2,600 protein–small-molecule complexes released after the training cutoffs of AlphaFold 3, Protenix, Chai-1 and Boltz-1. It scores each test complex from 0 to 100 by its similarity to the closest training complex, combining how much of the binding pocket matches with how much of the molecule’s shape and chemistry overlaps. By that measure, 87% of the PoseBusters benchmark, widely used to report small-molecule accuracy, scores above 50. Runs N’ Poses

A prediction counts as correct here when the molecule lies within 2 Å of its experimental position and its contacts with the protein are reproduced. Accuracy follows the similarity score closely. AlphaFold 3’s top-ranked prediction was correct for 23% of the least similar complexes and 90% of the most similar. Protenix, Chai-1 and Boltz-1 rise the same way from lower starting points, 8 to 14%. These figures are recomputed from the benchmark’s released predictions.

AlphaFold 3 is most accurate on complexes that resemble its training data

Share of protein–small-molecule complexes predicted correctly, grouped by how closely each resembles a complex in the training set (0 to 100). Complexes were all released after the training cutoff.

• Best of 25 samples, picked using the answer
• Top-ranked sample, picked by the model’s confidence

Values

Similarity	Complexes	Top-ranked	Best of 25
0–20	64	23%	31%
20–30	91	27%	33%
30–40	149	32%	39%
40–50	258	45%	56%
50–60	312	58%	70%
60–70	376	69%	80%
70–80	368	77%	85%
80–100	674	90%	95%

The figure’s two lines separate generation from ranking. If the models generated correct poses and merely failed to rank them first, picking the best of 25 samples with the answer in hand would flatten the curve. It does not. Across models and similarity bins, picking the best sample raises success by 2 to 16 percentage points and leaves the rise in place. In Runs N’ Poses, then, ranking costs some accuracy, but the main limit on unfamiliar complexes is generating a correct pose at all. The authors conclude that these models “largely memorise ligand poses from their training data.”

Across all test complexes, the four models reproduce the binding pocket itself accurately in about nine predictions in ten. The authors suggest that the harder problem is the molecule’s pose within familiar pockets. On the least similar complexes, though, the pocket is also modelled less well, accurately in 57 to 67% of predictions.

Two comparisons ask whether newer training data helps. Boltz-2 trained on PDB structures up to mid-2023 rather than late 2021. Newer data can move a test complex into a higher similarity group, so for this comparison Runs N’ Poses recomputed similarity against Boltz-2’s own cutoff. Boltz-2 improved on the most similar complexes and not on the least similar. It also differs from Boltz-1 in architecture, so the comparison does not isolate the data. Runs N’ Poses

OpenBind-0’s authors examined what four more years of PDB structures changed. Most test complexes stayed in the same similarity group, and those that did were predicted about equally well by OpenBind-0 and the earlier OpenFold3 preview. Complexes that moved into the most similar group were predicted substantially better. Some gains also appeared in the least similar group, which the authors tentatively attribute to broader coverage of chemical and structural space. The two models also differ in training and checkpoint choice, so this is not a controlled data ablation. OpenBind-0

Other studies probe the same question by changing the input. Masters and colleagues mutated every residue within 3.5 Å of a bound molecule: to glycine, removing the side chains; to phenylalanine, filling the pocket; or to residues of opposite size and polarity. These changes should weaken or prevent binding. Simulations found no binding in the filled and inverted pockets, and some in the glycine pocket. On 285 complexes, more than half of AlphaFold 3’s correct predictions kept the same pose after each change, and its confidence did not flag them.

The authors attribute this to the models learning statistical patterns rather than physical interactions. They point to an input the mutations do not change: a mutated protein retrieves nearly the same sequence alignment and templates as the original. If the models were simply recalling training structures, the complexes with the most training examples should be the least sensitive to the mutations. The paper found no correlation between the two, which argues against memorization as the whole explanation. Masters et al.

A study of 82 enzymes that close around a bound molecule found a similar dependence on the training data. Whether AlphaFold 3 predicted an enzyme open or closed followed how many open and closed structures of it the PDB holds. Adding the molecule to the input moved the share of closed predictions by 9 to 18 points, while enzymes mostly crystallized open and those mostly crystallized closed differed by about 40 points. Molecules known not to bind also made AlphaFold 3 predict the closed form. Yu et al.

Some evaluations found no such dependence. The CASP16 assessment included drug-discovery targets with bound molecules, and its assessors ran AlphaFold 3 afterwards as a baseline rather than as a blind entry. It reached a mean LDDT-PLI (a score from 0 to 1 of how well protein–ligand contacts are reproduced) of 0.80, against 0.69 for the best participating group. Its accuracy did not track similarity to PDB complexes, measured the same way as in Runs N’ Poses, and it placed one molecule correctly on a protein with no previously known binding site. The CASP16 targets are well-studied proteins, though, which narrows the range of similarity over which a trend could appear.

Studies of the Mac1 domain of SARS-CoV-2 and of receptors from G protein-coupled receptor families new to training found no trend with the molecule’s chemical similarity or the receptor’s sequence identity to training data. Those studies measure similarity differently from Runs N’ Poses, and how far their results conflict with it is unsettled. CASP16 ligands, Mac1, GPCRs

Binding affinity shows the dependence as well. MIRAGE, a September 2026 preprint, found that the affinity accuracy of two cofolding models, one of them Boltz-2, rose steeply with the number of structures from the same protein family in the public data. Control models that could not have learned the test families stayed flat. MIRAGE

The models also generate a narrow range of shapes. AlphaFold 3’s authors state that its samples, across seeds, do not approximate the range of shapes a complex takes in solution; they reproduce the static structures the PDB contains. KinConfBench, an independent kinase benchmark, found the same. For about two-thirds of kinase–ligand systems, all 20 predictions were right or all were wrong, and the correct predictions of a system varied by only about 0.1 Å. Boltz-2’s simulation data and method feature are attempts to change this in training. AlphaFold 3, KinConfBench

What is settled and what is open

Some parts of the recipe are established. AlphaFold 3’s design, a pair trunk feeding a diffusion module that denoises every heavy atom, trained on cropped examples with a confidence head learning from quick rollouts, has been reproduced by at least five groups from its written description. The groups that report the details corrected the same two printed errors. Nearly every cofolding model discussed here mixes large amounts of predicted structure into its training data.

Physical validity can be improved without retraining. Boltz-1x raised the share of its predictions passing PoseBusters’ physical checks from 43% to 97%, though Runs N’ Poses found no gain in pose accuracy for that pair of models. In Runs N’ Poses, small-molecule pose accuracy rose strongly with similarity to training complexes. CASP16 used the same similarity measure on a narrow set of well-studied targets and found no trend, and studies using other measures found none either, so the scope of this dependence remains open.

Most individual training choices have not been tested one at a time. The evidence that predicted protein structures help comes from one branch experiment in a cofolding model, SeedFold, and one controlled data sweep in a single-chain model, SimpleFold. The evidence for AlphaFold 3’s disordered-protein set rests on one ablation. Staged data mixtures, crop schedules and the several ways of training the confidence head have each been adopted without a comparison. Whether chirality belongs in the weights or in the sampler, and whether equivariance beats data augmentation, are argued on uncontrolled comparisons. Physics-generated training data has one controlled result, a gain of a few points on a smaller model. Fine-tuning a cofolding model on a reward, a standard step for language models, has one published attempt found for this digest, with no numbers that isolate its effect.

The largest open question is whether predicted structures, simulations, physics-generated complexes or a change to the objective can improve pose accuracy on protein–small-molecule complexes with little similarity to the training set.