Mode collapse in post-training

Sep 18, 2026 · 25 min read

Digests are written primarily by LLMs with only readability edits by me. If this one reads like slop, please shoot me an email at max.zhuang@berkeley.edu.

Ask four different language models to write a story, and about half the stories will have a lighthouse in them. A quarter will have someone named Elias. This is from a May 2026 study that sampled 20,000 stories from Claude Haiku 4.5, Gemini 3.1 Flash-Lite, GPT-5.4-Mini, and OLMo 3 Think at 7B parameters, using five prompts no more specific than “Write a story.” A lighthouse turned up in 51.2% of them and an Elias in 26.5%. In the published fiction the authors used for comparison, Elias shows up 2.7 times per million words, and in the generated stories 2,428 times. The one training set the authors could open, OLMo’s, contained those names at far lower rates than the outputs. The private models’ data could not be checked, so for three of the four the source is unknown. Hamilton and Mimno

That is mode collapse: the model’s outputs pile onto a few options when the task allows many. Related failures show up elsewhere. Ask for a random number and get the same one. Ask for a story and get the same story from four unrelated labs. Train a reasoning model and watch it get better on its first try while its own starting model beats it once both get a few hundred tries. These examples differ in mechanism, but they share a problem: removing bad answers and removing useful alternatives are two different things, post-training can do both at once, and a higher average score cannot tell you which one happened.

What mode collapse is

Mode collapse is too much concentration of a model’s outputs on a few options when the task admits many useful ones. The definition depends on the task. A model asked for the derivative of x² should say 2x every time, and nobody calls that collapse. A model asked for three different proofs, or three story premises, needs to have kept some alternatives. So measure the variety among the successful outputs, not just how often the model succeeds.

The same accuracy can hide very different variety

Illustrative probabilities for one problem with three valid solution methods.

Starting model

Incorrect: 45%
Correct method A: 25%
Correct method B: 20%
Correct method C: 10%

55% correct

Quality improves

Incorrect: 5%
Correct method A: 40%
Correct method B: 35%
Correct method C: 20%

95% correct

Useful variety collapses

Incorrect: 5%
Correct method A: 91%
Correct method B: 3%
Correct method C: 1%

95% correct

Both improved models answer correctly 95% of the time. One retains three useful methods; the other almost always chooses method A. Accuracy alone misses that difference. These are invented values, not measured training results or a claim that collapse is irreversible.

The term comes from generative adversarial networks, where a generator learns to turn random noise into images and a discriminator learns to tell its images from real ones. There, collapse means that different noise inputs produce too little variety in the images. A mode is a peak in a probability distribution; a collapsed model covers too few of the peaks it was supposed to cover. Partial collapse, where several peaks survive and the rest vanish, is more common than one image repeated forever. Goodfellow, GAN tutorial

The 2022 example that gave language models the word

The earliest widely read discussion came from the researcher known as janus, comparing OpenAI’s base model davinci with its tuned sibling text-davinci-002. Asked for a random integer between 0 and 100, the base model spread its probability fairly broadly. The tuned model favored 97. Asked whether bugs are real, the tuned model produced the same “there is no single answer” paragraph again and again, with many tokens above 99% probability at temperature 1. (Temperature 1 leaves the model’s token probabilities exactly as they are. It does not make an already concentrated distribution diverse; it samples faithfully from it.)

The post blamed reinforcement learning from human feedback. The author later reported that text-davinci-002 had not used it, and corrected the attribution. The observations held up and the attributed cause did not, which roughly describes the field four years later. janus, Mysteries of mode collapse

Two related terms get called mode collapse but describe different things.

Entropy collapse is a property of a training run, not of a model. Entropy measures how spread out the next-token probabilities are, and during reinforcement learning it can fall fast and early. That makes it a useful diagnostic. It is not the same as losing useful variety, in either direction: many wordings can express one idea, so high entropy can hide one story told a hundred ways, and a few decisive tokens can select between very different ideas, so low entropy can coexist with real choice.

Model collapse is a different mechanism with a confusingly similar name. Shumailov and colleagues studied models trained, generation after generation, on the outputs of earlier models. The rare parts of the original distribution disappear and the learned distribution degrades. That depends on how generated data feeds back into later training. Mode collapse needs no recursion at all; one model can do it to itself in one run. Narrow synthetic data can make things worse, but using synthetic data does not by itself tell you anything has collapsed. Shumailov et al.

Where the narrowing happens

If you want to know which stage lost the variety, you need checkpoints from every stage, and OLMo is one of the few lineages that publishes them. A quick map of the stages: supervised fine-tuning teaches the model to imitate examples. Preference training, including direct preference optimization (DPO), learns from comparisons between answers. Reinforcement learning (RL) updates the model with rewards for outputs it sampled. Any of the three can change diversity, and the question is which one did.

Karouzos, Tan, and Aletras used those checkpoints. They studied 13 OLMo checkpoints at 7B parameters. Relative to the base model, the Think lineage lost 62 percentage points of a semantic diversity measure at supervised fine-tuning alone. The Instruct lineage lost 38 there and another 23 at DPO. RL-Zero, which skipped both of those stages, kept 93% on average on this measure, and paid for it with lower quality on many tasks.

The largest diversity loss occurs at different stages

Semantic diversity index · Base model = 100 · Higher means more variety

Checkpoint	Think	Instruct
Base reference	100	100
After supervised fine-tuning	38	62
After preference training	34	39
After reinforcement learning	38	34

RL-Zero, which skips supervised and preference training, retained an average index of 93, with lower quality on many tasks.

Based on the rounded stage changes reported by Karouzos et al. The measure is average distance between sentence embeddings across 15 tasks, normalized to the base model. These are checkpoint comparisons, not two otherwise identical experiments: Instruct's supervised training begins from Think-SFT.

So the stage of greatest loss depends on the lineage. The comparison is not an experiment that changes only the breadth of the data: Instruct’s supervised stage starts from Think-SFT, and the two lineages differ in their examples, their reasoning format, and how their preference pairs were built. One more result from the same study: suppressing Think’s written-out reasoning at inference did not bring the diversity back. That rules out one cheap fix, not every inference-time intervention. Karouzos et al.

Why post-training narrows the distribution

The objective, the update rule, and the data can each favor concentration. How much each one matters depends on the setup, and none of them proves that every run must collapse. But it helps to see all three, because they push the same way and they compound.

The objective reweights the reference model

The usual objective in reinforcement learning from human feedback (RLHF) is to raise expected reward while penalizing movement away from a reference model, normally a frozen copy of an earlier checkpoint. DPO is derived from that same objective under a model of paired preferences. The penalty is a Kullback–Leibler (KL) divergence, a measure of how different two distributions are, with a positive weight written β. Not everything uses it: supervised fine-tuning has a different loss altogether, and some reinforcement learning with verifiable rewards (RLVR) runs drop the reference penalty entirely.

The optimum of that objective has a closed form, and the DPO paper is the usual place to read it. Each output gets its reference probability multiplied by exp(reward/β), where exp is the exponential function, and then everything is normalized to sum to one. It says three things. Equal rewards leave the reference proportions as they were. A smaller β amplifies reward differences more. And a big enough reward advantage can make a previously unlikely answer common; the only answers the formula excludes are the ones the reference gave exactly zero probability. That last clause is a fact about the formula, not a theorem about what a neural network can learn to reason about. DPO

The update rule can amplify frequent outcomes

Policy-gradient methods, which include proximal policy optimization (PPO) and group relative policy optimization (GRPO), work from sampled outputs and their advantage: how much better an output’s reward was than a baseline, usually the average for that prompt. An output that comes up often supplies more training examples than one that comes up rarely. Shared parameters complicate this, since an update on one output also moves outputs that were never sampled, but the asymmetry remains.

Sinha and colleagues worked it through in a simplified model with a separate parameter for each outcome. Among outcomes with the same positive advantage, the more probable one gets the larger expected update, so it pulls further ahead. This is not the same as saying collapse is inevitable when all rewards are identical: in that case every advantage is zero and the expected reward gradient is zero too. Their fix, inverse probability scaling, compensates for how often each outcome shows up. On their molecular benchmark, the mean count of unique candidates above a computational reward threshold went from 1 to 1,478 against the REINVENT baseline. Sinha et al.

Cui and colleagues looked at the same thing one token at a time. To a first-order approximation, entropy falls when tokens that are already probable receive the larger increases in their underlying scores. For plain policy gradient, their expression is a probability-weighted advantage; the exact form depends on the update rule, so treat the derivation as an explanation of the tendency rather than an identity that holds in every training run. In their measurements, across 2,400-step runs, 73% of the entropy that would be lost, and 76% of the performance gain, happened in the first 200 steps. They fit a performance-versus-entropy curve separately for each setup, and it describes saturation in those runs. It is not a ceiling on what the model could learn with different data or a different method. Cui et al.

Clipping, the mechanism that keeps PPO updates stable, has an asymmetry that matters here. With the usual upper ratio threshold of 1.2, the clipped loss stops rewarding a positive-advantage token for growing more than 20% relative to the policy that produced the sample. For a token at probability 0.01, that threshold is 0.012. For a token at 0.9, the implied 1.08 is outside the range of probabilities altogether, so the common token is never held back. Clipping changes the loss; it does not hard-cap the probabilities that come out. But the incentive is lopsided, and the rare tokens are the ones it discourages. DAPO raised the upper threshold to 1.28, kept the lower at 0.8, and reported better entropy retention and more exploration. DAPO

Preference data can favor the familiar

Human raters also contribute. Zhang and colleagues, in the Verbalized Sampling paper, took 6,874 HelpSteer response pairs with identical correctness ratings and asked which response the raters liked more. The one the base model found more likely tended to win. Matching correctness does not hold every aspect of quality fixed, so this supports a typicality bias without isolating it from every other preference a rater might have. But it is a real signal, and their mathematical model shows what it does: a reward for familiarity sharpens the reference distribution even when the task-quality reward has not changed. That is one plausible mechanism for narrowing. It is not, on its own, a causal account of why particular names keep showing up in stories. Verbalized Sampling

Hamilton and Mimno found a more specific pattern in OLMo 3’s data. Stories containing at least one of their eleven recurring words made up 3.0% of the supervised-stage stories, 8.0% of the DPO stories, and 10.9% of the RL stories. Every one of those rates is far below the rate in the generated stories. A small subset of the data seems to have an outsized effect; the paper leaves the exact cause open. Hamilton and Mimno

Sharpening and reasoning ability

Sharpening explains a good share of post-training’s gains: the model becomes more likely to produce answers it could already produce occasionally. Huang and colleagues formalize this for self-improvement, where the model’s own assessment of its answers is the training signal. Their framework fits that setting well, but does not cover every kind of post-training. Huang et al.

Karan and Du showed the effect with no training at all. They sampled from the base model’s own distribution raised to the fourth power, using repeated propose-and-accept steps over blocks of tokens to draw whole sequences from that sharpened distribution. On MATH500, a competition-math benchmark, Qwen2.5-Math-7B scored 74.8% this way against 78.5% after GRPO training. On HumanEval, a code benchmark, the sharpened base model beat the GRPO model, 57.3% to 53.7%. And their pass@k curves did not show the loss that their GRPO baseline showed. This is an empirical result rather than something guaranteed by leaving the weights alone: a sampler can concentrate outputs too. Karan and Du

Agarwal and colleagues got a related result with a reward that never looks at the answer: minimize the model’s own entropy. On a Qwen 7B model, that alone matched or beat GRPO trained on 60,000 labeled examples. Agarwal et al.

Yue and colleagues measured the tradeoff directly. Compare an RL-trained model with its base model on pass@k, the probability that at least one of k sampled answers solves a problem, averaged over the benchmark. In their comparisons the RL model was usually ahead at k = 1 and the base model overtook it as k grew. On the 2024 American Invitational Mathematics Examination (AIME), 30 problems, one comparison at k = 1,024 found 4 problems solved only by the base model and none solved only by its SimpleRLZoo counterpart. Their likelihood analysis was consistent with RL amplifying reasoning paths the base model already had. Distillation from a stronger teacher, which brings in examples from outside, improved coverage at every sampling budget they tested. Yue et al.

Pass@k for an RL-trained model and its base, schematic

Vertical axis: fraction of problems solved

A pattern observed by Yue et al.: the RL-trained model leads on the first attempt, while the base model covers more problems at larger sampling budgets. These curves are illustrative, not measured data. A crossover establishes a tradeoff at the tested budgets; it does not establish a permanent capability ceiling.

So sharpening can account for a lot of the gain in these settings. A pass@k crossover shows a tradeoff at the tested budgets; it does not prove that no new reasoning behavior was learned, because a finite sample cannot show that an unsolved problem has exactly zero success probability.

Other groups report coverage going up after RL. ProRL combines longer training with KL control and periodic reference resets, and reports gains across the sampling budgets it tested. Yuan and colleagues show how training can make previously unsolved problems solvable while excessive updates on already-solved problems drag down aggregate high-k performance, and that redirecting the updates improved pass@256 on hard benchmarks. Taken together, the evidence supports a practical risk of losing coverage. It does not support an immutable base-model ceiling. ProRL, Yuan et al.

Pretraining and model scale

Pretraining decides what RL has to work with. Zhao and colleagues pretrained small models from scratch on controlled data mixtures and then ran PPO, GRPO, and expert iteration, which fine-tunes on the model’s own best samples, on each of them. Every algorithm converged on a dominant output distribution that amplified patterns already in the pretraining mix, and models of different sizes trained on the same data converged to different ones. Zhao et al.

Shao and colleagues found the same dependence in public base models, with a result that matters for anyone who has demonstrated an RL method on only one model family. Training Qwen2.5-Math-7B with random rewards improved MATH-500 by 21.4 points. The true reward improved it by 29.1. The gain came from the clip amplifying a behavior the base model already had, reasoning in code-like steps, whose frequency rose from 65% to over 90% of responses. Llama and OLMo base models, which rarely did that, generally gained nothing from random rewards. Wang and colleagues found a matching result at the token level: RL mostly leaves the base model’s entropy pattern alone and adjusts the roughly 20% of tokens where the base model was already uncertain, and updating only those tokens matched or beat updating all of them. Shao et al., Wang et al.

In practice, if RL on your base model goes nowhere, the base model may not contain the behavior you are trying to amplify. OctoThinker took Llama models that scaled poorly under RL and fixed it with mid-training, continued pretraining on a mathematical corpus plus question-and-answer data with long reasoning chains. After that, the gap to Qwen under RL closed. OctoThinker

Bigger models can lose more. In Springer and colleagues’ controlled experiments, semantic collapse after supervised fine-tuning got worse with model size: a bigger base model starts out more diverse, and after fine-tuning on the same data it ends up less diverse than a smaller one, because it fits the small fine-tuning set’s narrow semantic prior more tightly. Lower fine-tuning loss went with lower diversity. NoveltyBench, which scores repeated sampling on open-ended prompts against human writers, saw the same ordering across 20 models: the larger model in a family was often the less diverse one. And the Verbalized Sampling authors found that their prompting fix recovered 1.5 to 2 times more diversity from the larger model of a pair, which fits the idea that the larger model still had more alternatives to recover. This is a pattern across a few studies with a plausible reason, not a scaling law. Springer et al., NoveltyBench, Verbalized Sampling

How to recognize it

The base model is the reference. Nearly every number in this digest is a comparison between a post-trained model and the model it came from, and a diversity score on its own tells you very little.

Start with a prompt where you know what the distribution should be: “sample uniformly from the integers 0 through 100,” say. Then you can check the model against the answer. A prompt that merely asks it to name a state has no uniquely correct distribution; corpus frequencies make a reasonable comparison, but they are not automatically the target.

For open-ended tasks, repeated sampling needs two things: a way to judge quality and a definition of what counts as different. The usual measures answer different questions, so choose the one that matches yours.

Measure	What it reveals	What it does not establish
Distinct short word sequences, adjusted for length	Repeated wording	Different ideas or useful answers
Distance between sentence embeddings, numerical representations of meaning	Semantic similarity under that representation	Whether the differences matter to the task
A classifier that tests whether answers contradict each other	Incompatible claims or positions	Desirable variety; contradictions can be errors
Vendi score, computed from pairwise similarities	An effective number of distinct outputs under the chosen similarity measure	A literal count of reasoning strategies or independent attempts

Kirk and colleagues distinguish per-input diversity, across repeated answers to one prompt, from across-input diversity, across different prompts. Both can fall after RLHF. Whichever you measure, measure it among the correct or otherwise acceptable outputs too, because getting rid of wrong answers is good and getting rid of alternative correct methods may not be. One caution on interpretation: a low Vendi score among the correct answers does not by itself put a mathematical limit on what majority voting can gain. Kirk et al., Karouzos et al.

For reasoning models, plot pass@k against k for the post-trained model and its base at several sampling budgets. A crossover shows a tradeoff between single-attempt accuracy and coverage at those budgets. Keep the prompt format, the decoding settings, and the answer checker the same on both sides, and report uncertainty. Remember what pass@k assumes: a checker that recognizes success. It says nothing about which of the k answers an ordinary user should trust. Yue et al.

During training, watch policy entropy next to validation performance and actual sampled outputs. A rapid fall flags reduced exploration. An empirical fit from the same setup can help you see saturation coming; its coefficients will not transfer to a different setup. Cui et al.

Sampling settings and prompts are part of the measurement, not something you set once and forget. Raising temperature can add variety, and it can add errors, and it can fail to recover the alternatives you wanted, so compare quality and diversity together at each setting. Do not expect a second model to serve as a control, either: Jiang and colleagues found substantial inter-model similarity on open-ended queries. Comparing models is still informative, but two models agreeing does not mean either of them covers the space.

The chat template, the wrapper of role markers and special tokens around a conversation, matters too. Yun and colleagues found large effects on open-ended diversity in the models they tested. That sits alongside the OLMo result rather than contradicting it: removing written-out reasoning and changing the structural tokens are different interventions. Jiang et al., Yun et al.

You can see the same homogenization in human writing, though it does not by itself pin down a cause inside the model. Kobak and colleagues counted excess vocabulary in more than 15 million PubMed abstracts and found that “delves” appeared 28 times as often in 2024 as its earlier trend predicted, with “underscores” at 14 times and “showcasing” at 11. By their lower bound, at least 13.5% of 2024 abstracts had been through a language model. Doshi and Hauser gave writers story ideas from GPT-4 and found the stories were rated more creative, most of all from writers with lower baseline creativity scores, while the set of stories became more similar to one another. Padmakumar and He found the same for essays written with InstructGPT but not with the base GPT-3 it was tuned from, and the lost diversity came from the model’s contributions, not the writers’. Kobak et al., Doshi and Hauser, Padmakumar and He

How to limit it

Pick the intervention by where the useful variety was lost. The studies below use different tasks and different metrics, so they do not add up to a ranking of fixes. Instead they offer different ways to shift the balance between quality and variety, one for each place it can go wrong.

Data

Broader supervised data is the natural thing to try when the examples cover too few valid approaches. The OLMo comparison motivates it, with the caveat from earlier that its lineages differ in more than the number of teachers. Karouzos et al.

Choose preference pairs for diversity. Diverse Preference Optimization (DivPO) builds each DPO pair from a pool of samples by taking the rarest response among the high-quality ones as the preferred example and the most common response among the low-quality ones as the rejected example, so the preference signal points away from the mode instead of toward it. Persona attributes came out 45.6% more diverse and stories 74.6% more diverse, with quality held. DivPO

Anchor the semantics to pretraining. Springer and colleagues pretrain on documents paired with annotations of their topics and entities, then mask the annotations out of the fine-tuning loss, so the model keeps its broad sense of what to say while learning from the fine-tuning data how to say it. On their Stories benchmark, the 2.5B-parameter model closed about 85% of the semantic diversity gap to its base model. The cost is that pretraining has to change, so the method cannot be added to a checkpoint you already have. Springer et al.

Objective

Reward useful variety directly. Several methods change what counts as a successful update:

Method	Change to the learning signal	Reported setting
DARLING	Combine quality with diversity across groups of semantically equivalent responses	Creative writing, instruction following, and math
Pass@k training	Optimize success across multiple attempts	Reasoning tasks with checkable answers
Unlikeliness reward, by He, Fried, and Welleck	Give additional credit to correct proofs the model assigned low probability	Formal theorem proving
Differential smoothing	Modify rewards on correct trajectories to reduce concentration	Mathematical reasoning
Inverse probability scaling	Compensate for how frequently outcomes are generated	Controlled environments and computational molecular benchmarks

Each reports improvements on its own evaluation. Keep the diversity measure in view when reading them: a method that preserves varied wording is not necessarily one that preserves distinct reasoning strategies, so the metric is part of the claim.

Change the divergence. Reverse KL, the standard choice, weights discrepancies by the trained policy, so a reference mode the policy has dropped contributes nothing directly from the dropped region; redistributing that mass can still cost KL elsewhere. Forward KL weights discrepancies by the reference instead, and punishes assigning zero probability to anything the reference allowed, harshly. Jensen–Shannon is symmetric and bounded, with no infinite penalty either way. The f-DPO paper swaps these into DPO and reports a controllable trade between alignment score and diversity. f-DPO

Change the supervised loss. Cross-entropy maximizes the likelihood of the one observed response and has no term for the alternatives. GEM uses a game-theoretic loss connected to entropy-regularized reverse-KL minimization. It matched cross-entropy on downstream benchmarks from 3B to 70B parameters and gave more diverse outputs, which improved the returns from repeated sampling on chat and code. GEM

Update dynamics

Let rare tokens grow. Clip-higher, the DAPO change, raises the upper probability-ratio threshold to 1.28 and leaves the lower at 0.8. Clip-Cov and KL-Cov, from Cui and colleagues, target the small fraction of tokens with the highest covariance between log-probability and advantage, a fraction of a percent of all tokens, by detaching their gradients or putting a KL penalty on them. In one reported comparison, KL-Cov kept over ten times the baseline’s entropy late in training; on Qwen2.5-32B it improved performance by 6.4 percentage points on average and by 15 on AIME 2024. STEER generalizes these by estimating each token’s contribution to the entropy change from theory and reweighting accordingly. DAPO, Cui et al., STEER

Stop updating on what is already solved. Bayesian boundary gating estimates each problem’s remaining contribution to pass@k and suppresses updates on the saturated ones. ProRL’s reference resets and KL control let training run much longer without using up its entropy. Yuan et al., ProRL

Inference

Ask for several alternatives at once. Verbalized Sampling requests several responses with associated probabilities instead of one response. On its creative-writing tests it raised semantic diversity 1.6 to 2.1 times over direct prompting. Read that as “a different prompt can reveal alternatives the model still has,” not as “every model retains its full pretraining distribution.” And the probabilities it writes next to each response are words the model produced, not calibrated probabilities of its own outputs. Verbalized Sampling

Change the sampler. Karan and Du’s power sampling recovered much of their RL baseline’s accuracy while keeping better high-k performance, at the cost of iterative inference. It reshapes the distribution over whole sequences, which is a different thing from lowering the token temperature. Karan and Du

Temperature stays useful, as long as you check quality alongside it. In janus’s examples, raising it eventually wrecked coherence without reliably escaping the recurring patterns, so the tradeoff needs measuring at each setting. janus

What the evidence supports

Post-training can improve first-attempt quality while reducing the useful alternatives. In some experiments most of the gain is explained by concentrating on behavior the model already had; in others coverage improves too. Which one you get depends on the data, the objective, the update rule, and how you evaluate.

So compare successive checkpoints, and measure quality and variety together. Use pass@k to assess coverage at a stated sampling budget, and entropy as a training diagnostic. And do not overstate negative results: neither a low entropy value nor a failure within a finite sample shows that a capability is gone for good.

Mode collapse in bioAI

For bioAI systems that generate proteins or molecules, mode collapse means producing many candidates that offer nearly the same biological possibilities. The failure is the same one as for stories or proofs: the useful alternatives are gone. But “different” is harder to define. Different sequences are not necessarily different solutions; they can share a structure, share properties, and fail for the same reason. And a single substitution can change a property a great deal. Sequence distance and functional distance are only loosely related.

This section is about generative design, not property prediction.

Why useful diversity is harder to establish

Biological measurements cover a small, selective sample of what works. A sequence database can hold billions of entries, while the experimental data for one property may cover a small set of related proteins, and evolutionary relatives inflate the sequence counts far beyond the number of independent examples. So a model can learn broad sequence patterns in pretraining and then specialize around a narrow experimental dataset during adaptation. That is a plausible route to collapse, though not evidence that any particular specialized model has taken it.

The reward is also harder to trust once you leave the measured examples. A model that predicts an experimental property can be accurate near its training data and unreliable on candidates that look different. The FLIP2 benchmark tests those shifts in data distribution and reports that generalization is still hard. Optimizing against such a predictor can concentrate generation around its errors as easily as around real successes. That failure has its own name, reward hacking, and it can happen with or without mode collapse. FLIP2

And useful biological performance has several dimensions at once. A candidate can score well on the property you measured and fail on another, or fail in a different context. Generate close relatives of one candidate over and over and you get correlated failures: many tests, little new information about the alternatives. That is the practical reason to care about diversity within the set of candidates that meet the task’s requirements, not sequence difference for its own sake.

What Profluent’s published work shows

Profluent’s ProGen3 work addresses data breadth and experimental alignment. During pretraining it rebalances related sequences so abundant families do not dominate as strongly. Larger models generate proteins from a broader range of families, and expression measurements support some of that breadth. That is a counterexample to treating “larger means less diverse” as a universal rule.

For alignment, ProGen3 uses iterative reasoning preference optimization (IRPO), adapted from language-model preference training. It learns from comparisons based on experimental measurements, with a loss that keeps the preferred examples probable. That regularizer keeps good candidates from being forgotten; it is not an explicit reward for generating diverse ones. In one generation evaluation, selected designs from the aligned 46B model showed improved expression for 8 of 32 prompts, decreased expression for 3, and no significant change for 21. Expression is a stability-related readout, not proof of every function you might want.

The results support broader generation with scale and useful adaptation to measurements. Whether alignment preserves the base model’s family coverage is untested. Settling it needs a matched diversity comparison before and after alignment. Bhatnagar et al., ProGen3

What would count as avoiding collapse?

Some of the language-model remedies do not transfer. A protein sequence model may have no instruction interface, so there is nothing to prompt for a list of alternatives. Raising the sampling temperature changes amino-acid choices, but more variation on its own says nothing about more functional designs. The principle that does transfer is to judge the variety of useful outcomes, rather than inferring it from sampling settings or raw sequence counts.

Three claims need different evidence:

Claim	Evidence it needs
The candidates differ	Diversity under a stated sequence or structural similarity measure, at a fixed generation budget
The candidates work	Relevant experimental measurements, with predicted scores reported separately
Post-training retains useful alternatives	A comparison with the starting model of diversity among successful candidates under comparable conditions

An independent comparison of 13 generative protein models illustrates this: structural diffusion models generally had higher predicted structural confidence and less diversity, while protein language models produced more varied designs with lower confidence. And predicted confidence is itself a prediction, not an experiment. Generative protein model benchmark

My reading is that bioAI makes the cost of collapse concrete in a way stories do not. A high-scoring candidate is valuable, but a set of useful alternatives is what you want when the score is incomplete and every experiment is expensive. ProGen3 shows real progress on breadth and on experimentally informed adaptation. Whether a given adaptation keeps that breadth is a separate empirical question, and for now an open one.