Reading
Papers and posts I have read, most recent first. Each has a short overview of the idea and where it could be useful, plus a rating and my thoughts as I add them. Titles link to the original. You can search the shelf, filter it by topic, sort it by rating or by when a paper came out, or let it pick one at random.
Scaling Unlocks Broader Generation and Deeper Functional Understanding of Proteins
Bhatnagar et al. · 2025 · bioRxiv v2; also published at NeurIPS 2025
Protein design · Model training · background · method to try
Overview
Use when evaluating model scale for protein design: larger models can cover more protein families and benefit more from experimental post-training even when their unadapted fitness predictions get worse.
My thoughts
I think it's a very bitter lesson-pilled paper. I think that it is very interesting to think about these ways to to get models that are better at representing and generalizing to more sparse distributions (as represented by the clusters) and how (unfortunately), so much of how we get there comes from scale: scaling the model size, scaling the amount of data. I did like the idea of how to make an AR model still be able to infill sections through the sentinels, but I was left thinking a lot about how we get to the sequences that are inherently harder (as mentioned in the paper, there was a super strong correlation between sequence identity and its relation to existing and known sequences).
My biggest takeaway is that these larger models are what is functionally necessary for describing the complexities of biology: they needed bigger models to have better coverage of the clusters, they needed a bigger and more trustworthy in order to run IRPO, rather than a weaker preference optimizer like IPO. Something interesting to me personally is how we can approximate this scale with concepts like co-distillation/specialist models/directing model learning, but I think that's a very long-run future. Great paper and, of course, loved the wet lab validation!
Robust deep learning–based protein sequence design using ProteinMPNN
Dauparas et al. · 2022 · Science
Protein design · baseline
Overview
Generates sequences for a given protein backbone, useful for designing new proteins or improving existing designs.
My thoughts
Out of a little bit of spite that modern LLMs tend to blow classical ML techniques out of the water, I am a bit of a hater of how so many problems are fixed by "errr just add more data and parameters."
To that end, I think it's really neat that ProteinMPNN is an effective small model and how accurate heurestics can often improve the ROI on output / parameters. The representation of the backbone changes and the outsized impact is felt (particularly since there is my most beloved "wet lab data validation.") I like having this to round out the other papers that focus more on scaling protein models; it gives me more to think about than model size alone.
It also makes sequence design feel like something other methods can build on. A backbone generator can hand its output to ProteinMPNN instead of solving sequence design too. As techniques like co-distillation and attention residuals (in frontier models!) come to the surface, I'm really interested in seeing how far that division of work can go as models that generate sequence and structure together improve.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Rafailov et al. · 2023 · NeurIPS
Model training · Protein design · background
Overview
Trains models directly on preferred and rejected examples, useful for post-training without a separate reward model.
My thoughts
I think DPO is pretty neat as a first foray into "real" post-training. It feels like a stepsize improvement over its "just a little worse" counterpart in SFT while still keeping the main benefits (no separate reward model). It's nice to be able to think about generalizable improvements and connecting data together.
I guess a concern is that there are definitely biases in this structure -- there's some sensitivity to be gained from knowing how much of a difference between preferred and dispreferred answers (like contrastive pretraining). The model gives margin but I think it's better if the data can give the margin.
Aligning protein-generative models to experimental fitness with ProteinDPO
Widatalla et al. · 2026 · Nature Methods
Protein design · Reinforcement learning · background · method to try
Overview
Uses experimental stability measurements to fine-tune a protein generator, useful for proposing more stable sequences.
My thoughts
I think that a preexisting problem with protein language models (perhaps bioML as a whole) is the messiness of the data: datasets are incredibly biased (ex. look at how many sequences there are for HER2 vs other proteins!) and there's constantly new papers coming out about how every biology foundation model is a memorizer. If we think about post-training as a way to reshape the distribution, I think that there's an incredibly strong natural fit for how to get these models to be even more applicable.
This paper was very interesting to me for that reason, it was able to unify this preexisting protein model with new information in order to reshape the outputs. I particularly liked the distinction between this and SFT, since (in my biased opinion) SFT has always seemed to produce models that can imitate, not understand.
However, (in connection with a lot of papers I've been reading about offline vs online), I do feel like this paper is a good step in the right direction but using a fixed dataset shows me that there's so much more to gain -- if we can find a way to make these foundation models learn from online (or even semi-online!) environments, the output would be monumental.
Proteína-Complexa: Scaling Atomistic Protein Binder Design with Generative Pretraining and Test-Time Compute
Kieran Didi, NVIDIA · 2026 · arXiv (ICLR 2026, oral)
Protein design
Overview
Generates protein sequences and structures together, using guided search to propose binders for a chosen target.
My thoughts
The part I liked the most is how compute is used more intelligently for this difficult domain rather than just "throw more GPUs at more parameters." I think the architecture is pretty sophisticated (more than my initial thought of "well just combine some ideas that work") and it's interesting to think about how compute is still a bottleneck for generating candidates that eventually go into this 10-year long approval process.
Reading it alongside the recently (9/18/26) released companion wet-lab validation definitely goes to show why it is so important to do wet-lab. For example, in silico, redesign improved the benchmark but, in practice, the codesigned sequences performed better in the wet lab. It does just goes to show how difficult and how much of a million dollar question it is to improve the throughput and veracity of verifiability in biology.
On-Policy Distillation
Kevin Lu, Thinking Machines Lab · 2025 · Thinking Machines Lab: Connectionism
Reinforcement learning · Model training · background
Overview
Trains a student on its own outputs using teacher feedback, useful for transferring capabilities from a stronger model.
My thoughts
Really interesting to think about the impact of having a strong teacher; particularly how it's really important (honestly, mandatory) to have a teacher that the student can get feedback on throughout an answer.
For biology specifically, I thought it's a really interesting question to think about where you can find a "good teacher." Just another reason why bioML specifically is pretty difficult and why distillation in particular has been very helpful (making discoveries cheaper to reproduce). It's really interesting to think about how the architecture for bioML (often initialized from encoder-type models that have no next-token conditional) along with how the compaction might be super disrupting for the delicate understanding of diversity for bio.
GermRL: Alleviating The Germline Bias In Autoregressive Antibody Language Models Through Reinforcement Learning
Ludwig et al. · 2026 · bioRxiv preprint
background · method to try
Overview
Adds terms to GRPO to have pretrained model (ProGen2) be steered away from germline while trying to mitigate the diversity collapse more traditional GRPO gave.
My thoughts
I think the most interesting component of this was talking about what it really meant for entropy collapse (making the clipping limit easier to reach). However, I felt like much of the work that was done in this paper was too optimized for the work they were doing with little room to explore how the method could be applied to more translational aspects of antibody design. They had the advantage of a very easy verifier (Levenshtein distance) and were able to customize diversity demands (reducing a sequence's reward when its V gene had already appeared frequently during the epoch). Leaves the question a lot more open about whether the steering produced more useful antibodies.
I think it would've been important to explore its downstream rewards rather than just its directly computed idea (don't favor germline sequences).
Nothing matches.