ProtoMech
Published:
Discovering Computational Circuits in Protein Language Models via Cross-Layer Transcoders
Mechanistic Interpretability · Protein Language Models · Cross-Layer Transcoders · Circuit Discovery — ICML 2026
Project website Paper (arXiv) ICML poster LessWrong
Protein language models (pLMs) have emerged as powerful predictors of protein structure and function, yet the computational circuits underlying their predictions remain poorly understood. ProtoMech applies cross-layer transcoders to discover interpretable circuits inside pLMs.
Applied to the pLM ESM2, ProtoMech recovers 82–89% of the original model’s performance while exposing human-interpretable circuits. Steering along these circuits enables high-fitness protein design, surpassing baseline methods in more than 70% of cases.
