Pre-train Your First Autoregressive Mixture-of-Expert Based Protein Language Model
中文摘要
分享如何构建并预训练基于混合专家（MoE）的稀疏自回归蛋白质语言模型。
English Summary
Learn how to build and pre-train a sparse autoregressive protein language model based on Mixture-of-Experts (MoE).
Original Excerpt
Protein language models are still dense while the rest of the field went sparse. Here is how to build and pre-train a sparse one. Continue reading on Medium »