Annotated Research Paper Implementations: Transformers, StyleGAN, Stable Diffusion, DDPM/DDIM, LayerNorm, Nucleus Sampling and more

This is a collection of simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations, and the website renders these as side-by-side formatted notes. We believe these would help you understand these algorithms better.

Screenshot

We are actively maintaining this repo and adding new implementations. Twitter for updates.

Translations

English (original)

Chinese (translated)

Japanese (translated)

Paper Implementations

Transformers

Eleuther GPT-NeoX

Diffusion models

Generative Adversarial Networks

Recurrent Highway Networks

LSTM

HyperNetworks - HyperLSTM

ResNet

ConvMixer

Capsule Networks

U-Net

Sketch RNN

✨ Graph Neural Networks

Reinforcement Learning

Counterfactual Regret Minimization (CFR)

Solving games with incomplete information such as poker with CFR.

Optimizers

Normalization Layers

Distillation

Adaptive Computation

Uncertainty

Activations

Language Model Sampling Techniques

Scalable Training/Inference

Highlighted Research Paper PDFs

Installation

pip install labml-nn

Citing LabML

If you use this for academic research, please cite it using the following BibTeX entry.

@misc{labml,
 author = {Varuna Jayasiri, Nipun Wijerathne},
 title = {labml.ai Annotated Paper Implementations},
 year = {2020},
 url = {},
}