Sampling Techniques for Language Models

• Greedy Sampling
• Temperature Sampling
• Top-k Sampling
• Nucleus Sampling

Here's an experiment that uses these sampling techniques.

Sampler base class

Sample from logits

• logits are the logits of the distribution of shape [..., n_tokens]