Loading...

Build your own Inference Engine

Learn about attention, the transformer, and more

COMING SOON

This is a preview of a CodeCrafters challenge that is launching soon.

Follow us on Twitter to be notified when this challenge is available.

Why does "The capital of France is" always continue with the same words? GPT-2 gives every possible next word a score, picks the word with the highest score, and repeats. An inference engine is the program that runs a trained model. It loads the model's weights, its tables of numbers, and does the arithmetic that turns a prompt into the next word. In this challenge, you'll build one for GPT-2, load its real weights, and get the same words GPT-2 writes.

Along the way, you'll learn about attention, the transformer, and more.

Stages

Read safetensors
Login via GitHub to view this stage
Embed tokens
Login via GitHub to view this stage
Causal attention
Login via GitHub to view this stage
Multi-head attention
Login via GitHub to view this stage
Residual connections
Login via GitHub to view this stage
Transformer block
Login via GitHub to view this stage
Logits
Login via GitHub to view this stage
Greedy decode
Login via GitHub to view this stage
Sample a token
Login via GitHub to view this stage
KV cache
Login via GitHub to view this stage
Run GPT-2
Login via GitHub to view this stage