Learn about attention, the transformer, and more
This is a preview of a CodeCrafters challenge that is launching soon.
Follow us on Twitter to be notified when this challenge is available.
Why does "The capital of France is" always continue with the same words? GPT-2 gives every possible next word a score, picks the word with the highest score, and repeats. An inference engine is the program that runs a trained model. It loads the model's weights, its tables of numbers, and does the arithmetic that turns a prompt into the next word. In this challenge, you'll build one for GPT-2, load its real weights, and get the same words GPT-2 writes.
Along the way, you'll learn about attention, the transformer, and more.
This is a preview of a CodeCrafters challenge that is launching soon.
Follow us on Twitter to be notified when this challenge is available.