CodeCrafters CodeCrafters

Build your own Kafka

Learn about TCP servers, the Kafka wire protocol and more.

Start Building
kafka
JavaScript
kafka
Gleam
kafka
Go
kafka
Python
kafka
Rust
This challenge is free to try when it's in beta. We keep challenges in beta for a few weeks to gather feedback.

Apache Kafka is a distributed event streaming platform often used for high-performance data pipelines. In this challenge, you'll build your own Kafka broker that's capable of serving basic requests.

Along the way you'll learn about TCP servers, the Kafka wire protocol and more.

Stages

Bind to a port
Login via GitHub to view this stage
Send Correlation ID
Login via GitHub to view this stage
Parse Correlation ID
Login via GitHub to view this stage
Parse API Version
Login via GitHub to view this stage
Handle APIVersions requests
Login via GitHub to view this stage

Concurrent Clients

Serial requests
Login via GitHub to view this stage
Concurrent requests
Login via GitHub to view this stage

Consuming Messages

Include Fetch in APIVersions
Login via GitHub to view this stage
Fetch with no topics
Login via GitHub to view this stage
Fetch with an unknown topic
Login via GitHub to view this stage
Fetch with an empty topic
Login via GitHub to view this stage
Fetch single message from disk
Login via GitHub to view this stage
Fetch multiple messages from disk
Login via GitHub to view this stage
“ There are few sites I like as much that have a step by step guide. The real-time feedback is so good, it's creepy! avatar Ananthalakshmi Sankar Automation Engineer at Apple “ I think the instant feedback right there in the git push is really cool. Didn't even know that was possible! avatar Patrick Burris Senior Software Developer, CenturyLink