CodeCrafters Git Server Internals

If you've ever done a git push to a CodeCrafters repository, you've seen output like this:

This workflow is heavily inspired by Heroku, a platform that supports deploying applications via git push.

Curious about how this works under the hood? Read on to learn more.

Server-side Git Hooks

We use Git's server-side hooks feature to send output in response to a Git push.

If you're familiar with pre-commit hooks, server-side hooks are similar, just that they run on the Git server instead of your machine.

Hooks are scripts that are run when a git push is received.

For example, here's the script that prints the ASCII art when we receive a git push:

#!/bin/bash

echo -e "------------------------------------------------------------------------"
echo -e ""
echo -e ""

echo -e "\e[32m     ___            _          ___              __  _                   \e[0m"
echo -e "\e[32m    / __\ ___    __| |  ___   / __\_ __  __ _  / _|| |_  ___  _ __  ___ \e[0m"
echo -e "\e[32m   / /   / _ \  / _\` | / _ \ / /  | '__|/ _\` || |_ | __|/ _ \| '__|/ __|\e[0m"
echo -e "\e[32m  / /___| (_) || (_| ||  __// /___| |  | (_| ||  _|| |_|  __/| |   \__ \\e[0m"
echo -e "\e[32m  \____/ \___/  \__,_| \___|\____/|_|   \__,_||_|   \__|\___||_|   |___/\e[0m"

echo -e ""
echo -e ""
echo -e "   Welcome to \e[32mCodeCrafters\e[0m! Your commit was received successfully."
echo -e ""

echo -e "------------------------------------------------------------------------"

Symbols like \e[32m and \e[0m are ANSI escape sequences. They're used to add colour to the output.

Scheduling a test run

After printing the ASCII art, we notify our backend servers to schedule a test run for your code.

We use Firecracker microVMs to run tests. MicroVMs are quick to boot, they're the same technology used to power AWS Lambda.

These microVMs already contain an older version of your Git repository. This allows us to run an equivalent of git pull to fetch the latest changes you pushed, instead of having to run git clone all over again and download your full repository.

After scheduling a test run, we also push a notification to the front-end app at app.codecrafters.io via websockets. That's how you see the test run status changing live:

Streaming Logs

Since tests run on a separate machine than the Git server, the logs aren't directly accessible from our server-side Git hooks.

To make logs accessible to the Git server, we stream logs from the test runners to a central Redis server. The Git server then streams logs to your git client by reading them from the central Redis server.

Why not stream logs directly to the Git server? Why go through the intermediary Redis server? Two reasons:

1. Easy distribution: The streamed logs are consumed by multiple places — the Git server and the web UI. Pushing to a central server makes the distribution easier to handle. We use Redis Streams for this.

2. Simplifies implementation: By relying on a central Redis server, we offload the complexity of handling connection errors, interruptions, and other reliability concerns.

Test Results

After running tests, the final test result (pass/fail) is sent to our backend servers and subsequently used to update the UI at app.codecrafters.io

Summary

We covered how CodeCrafters streams test results on Git push.

• A server-side Git hook is used to print ASCII art when a git push is received
• Another Git hook schedules a remote a test run and streams logs via a central Redis server
• Once tests are complete, the final test result is recorded and used to update the UI at app.codecrafters.io