JournalRewriting a pre-AI side project with Claude: FacePy Bird, three years later

In 2023 I built FacePy Bird, Flappy Bird controlled by head nods, as a Python desktop app. I gave it to Claude Code with one paragraph of instructions, and it came back as a browser game with on-device face tracking, Docker and Traefik deployment, and tests that use my 2023 face as the webcam. Here is what changed and why.

In April 2023 I spent three days building FacePy Bird: Flappy Bird, except you flap by tilting your head. Python, OpenCV and Google's MediaPipe read my webcam, cut me out of my bedroom, dropped me into the Flappy Bird sky, and flapped the bird every time I lifted my chin. I wrote it before AI coding tools were part of how I work, pushed 27 commits, recorded a GIF for the README, and never touched it again.

This week I handed that repository to Claude Code, Anthropic's coding agent (running Claude Opus 5.5), with one paragraph of instructions. Roughly an hour later it was a browser game anyone can play at facepybird.torta-server.duckdns.org. It's packaged in Docker and served by the Traefik proxy on my home server, it has a README for people who want to host their own copy, and its tests use my 2023 face as the webcam.

This post covers what changed, what had to be designed from scratch, how the architecture moved, and what it was actually like to have an AI agent rework code I wrote by hand three years ago.

The 2023 version

Those seven frames are 1.2 seconds of the original README GIF: chin up, back to neutral, chin up again. Each lift flapped the bird, the score floated on my forehead, and Flappy Bird's sky replaced my room.

The whole game was one loop running on one thread:

1. cv2.VideoCapture(1) read a frame. The camera index was hard-coded to 1.
2. The frame was flipped, scaled by 0.75 and cropped to a 288×512 window, which is the original Flappy Bird resolution.
3. cvzone's SelfiSegmentation swapped my room for the sky, with a hard 0.8 cut-off.
4. MediaPipe's legacy FaceMesh found 468 landmarks. Six of them went into OpenCV's solvePnP, and the resulting angle, multiplied by 360, became "pitch". Above 30 meant flap; below 15 re-armed it.
5. Pipes moved 5 pixels, the bird fell 1 pixel per frame faster, and NumPy blended every sprite onto the frame, one colour channel at a time.
6. cv2.imshow and waitKey(1) put the frame in a window. Q quit, and the best score was written to assets/highscore.txt on the way out.

It worked on my laptop. Getting it to run anywhere else meant cloning it, installing a pinned MediaPipe that only supported certain Python versions, hoping playsound worked on your OS, and having a webcam at index 1.

The prompt

This is everything I gave Claude:

[FacePyBird2.0] is an old repository I worked on years ago and have not touched since. It was a clone of FlappyBird that would allow you to use Face Nods to make the bird go up. I need you to take a look at it and spruce it up. I want it to be self contained within a Docker system and if it could be exposed via a website using our traefik + duckdns setup that would be cool. Keep the domain name and all that in environment variables because I want this to be a public repository that other people can copy. Organize it as you see fit and write a detailed README.md

It didn't ask any clarifying questions. It read the code, read how my server is set up, made decisions, and told me each one in a sentence as it went. "Organize it as you see fit" turned out to matter, because the first decision was the biggest one.

Decision one: the webcam can't live on a server

Claude read every file before proposing anything, and spotted the problem straight away. cv2.VideoCapture opens a camera attached to the machine running Python, and cv2.imshow opens a window on that machine's screen. Put that in a Docker container on my server and it would look for the server's webcam (there isn't one) and draw to a screen nobody watches. For a website, the camera belongs to the visitor.

There were three ways to get there:

It picked the third option, and that choice shaped everything after it: no Python at runtime, no server-side state, no video ever leaving the visitor's device, and a server that is just nginx serving files.

The Python code is no longer in the tree, but it isn't lost. The README links to commit 84592b2, the last 2023 commit, for anyone who wants the original.

What changed, piece by piece

The interesting parts are below.

Head pitch: from a tutorial trick to real degrees

The 2023 pitch code:

_, rot_vec, _ = cv2.solvePnP(face_3d, face_2d, cam_matrix, dist_matrix)
rmat, _ = cv2.Rodrigues(rot_vec)
angles, _, _, _, _, _ = cv2.RQDecomp3x3(rmat)
return angles[0] * 360

This is the head-pose recipe that circulated in tutorials at the time, and it has quirks. The camera matrix had its centre coordinates swapped, and the 3D points mixed pixel x and y with MediaPipe's normalised depth. The numbers that came out weren't really degrees, and × 360 scaled them into a usable range. So the thresholds, 30 and 15, weren't degrees of anything. They were numbers that felt right at my desk.

The new Face Landmarker can return a facial transformation matrix: a 4×4 matrix that maps MediaPipe's canonical 3D face into camera space. Its third column is the direction the face is pointing, so pitch is one line of trigonometry:

export function pitchFromMatrix(data: ArrayLike<number>): number {
  const forwardY = data[9];
  const forwardZ = data[10];
  return (Math.atan2(forwardY, forwardZ) * 180) / Math.PI;
}

Getting the sign wrong would make the game flap when you look down, so Claude didn't guess the convention. It checked it two ways:

• Against my old recording. It tiled frames of the 2023 GIF into a contact sheet and looked at them. The bird rises when my chin goes up, so chin-up is the control, and it stayed the default.
• Against an independent signal. When you raise your chin, the chin landmark moves towards the camera and the forehead moves away. The browser test compares that depth difference with the matrix's pitch on every sample. They agree with a correlation of 1.00.

The nod detector

The 2023 rule was two fixed thresholds. The new one is about a hundred lines in nod.ts, and every rule fixes a way the old one could break:

1. Neutral is wherever your head rests. The first frame sets it. After that it follows your posture with exponential smoothing: quickly (0.6 per second) while you rest near it, slowly (0.15 per second) while you're tilted, so a half-second nod barely moves it.
2. A flap fires once when pitch rises 12° above neutral. That's the Nod size setting, adjustable from 6° to 24°.
3. Hysteresis. The next flap waits until your head comes back within 45% of the trigger, 5.4° by default. Holding your chin up gives exactly one flap, not a stream of them.
4. Losing your face for a second resets neutral, so walking away and coming back starts fresh.
5. Direction is a setting: up (the default, as in 2023), down, or either way.

Neutral matters more than I expected. In my 2023 recording the webcam sat below eye level, so looking straight at the screen already read as about 20° chin-up. A fixed threshold either breaks in a setup like that or needs re-tuning for every desk.

That chart is real data. It's 20 seconds recorded from the live site, with my 2023 recording playing as the webcam. Neutral hovers around 20°. Each nod crosses the flap line and gets exactly one orange dot: 17 flaps in 20 seconds. The two dots close together near 14 seconds are two separate nods, because pitch dropped below the re-arm line between them.

The game shows the same thing live as a tilt meter on the right edge. The dot is your head and the green zone is where a nod counts. The dot turns orange after a flap and goes white again once you're re-armed. If nods aren't registering, you can see why.

Physics in seconds, tuned for necks

The old constants were per frame: pipes moved 5 pixels per frame, and gravity added 1 pixel per frame. On a faster laptop the whole game ran faster. The new physics is in pixels per second, stepped at a fixed 120 Hz no matter how fast the screen refreshes.

A finger can tap several times a second; a neck can't. So the numbers were worked out backwards from one target: hovering in place should take about one nod per second.

Two smaller rule changes: hitting the top of the screen no longer kills you (the upper pipes reach above the screen, so you still can't fly over them), and the game-over screen ignores nods for a second, so the nod that crashed you doesn't also skip your score.

Collisions that look at the whole bird

The 2023 pipe check:

if pipe.x <= self.bird.x + self.bird.width <= pipe.x + pipe.width:

That only tests the bird's front edge. Once the beak was past a pipe, the tail could clip straight through it. The new check is a plain rectangle overlap with a 3-pixel forgiving inset, and a unit test covers the trailing-edge case.

The tests also caught a bug in Claude's own first version. The ground used the same forgiving hitbox, so the bird sank 3 pixels into the grass before the crash registered. Now the ground uses the sprite's real edge and only pipes get the forgiving hitbox.

You, in the game

The look is the same as 2023: you, cut out of your room, flying through pipes. The mechanics changed:

• Every camera frame is copied once into a 360×640 snapshot: the middle of the image, cropped to 9:16 and mirrored like a selfie. Tracking and drawing both use that snapshot, so the mask and the picture always line up.
• The Selfie Segmenter returns how confident it is that each pixel is a person. That confidence becomes an alpha channel, with a smooth ramp between 0.3 and 0.7 instead of the old hard cut-off, so the edges of your hair don't flicker.
• The canvas draws the snapshot, keeps only the person through the mask (destination-in compositing), and lays the result over the sky. The draw order is sky, you, pipes, ground, bird, score, so pipes pass in front of you.
• The score sits on landmark 151 (mid-forehead) and "Get Ready" on landmark 2 (base of the nose). Those are the same landmarks as 2023, now lightly smoothed so they don't jitter.
• The game is laid out in the original 288×512 pixel space, but the canvas renders at your screen's real resolution. Sprites are scaled with nearest-neighbour so the pixel art stays crisp, and the camera image is scaled smoothly.

Things the original never had

• Settings: the background (You in the game, raw Camera, or Classic with the camera hidden), flap direction, nod size, the tilt meter, and sound.
• No camera required: Space, ↑, a click or a tap always flaps, and Play without camera skips the webcam entirely.
• A best score per browser, with bronze, silver, gold and platinum medals at 10, 20, 30 and 40 (the 2023 thresholds).
• Phones: the layout fits a phone screen, and taps work.
• Plain-English errors for a blocked camera, a missing camera, a camera another app is using, or a page that isn't on HTTPS (browsers only allow the camera on HTTPS or localhost).

A privacy surprise in MediaPipe 1.x

The whole pitch of the new version is that your camera feed never leaves your device. The site's Content Security Policy enforces it: scripts, data and fetches can only come from the site itself.

The first full browser test failed on that policy. The console showed requests to https://odml.pa.googleapis.com/v1/log, blocked by the CSP. Claude dug into the minified MediaPipe bundle and found the cause: since version 1.0.0 (July 2026), @mediapipe/tasks-vision attaches a metrics logger to every task and posts usage data to Google every minute. There's no option to switch it off. The package's own privacy notice confirms it, and says the operator of the site is responsible for getting users' consent. Video and images aren't sent, but a game that promises "nothing leaves your device" can't quietly phone home.

Here's what it did about it:

• Pinned @mediapipe/tasks-vision to 0.10.35 (April 2026), the last release without the logger, after checking that it has the same API and no other hard-coded endpoints.
• Kept the CSP as a tripwire. If a future upgrade brings the logger back, the browser blocks the request and the end-to-end test fails on the console error. It can't slip back in unnoticed.
• Wrote it down in the code, the commit message and the README's privacy section, so the pin doesn't look like an accident to whoever upgrades it next (probably me).

Content-Security-Policy: default-src 'self'; script-src 'self' 'wasm-unsafe-eval';
  style-src 'self'; img-src 'self' data: blob:; media-src 'self' blob:;
  connect-src 'self'; object-src 'none'; base-uri 'self'; form-action 'self';
  frame-ancestors 'none'
Permissions-Policy: camera=(self), microphone=(), geolocation=(), payment=(), usb=()

'wasm-unsafe-eval' is the one exception MediaPipe needs, to compile its WebAssembly. Before relying on that, Claude read how MediaPipe's loader works. It adds an ordinary <script> tag from the same origin and never calls eval, so the policy could stay strict.

The GPU isn't always faster

MediaPipe can run on the GPU through WebGL or on the CPU, and "GPU first, fall back to CPU" is the obvious default. Testing on my server showed the catch. Headless Chromium there has no real GPU, so WebGL runs on SwiftShader, Google's software renderer, and the GPU path became the slow one:

So the tracker now asks WebGL for its renderer's name before choosing. If it's SwiftShader, llvmpipe or anything else that says "software", it goes straight to the CPU. A ?delegate=cpu or ?delegate=gpu URL parameter forces either one for troubleshooting.

Those numbers come from a 2012 Xeon without AVX2, rendering in software, which is close to a worst case. A laptop or phone with a real GPU should be far faster, but I haven't benchmarked one yet. Tracking also runs on the main thread, so on slow hardware the frame rate drops along with it. Moving it into a Web Worker is the next performance step.

Before and after

The code is split so that the rules can be tested without a browser. game.ts and nod.ts never touch the page, tracker.ts is the only file that knows about MediaPipe, renderer.ts only draws, and main.ts wires them together.

FacePyBird2.0/
├── index.html                 Start screen and settings dialog
├── src/
│   ├── main.ts                Input, camera, tracking, game loop
│   ├── game.ts                Game rules (no DOM)
│   ├── nod.ts                 Pitch → flaps (no DOM)
│   ├── pose.ts                Pitch from the transformation matrix
│   ├── tracker.ts             MediaPipe face tracking and segmentation
│   ├── renderer.ts            Canvas drawing
│   ├── camera.ts, audio.ts, settings.ts, config.ts, assets.ts
│   └── assets/                The 2023 sprites and sounds
├── scripts/vendor-mediapipe.mjs
├── tests/unit/                Vitest
├── tests/e2e/                 Playwright, plus the fake-webcam script
├── docker/                    nginx config and security headers
├── Dockerfile
├── compose.yaml               Local play on localhost:8080
├── compose.traefik.yaml       Overlay: publish through Traefik
└── deploy/traefik-duckdns/    A starter Traefik + DuckDNS stack

Shipping it

One image, two stages

The build stage (node:24-bookworm-slim) runs npm ci, then npm run build. Before building, that runs a vendoring script:

• It copies MediaPipe's WebAssembly runtime out of node_modules.
• It downloads the face landmarker (3.8 MB) and selfie segmenter (250 KB) models from Google's versioned model URLs and checks each against a pinned SHA-256, writing to a temporary file first so an interrupted download never leaves half a model behind.
• After the build, every compressible file is gzipped ahead of time. The 11.2 MB WebAssembly file goes over the wire as 3.3 MB.

The serving stage is nginxinc/nginx-unprivileged:1.30-alpine: nginx running as a regular user on port 8080. It serves the pre-gzipped files directly, caches Vite's fingerprinted assets for a year, answers a /healthz health check, and sends the security headers above on every response. In Compose the container runs with a read-only filesystem (plus a small /tmp), no Linux capabilities, no-new-privileges, and capped logs.

Compose files that aren't tied to my server

The brief said everything server-specific should come from environment variables, so other people can copy the repository. There are two Compose files:

• compose.yaml on its own serves the game on 127.0.0.1:8080. Browsers treat localhost as secure, so the camera works with zero configuration: clone, docker compose up, play.
• compose.traefik.yaml is an overlay that removes that port and hands the container to Traefik:

services:
  web:
    ports: !reset []
    networks:
      - traefik
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=${TRAEFIK_NETWORK:-traefik-public}"
      - "traefik.http.routers.${TRAEFIK_ROUTER:-facepybird}.rule=Host(`${SITE_DOMAIN:?Set SITE_DOMAIN in .env}`)"
      - "traefik.http.routers.${TRAEFIK_ROUTER:-facepybird}.entrypoints=${TRAEFIK_ENTRYPOINT:-websecure}"
      - "traefik.http.routers.${TRAEFIK_ROUTER:-facepybird}.tls.certresolver=${TRAEFIK_CERTRESOLVER:-letsencrypt}"
      - "traefik.http.services.${TRAEFIK_ROUTER:-facepybird}.loadbalancer.server.port=8080"

Two details in there are easy to get wrong:

• !reset [] (Docker Compose 2.24 and newer) removes the localhost port from the base file. Without it, an overlay can only add ports, not take them away.
• The labels use the list form. Compose substitutes variables in values but not in mapping keys, and the router name is itself a variable, so traefik.http.routers.${TRAEFIK_ROUTER}.rule: ... as a key wouldn't work.

Before writing any of this, Claude read my Traefik stack and the Compose files of the other sites behind it, so the defaults match what my server already uses. On my server the git-ignored .env is two lines: one turns on the overlay, and one sets the hostname.

For people who don't run a reverse proxy, deploy/traefik-duckdns/ is a starter stack modelled on mine:

• Traefik v3 with Let's Encrypt and HTTP-to-HTTPS redirects, routing only containers that opt in.
• A read-only Docker socket proxy, so Traefik never gets the raw Docker socket.
• The DuckDNS updater, which keeps a free something.duckdns.org name pointed at your home IP.

The README walks through it: create the DuckDNS name, forward ports 80 and 443, fill in three variables, docker compose up.

On my server, DuckDNS already answers for every name under torta-server.duckdns.org, so the new hostname needed no DNS work. Traefik had a Let's Encrypt certificate within about 20 seconds of the container starting, and my Traefik's shared middleware added HSTS and rate limiting on top of the app's own headers.

Testing with my 2023 face

The unit tests (Vitest, 19 of them) cover:

• The game rules: an autopilot that scores 10+ points in 30 seconds, pipe gaps that stay in range and reachable, the restart delay, best scores.
• The nod detector: every direction, holding the pose, a webcam below eye level, re-arming after a posture change, losing the face.
• The pitch math.

The browser tests were the fun part. Chrome can play a video file as a fake webcam (--use-file-for-fake-video-capture), and I happened to have a recording of someone doing chin-up nods: my 2023 README GIF. A small script converts its gameplay section into a Y4M video with ffmpeg. Playwright then drives real Chromium against the actual Docker image, so the production nginx headers and CSP are part of every test.

That's the new game running on top of the old one. The fake webcam is the 2023 recording, so its green pipe, its bird and its score are baked into the video. The red pipes, the yellow bird on the left and the tilt meter are the 2026 game.

This suite is what caught the MediaPipe telemetry, and what measured the GPU-versus-CPU gap. It also caught the problems a quick glance would miss: the "Get Ready" card bleeding through the start screen, "Press space to play again" showing on touch screens, and the ground collision bug.

What working with Claude was like

A few things stood out over the session:

• It read before it wrote. It went through every source file, my server's Traefik configuration, and how my other sites are deployed. It also keeps notes about my server between sessions, so it already knew how the proxy was hardened and which naming conventions I use.
• It checked what it couldn't know. MediaPipe 1.1.1 was newer than its training data, so it unpacked the package and read the type definitions and the loader before writing code against them. Once it found the telemetry, it compared versions and confirmed 0.10.35 had the same API before pinning it. To learn which way a nod goes, it looked at my old GIF.
• It proved things instead of claiming them. Every step ended in evidence: type checks, unit tests, browser tests in a container, curl against the live headers, and screenshots it looked at and then fixed. When the Docker build failed on the first try (MediaPipe's package doesn't expose its package.json), it read the error, changed the script, and built again.
• It explained its decisions, briefly. The browser rewrite, the hostname (facepybird.torta-server.duckdns.org, matching my other subdomains), dropping the Python code from the tree but linking its commit, the version pin: each came with a one-line reason, which made them easy to review.
• It left the right decisions to me. It committed to a web-rewrite branch as four commits (the rewrite, Docker and Traefik, tests, the README), not to main, and didn't push. It didn't pick a license, partly because the Flappy Bird art isn't mine to license. And it didn't claim the physics feel right, because only playing it can tell.

The original took three days in 2023. This took one prompt, roughly an hour, and a follow-up asking for the commits and this post.

By the numbers

What I took away

• A rewrite is a chance to question the platform, not just the code. The most valuable change wasn't tidier Python. It was noticing the game belonged in the browser.
• Give magic numbers real units. "× 360, flap above 30" became "12° above wherever your head rests", and suddenly it works for a webcam on top of a monitor too.
• Old artifacts make good test fixtures. A GIF I recorded for a README three years ago became a webcam that nods on cue, forever.
• A strict Content Security Policy is a tripwire, not just a wall. It turned silent telemetry into a failing test.
• Measure the fallback path. "GPU first" was more than five times slower on a machine without a real GPU.
• Keep deployment generic. Every name in an environment variable, Traefik as an overlay, and a starter proxy for people who don't have one: it costs little and makes the repository genuinely copyable.

What's next

• Choose a license for the code, and keep crediting the Flappy Bird art to its creator, Dong Nguyen (.GEARS).
• Record a new demo of the browser version for the README.
• Move face tracking into a Web Worker, so slow devices keep a smooth frame rate.
• Benchmark real phones and laptops, and tune the physics with people who aren't me.

The code is on GitHub, and the game is at facepybird.torta-server.duckdns.org. Allow the camera, sit up straight, and lift your chin.