Another week of my RC batch wraps up. I'm done with five weeks, and seven weeks are left! Time is still flying by, and I've hit an inflection point. I have gotten what I want out of the two projects I've worked on so far, so I'm going to wrap them up and move on to one new project for the rest of the batch.

A very social week

This week I did a lot of social things:

• 7 pairing sessions
• 8 coffee chats
• Went to an ML event
• Went to a theorem proving event (my brain is melted)

This was good, but I want to tone it down a little on the pairing and chats next week. I need to recharge, and I need to build a little more time for individual think time. Also for walks. Walks are nice.

Winding down my current projects

So far, I've worked on a key-value store and a chess engine. My goal with these was to learn about how systems programs are written in a way that's efficient and extensible. I think I've gotten there!

The key-value store supports a basic but useful subset of Redis commands, and its performance outpaces Redis (admittedly by using multiple threads, but single-thread performance nearly matches Redis even with write-ahead logging enabled). The next steps here would be to add more Redis commands or decide on some other more interesting features to add. The performance constraints aren't particularly interesting; it's useful as-is if I made it complete and production ready, and I don't want to focus on making something production-ready.

And the chess engine is strong enough to beat me if I'm not very careful, although I can beat it if I pay attention and give it my best. The next steps here would be to add more things like quiescence search and iterative deepening. This is actually still very interesting to me! But I want to learn about data-intensive applications, and I can't focus on a new project while I keep working on the chess engine. I'm going to come back to the chess engine sometime, either at the end of my batch or after it. But I'm putting it on pause for now to give myself space to explore the new project.

The new project

I was toying with adding some features to the key-value store that would require some indexing, some disk read/writes, some user interactive queries. It all felt artificial, and it was a lot of design but in a space that had no real constraints, because there was no use case. Ultimately the goal isn't to add a feature to the key-value store. The goal is to learn about things like indexing, database query languages, data access patterns when it won't fit in memory.

What I need is a project that combines my learning goals with some practical, tangible problem that will impose constraints and give actual user requirements. I'm going to get that through creating a chess database. Sometimes "chess database" can mean "a bunch of PGN files in a collection," but here I mean a database tailored to holding chess games and doing analysis on them.

The raw data that I have for master-level games is about 7 GB on disk. Lichess also has an open database of games, with each month of data being about 25 GB. If I want to do any sort of database analysis across play below the master level, this will quickly be larger than fits in RAM. (But I'm going to start with master-level play for now.)

I've created the project hub for IsabellaDB (maybe I registered a domain as well, but let's not talk about that). If you're interested in pairing on it as it comes to live, reach out to me!

Some features I want to implement (some posed as questions I'd like to be able to answer):

• Standard chess database features:
  • Explore openings and see what the win/lose/draw percentages are
  • Detect when a given game has reached a novel position (eventually an integration into live broadcasts of games?)
  • Filter/search by player name, rating, event, and other metadata
• For a given position, what other main-line positions can it transpose back into?
• If a player plays X opening as white, what do they typically play as black?
• For a given player, what is their repertoire? (potential integration with chess.com and lichess APIs to do some prep on real-life opponents!)
• Find the games in which there's a queen sacrifice
• Find positions where there is a battery, a pin, or another tactic available
  • Can this be used to construct tactics puzzles from real games?

I'm super excited to start working on this! I'm also a little overwhelmed, because it's a lot, and it's very much at the edge of my abilities right now. I went down a rabbit hole today on how to approach the UI, since a chess database needs something visual. Ultimately, on the advice of a couple of folks, I settled on just doing the most basic thing I can for the UI and then if I find more need for interactivity later, adding it on. Static results pages it is!

What's in store for next week?

Next week is all about shifting focus to my new project. I'm going to write up an initial plan, get some review on the plan, and start implementation!

• Keep pairing every day, keep coffee chats every day, but not too many days of doubling up!
• Write a blog post! I'm switching from GitHub to Sourcehut, and I want to talk about why
• Start on IsabellaDB:
  • Write up a design for basic functionality: opening explorer, and searching games by metadata, and being able to scroll through the state of a game
  • Implement these features!
  • Pair with at least one person on implementing this stuff
• Read another Red Book paper
• Read the first chapter of Software Foundations: Logical Foundations. (We're switching to this instead of the Lean book. Hopefully brains are less melted.)
• Go to some of the other events, like creative coding and leetcode

It's going to be a full week, and I'm excited to get started. But first, it's time for a restful weekend. I'll be spending some time away from computers out in the workshop.

See you next week!