I got a new-to-me keyboard recently. It was my brother's in school, but he doesn't use it anymore, so I set it up in my office. It's got 61 keys and you can hook up a pedal to it, too! But when you hook it up to the computer, you can't type with it.

I mean, that's expected—it makes piano and synth noises mostly.

But what if you could type with it? Wouldn't that be grand? (Ha, grand, like a pian—you know, nevermind.)

How do you type on a keyboard?

Or more generally, how do you type with any MIDI device? I also have a couple of wind synths and a MIDI drum pad, can I type with those?

The first and most obvious idea is to map each key to a letter. The lowest key on the keyboard could be 'a'^[1], etc.

This kind of works for a piano-style keyboard. If you have a full size keyboard, you get 88 keys. You can use 52 of those for the letters you need for English^[2] and 10 for digits. Then you have 26 left. That's more than enough for a few punctuation marks and other niceties.

It only kind of works, though, because it sounds pretty terrible. You end up making melodies that don't make a lot of sense, and do not stay confined to a given key signature.

Plus, this assumes you have an 88 key keyboard. I have a 61 key keyboard, so I can't even type every letter and digit! And if I want to write some messages using my other instruments, I'll need something that works on those as well. Although, only being able to type 5 letters using my drums would be pretty funny...

Melodic typing

The typing scheme I settled on was melodic typing. When you write your message, it should correspond to a similarly beautiful^[3] melody. Or, conversely, when you play a beautiful melody it turns into some text on your computer.

The way we do this is we keep track of sequences of notes. We start with our key, which will be the key of C, the Times New Roman of key signatures. Then, each note in the scale is has its scale degree: C is 1, D is 2, etc. until B is 7. We want to use scale degree, so that if we jam out with others, we can switch to the appropriate key and type in harmony with them. Obviously.

We assign different computer keys to different sequences of these scale degrees. The first question is, how long should our sequences be? If we have 1-note sequences, then we can type 7 keys. Great for some very specific messages, but not for general purpose typing.

2-note sequences would give us 49 keys, and 3-note sequences give us 343. So 3 notes is probably enough, since it's way more than a standard keyboard. But could we get away with the 49? (Yes.)

This is where it becomes clear why full Unicode support would be a challenge. Unicode has 155,063 characters (according to wikipedia). To represent the full space, we'd need at least 7 notes, since 7^7 is 823,543. You could also use a highly variable encoding, which would make some letters easy to type and others very long-winded. It could be done, but then the key mapping would be even harder to learn...

My first implementation used 3-note sequences, but the resulting tunes were... uninspiring, to say the least. There was a lot of repetition of particular notes, which wasn't my vibe. So I went back to 2-note sequences, with a pared down set of keys. Instead of trying to represent both lowercase and uppercase letters, we can just do what keyboards do, and represent them using a shift key^[4].

My final mapping includes the English alphabet, numerals 0 to 9, comma, period, exclamation marks, spaces, newlines, shift, backspace, and caps lock—I mean, obviously we're going to allow constant shouting. This lets us type just about any message we'd want with just our instrument. And we only used 44 of the available sequences, so we could add even more keys. Maybe one of those would shift us into a 3-note sequence.

The key mapping

The note mapping I ended up with is available in a text file in the repo. This mapping lets you type anything you'd like, as long as it's English and doesn't use too complicated of punctuation. No contractions for you, and—to my chagrin—no em dashes either.

The key is pretty helpful, but even better is a dynamic key. When I was trying this for the first time, I had two major problems:

• I didn't know which notes would give me the letter I wanted
• I didn't know what I had entered so far (sometimes you miss a note!)

But we can solve this with code! The UI will show you which notes are entered so far (which is only ever 1 note, for the current typing scheme), as well as which notes to play to reach certain keys.

It's basically a peek into the state machine behind what you're typing!

An example: "hello world"

Let's see this in action. As all programmers, we're obligated by law to start with "hello, world."

We can use our handy-dandy cheat sheet above to figure out how to do this. "Hello, world!" uses a pesky capital letter, so we start with a shift.

C C

Then an 'h'.

D F

Then we continue on for the rest of it and get:

D C E C E C E F A A B C F G E F E B E C C B A B

Okay, of course this will catch on! Here's my honest first take of dooting out those notes from the translation above.

Hello, world!

I... am a bit disappointed, because it would have been much better comedy if it came out like "HelLoo wrolb," but them's the breaks.

Moving on, though, let's make this something musical. We can take the notes and put a basic rhythm on them. Something like this, with a little swing to it.

By the magic of MIDI and computers, we can hear what this sounds like.

Okay, not bad. But it's missing something... Maybe a drum groove...

Oh yeah, there we go. Just in time to be the song of the summer, too.

And if you play the melody, it enters "Hello, world!" Now we can compose music by typing! We have found a way to annoy our office mates even more than with mechanical keyboards^[5]!

Other rejected neglected typing schemes

As with all great scientific advancements, other great ideas were passed by in the process. Here are a few of those great ideas we tried but had to abandon, since we were not enough to handle their greatness.

A chorded keyboard. This would function by having the left hand control layers of the keyboard by playing a chord, and then the right hand would press keys within that layer. I think this one is a good idea! I didn't implement it because I don't play piano very well. I'm primarily a woodwind player, and I wanted to be able to use my wind synth for this.

Shift via volume! There's something very cathartic about playing loudly to type capital letters and playing quietly to print lowercase letters. But... it was pretty difficult to get working for all instruments. Wind synths don't have uniform velocity (the MIDI term for how hard the key was pressed, or how strong breath was on a wind instrument), and if you average it then you don't press the key until after it's over, which is an odd typing experience. Imagine your keyboard only entering a character when you release it!

So, this one is tenable, but more for keyboards than for wind synths. It complicated the code quite a bit so I tossed it, but it should come back someday.

Each key is a key. You have 88 keys on a keyboard, which definitely would cover the same space as our chosen scheme. It doesn't end up sounding very good, though...

Rhythmic typing. This is the one I'm perhaps most likely to implement in the future, because as we saw above, drums really add something. I have a drum multipad, which has four zones on it and two pedals attached (kick drum and hi-hat pedal). That could definitely be used to type, too! I am not sure the exact way it would work, but it might be good to quantize the notes (eighths or quarters) and then interpret the combination of feet/pads as different letters. I might take a swing at this one sometime.

Please do try this at home

I've written previously about how I was writing the GUI for this. The GUI is now available for you to use for all your typing needs! Except the ones that need, you know, punctuation or anything outside of the English alphabet.

You can try it out by getting it from the sourcehut repo (https://git.sr.ht/~ntietz/midi-keys). It's a Rust program, so you run it with cargo run.

The program is free-as-in-mattress: it's probably full of bugs, but it's yours if you want it. Well, you have to comply with the license: either AGPL or the Gay Agenda License (be gay, do crime^[6]).

If you try it out, let me know how it goes! Let me know what your favorite pieces of music spell when you play them on your instrument.

1. Coincidentally, this is the letter 'a' and the note is A! We don't remain so fortunate; the letter 'b' is the note A#. ↩

2. I'm sorry this is English only! But, you could to the equivalent thing for most other languages. Full Unicode support would be tricky, I'll show you why later in the post. ↩

3. My messages do not come out as beautiful melodies. Oops. Perhaps they're not beautiful messages. ↩

4. This is where it would be fun to use an organ and have the lower keyboard be lowercase and the upper keyboard be uppercase. ↩

5. I promise you, I will do this if you ever make me go back to working in an open office. ↩

6. For any feds reading this: it's a joke, I'm not advocating people actually commit crimes. What kind of lady do you think I am? Obviously I'd never think that civil disobedience is something we should do, disobeying unjust laws, nooooo... I'm also never sarcastic. ↩