Beginning of a MIDI GUI in Rust
Monday, January 13, 2025
A project I'm working on (which is definitely not my SIGBOVIK submission for this year, and definitely not about computer ergonomics) requires me to use MIDI. And to do custom handling of it. So I need something that receives those MIDI events and handles them.
But... I'm going to make mistakes along the way, and a terminal program isn't very interesting for a presentation. So of course, this program also needs a UI.
This should be simple, right? Just a little UI to show things as they come in, should be easy, yeah?
Hahahaha. Haha. Ha. Ha. Whoops.
The initial plan
Who am I kidding?
There was no plan. I sat down with egui's docs open in a tab and started just writing a UI.
After a few false starts with this—it turns out, sitting down without a plan is a recipe for not doing anything at all—I finally asked some friends to talk through it. I had two short meetings with two different friends. Talking it through with them forced me to figure out ahead of time what I wanted, which made the problems much clearer.
Laying out requirements
Our goal here is twofold: to provide a debug UI to show MIDI messages and connected devices, and to serve as scaffolding for future MIDI shenanigans. A few requirements fall out of this1:
- Display each incoming MIDI message in a list in the UI
- Display each connected MIDI device in a list in the UI
- Allow filtering of MIDI messages by type in the UI
- Provide a convenient hook to add subscribers, unrelated to the UI, which receive all MIDI messages
- Allow the subscribers to choose which device categories they receive messages from (these categories are things like "piano," "drums", or "wind synth")
- Dynamically detect MIDI devices, handling the attachment or removal of devices
- Minimize duplication of responsibility (cloning data is fine, but I want one source of truth for incoming data, not multiple)
Now that we have some requirements, we can think about how this would be implemented. We'll need some sort of routing system. And the UI will need state, which I think should be separate from the state of the core application which handles the routing. That is, I want to make it so that the UI is a subscriber just like all the other subscribers are.
Jumping ahead a bit, here's what it looks like when two of my MIDI devices are connected, after playing a few notes.
Current architecture
The architecture has three main components: the MIDI daemon, the message subscribers, and the GUI.
The MIDI daemon is the core that handles detecting MIDI devices and sets up message routing. This daemon owns the connections for each port2, and periodically checks if devices are still connected or not. When it detects a new device, it maps it onto a type of device (is this a piano? a drum pad? a wind synth?) Each new device gets a listener, which will take every message and send it into the global routing.
The message routing and device connection handling are done in the same loop, which is fine—originally I was separating them, but then I measured the timing and each refresh takes under 250 microseconds. That's more than fast enough for my purposes, and probably well within the latency requirements of most MIDI systems.
The next piece is message subscribers. Each subscriber can specify which type of messages it wants to get, and then their logic is applied to all incoming messages. Right now there's just the GUI subscriber and some debug subscribers, but there'll eventually be a better debug subscriber and there'll be the core handlers that this whole project is written around. (Is this GUI part of a giant yak shave? Maaaaybeeeee.)
The subscribers look pretty simple.
Here's one that echoes every message it receives (where dbg_recv
is its queue).
You just receive from the queue, then do whatever you want with that information!
std::thread::spawn(move || loop {
match dbg_recv.recv() {
Ok(m) => println!("debug: {m:?}"),
Err(err) => println!("err: {err:?}"),
}
});
Finally we reach the GUI, which has its own receiver (marginally more complicated than the debug print one, but not much).
The GUI has a background thread which handles message receiving, and stores these messages into state which the GUI uses.
This is all separated out so we won't block a frame render if a message takes some time to handle.
The GUI also contains state, in two pieces: the ports and MIDI messages are shared with the background thread, so they're in an Arc<Mutex>
.
And there is also the pure GUI state, like which fields are selected, which is not shared and is just used inside the GUI logic.
I think there's a rule that you can't bandy around the word "architecture" without drawing at least one diagram, so here's one diagram. This is the flow of messages through the system. Messages come in from each device, go into the daemon, then get routed to where they belong. Ultimately, they end up stored in the GUI state for updates (by the daemon) and display (by the GUI).
State of the code
This one's not quite ready to be used, but the code is available. In particular, the full GUI code can be found in src/ui.rs. Here are the highlights!
First let's look at some of the state handling.
Here's the state for the daemon.
CTM
is a tuple of the connection id, timestamp, and message; I abbreviate it since it's just littered all over the place.
pub struct State {
midi: MidiInput,
ports: Vec<MidiInputPort>,
connections: HashMap<String, MidiInputConnection<(ConnectionId, Sender<CTM>)>>,
conn_mapping: HashMap<ConnectionId, Category>,
message_receive: Receiver<CTM>,
message_send: Sender<CTM>,
ports_receive: Receiver<Vec<MidiInputPort>>,
ports_send: Sender<Vec<MidiInputPort>>,
}
And here's the state for the GUI.
Data that's written once or only by the GUI is in the struct directly, and anything which is shared is inside an Arc<Mutex>
.
/// State used to display the UI. It's intended to be shared between the
/// renderer and the daemon which updates the state.
#[derive(Clone)]
pub struct DisplayState {
pub midi_input_ports: Arc<Mutex<Vec<MidiInputPort>>>,
pub midi_messages: Arc<Mutex<VecDeque<CTM>>>,
pub selected_ports: HashMap<String, bool>,
pub max_messages: usize,
pub only_note_on_off: Arc<AtomicBool>,
}
I'm using egui, which is an immediate-mode GUI library. That means we do things a little differently, and we build what is to be rendered on each frame instead of retaining it between frames. It's a model which is different from things like Qt and GTK, but feels pretty intuitive to me, since it's just imperative code!
This comes through really clearly in the menu handling.
Here's the code for the top menu bar.
We show the top panel, and inside it we create a menu bar.
Inside that menu bar, we create menu buttons, which have an if
statement for the click hander.
egui::TopBottomPanel::top("menu_bar_panel").show(ctx, |ui| {
egui::menu::bar(ui, |ui| {
ui.menu_button("File", |ui| {
if ui.button("Quit").clicked() {
ctx.send_viewport_cmd(egui::ViewportCommand::Close);
}
});
ui.menu_button("Help", |ui| {
if ui.button("About").clicked() {
// TODO: implement something
}
});
});
});
Notice how we handle clicks on buttons. We don't give it a callback—we just check if it's currently clicked and then take action from there. This is run each frame, and it just... works.
After the top menu panel, we can add our left panel3.
egui::SidePanel::left("instrument_panel").show(ctx, |ui| {
ui.heading("Connections");
for port in ports.iter() {
let port_name = // ...snip!
let conn_id = port.id();
let selected = self.state.selected_ports.get(&conn_id).unwrap_or(&false);
if ui
.add(SelectableLabel::new(*selected, &port_name))
.clicked()
{
self.state.selected_ports.insert(conn_id, !selected);
}
}
});
Once again, we see pretty readable code—though very unfamiliar, if you're not used to immediate mode. We add a heading inside the panel, then we iterate over the ports and for each one we render its label. If it's selected it'll show up in a different color, and if we click on it, the state should toggle4.
The code for displaying the messages is largely the same, and you can check it out in the repo. It's longer, but only in tedious ways, so I'm omitting it here.
What's next?
Coming up, I'm going to focus on what I set out to in the first place and write the handlers. I have some fun logic to do for different MIDI messages!
I'm also going to build another tab in this UI to show the state of those handlers. Their logic will be... interesting... and I want to have a visual representation of it. Both for presentation reasons, and also for debugging, so I can see what state they're in while I try to use those handlers.
I think the next step is... encoding bytes in base 3 or base 45, then making my handlers interpret those. And also base 7 or 11.
And then I'll be able to learn how to use this weird program I'm building.
Sooo this has been fun, but back to work!
Thank you to Anya and Mary for helping me think through some of the problems here, and especially Anya for all the pair programming!
I really try to avoid bulleted lists when possible, but "list of requirements" is just about the perfect use case for them.
↩A MIDI port is essentially a physically connected, available device. It's attached to the system, but it's not one we're listening to yet.
↩The order you add panels matters for the final rendering result! This can feel a little different from what we're used to in a declarative model, but I really like it.
↩This is debounced by egui, I think! Otherwise it would result in lots of clicks, since we don't usually hold a button down for precisely the length of one frame render.
↩or in bass drum
↩If this post was enjoyable or useful for you, please share it! If you have comments, questions, or feedback, you can email my personal email. To get new posts and support my work, subscribe to the newsletter. There is also an RSS feed.
Want to become a better programmer?
Join the Recurse Center!
Want to hire great programmers?
Hire via Recurse Center!