Building Molecule Reader in one day

Wednesday, December 7, 2022

Reading on screens is very difficult for me. I just cannot focus on the articles, especially when there are notifications coming in or even just other content on the screen1. I have a reMarkable tablet (RM), which I love dearly2 and much prefer to read on. But it's annoying getting articles onto it.

To put a blog post onto my RM, I copy the link from Firefox (my usual browser), open Chromium, load the page, and print it with the "Read on reMarkable" printer (which is only for Chrome-based browsers). And when I have five or ten articles I want to read, I have to repeat this for each one manually. Ouch.

It's also annoying how the articles end up. I'd like to have them all tidy in one folder where I can read them, or even in one continuous document. If I send each one individually, they just litter the home screen (since you can't print to a specific folder) and displace other things I'm reading.

I decided to solve this by writing a web app to bundle up my reading and send it to my RM!

Preparing to build

I knew generally what I wanted: get a bunch of articles, merge them into one PDF, and send that PDF on my RM. And I wanted to do it in Rust. Oh, and only spend one day on it.

That last requirement is the tricky one. The scope could easily get too big, or I could end up cutting critical features for the deadline. Balance was going to be tricky, and this was ambitious. At this point, I was not expecting to finish, but hoping to get something working.

Since I came up with the idea last week and did the build yesterday, I was able to spend the weekend thinking about requirements and doing some idle searching on helpful resources. I went into the day pretty confident that I could find libraries to do the RSS feed parsing, but I was most unsure about generating the PDF.

My plan going into the build day was to pair program with other people to keep me making progress and start from the PDF generation, since that had the most unknowns. Everything after that was going to be improvised.

Like any good project, I also gave it a good name and made a repository. The project is Molecule Reader, because it bundles up Atom feeds, and what else would you call a bundle of atoms? You can check out the repo, with no apologies for the quality of code.

Building it

Yesterday, I started the day by announcing at my morning RC check-in that this was my plan, and I put out a call for pairing. The first thing I tackled was PDF generation. This turned out to be the easiest part, after I switched up my approach.

My original approach was to use fantoccini (or similar) to use WebDriver to control a browser to render the articles into individual PDFs. I wanted to do this since I knew browsers in print mode would render the articles pretty well, where I was less confident in any reader-mode shenanigans I might be able to pull off in pure Rust.

Instead, I wound up using headless Chromium to render the pages to PDFs! This took me some time to figure out since the command-line options for this were hard for me to find documentation for, but once I got it, it has worked pretty flawlessly. With one little Rust function, I can generate a PDF for the page at a given URL:

/// Takes an item and generates a PDF for it. Will return an error if it fails
/// and an empty result otherwise.
///
/// Assumes that you have `chromium-browser` installed on your system.
pub fn url_to_pdf(url: &str, output_filename: &str) {
    let print_arg = format!("--print-to-pdf={output_filename}");
    match Command::new("chromium-browser")
        .args(["--headless", &print_arg, "--virtual-time-budget=10000", url])
        .status()
    {
        Ok(_) => {}
        Err(e) => {
            tracing::error!(error=?e, url=url, output_filename=output_filename, "error while printing page");
        }
    };
}

I also found a simple utility for collating the individual PDFs into one. There's this tool called pdfunite which is installed on all my systems already. I run Fedora, but I don't know if this is standard or something that comes with another tool I have installed. At any rate, it was super convenient, and this only has to work on my machine(s)!


/// Takes a sequence of filenames and collates them into a combined PDF with the
/// specified string. Returns an error if it fails and an empty result otherwise.
///
/// Assumes that you have `pdfunite` installed on your system.
pub fn collate(filenames: &[String], combined_filename: &str) {
    let mut args: Vec<String> = filenames.to_vec();
    args.push(combined_filename.into());
    match Command::new("pdfunite").args(args).status() {
        Ok(_) => {}
        Err(e) => {
            tracing::error!(error=?e, combined_filename=combined_filename, "error while collating");
        }
    }
}

With the hard part out of the way by early morning, I was able to move on to the rest of the build. Scraping the RSS feeds was up next, and this was where things were a little hairy. I wanted to process both RSS feeds and Atom feeds, so I used a wrapper library which tries parsing in both formats and gives you a parsed feed in one or the other.

After that, I built the web application itself! This part was pretty straightforward with actix-web, and the type system made plumbing together the forms a delight. I created three pages, and all the templates are parsed and checked at compile time, so I always know that my page templates will render if the app compiles.

A few folks were around to pair with me for the feed parsing and for the web app build, and it was really nice having company and having ideas to keep me moving forward3!

I got something hacked together that worked, but had a lot of sins in it, before 5pm that day. That's a success. I did come back that evening and polished things up, including a refactoring that resolved the lingering performance problems from the original hacky data storage.

And it is pretty nice, in my totally unbiased opinion. There's almost no CSS (there's one rule, which strikes through any articles which have been printed; I keep them visible in case I need to reprint), but it works.

A screenshot of Molecule Reader, showing the feeds which are aggregated as well as the latest items.

Reflections

This was a really fun experience. I'll probably do it again, balancing it against the intensity of the experience. Today I was pretty tired, and I think the one-day build yesterday was a big part of why I'm tired today. (Also shoutout to my kids for why I'm tired today. Love you, and you're a lot of work.)

I took away from this a few things.

Building something useful in one day is possible. I've wanted a small suite of personal web apps for a while to do little things. This has made that possibility much more tangible by reminding me that building something useful quickly is doable. I didn't have to build some of the more complicated pieces (login, permissions), and I don't have to deal with robust error handling since I can always check the logs. I'm excited to see what I build next!

I can prototype quickly in Rust. I used to assume I'd better build a prototype in Python, or maybe Go. Those were the languages I can go quickly in, since the type checker isn't along for the ride. That's not the case anymore! Since I've spent the last 12 weeks working in Rust, I think I'm as productive in it as in any other language I know. My prototyping speed in Rust is on par with my other main languages, but with the advantage of the type system making later refactoring much safer.

Rapid prototyping is exhausting. I'll definitely do this again, but I'm pretty tired. There are two aspects of that. One was the intensity of it! Building something useful in one day is a lot of constant hard thinking, and that's draining. The other is that I was pairing for over half the day, and that's pretty draining for me as well. I love pairing, and I can't handle a ton in one go. This draining aspect is something I'll have to keep in mind and maybe stretch things into two-day builds!

I'm proud of what I've built, and excited that it exists. Now please excuse me while I go read 76 pages of blog posts on my RM.


1

For some reason, I don't have this problem while coding, which I can hyperfocus on, but I do not get into that same state while reading.

2

It's the only electronic device I've used every single day since I got it in 2018. It replaced a handful of paper notebooks I carried everywhere. I now have two, since I got the newer model as well! The original is my book/paper/blog reading/annotating device, and the newer one is for my notes, todo lists, sketches, illustrations, etc.

3

This was particularly helpful while debugging. Huge shoutout to Conner, who helped me debug something and sped me up a lot as a result, and spent a long time pairing with me yesterday.


If this post was enjoyable or useful for you, please share it! If you have comments, questions, or feedback, you can email my personal email. To get new posts and support my work, subscribe to the newsletter. There is also an RSS feed.

Want to become a better programmer? Join the Recurse Center!