handwritten.blog

EDSGER: A REMARKABLE REPL

First, a disclaimer: if you find my handwriting hard to read, there’s an accessible transcript available here.

With that said, let’s start with a video.

Yes, this is an actual Clojure REPL running on my reMarkable 2. Except for handwriting recognition, everything runs on device; there is no need to have a REPL running on your laptop, or anything like that.

As you might guess, in this post I’ll explain how it works. And what better medium for that than to resurrect my old handwritten blog?

Architecture sketch

Handwritten post: architecture

Here’s how it works:

  1. User writes a Clojure expression on the reMarkable. The device is running its official software (called Xochitl), with a small modification, of which more anon. The open notebook is called “Clojure REPL”.
  2. Several seconds later, Xochitl saves the updated notebook back to the filesystem. This is the primary source of this REPL's latency: I have not yet found a way of controlling this delay.
  3. Every 500ms, a Clojure (actually, let-go) process running on the device polls the notebook for changes (by stat()ing every page file within it – each page is a separate file, in reMarkable’s proprietary .lines format, version 6 – and comparing the mtime to the last one).
  4. If a change is detected, let-go reads the changed file, parses it and diffs with the previous parse to get the strokes for the most recently written expression only. It then converts these strokes to a .png file via a specially-crafted invocation of ImageMagick (which I have cross-compiled for armv7 and installed on the device).
  5. let-go then invokes the Anthropic API (Claude Sonnet 4.6 works beautifully with my handwriting; other models might work too, haven’t checked), asking it to “exactly transcribe this handwritten Clojure expression.”
  6. At this point, we finally have some text that we can pass to read, and then to eval. The twist here is what to do if read throws instead of producing a valid piece of data: is it (assuming the handwriting was OCR’d correctly) because the user wrote something that genuinely is not valid Clojure, or because they are still writing and xochitl just happened to sync the document to the filesystem at this point? We can't know for sure, so this REPL leniently assumes the second case.
  7. Our RE (as in REPL) are done, time for the P. let-go invokes ImageMagick again, this time to convert the eval’d result to an image. With a judiciously chosen font (coming from this REPL’s namesake) we simulate a handwritten response.
  8. It remains to inject the response as an image. Here’s where the Xochitl plugin comes in. It adds another polling loop to Xochitl: it listens to changes in a special file, and if it detects a change of the form: 
    
              </path/to/image.png, x, y>
    then it invokes an internal API to inject that image as a “floating” element. So let-go just needs to write to that file. But at what coordinates? Well, we’ve parsed the document in step 4, so we can figure out the y-coord of the bottommost stroke and apply a small delta, to make the response appear below the input, so the conversation flows naturally top-to-bottom.
  9. Success! GOTO 1.

Standing on the shoulders of giants

I’d like to call out some projects that made this possible.

First, the reMarkable itself. It is a relatively open device – unlike, say, most mobile phones, there's no “jailbreaking” required. Out of the box, it gives you an option to SSH into it as root, and there are official SDKs that allow you to cross-compile your favourite software for it. It runs Linux! On the more recent versions of rM (such as the Paper Pro), I’m told, one has to enable “Developer Mode,” but it’s still a minor hurdle. I have a reMarkable 2 that’s open by default.

The one thing that’s not open in the reMarkable ecosystem is Xochitl. Still, the community has come up with an ingenious way to alter its behaviour. It relies on a small library called XOVI – based on LD_PRELOAD, it gives programs a semi-declarative way to hook into global functions. Kind of like Emacs’ advice mechanism, or Common Lisp's :around method combinations, but for native ARM binaries and without the need to alter them.

Then there’s a XOVI-based extension for Xochitl, called qt-resource-rebuilder. It exploits the fact that Xochitl is written in Qt, whose modern versions are, to a certain extent, declarative – there’s a markup language called QML that allows to describe (parts of) the app’s UI in a DOM-like fashion, and attach JavaScript to individual elements to customize their behaviour. qt-resource-rebuilder hooks into Qt’s resource loading mechanism, so you can say things like “replace the splash screen with this image,” or “apply this structural diff to that QML before processing.” This way, one can add a polling timer to the piece of QML that describes a document page editor.

And finally, let-go. It is a new “almost Clojure” implementation written in Go that I’ve been tinkering with recently. One day, I was wondering how portable it was, and tried out of the blue to cross-compile it to ARM – which worked almost flawlessly: turns out Go has a great cross-compilation story. So I had a binary that I could run on the device over SSH. But then I thought: “does it really count if I can’t interact with it natively, using the stylus?” I kept thinking, one thing led to another, and here I am, writing this post.

On design

If you’ve seen the video, it might seem obvious to you how a handwritten REPL works. There’s basically no UI to speak of. Right? In reality, it was far from obvious to me when I embarked on this journey. It was a humbling design exercise: turns out it has a lot of nuance to it. Like an everyday object; like a chair, or a mug.

Some questions I needed to answer myself:

LLM disclosure

Yes, LLMs were involved in getting it all to work. However, this is very much not your average vibe-coded project: I was the front-seat driver, and although the LLM might have written most of the initial code, I then carefully reviewed it and simplified and refactored it to my taste. I know exactly what every line of code does, and I can vouch for it. The design is mine, and so is most of the architecture (with one exception: I initially thought it wouldn’t be necessary to monkey-patch Xochitl at all, because it would suffice to write back an updated notebook to the file system and notify Xochitl to re-render the page; however, the code I thought I had spotted in Xochitl’s decompiled bundle turned out to be a red herring).

Claude really shone while assisting me in the reverse-engineering part of the project. I used Ghidra for that, and gave Claude access to it via an MCP server: this greatly sped up discovering how to extract QML resources from the xochitl binary and where to hook the timer into.

That being said, I still consider myself a “conscious LLM-skeptic.” I described my stance here, with some further thoughts in this followup. And I hope I don't have to convince you that this writeup contains no LLM-written content.

Can I try it out on my device?

Sure! Let me introduce to you – Edsger:

https://codeberg.org/nathell/edsger

I’m sure you guessed who the namesake is: Edsger Wybe Dijkstra famously used his Mont Blanc fountain pen to write thousands of pages of his manuscripts, many of which were highly influential and are now considered classics of computer science literature.

Would Dijkstra want to use this REPL? I doubt it. Even so, I thought it appropriate to make it render responses in a font based on his handwriting by default.

FAQ

Will it work on Boox/Supernote/Kindle Scribe/...?
Probably not. You'd have to get let-go (or actual Clojure) and ImageMagick onto the device, and then do some equivalent hackery to parse user’s input and inject the rendered output. That said, if I did this on the rM, it’s likely doable elsewhere. Try it, it’s a fun side project!
Why is it so slow?
Xochitl takes approximately 12 seconds to update the notebook on-disk once the user stops writing; from there, parsing the strokes, OCR and rendering output adds another few seconds. Further reverse engineering might reveal a way to force Xochitl to write sooner or somehow bypass the write. If you find something, let me know!
Why do it? It's so impractical!
The only “why” I have to offer is that, to me, it has hack value. Dare I call it art? I’ll leave it to you to decide. I invite you to come up with practical applications of Edsger as a creative exercise.

World domination plans

Will I maintain Edsger? Probably not. But in case you’d like to take a stab at improving it (I’ll happily accept contributions!), here are some ideas, listed in rough order of difficulty (easiest first):