Jan Miksovsky’s BlogArchive AboutRSSJSONContact

Automating generation of the audio and video for an Origami intro screencast

I’ve posted a new Origami intro screencast that covers some of the basics of the Origami language:

This screencast doesn’t give a complete introduction yet, but I think the production process I’m using is itself interesting and worth sharing.

Videos are a vital form of documentation but:

Goal

I was inspired by how Grant Sanderson produces his 3Blue1Brown math animations programmatically. Grant controls the graphics, the camera movements, highlights, text — everything — in code. This makes it easier for him to iterate on the story he’s telling by rewriting the code.

What I really want is to be able to focus on the story I want to tell — to be a screenwriter. I want to write a screencast script with stage directions (“Click on index.html”) and dialogue (“Alice: This index.html file contains…”). I’d love to be able to generate a decent screencast video directly from that.

Process

I can’t find anything that lets me do what I want, so for now I’m trying to automate generation of the video and audio separately.

I’m trying a scriptable mouse/keyboard desktop automation product called Keyboard Maestro. I can write keyboard/mouse macros for specific common actions (open the VS Code terminal, select a file, etc.), then assemble these into an overall macro for the screencast.

This is very clunky for this purpose; I really wish some product like this offered a real programming language. In any event, I play the macro while recording the screen to get the video portion of the screencast.

I’m using Origami itself to map a script of dialogue to generated voice files via OpenAI. Having two alternating voices seems to reduce the fatigue of listening to generated text-to-speech.

The script is a YAML file with lines of dialogue for each of the “actors”:

- echo: >
    To illustrate the basic ideas in their plainest form, let's start by writing
    some expressions in Origami using the command-line interface called ori. If
    I type "ori 1 plus 1", it evaluates that and displays 2.
- echo: >
    If I type "ori hello”, oree displays hello. In the shell, you'll need to
    escape quotes or surround them with extra quotes because the shell itself
    consumes quote marks.
- shimmer: >
    In addition to basic numbers and strings, you can reference files. Think of
    each file as if it were a spreadsheet cell. Instead of the A1, B2 style cell
    references in a spreadsheet, we can use paths and file names to refer to
    things. Unlike most programming languages, names in Origami can include
    characters like periods and hyphens.

This produces the audio portion of the screencast. I then use Camtasia to merge the audio and video to create the final screencast file. This requires a fair bit of work to position the audio clips in relation to the video, and in many places to add delays in the video to give the audio time to play.

Lessons

This editing process still takes time but the recording of the audio and video took very little time compared to previous screencasts. And if Origami changes, it’s feasible to tweak the demo macro or dialogue, rerecord those, and splice those into the screencast.

I posted this video on the Origami discussion board to get feedback on a draft screencast. I used that feedback to refine the audio and video and then spliced in the updated parts. Being able to iterate on a screencast is fantastic.

One unpleasant surprise: After a second round of feedback, I went to rerecord the video — and discovered that VS Code had made changes to its window chrome! Uh, rats. That means that new video clips can’t be used alongside old ones, which means having to reposition all the audio in relation to the new video. That’s a real time sink.

Future

I hope this approach pans out so that I can make more useful screencasts that can stay relevant for a longer time. Towards that end, I’d love to be able to:

If you’re familiar with tools that can do any of those things, please let me know!

Looking for people to playtest a programming language for making websites

Source code in the Origami programming language for a basic blog

I’d love to find a few new people to try out the Origami programming language for creating websites — maybe you?

Maybe you have any of these goals:

and you:

The Origami programming language complements HTML and CSS to let you define the structure of a site. You write formulas or expressions at roughly the level of complexity of spreadsheet formulas. These fully determine the site you get; nothing happens unless you ask for it.

The language is concise and powerful. The above screenshot shows the code for a sample influencer lifestyle blog. Those ~15 lines of code establish the basic site structure, and another 20 lines in other files prepare the raw content and produce the blog feed. That’s all that’s required to define a blog engine completely from scratch. For comparison, a typical blog engine might require much more code in its configuration file alone and be much harder to reason about.

A playtest is generally done as a video call. You outline your goals and then either go through an Origami tutorial or we work together to start something from scratch. At this stage of the language’s evolution, direct observation is extremely helpful. Some people find that prospect intimidating, but this is a playtest! We will test the language — no one will be testing you.

The Origami documentation is complete enough that a motivated person could potentially get up and running on their own, but the onboarding won’t be as easy as with an already well-established, mature language.

If interested, please contact me!

Source code for the MomBoard project

Hacker News screenshot on November 14, 2024, showing the MomBoard post as the #1 story

I am deeply touched by the warm response to my recent MomBoard: E-ink display for a parent with amnesia post. It got much more circulation on Mastodon that I usually get, and overnight someone posted it to Hacker News — where it shot to the #1 story. I discovered this after waking up and finding my inbox full of LinkedIn requests.

The post stayed on the HN home page for two full days, which stunned me because their story ranking algorithm heavily discounts stories over time. The story got 2000+ points, more than every other post on HN that day, beating out the day’s news that The Onion bought InfoWars.

In response, I received a number of emails from people with a parent or loved one in a similar situation and wondering whether I could make the MomBoard source code available.

That was a small challenge, as the project has more in common with a home-cooked meal than a finished product. I’d written the project for a tiny audience of family members and had taken the liberty of shortcuts that wouldn’t work for others, like hard-coding people’s names.

I spent a day cleaning things up a bit and publishing the MomBoard source on GitHub.

I’ve done my best to document the “recipe” for this project in the README. It’s long and complicated, and perhaps only useful as inspiration; we’ll see. Like most home recipes, people will want to adapt the recipe to their own tastes and situation.

MomBoard: E-ink display for a parent with amnesia

E-ink display on a bathroom counter showing birthday messages

Today marks two years since I first set up an e-ink display in my mom’s apartment to help her live on her own with amnesia. The display has worked extremely well during those two years, so I’m sharing the basic set-up in case others find it useful for similar situations.

Note: unless you have specific experience caring for someone who has amnesia but not dementia, please do not offer care suggestions.

The patient

In June 2022 the side-effects of a long surgery left my mom with permanent anterograde amnesia: she can no longer form new long-term memories. Memory isn’t just one neurological system, so very occasionally she will be able to remember certain types of things. But for the most part, if she hears or sees something, a few minutes later she will no longer remember it.

To medical professionals her condition looks a lot like dementia — amnesia is a common symptom of dementia — but she doesn’t have dementia. One difference is that (as I understand it) dementia is a progressive disease, while this amnesia is stable. There is no cure.

Someday I might post about the experience about caring for her, but for now I’ll just say that this type of amnesia is not something one should wish on one’s worst enemies.

Needs

My mom still lives on her own in an apartment. Because she cannot remember things, she goes through each day in a state of low-grade anxiety about where her grown children are and whether they are all right. She feels she hasn’t heard from any of us in a long time. This anxiety manifests as extremely frequent attempts to call or text us.

Paper notes and other forms of reminders didn’t seem to help, and would become out of date even if they weren’t misplaced. My siblings and I would call to let her know we were okay, but five minutes later she’d be back to being worried. She wasn’t in the habit of scrolling back through text messages, so once she’d read a message, it was immediately forgotten and effectively lost.

I thought some sort of unobtrusive, always-on device installed in her apartment might be able to show her notes written by my siblings and me.

Design goals

My goal was to find a display that:

  1. Could stay on for months on end
  2. Would let my siblings and I easily post short messages to it that would remain visible until replaced
  3. Was large enough and easy enough to read without glasses
  4. Required no interaction to wake or read and was relatively foolproof (touching it wouldn’t disrupt it)
  5. Was resilient to network failures
  6. Didn’t glow at nighttime
  7. Didn’t require hardware hackery (I’m a software person)
  8. Would boot directly into displaying messages (no interaction needed to start an app)
  9. Was not enshittified with a subscription service or proprietary app store
  10. Was reasonably affordable
  11. Would not look out of place in a home

Device

Given the above design goals, I searched for a tablet-size electronic ink display with Wi-Fi connectivity and a decent web browser.

One device that seemed to fit my parameters was the BOOX Note Air2 Series. At the time it cost US$500, which is expensive but is still far cheaper than screens intended for use as commercial retail displays. It’s marketed as a note-taking device and ebook reader, but it also has a capable web browser. It’s big enough to read from a few feet away.

A critical question I couldn’t answer online was whether I’d be able to have the device automatically start its web browser and have that browser display a designated start page. Happily, when the device arrived I was able to confirm it could do both of those things.

The physical case of the Note Air2 looks reasonably nice and not particularly tech-y. The e-ink display is clear and legible; it refreshes quickly enough to not be distracting. By default the device’s backlight was turned on but I could turn it off.

I found a small metal stand to serve an easel for the display so that it felt more like a picture frame.

Web software

Since the physical device was satisfactory, the next step was writing a simple website that could drive the display. The site would have two pages:

  1. A Board page showing the messages. The e-ink device would boot into showing this page. This is the only page my mom needed to see.
  2. A Compose page my siblings and I write messages and save them to be displayed.

The device needed to run for months, and needed to be resilient in the case of network and service failures. At the same time, I also needed to be able to remotely update not only the messages being displayed, but the software displaying those messages.

With that in mind, I factored the Board page into an outer frame and an inner page:

  1. The top-level outer frame acts as a thin shell around the inner page. At top of every hour, the outer frame reloads the inner page to pick up potential software changes. If the network is down and the inner page doesn’t reload, the frame just tries again an hour later. To maximize reliability, the outer frame has very little logic and no external dependencies.
  2. The inner page actually displays the messages. Every 5 minutes it queries a simple web service for message data and displays the messages. The inner page contains a small amount of logic, but as few dependencies as possible.

Since it’s essentially impossible to debug anything that happens on the device, I made as much use of vanilla HTML and CSS as possible. I used a small amount of JavaScript but no framework or other libraries.

Compose form for posting a message to the display

The Compose page presents a simple web form my siblings and I can use to compose and save a message. I designed the form to work well on a phone screen so that we can write messages when we’re out and about. A small web app manifest lets us save the Compose page to a phone’s home screen as an icon for quick access.

The whole site is tiny, entails no build process, and with the exception of the service (below) is just static files.

Visual design

I was concerned about the possibility of e-ink burn-in, so the Board page randomly changes which message appears where. Other visual elements like the date and time alternate from side to side, with the intention that no single pixel is always on.

To style the note text I chose the free Architect’s Daughter font for a handwritten feel. This font works well on the e-ink display. Labels are displayed in Open Sans.

One small challenge was maximizing the size of the message text. Sometimes a message is just a word or two; other times it might be several sentences. A single font size can’t accommodate such a wide range of text content. I couldn’t find a pure CSS way to automatically maximize font size so that a text element with word wrapping would display without clipping.

I ended up writing a small JavaScript function to maximize font size: it makes the text invisible (via CSS visibility: hidden), tries displaying the text at a very large size, and then tries successively smaller font sizes until it finds a size that lets all the text fit. It then makes the text visible again.

Service

Just a tiny amount of text data is necessary to drive the display, so I was happy to find the minimalist JsonStorage service that was perfect for this project. A single JSON object stores the text and metadata for the current set of messages. The Compose page can save to the service with a POST request, and the Board page can retrieve the data with a GET.

The service has a free tier that I started with, but I liked the service so much that I eventually paid for a $1/month basic tier. (It appears that tier is now $5/month.)

Trial and installation

I spent a couple of weeks working on the software and letting it run for long periods of time. I was pleasantly surprised that the Boox display worked as well as it did and seemed to stay up indefinitely.

I brought the display over to my mom’s apartment on November 12, 2022, turned it on, joined it to her Wi-Fi, and rebooted it to confirm everything worked in the new environment.

I thought the bathroom counter might be a good place for it, but my mom thought she’d rather have it in her bedroom, so we found a home for it on a windowsill.

My mom was happy with the display right away.

Retrospective

Despite her amnesia, my mom came to remember that this display exists and what it’s for. She looks forward to seeing updates from her children on it.

If we tell her about something that’s coming up, she often asks whether we’ve already put that event on the MomBoard. On the flip side, we have to be careful to keep it up to date; if we fail to take down a message that no longer applies, it confuses her.

Looking back, the display is essentially the only intervention of any kind we’ve tried that’s actually been successful at improving her quality of life (and ours). One reason it’s worked so well is that it didn’t require her to learn anything new. Without the ability to remember new things, it’s virtually impossible for her to learn a new skill or to form new habits.

The device’s reliability has surpassed my expectations. There was one period where the device seemed to stop working, but I traced the problem to a faulty Wi-Fi hub; after that was replaced, it’s worked flawlessly since. For my part, keeping the software as simple as possible and sticking to vanilla web technologies surely helped avoid bugs.

The display still looks great, and it still displays messages day in and day out.

If you want to try to set up something similar to what I describe here, I’m happy to answer technical questions or share advice.

Update: I’ve published the MomBoard source code on GitHub.

Retire an old server to be a static site

Screenshot of the Elix web component library documentation

I was able retire an old #Heroku site by copying the content to a completely #static #website. Instead of resurrecting the source project and rewriting it, I used the #WebOrigami crawl command to retrieve the static files.

The @crawl picked up all but a few exotically-referenced resources that I copied over by hand. I dropped it all on Netlify. Some pages didn’t have an .html extension, so I added a small Netlify config to serve those as HTML. Then I pointed the old domain at the new site: Done!

Retiring this server as a static site reduces my monthly cost for it from $7/month to $0. The site’s for an old startup company but includes documentation for a web components library that I want to leave available.

If you’d like help retiring a site, let me know and I’ll see if I can assist.

The basic site auditing tool in #WebOrigami can work against a site defined in many ways: a data file, files in the file system, an Origami program — or a live #website running in production.

In the video I show an audit of the unbelievably massive Space Jam. For a site with 350+ handwritten HTML files, it contains surprisingly few broken internal links.

A simple protocol lets you make the contents of your #smallweb / #indieweb site more fully discoverable and explorable by interested users. This can also let site A see what resources site B offers.

I’ve been working on this in the spirit of the web’s simple, venerable, and immensely valuable View Source facility. View Source just lets you view a single HTML page; the web lacks a standard equivalent of a directory listing for a given route. This aims to fill that gap.

I made a short video walking through how I generate OpenGraph images for the pages on my blog using Origami.

This video is an experiment. It takes me ages to create polished documentation or videos — and that’s discouraged me from creating anything at all! I thought I’d try recording a video with only minimal rehearsal and no editing and see if this kind of thing can fill the gap until I can produce something better.

Cat test

I want to test RSS-to-Mastodon posting via MastoFeed, so here’s a test post with a photo of Mojo, who wants to come inside.

Update: I couldn’t get MastoFeed to work, and ended up following these instructions for Pipedream instead.

An ocicat with brown fur and green eyes stands on a yellow stucco banister and stares at the camera.

Sample influencer lifestyle blog for Henry David Thoreau

To demonstrate that Origami is a good language for building #smallweb / #indieweb blogs, I built a fun sample blog reenvisioning Henry David Thoreau has a modern influencer with a lifestyle blog about off-grid living.

Blog post titled Beans with text adapted from Thoreau's Walden

Creating a site in Origami is completely different than creating a site in any other tool I know of.

Origami isn’t a blog engine or framework, just like Microsoft Excel isn’t an invoicing or expense reporting framework. Excel is a general tool that transforms and aggregates numbers and text in tables. Origami is a general tool that transforms and aggregates data and content in trees.

So this project isn’t configuring a blog tool — it’s defining what a blog is from scratch. The source is endlessly malleable, and you can readily change what gets produced to achieve a wide variety of results.

To build in Origami, think about the starting tree of content you’ll write or gather by hand, and the final tree of resources in your running site. Step by step, you transform the former into the latter.

For a blog you might start with, say, a markdown folder containing markdown posts, an images folder for photos, and an assets folder for stylesheets.

Tree diagram of staring point with assets, images, and markdown folders

The final resource tree comprises the images and assets as is, plus generated pages for individual posts, list pages, and feeds.

Tree diagram of site resources showing generated pages, posts, and feed areas

To generate individual pages you can use a template language. Origami has a nice one built in; you can use others.

But the real magic is writing formulas to process a pile of content at once, like a pipeline that transforms markdown to HTML, calculates data, and sorts the posts to prepare them for rendering.

Origami pipeline transforming a markdown folder into data

Or a compact definition of the entire public portion of the site:

Origami program that defines the publicly-visible resources for a blog site

This Origami program generates the full tree of the resources for your blog, which you can browse immediately. The Origami runtime only does the work to generate a page when you ask for it.

The same Origami program can also produce a complete build folder with all your site files. Deploy those on a static web server or have a service do a build whenever you update your project.

Full source

Older posts