Published by

Inexactly benchmarking Eleventy vs Astro build times

The Eames Institute team is checking out some frameworks and static site generators for a project, and I wanted to see how Eleventy and Astro compare in terms of build time.

Zach Leatherman’s 2022 article “Which generator builds Markdown the fastest?” is probably the most thorough resource I’ve come across along these lines, and I’d recommend checking that out if you want to do some serious comparisons.

But I was curious about a “real world” test in 2024, so I decided to do some inexact benchmarking using a Markdown export of this blog. A caveat up front: I’m much more familiar with Eleventy than I am with Astro, which will likely be apparent when I get to the incremental build tests later in this post.

For Eleventy I used eleventy-netlify-boilerplate with zero modifications, and for Astro I used the blog template as described in their docs with some small modifications along the lines of this wordpress-to-astro repo to get categories and tags working. I didn’t want to use wordpress-to-astro directly since it was last updated two years ago, but it is a good reference point.

My blog has 770 posts which were exported to 770 Markdown files. With a paginated feed, categories, and tags, the total number of built pages is around 2550.*

Based on an average taken from 10 builds, Astro took 10.07 seconds and Eleventy took 4.29 seconds to build.

Incremental builds can speed things up significantly since the only built content is that which is relevant to the modified files.

Eleventy has supported incremental builds since December 2022 (I believe!), but it doesn’t yet support it on a CI server. There is an open issue for it which looks like it has traction.

To test incremental builds, I added and removed the same single tag on the same post 10 times.** Based on an average taken from 10 incremental builds, Eleventy took 2.17 seconds skipping 777 files. I would have expected it to skip more, but this might have to do with not being able to incrementally build paginated data.

I wanted to test the same content change in Astro… but it isn’t clear to me that there is an apples-to-apples comparison. Astro introduced an experimental Incremental Content Caching feature in v4.0 (not sure if this is supported on CI servers). When I added experimental.contentCollectionCache to the config, there was no difference between basic build times when I made a content change. I’m not sure if this is because having all of my content in Markdown makes the caching a mute point, or if it’s something else. If anyone has further context on how best to test Incremental Content Caching in Astro, would love to know.

For what it’s worth, running astro dev is extremely quick, just 125ms before it’s ready.

I’d be curious to do a similar benchmark using WordPress’s REST API but am not sure I’ll have the time… Will update here if I do.

* I give a rough number because the Eleventy boilerplate and Astro template generate a few additional pages, but the page total difference is in the single digits so I didn’t waste time evening them up perfectly.

** For my own future reference in case I do further tests: Add and remove the tag hello from this post.

Published by

Open source tools for multi-source and cross-format academic publishing

I’m working with Sasha Engelmann and Sophie Dyer on the Open Weather platform, an archive and learning resource related to NOAA satellite 🛰 imagery. Sasha just shared a few open source publication tools that were brought to her attention by a friend and fellow artist at her Akademie Schloss Solitude residency, wanted to add them here for further research and future reference.

Manifold: A platform for publishing academic texts online

Manifold is a free “intuitive, collaborative, open-source platform for scholarly publishing”. See their repo on GitHub.

Manifold powers the Fembot Collective including Ada, Fembot’s journal on gender, new media, and technology. Looks like Fembot has been working with Manifold since about a year ago when the platform launched their pilot. Read Ada 16: Emerging Gender, Media and Technology Scholarship in Africa.

It looks pretty cool (and so does Fembot + Ada!). Manifold can bring together a whole lot of different methods of writing such as Epub, Markdown, HTML, and Google Docs. Hence the name Manifold, I guess. This is incredibly useful when bringing the work of different researchers together. Also makes it clear to me that good markup in writing is so worth it.

Manifold wants to make a digital book much more than just a screen version of a physical book, something that can easily fold in explorations, supplements, and other resources that augment the main text. It also incorporates annotation and discussion settings to keep the conversation going.

I’d love to see a book that really heavily uses the platform’s unusual features. Metagaming: Playing, Competing, Spectating, Cheating, Trading, Making, and Breaking Videogames is a featured project that’s worth a look. The chapters are punctuated with metagames they’ve created that you can download and install.

As a reader, I feel that the typography lets it down a bit. I found it hard to read, particularly on larger screens. A slightly narrower maximum width to the main text column would help a lot. Losing the justification and greater paragraph indentations would help too. Manifold does have some theme options, but it doesn’t involve control over the typography.

If your priorities are bringing together content from a wide arrange of sources, incorporating the work of disparate researchers with varying levels of technical abilities, and relative ease of setup (the documentation seems comprehensive), then Manifold seems like an incredible tool. If you need to retain any control over the design though or if you also want print publishing tools, it might not be the right fit for the job.

And probably worth mentioning: I think you’d need at least a bit of technical know-how to get this set up safely and securely. Probably worth getting in touch with Manifold directly if you’re an org since they’re still in beta.

B-ber: A tool for single-source, cross-format, design-conscious publishing

Triple Canopy is a magazine that “resists the atomization of culture”. They’re responsible for b-ber, a tool for single-source, cross-format, design-conscious publishing. Here’s how they describe it in the b-ber GitHub repo:

b-ber is both a method and an application for producing publications in a variety of formats—EPUB 3, Mobi/KF8, static website, PDF, and XML file, which can be imported into InDesign for print layouts—from a single source that consists of plain-text files and other assets. b-ber also functions as a browser-based EPUB reader, which explains the name.

Their text introducing b-ber “Working on our thoughts”—title from the Nietzche quote “Our writing tools are also working on our thoughts” according to the footnotes—is a good read, explains the impetus and a bit about the ups and downs of how it evolved.

B-ber can only consume one input, an extended form of Markdown. This makes it more limited than Manifold in that regard, but the output options are substantial. It’s particularly strong for the design-conscious, the fact that you can import to InDesign and easily theme the browser-based EPUB reader is pretty fantastic. This is exactly the sort of thing I was looking for back when I was working at Occasional Papers!

The reading experience of the default b-ber theme (or whichever they use on their post) is nicer than Manifold in my opinion, it’s just a lot easier to read. There are some snags, but I imagine you could resolve these in a custom theme. Related to that, see their repository of b-ber demos and b-ber theme starter.

It’s definitely worth following the development of this project if you’re in to digital publishing. Their announcement post was published back in December, not very long ago! Excited to see how it develops.

As with Manifold, I think you’d need a reasonable amount of technical knowledge to get this set up. Since it seems to be more of an internal Triple Canopy tool that they’ve kindly made open source for wider use, they probably wouldn’t be able to provide as much support as Manifold might be able to. (This is just a guess though!)

My experience

Though I’ve been tempted, I’ve never built something that was meant to have a digital bookish-ness, everything I’ve developed has had online-first layouts and components in mind. Some sites have had fairly extensive print styles, but that’s usually as far as it goes.

The most common related problem I’ve run in to on sites with long-format academic writing is footnotes. I’ve never come across a CMS that handles footnotes well. Heck, even HTML doesn’t handle them all that well, there aren’t any appropriate semantic elements as far as I’m aware (though there were in HTML3?).

The only easily accessible markup system that works with footnotes AFAIK is extended Markdown syntax. To use extended Markdown on a client site though, A) I have to be sure that the client is on board with learning quite a bit of Markdown (they often are once they understand the benefit, but some are stubborn!), and B) it needs to be compatible with whatever layout system the designer has devised.

I used this approach a while ago on the Jock Kinneir Library site, as of right now they’re using footnotes on the Biography page.

This implementation wasn’t super straightforward since the site couldn’t use a single Markdown field for content, we needed more of a page builder to accomplish the layout. Because of that, I had to do some trickery to recompile the footnotes at the base of the page content as opposed to after each text section. Honestly I can’t 100% remember how I accomplished it… It’s on Craft so uses Twig templates, and I don’t think we had the time to make a custom module that would take advantage of server-side logic. I do remember that it was a bit hackier than I wanted, but it safely accomplished what needed to be done.

If I need to include footnotes or something similar in the future, I’ll probably refer to this comprehensive article on footnotes, endnotes, and sidenotes (via @s3ththompson).

Would be curious if others have come across similar free, open source tools, or if anyone knows of work being done on the HTML spec to get some progress with footnotes.

At any rate, all of the above just reinforces my opinion that anyone who writes, regardless of how tech-savvy, should learn how to write in Markdown at minimum, ideally the extended syntax. If your archive of writing is in a machine-readable format, you’re miles ahead should you ever wish to publish it somewhere remotely digital or want to convert it to an IDML file or something similar.

Edit 24 September 2020: Added link to article about sidenotes.