Packaging a site no longer holds the whole site in memory

ChangedCLI

TL;DR: The packaging tool ee-cdn-cli downloads now writes the package to disk as it builds it, instead of holding the site and the package in memory, so a large site no longer needs memory to match. Packages are unchanged.

To package your site, ee-cdn-cli downloads and runs a separate tool, ee-builder. Until now it built the whole package in memory: your site in one copy, every compressed block in another, then the finished package as a third. The memory a deploy needed grew with the site, so a large site could fail to package on a laptop or a small CI runner.

Now ee-builder writes each block to disk as soon as it’s built, and reads your files from disk as it goes instead of holding the site. The memory it needs no longer grows with the size of your site. Blocks are also compressed on several of your machine’s cores at once, up to 8, so memory depends on the cores in use rather than on the site.

Your package is unchanged

  • The same files give the same package as before, byte for byte, whatever the number of cores.
  • A file that changes while it is being packaged fails the build, rather than ending up in a package that doesn’t describe it.

Limits

  • With image conversion on, memory still grows with the site. Converting images means reading every file to find references to them. The site is no longer held twice, though.
  • A pinned builder version keeps the old builder until you change it.

What you need to do

Nothing. ee-cdn-cli replaces a cached latest builder automatically, so your next deploy uses the new one. If you pin --builder-version (builder-version in the GitHub Action), pin the builder published on 3 November or later, or remove the pin.

← All changes · More CLI changes · Subscribe to CLI

View as Markdown