diff --git a/src/_posts/nix-jekyll-derivation/2024-05-03-nix-jekyll-derivation.md b/src/_posts/nix-jekyll-derivation/2024-05-03-nix-jekyll-derivation.md new file mode 100644 index 0000000..3741c32 --- /dev/null +++ b/src/_posts/nix-jekyll-derivation/2024-05-03-nix-jekyll-derivation.md @@ -0,0 +1,202 @@ +--- +layout: post +title: "Building a Jekyll Static Website using Nix" +date: 2024-05-02 23:51:00 Europe/Amsterdam +categories: jekyll nix blog ruby +--- + +This blog has been down for several months, and I recently at last decided to revive it. +The blog is written in [Jekyll](https://jekyllrb.com/) and building it generally means: + +1. Installing the right version of the Ruby programming language. +2. Installing the correct version of Ruby libraries. +3. Building the website using `jekyll build`. + +In order to improve reproducability, I'll show how to create a Nix derivation for a Jekyll static website. +Much of this post also applies to other Ruby projects. + +# Generate a `gemset.nix` + +Ruby uses [Bundler](https://bundler.io/) to manage dependencies (called gems). +Each Ruby project generally has two files related to Bundler: `Gemfile` and `Gemfile.lock`. +These are similar to `flake.nix` and `flake.lock` in the Nix realm: the `Gemfile` specifies a project's high-level dependencies, which is converted to a `Gemfile.lock` file with a snapshot of exact versions these dependencies revolve to. + +Nix, however, doesn't understand this `Gemfile.lock`. +Therefore we will first have to convert this to a Nix definition. +We can do this using [Bundix](https://github.com/nix-community/bundix); simply execute `nix run nixpkgs#bundix` in the directory containing your `Gemfile.lock`. +If you don't have a `Gemfile.lock` yet, you can also have Bundix generate it using `nix run nixpkgs#bundix --lock` +This creates a `gemset.nix` file containing an attrset like: +```nix +{ + jekyll = { + dependencies = ["addressable" "colorator" "em-websocket" "i18n" "jekyll-sass-converter" "jekyll-watch" "kramdown" "kramdown-parser-gfm" "liquid" "mercenary" "pathutil" "rouge" "safe_yaml" "terminal-table"]; + groups = ["default" "jekyll_plugins"]; + platforms = []; + source = { + remotes = ["https://rubygems.org"]; + sha256 = "192k1ggw99slpqpxb4xamcvcm2pdahgnmygl746hmkrar0i3xa5r"; + type = "gem"; + }; + version = "4.1.1"; + }; +} +``` + +As you can see, apart from pinning the version (`4.1.1`), Bundix also pins the SHA256 hash of the Ruby gem guaranteeing reproducability. + +# Packaging the Ruby Gems + +Now that have all Ruby dependencies neatly version-pinned in Nix, we can have Nix build them for us. +Here, [`bundlerEnv`](https://github.com/NixOS/nixpkgs/blob/123ed385f88545e6037b20a6cda113b936614605/doc/languages-frameworks/ruby.section.md#developing-with-ruby-developing-with-ruby) does the heavy lifting for us: +```nix +{ + gems = pkgs.bundlerEnv { + name = "blog-pim"; + gemdir = ./src; + }; +} +``` + +The attribute `gemdir` specifies the directory containing our `gemset.nix`. +Using `bundlerEnv` like this, `gems` contains `/lib` and `/bin` of all direct and indirect dependencies and puts all executables from these gems on PATH. +Also, `gems.wrappedRuby` contains a Ruby package with access to each of these Ruby gems. + +# The Nix Derivation + +Having access to all Ruby dependencies as Nix packages, we can now build static website using Nix: + +```nix +{ + static-website = pkgs.stdenv.mkDerivation { + name = "blog-pim"; + src = ./src; + sourceRoot = "src"; + + buildInputs = [ + gems + gems.wrappedRuby + ]; + + buildPhase = '' + bundle exec jekyll build + ''; + + installPhase = '' + mkdir -p $out + cp -r _site/* $out/ + ''; + }; +} +``` + +Set the `src` attribute to the directory containing your Jekyll source code. +Also set `sourceRoot` to this directory, which `jekyll build` expects. +As inputs to our build process, we have the Ruby gems and the wrapped Ruby program from before. +The build phase should be obvious: with our setup we can just build Jekyll as normal. +The compiled static website is put in the `_site` directory, which we copy to our build result in the install phase. + +And that's it for the most part! +A fully working flake can be found on my [Git website](https://git.kun.is/home/blog-pim/src/commit/824b03b4763487efd39f0cc475f45fa9e3195b43/flake.nix#L53). +Let's build the derivation and check its result: +```bash +$ nix build .#packages.x86_64-linux.static-website +$ ll result/ +total 180 +-r--r--r-- 1 root root 7391 Jan 1 1970 404.html +dr-xr-xr-x 2 root root 4096 Jan 1 1970 about +dr-xr-xr-x 2 root root 4096 Jan 1 1970 ansible-edit-grub +dr-xr-xr-x 2 root root 4096 Jan 1 1970 archive +dr-xr-xr-x 7 root root 4096 Jan 1 1970 assets +dr-xr-xr-x 2 root root 4096 Jan 1 1970 backup-failure +-r--r--r-- 1 root root 262 Jan 1 1970 browserconfig.xml +dr-xr-xr-x 2 root root 4096 Jan 1 1970 concourse-apprise-notifier +... +``` + +# `jekyll-feed`'s Monkey Wrench + +The above works fine for the most part, but if you use the [jekyll-feed](https://github.com/jekyll/jekyll-feed) Ruby gem, you will unfortunately get a non-deterministic derivation! +`jekyll-feed` creates an Atom feed for your blog posts when compiling the static website. +Let's see why it creates non-deterministic builds: +```shell +$ nix build --rebuild --keep-failed .#packages.x86_64-linux.static-website +note: keeping build directory '/tmp/nix-build-blog-pim.drv-3' +error: derivation '/nix/store/sdphangdgj348ps9x93qwly7kzqalwqf-blog-pim.drv' may not be deterministic: output '/nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim' differs from '/nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim.check' +``` + +Nix complains our derivation differs between different runs. +Let's see what is different between these two executions: +```text +$ nix run nixpkgs#diffoscope /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim.check +--- /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim ++++ /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim.check +│ --- /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim/feed.xml +├── +++ /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim.check/feed.xml +│ │ --- /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim/feed.xml +│ ├── +++ /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim.check/feed.xml +│ │ @@ -1,13 +1,13 @@ +│ │ +│ │ +│ │ Jekyll +│ │ +│ │ +│ │ - 2024-05-03T11:12:54+00:00 +│ │ + 2024-05-03T11:12:58+00:00 +│ │ https://pim.kun.is/feed.xml +│ │ Pim Kunis +│ │ A pig's gotta fly +│ │ +│ │ Pim Kunis +│ │ +│ │ +``` + +Aha, it seems `jekyll-feed` uses the current datetime inside the feed to indicate the last change. +Patching this was quite easy: I simply replaced this date with the date of the last post. +I'll spare you my terrible Python code, but this can be read [here](https://git.kun.is/home/blog-pim/src/commit/824b03b4763487efd39f0cc475f45fa9e3195b43/patch-feed-date.py). + +I packaged this script like so: +```nix +{ + patch-feed-date = pkgs.stdenv.mkDerivation { + name = "path-feed-date"; + propagatedBuildInputs = [ pkgs.python3 ]; + dontUnpack = true; + installPhase = "install -Dm755 ${./patch-feed-date.py} $out/bin/patch-feed-date"; + }; +} +``` + +And I updated the derivation as follows: +```nix +{ + static-website = pkgs.stdenv.mkDerivation { + name = "blog-pim"; + src = ./src; + sourceRoot = "src"; + + buildInputs = [ + gems + gems.wrappedRuby + patch-feed-date # Updated + ]; + + buildPhase = '' + bundle exec jekyll build + ''; + + installPhase = '' + mkdir -p $out + cp -r _site/* $out/ + patch-feed-date --file _site/feed.xml > $out/feed.xml # Updated + ''; + }; +} +``` + +Let's check if this fixed our non-deterministic build: +```shell +$ nix build --rebuild --keep-failed .#packages.x86_64-linux.static-website +$ echo $? +0 +```