Add post about Nix Jekyll derivation
All checks were successful
/ blog-pim (push) Successful in 11m52s
All checks were successful
/ blog-pim (push) Successful in 11m52s
This commit is contained in:
parent
824b03b476
commit
4abcb61db4
1 changed files with 202 additions and 0 deletions
|
|
@ -0,0 +1,202 @@
|
||||||
|
---
|
||||||
|
layout: post
|
||||||
|
title: "Building a Jekyll Static Website using Nix"
|
||||||
|
date: 2024-05-02 23:51:00 Europe/Amsterdam
|
||||||
|
categories: jekyll nix blog ruby
|
||||||
|
---
|
||||||
|
|
||||||
|
This blog has been down for several months, and I recently at last decided to revive it.
|
||||||
|
The blog is written in [Jekyll](https://jekyllrb.com/) and building it generally means:
|
||||||
|
|
||||||
|
1. Installing the right version of the Ruby programming language.
|
||||||
|
2. Installing the correct version of Ruby libraries.
|
||||||
|
3. Building the website using `jekyll build`.
|
||||||
|
|
||||||
|
In order to improve reproducability, I'll show how to create a Nix derivation for a Jekyll static website.
|
||||||
|
Much of this post also applies to other Ruby projects.
|
||||||
|
|
||||||
|
# Generate a `gemset.nix`
|
||||||
|
|
||||||
|
Ruby uses [Bundler](https://bundler.io/) to manage dependencies (called gems).
|
||||||
|
Each Ruby project generally has two files related to Bundler: `Gemfile` and `Gemfile.lock`.
|
||||||
|
These are similar to `flake.nix` and `flake.lock` in the Nix realm: the `Gemfile` specifies a project's high-level dependencies, which is converted to a `Gemfile.lock` file with a snapshot of exact versions these dependencies revolve to.
|
||||||
|
|
||||||
|
Nix, however, doesn't understand this `Gemfile.lock`.
|
||||||
|
Therefore we will first have to convert this to a Nix definition.
|
||||||
|
We can do this using [Bundix](https://github.com/nix-community/bundix); simply execute `nix run nixpkgs#bundix` in the directory containing your `Gemfile.lock`.
|
||||||
|
If you don't have a `Gemfile.lock` yet, you can also have Bundix generate it using `nix run nixpkgs#bundix --lock`
|
||||||
|
This creates a `gemset.nix` file containing an attrset like:
|
||||||
|
```nix
|
||||||
|
{
|
||||||
|
jekyll = {
|
||||||
|
dependencies = ["addressable" "colorator" "em-websocket" "i18n" "jekyll-sass-converter" "jekyll-watch" "kramdown" "kramdown-parser-gfm" "liquid" "mercenary" "pathutil" "rouge" "safe_yaml" "terminal-table"];
|
||||||
|
groups = ["default" "jekyll_plugins"];
|
||||||
|
platforms = [];
|
||||||
|
source = {
|
||||||
|
remotes = ["https://rubygems.org"];
|
||||||
|
sha256 = "192k1ggw99slpqpxb4xamcvcm2pdahgnmygl746hmkrar0i3xa5r";
|
||||||
|
type = "gem";
|
||||||
|
};
|
||||||
|
version = "4.1.1";
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
As you can see, apart from pinning the version (`4.1.1`), Bundix also pins the SHA256 hash of the Ruby gem guaranteeing reproducability.
|
||||||
|
|
||||||
|
# Packaging the Ruby Gems
|
||||||
|
|
||||||
|
Now that have all Ruby dependencies neatly version-pinned in Nix, we can have Nix build them for us.
|
||||||
|
Here, [`bundlerEnv`](https://github.com/NixOS/nixpkgs/blob/123ed385f88545e6037b20a6cda113b936614605/doc/languages-frameworks/ruby.section.md#developing-with-ruby-developing-with-ruby) does the heavy lifting for us:
|
||||||
|
```nix
|
||||||
|
{
|
||||||
|
gems = pkgs.bundlerEnv {
|
||||||
|
name = "blog-pim";
|
||||||
|
gemdir = ./src;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The attribute `gemdir` specifies the directory containing our `gemset.nix`.
|
||||||
|
Using `bundlerEnv` like this, `gems` contains `/lib` and `/bin` of all direct and indirect dependencies and puts all executables from these gems on PATH.
|
||||||
|
Also, `gems.wrappedRuby` contains a Ruby package with access to each of these Ruby gems.
|
||||||
|
|
||||||
|
# The Nix Derivation
|
||||||
|
|
||||||
|
Having access to all Ruby dependencies as Nix packages, we can now build static website using Nix:
|
||||||
|
|
||||||
|
```nix
|
||||||
|
{
|
||||||
|
static-website = pkgs.stdenv.mkDerivation {
|
||||||
|
name = "blog-pim";
|
||||||
|
src = ./src;
|
||||||
|
sourceRoot = "src";
|
||||||
|
|
||||||
|
buildInputs = [
|
||||||
|
gems
|
||||||
|
gems.wrappedRuby
|
||||||
|
];
|
||||||
|
|
||||||
|
buildPhase = ''
|
||||||
|
bundle exec jekyll build
|
||||||
|
'';
|
||||||
|
|
||||||
|
installPhase = ''
|
||||||
|
mkdir -p $out
|
||||||
|
cp -r _site/* $out/
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Set the `src` attribute to the directory containing your Jekyll source code.
|
||||||
|
Also set `sourceRoot` to this directory, which `jekyll build` expects.
|
||||||
|
As inputs to our build process, we have the Ruby gems and the wrapped Ruby program from before.
|
||||||
|
The build phase should be obvious: with our setup we can just build Jekyll as normal.
|
||||||
|
The compiled static website is put in the `_site` directory, which we copy to our build result in the install phase.
|
||||||
|
|
||||||
|
And that's it for the most part!
|
||||||
|
A fully working flake can be found on my [Git website](https://git.kun.is/home/blog-pim/src/commit/824b03b4763487efd39f0cc475f45fa9e3195b43/flake.nix#L53).
|
||||||
|
Let's build the derivation and check its result:
|
||||||
|
```bash
|
||||||
|
$ nix build .#packages.x86_64-linux.static-website
|
||||||
|
$ ll result/
|
||||||
|
total 180
|
||||||
|
-r--r--r-- 1 root root 7391 Jan 1 1970 404.html
|
||||||
|
dr-xr-xr-x 2 root root 4096 Jan 1 1970 about
|
||||||
|
dr-xr-xr-x 2 root root 4096 Jan 1 1970 ansible-edit-grub
|
||||||
|
dr-xr-xr-x 2 root root 4096 Jan 1 1970 archive
|
||||||
|
dr-xr-xr-x 7 root root 4096 Jan 1 1970 assets
|
||||||
|
dr-xr-xr-x 2 root root 4096 Jan 1 1970 backup-failure
|
||||||
|
-r--r--r-- 1 root root 262 Jan 1 1970 browserconfig.xml
|
||||||
|
dr-xr-xr-x 2 root root 4096 Jan 1 1970 concourse-apprise-notifier
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
# `jekyll-feed`'s Monkey Wrench
|
||||||
|
|
||||||
|
The above works fine for the most part, but if you use the [jekyll-feed](https://github.com/jekyll/jekyll-feed) Ruby gem, you will unfortunately get a non-deterministic derivation!
|
||||||
|
`jekyll-feed` creates an Atom feed for your blog posts when compiling the static website.
|
||||||
|
Let's see why it creates non-deterministic builds:
|
||||||
|
```shell
|
||||||
|
$ nix build --rebuild --keep-failed .#packages.x86_64-linux.static-website
|
||||||
|
note: keeping build directory '/tmp/nix-build-blog-pim.drv-3'
|
||||||
|
error: derivation '/nix/store/sdphangdgj348ps9x93qwly7kzqalwqf-blog-pim.drv' may not be deterministic: output '/nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim' differs from '/nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim.check'
|
||||||
|
```
|
||||||
|
|
||||||
|
Nix complains our derivation differs between different runs.
|
||||||
|
Let's see what is different between these two executions:
|
||||||
|
```text
|
||||||
|
$ nix run nixpkgs#diffoscope /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim.check
|
||||||
|
--- /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim
|
||||||
|
+++ /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim.check
|
||||||
|
│ --- /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim/feed.xml
|
||||||
|
├── +++ /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim.check/feed.xml
|
||||||
|
│ │ --- /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim/feed.xml
|
||||||
|
│ ├── +++ /nix/store/193gdfza7d98l573b53sjqswhwfpc30g-blog-pim.check/feed.xml
|
||||||
|
│ │ @@ -1,13 +1,13 @@
|
||||||
|
│ │ <?xml version="1.0" encoding="utf-8"?>
|
||||||
|
│ │ <feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en-US">
|
||||||
|
│ │ <generator uri="https://jekyllrb.com/" version="4.1.1">Jekyll</generator>
|
||||||
|
│ │ <link href="https://pim.kun.is/feed.xml" rel="self" type="application/atom+xml"/>
|
||||||
|
│ │ <link href="https://pim.kun.is/" rel="alternate" type="text/html" hreflang="en-US"/>
|
||||||
|
│ │ - <updated>2024-05-03T11:12:54+00:00</updated>
|
||||||
|
│ │ + <updated>2024-05-03T11:12:58+00:00</updated>
|
||||||
|
│ │ <id>https://pim.kun.is/feed.xml</id>
|
||||||
|
│ │ <title type="html">Pim Kunis</title>
|
||||||
|
│ │ <subtitle>A pig's gotta fly</subtitle>
|
||||||
|
│ │ <author>
|
||||||
|
│ │ <name>Pim Kunis</name>
|
||||||
|
│ │ </author>
|
||||||
|
│ │ <entry>
|
||||||
|
```
|
||||||
|
|
||||||
|
Aha, it seems `jekyll-feed` uses the current datetime inside the feed to indicate the last change.
|
||||||
|
Patching this was quite easy: I simply replaced this date with the date of the last post.
|
||||||
|
I'll spare you my terrible Python code, but this can be read [here](https://git.kun.is/home/blog-pim/src/commit/824b03b4763487efd39f0cc475f45fa9e3195b43/patch-feed-date.py).
|
||||||
|
|
||||||
|
I packaged this script like so:
|
||||||
|
```nix
|
||||||
|
{
|
||||||
|
patch-feed-date = pkgs.stdenv.mkDerivation {
|
||||||
|
name = "path-feed-date";
|
||||||
|
propagatedBuildInputs = [ pkgs.python3 ];
|
||||||
|
dontUnpack = true;
|
||||||
|
installPhase = "install -Dm755 ${./patch-feed-date.py} $out/bin/patch-feed-date";
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
And I updated the derivation as follows:
|
||||||
|
```nix
|
||||||
|
{
|
||||||
|
static-website = pkgs.stdenv.mkDerivation {
|
||||||
|
name = "blog-pim";
|
||||||
|
src = ./src;
|
||||||
|
sourceRoot = "src";
|
||||||
|
|
||||||
|
buildInputs = [
|
||||||
|
gems
|
||||||
|
gems.wrappedRuby
|
||||||
|
patch-feed-date # Updated
|
||||||
|
];
|
||||||
|
|
||||||
|
buildPhase = ''
|
||||||
|
bundle exec jekyll build
|
||||||
|
'';
|
||||||
|
|
||||||
|
installPhase = ''
|
||||||
|
mkdir -p $out
|
||||||
|
cp -r _site/* $out/
|
||||||
|
patch-feed-date --file _site/feed.xml > $out/feed.xml # Updated
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Let's check if this fixed our non-deterministic build:
|
||||||
|
```shell
|
||||||
|
$ nix build --rebuild --keep-failed .#packages.x86_64-linux.static-website
|
||||||
|
$ echo $?
|
||||||
|
0
|
||||||
|
```
|
||||||
Loading…
Reference in a new issue