This commit is contained in:
parent
4abcb61db4
commit
8666240a9e
2 changed files with 144 additions and 1 deletions
|
|
@ -0,0 +1,143 @@
|
||||||
|
---
|
||||||
|
layout: post
|
||||||
|
title: "Notes on Forgejo Actions"
|
||||||
|
date: 2024-05-03 19:12:00 Europe/Amsterdam
|
||||||
|
categories: forgejo actions ci act nix nixos kubernetes
|
||||||
|
---
|
||||||
|
|
||||||
|
I recently revived this blog, which is now running as a pod in my Kubernetes cluster.
|
||||||
|
In order to deploy a new version, I had to (manualy) perform the following steps:
|
||||||
|
|
||||||
|
1. Build static website from the Jekyll source
|
||||||
|
2. Build container image using the static website
|
||||||
|
3. Push container image to my container registry
|
||||||
|
4. Update Kubernetes deployment manifest to use the new container image
|
||||||
|
5. Apply the new manifest to the running blog deployment
|
||||||
|
|
||||||
|
This quickly gets annoying and I started looking for a Continuous Integration (CI) system to automate this process.
|
||||||
|
I am using [Forgejo](https://forgejo.org/) as my git server, which recently had its own CI system, Forgejo Actions, [graduated](https://forgejo.org/2023-07-release-v1201-0/) to alpha status.
|
||||||
|
Even though this is still alpha software, I decided to try it out for myself.
|
||||||
|
|
||||||
|
# Setting up a Forgejo Actions runner
|
||||||
|
|
||||||
|
[Forgejo Actions](https://forgejo.org/docs/v1.20/user/actions/) is forked from [Gitea Actions](https://docs.gitea.com/usage/actions/overview) which in turn is forked from [act](https://github.com/nektos/act).
|
||||||
|
Act is project to run GitHub Actions locally, which is the reason why Forgejo Actions are quite similar to Github Actions.
|
||||||
|
In fact, there are quite a few references to GitHub still.
|
||||||
|
|
||||||
|
Forgejo Actions work roughly as follows:
|
||||||
|
1. You install a runner, which is a server that accepts workloads from Forgejo Actions.
|
||||||
|
2. You register this runner with Forgejo.
|
||||||
|
3. You define workflows on a git repository that execute steps.
|
||||||
|
4. These workflows are submitted to a runner.
|
||||||
|
5. Either a Docker container or an LXC container is spinned up to run the workflow to completion.
|
||||||
|
6. The result is communicated back to Forgejo.
|
||||||
|
|
||||||
|
All my workloads run on Kubernetes, and wanted the Forgejo runner to run on Kubernetes as well.
|
||||||
|
Unfortunately, there is no Kubernetes-native way of running this runner, like for example [GitLab](https://docs.gitlab.com/runner/install/operator.html) does.
|
||||||
|
However, there is [an example deployment](https://code.forgejo.org/forgejo/runner/src/branch/main/examples/kubernetes) using a Docker-in-Docker setup.
|
||||||
|
This does however require to run the image as a privileged container, and (spoiler) this setup seems to be very unreliable.
|
||||||
|
|
||||||
|
# Using Forgejo Actions to build Nix derivations
|
||||||
|
|
||||||
|
Both the static website and container image for my blog are built using Nix (which I wrote about [here]({% post_url nix-jekyll-derivation/2024-05-03-nix-jekyll-derivation.md %})).
|
||||||
|
Therefore, it made sense to me to use the [official Docker image](https://hub.docker.com/r/nixos/nix).
|
||||||
|
However, it seems Forgeo Actions has an undocumented dependency on `/bin/sleep` being present inside the container image.
|
||||||
|
It seems sleep is being used to check whether the image is working correctly.
|
||||||
|
Nix being very unconventional, does not have `/bin/sleep` by default in its Docker image.
|
||||||
|
Therefore, I extended the offical Docker image using Nix below.
|
||||||
|
The `coreutils` package contains the `sleep` binary which is linked inside the image using `pathsToLink`.
|
||||||
|
|
||||||
|
```nix
|
||||||
|
let
|
||||||
|
nixFromDockerHub = pkgs.dockerTools.pullImage {
|
||||||
|
imageName = "nixos/nix";
|
||||||
|
imageDigest = "sha256:b3dc72ab3216606d52357ee46f0830a0cc32f3e50e00bd490efa1a8304e9f99d";
|
||||||
|
sha256 = "sha256-FvDlbSnCmPtWTn4eG3hu8WVK1Wm3RSi2T+CdmIDLkG4=";
|
||||||
|
finalImageTag = "2.22.0";
|
||||||
|
finalImageName = "nix";
|
||||||
|
};
|
||||||
|
};
|
||||||
|
in
|
||||||
|
{
|
||||||
|
packages.forgejo-nix-action = pkgs.dockerTools.buildImage {
|
||||||
|
name = "forgejo-nix-action";
|
||||||
|
tag = "latest";
|
||||||
|
fromImage = nixFromDockerHub;
|
||||||
|
|
||||||
|
copyToRoot = pkgs.buildEnv {
|
||||||
|
name = "image-root";
|
||||||
|
paths = with pkgs; [ coreutils ];
|
||||||
|
pathsToLink = [ "/bin" ];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
This is sufficient to be able to run `nix build` commands inside Forgejo Action, for example:
|
||||||
|
```yaml
|
||||||
|
on: [ push ]
|
||||||
|
jobs:
|
||||||
|
blog-pim:
|
||||||
|
runs-on: docker
|
||||||
|
container:
|
||||||
|
image: git.kun.is/home/forgejo-nix-action:687d16c49ea7936068bac64ec68c480a9d681962
|
||||||
|
steps:
|
||||||
|
- name: Clone repository
|
||||||
|
run: git clone ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git src
|
||||||
|
- name: Build image
|
||||||
|
run: nix build --out-link image ./src#packages.x86_64-linux.container-image
|
||||||
|
```
|
||||||
|
|
||||||
|
This workflows clones the source code of my blog, then builds a container image for it using Nix.
|
||||||
|
|
||||||
|
# Pushing the image to a container registry
|
||||||
|
|
||||||
|
Being able to automatically build container images is useless if we don't publish them somewhere.
|
||||||
|
[Skopeo](https://github.com/containers/skopeo) is a real life-saver here, and I would strongly recommend this tool when interacting with container images.
|
||||||
|
Before settling with Skopeo, I first tried using just Docker.
|
||||||
|
However, for many operations, Docker needs a running Docker daemon and this is annoying to setup inside an already annoying Docker-in-Docker setup.
|
||||||
|
Another option I explored was Podman but it was not even able to read the container images built by Nix.
|
||||||
|
|
||||||
|
These are the steps I ended up with to push a container image to a registry using Skopeo:
|
||||||
|
```yaml
|
||||||
|
- name: Log into container registry
|
||||||
|
run: /bin/skopeo login --tls-verify --username ${{ vars.RUNNER_USER }} --password ${{ secrets.RUNNER_TOKEN }} ${GITHUB_SERVER_URL}
|
||||||
|
- name: Push image to container registry
|
||||||
|
run: |
|
||||||
|
/bin/skopeo --insecure-policy copy docker-archive:image docker://${GITHUB_SERVER_URL#https://}/${GITHUB_REPOSITORY_OWNER}/blog-pim:latest
|
||||||
|
```
|
||||||
|
|
||||||
|
# Failing to deploy to Kubernetes
|
||||||
|
|
||||||
|
Having pushed a new container image to a registry, we just have to updated the Kubernetes deployment to use the new image.
|
||||||
|
This is unfortunately something I have not been able to get working.
|
||||||
|
For some reason, anytime I run `nix run` inside a step, the k3s systemd service crashes.
|
||||||
|
It seems during the execution of `nix run`, `etcd` is continuously timing out.
|
||||||
|
After a few seconds of this, k3s deems it unhealthy and restarts itself.
|
||||||
|
|
||||||
|
I have a hunch running `nix run` puts so much strain on the disk that `etcd` is not able to quickly write to disk.
|
||||||
|
It is known that `etcd` needs very quick disk times to function correctly.
|
||||||
|
I have however not been able to reproduce this issue outside of the container.
|
||||||
|
This leads me to believe that perhaps the Docker-in-Docker setup is the cause of this issue.
|
||||||
|
|
||||||
|
# Conclusions
|
||||||
|
|
||||||
|
I think Forgejo Actions really needs a first-class Kubernetes treatment.
|
||||||
|
It seems to me having Action runners on Kubernetes gives free improvements in terms of scalability and management.
|
||||||
|
Going further, I will not be using Forgejo Actions for CI, but will explore some systems that run on Kubernetes natively.
|
||||||
|
In particular, I'm interested in [Argo Workflows](https://argo-workflows.readthedocs.io/en/latest/), and perhaps I will give [GitLab](https://docs.gitlab.com/runner/install/operator.html) a try.
|
||||||
|
|
||||||
|
However in general, and this is not a critique on Forgejo Actions in particular, I don't fully agree with the way these "Actions" systems work.
|
||||||
|
It seems to leading way to use these actions is to use a `runs-on` directive and specify some container image, perhaps the latest Ubuntu version (who thought it was a good idea to turn OSes into images?).
|
||||||
|
Then, people install some tools using the image's package manager after which their actual jobs runs.
|
||||||
|
Unless they are very well maintained, these actions will probably stop working after a while when the underlying container image is discontinued from support.
|
||||||
|
Or perhaps a package is updated, which is incompatable with the action.
|
||||||
|
Or there is some hidden dependency that changes in the underlying container image.
|
||||||
|
Or ...
|
||||||
|
|
||||||
|
I believe Nix could be an amazing tool to improve these "Actions" systems.
|
||||||
|
Using Nix, you wouldn't need a bloated container image with lots of tools you won't need.
|
||||||
|
You could take advantage of Nixpkgs and install only the tools you need to run your job.
|
||||||
|
Everything inside the image would be version-pinned and reproducible and will continue to work in 50 years.
|
||||||
|
I'm not entirely sure how such a system would even look, but my gut tells me it should be possible.
|
||||||
|
I'm unfortunately not aware whether such a system currently exists though.
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
---
|
---
|
||||||
layout: post
|
layout: post
|
||||||
title: "Building a Jekyll Static Website using Nix"
|
title: "Building a Jekyll Static Website using Nix"
|
||||||
date: 2024-05-02 23:51:00 Europe/Amsterdam
|
date: 2024-05-03 15:44:00 Europe/Amsterdam
|
||||||
categories: jekyll nix blog ruby
|
categories: jekyll nix blog ruby
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue