nixcon-2024-presentation-public/presentation.md

---
marp: true
theme: gaia
paginate: true
footer: "October 26th, 2024"
---

# Kubenix: Leveraging NixOS modules to generate Kubernetes manifests

![bg right:35% 100% Kubenix logo](assets/kubenix.svg)

**Pim Kunis**

---

# Agenda

![bg right:60% 80%](assets/kubenix-github.png)


1. My background
2. Kubernetes
3. Kubenix

---

# `$ whoami`

![bg right 60% drop-shadow:0,5px,10px,rgba(0,0,0,.7)](assets/me.jpg)

- Pim Kunis
    - Matrix: @pim:envs.net
    - GitHub: @pizzapim
- Consultant @ [Sue](https://sue.nl)
    - Currently doing Linux + Ansible stuff
- Relatively new to Nix
- Avid self-hoster

---

# Self-hosting

![bg Logos of services I self-host](assets/self-hosting/logos.png)

---

# Self-hosting

3 mini PCs (Gigabyte Brix with Intel Celeron J4105)

![w:900 My servers in their closet](assets/servers-services.jpg)

---

# NixOS for a cluster?

![w:900 My servers in their closet](assets/servers-kill.jpg)

---

# NixOS for a cluster?

![bg right assets/cat-chewing-cable.jpg](assets/cat-chewing-cable.jpg)

- Balance services
- Maintenance
- Services are tied to a particular host
    - Pets vs cattle

---

# Agenda

1. My background
2. **Kubernetes**
3. Kubenix

---

# Enter: Kubernetes

![bg left:40% 100% assets/cat-chewing-cable.jpg](assets/kubernetes.png)

- Abstracts infrastructure
- Container orchestration
    - Workloads are containers (pods)
- Distributed state (etcd)
    - Actual state
    - Desired state
    - Reconciliation

---

# Kubernetes

![w:900 My servers in their closet](assets/servers-k8s.jpg)

---

# Kubernetes

![w:900 My servers in their closet](assets/servers-k8s-kill.jpg)

---

# Kubernetes

![w:900 My servers in their closet](assets/servers-k8s-revive.jpg)

---

# Mutating state in Kubernetes

![bg 80% hi](assets/yaml-manifest/base.png)

---

# Mutating state in Kubernetes

![bg 80% hi](assets/yaml-manifest/highlight1.png)

---

# Mutating state in Kubernetes

![bg 80% hi](assets/yaml-manifest/highlight2.png)

---

# Mutating state in Kubernetes

![bg 80% hi](assets/yaml-manifest/highlight3.png)

---

# Helm

![bg right:50% 60%](assets/helm.png)

- Package manager
- Templating YAML
- Cursed

---

# Helm


![bg 80% hi](assets/helm-example/base.png)

---

# Helm


![bg 80% hi](assets/helm-example/highlight.png)

---

# Helm

![bg 80% hi](assets/helm-example/watermark.png)

---

# Agenda

1. My background
2. Kubernetes
3. **Kubenix**

---

# Kubenix ✨

- Generate manifests using NixOS modules!
- Benefits:
  - No ugly YAML templates
  - Functional language
  - Type and structure checking
  - Modularity

---

# How Kubenix works

1. My background
2. Kubernetes
3. **Kubenix**
    - **OpenAPI specification**
    - Generate NixOS module
    - Use generated module
    - Generate manifest

---


# The OpenAPI spec

- Standard for describing APIs
- YAML or JSON
- Specifies how to interact with Kubernetes API

---

# The OpenAPI spec


![bg 75% hi](assets/openapi-spec-example/base.png)

---

# The OpenAPI spec

![bg 75% hi](assets/openapi-spec-example/highlight1.png)

---

# The OpenAPI spec

![bg 75% hi](assets/openapi-spec-example/highlight2.png)

---

# How Kubenix works

1. My background
2. Kubernetes
3. **Kubenix**
    - OpenAPI specification
    - **Generate NixOS module**
    - Use generated module
    - Generate manifest

---

# Generating the NixOS module

- Generate Nix code
  - Templating?

---

# Generating the NixOS module

```nix
''
definitions = {
  # Here we loop over each "definition"
  ${concatStrings (mapAttrsToList (name: value: ''
  "${name}" = {
    ${optionalString (hasAttr "options" value) "
      options = {${concatStrings (mapAttrsToList (name: value: ''                                                            
      "${name}" = ${value};
    '')
    value.options)}};
      "}

    ${optionalString (hasAttr "config" value) ''
    config = {${concatStrings (mapAttrsToList (name: value: ''
        "${name}" = ${value};
      '')
      value.config)}};
  ''}
  };
'')
definitions)}
};
''
```

---

# The generated NixOS module

```nix
    "io.k8s.api.core.v1.Pod" = {
      options = {
        "apiVersion" = mkOption {
          description = "...";
          type = (types.nullOr types.str);
        };
        "kind" = mkOption {
          description = "...";
          type = (types.nullOr types.str);
        };
        "metadata" = mkOption {
          description = "...";
          type = (types.nullOr (submoduleOf "io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta"));                                  
        };
        "spec" = mkOption {
          description = "...";
          type = (types.nullOr (submoduleOf "io.k8s.api.core.v1.PodSpec"));
        };
        "status" = mkOption {
          description = "...";
          type = (types.nullOr (submoduleOf "io.k8s.api.core.v1.PodStatus"));
        };
      };
    };
```

---

# How Kubenix works

1. My background
2. Kubernetes
3. **Kubenix**
    - OpenAPI specification
    - Generate NixOS module
    - **Use generated module**
    - Generate manifest

---

# Using the module

![bg 70% hi](assets/kubenix-example/base.png)

---

# Using the module

![bg 70% hi](assets/kubenix-example/highlight1.png)

---

# Using the module

![bg 70% hi](assets/kubenix-example/highlight2.png)

---

# Using the module

![bg 70% hi](assets/kubenix-example/highlight3.png)

---

# Using the module

![bg 70% hi](assets/kubenix-example/highlight4.png)

---

# Example of type checking

```nix
{
    kubernetes.resources.pods.example-pod = {
        spec.containers.jellyfin-container = {
            image = "jellyfin/jellyfin:latest";
            ports = [{ containerPort = "foo"; }];
        };
    };
};
```

```
 error: A definition for option `kubernetes.(...).containerPort' is not of type `signed integer'.
```

---

# An undefined option

```nix
{
    kubernetes.resources = {
        foo.bar = "baz";
    }
}
```

```
error: The option `kubernetes.api.resources.foo' does not exist.
```

---

# How Kubenix works

1. My background
2. Kubernetes
3. **Kubenix**
    - OpenAPI specification
    - Generate NixOS module
    - Use generated module
    - **Generate manifest**

---

# `kubenix.evalModules`

```nix
{
  inputs.kubenix.url = "github:hall/kubenix";
  outputs = {self, kubenix, ... }@inputs: let
    system = "x86_64-linux";
  in {
    packages.${system}.default = (kubenix.evalModules.${system} {
      module = { kubenix, ... }: {
        imports = [ kubenix.modules.k8s ];
        kubernetes.resources.pods.example.spec.containers.jellyfin.image = "jellyfin/jellyfin:latest";
      };
    }).config.kubernetes.result;
  };
}
```

---

# `kubenix.evalModules`


```json
{
  "apiVersion": "v1",
  "items": [
    {
      "apiVersion": "v1",
      "kind": "Pod",
      "metadata": {
        "annotations": {
          "kubenix/k8s-version": "1.30",
          "kubenix/project-name": "kubenix"
        },
        "labels": {
          "kubenix/hash": "84130bb6a26b9066eb2bdcaf2e68148a6e0648b0"                                                                                                                  
        },
        "name": "example"
      },
      "spec": {
        "containers": [
          {
            "image": "jellyfin/jellyfin:latest",
            "name": "jellyfin"
          }
        ]
      }
    }
  ],
  "kind": "List",
  "labels": {
    "kubenix/hash": "84130bb6a26b9066eb2bdcaf2e68148a6e0648b0",
    "kubenix/k8s-version": "1.30",
    "kubenix/project-name": "kubenix"
  }
}
```

---

# Summary

- Kubenix generates Kubernetes manifests
- Converts OpenAPI spec to a NixOS module
- Build Kubernetes deployments with:
  - No ugly YAML templates
  - Modularity
  - In a functional language
  - Type and structure checking

---

# Terraform: OpenAPI Provider Spec Generator

![hoi](assets/openapi-spec-gen.png)

---

# Bonus: nix-snapshotter

![bg right:50% 100%](assets/nix-snapshotter.png)

- Images are in Nix store
- Fully declarative

---

# Bonus: NixNG

![bg right:50% 100%](assets/nixng.png)

- Nix~~OS~~ in a container
- init-system independent