--- marp: true theme: gaia paginate: true footer: "October 26th, 2024" --- # Kubenix: Leveraging NixOS modules to generate Kubernetes manifests ![bg right:35% 100% Kubenix logo](assets/kubenix.svg) **Pim Kunis** --- # Agenda ![bg right:60% 80%](assets/kubenix-github.png) 1. My background 2. Kubernetes 3. Kubenix --- # `$ whoami` ![bg right 60% drop-shadow:0,5px,10px,rgba(0,0,0,.7)](assets/me.jpg) - Pim Kunis - Matrix: @pim:envs.net - GitHub: @pizzapim - Consultant @ [Sue](https://sue.nl) - Currently doing Linux + Ansible stuff - Relatively new to Nix - Avid self-hoster --- # Self-hosting ![bg Logos of services I self-host](assets/self-hosting/logos.png) --- # Self-hosting 3 mini PCs (Gigabyte Brix with Intel Celeron J4105) ![w:900 My servers in their closet](assets/servers-services.jpg) --- # NixOS for a cluster? ![w:900 My servers in their closet](assets/servers-kill.jpg) --- # NixOS for a cluster? ![bg right assets/cat-chewing-cable.jpg](assets/cat-chewing-cable.jpg) - Balance services - Maintenance - Services are tied to a particular host - Pets vs cattle --- # Agenda 1. My background 2. **Kubernetes** 3. Kubenix --- # Enter: Kubernetes ![bg left:40% 100% assets/cat-chewing-cable.jpg](assets/kubernetes.png) - Abstracts infrastructure - Container orchestration - Workloads are containers (pods) - Distributed state (etcd) - Actual state - Desired state - Reconciliation --- # Kubernetes ![w:900 My servers in their closet](assets/servers-k8s.jpg) --- # Kubernetes ![w:900 My servers in their closet](assets/servers-k8s-kill.jpg) --- # Kubernetes ![w:900 My servers in their closet](assets/servers-k8s-revive.jpg) --- # Mutating state in Kubernetes ![bg 80% hi](assets/yaml-manifest/base.png) --- # Mutating state in Kubernetes ![bg 80% hi](assets/yaml-manifest/highlight1.png) --- # Mutating state in Kubernetes ![bg 80% hi](assets/yaml-manifest/highlight2.png) --- # Mutating state in Kubernetes ![bg 80% hi](assets/yaml-manifest/highlight3.png) --- # Helm ![bg right:50% 60%](assets/helm.png) - Package manager - Templating YAML - Cursed --- # Helm ![bg 80% hi](assets/helm-example/base.png) --- # Helm ![bg 80% hi](assets/helm-example/highlight.png) --- # Helm ![bg 80% hi](assets/helm-example/watermark.png) --- # Agenda 1. My background 2. Kubernetes 3. **Kubenix** --- # Kubenix ✨ - Generate manifests using NixOS modules! - Benefits: - No ugly YAML templates - Functional language - Type and structure checking - Modularity --- # How Kubenix works 1. My background 2. Kubernetes 3. **Kubenix** - **OpenAPI specification** - Generate NixOS module - Use generated module - Generate manifest --- # The OpenAPI spec - Standard for describing APIs - YAML or JSON - Specifies how to interact with Kubernetes API --- # The OpenAPI spec ![bg 75% hi](assets/openapi-spec-example/base.png) --- # The OpenAPI spec ![bg 75% hi](assets/openapi-spec-example/highlight1.png) --- # The OpenAPI spec ![bg 75% hi](assets/openapi-spec-example/highlight2.png) --- # How Kubenix works 1. My background 2. Kubernetes 3. **Kubenix** - OpenAPI specification - **Generate NixOS module** - Use generated module - Generate manifest --- # Generating the NixOS module - Generate Nix code - Templating? --- # Generating the NixOS module ```nix '' definitions = { # Here we loop over each "definition" ${concatStrings (mapAttrsToList (name: value: '' "${name}" = { ${optionalString (hasAttr "options" value) " options = {${concatStrings (mapAttrsToList (name: value: '' "${name}" = ${value}; '') value.options)}}; "} ${optionalString (hasAttr "config" value) '' config = {${concatStrings (mapAttrsToList (name: value: '' "${name}" = ${value}; '') value.config)}}; ''} }; '') definitions)} }; '' ``` --- # The generated NixOS module ```nix "io.k8s.api.core.v1.Pod" = { options = { "apiVersion" = mkOption { description = "..."; type = (types.nullOr types.str); }; "kind" = mkOption { description = "..."; type = (types.nullOr types.str); }; "metadata" = mkOption { description = "..."; type = (types.nullOr (submoduleOf "io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta")); }; "spec" = mkOption { description = "..."; type = (types.nullOr (submoduleOf "io.k8s.api.core.v1.PodSpec")); }; "status" = mkOption { description = "..."; type = (types.nullOr (submoduleOf "io.k8s.api.core.v1.PodStatus")); }; }; }; ``` --- # How Kubenix works 1. My background 2. Kubernetes 3. **Kubenix** - OpenAPI specification - Generate NixOS module - **Use generated module** - Generate manifest --- # Using the module ![bg 70% hi](assets/kubenix-example/base.png) --- # Using the module ![bg 70% hi](assets/kubenix-example/highlight1.png) --- # Using the module ![bg 70% hi](assets/kubenix-example/highlight2.png) --- # Using the module ![bg 70% hi](assets/kubenix-example/highlight3.png) --- # Using the module ![bg 70% hi](assets/kubenix-example/highlight4.png) --- # Example of type checking ```nix { kubernetes.resources.pods.example-pod = { spec.containers.jellyfin-container = { image = "jellyfin/jellyfin:latest"; ports = [{ containerPort = "foo"; }]; }; }; }; ``` ``` error: A definition for option `kubernetes.(...).containerPort' is not of type `signed integer'. ``` --- # An undefined option ```nix { kubernetes.resources = { foo.bar = "baz"; } } ``` ``` error: The option `kubernetes.api.resources.foo' does not exist. ``` --- # How Kubenix works 1. My background 2. Kubernetes 3. **Kubenix** - OpenAPI specification - Generate NixOS module - Use generated module - **Generate manifest** --- # `kubenix.evalModules` ```nix { inputs.kubenix.url = "github:hall/kubenix"; outputs = {self, kubenix, ... }@inputs: let system = "x86_64-linux"; in { packages.${system}.default = (kubenix.evalModules.${system} { module = { kubenix, ... }: { imports = [ kubenix.modules.k8s ]; kubernetes.resources.pods.example.spec.containers.jellyfin.image = "jellyfin/jellyfin:latest"; }; }).config.kubernetes.result; }; } ``` --- # `kubenix.evalModules` ```json { "apiVersion": "v1", "items": [ { "apiVersion": "v1", "kind": "Pod", "metadata": { "annotations": { "kubenix/k8s-version": "1.30", "kubenix/project-name": "kubenix" }, "labels": { "kubenix/hash": "84130bb6a26b9066eb2bdcaf2e68148a6e0648b0" }, "name": "example" }, "spec": { "containers": [ { "image": "jellyfin/jellyfin:latest", "name": "jellyfin" } ] } } ], "kind": "List", "labels": { "kubenix/hash": "84130bb6a26b9066eb2bdcaf2e68148a6e0648b0", "kubenix/k8s-version": "1.30", "kubenix/project-name": "kubenix" } } ``` --- # Summary - Kubenix generates Kubernetes manifests - Converts OpenAPI spec to a NixOS module - Build Kubernetes deployments with: - No ugly YAML templates - Modularity - In a functional language - Type and structure checking --- # Terraform: OpenAPI Provider Spec Generator ![hoi](assets/openapi-spec-gen.png) --- # Bonus: nix-snapshotter ![bg right:50% 100%](assets/nix-snapshotter.png) - Images are in Nix store - Fully declarative --- # Bonus: NixNG ![bg right:50% 100%](assets/nixng.png) - Nix~~OS~~ in a container - init-system independent