update documentation on k8s

This commit is contained in:
Pim Kunis 2024-04-13 13:35:18 +02:00
parent fed2708d13
commit d40150b97a
2 changed files with 11 additions and 51 deletions

View file

@ -8,7 +8,6 @@ Nix definitions to configure our servers at home.
- [disko](https://github.com/nix-community/disko): declarative disk partitioning - [disko](https://github.com/nix-community/disko): declarative disk partitioning
- [agenix](https://github.com/ryantm/agenix): deployment of encrypted secrets to NixOS machines - [agenix](https://github.com/ryantm/agenix): deployment of encrypted secrets to NixOS machines
- [dns.nix](https://github.com/kirelagin/dns.nix): A Nix DSL for defining DNS zones - [dns.nix](https://github.com/kirelagin/dns.nix): A Nix DSL for defining DNS zones
- [microvm.nix](https://github.com/astro/microvm.nix): Declarative virtual machine management in NixOS
- [flake-utils](https://github.com/numtide/flake-utils): Handy utilities to develop Nix flakes - [flake-utils](https://github.com/numtide/flake-utils): Handy utilities to develop Nix flakes
- [nixos-hardware](https://github.com/NixOS/nixos-hardware): Hardware-specific NixOS modules. Doing the heavy lifting for our Raspberry Pi. - [nixos-hardware](https://github.com/NixOS/nixos-hardware): Hardware-specific NixOS modules. Doing the heavy lifting for our Raspberry Pi.
@ -36,17 +35,19 @@ Additionally, it deploys an age identity, which is later used for decrypting sec
To deploy all servers at once: `nix run nixpkgs#deploy-rs -- .# -k` To deploy all servers at once: `nix run nixpkgs#deploy-rs -- .# -k`
To deploy only one server: `nix run nixpkgs#deploy-rs -- -k --targets .#<host>` To deploy only one server: `nix run nixpkgs#deploy-rs -- -k --targets .#<host>`
## Deploying to Kubernetes
To deploy to the Kubernetes cluster, first make sure you have an admin account on the cluster.
You can generate this using `nix run .#gen-k3s-cert <username> <servername> ~/.kube`, assuming you have SSH access to the master node.
This puts a private key, signed certificate and a kubeconfig in the kubeconfig directory
If the cluster has not been initialized yet, next run `nix run .#kubenix-bootstrap.x86_64-linux`.
⚠️ Do not do this if the cluster has been initialized already, as it will prune any deployed resources! ⚠️
Lastly, deploy everything to the cluster using `nix run .#kubenix.x86_64-linux`.
## Known bugs ## Known bugs
### Failed to connect to socket
When deploying a new virtiofs share, the error `Failed to connect to '<name>.sock': No such file or directory` can occur.
This seems to be a bug in `microvm.nix` and I opened a bug report [here](https://github.com/astro/microvm.nix/issues/200).
A workaround is to deploy the share without `deploy-rs`'s rollback feature enabled:
```
nix run nixpkgs#deploy-rs -- -k --targets .#<host> --auto-rollback false --magic-rollback false
```
### Rsync not available during bootstrap ### Rsync not available during bootstrap
The `rsync` command was removed from recent NixOS ISO which causes nixos-anywhere to fail when copying extra files. The `rsync` command was removed from recent NixOS ISO which causes nixos-anywhere to fail when copying extra files.

View file

@ -1,41 +0,0 @@
# Kubernetes
## Creating an admin certificate for k3s
Create the admin's private key:
```
openssl genpkey -algorithm ed25519 -out <username>-key.pem
```
Create a CSR for the admin:
```
openssl req -new -key <username>-key.pem -out <username>.csr -subj "/CN=<username>"
```
Create a Kubernetes CSR object on the cluster:
```
k3s kubectl create -f - <<EOF
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
name: <username>-csr
spec:
request: $(cat <username>.csr | base64 | tr -d '\n')
expirationSeconds: 307584000 # 10 years
signerName: kubernetes.io/kube-apiserver-client
usages:
- digital signature
- key encipherment
- client auth
EOF
```
Approve and sign the admin's CSR:
```
k3s kubectl certificate approve <username>-csr
```
Extract the resulting signed certificate from the CSR object:
```
k3s kubectl get csr <username>-csr -o jsonpath='{.status.certificate}' | base64 --decode > <username>.crt
```