62 lines
3.2 KiB
Markdown
62 lines
3.2 KiB
Markdown
|
---
|
||
|
|
layout: post
|
||
|
|
title: My Experiences with virtio-9p
|
||
|
|
date: 2023-05-31 14:18:00 Europe/Amsterdam
|
||
|
|
categories: libvirt virtio 9p
|
||
|
|
---
|
||
|
|
|
||
|
|
When I was scaling up my home lab, I started thinking more about data management.
|
||
|
|
I hadn't (and still haven't) set up any form of network storage.
|
||
|
|
I have, however, set up a backup mechanism using [Borg](https://borgbackup.readthedocs.io/en/stable/).
|
||
|
|
Still, I want to operate lots of virtual machines, and backing up each one of them separately seemed excessive.
|
||
|
|
So I started thinking, what if I just let the host machines back up the data?
|
||
|
|
After all, the amount of physical hosts I have in my home lab is unlikely to increase drastically.
|
||
|
|
|
||
|
|
# The Use Case for Sharing Directories
|
||
|
|
|
||
|
|
I started working out this idea further.
|
||
|
|
Without network storage, I needed a way for guest VMs to access the host's disks.
|
||
|
|
Here there are two possibilities, either expose some block device or a file system.
|
||
|
|
Creating a whole virtual disk for just the data of some VMs seemed wasteful, and from my experiences also increases backup times dramatically.
|
||
|
|
I therefore searched for a way to mount a directory from the host OS on the guest VM.
|
||
|
|
This is when I stumbled upon [this blog](https://rabexc.org/posts/p9-setup-in-libvirt) post talking about sharing directories with virtual machines.
|
||
|
|
|
||
|
|
# Sharing Directories with virtio-9p
|
||
|
|
|
||
|
|
virtio-9p is a way to map a directory on the host OS to a special device on the virtual machine.
|
||
|
|
In `virt-manager`, it looks like the following:
|
||
|
|
|
||
|
|
Under the hood, virtio-9p uses the 9pnet protocol.
|
||
|
|
Originally developed at Bell Labs, support for this is available in all modern Linux kernels.
|
||
|
|
If you share a directory with a VM, you can then mount it.
|
||
|
|
Below is an extract of my `/etc/fstab` to automatically mount the directory:
|
||
|
|
```
|
||
|
|
data /mnt/data 9p trans=virtio,rw 0 0
|
||
|
|
```
|
||
|
|
|
||
|
|
The first argument (`data`) refers to the name you gave this share from the host
|
||
|
|
With the `trans` option we specify that this is a virtio share.
|
||
|
|
|
||
|
|
# Problems with virtio-9p
|
||
|
|
|
||
|
|
At first I had no problems with my setup, but I am now contemplating just moving to a network storage based setup because of two problems.
|
||
|
|
|
||
|
|
The first problem is that some files have suddenly changed ownership from `libvirt-qemu` to `root`.
|
||
|
|
If the file is owned by `root`, the guest OS can still see it, but cannot access it.
|
||
|
|
I am not entirely sure the problem lies with virtio, but I suspect it is.
|
||
|
|
For anyone experiencing this problem, I wrote a small shell script to revert ownership to the `libvirt-qemu` user:
|
||
|
|
```shell
|
||
|
|
find -printf "%h/%f %u\n" | grep root | cut -d ' ' -f1 | xargs chown libvirt-qemu:libvirt-qemu
|
||
|
|
```
|
||
|
|
|
||
|
|
Another problem that I have experienced, is guests being unable to mount the directory at all.
|
||
|
|
I have only experienced this problem once, but it was highly annoying.
|
||
|
|
To fix it, I had to reboot the whole physical machine.
|
||
|
|
|
||
|
|
# Alternatives
|
||
|
|
|
||
|
|
virtio-9p seemed like a good idea, but as discussed, I had some problems with it.
|
||
|
|
It seems [virtioFS](https://virtio-fs.gitlab.io/) might be a an interesting alternative as it is designed specifically for sharing directories with VMs.
|
||
|
|
|
||
|
|
As for me, I will probably finally look into deploying network storage either with NFS or SSHFS.
|