Hello,
I am currently in the process of rebuilding my home server from scratch
using NetBSD and would appreciate a bit of sanity-checking from people
with more experience in this area.
The setup is a fairly typical Xen environment: one NetBSD Dom0 and a
small number of NetBSD guest VMs running various workloads. Some of
these are relatively data-intensive (e.g. a Samba file server and a
database), so three requirements are absolutely critical for me:
- Reliable, consistent online backups from running guests
- Good read/write performance, ideally close to raw disk access
- Flexibility in terms of free space allocation (on demand)
=== Background: backup workflow ===
For backups, I strongly prefer doing them from inside the guest, using
dump with FFS snapshots (dump -X / fssconfig), streaming the dump over
SSH to the host and writing it directly to an external USB disk. This
approach has worked very well for me over the years - when the storage
stack underneath behaves nicely.
Unfortunately, once LVM enters the picture, I have repeatedly run into
situations where triggering an FSS snapshot causes severe stalls or
complete lockups. ZFS zvols, while very attractive feature-wise, showed
significantly lower performance in my setup compared to raw partitions,
and also compared poorly to simpler approaches such as CCD or even
vnd-backed storage.
=== Performance analysis ===
Towards the end of last year I ran a series of simple but reproducible
performance tests to compare different Xen guest storage backends under
NetBSD. The results are described in more detail here:
https://www.petermann-digital.de/blog/netbsd-xen-storage/
In short, the observations were not too surprising:
- Raw partitions deliver the best and most predictable performance (as
expected).
- vnd-backed images introduce some overhead but are still reasonably close.
- ZFS zvols were significantly slower in my environment, especially for
write-heavy workloads.
- CCD-backed volumes, built from GPT partitions, came very close to raw
performance while remaining simple and transparent.
Given these results, CCD appeared to hit a sweet spot between
performance and flexibility, without introducing an additional
snapshotting or metadata layer that could interfere with FFS/FSS.
=== CCD-based approach and ccdtool ===
Based on this, I ended up using CCD volumes composed of multiple GPT
partitions, exposed to guests as block devices. To make this manageable,
I wrote a small shell tool (ccdtool,
https://forge.ext.d2ux.net/Polaris/toolbox/src/branch/main/ccdtool) that
provides very lightweight, LVM-like convenience:
- human-friendly aliases instead of raw ccdN names
- simple create/extend/destroy operations
- no hidden metadata, only GPT labels and /etc/ccd.conf
- no attempt to add snapshot logic — snapshots are still handled purely
at the FFS/FSS level
This setup has been running quietly and reliably for a few months now.
Why I’m posting? Now that I am about to rebuild everything cleanly and
migrate the data, I would like to pause for a moment and ask:
- Does this overall approach sound reasonable to you?
- Is there an important detail in my performance analysis that I might
have overlooked?
- Is there a simpler or more idiomatic NetBSD/Xen solution today that
would achieve the same goals (consistent guest-side backups + near-raw
performance + flexibility) without the complexity or issues I observed
with LVM and zvols?
Thanks a lot for your time and for any feedback you might have.
Best regards,
Matthias
--
Für alle, die digitale Systeme verstehen und gestalten wollen:
jede Woche neue Beiträge zu Architektur, Souveränität und Systemdesign.
👉 https://www.petermann-digital.de/blog
