On Sun, Jan 30, 2022 at 06:43:13PM +0100, Zdenek Kabelac wrote:
> Dne 30. 01. 22 v 17:45 Demi Marie Obenour napsal(a):
> > On Sun, Jan 30, 2022 at 11:52:52AM +0100, Zdenek Kabelac wrote:
> > > Dne 30. 01. 22 v 1:32 Demi Marie Obenour napsal(a):
> > > > On Sat, Jan 29, 2022 at 10:32:52PM +0100, Zdenek Kabelac wrote:
> > > > > Dne 29. 01. 22 v 21:34 Demi Marie Obenour napsal(a):
> > > > > > How much slower are operations on an LVM2 thin pool compared to
> > > > > > manually
> > > > > > managing a dm-thin target via ioctls? I am mostly concerned about
> > > > > > volume snapshot, creation, and destruction. Data integrity is very
> > > > > > important, so taking shortcuts that risk data loss is out of the
> > > > > > question. However, the application may have some additional
> > > > > > information
> > > > > > that LVM2 does not have. For instance, it may know that the volume
> > > > > > that
> > > > > > it is snapshotting is not in use, or that a certain volume it is
> > > > > > creating will never be used after power-off.
> > > > > >
> > > >
> > > > > So brave developers may always write their own management tools for
> > > > > their
> > > > > constrained environment requirements that will by significantly
> > > > > faster in
> > > > > terms of how many thins you could create per minute (btw you will
> > > > > need to
> > > > > also consider dropping usage of udev on such system)
> > > >
> > > > What kind of constraints are you referring to? Is it possible and safe
> > > > to have udev running, but told to ignore the thins in question?
> > >
> > > Lvm2 is oriented more towards managing set of different disks,
> > > where user is adding/removing/replacing them. So it's more about
> > > recoverability, good support for manual repair (ascii metadata),
> > > tracking history of changes, backward compatibility, support
> > > of conversion to different volume types (i.e. caching of thins, pvmove...)
> > > Support for no/udev & no/systemd, clusters and nearly every linux distro
> > > available... So there is a lot - and this all adds quite complexity.
> >
> > I am certain it does, and that makes a lot of sense. Thanks for the
> > hard work! Those features are all useful for Qubes OS, too — just not
> > in the VM startup/shutdown path.
> >
> > > So once you scratch all this - and you say you only care about single disc
> > > then you are able to use more efficient metadata formats which you could
> > > even keep permanently in memory during the lifetime - this all adds great
> > > performance.
> > >
> > > But it all depends how you could constrain your environment.
> > >
> > > It's worth to mention there is lvm2 support for 'external' 'thin volume'
> > > creators - so lvm2 only maintains 'thin-pool' data & metadata LV - but
> > > thin
> > > volume creation, activation, deactivation of thins is left to external
> > > tool.
> > > This has been used by docker for a while - later on they switched to
> > > overlayFs I believe..
> >
> > That indeeds sounds like a good choice for Qubes OS. It would allow the
> > data and metadata LVs to be any volume type that lvm2 supports, and
> > managed using all of lvm2’s features. So one could still put the
> > metadata on a RAID-10 volume while everything else is RAID-6, or set up
> > a dm-cache volume to store the data (please correct me if I am wrong).
> > Qubes OS has already moved to using a separate thin pool for virtual
> > machines, as it prevents dom0 (privileged management VM) from being run
> > out of disk space (by accident or malice). That means that the thin
> > pool use for guests is managed only by Qubes OS, and so the standard
> > lvm2 tools do not need to touch it.
> >
> > Is this a setup that you would recommend, and would be comfortable using
> > in production? As far as metadata is concerned, Qubes OS has its own
> > XML file containing metadata about all qubes, which should suffice for
> > this purpose. To prevent races during updates and ensure automatic
> > crash recovery, is it sufficient to store metadata for both new and old
> > transaction IDs, and pick the correct one based on the device-mapper
> > status line? I have seen lvm2 get in an inconsistent state (transaction
> > ID off by one) that required manual repair before, which is quite
> > unnerving for a desktop OS.
>
> My biased advice would be to stay with lvm2. There is lot of work, many
> things are not well documented and getting everything running correctly will
> take a lot of effort (Docker in fact did not managed to do it well and was
> incapable to provide any recoverability)
What did Docker do wrong? Would it be possible for a future version of
lvm2 to be able to automatically recover from off-by-one thin pool
transaction IDs?
> > One feature that would be nice is to be able to import an
> > externally-provided mapping of thin pool device numbers to LV names, so
> > that lvm2 could provide a (read-only, and not guaranteed fresh) view of
> > system stat