On Sat, Dec 1, 2018 at 12:00 AM Josh Paetzel <[email protected]> wrote:
>
>
>
>
>
> > On Nov 30, 2018, at 5:40 PM, Rich via openzfs-developer
> > <[email protected]> wrote:
> >
> > Hi all,
> > So, I spent a little bit pondering the idea from here [1] of a zpool
> > compat flag to allow people to minimize foot-gun interactions when
> > creating pools and wanting to migrate them around, and here's what I
> > came up with. I'd like comments on it, ideally, before I go cut a
> > proof of concept, but without them, I'll probably just go write it and
> > submit a PR.
> >
> > I started off thinking about what the things I wanted most to do with
> > this, and came up with:
> > - zpool create without fear that you're going to render the pool
> > unimportable on platform XYZ
> > - zfs send with the avoidance of feature flags that would be
> > unsupported by the other side
> > - zfs recv with the ability to avoid flipping feature bits that would
> > result in the pool not being importable by something
> > - zpool create not defaulting to all flags on as a reasonable change
> > without significant negative impact
> >
> > I don't know if I'll convince people of 2,3,4, or especially 5, but
> > that's what I'd like.
> >
> > Given that, I'd propose a flag -C on {zpool create, zfs send, zfs
> > recv} (since that's only ever already used in zfs project on ZoL
> > 0.8.x, that I can find quickly), with usage like:
> >
> > zpool create -C 'zol0.7,zol0.6.5,freebsd11'
> > zfs send -C 'zol0.7,zol0.6.5'
> >
> > which ends up:
> > - calculating a list of supported features for each platform
> > enumerated (in my head, a bitmask with AND across all the bitmasks
> > enumerated)
> > - erroring if you try to (or would need to, for recv) explicitly
> > enable a feature incompatible with those in the command (lest we end
> > up with people doing zfs send -Lcew -C 'zol0.6.5' and ending up with
> > Very Different streams than they expected)
> > - for at least zpool create, enabling all the features compatible with
> > that least-common-features calculation
> >
> > send/recv might be contentious for a few reasons, but my logic here is:
> > - if a send stream is generated requiring a feature on the receiver,
> > and sent to a pool with the feature disabled, receiving a stream into
> > that pool without explicit zpool upgrade/zpool set will already fail
> > - if a pool has a feature enabled but not active, you'd like a way to
> > receive sends without losing whatever backward compat constraints you
> > have (e.g. having a feature enabled but not active doesn't mean you
> > want something to be able to set it active)
> > - if pool1 has a feature active, and you want to backup to pool2 and
> > pool3, you'd like a way to readily generate a send stream with the
> > least-common-features of pool2 and pool3 without having to
> > hand-calculate it, if possible
> >
> > For zpool create, my logic for the enable-all-possible (and default to
> > all-off) with that specification is:
> > - zpool create defaulting to all-features-enabled is a recurring
> > compat problem for new users who later want to switch/move frequently
> > from one platform to another, but they didn't use the correct arcana
> > at create time and are stuck until the all-implemented-feature set is
> > what they have
> > - Making people explicitly enumerate every feature they want to use at
> > create time is a mild and error-prone pain, and will only get worse as
> > the enumerations get bigger
> > - Defaulting to all-features-off is less of a problem for
> > managed/expert systems than defaulting to all-features-on is for
> > personal/smaller users, but...
> > - It'd be useful to both to be able to say "all","none", or "common
> > subset of XYZ" without ever-growing parameter lists
> >
> > That's quite a lot of words for a fairly simple concept, and I don't
> > think there's anything I haven't conveyed here about the idea itself,
> > but here are two minor thoughts as well:
> > - something to print the enumeration of what "featuresets" it knows
> > (just names, or names and values), as well as being able to print
> > which of those enumerations it is, would probably be useful both for
> > people unfamiliar and for tooling to readily be able to do
> > create/send/recv -C 'sideA,sideB'
> > - I'd really like to rephrase or get rid of the "you can zpool
> > upgrade" notice in zpool status by default, because while it says "
> > Once this is done, the pool may no longer be accessible by software
> > that does not support the features.", people keep running zpool
> > upgrade because zpool status mentioned it, even when they didn't need
> > any of the new features involved, and are upset when they find out
> > what that meant. Maybe make it print that behind an explicit flag so
> > people who want that information can still get it, while not handing
> > unfamiliar users a sharp tool and suggesting they juggle it.
> >
> > Anyway, as I said, comments/criticisms welcomed, but please warn me
> > before gratuitous flaming, I need to go invest in asbestos
> > undergarments before you do that. :)
> >
> > - Rich
>
> I read this a couple times. I see where you are coming from and I think your
> points are valid.
First, thank you for taking the time to read it.
> I can’t help but feel that a priori knowledge of feature flags violates the
> whole point of having feature flags in the first place. (Which is to say a
> mapping between a specific version of an OS and what that means for feature
> flags)
I thought the whole point of feature flags was to have useful
awareness of features added in parallel (and enough tagging on-disk to
tell you you could/couldn't import it r/o, if it was an unknown one)
without relying on a monotonic version number as had been the behavior
before OpenZFS. Having a convenience enumeration shorthand for
compatibility doesn't particularly violate that, AFAICT
> It’s also very easy to get confused between “feature flags supported by
> FreeBSD 11.1-R” and “feature flags enabled on a pool created on FreeBSD 9.2-R
> which happens to be imported on a FreeBSD 11.1-R system” I think this would
> be the flu in the ointment for send/recv.
Howso? On the recv side, you're returning an error if it would require
a feature go from enabled->active; returning an error on requiring
disabled->active is the existing behavior anyway.
On the send side, AFAIK it's perfectly reasonable to specify e.g. -L,
-c, -e without actually having those features enabled (bugs in -L
aside).
> What about a simpler policy: “A port of OpenZFS should only enable feature
> flags by default that are common across all tier 1 ports at that point in
> time.” Where tier 1 would be defined for now as IllumOS, ZoL, and FreeBSD.
It took me a bit of thinking before I realized why this idea didn't
sit well with me. This would mean that the defaults for zpool create
change every release, so you only reduce the chance of getting stung
by moving to an incompatible OS featureset, not remove it. (Hell, if
ORCL hadn't broken the ability to interoperate with v28/v5 pools, I'd
probably be advocating defaulting to that, but that bridge is a crispy
cinder.)
Consider if you create a pool on a test release (e.g. ZoL 0.8-rc or
git, or FBSD HEAD). When should it change from defaulting to the flags
of {itself last stable + other things last stable it knows about} to
{itself nightly supported + other last stable it knows about}? What if
you run create on 0.8.0 and then realize there's a nasty bug in 0.8,
but you can't roll back to 0.7 because you already incremented the
default features used. (You could argue for OS $VERSION only
describing OS $VERSION-1's supported bits, but I still don't like the
idea of ever-rolling defaults, for the same reason I don't like the
ever-changing all-on. Also, this would require maintaining the
enumeration of supported features anyway, without the additional
benefits.)
I'd like, ideally, for there to be a system where
- you can specify what systems you know you're going to use this on,
and have the maximum number of features result, rather than having to
solve the set intersection yourself.
- people who were already doing explicit enumeration can keep doing that
- the default does not result in a bad experience 6 months later when
you go to do e.g. ZoL->FreeBSD and discover a feature you never needed
was default-on and you have no recourse.other than hoping zhack can
flip bits or recreate the pool
- zpool status does not suggest zpool upgrade every time you run it,
because that's similar to explicitly telling someone "don't press this
button" every time they enter a room.
So I suppose I'd also like an upgrade -C for shorthand rather than
upgrade just flipping every bit, but that still feels way too easy to
shoot yourself in the foot with. (I think I'd really like zpool
upgrade to require some very explicit flags before possibly screwing
the user over by flipping every bit, rather than it being the default
behavior of zpool upgrade [pool], but that's not the point I'd like to
discuss today.)
Anyway, I hope this seemed like a reasonable explanation of my
thoughts on your simpler proposal, and your disagreements with mine.
(Fundamentally, I just really want to get status to stop printing
zpool upgrade every time and zpool create to not default to SURPRISE
YOU DID IT WRONG O(months,years) later; the rest is less important to
me but seemed like a necessary level of convenience to get away from
e.g. possibly ending up hitting argv[] length limits as we keep
growing our zpool create -o -O lists.)
- Rich
------------------------------------------
openzfs: openzfs-developer
Permalink:
https://openzfs.topicbox.com/groups/developer/T04955dd2e8aa4e5d-M52933601ef8e0d4ac5c28b7e
Delivery options: https://openzfs.topicbox.com/groups/developer/subscription
