Bug#1050001: Unwinding directory aliasing [and 3 more messages]
On Sat, 26 Aug 2023 at 11:27, Ian Jackson
wrote:
>
> Helmut Grohne writes ("Bug#1050001: Unwinding directory aliasing"):
> > On Wed, Aug 23, 2023 at 05:04:36PM +0100, Ian Jackson wrote:
> > > And, the approach being taken very seriously privileges Debian itself,
> > > and those well-staffed derivatives able to do the necessary transition
> > > auditing (albeit, indeed, with tooling from Debian). I am
> > > firmly ideologically opposed to such a tradeoff.
> >
> > I have difficulties disagreeing with that. Getting this done is more
> > important to me though.
>
> I have hoisted this to the start of the mail. I think it is a hugely
> important point.
>
> Debian is not simply a technical project trying to thread its way in a
> complicated world. Debian is an ideological project. At its best,
> Debian is the infrastructrure that enables vast swathes of people to
> massively enhance their own technological autonomy. Many of our most
> controversial decisions served this goal, and stood the test of time.
Exactly, and usr-merge provides exactly that infrastructure, which is
one of the many reasons why it is necessary, as it should be beyond
obvious by now.
> If you want to think about it on more practical (or even, selfish)
> level, we want Debian to continue to be the preferred choice, when
> someone is choosing an upstream. We didn't get where we are today by
> following bad technical decisions made by others.
Very true, and a perfect example of such a "bad technical decision" is
the symlinks-farm layout that is proposed by a couple of people: it
doesn't solve any real problem, and just causes issues for developers
and users, and is purely designed to carefully tip-toe around one
singular hopelessly broken debian-specific tool, dpkg, which suffers
from long-standing bad design decisions that have never been fixed to
this day. It was attempted once, it failed, it was rolled back and
actual usr-merge was successfully deployed.
> > * Basically every other distro uses aliasing now. I expect that
> >a few upstreams have stopped paying attention to systems in your end
> >state. Therefore, they may not only hard code paths in /usr/bin, but
> >also the other way round assume that everything is to be found in /.
>
> This is indeed a plausible practical reason to do it the aliased way.
> >From my point of view, it amounts to saying "everyone else has made
> this mistake, so to be compatible, we must too".
>
> But I think that seriously underestimates our influence. Debian
> derivatives make up well over half of all Linux installations.
> They're the default basis for most CI images. If we decide this was a
> failed experiment, then indeed there will be some pain for a while,
> but fairly soon people will stop making this assumption.
In case you haven't noticed, it's not 1998 anymore. It's 2023, and
Debian is sliding fast into irrelevance, largely because it takes a
decade of fighting off trolls and saboteurs to achieve the most
innocuous of changesl, while most other projects can just implement
obviously correct and needed features such as this one and many others
in a couple of months and then move on to the next. In other words,
other distros innovate while we stagnate. Nobody would truly care if
somehow madness descended on this project and such a grievous act of
self-harm was actually perpetrated, at most some raised eyebrows and
some "they are at it again, aren't they" snarky remarks. Certainly
nobody would move back. For example, I can provide a cast-iron
guarantee that systemd will keep supporting only an usr-merged layout
and not work anywhere else starting from the next version (currently
in main), so there goes ~99.5% of Debian installations.
> > * A key motivation cited for doing the merged-/usr work is called
> >"hermetic /usr". [...] Setting up the aliasing symlinks is easier and
> >less prone to change over time than setting up the symlink farm that
> >you are proposing.
>
> I don't like the phrase "symlink farm" because it suggests that all,
> most, or even a substantial minority, of files have these symlinks.
> True, at the start, there will be at least a symlink allotment
> but I'm hoping that in the end it'll be a symlink flowerbed.
"I'm hoping" doesn't cut it. There is only one example of such an
attempt, in OpenSUSE, and it never materialized. It's been 10 years,
and still the handful of proponents of symlinks-farm have absolutely
nothing to show other than "I'm hoping". What are we even doing here?
> > And then we have a large body of people who simply
> > want this to be over and not having to thing about /usr-merge
> > consequences anymore.
>
> Well, of course it is very tempting to declare the matter as settled
> and hope that it will go away. I too want an end state where we will
> eventually be able to forget about all of this.
The matter is settled, and was settled long ago to boot.
> Or to put it another way, the delays to completion of this
Re: Bug#1050001: Unwinding directory aliasing
On Sat, 26 Aug 2023 at 23:19, gregor herrmann wrote: > > On Sat, 26 Aug 2023 15:47:51 +0100, Luca Boccassi wrote: > > > On Wed, 23 Aug 2023 at 20:12, Simon McVittie wrote: > > > I would very much prefer it if we can keep this conversation respectful, > […] > > > If you're frustrated, I understand that, but please can we take that > > > off-list? > > The perfect time to discuss "creative" alternative approaches was ~12 > > years ago, when everybody else was discussing where to move towards. > > It was perfectly acceptable to discuss it before the first CTTE vote. > > It was still ok-ish before the second CTTE vote. In August 2023: > […] > > That may well be true, but I think you missed smvc's point: > > > and hence interpreting attempts to propose yet another different and > > incompatible layout (only and solely for the benefit of the hopelessly > > broken dpkg tool) as sealioning is not disrespectful, on the contrary > > it is the most charitable interpretation possible. > > What Simon is asking for, in my interpretation, is some form of > civilised conversation; what you are delivering, as a direct reply to > Simon's request, causes yet another major pain in my back. > > Please stop and reconsider how you communicate with your peers. And I think you missed mine: calling out sealioning as such is not "uncivilized". Far from it: after all this time, it is necessary.
Re: Bug#1050001: Unwinding directory aliasing
On Sat, 26 Aug 2023 15:47:51 +0100, Luca Boccassi wrote: > On Wed, 23 Aug 2023 at 20:12, Simon McVittie wrote: > > I would very much prefer it if we can keep this conversation respectful, […] > > If you're frustrated, I understand that, but please can we take that > > off-list? > The perfect time to discuss "creative" alternative approaches was ~12 > years ago, when everybody else was discussing where to move towards. > It was perfectly acceptable to discuss it before the first CTTE vote. > It was still ok-ish before the second CTTE vote. In August 2023: […] That may well be true, but I think you missed smvc's point: > and hence interpreting attempts to propose yet another different and > incompatible layout (only and solely for the benefit of the hopelessly > broken dpkg tool) as sealioning is not disrespectful, on the contrary > it is the most charitable interpretation possible. What Simon is asking for, in my interpretation, is some form of civilised conversation; what you are delivering, as a direct reply to Simon's request, causes yet another major pain in my back. Please stop and reconsider how you communicate with your peers. Cheers, gregor -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- signature.asc Description: Digital Signature
Re: Lifting the moratorium
Re: Timo Röhling > * Helmut Grohne [2023-08-26 11:39]: > > > Who is this going to be? Do you, Helmut, feel comfortable as driver? > > > > Yes, though I'm not opposed to sharing the load. Thanks Helmut! > It does not even have to be much, maybe a > short paragraph such as > > "Please monitor the document at for the current transition > status. Helmut has graciously agreed to coordinate the transition. > Please reach out to him if you want to help with the effort, or if > you have affected packages and are unsure how you should proceed." > > in the CTTE announcement is sufficient. I like this very much. Christoph
Re: Bug#1050001: Unwinding directory aliasing
On Wed, 23 Aug 2023 at 20:12, Simon McVittie wrote: > > I would very much prefer it if we can keep this conversation respectful, > because mediating between angry people is exhausting (and the more > spoons I have to put into that, the less effectively I can advocate for > the technical opinions that I think you and I mostly agree on, which I > assume is not what you want). > > I'm keen to avoid the anti-pattern where a valid technical point gets > disregarded or even opposed because the way it was expressed puts other > project members on the defensive. > > If you're frustrated, I understand that, but please can we take that > off-list? The perfect time to discuss "creative" alternative approaches was ~12 years ago, when everybody else was discussing where to move towards. It was perfectly acceptable to discuss it before the first CTTE vote. It was still ok-ish before the second CTTE vote. In August 2023: - the matter has been settled for a decade - everyone that matters has switched to the same layout - Debian's CTTE voted twice on the matter - Debian has switched to the same layout for all installations since Bookworm and for new installations since Buster - the final quirks in packaging that we have to deal with because dpkg is hopelessly broken beyond repair are all planned out and handled thanks to Helmut's work - upstreams have already started to drop support for anything but the new layout and hence interpreting attempts to propose yet another different and incompatible layout (only and solely for the benefit of the hopelessly broken dpkg tool) as sealioning is not disrespectful, on the contrary it is the most charitable interpretation possible. > On Wed, 23 Aug 2023 at 10:35:42 +0100, Luca Boccassi wrote: > > Suse [...] tried [symlink farms similar to those that Ian proposes] > > for a few years, then had to backtrack and > > go the way everyone else successfully went instead, which quickly wrapped > > up as > > expected. > > I alluded to that in a previous mail to this thread, but I don't have > first-hand knowledge of the specifics of how that went, and it sounds > as though you might. Do you have references that you can point us to? > (To the list or as private email for me to summarize on-list later, > whichever seems more appropriate. https://en.opensuse.org/openSUSE:Usr_merge_preparation#initial_situation_in_openSUSE
Re: Lifting the moratorium
Hi Helmut, * Helmut Grohne [2023-08-26 11:39]: Who is this going to be? Do you, Helmut, feel comfortable as driver? Yes, though I'm not opposed to sharing the load. Ack. Do you prefer if the Release Team takes over with you in an advisory role? If you continue as main driver, should the DPL formally delegate that role somehow? On a micro level, the Release Team will be involved. The plan is for me to file RC bugs. There will be disagreement about such bugs and the Release Team is the arbiter of severities and testing migration here. Ack. I'm surprised. We did have some contentious questions such as whether changing dpkg would be part of the solution and how the bootstrap protocol should look like. These seem settled now. Beyond these, the feedback mostly was that we should simply get it done. We will still see individual disagreements on particular packages, but those cannot be settled by some form of delegation as in effect they touch the package ownership and overriding that currently requires CTTE involvement. I'd rather see us not using the constitution hammer just because we see a nail and hope that I'm right about the expected cooperation. Maybe DPL delegation invokes the wrong kind of image here. I'm not asking for some dictator who pushes the transition through no matter what. What I do like to see is transparent commmunication about the process; we should be upfront about the fact that you drive this transition and not just kind of let it happen (which is what it has felt like to me at times). It does not even have to be much, maybe a short paragraph such as "Please monitor the document at for the current transition status. Helmut has graciously agreed to coordinate the transition. Please reach out to him if you want to help with the effort, or if you have affected packages and are unsure how you should proceed." in the CTTE announcement is sufficient. All the usual channels for dispute resolution remain available, but everyone is brought to the same page, even if they missed (or ignored) the d-devel discussion up this point. Also, any developer who just wants to "get it done" but does not have the time or energy to fully grok DEP-17 has the option to defer to you / the developers coordinating the transition, similar to what has happened with the debhelper NMU. Cheers Timo -- ⢀⣴⠾⠻⢶⣦⠀ ╭╮ ⣾⠁⢠⠒⠀⣿⡁ │ Timo Röhling │ ⢿⡄⠘⠷⠚⠋⠀ │ 9B03 EBB9 8300 DF97 C2B1 23BF CC8C 6BDD 1403 F4CA │ ⠈⠳⣄ ╰╯ signature.asc Description: PGP signature
Bug#1050001: Unwinding directory aliasing [and 3 more messages]
Helmut Grohne writes ("Bug#1050001: Unwinding directory aliasing"):
> On Wed, Aug 23, 2023 at 05:04:36PM +0100, Ian Jackson wrote:
> > And, the approach being taken very seriously privileges Debian itself,
> > and those well-staffed derivatives able to do the necessary transition
> > auditing (albeit, indeed, with tooling from Debian). I am
> > firmly ideologically opposed to such a tradeoff.
>
> I have difficulties disagreeing with that. Getting this done is more
> important to me though.
I have hoisted this to the start of the mail. I think it is a hugely
important point.
Debian is not simply a technical project trying to thread its way in a
complicated world. Debian is an ideological project. At its best,
Debian is the infrastructrure that enables vast swathes of people to
massively enhance their own technological autonomy. Many of our most
controversial decisions served this goal, and stood the test of time.
That's why *I'm* involved in Debian. Our technical choices should
serve those goals, always.
(To an extent, this divergence in goals may explain why at times our
comments have been talking slightly past each other.)
If you want to think about it on more practical (or even, selfish)
level, we want Debian to continue to be the preferred choice, when
someone is choosing an upstream. We didn't get where we are today by
following bad technical decisions made by others.
> * Basically every other distro uses aliasing now. I expect that
>a few upstreams have stopped paying attention to systems in your end
>state. Therefore, they may not only hard code paths in /usr/bin, but
>also the other way round assume that everything is to be found in /.
This is indeed a plausible practical reason to do it the aliased way.
>From my point of view, it amounts to saying "everyone else has made
this mistake, so to be compatible, we must too".
But I think that seriously underestimates our influence. Debian
derivatives make up well over half of all Linux installations.
They're the default basis for most CI images. If we decide this was a
failed experiment, then indeed there will be some pain for a while,
but fairly soon people will stop making this assumption.
> * A key motivation cited for doing the merged-/usr work is called
>"hermetic /usr". [...] Setting up the aliasing symlinks is easier and
>less prone to change over time than setting up the symlink farm that
>you are proposing.
I don't like the phrase "symlink farm" because it suggests that all,
most, or even a substantial minority, of files have these symlinks.
True, at the start, there will be at least a symlink allotment
but I'm hoping that in the end it'll be a symlink flowerbed.
> And then we have a large body of people who simply
> want this to be over and not having to thing about /usr-merge
> consequences anymore.
Well, of course it is very tempting to declare the matter as settled
and hope that it will go away. I too want an end state where we will
eventually be able to forget about all of this.
But pushing ahead won't lead to such a state. As I say, I think
people will keep introducing new references to files by their
non-physical names, and we'll keep getting lossage, essentially
forever. (Adopting Simon's terminology.)
Or to put it another way, the delays to completion of this project
have not been due to the political opposition,. They have been
because the project encountered technical problems. Problems whose
existence was predicted by subject matter experts but dismissed at the
time as FUD. Problems which were apparently not regarded as real by the
non-expert decisionmakers on the TC. Problems which still remain in
large part unresolved, albeit in some caes "mitigated".
> > Aliasing is EBW, and "Only use canonical names" is not good enough
> > ==
> >
> > There is basically one underlying technical reason for preferring the
> > un-aliased usrmerge approach: aliasing directories in this way leads
> > to great complication in file management, especially in package
> > management software and in individual packages.
>
> I'm not sure I follow this argument precisely.
This argument is basically drawing together the common factor in the
DEP-17 problem list.
> these complications mostly affect ourselves and
> our package management while end users are mostly unaffected.
I think "package management" is the wrong term here. It's not just
our tools and packages that are affected. All forms of operating
system management are affected: anything that changes the software,
and not just its configuration.
Affected tooling includes not just our .debs, but also remote
configuration management systems like Ansible, image construction
(Dockerfiles), and 3rd-party software installation progrmas (including
both 3rd-party .debs and 3rd-party script-based installation systems).
And yes, actual *end users* (especially of something like Ubuntu)
Re: Lifting the moratorium
Hi Timo, On Sat, Aug 26, 2023 at 01:32:25AM +0200, Timo Röhling wrote: > Practically, lifting the moratorium means that the CTTE relinquishes > control of the /usr-merge transition to whoever drives the > transition, i.e., maintains that living document and thereby makes > decisions on how to proceed. Yes. > Who is this going to be? Do you, Helmut, feel comfortable as driver? Yes, though I'm not opposed to sharing the load. > Do you prefer if the Release Team takes over with you in an advisory > role? If you continue as main driver, should the DPL formally > delegate that role somehow? Isn't this "just" a transition? I mean it's a larger one spanning multiple releases and it's one that will not migrate to testing in one step, but other than that, it is a change affecting many packages in a coordinated way which is what we usually call transition. A transition bug probably is a good idea, but other than that, transitions are usually managed by those doing the work without exercising constitutional powers. I sincerely hope that we as a project can get this done in a consensual way without requiring the use of constitutional mechanisms to make it work. Yes, there have been a lot of bad feelings and even questionable actions and so forth about this, but on a lot of sides I also see an interest to listen to each other and cooperation. People who formerly have indicated to be too annoyed by this matter to consider discussing it start doing so again. Why not try another time to put trust in our community? The worst thing that could happen seems to be us messing up the transition and making some systems unbootable until we figure out a solution. Given dumat, we can likely keep that out of trixie. If things fail that way, we can still consider the use of more constitutional powers, but I'd rather see us getting done without needing them. On a micro level, the Release Team will be involved. The plan is for me to file RC bugs. There will be disagreement about such bugs and the Release Team is the arbiter of severities and testing migration here. > I think these are questions we need answered, because it would be > advisable to have someone with clearly defined responsibilities at > the wheel for the transition. Otherwise, we risk stale-mating > ourselves with endless debates on how to finish the transition > "properly". I'm surprised. We did have some contentious questions such as whether changing dpkg would be part of the solution and how the bootstrap protocol should look like. These seem settled now. Beyond these, the feedback mostly was that we should simply get it done. We will still see individual disagreements on particular packages, but those cannot be settled by some form of delegation as in effect they touch the package ownership and overriding that currently requires CTTE involvement. I'd rather see us not using the constitution hammer just because we see a nail and hope that I'm right about the expected cooperation. Helmut
