Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
Luca Boccassi writes: > And I am more than a bit sad that sensible, clear-cut, binding and > already-implemented decisions taken by our constitutional bodies get > constantly second-guessed and belittled because of an irrational > attachment to inconsequential accidents of history. But what can we do, > we'll just have to be sad together, I guess. > Aside from that, in the future please avoid using the word 'minorities' > when talking about silly things such as liking or disliking symlinks, as > in common English it is used to refer to much more serious matters. Okay, stop. We are not going to have a Policy conversation like this. Nothing about this discussion style helps in making a good decision. Luca, you have asked repeatedly what you can do to advance the status of your open Policy bugs. Most of the answer to that question is that a lot of it is outside of your control; I've been having a spectacularly annoying summer in a bunch of minor ways and, frankly, simply haven't done much of anything I'm responsible for in Debian. But the most helpful thing that you could possibly do would be to stop being utterly emotionally exhausting to deal with. Case in point: now I'm writing this email message rather than doing any of the other things that I am quite sure you would rather that I be doing and that you would consider a much higher priority. In this bug, about a point that from the Policy perspective is mostly not even normative, which appears to partly be a bug report against Lintian by proxy (the Policy Editors do not maintain Lintian), you started at an emotional level of eight out of ten before anyone even disagreed with you. Let me quote the start of this bug: I heard many times the policy maintainers mention something along the lines of 'policy should not be a hammer to beat other maintainers with'. Today I saw policy being used to force a maintainer to re-add support for the deprecated and unsupported split-usr filesystem layout, as 'policy only mentions /bin/sh, not /usr/bin/sh'. So let's update the policy to refer to modern and supported filesystem paths as adopted by Debian de-facto and de-jure, and stop other maintainers from getting beaten with it. So right from the first message, you're saying that the goal of this proposal is to stop other people from doing something socially you don't like that Policy doesn't even tell them to do. And you're using phrasing that you know is loaded and contentious and that you know the people you've been arguing with all over Debian will disagree with. I have no idea why. Because you like making them mad? Because you're mad and you want to keep rubbing in their face the fact you disagree with them? Maybe you think this is your strongest argument? (It is not.) This is pretty much guaranteed to turn the bug into a fight regardless of its technical merits, because this presentation essentially says that you're spoiling for a fight and want someone to attack you. I cannot remember the last time I have seen someone who is this unwilling to win an argument. It's driving me nuts. You are winning your technical arguments in Debian! The technical design that you support is being implemented because you and others have convinced people that it is a good idea, even though some people are very upset about this! There are even people who don't entirely agree with you investing significant amounts of time and energy to help you realize your technical vision of how Debian should be assembled. And yet, every time I see you in a Debian architectural discussion, you're fighting with everyone in sight as if you're losing. You come across as the most furious winner of an argument I can remember seeing. This is directly undermining what you're attempting to achieve, because the sheer vehemence of the way that you argue every point you care about means that even people who agree with you don't want to help you get things done because even being around this level of fighting is deeply exhausting. A lot of this hasn't been on Policy, although you do have some of the same tendency to reply to everything you disagree with until people stop disagreeing even here. But elsewhere, including the Technical Committee discussion that I assume you were referring to above, it happens a lot. I can only assume that you think you have to shove harder and harder to get things to move faster, but this isn't how anything works, particularly in a volunteer organization. People work on things they enjoy working on. Calm and productive technical work is enjoyable to a lot of us. Watching people fight is not enjoyable. What you're actually accomplishing is building a subconscious emotional association in the brains of people who are watching and reading between your name and angry arguments. You are literally training people to avoid you. You are never going to convince people to stop disagreeing with you. Never. We have a process for
Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
On Fri, 8 Sept 2023 at 00:37, gregor herrmann wrote: > > On Thu, 07 Sep 2023 21:28:15 +0100, Luca Boccassi wrote: > > > Yes, that is fine by me, as explained in later replies my main > > intention is to fix the issue that some wording is being used to > > reintroduce things that should not be reintroduced > > If I understand you correctly, "Reintroduc[ing] things that should not be > reintroduced" means not acting on bug reports where someone says > "Please change #!/usr/bin/sh back to #!/bin/sh". > > If I'm understanding the technical question correctly, "#!/bin/sh" > works for both merged-/usr and split-/usr, while "#!/usr/bin/sh" works > only for merged-/usr and breaks for split-/usr. > > Now, how to handle this situation? In my naïve point of view and a > bit late at night I see two options for maintainers. They can say … The entire point of the decision that applies since bookworm is that there is only one supported form. That's not an accident, it is the entire point of the whole thing. As in, there is no such thing as "breaks split-/usr" because split-usr is no longer a thing that exists in Debian stable/backports/testing/unstable/experimental. > I'm a bit sad that many discussions in Debian (like this one) turn > into the "my way or the highway" lane, and I'd rather see a way of > cooperation which gives more leeway to others and take small extra > steps to accomodate minorities, when it doesn't have any actual cost. And I am more than a bit sad that sensible, clear-cut, binding and already-implemented decisions taken by our constitutional bodies get constantly second-guessed and belittled because of an irrational attachment to inconsequential accidents of history. But what can we do, we'll just have to be sad together, I guess. Aside from that, in the future please avoid using the word 'minorities' when talking about silly things such as liking or disliking symlinks, as in common English it is used to refer to much more serious matters.
Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
On Thu, 07 Sep 2023 21:28:15 +0100, Luca Boccassi wrote: > Yes, that is fine by me, as explained in later replies my main > intention is to fix the issue that some wording is being used to > reintroduce things that should not be reintroduced If I understand you correctly, "Reintroduc[ing] things that should not be reintroduced" means not acting on bug reports where someone says "Please change #!/usr/bin/sh back to #!/bin/sh". If I'm understanding the technical question correctly, "#!/bin/sh" works for both merged-/usr and split-/usr, while "#!/usr/bin/sh" works only for merged-/usr and breaks for split-/usr. Now, how to handle this situation? In my naïve point of view and a bit late at night I see two options for maintainers. They can say … 1) "Well, right, *sigh*, ok, let's take the five minutes to change this back to "#!/bin/sh", and everyone's happy." … or … 2) "No way, we are usr-merge, resistence is futile, go yourself, "#!/usr/bin/sh" is the default, bad luck for you 111" I'm a bit sad that many discussions in Debian (like this one) turn into the "my way or the highway" lane, and I'd rather see a way of cooperation which gives more leeway to others and take small extra steps to accomodate minorities, when it doesn't have any actual cost. Cheers, gregor -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- signature.asc Description: Digital Signature
Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
On Thu, 7 Sept 2023 at 21:22, Helmut Grohne wrote: > > Hi Luca, > > On Wed, Sep 06, 2023 at 10:50:14PM +0100, Luca Boccassi wrote: > > Package: debian-policy > > X-Debbugs-Cc: j...@debian.org hel...@subdivi.de > > > > Debian only supports merged-usr since Bookworm. We should update policy > > to reference /usr/bin/sh and similar paths to describe recommended > > shebangs for scripts. > > I disagree. The promise of merged-/usr has always been that both paths > are valid. /bin/sh remains the location recommended by external > standards and (like the dynamic loader path) should remain the way it > is. > > > I heard many times the policy maintainers mention something along the > > lines of 'policy should not be a hammer to beat other maintainers > > with'. Today I saw policy being used to force a maintainer to re-add > > support for the deprecated and unsupported split-usr filesystem layout, > > as 'policy only mentions /bin/sh, not /usr/bin/sh'. > > This can also be addressed by adding a note to policy that allows > maintainers to rely on the aliasing. If there was a need to refer to the > shell via /usr/bin/sh, we would aim for eventually removing the aliasing > symlinks. That's not what we're up to. > > > So let's update the policy to refer to modern and supported filesystem > > paths as adopted by Debian de-facto and de-jure, and stop other > > maintainers from getting beaten with it. > > I don't think this is right. We intend to finalize the /usr-merge > transition by moving files from / to /usr. This is is an implementation > strategy that arises from the constraints set by the current > implementation of dpkg and other components. It is not a new filesystem > layout that we expect upstreams to support. Rather, we promised to > upstreams that both ways will work. The aspect that in a data.tar we'll > have to install files to /usr is a technical one and can be supported by > debhelper. Still, packages may assume that referencing files they > installed to /usr via aliased paths in / will continue to work. > > > Patch attached and also pushed to > > https://salsa.debian.org/bluca/policy/-/tree/bin_sh > > Nack to this particular change, but I agree that it is worth considering > two changes to policy sooner and later: > * Making it explicit that referring to files via either paths for >read-only consumption is ok. > * DEP17 aims for not installing any files in aliased locations and we >should encode that in policy once there is wide adoption of this rule >in binary packages. > > Would you agree to repurpose this bug to propose the former change? > While my variant is weaker, it still prevents people from using policy > to require supporting split-/usr. Yes, that is fine by me, as explained in later replies my main intention is to fix the issue that some wording is being used to reintroduce things that should not be reintroduced
Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
Hi Luca, On Wed, Sep 06, 2023 at 10:50:14PM +0100, Luca Boccassi wrote: > Package: debian-policy > X-Debbugs-Cc: j...@debian.org hel...@subdivi.de > > Debian only supports merged-usr since Bookworm. We should update policy > to reference /usr/bin/sh and similar paths to describe recommended > shebangs for scripts. I disagree. The promise of merged-/usr has always been that both paths are valid. /bin/sh remains the location recommended by external standards and (like the dynamic loader path) should remain the way it is. > I heard many times the policy maintainers mention something along the > lines of 'policy should not be a hammer to beat other maintainers > with'. Today I saw policy being used to force a maintainer to re-add > support for the deprecated and unsupported split-usr filesystem layout, > as 'policy only mentions /bin/sh, not /usr/bin/sh'. This can also be addressed by adding a note to policy that allows maintainers to rely on the aliasing. If there was a need to refer to the shell via /usr/bin/sh, we would aim for eventually removing the aliasing symlinks. That's not what we're up to. > So let's update the policy to refer to modern and supported filesystem > paths as adopted by Debian de-facto and de-jure, and stop other > maintainers from getting beaten with it. I don't think this is right. We intend to finalize the /usr-merge transition by moving files from / to /usr. This is is an implementation strategy that arises from the constraints set by the current implementation of dpkg and other components. It is not a new filesystem layout that we expect upstreams to support. Rather, we promised to upstreams that both ways will work. The aspect that in a data.tar we'll have to install files to /usr is a technical one and can be supported by debhelper. Still, packages may assume that referencing files they installed to /usr via aliased paths in / will continue to work. > Patch attached and also pushed to > https://salsa.debian.org/bluca/policy/-/tree/bin_sh Nack to this particular change, but I agree that it is worth considering two changes to policy sooner and later: * Making it explicit that referring to files via either paths for read-only consumption is ok. * DEP17 aims for not installing any files in aliased locations and we should encode that in policy once there is wide adoption of this rule in binary packages. Would you agree to repurpose this bug to propose the former change? While my variant is weaker, it still prevents people from using policy to require supporting split-/usr. Helmut
Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
> "Bill" == Bill Allombert writes: Bill> I would. Having two paths for the same thing is a technical Bill> debt going forward. I think the TC has made it clear we're committed to usrmerge at this point, and I think that one of the drivers behind usrmerge is that we gain more from having both /bin/python and /usr/bin/python work than we lose by having two paths. I understand people disagree, but I think we should fall in behind the TC decision and accept we've decided that in this instance we value aliasing.
Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
On Wed, Sep 06, 2023 at 04:51:10PM -0600, Sam Hartman wrote: > > "Luca" == Luca Boccassi writes: > Luca> /bin/sh is not universally compatible with non-Linux OSes. > > I claim it is more compatible. > > > Luca> Also I thought that policy should not be used to beat other > Luca> developers (it is because of this) and it should reflect the > Luca> common practices adopted in Debian (it does not because of > Luca> this). Is that no longer the case? > > I'd consider it a non-RC bug if someone were manually writing > #!/usr/bin/sh > > We can debate about normal vs minor. > > I do not think it should be a bug if some automated build process found > /usr/bin/sh and stuck that into a script. > I'd support a policy change to make that clear. I would. Having two paths for the same thing is a technical debt going forward. Cheers, -- Bill. Imagine a large red swirl here.
Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
> "Ansgar" == Ansgar writes: Ansgar> On Wed, 2023-09-06 at 16:51 -0600, Sam Hartman wrote: >> > > > > > "Luca" == Luca Boccassi writes: >> Luca> /bin/sh is not universally compatible with non-Linux OSes. >> >> I claim it is more compatible. Ansgar> Why should that matter to Debian? Debian has traditionally valued supporting common interfaces like posix, fhs, Linux ABIs, etc where that makes sense. We recently had a discussion of the value of interfaces in the discussion of changing the ABI to make merged /usr easier, and I do not want to revisit that. I do value this sort of interface stability, and Debian's alignment with my values is one of the things that drew me to Debian. So, yes, I do believe we should support encouraging portability where that is reasonable for us. I admit that I care more about OSes like FreeBSD than Mac OS. More over, when merged /usr was presented to the project, it was presented as a way to move the physical locations on files and as a way to create an alias so that we didn't need to argue when different distributions made different decisions about /bin vs /usr/bin. It was not presented as a change to common interface paths like /bin/sh. This request is new, and given the politics, is something I find highly problematic. It is not abusing maintainers to ask them to respect long-standing interfaces like the location of /bin/sh. As Simon has pointed out, in a number of cases it is still actually RC because it can break builds. It is not abusive to ask maintainers to fix issues that prevent their packages from building. We make mistakes. It is not abusive to get the severity of a bug wrong or even to disagree with the severity of a bug. I am sympathetic to the idea that after buildds are updated, we we might want to reduce the severity of not using longstanding interface paths, and in some cases not even treat it as a bug. I reject the idea that /usr/bin/sh should be preferred over /bin/sh or even the idea that it should be equally preferred. I am open to the idea that we may not care to record that as a bug or spend the time fixing it. However, the tone and approach in this discussion does not encourage me to participate. If the current tone continues, I will use up the energy I have for working toward a compromise and simply stand behind my support of longstanding practice and support of portable interfaces. --Sam signature.asc Description: PGP signature
Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
On Wed, 2023-09-06 at 16:51 -0600, Sam Hartman wrote: > > > > > > "Luca" == Luca Boccassi writes: > Luca> /bin/sh is not universally compatible with non-Linux OSes. > > I claim it is more compatible. Why should that matter to Debian? Should Debian invest resources to make upstream software more portable to systems like Mac OS X? For me this seems out of scope for Debian. > Luca> Also I thought that policy should not be used to beat other > Luca> developers (it is because of this) and it should reflect > the > Luca> common practices adopted in Debian (it does not because of > Luca> this). Is that no longer the case? > > I'd consider it a non-RC bug if someone were manually writing > #!/usr/bin/sh > > We can debate about normal vs minor. Should we require Debian maintainers to patch this in software packaged for Debian? Why? I think it is fine to wait with this until buildds use merged-/usr for builds, but I don't see any reason to spend resources on this after that. Ansgar
Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
On Thu, 7 Sept 2023 at 09:41, Simon McVittie wrote: > > On Wed, 06 Sep 2023 at 16:51:10 -0600, Sam Hartman wrote: > > I'd consider it a non-RC bug if someone were manually writing > > #!/usr/bin/sh > > As long as our official buildds are non-merged-/usr, I would consider use > of that path in scripts that get run at package build time to be > potentially RC, in fact. I have tried to arrange for our official buildds > for suites >= trixie to become merged-/usr (#1050868, #1025708) but that > work is awaiting review. I am aware, and we are targeting 12.2 for that (and I hope we manage to hit the target). On the other hand, changing Lintian takes several months usually. Policy is even slower, and moves at glacier speed, so when I opened this yesterday I did so with the intention that by starting now, we might get a chance to have it sorted in time for Debian 14 (Forky is it?). Further delaying means even that target might be missed. Fixing the current loophole in usr-is-merged that is being used to abuse maintainers is imperative and high priority, so I hope we will sort out the buildd situation within a month if all goes well, and in a couple of months at worst. I fully expect this to happen before anybody looks at the Lintian MR, and long long before this policy change has a chance to go anywhere. > Until that change lands, or our official buildds for > trixie/sid/experimental become merged-/usr some other way, encouraging > maintainers to use interpreters that were historically in /bin via their > /usr/bin paths is actively harmful and we should not do it. Please don't > make the transition to merged-/usr-everywhere more contentious than it > already is. The intention here was not to encourage to change or switch, but to stop bug reporters from using policy to beat maintainers. Given I am told these sections are not normative, I am perfectly fine with rewording in any way that achieves that objective - ie, mentioning both paths, or none, for example. > > > "Luca" == Luca Boccassi writes: > > Luca> /bin/sh is not universally compatible with non-Linux OSes. > > > > I claim it is more compatible. > > Hard-coding "#!/bin/sh" is not compatible with every operating system, > but it's much more widely compatible than hard-coding "#!/usr/bin/sh", > particularly for scripts that are meant to be backportable to older > versions of LTS distributions (notably Debian!). Even NixOS, an operating > system that is notable for the extent to which it intentionally avoids > using FHS paths, has made the pragmatic choice to provide /bin/sh. > > I would not want to encourage Debian maintainers to use > "#!/usr/bin/env sh" for shell scripts, even though it's more broadly > portable than assuming that /bin/sh is a POSIX shell. > This is analogous to how we encourage (and sometimes require) Debian > maintainers to use "#!/usr/bin/perl" and "#!/usr/bin/python3" in > preference to indirection via "#!/usr/bin/env perl" or similar, even > if their upstream uses "#!/usr/bin/env perl" for wider portability. As above, the intention here is not to encourage one way or the other, but to take the hammer away. These wordings are not normative anyway, but they are used as if they were, and I think that's something we want to fix regardless.
Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
On Thu, 07 Sep 2023 at 00:50:54 +0100, Luca Boccassi wrote: > On Thu, 7 Sept 2023 at 00:45, Sam Hartman wrote: > > I.E. in the cases you adjust, I think it is already > > not a bug, and it would be inappropriate to use existing policy language > > to complain about which interpreter path people use. > > In practice though they are used as normative, for example Lintian > raises an error (not even a warning, an actual error). In general, if Lintian is diagnosing paths as an error when they are allowed by Policy (therefore not a serious bug) and work in practice on all relevant systems (therefore not a grave bug), then the place to change that would be Lintian, not Policy. However, because our official buildds for trixie/sid/experimental are still using the transitional non-merged-/usr layout at the time of writing, it is still valuable for Lintian to care about whether interpreters are specified with their /usr path or not: so I think Lintian is still entirely correct to encourage #!/bin/sh over #!/usr/bin/sh, and so on. Also, the same version of Lintian is used to check packages targeting any supported suite (including bookworm-backports, bookworm, and even bullseye), and Lintian does not always have enough information to know which suite is being targeted (consider UNRELEASED packages). bookworm buildds are likely to remain non-merged-/usr indefinitely, to ensure we have a working upgrade path from bullseye, so using the non-traditional path to an interpreter (for example /usr/bin/sh or /bin/perl) is in practice still RC for bookworm if the script might be invoked during build. So I think it would be premature to change Lintian for this, even after our official buildds for suites >= trixie become merged-/usr. Using the non-traditional path to an interpreter is also *definitely* RC for *bullseye*, because bullseye systems that were upgraded from a sufficiently old release are non-merged-/usr unless the user has specifically taken steps to merge it. I think it would be appropriate to change Lintian to accept /bin/sh and /usr/bin/sh as interchangeable after trixie is released, at which point bullseye-backports will have been discontinued, bullseye will be LTS-maintained, and bookworm-backports will only be receiving targeted stable-release-suitable changes which are unlikely to regress the choice of interpreter path. smcv
Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters
On Wed, 06 Sep 2023 at 16:51:10 -0600, Sam Hartman wrote: > I'd consider it a non-RC bug if someone were manually writing > #!/usr/bin/sh As long as our official buildds are non-merged-/usr, I would consider use of that path in scripts that get run at package build time to be potentially RC, in fact. I have tried to arrange for our official buildds for suites >= trixie to become merged-/usr (#1050868, #1025708) but that work is awaiting review. Until that change lands, or our official buildds for trixie/sid/experimental become merged-/usr some other way, encouraging maintainers to use interpreters that were historically in /bin via their /usr/bin paths is actively harmful and we should not do it. Please don't make the transition to merged-/usr-everywhere more contentious than it already is. > > "Luca" == Luca Boccassi writes: > Luca> /bin/sh is not universally compatible with non-Linux OSes. > > I claim it is more compatible. Hard-coding "#!/bin/sh" is not compatible with every operating system, but it's much more widely compatible than hard-coding "#!/usr/bin/sh", particularly for scripts that are meant to be backportable to older versions of LTS distributions (notably Debian!). Even NixOS, an operating system that is notable for the extent to which it intentionally avoids using FHS paths, has made the pragmatic choice to provide /bin/sh. I would not want to encourage Debian maintainers to use "#!/usr/bin/env sh" for shell scripts, even though it's more broadly portable than assuming that /bin/sh is a POSIX shell. This is analogous to how we encourage (and sometimes require) Debian maintainers to use "#!/usr/bin/perl" and "#!/usr/bin/python3" in preference to indirection via "#!/usr/bin/env perl" or similar, even if their upstream uses "#!/usr/bin/env perl" for wider portability. We certainly shouldn't require shell scripts in Debian to go through the same contortions as Autoconf-generated configure scripts to find a POSIX shell (see `info autoconf 'Portable shell'`) because that's unnecessary runtime and maintenance overhead on any GNU/Linux system, any of the non-Linux ports that Debian has attempted to support in the past, and hopefully any free or proprietary Unix that is still relevant to Free Software authors this decade. smcv
