Bug#914897: affects private debs too
Andreas Henriksson wrote:
> On Wed, Nov 28, 2018 at 03:14:20PM +, Ian Jackson wrote:
> > Julien Cristau writes ("Re: Bug#914897: debootstrap, buster: Please
> > disabled merged /usr by default"):
> [...]
> > > I'd suggest that this should be fixed by not shipping any packages that
> > > aren't built on buildds.
> >
> > It would be quite a radical departure for Debian to no longer support
> > "I built this package for my mate to install on their computer".
>
> For the case of locally built binaries, bringing any problem
> that usrmerge would hit to the light would be preferable.
Any checks you can do may test only packages that reached the Debian
archive. We can discipline DDs, be it by requiring source-only, or by
catching misbuilt packages, but we can't do anything for local packages.
And these in a good part are not based on Debian sources.
I for one use a .deb package to distribute my .bashrc to machines under my
control. Joe from a derivative named Debuntituan provides an
uber-proprietary-drivers package to his users. Any PPA. A company-wide
repo. Etc, etc.
We'd need to have dpkg-dev Conflict: usrmerge, and even that won't catch
systems installed with recent versions of debootstrap.
Thus: sorry but there is no way we can possibly support usrmerged and
non-usrmerged systems at the same time. Usrmerge is not viable without a
flag day.
So it's up to you to decide:
* should there _be_ a flag day?
* should that flag day happen before Buster?
If the answer to the second question is "no", the debootstrap change
needs to be at least temporarily reverted.
Meow!
--
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Ivan was a wordly man: born in St. Petersburg, raised in
⢿⡄⠘⠷⠚⠋⠀ Petrograd, lived most of his life in Leningrad, then returned
⠈⠳⣄ to the city of his birth to die.
Bug#914897: affects private debs too
On 29/11/2018 17:13, Adam Borowski wrote:
> Andreas Henriksson wrote:
>> On Wed, Nov 28, 2018 at 03:14:20PM +, Ian Jackson wrote:
>>> Julien Cristau writes ("Re: Bug#914897: debootstrap, buster: Please
>>> disabled merged /usr by default"):
>> [...]
I'd suggest that this should be fixed by not shipping any packages that
aren't built on buildds.
>>>
>>> It would be quite a radical departure for Debian to no longer support
>>> "I built this package for my mate to install on their computer".
>>
>> For the case of locally built binaries, bringing any problem
>> that usrmerge would hit to the light would be preferable.
>
> Any checks you can do may test only packages that reached the Debian
> archive. We can discipline DDs, be it by requiring source-only, or by
> catching misbuilt packages, but we can't do anything for local packages.
>
> And these in a good part are not based on Debian sources.
>
> I for one use a .deb package to distribute my .bashrc to machines under my
> control. Joe from a derivative named Debuntituan provides an
> uber-proprietary-drivers package to his users. Any PPA. A company-wide
> repo. Etc, etc.
>
> We'd need to have dpkg-dev Conflict: usrmerge, and even that won't catch
> systems installed with recent versions of debootstrap.
>
> Thus: sorry but there is no way we can possibly support usrmerged and
> non-usrmerged systems at the same time. Usrmerge is not viable without a
> flag day.
>
>
> So it's up to you to decide:
> * should there _be_ a flag day?
> * should that flag day happen before Buster?
>
> If the answer to the second question is "no", the debootstrap change
> needs to be at least temporarily reverted.
With my release team hat on: if we have a flag day to switch to merged-/usr,
that should wait for after buster, and it should happen in the early stages of a
release cycle, not right before the freeze.
Also if you ask me, I think debootstrap should default to non-merged /usr for
the time being. That change is only causing trouble.
As Julien said, we should aim to have source-only uploads everywhere and forbit
or discard binaries from the uploads. But that doesn't prevent this change from
being reverted. Both things should happen, and they don't depend on each other.
Cheers,
Emilio
Bug#914897: affects private debs too
Adam Borowski writes:
> Andreas Henriksson wrote:
>> On Wed, Nov 28, 2018 at 03:14:20PM +, Ian Jackson wrote:
>> > Julien Cristau writes ("Re: Bug#914897: debootstrap, buster:
>> > Please disabled merged /usr by default"):
>> [...]
>> > > I'd suggest that this should be fixed by not shipping any packages that
>> > > aren't built on buildds.
>> >
>> > It would be quite a radical departure for Debian to no longer support
>> > "I built this package for my mate to install on their computer".
>>
>> For the case of locally built binaries, bringing any problem
>> that usrmerge would hit to the light would be preferable.
>
> Any checks you can do may test only packages that reached the Debian
> archive. We can discipline DDs, be it by requiring source-only, or by
> catching misbuilt packages, but we can't do anything for local packages.
>
> And these in a good part are not based on Debian sources.
>
> I for one use a .deb package to distribute my .bashrc to machines under my
> control. Joe from a derivative named Debuntituan provides an
> uber-proprietary-drivers package to his users. Any PPA. A company-wide
> repo. Etc, etc.
Debian is not responsible how third parties build their packages. We
don't even exclude /usr/local/bin from PATH when building packages...
(If you care about distributions not doing the same as Debian, one would
need to patch every package to build & work on both merged-/usr and
non-merged-/usr systems no matter what Debian does...)
> Thus: sorry but there is no way we can possibly support usrmerged and
> non-usrmerged systems at the same time. Usrmerge is not viable without a
> flag day.
Oh, it's possible. It makes life a bit harder than a flag day or clear
commitment to one setup, but so does supporting multiple init systems
(so the advantages of, for example, easier maintenance by having
declarative definitions is lost).
Regardless of debootstrap defaults or flag days, we could also consider
moving programs from /{s,}bin to /usr/{s,}bin with a compat symlink in
/{s,}bin; this does not need a flag day. That is useful either way as:
a) some people will use /usr/bin/${x} instead of /bin/${x} anyway (they
don't have to come from Debian), and b) it would also make supporting
merged-/usr and non-merged-/usr simpler as the programs would always be
available in both locations.
Some packages such as iptables have already done this ad-hoc.
I'm toying around with ideas for a dh_usrmove tool which would allow to
easily add this to existing packages. That would also allow to drop it
later in one go should one in the far future require the /bin ->
/usr/bin symlink.
Ansgar
Bug#914897: affects private debs too
On Thu, 29 Nov 2018 at 18:34:42 +0100, Ansgar Burchardt wrote:
> Regardless of debootstrap defaults or flag days, we could also consider
> moving programs from /{s,}bin to /usr/{s,}bin with a compat symlink in
> /{s,}bin
I'm not convinced this is a good idea: it seems like it causes as many
problems as it solves. It's effectively a gradual implementation of
making merged /usr mandatory, with a lot of moving parts (places where
it could go wrong) and a lot of packages and maintainers needing to be
involved, but without the simpler end result of *actually* merging /usr.
We've been migrating from non-multiarch to multiarch for more than 5
years and have still not got there, so I'm not confident that ad-hoc
migration from unmerged to merged /usr would go any quicker.
It can also cause the same failure modes with hard-coded executable paths
as merged /usr. Let's suppose we migrated grep from /bin to /usr/bin in
the way you suggest during the Debian 11 release cycle. If a package that
hard-codes the compile-time grep path (again, consider gzip, #914907)
is built with the new grep, it won't work correctly with the old grep
during a partial upgrade from Debian 10 to 11; but I don't see how it
would pick up a versioned Depends on the new grep without manual action
from the dependent package's maintainer, which isn't going to scale well.
> Debian is not responsible how third parties build their packages. We
> don't even exclude /usr/local/bin from PATH when building packages...
In particular, if you have a /usr/local/bin/grep, the same packages that
would hard-code /usr/bin/grep when built on a system with merged /usr
(for example gzip, #914907) will instead hard-code the path to your
/usr/local/bin/grep, resulting in them not working properly on systems
that don't have /usr/local/bin/grep. So perhaps bugs of the same class
as #914907 were already bugs in the source package before merged /usr
was ever suggested?
(To be fair, debuild(1) does sanitize PATH, but it's true that
dpkg-buildpackage doesn't.)
smcv
Bug#914897: affects private debs too
Simon McVittie writes:
> On Thu, 29 Nov 2018 at 18:34:42 +0100, Ansgar Burchardt wrote:
>> Regardless of debootstrap defaults or flag days, we could also consider
>> moving programs from /{s,}bin to /usr/{s,}bin with a compat symlink in
>> /{s,}bin
>
> I'm not convinced this is a good idea: it seems like it causes as many
> problems as it solves. It's effectively a gradual implementation of
> making merged /usr mandatory, with a lot of moving parts (places where
> it could go wrong) and a lot of packages and maintainers needing to be
> involved, but without the simpler end result of *actually* merging /usr.
Well, people don't like the simpler solution that someone else
proposed...
> We've been migrating from non-multiarch to multiarch for more than 5
> years and have still not got there, so I'm not confident that ad-hoc
> migration from unmerged to merged /usr would go any quicker.
Migrating /{s,}bin involves far fewer packages (only ~200 binary
packages). There is still /lib though.
> It can also cause the same failure modes with hard-coded executable paths
> as merged /usr. Let's suppose we migrated grep from /bin to /usr/bin in
> the way you suggest during the Debian 11 release cycle. If a package that
> hard-codes the compile-time grep path (again, consider gzip, #914907)
> is built with the new grep, it won't work correctly with the old grep
> during a partial upgrade from Debian 10 to 11; but I don't see how it
> would pick up a versioned Depends on the new grep without manual action
> from the dependent package's maintainer, which isn't going to scale well.
You will always have this problem with partial upgrades unless *every*
package depends on a package that would ensure that the system has all
binaries previously in /bin also in /usr/bin.
In particular, if we want to treat this as a (release critical) bug,
iptables or any other package ever moving from /bin to /usr/bin (or the
other way) will always be buggy, regardless of any merged-/usr. Same
for any binary ever moving between .../bin and .../sbin.
Ansgar
Bug#914897: affects private debs too
On Nov 29, Adam Borowski wrote: > Thus: sorry but there is no way we can possibly support usrmerged and > non-usrmerged systems at the same time. Usrmerge is not viable without a > flag day. We have being doing exactly this opt-in (the usrmerge package) for over three years and opt-out (the debootstrap default) for six months with only minor problems. With growing adoptions we have found and addressed the bugs, and currently the only significant issue still open is #913883 (iptables doing diversions, which I am sure can be fixed). How much experience do you have in using merged-/usr systems? -- ciao, Marco signature.asc Description: PGP signature
