Re: Going ahead with non-free-firmware

[Hideki Yamane]

Hi,

On Sun, 24 Feb 2019 16:44:39 +0100
Ansgar  wrote:
> Sadly no; I think some later discussion made me doubt that there was
> indeed consensus about having non-free-firmware (and only that and not
> non-free-doc, non-free-drivers, non-free-*).  Nor about how it would
> work.
> 
> I don't think we should add a new component without having that
> (component meaning main, contrib, non-free, non-free-firmware here).
> They are not nice to handle on the archive side.

 Thanks for your reply. Well, then is there any proposal to improve
 setting non-free firmware in installer now?


-- 
Hideki Yamane 

Bug#923091: base-installer: Allow installing w/o the broken merged-usr-via-symlinks

[Holger Wansing]

Hi,

Cyril Brulebois  wrote:
> 
> If that were to be implemented: the question must not be asked in a
> normal installation, I think it shouldn't be translatable (at least at
> first) so that it doesn't put more burden on l10n people (coordinator
> and translators) for an expert-only option. Cc-ing Holger for input.

I'm unable to completely follow this whole 'merged-usr' thing, but to me
personally it seems like a somewhat controversial issue, which may see some
more discussion and changings.

So I tend to follow Kibi's proposal to keep the strings untranslateable for 
now, to allow everything to settle down, and make them translatable within 
the Buster+1 development cycle.

Filing a separate bugreport for this untranslatable -> translatable change
will be the best way, to make sure we don't forget about this.


So long
Holger


-- 
Holger Wansing 
PGP-Fingerprint: 496A C6E8 1442 4B34 8508  3529 59F1 87CA 156E B076

Bug#923091: base-installer: Allow installing w/o the broken merged-usr-via-symlinks

[Cyril Brulebois]

Hi,

Guillem Jover  (2019-02-24):
> The current base-installer uses the default debootstrap settings
> which end up unconditionally installing systems with the
> merged-usr-via-symlinks deployment method which is broken by design,
> please see:
> 
>   

I won't be commenting on the brokenness you claim in this bug report,
and in the mail you linked to, and in the subthread; that doesn't mean
I agree with your assessment.

Instead, I'll concentrate on the actual request for a change in
bootstrap-base below:

> I'm aware the original request to change the debootstrap default got
> unfortunately moved to the tech-ctte. :(
> 
> But regardless of that outcome I'd like to request to have a way to
> install using debootstrap's --no-merged-usr option, to not have to
> install from stretch and then upgrade to buster, or having to drop into
> a shell and do manual stuff from within the installer.

I'm not sure what costs (initial addition then maintenance) an
expert-only option would have, but I'm naively expect them to be rather
low? It seems to me that this would suit your needs and also make that
configurable through preseeding?

If that were to be implemented: the question must not be asked in a
normal installation, I think it shouldn't be translatable (at least at
first) so that it doesn't put more burden on l10n people (coordinator
and translators) for an expert-only option. Cc-ing Holger for input.

> In addition it would be also nice if that option was passed whenever
> /usr is not on its own partition, because then the properties from
> the merged-/usr concept are not relevant anymore, but we get all the
> downsides of the broken deployment method.

That shouldn't be tracked in this bug report as that seems quite
orthogonal (base-installer runs in d-i, not on installed systems)?

> If this was to be applied for buster, I'd be happy to provide patches.

Not promising anything, but I think I should be able to test/review such
patches once I'm done with updating the test suite.

> Otherwise I guess I might need to end up looking to generate
> alternative netinst somewhere else or something. :(

That won't be a factor regarding the inclusion of possible patches into
d-i, for buster or afterward.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Bug#914897: tech-ctte: Should debootstrap disable merged /usr by default?

[Didier 'OdyX' Raboud]

Le jeudi, 21 février 2019, 15.28:23 h CET Ian Jackson a écrit :
> Back in the wider world, of course many people build packages on
> Debian systems for installation `somewhere else'.  I have done it
> myself and probably many of the people reading this message have too.
> 
> What is implicitly going on here is that it is mostly-tacitly[2] being
> assumed (or declared) that `I built this .deb on my laptop[1] and gave
> it to my friend' is wrongheaded.

I don't think it is wrongheaded, but I do think that assuming that it is meant 
to work consistently under any circumstances, is.  There are _so many_ 
circumstances that make this exercise error-prone:

"I built this amd64 .deb and gave it to my friend for use on their RasbperryPi 
arm64 host"

"I built this .deb on my laptop on which I had installed a /usr/local/bin/
grep"

"I built this .deb an gave it to my friend who runs CentOS"

etc
 
> I don't think doing that is wrongheaded.  Certainly it is extremely
> common; we don't have any public pronouncements saying it is somehow
> wrong; and we have had little discussion where we as a project decided
> that this was now a use case which we feel free to just break.

Frankly, while it's not _broken_ per se, it is and has always really been 
fragile.  True, merged-/usr arguably worsens _one_ aspect of building packages 
on one's host, but in ways that are really easy to detect, and warn for.

> Personally I think that this is a very important thing to keep
> working.  It is the very core of users' collective software freedom.

You seem to be conflating "building on one's host" with "outside of any 
chroot", and equating this with "users' collective software freedom" is really 
making your point a disservice.  Being able to improve, build and share binary 
artifacts of free software is _vital_, and the whole point of building 
software distributions in the first place, but insisting that these rights are 
only "truly" exercised when builds are done outside of chroots is 
disingenuine. .deb packages already have to be built using certain tools, so 
we do set the limit (of what minimal set of tools are mandatory) somewhere, 
and my point is that this limit might not be at the right place. I'd be 
totally OK with saying "the only supported way to build .deb packages from 
buster on is using by pdebuild; however, you _can_ use dpkg-buildpackage 
alone, but be aware (and you'll get warnings) that this can lead to building 
.deb packages with undesireable properties."  The second half of the sentence 
is already true, and has always been; we probably just failed at communicating 
it sufficiently clearly.

> And that software freedom needs to be easy to exercise in practice.
> Despite a lot of excellent tooling, chroots are still not entirely
> trivial to set up and maintain.

Then that's maybe what we should be fixing.  

> I would invite the TC to read this manpage, which is trying to explain
> to a Debian user how to change the programs on their own computer
>   https://manpages.debian.org/stretch-backports/dgit/dgit-user.7.en.html
> and then consider how much even worse it would be if we had to write
> about chroots too.

Perhaps dpkg-buildpackage should be grown to build in a chroot by default? Or 
get an option to do that?

Really, I think we should stop considering that .debs built outside of 
"controlled environments" are more than temporary software transports. Our 
"standard" package building tools should build in such environments by 
default.

> To TC members who are minded to uphold the current approach, I would
> ask: can you please explicitly state your opinion on this ?  That is:
> 
>   Is it OK for a Debian user to build .deb on their laptop and give it
>   to their friend ?

Yes; provided that said .deb is built in a sane (sanitized / controlled) 
environment.  As you're talking about non-chroot builds; it is OK, provided 
that the tools warn about that. In that area, I think the "Build-Tainted-By" 
are steps in the right direction.

>   If it is OK, how will we make sure that it does not pointlessly break ?

As it is already broken in many ways, your question is biased. But I think the 
good way is to normalize "proper builds", by making sure our standard tools 
"do the right thing" by default.

> I readily admit that there are many ways that this can produce a
> result significantly different to the official Debian package,
> particularly because the package build system may configure itself
> differently in the the presence of unexpected.  The resulting package
> is probably not going to be suitable for wide distribution.
> 
> But those kind of problems are (a) not serious in practice
> (b) generally have obvious symptoms and (c) are easily worked around
> by various means.  Working around a usrmerge-generated failure is
> difficult; it usually involves doing serious violence to the upstream
> build system, or perhaps horrific rules file bodges.

Given a Build-Tainted-By flag in the binary artifact

RFC: Make dh_makeshlibs auto-detect udebs in common cases

[Niels Thykier]

Hi,

(Please CC me on replies per mail).

I am seeing feedback on a PoC patch for dh_makeshlibs to have it
auto-detect udebs in the common case, where the udeb has the same name
as the deb followed by the suffix "-udeb".  AFAICT, this will make
--add-udeb obsolete in many cases as many udebs follow the previous
mentioned naming convention.

The patch is available at [SALSA].  Please let me know what you think
about it.

Thanks,
~Niels

[SALSA]: https://salsa.debian.org/debian/debhelper/merge_requests/21




signature.asc
Description: OpenPGP digital signature

Re: Bug#914897: tech-ctte: Should debootstrap disable merged /usr by default?

[Didier 'OdyX' Raboud]

Le mardi, 19 février 2019, 01.59:18 h CET Steve McIntyre a écrit :
> While I'm not necessarily disagreeing with the overall point(s) here,
> some of the points in this list of arguments seem dubious at
> best. Maybe you could expand?

Sure. Sorry for publishing a new ballot before answering.

> >* booting with `/` only is not systematically tested in Debian anymore;
> 
> I'm assuming you mean "/ without /usr" here?

Yes. Clarified this point by merging it with the "separate / and /usr" one 
above.

> >* the packaging infrastructure to install files outside of `/usr` is not
> >  standard and represents technical debt:
> 
> I have no idea what you're suggesting here.

Same here (clarified in the ballot). What I'm trying to say here is that 
installing (e.g.) libs to /lib instead of /usr/lib is usually a deviation from 
standard software building (either in upstream packaging, or in debian/rules), 
that needs to be maintained on the long-term.

Cheers,
OdyX

signature.asc
Description: This is a digitally signed message part.

Re: Going ahead with non-free-firmware

[Ansgar]

Hi,

Hideki Yamane  writes:
>  Is there any progress about non-free-firmware section?

Sadly no; I think some later discussion made me doubt that there was
indeed consensus about having non-free-firmware (and only that and not
non-free-doc, non-free-drivers, non-free-*).  Nor about how it would
work.

I don't think we should add a new component without having that
(component meaning main, contrib, non-free, non-free-firmware here).
They are not nice to handle on the archive side.

Ansgar

>> Hi,
>> 
>> I think there was consensus to introduce the non-free-firmware section
>> and move the non-free firmware blobs there.  I'm wondering what we need
>> to do next?
>> 
>> Besides the ftp team setting the new section up, I expect the installer
>> would need changes to enable it instead of non-free when non-free
>> firmware is required; maybe it still needs to ask for non-free as well
>> for other reasons?  Other teams might also need to add the new section,
>> e.g. the release team, packages.d.o, ...  I expect the list to be
>> hard-coded in quite a few places.
>> 
>> Then the release notes need to document that "non-free-firmware" might
>> have to be added to sources.list.
>> 
>> Finally we need to identify the packages that should move there.  I
>> guess all non-free packages named "firmware-*" would be a good match.
>> 
>> Ansgar

Re: Going ahead with non-free-firmware

[Hideki Yamane]

Hi,

 Is there any progress about non-free-firmware section?


> Hi,
> 
> I think there was consensus to introduce the non-free-firmware section
> and move the non-free firmware blobs there.  I'm wondering what we need
> to do next?
> 
> Besides the ftp team setting the new section up, I expect the installer
> would need changes to enable it instead of non-free when non-free
> firmware is required; maybe it still needs to ask for non-free as well
> for other reasons?  Other teams might also need to add the new section,
> e.g. the release team, packages.d.o, ...  I expect the list to be
> hard-coded in quite a few places.
> 
> Then the release notes need to document that "non-free-firmware" might
> have to be added to sources.list.
> 
> Finally we need to identify the packages that should move there.  I
> guess all non-free packages named "firmware-*" would be a good match.
> 
> Ansgar

-- 
Regards,

 Hideki Yamane henrich @ debian.org/iijmio-mail.jp

Re: [rb-general] debian-installer: Ensure build is reproducible regardless of the user's umask(2)

[Chris Lamb]

Dear all,

> > This seems to have broken all daily builds.
[…]
> They seemed innocuous, but obvisouly this is another important lesson to
> always test.

Apologies on my part for this breakage although I do hope my pings
on this issue were interpreted as requests along the lines of "is
there further testing or things that I might be missing?" rather
than an explicit sign-off that they are perfect and simply need
merging. :)

Naturally, I was doing some testing with calls to "make build_hd-
media" and friends but clearly I was missing the build of
src:debian-installer itself — noted for any future change.

> >   
> > https://salsa.debian.org/installer-team/debian-installer/commit/d0868ed0bfd3d31a7fc5c3eaf6509d58e74cc2b1
> 
> Thanks for cleaning up the mess, Samuel!

Indeed, thank you for this change. Alas, I believe it introduces a
reproducibility regression but I will follow-up there.


Best wishes,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org 🍥 chris-lamb.co.uk
   `-