Re: Removing more packages from unstable
Noah wrote: >On Tue, Aug 20, 2024 at 11:09:51AM +0200, Bastian Venthur wrote: >> I think popcon does give a good approximation of how much percent of people >> installed a certain package even if not everyone uses it, don't you agree? > >No, definitely not. There are hundreds of thousands of Debian systems >running in various cloud environments, and I'd be surprised if any of >them have ever submitted popcon data. > >> Last time I installed Debian it was still enabled by default. > >Oh? I don't think it has ever been enabled by default. It hasn't been AFAIK, no. d-i always asks, with the default being "no". -- Steve McIntyre, Cambridge, UK.st...@einval.com Can't keep my eyes from the circling sky, Tongue-tied & twisted, Just an earth-bound misfit, I...
Re: what about Netplan?
Luca Boccassi wrote: > >Networking is not static, it constantly changes in the kernel, >sometimes in dramatic and incompatible ways. Sorry, but no. Networking clearly is *not* changing that fast, in software terms. Many old tools still continue to work just fine after a decade or more. >A widely used, well maintained stack with large amounts of >contributors is fundamental for the default choice, because we have >to keep up, as the rest of the world will not sit and wait for us. > >Here's some stats from 'git shortlog --after="2021-12-31" -sn --all'. >In the last ~2.5 years, in netplan.io's github repo, there are only 2 >contributors with more than 100 commits, and 2 with more than 10, and >2 of them are Canonical employees: > > 569 Lukas Märdian > 310 Danilo Egea Gondolfo >39 Simon Chopin >38 Danilo Egêa Gondolfo >11 Robert Krátký > >Same stat, for the same period, for systemd: > > 6650 Yu Watanabe > 5415 Lennart Poettering > 2884 Luca Boccassi > 2772 Zbigniew JÄdrzejewski-Szmek ... >3 companies and one independent in the 4 digits, and too many to be >bothered to check between 10 and 999 commits. I understand what you're trying to say, but that's a disingenuous comparison. systemd is a massive (some would say *too* massive) project with fingers in many pies. How many of those people have touched *networking* bits? -- Steve McIntyre, Cambridge, UK.st...@einval.com Can't keep my eyes from the circling sky, Tongue-tied & twisted, Just an earth-bound misfit, I...
Re: Mini-DebConf in Cambridge, UK - October 10-13 2024
On Mon, Jun 24, 2024 at 12:18:56PM +0200, somebody *claiming* to be Luna Jernberg wrote: Just to be 100% clear, that mail didn't come from Luna's normal gmail account but was instead spoofed and sent via emkei.cz, a "free online fake mailer". It's now blocked from Debian lists. -- Steve McIntyre, Cambridge, UK.st...@einval.com "...In the UNIX world, people tend to interpret `non-technical user' as meaning someone who's only ever written one device driver." -- Daniel Pead
Re: Debian
You've written a lot of text here in a few mails, replying to yourself several times. This is not a positive pattern. On Fri, Apr 19, 2024 at 11:58:18AM +0200, José Luis González González wrote: >> There are similar issues with boa and dhttpd, and it seems Apache is going >> that way. > >nvi adds to the subversive ones, with bash, etc. What on earth do you mean by "subversive" here?? -- Steve McIntyre, Cambridge, UK.st...@einval.com "Arguing that you don't care about the right to privacy because you have nothing to hide is no different than saying you don't care about free speech because you have nothing to say." -- Edward Snowden
Re: finally end single-person maintainership
Bill Alombert wrote: >On Sun, Apr 07, 2024 at 04:04:18PM +0200, Andreas Tille wrote: >> Hi Wouter, >> >> Am Sun, Apr 07, 2024 at 03:31:43PM +0200 schrieb Wouter Verhelst: >> > [Feel free to quote any part of this email which I wrote outside of this >> > mailinglist] >> >> OK, moving the discussion to debian-devel where it should belong. >> >> > Debian packages need to be well maintained. In some cases, having >> > multiple maintainers on a package improves the resulting quality of >> > packages. But in some other cases, it does not; one example for this >> > second case is my package "logtool", which I'm going to upload to fix >> > #1066251 soon and for which by the simple act of doing that I will >> > double the amount of uploads it's seen in the past five years (and the >> > number of uploads in the past 10 can still be counted on the fingers of >> > a single hand). >> > >> > This is not because it's not well maintained; it's because the package >> > just *does not require* a lot of work to be kept up to date: upstream >> > has not been active for over 20 years, but it still performs the job it >> > was designed to do, as it was designed to, and I see no need to have it >> > removed from the archive. >> >> What is your opinion about pushing logtool to Salsa? > >Not speaking for logtool obviously, but maintaining simple packages on salsa is >just useless bureaucracy. So that's OK for *you* only in this case. Now consioder for the project as a whole. Every package that differs from the norm is more effort for anybody else to maintain/bugfix/NMU/whatever. We have a history of this (in so many ways), and it's a significant drain. Please consider the bigger picture. -- Steve McIntyre, Cambridge, UK.st...@einval.com Can't keep my eyes from the circling sky, Tongue-tied & twisted, Just an earth-bound misfit, I...
Re: Reaction to potential PGP schism
Enrico Zini wrote: > >I maintain critical code that calls out to gnupg, in part because at the >time I wrote it that was the only thing available, and in part because >I'm supposed to offer the broadest possible compatibility with what >other people in Debian are using, so if everyone else seems to use >gnupg, gnupg is the first thing I would consider. > >I hated and still hate every single moment I spent having to interface >with gnupg. The protocol to interact with it is custom, hydiosincratic, >poorly documented, and very hard to speak correctly. When in the end I >managed to make things work, I was always left with the feeling that >there would still be a corner case that I missed, or that will be >introduced in a future gnupg release, waiting to become a security issue >in our infrastructure, despite having asked for peer review from >appropriate people in Debian. > >New releases make things harder rather than easier. Now gnupg is a >mini-ecosystem of security-critical daemons that need to be brought up >and killed, that may time out or run partly off sync with configuration, >which adds even more know-how to the amount require to survive as >downstream consumer of that one single "API". ^^^ 100% on all of that. Trying to drive gnupg with automation is *such* a hateful, error-prone, *inconsistent* experience. Trying to do (what should be) simple things is really hard and messy. Simply trying to identify keys reliably is harder than it should be. Semantics of command lines change from one version to the next. >I've been wanting for literally decades something with language >bindings, or with a protocol that is built on existing well-known >standards, outputting data that I can parse with an existing and tested >parser library, using I/O channels that I can manage using an existing >and tested communication library. Again 100% on this. Just like you, I've spent countless hours writing code to parse GPG output because of this mess. >I hate it every single time I need to use gnupg, but still I use it >because I understand it's what Debian has been expecting me to use, so I >add that requirement to the pile of historical quirks that geologically >accrete in our community, which make our barrier of entry so stylishly >high, and make us appear oh so fearfully smart. > > >> # What Can Debian Do About This? >> >> If you are implementing or maintaining an OpenPGP implementation in >> debian, please consider encouraging upsteam to add a sop frontend, and >> get it tested in the interop test suite! > >This. > >I don't know if it should be sop or a protocol or a standard, but I'd >like to see Debian clearly document its expectations on its crypto >requirements, and stand behind it. > >I personally believe that we should depend, for our core security, on an >interoperable standard with multiple implementations rather than a >project that follows the hydiosincracies of a single isolated upstream. > >Whatever we do, though, I want that to be official. As things stand I'll >keep suffering with gnupg until at a DebConf I'll have at least 5 people >look at me wide-eyed and say "are you still using THAT? Everyone moved >to THIS instead!" > >I'd like to ask for what mature OpenPGP implementations exist today, >pick one I feel I can confidently control, and then when somebody comes >and says "my gpg/$TOOL segfaults on your input", I want to be able to >point them at a documented decision and say "please report a bug to >$TOOL" instead of taking a week off to port everything again to gpg. > >Thank you for all the work you've done on this over the years! I've >appreciated it with great gratitude and a big hope that some day, thanks >to you and others like you, those >=5 people at a DebConf will really >look at me wide-eyed and show me a way out of the pit. *grin* I look forwards to it! -- Steve McIntyre, Cambridge, UK.st...@einval.com Can't keep my eyes from the circling sky, Tongue-tied & twisted, Just an earth-bound misfit, I...
Re: /usr-move: Do we support upgrades without apt?
On Sat, Dec 23, 2023 at 01:22:48PM +, Richard Lewis wrote: > >Perhaps release-notes should suggest to run dpkg --verify after a >dist-upgrade anyway - i assume it doesnt hurt to do so? That's a really good suggestion, yes! I don't know why nobody hasn't thought of this before. :-) >Happy to suggest and edit text for release notes: i would want to know: > >- how do i know if the system is fine? > > i was assuming that it would output nothing if everything is fine, > but that seems to be far from the case. I get huge amounts of output > that is very hard to interpret, i assume it's showing a 'c' for every > single changed conffile, and 'missing' where i deleted conffiles. > But why are some lines contain question marks? we would need a lot > of explanation here to make this useful for users (unfortunately, the > dpkg man page is very confusing about this option, and doesnt have > any of this, as far as i can understand) > >- if something went wrong: > a) how do i know? would there be an obvious error message? do i need to > check the exit status? > b) what would i do?: reinstall some packages? reinstall the whole system? > >(maybe this should be in a bug against release-notes) Maybe a wrapper script to just report likely problems would be a good plan. -- Steve McIntyre, Cambridge, UK.st...@einval.com We don't need no education. We don't need no thought control.
Re: RFC: advise against using Proton Mail for Debian work?
I wrote: >nil...@mailbox.org wrote: >> >>>2. The Proton Mail web client automatically encrypts email to anyone who >>>it has a key for. Usually, this would be a great thing, but it means >>>that emailing 1234 at bugs.debian.org while CCing >>>uploader_since_this_is_an_rc_...@debian.org will encrypt the email that >>>is sent to the BTSe...which has the effect of making Debian development >>>veiled in plain sight rather than "in the open". >> >>Does it not encrypt email per-sender? > >Ewww, I certainly hope so! So I've just set up a ProtonMail test account to verify. Mailing to myself, it already clearly knew about my PGP key (I've already had lots of encrypted mails from ProtonMail users), but sending to a different address that came through in plain text. So *that* doesn't seem to be a worry, but the concern about people sharing private keys is still very real... :-/ -- Steve McIntyre, Cambridge, UK.st...@einval.com Can't keep my eyes from the circling sky, Tongue-tied & twisted, Just an earth-bound misfit, I...
Re: RFC: advise against using Proton Mail for Debian work?
nil...@mailbox.org wrote: > >>2. The Proton Mail web client automatically encrypts email to anyone who >>it has a key for. Usually, this would be a great thing, but it means >>that emailing 1234 at bugs.debian.org while CCing >>uploader_since_this_is_an_rc_...@debian.org will encrypt the email that >>is sent to the BTSe...which has the effect of making Debian development >>veiled in plain sight rather than "in the open". > >Does it not encrypt email per-sender? Ewww, I certainly hope so! Do we have any examples of encrypted mails hitting the BTS? -- Steve McIntyre, Cambridge, UK.st...@einval.com Can't keep my eyes from the circling sky, Tongue-tied & twisted, Just an earth-bound misfit, I...
Re: booststrapping /usr-merged systems (was: Re: DEP 17: Improve support for directory aliasing in dpkg)
Raphaël Hertzog wrote: > >In the same spirit, I'd like to throw an idea... could we decide that >base-files is the first package to be configured as part of the bootstrap >protocol and change base-files maintainer's scripts into statically linked >executables so that they can work even if we don't have the library loader >on the ABI-compliant path? What exactly do you mean here? You know that even a statically linked executable needs an interpreter defined in the ELF header? -- Steve McIntyre, Cambridge, UK.st...@einval.com < sladen> I actually stayed in a hotel and arrived to find a post-it note stuck to the mini-bar saying "Paul: This fridge and fittings are the correct way around and do not need altering"
Re: i386 in the future (was Re: 64-bit time_t transition for 32-bit archs: a proposal)
Steve Langasek wrote: > >On Fri, May 19, 2023 at 12:42:32PM +0100, Steve McIntyre wrote: >> >If the main reason is to support non-free binaries, at least to me >> >that does not seem like a very compelling reason. And people can >> >always use old chroots or similar I guess? > >> i386 is in a really awkward situation here, I think. Nobody is working >> on it explicitly any more (AFAICS?), but its history as by far the >> most common architecture means that: > >> * we still have a (very!) long tail of installations using it >> * there are *massively* more old binaries available for it, free, >>proprietary *and* locally-built > >FTR this includes wine, and 30 years of 32-bit Windows executables that >people want to be able to run, including games. (for which inaccurate times >are not going to be hugely important, in general.) And some of those games >are going to require e.g. library packages for 3d acceleration that are in >sync with kernel drivers (nvidia). This was ultimately what made "just use >an older version in a chroot/container" untenable for Ubuntu and led to >keeping i386 as a partial port. ACK! >So one may not think that support for legacy, proprietary programs is a >compelling reason to keep binary-compatibility on i386. But I counter that >unless you care about this, there's no reason to keep i386 as an >architecture *at all*. That's exactly my reasoning, yup! -- Steve McIntyre, Cambridge, UK.st...@einval.com < sladen> I actually stayed in a hotel and arrived to find a post-it note stuck to the mini-bar saying "Paul: This fridge and fittings are the correct way around and do not need altering"
Re: i386 in the future (was Re: 64-bit time_t transition for 32-bit archs: a proposal)
Andrew Cater wrote: >On Fri, May 19, 2023 at 03:03:40PM +0100, Steve McIntyre wrote: >> >> I had been thinking about doing similar for installer images too, but >> with other work going on too I think it got too late in the cycle to >> make that change. My plan is therefore to ship i386 installer images >> for bookworm as normal (including bookworm point releases going >> forwards), but to disable those builds for testing/trixie ~immediately >> after the release. > >I'd honestly suggest *just* publishing DVD1 for i386. > >Netinst requires internet access: DVD1 can be used to install a basic >system without this. Scrap *everything else* for i386 installation media. We've had this discussion before. I don't see the point in removing choice here at *really* short notice before bookworm, but still keeping a non-zero number of installer images for the architecture. It saves us very little effort and doesn't really gain us anything. This is why I'm talking *now* about dropping things for trixie. -- Steve McIntyre, Cambridge, UK.st...@einval.com < sladen> I actually stayed in a hotel and arrived to find a post-it note stuck to the mini-bar saying "Paul: This fridge and fittings are the correct way around and do not need altering"
Re: i386 in the future (was Re: 64-bit time_t transition for 32-bit archs: a proposal)
Colin Watson wrote:
>On Fri, May 19, 2023 at 09:19:35AM -0500, G. Branden Robinson wrote:
>> Well, maybe not a strong view, but a sense of vague unease--possibly an
>> ill-informed one. As someone who has used SIMH for "real" work[1], I
>> have to ask how someone would conduct an install to a 32-bit x86 machine
>> running under emulation, assuming no OS on the simulated machine.
>
>I occasionally use 32-bit x86 even today (mostly for not very good
>historical reasons, but nevertheless), and I do it by using a 32-bit
>container on a 64-bit x86 machine instead. It's much faster to run, and
>it doesn't depend on installer support. There are doubtless edge cases
>where you need a completely separate kernel, but they aren't really ones
>I run into.
ACK. For people needing/testing i386 stuff, even just a simple
debootstrap and {s,}chroot will cover the vast majority of
needs. That's how we've been building i386 software already for ages
in Debian already.
More complex things can be done if needed: loopback mount an image,
debootstrap, install a kernel, etc. I don't see this as something we
should be spending much effort on in the future.
--
Steve McIntyre, Cambridge, UK.st...@einval.com
< sladen> I actually stayed in a hotel and arrived to find a post-it
note stuck to the mini-bar saying "Paul: This fridge and
fittings are the correct way around and do not need altering"
Re: i386 in the future (was Re: 64-bit time_t transition for 32-bit archs: a proposal)
Luca Boccassi wrote: >On Fri, 19 May 2023 at 12:42, Steve McIntyre wrote: >> >> I'm planning on stopping publishing installer images for i386 >> soon. Why? We should be strongly encouraging users to move away from >> it as a main architecture. If they're still installing i386 on 64-bit >> hardware, then that's a horrible mistake. If they're still running >> i386 *hardware*, then they should be replacing that hardware with more >> modern, more capable, more *efficient* stuff. >> >> As and when we switch i386 to a secondary status like this (however we >> label it!), then I think we should *only* consider it as a >> compatibility layer for older software. People *could* just use old >> chroots or similar, but the need is likely to be around for a >> while. > >+1 for stopping publishing installers for i386, it has been mentioned >many times but it's always worth repeating: electricity costs to keep >running i386 hardware are already way higher than what it costs to buy >a cheap, low-power replacement like a raspberry pi, that also provides >better performance. Exactly. >Just to clarify, by 'soon' here, do you mean for Bookworm too, or >post-Bookworm? We've already switched off i386 *live* images for Bookworm. Honestly, we should probably have done that a while ago; IMHO they've not been useful in some time. I had been thinking about doing similar for installer images too, but with other work going on too I think it got too late in the cycle to make that change. My plan is therefore to ship i386 installer images for bookworm as normal (including bookworm point releases going forwards), but to disable those builds for testing/trixie ~immediately after the release. If people have strong opinions about that plan, let us know please. -- Steve McIntyre, Cambridge, UK.st...@einval.com < sladen> I actually stayed in a hotel and arrived to find a post-it note stuck to the mini-bar saying "Paul: This fridge and fittings are the correct way around and do not need altering"
i386 in the future (was Re: 64-bit time_t transition for 32-bit archs: a proposal)
Guillem Jover wrote: >On Thu, 2023-05-18 at 12:01:40 -0700, Steve Langasek wrote: ... >> > > * ⦠but NOT on i386. Because i386 as an architecture is primarily of >> > > interest for running legacy binaries which cannot be rebuilt against a >> > > new >> > > ABI, changing the ABI on i386 would be counterproductive, as mentioned >> > > in >> > > https://wiki.debian.org/ReleaseGoals/64bit-time. >> >> > Like Russ, I'm also dubious about this, and introduces a special case >> > and complexity that does not seem warranted TBH. If this was the case it >> > would seem to me it would disallow SOVERSION bumps for example, which we >> > have never been concerned with. >> >> I didn't see anything in Russ's email in this thread that implied he was >> dubious of this approach? He simply has a library he maintains for which he >> believes binary compatibility is uninteresting. > >Ah, indeed, just reread the specific paragraph now, sorry Russ! :) > >> FWIW in Ubuntu where we maintain i386 strictly as an architecture for >> compatibility with third-party binaries, we have 1034 source packages that >> build Arch: i386 debs. Not all of those source packages are shared >> libraries, but enough of them are that manually updating them to maintain >> ABI compatibility on i386 would be a substantial amount of work. In terms >> of overall complexity, I think the scales tip in favor of the toolchain not >> defaulting to _TIME_BITS=64 on i386. > >The problem with obsolete packages is also shared by all other arches, >and for those and for local packages the dependency system works for >the user and should let them know whether they can upgrade or they >would need to remove such packages. For other local FOSS packages >they might just be able to rebuild them. > >Excluding i386 from this transition seems to me will pretty much >sentence it, and would also make it rather hard to perform that >transition cleanly going forward if people want to keep it alive. And >while Debian might eventually remove it from its official ports, we >have multiple old ports that are still maintained and used. > >If the main reason is to support non-free binaries, at least to me >that does not seem like a very compelling reason. And people can >always use old chroots or similar I guess? i386 is in a really awkward situation here, I think. Nobody is working on it explicitly any more (AFAICS?), but its history as by far the most common architecture means that: * we still have a (very!) long tail of installations using it * there are *massively* more old binaries available for it, free, proprietary *and* locally-built Moving forwards, we need to make a call on what we want i386 for. I was hoping to wait until after bookworm is released to have the meat of that discussion, but... I'm planning on stopping publishing installer images for i386 soon. Why? We should be strongly encouraging users to move away from it as a main architecture. If they're still installing i386 on 64-bit hardware, then that's a horrible mistake. If they're still running i386 *hardware*, then they should be replacing that hardware with more modern, more capable, more *efficient* stuff. As and when we switch i386 to a secondary status like this (however we label it!), then I think we should *only* consider it as a compatibility layer for older software. People *could* just use old chroots or similar, but the need is likely to be around for a while. There's a tension here: I think it's important to keep the old ABI around for those old binaries, and I genuinely don't see a use case for a new incompatible ABI on a mostly-dead architecture that won't support those binaries. *But* I think we'll also need to keep the port going with security fixes - it's still likely to be quite common and we need to keep users safe. People are even likely to want to keep old software running beyond 2038, in which case I envisage clock hacks coming to keep things limping on. :-/ -- Steve McIntyre, Cambridge, UK.st...@einval.com < sladen> I actually stayed in a hotel and arrived to find a post-it note stuck to the mini-bar saying "Paul: This fridge and fittings are the correct way around and do not need altering"
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited)
Hey Johannes, On Mon, May 15, 2023 at 06:48:04AM +0200, Johannes Schauer Marin Rodrigues wrote: >Quoting Steve McIntyre (2023-05-15 02:54:02) >> >> Pointing at gentoo or nixos as examples of projects that have decided >> to break compatibility doesn't cut it, I'm afraid. They're well known >> for changing fundamental things around Linux and (basically) not caring about >> interoperability. That attitude is *not* Debian's. > >me and Luca have different ideas about how bootstrapping a Debian chroot should >look like and I don't want to make an argument *for* changing PT_INTERP here as >I think that keeping compatibility with others by following ABI is a good thing >and because I think (and hope -- but Helmut is doing that analysis right now) >that the debootstrap problem can be solved in a way I envision without changing >PT_INTERP. But what I do not understand about the argument against Luca's >proposal is: > >Obviously, with Luca's proposal, binaries from packages built with a different >dynamic linker path in them would not work on distributions without merged-/usr >symlinks. But if the property of stuff from Debian being able to run on >non-Debian non-merged-/usr systems is an important one, then why was it okay to >have merged-/usr as the default? Because with merged-/usr we already changed >the interface contract for a lot of things because now binaries and libraries >can also be found at other locations than on non-merged-/usr systems. A script >with a /usr/bin/bash shebang built on and for Debian will not work on a system >without the symlinks. Despite the massive upheavals of merged-/usr in *other* ways, it's actually a *minor* change as far as compatibility is concerned here. The runtime linker (aka interpreter) is responsible for resolving symbols and finding needed libraries. So long as *that* bit works OK, then ~everything else should follow OK. This is how it's possible to have things work across distros: binaries don't actually care where the libraries live, or whether they came from rpm or deb packaging, etc. The issue at hand here is that the interpreter path is the most basic thing that matters for this compatibility. Break this and *nothing* can work. >So did we not years ago decide, that the result of the "cross- and >inter-project discussion" is, that everybody is going merged-/usr and that's >why we need it too and that's why it is okay to build a system where binaries >and scripts built for it just may not run on those other systems that do not do >it? With merged-/usr we already *did* "change fundamental things around" for >reasons that are really not clear to me (but which i do not want to discuss >here) and as a result did not "care about interoperability" (with those who do >not also adopt it). In my own Debian work I so far only got extra work because >of merged-/usr and I do not see the benefits (yet) and I was hoping that >"changing fundamental things around Linux and (basically) not caring about >interoperability" was *not* Debian's attitude but alas here we are. > >So have we not already burned the bridges to the non-merged-/usr world? Why was >it okay back then to say "we can make this change because all other important >players are doing merged-/usr so we can/have to as well". And now in the >PT_INTERP discussion somehow we care again about those systems? I thought we >already had the "cross- and inter-project discussion" about merged-/usr and >because the result was "yes, go for it" we did it too. But if that is the case, >why do we now care for a subset of the interoperability problems caused by >merged-/usr for systems that don't have it? This change is absolutely *not* needed to make merged-/usr work; if anybody is claiming that it is, then they are not being 100% honest with us. All the other distros doing merged-/usr have done it without making this change, and it's also been working OK for us so far without this change. Breaking an agreed interface contract like this is axiomatically *wrong* and will hurt us and the rest of the Linux ecosystem. -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited)
I'm *trying* to assume good faith here, but I'm running out of energy to do so. On Mon, May 15, 2023 at 01:42:27AM +0100, Luca Boccassi wrote: >On Mon, 15 May 2023 at 01:14, Russ Allbery wrote: > >> Incidentally, that remains true even if we only do that in distribution >> packages. I certainly have copied binaries from a Debian package to other >> Linux systems before for various reasons and expected them to run. Sure, >> this might not work for other reasons outside of our control, but that's >> no reason to be gratuitously incompatible by breaking the ABI, >> particularly for what seem to be annoyances of our own creation with known >> workarounds. > >Thanks, that's the first actual real example mentioned so far. And >it's an interesting one: taking a $random Debian package and using it >on a completely different, non-Debian system. Is that a supported use >case? If so, does that mean that I can go ahead and raise a Severity: >serious bug on any package that doesn't work in such a way? Russ has described copying *binaries* out of packages and running them elsewhere. I've done that too, from time to time. This is one of the things made possible by the ABI contract being followed. You are the one proposing to break that contract, thereby *guaranteeing* this will fail on systems where otherwise it could work. I think the onus is on *you* to justify why this is a valid and useful thing to do. Your apparent lack of care for agreed standards here is horrifying. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited)
On Mon, May 15, 2023 at 12:24:15AM +0100, Luca Boccassi wrote: >On Sun, 14 May 2023 at 22:37, Josh Triplett wrote: > >> The x86-64 ABI is set. Feel free to make the case to the next >> architecture designer that their new ABI should have the dynamic linker >> in `/usr/lib`. That would *not* have the same downsides, as long as >> everyone agrees on a path. > >In practice it is not, though. There are other distributions that >change PT_INTERP for their own purposes, they've already been listed >in this thread. And I am still not hearing any concrete, factual use >case that would be impaired by such a change. I'm beginning to >seriously think there aren't any? Is that really the case? The ABI has been agreed and set down in documentation that *just about* everybody has been following since its inception. This includes the most basic set of definitions of what an x86-64 program must look like, including the interpreter path. If this path is changed, then *at the most basic level* we'd be making programs that are not valid by the ABI we've agreed to. This is an *external interface contract*, not something we should ever consider changing without significant cross- and inter-project discussion. Pointing at gentoo or nixos as examples of projects that have decided to break compatibility doesn't cut it, I'm afraid. They're well known for changing fundamental things around Linux and (basically) not caring about interoperability. That attitude is *not* Debian's. -- Steve McIntyre, Cambridge, UK.st...@einval.com "The problem with defending the purity of the English language is that English is about as pure as a cribhouse whore. We don't just borrow words; on occasion, English has pursued other languages down alleyways to beat them unconscious and rifle their pockets for new vocabulary." -- James D. Nicoll
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited)
On Fri, May 12, 2023 at 01:11:38PM +0100, Luca Boccassi wrote: >On Fri, 12 May 2023 at 12:08, Steve McIntyre wrote: >> >> On Fri, May 12, 2023 at 11:40:05AM +0100, Steve McIntyre wrote: >> >On Fri, May 12, 2023 at 10:49:32AM +0100, Luca Boccassi wrote: >> >>On Fri, 12 May 2023 at 09:40, Steve McIntyre wrote: >> >>> >> >>> On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote: >> >>> > >> >>> >The core issue as I see it is as follows: >> >>> > >> >>> >- Debian has decided to support only merged-/usr, including possibly >> >>> > moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2 >> >>> > as the interpreter in binaries. >> >>> >> >>> WTF? *Nobody* has been talking about breaking ABI like this, that I've >> >>> seen. The interpreter must *not* be changed willy-nilly. >> >> >> >>Nothing's happening 'willy-nilly'. We are discussing a bunch of >> >>seemingly crazy options, as in, "what would _actually_ explode if we >> >>do this or do that?", on this very d-devel thread. I posted a longer >> >>version here some days ago: >> >> >> >>https://lists.debian.org/debian-gcc/2023/05/msg00030.html >> > >> >Oh holy fuck. >> > >> >You're talking about changing ABI by doing this. That *is* utterly >> >crazy. No. >> >> People have asked me to expand on this further... >> >> I've been involved in defining ABI before, specifically for >> armhf. It's not a quick and easy process. It needs buy-in from all >> sides to make things work, and people *really* value the >> interoperability that it enables. >> >> The interpreter path is one of the most important parts of the ABI >> spec, the bit that makes binaries compatible between all the various >> stakeholders: compiler/tools people, distros, software vendors, >> etc. Lots of the rest of the details downstream of this can be >> changed, and people do this all the time - compare multilib to >> multi-arch for example. That all works fine *so long as* the runtime >> linker can be located and started OK. > >The loader is still available via the old path, so external/third >party/local/other software works unchanged. This should negatively >only affect our 1st party packages, when running on a non-merged >distro. So why the hell do you want to break this in the first place? Does a symlink in the "wrong" place offend you for some reason? For that you want to change a core assumption in *every single binary* in Debian? Believe me, I've been here in the past when we made changes in armhf to accommodate earlier mistakes. That was just for one architecture. What possible benefit do you see in this change? >And are _all_ our packages really 100% compatible with other distros >at all? Are they even supposed to be? > >For example, if I download efibootmgr from Bookworm on an Ubuntu Focal >machine, when I try to run it, it fails: > >root@focal:/tmp# wget >http://ftp.de.debian.org/debian/pool/main/e/efibootmgr/efibootmgr_17-2_amd64.deb >--2023-05-12 12:46:17-- >http://ftp.de.debian.org/debian/pool/main/e/efibootmgr/efibootmgr_17-2_amd64.deb >Resolving ftp.de.debian.org (ftp.de.debian.org)... 141.76.2.4 >Connecting to ftp.de.debian.org (ftp.de.debian.org)|141.76.2.4|:80... >connected. >HTTP request sent, awaiting response... 200 OK >Length: 27572 (27K) [application/vnd.debian.binary-package] >Saving to: 'efibootmgr_17-2_amd64.deb' > >efibootmgr_17-2_amd64.deb >100%[===>] 26.93K >--.-KB/sin 0.04s > >2023-05-12 12:46:17 (740 KB/s) - 'efibootmgr_17-2_amd64.deb' saved >[27572/27572] > >root@focal:/tmp# dpkg -x efibootmgr_17-2_amd64.deb ebm >root@focal:/tmp# ./ebm/bin/efibootmgr >./ebm/bin/efibootmgr: /lib/x86_64-linux-gnu/libc.so.6: version >`GLIBC_2.34' not found (required by ./ebm/bin/efibootmgr) > >Should I file a severity: serious bug against efibootmgr because it is >not interoperable? You're wilfully missing the point, and you know it. >The answer is obviously not, because it would be absurd to expect a >binary compiled against libraries from one distro to "just work" on an >entirely different distro. Glibc itself is not forward compatible, and >is allowed to add new symbols, that are not present in older versions, >and packages are allowed to depend on them. Aren't those also ABI >breakages? What about all the libraries that bump soname? What about &
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited)
On Fri, May 12, 2023 at 11:40:05AM +0100, Steve McIntyre wrote: >On Fri, May 12, 2023 at 10:49:32AM +0100, Luca Boccassi wrote: >>On Fri, 12 May 2023 at 09:40, Steve McIntyre wrote: >>> >>> On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote: >>> > >>> >The core issue as I see it is as follows: >>> > >>> >- Debian has decided to support only merged-/usr, including possibly >>> > moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2 >>> > as the interpreter in binaries. >>> >>> WTF? *Nobody* has been talking about breaking ABI like this, that I've >>> seen. The interpreter must *not* be changed willy-nilly. >> >>Nothing's happening 'willy-nilly'. We are discussing a bunch of >>seemingly crazy options, as in, "what would _actually_ explode if we >>do this or do that?", on this very d-devel thread. I posted a longer >>version here some days ago: >> >>https://lists.debian.org/debian-gcc/2023/05/msg00030.html > >Oh holy fuck. > >You're talking about changing ABI by doing this. That *is* utterly >crazy. No. People have asked me to expand on this further... I've been involved in defining ABI before, specifically for armhf. It's not a quick and easy process. It needs buy-in from all sides to make things work, and people *really* value the interoperability that it enables. The interpreter path is one of the most important parts of the ABI spec, the bit that makes binaries compatible between all the various stakeholders: compiler/tools people, distros, software vendors, etc. Lots of the rest of the details downstream of this can be changed, and people do this all the time - compare multilib to multi-arch for example. That all works fine *so long as* the runtime linker can be located and started OK. Changing the interpreter path would mean moving to a Debian-specific ABI, breaking that compatibility. Hand-waving that away with (and I quote): "The vast majority of distros today ship the loader in /usr/lib as /lib is just a symlink, so it would be interoperable." is appalling arrogance. No. You do *not* get to break ABI with that argument. The point of the ABI spec is that *everybody* follows it. You don't change it just because you think it'll make your life a little easier when bootstrapping a system. -- Steve McIntyre, Cambridge, UK.st...@einval.com Welcome my son, welcome to the machine.
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited)
On Fri, May 12, 2023 at 10:49:32AM +0100, Luca Boccassi wrote: >On Fri, 12 May 2023 at 09:40, Steve McIntyre wrote: >> >> On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote: >> > >> >The core issue as I see it is as follows: >> > >> >- Debian has decided to support only merged-/usr, including possibly >> > moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2 >> > as the interpreter in binaries. >> >> WTF? *Nobody* has been talking about breaking ABI like this, that I've >> seen. The interpreter must *not* be changed willy-nilly. > >Nothing's happening 'willy-nilly'. We are discussing a bunch of >seemingly crazy options, as in, "what would _actually_ explode if we >do this or do that?", on this very d-devel thread. I posted a longer >version here some days ago: > >https://lists.debian.org/debian-gcc/2023/05/msg00030.html Oh holy fuck. You're talking about changing ABI by doing this. That *is* utterly crazy. No. -- Steve McIntyre, Cambridge, UK.st...@einval.com "...In the UNIX world, people tend to interpret `non-technical user' as meaning someone who's only ever written one device driver." -- Daniel Pead
Re: Bug#1035904: dpkg currently warning about merged-usr systems (revisited)
On Fri, May 12, 2023 at 07:40:00AM +0200, Ansgar wrote: > >The core issue as I see it is as follows: > >- Debian has decided to support only merged-/usr, including possibly > moving /bin/sh to /usr/bin/sh or using /usr/lib*/ld-linux-x86-64.so.2 > as the interpreter in binaries. WTF? *Nobody* has been talking about breaking ABI like this, that I've seen. The interpreter must *not* be changed willy-nilly. -- Steve McIntyre, Cambridge, UK.st...@einval.com "I've only once written 'SQL is my bitch' in a comment. But that code is in use on a military site..." -- Simon Booth
Re: Re-enabling os-prober for live images?
[ Following up on the older discussion... ] t...@debian.org wrote: >kilobyte wrote: > >>At this point, I'd just enable os-prober unconditionally, and think of a > >Erm, *no*?! > >os-prober corrupts data when called (in virtualisation/emulation >guests, at the very least). > > >Steve wrote: > >>I'm also pondering tweaking things in d-i to re-enable os-prober if >>the system looks like it might have some other OS installed. Yes, I > >But what if the system has *both* other OSes installed *and* is >used as virtualisation host (when booting Debian) at the same time? > >Youâll still get data corruption of unrelated data (VM guests). >I consider that inacceptable, but apparently YMMV⦠And this debate is exactly the problem. There is no single good answer here, and two different sets of people will lose, depending on what we choose as a default: * (Potentially new) users install Debian as part of a dual-/multi- boot system and now (as os-prober is disabled by default) they don't get an option to boot the other OS(es) easily. I've seen situations like this before, and I understand that people worry here: "OMG what happened to the Windows system I still need?" * People with currently-running guests potentially end up with data corruption problems when updating grub on the host. What I've done now is: 1. Added debconf handling for enabling/disabling os-prober in /etc/default/grub to make it easier to handle things. 2. Made some *limited* tweaks to grub-installer, the d-i component that sets up GRUB. It already runs os-prober in a massively over-complicated way (see below). If *that* os-prober run detects other OSes installed, we will now use the new grub debconf setting to enable os-prober on the new system. If no other OS is detected during installation, we will disable os-prober there. Either way, the user will be prompted about os-prober *at a low priority* so that they can override things via preseed or answering the question in d-i expert mode. I think this is the best route forward, from the available options. If you're in the tiny set of people running Debian on a dual-boot *and* running guests while you update grub then you'll need to manage that on your own: disable os-prober and come up with your own method of adding extra boot options (or similar). /etc/grub.d is designed for this kind of thing if you need it. The current set of options here with grub and grub-installer are *huge*, and IMHO the code is getting impossible to follow or maintain. :-( *After bookworm*, I plan to take a machete to grub-installer and do some long-needed cleanup. 1. There's currently support in grub-installer for doing a *one-off* os-prober run and adding a semi-hard-coded list of other OSes found there, then not running os-prober in future on the installed system. This is going away; I'm not convinced that it's *ever* worked proparly and of course it doesn't deal with future changes to the system. 2. I'm planning on killing support for grub-legacy, which is *way* overdue. It's been dead upstream for over a decade already... For GRUB itself: 1. We have a *lot* of patches on top of upstream GRUB, which makes it hard to keep in step with security updates etc. I hope to find time to rationalise things, following on from work that Colin was doing earlier. 2. I'd like to simplify options more and reduce the number of packages we ship here too. Let's see... -- Steve McIntyre, Cambridge, UK.st...@einval.com < sladen> I actually stayed in a hotel and arrived to find a post-it note stuck to the mini-bar saying "Paul: This fridge and fittings are the correct way around and do not need altering"
Re: Reducing allowed Vcs for packaging?
Sean Whitton wrote: > >On Sat 04 Mar 2023 at 10:58PM +01, Joerg Jaspert wrote: > >> On 16792 March 1977, Adrian Bunk wrote: >> >>> for the contents of packages in the archive the ftp team requires that >>> everything is in the preferred form of modification. >> >> Yes. Of course. >> >> But git or svn or even sccs and rcs is NOT, in any way, preferred for of >> modification. Only one way of storage and handling some metadata. > >This is Debian's official position, but it's debateable. > >There is the preferred form for modification for an individual *file*, >and that probably doesn't include version control. But the preferred >form for modifying a *project* arguably does. I think you're *reaching* here. I know of quite a few projects where they consider their CI setup to be an intergral part of project development. Should we therefore declare that "preferred form of modification" could include all of the github or gitlab infrastructure? -- Steve McIntyre, Cambridge, UK.st...@einval.com < sladen> I actually stayed in a hotel and arrived to find a post-it note stuck to the mini-bar saying "Paul: This fridge and fittings are the correct way around and do not need altering"
Re: Starting the weekly live images for Bookworm building again
Hey again, On Tue, Jan 31, 2023 at 03:36:53PM +, Steve McIntyre wrote: >> >>As you can see, this affects many teams: >>* live-setup: a MR to generate all live images for Bookworm [A2] So, after some delay from me and some further delays from various Debian machines committing suicide [1], I've got bookworm live builds running again. \o/ I've taken Roland's updated patches and tweaked a little more in the setup.git and live-setup.git repos, and we now have live builds integrated. Changes I've added: * turned on source tarball generation using LB_SOURCE=true, and disable the external source build that we dod for the older live-wrapper builds * when building on casulana, warn about archive updates rather than restarting builds * don't attempt to build i386 live images any more, they're not useful * tweaked logging So, *builds* work fine but I've not *yet* tested actually booting/using one of these images in any way. I've just triggered a full build of "testing" live images now, please help test if you can once they're in place at [2] in a couple of hours from now. I don't yet know how close we are to having full non-free-firmware integration with the live images; I expect there might be some more work needed there yet, but I'd love to be proven wrong. :-) [1] "yay" for the long-standing tradition of services failing as we get close to a release: this time it was casulana and salsa... [2] https://get.debian.org/images/weekly-live-builds/ -- Steve McIntyre, Cambridge, UK.st...@einval.com "...In the UNIX world, people tend to interpret `non-technical user' as meaning someone who's only ever written one device driver." -- Daniel Pead
Re: Re-enabling os-prober for live images?
j...@debian.org wrote: > >Since the grub 2.06 upload, os-prober is now disabled by default. This >means that other operating systems are no longer detected and added to >grub by default in Debian 12. ... >I haven't followed further on to which solution they went with, but >since it's so late in the development cycle, wouldn't it make sense to >enable os-prober for Live systems for Debian 12, and then look at some >of the better solutions (like only probing at install time, and keeping >a cached results for grub) for Debian 13? I'm also pondering tweaking things in d-i to re-enable os-prober if the system looks like it might have some other OS installed. Yes, I realise that may sound odd(!), but I can see a number of users complaining that their dual-boot system doesn't work any more... :-/ -- Steve McIntyre, Cambridge, UK.st...@einval.com < sladen> I actually stayed in a hotel and arrived to find a post-it note stuck to the mini-bar saying "Paul: This fridge and fittings are the correct way around and do not need altering"
Non-free-firmware changes - initial cut released!
Hey all! Here's a status update on the non-free-firmware changes that we voted for last year [1]. Cyril has done a huge amount of work [2], implementing the bulk of what we need. We released d-i bookworm alpha 2 last weekend [3], including those changes. Our own testing shows that things work well on *our* test hardware, but we'd like some more assistance in testing! If you would like to help and you have a machine that wants firmware, please: 1. Boot the installer and verify that it identifies the necessary firmware correctly - go as far as configuring the network. If there are still missing firmware blobs, d-i will complain. If you stop before making any changes in the partitioner (partman), then this should a safe thing to do on your existing machine and won't make any permanent changes. 2. (If you can) complete an installation and check that all the hardware works as expected after normal bootup. We'd love people to do this to verify the more awkward blobs: audio and GPU. We're especially interested in wifi, NIC and GPU firmware here, as they have been the things blocking people installing Debian in the past. Please file bugs against "installation-reports" with whatever you find! Thanks in advance! We're also planning more updates before the full bookworm release - watch this space! [1] https://www.debian.org/vote/2022/vote_003 [2] https://debamax.com/blog/2023/02/27/debian-versus-non-free-firmware/ [3] https://lists.debian.org/debian-devel-announce/2023/02/msg5.html -- Steve McIntyre, Cambridge, UK.st...@einval.com "Because heaters aren't purple!" -- Catherine Pitt
Re: Starting the weekly live images for Bookworm building again
Hey Roland, Apologies for leaving you waiting a while :-/ On Mon, Jan 16, 2023 at 05:35:50PM +0100, Roland Clobus wrote: > >This is a follow-up of my mail from 2022-11-21 [A1]. > >I've made progress in the last two months, but would like to have some merge >requests approved, to get more traction and to allow others to jump aboard >and make the live images for Bookworm possible. > >As you can see, this affects many teams: >* live-setup: a MR to generate all live images for Bookworm [A2] ACK, I'll take a look at this again shortly. >* localechooser: A minor fix [A3] No idea about that, leaving for somebody else. >* live-installer: A better user experience after the installer is finished >[A4] Merred just now. >* live-build: Various installer improvements, including off-line installation >[A5] Not sure who might review that, let's see -- Steve McIntyre, Cambridge, UK.st...@einval.com < Aardvark> I dislike C++ to start with. C++11 just seems to be handing rope-creating factories for users to hang multiple instances of themselves.
Re: need we support unshadowed passwords from the installer
On Sat, Jan 14, 2023 at 09:18:59AM -0500, nick black wrote: >Marc Haber left as an exercise for the reader: >> On Fri, 13 Jan 2023 21:11:40 -0500, nick black >> wrote: >> >i'm absolutely not suggesting we stop supporting NIS or other >> >programs which rely on unshadowed passwords. it's a big ol' >> >tent, and we have more than enough room for you to carry forth >> >the torch of Solaris 2. i just don't think this belongs in the >> >installer anymore. >> >> Amen. NIS-based systems usually have professional administrators who >> are well able to change the configuration. > >hahah, yes i thought you might support the idea based off >adduser changelogs circa 2005 =]. > >thanks to you and peter for voicing your support. i will head >off to #debian-boot and try to drum up a merge. I'll be honest, I've been horrified for years that we can still ask the shadow question. I hadn't realised it might be relevant for NIS. Even so, +1 from me. Let's get this done, I think... -- Steve McIntyre, Cambridge, UK.st...@einval.com Is there anybody out there?
Re: Firmware GR result - what happens next?
On Thu, Oct 13, 2022 at 05:08:57PM +0200, Julien Cristau wrote: >On Thu, Oct 06, 2022 at 05:13:22PM +0200, Tobias Frost wrote: >> Maybe and idea would to do something like isa-support does for e.g >> sseX-support >> on CPUs that does not have that feature: It fails on installation with an >> debconf message, IIRC. >> So that would allow something like "new package" | >> "you-need-to-enable-nonfree-firmware-reminder-package" >> >Failing on installation is a terrible user experience, let's not, pretty >please. It's not great, no. Do you have a better suggestion for making sure people update sources.list? -- Steve McIntyre, Cambridge, UK.st...@einval.com "Since phone messaging became popular, the young generation has lost the ability to read or write anything that is longer than one hundred and sixty characters." -- Ignatios Souvatzis
Re: Firmware GR result - what happens next?
On Wed, Oct 05, 2022 at 10:11:27PM +0200, Julien Cristau wrote: >On Sun, Oct 02, 2022 at 08:21:31PM +0100, Steve McIntyre wrote: >> On Sun, Oct 02, 2022 at 11:08:47AM -0400, Michael Stone wrote: >> >On Sun, Oct 02, 2022 at 03:53:00PM +0100, Steve McIntyre wrote: >> >> On Sun, Oct 02, 2022 at 04:43:47PM +0200, Michael Biebl wrote: >> >> > What's the plan for upgraded systems with an existing >> >> > /etc/apt/sources.list. >> >> > Will the new n-f-f section added on upgrades automatically(if non-free >> >> > was >> >> > enabled before)? >> >> >> >> So this is the one bit that I don't think we currently have a good >> >> answer for. We've never had a specific script to run on upgrades (like >> >> Ubuntu do), so this kind of potentially breaking change doesn't really >> >> have an obvious place to be fixed. >> > >> >Is there a reason to not continue to make the packages available in >> >non-free? >> >I don't see a reason to force any change on existing systems. >> >> Two things: >> >> 1. I'm worried what bugs we might expose by having packages be in two >> components at once. >> 2. I really don't like the idea of leaving two different >> configurations in the wild; it'll confuse people and is more >> likely to cause issues in the future IMHO. >> >> Plus, as Shengjing Zhu points out: we already expect people to manage >> the sources.list anyway on upgrades. >> >I think in the absence of a release upgrade script (which I very much >doubt will happen, and be tested, and we can rely will be used, for >bookworm), Michael's suggestion seems like a reasonable way forward. I >imagine we'll need to patch dak to allow that, but it seems like it >should be tractable? I'm also worried what effect this will have on other tools that have to grok the archive (mirror tools, debian-cd, etc.). I'm not going to try and veto having things in more than one component, but (ugh!) I really think it's ugly. Actually, I think I'd much prefer Santiago's idea: > Couldn't we handle this via transitional firware* non-free packages, > that depend on bookworm non-free-firmware packages? We'd need to add some transitional binary packages for the small number of n-f-f source packages. That way people would get errors from apt if they don't read our warnings and update. Maybe this is a way forward? -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Re: Firmware GR result - what happens next?
On Sun, Oct 02, 2022 at 11:08:47AM -0400, Michael Stone wrote: >On Sun, Oct 02, 2022 at 03:53:00PM +0100, Steve McIntyre wrote: >> On Sun, Oct 02, 2022 at 04:43:47PM +0200, Michael Biebl wrote: >> > What's the plan for upgraded systems with an existing >> > /etc/apt/sources.list. >> > Will the new n-f-f section added on upgrades automatically(if non-free was >> > enabled before)? >> >> So this is the one bit that I don't think we currently have a good >> answer for. We've never had a specific script to run on upgrades (like >> Ubuntu do), so this kind of potentially breaking change doesn't really >> have an obvious place to be fixed. > >Is there a reason to not continue to make the packages available in non-free? >I don't see a reason to force any change on existing systems. Two things: 1. I'm worried what bugs we might expose by having packages be in two components at once. 2. I really don't like the idea of leaving two different configurations in the wild; it'll confuse people and is more likely to cause issues in the future IMHO. Plus, as Shengjing Zhu points out: we already expect people to manage the sources.list anyway on upgrades. -- Steve McIntyre, Cambridge, UK.st...@einval.com You raise the blade, you make the change... You re-arrange me 'til I'm sane...
Re: Firmware GR result - what happens next?
On Sun, Oct 02, 2022 at 05:31:16PM +0200, Cyril Brulebois wrote: >Steve McIntyre (2022-10-02): >> * Extra d-i code to inform users about what firmware blobs have been >> loaded and the matching non-free-firmware packages. Plus information >> about the hardware involved. Maybe a new d-i module / udeb for this? >> Exact details here still TBD. Probably the biggest individual piece >> of work here. > >Not just blobs that were loaded, but also those which might get loaded >in the installed system (since we don't have all modules around), see >hw-detect.post-base-installer.d/50install-firmware in hw-detect (udevadm >vs. modalias stuff). ACK. >> * Tweaks to add the non-free-firmware section in the apt-setup module >> if desired/needed. >> >> * An extra boot option (a debconf variable) to disable loading extra >> firmware automatically, then exposed as an extra option through the >> isolinux and GRUB menus. d-i "expert mode" can also be used to tweak >> this behaviour later, except (obviously) for things like audio >> firmware that *must* be loaded early if they're going to be useful >> at all. > >The option/variable looks easy enough (once we decide what to call it)… >Not sure what would be best to expose it to users: bootloader menus >already have many options (text vs. graphical, normal vs. rescue, >accessibility settings, etc.), and don't get internationalization >support anyway. On the flip side, having to go through a full expert >installation is very nice. > >Maybe start by documenting the option (probably installation guide plus >release notes), and of course implementing it; then see if/how we expose >it? Yup. Very much in that order... :-) -- Steve McIntyre, Cambridge, UK.st...@einval.com You lock the door And throw away the key There's someone in my head but it's not me
Re: Firmware GR result - what happens next?
On Sun, Oct 02, 2022 at 04:43:47PM +0200, Michael Biebl wrote: > >Hi Steve, > >thanks for the update! > >Am 02.10.22 um 16:27 schrieb Steve McIntyre: > >> * Tweaks to add the non-free-firmware section in the apt-setup module >>if desired/needed. > >... > >> If you think I've missed anything here, please let me and Cyril know - >> the best place would be the mailing list >> (debian-b...@lists.debian.org). > >What's the plan for upgraded systems with an existing /etc/apt/sources.list. >Will the new n-f-f section added on upgrades automatically(if non-free was >enabled before)? So this is the one bit that I don't think we currently have a good answer for. We've never had a specific script to run on upgrades (like Ubuntu do), so this kind of potentially breaking change doesn't really have an obvious place to be fixed. Obviously we'll need to mention this in the release notes for bookworm. Should we maybe talk about adding an upgrade helper tool? -- Steve McIntyre, Cambridge, UK.st...@einval.com Into the distance, a ribbon of black Stretched to the point of no turning back
Firmware GR result - what happens next?
Hi all! [ I've also blogged about this - see https://blog.einval.com/2022/10/02#firmware-vote-result ] I'm assuming that there will be no surprises and Kurt will shortly confirm the result that devotee reported already [1]. :-) We have quite a few things to do now, ideally before the freeze for Debian 12 (bookworm), due January 2023 [2]. This list of work items is almost definitely not complete, and Cyril and I are aiming to get together this week and do more detailed planning for the d-i pieces. Off the top of my head I can think of the following: * Update the SC with the new text, update the website. * Check/add support for the non-free-firmware section in various places: + d-i build + debian-cd + debmirror (?) + ftpsync (?) + Any others? * Uploads of firmware packages targeting non-free-firmware. * Extra d-i code to inform users about what firmware blobs have been loaded and the matching non-free-firmware packages. Plus information about the hardware involved. Maybe a new d-i module / udeb for this? Exact details here still TBD. Probably the biggest individual piece of work here. * Tweaks to add the non-free-firmware section in the apt-setup module if desired/needed. * An extra boot option (a debconf variable) to disable loading extra firmware automatically, then exposed as an extra option through the isolinux and GRUB menus. d-i "expert mode" can also be used to tweak this behaviour later, except (obviously) for things like audio firmware that *must* be loaded early if they're going to be useful at all. * Update the image build scripts to include the n-f-f packages, only build one type of image. I'll do my best to keep config and support around too for images without n-f-f included, to make it easier to still build those for people who still want them. * Matching updates to docs, website, wiki etc. * ... If you think I've missed anything here, please let me and Cyril know - the best place would be the mailing list (debian-b...@lists.debian.org). If you'd like to help implement any of these changes, that would be lovely too! [1] https://lists.debian.org/debian-vote/2022/10/msg0.html [2] https://lists.debian.org/debian-devel/2022/03/msg00251.html -- Steve McIntyre, Cambridge, UK.st...@einval.com There's no sensation to compare with this Suspended animation, A state of bliss
Re: General resolution: non-free firmware
Thorsten Glaser wrote: >Phil Morrell dixit: >>> Is there support for something like A but not enabled by default? >>> That is, you have to actively select a nÅn-default option >>I don't believe so, because that doesn't solve the problem at hand. How >>exactly do you select a non-default option if you cannot see or hear it? > >You see it in the bootloader, or if not, can append it manually to the >kernel command line so that the initrd and d-i see it both. (Iâm willing >to compromise on having early pre-initrd microcode updates always loaded >which would be the one thing we could not easily toggle on/off from the >initrd and/or d-i.) Please go and *read* and *respond* in debian-vote. The discussion is there, not here. You've utterly missed Phil's point about people not seeing or hearing boot options. Go and read my first mail in the thread for context. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Starting the firmware GR - see mail on d-vote
Hi all! Sorry for the delay on this, I've been really really busy. :-( I think it's time we started on the firmware GR, so I've mailed the -vote list: https://lists.debian.org/debian-vote/2022/08/msg00001.html -- Steve McIntyre, Cambridge, UK.st...@einval.com "C++ ate my sanity" -- Jon Rabone
Re: Bug#1014908: ITP: gender-guesser -- Guess the gender from first name
Roberto C. Sánchez wrote: >On Thu, Jul 14, 2022 at 11:14:43AM +0100, Steve McIntyre wrote: >> >> Do you have a real use for this package? > >Why in the world is that even a relevant question? There are plenty of >packages in the archive which are useful to essentially nobody apart >from the maintainer and there are even packages which are maintained >without being useful to the maintainer at all (but rather useful to >others). I think it's a valid question to ask. >> There are a *lot* of issues >> in this area, and mis-gendering people is not something to risk >> lightly... > >"There are a *lot* of issues in this area" seems rather nebulous. In >which area? Given the fact that we have clear and rather unambiguous >guidelines for what constitutes software which is appropriate for >inclusion in the archive, and given that on its face this software does >not seem to be in conflict with any of those guidelines, what then is >the problem? I'll link to https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ again, as a start. Assuming *anything* about names is iffy at best, and trying to derive other information (whether that's gender, age, nationality, whatever) from names is unreliable as all hell. When there is the potential for *also* causing offense from that unreliable information then I'd hope that people would know better. >BTW, I'm not interested in any sort of "well I don't like ..." or >"such and such could offend so and so ..." sort of arguments. Thanks, I'm well aware that you don't care. Maybe you could try? -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Bug#1014908: ITP: gender-guesser -- Guess the gender from first name
Edward wrote: >Steve McIntyre wrote: >> >> Oh, not *another* package that tries to guess things from names. >> >> Do you have a real use for this package? There are a *lot* of issues >> in this area, and mis-gendering people is not something to risk >> lightly... > >I can add a warning to the package about problems guessing things from names, >or I'm happy to retract the ITP. At the very least I think a warning would be useful, yes. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Bug#1014908: ITP: gender-guesser -- Guess the gender from first name
Adrian Bunk wrote: >On Thu, Jul 14, 2022 at 04:05:35PM +0200, Jeremy Bicha wrote: >>... >> Debian has a Diversity Statement [1] which says that Debian welcomes >> people regardless of how they identify themselves. Trans people and >> non-binary people face a lot of discrimination, harrassment and >> bullying around the world. > >Our Diversity Statement says that Debian "welcomes and encourages >participation by everyone". > >People who express how they identify themselves by having a swastika >tattoo on their forehead also face a lot of discrimination, harrassment >and bullying around the world. Our Diversity Statement makes it clear >that we are welcoming and encouraging their participation and are not >ourselves discriminating against them. And, to add the bit that a lot of people conveniently ignore when trying to make strawman arguments: "We welcome contributions from everyone as long as they interact constructively with our community." -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Bug#1014908: ITP: gender-guesser -- Guess the gender from first name
Adam Borowski wrote: >On Thu, Jul 14, 2022 at 04:05:35PM +0200, Jeremy Bicha wrote: >> > > >* Package name: gender-guesser > >> Debian has a Diversity Statement [1] which says that Debian welcomes >> people regardless of how they identify themselves. Trans people and >> non-binary people face a lot of discrimination, harrassment and >> bullying around the world. That bad treatment of these people is >> against Debian's core values. > >Unless they're Jewish, believe that a woman should be allowed to abort a >Down syndrome fetus, believe that there's more than just a name to the >gender, or have a kind of transsexualism that matches their life's >experiences and is detectable by brain imaging but the loud group says >doesn't exist. > >The inconsistency here is astounding. I genuinely have no clue what you're trying to say here. Are you actually somehow claiming that Debian's core values include bad treatment of Jews and those other groups? Seriously, WTF? -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Bug#1014908: ITP: gender-guesser -- Guess the gender from first name
Marc Haber wrote: >On Thu, 14 Jul 2022 12:54:52 +0100, Steve McIntyre >wrote: >>And I see you uploaded ~immediately - why even bother with an ITP? > >Is it proper procedure to upload without an ITP? IMHO there are 2 points to an ITP: * to save effort in case two people might be working on the same package * to invite discussion on debian-devel / elsewhere If people post an ITP and upload iummediately, then I don't think that helps on either count. If the only reason for the ITP is to make lintian quiet then I think that's a total waste of time - it's following a guideline blindly without understanding the reason for it. How do others feel? -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Bug#1014908: ITP: gender-guesser -- Guess the gender from first name
Steve McIntyre wrote: >edw...@4angle.com wrote: > >>Package: wnpp >>Severity: wishlist >>Owner: Edward Betts >>X-Debbugs-Cc: debian-devel@lists.debian.org, debian-pyt...@lists.debian.org >> >>* Package name: gender-guesser >> Version : 0.4.0 >> Upstream Author : Israel Saeta Pérez >>* URL : https://github.com/lead-ratings/gender-guesser >>* License : GPL-3 & GFDL-1.2+ >> Programming Lang: Python >> Description : Guess the gender from first name > >Oh, not *another* package that tries to guess things from names. > >Do you have a real use for this package? There are a *lot* of issues >in this area, and mis-gendering people is not something to risk >lightly... And I see you uploaded ~immediately - why even bother with an ITP? -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Bug#1014908: ITP: gender-guesser -- Guess the gender from first name
edw...@4angle.com wrote: >Package: wnpp >Severity: wishlist >Owner: Edward Betts >X-Debbugs-Cc: debian-devel@lists.debian.org, debian-pyt...@lists.debian.org > >* Package name: gender-guesser > Version : 0.4.0 > Upstream Author : Israel Saeta Pérez >* URL : https://github.com/lead-ratings/gender-guesser >* License : GPL-3 & GFDL-1.2+ > Programming Lang: Python > Description : Guess the gender from first name Oh, not *another* package that tries to guess things from names. Do you have a real use for this package? There are a *lot* of issues in this area, and mis-gendering people is not something to risk lightly... -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: feedback for NEW packages: switch to using the BTS?
Paul Wise wrote: > >During the discussions about NEW on debian-devel in recent times, I had >the idea that instead of the current mechanism of sending REJECT mails, >Debian could switch to using the BTS for most feedback on NEW packages. > >This means that most discussion about NEW packages would become public, >but of course the ftpmasters could opt to send private mail instead if >in some cases if there were sensitive issues to be discussed. > >The ftpmasters could simply file severity serious bug reports against >NEW packages that have issues blocking their entry into Debian. When >there are minor issues noticed at the same time, then file bugs of a >lower severity. Only when a NEW package has not had its serious bugs >fixed in a long time would an eventual removal and REJECT mail happen, >perhaps after a few months of zero action on the bug reports. Just to clarify: is this suggesting that packages from NEW would end up in the archive even with serious bugs? If not, what's the point of the "eventual removal" above? I'm not following you here... -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: shim-signed
Tollef Fog Heen wrote: >]] Hanno 'Rince' Wagner >> >> I am a very firm believer of giving people as much information as >> possible while being responsible. Meaning, that I would love to have >> that documentation - including a big warning sign which sais "if you >> follow this path, you may brick your machine and this is your problem, >> not ours". If someone is interested to learn _how_ the security is >> done and implemented, why should this be unavailable? > >Sadly, warnings generally don't work because people will ignore them and >break their systems and then blame the people writing the >documentation, causing load and grief on people were trying to be >helpful by writing the docs. > >I don't think we should invest a lot into writing the docs required >because we can use that effort elsewhere. Documentation requires >maintenance, just like everything else and if it's rarely used, we get >little value out of that effort. If somebody is very eager to write and >maintain that documentation over time, by all means, but we haven't seen >anyone step up to do so. +1 Alongside doing technical stuff, I've written a fair amount of user-facing docs for UEFI, shim and secure boot over the last few years. In the limited time I have available, I've tried to focus on the common cases for people using an expected simple setup. I'd *love* some help to cover more of the space. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: shim-signed
The Wanderer wrote: >On 2022-04-26 at 10:14, Marc Haber wrote: > >> On Sat, 23 Apr 2022 18:21:47 +0100, Steve McIntyre >> wrote: > >>> Alternatively, people can build replacement shim-signed packages >>> using their own root of trust if desired. If we had a large enough >>> number of users wanting a different root of trust, we could even >>> offer our own different shim-signed package to match. >> >> I would probably prefer to have grub an/or the kernel signed, >> avoiding additional code, but maybe having some explanation would >> convince me that the shim actually improves things additionally to >> adding complexity. > >My understanding has always been that the point of having a >Microsoft-signed shim, rather than having Microsoft sign GRUB or the >kernel, is to A: avoid the need for round-trip with Microsoft's signing >facilities every time the GRUB or kernel packages are updated, and B: >ensure that end-users can still build their own kernels (et cetera) >without having to go through the Microsoft signing process, even if >their systems are set up to take advantage of the signed shim. Correct on both counts. There's also: C: Microsoft will not sign GPL software here Shim is deliberately licenced permissively to get around that issue. >(And the point of having Microsoft sign it, rather than using a signing >key under the control of e.g. Debian, is that Microsoft's key is already >considered valid by the relevant firmware environments - including the >ones that can't be told to add another key to the list of valid ones. >That avoids having another barrier to entry, in the form of a set of >steps at the start of the install process which is going to be different >per UEFI designer, and is also going to be complex and unintuitive from >the perspective of a non-technical potential new user.) Nod. As we found in the Arm ecosystem a few years ago when discussing a similar setup for SB, it's also *very* difficult to find people who are capable, willing, large and trusted enough to act as the CA. Linux users may not like it that Microsoft are involved here, but system vendors also care here. There's potentially a huge liability for broken systems if somebody screws up... -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: shim-signed
The Wanderer wrote: >On 2022-04-26 at 18:05, Paul Wise wrote: > >> On Tue, 2022-04-26 at 20:41 +0200, Bastian Blank wrote: >> >>> secure boot signing process at Microsoft is a review-sign process >> >> What kind of review are Microsoft doing of the Debian shim? >> >> Are they reviewing the source and checking for a reproducible build? > >I'd be curious to have a more in-depth answer to this, myself. > >My understanding has always been that they check to make sure that what >they're signing is not visibly malicious, and in most cases also that it >can't chain to load something else (which isn't signed, and might be >malicious). Since the entire purpose of the shim - at least as I >understand it - is to chain to load something else, clearly either that >understanding is not correct, or they're making an exception for the >case of the shim. Microsoft themselves *don't* do direct code review of the shim submissions; they acknowledged some time ago that they didn't have direct knowledge good enough to make this sensible. Instead, there is a team of trusted distro maintainers who have stepped up to revies submissions. See * https://github.com/rhboot/shim-review * https://github.com/rhboot/shim/wiki for more information about what we look for. Every single patch that's applied to a signed shim will be reviewed by the community, and we also want to see what patches people have aplied to Grub, Linux, etc. too. We need a reproducible build for shim so that we can check that the shipped binary for signing matches what we can rebuild ourselves. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: shim-signed (was: Firmware - what are we going to do about it?)
Marc Haber wrote: >On Sat, 23 Apr 2022 18:21:47 +0100, Steve McIntyre > >>Better than that, our shim-signed source package always double-checks >>things here. At build time it removes the Microsoft signature and >>compares that shim binary to the binary that we submitted for >>signing. We would spot immediately if there was any code added. > >And if that check fails at build time, the Debian process refrains >from putting a Debian signature on the deb and from uploading? Can the >end user build the shim herself, remove the signature from the signed >shim and compare the binary, preferably in a documented way? Look at the shim-signed source - the build will fail if the code has changed. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Firmware - what are we going to do about it
Steven Robbins wrote: > >Luca Boccassi wrote: > >> Personally, I'd even go for option 4, so that other drivers are covered >> too (the general advice that can be found on the internet for users >> with nvidia hardware seems to be: "avoid Debian, go Ubuntu/Mint/etc", >> which is just a sad state of affairs). But option 5 is already a great >> improvement upon the status quo. > >Agree! > >The original post did mention video cards, but I'm left unsure whether the >non-free stuff in the NVidia case, for example, would fall into "firmware" >(hence included) or "drivers". If the latter, my sense is that Option 5 was >intended to be narrowly focused and exclude such drivers. I'd rather support >a wider focus that would include drivers -- basically any "non-free hardware >support package". So if Option 5 is narrow and Option 4 is wide-open "non- >free" maybe there's room for an option in between? I have no desire to add a wider set of packages from non-free onto our media, I'm afraid. I'm entirely focused on firmware and **not** drivers. As and when we start to draft a GR to formalise what the project wants, feel free to add an option for that too but I personally wouldn't expect it to gain much traction. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: shim-signed (was: Firmware - what are we going to do about it?)
Marc Haber wrote: >On Fri, 22 Apr 2022 11:16:42 +0200, Philip Hands >wrote: >>I understand the urge to insist upon absolute DFSG purity in the media >>we produce, but when it comes to wanting to avoid every last shred of >>data that we could not regenerate ourselves, I think we crossed that >>line some time ago. >> >>I'm thinking of shim-signed, which is included in our official media. >> >>Despite being free software in source form, it is signed by Microsoft, >>and can only be expected to work with that signature ... which we cannot >>create. >> >>On most (all?) hardware one is able to avoid UEFI secure-boot, so won't >>need to use shim-signed, but I'd imagine that some hardware insists on >>secure-boot, or the opt-outs are somehow broken and so is not usable >>without shim-signed. > >Excuse me for asking a user question on -devel, but do we have any >docs where someone explains how much a security trade off is >shim-signed relativ to the optimum? I think that using shim-signed is >surely worse than a directly signed kernel, but I don't know whether I >can tell my system (or shim-signed?) to accept MY or Debian's signed >kernel without the Microsoft intermediate signature, and whether this >is any more secure than running an encrypted system without secure >boot at all. We don't have good docs around this in general (hey, it's security software - it's against the law to write good and complete docs!), but I've certainly discussed this with a few folks over the years. Fundamentally, shim is a compromise. It lets us use an existing root of trust (that's already installed on the vast majority of x86 machines) to bootstrap our own secure setup. If you don't like the fact that Microsoft's keys are involved. it's possible on a lot of machines to enrol your own keys and remove Microsoft's entirely. If you go that way, you could absolutely sign grub and/or the kernel directly. But that's not something that's easy to do - it's not likely to work well for the vast majority of users. Running an encrypted system without secure boot *at all* is a difficult thing to support well. The entire point of SB is that the firmware can use keys to validate the next stage in the boot process. An *encrypted* kernel stops other people logging in to your system, but it does not give you protection against somebody tampering with the system behind your back and doing a MitM attack. Alternatively, people can build replacement shim-signed packages using their own root of trust if desired. If we had a large enough number of users wanting a different root of trust, we could even offer our own different shim-signed package to match. ... >>If not, is the problem having other blobs of data on the media that we >>also cannot generate, or is it the licensing of that data, or something >>else? > >We can compile shim-signed and compare the signed code with our own >object code, can't we? That we we would only have to worry about the >validity and benignness of the signature and not worry about having >undocumented functionality in the signed code. Better than that, our shim-signed source package always double-checks things here. At build time it removes the Microsoft signature and compares that shim binary to the binary that we submitted for signing. We would spot immediately if there was any code added. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Firmware - what are we going to do about it?
Andrey Rahmatullin wrote: > >On Wed, Apr 20, 2022 at 12:53:46PM -0500, Devin Prater wrote: >> But back on topic, would the nonfree DVD ISO's have more firmware on them >> than the CD version? Or is that just for offline installs? >As far as I understand it there is just one set of non-free firmware for >including in the ISOs and separate drives, which consists of all non-free >firmware found in the archive. That's definitely the intention, yes. There may be *minor* differences in the lists that are in use in various places, as different people have written the scripts involved. This is actually another place where we'd benefit from the new non-free-firmware component - we'd be able to just consistently rely on *that* list of packages. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Firmware - what are we going to do about it?
Russ Allbery wrote: >Steve McIntyre writes: > >> Thanks! That's a really good question, and one that we should also >> include on the GR. I'd split my option 5 into two: > >> 5. Include non-free firmware but do not enable it by default >> 6. Include non-free firmware and enable it by default > >> In either case, we'd make the opposite choice available via a d-i kernel >> command line option and a boot menu option. I think that makes sense? > >I agree with this option split, but that reminds me of a different >procedural note. > >While I recognize and respect the desire to create a comprehensive ballot, >I'm still going to advocate for proposing a GR only with the options that >the person proposing the GR would vote above NOTA, and possibly only your >preferred options. ACK, that's fair and reasonable. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Firmware - what are we going to do about it?
Hey Ansgar! Ansgar wrote: >On Wed, 2022-04-20 at 17:11 +0100, Steve McIntyre wrote: >> > Russ Allbery wrote: >> > 1. Purely free installation. >[ Other options ] >> > 4. Enable non-free firmware and enable normal upgrades, [...] >[...] >> Now, the *default* is going to be the hard choice for us to make. > >Do you think the choice for the default should be part of a GR too, a >separate GR or decided some other way? Thanks! That's a really good question, and one that we should also include on the GR. I'd split my option 5 into two: 5. Include non-free firmware but do not enable it by default 6. Include non-free firmware and enable it by default In either case, we'd make the opposite choice available via a d-i kernel command line option and a boot menu option. I think that makes sense? -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Firmware - what are we going to do about it?
Russ Allbery wrote: >Jonas Smedegaard writes: > >In other words, rather than having to do what one does now and choose >between the free installer and the non-free installer, my understanding of >option #5 is that there would be one install image, but there could then >be a prompt asking you whether you want to install non-free firmware. We >could even offer a few different options (with the caveat that options >tend to confuse users, so we may not want to add too many or gate them >behind an advanced mode): > >1. Purely free installation. >2. Enable non-free firmware in the installer but don't put it on the > installed system. (Not sure how useful this is, but I could see > needing non-free firmware to bootstrap from wifi but the running system > may eventually not use the non-free firmware.) >3. Enable non-free firmware and install it on the system but pin it so > that it's never upgraded by default. >4. Enable non-free firmware and enable normal upgrades, similar to adding > the non-free archive area today but only adding the firmware archive > area. > >I think 1 and 4 are the most useful options, and I'm not sure how many >people really want 2 or 3, but if there are enough people who want them, I >don't see any technical barriers to adding them. Nod, exactly. We can add those options via boot flags and menu options, with later d-i screens too to allow the choice (maybe in advanced mode). That's probably the easiest way to manage it. Now, the *default* is going to be the hard choice for us to make. With the example of blind people using d-i, we'll want to make an easy option for those people to boot the installer with all firmware enabled and installed - see the firmware-sof-signed package that they'll need to get audio prompts during installation. >I feel professionally obligated to argue that Debian should, *by default*, >upgrade anything that it installs, since from a security standpoint that >is the least risky default configuration (with, as always, the caveat that >there are special cases with different security models for which this >default isn't appropriate). But that doesn't rule out a prompt or >allowing a user to turn this off if they want to. Yup. >> I agree that we should make it easier for our users to choose to trust >> black magic "stuff" that they need to enable their devices. > >> I do not think that we should impose on our users to trust black magic >> by default, though. > >I think this is a somewhat different question than whether we put the >firmware on the default installation media so that it's *available* if >users want it. Nod. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Firmware - what are we going to do about it?
Paul Wise wrote: > >On Tue, 2022-04-19 at 01:27 +0100, Steve McIntyre wrote: > >> There are a number of issues here that make developers and users unhappy: > >There are a couple more issues related to unredistributable firmware: Oh, I'm quite sure there are more than that even! :-) >Some firmware is only available in the operating system preinstalled on >the device and needs to be manually extracted before d-i is run, >potentially even only from processes running on the preinstalled >operating system in cases where the storage must be wiped (such as >Android devices) before an alternative OS like Debian can be installed. >IIRC there is some laptop WiFi firmware that is like this. Yup. >Some firmware is not redistributable and is only available in >proprietary drivers on websites and is hard to extract from those >drivers. IIRC some of the proprietary nvidia firmware for use by the >libre nouveau GPU driver is like this, both signed firmware for very >new nvidia hardware and unsigned firmware for very old nvidia hardware, >although the firmware for the old nvidia hardware has libre firmware in >nouveau, but the libre firmware is/was buggier than the proprietary >firmware. The tools for extracting the old firmware aren't in Debian. Yup. I don't see us fixing *all* of these issues any time soon. But in those cases where we *can* redistribute firmware we can make things easier for users who need it. And *then* we can explain to them why the non-free firmware is bad for their freedom, with examples of what they can do to improve things. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Firmware - what are we going to do about it?
Hi Polyna-Maude, Polyna-Maude R.-Summerside wrote: >bwt ! >1st I've always saw Debian having brltty support from the start >2nd Just install the firmware instruction here and your problem will be >solved. >https://wiki.debian.org/Firmware > >Stop blaiming other people when the problem is a lack of research on >your part and expectation all work "out of the box" in all situation. > >Take destiny into your own hand. Please tone your argument down here. As developers we're having a discussion about how to make things work better and more easily for our users. Devin's description of the troubles they had are entirely on-topic and useful in the context of that discussion. Castigating them here is *not* helpful. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Firmware - what are we going to do about it?
Jeremy Stanley wrote: >On 2022-04-19 01:27:46 +0100 (+0100), Steve McIntyre wrote: >[...] >> Along with adding non-free firmware onto media, when the installer >> (or live image) runs, we should make it clear exactly which >> firmware packages have been used/installed to support detected >> hardware. We could link to docs about each, and maybe also to >> projects working on Free re-implementations. >[...] > >It's probably what you meant, but just to be clear, as a user I'd >also want to know which of the firmware packages used/installed were >pulled from non-free and what devices prompted their addition. >Something like "The following packages do not meet Debian Free >Software Guidelines but were installed because they're necessary in >order to enable or secure some of your hardware: foo(CPUX21 >processor microcode), bar(PM2743 power management controller), >baz(RF17G wireless interface), ..." > >This would allow me to make better informed decisions, such as: > >1. Disable some of these devices if I don't actually use them. > >2. Seek out alternative open peripherals which do work without >proprietary firmware. > >3. Reach out to the manufacturer of my device and extol the virtues >of open firmware. > >4. Consider (as you mentioned) working on my own reimplementation. Nod, that's exactly what I was suggesting. More information for users here helps them to make decisions like this, absolutely. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Firmware - what are we going to do about it?
w...@debian.org wrote: > >Also, drivers and firmware are different things. *Totally*. This is one of my pet peeves - many *many* people confuse the two and talk about "non-free drivers" in Debian when they actually mean firmware... ARGH. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Firmware - what are we going to do about it?
er suggestion, please let us know! I'd like to take this set of options to a GR, and do it soon. I want to get a clear decision from the wider Debian project as to how to organise firmware and installation images. If we do end up changing how we do things, I want a clear mandate from the project to do that. My preference, and rationale Mainly, I want to see how the project as a whole feels here - this is a big issue that we're overdue solving. What would I choose to do? My personal preference would be to go with option 5: split the non-free firmware into a special new component and include that on official media. Does that make me a sellout? I don't think so. I've been passionately supporting and developing Free Software for more than half my life. My philosophy here has not changed. However, this is a complex and nuanced situation. I firmly believe that sharing software freedom with our users comes with a responsibility to also make our software useful. If users can't easily install and use Debian, that helps nobody. By splitting things out here, we would enable users to install and use Debian on their hardware, without promoting/pushing higher-level non-free software in general. I think that's a reasonable compromise. This is simply a change to recognise that hardware requirements have moved on over the years. Further work If we do go with the changes in option 5, there are other things we could do here for better control of and information about non-free firmware: 1. Along with adding non-free firmware onto media, when the installer (or live image) runs, we should make it clear exactly which firmware packages have been used/installed to support detected hardware. We could link to docs about each, and maybe also to projects working on Free re-implementations. 2. Add an option at boot to explicitly disable the use of the non-free firmware packages, so that users can choose to avoid them. Acknowledgements ==== Thanks to people who reviewed earlier versions of this document and/or made suggestions for improvement, in particular: • Cyril Brulebois • Matthew Garrett • David Leggett • Martin Michlmayr • Andy Simpkins • Neil Williams -- Steve McIntyre, Cambridge, UK.st...@einval.com Who needs computer imagery when you've got Brian Blessed? signature.asc Description: PGP signature
Re: proposed MBF: packages still using source format 1.0
Ian Jackson wrote: > >1. Why is 1.0-without-diff not always worse than 3.0 (native) ? > >1.0 native is sometimes better than 3.0 (native) because dpkg-source >refuses to build a 3.0 native package with a Debian revision in its >version number. > >This prohibition exists solely because of a doctrinal objection to >native-format packages with Debian revisions. There is no technical >reason why this restriction could not be lifted. I sometimes upload >this way and I have never had anyone report problems[1] with it. > >IMO there is nothing wrong with native format packages with Debian >revisions. They work just fine. For a small paockage, this is often >a good choice, because it avoids dealing with patches at all. Why on earth *would* you mess around using Debian revisions on a native-format package, though? IMHO it's pointless and is just going to confuse people. Unless you can explain a good reason to need this, I'd argue strongly that the 3.0 native support is DTRT for the principle of least surprise if nothing else! -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Gmail bounce unauthenticated @debian.org addresses
Baptiste Beauplat wrote: >We recently discovered that Gmail started to bounce email from >mentors.debian.net with the following message: > >550-5.7.26 This message does not have authentication information or >fails to 550-5.7.26 pass authentication > checks. To best protect our users from spam, the 550-5.7.26 message has >been blocked. Please visit 550-5.7.26 >https://support.google.com/mail/answer/81126#authentication for more 5 >50 5.7.26 information. Yup. I've seen this too. Thanks for starting the thread here, which has prompted useful clues on how to deal with this. It's maddening to see Google continue to f*ck up mail requirements for everybody else. Of course, they continue to be (one of?) the biggest sources of spam on the net and show no interest in doing anything about it. "Don't be evil" indeed... :-( -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Next attempt to add Blends to Debian installer
Hi Andreas! Apologies for keeping you waiting here. For a range of reasons, I've been struggling to find *any* time to look at this. Much as I hate to let you down, I think the best policy now is to be honest (with myself and you!) and say that I'm not going to find time to do this any time soon. Sorry. :-( I'm just orphaning some of my packages now, as I've been failing to keep on top of those already. Other stuff has had to take priority for too long. I still think the best route forward is to add new functionality into debconf and then update tasksel use that. But please don't wait on me any more. If you can find other volunteers to pick this up, *please* do. -- Steve McIntyre, Cambridge, UK.st...@einval.com Google-bait: https://www.debian.org/CD/free-linux-cd Debian does NOT ship free CDs. Please do NOT contact the mailing lists asking us to send them to you.
Re: Status of weekly live builds
Peter wrote: > >I've just noticed that the weekly live builds (both the free ones [0] >and the unofficial non-free ones [1]) have not been updated since August >9, 2021. Is this on purpose or did some machinery get stuck? I disabled the testing live image builds after the bullseye release, and I'm waiting for somebody to take over maintaining them. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: partman, growlight, discoverable partitions, and fun
Hi Nick! On Tue, Sep 28, 2021 at 12:47:11PM +0200, Cyril Brulebois wrote: >Hi nick, > >[ cc += debian-boot@ ] > >nick black (2021-09-27): >> Marc Haber left as an exercise for the reader: >> > But maybe an alternative? I find the partitioning step one of the >> > most error-prone and hard-to-use parts of non-trivial Debian >> > installations. >> >> so overall, i've got to say the feedback i heard here was a lot more >> positive than i was expecting, though there was a bit less than i had >> hoped for. but perhaps something that can be touched would see more >> interest? > >FWIW I've followed the answers to your mail over the last few days, >but I haven't had a chance to look at either the video or the slides >(only 4 days before your initial mail and the one I'm replying to…). Amen to that! >At first glance, it seems fine to be experimenting with a different >partitioner; of course all people are somewhere on the love/hate >spectrum regarding partman and the zillions of partman-* packages, but >it's indeed a shell maze and it *probably* could use some heavy lifting. >I keep hoping that simple use cases are made simple(r)… maybe growlight >could help there (e.g. “I'd like soft RAID1 with encrypted LVM, all that >with a UEFI boot — therefore ESP —, without being a storage wizard”). ACK. I've done a *bit* of hacking around partman over the last few years - it's heavy going and probably the least well understood part of d-i... :-/ >I suppose some step (once you've experimented on a growlight-only >d-i) would be to have both partman and growlight, and let people >choose (maybe with a flag at boot time, or by entering expert mode or >whatever), until we have a better idea what works, what doesn't, what >can be fixed, what cannot, and until a decision is made for the next >Debian stable release. > >Leaving the technical integration aside for a moment, one question >that comes to mind is whether this would be a one-shot contribution or >some kind of longer commitment to maintain that different partitioner. >I seem to remember earlier attempts at revamping the partitioning >step, which stalled eventually. > >(Your recent mail to debian-newmaint@ is a hint; your apparent >steadiness of those packages maintenance-wise is another; and your >apparent interest in adding support to possibly missing features yet >another.) > >Of course, one might object that the question is moot as there isn't >much active development on partman components (even if some patches have >been submitted over the last few days), but at least that's a known >(imperfect, but…) beast. Nod! >> given that no one seemed to reject the idea out of hand, i'm going to >> go ahead and rebase my work from 2012 (or more likely look at it once >> and redo it) in a Salsa fork of d-i, and make some installation media >> available. forgive me for likely only having x86 available at first. >> i'll try to have this done within a week or so, and put it up on my >> server. people can then give it a whirl. > >Feel free to touch base with debian-boot@ and/or debian-cd@ once you >have a working proof of concept that some people have toyed with; we can >discuss how the “alternative” part could be implemented (different >images, or both partitioners ship together, with some option/selection — >people might remember GRUB vs. LILO). I cannot guarantee a timely answer >(life tends to keep me busy), but you shouldn't see a lack of answer >after just a few days as if people were not interested. 100% true - expect people are busy, rather than hostile! :-) -- Steve McIntyre, Cambridge, UK.st...@einval.com Google-bait: https://www.debian.org/CD/free-linux-cd Debian does NOT ship free CDs. Please do NOT contact the mailing lists asking us to send them to you.
Re: Arch triplet for uefi applications
On Tue, Aug 10, 2021 at 03:19:10PM -0700, Josh Triplett wrote: >Bastien Roucariès wrote: >> I am going to compile shell.efi from source. >> >> I whish to install to something stable, but I need an arch triplet in order >> to >> put in a multiarch (like) location. >> >> I suppose that it will be x86_64-efi-none (or maybe x86_64-windows-efi ) >> and >> i686-uefi-none ? As Simon says, *definitely* not *windows* here please! :-) >I don't think GRUB's x86_64-efi is an architecture triplet, just a >GRUB-specific target. > >UEFI is effectively an "operating system", insofar as it provides a >runtime environment with some baseline properties and a set of system >calls. uefi definitely isn't a "vendor", and the OS shouldn't be "none" >since there is an OS of sorts. > >For that reason, Rust uses the target names "aarch64-unknown-uefi", >"i686-unknown-uefi", and "x86_64-unknown-uefi". Those seem like the >right names for these targets in other toolchains, as well. Nod, agreed. I think that makes sense. -- Steve McIntyre, Cambridge, UK.st...@einval.com < sladen> I actually stayed in a hotel and arrived to find a post-it note stuck to the mini-bar saying "Paul: This fridge and fittings are the correct way around and do not need altering"
Re: Reconsider sending ITP bugs to debian-devel: a new list?
Hi Jon! Jon Dowland wrote: > >ITP bugs are copied to debian-devel@. The intention, I think, is to make >sure that they have many eyes on them. ITP bugs often get feedback from >readers of debian-devel. > >I think this is valuable. However, it's one job/task/role, and sometimes >One wishes to focus on other jobs/tasks/roles instead. When I subscribed >to debian-devel directly, I most often filtered ITP mail into a separate >mailbox, to read at separate times. Nowadays, I read most Debian lists >via NNTP gateways, and filtering ITPs is not quite so convenient (not >least, because I don't easily have an analogue of the ITP-dedicated >mailbox I used to.). But besides me, I think a better "default" for >debian-devel would be not to have the ITPs. > >I think the ITP mails can make reading the rest of the list difficult >without extra local filtering or steps. Some times they are the >majority of the list traffic. I think it would be better if >ITP mail went to a separate, dedicated list, e.g. "debian-itp" to which >contributors are encouraged to subscribe and participate. To be honest, I think if we did that we'd lose just about all the reviews that currently happen. The whole point of sending ITPs to d-devel is that they will be seen by a wider audience, but I can't see many signing up for YA mailing list for them. -- Steve McIntyre, Cambridge, UK.st...@einval.com "We're the technical experts. We were hired so that management could ignore our recommendations and tell us how to do our jobs." -- Mike Andrews
Re: Next attempt to add Blends to Debian installer
Hey Andreas! On Tue, Mar 02, 2021 at 02:46:52PM +0100, Andreas Tille wrote: >On Thu, Oct 08, 2020 at 10:23:19AM +0100, Steve McIntyre wrote: >> Hey Andreas! I hope you're keeping well! >> >> On Tue, Oct 06, 2020 at 06:08:02PM +0200, Andreas Tille wrote: >> >> (Overdue!) update: I've been hacking on this for a while, and I hope >> >> to have a prototype for testing up shortly. It works fine on my local >> >> system, but in a test d-i build it fails totally so I've clearly >> >> missed something! Debugging that now... >> > >> >I wonder whether I might have missed some information whether there >> >is something I could test meanwhile. >> >> I'm afraid that various higher-priority interrupts came up (new job, >> UEFI security work) and I got side-tracked for a while. You must be >> psychic - I just started picking things up again last weekend. > >I admit I did not payed much attention on the development of tasksel and >thus the chances to select Blends right from the installer. The topic >remains to be urgent for all Blends - but I'm afraid it will be to late >for Debian 10. Or did I missed something and the status is promising >for this release? Apologies, I think I've let you down :-( . I've made a *small* amount of progress at hacking on debconf (route #2). But again I've had other things come up, not least another round of Secure Boot fixes. We're not going to have changes in for Bullseye. Sorry. :-( -- Steve McIntyre, Cambridge, UK.st...@einval.com Getting a SCSI chain working is perfectly simple if you remember that there must be exactly three terminations: one on one end of the cable, one on the far end, and the goat, terminated over the SCSI chain with a silver-handled knife whilst burning *black* candles. --- Anthony DeBoer
Re: -1 (Re: Making Debian available)
Paul wrote: > >During install, the installer asks if you have a disk with drivers on >for this closed hardware, I don't know what it wants at this point. > >If we could insert a 2nd usb disk, or anything with the correct drivers >on, it may help. Argh. Pet peeve. It asks for a disk with *firmware* on it, not *drivers*. Please stop confusing the two. -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: RFC: raising ca-certificates package Priority to standard or important
Hey Julien, On Fri, Jan 22, 2021 at 12:00:56PM +0100, Julien Cristau wrote: >On Thu, Jan 21, 2021 at 02:47:25PM -0300, Antonio Terceiro wrote: >> On Thu, Jan 21, 2021 at 03:10:47PM +0100, Julien Cristau wrote: >> > And which of standard or important made most sense (AIUI, standard >> > means "installed by default in d-i" and important means "installed by >> > default in debootstrap"). >> >> wget is already Priority: standard and recommends ca-certificates, so it >> seems to me that making it standard would be a noop in practice for most >> of the systems installed by d-i. >> >> On the other hand, all cases that I remember seeing a problem caused by >> missing ca-certificates was in systems not installed by d-i, such as >> containers, vm images, etc. Based on that, I would make it important. > >Here's my thinking on this: >I would expect "standard" to get installed on "general purpose" VM >images, and "important" *not* to get installed on "minimal" container or >VM images. Looking at the docker debian image build script just now[1], >it seems to pull in required packages + iproute2 and ping, so it has its >own selection that doesn't include "important" priority. So changing >the severity, by itself, won't change anything unless we go all the way >to "required" which feels like it'd be going too far (but then I also >don't think apt should be "required"). >If there are specific examples where you think "important" would help >I'd be interested; right now I'm sort of favouring "standard" as good >enough. Sounds like good logic to me. Thanks for looking into this! -- Steve McIntyre, Cambridge, UK.st...@einval.com Can't keep my eyes from the circling sky, Tongue-tied & twisted, Just an earth-bound misfit, I...
Re: Making Debian available
Bjørn wrote: >Steve McIntyre writes: > >> Marc Haber wrote: >>> >>>I was not aware of that feature. It is good to have that, but I would >>>be embarrassed to seriously suggest this way because we can't manage >>>to get WLAN working in the installer for political reasons. >> >> Are we seriously just going to describe our Free Software goals as >> "political reasons"? Should we just abandon them? > >FWIW, I did not read Marc that way. Fair enough. >Somehow, the Free Software goals have been interpreted to mean that >there is a difference in "free" between firmware on device internal >flash and firmware on host ssd. This makes no sense from a technical >point of view. Which makes the interpretation political if we assume >that the decision is made by otherwise sane people ignoring technical >arguments. To a simple end user, there might not be a difference. They have firmware for their device. However, in the latter case Debian has shipped non-free stuff. That is a big shift in our position. Don't get me wrong: I'm not saying this is an impossible place for us to go to. But before we do that we should have an open and honest debate about it. -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Making Debian available
Marc Haber wrote: >On Mon, 18 Jan 2021 16:35:01 +0000, Steve McIntyre >wrote: >>Marc Haber wrote: >>>I was not aware of that feature. It is good to have that, but I would >>>be embarrassed to seriously suggest this way because we can't manage >>>to get WLAN working in the installer for political reasons. >> >>Are we seriously just going to describe our Free Software goals as >>"political reasons"? Should we just abandon them? > >No, we shouldn't, but we should drop the double standards that we >obviously apply towards docs and firmware. Sorry, I'm not following you on that. Can you expand on that please? How are we treating docs differently? -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Making Debian available
[ Catching up on this thread a little late... ] Ansgar wrote: >On Fri, 2021-01-15 at 15:30 +, Jeremy Stanley wrote: >> On 2021-01-15 12:11:06 +0100 (+0100), Emanuele Rocca wrote: >> [...] >> > So the current situation is that we make an active effort to >> > produce two different types of installation media: one that works >> > for all users, and one broken for most laptops. >> [...] >> >> The one you say "works for all users" doesn't "work" for me because >> it contains proprietary closed-source software I don't want. > >Then don't install them? > >Debian's Social Contract states "We encourage CD manufacturers to read >the licenses of the packages in these areas [non-free & contrib] and >determine if they can distribute the packages on their CDs." Maybe we >should do that for the CD images we manufacture? :) There's a major difference here - do we want Debian's *official* media to include non-free stuff? We've had this discussion a few times, including in person back at DC15 at least. Back then, the overwhelming response was *no*. We can change that, but it's not something to do lightly. We *can* do a better job of pointing people at the media with non-free firmware included. I've added a few more links in various places, but IMHO we desperately need a better set of download (etc.) pages rather than just bodging changes in to the existing inadequate pages. See #819664, for example. -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Making Debian available
Marc Haber wrote: > >I was not aware of that feature. It is good to have that, but I would >be embarrassed to seriously suggest this way because we can't manage >to get WLAN working in the installer for political reasons. Are we seriously just going to describe our Free Software goals as "political reasons"? Should we just abandon them? -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Release status of i386 for Bullseye and long term support for 3 years?
Stephan wrote: >Am Sa, Dez 12, 2020 at 18:09:02 +0200 schrieb Adrian Bunk: >>4. People who wrongly installed i386 on amd64-capable hardware. > >Well, some releases ago befor multi-arch I used to install i386 even on >am64-capable hardware if ram was quite low (=< 8GB) and if the chance >wasnât that low that you needed to install ia32-codec to watch ancient >video formats. > >I wouldnât do it anymore but at least one system is still in use, and >there isnât a real supported way to upgrade from i386 to amd64. It's still quite new, but we have a package in the archive for this now: https://tracker.debian.org/pkg/debian-crossgrader It's targeted at systems exactly like yours, tbh. Maybe give it a try? -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Debconf - adding "treeselect" type(s)?
Thomas Goirand wrote: >On 11/29/20 11:37 PM, Timo Weingärtner wrote: >> >> Would checking "a" automatically also check "a/*"? >> Is it only about UI, meaning "a/*" would be collapsed under "a"? >> Shall it be possible to check "a", but uncheck "a/b"? >> >> GrüÃe >> Timo > >I very much agree that this needs to be discussed, then specified correctly. I think I've been clear on this bit in the thread? What's missing? >Also, would that sound crazy if we were to introduce the tree as >described with json? That's simple enough, so that even a human can do >it by hand, and also understood by all languages. json is all well and good for more powerful languages (and yes, I'd be convinced there!), but nigh-on impossible to do reliably in shell AFAICS. Shell is (one of) the most common environments for driving debconf, in package config and postinst scripts. I don't think that's going to fly. Oh, except... We'd only have to generate the json from shell, not parse it. That may be more tractable. Thanks, that could be a good suggestion! >Otherwise, what would be the way to describe a tree? I'm playing with ideas so far, let's see shortly. :-) -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Debconf - adding "treeselect" type(s)?
Thomas Goirand wrote: >On 11/30/20 11:18 AM, Steve McIntyre wrote: >> I'm envisaging treating the Choices *mostly* like in an existing >> select/multiselect, but with extra decoration to indicate the position >> in the tree for display. It would then be up to the frontend to decode >> that and work out a sensible way to display things as best it can. Not >> sure on the best way to do the decoration, hoping there'll be an >> available special character or similar that we could use in strings to >> avoid too big an upheaval here. > >å§ [1] Maybe... >>> Although these require manual selection, I think they do have at least >>> some use, and I'd rather keep them going. It shouldn't be too hard to >>> get full coverage, pulling in help from specialists if necessary. >> >> I guess so. > >How would it be pre-seeded? Just like a multiselect? With only leafs >that could potentially be in the pre-seed? That's exactly my plan, yes. The non-leaf bits are just syntactic sugar for the UI. -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Debconf - adding "treeselect" type(s)?
Timo wrote: >-=-=-=-=-=- > >Hallo Steve McIntyre, > >30.11.20 16:36 Steve McIntyre: >> I'll admit that I'm thinking of this *mostly* in terms of the needs of >> tasksel so far, but I'd expect switching to a new template type is >> likely to need a rethink to make best use of them anyway. > >In my mind there is also locales where we should have three levels: >de > de_AT > de_DE >[ ] de_DE >[ ] de_DE@euro >[ ] de_DE.UTF-8 >en > en_DK > en_GB > en_US ACK. For the tasksel work I'm also looking at a 3-level tree to organise the Blends stuff, at least. -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Debconf - adding "treeselect" type(s)?
Colin Watson wrote: >On Mon, Nov 30, 2020 at 10:24:56AM +0000, Steve McIntyre wrote: >> Timo wrote: >> >What are the proposed semantics of this multitreeselect? >> > >> >If we imagine something like: >> > >> >[ ] a >> > [ ] a/b >> > [ ] a/c >> >[ ] b >> > [ ] b/a >> > >> >Would checking "a" automatically also check "a/*"? >> >Is it only about UI, meaning "a/*" would be collapsed under "a"? >> >Shall it be possible to check "a", but uncheck "a/b"? >> >> Yes, I'm thinking just about UI here: my plan would be to not have >> anything *selectable* at "a" - it would just toggle the display of the >> "a/*" subtree in those frontends that can display such a thing. > >If we took that approach then it wouldn't work so well for the grub2 >case, where "a" would be a whole disk and "a/b" and "a/c" would be >individual partitions. (Unless we rephrased the choices to include an >explicit "Whole disk" option or something.) That might be the best bet anyway, I think. I'll admit that I'm thinking of this *mostly* in terms of the needs of tasksel so far, but I'd expect switching to a new template type is likely to need a rethink to make best use of them anyway. -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Debconf - adding "treeselect" type(s)?
Colin Watson wrote: >On Sun, Nov 29, 2020 at 07:21:54PM +0000, Steve McIntyre wrote: >> On and off, I've been hacking on tasksel for quite a while to improve >> the UI there and add better support for things like Blends. I've made >> some progress in my hacking, but I think I've hit a brick wall and I >> need to change tack. :-/ >> >> What I've ended up doing so far is hacking tasksel to give a poor >> *approximation* of a tree-style layout: classifying some existing >> tasks under headers and building a tree, then displaying each of the >> nodes of the tree one level at a time via the existing debconf >> setup. It just about works, but it's ugly as all hell and I'm not >> happy with where I've got to. I've sunk a lot of development time into >> this, but I don't think it's ready to fly this way. :-( >> >> What I *actually* need here is proper support in debconf for >> tree-style selection. I'm thinking of adding that, adding new types >> "treeselect" as a tree-equivalent of "select" and "multitreeselect" as >> an extension of "multitselect". The first one may not even be needed, >> but would be a trivial simplification of the second, so *meh*. > >grub2's maintainer scripts attempt to emulate something a bit like a >tree-style multiselect by putting "- " at the start of the partition >descriptions to indicate that they're under the disks. It's certainly >not ideal; I'd be open to considering something better, although it >might take some time to be usable by packages in general. (tasksel has >an easier time of it than some, since it's a full debconf-based >application rather than needing to work at the preconfiguration stage.) Yup. >> AFAICS I'd need to add at least basic support for the new type(s) into >> all the frontends that *can* support it. So, I have a couple of >> questions: >> >> 1. How flexible is Debconf at coping with a frontend not including >> support for a type?? > >Not hugely. The INPUT command would return an internal error with the >text "unable to make an input element". I think we'll need at least >minimal support across all the frontends, which may need to inform the >design of the element. Bah, I was afraid you were going to say that. :-( For frontends that can't meaningfully deal with a tree (like showing/hiding sub-trees), I think the best way is to maybe just display the entire tree and treat it as the equivalent select/multiselect. It's not great, but at least it should work. >How were you imagining Choices working here? I'm envisaging treating the Choices *mostly* like in an existing select/multiselect, but with extra decoration to indicate the position in the tree for display. It would then be up to the frontend to decode that and work out a sensible way to display things as best it can. Not sure on the best way to do the decoration, hoping there'll be an available special character or similar that we could use in strings to avoid too big an upheaval here. >> 2. Is anybody actually ever using the more obscure (to me!) frontends >> (e.g. kde, editor)? Is it worth spending time there? > >Although these require manual selection, I think they do have at least >some use, and I'd rather keep them going. It shouldn't be too hard to >get full coverage, pulling in help from specialists if necessary. I guess so. -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Debconf - adding "treeselect" type(s)?
Hi!
Timo wrote:
>29.11.20 20:21 Steve McIntyre:
>> On and off, I've been hacking on tasksel for quite a while to improve
>> the UI there and add better support for things like Blends. I've made
>> some progress in my hacking, but I think I've hit a brick wall and I
>> need to change tack. :-/
>>
>> What I've ended up doing so far is hacking tasksel to give a poor
>> *approximation* of a tree-style layout: classifying some existing
>> tasks under headers and building a tree, then displaying each of the
>> nodes of the tree one level at a time via the existing debconf
>> setup. It just about works, but it's ugly as all hell and I'm not
>> happy with where I've got to. I've sunk a lot of development time into
>> this, but I don't think it's ready to fly this way. :-(
>>
>> What I *actually* need here is proper support in debconf for
>> tree-style selection. I'm thinking of adding that, adding new types
>> "treeselect" as a tree-equivalent of "select" and "multitreeselect" as
>> an extension of "multitselect". The first one may not even be needed,
>> but would be a trivial simplification of the second, so *meh*.
>
>What are the proposed semantics of this multitreeselect?
>
>If we imagine something like:
>
>[ ] a
> [ ] a/b
> [ ] a/c
>[ ] b
> [ ] b/a
>
>Would checking "a" automatically also check "a/*"?
>Is it only about UI, meaning "a/*" would be collapsed under "a"?
>Shall it be possible to check "a", but uncheck "a/b"?
Yes, I'm thinking just about UI here: my plan would be to not have
anything *selectable* at "a" - it would just toggle the display of the
"a/*" subtree in those frontends that can display such a thing. So,
yes: it would be possible to hit "a" and then not show have any of
"a/*" selected.
For simpler frontends that can't handle that kind of toggling, I'm
thinking we'd just display the tree nodes ("a", "b") as headings to
the user to group things together. Maybe we could do better - ask an
extra question at that point in the listing to show the user that
sub-tree or not.
--
Steve McIntyre, Cambridge, UK.st...@einval.com
"You can't barbecue lettuce!" -- Ellie Crane
Debconf - adding "treeselect" type(s)?
Hey folks, On and off, I've been hacking on tasksel for quite a while to improve the UI there and add better support for things like Blends. I've made some progress in my hacking, but I think I've hit a brick wall and I need to change tack. :-/ What I've ended up doing so far is hacking tasksel to give a poor *approximation* of a tree-style layout: classifying some existing tasks under headers and building a tree, then displaying each of the nodes of the tree one level at a time via the existing debconf setup. It just about works, but it's ugly as all hell and I'm not happy with where I've got to. I've sunk a lot of development time into this, but I don't think it's ready to fly this way. :-( What I *actually* need here is proper support in debconf for tree-style selection. I'm thinking of adding that, adding new types "treeselect" as a tree-equivalent of "select" and "multitreeselect" as an extension of "multitselect". The first one may not even be needed, but would be a trivial simplification of the second, so *meh*. Am I crazy to be considering this? I'm *not* sure that we need this to work in the *general* case (i.e. supporting more packages using this functionality, supporting all the panoply of Debconf frontends). However, I'd also expect that other people might disagree and see more general use cases than just tasksel. :-) AFAICS I'd need to add at least basic support for the new type(s) into all the frontends that *can* support it. So, I have a couple of questions: 1. How flexible is Debconf at coping with a frontend not including support for a type?? 2. Is anybody actually ever using the more obscure (to me!) frontends (e.g. kde, editor)? Is it worth spending time there? -- Steve McIntyre, Cambridge, UK.st...@einval.com We don't need no education. We don't need no thought control.
Re: Next attempt to add Blends to Debian installer
Hey Andreas! I hope you're keeping well! On Tue, Oct 06, 2020 at 06:08:02PM +0200, Andreas Tille wrote: >On Mon, Mar 23, 2020 at 07:16:11PM +0000, Steve McIntyre wrote: >> >Not yet, I'm afraid. A little too swamped so far, but you're near the >> >top of my TODO list. I'm hoping to get some time for development on >> >this in the next couple of months. >> >> (Overdue!) update: I've been hacking on this for a while, and I hope >> to have a prototype for testing up shortly. It works fine on my local >> system, but in a test d-i build it fails totally so I've clearly >> missed something! Debugging that now... > >I wonder whether I might have missed some information whether there >is something I could test meanwhile. I'm afraid that various higher-priority interrupts came up (new job, UEFI security work) and I got side-tracked for a while. You must be psychic - I just started picking things up again last weekend. -- Steve McIntyre, Cambridge, UK.st...@einval.com "Further comment on how I feel about IBM will appear once I've worked out whether they're being malicious or incompetent. Capital letters are forecast." Matthew Garrett, http://www.livejournal.com/users/mjg59/30675.html
Re: Preferred form of modification for binary data used in unit testing?
Hey Philipp, Philipp Hahn wrote: > >if a *previous* version of a software generated a *buggy* binary >database, that bug got fixed in a *newer* version and also some >*recovery* mechanism was added to allow reading that broken format >*once*, but there is no code the write the *broken* file again. For >*unit testing* the upstream developers added an *example* of such a >broken database to their test data. >What's the preferred form of modification for that data set? > >* Should I include a copy of the *broken code* to generate that data? >* Declare that there in no preferred form for modification, as a >"open-save"-cycle with the current code will not re-create the bit >idencial file again. >* Remove the test data because it is not DFSG conformant and hope the >Debian build will never break the recovery code. >* Include instructions on how to re-build the broken version and give >instructions on how to maybe rebuild a similar broken file. Firstly, removing the test data would be absurd - less-tested code does not serve us or our users well. If it happens to be a binary artifact that cannot be easily recreated, then explain that. The binary artifact has become the preferred form for modification once you have it. In some cases you won't be able to sensibly reproduce the artifact that causes a problem, but you keep it around to ensure test coverage. IMHO there is no issue here. -- Steve McIntyre, Cambridge, UK.st...@einval.com "You can't barbecue lettuce!" -- Ellie Crane
Re: Next attempt to add Blends to Debian installer
On Mon, Jan 06, 2020 at 04:23:08PM +, Steve McIntyre wrote: >Hey Andreas! > >On Wed, Dec 18, 2019 at 08:23:39AM +0100, Andreas Tille wrote: >> >>the first alpha of the installer of Debian 11 is out. As we talked at >>DebConf about better Blends support: Is there anything we can test >>regarding tasksel? > >Not yet, I'm afraid. A little too swamped so far, but you're near the >top of my TODO list. I'm hoping to get some time for development on >this in the next couple of months. (Overdue!) update: I've been hacking on this for a while, and I hope to have a prototype for testing up shortly. It works fine on my local system, but in a test d-i build it fails totally so I've clearly missed something! Debugging that now... -- Steve McIntyre, Cambridge, UK.st...@einval.com Getting a SCSI chain working is perfectly simple if you remember that there must be exactly three terminations: one on one end of the cable, one on the far end, and the goat, terminated over the SCSI chain with a silver-handled knife whilst burning *black* candles. --- Anthony DeBoer
Re: Y2038 - best way forward in Debian?
On Mon, Feb 17, 2020 at 09:03:22AM +0100, Wouter Verhelst wrote: >On Sat, Feb 15, 2020 at 11:41:32PM +0000, Steve McIntyre wrote: >> Hey John, >> >> John Goarzen wrote: >> >On Tue, Feb 04 2020, Steve McIntyre wrote: >> > >> >The thing that we have to remember is that an operating system is a >> >platform for running software. This problem is rather thorny, because: >> > >> >1) Some software is provided in only binary form and cannot be >> >recompiled >> >> Oh, absolutely. In that situation there's not a lot we can sensibly >> do, modulo telling people to run such things in a time-shifted VM. I'm >> more worried about making *our* software work in the future. > >This feels like a waste of effort, then. The only reason why people want >to run i386 is "multiarch, because I have this old binary that won't go >away". For all other stuff, there's amd64. Especially since RHEL doesn't >even do i386 anymore these days, so ISVs will have to compile for amd64 >if they want it to work on whatever their customers run. > >In my opinion, there are really only two viable options: > >- Throw away the i386 port, and tell people that we no longer support > it; >- Figure out a way for 32-bit binary-only programs to keep working when > they touch a time_t beyond 2038. Right, that's it for our existing i386 port then. But we do have other 32-bit ports with different ecosystems and requirements - hence why I started this thread. -- Steve McIntyre, Cambridge, UK.st...@einval.com "War does not determine who is right - only who is left." -- Bertrand Russell
Re: Y2038 - best way forward in Debian?
anth...@derobert.net wrote: ... >Lastly, 64-bit time_t has been tested widely (e.g., on amd64). That's not >perfect of >course since 32-bit archs have smaller basic integer types, but likely a lot >less work >fixing the stray "int"s than adding epochs all over the place. > >PS: Debian-devel is likely the wrong place to redesign C/POSIX functions. +1000 -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
Hey John, John Goarzen wrote: >On Tue, Feb 04 2020, Steve McIntyre wrote: > >The thing that we have to remember is that an operating system is a >platform for running software. This problem is rather thorny, because: > >1) Some software is provided in only binary form and cannot be >recompiled Oh, absolutely. In that situation there's not a lot we can sensibly do, modulo telling people to run such things in a time-shifted VM. I'm more worried about making *our* software work in the future. >2) Some software can be recompiled but makes assumptions about the size >of variables, may use int instead of time_t, and other assorted >messiness Nod. I'm sure we'll find quite a bit of that, and need to file (and maybe fix) those bugs. Hell, we're also likely to find some places where we won't be able to sensibly fix things. But it's better to know and document those places than not. >3) Some software is going to break now, due to forward-looking time >calculations, and for others, it may be fine (or nearly so) even past >2038 due to timekeeping being only ancillary to its purpose. For >instance, I have some old games that are binary-only and really don't >care what time it is. Yup. >This option #1 sounds like a significant effort (because not only would >we need two versions of libraries, but also of include files). But it >certainly passes the "correctness" test better than your option #2. Sure. It comes down to how long we think we can / want to spend on this. I'd *hope* that a lot of software *will* work correctly with a rebootstrap, but of course we know it won't be 100%. I'm concerned that we don't leave it too long to get on with fixing things - that will continue to build an ever-growing corpus of broken systems that people will need to deal with later. -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Crossgrading support (was Re: Y2038 - best way forward in Debian?)
Adam Borowski wrote: >On Wed, Feb 12, 2020 at 06:10:53PM +0000, Steve McIntyre wrote: >> >> /me ponders - this could be a fun task for a GSoC/outreachy student to >> hack on, maybe? > >Prior art: > >* my unfinished https://github.com/kilobyte/crossgrade -- does a lot of > error checking and continue-after-problem, but currently stops after > crossgrading the essential set > >* stuff linked from https://wiki.debian.org/CrossGrading Ah, cool! >Problems: > >* crossgrades fail _badly_ in a hard to recover way if packages for the > two architectures come in different versions (including binNMUs). This > hurts especially if -ports archs are involved. Yup. Probably best done on top of a stable release, by default, to avoid the worst of this, >* arch-specific or local packages are not as bad but 1. crossgrade must > know what's safe to remove, 2. what should be left unchanged (and their > recursive deps unremoved), 3. there may be non-multiarchable packages > within 1. or 2.'s dependency chain Yup. So I think it would be good to have a tool which can work through the existing state of a system up-front and analyse what's likely to work or not. >* systemd can't handle being crossgraded (I'm a systemd hater, but same was > noticed by eg. anarcat and Simon Richter). On a minimal system it may > survive but usually it starts killing daemons (including X), unmounting > stuff, choking and forcing an unclean reboot, etc. This could be worked > around by: > ⢠switching to sysvinit > ⢠booting from a different media then chrooting to crossgrade (not for >ordinary users unless it's eg. scripted from d-i) > ⢠having a package bring a grub entry that boots with >init=/usr/sbin/crossgrade to do the thing? ACK. Booting in a *simple* state is likely to be key. -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
Andrej Shadura wrote:
>On Thu, 13 Feb 2020, 10:40 YunQiang Su, wrote:
>>
>> just redefine time_t to 64bit may also cause a problem:
>>a bad designed and old network protocol which aims only target 32bit
>> system,
>>a binary data packet, may contain time_t:
>> struct {
>> int a;
>> time_t b;
>> }
>> just define time_t to 64 will break this protocol, although it is bad
>> designed.
>
>Aren't such things already broken on amd64?
Of course they are, yes. We may have cases where nobody has tested
interop between 32 and 64 bit machines, though. :-/
--
Steve McIntyre, Cambridge, UK.st...@einval.com
Armed with "Valor": "Centurion" represents quality of Discipline,
Honor, Integrity and Loyalty. Now you don't have to be a Caesar to
concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
YunQiang Su wrote:
>Ansgar 于2020年2月13日周四 下午5:29写道：
>>
>> For arm* and mips*, we mostly seem to be talking about special-purpose
>> systems where just switching to a new architecture/port doesn't seem to
>> be that much as a problem as for i386. I think rebuilding the world and
>> breaking ABI might thus be acceptable there.
>>
>> i386 seems different. I think option C above would be the only
>> realistic proposal so far to fix the time_t problem for (parts of) i386,
>> but if glibc upstream doesn't want to expose two interfaces then i386
>> will probably just break.
>>
>
>just redefine time_t to 64bit may also cause a problem:
> a bad designed and old network protocol which aims only target 32bit
> system,
> a binary data packet, may contain time_t:
>struct {
>int a;
>time_t b;
>}
>just define time_t to 64 will break this protocol, although it is bad designed.
Oh, sure. We'll find bugs like this, guaranteed.
>Currently, the major task of 32bit ports is to keep compatible with
>old system/binary.
>Should we really want to break them?
Well, that's the question. AIUI people seem to be wanting to keep i386
as-is, due to the existing ecosystem of binaries (both free and
proprietary), and I've not seen anybody really saying that i386 needs
to live beyond 2038.
armhf is different, and we want to fix it (/replace it with
armhf_) with a 64-bit clean ABI. Where do the other existing
32-bit ports sit?
* armel? anybody want to chime in?
* mipsel?
I'd like to start making decisions *soon* on what we want to do, so we
can start work. I'm *hoping* that we might be able to get a new armhf
port done and released with bullseye, but that's clearly up to the
release team to make a call on. The longer we leave things, the harder
that target will be.
--
Steve McIntyre, Cambridge, UK.st...@einval.com
Armed with "Valor": "Centurion" represents quality of Discipline,
Honor, Integrity and Loyalty. Now you don't have to be a Caesar to
concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
Florian Weimer wrote: >* Steve McIntyre: > >>>In addition if we are using a new multiarch triplet, and need to >>>rebuild the world, are going to be ABI incompatible anyway, we might >>>as well use a proper multiarch-qualified ld.so pathname that does >>>not collide with anything. >> >> Hmmm. Moving ld.so is *hard* - we were already bitten by stuff here >> when we bootstrapped armhf initially. What we didn't know then (but >> know now!) is that the final element of the path (i.e. the filename) >> must be globally unique for glibc's code to work. We can't (for >> example) just move ld-linux-armhf.so.3 to a new directory, we'd have >> to rename the file itself. (Apologies if this is stuff you already >> know - I think it's worth explaining for others too!) > >These changes also require updates to the ABI manual and upstream >patches to the toolchain. Nod. (and "yay" - been there, done that before...) >I don't think Debian's multi-arch path changes have been properly >upstreamed yet, either. Of course it does not help that key parts of >the GNU toolchain are more or less dead upstream; the lack of an >autoconf release is a real problem. ACK. :-( -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
Hey Simon, Simon McVittie wrote: >On Tue, 04 Feb 2020 at 13:14:10 +0000, Steve McIntyre wrote: >> Arnd scanned the library packages in the Debian archive and identified >> that about one third of our library packages would need rebuilding >> (and tracking) to make a (recursive) transition. > >Is a list of affected/unaffected packages available? (I know they'll >be long lists.) Depending on how high up the stack they are, the right >approach might change. I don't have the list, I'm afraid. Arnd? >Thinking about i386 specifically, since the tradeoffs for that architecture >are likely to be a little different due to the prevalence of proprietary >binaries, the use-cases I am aware of are: > >- Old hardware that doesn't implement amd64. > Breaking ABI would help here, but equally, this use-case seems likely > to go away "naturally" in the next 10? years (and for example Ubuntu is > already relegating i386 to a second-class-citizen status as a multiarch > library stack on amd64 systems, with no kernel or full bootable OS). > >- amd64-capable machines that have inherited a legacy i386 installation > because their owner doesn't want to reinstall or sidegrade to amd64. > Breaking the i386 ABI seems like it would be counterproductive here: > they'll still need to reinstall or sidegrade, and sidegrading from > i386 to i386t would probably be very crash-prone, since those will > presumably both have to be represented by ELF32 x86 binaries? > >- 32-bit Wine (to run 32-bit Windows programs). > On the Unix side, breaking ABI would maybe help. On the Windows side, > Wine needs to match whatever Microsoft does to solve this problem > (apparently time_t already defaults to 64-bit on Windows, but older > binaries will presumably use 32-bit time_t forever). Hmmm. I didn't think Microsoft used *time_t* as such, but had their own 64-bit format based on an epoch in ~1600? >- Running old proprietary or otherwise sourceless binaries (e.g. games). > Breaking ABI is counterproductive: we can't recompile the games anyway. > Some solutions for running these (e.g. the Steam Runtime used by Steam) > rely on host libraries being approximately ABI-compatible with the ones > the binaries were compiled against, because they need to fetch graphics > drivers and their dependencies from the host system, otherwise they > could not work on recent GPUs (although by 2038, with faster CPUs, we'll > hopefully be able to run 2010s games at acceptable performance with > software rendering like llvmpipe, which would sidestep this issue). > (See the slides/video from my recent FOSDEM talk for more details) ACK. I'm in agreement here. I also personally don't see the same benefit to doing a 64-bit time_t transition for i386, using any of the models we've suggested. It's a legacy ABI that we should explicitly warn people will break. Have the Steam folks considered this issue at all yet, OOC? armhf is a different kettle of fish, without the same set of legacy binaries. There are *many* legacy proprietary binaries for the architecture, but the only ones that really matter are on other platforms (Android or iOS) and really not our problem here. :-) -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
[ Skimming through this - Arnd already responded ... ] Guillem Jover wrote: >On Tue, 2020-02-04 at 13:14:10 +0000, Steve McIntyre wrote: ... >> So, we're all fine? Not so much: for our 32-bit Debian arches, we will >> need to basically rebuild the world to be 2038-safe. When we had to do >> something like this in the past, to deal with the libc5->libc6 >> transition, we had an SONAME change in libc to work with. We decided >> to append the "g" tag to the names of our library binary packages to >> signal that a library was built against libc6. We can still see some >> of the effects of this in the archive, many years later >> (e.g. zlib1g). The problem now is that we will *not* have an soname >> change here to help identify code compatibility on the 32-bit front. > >Well, I guess such a new (conditinally selectable) name could be >coordinated with glibc upstream? Say bump 32-bit ports to use libc6.1? >(We already have precedent in some ports that do not use the same >prevalent SONAME, such as libc6.1, libc0.1 and libc0.1.) But it would >indeed involve lots of work, with massive amounts of package renames. :/ Nod. :-( >>* users would *not* be able to simply upgrade from one to the >> other, due to lack of binary compatibility > >I think this conclusion stems from an incorrect premise. See below. > >>We'd need to decide exactly which of our 32-bit ports we would want >>to do this path with (probably armhf->arhmft?, maybe >>armel->armelt?, i386->i386t?. mipsel???). Due to the libc6 soname >>continuity, we will most likely *not* end up with a different >>visible ABI via the triplet and the runtime linker, so old/new >>multi-arch will be impossible. > >A new dpkg architecture must use a different triplet, as these are >required to be bijective. See âlpiaâ for a previous example of this. >(This would then make it possible to cross-grade.) ACK, that's feasible of course. It's not a *simple* upgrade, though. >In addition if we are using a new multiarch triplet, and need to >rebuild the world, are going to be ABI incompatible anyway, we might >as well use a proper multiarch-qualified ld.so pathname that does >not collide with anything. Hmmm. Moving ld.so is *hard* - we were already bitten by stuff here when we bootstrapped armhf initially. What we didn't know then (but know now!) is that the final element of the path (i.e. the filename) must be globally unique for glibc's code to work. We can't (for example) just move ld-linux-armhf.so.3 to a new directory, we'd have to rename the file itself. (Apologies if this is stuff you already know - I think it's worth explaining for others too!) >I also think we have a C option, which would be something like: > > C Do a transition equivalent to the LFS one, by either switching >libraries opportunistically to 64-bit time_t on their next SONAME >bumps, or by updating them to provide 64-bit variants of their >interfaces along their 32-bit ones, as done in glibc. Of course >the main problem here is that the LFS "transition" is not one we >should be very proud of, as it's been dragging on for a very long >time, and it's not even close to be finished⦠> > > <https://lintian.debian.org/tags/binary-file-built-without-LFS-support.html> >(Hmm there seems to be something borked with lintian.d.o, as the >general tag numbers seem extremely low. :) I don't think this helps very much though - it's the embedded copies of "time_t" all the way up the library stack that will break... -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
Florian Weimer wrote: >* Steve McIntyre: > >> The kernel is *basically* fixed now. Internally, data structures >> should now be safe. There are a small number places where 32-bit time >> is still a thing, but it's in hand. A number of syscalls, ioctls, >> etc. have needed updates for the user-kernel interface level. > >XFS is the elephant in the room, though. Oh, sure. There are a few other places that will break too, I'm sure. >> glibc is the place that needs to care about most of this. > >I'm not so sure. I really do not want glibc to parse and rewrite >ioctl arguments and return values, or ancillary socket data. Sorry - to be clear, I'm talking about in terms of the definitions of various structures and library routines that are used further up the stack. If we set our new 64-bit glibc to *always* expose 64-bit time_t and use the right 64-bit syscalls here, then the vast majority of the libraries and applications above it will just inherit the right answers by default. There will (of course!) be some other packages that will need work - we'll need to look for those too but they're much smaller targets. >> The glibc folks have taken an interesting approach. >> >> * 64-bit ABIs/architectures are already using 64-bit time_t >>throughout. The design is sane and so we should already be mostly >>safe here, modulo silly code bugs (we'll always have those!) > >With the exception of utmp/utmpx, see __WORDSIZE_TIME64_COMPAT32 in >the headers. Unfortuantely, this also affects on-disk data >structures. ACK. :-( >> * 32-bit ABIs/arches are more awkward. glibc will continue *by >>default* to use 32-bit time_t to keep compatibility with existing >>code. This will *not* be safe as we approach 2038. >> >> * 32-bit ABIs/arches *can* be told to use 64-bit time_t from glibc >>upwards, but this will of course affect the ABI. Embedded uses of >>time_t in libraries will change size, etc. This *will* be safe for >>2038. >> >> So, we're all fine? Not so much: for our 32-bit Debian arches, we will >> need to basically rebuild the world to be 2038-safe. > >The question is whether anyone wants an i386 port where this has >happened. > >My opinion (professional in this case, even) is that i386 users want >compatibility with their binaries from 1998. Otherwise they would >have rebuilt them for x86-64 by now. Under this worldview, i386 is >for backwards compatibility with existing software. Users will want >to run these old programs in a time namespace with shifted time, too. Agreed! >There is also substantial commercial interest in those old binaries >(which becomes apparent when you break glibc compatibility with them >8-/). > >> B Decide which 32-bit ports we expect to care about by 2038, and >>re-bootstrap new versions of those ports *with different >>names*. This would allow most of our developers to ignore the >>problem here (as 64-bit arches are not affected) and let a smaller >>number of people re-bootstrap with new ABIs with 64-bit time_t >>embedded. There are some downsides here: >> >>* we would end up with two versions of some ports for a short time >> - probably one release of overlap >> >>* users would *not* be able to simply upgrade from one to the >> other, due to lack of binary compatibility >> >>We *would* be able to run old and new ABIs on top of the same (new >>enough) kernel (e.g. for buildds), so a lump of this would just be >>simply building the new world and filing bugs where needed. >> >>We'd need to decide exactly which of our 32-bit ports we would want >>to do this path with (probably armhf->arhmft?, maybe >>armel->armelt?, i386->i386t?. mipsel???). Due to the libc6 soname >>continuity, we will most likely *not* end up with a different >>visible ABI via the triplet and the runtime linker, so old/new >>multi-arch will be impossible. > >Yes, it really has to be B for i386 at least. > >The other issue is that the Y2038 work has tentacles everywhere. >Porting to new architectures (whether they are 32-bit or 64-bit) will >require additional changes to those applications which make direct >system calls, in some cases even if they do not even use time_t or >struct timespec with those system calls at all. For example, new >ports will not define __NR_futex, only __NR_futex_time64. Nod. -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
Simon McVittie wrote: >On Fri, 07 Feb 2020 at 09:28:24 +0200, Wouter Verhelst wrote: >> Why not? This seems like the type of problem that SONAMEs are made for. >> What am I missing? > >SONAMEs are set by the upstream developer in their build system (building >the same source code produces the same SONAME), and are cross-distro >comparable/compatible. This makes them good for representing ABI breaks >that result from changes to the upstream source code (like deleting >a deprecated function or changing a struct's members), but bad for >representing ABI breaks that result from external factors like a toolchain >or compilation-option change. > >We didn't/couldn't change SONAMEs for the C++11 std::string transition >(g++ 5, "v5" suffix), which was a similar situation. Nod, exactly. This involves tracking all the way up the library stack, which makes it hard. We *can* go the suffix route if preferred, but that's a *lot* of effort - see proposal A... -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
Sam Hartman wrote: >Steve, you're presuming that we would not create a new soname for libc6 >on architectures where we want a new time ABI. > >That's not at all obvious to me. >It seems in the same level of drastic as the other options you are >considering. >So taking it off the table without discussion or consideration doesn't >seem like a good call. As Arnd has mentioned, it seems the (upstream) glibc maintainers are not keen to change soname. The last thing we want is to diverge from them (and hence other distros), so it's a discussion we would need to have. -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
wzss...@gmail.com wrote: >Steve McIntyre äº2020å¹´2æ4æ¥å¨äº ä¸å9:15åéï¼ ... >> A Follow a similar path to last time (rename library packages). This >>will allow us to do partial upgrades, but the cost is that a vast >>number of packages will need work to make this happen, >>*potentially* building library packages twice to allow us to >>continue both 32-bit and 64-bit time_t support forwards for a >>while. This effort will be *needed* only for the sake of our 32-bit >>ports, but would affect *everybody*. >> >> *** OR *** >> >> B Decide which 32-bit ports we expect to care about by 2038, and >>re-bootstrap new versions of those ports *with different >>names*. This would allow most of our developers to ignore the >>problem here (as 64-bit arches are not affected) and let a smaller >>number of people re-bootstrap with new ABIs with 64-bit time_t >>embedded. There are some downsides here: ... >> So, which way should we go? My own personal gut feel matches Arnd's, >> which would be route B. Some of us already have fair experience with >> bootstrapping new arches, and we could almost just crank the handle >> for most of this work. > >Is there the option C? > >Draft: > 1. define time64_t > 2. for data_struct which act as a part of ABI, define a new data_struct64 > 3. for function which act as part of ABI, define a new version func64. > 4. patch all packages to use time64_t instead of time_t step by step. > 5. set a time as deadline (2030?), and then treat all packages >haven't finished > the migration as rc-buggy. > > Since we have enough time, we can patch all packages in that period. It's possible, but... The problem here is that we have many thousands of packages to work on, glibc up through other libs to all the applications that use them. It's invasive work, and likely to take a very long time. Since it's work that would *not* be needed for 64-bit architectures, we're also likely to see push-back from some upstreams. 2030 is also too late to fix the problem - people are already starting to see issues in some applications. We want to make a difference soon: the earlier that we have fixes available, the fewer broken systems will have to be fixed / removed / replaced later. -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
Steve McIntyre wrote: >Russ Allbery wrote: >> >>If we go down this path, can we make cross-grading a supported feature for >>the next stable release? I'm sure I'm not the only one who is stuck with >>continuously-upgraded i386 hosts who has been wanting to switch but has >>been waiting until cross-grading is a little bit less scary. > >ACK - I've heard quite a few people asking for this over the last few >years. I've personally cross-graded a couple of systems (and my home >server has moved twice, i386->amd64->arm64), but it's not for the >faint-hearted. > >/me ponders - this could be a fun task for a GSoC/outreachy student to >hack on, maybe? https://wiki.debian.org/SummerOfCode2020/UnApprovedProjects/ArchitectureCrossGrade -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
Re: Y2038 - best way forward in Debian?
Russ Allbery wrote: >Ansgar writes: > >> So maybe just recommend people to move to 64-bit architectures and put >> 32-bit applications in a time namespace so they believe they are still >> in 2001 ;-) 32-bit architectures will probably still be useful in >> embedded contexts for a long time and there it might be easier to just >> change the ABI, but for a general-purpose distribution we start seeing >> more and more problems and I don't really see us supporting them as a >> full architecture in 10+ years. > >If we go down this path, can we make cross-grading a supported feature for >the next stable release? I'm sure I'm not the only one who is stuck with >continuously-upgraded i386 hosts who has been wanting to switch but has >been waiting until cross-grading is a little bit less scary. ACK - I've heard quite a few people asking for this over the last few years. I've personally cross-graded a couple of systems (and my home server has moved twice, i386->amd64->arm64), but it's not for the faint-hearted. /me ponders - this could be a fun task for a GSoC/outreachy student to hack on, maybe? -- Steve McIntyre, Cambridge, UK.st...@einval.com Armed with "Valor": "Centurion" represents quality of Discipline, Honor, Integrity and Loyalty. Now you don't have to be a Caesar to concord the digital world while feeling safe and proud.
