Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
Hi, Thank you for your answer. Le mercredi 19 août à 19h 06mn 39s (+0200), Samuel Thibault a écrit : > jhcha54008, le mer. 19 août 2020 19:04:30 +0200, a ecrit: > > Le lundi 17 août à 01h 06mn 05s (+0200), Samuel Thibault a écrit : > > > My debian-cd changes create empty files so that d-i is happy (exactly > > > like is done for -backports). > > > > I am just curious : what are the expected benefits > > of an 'unreleased' distribution on install cdrom ? > > Err, containing the content of the unreleased distribution? > > So that arch-specific packages are available, as well as patched > packages uploaded there (yes, I know, patches should be uploaded to > sid, but there's the maintainer latency there which hinders port > progression when not using unreleased as a staging area to be able to > move forward without having to wait). > > Samuel I agree that this is what the 'unreleased' distribution on debian-ports mirrors is for. But my question was : what about the install cdrom (and not the debian-ports mirrors) ? What are the expected benefits of keeping a separated 'unreleased' distribution on the cdrom ? Currently, debian-cd (when setting UNRELEASED=1) is supposed to pull all needed packages from both suites 'unstable' and 'unreleased', and to put everything in 'sid' on the built cdrom image. Are there use cases where this scheme is unsuitable ? Regards, JH Chatenet
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
On 19.08.20 19:01, jhcha54008 wrote: > Le lundi 17 août à 00h 55mn 08s (+0200), John Paul Adrian Glaubitz a écrit : >> What happens when a port does not have any packages in "unreleased"? >> >> ... to dicuss first whether enabling "unreleased" unconditionally will work >> even if a port doesn't have any package in this suite. > > Hi again, > > I doubt that an empty 'unreleased' distribution is the cause > of the problem (but an inexistent one causes an error with > --extra-suites=unreleased for sure). > > hppa is an example of architecture with no regular deb packages in > 'unreleased', only udeb. Trying : > > # debootstrap --arch=hppa --variant=minbase --foreign > --extra-suites=unreleased sid my_chroot http://ftp.de.debian.org/debian-ports > [ ... ] > : Extracting zlib1g... > 0 dtl64-seine:/home/moi/cages# echo $? > 0 > > So, it works (actually, I haven't tested in the context of the > debian-installer. I expect a similar result). > > In the paricular case of hppa, --extra-suite is definitely useless but > seems harmless. Should it be enabled ? If it works without issues (I mean if it doesn't abort during installation), then I think the unreleased suite should be included for the hppa architecture and all others by default. Specifically for hppa it could be that we need to add packages in the future - even if it's best to avoid adding packages there. Helge
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
On 8/19/20 7:08 PM, jhcha54008 wrote: > * powerpc and ppc64 : > > base package powerpc-utils(1.3.7-1) has a missing dependency : > pmac-utils > and pmac-utils(1.1.3-27+ports) is in unreleased Both powerpc and ppc64 are perfectly installable as pmac-utils is part of the installation CD. I may also modify the powerpc-utils package in the future to degrade pmac-utils to Recommends. As I said, debian-cd allows packages from "unreleased" to be included in the installation CD image. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
jhcha54008, le mer. 19 août 2020 19:04:30 +0200, a ecrit: > Le lundi 17 août à 01h 06mn 05s (+0200), Samuel Thibault a écrit : > > My debian-cd changes create empty files so that d-i is happy (exactly > > like is done for -backports). > > I am just curious : what are the expected benefits > of an 'unreleased' distribution on install cdrom ? Err, containing the content of the unreleased distribution? So that arch-specific packages are available, as well as patched packages uploaded there (yes, I know, patches should be uploaded to sid, but there's the maintainer latency there which hinders port progression when not using unreleased as a staging area to be able to move forward without having to wait). Samuel
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
On 8/19/20 6:55 PM, jhcha54008 wrote: > I think this is a misunderstanding. > > There is no 'unreleased' suite on install cdrom > (and I am still unconvinced that one would be of any help there). There are individual packages from "unrelased" on the CD image when building a CD with debian-cd because I added such a feature to debian-cd. > The option : --extra-suite=unreleased may be useful > when running debootstrap and installing from a > debian-ports mirror (ie through http or https, not > from a cdrom). This option shouldn't be used on > netinst and larger cdrom, only in netboot (and mini iso) > and businesscard. It was added like that to base-installer. Hence base-installer was run with that argument when run during a CD installation. And I already mentioned, "unreleased" is not really needed for most ports, so I think it should rather be optional than default. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
Le lundi 17 août à 10h 17mn 24s (+0200), John Paul Adrian Glaubitz a écrit : > > ... I don't quite understand why "unreleased" is supposed to be part of the > base installation. "unreleased" is not a regular package archive and all > packages in there are usually only added temporarily. > > Does Debian Hurd require packages from "unreleased" for it's base system? > > Since "unreleased" is not properly maintained due to the fact that all > packages there are built and uploaded manually, I rather prefer them not > to be enabled by default, especially not for the base system. > Hi again, At the time of writing (2020-08-19 16:00 UTC), it seems that hurd-i386, powerpc and ppc64 are uninstallable from 'unstable' but would be with option : --extra-suites=unreleased * hurd-i386 : base package bsdmainutils(12.1.7) has a missing dependency : bsdextrautils (>= 2.35.2-7) and bsdextrautils(2.35.2-7+hurd.1) is in 'unreleased' required package libgnutls30(3.6.14-2+b1) has a missing dependency : libp11-kit0 (>= 0.23.18.1) and libp11-kit0(0.23.18.1-2+hurd.2) (actually, I am unsure of the installability of rsyslog) * powerpc and ppc64 : base package powerpc-utils(1.3.7-1) has a missing dependency : pmac-utils and pmac-utils(1.1.3-27+ports) is in unreleased I hope it will help ! Regards, JH Chatenet
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
Le lundi 17 août à 01h 06mn 05s (+0200), Samuel Thibault a écrit : > > My debian-cd changes create empty files so that d-i is happy (exactly > like is done for -backports). > Hi again, I am just curious : what are the expected benefits of an 'unreleased' distribution on install cdrom ? Regards, JH Chatenet
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
Le lundi 17 août à 00h 55mn 08s (+0200), John Paul Adrian Glaubitz a écrit : > What happens when a port does not have any packages in "unreleased"? > > ... to dicuss first whether enabling "unreleased" unconditionally will work > even if a port doesn't have any package in this suite. Hi again, I doubt that an empty 'unreleased' distribution is the cause of the problem (but an inexistent one causes an error with --extra-suites=unreleased for sure). hppa is an example of architecture with no regular deb packages in 'unreleased', only udeb. Trying : # debootstrap --arch=hppa --variant=minbase --foreign --extra-suites=unreleased sid my_chroot http://ftp.de.debian.org/debian-ports [ ... ] : Extracting zlib1g... 0 dtl64-seine:/home/moi/cages# echo $? 0 So, it works (actually, I haven't tested in the context of the debian-installer. I expect a similar result). In the paricular case of hppa, --extra-suite is definitely useless but seems harmless. Should it be enabled ? Regards, JH Chatenet
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
Le lundi 17 août à 00h 38mn 15s (+0200), John Paul Adrian Glaubitz a écrit : > Hi! > > This seems to have broken debian-installer on Debian Ports. > > Installing the base system now fails with: > > Aug 16 22:34:35 debootstrap: /usr/sbin/debootstrap --components=main > --debian-installer --resolve-deps > --include=debian-ports-archive-keyring,debian-ports-archive-keyring > --extra-suites=unreleased --no-check-gpg sid /target file:///cdrom/ > Aug 16 22:34:40 base-installer: error: exiting on error > base-installer/debootstrap-failed > > Aug 16 22:34:41 main-menu[235]: WARNING **: Configuring 'bootstrap-base' > failed with error code 1 > > Aug 16 22:34:41 main-menu[235]: WARNING **: Menu item 'bootstrap-base' failed. > > Has this actually been tested to work? > > Thanks, > Adrian > > -- > .''`. John Paul Adrian Glaubitz > : :' : Debian Developer - glaub...@debian.org > `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de > `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913 Hi Adrian, I think this is a misunderstanding. There is no 'unreleased' suite on install cdrom (and I am still unconvinced that one would be of any help there). The option : --extra-suite=unreleased may be useful when running debootstrap and installing from a debian-ports mirror (ie through http or https, not from a cdrom). This option shouldn't be used on netinst and larger cdrom, only in netboot (and mini iso) and businesscard. Hence the test in the proposal [1] : if [ "$PROTOCOL" = http ] || [ "$PROTOCOL" = https ] ||[ "$PROTOCOL" = ftp ]; then I still hope it will help to find a solution. Regards, JH Chatenet [1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=964248#134
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
I have reverted the change for now. But we need to discuss among various porters, I have really seen desire for enabling the unreleased suite in ports other than the hurd port. Samuel
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
Samuel Thibault, le lun. 17 août 2020 10:28:54 +0200, a ecrit:
> John Paul Adrian Glaubitz, le lun. 17 août 2020 10:17:24 +0200, a ecrit:
> > On 8/17/20 1:06 AM, Samuel Thibault wrote:
> > >> I checked the patch and it enables "unreleased" unconditionally. I'm
> > >> not sure whether that's a good idea.
> > >
> > > I don't think we really have a better choice: writing yet another arch
> > > list looks dirty, and trying whether the suite is available is probably
> > > quite complex to implement.
> >
> > ... I don't quite understand why "unreleased" is supposed to be part of the
> > base installation.
>
> For ports architectures, it often is, because the sid archive does not
> contain various packages that ports need.
>
> > "unreleased" is not a regular package archive and all
> > packages in there are usually only added temporarily.
>
> AFAIK that's not what ports' daily life really is.
See also
9bb019085356 ('Enable the "unreleased" suite for ports architectures.')
in the apt-setup package.
Samuel
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
Hello Samuel! On 8/17/20 1:06 AM, Samuel Thibault wrote: > John Paul Adrian Glaubitz, le lun. 17 août 2020 00:55:08 +0200, a ecrit: >>> and then a couple other changes, I'm waiting for the above to be merged >>> before I can submit them. >> >> What happens when a port does not have any packages in "unreleased"? > > My debian-cd changes create empty files so that d-i is happy (exactly > like is done for -backports). OK, that's a possible workaround. However, ... >> I checked the patch and it enables "unreleased" unconditionally. I'm >> not sure whether that's a good idea. > > I don't think we really have a better choice: writing yet another arch > list looks dirty, and trying whether the suite is available is probably > quite complex to implement. ... I don't quite understand why "unreleased" is supposed to be part of the base installation. "unreleased" is not a regular package archive and all packages in there are usually only added temporarily. Does Debian Hurd require packages from "unreleased" for it's base system? Since "unreleased" is not properly maintained due to the fact that all packages there are built and uploaded manually, I rather prefer them not to be enabled by default, especially not for the base system. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
John Paul Adrian Glaubitz, le lun. 17 août 2020 10:36:43 +0200, a ecrit: > On 8/17/20 10:28 AM, Samuel Thibault wrote: > > John Paul Adrian Glaubitz, le lun. 17 août 2020 10:17:24 +0200, a ecrit: > >> "unreleased" is not a regular package archive and all > >> packages in there are usually only added temporarily. > > > > AFAIK that's not what ports' daily life really is. > > Yes, it is. I actually think I have a good overview over this. Then opinions differ and we need to discuss about it among porters because that's really not something I have seen specific to the Hurd, I have seen other porters wanting this. > >> Does Debian Hurd require packages from "unreleased" for it's base system? > > > > The Hurd needs netdde from there. It happens that the hurd package > > itself is still in sid but that's only by luck because there's the > > arch:all hurd-doc package. > > And that package is required for the base-system and cannot be installed > later? netdde contains the network drivers, so yes, it is pretty much part of the base system. Strictly speaking it could be installed "later" at the task installation stage, but people are supposed to be able to skip that step and still having a system that works. > > But that's far from being specific to the Hurd. Various ports have > > various requirements such as bootloaders. > > These are always shipped on the installation CD. It also affects m68k > only as alpha, hppa, powerpc, ppc64, sh4 and sparc64 use either GRUB or have > their bootloaders in unstable. > > Does Hurd not use GRUB? It does. I'm just here mentioning one of the cases that I have seen mentioned from other ports. I don't know the details there, but there are probably others like firmware packages, etc. Yes probably these can also be shipped by hand on debian-cd, but that looks to me like duplicate work while we can simply enable unreleased. > >> Since "unreleased" is not properly maintained due to the fact that all > >> packages there are built and uploaded manually, I rather prefer them not > >> to be enabled by default, especially not for the base system. > > > > They are uploaded manually by the porters themselves. They are > > responsible for what they upload there, so I don't see why it would > > be harmful? On the contrary, it provides flexibility for fixing > > installability without having to wait for sid to have fixes uploaded. > > The problem with "unreleased" is that there are no binNMUs and no proper > maintenance by a package maintainer. I always try to avoid using "unreleased" > whenever possible. Sure. But that's very convenient to keep things installable without having to wait for sid to catch up with whatever issue is encountered by the port. Otherwise the sid upload latency makes the port uninstallable for long periods of time, possibly all the time when these periods overlap. Samuel
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
John Paul Adrian Glaubitz, le lun. 17 août 2020 00:55:08 +0200, a ecrit: > > and then a couple other changes, I'm waiting for the above to be merged > > before I can submit them. > > What happens when a port does not have any packages in "unreleased"? My debian-cd changes create empty files so that d-i is happy (exactly like is done for -backports). > I checked the patch and it enables "unreleased" unconditionally. I'm > not sure whether that's a good idea. I don't think we really have a better choice: writing yet another arch list looks dirty, and trying whether the suite is available is probably quite complex to implement. Samuel
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
John Paul Adrian Glaubitz, le lun. 17 août 2020 10:17:24 +0200, a ecrit: > On 8/17/20 1:06 AM, Samuel Thibault wrote: > >> I checked the patch and it enables "unreleased" unconditionally. I'm > >> not sure whether that's a good idea. > > > > I don't think we really have a better choice: writing yet another arch > > list looks dirty, and trying whether the suite is available is probably > > quite complex to implement. > > ... I don't quite understand why "unreleased" is supposed to be part of the > base installation. For ports architectures, it often is, because the sid archive does not contain various packages that ports need. > "unreleased" is not a regular package archive and all > packages in there are usually only added temporarily. AFAIK that's not what ports' daily life really is. > Does Debian Hurd require packages from "unreleased" for it's base system? The Hurd needs netdde from there. It happens that the hurd package itself is still in sid but that's only by luck because there's the arch:all hurd-doc package. But that's far from being specific to the Hurd. Various ports have various requirements such as bootloaders. > Since "unreleased" is not properly maintained due to the fact that all > packages there are built and uploaded manually, I rather prefer them not > to be enabled by default, especially not for the base system. They are uploaded manually by the porters themselves. They are responsible for what they upload there, so I don't see why it would be harmful? On the contrary, it provides flexibility for fixing installability without having to wait for sid to have fixes uploaded. Samuel
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
Hello! On 8/17/20 10:28 AM, Samuel Thibault wrote: > John Paul Adrian Glaubitz, le lun. 17 août 2020 10:17:24 +0200, a ecrit: >> On 8/17/20 1:06 AM, Samuel Thibault wrote: I checked the patch and it enables "unreleased" unconditionally. I'm not sure whether that's a good idea. >>> >>> I don't think we really have a better choice: writing yet another arch >>> list looks dirty, and trying whether the suite is available is probably >>> quite complex to implement. >> >> ... I don't quite understand why "unreleased" is supposed to be part of the >> base installation. > > For ports architectures, it often is, because the sid archive does not > contain various packages that ports need. Well, that's not correct since the only additional packages needed are some bootloaders (emile for m68k, for example) and those are shipped on the installation CD. In the case of Debian Hurd, the case here is a package temporarily being sourced from "unreleased" because the version from unstable FTBFS on unstable. >> "unreleased" is not a regular package archive and all >> packages in there are usually only added temporarily. > > AFAIK that's not what ports' daily life really is. Yes, it is. I actually think I have a good overview over this. >> Does Debian Hurd require packages from "unreleased" for it's base system? > > The Hurd needs netdde from there. It happens that the hurd package > itself is still in sid but that's only by luck because there's the > arch:all hurd-doc package. And that package is required for the base-system and cannot be installed later? > But that's far from being specific to the Hurd. Various ports have > various requirements such as bootloaders. These are always shipped on the installation CD. It also affects m68k only as alpha, hppa, powerpc, ppc64, sh4 and sparc64 use either GRUB or have their bootloaders in unstable. Does Hurd not use GRUB? >> Since "unreleased" is not properly maintained due to the fact that all >> packages there are built and uploaded manually, I rather prefer them not >> to be enabled by default, especially not for the base system. > > They are uploaded manually by the porters themselves. They are > responsible for what they upload there, so I don't see why it would > be harmful? On the contrary, it provides flexibility for fixing > installability without having to wait for sid to have fixes uploaded. The problem with "unreleased" is that there are no binNMUs and no proper maintenance by a package maintainer. I always try to avoid using "unreleased" whenever possible. -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
Hi! On 8/17/20 12:49 AM, Samuel Thibault wrote: > AIUI it was only tested on the mini iso. Well, then it should not have been merged :(. > I noticed the issue, and worked on it, there is: > > https://salsa.debian.org/images-team/debian-cd/-/merge_requests/7 > > and then a couple other changes, I'm waiting for the above to be merged > before I can submit them. What happens when a port does not have any packages in "unreleased"? I checked the patch and it enables "unreleased" unconditionally. I'm not sure whether that's a good idea. FWIW, I have commit access to debian-cd, so I can merge your changes but I would like to discuss first whether enabling "unreleased" unconditionally will work even if a port doesn't have any package in this suite. Thanks, Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
Hello, John Paul Adrian Glaubitz, le lun. 17 août 2020 00:38:15 +0200, a ecrit: > This seems to have broken debian-installer on Debian Ports. > > Installing the base system now fails with: > > Aug 16 22:34:35 debootstrap: /usr/sbin/debootstrap --components=main > --debian-installer --resolve-deps > --include=debian-ports-archive-keyring,debian-ports-archive-keyring > --extra-suites=unreleased --no-check-gpg sid /target file:///cdrom/ > Aug 16 22:34:40 base-installer: error: exiting on error > base-installer/debootstrap-failed > > Aug 16 22:34:41 main-menu[235]: WARNING **: Configuring 'bootstrap-base' > failed with error code 1 > > Aug 16 22:34:41 main-menu[235]: WARNING **: Menu item 'bootstrap-base' failed. > > Has this actually been tested to work? AIUI it was only tested on the mini iso. I noticed the issue, and worked on it, there is: https://salsa.debian.org/images-team/debian-cd/-/merge_requests/7 and then a couple other changes, I'm waiting for the above to be merged before I can submit them. Samuel
Bug#964248: marked as done (base-installer: Pass "--extra-suites=unreleased" to debootstrap on ports arches)
Hi! This seems to have broken debian-installer on Debian Ports. Installing the base system now fails with: Aug 16 22:34:35 debootstrap: /usr/sbin/debootstrap --components=main --debian-installer --resolve-deps --include=debian-ports-archive-keyring,debian-ports-archive-keyring --extra-suites=unreleased --no-check-gpg sid /target file:///cdrom/ Aug 16 22:34:40 base-installer: error: exiting on error base-installer/debootstrap-failed Aug 16 22:34:41 main-menu[235]: WARNING **: Configuring 'bootstrap-base' failed with error code 1 Aug 16 22:34:41 main-menu[235]: WARNING **: Menu item 'bootstrap-base' failed. Has this actually been tested to work? Thanks, Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
