Bug#985957: usrmerge has 47.3MB of dependencies
Now that we have the usr-is-merged metapackage, that debootstrap/mmdebstrap install, can we close this? -- Kind regards, Luca Boccassi signature.asc Description: This is a digitally signed message part
Bug#985957: Bug#987615: Bug#985957: usrmerge has 47.3MB of dependencies
On Sun, 24 Oct 2021 17:12:40 +0100, Niko Tyni wrote: > The libnumber-compare-perl and libtext-glob-perl modifications are trivial > (override dh_perl to dh_perl -d). I suggest modifying at least those > two packages instead of bundling them. Both packages uploaded with the suggested 'dh_perl -d' change. Cheers, gregor -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- signature.asc Description: Digital Signature
Bug#987615: Bug#985957: usrmerge has 47.3MB of dependencies
On Thu, Oct 07, 2021 at 03:59:11AM +0200, Michael Biebl wrote: > On Mon, 16 Aug 2021 07:46:49 +0100 Niko Tyni wrote: > > On Mon, Aug 16, 2021 at 12:11:26AM +0200, Marco d'Itri wrote: > > > > > I see no reason to move the usrmerge dependencies to perl-base: usrmerge > > > is supposed to be installed, run and then removed again. > > > If something is moved to perl-base then it will use space on every > > > Debian system. > > > > My suggestion in #987615 (which I also copied to usrmerge@pdo, being > > unaware of #985957) is to introduce separate packages of the usrmerge > > dependencies, and limit their dependencies to perl-base only. This does > > not grow perl-base and can be phased out later, so it does not impose > > a permanent cost on everybody. > > Seems to me, that simply vendoring the necessary perl modules in the > usrmerge package would be a better, simpler and more lightweight approach to > packaging those all invidually as separate packages. Yeah, you're probably right, particularly as I haven't got around to doing much about this so far. I'm fine with that approach FWIW. I did try out the separate package plan at home at one point, and got things working with seven modified or introduced packages (well, eight if you count the usrmerge package too.) Here's some notes, maybe they will save somebody some work even if this is not the chosen path. There's two branches to the usrmerge dependencies: libautodie-perl (would need a separate package) libif-perl (would need a separate package) libtie-refhash-perl (would need a separate package) libfile-find-rule-perl (would need dependency modifications) libfile-find-perl (would need a separate package) libnumber-compare-perl (would need dependency modifications) libtext-glob-perl (would need dependency modifications) Happily all these modules are pure Perl, which makes things a bit easier. The libnumber-compare-perl and libtext-glob-perl modifications are trivial (override dh_perl to dh_perl -d). I suggest modifying at least those two packages instead of bundling them. I'm copying the debian-perl list for some visibility. -- Niko
Bug#985957: usrmerge has 47.3MB of dependencies
On Mon, 16 Aug 2021 07:46:49 +0100 Niko Tyni wrote: > On Mon, Aug 16, 2021 at 12:11:26AM +0200, Marco d'Itri wrote: > > > I see no reason to move the usrmerge dependencies to perl-base: usrmerge > > is supposed to be installed, run and then removed again. > > If something is moved to perl-base then it will use space on every > > Debian system. > > My suggestion in #987615 (which I also copied to usrmerge@pdo, being > unaware of #985957) is to introduce separate packages of the usrmerge > dependencies, and limit their dependencies to perl-base only. This does > not grow perl-base and can be phased out later, so it does not impose > a permanent cost on everybody. Seems to me, that simply vendoring the necessary perl modules in the usrmerge package would be a better, simpler and more lightweight approach to packaging those all invidually as separate packages. Regards, Michael signature.asc Description: This is a digitally signed message part
Bug#985957: usrmerge has 47.3MB of dependencies
On Mon, Aug 16, 2021 at 12:11:26AM +0200, Marco d'Itri wrote: > I see no reason to move the usrmerge dependencies to perl-base: usrmerge > is supposed to be installed, run and then removed again. > If something is moved to perl-base then it will use space on every > Debian system. My suggestion in #987615 (which I also copied to usrmerge@pdo, being unaware of #985957) is to introduce separate packages of the usrmerge dependencies, and limit their dependencies to perl-base only. This does not grow perl-base and can be phased out later, so it does not impose a permanent cost on everybody. -- Niko Tyni nt...@debian.org
Bug#985957: usrmerge has 47.3MB of dependencies
On Aug 15, Niko Tyni wrote: > > > I think it would be a fair request to ask for usrmerge's dependencies > > > to be included there if we install usrmerge by default on upgrades to > > > covert existing systems. The dependencies could probably be dropped > > > again for bookworm+1 (with perl-base having `Breaks: usrmerge (<< X)` > > > where `X` is the version of usrmerge depending on perl instead of perl- > > > base again). > Just a note that this #985957 is also #987615 (cc'd) on the perl side. > > Now that bookworm development is open I'll see what we can do there. I see no reason to move the usrmerge dependencies to perl-base: usrmerge is supposed to be installed, run and then removed again. If something is moved to perl-base then it will use space on every Debian system. usrmerge being installed DOES NOT mean that the system has been converted to merged-/usr, so there is no reason at all to keep it around after it has been used. -- ciao, Marco signature.asc Description: PGP signature
Bug#987615: Bug#985957: usrmerge has 47.3MB of dependencies
On Tue, Apr 06, 2021 at 02:17:00PM +0100, Dimitri John Ledkov wrote: > On Tue, 6 Apr 2021 at 14:04, Ansgar wrote: > > > > On Fri, 26 Mar 2021 22:12:33 + Dimitri John Ledkov wrote: > > > I don't know what is the correct process to follow here. For example, > > > could the 5.32 things be promoted from modules to perl-base? > > > > What gets included in the (essential) perl-base package is decided by > > Debian. > > > > I think it would be a fair request to ask for usrmerge's dependencies > > to be included there if we install usrmerge by default on upgrades to > > covert existing systems. The dependencies could probably be dropped > > again for bookworm+1 (with perl-base having `Breaks: usrmerge (<< X)` > > where `X` is the version of usrmerge depending on perl instead of perl- > > base again). Just a note that this #985957 is also #987615 (cc'd) on the perl side. Now that bookworm development is open I'll see what we can do there. -- Niko
Bug#985957: usrmerge has 47.3MB of dependencies
On Tue, 6 Apr 2021 at 14:04, Ansgar wrote: > > On Fri, 26 Mar 2021 22:12:33 + Dimitri John Ledkov wrote: > > I don't know what is the correct process to follow here. For example, > > could the 5.32 things be promoted from modules to perl-base? > > What gets included in the (essential) perl-base package is decided by > Debian. > > I think it would be a fair request to ask for usrmerge's dependencies > to be included there if we install usrmerge by default on upgrades to > covert existing systems. The dependencies could probably be dropped > again for bookworm+1 (with perl-base having `Breaks: usrmerge (<< X)` > where `X` is the version of usrmerge depending on perl instead of perl- > base again). > > For this to work, usrmerge would need to stop using "libfile-find-rule- > perl" as well as that depends on the full "perl" package. All of the above is very interesting, especially in the context of how Debian should approach installing / converting to usrmerge by default on upgrades. In Ubuntu, I have left usrmerge package as is. Instead of making ubuntu-minimal metapackage depend on usrmerge, I made it recommend it only. This way on upgrades, it will be installed by default. And it is also allowed to remove it after the install. And I have adjusted Ubuntu's debootstrap to always create things with merged usr, without installing usrmerge package. The remaining piece is to remove usrmerge, if conversion was successful. I am pondering if `usrmerged` package might still be a good idea, which is an empty package that fails to install if the system is not usrmerged. That way essential packages can predepend on that, once split-usr systems are no longer supported. -- Regards, Dimitri.
Bug#985957: usrmerge has 47.3MB of dependencies
On Fri, 26 Mar 2021 22:12:33 + Dimitri John Ledkov wrote: > I don't know what is the correct process to follow here. For example, > could the 5.32 things be promoted from modules to perl-base? What gets included in the (essential) perl-base package is decided by Debian. I think it would be a fair request to ask for usrmerge's dependencies to be included there if we install usrmerge by default on upgrades to covert existing systems. The dependencies could probably be dropped again for bookworm+1 (with perl-base having `Breaks: usrmerge (<< X)` where `X` is the version of usrmerge depending on perl instead of perl- base again). For this to work, usrmerge would need to stop using "libfile-find-rule- perl" as well as that depends on the full "perl" package. Ansgar
Bug#985957: usrmerge has 47.3MB of dependencies
On Fri, 26 Mar 2021 at 20:58, Marco d'Itri wrote: > > On Mar 26, Dimitri John Ledkov wrote: > > > Is it possible to rewrite the code to only use perl-base? without the > > full perl? > Probably it would not be too hard to reimplement what File::Find::Rule > does and remove these dependencies. Maybe by running find(1). > It actually is kind of nice, and I don't think that's the largest culprit. > > Or maybe even pure dash or C? > Hard to believe. > I mean a boy can dream right?! =) > > Can we vendor the needed perl bits, and then remove them from disk if > > the conversion was successful? > I am not sure how much space this would save, do you want to try? Add: > > use lib "/usr/lib/usrmerge/lib/"; > > and then start copying the dependencies to the directory until it > works... (Also, there is App::FatPacker.) > > I do not know about Ubuntu, but I do not think that a self-modifying > package would be be following Debian policy. > I ran strace to try to figure out which files are used from perl-modules, which are not from perl-base. I think it's just this: openat(AT_FDCWD, "/usr/share/perl/5.32/Fatal.pm", O_RDONLY|O_CLOEXEC) openat(AT_FDCWD, "/usr/share/perl/5.32/File/Find.pm", O_RDONLY|O_CLOEXEC) openat(AT_FDCWD, "/usr/share/perl/5.32/Tie/RefHash.pm", O_RDONLY|O_CLOEXEC) openat(AT_FDCWD, "/usr/share/perl/5.32/autodie.pm", O_RDONLY|O_CLOEXEC) openat(AT_FDCWD, "/usr/share/perl/5.32/autodie/Scope/Guard.pm", O_RDONLY|O_CLOEXEC) openat(AT_FDCWD, "/usr/share/perl/5.32/autodie/Scope/GuardStack.pm", O_RDONLY|O_CLOEXEC) openat(AT_FDCWD, "/usr/share/perl/5.32/autodie/Util.pm", O_RDONLY|O_CLOEXEC) openat(AT_FDCWD, "/usr/share/perl/5.32/if.pm", O_RDONLY|O_CLOEXEC) openat(AT_FDCWD, "/usr/share/perl5/File/Find/Rule.pm", O_RDONLY|O_CLOEXEC) openat(AT_FDCWD, "/usr/share/perl5/Number/Compare.pm", O_RDONLY|O_CLOEXEC) openat(AT_FDCWD, "/usr/share/perl5/Text/Glob.pm", O_RDONLY|O_CLOEXEC) I don't know what is the correct process to follow here. For example, could the 5.32 things be promoted from modules to perl-base? Not sure why "if.pm" is used. I will see if I can vendor all of the above and thus make usrmerge be installable with just perl-base. Probably as an Ubuntu only change, cause yeah vendoring things like that sounds like embedded copies of code against the Debian Policy. > > Can we add a usrmerge-completed package, which is empty, but ensures > > in preinst that the system is usrmerged? That way in minimal > > containers said package will be preinstalled by default > > satifying/providing usrmerge. > Sure: since you apparently have a design in mind, could you provide > a rough patch? > Which packages do you think would depend on it? > In Ubuntu we use metapackages, hence in Ubuntu to ensure that all systems upon upgrade are converted. I wanted to make ubuntu-minimal to depend on usrmerge. However, that exploded the size of the minimal tarballs/containers with just apt & dpkg and nothing else. I was thinking to have a package called: usrmerged Which Conflicts/Replaces/Provides: usrmerge, is empty, has no deps, and in the preinst tests that the system is usrmerged. That way ubuntu-minimal would be able to depend on usrmerge | usrmerged => with fresh installs having no extra deps, whilst systems that try to upgrade will eventually install usrmerge. > > At the moment, I'm trying to make usrmerge required such that systems > > that upgrade get it installed, but it is becomming problematic on the > > minimal containers. Especially, since after installation its no longer > > needed. > I am all for improving Debian and Ubuntu to make them optimal for > building tiny containers, but I am not sure that it is a good idea to > do this by spending time adding complexity and bugs to usrmerge which is > not even supposed to be installed on them since it is only needed for > the one time conversion. Yeah that. I'm now experimenting with makeing ubuntu-minimal recommend usrmerge; such that it gets installed on upgrades, but not on fresh installs (which are usrmerged anyway). Maybe that will be good enough and will then require no minimization surgery to usrmerge. -- Regards, Dimitri.
Bug#985957: usrmerge has 47.3MB of dependencies
On Mar 26, Dimitri John Ledkov wrote: > Is it possible to rewrite the code to only use perl-base? without the > full perl? Probably it would not be too hard to reimplement what File::Find::Rule does and remove these dependencies. Maybe by running find(1). > Or maybe even pure dash or C? Hard to believe. > Can we vendor the needed perl bits, and then remove them from disk if > the conversion was successful? I am not sure how much space this would save, do you want to try? Add: use lib "/usr/lib/usrmerge/lib/"; and then start copying the dependencies to the directory until it works... (Also, there is App::FatPacker.) I do not know about Ubuntu, but I do not think that a self-modifying package would be be following Debian policy. > Can we add a usrmerge-completed package, which is empty, but ensures > in preinst that the system is usrmerged? That way in minimal > containers said package will be preinstalled by default > satifying/providing usrmerge. Sure: since you apparently have a design in mind, could you provide a rough patch? Which packages do you think would depend on it? > At the moment, I'm trying to make usrmerge required such that systems > that upgrade get it installed, but it is becomming problematic on the > minimal containers. Especially, since after installation its no longer > needed. I am all for improving Debian and Ubuntu to make them optimal for building tiny containers, but I am not sure that it is a good idea to do this by spending time adding complexity and bugs to usrmerge which is not even supposed to be installed on them since it is only needed for the one time conversion. -- ciao, Marco signature.asc Description: PGP signature
Bug#985957: usrmerge has 47.3MB of dependencies
Package: usrmerge Version: 23 Severity: important Dear Maintainer, In minimal chroot/container scenarios usrmerge has a huge amount of dependencies, doubling the total size of the container. # apt install usrmerge --no-install-recommends Reading package lists... Done Building dependency tree... Done Reading state information... Done The following additional packages will be installed: libfile-find-rule-perl libnumber-compare-perl libperl5.32 libtext-glob-perl perl perl-modules-5.32 Suggested packages: perl-doc libterm-readline-gnu-perl | libterm-readline-perl-perl make libtap-harness-archive-perl Recommended packages: netbase The following NEW packages will be installed: libfile-find-rule-perl libnumber-compare-perl libperl5.32 libtext-glob-perl perl perl-modules-5.32 usrmerge 0 upgraded, 7 newly installed, 0 to remove and 0 not upgraded. Need to get 7085 kB of archives. After this operation, 47.3 MB of additional disk space will be used. That is not acceptable for the tiny containers. Is it possible to rewrite the code to only use perl-base? without the full perl? Or maybe even pure dash or C? Can we vendor the needed perl bits, and then remove them from disk if the conversion was successful? Can we add a usrmerge-completed package, which is empty, but ensures in preinst that the system is usrmerged? That way in minimal containers said package will be preinstalled by default satifying/providing usrmerge. At the moment, I'm trying to make usrmerge required such that systems that upgrade get it installed, but it is becomming problematic on the minimal containers. Especially, since after installation its no longer needed. Regards, Dimitri.
