Re: Supporting LZ4 as initramfs compressor
On Wed, 5 Jun 2019 at 19:59, Steve Langasek wrote: > > Hi Dimitri, > > One point here: > > On Wed, Jun 05, 2019 at 01:15:48PM +0100, Dimitri John Ledkov wrote: > > - lz4 size weight over gzip is marginal (14%) but imho worth the > > improved boot time & initrd creation time > > A 14% increase in initramfs size is NOT marginal. Since there are (and will > always be) various scenarios which require a separate /boot partition, we > have over several cycles been contending with how to ensure a clean upgrade > as kernel+initramfs sizes increase over time and push up against the space > limits of the boot partitions that have historically been created. > > If you are making a change that increases the size of initramfses by 14% > across the board, you must also: > > - update the ubuntu-release-upgrader code to take this into account so >users don't run out of space in the middle of the upgrade >- possibly in a way that is smart enough to know if a user is already > configuring initramfs-tools to use a non-default compressor, to avoid > false-positives > - check that the minimum size requirements for /boot that are encoded in >the installers are still adequate, or if they need updating (and if they >need updating, update this back to 18.04) > - document this issue in the release notes agree. -- Regards, Dimitri. -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Add "dis_ucode_ldr" to linux boot options for Recovery Mode
On 2019-06-05 20:00:54, Jeroen Bobbeldijk wrote: > Thanks. I'll open a report for grub2. > According to the docs, dis_ucode_ldr is x86 only > (arch/x86/kernel/cpu/microcode/core.c), so my guess is that it will be > ignored for other architectures. According to the kernel code, > dis_ucode_ldr works for both AMD and Intel. Agreed on all the above. For anyone else following along, Jeroen opened the following bug report: https://launchpad.net/bugs/1831789 > I have some time tomorrow on test this on a ARM device. After taking a look at the grub2 code, it looks like it'll be easy enough to only inject the dis_ucode_ldr option on x86. There's no need to test that option on other architectures at this time. Thanks for the offer, though. Tyler > > Jeroen > > On Wed, Jun 5, 2019 at 7:49 PM Tyler Hicks wrote: > > > On 2019-06-05 10:03:06, Jeroen Bobbeldijk wrote: > > > Hi, > > > > > > Due to some recent problems with a microcode update which broke booting > > of > > > some laptops I suggest adding "dis_ucode_ldr" to recovery mode boot > > options > > > to allow booting after a bad microcode update. > > > > > > See the following topics that describe the problems: > > > - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1829620 > > > - > > https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files/issues/1 > > > > > > When such problems occur it's very hard to see that the microcode loading > > > is the issue, the booting just hangs at the purple screen, even removing > > > "quiet splash" will not show the error. > > > > I really like this idea. Thanks for mentioning it. > > > > Please open a bug report here: > > > > https://bugs.launchpad.net/ubuntu/+source/grub2/+filebug > > > > We'll need to double check that dis_ucode_ldr works as expected across > > all architectures. Otherwise, I can't think of any other technical > > blockers to doing this. > > > > Tyler > > > > > > > > Kind regards, > > > > > > Jeroen > > > > > -- > > > ubuntu-devel mailing list > > > ubuntu-devel@lists.ubuntu.com > > > Modify settings or unsubscribe at: > > https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel > > > > -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Supporting LZ4 as initramfs compressor
Hi Dimitri, One point here: On Wed, Jun 05, 2019 at 01:15:48PM +0100, Dimitri John Ledkov wrote: > - lz4 size weight over gzip is marginal (14%) but imho worth the > improved boot time & initrd creation time A 14% increase in initramfs size is NOT marginal. Since there are (and will always be) various scenarios which require a separate /boot partition, we have over several cycles been contending with how to ensure a clean upgrade as kernel+initramfs sizes increase over time and push up against the space limits of the boot partitions that have historically been created. If you are making a change that increases the size of initramfses by 14% across the board, you must also: - update the ubuntu-release-upgrader code to take this into account so users don't run out of space in the middle of the upgrade - possibly in a way that is smart enough to know if a user is already configuring initramfs-tools to use a non-default compressor, to avoid false-positives - check that the minimum size requirements for /boot that are encoded in the installers are still adequate, or if they need updating (and if they need updating, update this back to 18.04) - document this issue in the release notes Thanks, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Add "dis_ucode_ldr" to linux boot options for Recovery Mode
On 2019-06-05 10:03:06, Jeroen Bobbeldijk wrote: > Hi, > > Due to some recent problems with a microcode update which broke booting of > some laptops I suggest adding "dis_ucode_ldr" to recovery mode boot options > to allow booting after a bad microcode update. > > See the following topics that describe the problems: > - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1829620 > - > https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files/issues/1 > > When such problems occur it's very hard to see that the microcode loading > is the issue, the booting just hangs at the purple screen, even removing > "quiet splash" will not show the error. I really like this idea. Thanks for mentioning it. Please open a bug report here: https://bugs.launchpad.net/ubuntu/+source/grub2/+filebug We'll need to double check that dis_ucode_ldr works as expected across all architectures. Otherwise, I can't think of any other technical blockers to doing this. Tyler > > Kind regards, > > Jeroen > -- > ubuntu-devel mailing list > ubuntu-devel@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Supporting LZ4 as initramfs compressor
Hi Dimitri, Thanks for keeping this thread alive. On Wed, Jun 5, 2019 at 2:16 PM Dimitri John Ledkov wrote: > > On Fri, 31 May 2019 at 09:13, Seth Arnold wrote: > > > > On Thu, May 30, 2019 at 11:46:57AM +0100, Dimitri John Ledkov wrote: > > > As if lz4 kernel & xz initrd would yield the fastest boot time? That > > > > I'm lacking some context here, but I think building the initrds is already > > too slow and I'm afraid xz on initrd rebuilds would be significantly > > worse than lz4 or zstd or even low-compression levels of zlib. > > > > Boot times are important but every now and then people do run apt upgrade > > by hand and waiting forever to rebuild an initrd that may or may not be > > used is pretty tedious. xz is great if the really slow compress times will > > be made up for by better transfers or disk savings or similar. > > > > I've tried to dig up past benchmarks in private discussions, public > discussions and kernel mailing list. > > Zstd patches have not made it into the upstream kernel yet. > > As used by mkinitramfs: > - lz4 is faster to compress than gzip > - lz4 is blazingly fast to decompress > - lzma is dog slow to compress and decompress, but is tiny > - lz4 size weight over gzip is marginal (14%) but imho worth the > improved boot time & initrd creation time > - xz is potentially even slower and even smaller than lzma > > In places where size is an absolute premium (tiny embedded iot > devices) and performance is irrelevant, xz or lzma should be used. > > In all other places, our performance profile is in favor of lz4. > > Imho that includes the kernel image itself, thus we should consider switching: > - initramfs tools to default to lz4 > - livecd-rootfs to default to lz4 > - kernels to compress kernel image with lz4 > - grub to include lz4 support Please also note that it is not just the compressor that should be tweaked but the parameters, too. Initramfs-tools uses lz -9 because it provided the best space - speed tradeoff, but comparisons testing the kernel compressors were using the defaults which were not -9. IMO lz4 (-9) would be the best choice for kernels, too, but to see more accurate measurements it may be a good idea to run tests again with setting compression level. Cheers, Balint > > I shall proceed with changing the defaults on the above to improve our > responsiveness experience on installer, cloud, core and classic > devices. > If our firstboot & subsequent boot speed degrades or disk space > becomes a concern, we can look into tweaking these changes further. > > -- > Regards, > > Dimitri. > > -- > ubuntu-devel mailing list > ubuntu-devel@lists.ubuntu.com > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel -- Balint Reczey Ubuntu & Debian Developer -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: Supporting LZ4 as initramfs compressor
On Fri, 31 May 2019 at 09:13, Seth Arnold wrote: > > On Thu, May 30, 2019 at 11:46:57AM +0100, Dimitri John Ledkov wrote: > > As if lz4 kernel & xz initrd would yield the fastest boot time? That > > I'm lacking some context here, but I think building the initrds is already > too slow and I'm afraid xz on initrd rebuilds would be significantly > worse than lz4 or zstd or even low-compression levels of zlib. > > Boot times are important but every now and then people do run apt upgrade > by hand and waiting forever to rebuild an initrd that may or may not be > used is pretty tedious. xz is great if the really slow compress times will > be made up for by better transfers or disk savings or similar. > I've tried to dig up past benchmarks in private discussions, public discussions and kernel mailing list. Zstd patches have not made it into the upstream kernel yet. As used by mkinitramfs: - lz4 is faster to compress than gzip - lz4 is blazingly fast to decompress - lzma is dog slow to compress and decompress, but is tiny - lz4 size weight over gzip is marginal (14%) but imho worth the improved boot time & initrd creation time - xz is potentially even slower and even smaller than lzma In places where size is an absolute premium (tiny embedded iot devices) and performance is irrelevant, xz or lzma should be used. In all other places, our performance profile is in favor of lz4. Imho that includes the kernel image itself, thus we should consider switching: - initramfs tools to default to lz4 - livecd-rootfs to default to lz4 - kernels to compress kernel image with lz4 - grub to include lz4 support I shall proceed with changing the defaults on the above to improve our responsiveness experience on installer, cloud, core and classic devices. If our firstboot & subsequent boot speed degrades or disk space becomes a concern, we can look into tweaking these changes further. -- Regards, Dimitri. -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Add "dis_ucode_ldr" to linux boot options for Recovery Mode
Hi, Due to some recent problems with a microcode update which broke booting of some laptops I suggest adding "dis_ucode_ldr" to recovery mode boot options to allow booting after a bad microcode update. See the following topics that describe the problems: - https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1829620 - https://github.com/intel/Intel-Linux-Processor-Microcode-Data-Files/issues/1 When such problems occur it's very hard to see that the microcode loading is the issue, the booting just hangs at the purple screen, even removing "quiet splash" will not show the error. Kind regards, Jeroen -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
