Chris, Thanks for your feedback and comments, it's very valuable to me.
In my previous message, I mentioned that CPU is *underutilized* during installation. I haven't investigated further why, but I suspect it's due to the inefficiency caused by the usage of the *loop* device and/or inefficiency in the rsync itself. In fact, I have an optimization to file next weekend on my to do list. > All of the Live installations use rsync. And that's what I propose to change: to use unsquashfs instead of rsync, preliminary benchmarks show 8x improvement in decompressing speed on local media for XZ on local storage. On my PC configuration, that will require approx. 52.98 MiB/s sequential read performance from local media and approx. 181.19MiB writing speed to the destination media. That level performance is not common among today's USB drives or optical media -- the installation speed will not be capped due to the CPU limits, but otherwise limited by the sequential read speed of the installation media. It means, selecting an algorithm with better compression ratio should reduce the installation time from commonly used USB storage. Yes, Zstd consumes 12.24x less CPU user time while unsquashfs, but let's consider the practical application. Will Zstd decrease the installation time, given the constraints and optimization above -- that's what I plan to investigate in upcoming weekends. My proposal focuses on reducing the installation media size, and *recommends *to use certain compression options. But, I think, the final decision is to be made by FESCO. On Mon, Jan 20, 2020 at 12:42 AM Chris Murphy <li...@colorremedies.com> wrote: > On Sun, Jan 19, 2020 at 8:41 AM Bohdan Khomutskyi <bkhom...@redhat.com> > wrote: > > > > Hello, > > > > Thanks everyone for posting feedback. > > More benchmarking results are available at > https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS, > including the 'plain' SquashFS filesystem. > > After performing the tests, I personally recommend to use xz compression > with 1MiB block size, without bcj, on a 'plain' squash filesystem -- this > will lead to a reduction of 142MiB on the ISO, compared to the stock Fedora > 31 Workstation x86_64 image. > > Alternative compression options, such as Zstd, are also mentioned in the > change proposal. > > Thanks for all the tests. > > While I see the meaningfully reduced CPU hit of xz compressed images, > the proposal leaves a lot of performance improvement on the table by > not also enabling zstd as an option in the compose process. The tests > show zstd results, but the proposal doesn't mention zstd at all. > > In particular for Workstation ISO, the CPU hit isn't worth the size > savings for regular users, let alone the recurring hit for releng > composes and QA's automated installation tests. It's a lot of CPU burn > at both ends of the candle, for not a lot of size savings. I'm not > convinced it's worth the extra hit on the create side for Zstd level > 22, compared to Zstd 15 or 17. > > I admit I'm biased toward the two endpoints: create and consume, not > distribution ,i.e the mirror donors. Their storage and bandwidth > concerns were evaluated with the RPM change from xz to zstd. So I'm > mystified by the bias for image size. > > Anyway, I approve of the change but disappointed if it really doesn't > let Fedora release engineering the ability to choose (possibly based > on image type - maybe there's some benefit to using xz for raw and > qcow2 images). > > > > -- > Chris Murphy > _______________________________________________ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > -- Bohdan Khomutskyi, RHCE Release configuration management engineer, PnT DevOps Red Hat Czech s.r.o T: +420532270289 IRC: bkhomuts
_______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
