On Sat, Aug 5, 2023 at 4:12 PM Richard W.M. Jones <rjo...@redhat.com> wrote: > > The background to this is I've spent far too long trying to optimize > the conversion of qcow2 files to raw files. Most existing qcow2 files > that you can find online are zlib compressed, including the qcow2 > images provided by Fedora. Each cluster in the file is separately > compressed as a zlib stream, and qemu uses zlib library functions to > decompress them. When downloading and decompressing these files, I > measured 40%+ of the total CPU time is doing zlib decompression. > > [You don't need to tell me how great Zstd is, qcow2 supports this for > compression also, but it is not widely used by existing content.] > > I was looking around for alternative zlib implementations and there > are several. This AWS blog is a decent summary: > > > https://aws.amazon.com/blogs/opensource/improving-zlib-cloudflare-and-comparing-performance-with-other-zlib-forks/ > > We already package zlib-ng in Fedora: > > https://src.fedoraproject.org/rpms/zlib-ng/blob/rawhide/f/zlib-ng.spec > https://github.com/zlib-ng/zlib-ng > > In my test zlib-ng is about 40% faster. > > Sadly zlib-ng is not compiled with the ZLIB_COMPAT option. What this > means in practice is that the zlib functions have different names > (eg. zng_inflateInit instead of inflateInit). It is not a drop-in > replacement for zlib and software would need to be adjusted to use it. > > However there is this bug / RFE to package the compat library. > > https://bugzilla.redhat.com/show_bug.cgi?id=2145239 > > It literally replaces /usr/lib64/libz.so.1, so pretty high stakes. It > has the appropriate Provides/Conflicts. > > Anyway, 40% (or whatever, but significant) performance improvement > sounds good for a very widely used operation. > > So I'd like to ask Fedora ... > > What do we think about the opt in approach of adding a patch similar > to the one proposed in bug 2145239, where I think you could "simply" > install zlib-ng to get better performance with existing software? > ("simply" because it seems high risk of going wrong) > > What about replacing zlib with zlib-ng? > > Other ideas ...? Does anyone know what other distros are doing? >
It's certainly an idea. In CentOS Hyperscale, we've discussed proposing this for Fedora because of the better maintainability and performance. openSUSE ships zlib-ng-compat as a provider of zlib, though it is not yet the default. They are working on build failures caused by switching to zlib-ng as the provider of zlib. I think it'd be worth us moving to zlib-ng as the sole zlib provider. -- 真実はいつも一つ！/ Always, there's only one truth! _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
