Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Richard W.M. Jones
On Fri, Sep 01, 2023 at 06:45:22PM +0200, Florian Weimer wrote: > * Richard W. M. Jones: > > > I tested the speed of decompression using: > > > > $ hyperfine 'qemu-img convert -W -m 16 -f qcow2 test.qcow2.XXX -O raw > > test.out' > > (qemu 8.0.0-4.fc39.x86_64) > > > > $ hyperfine 'nbdkit -U

Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Florian Weimer
* Richard W. M. Jones: > I tested the speed of decompression using: > > $ hyperfine 'qemu-img convert -W -m 16 -f qcow2 test.qcow2.XXX -O raw > test.out' > (qemu 8.0.0-4.fc39.x86_64) > > $ hyperfine 'nbdkit -U - --filter=qcow2dec file test.qcow2.XXX --run > '\''nbdcopy --request-size "$uri

Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Richard W.M. Jones
On Fri, Sep 01, 2023 at 01:10:16PM +0200, Kevin Wolf wrote: > And nbdkit seems to get worse instead of better with larger cluster > size, no matter whether zlib or zstd is used. It's caused by nbdcopy's default request size being 256k. Increasing it to 2M cures the scaling problem - see updated r

Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Kevin Wolf
Am 01.09.2023 um 12:03 hat Richard W.M. Jones geschrieben: > On Fri, Sep 01, 2023 at 10:55:50AM +0100, Daniel P. Berrangé wrote: > > On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote: > > > On Fri, Sep 01, 2023 at 10:48:14AM +0200, Kevin Wolf wrote: > > > > I understand the context

Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Richard W.M. Jones
On Fri, Sep 01, 2023 at 11:06:20AM +0100, Daniel P. Berrangé wrote: > On Fri, Sep 01, 2023 at 11:03:54AM +0100, Richard W.M. Jones wrote: > > I forgot to say that nbdkit is using zlib-ng, since I made the source > > level changes a few weeks back (but most of the nbdkit performance > > improvement

Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Daniel P . Berrangé
On Fri, Sep 01, 2023 at 11:03:54AM +0100, Richard W.M. Jones wrote: > On Fri, Sep 01, 2023 at 10:55:50AM +0100, Daniel P. Berrangé wrote: > > On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote: > > > On Fri, Sep 01, 2023 at 10:48:14AM +0200, Kevin Wolf wrote: > > > > Am 31.08.2023 u

Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Richard W.M. Jones
On Fri, Sep 01, 2023 at 10:55:50AM +0100, Daniel P. Berrangé wrote: > On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote: > > On Fri, Sep 01, 2023 at 10:48:14AM +0200, Kevin Wolf wrote: > > > Am 31.08.2023 um 11:20 hat Richard W.M. Jones geschrieben: > > > > On Thu, Aug 31, 2023 at

Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Richard W.M. Jones
On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote: > Results: > > Cluster Compression Compressed size Prog Decompression speed > > 4k zlib 3228811264 qemu 5.921 s ± 0.074 s > 4k zstd 3258097664 qemu 5.189 s ± 0.158 s > >

Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Daniel P . Berrangé
On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote: > On Fri, Sep 01, 2023 at 10:48:14AM +0200, Kevin Wolf wrote: > > Am 31.08.2023 um 11:20 hat Richard W.M. Jones geschrieben: > > > On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote: > > > > [ Cc: qemu-block ] > > > > > >

Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Richard W.M. Jones
On Fri, Sep 01, 2023 at 10:42:16AM +0100, Richard W.M. Jones wrote: > $ hyperfine 'nbdkit -U - --filter=qcow2dec file test.qcow2.XXX --run > '\''nbdcopy --request-size "$uri" test.out'\'' ' Sorry, copy and paste error, the command is: hyperfine 'nbdkit -U - --filter=qcow2dec file test.qcow2.XX

Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Richard W.M. Jones
On Fri, Sep 01, 2023 at 10:48:14AM +0200, Kevin Wolf wrote: > Am 31.08.2023 um 11:20 hat Richard W.M. Jones geschrieben: > > On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote: > > > [ Cc: qemu-block ] > > > > > > Am 30.08.2023 um 20:26 hat Richard W.M. Jones geschrieben: > > > > On Tue, A

Re: zlib-ng as a compat replacement for zlib

2023-09-01 Thread Kevin Wolf
Am 31.08.2023 um 11:20 hat Richard W.M. Jones geschrieben: > On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote: > > [ Cc: qemu-block ] > > > > Am 30.08.2023 um 20:26 hat Richard W.M. Jones geschrieben: > > > On Tue, Aug 29, 2023 at 05:49:24PM -, Daniel Alley wrote: > > > > > The backg

Re: zlib-ng as a compat replacement for zlib

2023-08-31 Thread Florian Weimer
* Richard W. M. Jones: > On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote: >> Unfortunately, we seem to build the RHEL packages with --disable-zstd >> (I suppose just because we tend to disable everything nobody explicitly >> asked for). Maybe we should check other distros. If zstd is co

Re: zlib-ng as a compat replacement for zlib

2023-08-31 Thread Richard W.M. Jones
On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote: > [ Cc: qemu-block ] > > Am 30.08.2023 um 20:26 hat Richard W.M. Jones geschrieben: > > On Tue, Aug 29, 2023 at 05:49:24PM -, Daniel Alley wrote: > > > > The background to this is I've spent far too long trying to optimize > > > > the

Re: zlib-ng as a compat replacement for zlib

2023-08-31 Thread Kevin Wolf
[ Cc: qemu-block ] Am 30.08.2023 um 20:26 hat Richard W.M. Jones geschrieben: > On Tue, Aug 29, 2023 at 05:49:24PM -, Daniel Alley wrote: > > > The background to this is I've spent far too long trying to optimize > > > the conversion of qcow2 files to raw files. Most existing qcow2 files > >