Re: [Libguestfs] [PATCH libnbd 4/9] golang: aio_buffer.go: Add MakeAioBufferZero()
On Tue, Feb 8, 2022 at 10:45 PM Eric Blake wrote: > > On Sun, Jan 30, 2022 at 01:33:32AM +0200, Nir Soffer wrote: > > Make it easy to create a zeroed buffer via calloc(), preventing leaking > > sensitive info from the heap. > > > > Benchmarking show that creating a zeroed buffer is much slower compared > > shows Will fix > > > with uninitialized buffer, but much faster compared with manually > > initializing the buffer with a loop. > > > > BenchmarkMakeAioBuffer-12 7252674 148.1 ns/op > > BenchmarkMakeAioBufferZero-12 262107 4181 ns/op > > BenchmarkAioBufferZero-1217581 68759 ns/op > > > > It is interesting that creating a zeroed buffer is 3 times faster > > compared with making a new []byte slice: > > > > BenchmarkMakeAioBufferZero-12 247710 4440 ns/op > > BenchmarkMakeByteSlice-1284117 13733 ns/op > > Some of this is due to how much vectorization the standard library > (whether libc or Go's core libraries) can do when bulk-zeroing > (zeroing 64 bits, or even a cache line at a time, in an unrolled loop, > is always going to be more performant than a naive loop of one byte at > a time). > > > > > Signed-off-by: Nir Soffer > > --- > > golang/aio_buffer.go | 6 ++ > > golang/libnbd_620_aio_buffer_test.go | 16 > > Another file that may fit better in the 0xx naming, especially if we > decide to duplicate similar functionality into the python or OCaml > bindings of being able to pre-generate a known-zero buffer for use in > passing to nbd_pread. > > As a helper API, this seems useful. But do we need any man page > documentation of a language-specific helper function? The AioBuffer type is documented here: https://pkg.go.dev/libguestfs.org/libnbd#AioBuffer Patch #3 golang: aio_buffer.go: Add missing documentation adds the missing documentation for the functions, We can add more documentation for the type. If we need task-based documentation I think improving libnbd-golang is the best way. ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
Re: [Libguestfs] [PATCH libnbd 4/9] golang: aio_buffer.go: Add MakeAioBufferZero()
On Sun, Jan 30, 2022 at 01:33:32AM +0200, Nir Soffer wrote: > Make it easy to create a zeroed buffer via calloc(), preventing leaking > sensitive info from the heap. > > Benchmarking show that creating a zeroed buffer is much slower compared shows > with uninitialized buffer, but much faster compared with manually > initializing the buffer with a loop. > > BenchmarkMakeAioBuffer-12 7252674 148.1 ns/op > BenchmarkMakeAioBufferZero-12 262107 4181 ns/op > BenchmarkAioBufferZero-1217581 68759 ns/op > > It is interesting that creating a zeroed buffer is 3 times faster > compared with making a new []byte slice: > > BenchmarkMakeAioBufferZero-12 247710 4440 ns/op > BenchmarkMakeByteSlice-1284117 13733 ns/op Some of this is due to how much vectorization the standard library (whether libc or Go's core libraries) can do when bulk-zeroing (zeroing 64 bits, or even a cache line at a time, in an unrolled loop, is always going to be more performant than a naive loop of one byte at a time). > > Signed-off-by: Nir Soffer > --- > golang/aio_buffer.go | 6 ++ > golang/libnbd_620_aio_buffer_test.go | 16 Another file that may fit better in the 0xx naming, especially if we decide to duplicate similar functionality into the python or OCaml bindings of being able to pre-generate a known-zero buffer for use in passing to nbd_pread. As a helper API, this seems useful. But do we need any man page documentation of a language-specific helper function? -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [PATCH libnbd 4/9] golang: aio_buffer.go: Add MakeAioBufferZero()
Make it easy to create a zeroed buffer via calloc(), preventing leaking sensitive info from the heap. Benchmarking show that creating a zeroed buffer is much slower compared with uninitialized buffer, but much faster compared with manually initializing the buffer with a loop. BenchmarkMakeAioBuffer-127252674 148.1 ns/op BenchmarkMakeAioBufferZero-12 262107 4181 ns/op BenchmarkAioBufferZero-12 17581 68759 ns/op It is interesting that creating a zeroed buffer is 3 times faster compared with making a new []byte slice: BenchmarkMakeAioBufferZero-12 247710 4440 ns/op BenchmarkMakeByteSlice-12 84117 13733 ns/op Signed-off-by: Nir Soffer --- golang/aio_buffer.go | 6 ++ golang/libnbd_620_aio_buffer_test.go | 16 2 files changed, 22 insertions(+) diff --git a/golang/aio_buffer.go b/golang/aio_buffer.go index 8f33f500..dcb036a5 100644 --- a/golang/aio_buffer.go +++ b/golang/aio_buffer.go @@ -38,20 +38,26 @@ type AioBuffer struct { Punsafe.Pointer Size uint } // MakeAioBuffer makes a new buffer backed by an unitilialized C allocated // array. func MakeAioBuffer(size uint) AioBuffer { return AioBuffer{C.malloc(C.ulong(size)), size} } +// MakeAioBuffer makes a new buffer backed by a C allocated array. The +// underlying buffer is set to zero. +func MakeAioBufferZero(size uint) AioBuffer { + return AioBuffer{C.calloc(C.ulong(1), C.ulong(size)), size} +} + // FromBytes makes a new buffer backed by a C allocated array, initialized by // copying the given Go slice. func FromBytes(buf []byte) AioBuffer { size := len(buf) ret := MakeAioBuffer(uint(size)) for i := 0; i < len(buf); i++ { *ret.Get(uint(i)) = buf[i] } return ret } diff --git a/golang/libnbd_620_aio_buffer_test.go b/golang/libnbd_620_aio_buffer_test.go index 89badfc6..53d6233b 100644 --- a/golang/libnbd_620_aio_buffer_test.go +++ b/golang/libnbd_620_aio_buffer_test.go @@ -51,20 +51,28 @@ func TestAioBuffer(t *testing.T) { t.Fatalf("Expected %v, got %v", zeroes, buf.Bytes()) } /* Create a nother buffer from Go slice. */ buf2 := FromBytes(zeroes) defer buf2.Free() if !bytes.Equal(buf2.Bytes(), zeroes) { t.Fatalf("Expected %v, got %v", zeroes, buf2.Bytes()) } + + /* Crated a zeroed buffer. */ + buf3 := MakeAioBufferZero(uint(32)) + defer buf.Free() + + if !bytes.Equal(buf3.Bytes(), zeroes) { + t.Fatalf("Expected %v, got %v", zeroes, buf2.Bytes()) + } } func TestAioBufferFree(t *testing.T) { buf := MakeAioBuffer(uint(32)) /* Free the underlying C array. */ buf.Free() /* And clear the pointer. */ if buf.P != nil { @@ -105,20 +113,28 @@ func TestAioBufferGetAfterFree(t *testing.T) { const bufferSize uint = 256 * 1024 // Benchmark creating uninitilized buffer. func BenchmarkMakeAioBuffer(b *testing.B) { for i := 0; i < b.N; i++ { buf := MakeAioBuffer(bufferSize) buf.Free() } } +// Benchmark creating zeroed buffer. +func BenchmarkMakeAioBufferZero(b *testing.B) { + for i := 0; i < b.N; i++ { + buf := MakeAioBufferZero(bufferSize) + buf.Free() + } +} + // Benchmark zeroing a buffer. func BenchmarkAioBufferZero(b *testing.B) { for i := 0; i < b.N; i++ { buf := MakeAioBuffer(bufferSize) for i := uint(0); i < bufferSize; i++ { *buf.Get(i) = 0 } buf.Free() } } -- 2.34.1 ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs