Re: [Libguestfs] [PATCH libnbd 4/9] golang: aio_buffer.go: Add MakeAioBufferZero()
On Tue, Feb 8, 2022 at 10:45 PM Eric Blake wrote: > > On Sun, Jan 30, 2022 at 01:33:32AM +0200, Nir Soffer wrote: > > Make it easy to create a zeroed buffer via calloc(), preventing leaking > > sensitive info from the heap. > > > > Benchmarking show that creating a zeroed buffer is much slower compared > > shows Will fix > > > with uninitialized buffer, but much faster compared with manually > > initializing the buffer with a loop. > > > > BenchmarkMakeAioBuffer-12 7252674 148.1 ns/op > > BenchmarkMakeAioBufferZero-12 262107 4181 ns/op > > BenchmarkAioBufferZero-1217581 68759 ns/op > > > > It is interesting that creating a zeroed buffer is 3 times faster > > compared with making a new []byte slice: > > > > BenchmarkMakeAioBufferZero-12 247710 4440 ns/op > > BenchmarkMakeByteSlice-1284117 13733 ns/op > > Some of this is due to how much vectorization the standard library > (whether libc or Go's core libraries) can do when bulk-zeroing > (zeroing 64 bits, or even a cache line at a time, in an unrolled loop, > is always going to be more performant than a naive loop of one byte at > a time). > > > > > Signed-off-by: Nir Soffer > > --- > > golang/aio_buffer.go | 6 ++ > > golang/libnbd_620_aio_buffer_test.go | 16 > > Another file that may fit better in the 0xx naming, especially if we > decide to duplicate similar functionality into the python or OCaml > bindings of being able to pre-generate a known-zero buffer for use in > passing to nbd_pread. > > As a helper API, this seems useful. But do we need any man page > documentation of a language-specific helper function? The AioBuffer type is documented here: https://pkg.go.dev/libguestfs.org/libnbd#AioBuffer Patch #3 golang: aio_buffer.go: Add missing documentation adds the missing documentation for the functions, We can add more documentation for the type. If we need task-based documentation I think improving libnbd-golang is the best way. ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
Re: [Libguestfs] [PATCH libnbd 4/9] golang: aio_buffer.go: Add MakeAioBufferZero()
On Sun, Jan 30, 2022 at 01:33:32AM +0200, Nir Soffer wrote: > Make it easy to create a zeroed buffer via calloc(), preventing leaking > sensitive info from the heap. > > Benchmarking show that creating a zeroed buffer is much slower compared shows > with uninitialized buffer, but much faster compared with manually > initializing the buffer with a loop. > > BenchmarkMakeAioBuffer-12 7252674 148.1 ns/op > BenchmarkMakeAioBufferZero-12 262107 4181 ns/op > BenchmarkAioBufferZero-1217581 68759 ns/op > > It is interesting that creating a zeroed buffer is 3 times faster > compared with making a new []byte slice: > > BenchmarkMakeAioBufferZero-12 247710 4440 ns/op > BenchmarkMakeByteSlice-1284117 13733 ns/op Some of this is due to how much vectorization the standard library (whether libc or Go's core libraries) can do when bulk-zeroing (zeroing 64 bits, or even a cache line at a time, in an unrolled loop, is always going to be more performant than a naive loop of one byte at a time). > > Signed-off-by: Nir Soffer > --- > golang/aio_buffer.go | 6 ++ > golang/libnbd_620_aio_buffer_test.go | 16 Another file that may fit better in the 0xx naming, especially if we decide to duplicate similar functionality into the python or OCaml bindings of being able to pre-generate a known-zero buffer for use in passing to nbd_pread. As a helper API, this seems useful. But do we need any man page documentation of a language-specific helper function? -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [PATCH libnbd 4/9] golang: aio_buffer.go: Add MakeAioBufferZero()
Make it easy to create a zeroed buffer via calloc(), preventing leaking
sensitive info from the heap.
Benchmarking show that creating a zeroed buffer is much slower compared
with uninitialized buffer, but much faster compared with manually
initializing the buffer with a loop.
BenchmarkMakeAioBuffer-127252674 148.1 ns/op
BenchmarkMakeAioBufferZero-12 262107 4181 ns/op
BenchmarkAioBufferZero-12 17581 68759 ns/op
It is interesting that creating a zeroed buffer is 3 times faster
compared with making a new []byte slice:
BenchmarkMakeAioBufferZero-12 247710 4440 ns/op
BenchmarkMakeByteSlice-12 84117 13733 ns/op
Signed-off-by: Nir Soffer
---
golang/aio_buffer.go | 6 ++
golang/libnbd_620_aio_buffer_test.go | 16
2 files changed, 22 insertions(+)
diff --git a/golang/aio_buffer.go b/golang/aio_buffer.go
index 8f33f500..dcb036a5 100644
--- a/golang/aio_buffer.go
+++ b/golang/aio_buffer.go
@@ -38,20 +38,26 @@ type AioBuffer struct {
Punsafe.Pointer
Size uint
}
// MakeAioBuffer makes a new buffer backed by an unitilialized C allocated
// array.
func MakeAioBuffer(size uint) AioBuffer {
return AioBuffer{C.malloc(C.ulong(size)), size}
}
+// MakeAioBuffer makes a new buffer backed by a C allocated array. The
+// underlying buffer is set to zero.
+func MakeAioBufferZero(size uint) AioBuffer {
+ return AioBuffer{C.calloc(C.ulong(1), C.ulong(size)), size}
+}
+
// FromBytes makes a new buffer backed by a C allocated array, initialized by
// copying the given Go slice.
func FromBytes(buf []byte) AioBuffer {
size := len(buf)
ret := MakeAioBuffer(uint(size))
for i := 0; i < len(buf); i++ {
*ret.Get(uint(i)) = buf[i]
}
return ret
}
diff --git a/golang/libnbd_620_aio_buffer_test.go
b/golang/libnbd_620_aio_buffer_test.go
index 89badfc6..53d6233b 100644
--- a/golang/libnbd_620_aio_buffer_test.go
+++ b/golang/libnbd_620_aio_buffer_test.go
@@ -51,20 +51,28 @@ func TestAioBuffer(t *testing.T) {
t.Fatalf("Expected %v, got %v", zeroes, buf.Bytes())
}
/* Create a nother buffer from Go slice. */
buf2 := FromBytes(zeroes)
defer buf2.Free()
if !bytes.Equal(buf2.Bytes(), zeroes) {
t.Fatalf("Expected %v, got %v", zeroes, buf2.Bytes())
}
+
+ /* Crated a zeroed buffer. */
+ buf3 := MakeAioBufferZero(uint(32))
+ defer buf.Free()
+
+ if !bytes.Equal(buf3.Bytes(), zeroes) {
+ t.Fatalf("Expected %v, got %v", zeroes, buf2.Bytes())
+ }
}
func TestAioBufferFree(t *testing.T) {
buf := MakeAioBuffer(uint(32))
/* Free the underlying C array. */
buf.Free()
/* And clear the pointer. */
if buf.P != nil {
@@ -105,20 +113,28 @@ func TestAioBufferGetAfterFree(t *testing.T) {
const bufferSize uint = 256 * 1024
// Benchmark creating uninitilized buffer.
func BenchmarkMakeAioBuffer(b *testing.B) {
for i := 0; i < b.N; i++ {
buf := MakeAioBuffer(bufferSize)
buf.Free()
}
}
+// Benchmark creating zeroed buffer.
+func BenchmarkMakeAioBufferZero(b *testing.B) {
+ for i := 0; i < b.N; i++ {
+ buf := MakeAioBufferZero(bufferSize)
+ buf.Free()
+ }
+}
+
// Benchmark zeroing a buffer.
func BenchmarkAioBufferZero(b *testing.B) {
for i := 0; i < b.N; i++ {
buf := MakeAioBuffer(bufferSize)
for i := uint(0); i < bufferSize; i++ {
*buf.Get(i) = 0
}
buf.Free()
}
}
--
2.34.1
___
Libguestfs mailing list
Libguestfs@redhat.com
https://listman.redhat.com/mailman/listinfo/libguestfs
