Re: Release of libvirt-6.5.0

On 7/3/20 3:56 AM, Daniel Veillard wrote: > Half a day late, but I pushed the 6.5.0 release out, it is as usual > available as a signed tarball and source rpms from the server: > > https://libvirt.org/sources/ > > I also tagged and pushed the 6.5.0 python bindings that one can find at > >

[GSoC][PATCH] qemu_domainjob: removal of its dependency on other qemu-files

It was seen that `qemu_domain.h` file depended upon `qemu_migration_params.h` and `qmeu_monitor.h` as they were required by some qemu_domainjob stuctures. This dependency was removed by the introduction of a `void *privateData` pointer. This privateData pointer was handled using a structure of

Re: [PATCH 0/9] Allow sparse streams for block devices

On 7/3/20 3:59 PM, Andrea Bolognani wrote: On Fri, 2020-07-03 at 12:28 +0200, Michal Privoznik wrote: Michal Prívozník (9): libvirt-storage: Document volume upload/download stream format virstring: Introduce virStringIsNull() virfile: Introduce virFileInDataDetectZeroes() virsh:

Re: [PATCH 0/9] Allow sparse streams for block devices

On Fri, 2020-07-03 at 12:28 +0200, Michal Privoznik wrote: > Michal Prívozník (9): > libvirt-storage: Document volume upload/download stream format > virstring: Introduce virStringIsNull() > virfile: Introduce virFileInDataDetectZeroes() > virsh: Pass virshStreamCallbackDataPtr to

Re: [libvirt PATCH 1/1] qemu: Use virQEMUCapsCacheLookupDefault instead of lookup by arch

On Fri, Jul 03, 2020 at 03:14:48PM +0200, Erik Skultety wrote: > Firstly, SEV is present only on AMD, so we can safely assume x86. > Secondly, the problem with looking up capabilities in the cache by arch > is that it's using virHashSearch with a callback to find the right > capabilities and get

Re: [PATCH 2/9] virstring: Introduce virStringIsNull()

On Fri, Jul 03, 2020 at 12:28:43PM +0200, Michal Privoznik wrote: > This function will be used to detect zero buffers (which are > going to be interpreted as hole in virStream later). > > I shamelessly took inspiration from coreutils. > > Signed-off-by: Michal Privoznik > --- >

Re: [PATCH 2/9] virstring: Introduce virStringIsNull()

On Fri, Jul 03, 2020 at 03:13:29PM +0200, Michal Privoznik wrote: > On 7/3/20 2:08 PM, Ján Tomko wrote: > > On a Friday in 2020, Michal Privoznik wrote: > > > This function will be used to detect zero buffers (which are > > > going to be interpreted as hole in virStream later). > > > > > > I

[libvirt PATCH 0/1] Use the default capabilities cache lookup rather than an arch based lookup

https://bugzilla.redhat.com/show_bug.cgi?id=1852311 Erik Skultety (1): qemu: Use virQEMUCapsCacheLookupDefault instead of lookup by arch src/qemu/qemu_driver.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) -- 2.26.2

Re: [PATCH 2/9] virstring: Introduce virStringIsNull()

On 7/3/20 2:08 PM, Ján Tomko wrote: On a Friday in 2020, Michal Privoznik wrote: This function will be used to detect zero buffers (which are going to be interpreted as hole in virStream later). I shamelessly took inspiration from coreutils. How is that okay, isn't coreutils GPL-licensed?

[libvirt PATCH 1/1] qemu: Use virQEMUCapsCacheLookupDefault instead of lookup by arch

Firstly, SEV is present only on AMD, so we can safely assume x86. Secondly, the problem with looking up capabilities in the cache by arch is that it's using virHashSearch with a callback to find the right capabilities and get the binary name from it as well, but since the cache is empty, it will

Re: [RFC v2 1/1] memory: Delete assertion in memory_region_unregister_iommu_notifier

On Fri, Jul 03, 2020 at 03:24:19PM +0800, Jason Wang wrote: > > On 2020/7/2 下午11:45, Peter Xu wrote: > > On Thu, Jul 02, 2020 at 11:01:54AM +0800, Jason Wang wrote: > > > So I think we agree that a new notifier is needed? > > Good to me, or a new flag should be easier (IOMMU_NOTIFIER_DEV_IOTLB)?

Re: Re: [RFC 00/21] RFC: Generate parsexml/formatbuf functions based on directives

On Wed, Jul 01, 2020 at 12:06:32AM +0800, Shi Lei wrote: > >On Wed, Jun 10, 2020 at 09:20:28AM +0800, Shi Lei wrote: > >> > >> Last RFC: > >> [https://www.redhat.com/archives/libvir-list/2020-April/msg00970.html] > >> In last RFC, I suggested we can generate object-model code based on > >>

Re: [libvirt-python][PATCH 1/3] MANIFEST: Distribute sparsestream.py example

On Fri, Jul 03, 2020 at 01:30:58PM +0200, Michal Privoznik wrote: > Signed-off-by: Michal Privoznik > --- > MANIFEST.in | 1 + > 1 file changed, 1 insertion(+) Reviewed-by: Pavel Hrdina signature.asc Description: PGP signature

Re: [libvirt-python][PATCH 2/3] virStream: Use larger buffer for sendAll/recvAll methods

On Fri, Jul 03, 2020 at 01:30:59PM +0200, Michal Privoznik wrote: > There are four methods which receive/send entire stream > (sendAll(), recvAll(), sparseSendAll() and sparseRecvAll()). All > these have an intermediary buffer which is either filled by > incoming stream and passed to a user

Re: [PATCH 2/9] virstring: Introduce virStringIsNull()

On a Friday in 2020, Michal Privoznik wrote: This function will be used to detect zero buffers (which are going to be interpreted as hole in virStream later). I shamelessly took inspiration from coreutils. How is that okay, isn't coreutils GPL-licensed? Jano Signed-off-by: Michal

Re: [PATCH 0/9] Allow sparse streams for block devices

On Fri, Jul 03, 2020 at 01:50:22PM +0200, Michal Privoznik wrote: > On 7/3/20 1:34 PM, Daniel P. Berrangé wrote: > > On Fri, Jul 03, 2020 at 12:28:41PM +0200, Michal Privoznik wrote: > > > The way our sparse streams are implemented is: > > > > > > 1) user facing APIs (virStreamSparseRecvAll() and

Re: [PATCH 0/9] Allow sparse streams for block devices

On 7/3/20 1:34 PM, Daniel P. Berrangé wrote: On Fri, Jul 03, 2020 at 12:28:41PM +0200, Michal Privoznik wrote: The way our sparse streams are implemented is: 1) user facing APIs (virStreamSparseRecvAll() and virStreamSparseSendAll()) take callbacks as arguments. These callbacks read/write

Re: [PATCH 0/9] Allow sparse streams for block devices

On Fri, Jul 03, 2020 at 12:28:41PM +0200, Michal Privoznik wrote: > The way our sparse streams are implemented is: > > 1) user facing APIs (virStreamSparseRecvAll() and virStreamSparseSendAll()) > take >callbacks as arguments. These callbacks read/write data or determine if > there >is

[libvirt-python][PATCH 2/3] virStream: Use larger buffer for sendAll/recvAll methods

There are four methods which receive/send entire stream (sendAll(), recvAll(), sparseSendAll() and sparseRecvAll()). All these have an intermediary buffer which is either filled by incoming stream and passed to a user provided callback to handle the data, or the other way round - user fills it

[libvirt-python][PATCH 1/3] MANIFEST: Distribute sparsestream.py example

Signed-off-by: Michal Privoznik --- MANIFEST.in | 1 + 1 file changed, 1 insertion(+) diff --git a/MANIFEST.in b/MANIFEST.in index da7cbae..fd76f3c 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -13,6 +13,7 @@ include examples/event-test.py include examples/guest-vcpus/guest-vcpu-daemon.py

[libvirt-python][PATCH 3/3] examples: Update sparsestream.py so that it handles block devices

This patch is similar to what I sent for libvirt. It extends callbacks so that they work with block devices. Signed-off-by: Michal Privoznik --- examples/sparsestream.py | 73 +--- 1 file changed, 54 insertions(+), 19 deletions(-) diff --git

[libvirt-python][PATCH 0/3] Improve streams a bit

Inspired by patches I've sent for libvirt: https://www.redhat.com/archives/libvir-list/2020-July/msg00145.html Michal Privoznik (3): MANIFEST: Distribute sparsestream.py example virStream: Use larger buffer for sendAll/recvAll methods examples: Update sparsestream.py so that it handles

[PATCH 4/9] virsh: Pass virshStreamCallbackDataPtr to virshStreamSink() and virshStreamSkip()

These callback will need to know more that the FD they are working on. Pass the structure that is passed to other stream callbacks (e.g. virshStreamSource() or virshStreamSourceSkip()) instead of inventing a new one. Signed-off-by: Michal Privoznik --- tools/virsh-util.c | 10 +-

[PATCH 9/9] virfdstream: Emulate skip for block devices

This is similar to one of previous patches. When receiving stream (on virStorageVolUpload() and subsequent virStreamSparseSendAll()) we may receive a hole. If the volume we are saving the incoming data into is a regular file we just lseek() and ftruncate() to create the hole. But this won't work

[PATCH 0/9] Allow sparse streams for block devices

The way our sparse streams are implemented is: 1) user facing APIs (virStreamSparseRecvAll() and virStreamSparseSendAll()) take callbacks as arguments. These callbacks read/write data or determine if there is a hole in the underlying file and big it is. 2) libvirtd has something similar -

[PATCH 5/9] virsh: Track if vol-upload or vol-download work over a block device

We can't use virFileInData() with block devices, but we could use new virFileInDataDetectZeroes(). But to decide we need to know if the FD we are reading data from / writing data to is a block device. Store this information in _virshStreamCallbackData. Signed-off-by: Michal Privoznik ---

[PATCH 7/9] virfdstream: Allow sparse stream vol-download

When handling sparse stream, a thread is executed. This thread runs a read() or write() loop (depending what API is called; in this case it's virStorageVolDownload() and this the thread run read() loop). The read() is handled in virFDStreamThreadDoRead() which is then data/hole section aware,

[PATCH 2/9] virstring: Introduce virStringIsNull()

This function will be used to detect zero buffers (which are going to be interpreted as hole in virStream later). I shamelessly took inspiration from coreutils. Signed-off-by: Michal Privoznik --- src/libvirt_private.syms | 1 + src/util/virstring.c | 40

[PATCH 6/9] virshStreamSkip: Emulate skip for block devices

This callback is called when the server sends us STREAM_HOLE meaning there is no real data, only zeroes. For regular files we would just seek() beyond EOF and ftruncate() to create the hole. But for block devices this won't work. Not only we can't seek() beyond EOF, and ftruncate() will fail, this

[PATCH 8/9] virshStreamInData: Handle block devices

This is very similar to previous commit. The virshStreamInData() callback is used by virStreamSparseSendAll() to detect whether the file the data is read from is in data or hole section. The SendAll() will then send corresponding type of virStream message to make server create a hole or write

[PATCH 1/9] libvirt-storage: Document volume upload/download stream format

For libvirt, the volume is just a binary blob and it doesn't interpret data on volume upload/download. But as it turns out, this unspoken assumption is not clear to our users. Document it explicitly. Suggested in: https://bugzilla.redhat.com/show_bug.cgi?id=1851023#c17 Signed-off-by: Michal

[PATCH 3/9] virfile: Introduce virFileInDataDetectZeroes()

For given file descriptor determine if the current position it is in plus 1MiB (arbitrary chosen value) consists solely from zero bytes or not. This is a block device friendly version of virFileInData(). Signed-off-by: Michal Privoznik --- src/libvirt_private.syms | 1 + src/util/virfile.c

Re: [PATCH 08/13] conf: Parse and format HMAT

On 7/3/20 11:23 AM, Daniel P. Berrangé wrote: On Wed, Jun 24, 2020 at 03:49:06PM +0200, Michal Privoznik wrote: To cite ACPI specification: Heterogeneous Memory Attribute Table describes the memory attributes, such as memory side cache attributes and bandwidth and latency details,

Re: [PATCH 08/13] conf: Parse and format HMAT

On Wed, Jun 24, 2020 at 03:49:06PM +0200, Michal Privoznik wrote: > To cite ACPI specification: > > Heterogeneous Memory Attribute Table describes the memory > attributes, such as memory side cache attributes and bandwidth > and latency details, related to the System Physical Address >

Release of libvirt-6.5.0

Half a day late, but I pushed the 6.5.0 release out, it is as usual available as a signed tarball and source rpms from the server: https://libvirt.org/sources/ I also tagged and pushed the 6.5.0 python bindings that one can find at https://libvirt.org/sources/python/ This release includes a

[PATCH] maint: Post-release version bump to 6.6.0

Signed-off-by: Michal Privoznik --- Pushed under trivial rule. NEWS.rst | 10 ++ configure.ac | 2 +- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/NEWS.rst b/NEWS.rst index 1663d11af7..7baf742e03 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -8,6 +8,16 @@ the changes

Re: [RFC v2 1/1] memory: Delete assertion in memory_region_unregister_iommu_notifier

On 2020/7/2 下午11:45, Peter Xu wrote: On Thu, Jul 02, 2020 at 11:01:54AM +0800, Jason Wang wrote: So I think we agree that a new notifier is needed? Good to me, or a new flag should be easier (IOMMU_NOTIFIER_DEV_IOTLB)? That should work but I wonder something as following is better.