[libvirt PATCH 1/3] qemu: add capability for qemu-vdagent chardev

2022-03-22 Thread Jonathon Jongsma
Detect whether qemu supports the qemu-vdagent character device. This enables support for copy/paste with VNC graphics. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_capabilities.c | 2 ++ src/qemu/qemu_capabilities.h | 1 +

[libvirt PATCH 3/3] qemu: add support for qemu-vdagent channel

2022-03-22 Thread Jonathon Jongsma
Implement the qemu-vdagent channel introduced in the previous commit. Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_command.c | 13 + src/qemu/qemu_monitor_json.c | 10 src/qemu/qemu_process.c | 1 +

[libvirt PATCH 0/3] Enable copy/paste for vnc displays

2022-03-22 Thread Jonathon Jongsma
This patch series enables support for the qemu-vdagent character device which enables copy/paste support between guest and client when using vnc graphics. The guest must be configured with something like the following: Copy/paste sync requires a vnc client that has

[libvirt PATCH 2/3] conf: add qemu-vdagent channel

2022-03-22 Thread Jonathon Jongsma
Add the ability to configure a qemu-vdagent in guest domains. This device is similar to the spice vdagent channel except that qemu handles the spice-vdagent protocol messages itself rather than routing them over a spice protocol channel. The qemu-vdagent device has two notable configuration

Re: [libvirt PATCH] ci: integration: Rename all Avocado standard stream log files to *.log

2022-03-22 Thread Andrea Bolognani
On Tue, Mar 22, 2022 at 06:23:47PM +0100, Erik Skultety wrote: > On Tue, Mar 22, 2022 at 05:17:28PM +, Andrea Bolognani wrote: > > On Tue, Mar 22, 2022 at 01:11:22PM +0100, Erik Skultety wrote: > > > Since GitLab sets a proper MIME for .txt and .log file extensions, > > > rename all Avocado

Re: [libvirt PATCH] ci: integration: Rename all Avocado standard stream log files to *.log

2022-03-22 Thread Erik Skultety
On Tue, Mar 22, 2022 at 05:17:28PM +, Andrea Bolognani wrote: > On Tue, Mar 22, 2022 at 01:11:22PM +0100, Erik Skultety wrote: > > Since GitLab sets a proper MIME for .txt and .log file extensions, > > rename all Avocado log files without an extension to *.log . This pairs > > nicely with the

Re: [libvirt PATCH] ci: integration: Rename all Avocado standard stream log files to *.log

2022-03-22 Thread Andrea Bolognani
On Tue, Mar 22, 2022 at 01:11:22PM +0100, Erik Skultety wrote: > Since GitLab sets a proper MIME for .txt and .log file extensions, > rename all Avocado log files without an extension to *.log . This pairs > nicely with the coredumpctl info file which we already name as > 'coredumpctl.txt' because

Re: [libvirt PATCH] ci: integration: Rename all Avocado standard stream log files to *.log

2022-03-22 Thread Erik Skultety
On Tue, Mar 22, 2022 at 01:47:32PM +, Daniel P. Berrangé wrote: > On Tue, Mar 22, 2022 at 01:11:22PM +0100, Erik Skultety wrote: > > By default, stdout/stderr Avocado test log files do not have any file > > extension which confuses GitLab's web UI to mangle the MIME type for > > these and so

Re: [libvirt PATCH] keycodemapdb: Update git submodule

2022-03-22 Thread Michal Prívozník
On 3/22/22 14:51, Andrea Bolognani wrote: > This brings in all the fixes made since April 2020. > > Signed-off-by: Andrea Bolognani > --- > src/keycodemapdb | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/src/keycodemapdb b/src/keycodemapdb > index

[PATCH v2 1/7] qemu_tpm: Do async IO when starting swtpm emulator

2022-03-22 Thread Michal Privoznik
When vTPM is secured via virSecret libvirt passes the secret value via an FD when swtpm is started (arguments --key and --migration-key). The writing of the secret into the FDs is handled via virCommand, specifically qemu_tpm calls virCommandSetSendBuffer()) and then virCommandRunAsync() spawns a

[PATCH v2 7/7] commandtest: Use virTestCompareToFile() in checkoutput()

2022-03-22 Thread Michal Privoznik
In the commandtest there is checkoutput() function which checks the latest log of commandhelper (containing things like cmd line arguments, env vars, FDs, CWD, etc.) and compares that against expected output. Well, the way this function implements that is effectively by open coding

[PATCH v2 6/7] commandtest: Test virCommandSetSendBuffer() with virCommandDoAsyncIO()

2022-03-22 Thread Michal Privoznik
Introduce a test case which ensures that a daemonized process can work with virCommandSetSendBuffer() when async IO is enabled. Signed-off-by: Michal Privoznik --- tests/commanddata/test29.log | 19 tests/commandtest.c | 84 2 files

[PATCH v2 5/7] virCommandSetSendBuffer: Take double pointer of @buffer

2022-03-22 Thread Michal Privoznik
The virCommandSetSendBuffer() function consumes passed @buffer, but takes it only as plain pointer. Switch to a double pointer to make this obvious. This allows us then to drop all g_steal_pointer() in callers. Signed-off-by: Michal Privoznik --- src/qemu/qemu_tpm.c | 2 +-

[PATCH v2 3/7] vircommand: Don't set nonblocking FDs in virCommandDoAsyncIO()

2022-03-22 Thread Michal Privoznik
Back in 2013 (v1.0.3-rc1~235) when I introduced virCommandDoAsyncIO() things were way different than today. We had one event loop for everything and asynchronous IO for virCommand was handled by the event loop. Therefore, it made sense to enable VIR_EXEC_NONBLOCK flag alongside with

[PATCH v2 4/7] commandtest: Use unsigned char in test27()

2022-03-22 Thread Michal Privoznik
In test27() the virCommandSetSendBuffer() is used, which expects unsigned char. Use that type for variables which are passed to the function. Signed-off-by: Michal Privoznik --- tests/commandtest.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git

[PATCH v2 2/7] vircommand: Document virCommandSetSendBuffer() behaviour wrt daemonize

2022-03-22 Thread Michal Privoznik
When virCommandSetSendBuffer() is used over a virCommand that is (or will be) daemonized, then VIR_EXEC_ASYNC_IO the command must have VIR_EXEC_ASYNC_IO flag set no later than at virCommandRunAsync() phase so that the thread that's doing IO is spawned and thus buffers can be sent to the process.

[PATCH v2 0/7] Do async IO when starting swtpm emulator

2022-03-22 Thread Michal Privoznik
v2 of: https://listman.redhat.com/archives/libvir-list/2022-March/229433.html diff to v1: - More fixes - New test case, yay! Michal Prívozník (7): qemu_tpm: Do async IO when starting swtpm emulator vircommand: Document virCommandSetSendBuffer() behaviour wrt daemonize vircommand: Don't

[PATCH 2/4] qemu_capabilities: Detect memory-backend-*.prealloc-threads property

2022-03-22 Thread Michal Privoznik
The prealloc-threads is property of memory-backend class which is parent to the other three classes memory-backend-{ram,file,memfd}. Therefore the property is present for all, or none if QEMU is older than v5.0.0-rc0~75^2~1^2~3 which introduced the property. Anyway, the .reserve property is the

[PATCH 4/4] qemu_command: Generate prealloc-threads property

2022-03-22 Thread Michal Privoznik
Let's generate prealloc-threads property onto the cmd line if domain configuration requests so. Signed-off-by: Michal Privoznik --- src/qemu/qemu_command.c | 5 - tests/qemuxml2argvdata/memfd-memory-numa.x86_64-latest.args | 4 ++-- 2 files changed, 6

[PATCH 3/4] qemu_validate: Validate prealloc threads against qemuCpas

2022-03-22 Thread Michal Privoznik
Only fairly new QEMUs are capable of user provided number of preallocation threads. Validate this assumption. Signed-off-by: Michal Privoznik --- src/qemu/qemu_validate.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c index

[PATCH 1/4] conf: Introduce memory allocation threads

2022-03-22 Thread Michal Privoznik
Since its commit v5.0.0-rc0~75^2~1^2~3 QEMU is capable of specifying number of threads used to allocate memory. While it defaults to the number of vCPUs, users might want to use a different value (especially for humongous guests with gigantic pages). In general, on QEMU cmd line level it is

[PATCH 0/4] Introduce memory allocation threads

2022-03-22 Thread Michal Privoznik
*** BLURB HERE *** Michal Prívozník (4): conf: Introduce memory allocation threads qemu_capabilities: Detect memory-backend-*.prealloc-threads property qemu_validate: Validate prealloc threads against qemuCpas qemu_command: Generate prealloc-threads property docs/formatdomain.rst

[libvirt PATCH] keycodemapdb: Update git submodule

2022-03-22 Thread Andrea Bolognani
This brings in all the fixes made since April 2020. Signed-off-by: Andrea Bolognani --- src/keycodemapdb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/keycodemapdb b/src/keycodemapdb index 27acf0ef82..57ba70da53 16 --- a/src/keycodemapdb +++ b/src/keycodemapdb @@

Re: [libvirt PATCH] ci: integration: Rename all Avocado standard stream log files to *.log

2022-03-22 Thread Daniel P . Berrangé
On Tue, Mar 22, 2022 at 01:11:22PM +0100, Erik Skultety wrote: > By default, stdout/stderr Avocado test log files do not have any file > extension which confuses GitLab's web UI to mangle the MIME type for > these and so the browser will never offer the option to open such file > from in a text

[libvirt PATCH] ci: integration: Rename all Avocado standard stream log files to *.log

2022-03-22 Thread Erik Skultety
By default, stdout/stderr Avocado test log files do not have any file extension which confuses GitLab's web UI to mangle the MIME type for these and so the browser will never offer the option to open such file from in a text editor rather than dowloading it. Since GitLab sets a proper MIME for

Re: [libvirt PATCH 2/2] ci: integration: Collect stack traces with coredumpctl

2022-03-22 Thread Erik Skultety
On Tue, Mar 22, 2022 at 10:53:27AM +, Daniel P. Berrangé wrote: > On Tue, Mar 22, 2022 at 07:52:21AM +0100, Erik Skultety wrote: > > Some Red Hat-like distros have cores limited with a soft limit of 0 > > which means that neither a stack trace nor a core file will be > > available. Since we

[PATCH v2 0/6] qemu: Add qemu-namespace XML to override device properties (-set replacement)

2022-03-22 Thread Peter Krempa
v2: - patch 1/7 from last time was pushed - XML changes: - top level element renamed to "qemu:override" to accomodate possible extension in the future - wrapped 'qemu:props' meant to override the frontend under a 'qemu:frontend' element - code changes: - renamed few bits

[PATCH v2 6/6] NEWS: Mention the qemu device property override feature

2022-03-22 Thread Peter Krempa
Signed-off-by: Peter Krempa --- NEWS.rst | 7 +++ 1 file changed, 7 insertions(+) diff --git a/NEWS.rst b/NEWS.rst index b9b9807625..754687ac4d 100644 --- a/NEWS.rst +++ b/NEWS.rst @@ -25,6 +25,13 @@ v8.2.0 (unreleased) migration allowing migration to proceed even when the user expects

[PATCH v2 5/6] qemu: command: Override device definition according to the namespace config

2022-03-22 Thread Peter Krempa
Apply the user-requested changes to the device definition as requested by the element from the custom qemu XML namespace. Closes: https://gitlab.com/libvirt/libvirt/-/issues/287 Signed-off-by: Peter Krempa --- src/qemu/qemu_command.c | 34 ++-

[PATCH v2 2/6] qemu: domain: Add XML namespace code for overriding device config

2022-03-22 Thread Peter Krempa
Implement the XML parser and formatter for overriding of device properties such as: Signed-off-by: Peter Krempa --- src/conf/schemas/domaincommon.rng | 42 src/qemu/qemu_domain.c

[PATCH v2 4/6] qemuBuildDeviceCommandlineFromJSON: Pass 'virDomainDef' into the function

2022-03-22 Thread Peter Krempa
The definition object will be later used to access the qemu namespace definition used to override device properties. Signed-off-by: Peter Krempa --- src/qemu/qemu_command.c | 101 +--- 1 file changed, 53 insertions(+), 48 deletions(-) diff --git

[PATCH v2 3/6] conf: Introduce VIR_DOMAIN_TAINT_CUSTOM_DEVICE and use it in qemu

2022-03-22 Thread Peter Krempa
Taint the domain object when the user requests custom device properties. Signed-off-by: Peter Krempa --- src/conf/domain_conf.c | 2 ++ src/conf/domain_conf.h | 1 + src/qemu/qemu_domain.c | 3 +++ 3 files changed, 6 insertions(+) diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c

[PATCH v2 1/6] docs: drvqemu: Document overriding of device properties

2022-03-22 Thread Peter Krempa
Upcoming patches will add possibility to override configuration of a device with custom properties as a more versatile replacement to using QEMU's '-set' parameter, which doesn't work when we use JSON to instantiate devices. Describe the XML used for the override as well as expectations of

Re: [PATCH] gitpublish: Drop "libvirt" from prefix

2022-03-22 Thread Andrea Bolognani
On Tue, Mar 22, 2022 at 10:35:41AM +, Daniel P. Berrangé wrote: > On Tue, Mar 22, 2022 at 11:11:42AM +0100, Andrea Bolognani wrote: > > We're now using GitLab MRs for all subprojects, which makes > > having "libvirt" in the prefix no longer necessary and a > > pointless deviation from

Re: [libvirt PATCH 2/2] ci: integration: Collect stack traces with coredumpctl

2022-03-22 Thread Daniel P . Berrangé
On Tue, Mar 22, 2022 at 07:52:21AM +0100, Erik Skultety wrote: > Some Red Hat-like distros have cores limited with a soft limit of 0 > which means that neither a stack trace nor a core file will be > available. Since we want the stack trace we need to set the core limit > with systemd globally to

Re: [PATCH] gitpublish: Drop "libvirt" from prefix

2022-03-22 Thread Daniel P . Berrangé
On Tue, Mar 22, 2022 at 11:11:42AM +0100, Andrea Bolognani wrote: > We're now using GitLab MRs for all subprojects, which makes > having "libvirt" in the prefix no longer necessary and a > pointless deviation from git-format-patch's default, which > some people still use. > > Signed-off-by:

Re: [PATCH] gitpublish: Drop "libvirt" from prefix

2022-03-22 Thread Pavel Hrdina
On Tue, Mar 22, 2022 at 11:11:42AM +0100, Andrea Bolognani wrote: > We're now using GitLab MRs for all subprojects, which makes > having "libvirt" in the prefix no longer necessary and a > pointless deviation from git-format-patch's default, which > some people still use. > > Signed-off-by:

[PATCH] gitpublish: Drop "libvirt" from prefix

2022-03-22 Thread Andrea Bolognani
We're now using GitLab MRs for all subprojects, which makes having "libvirt" in the prefix no longer necessary and a pointless deviation from git-format-patch's default, which some people still use. Signed-off-by: Andrea Bolognani --- .gitpublish | 2 +- 1 file changed, 1 insertion(+), 1

[PATCH] docs: Update obsolete reference to formatdomain.html.in

2022-03-22 Thread Andrea Bolognani
Signed-off-by: Andrea Bolognani --- Pushed as trivial. docs/api_extension.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/api_extension.rst b/docs/api_extension.rst index d70c244fd4..152e64487e 100644 --- a/docs/api_extension.rst +++ b/docs/api_extension.rst @@

Re: [libvirt PATCH v2] schemas: move out of docs, fix no-docs build

2022-03-22 Thread Andrea Bolognani
On Tue, Mar 22, 2022 at 09:18:57AM +0100, Claudio Fontana wrote: > schemas are used for more than just documentation, > virsh edit fails if schemas are not available. > > Therefore, fix the no-docs build by moving schemas/ > to the parsing code inside src/conf/. > > Signed-off-by: Claudio Fontana

Re: [PATCH] qemu_tpm: Do async IO when starting swtpm emulator

2022-03-22 Thread Michal Prívozník
On 3/21/22 20:56, Marc-André Lureau wrote: > Hi > > On Mon, Mar 21, 2022 at 6:59 PM Michal Privoznik > wrote: > > When vTPM is secured via virSecret libvirt passes the secret > value via an FD when swtpm is started (arguments --key and > --migration-key).

[libvirt PATCH v2] schemas: move out of docs, fix no-docs build

2022-03-22 Thread Claudio Fontana
schemas are used for more than just documentation, virsh edit fails if schemas are not available. Therefore, fix the no-docs build by moving schemas/ to the parsing code inside src/conf/. Signed-off-by: Claudio Fontana --- changes v1 -> v2: * move to src/conf/ instead of top_srcdir (Daniel) *

Re: [PATCH 3/7] qemu: domain: Add XML namespace code for overriding device config

2022-03-22 Thread Peter Krempa
On Mon, Mar 21, 2022 at 15:34:46 +, Daniel P. Berrangé wrote: > On Mon, Mar 21, 2022 at 04:24:36PM +0100, Peter Krempa wrote: > > Implement the XML parser and formatter for overriding of device > > properties such as: > > > > > > s/deviceOverride/devices/ > > > > > > >

Re: [libvirt PATCH 0/2] [CI] Fix running integration pipeline in forks + collect daemon stack trace

2022-03-22 Thread Michal Prívozník
On 3/22/22 07:52, Erik Skultety wrote: > Even if users set the LIBVIRT_CI_INTEGRATION variable to 1 to enable the > integration test suite with their own private runner, the pipeline will be > stuck > because we expect the runner to have the 'redhat-vm-host' tag. We need to > define > the job

[libvirt PATCH 1/2] ci: Define the integration job tag dynamically via a variable

2022-03-22 Thread Erik Skultety
Custom runners are private to a project, so naturally forks cannot run any workloads on these. The integration test suite which requires access to our custom runner is naturally disabled on forks and can be enabled by setting LIBVIRT_CI_INTEGRATION=1. The problem is that the current integration

[libvirt PATCH 2/2] ci: integration: Collect stack traces with coredumpctl

2022-03-22 Thread Erik Skultety
Some Red Hat-like distros have cores limited with a soft limit of 0 which means that neither a stack trace nor a core file will be available. Since we want the stack trace we need to set the core limit with systemd globally to unlimited/infinity. Signed-off-by: Erik Skultety ---

[libvirt PATCH 0/2] [CI] Fix running integration pipeline in forks + collect daemon stack trace

2022-03-22 Thread Erik Skultety
Even if users set the LIBVIRT_CI_INTEGRATION variable to 1 to enable the integration test suite with their own private runner, the pipeline will be stuck because we expect the runner to have the 'redhat-vm-host' tag. We need to define the job tag dynamically instead, so that the users can set the