On Mon, Aug 28, 2023 at 04:44:35PM -0500, Jonathon Jongsma wrote:
> This is the seventh version of this patch series. See
> https://bugzilla.redhat.com/show_bug.cgi?id=2016527 for more information.
>
> Note that testing this requires selinux policy changes which are not fully
> done, but there is
We were testing the arguments that were being passed to qemu when a disk
was being served by nbdkit, but the arguments used to start nbdkit
itself were not testable. This adds a test to ensure that we're invoking
nbdkit correctly for various disk source definitions.
Signed-off-by: Jonathon Jongsma
Since the libvirt documentation suggests to prefer GObject over
virObject, and since virObject is a GObject, change virFileCache to
allow GObjects as data.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/util/virfilecache.c | 14 --
src/util/virfilecache.h | 2 +-
Right now, ssh network disks are not usable. There is some basic support
in libvirt that is meant to support disk chains that have backing disks
located at ssh urls, but there is no real way for a user to configure a
ssh-based disk. This commit allows users to configure an ssh disk with
password a
For ssh disks that are served by nbdkit, we can support logging in with
an ssh key file. Pass the path to the configured key file and the
username to the nbdkit process.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/conf/domain_conf.c| 30
Add xml to the private data for a disk source to represent the nbdkit
process so that the state can be re-created if the libvirt daemon is
restarted. Format:
/path/to/nbdkit.pid
/path/to/nbdkit.socket
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/qemu/qemu
Preparatory step for caching nbdkit capabilities. This patch implements
the newData and isValid virFileCacheHandlers callback functions.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/qemu/qemu_nbdkit.c | 89 +-
src/qemu/qemu_nbdkit.h |
log stderr and stdout from nbdkit into its own log so that
nbdkit-related issues can be debugged more easily.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/qemu/qemu_nbdkit.c | 16 +++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/src/qemu/qemu_n
Add the ability to specify a path to a ssh-agent socket in order to use
the ssh-agent to authenticate to remote ssh disks. Example
configuration:
...
...
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
docs/formatdomain
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
tests/qemunbdkitdata/disk-network-source-curl.args.disk1 | 4 +++-
.../disk-network-source-curl.args.disk1.pipe.780 | 2 +-
.../disk-network-source-curl.args.disk1.pipe.782 | 1 +
tests/qemunbdkitdata/di
All users of virCommandSetSendBuffer() are using it to send sensitive
data to a child process. So, since these buffers contain sensitive
information, clear it with virSecureErase().
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/util/vircommand.c | 2 ++
1 file changed, 2 ins
Authenticating via key file to an ssh server is often preferable to
logging in via password. In order to support this functionality add a
new xml element for ssh disks that allows the user to specify
a keyfile and username. Example configuration:
...
...
Rather than passing passwords and cookies (which could contain
passwords) to nbdkit via commandline arguments, use the alternate format
that nbdkit supports where we can specify a file descriptor which nbdkit
will read to get the password or cookies.
Signed-off-by: Jonathon Jongsma
Reviewed-by: P
Implement the loadFile and saveFile virFileCacheHandlers callbacks so
that nbdkit capabilities are cached perstistently across daemon
restarts. The format and implementation is modeled on the qemu
capabilities, but simplified slightly.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
--
Add a private function to peek at the list of send buffers in virCommand
so that it is testable
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/libvirt_private.syms | 1 +
src/util/vircommand.c | 17 +
src/util/vircommand.h | 8
src/util/vir
Adds the ability to monitor the nbdkit process so that we can take
action in case the child exits unexpectedly.
When the nbdkit process exits, we pause the vm, restart nbdkit, and then
resume the vm. This allows the vm to continue working in the event of a
nbdkit failure.
Eventually we may want t
Require libnbd-devel when building the qemu driver, recommend nbdkit
packages.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
libvirt.spec.in | 8
1 file changed, 8 insertions(+)
diff --git a/libvirt.spec.in b/libvirt.spec.in
index 7157cfe3b4..94cae34496 100644
--- a/li
For virStorageSource objects that contain an nbdkitProcess, start that
nbdkit process to serve that network drive and then pass the nbdkit
socket to qemu rather than sending the network url to qemu directly.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/qemu/qemu_block.c
For ssh disks that are served by nbdkit, lookup the password from the
configured secret and securely pass it to the nbdkit process using fd
passing.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/qemu/qemu_nbdkit.c| 84 ++-
.../disk-net
It's not possible to use password-protected ssh keys directly with
libvirt because libvirt doesn't have any way to prompt a user for the
password. To accomodate password-protected key files, an administrator
can add these keys to an ssh agent and then configure the domain with
the path to the ssh-a
Since the restart handler will trigger at an arbitrary time (when the
nbdkit process crashes, for instance), it's difficult to provide
feedback to the user if the restart is unsuccessful. Rather than just
relying on a warning in the log, taint the domain so that there will be
a slightly more user-v
This code can be used by the nbdkit implementation for reading back
filtered log data for error reporting. Move it to qemuLogContext so that
it can be shared. Renamed to qemuLogContextReadFiltered().
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/qemu/qemu_logcontext.c | 65 +
This prepares encryption secrets and authentication secrets. When we add
nbdkit-backed network storage sources, we will not need to send
authentication secrets to qemu, since they will be sent to nbdkit
instead. So split this into two different functions.
Signed-off-by: Jonathon Jongsma
Reviewed-
For ssh disks that are served by nbdkit, use the configured value for
knownHosts and pass it to the nbdkit process.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/conf/domain_conf.c| 8 ++
src/conf/storage_source_conf.c| 2 ++
src
When using nbdkit to serve a network disk source, the nbdkit process
will start and wait for an nbd connection before actually attempting to
connect to the (remote) disk location. Because of this, nbdkit will not
report an error until after qemu is launched and tries to read from the
disk. This res
Add the virFileCache implementation for nbdkit capabilities to the qemu
driver. This allows us to determine whether nbdkit is installed and
which plugins are supported. it also has persistent caching and the
capabilities are re-queried whenever something changes.
Signed-off-by: Jonathon Jongsma
R
In order to make ssh disks usable, we need to be able to validate a
remote host. To do this, add a xml element for ssh disks to
allow the user to specify a location for a file that contains known host
keys. Implementation to follow.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
Add some helper functions to build a virCommand object and run the
nbdkit process for a given virStorageSource.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/qemu/qemu_nbdkit.c | 250 +
src/qemu/qemu_nbdkit.h | 10 ++
2 files changed,
Add new DO_TEST_CAPS_LATEST_NBDKIT macro to test xml2argv for various
nbdkit capability scenarios.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/qemu/qemu_nbdkit.c | 20 +---
tests/qemuxml2argvtest.c | 11 +++
tests/testutilsqemu.c| 26 +
Allow to specify a basename for the log file so that
qemuDomainLogContextNew() can be used to create log contexts for
secondary loggers.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/qemu/qemu_domain.c | 5 +++--
src/qemu/qemu_domain.h | 3 ++-
src/qemu/qemu_process.c | 2
This will allow us to use it for nbdkit logging in upcoming commits.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
po/POTFILES| 1 +
src/qemu/meson.build | 1 +
src/qemu/qemu_domain.c | 247 ++
src/qemu/qemu_domain.h
An object for storing information about a nbdkit process that is serving
a specific virStorageSource. At the moment, this information is just
stored in the private data of virStorageSource and not used at all.
Future commits will use this data to actually start a nbdkit process.
Signed-off-by: Jon
Rather than hard-coding the nbdkit module directory, query the nbdkit
binary for the location to these directories. nbdkit provides a
--dump-config optiont that outputs this information and can be easily
parsed. We can also get the version from this output rather than
executing `nbdkit --version` s
In order to add caching of the nbdkit capabilities, we will need to
compare against file modification times, etc. So look up this
information when creating the nbdkit caps.
Add a nbdkit_moddir build option to allow the builder to specify the
location to look for nbdkit plugins and filters.
Signed
There was support in the code for parsing protocol='ssh' on network disk
sources, but it was not present in the xml schema. Add this to the
schema.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/conf/schemas/domaincommon.rng | 1 +
tests/qemublocktest.c
In future commits, we will optionally use nbdkit to serve some remote
disk sources. This patch queries to see whether nbdkit is installed on
the host and queries it for capabilities. The data will be used in later
commits.
Signed-off-by: Jonathon Jongsma
Reviewed-by: Peter Krempa
---
src/qemu/m
This is the seventh version of this patch series. See
https://bugzilla.redhat.com/show_bug.cgi?id=2016527 for more information.
Note that testing this requires selinux policy changes which are not fully
done, but there is a new policy in development that has allowed me to run with
selinux in enfor
On Wed, Aug 16, 2023 at 05:41:37PM -0600, x...@trimaso.com.mx wrote:
I'll be direct.
After learning that QEMU itself has never officially implemented options
such as "-vcpu", "-vcpupin", or similar, remaining only as unimplemented
commits at their mailing lists, I felt the need of asking here on
Rewrite the old-style parser to use virXMLNodeGetSubelementList
Signed-off-by: Peter Krempa
---
src/conf/domain_conf.c | 17 +
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 3a08034b9d..02bba77768 100644
--- a/s
Rewrite the old-style parser to use virXMLNodeGetSubelementList
Signed-off-by: Peter Krempa
---
src/conf/domain_conf.c | 13 +++--
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index bb4f1fdb94..d510279472 100644
--- a/src/c
Peter Krempa (5):
virNetworkDNSHostDefParseXML: Refactor parsing
virsh: domain: Refactor XML handling for disk changes
virDomainFeaturesCapabilitiesDefParse: Use virXMLNodeGetSubelementList
virDomainFeaturesKVMDefParse: Use virXMLNodeGetSubelementList
virDomainFeaturesXENDefParse: Use vir
Use 'virXMLNodeGetSubelementList' instead of looping through XML nodes
and modernize the code.
Signed-off-by: Peter Krempa
---
src/conf/network_conf.c | 94 -
1 file changed, 46 insertions(+), 48 deletions(-)
diff --git a/src/conf/network_conf.c b/src/con
Use virXMLNodeGetSubelement to find needed subelements.
In virshUpdateDiskXML this commit removes the code which keeps XML
formatting tidy, but that is not needed for the code to format proper
XMLs.
Signed-off-by: Peter Krempa
---
tools/virsh-domain.c | 63 +-
Rewrite the old-style parser to use virXMLNodeGetSubelementList
Signed-off-by: Peter Krempa
---
src/conf/domain_conf.c | 20 +---
1 file changed, 9 insertions(+), 11 deletions(-)
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index d510279472..3a08034b9d 100644
---
Nowadays all tests were considered 'modern' so it makes no longer sense
to have that field.
Signed-off-by: Peter Krempa
---
tests/qemuhotplugtest.c | 40 ++--
1 file changed, 14 insertions(+), 26 deletions(-)
diff --git a/tests/qemuhotplugtest.c b/tests/qemuh
Upcoming patch will re-probe machines from the current qemu instance to
populate the private copy of qemuCaps after reconnecting to a running
instance. This is needed to be able to access the machine type data,
while storing them in the status XML seems to be an overkill, for
information which can
All qemu versions have that command and cpu hotplug code now directly
probes the machine type.
Signed-off-by: Peter Krempa
---
src/qemu/qemu_capabilities.c | 8 ++--
src/qemu/qemu_capabilities.h | 2 +-
tests/qemucapabilitiesdata/caps_4.2.0_a
The filtering of qemu capabilities by machine type doesn't seem to be
ever used, remove it and adjust callers.
Signed-off-by: Peter Krempa
---
src/qemu/qemu_capabilities.c | 41 +---
src/qemu/qemu_capabilities.h | 7 +-
src/qemu/qemu_domain.c | 4 +---
When reconnecting we populate only the capability flags from the XML as
we need to know the exact flags that were present when starting the VM.
On the other hand the machine type data is not stored as it wasn't
really used after startup. While storing all of the data into the status
XML would be t
The tests were using a copy of a x86_64 based XML and thus
'qemuhotplugtest' was selecting wrong capabilities to use for that
specific test.
Signed-off-by: Peter Krempa
---
.../ppc64-modern-bulk-domain.xml | 4 ++--
.../ppc64-modern-bulk-result-conf.xml | 18 ++-
The QEMU_CAPS_QUERY_HOTPLUGGABLE_CPUS flag is always asserted as all
qemu versions support the command and selectively cleared when copying
the capabilities for VM use if given machine type does not support cpu
hotplug.
Rework this to directly probe the machine as we now populate the data
also whe
Signed-off-by: Peter Krempa
---
...args => cpu-hotplug-startup.x86_64-latest.args} | 14 --
tests/qemuxml2argvtest.c | 2 +-
2 files changed, 9 insertions(+), 7 deletions(-)
rename tests/qemuxml2argvdata/{cpu-hotplug-startup.args =>
cpu-hotplug-startup.x86
Support for legacy cpu hotplug was removed a long time ago. At this
point this function only checks whether the current machine type
supports cpu hotplug.
Signed-off-by: Peter Krempa
---
src/qemu/qemu_domain.c | 4 ++--
src/qemu/qemu_domain.h | 2 +-
src/qemu/qemu_hotplug.c | 4 ++--
3 files c
Nowadays all qemu's support the command which was used as witness, but
was gated on machine type's support of vCPU hotplug. Directly probe the
machine type.
Peter Krempa (9):
qemu: Rename qemuDomainSupportsNewVcpuHotplug to
qemuDomainSupportsVcpuHotplug
qemu: capabilities: Export functio
This change was supposed to be part of commit 120a674f , but was
proposed against the libvirt TCK project instead. Since we're running
the TCK test suite as part of this project, this is the right place for
the TCK runtime deps list config.
Signed-off-by: Erik Skultety
---
ci/lcitool/projects/li
I have just tagged v9.7.0-rc1 in the repository and pushed signed
tarballs and source RPMs to https://download.libvirt.org/
Please give the release candidate some testing and in case you find a
serious issue which should have a fix in the upcoming release, feel
free to reply to this thread to make
56 matches
Mail list logo
