Re: [PATCH v5 4/5] qmp: Added new command to retrieve eBPF blob.
Andrew Melnichenko writes: > Hi all, > Thanks for the comments - I'll update and send new patches. > > On Sat, Aug 5, 2023 at 10:34 AM Markus Armbruster wrote: >> >> Andrew Melnychenko writes: >> >> > Now, the binary objects may be retrieved by id. >> > It would require for future qmp commands that may require specific >> > eBPF blob. >> > >> > Added command "request-ebpf". This command returns >> > eBPF program encoded base64. The program taken from the >> > skeleton and essentially is an ELF object that can be >> > loaded in the future with libbpf. >> > >> > The reason to use the command to provide the eBPF object >> > instead of a separate artifact was to avoid issues related >> > to finding the eBPF itself. eBPF object is an ELF binary >> > that contains the eBPF program and eBPF map description(BTF). >> > Overall, eBPF object should contain the program and enough >> > metadata to create/load eBPF with libbpf. As the eBPF >> > maps/program should correspond to QEMU, the eBPF can't >> > be used from different QEMU build. >> > >> > The first solution was a helper that comes with QEMU >> > and loads appropriate eBPF objects. And the issue is >> > to find a proper helper if the system has several >> > different QEMUs installed and/or built from the source, >> > which helpers may not be compatible. >> > >> > Another issue is QEMU updating while there is a running >> > QEMU instance. With an updated helper, it may not be >> > possible to hotplug virtio-net device to the already >> > running QEMU. Overall, requesting the eBPF object from >> > QEMU itself solves possible failures with acceptable effort. >> > >> > Links: >> > [PATCH 3/5] qmp: Added the helper stamp check. >> > https://lore.kernel.org/all/20230219162100.174318-4-and...@daynix.com/ >> > >> > Signed-off-by: Andrew Melnychenko >> > --- >> >> [...] >> >> > diff --git a/qapi/ebpf.json b/qapi/ebpf.json >> > new file mode 100644 >> > index 000..40851f8c177 >> > --- /dev/null >> > +++ b/qapi/ebpf.json [...] >> > +## >> > +# @request-ebpf: >> > +# >> > +# Returns eBPF object that can be loaded with libbpf. >> > +# Management applications (g.e. libvirt) may load it and pass file >> > +# descriptors to QEMU. Which allows running QEMU without BPF capabilities. >> > +# It's crucial that eBPF program/map is compatible with QEMU, so it's >> > +# provided through QMP. >> > +# >> > +# Returns: RSS eBPF object encoded in base64. >> >> What does "RSS" mean? > > RSS - Receive-side Scaling. Suggest to use something like "receive-side scaling (RSS)" the first time. You could also put a general introduction right below the header, like ## # = eBPF Objects # # Text goes here ## This is not a demand. [...]
Re: [PATCH v5 4/5] qmp: Added new command to retrieve eBPF blob.
Hi all,
Thanks for the comments - I'll update and send new patches.
On Sat, Aug 5, 2023 at 10:34 AM Markus Armbruster wrote:
>
> Andrew Melnychenko writes:
>
> > Now, the binary objects may be retrieved by id.
> > It would require for future qmp commands that may require specific
> > eBPF blob.
> >
> > Added command "request-ebpf". This command returns
> > eBPF program encoded base64. The program taken from the
> > skeleton and essentially is an ELF object that can be
> > loaded in the future with libbpf.
> >
> > The reason to use the command to provide the eBPF object
> > instead of a separate artifact was to avoid issues related
> > to finding the eBPF itself. eBPF object is an ELF binary
> > that contains the eBPF program and eBPF map description(BTF).
> > Overall, eBPF object should contain the program and enough
> > metadata to create/load eBPF with libbpf. As the eBPF
> > maps/program should correspond to QEMU, the eBPF can't
> > be used from different QEMU build.
> >
> > The first solution was a helper that comes with QEMU
> > and loads appropriate eBPF objects. And the issue is
> > to find a proper helper if the system has several
> > different QEMUs installed and/or built from the source,
> > which helpers may not be compatible.
> >
> > Another issue is QEMU updating while there is a running
> > QEMU instance. With an updated helper, it may not be
> > possible to hotplug virtio-net device to the already
> > running QEMU. Overall, requesting the eBPF object from
> > QEMU itself solves possible failures with acceptable effort.
> >
> > Links:
> > [PATCH 3/5] qmp: Added the helper stamp check.
> > https://lore.kernel.org/all/20230219162100.174318-4-and...@daynix.com/
> >
> > Signed-off-by: Andrew Melnychenko
> > ---
>
> [...]
>
> > diff --git a/qapi/ebpf.json b/qapi/ebpf.json
> > new file mode 100644
> > index 000..40851f8c177
> > --- /dev/null
> > +++ b/qapi/ebpf.json
> > @@ -0,0 +1,57 @@
> > +# -*- Mode: Python -*-
> > +# vim: filetype=python
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> > +# See the COPYING file in the top-level directory.
> > +
> > +##
> > +# = eBPF Objects
> > +##
> > +
> > +{ 'include': 'common.json' }
>
> This looks superfluous.
>
> > +
> > +##
> > +# @EbpfObject:
> > +#
> > +# Structure that holds eBPF ELF object encoded in base64.
> > +#
> > +# Since: 8.3
>
> 8.2
>
> More of the same below, not noting it again.
>
> > +#
> > +##
>
> You're not documenting member @object. Leaving a member undocumented
> should be a hard error. It isn't only because we have hundreds of
> instances to fix.
>
> Generated documentation looks like
>
> "EbpfObject" (Object)
> -
>
> Structure that holds eBPF ELF object encoded in base64.
>
>
> Members
> ~~~
>
> "object": "string"
>Not documented
>
> [...]
>
> This isn't what you want :)
>
> Better:
>
>##
># @EbpfObject:
>#
># An eBPF ELF object.
>#
># @object: the eBPF object encoded in base64
>#
># Since: 8.2
>##
>
> > +{ 'struct': 'EbpfObject',
> > + 'data': {'object': 'str'},
> > + 'if': 'CONFIG_EBPF' }
> > +
> > +##
> > +# @EbpfProgramID:
> > +#
> > +# The eBPF programs that can be gotten with request-ebpf.
> > +#
> > +# @rss: Receive side scaling, technology that allows steering traffic
> > +# between queues by calculation hash. Users may set up indirection table
> > +# and hash/packet types configurations. Used with virtio-net.
>
> Please format like
>
># @rss: Receive side scaling, technology that allows steering traffic
># between queues by calculation hash. Users may set up
># indirection table and hash/packet types configurations. Used
># with virtio-net.
>
> to blend in with recent commit a937b6aa739 (qapi: Reformat doc comments
> to conform to current conventions).
Thank you, I'll check it!
>
> > +#
> > +# Since: 8.3
> > +##
> > +{ 'enum': 'EbpfProgramID',
> > + 'if': 'CONFIG_EBPF',
> > + 'data': [ { 'name': 'rss' } ] }
> > +
> > +##
> > +# @request-ebpf:
> > +#
> > +# Returns eBPF object that can be loaded with libbpf.
> > +# Management applications (g.e. libvirt) may load it and pass file
> > +# descriptors to QEMU. Which allows running QEMU without BPF capabilities.
> > +# It's crucial that eBPF program/map is compatible with QEMU, so it's
> > +# provided through QMP.
> > +#
> > +# Returns: RSS eBPF object encoded in base64.
>
> What does "RSS" mean?
RSS - Receive-side Scaling.
>
> > +#
> > +# Since: 8.3
> > +#
> > +##
>
> You're not documenting argument @id.
>
> Generated documentation looks like
>
> "request-ebpf" (Command)
>
>
> Returns eBPF object that can be loaded with libbpf. Management
> applications (g.e. libvirt) may load it and pass file descriptors to
> QEMU. Which allows running QEMU without BPF capabilities. It's crucial
> that eBPF program/map is compatible with QEMU, so it's provided
Re: [PATCH v5 4/5] qmp: Added new command to retrieve eBPF blob.
Andrew Melnychenko writes:
> Now, the binary objects may be retrieved by id.
> It would require for future qmp commands that may require specific
> eBPF blob.
>
> Added command "request-ebpf". This command returns
> eBPF program encoded base64. The program taken from the
> skeleton and essentially is an ELF object that can be
> loaded in the future with libbpf.
>
> The reason to use the command to provide the eBPF object
> instead of a separate artifact was to avoid issues related
> to finding the eBPF itself. eBPF object is an ELF binary
> that contains the eBPF program and eBPF map description(BTF).
> Overall, eBPF object should contain the program and enough
> metadata to create/load eBPF with libbpf. As the eBPF
> maps/program should correspond to QEMU, the eBPF can't
> be used from different QEMU build.
>
> The first solution was a helper that comes with QEMU
> and loads appropriate eBPF objects. And the issue is
> to find a proper helper if the system has several
> different QEMUs installed and/or built from the source,
> which helpers may not be compatible.
>
> Another issue is QEMU updating while there is a running
> QEMU instance. With an updated helper, it may not be
> possible to hotplug virtio-net device to the already
> running QEMU. Overall, requesting the eBPF object from
> QEMU itself solves possible failures with acceptable effort.
>
> Links:
> [PATCH 3/5] qmp: Added the helper stamp check.
> https://lore.kernel.org/all/20230219162100.174318-4-and...@daynix.com/
>
> Signed-off-by: Andrew Melnychenko
> ---
[...]
> diff --git a/qapi/ebpf.json b/qapi/ebpf.json
> new file mode 100644
> index 000..40851f8c177
> --- /dev/null
> +++ b/qapi/ebpf.json
> @@ -0,0 +1,57 @@
> +# -*- Mode: Python -*-
> +# vim: filetype=python
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> +# See the COPYING file in the top-level directory.
> +
> +##
> +# = eBPF Objects
> +##
> +
> +{ 'include': 'common.json' }
This looks superfluous.
> +
> +##
> +# @EbpfObject:
> +#
> +# Structure that holds eBPF ELF object encoded in base64.
> +#
> +# Since: 8.3
8.2
More of the same below, not noting it again.
> +#
> +##
You're not documenting member @object. Leaving a member undocumented
should be a hard error. It isn't only because we have hundreds of
instances to fix.
Generated documentation looks like
"EbpfObject" (Object)
-
Structure that holds eBPF ELF object encoded in base64.
Members
~~~
"object": "string"
Not documented
[...]
This isn't what you want :)
Better:
##
# @EbpfObject:
#
# An eBPF ELF object.
#
# @object: the eBPF object encoded in base64
#
# Since: 8.2
##
> +{ 'struct': 'EbpfObject',
> + 'data': {'object': 'str'},
> + 'if': 'CONFIG_EBPF' }
> +
> +##
> +# @EbpfProgramID:
> +#
> +# The eBPF programs that can be gotten with request-ebpf.
> +#
> +# @rss: Receive side scaling, technology that allows steering traffic
> +# between queues by calculation hash. Users may set up indirection table
> +# and hash/packet types configurations. Used with virtio-net.
Please format like
# @rss: Receive side scaling, technology that allows steering traffic
# between queues by calculation hash. Users may set up
# indirection table and hash/packet types configurations. Used
# with virtio-net.
to blend in with recent commit a937b6aa739 (qapi: Reformat doc comments
to conform to current conventions).
> +#
> +# Since: 8.3
> +##
> +{ 'enum': 'EbpfProgramID',
> + 'if': 'CONFIG_EBPF',
> + 'data': [ { 'name': 'rss' } ] }
> +
> +##
> +# @request-ebpf:
> +#
> +# Returns eBPF object that can be loaded with libbpf.
> +# Management applications (g.e. libvirt) may load it and pass file
> +# descriptors to QEMU. Which allows running QEMU without BPF capabilities.
> +# It's crucial that eBPF program/map is compatible with QEMU, so it's
> +# provided through QMP.
> +#
> +# Returns: RSS eBPF object encoded in base64.
What does "RSS" mean?
> +#
> +# Since: 8.3
> +#
> +##
You're not documenting argument @id.
Generated documentation looks like
"request-ebpf" (Command)
Returns eBPF object that can be loaded with libbpf. Management
applications (g.e. libvirt) may load it and pass file descriptors to
QEMU. Which allows running QEMU without BPF capabilities. It's crucial
that eBPF program/map is compatible with QEMU, so it's provided
through QMP.
Arguments
~
"id": "EbpfProgramID"
Not documented
Returns
~~~
RSS eBPF object encoded in base64.
[...]
Here's my try:
##
# @request-ebpf:
#
# Retrieve an eBPF object that can be loaded with libbpf. Management
# applications (g.e. libvirt) may load it and pass file descriptors to
# QEMU, so they can run running QEMU without BPF capabilities.
#
# @id: The ID of the program to return.
#
Re: [PATCH v5 4/5] qmp: Added new command to retrieve eBPF blob.
On Wed, Aug 02, 2023 at 11:41:22PM +0300, Andrew Melnychenko wrote:
> Now, the binary objects may be retrieved by id.
> It would require for future qmp commands that may require specific
> eBPF blob.
>
> Added command "request-ebpf". This command returns
> eBPF program encoded base64. The program taken from the
> skeleton and essentially is an ELF object that can be
> loaded in the future with libbpf.
>
> The reason to use the command to provide the eBPF object
> instead of a separate artifact was to avoid issues related
> to finding the eBPF itself. eBPF object is an ELF binary
> that contains the eBPF program and eBPF map description(BTF).
> Overall, eBPF object should contain the program and enough
> metadata to create/load eBPF with libbpf. As the eBPF
> maps/program should correspond to QEMU, the eBPF can't
> be used from different QEMU build.
>
> The first solution was a helper that comes with QEMU
> and loads appropriate eBPF objects. And the issue is
> to find a proper helper if the system has several
> different QEMUs installed and/or built from the source,
> which helpers may not be compatible.
>
> Another issue is QEMU updating while there is a running
> QEMU instance. With an updated helper, it may not be
> possible to hotplug virtio-net device to the already
> running QEMU. Overall, requesting the eBPF object from
> QEMU itself solves possible failures with acceptable effort.
>
> Links:
> [PATCH 3/5] qmp: Added the helper stamp check.
> https://lore.kernel.org/all/20230219162100.174318-4-and...@daynix.com/
>
> Signed-off-by: Andrew Melnychenko
> ---
> ebpf/ebpf.c | 70 +++
> ebpf/ebpf.h | 31 +++
> ebpf/ebpf_rss.c | 6
> ebpf/meson.build | 2 +-
> qapi/ebpf.json| 57 +++
> qapi/meson.build | 1 +
> qapi/qapi-schema.json | 1 +
> 7 files changed, 167 insertions(+), 1 deletion(-)
> create mode 100644 ebpf/ebpf.c
> create mode 100644 ebpf/ebpf.h
> create mode 100644 qapi/ebpf.json
> diff --git a/qapi/ebpf.json b/qapi/ebpf.json
> new file mode 100644
> index 000..40851f8c177
> --- /dev/null
> +++ b/qapi/ebpf.json
> @@ -0,0 +1,57 @@
> +# -*- Mode: Python -*-
> +# vim: filetype=python
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> +# See the COPYING file in the top-level directory.
> +
> +##
> +# = eBPF Objects
> +##
> +
> +{ 'include': 'common.json' }
> +
> +##
> +# @EbpfObject:
> +#
> +# Structure that holds eBPF ELF object encoded in base64.
> +#
> +# Since: 8.3
We're just releasing 8.1, so next will be 8.2
There will never be any 8.3 because we reset to next major
version in the first release of each year.
With regards,
Daniel
--
|: https://berrange.com -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
[PATCH v5 4/5] qmp: Added new command to retrieve eBPF blob.
Now, the binary objects may be retrieved by id.
It would require for future qmp commands that may require specific
eBPF blob.
Added command "request-ebpf". This command returns
eBPF program encoded base64. The program taken from the
skeleton and essentially is an ELF object that can be
loaded in the future with libbpf.
The reason to use the command to provide the eBPF object
instead of a separate artifact was to avoid issues related
to finding the eBPF itself. eBPF object is an ELF binary
that contains the eBPF program and eBPF map description(BTF).
Overall, eBPF object should contain the program and enough
metadata to create/load eBPF with libbpf. As the eBPF
maps/program should correspond to QEMU, the eBPF can't
be used from different QEMU build.
The first solution was a helper that comes with QEMU
and loads appropriate eBPF objects. And the issue is
to find a proper helper if the system has several
different QEMUs installed and/or built from the source,
which helpers may not be compatible.
Another issue is QEMU updating while there is a running
QEMU instance. With an updated helper, it may not be
possible to hotplug virtio-net device to the already
running QEMU. Overall, requesting the eBPF object from
QEMU itself solves possible failures with acceptable effort.
Links:
[PATCH 3/5] qmp: Added the helper stamp check.
https://lore.kernel.org/all/20230219162100.174318-4-and...@daynix.com/
Signed-off-by: Andrew Melnychenko
---
ebpf/ebpf.c | 70 +++
ebpf/ebpf.h | 31 +++
ebpf/ebpf_rss.c | 6
ebpf/meson.build | 2 +-
qapi/ebpf.json| 57 +++
qapi/meson.build | 1 +
qapi/qapi-schema.json | 1 +
7 files changed, 167 insertions(+), 1 deletion(-)
create mode 100644 ebpf/ebpf.c
create mode 100644 ebpf/ebpf.h
create mode 100644 qapi/ebpf.json
diff --git a/ebpf/ebpf.c b/ebpf/ebpf.c
new file mode 100644
index 000..ea97c0403ee
--- /dev/null
+++ b/ebpf/ebpf.c
@@ -0,0 +1,70 @@
+/*
+ * QEMU eBPF binary declaration routine.
+ *
+ * Developed by Daynix Computing LTD (http://www.daynix.com)
+ *
+ * Authors:
+ * Andrew Melnychenko
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/queue.h"
+#include "qapi/error.h"
+#include "qapi/qapi-commands-ebpf.h"
+#include "ebpf/ebpf.h"
+
+struct ElfBinaryDataEntry {
+int id;
+const void *data;
+size_t datalen;
+
+QSLIST_ENTRY(ElfBinaryDataEntry) node;
+};
+
+static QSLIST_HEAD(, ElfBinaryDataEntry) ebpf_elf_obj_list =
+QSLIST_HEAD_INITIALIZER();
+
+void ebpf_register_binary_data(int id, const void *data, size_t datalen)
+{
+struct ElfBinaryDataEntry *dataentry = NULL;
+
+dataentry = g_new0(struct ElfBinaryDataEntry, 1);
+dataentry->data = data;
+dataentry->datalen = datalen;
+dataentry->id = id;
+
+QSLIST_INSERT_HEAD(&ebpf_elf_obj_list, dataentry, node);
+}
+
+const void *ebpf_find_binary_by_id(int id, size_t *sz, Error **errp)
+{
+struct ElfBinaryDataEntry *it = NULL;
+QSLIST_FOREACH(it, &ebpf_elf_obj_list, node) {
+if (id == it->id) {
+*sz = it->datalen;
+return it->data;
+}
+}
+
+error_setg(errp, "can't find eBPF object with id: %d", id);
+
+return NULL;
+}
+
+EbpfObject *qmp_request_ebpf(EbpfProgramID id, Error **errp)
+{
+EbpfObject *ret = NULL;
+size_t size = 0;
+const void *data = ebpf_find_binary_by_id(id, &size, errp);
+if (!data) {
+return NULL;
+}
+
+ret = g_new0(EbpfObject, 1);
+ret->object = g_base64_encode(data, size);
+
+return ret;
+}
diff --git a/ebpf/ebpf.h b/ebpf/ebpf.h
new file mode 100644
index 000..b6266b28b8c
--- /dev/null
+++ b/ebpf/ebpf.h
@@ -0,0 +1,31 @@
+/*
+ * QEMU eBPF binary declaration routine.
+ *
+ * Developed by Daynix Computing LTD (http://www.daynix.com)
+ *
+ * Authors:
+ * Andrew Melnychenko
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+
+#ifndef EBPF_H
+#define EBPF_H
+
+struct Error;
+
+void ebpf_register_binary_data(int id, const void *data,
+ size_t datalen);
+const void *ebpf_find_binary_by_id(int id, size_t *sz,
+ struct Error **errp);
+
+#define ebpf_binary_init(id, fn) \
+static void __attribute__((constructor)) ebpf_binary_init_ ## fn(void) \
+{ \
+size_t datalen = 0;\
+const void *data = fn(&datalen); \
+ebpf_register_binary_data(id, data, datalen); \
+}
+
+#end
