Re: Libvirt Open Source Contribution

2020-11-07 Thread Barrett J Schonefeld 
Michal,

We've struggled with the email system so far. You may have seen when we
submitted a chain of 65 patches when we meant to send only the last commit.
We tested this by sending it to ourselves first but must have made a
mistake when submitting to libvir-list.

We submitted an initial patch for issue-11, which was rejected because Ryan
emailed the patch, but I was CCed. I am working on getting my computer set
up to submit this patch myself.

Additionally, we submitted a patch for issue-16 (from Ryan Gahagan), and we
feel confident we did this correctly (even received a confirmation email).
However, we have not seen the patch posted on the forum nor have we
received any email that the patch was rejected. Is it possible it has not
been posted because we left outstanding questions in the patch?

Best regards,
Barrett Schonefeld

On Fri, Nov 6, 2020 at 2:03 AM Michal Prívozník  wrote:

> On 11/5/20 10:04 PM, Barrett J Schonefeld wrote:
> > Hey folks,
> >
> > We appreciate the feedback, and we've used this to complete some initial
> > work on issue 11.
> >
> > We started with small changes in src/util, and we submitted them via
> > email (following these guidelines on submitting patches
> > ) to ensure we are on the
> > right path.
> >
> > We do not see the patch we submitted via email showing up in the
> > threads, and we are wondering if we are submitting incorrectly or if the
> > patch goes through some approval process before appearing in the threads.
> >
> > Best regards,
> > Barrett Schonefeld
>
> So I checked the moderator's queue and there are no patches stuck, so
> probably it is something else. Did you see any error when 'git
> send-email'-ing the patches? When I was setting up my machine I used to
> send patches just to me to verify send-email is working and only then I
> tried to send patches to the list.
>
> Also, are you sure you (the sender) are subscribed to the list?
> Generally, if you aren't then list won't forward your e-mails/patches.
>
> Michal
>
>

Re: [PATCH v2] virsh: Allow listing just domain IDs

2020-11-07 Thread Michal Prívozník 

On 11/6/20 2:59 PM, Martin Kletzander wrote:

On Thu, Oct 22, 2020 at 04:41:18PM +0200, Michal Privoznik wrote:

Some completers for libvirt related tools might want to list
domain IDs only. Just like the one I've implemented for
virt-viewer [1]. I've worked around it using some awk magic,
but if it was possible to just 'virsh list --id' then I could
drop awk.

1: https://www.redhat.com/archives/virt-tools-list/2019-May/msg00014.html

Signed-off-by: Michal Privoznik 
---

Diff to v1:
- Documented the new switch in the manpage
- Allowed --id to be used with --all

docs/manpages/virsh.rst      | 21 +-
tools/virsh-domain-monitor.c | 42 +---
2 files changed, 39 insertions(+), 24 deletions(-)

diff --git a/docs/manpages/virsh.rst b/docs/manpages/virsh.rst
index d34a1c8684..848e1a6458 100644
--- a/docs/manpages/virsh.rst
+++ b/docs/manpages/virsh.rst
@@ -621,7 +621,7 @@ list

   list [--inactive | --all]
        [--managed-save] [--title]
-        { [--table] | --name | --uuid }
+        { [--table] | --name | --uuid | --id }
        [--persistent] [--transient]
        [--with-managed-save] [--without-managed-save]
        [--autostart] [--no-autostart]
@@ -758,16 +758,15 @@ If *--managed-save* is specified, then domains 
that have managed save state
in the listing. This flag is usable only with the default *--table* 
output.

Note that this flag does not filter the list of domains.

-If *--name* is specified, domain names are printed instead of the table
-formatted one per line. If *--uuid* is specified domain's UUID's are 
printed
-instead of names. Flag *--table* specifies that the legacy 
table-formatted

-output should be used. This is the default.
-
-If both *--name* and *--uuid* are specified, domain UUID's and names
-are printed side by side without any header. Flag *--table* specifies
-that the legacy table-formatted output should be used. This is the
-default if neither *--name* nor *--uuid* are specified. Option
-*--table* is mutually exclusive with options *--uuid* and *--name*.
+If *--name* is specified, domain names are printed instead of the
+table formatted one per line. If *--uuid* is specified domain's UUID's
+are printed instead of names. If *--id* is specified then domain's ID's
+are printed indead of names. However, it is possible to combine
+*--name*, *--uuid* and *--id* to select only desired fields for
+printing. Flag *--table* specifies that the legacy table-formatted
+output should be used, but it is mutually exclusive with *--name*,
+*--uuid* and *--id*. This is the default and will be used if neither of
+*--name*, *--uuid* or *--id* is specified.



Didn't you want to add something like the following?

   If neither *--name* nor *--uuid* is specified, but *--id* is, then 
only active
   domains are listed, even with the *--all* parameter as otherwise the 
output

   would just contain bunch of lines with just *-1*. >
Otherwise the code is very unclear for no reason.


Yes, this behaviour is not documented. Mostly because I don't expect 
regular users to use 'virsh list --id'. It was meant for completers, 
e.g. virt-viewer has --id argument and its completer could list domain 
IDs. Is it okay if I squash your suggestion in and push? Or do you want 
me to send v3?


Michal

[PATCH v4 0/7] migration/dirtyrate: Introduce APIs for getting domain memory dirty rate

2020-11-07 Thread Hao Wang 
V3 -> V4:
define flags to unsigned int
fix some compile warnings

V2 -> V3:
reorganize patchset to fix compile warning

V1 -> V2:
replace QEMU_JOB_ASYNC with QEMU_JOB_QUERY


Sometimes domain's memory dirty rate is expected by user in order to
decide whether it's proper to be migrated out or not.

We have already completed the QEMU part of the capability:
https://patchew.org/QEMU/1600237327-33618-1-git-send-email-zhengch...@huawei.com/
And this serial of patches introduce the corresponding LIBVIRT part --
DomainGetDirtyRateInfo API and corresponding virsh api -- "getdirtyrate".


instructions:
bash# virsh getdirtyrate --help
  NAME
getdirtyrate - Get a vm's memory dirty rate

  SYNOPSIS
getdirtyrate  [--seconds ] [--calculate] [--query]

  DESCRIPTION
Get memory dirty rate of a domain in order to decide whether it's proper to 
be migrated out or not.

  OPTIONS
[--domain]   domain name, id or uuid
--seconds   calculate memory dirty rate within specified seconds, a 
valid range of values is [1, 60], and would default to 1s.
--calculate  calculate dirty rate only, can be used together with 
--query, either or both is expected, otherwise would default to both.
--query  query dirty rate only, can be used together with 
--calculate, either or both is expected, otherwise would default to both.


example:
bash# virsh getdirtyrate --calculate --query --domain vm0 --seconds 1
status:measured
startTime: 820148
calcTime:  1 s
dirtyRate: 6 MB/s


*** BLURB HERE ***

Hao Wang (7):
  migration/dirtyrate: Introduce virDomainDirtyRateInfo structure
  migration/dirtyrate: set up framwork of domainGetDirtyRateInfo API
  migration/dirtyrate: Implement qemuDomainCalculateDirtyRate
  migration/dirtyrate: Implement qemuDomainQueryDirtyRate
  migration/dirtyrate: Implement qemuMonitorJSONExtractDirtyRateInfo
  migration/dirtyrate: Implement qemuDomainGetDirtyRateInfo
  migration/dirtyrate: Introduce getdirtyrate virsh api

 include/libvirt/libvirt-domain.h |  57 
 src/driver-hypervisor.h  |   7 ++
 src/libvirt-domain.c |  46 +
 src/libvirt_public.syms  |   5 ++
 src/qemu/qemu_driver.c   |  68 +++
 src/qemu/qemu_migration.c|  59 
 src/qemu/qemu_migration.h|  10 +++
 src/qemu/qemu_monitor.c  |  24 +++
 src/qemu/qemu_monitor.h  |   8 +++
 src/qemu/qemu_monitor_json.c |  97 ++
 src/qemu/qemu_monitor_json.h |   8 +++
 src/remote/remote_driver.c   |   1 +
 src/remote/remote_protocol.x |  21 +-
 tools/virsh-domain.c | 112 +++
 14 files changed, 522 insertions(+), 1 deletion(-)

-- 
2.23.0

[PATCH v4 6/7] migration/dirtyrate: Implement qemuDomainGetDirtyRateInfo

2020-11-07 Thread Hao Wang 
Implement qemuDomainGetDirtyRateInfo:
using flags to control behaviors -- calculate and/or query dirtyrate.

Signed-off-by: Hao Wang 
Reviewed-by: Chuan Zheng 
---
 include/libvirt/libvirt-domain.h | 11 ++
 src/qemu/qemu_driver.c   | 68 
 2 files changed, 79 insertions(+)

diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h
index 51d8685086..fc45f42dcf 100644
--- a/include/libvirt/libvirt-domain.h
+++ b/include/libvirt/libvirt-domain.h
@@ -5096,6 +5096,17 @@ int virDomainBackupBegin(virDomainPtr domain,
 char *virDomainBackupGetXMLDesc(virDomainPtr domain,
 unsigned int flags);
 
+/**
+ * virDomainDirtyRateFlags:
+ *
+ * Details on the flags used by getdirtyrate api.
+ */
+
+typedef enum {
+VIR_DOMAIN_DIRTYRATE_CALC = 1 << 0,  /* calculate domain's dirtyrate */
+VIR_DOMAIN_DIRTYRATE_QUERY = 1 << 1, /* query domain's dirtyrate */
+} virDomainDirtyRateFlags;
+
 /**
  * virDomainDirtyRateStatus:
  *
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index cb56fbbfcf..93d5a23630 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -20121,6 +20121,73 @@ qemuDomainAgentSetResponseTimeout(virDomainPtr dom,
 }
 
 
+#define MIN_DIRTYRATE_CALCULATION_PERIOD1  /* supported min dirtyrate calc 
time: 1s */
+#define MAX_DIRTYRATE_CALCULATION_PERIOD60 /* supported max dirtyrate calc 
time: 60s */
+
+static int
+qemuDomainGetDirtyRateInfo(virDomainPtr dom,
+   virDomainDirtyRateInfoPtr info,
+   long long sec,
+   unsigned int flags)
+{
+virDomainObjPtr vm = NULL;
+virQEMUDriverPtr driver = dom->conn->privateData;
+int ret = -1;
+
+if (!(vm = qemuDomainObjFromDomain(dom)))
+return ret;
+
+if (virDomainGetDirtyRateInfoEnsureACL(dom->conn, vm->def) < 0)
+goto cleanup;
+
+if (qemuDomainObjBeginJob(driver, vm, QEMU_JOB_QUERY) < 0)
+goto cleanup;
+
+if (!qemuMigrationSrcIsAllowed(driver, vm, false, 0))
+goto endjob;
+
+if (flags & VIR_DOMAIN_DIRTYRATE_CALC) {
+if (sec < MIN_DIRTYRATE_CALCULATION_PERIOD || sec > 
MAX_DIRTYRATE_CALCULATION_PERIOD) {
+virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
+   _("seconds=%lld is invalid, please choose value 
within [1, 60]."),
+   sec);
+goto endjob;
+}
+
+if (qemuDomainCalculateDirtyRate(dom, vm, sec) < 0) {
+virReportError(VIR_ERR_OPERATION_FAILED, "%s",
+   _("can't calculate domain's dirty rate"));
+goto endjob;
+}
+}
+
+if (flags & VIR_DOMAIN_DIRTYRATE_QUERY) {
+if (flags & VIR_DOMAIN_DIRTYRATE_CALC) {
+struct timespec ts = { .tv_sec = sec, .tv_nsec = 50 * 1000 * 
1000ull };
+
+virObjectUnlock(vm);
+nanosleep(, NULL);
+virObjectLock(vm);
+}
+
+if (qemuDomainQueryDirtyRate(dom, vm, info) < 0) {
+virReportError(VIR_ERR_OPERATION_FAILED, "%s",
+   _("can't query domain's dirty rate"));
+goto endjob;
+}
+}
+
+ret = 0;
+
+ endjob:
+qemuDomainObjEndJob(driver, vm);
+
+ cleanup:
+virDomainObjEndAPI();
+return ret;
+}
+
+
 static virHypervisorDriver qemuHypervisorDriver = {
 .name = QEMU_DRIVER_NAME,
 .connectURIProbe = qemuConnectURIProbe,
@@ -20360,6 +20427,7 @@ static virHypervisorDriver qemuHypervisorDriver = {
 .domainAgentSetResponseTimeout = qemuDomainAgentSetResponseTimeout, /* 
5.10.0 */
 .domainBackupBegin = qemuDomainBackupBegin, /* 6.0.0 */
 .domainBackupGetXMLDesc = qemuDomainBackupGetXMLDesc, /* 6.0.0 */
+.domainGetDirtyRateInfo = qemuDomainGetDirtyRateInfo, /* 6.9.0 */
 };
 
 
-- 
2.23.0

[PATCH v4 2/7] migration/dirtyrate: set up framwork of domainGetDirtyRateInfo API

2020-11-07 Thread Hao Wang 
Introduce DomainGetDirtyRateInfo API for domain's memory dirty rate
calculation and query.

Signed-off-by: Hao Wang 
Signed-off-by: Zhou Yimin 
Reviewed-by: Chuan Zheng 
---
 include/libvirt/libvirt-domain.h |  5 
 src/driver-hypervisor.h  |  7 +
 src/libvirt-domain.c | 46 
 src/libvirt_public.syms  |  5 
 src/remote/remote_driver.c   |  1 +
 src/remote/remote_protocol.x | 21 ++-
 6 files changed, 84 insertions(+), 1 deletion(-)

diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h
index 145f517068..b950736b67 100644
--- a/include/libvirt/libvirt-domain.h
+++ b/include/libvirt/libvirt-domain.h
@@ -5120,4 +5120,9 @@ struct _virDomainDirtyRateInfo {
 
 typedef virDomainDirtyRateInfo *virDomainDirtyRateInfoPtr;
 
+int virDomainGetDirtyRateInfo(virDomainPtr domain,
+  virDomainDirtyRateInfoPtr info,
+  long long sec,
+  unsigned int flags);
+
 #endif /* LIBVIRT_DOMAIN_H */
diff --git a/src/driver-hypervisor.h b/src/driver-hypervisor.h
index bce023017d..a77c29de54 100644
--- a/src/driver-hypervisor.h
+++ b/src/driver-hypervisor.h
@@ -1387,6 +1387,12 @@ typedef char *
 (*virDrvDomainBackupGetXMLDesc)(virDomainPtr domain,
 unsigned int flags);
 
+typedef int
+(*virDrvDomainGetDirtyRateInfo)(virDomainPtr domain,
+virDomainDirtyRateInfoPtr info,
+long long sec,
+unsigned int flags);
+
 typedef struct _virHypervisorDriver virHypervisorDriver;
 typedef virHypervisorDriver *virHypervisorDriverPtr;
 
@@ -1650,4 +1656,5 @@ struct _virHypervisorDriver {
 virDrvDomainAgentSetResponseTimeout domainAgentSetResponseTimeout;
 virDrvDomainBackupBegin domainBackupBegin;
 virDrvDomainBackupGetXMLDesc domainBackupGetXMLDesc;
+virDrvDomainGetDirtyRateInfo domainGetDirtyRateInfo;
 };
diff --git a/src/libvirt-domain.c b/src/libvirt-domain.c
index 3c5f55176a..ce3c40edf8 100644
--- a/src/libvirt-domain.c
+++ b/src/libvirt-domain.c
@@ -12758,3 +12758,49 @@ virDomainBackupGetXMLDesc(virDomainPtr domain,
 virDispatchError(conn);
 return NULL;
 }
+
+
+/**
+ * virDomainGetDirtyRateInfo:
+ * @domain: a domain object.
+ * @info: return value of current domain's memory dirty rate info.
+ * @sec: show dirty rate within specified seconds.
+ * @flags: the flags of getdirtyrate action -- calculate and/or query.
+ *
+ * Get the current domain's memory dirty rate (in MB/s).
+ *
+ * Returns 0 in case of success, -1 otherwise.
+ */
+int
+virDomainGetDirtyRateInfo(virDomainPtr domain,
+  virDomainDirtyRateInfoPtr info,
+  long long sec,
+  unsigned int flags)
+{
+virConnectPtr conn;
+
+VIR_DOMAIN_DEBUG(domain, "info = %p, seconds=%lld", info, sec);
+
+virResetLastError();
+
+virCheckDomainReturn(domain, -1);
+conn = domain->conn;
+
+virCheckNonNullArgGoto(info, error);
+virCheckReadOnlyGoto(conn->flags, error);
+
+if (info)
+memset(info, 0, sizeof(*info));
+
+if (conn->driver->domainGetDirtyRateInfo) {
+if (conn->driver->domainGetDirtyRateInfo(domain, info, sec, flags) < 0)
+goto error;
+VIR_DOMAIN_DEBUG(domain, "info = %p, seconds=%lld", info, sec);
+return 0;
+}
+
+virReportUnsupportedError();
+ error:
+virDispatchError(conn);
+return -1;
+}
diff --git a/src/libvirt_public.syms b/src/libvirt_public.syms
index 539d2e3943..11864f48b1 100644
--- a/src/libvirt_public.syms
+++ b/src/libvirt_public.syms
@@ -873,4 +873,9 @@ LIBVIRT_6.0.0 {
 virDomainBackupGetXMLDesc;
 } LIBVIRT_5.10.0;
 
+LIBVIRT_6.9.0 {
+global:
+virDomainGetDirtyRateInfo;
+} LIBVIRT_6.0.0;
+
 #  define new API here using predicted next version number 
diff --git a/src/remote/remote_driver.c b/src/remote/remote_driver.c
index 9cd2fd36ae..325968a22b 100644
--- a/src/remote/remote_driver.c
+++ b/src/remote/remote_driver.c
@@ -8458,6 +8458,7 @@ static virHypervisorDriver hypervisor_driver = {
 .domainAgentSetResponseTimeout = remoteDomainAgentSetResponseTimeout, /* 
5.10.0 */
 .domainBackupBegin = remoteDomainBackupBegin, /* 6.0.0 */
 .domainBackupGetXMLDesc = remoteDomainBackupGetXMLDesc, /* 6.0.0 */
+.domainGetDirtyRateInfo = remoteDomainGetDirtyRateInfo, /* 6.9.0 */
 };
 
 static virNetworkDriver network_driver = {
diff --git a/src/remote/remote_protocol.x b/src/remote/remote_protocol.x
index 5e5e781e76..5e3a75f1b9 100644
--- a/src/remote/remote_protocol.x
+++ b/src/remote/remote_protocol.x
@@ -3779,6 +3779,19 @@ struct remote_domain_backup_get_xml_desc_ret {
 remote_nonnull_string xml;
 };
 
+struct remote_domain_get_dirty_rate_info_args {
+remote_nonnull_domain dom;
+hyper sec;
+unsigned int flags;
+};
+
+struct

[PATCH v4 7/7] migration/dirtyrate: Introduce getdirtyrate virsh api

2020-11-07 Thread Hao Wang 
Signed-off-by: Hao Wang 
Signed-off-by: Zhou Yimin 
Reviewed-by: Chuan Zheng 
---
 tools/virsh-domain.c | 112 +++
 1 file changed, 112 insertions(+)

diff --git a/tools/virsh-domain.c b/tools/virsh-domain.c
index ef347585e8..52608e2c1b 100644
--- a/tools/virsh-domain.c
+++ b/tools/virsh-domain.c
@@ -14374,6 +14374,112 @@ cmdGuestInfo(vshControl *ctl, const vshCmd *cmd)
 return ret;
 }
 
+/*
+ * "querydirtyrate" command
+ */
+static const vshCmdInfo info_getdirtyrate[] = {
+{.name = "help",
+ .data = N_("Get a vm's memory dirty rate")
+},
+{.name = "desc",
+ .data = N_("Get memory dirty rate of a domain in order to decide"
+" whether it's proper to be migrated out or not.")
+},
+{.name = NULL}
+};
+
+static const vshCmdOptDef opts_getdirtyrate[] = {
+VIRSH_COMMON_OPT_DOMAIN_FULL(0),
+{.name = "seconds",
+ .type = VSH_OT_INT,
+ .help = N_("calculate memory dirty rate within specified seconds,"
+" a valid range of values is [1, 60], and would default to 
1s.")
+},
+{.name = "calculate",
+ .type = VSH_OT_BOOL,
+ .help = N_("calculate dirty rate only, can be used together with --query,"
+" either or both is expected, otherwise would default to 
both.")
+},
+{.name = "query",
+ .type = VSH_OT_BOOL,
+ .help = N_("query dirty rate only, can be used together with --calculate,"
+" either or both is expected, otherwise would default to 
both.")
+},
+{.name = NULL}
+};
+
+static bool
+cmdGetDirtyRateInfo(vshControl *ctl, const vshCmd *cmd)
+{
+virDomainPtr dom = NULL;
+virDomainDirtyRateInfo info;
+long long sec = 0;
+const char *status = NULL;
+unsigned int flags = 0;
+int rc;
+bool ret = false;
+bool calc = vshCommandOptBool(cmd, "calculate");
+bool query = vshCommandOptBool(cmd, "query");
+
+if (calc)
+flags |= VIR_DOMAIN_DIRTYRATE_CALC;
+if (query)
+flags |= VIR_DOMAIN_DIRTYRATE_QUERY;
+
+/* if flag option is missing, default to both --calculate and --query */
+if (!calc && !query)
+flags |= VIR_DOMAIN_DIRTYRATE_CALC | VIR_DOMAIN_DIRTYRATE_QUERY;
+
+if (!(dom = virshCommandOptDomain(ctl, cmd, NULL)))
+return false;
+
+rc = vshCommandOptLongLong(ctl, cmd, "seconds", );
+if (rc < 0)
+goto done;
+
+/* if --seconds option is missing, default to 1s */
+if (!rc)
+sec = 1;
+
+if (virDomainGetDirtyRateInfo(dom, , sec, flags) < 0) {
+vshError(ctl, "%s", _("Get memory dirty-rate failed."));
+goto done;
+}
+
+if (flags & VIR_DOMAIN_DIRTYRATE_QUERY) {
+switch (info.status) {
+case VIR_DOMAIN_DIRTYRATE_UNSTARTED:
+status = _("unstarted");
+break;
+case VIR_DOMAIN_DIRTYRATE_MEASURING:
+status = _("measuring");
+break;
+case VIR_DOMAIN_DIRTYRATE_MEASURED:
+status = _("measured");
+break;
+default:
+status = _("unknown");
+}
+
+vshPrint(ctl, _("status:%s\n"), status);
+vshPrint(ctl, _("start time:%lld\n"), info.startTime);
+vshPrint(ctl, _("calc time: %lld s\n"), info.calcTime);
+
+if (info.status == VIR_DOMAIN_DIRTYRATE_MEASURED)
+vshPrint(ctl, _("dirty rate:%lld MB/s\n"), info.dirtyRate);
+else
+vshPrint(ctl, _("dirty rate:the calculation is %s, please 
query results later\n"),
+ status);
+} else {
+vshPrint(ctl, _("Memory dirty rate is calculating, use --query option 
to display results.\n"));
+}
+
+ret = true;
+ done:
+virshDomainFree(dom);
+return ret;
+}
+
 const vshCmdDef domManagementCmds[] = {
 {.name = "attach-device",
  .handler = cmdAttachDevice,
@@ -15001,5 +15107,11 @@ const vshCmdDef domManagementCmds[] = {
  .info = info_guestinfo,
  .flags = 0
 },
+{.name = "getdirtyrate",
+ .handler = cmdGetDirtyRateInfo,
+ .opts = opts_getdirtyrate,
+ .info = info_getdirtyrate,
+ .flags = 0
+},
 {.name = NULL}
 };
-- 
2.23.0

[PATCH v4 5/7] migration/dirtyrate: Implement qemuMonitorJSONExtractDirtyRateInfo

2020-11-07 Thread Hao Wang 
Implement qemuMonitorJSONExtractDirtyRateInfo to deal with the return from
qmp "query-dirty-rate", and store them in virDomainDirtyRateInfo.

Signed-off-by: Hao Wang 
Reviewed-by: Chuan Zheng 
---
 include/libvirt/libvirt-domain.h | 17 +
 src/qemu/qemu_monitor_json.c | 59 ++--
 2 files changed, 73 insertions(+), 3 deletions(-)

diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h
index b950736b67..51d8685086 100644
--- a/include/libvirt/libvirt-domain.h
+++ b/include/libvirt/libvirt-domain.h
@@ -5096,6 +5096,23 @@ int virDomainBackupBegin(virDomainPtr domain,
 char *virDomainBackupGetXMLDesc(virDomainPtr domain,
 unsigned int flags);
 
+/**
+ * virDomainDirtyRateStatus:
+ *
+ * Details on the cause of a dirtyrate calculation status.
+ */
+
+typedef enum {
+VIR_DOMAIN_DIRTYRATE_UNSTARTED = 0, /* the dirtyrate calculation has not 
been started */
+VIR_DOMAIN_DIRTYRATE_MEASURING = 1, /* the dirtyrate calculation is 
measuring */
+VIR_DOMAIN_DIRTYRATE_MEASURED  = 2, /* the dirtyrate calculation is
+completed */
+
+# ifdef VIR_ENUM_SENTINELS
+VIR_DOMAIN_DIRTYRATE_LAST
+# endif
+} virDomainDirtyRateStatus;
+
 /**
  * virDomainDirtyRateInfo:
  *
diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c
index 1924c7229b..ca7d8d23c0 100644
--- a/src/qemu/qemu_monitor_json.c
+++ b/src/qemu/qemu_monitor_json.c
@@ -9659,12 +9659,61 @@ qemuMonitorJSONCalculateDirtyRate(qemuMonitorPtr mon,
 }
 
 
+VIR_ENUM_DECL(qemuDomainDirtyRateStatus);
+VIR_ENUM_IMPL(qemuDomainDirtyRateStatus,
+  VIR_DOMAIN_DIRTYRATE_LAST,
+  "unstarted",
+  "measuring",
+  "measured");
+
+static int
+qemuMonitorJSONExtractDirtyRateInfo(virJSONValuePtr data,
+virDomainDirtyRateInfoPtr info)
+{
+const char *status;
+int statusID;
+
+if (!(status = virJSONValueObjectGetString(data, "status"))) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("query-dirty-rate reply was missing 'status' data"));
+return -1;
+}
+
+if ((statusID = qemuDomainDirtyRateStatusTypeFromString(status)) < 0) {
+return -1;
+}
+info->status = statusID;
+
+if ((info->status == VIR_DOMAIN_DIRTYRATE_MEASURED) &&
+(virJSONValueObjectGetNumberLong(data, "dirty-rate", 
&(info->dirtyRate)) < 0)) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("query-dirty-rate reply was missing 'dirty-rate' 
data"));
+return -1;
+}
+
+if (virJSONValueObjectGetNumberLong(data, "start-time", 
&(info->startTime)) < 0) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("query-dirty-rate reply was missing 'start-time' 
data"));
+return -1;
+}
+
+if (virJSONValueObjectGetNumberLong(data, "calc-time", &(info->calcTime)) 
< 0) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("query-dirty-rate reply was missing 'calc-time' 
data"));
+return -1;
+}
+
+return 0;
+}
+
+
 int
 qemuMonitorJSONQueryDirtyRate(qemuMonitorPtr mon,
   virDomainDirtyRateInfoPtr info)
 {
 g_autoptr(virJSONValue) cmd = NULL;
 g_autoptr(virJSONValue) reply = NULL;
+virJSONValuePtr data = NULL;
 
 if (!(cmd = qemuMonitorJSONMakeCommand("query-dirty-rate", NULL)))
 return -1;
@@ -9675,7 +9724,11 @@ qemuMonitorJSONQueryDirtyRate(qemuMonitorPtr mon,
 if (qemuMonitorJSONCheckError(cmd, reply) < 0)
 return -1;
 
-/* TODO: extract dirtyrate data from reply and store in 
virDomainDirtyRateInfoPtr */
-info->status = 0;
-return 0;
+if (!(data = virJSONValueObjectGetObject(reply, "return"))) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("query-dirty-rate reply was missing 'return' data"));
+return -1;
+}
+
+return qemuMonitorJSONExtractDirtyRateInfo(data, info);
 }
-- 
2.23.0

[PATCH v4 3/7] migration/dirtyrate: Implement qemuDomainCalculateDirtyRate

2020-11-07 Thread Hao Wang 
Implement qemuDomainCalculateDirtyRate which calculates domain's memory
dirty rate calling qmp "calc-dirty-rate".

Signed-off-by: Hao Wang 
Signed-off-by: Zhou Yimin 
Reviewed-by: Chuan Zheng 
---
 src/qemu/qemu_migration.c| 28 
 src/qemu/qemu_migration.h|  5 +
 src/qemu/qemu_monitor.c  | 12 
 src/qemu/qemu_monitor.h  |  4 
 src/qemu/qemu_monitor_json.c | 22 ++
 src/qemu/qemu_monitor_json.h |  4 
 6 files changed, 75 insertions(+)

diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c
index 6f764b0c73..8029e24415 100644
--- a/src/qemu/qemu_migration.c
+++ b/src/qemu/qemu_migration.c
@@ -5836,3 +5836,31 @@ qemuMigrationSrcFetchMirrorStats(virQEMUDriverPtr driver,
 virHashFree(blockinfo);
 return 0;
 }
+
+
+int
+qemuDomainCalculateDirtyRate(virDomainPtr dom,
+ virDomainObjPtr vm,
+ long long sec)
+{
+virQEMUDriverPtr driver = dom->conn->privateData;
+qemuDomainObjPrivatePtr priv;
+int ret = -1;
+
+if (!virDomainObjIsActive(vm)) {
+virReportError(VIR_ERR_OPERATION_INVALID,
+   "%s", _("domain is not running"));
+return ret;
+}
+
+priv = vm->privateData;
+
+VIR_DEBUG("Calculate dirty rate during %lld seconds", sec);
+qemuDomainObjEnterMonitor(driver, vm);
+
+ret = qemuMonitorCalculateDirtyRate(priv->mon, sec);
+if (qemuDomainObjExitMonitor(driver, vm) < 0)
+ret = -1;
+
+return ret;
+}
diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h
index fd9eb7cab0..0522b375c0 100644
--- a/src/qemu/qemu_migration.h
+++ b/src/qemu/qemu_migration.h
@@ -258,3 +258,8 @@ qemuMigrationSrcFetchMirrorStats(virQEMUDriverPtr driver,
  virDomainObjPtr vm,
  qemuDomainAsyncJob asyncJob,
  qemuDomainJobInfoPtr jobInfo);
+
+int
+qemuDomainCalculateDirtyRate(virDomainPtr dom,
+ virDomainObjPtr vm,
+ long long sec);
diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c
index 5481bd99a0..06603b8691 100644
--- a/src/qemu/qemu_monitor.c
+++ b/src/qemu/qemu_monitor.c
@@ -4769,3 +4769,15 @@ qemuMonitorTransactionBackup(virJSONValuePtr actions,
 return qemuMonitorJSONTransactionBackup(actions, device, jobname, target,
 bitmap, syncmode);
 }
+
+
+int
+qemuMonitorCalculateDirtyRate(qemuMonitorPtr mon,
+  long long sec)
+{
+VIR_DEBUG("seconds=%lld", sec);
+
+QEMU_CHECK_MONITOR(mon);
+
+return qemuMonitorJSONCalculateDirtyRate(mon, sec);
+}
diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h
index 54fbb41ef7..afb97780cf 100644
--- a/src/qemu/qemu_monitor.h
+++ b/src/qemu/qemu_monitor.h
@@ -1527,3 +1527,7 @@ qemuMonitorTransactionBackup(virJSONValuePtr actions,
  const char *target,
  const char *bitmap,
  qemuMonitorTransactionBackupSyncMode syncmode);
+
+int
+qemuMonitorCalculateDirtyRate(qemuMonitorPtr mon,
+  long long sec);
diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c
index 843a555952..65691522fb 100644
--- a/src/qemu/qemu_monitor_json.c
+++ b/src/qemu/qemu_monitor_json.c
@@ -9635,3 +9635,25 @@ qemuMonitorJSONGetCPUMigratable(qemuMonitorPtr mon,
 return virJSONValueGetBoolean(virJSONValueObjectGet(reply, "return"),
   migratable);
 }
+
+
+int
+qemuMonitorJSONCalculateDirtyRate(qemuMonitorPtr mon,
+  long long sec)
+{
+g_autoptr(virJSONValue) cmd = NULL;
+g_autoptr(virJSONValue) reply = NULL;
+
+if (!(cmd = qemuMonitorJSONMakeCommand("calc-dirty-rate",
+   "I:calc-time", (long)sec,
+   NULL)))
+return -1;
+
+if (qemuMonitorJSONCommand(mon, cmd, ) < 0)
+return -1;
+
+if (qemuMonitorJSONCheckError(cmd, reply) < 0)
+return -1;
+
+return 0;
+}
diff --git a/src/qemu/qemu_monitor_json.h b/src/qemu/qemu_monitor_json.h
index 38e23ef3c5..c9556fc19a 100644
--- a/src/qemu/qemu_monitor_json.h
+++ b/src/qemu/qemu_monitor_json.h
@@ -710,3 +710,7 @@ int qemuMonitorJSONSetDBusVMStateIdList(qemuMonitorPtr mon,
 int
 qemuMonitorJSONGetCPUMigratable(qemuMonitorPtr mon,
 bool *migratable);
+
+int
+qemuMonitorJSONCalculateDirtyRate(qemuMonitorPtr mon,
+  long long sec);
-- 
2.23.0

[PATCH v4 4/7] migration/dirtyrate: Implement qemuDomainQueryDirtyRate

2020-11-07 Thread Hao Wang 
Implement qemuDomainQueryDirtyRate which query domain's memory
dirty rate calling qmp "query-dirty-rate".

Signed-off-by: Hao Wang 
Signed-off-by: Zhou Yimin 
Reviewed-by: Chuan Zheng 
---
 src/qemu/qemu_migration.c| 31 +++
 src/qemu/qemu_migration.h|  5 +
 src/qemu/qemu_monitor.c  | 12 
 src/qemu/qemu_monitor.h  |  4 
 src/qemu/qemu_monitor_json.c | 22 ++
 src/qemu/qemu_monitor_json.h |  4 
 6 files changed, 78 insertions(+)

diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c
index 8029e24415..3d07ba3ac4 100644
--- a/src/qemu/qemu_migration.c
+++ b/src/qemu/qemu_migration.c
@@ -5864,3 +5864,34 @@ qemuDomainCalculateDirtyRate(virDomainPtr dom,
 
 return ret;
 }
+
+
+int
+qemuDomainQueryDirtyRate(virDomainPtr dom,
+ virDomainObjPtr vm,
+ virDomainDirtyRateInfoPtr info)
+{
+virQEMUDriverPtr driver = dom->conn->privateData;
+qemuDomainObjPrivatePtr priv;
+int ret = -1;
+
+if (!virDomainObjIsActive(vm)) {
+virReportError(VIR_ERR_OPERATION_INVALID,
+   "%s", _("domain is not running"));
+return ret;
+}
+
+priv = vm->privateData;
+
+qemuDomainObjEnterMonitor(driver, vm);
+
+ret = qemuMonitorQueryDirtyRate(priv->mon, info);
+if (ret < 0) {
+virReportError(VIR_ERR_OPERATION_FAILED,
+   "%s", _("get vm's dirty rate failed."));
+}
+if (qemuDomainObjExitMonitor(driver, vm) < 0)
+ret = -1;
+
+return ret;
+}
diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h
index 0522b375c0..8baae512b7 100644
--- a/src/qemu/qemu_migration.h
+++ b/src/qemu/qemu_migration.h
@@ -263,3 +263,8 @@ int
 qemuDomainCalculateDirtyRate(virDomainPtr dom,
  virDomainObjPtr vm,
  long long sec);
+
+int
+qemuDomainQueryDirtyRate(virDomainPtr dom,
+ virDomainObjPtr vm,
+ virDomainDirtyRateInfoPtr info);
diff --git a/src/qemu/qemu_monitor.c b/src/qemu/qemu_monitor.c
index 06603b8691..2fa6879467 100644
--- a/src/qemu/qemu_monitor.c
+++ b/src/qemu/qemu_monitor.c
@@ -4781,3 +4781,15 @@ qemuMonitorCalculateDirtyRate(qemuMonitorPtr mon,
 
 return qemuMonitorJSONCalculateDirtyRate(mon, sec);
 }
+
+
+int
+qemuMonitorQueryDirtyRate(qemuMonitorPtr mon,
+  virDomainDirtyRateInfoPtr info)
+{
+VIR_DEBUG("info=%p", info);
+
+QEMU_CHECK_MONITOR(mon);
+
+return qemuMonitorJSONQueryDirtyRate(mon, info);
+}
diff --git a/src/qemu/qemu_monitor.h b/src/qemu/qemu_monitor.h
index afb97780cf..25105c3ad9 100644
--- a/src/qemu/qemu_monitor.h
+++ b/src/qemu/qemu_monitor.h
@@ -1531,3 +1531,7 @@ qemuMonitorTransactionBackup(virJSONValuePtr actions,
 int
 qemuMonitorCalculateDirtyRate(qemuMonitorPtr mon,
   long long sec);
+
+int
+qemuMonitorQueryDirtyRate(qemuMonitorPtr mon,
+  virDomainDirtyRateInfoPtr info);
diff --git a/src/qemu/qemu_monitor_json.c b/src/qemu/qemu_monitor_json.c
index 65691522fb..1924c7229b 100644
--- a/src/qemu/qemu_monitor_json.c
+++ b/src/qemu/qemu_monitor_json.c
@@ -9657,3 +9657,25 @@ qemuMonitorJSONCalculateDirtyRate(qemuMonitorPtr mon,
 
 return 0;
 }
+
+
+int
+qemuMonitorJSONQueryDirtyRate(qemuMonitorPtr mon,
+  virDomainDirtyRateInfoPtr info)
+{
+g_autoptr(virJSONValue) cmd = NULL;
+g_autoptr(virJSONValue) reply = NULL;
+
+if (!(cmd = qemuMonitorJSONMakeCommand("query-dirty-rate", NULL)))
+return -1;
+
+if (qemuMonitorJSONCommand(mon, cmd, ) < 0)
+return -1;
+
+if (qemuMonitorJSONCheckError(cmd, reply) < 0)
+return -1;
+
+/* TODO: extract dirtyrate data from reply and store in 
virDomainDirtyRateInfoPtr */
+info->status = 0;
+return 0;
+}
diff --git a/src/qemu/qemu_monitor_json.h b/src/qemu/qemu_monitor_json.h
index c9556fc19a..487f2e6e58 100644
--- a/src/qemu/qemu_monitor_json.h
+++ b/src/qemu/qemu_monitor_json.h
@@ -714,3 +714,7 @@ qemuMonitorJSONGetCPUMigratable(qemuMonitorPtr mon,
 int
 qemuMonitorJSONCalculateDirtyRate(qemuMonitorPtr mon,
   long long sec);
+
+int
+qemuMonitorJSONQueryDirtyRate(qemuMonitorPtr mon,
+  virDomainDirtyRateInfoPtr info);
-- 
2.23.0

[PATCH v4 1/7] migration/dirtyrate: Introduce virDomainDirtyRateInfo structure

2020-11-07 Thread Hao Wang 
Introduce virDomainDirtyRateInfo structure used for domain's memory dirty rate 
query.

Signed-off-by: Hao Wang 
Reviewed-by: Chuan Zheng 
---
 include/libvirt/libvirt-domain.h | 24 
 1 file changed, 24 insertions(+)

diff --git a/include/libvirt/libvirt-domain.h b/include/libvirt/libvirt-domain.h
index b3310729bf..145f517068 100644
--- a/include/libvirt/libvirt-domain.h
+++ b/include/libvirt/libvirt-domain.h
@@ -5096,4 +5096,28 @@ int virDomainBackupBegin(virDomainPtr domain,
 char *virDomainBackupGetXMLDesc(virDomainPtr domain,
 unsigned int flags);
 
+/**
+ * virDomainDirtyRateInfo:
+ *
+ * a virDomainDirtyRateInfo is a structure filled by virDomainGetDirtyRate() 
and
+ * extracting dirty rate infomation for a given active Domain.
+ */
+
+typedef struct _virDomainDirtyRateInfo virDomainDirtyRateInfo;
+
+struct _virDomainDirtyRateInfo {
+int status; /* the status of dirtyrate calculation */
+long long dirtyRate;/* the dirtyrate in MB/s */
+long long startTime;/* the start time of dirtyrate calculation */
+long long calcTime; /* the period of dirtyrate calculation */
+};
+
+/**
+ * virDomainDirtyRateInfoPtr:
+ *
+ * a virDomainDirtyRateInfoPtr is a pointer to a virDomainDirtyRateInfo 
structure.
+ */
+
+typedef virDomainDirtyRateInfo *virDomainDirtyRateInfoPtr;
+
 #endif /* LIBVIRT_DOMAIN_H */
-- 
2.23.0

[libvirt PATCH] qemu: add qemuAgentSSH{Add,Remove,Get}AuthorizedKeys

2020-11-07 Thread marcandre . lureau 
From: Marc-André Lureau 

In QEMU 5.2, the guest agent learned to manipulate a user
~/.ssh/authorized_keys. Bind the JSON API to libvirt.

https://wiki.qemu.org/ChangeLog/5.2#Guest_agent

Fixes:
https://bugzilla.redhat.com/show_bug.cgi?id=1888537

Signed-off-by: Marc-André Lureau 
---
 src/qemu/qemu_agent.c | 158 ++
 src/qemu/qemu_agent.h |  26 +++
 tests/qemuagenttest.c |  80 +
 3 files changed, 264 insertions(+)

diff --git a/src/qemu/qemu_agent.c b/src/qemu/qemu_agent.c
index 7fbb4a9431..75e7fea9e4 100644
--- a/src/qemu/qemu_agent.c
+++ b/src/qemu/qemu_agent.c
@@ -2496,3 +2496,161 @@ qemuAgentSetResponseTimeout(qemuAgentPtr agent,
 {
 agent->timeout = timeout;
 }
+
+void qemuAgentSSHAuthorizedKeyFree(qemuAgentSSHAuthorizedKeyPtr key)
+{
+if (!key)
+return;
+
+g_free(key->key);
+g_free(key);
+}
+
+/* Returns: 0 on success
+ *  -2 when agent command is not supported by the agent and
+ * 'report_unsupported' is false (libvirt error is not reported)
+ *  -1 otherwise (libvirt error is reported)
+ */
+int qemuAgentSSHGetAuthorizedKeys(qemuAgentPtr agent,
+  const char *user,
+  qemuAgentSSHAuthorizedKeyPtr **keys,
+  bool report_unsupported)
+{
+g_autoptr(virJSONValue) cmd = NULL;
+g_autoptr(virJSONValue) reply = NULL;
+virJSONValuePtr data = NULL;
+size_t ndata;
+size_t i;
+int rc;
+qemuAgentSSHAuthorizedKeyPtr *keys_ret = NULL;
+
+if (!(cmd = qemuAgentMakeCommand("guest-ssh-get-authorized-keys",
+ "s:username", user,
+  NULL)))
+return -1;
+
+if ((rc = qemuAgentCommandFull(agent, cmd, , agent->timeout,
+   report_unsupported)) < 0)
+return rc;
+
+if (!(data = virJSONValueObjectGetArray(reply, "return"))) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("qemu agent didn't return an array of keys"));
+return -1;
+}
+ndata = virJSONValueArraySize(data);
+
+keys_ret = g_new0(qemuAgentSSHAuthorizedKeyPtr, ndata);
+
+for (i = 0; i < ndata; i++) {
+virJSONValuePtr entry = virJSONValueArrayGet(data, i);
+
+if (!entry) {
+virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+   _("array element missing in 
guest-ssh-get-authorized-keys return "
+ "value"));
+goto cleanup;
+}
+
+keys_ret[i] = g_new0(qemuAgentSSHAuthorizedKey, 1);
+keys_ret[i]->key = g_strdup(virJSONValueGetString(entry));
+}
+
+*keys = g_steal_pointer(_ret);
+return ndata;
+
+ cleanup:
+if (keys_ret) {
+for (i = 0; i < ndata; i++)
+qemuAgentSSHAuthorizedKeyFree(keys_ret[i]);
+g_free(keys_ret);
+}
+return -1;
+}
+
+static virJSONValuePtr
+makeJSONArrayFromKeys(qemuAgentSSHAuthorizedKeyPtr *keys,
+  size_t nkeys)
+{
+g_autoptr(virJSONValue) jkeys = NULL;
+size_t i;
+
+jkeys = virJSONValueNewArray();
+
+for (i = 0; i < nkeys; i++) {
+qemuAgentSSHAuthorizedKeyPtr k = keys[i];
+
+if (virJSONValueArrayAppendString(jkeys, k->key) < 0)
+return NULL;
+}
+
+return g_steal_pointer();
+}
+
+/* Returns: 0 on success
+ *  -2 when agent command is not supported by the agent and
+ * 'report_unsupported' is false (libvirt error is not reported)
+ *  -1 otherwise (libvirt error is reported)
+ */
+int qemuAgentSSHAddAuthorizedKeys(qemuAgentPtr agent,
+  const char *user,
+  qemuAgentSSHAuthorizedKeyPtr *keys,
+  size_t nkeys,
+  bool reset,
+  bool report_unsupported)
+{
+g_autoptr(virJSONValue) cmd = NULL;
+g_autoptr(virJSONValue) reply = NULL;
+g_autoptr(virJSONValue) jkeys = NULL;
+int rc;
+
+jkeys = makeJSONArrayFromKeys(keys, nkeys);
+if (jkeys == NULL)
+return -1;
+
+if (!(cmd = qemuAgentMakeCommand("guest-ssh-add-authorized-keys",
+ "s:username", user,
+ "a:keys", ,
+ "b:reset", reset,
+  NULL)))
+return -1;
+
+if ((rc = qemuAgentCommandFull(agent, cmd, , agent->timeout,
+   report_unsupported)) < 0)
+return rc;
+
+return 0;
+}
+
+/* Returns: 0 on success
+ *  -2 when agent command is not supported by the agent and
+ * 'report_unsupported' is false (libvirt error is not reported)
+ *  -1 otherwise (libvirt error is reported)
+ */
+int