Re: [Xen-devel] [PATCH v2] docs: clarify xl mem-max semantics
On Fri, Jan 27, 2017 at 10:37:51AM +0100, Juergen Gross wrote: > On 24/01/17 22:47, Jim Fehlig wrote: > > On 01/23/2017 02:49 AM, Juergen Gross wrote: > >> The information given in the xl man page for the mem-max command is > >> rather brief. Expand it in order to let the reader understand what it > >> is really doing. > >> > >> As the related libxl function libxl_domain_setmaxmem() isn't much > >> clearer add a comment to it explaining the desired semantics. > >> > >> Signed-off-by: Juergen Gross> >> --- > >> V2: rewording as suggested by Jim Fehlig and Ian Jackson > >> --- > >> docs/man/xl.pod.1.in | 10 ++ > >> tools/libxl/libxl.c | 6 ++ > >> 2 files changed, 16 insertions(+) > >> > >> diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in > >> index 09c1faa..7caed08 100644 > >> --- a/docs/man/xl.pod.1.in > >> +++ b/docs/man/xl.pod.1.in > >> @@ -401,6 +401,16 @@ for bytes. > >> The mem-max value may not correspond to the actual memory used in the > >> domain, as it may balloon down its memory to give more back to the OS. > >> > >> +The value given just sets the memory amount the domain is allowed to > >> allocate > >> +in the hypervisor. It can't be set lower than the current > >> reservation, but > >> +it is allowed to be higher than the configured maximum memory size of > >> the > >> +domain (B parameter in the domain's configuration). Using > >> B > >> +to set the maximum memory above the initial B value will not > >> allow the > >> +additional memory to be used via B. The initial B > >> value is > >> +still used as an upper limit for B. > >> + > >> +The domain is not receiving any signal regarding the changed memory > >> limit. > >> + > >> =item B I I > >> > >> Set the domain's used memory using the balloon driver; append 't' for > >> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c > >> index 0622311..ed59510 100644 > >> --- a/tools/libxl/libxl.c > >> +++ b/tools/libxl/libxl.c > >> @@ -4018,6 +4018,12 @@ out: > >> > >> > >> /**/ > >> > >> > >> +/* > >> + * Set the maximum memory size of the domain in the hypervisor. There > >> is no > >> + * change of the current memory size involved. The allowed memory > >> size can > > > > s/allowed/specified/ ? Or perhaps "The value of max_memkb can even..." ? > > Sorry I didn't look closely at this hunk in V1. > > Okay, I don't mind. > > Can this be changed when committing, or should I send V3? > V3 is better. I can't work out how you would like me to change the wording. Wei. > > Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2] docs: clarify xl mem-max semantics
On 24/01/17 22:47, Jim Fehlig wrote: > On 01/23/2017 02:49 AM, Juergen Gross wrote: >> The information given in the xl man page for the mem-max command is >> rather brief. Expand it in order to let the reader understand what it >> is really doing. >> >> As the related libxl function libxl_domain_setmaxmem() isn't much >> clearer add a comment to it explaining the desired semantics. >> >> Signed-off-by: Juergen Gross>> --- >> V2: rewording as suggested by Jim Fehlig and Ian Jackson >> --- >> docs/man/xl.pod.1.in | 10 ++ >> tools/libxl/libxl.c | 6 ++ >> 2 files changed, 16 insertions(+) >> >> diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in >> index 09c1faa..7caed08 100644 >> --- a/docs/man/xl.pod.1.in >> +++ b/docs/man/xl.pod.1.in >> @@ -401,6 +401,16 @@ for bytes. >> The mem-max value may not correspond to the actual memory used in the >> domain, as it may balloon down its memory to give more back to the OS. >> >> +The value given just sets the memory amount the domain is allowed to >> allocate >> +in the hypervisor. It can't be set lower than the current >> reservation, but >> +it is allowed to be higher than the configured maximum memory size of >> the >> +domain (B parameter in the domain's configuration). Using >> B >> +to set the maximum memory above the initial B value will not >> allow the >> +additional memory to be used via B. The initial B >> value is >> +still used as an upper limit for B. >> + >> +The domain is not receiving any signal regarding the changed memory >> limit. >> + >> =item B I I >> >> Set the domain's used memory using the balloon driver; append 't' for >> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c >> index 0622311..ed59510 100644 >> --- a/tools/libxl/libxl.c >> +++ b/tools/libxl/libxl.c >> @@ -4018,6 +4018,12 @@ out: >> >> >> /**/ >> >> >> +/* >> + * Set the maximum memory size of the domain in the hypervisor. There >> is no >> + * change of the current memory size involved. The allowed memory >> size can > > s/allowed/specified/ ? Or perhaps "The value of max_memkb can even..." ? > Sorry I didn't look closely at this hunk in V1. Okay, I don't mind. Can this be changed when committing, or should I send V3? Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2] docs: clarify xl mem-max semantics
On 24/01/2017 21:47, Jim Fehlig wrote: > On 01/23/2017 02:49 AM, Juergen Gross wrote: >> The information given in the xl man page for the mem-max command is >> rather brief. Expand it in order to let the reader understand what it >> is really doing. >> >> As the related libxl function libxl_domain_setmaxmem() isn't much >> clearer add a comment to it explaining the desired semantics. >> >> Signed-off-by: Juergen Gross>> --- >> V2: rewording as suggested by Jim Fehlig and Ian Jackson >> --- >> docs/man/xl.pod.1.in | 10 ++ >> tools/libxl/libxl.c | 6 ++ >> 2 files changed, 16 insertions(+) >> >> diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in >> index 09c1faa..7caed08 100644 >> --- a/docs/man/xl.pod.1.in >> +++ b/docs/man/xl.pod.1.in >> @@ -401,6 +401,16 @@ for bytes. >> The mem-max value may not correspond to the actual memory used in the >> domain, as it may balloon down its memory to give more back to the OS. >> >> +The value given just sets the memory amount the domain is allowed to >> allocate >> +in the hypervisor. It can't be set lower than the current >> reservation, but >> +it is allowed to be higher than the configured maximum memory size >> of the >> +domain (B parameter in the domain's configuration). Using >> B >> +to set the maximum memory above the initial B value will not >> allow the >> +additional memory to be used via B. The initial >> B value is >> +still used as an upper limit for B. >> + >> +The domain is not receiving any signal regarding the changed memory >> limit. >> + >> =item B I I >> >> Set the domain's used memory using the balloon driver; append 't' for >> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c >> index 0622311..ed59510 100644 >> --- a/tools/libxl/libxl.c >> +++ b/tools/libxl/libxl.c >> @@ -4018,6 +4018,12 @@ out: >> >> >> /**/ >> >> >> +/* >> + * Set the maximum memory size of the domain in the hypervisor. >> There is no >> + * change of the current memory size involved. The allowed memory >> size can > > s/allowed/specified/ ? Or perhaps "The value of max_memkb can even..." > ? Sorry I didn't look closely at this hunk in V1. > >> + * even be above the configured maxmem size of the domain, but the >> related >> + * Xenstore entry memory/static-max isn't modified! >> + */ >> int libxl_domain_setmaxmem(libxl_ctx *ctx, uint32_t domid, uint64_t >> max_memkb) >> { >> GC_INIT(ctx); > > I have a related question for you (or Ian, Wei, ...): What does > max_pages (as seen with 'q' debug key') represent? How is it related > to memory/static-max in xenstore, if at all? That is a can of worms. They have grown teeth and are biting XenServer as well. :( Accurate accounting of memory is basically impossible at the moment, because it is an $N variable equation (for $N = 14 IIRC last time it was counted) with $M entities knowing some but not all of the information. There is active work underway to try and quantify and remedy the situation. As to your specific question though... > > When a domain is running, libvirt reports maximum memory through > max_memkb field of libxl_dominfo struct. max_memkb is filled with > PAGE_TO_MEMKB(xcinfo->max_pages). AFAICT, xcinfo->max_pages contains > the same value as max_pages from 'q' debug key, but neither of those > represent the maxmem value used at domain creation. E.g. > > # cat test.xl | grep mem > maxmem = 8192 > memory = 2048 > # xl create test.xl > # xl debug-key q && xl dmesg | grep max_pages > (XEN) nr_pages=524205 xenheap_pages=6 shared_pages=0 paged_pages=0 > dirty_cpus={20,42} max_pages=524544 > > On this system, 524544*4096=2049M which is closer to 'memory' than > 'maxmem'. > > I have a patch which fixes this in libvirt, but I'm having a hard time > articulating why the current use of max_pages is incorrect :-). nr_pages is (to an approximation) the number of pages currently making up guest RAM. It becomes complicated/fuzzy with PoD, VRAM, option roms, ballooned out pages, etc. (Part of the problem for XenServer is that it doesn't relate to "the amount RAM is Xen using to look after this domain"; there is absolutely no quantification of the overhead of running a VM.) max_pages is a value set directly by the toolstack (XEN_DOMCTL_set_maxmem iirc). It is an upper limit, which the guest may not exceed. However, it may be lower than nr_pages, at which point the VM may release pages (i.e. balloon them out), but cant claim them back. Part of the fuzzy problems come from the fact that multiple entities play with set_maxmem, including qemu when trying to allocate space for VRAM/option roms. This particular information is never conveyed back to libxl, which is most likely where your discrepancies originate. Having said that, by having maxmem and memory different, you have invoked PoD and that is a separate world of fun. A value which isn't printed is the size of the PoD cache, which is
Re: [Xen-devel] [PATCH v2] docs: clarify xl mem-max semantics
On 01/23/2017 02:49 AM, Juergen Gross wrote: The information given in the xl man page for the mem-max command is rather brief. Expand it in order to let the reader understand what it is really doing. As the related libxl function libxl_domain_setmaxmem() isn't much clearer add a comment to it explaining the desired semantics. Signed-off-by: Juergen Gross--- V2: rewording as suggested by Jim Fehlig and Ian Jackson --- docs/man/xl.pod.1.in | 10 ++ tools/libxl/libxl.c | 6 ++ 2 files changed, 16 insertions(+) diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in index 09c1faa..7caed08 100644 --- a/docs/man/xl.pod.1.in +++ b/docs/man/xl.pod.1.in @@ -401,6 +401,16 @@ for bytes. The mem-max value may not correspond to the actual memory used in the domain, as it may balloon down its memory to give more back to the OS. +The value given just sets the memory amount the domain is allowed to allocate +in the hypervisor. It can't be set lower than the current reservation, but +it is allowed to be higher than the configured maximum memory size of the +domain (B parameter in the domain's configuration). Using B +to set the maximum memory above the initial B value will not allow the +additional memory to be used via B. The initial B value is +still used as an upper limit for B. + +The domain is not receiving any signal regarding the changed memory limit. + =item B I I Set the domain's used memory using the balloon driver; append 't' for diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 0622311..ed59510 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -4018,6 +4018,12 @@ out: /**/ +/* + * Set the maximum memory size of the domain in the hypervisor. There is no + * change of the current memory size involved. The allowed memory size can s/allowed/specified/ ? Or perhaps "The value of max_memkb can even..." ? Sorry I didn't look closely at this hunk in V1. + * even be above the configured maxmem size of the domain, but the related + * Xenstore entry memory/static-max isn't modified! + */ int libxl_domain_setmaxmem(libxl_ctx *ctx, uint32_t domid, uint64_t max_memkb) { GC_INIT(ctx); I have a related question for you (or Ian, Wei, ...): What does max_pages (as seen with 'q' debug key') represent? How is it related to memory/static-max in xenstore, if at all? When a domain is running, libvirt reports maximum memory through max_memkb field of libxl_dominfo struct. max_memkb is filled with PAGE_TO_MEMKB(xcinfo->max_pages). AFAICT, xcinfo->max_pages contains the same value as max_pages from 'q' debug key, but neither of those represent the maxmem value used at domain creation. E.g. # cat test.xl | grep mem maxmem = 8192 memory = 2048 # xl create test.xl # xl debug-key q && xl dmesg | grep max_pages (XEN) nr_pages=524205 xenheap_pages=6 shared_pages=0 paged_pages=0 dirty_cpus={20,42} max_pages=524544 On this system, 524544*4096=2049M which is closer to 'memory' than 'maxmem'. I have a patch which fixes this in libvirt, but I'm having a hard time articulating why the current use of max_pages is incorrect :-). TIA for any comments you might have on that. Regards, Jim ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel