Re: [Xen-devel] [PATCH v2 12/19] hvmloader: retrieve vNUMA information from hypervisor
>>> On 09.12.14 at 18:52, wrote: > On Tue, Dec 09, 2014 at 04:46:22PM +, Jan Beulich wrote: >> >>> On 01.12.14 at 16:33, wrote: >> > + */ >> > + >> > +#ifndef __HVMLOADER_VNUMA_H__ >> > +#define __HVMLOADER_VNUMA_H__ >> > + >> > +#include >> > + >> > +#define MAX_VNODES 64 >> > +#define MAX_VDISTANCE (MAX_VNODES * MAX_VNODES) >> > +#define MAX_VMEMRANGES (MAX_VNODES * 2) >> >> These look arbitrary - please add a (brief) comment giving some >> rationale so that people needing to change them know eventual >> consequences. Would it perhaps make sense to derive >> MAX_VNODES from HVM_MAX_VCPUS? Additionally I think the > > I don't think these two have very strong connection though. > > And I remember you saying HVM_MAX_VCPUS will be removed. Removed? I recall myself saying increased... >> code wouldn't become much more difficult if you didn't build in >> static upper limits. >> > > Yes I can make two hypercalls. First one to retrieve the number of nodes > / vmemranges configured, allocate memory then fill in those arrays with > second hypercall. That'd be great, as it would eliminate the point above at once. Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2 12/19] hvmloader: retrieve vNUMA information from hypervisor
On Tue, Dec 09, 2014 at 04:46:22PM +, Jan Beulich wrote: > >>> On 01.12.14 at 16:33, wrote: > > @@ -261,6 +262,8 @@ int main(void) > > > > init_hypercalls(); > > > > +init_vnuma_info(); > > This is very early, and I don't think it needs to be - I guess it could be > done right before doing ACPI stuff? > Yes, it can be moved right before ACPI stuff. > > --- /dev/null > > +++ b/tools/firmware/hvmloader/vnuma.c > > @@ -0,0 +1,84 @@ > > +/* > > + * vnuma.c: obtain vNUMA information from hypervisor > > + * > > + * Copyright (c) 2014 Wei Liu, Citrix Systems (R&D) Ltd. > > + * > > + * Redistribution and use in source and binary forms, with or without > > + * modification, are permitted provided that the following conditions > > + * are met: > > + * 1. Redistributions of source code must retain the above copyright > > + *notice, this list of conditions and the following disclaimer. > > + * 2. Redistributions in binary form must reproduce the above copyright > > + *notice, this list of conditions and the following disclaimer in the > > + *documentation and/or other materials provided with the distribution. > > + * > > + * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND > > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR > > PURPOSE > > + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE > > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR > > CONSEQUENTIAL > > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, > > STRICT > > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY > > WAY > > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > > + * SUCH DAMAGE. > > + */ > > + > > +#include "util.h" > > +#include "hypercall.h" > > +#include "vnuma.h" > > +#include > > This is the system header, not the Xen one. See the discussion at > http://lists.xenproject.org/archives/html/xen-devel/2014-10/msg03206.html > and perhaps build on the resulting patch > http://lists.xenproject.org/archives/html/xen-devel/2014-12/msg00013.html. > I will cherry pick that patch and build on top of it. > > + > > +unsigned int nr_vnodes, nr_vmemranges; > > +unsigned int *vcpu_to_vnode, *vdistance; > > +xen_vmemrange_t *vmemrange; > > + [...] > > + */ > > + > > +#ifndef __HVMLOADER_VNUMA_H__ > > +#define __HVMLOADER_VNUMA_H__ > > + > > +#include > > + > > +#define MAX_VNODES 64 > > +#define MAX_VDISTANCE (MAX_VNODES * MAX_VNODES) > > +#define MAX_VMEMRANGES (MAX_VNODES * 2) > > These look arbitrary - please add a (brief) comment giving some > rationale so that people needing to change them know eventual > consequences. Would it perhaps make sense to derive > MAX_VNODES from HVM_MAX_VCPUS? Additionally I think the I don't think these two have very strong connection though. And I remember you saying HVM_MAX_VCPUS will be removed. > code wouldn't become much more difficult if you didn't build in > static upper limits. > Yes I can make two hypercalls. First one to retrieve the number of nodes / vmemranges configured, allocate memory then fill in those arrays with second hypercall. Wei. > Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2 12/19] hvmloader: retrieve vNUMA information from hypervisor
>>> On 01.12.14 at 16:33, wrote: > @@ -261,6 +262,8 @@ int main(void) > > init_hypercalls(); > > +init_vnuma_info(); This is very early, and I don't think it needs to be - I guess it could be done right before doing ACPI stuff? > --- /dev/null > +++ b/tools/firmware/hvmloader/vnuma.c > @@ -0,0 +1,84 @@ > +/* > + * vnuma.c: obtain vNUMA information from hypervisor > + * > + * Copyright (c) 2014 Wei Liu, Citrix Systems (R&D) Ltd. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * 1. Redistributions of source code must retain the above copyright > + *notice, this list of conditions and the following disclaimer. > + * 2. Redistributions in binary form must reproduce the above copyright > + *notice, this list of conditions and the following disclaimer in the > + *documentation and/or other materials provided with the distribution. > + * > + * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND > + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE > + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE > + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE > + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL > + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS > + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) > + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT > + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY > + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF > + * SUCH DAMAGE. > + */ > + > +#include "util.h" > +#include "hypercall.h" > +#include "vnuma.h" > +#include This is the system header, not the Xen one. See the discussion at http://lists.xenproject.org/archives/html/xen-devel/2014-10/msg03206.html and perhaps build on the resulting patch http://lists.xenproject.org/archives/html/xen-devel/2014-12/msg00013.html. > + > +unsigned int nr_vnodes, nr_vmemranges; > +unsigned int *vcpu_to_vnode, *vdistance; > +xen_vmemrange_t *vmemrange; > + > +void init_vnuma_info(void) > +{ > +int rc, retry = 0; > +struct xen_vnuma_topology_info vnuma_topo = { > +.domid = DOMID_SELF, > +}; > + > +vcpu_to_vnode = scratch_alloc(sizeof(uint32_t) * hvm_info->nr_vcpus, 0); > +vdistance = scratch_alloc(sizeof(uint32_t) * MAX_VDISTANCE, 0); > +vmemrange = scratch_alloc(sizeof(xen_vmemrange_t) * MAX_VMEMRANGES, 0); > + > +vnuma_topo.nr_vnodes = MAX_VNODES; > +vnuma_topo.nr_vcpus = hvm_info->nr_vcpus; > +vnuma_topo.nr_vmemranges = MAX_VMEMRANGES; > + > +set_xen_guest_handle(vnuma_topo.vdistance.h, vdistance); > +set_xen_guest_handle(vnuma_topo.vcpu_to_vnode.h, vcpu_to_vnode); > +set_xen_guest_handle(vnuma_topo.vmemrange.h, vmemrange); > + > +rc = hypercall_memory_op(XENMEM_get_vnumainfo, &vnuma_topo); > +while ( rc == -EAGAIN && retry < 10 ) > +{ > +rc = hypercall_memory_op(XENMEM_get_vnumainfo, &vnuma_topo); > +retry++; > +} > +if ( rc < 0 ) > +{ > +printf("Failed to retrieve vNUMA information, rc = %d\n", rc); > +goto out; return; > +} > + > +nr_vnodes = vnuma_topo.nr_vnodes; > +nr_vmemranges = vnuma_topo.nr_vmemranges; > + > +out: > +return; Drop these two (or really three) unnecessary lines please. > --- /dev/null > +++ b/tools/firmware/hvmloader/vnuma.h > @@ -0,0 +1,56 @@ > +/** > + * vnuma.h > + * > + * Copyright (c) 2014, Wei Liu > + * > + * This program is free software; you can redistribute it and/or > + * modify it under the terms of the GNU General Public License version 2 > + * as published by the Free Software Foundation; or, when distributed > + * separately from the Linux kernel or incorporated into other > + * software packages, subject to the following license: > + * > + * Permission is hereby granted, free of charge, to any person obtaining a > copy > + * of this source file (the "Software"), to deal in the Software without > + * restriction, including without limitation the rights to use, copy, modify, > + * merge, publish, distribute, sublicense, and/or sell copies of the > Software, > + * and to permit persons to whom the Software is furnished to do so, subject > to > + * the following conditions: > + * > + * The above copyright notice and this permission notice shall be included in > + * all copies or substantial portions of the Software. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL > THE > + * AUTHORS OR COPYRIGHT HOLDE
[Xen-devel] [PATCH v2 12/19] hvmloader: retrieve vNUMA information from hypervisor
Hvmloader issues XENMEM_get_vnumainfo hypercall and stores the information retrieved in scratch space for later use. Signed-off-by: Wei Liu Cc: Jan Beulich --- tools/firmware/hvmloader/Makefile|2 +- tools/firmware/hvmloader/hvmloader.c |3 ++ tools/firmware/hvmloader/vnuma.c | 84 ++ tools/firmware/hvmloader/vnuma.h | 56 +++ 4 files changed, 144 insertions(+), 1 deletion(-) create mode 100644 tools/firmware/hvmloader/vnuma.c create mode 100644 tools/firmware/hvmloader/vnuma.h diff --git a/tools/firmware/hvmloader/Makefile b/tools/firmware/hvmloader/Makefile index 46a79c5..a5af34b 100644 --- a/tools/firmware/hvmloader/Makefile +++ b/tools/firmware/hvmloader/Makefile @@ -29,7 +29,7 @@ LOADADDR = 0x10 CFLAGS += $(CFLAGS_xeninclude) OBJS = hvmloader.o mp_tables.o util.o smbios.o -OBJS += smp.o cacheattr.o xenbus.o +OBJS += smp.o cacheattr.o xenbus.o vnuma.o OBJS += e820.o pci.o pir.o ctype.o OBJS += hvm_param.o ifeq ($(debug),y) diff --git a/tools/firmware/hvmloader/hvmloader.c b/tools/firmware/hvmloader/hvmloader.c index 7b0da38..2aa2f76 100644 --- a/tools/firmware/hvmloader/hvmloader.c +++ b/tools/firmware/hvmloader/hvmloader.c @@ -26,6 +26,7 @@ #include "pci_regs.h" #include "apic_regs.h" #include "acpi/acpi2_0.h" +#include "vnuma.h" #include #include @@ -261,6 +262,8 @@ int main(void) init_hypercalls(); +init_vnuma_info(); + xenbus_setup(); bios = detect_bios(); diff --git a/tools/firmware/hvmloader/vnuma.c b/tools/firmware/hvmloader/vnuma.c new file mode 100644 index 000..cd26f4f --- /dev/null +++ b/tools/firmware/hvmloader/vnuma.c @@ -0,0 +1,84 @@ +/* + * vnuma.c: obtain vNUMA information from hypervisor + * + * Copyright (c) 2014 Wei Liu, Citrix Systems (R&D) Ltd. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + *notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + *notice, this list of conditions and the following disclaimer in the + *documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include "util.h" +#include "hypercall.h" +#include "vnuma.h" +#include + +unsigned int nr_vnodes, nr_vmemranges; +unsigned int *vcpu_to_vnode, *vdistance; +xen_vmemrange_t *vmemrange; + +void init_vnuma_info(void) +{ +int rc, retry = 0; +struct xen_vnuma_topology_info vnuma_topo = { +.domid = DOMID_SELF, +}; + +vcpu_to_vnode = scratch_alloc(sizeof(uint32_t) * hvm_info->nr_vcpus, 0); +vdistance = scratch_alloc(sizeof(uint32_t) * MAX_VDISTANCE, 0); +vmemrange = scratch_alloc(sizeof(xen_vmemrange_t) * MAX_VMEMRANGES, 0); + +vnuma_topo.nr_vnodes = MAX_VNODES; +vnuma_topo.nr_vcpus = hvm_info->nr_vcpus; +vnuma_topo.nr_vmemranges = MAX_VMEMRANGES; + +set_xen_guest_handle(vnuma_topo.vdistance.h, vdistance); +set_xen_guest_handle(vnuma_topo.vcpu_to_vnode.h, vcpu_to_vnode); +set_xen_guest_handle(vnuma_topo.vmemrange.h, vmemrange); + +rc = hypercall_memory_op(XENMEM_get_vnumainfo, &vnuma_topo); +while ( rc == -EAGAIN && retry < 10 ) +{ +rc = hypercall_memory_op(XENMEM_get_vnumainfo, &vnuma_topo); +retry++; +} +if ( rc < 0 ) +{ +printf("Failed to retrieve vNUMA information, rc = %d\n", rc); +goto out; +} + +nr_vnodes = vnuma_topo.nr_vnodes; +nr_vmemranges = vnuma_topo.nr_vmemranges; + +out: +return; +} + + +/* + * Local variables: + * mode: C + * c-file-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/tools/firmware/hvmloader/vnuma.h b/tools/firmware/hvmloader/vnuma.h new file mode 100644 index 000..0779b83 --- /dev/null +++ b/tools/firmware/hvmloader/vnuma.h @@ -0,0 +1,56 @@ +/** + * vnuma.h + * + * Copyright (c) 2014, Wei Liu + * + * This program is free software; you can re