Re: [Engine-devel] NUMA support action items
On 4/3/2014 7:11 AM, Gilad Chaplik wrote: - Original Message - From: "Chegu Vinod" To: "Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ)" Cc: "Einav Cohen" , "Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC)" , "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)" , msi...@redhat.com, "Da-huai Tang (Gary, MCXS-CQ)" , "Malini Rao" , "Eldan Hildesheim" , "Doron Fediuck" , sher...@redhat.com, "Alexander Wels" , "Gilad Chaplik" Sent: Thursday, April 3, 2014 3:28:03 PM Subject: RE: NUMA support action items Hi Bruce, The virtual NUMA layout in the guest is a very simple one (not multi-level etc). It is generated by qemu+seabios... and there is no relationship with the host NUMA node distances etc. Let us not worry about gathering Virtual NUMA node distances for now. Vinod CC'ing devel list as well. Having said that, I don't see a reason why not to prepare an infrastructure for that (if it's free) for future versions (guest agent will collect vNuma data in some point in time). If you think having this Virtual NUMA topology (along with the virtual numa node *distance* info.) really helps some future use cases then pl. go ahead... Vinod Thanks, Gilad. -Original Message- From: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ) Sent: Thursday, April 03, 2014 12:41 AM To: Vinod, Chegu Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Tang, Da-huai (Gary, MCXS-CQ); Malini Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels; Gilad Chaplik Subject: RE: NUMA support action items Hi Vinod, Is it meaningful for us to collect the distance information of vm numa node (maybe in future, not now)? In my understanding, vm numa topology is a simulation of numa topology, since the vcpus are just threads, I don't know how the vm numa node distances are calculated in vm. Is there any relationship between the vNode distances and host node distances? Thanks & Best Regards Shi, Xiao-Lei (Bruce) Hewlett-Packard Co., Ltd. HP Servers Core Platform Software China Telephone +86 23 65683093 Mobile +86 18696583447 Email xiao-lei@hp.com -Original Message- From: Vinod, Chegu Sent: Thursday, April 03, 2014 7:18 AM To: Gilad Chaplik Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels Subject: RE: NUMA support action items Not sure what the correct way to do this isbut here is a suggestion. Let a given host server diagram shown be very generic...i.e. show the N sockets/nodes numbered from 0 thru N-1. Show the amount of memory and the list of CPUs in each of those sockets/nodes. Draw a generic Interconnect fabric [box] in between which all the sockets connect to Ideally ... Under that host diagram we could show the NUMA node distances in text format (as you know this is derived from the "numactl -H" and then conveyed from VDSM-> oVIrt engine etc). That distance info. will tell the user what the distance between a pair of sockets/nodes are (and they can then do what they wish after that :)). Vinod -Original Message- From: Gilad Chaplik [mailto:gchap...@redhat.com] Sent: Wednesday, April 02, 2014 4:09 PM To: Vinod, Chegu Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels Subject: Re: NUMA support action items Thank you Vinod for the much elaborate explanation. GUI-wise, do you want to show those numbers? maybe for first phase, enough to show them via API? A thought, According to your example there could be up to 2 distances, so maybe the 'closer' nodes can be on the same column or sth; I mean to try an illustrate it graphically rather than with numbers (we have enough of those :)). Thanks, Gilad. - Original Message - From: "Chegu Vinod" To: "Einav Cohen" Cc: "Gilad Chaplik" , "Shang-Chun Liang (David Liang, HPservers-Core-OE-PSC)" , "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)" , msi...@redhat.com, "Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ)" , "Da-huai Tang (Gary, MCXS-CQ)" , "Malini Rao" , "Eldan Hildesheim" , "Doron Fediuck" , sher...@redhat.com, "Alexander Wels" Sent: Saturday, March 29, 2014 8:15:56 AM Subject: Re: NUMA support action items On 3/27/2014 10:42 AM, Einav Cohen wrote: Hi Vinod, thank you very much for that extra information. unfortunately, we are not familiar with what are levels of NUMA (local socket/node, buddy socket/node, remote socket/ node) and/or what "distance" is - I assume that these are definitions that are related to the physical layout of the sock
Re: [Engine-devel] NUMA support action items
- Original Message - > From: "Chegu Vinod" > To: "Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ)" > Cc: "Einav Cohen" , "Shang-Chun Liang (David Liang, > HPservers-Core-OE-PSC)" > , "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)" > , msi...@redhat.com, > "Da-huai Tang (Gary, MCXS-CQ)" , "Malini Rao" > , "Eldan Hildesheim" > , "Doron Fediuck" , > sher...@redhat.com, "Alexander Wels" > , "Gilad Chaplik" > Sent: Thursday, April 3, 2014 3:28:03 PM > Subject: RE: NUMA support action items > > Hi Bruce, > > The virtual NUMA layout in the guest is a very simple one (not multi-level > etc). It is generated by qemu+seabios... and there is no relationship with > the host NUMA node distances etc. Let us not worry about gathering Virtual > NUMA node distances for now. > > Vinod > CC'ing devel list as well. Having said that, I don't see a reason why not to prepare an infrastructure for that (if it's free) for future versions (guest agent will collect vNuma data in some point in time). Thanks, Gilad. > > -Original Message- > From: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ) > Sent: Thursday, April 03, 2014 12:41 AM > To: Vinod, Chegu > Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); > Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Tang, > Da-huai (Gary, MCXS-CQ); Malini Rao; Eldan Hildesheim; Doron Fediuck; > sher...@redhat.com; Alexander Wels; Gilad Chaplik > Subject: RE: NUMA support action items > > Hi Vinod, > > Is it meaningful for us to collect the distance information of vm numa node > (maybe in future, not now)? > In my understanding, vm numa topology is a simulation of numa topology, since > the vcpus are just threads, I don't know how the vm numa node distances are > calculated in vm. Is there any relationship between the vNode distances and > host node distances? > > Thanks & Best Regards > Shi, Xiao-Lei (Bruce) > > Hewlett-Packard Co., Ltd. > HP Servers Core Platform Software China Telephone +86 23 65683093 Mobile +86 > 18696583447 Email xiao-lei@hp.com > > > -Original Message- > From: Vinod, Chegu > Sent: Thursday, April 03, 2014 7:18 AM > To: Gilad Chaplik > Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); > Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi, > Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini > Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels > Subject: RE: NUMA support action items > > Not sure what the correct way to do this isbut here is a suggestion. > > Let a given host server diagram shown be very generic...i.e. show the N > sockets/nodes numbered from 0 thru N-1. Show the amount of memory and the > list of CPUs in each of those sockets/nodes. > Draw a generic Interconnect fabric [box] in between which all the sockets > connect to > > Ideally ... Under that host diagram we could show the NUMA node distances in > text format (as you know this is derived from the "numactl -H" and then > conveyed from VDSM-> oVIrt engine etc). > That distance info. will tell the user what the distance between a pair of > sockets/nodes are (and they can then do what they wish after that :)). > > Vinod > > -Original Message- > From: Gilad Chaplik [mailto:gchap...@redhat.com] > Sent: Wednesday, April 02, 2014 4:09 PM > To: Vinod, Chegu > Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); > Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi, > Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini > Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels > Subject: Re: NUMA support action items > > Thank you Vinod for the much elaborate explanation. > GUI-wise, do you want to show those numbers? maybe for first phase, enough to > show them via API? > > A thought, According to your example there could be up to 2 distances, so > maybe the 'closer' nodes can be on the same column or sth; I mean to try an > illustrate it graphically rather than with numbers (we have enough of those > :)). > > Thanks, > Gilad. > > - Original Message - > > From: "Chegu Vinod" > > To: "Einav Cohen" > > Cc: "Gilad Chaplik" , "Shang-Chun Liang (David Liang, > > HPservers-Core-OE-PSC)" > > , "Chuan Liao (Jason Liao, > > HPservers-Core-OE-PSC)" , msi...@redhat.com, "Xiao-Lei > > Shi (Bruce, HP Servers-PSC-CQ)" , "Da-huai Tang > > (Gary, MCXS-CQ)" > > , "Malini Rao" , "Eldan Hildesheim" > > , "Doron Fediuck" > > , sher...@redhat.com, "Alexander Wels" > > > > Sent: Saturday, March 29, 2014 8:15:56 AM > > Subject: Re: NUMA support action items > > > > On 3/27/2014 10:42 AM, Einav Cohen wrote: > > > Hi Vinod, thank you very much for that extra information. > > > > > > unfortunately, we are not familiar with what are levels of NUMA > > > (local socket/node, buddy socket/node, remote socket/ > > > node) and/or what "distance" is - I assume th
Re: [Engine-devel] NUMA support action items
- Original Message - > From: "Chegu Vinod" > To: "Gilad Chaplik" > Cc: "Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ)" , "Einav > Cohen" , "Shang-Chun > Liang (David Liang, HPservers-Core-OE-PSC)" , "Chuan > Liao (Jason Liao, > HPservers-Core-OE-PSC)" , msi...@redhat.com, "Da-huai Tang > (Gary, MCXS-CQ)" > , "Malini Rao" , "Eldan Hildesheim" > , "Doron Fediuck" > , sher...@redhat.com, "Alexander Wels" > , "engine-devel" > > Sent: Thursday, April 3, 2014 5:21:49 PM > Subject: Re: NUMA support action items > > On 4/3/2014 7:11 AM, Gilad Chaplik wrote: > > - Original Message - > >> From: "Chegu Vinod" > >> To: "Xiao-Lei Shi (Bruce, HP Servers-PSC-CQ)" > >> Cc: "Einav Cohen" , "Shang-Chun Liang (David Liang, > >> HPservers-Core-OE-PSC)" > >> , "Chuan Liao (Jason Liao, HPservers-Core-OE-PSC)" > >> , msi...@redhat.com, > >> "Da-huai Tang (Gary, MCXS-CQ)" , "Malini Rao" > >> , "Eldan Hildesheim" > >> , "Doron Fediuck" , > >> sher...@redhat.com, "Alexander Wels" > >> , "Gilad Chaplik" > >> Sent: Thursday, April 3, 2014 3:28:03 PM > >> Subject: RE: NUMA support action items > >> > >> Hi Bruce, > >> > >> The virtual NUMA layout in the guest is a very simple one (not multi-level > >> etc). It is generated by qemu+seabios... and there is no relationship with > >> the host NUMA node distances etc. Let us not worry about gathering > >> Virtual > >> NUMA node distances for now. > >> > >> Vinod > >> > > CC'ing devel list as well. > > > > Having said that, I don't see a reason why not to prepare an infrastructure > > for that (if it's free) for future versions (guest agent will collect > > vNuma data in some point in time). > > If you think having this Virtual NUMA topology (along with the virtual > numa node *distance* info.) really helps some future use cases then pl. > go ahead... > > Vinod > > I really don't know. but IMO, me as a user that gets some machine (transparent to the fact it's VM or bare metal), it would be very nice to see the NUMA stats outside of the machine. Thanks, Gilad. > > > > > Thanks, > > Gilad. > > > >> -Original Message- > >> From: Shi, Xiao-Lei (Bruce, HP Servers-PSC-CQ) > >> Sent: Thursday, April 03, 2014 12:41 AM > >> To: Vinod, Chegu > >> Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); > >> Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Tang, > >> Da-huai (Gary, MCXS-CQ); Malini Rao; Eldan Hildesheim; Doron Fediuck; > >> sher...@redhat.com; Alexander Wels; Gilad Chaplik > >> Subject: RE: NUMA support action items > >> > >> Hi Vinod, > >> > >> Is it meaningful for us to collect the distance information of vm numa > >> node > >> (maybe in future, not now)? > >> In my understanding, vm numa topology is a simulation of numa topology, > >> since > >> the vcpus are just threads, I don't know how the vm numa node distances > >> are > >> calculated in vm. Is there any relationship between the vNode distances > >> and > >> host node distances? > >> > >> Thanks & Best Regards > >> Shi, Xiao-Lei (Bruce) > >> > >> Hewlett-Packard Co., Ltd. > >> HP Servers Core Platform Software China Telephone +86 23 65683093 Mobile > >> +86 > >> 18696583447 Email xiao-lei@hp.com > >> > >> > >> -Original Message- > >> From: Vinod, Chegu > >> Sent: Thursday, April 03, 2014 7:18 AM > >> To: Gilad Chaplik > >> Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); > >> Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi, > >> Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini > >> Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels > >> Subject: RE: NUMA support action items > >> > >> Not sure what the correct way to do this isbut here is a suggestion. > >> > >> Let a given host server diagram shown be very generic...i.e. show the N > >> sockets/nodes numbered from 0 thru N-1. Show the amount of memory and the > >> list of CPUs in each of those sockets/nodes. > >> Draw a generic Interconnect fabric [box] in between which all the sockets > >> connect to > >> > >> Ideally ... Under that host diagram we could show the NUMA node distances > >> in > >> text format (as you know this is derived from the "numactl -H" and then > >> conveyed from VDSM-> oVIrt engine etc). > >> That distance info. will tell the user what the distance between a pair of > >> sockets/nodes are (and they can then do what they wish after that :)). > >> > >> Vinod > >> > >> -Original Message- > >> From: Gilad Chaplik [mailto:gchap...@redhat.com] > >> Sent: Wednesday, April 02, 2014 4:09 PM > >> To: Vinod, Chegu > >> Cc: Einav Cohen; Liang, Shang-Chun (David Liang, HPservers-Core-OE-PSC); > >> Liao, Chuan (Jason Liao, HPservers-Core-OE-PSC); msi...@redhat.com; Shi, > >> Xiao-Lei (Bruce, HP Servers-PSC-CQ); Tang, Da-huai (Gary, MCXS-CQ); Malini > >> Rao; Eldan Hildesheim; Doron Fediuck; sher...@redhat.com; Alexander Wels > >> Subject: Re: NUMA support action items
