Change in vdsm[master]: Move NUMA collecting code to stats thread
automat...@ovirt.org has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 3: * Update tracker::#1182094::OK * Set MODIFIED::bug 1182094#1182094IGNORE, not oVirt prod but Red Hat Enterprise Virtualization Manager -- To view, visit https://gerrit.ovirt.org/36906 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
automat...@ovirt.org has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 2: * Update tracker::#1182094::OK * Check Bug-Url::OK * Check Public Bug::#1182094::OK, public bug * Check Product::#1182094::OK, Correct product Red Hat Enterprise Virtualization Manager * Check TR::SKIP, not in a monitored branch (ovirt-3.5 ovirt-3.4 ovirt-3.3 ovirt-3.2) * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3']) -- To view, visit https://gerrit.ovirt.org/36906 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
Piotr Kliczewski has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.ovirt.org/36906 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
automat...@ovirt.org has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 1: * Update tracker::#1182094::OK * Check Bug-Url::OK * Check Public Bug::#1182094::OK, public bug * Check Product::#1182094::OK, Correct product Red Hat Enterprise Virtualization Manager * Check TR::SKIP, not in a monitored branch (ovirt-3.5 ovirt-3.4 ovirt-3.3 ovirt-3.2) * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3']) -- To view, visit http://gerrit.ovirt.org/36906 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
oVirt Jenkins CI Server has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 3: Build Successful http://jenkins.ovirt.org/job/vdsm_master_create-rpms-el6-x86_64_merged/732/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master-libgfapi_create-rpms-fc21-x86_64_merged/328/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_unit-tests_merged/4712/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_create-rpms-fc21-x86_64_merged/705/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master-libgfapi_create-rpms-el6-x86_64_merged/333/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_create-rpms-el7-x86_64_merged/730/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master-libgfapi_create-rpms-el7-x86_64_merged/332/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master-libgfapi_create-rpms-fc20-x86_64_merged/322/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_create-rpms-fc20-x86_64_merged/737/ : SUCCESS -- To view, visit https://gerrit.ovirt.org/36906 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
Martin Sivák has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 2: Verified+1 I was able to start a NUMA enabled VM and I saw an improvement in getAllVmStats performance. -- To view, visit https://gerrit.ovirt.org/36906 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
oVirt Jenkins CI Server has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 2: Build Failed http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/16485/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/16656/ : FAILURE http://jenkins.ovirt.org/job/vdsm_master_unit_tests_el_gerrit/15685/ : FAILURE -- To view, visit https://gerrit.ovirt.org/36906 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
Dan Kenigsberg has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 2: Code-Review+2 So basically, all this gives us is a 13% improvement of latency. This is basically peanuts; we should have the supervdsm cache the parsed file, and invalidate it in case of mtime change. However, this patch is mostly harmless. Let's take it in. -- To view, visit https://gerrit.ovirt.org/36906 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
Dan Kenigsberg has submitted this change and it was merged. Change subject: Move NUMA collecting code to stats thread .. Move NUMA collecting code to stats thread NUMA collecting was performed any time the .get() method was invoked. That caused a serious slow down. This patch moves the collecting to where all the other stats are collected. I measured the performance of getAllVmStats before and after this patch was applied on a host with local storage and two NUMA enabled VMs using: # time for x in $(seq 100); do # vdsClient -s 0 getAllVmStats /dev/null; # done The results for unpatched VDSM: real0m21.093s user0m11.998s sys 0m1.690s The results for patched VDSM: real0m18.485s user0m12.009s sys 0m1.846s And the results for both patched and unpatched VDSM when no NUMA enabled VM is started are very similar: real0m18.298s user0m11.878s sys 0m1.699s Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1182094 Signed-off-by: Martin Sivak msi...@redhat.com Reviewed-on: https://gerrit.ovirt.org/36906 Reviewed-by: Francesco Romani from...@redhat.com Reviewed-by: Piotr Kliczewski piotr.kliczew...@gmail.com Reviewed-by: Dan Kenigsberg dan...@redhat.com --- M lib/vdsm/config.py.in M vdsm/virt/vm.py 2 files changed, 17 insertions(+), 2 deletions(-) Approvals: Piotr Kliczewski: Looks good to me, but someone else must approve Martin Sivák: Verified Dan Kenigsberg: Looks good to me, approved Francesco Romani: Looks good to me, but someone else must approve -- To view, visit https://gerrit.ovirt.org/36906 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
Dan Kenigsberg has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 1: (1 comment) http://gerrit.ovirt.org/#/c/36906/1//COMMIT_MSG Commit Message: Line 7: Move NUMA collecting code to stats thread Line 8: Line 9: NUMA collecting was performed any time the .get() method was Line 10: invoked. That caused a serious slow down. This patch moves Line 11: the collecting to where all the other stats are collected. yes, it makes sense that going over to supervdsm asynchronously would reduce getVmStats's latency. Is it really the solution to the cited bug? If so, please provide hard numbers in the commit message. Line 12: Line 13: Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Line 14: Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1182094 -- To view, visit http://gerrit.ovirt.org/36906 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: Yes ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
Dan Kenigsberg has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.ovirt.org/36906 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
Francesco Romani has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 1: Code-Review+1 will help with latency until we figure out how to do deeper improvements. -- To view, visit http://gerrit.ovirt.org/36906 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
oVirt Jenkins CI Server has posted comments on this change. Change subject: Move NUMA collecting code to stats thread .. Patch Set 1: Build Failed http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/15038/ : FAILURE http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/14869/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_virt_functional_tests_gerrit/2176/ : There was an infra issue, please contact in...@ovirt.org http://jenkins.ovirt.org/job/vdsm_master_unit_tests_gerrit_el/14081/ : FAILURE -- To view, visit http://gerrit.ovirt.org/36906 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Sivák msi...@redhat.com Gerrit-Reviewer: Dan Kenigsberg dan...@redhat.com Gerrit-Reviewer: Francesco Romani from...@redhat.com Gerrit-Reviewer: Piotr Kliczewski piotr.kliczew...@gmail.com Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: Move NUMA collecting code to stats thread
Martin Sivák has uploaded a new change for review.
Change subject: Move NUMA collecting code to stats thread
..
Move NUMA collecting code to stats thread
NUMA collecting was performed any time the .get() method was
invoked. That caused a serious slow down. This patch moves
the collecting to where all the other stats are collected.
Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1182094
Signed-off-by: Martin Sivak msi...@redhat.com
---
M lib/vdsm/config.py.in
M vdsm/virt/vm.py
2 files changed, 17 insertions(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/06/36906/1
diff --git a/lib/vdsm/config.py.in b/lib/vdsm/config.py.in
index 94c6782..cf8cadd 100644
--- a/lib/vdsm/config.py.in
+++ b/lib/vdsm/config.py.in
@@ -167,6 +167,9 @@
('vm_sample_cpu_tune_interval', '15', None),
+('vm_sample_numa_interval', '15',
+'How often should we sample NUMA CPU assignments'),
+
('trust_store_path', '@TRUSTSTORE@',
'Where the certificates and keys are situated.'),
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index 2883517..dd29ceb 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -225,11 +225,16 @@
AdvancedStatsFunction(
self._sampleCpuTune,
config.getint('vars', 'vm_sample_cpu_tune_interval'), 1))
+self.sampleNuma = (
+AdvancedStatsFunction(
+self._sampleNuma,
+config.getint('vars', 'vm_sample_numa_interval'), 1))
self.addStatsFunction(
self.highWrite, self.updateVolumes, self.sampleCpu,
self.sampleDisk, self.sampleNet, self.sampleBalloon,
-self.sampleVmJobs, self.sampleVcpuPinning, self.sampleCpuTune)
+self.sampleVmJobs, self.sampleVcpuPinning, self.sampleCpuTune,
+self.sampleNuma)
def _highWrite(self):
if not self._vm.isDisksStatsCollectionEnabled():
@@ -265,6 +270,13 @@
cpuStats = self._vm._dom.getCPUStats(True, 0)
return cpuStats[0]
+
+def _sampleNuma(self):
+
+Numa CPU assignments.
+
+vmNumaNodeRuntimeMap = numaUtils.getVmNumaNodeRuntimeInfo(self._vm)
+return vmNumaNodeRuntimeMap
def _sampleDisk(self):
@@ -635,7 +647,7 @@
stats[vmDrive.name] = dStats
def _getNumaStats(self, stats):
-vmNumaNodeRuntimeMap = numaUtils.getVmNumaNodeRuntimeInfo(self._vm)
+vmNumaNodeRuntimeMap = self.sampleNuma.getLastSample()
if vmNumaNodeRuntimeMap:
stats['vNodeRuntimeInfo'] = vmNumaNodeRuntimeMap
--
To view, visit http://gerrit.ovirt.org/36906
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3fff147d892ecbe7e01629a6c9428f911672fcdd
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Sivák msi...@redhat.com
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
