Change in vdsm[master]: virt: sampling: replace flag with Stage
Francesco Romani has abandoned this change. Change subject: virt: sampling: replace flag with Stage .. Abandoned replaced by https://gerrit.ovirt.org/#/c/40353/ -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: abandon Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 4 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: automat...@ovirt.org ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
automat...@ovirt.org has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 4: * Update tracker::IGNORE, no Bug-Url found -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 4 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: automat...@ovirt.org Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 4: Verified-1 Build Failed http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18344/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1574/ : FAILURE -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 4 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 4: Build Started (2/2) -> http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1574/ -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 4 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 4: Build Started (1/2) -> http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18344/ -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 4 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
automat...@ovirt.org has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 4: * Update tracker::IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3']) -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 4 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 3: Build Successful http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18331/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1561/ : SUCCESS -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 3: Build Started (1/2) -> http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18331/ -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 3: Build Started (2/2) -> http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1561/ -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
automat...@ovirt.org has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 3: * Update tracker::IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3']) -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 2: Build Failed http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18253/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1483/ : 0 -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 2: Build Started (2/2) 0 -> http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1483/ -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 2: Build Started (1/2) -> http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18253/ -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
automat...@ovirt.org has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 2: * Update tracker::IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3']) -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 1: Build Successful http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18239/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1469/ : 0 -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 1: Build Started (2/2) 0 -> http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1469/ -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
oVirt Jenkins CI Server has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 1: Build Started (1/2) -> http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18239/ -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
automat...@ovirt.org has posted comments on this change. Change subject: virt: sampling: replace flag with Stage .. Patch Set 1: * Update tracker::IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3']) -- To view, visit https://gerrit.ovirt.org/40322 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani Gerrit-Reviewer: automat...@ovirt.org Gerrit-HasComments: No ___ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
Change in vdsm[master]: virt: sampling: replace flag with Stage
Francesco Romani has uploaded a new change for review.
Change subject: virt: sampling: replace flag with Stage
..
virt: sampling: replace flag with Stage
In order to detect stuck sampling calls, the SampleVM operation used
a simple boolean flag to mark business or not.
However, this may lead to races if an operation unblocks just after the
flag was set.
This race is expected to do little harm, but could and should be improved
anyway.
We can consider threading.Semaphore (or BoundedSemaphore), but we don't
want to rate-limit the number of working doing sampling, due to stuck
calls which can hold the semaphore indefinitely;
Moreover, we definitely don't want to fresh thread to wait for the
stuck ones.
We just need a safe object which can tell us if a critical section is
busy or not, because calling code will need to react in a different way than
just block.
So, we introduce the simple 'Stage' utility class, which just guards a
critical section (renamed 'stage'), and which keeps track of the performers
on the stage at any given time.
Client code can query the stage for emptiness, and react properly.
'Stage' is somehow reminiscent of a Semaphore, but just do accouting.
Change-Id: I09b4fec4d609fe22e890de911342f80505de88c9
Signed-off-by: Francesco Romani
---
M tests/vmUtilsTests.py
M vdsm/virt/sampling.py
M vdsm/virt/utils.py
3 files changed, 81 insertions(+), 19 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/22/40322/1
diff --git a/tests/vmUtilsTests.py b/tests/vmUtilsTests.py
index 74bf378..699b74e 100644
--- a/tests/vmUtilsTests.py
+++ b/tests/vmUtilsTests.py
@@ -18,6 +18,9 @@
# Refer to the README and COPYING files for full details of the license
#
+import threading
+import time
+
from virt import utils
from testlib import VdsmTestCase as TestCaseBase
@@ -125,3 +128,40 @@
clock.now = 3.0
self.assertFalse(cache)
+
+
+class StageTests(TestCaseBase):
+def setUp(self):
+self.stage = utils.Stage()
+
+def test_default_empty(self):
+self.assertTrue(self.stage.empty)
+
+def test_busy(self):
+with self.stage.performer():
+self.assertFalse(self.stage.empty)
+self.assertTrue(self.stage.empty)
+
+def test_busy_different_thread(self):
+
+run = True
+
+def _fun():
+with self.stage.performer():
+while run:
+time.sleep(0.1)
+
+t = threading.Thread(target=_fun)
+
+self.assertTrue(self.stage.empty)
+
+t.start()
+self.assertFalse(self.stage.empty)
+
+with self.stage.performer():
+self.assertFalse(self.stage.empty)
+
+run = False
+t.join()
+
+self.assertTrue(self.stage.empty)
diff --git a/vdsm/virt/sampling.py b/vdsm/virt/sampling.py
index 746f059..09ce2b3 100644
--- a/vdsm/virt/sampling.py
+++ b/vdsm/virt/sampling.py
@@ -37,7 +37,7 @@
from vdsm.config import config
import v2v
-from .utils import ExpiringCache
+from .utils import ExpiringCache, Stage
import caps
@@ -499,36 +499,35 @@
self._stats_cache = stats_cache
self._stats_flags = stats_flags
self._skip_doms = ExpiringCache(timeout)
-self._sampling = False
+self._sampling = Stage()
self._log = logging.getLogger("sampling.SampleVMs")
def __call__(self):
timestamp = self._stats_cache.clock()
# we are deep in the hot path. bool(ExpiringCache)
# *is* costly so we should avoid it if we can.
-fast_path = (not self._sampling and not self._skip_doms)
-self._sampling = True
+fast_path = (self._sampling.empty and not self._skip_doms)
try:
-if fast_path:
-# This is expected to be the common case.
-# If everything's ok, we can skip all the costly checks.
-bulk_stats = self._conn.getAllDomainStats(self._stats_flags)
-else:
-# A previous call got stuck, or not every domain
-# has properly recovered. Thus we must whitelist domains.
-doms = self._get_responsive_doms()
-self._log.debug('sampling %d domains', len(doms))
-if doms:
-bulk_stats = self._conn.domainListGetStats(
-doms, self._stats_flags)
+with self._sampling.performer():
+if fast_path:
+# This is expected to be the common case.
+# If everything's ok, we can skip all the costly checks.
+bulk_stats = self._conn.getAllDomainStats(
+self._stats_flags)
else:
-bulk_stats = []
+# A previous call got stuck, or not every domain
+# has properly recovered. Thus we must whitelist domains.
+dom
