Change in vdsm[master]: DONTMERGE virt: use "run_async" helper

[automation]

gerrit-hooks has posted comments on this change.

Change subject: DONTMERGE virt: use "run_async" helper
..


Patch Set 4:

* Update tracker: IGNORE, no Bug-Url found

-- 
To view, visit https://gerrit.ovirt.org/49640
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3147b6e860b9839b656257fcc22bf6f4b3c58914
Gerrit-PatchSet: 4
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: DONTMERGE virt: use "run_async" helper

[fromani]

Francesco Romani has abandoned this change.

Change subject: DONTMERGE virt: use "run_async" helper
..


Abandoned

no longer needed

-- 
To view, visit https://gerrit.ovirt.org/49640
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I3147b6e860b9839b656257fcc22bf6f4b3c58914
Gerrit-PatchSet: 4
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: gerrit-hooks 
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: DONTMERGE virt: use "run_async" helper

[automation]

gerrit-hooks has posted comments on this change.

Change subject: DONTMERGE virt: use "run_async" helper
..


Patch Set 4:

* Update tracker: IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 
'ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/49640
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3147b6e860b9839b656257fcc22bf6f4b3c58914
Gerrit-PatchSet: 4
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: DONTMERGE virt: use "run_async" helper

[automation]

gerrit-hooks has posted comments on this change.

Change subject: DONTMERGE virt: use "run_async" helper
..


Patch Set 3:

* Update tracker: IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 
'ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/49640
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3147b6e860b9839b656257fcc22bf6f4b3c58914
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: DONTMERGE virt: use "run_async" helper

[automation]

gerrit-hooks has posted comments on this change.

Change subject: DONTMERGE virt: use "run_async" helper
..


Patch Set 2:

* Update tracker: IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 
'ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/49640
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3147b6e860b9839b656257fcc22bf6f4b3c58914
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: DONTMERGE virt: use "run_async" helper

[fromani]

Francesco Romani has uploaded a new change for review.

Change subject: DONTMERGE virt: use "run_async" helper
..

DONTMERGE virt: use "run_async" helper

Quick hack demo to see how it will look when we use run_async
to throttle the migrations.

Change-Id: I3147b6e860b9839b656257fcc22bf6f4b3c58914
Wiki: http://www.ovirt.org/Features/Migration_Enhancements
Signed-off-by: Francesco Romani 
---
M lib/vdsm/config.py.in
M lib/vdsm/define.py
M vdsm/API.py
M vdsm/clientIF.py
M vdsm/virt/migration.py
5 files changed, 25 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/40/49640/1

diff --git a/lib/vdsm/config.py.in b/lib/vdsm/config.py.in
index 73bcb2b..012e5c9 100644
--- a/lib/vdsm/config.py.in
+++ b/lib/vdsm/config.py.in
@@ -122,6 +122,9 @@
 ('max_outgoing_migrations', '3',
 'Maximum concurrent outgoing migrations'),
 
+('max_incoming_migrations', '3',
+'Maximum concurrent incoming migrations'),
+
 ('sys_shutdown_timeout', '120',
 'Destroy and shutdown timeouts (in sec) before completing the '
 'action.'),
diff --git a/lib/vdsm/define.py b/lib/vdsm/define.py
index 0943939..aee6f09 100644
--- a/lib/vdsm/define.py
+++ b/lib/vdsm/define.py
@@ -193,6 +193,9 @@
 'destVolumeTooSmall': {'status': {
 'code': 79,
 'message': 'Destination volume is too small'}},
+'migrationLimit': {'status': {
+'code': 80,
+'message': 'Incoming migration limit exceeded'}},
 'recovery': {'status': {
 'code': 99,
 'message': 'Recovering from crash or Initializing'}},
diff --git a/vdsm/API.py b/vdsm/API.py
index 9936d1e..739ddd2 100644
--- a/vdsm/API.py
+++ b/vdsm/API.py
@@ -21,6 +21,7 @@
 # pylint: disable=R0904
 
 from contextlib import contextmanager
+import functools
 import os
 import signal
 import six
@@ -44,8 +45,10 @@
 import storage.image
 from virt import hoststats
 from virt import sampling
+from virt import migration
 from virt import vmstatus
 from virt import secret
+from virt import utils as virtutils
 from virt.vmdevices import graphics
 from virt.vmdevices import hwclass
 from vdsm.compat import pickle
@@ -174,7 +177,7 @@
 return errCode['noVM']
 return v.cont()
 
-def create(self, vmParams):
+def create(self, vmParams, spawn=virtutils.run_async):
 """
 Start up a virtual machine.
 
@@ -264,7 +267,7 @@
   vmParams.get('display')}}
 if 'nicModel' not in vmParams:
 vmParams['nicModel'] = config.get('vars', 'nic_model')
-return self._cif.createVm(vmParams)
+return self._cif.createVm(vmParams, spawn=spawn)
 
 except OSError as e:
 self.log.debug("OS Error creating VM", exc_info=True)
@@ -576,8 +579,13 @@
 """
 self.log.debug('Migration create')
 
+spawn = functools.partial(
+virtutils.run_async,
+sempahore=migration.incomingMigrations,
+error='migrationLimit',
+)
 params['vmId'] = self._UUID
-result = self.create(params)
+result = self.create(params, spawn)
 if result['status']['code']:
 self.log.debug('Migration create - Failed')
 return result
diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py
index fc99592..2c65ff9 100644
--- a/vdsm/clientIF.py
+++ b/vdsm/clientIF.py
@@ -54,7 +54,7 @@
 from virt.vm import Vm
 from virt.vmchannels import Listener
 from virt.vmdevices import hwclass
-from virt.utils import isVdsmImage
+from virt.utils import isVdsmImage, run_async
 try:
 import gluster.api as gapi
 _glusterEnabled = True
@@ -427,14 +427,14 @@
 
 return {'status': doneCode, 'alignment': aligning}
 
-def createVm(self, vmParams, vmRecover=False):
+def createVm(self, vmParams, vmRecover=False, spawn=run_async):
 with self.vmContainerLock:
 if not vmRecover:
 if vmParams['vmId'] in self.vmContainer:
 return errCode['exist']
 vm = Vm(self, vmParams, vmRecover)
 self.vmContainer[vmParams['vmId']] = vm
-vm.run()
+vm.run(run_async)
 return {'status': doneCode, 'vmList': vm.status()}
 
 def getAllVmStats(self):
diff --git a/vdsm/virt/migration.py b/vdsm/virt/migration.py
index 7d8b374..8a8b06b 100644
--- a/vdsm/virt/migration.py
+++ b/vdsm/virt/migration.py
@@ -22,6 +22,7 @@
 import time
 import libvirt
 
+import caps
 import hooks
 import kaxmlrpclib
 from vdsm import response
@@ -52,6 +53,10 @@
 VIR_MIGRATE_PARAM_BANDWIDTH = 'bandwidth'
 VIR_MIGRATE_PARAM_GRAPHICS_URI = 'graphics_uri'
 
+incomingMigrations = threading.BoundedSemaphore(
+min(config.getint('vars', 'max_incoming_migrations'),
+caps.CpuTopology().cores()))
+
 
 class 

Change in vdsm[master]: DONTMERGE virt: use "run_async" helper

[automation]

gerrit-hooks has posted comments on this change.

Change subject: DONTMERGE virt: use "run_async" helper
..


Patch Set 1:

* Update tracker: IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 
'ovirt-3.5', 'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/49640
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3147b6e860b9839b656257fcc22bf6f4b3c58914
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani 
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches