Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[mbetak]

Martin Betak has abandoned this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Abandoned

superceeded by I79ab97f15788e4024c94d051e4aade713d760acf

-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 8
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Martin Betak 
Gerrit-Reviewer: Martin Polednik 
Gerrit-Reviewer: Tomas Jelinek 
Gerrit-Reviewer: gerrit-hooks 
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[automation]

gerrit-hooks has posted comments on this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Patch Set 8:

* Update tracker: IGNORE, no Bug-Url found

-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 8
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Martin Betak 
Gerrit-Reviewer: Martin Polednik 
Gerrit-Reviewer: Tomas Jelinek 
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[automation]

automat...@ovirt.org has posted comments on this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Patch Set 8:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 8
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Martin Betak 
Gerrit-Reviewer: Martin Polednik 
Gerrit-Reviewer: Tomas Jelinek 
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[automation]

automat...@ovirt.org has posted comments on this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Patch Set 7:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 7
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Martin Polednik 
Gerrit-Reviewer: Tomas Jelinek 
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[mbetak]

Martin Betak has posted comments on this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Patch Set 6:

(1 comment)

https://gerrit.ovirt.org/#/c/47409/6/vdsm/virt/utils.py
File vdsm/virt/utils.py:

Line 160: 
Line 161: return decorator
Line 162: 
Line 163: 
Line 164: class DynamicSemaphore(threading._Semaphore):
> We learned the hard way from the cpopen saga that messing with python imple
another alternative would be to implement our own semaphore that would support 
this feature (as opposed to wrapping/inheriting from Python's)

please see patch set 7 for my implementation of DynamicBoundedSemaphore
Line 165: """
Line 166: Extends standard semaphore with the ability to alter the current 
"bound"
Line 167: by external actors.
Line 168: 


-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 6
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Martin Betak 
Gerrit-Reviewer: Martin Polednik 
Gerrit-Reviewer: Tomas Jelinek 
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: Yes
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[fromani]

Francesco Romani has posted comments on this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Patch Set 6:

(3 comments)

The whole "dynamic semaphore" concept is a nice hack, but I'm not convinced it 
is the best way forward.

https://gerrit.ovirt.org/#/c/47409/6/vdsm/API.py
File vdsm/API.py:

Line 1762: max_outgoing = params.get('maxOutgoingMigrations')
Line 1763: if max_outgoing is not None:
Line 1764: self.log('Updating maxOutgoingMigrations to %s',
Line 1765:  max_outgoing)
Line 1766: 
migration.SourceThread.ongoingMigrations.setValue(max_outgoing)
for another patch: let's move this sempahore in the module.
Line 1767: 
Line 1768: max_incoming = params.get('maxIncomingMigrations')
Line 1769: if max_incoming is not None:
Line 1770: self.log('Updating maxIncomingMigrations to %s',


Line 1768: max_incoming = params.get('maxIncomingMigrations')
Line 1769: if max_incoming is not None:
Line 1770: self.log('Updating maxIncomingMigrations to %s',
Line 1771:  max_incoming)
Line 1772: migration.incomingMigrations.setValue(max_incoming)
for another patch: once both semaphores are module variables, let's rename them

 - migration.incoming
 - migration.outgoing
Line 1773: 
Line 1774: return response.success()
Line 1775: except migration.MigrationConfigurationError:
Line 1776: return response.error('migrateChangeParamsErr')


https://gerrit.ovirt.org/#/c/47409/6/vdsm/virt/utils.py
File vdsm/virt/utils.py:

Line 160: 
Line 161: return decorator
Line 162: 
Line 163: 
Line 164: class DynamicSemaphore(threading._Semaphore):
We learned the hard way from the cpopen saga that messing with python 
implementation details is a bad idea.

Any other option to avoid this?
Line 165: """
Line 166: Extends standard semaphore with the ability to alter the current 
"bound"
Line 167: by external actors.
Line 168: 


-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 6
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Martin Polednik 
Gerrit-Reviewer: Tomas Jelinek 
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: Yes
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[automation]

automat...@ovirt.org has posted comments on this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Patch Set 5:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 5
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Martin Polednik 
Gerrit-Reviewer: Tomas Jelinek 
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[automation]

automat...@ovirt.org has posted comments on this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Patch Set 6:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 6
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Martin Polednik 
Gerrit-Reviewer: Tomas Jelinek 
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[automation]

automat...@ovirt.org has posted comments on this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Patch Set 3:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Martin Polednik 
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[automation]

automat...@ovirt.org has posted comments on this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Patch Set 4:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 4
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: Francesco Romani 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Martin Polednik 
Gerrit-Reviewer: Tomas Jelinek 
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[mbetak]

Martin Betak has uploaded a new change for review.

Change subject: migration: Add migrateChangeGlobalParams verb
..

migration: Add migrateChangeGlobalParams verb

Added new verb for setting values of incoming and outgoing migration semaphores.

Added a custom semaphore class that allows for dynamic changes to its value.

Wiki: http://www.ovirt.org/Features/Migration_Enhancements
Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Signed-off-by: Martin Betak 
---
M lib/vdsm/define.py
M vdsm/API.py
M vdsm/clientIF.py
M vdsm/virt/migration.py
M vdsm/virt/utils.py
5 files changed, 81 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/09/47409/1

diff --git a/lib/vdsm/define.py b/lib/vdsm/define.py
index aee6f09..5d47c6c 100644
--- a/lib/vdsm/define.py
+++ b/lib/vdsm/define.py
@@ -196,6 +196,9 @@
 'migrationLimit': {'status': {
 'code': 80,
 'message': 'Incoming migration limit exceeded'}},
+'migrateChangeParamsErr': {'status': {
+'code': 81,
+'message': 'Error setting migration parameters'}},
 'recovery': {'status': {
 'code': 99,
 'message': 'Recovering from crash or Initializing'}},
diff --git a/vdsm/API.py b/vdsm/API.py
index 8fc9413..b49708e 100644
--- a/vdsm/API.py
+++ b/vdsm/API.py
@@ -594,6 +594,32 @@
 return {'status': doneCode, 'migrationPort': 0,
 'params': result['vmList']}
 
+def migrateChangeGlobalParams(self, params):
+"""
+Change parameters that apply to all migrations.
+
+:param params: a dictionary containing:
+*max_outgoing_migrations* - maximum concurrent outgoing migrations
+"""
+
+try:
+max_outgoing = params.get('max_outgoing_migrations')
+if max_outgoing is not None:
+self.log('Updating max_outgoing_migrations to %s',
+ max_outgoing)
+migration.SourceThread.ongoingMigrations.setValue(max_outgoing)
+
+max_incoming = params.get('max_incoming_migrations')
+if max_incoming is not None:
+self.log('Updating max_incoming_migrations to %s',
+ max_incoming)
+migration.incomingMigrations.setValue(max_incoming)
+
+return response.success()
+except migration.MigrationConfigurationError:
+return response.error('migrateChangeParamsErr')
+
+
 def monitorCommand(self, command):
 """
 Send a monitor command to the specified VM and wait for the answer.
diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py
index aca99b2..4f2c164 100644
--- a/vdsm/clientIF.py
+++ b/vdsm/clientIF.py
@@ -464,7 +464,7 @@
 # API response.
 mog = min(config.getint('vars', 'max_outgoing_migrations'),
   caps.CpuTopology().cores())
-migration.SourceThread.setMaxOutgoingMigrations(mog)
+migration.SourceThread.ongoingMigrations.setValue(mog)
 
 # Recover stage 1: domains from libvirt
 doms = getVDSMDomains()
diff --git a/vdsm/virt/migration.py b/vdsm/virt/migration.py
index c8d87e5..b583107 100644
--- a/vdsm/virt/migration.py
+++ b/vdsm/virt/migration.py
@@ -33,6 +33,7 @@
 from vdsm.compat import pickle
 from vdsm.config import config
 from vdsm.define import NORMAL, Mbytes
+from virt.utils import DynamicSemaphore
 from yajsonrpc import \
 JsonRpcNoResponseError, \
 JsonRpcBindingsError
@@ -53,9 +54,15 @@
 VIR_MIGRATE_PARAM_BANDWIDTH = 'bandwidth'
 VIR_MIGRATE_PARAM_GRAPHICS_URI = 'graphics_uri'
 
-incomingMigrations = threading.BoundedSemaphore(
+incomingMigrations = DynamicSemaphore(
 min(config.getint('vars', 'max_incoming_migrations'),
 caps.CpuTopology().cores()))
+
+
+class MigrationConfigurationError(RuntimeError):
+"""
+Failed to set requested global migration option(s)
+"""
 
 
 class MigrationDestinationSetupError(RuntimeError):
@@ -68,14 +75,7 @@
 """
 A thread that takes care of migration on the source vdsm.
 """
-_ongoingMigrations = threading.BoundedSemaphore(1)
-
-@classmethod
-def setMaxOutgoingMigrations(cls, n):
-"""Set the initial value of the _ongoingMigrations semaphore.
-
-must not be called after any vm has been run."""
-cls._ongoingMigrations = threading.BoundedSemaphore(n)
+ongoingMigrations = DynamicSemaphore(1)
 
 def __init__(self, vm, dst='', dstparams='',
  mode=MODE_REMOTE, method=METHOD_ONLINE,
diff --git a/vdsm/virt/utils.py b/vdsm/virt/utils.py
index c4f47b5..29868c1 100644
--- a/vdsm/virt/utils.py
+++ b/vdsm/virt/utils.py
@@ -159,3 +159,45 @@
 return wrapper
 
 return decorator
+
+
+class DynamicSemaphore(threading.Semaphore):
+"""
+Extends standard semaphore with the ability to alter the current "bound"
+by 

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[automation]

automat...@ovirt.org has posted comments on this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Patch Set 1:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: migration: Add migrateChangeGlobalParams verb

[automation]

automat...@ovirt.org has posted comments on this change.

Change subject: migration: Add migrateChangeGlobalParams verb
..


Patch Set 2:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/47409
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I4f6d1bcdc29f144d9fcf28a085b7014127cc4f41
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Martin Betak 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches