Change in vdsm[master]: HACK: run GC in domain monitor

[automation]

gerrit-hooks has posted comments on this change.

Change subject: HACK: run GC in domain monitor
..


Patch Set 3:

* Update tracker: IGNORE, no Bug-Url found

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Jenkins CI RO
Gerrit-Reviewer: Nir Soffer 
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/admin/lists/vdsm-patches@lists.fedorahosted.org

Change in vdsm[master]: HACK: run GC in domain monitor

[Jenkins CI RO]

Jenkins CI RO has abandoned this change.

Change subject: HACK: run GC in domain monitor
..


Abandoned

Abandoned due to no activity - please restore if still relevant

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Jenkins CI RO
Gerrit-Reviewer: Nir Soffer 
Gerrit-Reviewer: gerrit-hooks 
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/admin/lists/vdsm-patches@lists.fedorahosted.org

Change in vdsm[master]: HACK: run GC in domain monitor

[alitke]

Adam Litke has restored this change.

Change subject: HACK: run GC in domain monitor
..


Restored

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: restore
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Jenkins CI RO
Gerrit-Reviewer: Nir Soffer 
Gerrit-Reviewer: gerrit-hooks 
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: HACK: run GC in domain monitor

[Jenkins CI RO]

Jenkins CI RO has abandoned this change.

Change subject: HACK: run GC in domain monitor
..


Abandoned

Abandoned due to no activity - please restore if still relevant

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Jenkins CI RO
Gerrit-Reviewer: Nir Soffer 
Gerrit-Reviewer: gerrit-hooks 
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: HACK: run GC in domain monitor

[automation]

gerrit-hooks has posted comments on this change.

Change subject: HACK: run GC in domain monitor
..


Patch Set 3:

* Update tracker: IGNORE, no Bug-Url found

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Jenkins CI RO
Gerrit-Reviewer: Nir Soffer 
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: HACK: run GC in domain monitor

[Jenkins CI RO]

Jenkins CI RO has posted comments on this change.

Change subject: HACK: run GC in domain monitor
..


Patch Set 3:

Abandoned due to no activity - please restore if still relevant

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke 
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Jenkins CI RO
Gerrit-Reviewer: Nir Soffer 
Gerrit-Reviewer: gerrit-hooks 
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: HACK: run GC in domain monitor

[automation]

automat...@ovirt.org has posted comments on this change.

Change subject: HACK: run GC in domain monitor
..


Patch Set 3:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke ali...@redhat.com
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer nsof...@redhat.com
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: HACK: run GC in domain monitor

[automation]

automat...@ovirt.org has posted comments on this change.

Change subject: HACK: run GC in domain monitor
..


Patch Set 2:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke ali...@redhat.com
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer nsof...@redhat.com
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: HACK: run GC in domain monitor

[nsoffer]

Nir Soffer has posted comments on this change.

Change subject: HACK: run GC in domain monitor
..


Patch Set 1:

(2 comments)

https://gerrit.ovirt.org/#/c/40380/1//COMMIT_MSG
Commit Message:

Line 4: Commit: Adam Litke ali...@redhat.com
Line 5: CommitDate: 2015-04-28 16:47:13 -0400
Line 6: 
Line 7: HACK: run GC in domain monitor
Line 8: 
I like this - this is not a hack! We have a thread dedicated to each storage 
domain, but we do not use it for most storage domain operations.
Line 9: Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376


https://gerrit.ovirt.org/#/c/40380/1/vdsm/storage/monitor.py
File vdsm/storage/monitor.py:

Line 249: 
Line 250: self._performDomainSelftest()
Line 251: self._checkReadDelay()
Line 252: self._collectStatistics()
Line 253: self._garbageCollect()
The domain monitor must wake up every 10 seconds for monitoring the storage, so 
this must be short operation that never blocks for long time, unless the 
blocking means that the domain is not accessible.

Collecting garbage here means also that notifying state changes, acquiring host 
id and setting next status will be delayed.

I think garbage collections should be integrated in the the monitor loop, after 
_monitorDomain returns, something like this:

deadline = monotonic_time()

while True:
deadline += interval
monitor domain
collect garbage before deadline
sleep until deadline

So gc is performed in the time the monitor used to sleep before, and cannot 
disturb reporting, unless the storage is stuck, which is a fine reason to 
disturb reporting.
Line 254: except Exception as e:
Line 255: self.log.exception(Error monitoring domain %s, 
self.sdUUID)
Line 256: self.nextStatus.error = e
Line 257: 


-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke ali...@redhat.com
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Nir Soffer nsof...@redhat.com
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-HasComments: Yes
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: HACK: run GC in domain monitor

[oVirt Jenkins CI Server]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: HACK: run GC in domain monitor
..


Patch Set 1: Verified-1

Build Failed 

http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18316/ : SUCCESS

http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1546/ : FAILURE

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke ali...@redhat.com
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: HACK: run GC in domain monitor

[oVirt Jenkins CI Server]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: HACK: run GC in domain monitor
..


Patch Set 1:

Build Started (1/2) - 
http://jenkins.ovirt.org/job/vdsm_master_pep8_gerrit/18316/

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke ali...@redhat.com
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: HACK: run GC in domain monitor

[alitke]

Adam Litke has uploaded a new change for review.

Change subject: HACK: run GC in domain monitor
..

HACK: run GC in domain monitor

Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Signed-off-by: Adam Litke ali...@redhat.com
---
M vdsm/storage/monitor.py
M vdsm/storage/sdm/__init__.py
2 files changed, 22 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/80/40380/1

diff --git a/vdsm/storage/monitor.py b/vdsm/storage/monitor.py
index ef032a5..a2a1bd7 100644
--- a/vdsm/storage/monitor.py
+++ b/vdsm/storage/monitor.py
@@ -28,6 +28,7 @@
 
 from . import clusterlock
 from . import misc
+from . import sdm
 from .sdc import sdCache
 
 
@@ -249,6 +250,7 @@
 self._performDomainSelftest()
 self._checkReadDelay()
 self._collectStatistics()
+self._garbageCollect()
 except Exception as e:
 self.log.exception(Error monitoring domain %s, self.sdUUID)
 self.nextStatus.error = e
@@ -340,6 +342,14 @@
 self.nextStatus.isoPrefix = self.isoPrefix
 self.nextStatus.version = self.domain.getVersion()
 
+def _garbageCollect(self):
+if True:  # XXX: limit this to domain ver 4 or later
+try:
+sdm.garbageCollectStorageDomain(self.domain)
+except:
+self.log.exception(Garbage collection failed for domain %s,
+   self.domain.sdUUID)
+
 # Managing host id
 
 def _shouldAcquireHostId(self):
diff --git a/vdsm/storage/sdm/__init__.py b/vdsm/storage/sdm/__init__.py
index 1636e63..4a31332 100644
--- a/vdsm/storage/sdm/__init__.py
+++ b/vdsm/storage/sdm/__init__.py
@@ -226,3 +226,15 @@
 cls.isolateVolumes(domain, srcImgUUID, dstImgUUID, volumeList)
 finally:
 domain.releaseClusterLock()
+
+
+def garbageCollectStorageDomain(domain):
+if domain.isISO():
+return
+cls = __getStoreClass(domain)
+hostId = getDomainHostId(domain.sdUUID)
+domain.acquireClusterLock(hostId)
+try:
+cls.garbageCollectStorageDomain(domain)
+finally:
+domain.releaseClusterLock()


-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke ali...@redhat.com
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: HACK: run GC in domain monitor

[oVirt Jenkins CI Server]

oVirt Jenkins CI Server has posted comments on this change.

Change subject: HACK: run GC in domain monitor
..


Patch Set 1:

Build Started (2/2) - 
http://jenkins.ovirt.org/job/vdsm_master_unit-tests_created/1546/

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke ali...@redhat.com
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches

Change in vdsm[master]: HACK: run GC in domain monitor

[automation]

automat...@ovirt.org has posted comments on this change.

Change subject: HACK: run GC in domain monitor
..


Patch Set 1:

* Update tracker::IGNORE, no Bug-Url found
* Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' 
and is a valid url.
* Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.5', 
'ovirt-3.4', 'ovirt-3.3'])

-- 
To view, visit https://gerrit.ovirt.org/40380
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3c560e6fbccdf50b135cc9c90b23824ae04b0376
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke ali...@redhat.com
Gerrit-Reviewer: automat...@ovirt.org
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No
___
vdsm-patches mailing list
vdsm-patches@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches