Nir Soffer has submitted this change and it was merged. Change subject: virt: Limit the number of workers in executor ......................................................................
virt: Limit the number of workers in executor In some situations, periodic operations may hang inside executor worker threads. We have observed that in real situations when libvirt calls started being blocked due to some bug. Such situations shouldn't happen when everything works correctly, but actually not everything works correctly and there are unexpected bugs. We should try to minimize the possible damages. The result of the hanging periodic calls is that the number of worker threads in the executor grows above all limits. This is not good. So we limit the maximum number of the worker threads in the executor with this patch. We set the maximum number of the threads to a value sufficiently high for harmless temporary problems. If the number of the workers is higher than that then there is probably a real problem and we should start emergency actions, i.e. not spawning further threads in this case. Note that a single isolated problem of this kind may block all the periodic operations. We introduce a bit more fine grained approach to handle that in a followup patch. Change-Id: Iba56d91474c6b14a1cfe2db827b6fd61843a1db2 Signed-off-by: Milan Zamazal <mzama...@redhat.com> Backport-To: 4.0 Backport-To: 3.6 Reviewed-on: https://gerrit.ovirt.org/57754 Reviewed-by: Francesco Romani <from...@redhat.com> Reviewed-by: Nir Soffer <nsof...@redhat.com> Continuous-Integration: Jenkins CI --- M lib/vdsm/config.py.in M lib/vdsm/executor.py M lib/vdsm/virt/periodic.py M tests/executorTests.py 4 files changed, 154 insertions(+), 8 deletions(-) Approvals: Nir Soffer: Looks good to me, approved Jenkins CI: Passed CI tests Francesco Romani: Looks good to me, approved Milan Zamazal: Verified -- To view, visit https://gerrit.ovirt.org/57754 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iba56d91474c6b14a1cfe2db827b6fd61843a1db2 Gerrit-PatchSet: 22 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Milan Zamazal <mzama...@redhat.com> Gerrit-Reviewer: Francesco Romani <from...@redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Martin Polednik <mpoled...@redhat.com> Gerrit-Reviewer: Milan Zamazal <mzama...@redhat.com> Gerrit-Reviewer: Nir Soffer <nsof...@redhat.com> Gerrit-Reviewer: gerrit-hooks <automat...@ovirt.org> _______________________________________________ vdsm-patches mailing list vdsm-patches@lists.fedorahosted.org https://lists.fedorahosted.org/admin/lists/vdsm-patches@lists.fedorahosted.org
