[MediaWiki-commits] [Gerrit] Decrease throttle for webrequest log verification - change (analytics/refinery)
Ottomata has submitted this change and it was merged. Change subject: Decrease throttle for webrequest log verification .. Decrease throttle for webrequest log verification The cluster default is unnecessarily high for this job, and only bought us jobs that unnecessarily check hdfs again and again. Change-Id: I254e96c4c92c3e842581570c54fc378e609747de --- M oozie/webrequest/partition/add/coordinator.xml 1 file changed, 11 insertions(+), 0 deletions(-) Approvals: Ottomata: Verified; Looks good to me, approved diff --git a/oozie/webrequest/partition/add/coordinator.xml b/oozie/webrequest/partition/add/coordinator.xml index db3577c..ab050ab 100644 --- a/oozie/webrequest/partition/add/coordinator.xml +++ b/oozie/webrequest/partition/add/coordinator.xml @@ -56,6 +56,17 @@ even-though the concurrency is low. -- concurrency2/concurrency + + +!-- +Since we expect only one incarnation per hourly dataset, the +default throttle of 12 is way to high, and there is not need +to keep that many materialized jobs around. + +By resorting to 2, we keep the hdfs checks on the datasets +low, while still being able to easily feed the concurrency. +-- +throttle2/throttle /controls datasets -- To view, visit https://gerrit.wikimedia.org/r/148700 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I254e96c4c92c3e842581570c54fc378e609747de Gerrit-PatchSet: 2 Gerrit-Project: analytics/refinery Gerrit-Branch: master Gerrit-Owner: QChris christ...@quelltextlich.at Gerrit-Reviewer: Ottomata o...@wikimedia.org Gerrit-Reviewer: QChris christ...@quelltextlich.at ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
[MediaWiki-commits] [Gerrit] Decrease throttle for webrequest log verification - change (analytics/refinery)
Hello Ottomata, I'd like you to do a code review. Please visit https://gerrit.wikimedia.org/r/148700 to review the following change. Change subject: Decrease throttle for webrequest log verification .. Decrease throttle for webrequest log verification The cluster default is unnecessarily high for this job, and only bought us jobs that unnecessarily check hdfs again and again. Change-Id: I254e96c4c92c3e842581570c54fc378e609747de --- M oozie/webrequest/partition/add/coordinator.xml 1 file changed, 11 insertions(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/analytics/refinery refs/changes/00/148700/1 diff --git a/oozie/webrequest/partition/add/coordinator.xml b/oozie/webrequest/partition/add/coordinator.xml index db3577c..ab050ab 100644 --- a/oozie/webrequest/partition/add/coordinator.xml +++ b/oozie/webrequest/partition/add/coordinator.xml @@ -56,6 +56,17 @@ even-though the concurrency is low. -- concurrency2/concurrency + + +!-- +Since we expect only one incarnation per hourly dataset, the +default throttle of 12 is way to high, and there is not need +to keep that many materialized jobs around. + +By resorting to 2, we keep the hdfs checks on the datasets +low, while still being able to easily feed the concurrency. +-- +throttle2/throttle /controls datasets -- To view, visit https://gerrit.wikimedia.org/r/148700 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I254e96c4c92c3e842581570c54fc378e609747de Gerrit-PatchSet: 1 Gerrit-Project: analytics/refinery Gerrit-Branch: master Gerrit-Owner: QChris christ...@quelltextlich.at Gerrit-Reviewer: Ottomata o...@wikimedia.org ___ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
