EBernhardson has uploaded a new change for review.
https://gerrit.wikimedia.org/r/270339
Change subject: Increase materialization timeout for transfer_to_es
......................................................................
Increase materialization timeout for transfer_to_es
It seems we can expect to wait 3 to 4 hours for the initial
oozie job to submit and complete. This import takes so long
it will run during the busy part of the day anyways. As a
once a week job it seems ok to allow this to wait up to 24
hours for the full data inputs before giving up.
Change-Id: I1c6655ffa580ff787382ad92f217fbcec761182f
---
M oozie/transfer_to_es/coordinator.xml
1 file changed, 3 insertions(+), 5 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/wikimedia/discovery/analytics
refs/changes/39/270339/1
diff --git a/oozie/transfer_to_es/coordinator.xml
b/oozie/transfer_to_es/coordinator.xml
index 230f7f9..390184d 100644
--- a/oozie/transfer_to_es/coordinator.xml
+++ b/oozie/transfer_to_es/coordinator.xml
@@ -38,12 +38,10 @@
<controls>
<!--
- If a materialized job sits around for too long we could end up
- doing the export during the busiest part of the day for the ES
- servers. Rather than risking an overload of the servers timeout
- the job after two hours.
+ This job only runs once a week, we can be patient and wait up to
+ 24 hours for the data to be ready.
-->
- <timeout>120</timeout>
+ <timeout>1440</timeout>
<!--
transfering these documents can load up elasticsearch, additionally
--
To view, visit https://gerrit.wikimedia.org/r/270339
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1c6655ffa580ff787382ad92f217fbcec761182f
Gerrit-PatchSet: 1
Gerrit-Project: wikimedia/discovery/analytics
Gerrit-Branch: master
Gerrit-Owner: EBernhardson <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
