Ottomata has submitted this change and it was merged. Change subject: Now importing all webrequest data into a webrequest/ subdirectory ......................................................................
Now importing all webrequest data into a webrequest/ subdirectory This is so we can map a single hive table onto all webrequests Change-Id: I830a582aeac2c497be0a492cc3a3ae2c37c04d91 --- M kraken-etl/conf/camus.webrequest.properties 1 file changed, 2 insertions(+), 2 deletions(-) Approvals: Ottomata: Verified; Looks good to me, approved diff --git a/kraken-etl/conf/camus.webrequest.properties b/kraken-etl/conf/camus.webrequest.properties index 9472cb7..ee512bb 100644 --- a/kraken-etl/conf/camus.webrequest.properties +++ b/kraken-etl/conf/camus.webrequest.properties @@ -1,7 +1,7 @@ # Needed Camus properties, more cleanup to come # final top-level data output directory, sub-directory will be dynamically created for each topic pulled -etl.destination.path=hdfs://kraken/wmf/data/external +etl.destination.path=hdfs://kraken/wmf/data/external/webrequest # HDFS location where you want to keep execution files, i.e. offsets, error logs, and count files etl.execution.base.path=hdfs://kraken/wmf/camus/webrequest # where completed Camus job output directories are kept, usually a sub-dir in the base.path @@ -48,7 +48,7 @@ kafka.max.historical.days=7 # Max minutes for each mapper to pull messages (-1 means no limit) # Let each mapper run for no more than 55 minutes. -# Camus is scheduled to run hourly, and we don't want a single +# Camus creates hourly directories, and we don't want a single # long running mapper keep other Camus jobs from being launched. kafka.max.pull.minutes.per.task=55 -- To view, visit https://gerrit.wikimedia.org/r/133229 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I830a582aeac2c497be0a492cc3a3ae2c37c04d91 Gerrit-PatchSet: 2 Gerrit-Project: analytics/kraken Gerrit-Branch: master Gerrit-Owner: Ottomata <o...@wikimedia.org> Gerrit-Reviewer: Ottomata <o...@wikimedia.org> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
