[MediaWiki-commits] [Gerrit] Using separate job for eventlogging camus imports - change (operations/puppet)

[Ottomata (Code Review)]

Ottomata has submitted this change and it was merged.

Change subject: Using separate job for eventlogging camus imports
......................................................................


Using separate job for eventlogging camus imports

Change-Id: I42c09d5241ac41c39e7330e6bc535418062dc2cf
---
M manifests/role/analytics/kraken.pp
1 file changed, 12 insertions(+), 5 deletions(-)

Approvals:
  Ottomata: Verified; Looks good to me, approved
  jenkins-bot: Verified



diff --git a/manifests/role/analytics/kraken.pp 
b/manifests/role/analytics/kraken.pp
index c73d29e..54abd83 100644
--- a/manifests/role/analytics/kraken.pp
+++ b/manifests/role/analytics/kraken.pp
@@ -46,15 +46,22 @@
 }
 
 # == Class role::analytics::kraken::jobs::import::kafka
-# Submits Camus MapReduce job hourly to import data from Kafka.
+# Submits Camus MapReduce jobs to import data from Kafka.
 class role::analytics::kraken::jobs::import::kafka {
     require role::analytics::kraken
 
-    $camus_properties = 
"${::role::analytics::kraken::path}/kraken-etl/conf/camus.properties"
-    $camus_log_file   = "${::role::analytics::kraken::log_dir}/camus.log"
+    $camus_webrequest_properties = 
"${::role::analytics::kraken::path}/kraken-etl/conf/camus.webrequest.properties"
+    $camus_webrequest_log_file   = 
"${::role::analytics::kraken::log_dir}/camus-webrequest.log"
+    cron { 'kraken-import-hourly-webrequest':
+        command => "${::role::analytics::kraken::path}/kraken-etl/camus 
--job-name camus-webrequest-import ${camus_webrequest_properties} >> 
${camus_webrequest_log_file} 2>&1",
+        user    => 'hdfs',  # we might want to use a different user for this, 
not sure.
+        minute  => '*/10',
+    }
 
-    cron { 'kraken-import-hourly-kafka':
-        command => "${::role::analytics::kraken::path}/kraken-etl/camus 
--job-name camus-import ${camus_properties} >> ${camus_log_file} 2>&1",
+    $camus_eventlogging_properties = 
"${::role::analytics::kraken::path}/kraken-etl/conf/camus.eventlogging.properties"
+    $camus_eventlogging_log_file   = 
"${::role::analytics::kraken::log_dir}/camus-eventlogging.log"
+    cron { 'kraken-import-hourly-eventlogging':
+        command => "${::role::analytics::kraken::path}/kraken-etl/camus 
--job-name camus-eventlogging-import ${camus_eventlogging_properties} >> 
${camus_eventlogging_log_file} 2>&1",
         user    => 'hdfs',  # we might want to use a different user for this, 
not sure.
         minute  => '*/10',
     }

-- 
To view, visit https://gerrit.wikimedia.org/r/130347
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I42c09d5241ac41c39e7330e6bc535418062dc2cf
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ottomata <o...@wikimedia.org>
Gerrit-Reviewer: Ottomata <o...@wikimedia.org>
Gerrit-Reviewer: jenkins-bot <>

_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits