Repository: incubator-samza Updated Branches: refs/heads/master f4876e35e -> b934aa873
SAMZA-415: Add a tutorial for running Samza from CDH Project: http://git-wip-us.apache.org/repos/asf/incubator-samza/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-samza/commit/b934aa87 Tree: http://git-wip-us.apache.org/repos/asf/incubator-samza/tree/b934aa87 Diff: http://git-wip-us.apache.org/repos/asf/incubator-samza/diff/b934aa87 Branch: refs/heads/master Commit: b934aa8738bd72f73518fbd595767426ca33e307 Parents: f4876e3 Author: Yan Fang <[email protected]> Authored: Tue Sep 30 00:08:23 2014 -0700 Committer: Yan Fang <[email protected]> Committed: Tue Sep 30 00:08:23 2014 -0700 ---------------------------------------------------------------------- .../tutorials/versioned/deploy-samza-to-CDH.md | 62 ++++++++++++++++++++ docs/learn/tutorials/versioned/index.md | 2 + 2 files changed, 64 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-samza/blob/b934aa87/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md b/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md new file mode 100644 index 0000000..8c19150 --- /dev/null +++ b/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md @@ -0,0 +1,62 @@ +--- +layout: page +title: Deploy Samza Job To CDH +--- +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +The tutorial assumes you have successfully run [hello-samza](../../../startup/hello-samza/{{site.version}}/) and now you want to deploy the job to your Cloudera Data Hub ([CDH](http://www.cloudera.com/content/cloudera/en/products-and-services/cdh.html)). This tutorial is based on CDH 5.0.0 and uses hello-samza as the example job. + +### Upload Package to Cluster + +There are a few ways of uploading the package to the cluster's HDFS. If you do not have the job package in your cluster, **scp** from you local machine to the cluster. Then run + +{% highlight bash %} +hadoop fs -put path/to/hello-samza-0.8.0-dist.tar.gz /path/for/tgz +{% endhighlight %} + +### Get Deloying Scripts + +Untar the job package (assume you will run from the current directory) + +{% highlight bash %} +tar -xvf path/to/samza-job-package-0.8.0-dist.tar.gz -C ./ +{% endhighlight %} + +### Add Package Path to Properties File + +{% highlight bash %} +vim config/wikipedia-parser.properties +{% endhighlight %} + +Change the yarn package path: + +{% highlight jproperties %} +yarn.package.path=hdfs://<hdfs name node ip>:<hdfs name node port>/path/to/tgz +{% endhighlight %} + +### Set Yarn Environment Variable + +{% highlight bash %} +export HADOOP_CONF_DIR=/etc/hadoop/conf +{% endhighlight %} + +### Run Samza Job + +{% highlight bash %} +bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/config/wikipedia-parser.properties +{% endhighlight %} http://git-wip-us.apache.org/repos/asf/incubator-samza/blob/b934aa87/docs/learn/tutorials/versioned/index.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/index.md b/docs/learn/tutorials/versioned/index.md index 91bddc5..b4d687a 100644 --- a/docs/learn/tutorials/versioned/index.md +++ b/docs/learn/tutorials/versioned/index.md @@ -23,6 +23,8 @@ title: Tutorials [Deploying a Samza Job from HDFS](deploy-samza-job-from-hdfs.html) +[Deoloy Samza to CDH](deploy-samza-to-CDH.html) + [Run Hello-samza in Multi-node YARN](run-in-multi-node-yarn.html) [Run Hello-samza without Internet](run-hello-samza-without-internet.html)
