Modified: incubator/samza/site/learn/tutorials/0.7.0/remote-debugging-samza.html URL: http://svn.apache.org/viewvc/incubator/samza/site/learn/tutorials/0.7.0/remote-debugging-samza.html?rev=1602533&r1=1602532&r2=1602533&view=diff ============================================================================== --- incubator/samza/site/learn/tutorials/0.7.0/remote-debugging-samza.html (original) +++ incubator/samza/site/learn/tutorials/0.7.0/remote-debugging-samza.html Fri Jun 13 22:21:06 2014 @@ -1,4 +1,20 @@ <!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> <html lang="en"> <head> <meta charset="utf-8"> @@ -70,11 +86,45 @@ </div> <div class="content"> - <h2>Remote Debugging with Samza</h2> + <!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +<h2>Remote Debugging with Samza</h2> + +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> -<p>Let's use Eclipse to attach a remote debugger to a Samza container. If you're an IntelliJ user, you'll have to fill in the blanks, but the process should be pretty similar. This tutorial assumes you've already run through the <a href="../../../startup/hello-samza/0.7.0/">Hello Samza</a> tutorial.</p> +<p>Let’s use Eclipse to attach a remote debugger to a Samza container. If you’re an IntelliJ user, you’ll have to fill in the blanks, but the process should be pretty similar. This tutorial assumes you’ve already run through the <a href="../../../startup/hello-samza/0.7.0/">Hello Samza</a> tutorial.</p> -<h3>Get the Code</h3> +<h3 id="toc_0">Get the Code</h3> <p>Start by checking out Samza, so we have access to the source.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">git clone http://git-wip-us.apache.org/repos/asf/incubator-samza.git @@ -82,32 +132,32 @@ <p>Next, grab hello-samza.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">git clone git://git.apache.org/incubator-samza-hello-samza.git </code></pre></div> -<h3>Setup the Environment</h3> +<h3 id="toc_1">Setup the Environment</h3> -<p>Now, let's setup the Eclipse project files.</p> +<p>Now, let’s setup the Eclipse project files.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">cd incubator-samza ./gradlew eclipse </code></pre></div> -<p>Let's also release Samza to Maven's local repository, so hello-samza has access to the JARs that it needs.</p> +<p>Let’s also release Samza to Maven’s local repository, so hello-samza has access to the JARs that it needs.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">./gradlew -PscalaVersion=2.9.2 clean publishToMavenLocal </code></pre></div> -<p>Next, open Eclipse, and import the Samza source code into your workspace: "File" > "Import" > "Existing Projects into Workspace" > "Browse". Select 'incubator-samza' folder, and hit 'finish'.</p> +<p>Next, open Eclipse, and import the Samza source code into your workspace: “File” > “Import” > “Existing Projects into Workspace” > “Browse”. Select ‘incubator-samza’ folder, and hit ‘finish’.</p> -<h3>Enable Remote Debugging</h3> +<h3 id="toc_2">Enable Remote Debugging</h3> <p>Now, go back to the hello-samza project, and edit ./samza-job-package/src/main/config/wikipedia-feed.properties to add the following line:</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">task.opts=-agentlib:jdwp=transport=dt_socket,address=localhost:9009,server=y,suspend=y </code></pre></div> -<p>The <a href="../../documentation/0.7.0/jobs/configuration-table.html">task.opts</a> configuration parameter is a way to override Java parameters at runtime for your Samza containers. In this example, we're setting the agentlib parameter to enable remote debugging on localhost, port 9009. In a more realistic environment, you might also set Java heap settings (-Xmx, -Xms, etc), as well as garbage collection and logging settings.</p> +<p>The <a href="../../documentation/0.7.0/jobs/configuration-table.html">task.opts</a> configuration parameter is a way to override Java parameters at runtime for your Samza containers. In this example, we’re setting the agentlib parameter to enable remote debugging on localhost, port 9009. In a more realistic environment, you might also set Java heap settings (-Xmx, -Xms, etc), as well as garbage collection and logging settings.</p> -<p><em>NOTE: If you're running multiple Samza containers on the same machine, there is a potential for port collisions. You must configure your task.opts to assign different ports for different Samza jobs. If a Samza job has more than one container (e.g. if you're using YARN with yarn.container.count=2), those containers must be run on different machines.</em></p> +<p><em>NOTE: If you’re running multiple Samza containers on the same machine, there is a potential for port collisions. You must configure your task.opts to assign different ports for different Samza jobs. If a Samza job has more than one container (e.g. if you’re using YARN with yarn.container.count=2), those containers must be run on different machines.</em></p> -<h3>Start the Grid</h3> +<h3 id="toc_3">Start the Grid</h3> -<p>Now that the Samza job has been setup to enable remote debugging when a Samza container starts, let's start the ZooKeeper, Kafka, and YARN.</p> +<p>Now that the Samza job has been setup to enable remote debugging when a Samza container starts, let’s start the ZooKeeper, Kafka, and YARN.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">bin/grid </code></pre></div> -<p>If you get a complaint that JAVA_HOME is not set, then you'll need to set it. This can be done on OSX by running:</p> +<p>If you get a complaint that JAVA_HOME is not set, then you’ll need to set it. This can be done on OSX by running:</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">export JAVA_HOME=$(/usr/libexec/java_home) </code></pre></div> <p>Once the grid starts, you can start the wikipedia-feed Samza job.</p> @@ -116,17 +166,17 @@ mkdir -p deploy/samza tar -xvf ./samza-job-package/target/samza-job-package-0.7.0-dist.tar.gz -C deploy/samza deploy/samza/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties </code></pre></div> -<p>When the wikipedia-feed job starts up, a single Samza container will be created to process all incoming messages. This is the container that we'll want to connect to from the remote debugger.</p> +<p>When the wikipedia-feed job starts up, a single Samza container will be created to process all incoming messages. This is the container that we’ll want to connect to from the remote debugger.</p> -<h3>Connect the Remote Debugger</h3> +<h3 id="toc_4">Connect the Remote Debugger</h3> -<p>Switch back to Eclipse, and set a break point in TaskInstance.process by clicking on a line inside TaskInstance.process, and clicking "Run" > "Toggle Breakpoint". A blue circle should appear to the left of the line. This will let you see incoming messages as they arrive.</p> +<p>Switch back to Eclipse, and set a break point in TaskInstance.process by clicking on a line inside TaskInstance.process, and clicking “Run” > “Toggle Breakpoint”. A blue circle should appear to the left of the line. This will let you see incoming messages as they arrive.</p> -<p>Setup a remote debugging session: "Run" > "Debug Configurations..." > right click on "Remote Java Application" > "New". Set the name to 'wikipedia-feed-debug'. Set the port to 9009 (matching the port in the task.opts configuration). Click "Source" > "Add..." > "Java Project". Select all of the Samza projects that you imported (i.e. samza-api, samza-core, etc). If you would like to set breakpoints in your own Stream task, also add the project that contains your StreamTask implementation. Click 'Debug'.</p> +<p>Setup a remote debugging session: “Run” > “Debug Configurations…” > right click on “Remote Java Application” > “New”. Set the name to ‘wikipedia-feed-debug’. Set the port to 9009 (matching the port in the task.opts configuration). Click “Source” > “Add…” > “Java Project”. Select all of the Samza projects that you imported (i.e. samza-api, samza-core, etc). If you would like to set breakpoints in your own Stream task, also add the project that contains your StreamTask implementation. Click ‘Debug’.</p> -<p>After a few moments, Eclipse should connect to the wikipedia-feed job, and ask you to switch to Debug mode. Once in debug, you'll see that it's broken at the TaskInstance.process method. From here, you can step through code, inspect variable values, etc.</p> +<p>After a few moments, Eclipse should connect to the wikipedia-feed job, and ask you to switch to Debug mode. Once in debug, you’ll see that it’s broken at the TaskInstance.process method. From here, you can step through code, inspect variable values, etc.</p> -<p>Congratulations, you've got a remote debug connection to your StreamTask!</p> +<p>Congratulations, you’ve got a remote debug connection to your StreamTask!</p> </div>
Modified: incubator/samza/site/learn/tutorials/0.7.0/run-hello-samza-without-internet.html URL: http://svn.apache.org/viewvc/incubator/samza/site/learn/tutorials/0.7.0/run-hello-samza-without-internet.html?rev=1602533&r1=1602532&r2=1602533&view=diff ============================================================================== --- incubator/samza/site/learn/tutorials/0.7.0/run-hello-samza-without-internet.html (original) +++ incubator/samza/site/learn/tutorials/0.7.0/run-hello-samza-without-internet.html Fri Jun 13 22:21:06 2014 @@ -1,4 +1,20 @@ <!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> <html lang="en"> <head> <meta charset="utf-8"> @@ -70,11 +86,45 @@ </div> <div class="content"> - <h2>Run Hello Samza without Internet</h2> + <!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +<h2>Run Hello Samza without Internet</h2> + +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> <p>This tutorial is to help you run <a href="../../../startup/hello-samza/0.7.0/">Hello Samza</a> if you can not connect to the internet. </p> -<h3>Test Your Connection</h3> +<h3 id="toc_0">Test Your Connection</h3> <p>Ping irc.wikimedia.org. Sometimes the firewall in your company blocks this service.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">telnet irc.wikimedia.org 6667 @@ -90,7 +140,7 @@ NOTICE AUTH :*** Found your hostname </code></pre></div> <p>Otherwise, you may have the connection problem.</p> -<h3>Use Local Data to Run Hello Samza</h3> +<h3 id="toc_1">Use Local Data to Run Hello Samza</h3> <p>We provide an alternative to get wikipedia feed data. Instead of running</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">deploy/samza/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties @@ -105,7 +155,7 @@ bin/produce-wikipedia-raw-data.sh </code></pre></div> <p>Now you can go back to Generate Wikipedia Statistics section in <a href="../../../startup/hello-samza/0.7.0/">Hello Samza</a> and follow the remaining steps.</p> -<h3>A Little Explanation</h3> +<h3 id="toc_2">A Little Explanation</h3> <p>The goal of </p> <div class="highlight"><pre><code class="text language-text" data-lang="text">deploy/samza/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties Modified: incubator/samza/site/learn/tutorials/0.7.0/run-in-multi-node-yarn.html URL: http://svn.apache.org/viewvc/incubator/samza/site/learn/tutorials/0.7.0/run-in-multi-node-yarn.html?rev=1602533&r1=1602532&r2=1602533&view=diff ============================================================================== --- incubator/samza/site/learn/tutorials/0.7.0/run-in-multi-node-yarn.html (original) +++ incubator/samza/site/learn/tutorials/0.7.0/run-in-multi-node-yarn.html Fri Jun 13 22:21:06 2014 @@ -1,4 +1,20 @@ <!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> <html lang="en"> <head> <meta charset="utf-8"> @@ -70,15 +86,49 @@ </div> <div class="content"> - <h2>Run Hello-samza in Multi-node YARN</h2> + <!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +<h2>Run Hello-samza in Multi-node YARN</h2> + +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> -<p>You must successfully run the <a href="../../../startup/hello-samza/0.7.0/">hello-samza</a> project in a single-node YARN by following the <a href="../../../startup/hello-samza/0.7.0/">hello-samza</a> tutorial. Now it's time to run the Samza job in a "real" YARN grid (with more than one node).</p> +<p>You must successfully run the <a href="../../../startup/hello-samza/0.7.0/">hello-samza</a> project in a single-node YARN by following the <a href="../../../startup/hello-samza/0.7.0/">hello-samza</a> tutorial. Now it’s time to run the Samza job in a “real” YARN grid (with more than one node).</p> -<h2>Set Up Multi-node YARN</h2> +<h2 id="toc_0">Set Up Multi-node YARN</h2> <p>If you already have a multi-node YARN cluster (such as CDH5 cluster), you can skip this set-up section.</p> -<h3>Basic YARN Setting</h3> +<h3 id="toc_1">Basic YARN Setting</h3> <p>1. Dowload <a href="http://mirror.symnds.com/software/Apache/hadoop/common/hadoop-2.3.0/hadoop-2.3.0.tar.gz";>YARN 2.3</a> to /tmp and untar it.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">cd /tmp @@ -104,7 +154,7 @@ vi conf/yarn-site.xml <p>Download and add capacity-schedule.xml.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">curl http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/capacity-scheduler.xml?view=co > conf/capacity-scheduler.xml </code></pre></div> -<h3>Set Up Http Filesystem for YARN</h3> +<h3 id="toc_2">Set Up Http Filesystem for YARN</h3> <p>The goal of these steps is to configure YARN to read http filesystem because we will use Http server to deploy Samza job package. If you want to use HDFS to deploy Samza job package, you can skip step 4~6 and follow <a href="deploy-samza-job-from-hdfs.html">Deploying a Samza Job from HDFS</a></p> @@ -130,7 +180,7 @@ curl http://search.maven.org/remoteconte </property> </configuration> </code></pre></div> -<h3>Distribute Hadoop File to Slaves</h3> +<h3 id="toc_3">Distribute Hadoop File to Slaves</h3> <p>7. Basically, you copy the hadoop file in your host machine to slave machines. (172.21.100.35, in my case):</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">scp -r . 172.21.100.35:/tmp/hadoop-2.3.0 @@ -138,12 +188,12 @@ echo 172.21.100.35 > conf/slaves sbin/start-yarn.sh </code></pre></div> <ul> -<li>If you get "172.21.100.35: Error: JAVA<em>HOME is not set and could not be found.", you'll need to add a conf/hadoop-env.sh file to the machine with the failure (172.21.100.35, in this case), which has "export JAVA</em>HOME=/export/apps/jdk/JDK-1<em>6</em>0<em>27" (or wherever your JAVA</em>HOME actually is).</li> +<li>If you get “172.21.100.35: Error: JAVA<em>HOME is not set and could not be found.”, you’ll need to add a conf/hadoop-env.sh file to the machine with the failure (172.21.100.35, in this case), which has “export JAVA</em>HOME=/export/apps/jdk/JDK-1<em>6</em>0<em>27” (or wherever your JAVA</em>HOME actually is).</li> </ul> <p>8. Validate that your nodes are up by visiting http://yourHostname:8088/cluster/nodes.</p> -<h2>Deploy Samza Job</h2> +<h2 id="toc_4">Deploy Samza Job</h2> <p>Some of the following steps are exactlly identical to what you have seen in <a href="../../../startup/hello-samza/0.7.0/">hello-samza</a>. You may skip them if you have already done so.</p> @@ -175,9 +225,9 @@ tar -xvf ./samza-job-package/target/samz <p>Go back to the original terminal (not the one running the HTTP server):</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">deploy/samza/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties </code></pre></div> -<p>Go to http://yourHostname:8088 and find the wikipedia-feed job. Click on the ApplicationMaster link to see that it's running.</p> +<p>Go to http://yourHostname:8088 and find the wikipedia-feed job. Click on the ApplicationMaster link to see that it’s running.</p> -<p>Congratulations! You now run the Samza job in a "real" YARN grid!</p> +<p>Congratulations! You now run the Samza job in a “real” YARN grid!</p> </div> Modified: incubator/samza/site/sitemap.xml URL: http://svn.apache.org/viewvc/incubator/samza/site/sitemap.xml?rev=1602533&r1=1602532&r2=1602533&view=diff ============================================================================== --- incubator/samza/site/sitemap.xml (original) +++ incubator/samza/site/sitemap.xml Fri Jun 13 22:21:06 2014 @@ -1,10 +1,26 @@ <?xml version="1.0" encoding="UTF-8"?> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> <urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd"; xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";> <url> <loc>http://samza.incubator.apache.org/</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> <changefreq>daily</changefreq> <priority>1.0</priority> </url> @@ -14,301 +30,308 @@ <url> <loc>http://samza.incubator.apache.org/community/committers.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/community/irc.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/community/mailing-lists.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/contribute/code.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/contribute/coding-guide.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/contribute/disclaimer.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/contribute/projects.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/contribute/rules.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/contribute/seps.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/index.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/api/overview.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/comparisons/introduction.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/comparisons/mupd8.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/comparisons/storm.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/container/checkpointing.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/container/event-loop.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/container/jmx.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/container/metrics.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/container/samza-container.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/container/serialization.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/container/state-management.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/container/streams.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/container/windowing.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/index.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/introduction/architecture.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/introduction/background.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/introduction/concepts.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/jobs/configuration.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/jobs/job-runner.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/jobs/logging.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/jobs/packaging.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> + + + </url> + + <url> + <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/jobs/reprocessing.html</loc> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/jobs/yarn-jobs.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/operations/kafka.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/operations/security.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/yarn/application-master.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/documentation/0.7.0/yarn/isolation.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/tutorials/0.7.0/deploy-samza-job-from-hdfs.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/tutorials/0.7.0/index.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/tutorials/0.7.0/remote-debugging-samza.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/tutorials/0.7.0/run-hello-samza-without-internet.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/learn/tutorials/0.7.0/run-in-multi-node-yarn.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/startup/download/index.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> <url> <loc>http://samza.incubator.apache.org/startup/hello-samza/0.7.0/index.html</loc> - <lastmod>2014-06-09</lastmod> + <lastmod>2014-06-13</lastmod> </url> Modified: incubator/samza/site/startup/download/index.html URL: http://svn.apache.org/viewvc/incubator/samza/site/startup/download/index.html?rev=1602533&r1=1602532&r2=1602533&view=diff ============================================================================== --- incubator/samza/site/startup/download/index.html (original) +++ incubator/samza/site/startup/download/index.html Fri Jun 13 22:21:06 2014 @@ -1,4 +1,20 @@ <!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> <html lang="en"> <head> <meta charset="utf-8"> @@ -70,7 +86,24 @@ </div> <div class="content"> - <h2>Download</h2> + <!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +<h2>Download</h2> <!-- TODO update maven dependency versions appropriately --> @@ -131,11 +164,11 @@ Snapshot builds are available in the Apa <id>apache-snapshots</id> <url>https://repository.apache.org/content/groups/snapshots</url> </repository> ---> +–> -<h3>Checking out and Building</h3> +<h3 id="toc_0">Checking out and Building</h3> -<p>If you're interested in working on Samza, or building the JARs from scratch, then you'll need to checkout and build the code. Samza does not have a binary release at this time. To check out and build Samza, run these commands.</p> +<p>If you’re interested in working on Samza, or building the JARs from scratch, then you’ll need to checkout and build the code. Samza does not have a binary release at this time. To check out and build Samza, run these commands.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">git clone http://git-wip-us.apache.org/repos/asf/incubator-samza.git cd incubator-samza ./gradlew clean build Modified: incubator/samza/site/startup/hello-samza/0.7.0/index.html URL: http://svn.apache.org/viewvc/incubator/samza/site/startup/hello-samza/0.7.0/index.html?rev=1602533&r1=1602532&r2=1602533&view=diff ============================================================================== --- incubator/samza/site/startup/hello-samza/0.7.0/index.html (original) +++ incubator/samza/site/startup/hello-samza/0.7.0/index.html Fri Jun 13 22:21:06 2014 @@ -1,4 +1,20 @@ <!DOCTYPE html> +<!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> <html lang="en"> <head> <meta charset="utf-8"> @@ -70,51 +86,68 @@ </div> <div class="content"> - <h2>Hello Samza</h2> + <!-- + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--> + +<h2>Hello Samza</h2> <p>The <a href="https://github.com/apache/incubator-samza-hello-samza";>hello-samza</a> project is a stand-alone project designed to help you run your first Samza job.</p> -<h3>Get the Code</h3> +<h3 id="toc_0">Get the Code</h3> <p>Check out the hello-samza project:</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">git clone git://git.apache.org/incubator-samza-hello-samza.git hello-samza cd hello-samza </code></pre></div> -<p>This project contains everything you'll need to run your first Samza jobs.</p> +<p>This project contains everything you’ll need to run your first Samza jobs.</p> -<h3>Start a Grid</h3> +<h3 id="toc_1">Start a Grid</h3> -<p>A Samza grid usually comprises three different systems: <a href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html";>YARN</a>, <a href="http://kafka.apache.org/";>Kafka</a>, and <a href="http://zookeeper.apache.org/";>ZooKeeper</a>. The hello-samza project comes with a script called "grid" to help you setup these systems. Start by running:</p> +<p>A Samza grid usually comprises three different systems: <a href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html";>YARN</a>, <a href="http://kafka.apache.org/";>Kafka</a>, and <a href="http://zookeeper.apache.org/";>ZooKeeper</a>. The hello-samza project comes with a script called “grid” to help you setup these systems. Start by running:</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">bin/grid bootstrap </code></pre></div> -<p>This command will download, install, and start ZooKeeper, Kafka, and YARN. It will also check out the latest version of Samza and build it. All package files will be put in a sub-directory called "deploy" inside hello-samza's root folder.</p> +<p>This command will download, install, and start ZooKeeper, Kafka, and YARN. It will also check out the latest version of Samza and build it. All package files will be put in a sub-directory called “deploy” inside hello-samza’s root folder.</p> -<p>If you get a complaint that JAVA_HOME is not set, then you'll need to set it to the path where Java is installed on your system.</p> +<p>If you get a complaint that JAVA_HOME is not set, then you’ll need to set it to the path where Java is installed on your system.</p> <p>Once the grid command completes, you can verify that YARN is up and running by going to <a href="http://localhost:8088";>http://localhost:8088</a>. This is the YARN UI.</p> -<h3>Build a Samza Job Package</h3> +<h3 id="toc_2">Build a Samza Job Package</h3> <p>Before you can run a Samza job, you need to build a package for it. This package is what YARN uses to deploy your jobs on the grid.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">mvn clean package mkdir -p deploy/samza tar -xvf ./samza-job-package/target/samza-job-package-0.7.0-dist.tar.gz -C deploy/samza </code></pre></div> -<h3>Run a Samza Job</h3> +<h3 id="toc_3">Run a Samza Job</h3> -<p>After you've built your Samza package, you can start a job on the grid using the run-job.sh script.</p> +<p>After you’ve built your Samza package, you can start a job on the grid using the run-job.sh script.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">deploy/samza/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties </code></pre></div> -<p>The job will consume a feed of real-time edits from Wikipedia, and produce them to a Kafka topic called "wikipedia-raw". Give the job a minute to startup, and then tail the Kafka topic:</p> +<p>The job will consume a feed of real-time edits from Wikipedia, and produce them to a Kafka topic called “wikipedia-raw”. Give the job a minute to startup, and then tail the Kafka topic:</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">deploy/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wikipedia-raw </code></pre></div> -<p>Pretty neat, right? Now, check out the YARN UI again (<a href="http://localhost:8088";>http://localhost:8088</a>). This time around, you'll see your Samza job is running!</p> +<p>Pretty neat, right? Now, check out the YARN UI again (<a href="http://localhost:8088";>http://localhost:8088</a>). This time around, you’ll see your Samza job is running!</p> <p>If you can not see any output from Kafka consumer, you may have connection problem. Check <a href="../../../learn/tutorials/0.7.0/run-hello-samza-without-internet.html">here</a>.</p> -<h3>Generate Wikipedia Statistics</h3> +<h3 id="toc_4">Generate Wikipedia Statistics</h3> -<p>Let's calculate some statistics based on the messages in the wikipedia-raw topic. Start two more jobs:</p> +<p>Let’s calculate some statistics based on the messages in the wikipedia-raw topic. Start two more jobs:</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">deploy/samza/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-parser.properties deploy/samza/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-stats.properties </code></pre></div> @@ -130,14 +163,14 @@ deploy/samza/bin/run-job.sh --config-fac {"bytes-added":3180,"edits":19,"unique-titles":19,"is-unpatrolled":1,"is-new":1,"is-minor":3} {"bytes-added":2218,"edits":18,"unique-titles":18,"is-unpatrolled":2,"is-new":2,"is-minor":3} </code></pre></div> -<p>If you check the YARN UI, again, you'll see that all three jobs are now listed.</p> +<p>If you check the YARN UI, again, you’ll see that all three jobs are now listed.</p> -<h3>Shutdown</h3> +<h3 id="toc_5">Shutdown</h3> -<p>After you're done, you can clean everything up using the same grid script.</p> +<p>After you’re done, you can clean everything up using the same grid script.</p> <div class="highlight"><pre><code class="text language-text" data-lang="text">bin/grid stop all </code></pre></div> -<p>Congratulations! You've now setup a local grid that includes YARN, Kafka, and ZooKeeper, and run a Samza job on it. Next up, check out the <a href="/learn/documentation/0.7.0/introduction/background.html">Background</a> and <a href="/learn/documentation/0.7.0/api/overview.html">API Overview</a> pages.</p> +<p>Congratulations! You’ve now setup a local grid that includes YARN, Kafka, and ZooKeeper, and run a Samza job on it. Next up, check out the <a href="/learn/documentation/0.7.0/introduction/background.html">Background</a> and <a href="/learn/documentation/0.7.0/api/overview.html">API Overview</a> pages.</p> </div>
