Modified: samza/site/learn/tutorials/latest/run-in-multi-node-yarn.html URL: http://svn.apache.org/viewvc/samza/site/learn/tutorials/latest/run-in-multi-node-yarn.html?rev=1832281&r1=1832280&r2=1832281&view=diff ============================================================================== --- samza/site/learn/tutorials/latest/run-in-multi-node-yarn.html (original) +++ samza/site/learn/tutorials/latest/run-in-multi-node-yarn.html Fri May 25 22:28:38 2018 @@ -44,9 +44,7 @@ <!-- this icon only shows in versioned pages --> - - - <a href="http://samza.apache.org/learn/tutorials/0.14/run-in-multi-node-yarn.html";><i id="switch-version-button"></i></a> + <a href="http://samza.apache.org/learn/tutorials/0.14/run-in-multi-node-yarn";><i id="switch-version-button"></i></a> <!-- links for the navigation bar --> @@ -157,31 +155,31 @@ <p>1. Download <a href="http://mirror.symnds.com/software/Apache/hadoop/common/hadoop-2.6.1/hadoop-2.6.1.tar.gz";>YARN 2.6</a> to /tmp and untar it.</p> -<div class="highlight"><pre><code class="bash"><span class="nb">cd</span> /tmp +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span><span class="nb">cd</span> /tmp tar -xvf hadoop-2.6.1.tar.gz -<span class="nb">cd </span>hadoop-2.6.1</code></pre></div> +<span class="nb">cd</span> hadoop-2.6.1</code></pre></figure> <p>2. Set up environment variables.</p> -<div class="highlight"><pre><code class="bash"><span class="nb">export </span><span class="nv">HADOOP_YARN_HOME</span><span class="o">=</span><span class="k">$(</span><span class="nb">pwd</span><span class="k">)</span> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span><span class="nb">export</span> <span class="nv">HADOOP_YARN_HOME</span><span class="o">=</span><span class="k">$(</span><span class="nb">pwd</span><span class="k">)</span> mkdir conf -<span class="nb">export </span><span class="nv">HADOOP_CONF_DIR</span><span class="o">=</span><span class="nv">$HADOOP_YARN_HOME</span>/conf</code></pre></div> +<span class="nb">export</span> <span class="nv">HADOOP_CONF_DIR</span><span class="o">=</span><span class="nv">$HADOOP_YARN_HOME</span>/conf</code></pre></figure> <p>3. Configure YARN setting file.</p> -<div class="highlight"><pre><code class="bash">cp ./etc/hadoop/yarn-site.xml conf -vi conf/yarn-site.xml</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>cp ./etc/hadoop/yarn-site.xml conf +vi conf/yarn-site.xml</code></pre></figure> <p>Add the following property to yarn-site.xml:</p> -<div class="highlight"><pre><code class="xml"><span class="nt"><property></span> +<figure class="highlight"><pre><code class="language-xml" data-lang="xml"><span></span><span class="nt"><property></span> <span class="nt"><name></span>yarn.resourcemanager.hostname<span class="nt"></name></span> <span class="c"><!-- hostname that is accessible from all NMs --></span> <span class="nt"><value></span>yourHostname<span class="nt"></value></span> -<span class="nt"></property></span></code></pre></div> +<span class="nt"></property></span></code></pre></figure> <p>Download and add capacity-schedule.xml.</p> -<div class="highlight"><pre><code class="language-text" data-lang="text">curl http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/capacity-scheduler.xml?view=co > conf/capacity-scheduler.xml +<div class="highlight"><pre><code class="language-text" data-lang="text"><span></span>curl http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/resources/capacity-scheduler.xml?view=co > conf/capacity-scheduler.xml </code></pre></div> <h3 id="set-up-http-filesystem-for-yarn">Set Up Http Filesystem for YARN</h3> @@ -189,39 +187,39 @@ vi conf/yarn-site.xml</code></pre></div> <p>4. Download Scala package and untar it.</p> -<div class="highlight"><pre><code class="bash"><span class="nb">cd</span> /tmp +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span><span class="nb">cd</span> /tmp curl http://www.scala-lang.org/files/archive/scala-2.11.8.tgz > scala-2.11.8.tgz -tar -xvf scala-2.11.8.tgz</code></pre></div> +tar -xvf scala-2.11.8.tgz</code></pre></figure> <p>5. Add Scala, its log jars, and Samza’s HttpFileSystem implementation.</p> -<div class="highlight"><pre><code class="bash">cp /tmp/scala-2.11.8/lib/scala-compiler.jar <span class="nv">$HADOOP_YARN_HOME</span>/share/hadoop/hdfs/lib +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>cp /tmp/scala-2.11.8/lib/scala-compiler.jar <span class="nv">$HADOOP_YARN_HOME</span>/share/hadoop/hdfs/lib cp /tmp/scala-2.11.8/lib/scala-library.jar <span class="nv">$HADOOP_YARN_HOME</span>/share/hadoop/hdfs/lib curl -L http://search.maven.org/remotecontent?filepath<span class="o">=</span>org/clapper/grizzled-slf4j_2.10/1.0.1/grizzled-slf4j_2.10-1.0.1.jar > <span class="nv">$HADOOP_YARN_HOME</span>/share/hadoop/hdfs/lib/grizzled-slf4j_2.10-1.0.1.jar curl -L http://search.maven.org/remotecontent?filepath<span class="o">=</span>org/apache/samza/samza-yarn_2.11/0.12.0/samza-yarn_2.11-0.12.0.jar > <span class="nv">$HADOOP_YARN_HOME</span>/share/hadoop/hdfs/lib/samza-yarn_2.11-0.12.0.jar -curl -L http://search.maven.org/remotecontent?filepath<span class="o">=</span>org/apache/samza/samza-core_2.11/0.12.0/samza-core_2.11-0.12.0.jar > <span class="nv">$HADOOP_YARN_HOME</span>/share/hadoop/hdfs/lib/samza-core_2.11-0.12.0.jar</code></pre></div> +curl -L http://search.maven.org/remotecontent?filepath<span class="o">=</span>org/apache/samza/samza-core_2.11/0.12.0/samza-core_2.11-0.12.0.jar > <span class="nv">$HADOOP_YARN_HOME</span>/share/hadoop/hdfs/lib/samza-core_2.11-0.12.0.jar</code></pre></figure> <p>6. Add http configuration in core-site.xml (create the core-site.xml file and add content).</p> -<div class="highlight"><pre><code class="xml">vi $HADOOP_YARN_HOME/conf/core-site.xml</code></pre></div> +<figure class="highlight"><pre><code class="language-xml" data-lang="xml"><span></span>vi $HADOOP_YARN_HOME/conf/core-site.xml</code></pre></figure> <p>Add the following code:</p> -<div class="highlight"><pre><code class="xml"><span class="cp"><?xml-stylesheet type="text/xsl" href="configuration.xsl"?></span> +<figure class="highlight"><pre><code class="language-xml" data-lang="xml"><span></span><span class="cp"><?xml-stylesheet type="text/xsl" href="configuration.xsl"?></span> <span class="nt"><configuration></span> <span class="nt"><property></span> <span class="nt"><name></span>fs.http.impl<span class="nt"></name></span> <span class="nt"><value></span>org.apache.samza.util.hadoop.HttpFileSystem<span class="nt"></value></span> <span class="nt"></property></span> -<span class="nt"></configuration></span></code></pre></div> +<span class="nt"></configuration></span></code></pre></figure> <h3 id="distribute-hadoop-file-to-slaves">Distribute Hadoop File to Slaves</h3> <p>7. Basically, you copy the hadoop file in your host machine to slave machines. (172.21.100.35, in my case):</p> -<div class="highlight"><pre><code class="bash">scp -r . 172.21.100.35:/tmp/hadoop-2.6.1 -<span class="nb">echo </span>172.21.100.35 > conf/slaves -sbin/start-yarn.sh</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>scp -r . <span class="m">172</span>.21.100.35:/tmp/hadoop-2.6.1 +<span class="nb">echo</span> <span class="m">172</span>.21.100.35 > conf/slaves +sbin/start-yarn.sh</code></pre></figure> <ul> <li>If you get “172.21.100.35: Error: JAVA<em>HOME is not set and could not be found.”, you’ll need to add a conf/hadoop-env.sh file to the machine with the failure (172.21.100.35, in this case), which has “export JAVA</em>HOME=/export/apps/jdk/JDK-1<em>8</em>0<em>45” (or wherever your JAVA</em>HOME actually is).</li> @@ -235,37 +233,37 @@ sbin/start-yarn.sh</code></pre></div> <p>1. Download Samza and publish it to Maven local repository.</p> -<div class="highlight"><pre><code class="bash"><span class="nb">cd</span> /tmp +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span><span class="nb">cd</span> /tmp git clone http://git-wip-us.apache.org/repos/asf/samza.git -<span class="nb">cd </span>samza +<span class="nb">cd</span> samza ./gradlew clean publishToMavenLocal -<span class="nb">cd</span> ..</code></pre></div> +<span class="nb">cd</span> ..</code></pre></figure> <p>2. Download hello-samza project and change the job properties file.</p> -<div class="highlight"><pre><code class="bash">git clone git://github.com/linkedin/hello-samza.git -<span class="nb">cd </span>hello-samza -vi src/main/config/wikipedia-feed.properties</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>git clone git://github.com/linkedin/hello-samza.git +<span class="nb">cd</span> hello-samza +vi src/main/config/wikipedia-feed.properties</code></pre></figure> <p>Change the yarn.package.path property to be:</p> -<div class="highlight"><pre><code class="jproperties"><span class="na">yarn.package.path</span><span class="o">=</span><span class="s">http://yourHostname:8000/target/hello-samza-0.14.0-dist.tar.gz</span></code></pre></div> +<figure class="highlight"><pre><code class="language-jproperties" data-lang="jproperties"><span></span><span class="na">yarn.package.path</span><span class="o">=</span><span class="s">http://yourHostname:8000/target/hello-samza-0.14.1-dist.tar.gz</span></code></pre></figure> <p>3. Compile hello-samza.</p> -<div class="highlight"><pre><code class="bash">mvn clean package +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>mvn clean package mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.14.0-dist.tar.gz -C deploy/samza</code></pre></div> +tar -xvf ./target/hello-samza-0.14.1-dist.tar.gz -C deploy/samza</code></pre></figure> <p>4. Deploy Samza job package to Http server..</p> <p>Open a new terminal, and run:</p> -<div class="highlight"><pre><code class="bash"><span class="nb">cd</span> /tmp/hello-samza <span class="o">&&</span> python -m SimpleHTTPServer</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span><span class="nb">cd</span> /tmp/hello-samza <span class="o">&&</span> python -m SimpleHTTPServer</code></pre></figure> <p>Go back to the original terminal (not the one running the HTTP server):</p> -<div class="highlight"><pre><code class="bash">deploy/samza/bin/run-job.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/wikipedia-feed.properties</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>deploy/samza/bin/run-job.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/wikipedia-feed.properties</code></pre></figure> <p>Go to http://yourHostname:8088 and find the wikipedia-feed job. Click on the ApplicationMaster link to see that it’s running.</p> @@ -287,7 +285,7 @@ tar -xvf ./target/hello-samza-0.14.0-dis <script> $( document ).ready(function() { - if ( $.fn.urlExists( "/learn/tutorials/0.14/run-in-multi-node-yarn.html" ) ) { + if ( $.fn.urlExists( "/learn/tutorials/0.14/run-in-multi-node-yarn" ) ) { $("#switch-version-button").addClass("fa fa-history masthead-icon"); } });
Modified: samza/site/learn/tutorials/latest/samza-async-user-guide.html URL: http://svn.apache.org/viewvc/samza/site/learn/tutorials/latest/samza-async-user-guide.html?rev=1832281&r1=1832280&r2=1832281&view=diff ============================================================================== --- samza/site/learn/tutorials/latest/samza-async-user-guide.html (original) +++ samza/site/learn/tutorials/latest/samza-async-user-guide.html Fri May 25 22:28:38 2018 @@ -44,9 +44,7 @@ <!-- this icon only shows in versioned pages --> - - - <a href="http://samza.apache.org/learn/tutorials/0.14/samza-async-user-guide.html";><i id="switch-version-button"></i></a> + <a href="http://samza.apache.org/learn/tutorials/0.14/samza-async-user-guide";><i id="switch-version-button"></i></a> <!-- links for the navigation bar --> @@ -153,7 +151,7 @@ <p>If your job process involves synchronous IO, or blocking IO, you can simply configure the Samza build-in thread pool to run your tasks in parallel. In the following example, SyncRestTask uses Jersey client to makes rest calls in each process().</p> -<div class="highlight"><pre><code class="java"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">SyncRestTask</span> <span class="kd">implements</span> <span class="n">StreamTask</span><span class="o">,</span> <span class="n">InitableTask</span><span class="o">,</span> <span class="n">ClosableTask</span> <span class="o">{</span> +<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="kd">public</span> <span class="kd">class</span> <span class="nc">SyncRestTask</span> <span class="kd">implements</span> <span class="n">StreamTask</span><span class="o">,</span> <span class="n">InitableTask</span><span class="o">,</span> <span class="n">ClosableTask</span> <span class="o">{</span> <span class="kd">private</span> <span class="n">Client</span> <span class="n">client</span><span class="o">;</span> <span class="kd">private</span> <span class="n">WebTarget</span> <span class="n">target</span><span class="o">;</span> @@ -173,12 +171,12 @@ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">close</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span> <span class="n">client</span><span class="o">.</span><span class="na">close</span><span class="o">();</span> <span class="o">}</span> -<span class="o">}</span></code></pre></div> +<span class="o">}</span></code></pre></figure> <p>By default Samza will run this task sequentially in a single thread. In below we configure the thread pool of size 16 to run the tasks in parallel:</p> -<div class="highlight"><pre><code class="jproperties"><span class="c"># Thread pool to run synchronous tasks in parallel.</span> -<span class="na">job.container.thread.pool.size</span><span class="o">=</span><span class="s">16</span></code></pre></div> +<figure class="highlight"><pre><code class="language-jproperties" data-lang="jproperties"><span></span><span class="c"># Thread pool to run synchronous tasks in parallel.</span> +<span class="na">job.container.thread.pool.size</span><span class="o">=</span><span class="s">16</span></code></pre></figure> <p><strong>NOTE:</strong> The thread pool will be used to run all the synchronous operations of a task, including StreamTask.process(), WindowableTask.window(), and internally Task.commit(). This is for maximizing the parallelism between tasks as well as reducing the blocking time. When running tasks in multithreading, Samza still guarantees the in-order processing of the messages within a task by default.</p> @@ -186,7 +184,7 @@ <p>If your job process is asynchronous, e.g. making non-blocking remote IO calls, <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/task/AsyncStreamTask.html">AsyncStreamTask</a> interface provides the support for it. In the following example AsyncRestTask makes asynchronous rest call and triggers callback once it’s complete.</p> -<div class="highlight"><pre><code class="java"><span class="kd">public</span> <span class="kd">class</span> <span class="nc">AsyncRestTask</span> <span class="kd">implements</span> <span class="n">AsyncStreamTask</span><span class="o">,</span> <span class="n">InitableTask</span><span class="o">,</span> <span class="n">ClosableTask</span> <span class="o">{</span> +<figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="kd">public</span> <span class="kd">class</span> <span class="nc">AsyncRestTask</span> <span class="kd">implements</span> <span class="n">AsyncStreamTask</span><span class="o">,</span> <span class="n">InitableTask</span><span class="o">,</span> <span class="n">ClosableTask</span> <span class="o">{</span> <span class="kd">private</span> <span class="n">Client</span> <span class="n">client</span><span class="o">;</span> <span class="kd">private</span> <span class="n">WebTarget</span> <span class="n">target</span><span class="o">;</span> @@ -218,12 +216,12 @@ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">close</span><span class="o">()</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span> <span class="n">client</span><span class="o">.</span><span class="na">close</span><span class="o">();</span> <span class="o">}</span> -<span class="o">}</span></code></pre></div> +<span class="o">}</span></code></pre></figure> <p>In the above example, the process is not complete when processAsync() returns. In the callback thread from Jersey client, we trigger <a href="/learn/documentation/latest/api/javadocs/org/apache/samza/task/TaskCallback.html">TaskCallback</a> to indicate the process is done. In order to make sure the callback will be triggered within certain time interval, e.g. 5 seconds, you can config the following property:</p> -<div class="highlight"><pre><code class="jproperties"><span class="c"># Timeout for processAsync() callback. When the timeout happens, it will throw a TaskCallbackTimeoutException and shut down the container.</span> -<span class="na">task.callback.timeout.ms</span><span class="o">=</span><span class="s">5000</span></code></pre></div> +<figure class="highlight"><pre><code class="language-jproperties" data-lang="jproperties"><span></span><span class="c"># Timeout for processAsync() callback. When the timeout happens, it will throw a TaskCallbackTimeoutException and shut down the container.</span> +<span class="na">task.callback.timeout.ms</span><span class="o">=</span><span class="s">5000</span></code></pre></figure> <p><strong>NOTE:</strong> Samza also guarantees the in-order process of the messages within an AsyncStreamTask by default, meaning the next processAsync() of a task won’t be called until the previous processAsync() callback has been triggered.</p> @@ -231,8 +229,8 @@ <p>In both cases above, Samza supports in-order process by default. Further parallelism is also supported by allowing a task to process multiple outstanding messages in parallel. The following config allows one task to process at most 4 outstanding messages in parallel at a time: </p> -<div class="highlight"><pre><code class="jproperties"><span class="c"># Max number of outstanding messages being processed per task at a time, applicable to both StreamTask and AsyncStreamTask.</span> -<span class="na">task.max.concurrency</span><span class="o">=</span><span class="s">4</span></code></pre></div> +<figure class="highlight"><pre><code class="language-jproperties" data-lang="jproperties"><span></span><span class="c"># Max number of outstanding messages being processed per task at a time, applicable to both StreamTask and AsyncStreamTask.</span> +<span class="na">task.max.concurrency</span><span class="o">=</span><span class="s">4</span></code></pre></figure> <p><strong>NOTE:</strong> In case of AsyncStreamTask, processAsync() is still invoked in the order of the message arrivals, but the completion can go out of order. In case of StreamTask with multithreading, process() can run out-of-order since they are dispatched to a thread pool. This option should <strong>NOT</strong> be used when strict ordering of the output is required.</p> @@ -262,7 +260,7 @@ <script> $( document ).ready(function() { - if ( $.fn.urlExists( "/learn/tutorials/0.14/samza-async-user-guide.html" ) ) { + if ( $.fn.urlExists( "/learn/tutorials/0.14/samza-async-user-guide" ) ) { $("#switch-version-button").addClass("fa fa-history masthead-icon"); } }); Modified: samza/site/learn/tutorials/latest/samza-rest-getting-started.html URL: http://svn.apache.org/viewvc/samza/site/learn/tutorials/latest/samza-rest-getting-started.html?rev=1832281&r1=1832280&r2=1832281&view=diff ============================================================================== --- samza/site/learn/tutorials/latest/samza-rest-getting-started.html (original) +++ samza/site/learn/tutorials/latest/samza-rest-getting-started.html Fri May 25 22:28:38 2018 @@ -44,9 +44,7 @@ <!-- this icon only shows in versioned pages --> - - - <a href="http://samza.apache.org/learn/tutorials/0.14/samza-rest-getting-started.html";><i id="switch-version-button"></i></a> + <a href="http://samza.apache.org/learn/tutorials/0.14/samza-rest-getting-started";><i id="switch-version-button"></i></a> <!-- links for the navigation bar --> @@ -161,28 +159,28 @@ <p>The source code for Samza REST is in the samza-rest module of the Samza repository. To build it, execute the following gradle task from the root of the project.</p> -<div class="highlight"><pre><code class="bash">./gradlew samza-rest:clean releaseRestServiceTar</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>./gradlew samza-rest:clean releaseRestServiceTar</code></pre></figure> <h4 id="deploy-the-samza-rest-service-locally">Deploy the Samza REST Service Locally</h4> <p>To deploy the service, you simply extract the tarball to the desired location. Here, we will deploy the tarball on the local host in</p> -<div class="highlight"><pre><code class="language-text" data-lang="text">SAMZA_ROOT/samza-rest/build/distributions/deploy/samza-rest +<div class="highlight"><pre><code class="language-text" data-lang="text"><span></span>SAMZA_ROOT/samza-rest/build/distributions/deploy/samza-rest </code></pre></div> <p>where <code>SAMZA_ROOT</code> is the path to the root of your Samza project.</p> <p>Run the following commands:</p> -<div class="highlight"><pre><code class="bash"><span class="nb">cd </span>samza-rest/build/distributions/ +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span><span class="nb">cd</span> samza-rest/build/distributions/ mkdir -p deploy/samza-rest -tar -xvf ./samza-rest-0.14.1-SNAPSHOT.tgz -C deploy/samza-rest</code></pre></div> +tar -xvf ./samza-rest-0.14.1.tgz -C deploy/samza-rest</code></pre></figure> <h4 id="configure-the-installations-path">Configure the Installations Path</h4> <p>The JobsResource has a required config <a href="../../documentation/latest/rest/resources/jobs.html#configuration">job.installations.path</a> which specifies the path where the jobs are installed. Edit the configuration file:</p> -<div class="highlight"><pre><code class="language-text" data-lang="text">deploy/samza-rest/config/samza-rest.properties +<div class="highlight"><pre><code class="language-text" data-lang="text"><span></span>deploy/samza-rest/config/samza-rest.properties </code></pre></div> <p>Set the job.installations.path to:</p> -<div class="highlight"><pre><code class="language-text" data-lang="text">job.installations.path=/hello-samza-ROOT/deploy +<div class="highlight"><pre><code class="language-text" data-lang="text"><span></span>job.installations.path=/hello-samza-ROOT/deploy </code></pre></div> <p>where <code>hello-samza-ROOT</code> is the path to your hello-samza clone, noted above. This tells the JobsResource to crawl this location to find all the installed jobs.</p> @@ -190,10 +188,10 @@ tar -xvf ./samza-rest-0.14.1-SNAPSHOT.tg <p>To deploy the service, run the run-samza-rest-service.sh script from the extracted directory.</p> -<div class="highlight"><pre><code class="bash"><span class="nb">cd </span>deploy/samza-rest +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span><span class="nb">cd</span> deploy/samza-rest ./bin/run-samza-rest-service.sh <span class="se">\</span> --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory <span class="se">\</span> - --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/config/samza-rest.properties</code></pre></div> + --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/config/samza-rest.properties</code></pre></figure> <p>You provide two parameters to the run-samza-rest-service.sh script. One is the config location, and the other, optional, parameter is a factory class that is used to read your configuration file. The SamzaRestService uses your ConfigFactory to get a Config object from the config path. The ConfigFactory is covered in more detail on the <a href="../../documentation/latest/jobs/job-runner.html">Job Runner page</a>. The run-samza-rest-service.sh script will block until the SamzaRestService terminates.</p> @@ -202,11 +200,11 @@ tar -xvf ./samza-rest-0.14.1-SNAPSHOT.tg <h3 id="curl-the-default-rest-service">Curl the Default REST Service</h3> <p>Curl the JobsResource to get all installed jobs</p> -<div class="highlight"><pre><code class="language-text" data-lang="text">curl localhost:9139/v1/jobs +<div class="highlight"><pre><code class="language-text" data-lang="text"><span></span>curl localhost:9139/v1/jobs [{"jobName":"wikipedia-stats","jobId":"1","status":"STARTED","statusDetail":RUNNING},{"jobName":"wikipedia-parser","jobId":"1","status":"STARTED","statusDetail":RUNNING},{"jobName":"wikipedia-feed","jobId":"1","status":"STARTED","statusDetail":RUNNING} </code></pre></div> <p>Now curl the JobsResource to stop one of the jobs</p> -<div class="highlight"><pre><code class="language-text" data-lang="text">curl -X PUT localhost:9139/v1/jobs/wikipedia-feed/1?status=stopped +<div class="highlight"><pre><code class="language-text" data-lang="text"><span></span>curl -X PUT localhost:9139/v1/jobs/wikipedia-feed/1?status=stopped {"jobId":"1","jobName":"wikipedia-feed","status":"STOPPED","statusDetail":"FINISHED"} </code></pre></div> <p>Congratulations, you’ve successfully deployed the Samza REST Service and used the JobsResource to list jobs and stop a job!</p> @@ -231,7 +229,7 @@ tar -xvf ./samza-rest-0.14.1-SNAPSHOT.tg <script> $( document ).ready(function() { - if ( $.fn.urlExists( "/learn/tutorials/0.14/samza-rest-getting-started.html" ) ) { + if ( $.fn.urlExists( "/learn/tutorials/0.14/samza-rest-getting-started" ) ) { $("#switch-version-button").addClass("fa fa-history masthead-icon"); } }); Modified: samza/site/learn/tutorials/latest/samza-sql.html URL: http://svn.apache.org/viewvc/samza/site/learn/tutorials/latest/samza-sql.html?rev=1832281&r1=1832280&r2=1832281&view=diff ============================================================================== --- samza/site/learn/tutorials/latest/samza-sql.html (original) +++ samza/site/learn/tutorials/latest/samza-sql.html Fri May 25 22:28:38 2018 @@ -44,9 +44,7 @@ <!-- this icon only shows in versioned pages --> - - - <a href="http://samza.apache.org/learn/tutorials/0.14/samza-sql.html";><i id="switch-version-button"></i></a> + <a href="http://samza.apache.org/learn/tutorials/0.14/samza-sql";><i id="switch-version-button"></i></a> <!-- links for the navigation bar --> @@ -166,27 +164,27 @@ <p>The below sql statements requires a topic named ProfileChangeStream to be created on the Kafka broker. You can follow the instructions in the <a href="http://kafka.apache.org/quickstart";>Kafka quick start guide</a> to create a topic named “ProfileChangeStream”.</p> -<div class="highlight"><pre><code class="bash">./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic ProfileChangeStream</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor <span class="m">1</span> --partitions <span class="m">1</span> --topic ProfileChangeStream</code></pre></figure> <h2 id="generate-events-into-profilechangestream-topic">Generate events into ProfileChangeStream topic</h2> <p>Use generate-kafka-events from <a href="samza-tools.html">Samza tools</a> to generate events into the ProfileChangeStream</p> -<div class="highlight"><pre><code class="bash"><span class="nb">cd </span>samza-tools-<version> -./scripts/generate-kafka-events.sh -t ProfileChangeStream -e ProfileChange</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span><span class="nb">cd</span> samza-tools-<version> +./scripts/generate-kafka-events.sh -t ProfileChangeStream -e ProfileChange</code></pre></figure> <h2 id="using-samza-sql-console-to-run-samza-sql-on-your-local-machine">Using Samza SQL Console to run Samza sql on your local machine</h2> <p>Below are some of the sql queries that you can execute using the samza-sql-console tool from <a href="samza-tools.html">Samza tools</a> package.</p> -<div class="highlight"><pre><code class="bash"><span class="c"># This command just prints out all the events in the Kafka topic ProfileChangeStream into console output as a json serialized payload.</span> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span><span class="c1"># This command just prints out all the events in the Kafka topic ProfileChangeStream into console output as a json serialized payload.</span> ./scripts/samza-sql-console.sh --sql <span class="s2">"insert into log.consoleoutput select * from kafka.ProfileChangeStream"</span> -<span class="c"># This command prints out the fields that are selected into the console output as a json serialized payload.</span> +<span class="c1"># This command prints out the fields that are selected into the console output as a json serialized payload.</span> ./scripts/samza-sql-console.sh --sql <span class="s2">"insert into log.consoleoutput select Name, OldCompany, NewCompany from kafka.ProfileChangeStream"</span> -<span class="c"># This command showcases the RegexMatch udf and filtering capabilities.</span> -./scripts/samza-sql-console.sh --sql <span class="s2">"insert into log.consoleoutput select Name as __key__, Name, NewCompany, RegexMatch('.*soft', OldCompany) from kafka.ProfileChangeStream where NewCompany = 'LinkedIn'"</span></code></pre></div> +<span class="c1"># This command showcases the RegexMatch udf and filtering capabilities.</span> +./scripts/samza-sql-console.sh --sql <span class="s2">"insert into log.consoleoutput select Name as __key__, Name, NewCompany, RegexMatch('.*soft', OldCompany) from kafka.ProfileChangeStream where NewCompany = 'LinkedIn'"</span></code></pre></figure> <h1 id="running-samza-sql-on-yarn">Running Samza SQL on YARN</h1> @@ -210,11 +208,11 @@ <p>After you’ve built your Samza package, you can start the app on the grid using the run-app.sh script.</p> -<div class="highlight"><pre><code class="bash">./deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/page-view-filter-sql.properties</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>./deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/page-view-filter-sql.properties</code></pre></figure> <p>The app executes the following SQL command :</p> -<div class="highlight"><pre><code class="sql"><span class="k">insert</span> <span class="k">into</span> <span class="n">kafka</span><span class="p">.</span><span class="n">NewLinkedInEmployees</span> <span class="k">select</span> <span class="n">Name</span> <span class="k">from</span> <span class="n">ProfileChangeStream</span> <span class="k">where</span> <span class="n">NewCompany</span> <span class="o">=</span> <span class="s1">'LinkedIn'</span></code></pre></div> +<figure class="highlight"><pre><code class="language-sql" data-lang="sql"><span></span><span class="k">insert</span> <span class="k">into</span> <span class="n">kafka</span><span class="p">.</span><span class="n">NewLinkedInEmployees</span> <span class="k">select</span> <span class="n">Name</span> <span class="k">from</span> <span class="n">ProfileChangeStream</span> <span class="k">where</span> <span class="n">NewCompany</span> <span class="o">=</span> <span class="s1">'LinkedIn'</span></code></pre></figure> <p>This SQL performs the following</p> @@ -226,7 +224,7 @@ <p>Give the job a minute to startup, and then tail the Kafka topic:</p> -<div class="highlight"><pre><code class="bash">./deploy/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic NewLinkedInEmployees</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>./deploy/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic NewLinkedInEmployees</code></pre></figure> <p>Congratulations! You’ve now setup a local grid that includes YARN, Kafka, and ZooKeeper, and run a Samza SQL application on it.</p> @@ -234,7 +232,7 @@ <p>To shutdown the app, use the same <em>run-app.sh</em> script with an extra <em>–operation=kill</em> argument</p> -<div class="highlight"><pre><code class="bash">./deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/page-view-filter-sql.properties --operation<span class="o">=</span><span class="nb">kill</span></code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>./deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/page-view-filter-sql.properties --operation<span class="o">=</span><span class="nb">kill</span></code></pre></figure> <p>Please follow the instructions from <a href="hello-samza-high-level-yarn.html">Hello Samza High Level API - YARN Deployment</a> on how to shutdown and cleanup the app.</p> @@ -254,7 +252,7 @@ <script> $( document ).ready(function() { - if ( $.fn.urlExists( "/learn/tutorials/0.14/samza-sql.html" ) ) { + if ( $.fn.urlExists( "/learn/tutorials/0.14/samza-sql" ) ) { $("#switch-version-button").addClass("fa fa-history masthead-icon"); } }); Modified: samza/site/learn/tutorials/latest/samza-tools.html URL: http://svn.apache.org/viewvc/samza/site/learn/tutorials/latest/samza-tools.html?rev=1832281&r1=1832280&r2=1832281&view=diff ============================================================================== --- samza/site/learn/tutorials/latest/samza-tools.html (original) +++ samza/site/learn/tutorials/latest/samza-tools.html Fri May 25 22:28:38 2018 @@ -44,9 +44,7 @@ <!-- this icon only shows in versioned pages --> - - - <a href="http://samza.apache.org/learn/tutorials/0.14/samza-tools.html";><i id="switch-version-button"></i></a> + <a href="http://samza.apache.org/learn/tutorials/0.14/samza-tools";><i id="switch-version-button"></i></a> <!-- links for the navigation bar --> @@ -151,8 +149,8 @@ <p>Please visit the <a href="/startup/download">Download page</a> to download the Samza tools package</p> -<div class="highlight"><pre><code class="bash">tar -xvzf samza-tools-*.tgz -<span class="nb">cd </span>samza-tools-<version></code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>tar -xvzf samza-tools-*.tgz +<span class="nb">cd</span> samza-tools-<version></code></pre></figure> <h1 id="using-samza-tools">Using Samza tools</h1> @@ -164,7 +162,7 @@ <p>You can follow below instructions on how to use Generate kafka events tool.</p> -<div class="highlight"><pre><code class="bash"><span class="c"># Usage of the tool</span> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span><span class="c1"># Usage of the tool</span> ./scripts/generate-kafka-events.sh usage: Error: Missing required options: t, e @@ -173,19 +171,19 @@ usage: Error: Missing required options: -n,--numEvents <NUM_EVENTS> Number of events to be produced, Default - Produces events continuously every second. -p,--partitions <NUM_PARTITIONS> Number of partitions in the topic, - Default <span class="o">(</span>4<span class="o">)</span>. + Default <span class="o">(</span><span class="m">4</span><span class="o">)</span>. -t,--topic <TOPIC_NAME> Name of the topic to write events to. -e,--eventtype <EVENT_TYPE> Type of the event values can be <span class="o">(</span>PageView<span class="p">|</span>ProfileChange<span class="o">)</span>. -<span class="c"># Example command to generate 100 events of type PageViewEvent into topic named PageViewStream</span> +<span class="c1"># Example command to generate 100 events of type PageViewEvent into topic named PageViewStream</span> - ./scripts/generate-kafka-events.sh -t PageViewStream -e PageView -n 100 + ./scripts/generate-kafka-events.sh -t PageViewStream -e PageView -n <span class="m">100</span> -<span class="c"># Example command to generate ProfileChange events continuously into topic named ProfileChangeStream</span> +<span class="c1"># Example command to generate ProfileChange events continuously into topic named ProfileChangeStream</span> - ./scripts/generate-kafka-events.sh -t ProfileChangeStream -e ProfileChange</code></pre></div> + ./scripts/generate-kafka-events.sh -t ProfileChangeStream -e ProfileChange </code></pre></figure> <h2 id="samza-sql-console-tool">Samza SQL console tool</h2> @@ -204,21 +202,21 @@ usage: Error: Missing required options: <p>In the absence of schema registry, Samza SQL console tool uses the convention to identify the schemas associated with the topic. If the topic name has string “page” it assumes the topic has PageViewEvents else ProfileChangeEvents. </p> -<div class="highlight"><pre><code class="bash"><span class="c"># Usage of the tool</span> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span><span class="c1"># Usage of the tool</span> ./scripts/samza-sql-console.sh usage: Error: One of the <span class="o">(</span>f or s<span class="o">)</span> options needs to be <span class="nb">set</span> -<span class="nb"> </span>samza-sql-console.sh + samza-sql-console.sh -f,--file <SQL_FILE> Path to the SQL file to execute. -s,--sql <SQL_STMT> SQL statement to execute. -<span class="c"># Example command to filter out all the users who have moved to LinkedIn</span> +<span class="c1"># Example command to filter out all the users who have moved to LinkedIn</span> -./scripts/samza-sql-console.sh --sql <span class="s2">"Insert into log.consoleOutput select Name, OldCompany from kafka.ProfileChangeStream where NewCompany = 'LinkedIn'"</span></code></pre></div> +./scripts/samza-sql-console.sh --sql <span class="s2">"Insert into log.consoleOutput select Name, OldCompany from kafka.ProfileChangeStream where NewCompany = 'LinkedIn'"</span></code></pre></figure> <p>You can run below sql commands using Samza sql console. Please make sure you are running generate-kafka-events tool to generate events into ProfileChangeStream before running the below command.</p> -<div class="highlight"><pre><code class="bash">./scripts/samza-sql-console.sh --sql <span class="s2">"Insert into log.consoleOutput select Name, OldCompany from kafka.ProfileChangeStream where NewCompany = 'LinkedIn'"</span></code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>./scripts/samza-sql-console.sh --sql <span class="s2">"Insert into log.consoleOutput select Name, OldCompany from kafka.ProfileChangeStream where NewCompany = 'LinkedIn'"</span></code></pre></figure> </div> @@ -236,7 +234,7 @@ usage: Error: One of the <span class="o" <script> $( document ).ready(function() { - if ( $.fn.urlExists( "/learn/tutorials/0.14/samza-tools.html" ) ) { + if ( $.fn.urlExists( "/learn/tutorials/0.14/samza-tools" ) ) { $("#switch-version-button").addClass("fa fa-history masthead-icon"); } }); Modified: samza/site/sitemap.xml URL: http://svn.apache.org/viewvc/samza/site/sitemap.xml?rev=1832281&r1=1832280&r2=1832281&view=diff ============================================================================== --- samza/site/sitemap.xml (original) +++ samza/site/sitemap.xml Fri May 25 22:28:38 2018 @@ -20,7 +20,7 @@ <url> <loc>http://samza.apache.org/</loc> - <lastmod>2018-01-04</lastmod> + <lastmod>2018-05-25</lastmod> <changefreq>daily</changefreq> <priority>1.0</priority> </url> @@ -29,505 +29,505 @@ <url> - <loc>http://samza.apache.org/learn/documentation/versioned/yarn/application-master.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/yarn/application-master</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/introduction/architecture.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/introduction/architecture</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/introduction/background.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/introduction/background</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/container/checkpointing.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/container/checkpointing</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/contribute/code.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/contribute/code</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/contribute/coding-guide.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/contribute/coding-guide</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/community/committers.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/community/committers</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/introduction/concepts.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/introduction/concepts</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/jobs/configuration.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/jobs/configuration</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/hadoop/consumer.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/hadoop/consumer</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/contribute/contributors-corner.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/contribute/contributors-corner</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/container/coordinator-stream.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/container/coordinator-stream</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/deploy-samza-job-from-hdfs.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/deploy-samza-job-from-hdfs</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/deploy-samza-to-CDH.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/deploy-samza-to-CDH</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/contribute/design-documents.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/contribute/design-documents</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/container/event-loop.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/container/event-loop</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/azure/eventhubs.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/azure/eventhubs</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/hello-samza-high-level-code.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/hello-samza-high-level-code</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/hello-samza-high-level-yarn.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/hello-samza-high-level-yarn</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/hello-samza-high-level-zk.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/hello-samza-high-level-zk</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/index.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/archive/index.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/archive/</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/startup/preview/index.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/startup/hello-samza/versioned/</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/index.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/startup/download/</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/index.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/startup/download/index.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/startup/preview/</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/startup/hello-samza/versioned/index.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/comparisons/introduction.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/comparisons/introduction</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/community/irc.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/community/irc</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/yarn/isolation.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/yarn/isolation</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/container/jmx.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/container/jmx</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/jobs/job-runner.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/jobs/job-runner</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/rest/resources/jobs.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/rest/resources/jobs</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/operations/kafka.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/operations/kafka</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/aws/kinesis.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/aws/kinesis</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/jobs/logging.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/jobs/logging</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/community/mailing-lists.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/community/mailing-lists</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/container/metrics.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/container/metrics</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/rest/monitors.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/rest/monitors</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/comparisons/mupd8.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/comparisons/mupd8</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/rest/overview.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/hadoop/overview</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/hadoop/overview.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/rest/overview</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/api/overview.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/api/overview</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/jobs/packaging.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/jobs/packaging</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/hadoop/producer.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/hadoop/producer</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/startup/releases/versioned/release-notes.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/startup/releases/versioned/release-notes</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/remote-debugging-samza.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/remote-debugging-samza</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/jobs/reprocessing.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/jobs/reprocessing</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/rest/resource-directory.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/rest/resource-directory</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/rest/resources.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/rest/resources</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/run-hello-samza-without-internet.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/run-hello-samza-without-internet</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/run-in-multi-node-yarn.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/run-in-multi-node-yarn</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/samza-async-user-guide.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/samza-async-user-guide</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/container/samza-container.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/container/samza-container</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/samza-rest-getting-started.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/samza-rest-getting-started</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/samza-sql.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/samza-sql</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/tutorials/versioned/samza-tools.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/tutorials/versioned/samza-tools</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/operations/security.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/operations/security</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/container/serialization.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/container/serialization</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/comparisons/spark-streaming.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/comparisons/spark-streaming</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/jobs/split-deployment.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/jobs/split-deployment</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/container/state-management.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/container/state-management</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/comparisons/storm.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/comparisons/storm</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/container/streams.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/container/streams</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/rest/resources/tasks.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/rest/resources/tasks</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/contribute/tests.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/contribute/tests</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/jobs/web-ui-rest-api.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/jobs/web-ui-rest-api</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/container/windowing.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/container/windowing</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/yarn/yarn-host-affinity.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/yarn/yarn-host-affinity</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/jobs/yarn-jobs.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/jobs/yarn-jobs</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/yarn/yarn-resource-localization.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/yarn/yarn-resource-localization</loc> + <lastmod>2018-05-25</lastmod> </url> <url> - <loc>http://samza.apache.org/learn/documentation/versioned/yarn/yarn-security.html</loc> - <lastmod>2018-01-04</lastmod> + <loc>http://samza.apache.org/learn/documentation/versioned/yarn/yarn-security</loc> + <lastmod>2018-05-25</lastmod> </url> Modified: samza/site/startup/download/index.html URL: http://svn.apache.org/viewvc/samza/site/startup/download/index.html?rev=1832281&r1=1832280&r2=1832281&view=diff ============================================================================== --- samza/site/startup/download/index.html (original) +++ samza/site/startup/download/index.html Fri May 25 22:28:38 2018 @@ -156,12 +156,13 @@ <p>Samza tools package contains command line tools that user can run to use Samza and it’s input/output systems. </p> <ul> -<li><a href="http://www-us.apache.org/dist/samza/0.14.0/samza-tools-0.14.0.tgz";>samza-tools-0.14.0.tgz</a></li> +<li><a href="http://www-us.apache.org/dist/samza/0.14.1/samza-tools-0.14.1.tgz";>samza-tools-0.14.1.tgz</a></li> </ul> <h3 id="source-releases">Source Releases</h3> <ul> +<li><a href="http://www.apache.org/dyn/closer.lua/samza/0.14.1";>samza-sources-0.14.1.tgz</a></li> <li><a href="http://www.apache.org/dyn/closer.lua/samza/0.14.0";>samza-sources-0.14.0.tgz</a></li> <li><a href="http://www.apache.org/dyn/closer.lua/samza/0.13.1";>samza-sources-0.13.1.tgz</a></li> <li><a href="http://www.apache.org/dyn/closer.lua/samza/0.13.0";>samza-sources-0.13.0.tgz</a></li> @@ -183,15 +184,15 @@ <p>A Maven-based Samza project can pull in all required dependencies Samza dependencies this XML block:</p> -<div class="highlight"><pre><code class="xml"><span class="nt"><dependency></span> +<figure class="highlight"><pre><code class="language-xml" data-lang="xml"><span></span><span class="nt"><dependency></span> <span class="nt"><groupId></span>org.apache.samza<span class="nt"></groupId></span> <span class="nt"><artifactId></span>samza-api<span class="nt"></artifactId></span> - <span class="nt"><version></span>0.14.0<span class="nt"></version></span> + <span class="nt"><version></span>0.14.1<span class="nt"></version></span> <span class="nt"></dependency></span> <span class="nt"><dependency></span> <span class="nt"><groupId></span>org.apache.samza<span class="nt"></groupId></span> <span class="nt"><artifactId></span>samza-core_2.11<span class="nt"></artifactId></span> - <span class="nt"><version></span>0.14.0<span class="nt"></version></span> + <span class="nt"><version></span>0.14.1<span class="nt"></version></span> <span class="nt"><scope></span>runtime<span class="nt"></scope></span> <span class="nt"></dependency></span> <span class="nt"><dependency></span> @@ -199,55 +200,55 @@ <span class="nt"><artifactId></span>samza-shell<span class="nt"></artifactId></span> <span class="nt"><classifier></span>dist<span class="nt"></classifier></span> <span class="nt"><type></span>tgz<span class="nt"></type></span> - <span class="nt"><version></span>0.14.0<span class="nt"></version></span> + <span class="nt"><version></span>0.14.1<span class="nt"></version></span> <span class="nt"><scope></span>runtime<span class="nt"></scope></span> <span class="nt"></dependency></span> <span class="nt"><dependency></span> <span class="nt"><groupId></span>org.apache.samza<span class="nt"></groupId></span> <span class="nt"><artifactId></span>samza-yarn_2.11<span class="nt"></artifactId></span> - <span class="nt"><version></span>0.14.0<span class="nt"></version></span> + <span class="nt"><version></span>0.14.1<span class="nt"></version></span> <span class="nt"><scope></span>runtime<span class="nt"></scope></span> <span class="nt"></dependency></span> <span class="nt"><dependency></span> <span class="nt"><groupId></span>org.apache.samza<span class="nt"></groupId></span> <span class="nt"><artifactId></span>samza-kv_2.11<span class="nt"></artifactId></span> - <span class="nt"><version></span>0.14.0<span class="nt"></version></span> + <span class="nt"><version></span>0.14.1<span class="nt"></version></span> <span class="nt"><scope></span>runtime<span class="nt"></scope></span> <span class="nt"></dependency></span> <span class="nt"><dependency></span> <span class="nt"><groupId></span>org.apache.samza<span class="nt"></groupId></span> <span class="nt"><artifactId></span>samza-kv-rocksdb_2.11<span class="nt"></artifactId></span> - <span class="nt"><version></span>0.14.0<span class="nt"></version></span> + <span class="nt"><version></span>0.14.1<span class="nt"></version></span> <span class="nt"><scope></span>runtime<span class="nt"></scope></span> <span class="nt"></dependency></span> <span class="nt"><dependency></span> <span class="nt"><groupId></span>org.apache.samza<span class="nt"></groupId></span> <span class="nt"><artifactId></span>samza-kv-inmemory_2.11<span class="nt"></artifactId></span> - <span class="nt"><version></span>0.14.0<span class="nt"></version></span> + <span class="nt"><version></span>0.14.1<span class="nt"></version></span> <span class="nt"><scope></span>runtime<span class="nt"></scope></span> <span class="nt"></dependency></span> <span class="nt"><dependency></span> <span class="nt"><groupId></span>org.apache.samza<span class="nt"></groupId></span> <span class="nt"><artifactId></span>samza-kafka_2.11<span class="nt"></artifactId></span> - <span class="nt"><version></span>0.14.0<span class="nt"></version></span> + <span class="nt"><version></span>0.14.1<span class="nt"></version></span> <span class="nt"><scope></span>runtime<span class="nt"></scope></span> -<span class="nt"></dependency></span></code></pre></div> +<span class="nt"></dependency></span></code></pre></figure> <p>Samza versions less than 0.12 should use artifacts with scala version 2.10 as suffix. For example,</p> -<div class="highlight"><pre><code class="xml"><span class="nt"><dependency></span> +<figure class="highlight"><pre><code class="language-xml" data-lang="xml"><span></span><span class="nt"><dependency></span> <span class="nt"><groupId></span>org.apache.samza<span class="nt"></groupId></span> <span class="nt"><artifactId></span>samza-yarn_2.10<span class="nt"></artifactId></span> <span class="nt"><version></span>0.11.0<span class="nt"></version></span> -<span class="nt"></dependency></span></code></pre></div> +<span class="nt"></dependency></span></code></pre></figure> <p>Samza versions less than 0.9 should include this additional dependency.</p> -<div class="highlight"><pre><code class="xml"><span class="nt"><dependency></span> +<figure class="highlight"><pre><code class="language-xml" data-lang="xml"><span></span><span class="nt"><dependency></span> <span class="nt"><groupId></span>org.apache.samza<span class="nt"></groupId></span> <span class="nt"><artifactId></span>samza-serializers_2.10<span class="nt"></artifactId></span> <span class="nt"><version></span>0.8.1<span class="nt"></version></span> -<span class="nt"></dependency></span></code></pre></div> +<span class="nt"></dependency></span></code></pre></figure> <p><a href="/startup/hello-samza/latest">Hello Samza</a> is a working Maven project that illustrates how to build projects that have Samza jobs in them.</p> @@ -255,25 +256,25 @@ <p>Samza is available in the Apache Maven repository.</p> -<div class="highlight"><pre><code class="xml"><span class="nt"><repository></span> +<figure class="highlight"><pre><code class="language-xml" data-lang="xml"><span></span><span class="nt"><repository></span> <span class="nt"><id></span>apache-releases<span class="nt"></id></span> <span class="nt"><url></span>https://repository.apache.org/content/groups/public<span class="nt"></url></span> -<span class="nt"></repository></span></code></pre></div> +<span class="nt"></repository></span></code></pre></figure> <p>Snapshot builds are available in the Apache Maven snapshot repository.</p> -<div class="highlight"><pre><code class="xml"><span class="nt"><repository></span> +<figure class="highlight"><pre><code class="language-xml" data-lang="xml"><span></span><span class="nt"><repository></span> <span class="nt"><id></span>apache-snapshots<span class="nt"></id></span> <span class="nt"><url></span>https://repository.apache.org/content/groups/snapshots<span class="nt"></url></span> -<span class="nt"></repository></span></code></pre></div> +<span class="nt"></repository></span></code></pre></figure> <h3 id="checking-out-and-building">Checking out and Building</h3> <p>If you’re interested in working on Samza, or building the JARs from scratch, then you’ll need to checkout and build the code. Samza does not have a binary release at this time. To check out and build Samza, run these commands.</p> -<div class="highlight"><pre><code class="bash">git clone http://git-wip-us.apache.org/repos/asf/samza.git -<span class="nb">cd </span>samza -./gradlew clean build</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>git clone http://git-wip-us.apache.org/repos/asf/samza.git +<span class="nb">cd</span> samza +./gradlew clean build</code></pre></figure> <p>See the README.md file for details on building.</p> Modified: samza/site/startup/hello-samza/latest/index.html URL: http://svn.apache.org/viewvc/samza/site/startup/hello-samza/latest/index.html?rev=1832281&r1=1832280&r2=1832281&view=diff ============================================================================== --- samza/site/startup/hello-samza/latest/index.html (original) +++ samza/site/startup/hello-samza/latest/index.html Fri May 25 22:28:38 2018 @@ -44,9 +44,7 @@ <!-- this icon only shows in versioned pages --> - - - <a href="http://samza.apache.org/startup/hello-samza/0.14/index.html";><i id="switch-version-button"></i></a> + <a href="http://samza.apache.org/startup/hello-samza/0.14/";><i id="switch-version-button"></i></a> <!-- links for the navigation bar --> @@ -153,9 +151,8 @@ <p>Check out the hello-samza project:</p> -<div class="highlight"><pre><code class="bash">git clone https://git.apache.org/samza-hello-samza.git hello-samza -<span class="nb">cd </span>hello-samza -git checkout latest</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>git clone https://git.apache.org/samza-hello-samza.git hello-samza +<span class="nb">cd</span> hello-samza</code></pre></figure> <p>This project contains everything you’ll need to run your first Samza jobs.</p> @@ -163,7 +160,7 @@ git checkout latest</code></pre></div> <p>A Samza grid usually comprises three different systems: <a href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html";>YARN</a>, <a href="http://kafka.apache.org/";>Kafka</a>, and <a href="http://zookeeper.apache.org/";>ZooKeeper</a>. The hello-samza project comes with a script called “grid” to help you setup these systems. Start by running:</p> -<div class="highlight"><pre><code class="bash">bin/grid bootstrap</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>bin/grid bootstrap</code></pre></figure> <p>This command will download, install, and start ZooKeeper, Kafka, and YARN. It will also check out the latest version of Samza and build it. All package files will be put in a sub-directory called “deploy” inside hello-samza’s root folder.</p> @@ -177,23 +174,23 @@ git checkout latest</code></pre></div> <p>NOTE: if you are building from the latest branch of hello-samza project, make sure that you run the following step from your local Samza project first:</p> -<div class="highlight"><pre><code class="bash">./gradlew publishToMavenLocal</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>./gradlew publishToMavenLocal</code></pre></figure> <p>Then, you can continue w/ the following command in hello-samza project:</p> -<div class="highlight"><pre><code class="bash">mvn clean package +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>mvn clean package mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.14.1-SNAPSHOT-dist.tar.gz -C deploy/samza</code></pre></div> +tar -xvf ./target/hello-samza-0.14.1-dist.tar.gz -C deploy/samza</code></pre></figure> <h3 id="run-a-samza-job">Run a Samza Job</h3> <p>After you’ve built your Samza package, you can start a job on the grid using the run-app.sh script.</p> -<div class="highlight"><pre><code class="bash">deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/wikipedia-feed.properties</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/wikipedia-feed.properties</code></pre></figure> <p>The job will consume a feed of real-time edits from Wikipedia, and produce them to a Kafka topic called “wikipedia-raw”. Give the job a minute to startup, and then tail the Kafka topic:</p> -<div class="highlight"><pre><code class="bash">deploy/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wikipedia-raw</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>deploy/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wikipedia-raw</code></pre></figure> <p>Pretty neat, right? Now, check out the YARN UI again (<a href="http://localhost:8088";>http://localhost:8088</a>). This time around, you’ll see your Samza job is running!</p> @@ -203,23 +200,23 @@ tar -xvf ./target/hello-samza-0.14.1-SNA <p>Let’s calculate some statistics based on the messages in the wikipedia-raw topic. Start two more jobs:</p> -<div class="highlight"><pre><code class="bash">deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/wikipedia-parser.properties -deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/wikipedia-stats.properties</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/wikipedia-parser.properties +deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/wikipedia-stats.properties</code></pre></figure> <p>The first job (wikipedia-parser) parses the messages in wikipedia-raw, and extracts information about the size of the edit, who made the change, etc. You can take a look at its output with:</p> -<div class="highlight"><pre><code class="bash">deploy/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wikipedia-edits</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>deploy/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wikipedia-edits</code></pre></figure> <p>The last job (wikipedia-stats) reads messages from the wikipedia-edits topic, and calculates counts, every ten seconds, for all edits that were made during that window. It outputs these counts to the wikipedia-stats topic.</p> -<div class="highlight"><pre><code class="bash">deploy/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wikipedia-stats</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>deploy/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wikipedia-stats</code></pre></figure> <p>The messages in the stats topic look like this:</p> -<div class="highlight"><pre><code class="json"><span class="p">{</span><span class="nt">"is-talk"</span><span class="p">:</span><span class="mi">2</span><span class="p">,</span><span class="nt">"bytes-added"</span><span class="p">:</span><span class="mi">5276</span><span class="p">,</span><span class="nt">"edits"</span><span class="p">:</span><span class="mi">13</span><span class="p">,</span><span class="nt">"unique-titles"</span><span class="p">:</span><span class="mi">13</span><span class="p">}</span> +<figure class="highlight"><pre><code class="language-json" data-lang="json"><span></span><span class="p">{</span><span class="nt">"is-talk"</span><span class="p">:</span><span class="mi">2</span><span class="p">,</span><span class="nt">"bytes-added"</span><span class="p">:</span><span class="mi">5276</span><span class="p">,</span><span class="nt">"edits"</span><span class="p">:</span><span class="mi">13</span><span class="p">,</span><span class="nt">"unique-titles"</span><span class="p">:</span><span class="mi">13</span><span class="p">}</span> <span class="p">{</span><span class="nt">"is-bot-edit"</span><span class="p">:</span><span class="mi">1</span><span class="p">,</span><span class="nt">"is-talk"</span><span class="p">:</span><span class="mi">3</span><span class="p">,</span><span class="nt">"bytes-added"</span><span class="p">:</span><span class="mi">4211</span><span class="p">,</span><span class="nt">"edits"</span><span class="p">:</span><span class="mi">30</span><span class="p">,</span><span class="nt">"unique-titles"</span><span class="p">:</span><span class="mi">30</span><span class="p">,</span><span class="nt">"is-unpatrolled"</span><span class="p">:</span><span class="mi">1</span><span class="p">,</span><span class="nt">"is-new"</span><span class="p">:</span><span class="mi">2</span><span class="p">,</span><span class="nt">"is-minor"</span><span class="p">:</span><span class="mi">7</span><span class="p">}</span> <span class="p">{</span><span class="nt">"bytes-added"</span><span class="p">:</span><span class="mi">3180</span><span class="p">,</span><span class="nt">"edits"</span><span class="p">:</span><span class="mi">19</span><span class="p">,</span><span class="nt">"unique-titles"</span><span class="p">:</span><span class="mi">19</span><span class="p">,</span><span class="nt">"is-unpatrolled"</span><span class="p">:</span><span class="mi">1</span><span class="p">,</span><span class="nt">"is-new"</span><span class="p">:</span><span class="mi">1</span><span class="p">,</span><span class="nt">"is-minor"</span><span class="p">:</span><span class="mi">3</span><span class="p">}</span> -<span class="p">{</span><span class="nt">"bytes-added"</span><span class="p">:</span><span class="mi">2218</span><span class="p">,</span><span class="nt">"edits"</span><span class="p">:</span><span class="mi">18</span><span class="p">,</span><span class="nt">"unique-titles"</span><span class="p">:</span><span class="mi">18</span><span class="p">,</span><span class="nt">"is-unpatrolled"</span><span class="p">:</span><span class="mi">2</span><span class="p">,</span><span class="nt">"is-new"</span><span class="p">:</span><span class="mi">2</span><span class="p">,</span><span class="nt">"is-minor"</span><span class="p">:</span><span class="mi">3</span><span class="p">}</span></code></pre></div> +<span class="p">{</span><span class="nt">"bytes-added"</span><span class="p">:</span><span class="mi">2218</span><span class="p">,</span><span class="nt">"edits"</span><span class="p">:</span><span class="mi">18</span><span class="p">,</span><span class="nt">"unique-titles"</span><span class="p">:</span><span class="mi">18</span><span class="p">,</span><span class="nt">"is-unpatrolled"</span><span class="p">:</span><span class="mi">2</span><span class="p">,</span><span class="nt">"is-new"</span><span class="p">:</span><span class="mi">2</span><span class="p">,</span><span class="nt">"is-minor"</span><span class="p">:</span><span class="mi">3</span><span class="p">}</span></code></pre></figure> <p>If you check the YARN UI, again, you’ll see that all three jobs are now listed.</p> @@ -227,11 +224,11 @@ deploy/samza/bin/run-app.sh --config-fac <p>To shutdown one of the jobs, use the same script with an extra ‘–operation=kill’ argument</p> -<div class="highlight"><pre><code class="bash">deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/wikipedia-feed.properties --operation<span class="o">=</span><span class="nb">kill</span></code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>deploy/samza/bin/run-app.sh --config-factory<span class="o">=</span>org.apache.samza.config.factories.PropertiesConfigFactory --config-path<span class="o">=</span>file://<span class="nv">$PWD</span>/deploy/samza/config/wikipedia-feed.properties --operation<span class="o">=</span><span class="nb">kill</span></code></pre></figure> <p>After you’re done, you can clean everything up using the same grid script.</p> -<div class="highlight"><pre><code class="bash">bin/grid stop all</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>bin/grid stop all</code></pre></figure> <p>Congratulations! You’ve now setup a local grid that includes YARN, Kafka, and ZooKeeper, and run a Samza job on it. Next up, check out the <a href="/learn/documentation/latest/introduction/background.html">Background</a> and <a href="/learn/documentation/latest/api/overview.html">API Overview</a> pages.</p> @@ -251,7 +248,7 @@ deploy/samza/bin/run-app.sh --config-fac <script> $( document ).ready(function() { - if ( $.fn.urlExists( "/startup/hello-samza/0.14/index.html" ) ) { + if ( $.fn.urlExists( "/startup/hello-samza/0.14/" ) ) { $("#switch-version-button").addClass("fa fa-history masthead-icon"); } });
