Update 2.1.0 docs to include https://github.com/apache/spark/pull/16294
This version is built from the docs source code generated by applying https://github.com/apache/spark/pull/16294 to v2.1.0 (so, other changes in branch 2.1 will not affect the doc). Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/d2bcf185 Tree: http://git-wip-us.apache.org/repos/asf/spark-website/tree/d2bcf185 Diff: http://git-wip-us.apache.org/repos/asf/spark-website/diff/d2bcf185 Branch: refs/heads/asf-site Commit: d2bcf1854b0e0409495e2f1d3c6beaad923f6e6b Parents: ecf94f2 Author: Yin Huai <yh...@databricks.com> Authored: Wed Dec 28 14:32:43 2016 -0800 Committer: Yin Huai <yh...@databricks.com> Committed: Wed Dec 28 14:32:43 2016 -0800 ---------------------------------------------------------------------- site/docs/2.1.0/building-spark.html | 46 +- site/docs/2.1.0/building-with-maven.html | 14 +- site/docs/2.1.0/configuration.html | 52 +- site/docs/2.1.0/ec2-scripts.html | 174 ++++ site/docs/2.1.0/graphx-programming-guide.html | 198 ++--- site/docs/2.1.0/hadoop-provided.html | 14 +- .../img/structured-streaming-watermark.png | Bin 0 -> 252000 bytes site/docs/2.1.0/img/structured-streaming.pptx | Bin 1105413 -> 1113902 bytes site/docs/2.1.0/job-scheduling.html | 40 +- site/docs/2.1.0/ml-advanced.html | 10 +- .../2.1.0/ml-classification-regression.html | 838 +++++++++--------- site/docs/2.1.0/ml-clustering.html | 124 +-- site/docs/2.1.0/ml-collaborative-filtering.html | 56 +- site/docs/2.1.0/ml-features.html | 764 ++++++++-------- site/docs/2.1.0/ml-migration-guides.html | 16 +- site/docs/2.1.0/ml-pipeline.html | 178 ++-- site/docs/2.1.0/ml-tuning.html | 172 ++-- site/docs/2.1.0/mllib-clustering.html | 186 ++-- .../2.1.0/mllib-collaborative-filtering.html | 48 +- site/docs/2.1.0/mllib-data-types.html | 208 ++--- site/docs/2.1.0/mllib-decision-tree.html | 94 +- .../2.1.0/mllib-dimensionality-reduction.html | 28 +- site/docs/2.1.0/mllib-ensembles.html | 182 ++-- site/docs/2.1.0/mllib-evaluation-metrics.html | 302 +++---- site/docs/2.1.0/mllib-feature-extraction.html | 122 +-- .../2.1.0/mllib-frequent-pattern-mining.html | 28 +- site/docs/2.1.0/mllib-isotonic-regression.html | 38 +- site/docs/2.1.0/mllib-linear-methods.html | 174 ++-- site/docs/2.1.0/mllib-naive-bayes.html | 24 +- site/docs/2.1.0/mllib-optimization.html | 50 +- site/docs/2.1.0/mllib-pmml-model-export.html | 35 +- site/docs/2.1.0/mllib-statistics.html | 180 ++-- site/docs/2.1.0/programming-guide.html | 302 +++---- site/docs/2.1.0/quick-start.html | 166 ++-- site/docs/2.1.0/running-on-mesos.html | 52 +- site/docs/2.1.0/running-on-yarn.html | 27 +- site/docs/2.1.0/spark-standalone.html | 30 +- site/docs/2.1.0/sparkr.html | 145 ++-- site/docs/2.1.0/sql-programming-guide.html | 819 +++++++++--------- site/docs/2.1.0/storage-openstack-swift.html | 12 +- site/docs/2.1.0/streaming-custom-receivers.html | 26 +- .../2.1.0/streaming-kafka-0-10-integration.html | 52 +- .../docs/2.1.0/streaming-programming-guide.html | 416 ++++----- .../structured-streaming-kafka-integration.html | 44 +- .../structured-streaming-programming-guide.html | 864 ++++++++++++------- site/docs/2.1.0/submitting-applications.html | 36 +- site/docs/2.1.0/tuning.html | 30 +- 47 files changed, 3926 insertions(+), 3490 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark-website/blob/d2bcf185/site/docs/2.1.0/building-spark.html ---------------------------------------------------------------------- diff --git a/site/docs/2.1.0/building-spark.html b/site/docs/2.1.0/building-spark.html index b3a720c..5c20245 100644 --- a/site/docs/2.1.0/building-spark.html +++ b/site/docs/2.1.0/building-spark.html @@ -127,33 +127,33 @@ <ul id="markdown-toc"> - <li><a href="#building-apache-spark" id="markdown-toc-building-apache-spark">Building Apache Spark</a> <ul> - <li><a href="#apache-maven" id="markdown-toc-apache-maven">Apache Maven</a> <ul> - <li><a href="#setting-up-mavens-memory-usage" id="markdown-toc-setting-up-mavens-memory-usage">Setting up Maven’s Memory Usage</a></li> - <li><a href="#buildmvn" id="markdown-toc-buildmvn">build/mvn</a></li> + <li><a href="#building-apache-spark">Building Apache Spark</a> <ul> + <li><a href="#apache-maven">Apache Maven</a> <ul> + <li><a href="#setting-up-mavens-memory-usage">Setting up Maven’s Memory Usage</a></li> + <li><a href="#buildmvn">build/mvn</a></li> </ul> </li> - <li><a href="#building-a-runnable-distribution" id="markdown-toc-building-a-runnable-distribution">Building a Runnable Distribution</a></li> - <li><a href="#specifying-the-hadoop-version" id="markdown-toc-specifying-the-hadoop-version">Specifying the Hadoop Version</a></li> - <li><a href="#building-with-hive-and-jdbc-support" id="markdown-toc-building-with-hive-and-jdbc-support">Building With Hive and JDBC Support</a></li> - <li><a href="#packaging-without-hadoop-dependencies-for-yarn" id="markdown-toc-packaging-without-hadoop-dependencies-for-yarn">Packaging without Hadoop Dependencies for YARN</a></li> - <li><a href="#building-with-mesos-support" id="markdown-toc-building-with-mesos-support">Building with Mesos support</a></li> - <li><a href="#building-for-scala-210" id="markdown-toc-building-for-scala-210">Building for Scala 2.10</a></li> - <li><a href="#building-submodules-individually" id="markdown-toc-building-submodules-individually">Building submodules individually</a></li> - <li><a href="#continuous-compilation" id="markdown-toc-continuous-compilation">Continuous Compilation</a></li> - <li><a href="#speeding-up-compilation-with-zinc" id="markdown-toc-speeding-up-compilation-with-zinc">Speeding up Compilation with Zinc</a></li> - <li><a href="#building-with-sbt" id="markdown-toc-building-with-sbt">Building with SBT</a></li> - <li><a href="#encrypted-filesystems" id="markdown-toc-encrypted-filesystems"> Encrypted Filesystems</a></li> - <li><a href="#intellij-idea-or-eclipse" id="markdown-toc-intellij-idea-or-eclipse">IntelliJ IDEA or Eclipse</a></li> + <li><a href="#building-a-runnable-distribution">Building a Runnable Distribution</a></li> + <li><a href="#specifying-the-hadoop-version">Specifying the Hadoop Version</a></li> + <li><a href="#building-with-hive-and-jdbc-support">Building With Hive and JDBC Support</a></li> + <li><a href="#packaging-without-hadoop-dependencies-for-yarn">Packaging without Hadoop Dependencies for YARN</a></li> + <li><a href="#building-with-mesos-support">Building with Mesos support</a></li> + <li><a href="#building-for-scala-210">Building for Scala 2.10</a></li> + <li><a href="#building-submodules-individually">Building submodules individually</a></li> + <li><a href="#continuous-compilation">Continuous Compilation</a></li> + <li><a href="#speeding-up-compilation-with-zinc">Speeding up Compilation with Zinc</a></li> + <li><a href="#building-with-sbt">Building with SBT</a></li> + <li><a href="#encrypted-filesystems"> Encrypted Filesystems</a></li> + <li><a href="#intellij-idea-or-eclipse">IntelliJ IDEA or Eclipse</a></li> </ul> </li> - <li><a href="#running-tests" id="markdown-toc-running-tests">Running Tests</a> <ul> - <li><a href="#testing-with-sbt" id="markdown-toc-testing-with-sbt">Testing with SBT</a></li> - <li><a href="#running-java-8-test-suites" id="markdown-toc-running-java-8-test-suites">Running Java 8 Test Suites</a></li> - <li><a href="#pyspark-pip-installable" id="markdown-toc-pyspark-pip-installable">PySpark pip installable</a></li> - <li><a href="#pyspark-tests-with-maven" id="markdown-toc-pyspark-tests-with-maven">PySpark Tests with Maven</a></li> - <li><a href="#running-r-tests" id="markdown-toc-running-r-tests">Running R Tests</a></li> - <li><a href="#running-docker-based-integration-test-suites" id="markdown-toc-running-docker-based-integration-test-suites">Running Docker-based Integration Test Suites</a></li> + <li><a href="#running-tests">Running Tests</a> <ul> + <li><a href="#testing-with-sbt">Testing with SBT</a></li> + <li><a href="#running-java-8-test-suites">Running Java 8 Test Suites</a></li> + <li><a href="#pyspark-pip-installable">PySpark pip installable</a></li> + <li><a href="#pyspark-tests-with-maven">PySpark Tests with Maven</a></li> + <li><a href="#running-r-tests">Running R Tests</a></li> + <li><a href="#running-docker-based-integration-test-suites">Running Docker-based Integration Test Suites</a></li> </ul> </li> </ul> http://git-wip-us.apache.org/repos/asf/spark-website/blob/d2bcf185/site/docs/2.1.0/building-with-maven.html ---------------------------------------------------------------------- diff --git a/site/docs/2.1.0/building-with-maven.html b/site/docs/2.1.0/building-with-maven.html index dea0259..0aafc64 100644 --- a/site/docs/2.1.0/building-with-maven.html +++ b/site/docs/2.1.0/building-with-maven.html @@ -1,8 +1,10 @@ <!DOCTYPE html> -<meta charset=utf-8> -<title>Redirecting...</title> -<link rel=canonical href="/building-spark.html"> -<meta http-equiv=refresh content="0; url=/building-spark.html"> -<h1>Redirecting...</h1> +<html lang="en-US"> +<meta charset="utf-8"> +<title>Redirectingâ¦</title> +<link rel="canonical" href="/building-spark.html"> +<meta http-equiv="refresh" content="0; url=/building-spark.html"> +<h1>Redirectingâ¦</h1> <a href="/building-spark.html">Click here if you are not redirected.</a> -<script>location='/building-spark.html'</script> +<script>location="/building-spark.html"</script> +</html> http://git-wip-us.apache.org/repos/asf/spark-website/blob/d2bcf185/site/docs/2.1.0/configuration.html ---------------------------------------------------------------------- diff --git a/site/docs/2.1.0/configuration.html b/site/docs/2.1.0/configuration.html index 8336326..2adc859 100644 --- a/site/docs/2.1.0/configuration.html +++ b/site/docs/2.1.0/configuration.html @@ -127,19 +127,19 @@ <ul id="markdown-toc"> - <li><a href="#spark-properties" id="markdown-toc-spark-properties">Spark Properties</a> <ul> - <li><a href="#dynamically-loading-spark-properties" id="markdown-toc-dynamically-loading-spark-properties">Dynamically Loading Spark Properties</a></li> - <li><a href="#viewing-spark-properties" id="markdown-toc-viewing-spark-properties">Viewing Spark Properties</a></li> - <li><a href="#available-properties" id="markdown-toc-available-properties">Available Properties</a> <ul> - <li><a href="#application-properties" id="markdown-toc-application-properties">Application Properties</a></li> - <li><a href="#runtime-environment" id="markdown-toc-runtime-environment">Runtime Environment</a></li> - <li><a href="#shuffle-behavior" id="markdown-toc-shuffle-behavior">Shuffle Behavior</a></li> - <li><a href="#spark-ui" id="markdown-toc-spark-ui">Spark UI</a></li> - <li><a href="#compression-and-serialization" id="markdown-toc-compression-and-serialization">Compression and Serialization</a></li> - <li><a href="#memory-management" id="markdown-toc-memory-management">Memory Management</a></li> - <li><a href="#execution-behavior" id="markdown-toc-execution-behavior">Execution Behavior</a></li> - <li><a href="#networking" id="markdown-toc-networking">Networking</a></li> - <li><a href="#scheduling" id="markdown-toc-scheduling">Scheduling</a></li> + <li><a href="#spark-properties">Spark Properties</a> <ul> + <li><a href="#dynamically-loading-spark-properties">Dynamically Loading Spark Properties</a></li> + <li><a href="#viewing-spark-properties">Viewing Spark Properties</a></li> + <li><a href="#available-properties">Available Properties</a> <ul> + <li><a href="#application-properties">Application Properties</a></li> + <li><a href="#runtime-environment">Runtime Environment</a></li> + <li><a href="#shuffle-behavior">Shuffle Behavior</a></li> + <li><a href="#spark-ui">Spark UI</a></li> + <li><a href="#compression-and-serialization">Compression and Serialization</a></li> + <li><a href="#memory-management">Memory Management</a></li> + <li><a href="#execution-behavior">Execution Behavior</a></li> + <li><a href="#networking">Networking</a></li> + <li><a href="#scheduling">Scheduling</a></li> </ul> </li> </ul> @@ -169,10 +169,10 @@ application. These properties can be set directly on a <p>Note that we run with local[2], meaning two threads - which represents “minimal” parallelism, which can help detect bugs that only exist when we run in a distributed context.</p> -<div class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="k">val</span> <span class="n">conf</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">SparkConf</span><span class="o">()</span> +<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span></span><span class="k">val</span> <span class="n">conf</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">SparkConf</span><span class="o">()</span> <span class="o">.</span><span class="n">setMaster</span><span class="o">(</span><span class="s">"local[2]"</span><span class="o">)</span> <span class="o">.</span><span class="n">setAppName</span><span class="o">(</span><span class="s">"CountingSheep"</span><span class="o">)</span> -<span class="k">val</span> <span class="n">sc</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">SparkContext</span><span class="o">(</span><span class="n">conf</span><span class="o">)</span></code></pre></div> +<span class="k">val</span> <span class="n">sc</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">SparkContext</span><span class="o">(</span><span class="n">conf</span><span class="o">)</span></code></pre></figure> <p>Note that we can have more than 1 thread in local mode, and in cases like Spark Streaming, we may actually require more than 1 thread to prevent any sort of starvation issues.</p> @@ -204,12 +204,12 @@ The following format is accepted:</p> instance, if you’d like to run the same application with different masters or different amounts of memory. Spark allows you to simply create an empty conf:</p> -<div class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="k">val</span> <span class="n">sc</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">SparkContext</span><span class="o">(</span><span class="k">new</span> <span class="nc">SparkConf</span><span class="o">())</span></code></pre></div> +<figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span></span><span class="k">val</span> <span class="n">sc</span> <span class="k">=</span> <span class="k">new</span> <span class="nc">SparkContext</span><span class="o">(</span><span class="k">new</span> <span class="nc">SparkConf</span><span class="o">())</span></code></pre></figure> <p>Then, you can supply configuration values at runtime:</p> -<div class="highlight"><pre><code class="language-bash" data-lang="bash">./bin/spark-submit --name <span class="s2">"My app"</span> --master <span class="nb">local</span><span class="o">[</span>4<span class="o">]</span> --conf spark.eventLog.enabled<span class="o">=</span><span class="nb">false</span> - --conf <span class="s2">"spark.executor.extraJavaOptions=-XX:+PrintGCDetails -XX:+PrintGCTimeStamps"</span> myApp.jar</code></pre></div> +<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span></span>./bin/spark-submit --name <span class="s2">"My app"</span> --master local<span class="o">[</span><span class="m">4</span><span class="o">]</span> --conf spark.eventLog.enabled<span class="o">=</span><span class="nb">false</span> + --conf <span class="s2">"spark.executor.extraJavaOptions=-XX:+PrintGCDetails -XX:+PrintGCTimeStamps"</span> myApp.jar</code></pre></figure> <p>The Spark shell and <a href="submitting-applications.html"><code>spark-submit</code></a> tool support two ways to load configurations dynamically. The first are command line options, @@ -1856,30 +1856,30 @@ Running the <code>SET -v</code> command will show the entire list of the SQL con <div class="codetabs"> <div data-lang="scala"> - <div class="highlight"><pre><code class="language-scala" data-lang="scala"><span class="c1">// spark is an existing SparkSession</span> -<span class="n">spark</span><span class="o">.</span><span class="n">sql</span><span class="o">(</span><span class="s">"SET -v"</span><span class="o">).</span><span class="n">show</span><span class="o">(</span><span class="n">numRows</span> <span class="k">=</span> <span class="mi">200</span><span class="o">,</span> <span class="n">truncate</span> <span class="k">=</span> <span class="kc">false</span><span class="o">)</span></code></pre></div> + <figure class="highlight"><pre><code class="language-scala" data-lang="scala"><span></span><span class="c1">// spark is an existing SparkSession</span> +<span class="n">spark</span><span class="o">.</span><span class="n">sql</span><span class="o">(</span><span class="s">"SET -v"</span><span class="o">).</span><span class="n">show</span><span class="o">(</span><span class="n">numRows</span> <span class="k">=</span> <span class="mi">200</span><span class="o">,</span> <span class="n">truncate</span> <span class="k">=</span> <span class="kc">false</span><span class="o">)</span></code></pre></figure> </div> <div data-lang="java"> - <div class="highlight"><pre><code class="language-java" data-lang="java"><span class="c1">// spark is an existing SparkSession</span> -<span class="n">spark</span><span class="o">.</span><span class="na">sql</span><span class="o">(</span><span class="s">"SET -v"</span><span class="o">).</span><span class="na">show</span><span class="o">(</span><span class="mi">200</span><span class="o">,</span> <span class="kc">false</span><span class="o">);</span></code></pre></div> + <figure class="highlight"><pre><code class="language-java" data-lang="java"><span></span><span class="c1">// spark is an existing SparkSession</span> +<span class="n">spark</span><span class="o">.</span><span class="na">sql</span><span class="o">(</span><span class="s">"SET -v"</span><span class="o">).</span><span class="na">show</span><span class="o">(</span><span class="mi">200</span><span class="o">,</span> <span class="kc">false</span><span class="o">);</span></code></pre></figure> </div> <div data-lang="python"> - <div class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># spark is an existing SparkSession</span> -<span class="n">spark</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="s">"SET -v"</span><span class="p">)</span><span class="o">.</span><span class="n">show</span><span class="p">(</span><span class="n">n</span><span class="o">=</span><span class="mi">200</span><span class="p">,</span> <span class="n">truncate</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span></code></pre></div> + <figure class="highlight"><pre><code class="language-python" data-lang="python"><span></span><span class="c1"># spark is an existing SparkSession</span> +<span class="n">spark</span><span class="o">.</span><span class="n">sql</span><span class="p">(</span><span class="s2">"SET -v"</span><span class="p">)</span><span class="o">.</span><span class="n">show</span><span class="p">(</span><span class="n">n</span><span class="o">=</span><span class="mi">200</span><span class="p">,</span> <span class="n">truncate</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span></code></pre></figure> </div> <div data-lang="r"> - <div class="highlight"><pre><code class="language-r" data-lang="r">sparkR.session<span class="p">()</span> + <figure class="highlight"><pre><code class="language-r" data-lang="r"><span></span>sparkR.session<span class="p">()</span> properties <span class="o"><-</span> sql<span class="p">(</span><span class="s">"SET -v"</span><span class="p">)</span> -showDF<span class="p">(</span>properties<span class="p">,</span> numRows <span class="o">=</span> <span class="m">200</span><span class="p">,</span> truncate <span class="o">=</span> <span class="kc">FALSE</span><span class="p">)</span></code></pre></div> +showDF<span class="p">(</span>properties<span class="p">,</span> numRows <span class="o">=</span> <span class="m">200</span><span class="p">,</span> truncate <span class="o">=</span> <span class="kc">FALSE</span><span class="p">)</span></code></pre></figure> </div> </div> http://git-wip-us.apache.org/repos/asf/spark-website/blob/d2bcf185/site/docs/2.1.0/ec2-scripts.html ---------------------------------------------------------------------- diff --git a/site/docs/2.1.0/ec2-scripts.html b/site/docs/2.1.0/ec2-scripts.html new file mode 100644 index 0000000..eb1bb60 --- /dev/null +++ b/site/docs/2.1.0/ec2-scripts.html @@ -0,0 +1,174 @@ + +<!DOCTYPE html> +<!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]--> +<!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]--> +<!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]--> +<!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]--> + <head> + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"> + <title>Running Spark on EC2 - Spark 2.1.0 Documentation</title> + + + + <meta http-equiv="refresh" content="0; url=https://github.com/amplab/spark-ec2#readme"> + <link rel="canonical" href="https://github.com/amplab/spark-ec2#readme" /> + + + <link rel="stylesheet" href="css/bootstrap.min.css"> + <style> + body { + padding-top: 60px; + padding-bottom: 40px; + } + </style> + <meta name="viewport" content="width=device-width"> + <link rel="stylesheet" href="css/bootstrap-responsive.min.css"> + <link rel="stylesheet" href="css/main.css"> + + <script src="js/vendor/modernizr-2.6.1-respond-1.1.0.min.js"></script> + + <link rel="stylesheet" href="css/pygments-default.css"> + + + <!-- Google analytics script --> + <script type="text/javascript"> + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-32518208-2']); + _gaq.push(['_trackPageview']); + + (function() { + var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); + })(); + </script> + + + </head> + <body> + <!--[if lt IE 7]> + <p class="chromeframe">You are using an outdated browser. <a href="http://browsehappy.com/">Upgrade your browser today</a> or <a href="http://www.google.com/chromeframe/?redirect=true">install Google Chrome Frame</a> to better experience this site.</p> + <![endif]--> + + <!-- This code is taken from http://twitter.github.com/bootstrap/examples/hero.html --> + + <div class="navbar navbar-fixed-top" id="topbar"> + <div class="navbar-inner"> + <div class="container"> + <div class="brand"><a href="index.html"> + <img src="img/spark-logo-hd.png" style="height:50px;"/></a><span class="version">2.1.0</span> + </div> + <ul class="nav"> + <!--TODO(andyk): Add class="active" attribute to li some how.--> + <li><a href="index.html">Overview</a></li> + + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown">Programming Guides<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="quick-start.html">Quick Start</a></li> + <li><a href="programming-guide.html">Spark Programming Guide</a></li> + <li class="divider"></li> + <li><a href="streaming-programming-guide.html">Spark Streaming</a></li> + <li><a href="sql-programming-guide.html">DataFrames, Datasets and SQL</a></li> + <li><a href="structured-streaming-programming-guide.html">Structured Streaming</a></li> + <li><a href="ml-guide.html">MLlib (Machine Learning)</a></li> + <li><a href="graphx-programming-guide.html">GraphX (Graph Processing)</a></li> + <li><a href="sparkr.html">SparkR (R on Spark)</a></li> + </ul> + </li> + + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown">API Docs<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="api/scala/index.html#org.apache.spark.package">Scala</a></li> + <li><a href="api/java/index.html">Java</a></li> + <li><a href="api/python/index.html">Python</a></li> + <li><a href="api/R/index.html">R</a></li> + </ul> + </li> + + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown">Deploying<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="cluster-overview.html">Overview</a></li> + <li><a href="submitting-applications.html">Submitting Applications</a></li> + <li class="divider"></li> + <li><a href="spark-standalone.html">Spark Standalone</a></li> + <li><a href="running-on-mesos.html">Mesos</a></li> + <li><a href="running-on-yarn.html">YARN</a></li> + </ul> + </li> + + <li class="dropdown"> + <a href="api.html" class="dropdown-toggle" data-toggle="dropdown">More<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="configuration.html">Configuration</a></li> + <li><a href="monitoring.html">Monitoring</a></li> + <li><a href="tuning.html">Tuning Guide</a></li> + <li><a href="job-scheduling.html">Job Scheduling</a></li> + <li><a href="security.html">Security</a></li> + <li><a href="hardware-provisioning.html">Hardware Provisioning</a></li> + <li class="divider"></li> + <li><a href="building-spark.html">Building Spark</a></li> + <li><a href="http://spark.apache.org/contributing.html">Contributing to Spark</a></li> + <li><a href="http://spark.apache.org/third-party-projects.html">Third Party Projects</a></li> + </ul> + </li> + </ul> + <!--<p class="navbar-text pull-right"><span class="version-text">v2.1.0</span></p>--> + </div> + </div> + </div> + + <div class="container-wrapper"> + + + <div class="content" id="content"> + + <h1 class="title">Running Spark on EC2</h1> + + + <p>This document has been superseded and replaced by documentation at https://github.com/amplab/spark-ec2#readme</p> + + + </div> + + <!-- /container --> + </div> + + <script src="js/vendor/jquery-1.8.0.min.js"></script> + <script src="js/vendor/bootstrap.min.js"></script> + <script src="js/vendor/anchor.min.js"></script> + <script src="js/main.js"></script> + + <!-- MathJax Section --> + <script type="text/x-mathjax-config"> + MathJax.Hub.Config({ + TeX: { equationNumbers: { autoNumber: "AMS" } } + }); + </script> + <script> + // Note that we load MathJax this way to work with local file (file://), HTTP and HTTPS. + // We could use "//cdn.mathjax...", but that won't support "file://". + (function(d, script) { + script = d.createElement('script'); + script.type = 'text/javascript'; + script.async = true; + script.onload = function(){ + MathJax.Hub.Config({ + tex2jax: { + inlineMath: [ ["$", "$"], ["\\\\(","\\\\)"] ], + displayMath: [ ["$$","$$"], ["\\[", "\\]"] ], + processEscapes: true, + skipTags: ['script', 'noscript', 'style', 'textarea', 'pre'] + } + }); + }; + script.src = ('https:' == document.location.protocol ? 'https://' : 'http://') + + 'cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'; + d.getElementsByTagName('head')[0].appendChild(script); + }(document)); + </script> + </body> +</html> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org