Repository: spark Updated Branches: refs/heads/master cdd53b762 -> 0d7aac99d
[SPARK-9641] [DOCS] spark.shuffle.service.port is not documented Document spark.shuffle.service.{enabled,port} CC sryza tgravescs This is pretty minimal; is there more to say here about the service? Author: Sean Owen <so...@cloudera.com> Closes #7991 from srowen/SPARK-9641 and squashes the following commits: 3bb946e [Sean Owen] Add link to docs for setup and config of external shuffle service 2302e01 [Sean Owen] Document spark.shuffle.service.{enabled,port} Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0d7aac99 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/0d7aac99 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/0d7aac99 Branch: refs/heads/master Commit: 0d7aac99da660cc42eb5a9be8e262bd9bd8a770f Parents: cdd53b7 Author: Sean Owen <so...@cloudera.com> Authored: Thu Aug 6 19:29:42 2015 +0100 Committer: Sean Owen <so...@cloudera.com> Committed: Thu Aug 6 19:29:42 2015 +0100 ---------------------------------------------------------------------- docs/configuration.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/0d7aac99/docs/configuration.md ---------------------------------------------------------------------- diff --git a/docs/configuration.md b/docs/configuration.md index 24b6063..c60dd16 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -474,6 +474,25 @@ Apart from these, the following properties are also available, and may be useful </td> </tr> <tr> + <td><code>spark.shuffle.service.enabled</code></td> + <td>false</td> + <td> + Enables the external shuffle service. This service preserves the shuffle files written by + executors so the executors can be safely removed. This must be enabled if + <code>spark.dynamicAllocation.enabled</code> is "true". The external shuffle service + must be set up in order to enable it. See + <a href="job-scheduling.html#configuration-and-setup">dynamic allocation + configuration and setup documentation</a> for more information. + </td> +</tr> +<tr> + <td><code>spark.shuffle.service.port</code></td> + <td>7337</td> + <td> + Port on which the external shuffle service will run. + </td> +</tr> +<tr> <td><code>spark.shuffle.sort.bypassMergeThreshold</code></td> <td>200</td> <td> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org