spark git commit: [SPARK-6391][DOCS] Document Tachyon compatibility.
Repository: spark Updated Branches: refs/heads/branch-1.4 7edb17bf0 -> d5572ab79 [SPARK-6391][DOCS] Document Tachyon compatibility. Adds a section in the RDD persistence section of the programming-guide docs detailing Spark-Tachyon version compatibility as discussed in [[SPARK-6391]](https://issues.apache.org/jira/browse/SPARK-6391). Author: Calvin Jia Closes #6382 from calvinjia/spark-6391 and squashes the following commits: 113e863 [Calvin Jia] Move compatibility info to the offheap storage level section. 7942dc5 [Calvin Jia] Add a section in the programming-guide docs for Tachyon compatibility. (cherry picked from commit ce0051d6f7e4ed54076676644c78b52b527ba190) Signed-off-by: Reynold Xin Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d5572ab7 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d5572ab7 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d5572ab7 Branch: refs/heads/branch-1.4 Commit: d5572ab79ac48fdd28aada960391ed7bc79df8e6 Parents: 7edb17b Author: Calvin Jia Authored: Mon May 25 16:50:43 2015 -0700 Committer: Reynold Xin Committed: Mon May 25 16:50:50 2015 -0700 -- docs/programming-guide.md | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/spark/blob/d5572ab7/docs/programming-guide.md -- diff --git a/docs/programming-guide.md b/docs/programming-guide.md index 5d9df28..10f474f 100644 --- a/docs/programming-guide.md +++ b/docs/programming-guide.md @@ -1214,9 +1214,11 @@ storage levels is: Compared to MEMORY_ONLY_SER, OFF_HEAP reduces garbage collection overhead and allows executors to be smaller and to share a pool of memory, making it attractive in environments with large heaps or multiple concurrent applications. Furthermore, as the RDDs reside in Tachyon, -the crash of an executor does not lead to losing the in-memory cache. In this mode, the memory +the crash of an executor does not lead to losing the in-memory cache. In this mode, the memory in Tachyon is discardable. Thus, Tachyon does not attempt to reconstruct a block that it evicts -from memory. +from memory. If you plan to use Tachyon as the off heap store, Spark is compatible with Tachyon +out-of-the-box. Please refer to this http://tachyon-project.org/master/Running-Spark-on-Tachyon.html";>page +for the suggested version pairings. - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
spark git commit: [SPARK-6391][DOCS] Document Tachyon compatibility.
Repository: spark Updated Branches: refs/heads/master 8af1bf10b -> ce0051d6f [SPARK-6391][DOCS] Document Tachyon compatibility. Adds a section in the RDD persistence section of the programming-guide docs detailing Spark-Tachyon version compatibility as discussed in [[SPARK-6391]](https://issues.apache.org/jira/browse/SPARK-6391). Author: Calvin Jia Closes #6382 from calvinjia/spark-6391 and squashes the following commits: 113e863 [Calvin Jia] Move compatibility info to the offheap storage level section. 7942dc5 [Calvin Jia] Add a section in the programming-guide docs for Tachyon compatibility. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ce0051d6 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ce0051d6 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ce0051d6 Branch: refs/heads/master Commit: ce0051d6f7e4ed54076676644c78b52b527ba190 Parents: 8af1bf1 Author: Calvin Jia Authored: Mon May 25 16:50:43 2015 -0700 Committer: Reynold Xin Committed: Mon May 25 16:50:43 2015 -0700 -- docs/programming-guide.md | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/spark/blob/ce0051d6/docs/programming-guide.md -- diff --git a/docs/programming-guide.md b/docs/programming-guide.md index 5d9df28..10f474f 100644 --- a/docs/programming-guide.md +++ b/docs/programming-guide.md @@ -1214,9 +1214,11 @@ storage levels is: Compared to MEMORY_ONLY_SER, OFF_HEAP reduces garbage collection overhead and allows executors to be smaller and to share a pool of memory, making it attractive in environments with large heaps or multiple concurrent applications. Furthermore, as the RDDs reside in Tachyon, -the crash of an executor does not lead to losing the in-memory cache. In this mode, the memory +the crash of an executor does not lead to losing the in-memory cache. In this mode, the memory in Tachyon is discardable. Thus, Tachyon does not attempt to reconstruct a block that it evicts -from memory. +from memory. If you plan to use Tachyon as the off heap store, Spark is compatible with Tachyon +out-of-the-box. Please refer to this http://tachyon-project.org/master/Running-Spark-on-Tachyon.html";>page +for the suggested version pairings. - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
