[
https://issues.apache.org/jira/browse/SPARK-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14066465#comment-14066465
]
Denis Serduik edited comment on SPARK-1305 at 7/18/14 4:00 PM:
---------------------------------------------------------------
I'm interested in using this feature especially with SchemeRDD to be able cache
intermediate results.
Where I can find some code examples how to use it ?
>From SparkContext's code I see the following:
private[spark] def persistRDD(rdd: RDD[_]) {
persistentRdds(rdd.id) = rdd
}
/**
* Unpersist an RDD from memory and/or disk storage
*/
private[spark] def unpersistRDD(rddId: Int, blocking: Boolean = true) {
env.blockManager.master.removeRdd(rddId, blocking)
persistentRdds.remove(rddId)
listenerBus.post(SparkListenerUnpersistRDD(rddId))
}
So persist don't put RDD into tachyonStore of blockmanager ? How does this
feature work ?
was (Author: dmaverick):
I'm interesting in using this feature especially with SchemeRDD to be able
cache intermediate results.
Where I can find some code examples how to use it ?
>From SparkContext's code I see the following:
private[spark] def persistRDD(rdd: RDD[_]) {
persistentRdds(rdd.id) = rdd
}
/**
* Unpersist an RDD from memory and/or disk storage
*/
private[spark] def unpersistRDD(rddId: Int, blocking: Boolean = true) {
env.blockManager.master.removeRdd(rddId, blocking)
persistentRdds.remove(rddId)
listenerBus.post(SparkListenerUnpersistRDD(rddId))
}
So persist don't put RDD into tachyonStore of blockmanager ? How does this
feature work ?
> Support persisting RDD's directly to Tachyon
> --------------------------------------------
>
> Key: SPARK-1305
> URL: https://issues.apache.org/jira/browse/SPARK-1305
> Project: Spark
> Issue Type: New Feature
> Components: Block Manager
> Reporter: Patrick Wendell
> Assignee: Haoyuan Li
> Priority: Blocker
> Fix For: 1.0.0
>
>
> This is already an ongoing pull request - in a nutshell we want to support
> Tachyon as a storage level in Spark.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
