[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-09-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-54375719 @colorant Can you file a JIRA to describe what your'e trying to achieve here? Also, how is this related to #1209? If this is still relevant, can you up merge this to

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-09-03 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-54389553 @andrewor14, JIRA ticket is: https://issues.apache.org/jira/browse/SPARK-1957. And #1209 is about to make the data format more general for BM. which also modified a lot

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-09-03 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-54393598 Can you add it to the title? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-06-25 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-47070411 Hi @andrewor14 , other than #1209 , also this one is related to BM, could you also take a look on the general idea ? I know the code need a rebase to the latest code,

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-45967842 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-45968211 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-45970155 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15748/ --- If your project is set up for it, you can

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-06-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-45970153 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-06-03 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-45040787 @pwendell would you mind take look on this one ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-05-28 Thread colorant
GitHub user colorant opened a pull request: https://github.com/apache/spark/pull/907 Pluggable Diskstore for BlockManager You can merge this pull request into a Git repository by running: $ git pull https://github.com/colorant/spark diskstore Alternatively you can review and

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-05-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-44483647 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15271/ --- If your project is set up for it, you can

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-05-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-44483646 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-05-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-44486254 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-05-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-44486261 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-05-28 Thread witgo
Github user witgo commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-44487759 @colorant This is a big changes. Can you explain this change reason? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-05-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-44488108 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-05-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-44488109 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15274/ --- If your project

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-05-28 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-44495966 Hi @witgo This is aim to aligned with the spark pluggable storage roadmap. say support different storage media across different existing / emerging hardware. ( for now,

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-05-28 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-44496055 Hi, jira here: https://issues.apache.org/jira/browse/SPARK-1957 --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: Pluggable Diskstore for BlockManager

2014-05-28 Thread colorant
Github user colorant commented on the pull request: https://github.com/apache/spark/pull/907#issuecomment-44496929 And the implementation here is try to minimize the impact to current spark code base. Especially the shuffle part of code as described in the jira. which might be