[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-29 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-203076036 ttp --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202696168 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202696162 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202696165 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202696161 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202695926 **[Test build #54400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54400/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202695991 **[Test build #54401 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54401/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202675749 **[Test build #54404 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54404/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202675826 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202675819 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202675332 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202675334 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202675186 **[Test build #54388 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54388/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202671868 Alright, this should now be ready for an initial review pass. I just pushed a somewhat hacky patch to work around the direct buffer allocation limit (42d0356) and

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202671881 /cc @andrewor14 @davies for review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202671558 **[Test build #54404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54404/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202657707 **[Test build #54401 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54401/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202651594 **[Test build #54400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54400/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202640177 **[Test build #54388 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54388/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-202616817 Discovered a potentially major problem: if we bail out of a getOrElseUpdate call and don't end up caching a block and then do not fully consume or dispose of the

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201984448 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201984454 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201984028 **[Test build #54272 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54272/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201971414 **[Test build #54272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54272/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201676097 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201676104 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201675517 **[Test build #54239 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54239/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-25 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201617490 Looks pretty good actually. There are some details about naming and on/off defaults we need to figure out, but that can go in later. --- If your project is set up for

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201601318 **[Test build #54239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54239/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-25 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201586342 /cc @andrewor14, @nongli, @rxin, can you glance over the PR description to check whether the user-facing semantics are okay for now? We might want to change some

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201569804 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201569812 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201569358 **[Test build #54215 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54215/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201506253 **[Test build #54215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54215/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201113965 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201113964 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201113647 **[Test build #54125 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54125/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201086372 **[Test build #54125 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54125/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201073322 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201073319 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201073194 **[Test build #54096 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54096/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-201037802 **[Test build #54096 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54096/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-200081984 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-200081981 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-200081722 **[Test build #53825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53825/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-200047288 **[Test build #53825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53825/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-200037304 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-200037294 **[Test build #53819 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53819/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-200037299 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-200036926 **[Test build #53819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53819/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-198142114 **[Test build #53483 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53483/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-198141906 **[Test build #53483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53483/consoleFull)** for PR 11805 at commit

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-198142116 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-19 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/11805 [SPARK-13992][WIP] Add support for off-heap caching This patch adds support for caching blocks in the executor processes using direct / off-heap memory. I'll expand the description later; this

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-198142117 Test FAILed. Refer to this link for build results (access rights to CI server needed):