[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8068#issuecomment-129734537 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8068#issuecomment-129734511 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/8068#discussion_r36713622 --- Diff: core/src/main/java/org/apache/spark/util/collection/TimSort.java --- @@ -914,7 +915,7 @@ private void mergeHi(int base1, int len1, int base2, int le

[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-10 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/8068#discussion_r36711245 --- Diff: core/src/main/java/org/apache/spark/util/collection/TimSort.java --- @@ -914,7 +915,7 @@ private void mergeHi(int base1, int len1, int base2, int le

[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-10 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/8068#issuecomment-129690243 Yes. I will update this later. Because `TimSort` does allocation via `SortDataFormat`, I will implement corresponding freeing in it. I am not sure if we want to modify `T

[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-10 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/8068#issuecomment-129605412 I think the problem here is that TimSort does allocations under the hood, and it doesn't support any explicit freeing. The proper solutions is probably to create a speciali

[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8068#issuecomment-129584352 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8068#issuecomment-129584133 **[Test build #40292 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40292/console)** for PR 8068 at commit [`732b588`](https://github

[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8068#issuecomment-129526928 [Test build #40292 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/40292/consoleFull) for PR 8068 at commit [`732b588`](https://gith

[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8068#issuecomment-129525262 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8068#issuecomment-129462165 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-7542][SQL][WIP] Support off-heap buffer...

2015-08-10 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/8068 [SPARK-7542][SQL][WIP] Support off-heap buffer in UnsafeInMemorySorter JIRA: https://issues.apache.org/jira/browse/SPARK-7542 This patch tries to add off-heap sort buffer in `UnsafeInMemorySo