[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42711166 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -58,6 +58,10 @@ private void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42711131 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -95,41 +99,75 @@ public void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-149998301 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-149998334 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-15251 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-15256 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-15244 **[Test build #44083 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44083/consoleFull)** for PR 9203 at commit

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-14732 **[Test build #44083 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44083/consoleFull)** for PR 9203 at commit

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150016227 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread davies
GitHub user davies opened a pull request: https://github.com/apache/spark/pull/9203 [SPARK-11243] [SQL] output UnsafeRow from columnar cache You can merge this pull request into a Git repository by running: $ git pull https://github.com/davies/spark unsafe_cache

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150019732 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150019702 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150020994 **[Test build #44092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44092/consoleFull)** for PR 9203 at commit

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42684787 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -95,41 +99,75 @@ public void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42686848 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java --- @@ -618,6 +618,25 @@ public void writeTo(ByteBuffer buffer) {

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150016965 **[Test build #44089 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44089/consoleFull)** for PR 9203 at commit

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150016246 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42686721 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java --- @@ -618,6 +618,25 @@ public void writeTo(ByteBuffer buffer) {

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150034498 I'm really excited to see how much of a performance difference this makes when scanning string columns, since this could potentially provide a big perf. boost to the

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42687247 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -58,6 +58,10 @@ private void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150041762 **[Test build #44089 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44089/consoleFull)** for PR 9203 at commit

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150041882 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150041884 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42695693 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -95,41 +99,75 @@ public void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42695679 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java --- @@ -618,6 +618,25 @@ public void writeTo(ByteBuffer buffer)

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42687468 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -95,41 +99,75 @@ public void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42688421 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/columnar/GenerateColumnAccessor.scala --- @@ -20,18 +20,42 @@ package org.apache.spark.sql.columnar

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150038460 LGTM otherwise. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150044864 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150044863 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42692096 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -95,41 +99,75 @@ public void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42692044 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -58,6 +58,10 @@ private void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150044370 @JoshRosen If we only scan the cached string and access it (without adding a ConvertToUnsafe), this could be a little bit slower, because we copy the bytes into

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150044743 **[Test build #44092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44092/consoleFull)** for PR 9203 at commit

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150075430 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150075426 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150075109 **[Test build #44100 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44100/consoleFull)** for PR 9203 at commit

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150084996 I'm going to merge this first. @davies please take a look at @cloud-fan 's comments, and address them in follow-up prs if necessary. --- If your project is set up for

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9203 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42700611 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java --- @@ -64,29 +63,72 @@ public void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42701316 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java --- @@ -64,29 +63,72 @@ public void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42701494 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java --- @@ -64,29 +63,72 @@ public void setOffset(int

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42701674 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -58,6 +58,10 @@ private void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42702161 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -95,41 +99,75 @@ public void

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42702305 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala --- @@ -124,17 +124,10 @@ object

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150070715 **[Test build #44098 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44098/consoleFull)** for PR 9203 at commit

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150070861 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150070859 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/9203#discussion_r42703195 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala --- @@ -124,17 +124,10 @@ object

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150053054 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150053026 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150054848 **[Test build #44100 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44100/consoleFull)** for PR 9203 at commit

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150053373 **[Test build #44098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44098/consoleFull)** for PR 9203 at commit

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150053787 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-11243] [SQL] output UnsafeRow from colu...

2015-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9203#issuecomment-150053767 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not