[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13569 Merging in master/2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #3073 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3073/consoleFull)** for PR 13569 at commit

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #3073 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3073/consoleFull)** for PR 13569 at commit

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13569 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60255/ Test PASSed. ---

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #60255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60255/consoleFull)** for PR 13569 at commit

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13569 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60258/ Test FAILed. ---

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13569 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #60258 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60258/consoleFull)** for PR 13569 at commit

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13569 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60252/ Test PASSed. ---

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #60252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60252/consoleFull)** for PR 13569 at commit

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13569 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #60258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60258/consoleFull)** for PR 13569 at commit

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/13569 Done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #60255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60255/consoleFull)** for PR 13569 at commit

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13569 LGTM other than one last comment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/13569 Also added an explicit test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #60252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60252/consoleFull)** for PR 13569 at commit

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/13569 Ok I figured it out, the NPE was trigged by calling `.rdd` on the result DataFrame. This was easily reproducible in unit tests, so I added `.rdd.count()` to all QueryPlan tests to verify this issue

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/13569 Yeah I'm looking into how to reproduce this in a unit test. It seems we have to go through `DeserializeToObjectExec` to reproduce the npe: ``` Execution 'q9-v1.4' failed: Job aborted due

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-09 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/13569 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-08 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13569 Can you try create a unit test? Maybe it's related to ConvertToLocalRelation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-08 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/13569 It's possible this was going through the local execution short-cut? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13569 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60205/ Test PASSed. ---

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13569 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #60205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60205/consoleFull)** for PR 13569 at commit

[GitHub] spark issue #13569: [SPARK-15791] Fix NPE in ScalarSubquery

2016-06-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13569 **[Test build #60205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60205/consoleFull)** for PR 13569 at commit