[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10001 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-160034260 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-160034258 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-160034212 **[Test build #46791 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46791/consoleFull)** for PR 10001 at commit [`4de7697`](https://g

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-160017383 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-160017247 **[Test build #46791 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46791/consoleFull)** for PR 10001 at commit [`4de7697`](https://gi

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/10001#discussion_r46012580 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -606,9 +606,17 @@ abstract class HadoopFsRelation private[sql](

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/10001#discussion_r46011770 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -606,9 +606,17 @@ abstract class HadoopFsRelation private[sql](

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/10001#discussion_r46011485 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -606,9 +606,17 @@ abstract class HadoopFsRelation private[sql](

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-160011568 @yhuai Yeah.. this function discoverPartitions() was changed recently (10/22/2015) as part of spark-9735. --- If your project is set up for it, you can reply to t

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-160011060 @dilipbiswal I guess I did not ask my question clearly. I meant why 1.5 is good but 1.6 is broken. There must be a change that exposed this issue. --- If your project is

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/10001#discussion_r46011037 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -606,9 +606,17 @@ abstract class HadoopFsRelation private[sql](

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-15341 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-15342 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-15278 **[Test build #46777 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46777/consoleFull)** for PR 10001 at commit [`af508de`](https://g

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-159998104 @yhuai Hi Yin, in the discoverPartitions method, we are trying to create the partition spec and are trying to cast a partition value to the corresponding user speci

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-159988308 @dilipbiswal Do you know why it is broken? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-159987537 **[Test build #46777 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46777/consoleFull)** for PR 10001 at commit [`af508de`](https://gi

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-159986618 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-159976397 @dilipbiswal , yours is the first PR after PR #1 ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-159877965 @davies Can you please let me know your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10001#issuecomment-159877629 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-11997][SQL] NPE when save a DataFrame a...

2015-11-26 Thread dilipbiswal
GitHub user dilipbiswal opened a pull request: https://github.com/apache/spark/pull/10001 [SPARK-11997][SQL] NPE when save a DataFrame as parquet and partitioned by long column Check for partition column null-ability while building the partition spec. You can merge this pull reque