[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-218415457 I am closing this due to not answering which I guess means not decided or not worth to doing this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/12629 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-218360303 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58321/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-218360301 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-218360205 **[Test build #58321 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58321/consoleFull)** for PR 12629 at commit [`ef4e6be`](https://github.com/apache/spark/commit/ef4e6be908a08840884e77f0ad0620f79026b4ad). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-218350959 **[Test build #58321 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58321/consoleFull)** for PR 12629 at commit [`ef4e6be`](https://github.com/apache/spark/commit/ef4e6be908a08840884e77f0ad0620f79026b4ad). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-216094824 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57492/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-216094823 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-216094731 **[Test build #57492 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57492/consoleFull)** for PR 12629 at commit [`a17c588`](https://github.com/apache/spark/commit/a17c5885b3254e8457bfb21e9347a402a0c7e87a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-216086947 cc @rxin as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-216086878 Hi @davies @viirya , If you are not sure of handling `null`s, I can close this for meanwhile. But, this PR includes - adding `OrcOptions` just like `ParquetOptions` - fixing a little bit inappropriate test for ORC compression So, I can close this but create another PR for them because I believe the two items are pretty clear to do so. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-216086817 **[Test build #57492 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57492/consoleFull)** for PR 12629 at commit [`a17c588`](https://github.com/apache/spark/commit/a17c5885b3254e8457bfb21e9347a402a0c7e87a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-215911793 Ping @davies and @viirya. It sets a default value for `null` to the options that throw an `NullPointExceptiom` and just passes `null` for some options that take `null`. If you think `null` should not be passed at all, then I can fix them. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214990643 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57093/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214990642 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214990430 **[Test build #57093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57093/consoleFull)** for PR 12629 at commit [`f9837d8`](https://github.com/apache/spark/commit/f9837d83cf0e93c9809ff39a3500d373f8330354). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214976458 **[Test build #57093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57093/consoleFull)** for PR 12629 at commit [`f9837d8`](https://github.com/apache/spark/commit/f9837d83cf0e93c9809ff39a3500d373f8330354). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214976019 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214976015 **[Test build #57092 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57092/consoleFull)** for PR 12629 at commit [`0c32f4e`](https://github.com/apache/spark/commit/0c32f4ee6f3e5dd630ed663a1fc1e21d85d0541d). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214976020 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57092/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214975784 It looks Hadoop configuration can be set via `option()` now. It looks when it sets `null` for Hadoop configurations, it uses default values which is consistent with what this PR proposes. So, I think we don't need a special handling for Hadoop configuration. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214975697 **[Test build #57092 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57092/consoleFull)** for PR 12629 at commit [`0c32f4e`](https://github.com/apache/spark/commit/0c32f4ee6f3e5dd630ed663a1fc1e21d85d0541d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214619175 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214619176 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56954/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214619052 **[Test build #56954 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56954/consoleFull)** for PR 12629 at commit [`2fa4a12`](https://github.com/apache/spark/commit/2fa4a128fcb576914d4632ab4a71f135839ab287). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214604317 **[Test build #56954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56954/consoleFull)** for PR 12629 at commit [`2fa4a12`](https://github.com/apache/spark/commit/2fa4a128fcb576914d4632ab4a71f135839ab287). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214554156 Could you review this please? @davies and @viirya --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214105426 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214105430 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56867/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214104876 **[Test build #56867 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56867/consoleFull)** for PR 12629 at commit [`395ad1d`](https://github.com/apache/spark/commit/395ad1d414b923e5940c442039744a8d11fcf731). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214088358 **[Test build #56867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56867/consoleFull)** for PR 12629 at commit [`395ad1d`](https://github.com/apache/spark/commit/395ad1d414b923e5940c442039744a8d11fcf731). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214086389 **[Test build #56866 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56866/consoleFull)** for PR 12629 at commit [`90a2d51`](https://github.com/apache/spark/commit/90a2d5111c564d6138a5846d1e0e9e521c4e491b). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214086391 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56866/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214086390 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-214086309 **[Test build #56866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56866/consoleFull)** for PR 12629 at commit [`90a2d51`](https://github.com/apache/spark/commit/90a2d5111c564d6138a5846d1e0e9e521c4e491b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-213670420 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56775/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-213670418 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-213670329 **[Test build #56775 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56775/consoleFull)** for PR 12629 at commit [`e5dec86`](https://github.com/apache/spark/commit/e5dec86845cbf25eb606ceea7a81151c0ed638de). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-213662696 **[Test build #56775 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56775/consoleFull)** for PR 12629 at commit [`e5dec86`](https://github.com/apache/spark/commit/e5dec86845cbf25eb606ceea7a81151c0ed638de). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-213661096 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-213661097 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/56765/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-213661066 **[Test build #56765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56765/consoleFull)** for PR 12629 at commit [`ccd3c7b`](https://github.com/apache/spark/commit/ccd3c7b43c0247e345b714210f7421d7dc484718). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-213656756 **[Test build #56765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56765/consoleFull)** for PR 12629 at commit [`ccd3c7b`](https://github.com/apache/spark/commit/ccd3c7b43c0247e345b714210f7421d7dc484718). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
Github user HyukjinKwon commented on the pull request: https://github.com/apache/spark/pull/12629#issuecomment-213656738 cc @davies @viirya --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-14800][SQL] Dealing with null as a valu...
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/12629 [SPARK-14800][SQL] Dealing with null as a value in options for each internal data source ## What changes were proposed in this pull request? https://issues.apache.org/jira/browse/SPARK-14800 This PR add the support for `null` for values as options (as a default value) for all the internal data source in Spark. This PR introduces two classes - `PrameterUtils`: This has some functions used in `CSVOptions` to check `null` for other data sources. - `OrcOptions`: Just like `ParquetOptions` this was separated (actually they are almost identical). ## How was this patch tested? Unit tests in `CSVSuite`, `JsonSuite`, `OrcHadoopFsRelationSuite, `ParquetHadoopFsRelationSuite` and `LibSVMRelation`. Also,`sbt scalastyle` You can merge this pull request into a Git repository by running: $ git pull https://github.com/HyukjinKwon/spark SPARK-14800 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/12629.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #12629 commit 8fb3a23ef61353749c35f523dbfc7d8f5d739fbf Author: hyukjinkwon Date: 2016-04-23T02:09:36Z CSV and JSON are now safe with null options commit ccd3c7b43c0247e345b714210f7421d7dc484718 Author: hyukjinkwon Date: 2016-04-23T02:55:02Z text, ORC, Parquet and libsvm are also okay --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
