[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/13008 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-220201533 Thanks! Merging to master and branch 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r63803944
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/fpm/FPGrowth.scala ---
@@ -116,7 +116,7 @@ object FPGrowthModel extends Loader[FPGrowthModel[_]] {
StructField("freq", LongType))
val schema = StructType(fields)
val rowDataRDD = model.freqItemsets.map { x =>
-Row(x.items, x.freq)
+Row(x.items.toSeq, x.freq)
--- End diff --
We need. This is a special case, `FPGrowthModel` has a type parameter and
we use `FPGrowthModel[_]` here. So `x.items` returns `Object[]` instead of
`T[]` as we expected and doesn't match the schema.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r63726478
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/fpm/FPGrowth.scala ---
@@ -116,7 +116,7 @@ object FPGrowthModel extends Loader[FPGrowthModel[_]] {
StructField("freq", LongType))
val schema = StructType(fields)
val rowDataRDD = model.freqItemsets.map { x =>
-Row(x.items, x.freq)
+Row(x.items.toSeq, x.freq)
--- End diff --
Do we need to call `toSeq` at here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-220002209 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58771/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-220002208 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-220001928 **[Test build #58771 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58771/consoleFull)** for PR 13008 at commit [`f533188`](https://github.com/apache/spark/commit/f533188c8c7bb295307ce141d0ba1f74788bd21c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219983946 **[Test build #58771 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58771/consoleFull)** for PR 13008 at commit [`f533188`](https://github.com/apache/spark/commit/f533188c8c7bb295307ce141d0ba1f74788bd21c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219929255 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58741/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219929253 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219929172 **[Test build #58741 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58741/consoleFull)** for PR 13008 at commit [`57efddb`](https://github.com/apache/spark/commit/57efddbf737686c536933def1937011575caec24). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219920209 **[Test build #58741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58741/consoleFull)** for PR 13008 at commit [`57efddb`](https://github.com/apache/spark/commit/57efddbf737686c536933def1937011575caec24). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219734656 **[Test build #58695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58695/consoleFull)** for PR 13008 at commit [`225128d`](https://github.com/apache/spark/commit/225128d48c9a9c913339a6291947675509991317). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219734722 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58695/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219734720 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219725454 **[Test build #58695 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58695/consoleFull)** for PR 13008 at commit [`225128d`](https://github.com/apache/spark/commit/225128d48c9a9c913339a6291947675509991317). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219724329 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219669778 **[Test build #58678 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58678/consoleFull)** for PR 13008 at commit [`0fd8a90`](https://github.com/apache/spark/commit/0fd8a90eacb1c6e3380ffec7b2215ae714ce4528). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219669826 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219669830 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58678/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219664628 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219664603 **[Test build #58680 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58680/consoleFull)** for PR 13008 at commit [`225128d`](https://github.com/apache/spark/commit/225128d48c9a9c913339a6291947675509991317). * This patch **fails MiMa tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219664629 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58680/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219662340 **[Test build #58680 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58680/consoleFull)** for PR 13008 at commit [`225128d`](https://github.com/apache/spark/commit/225128d48c9a9c913339a6291947675509991317). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-219661237 **[Test build #58678 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58678/consoleFull)** for PR 13008 at commit [`0fd8a90`](https://github.com/apache/spark/commit/0fd8a90eacb1c6e3380ffec7b2215ae714ce4528). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218706303 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218706306 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58462/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218706135 **[Test build #58462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58462/consoleFull)** for PR 13008 at commit [`3acf24f`](https://github.com/apache/spark/commit/3acf24f1ed256d4f2411b5c67a688587cba589ab). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218688181 **[Test build #58462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58462/consoleFull)** for PR 13008 at commit [`3acf24f`](https://github.com/apache/spark/commit/3acf24f1ed256d4f2411b5c67a688587cba589ab). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218686965 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218686962 **[Test build #58461 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58461/consoleFull)** for PR 13008 at commit [`114f362`](https://github.com/apache/spark/commit/114f3620337cc3c7c4f5699e1c4f311255622495). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218686967 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58461/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218686177 **[Test build #58461 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58461/consoleFull)** for PR 13008 at commit [`114f362`](https://github.com/apache/spark/commit/114f3620337cc3c7c4f5699e1c4f311255622495). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218672986 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58451/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218672864 **[Test build #58451 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58451/consoleFull)** for PR 13008 at commit [`0915a71`](https://github.com/apache/spark/commit/0915a71e2834617db28dc13f1a68c11c71c35009). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218672983 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218670622 Oh actually we did document it in the java doc of `Row`, and says users should use `Seq` for array type. see https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/Row.scala#L139-L154 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218670057 looks like we haven't documented what kind of field object types is allowed in a `Row`, let me find a place to document it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218669218 I see. Seems like a API change that at least we need to document. Is there any performance implication? also cc @mengxr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218668967 Unlike `CatalystConverter`, `RowEncoder` is stricter about the input external type, e.g. users must use `Seq` for `ArrayType`, but `CatalystConvert` also allows `Array`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218665830 what is the cause of those failed tests? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-21866 yea, the new commit should fixed it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218665223 legitimate issue? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218664669 **[Test build #58450 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58450/consoleFull)** for PR 13008 at commit [`7419a52`](https://github.com/apache/spark/commit/7419a525f3471b7a225721436150be78ee94e22b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218664688 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58450/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218664687 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218662389 **[Test build #58451 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58451/consoleFull)** for PR 13008 at commit [`0915a71`](https://github.com/apache/spark/commit/0915a71e2834617db28dc13f1a68c11c71c35009). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218661393 **[Test build #58450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58450/consoleFull)** for PR 13008 at commit [`7419a52`](https://github.com/apache/spark/commit/7419a525f3471b7a225721436150be78ee94e22b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218661098 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218661110 LGTM pending jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r62965499
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala
---
@@ -505,7 +505,7 @@ class DatasetSuite extends QueryTest with
SharedSQLContext {
val schema = StructType(Seq(
StructField("f", StructType(Seq(
StructField("a", StringType, nullable = true),
-StructField("b", IntegerType, nullable = false)
+StructField("b", IntegerType, nullable = true)
--- End diff --
thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r62965137
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala
---
@@ -505,7 +505,7 @@ class DatasetSuite extends QueryTest with
SharedSQLContext {
val schema = StructType(Seq(
StructField("f", StructType(Seq(
StructField("a", StringType, nullable = true),
-StructField("b", IntegerType, nullable = false)
+StructField("b", IntegerType, nullable = true)
--- End diff --
yea, `row nullability mismatch` checks the error that we pass in a null
column while this column is declared as not nullable.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r62964943
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala
---
@@ -505,7 +505,7 @@ class DatasetSuite extends QueryTest with
SharedSQLContext {
val schema = StructType(Seq(
StructField("f", StructType(Seq(
StructField("a", StringType, nullable = true),
-StructField("b", IntegerType, nullable = false)
+StructField("b", IntegerType, nullable = true)
--- End diff --
(just want to make sure we are not losing test coverage)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r62964815
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala
---
@@ -505,7 +505,7 @@ class DatasetSuite extends QueryTest with
SharedSQLContext {
val schema = StructType(Seq(
StructField("f", StructType(Seq(
StructField("a", StringType, nullable = true),
-StructField("b", IntegerType, nullable = false)
+StructField("b", IntegerType, nullable = true)
--- End diff --
ok. so the new test (`row nullability mismatch`) is effectively covering
this case? Then, should we change the name of this test? Will we hit the
exception that is checked by this test in any other cases?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r62964694
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala
---
@@ -505,7 +505,7 @@ class DatasetSuite extends QueryTest with
SharedSQLContext {
val schema = StructType(Seq(
StructField("f", StructType(Seq(
StructField("a", StringType, nullable = true),
-StructField("b", IntegerType, nullable = false)
+StructField("b", IntegerType, nullable = true)
--- End diff --
With the new null check, we will trigger error earlier than this test
expected. This test is testing the `AssertNotNull` expression, which is used
for converting nullable column to not-nullable object field(like primitive int).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r62963859
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala
---
@@ -505,7 +505,7 @@ class DatasetSuite extends QueryTest with
SharedSQLContext {
val schema = StructType(Seq(
StructField("f", StructType(Seq(
StructField("a", StringType, nullable = true),
-StructField("b", IntegerType, nullable = false)
+StructField("b", IntegerType, nullable = true)
--- End diff --
Why change this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218649776 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218649754 **[Test build #58437 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58437/consoleFull)** for PR 13008 at commit [`7419a52`](https://github.com/apache/spark/commit/7419a525f3471b7a225721436150be78ee94e22b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218649778 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58437/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-218646586 **[Test build #58437 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58437/consoleFull)** for PR 13008 at commit [`7419a52`](https://github.com/apache/spark/commit/7419a525f3471b7a225721436150be78ee94e22b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r62947902
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -113,8 +113,8 @@ object ScalaReflection extends ScalaReflection {
* Returns true if the value of this data type is same between internal
and external.
*/
def isNativeType(dt: DataType): Boolean = dt match {
-case BooleanType | ByteType | ShortType | IntegerType | LongType |
- FloatType | DoubleType | BinaryType => true
+case NullType | BooleanType | ByteType | ShortType | IntegerType |
LongType |
+ FloatType | DoubleType | BinaryType | CalendarIntervalType => true
--- End diff --
Because we don't have an external representation of it
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r62947857
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
---
@@ -490,8 +490,8 @@ class SparkSession private(
// TODO: use MutableProjection when rowRDD is another DataFrame and
the applied
// schema differs from the existing schema on any field data type.
val catalystRows = if (needsConversion) {
- val converter =
CatalystTypeConverters.createToCatalystConverter(schema)
- rowRDD.map(converter(_).asInstanceOf[InternalRow])
+ val encoder = RowEncoder(schema)
--- End diff --
yea
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r62939213
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -113,8 +113,8 @@ object ScalaReflection extends ScalaReflection {
* Returns true if the value of this data type is same between internal
and external.
*/
def isNativeType(dt: DataType): Boolean = dt match {
-case BooleanType | ByteType | ShortType | IntegerType | LongType |
- FloatType | DoubleType | BinaryType => true
+case NullType | BooleanType | ByteType | ShortType | IntegerType |
LongType |
+ FloatType | DoubleType | BinaryType | CalendarIntervalType => true
--- End diff --
Why CalendarIntervalType?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user yhuai commented on a diff in the pull request:
https://github.com/apache/spark/pull/13008#discussion_r62939286
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
---
@@ -490,8 +490,8 @@ class SparkSession private(
// TODO: use MutableProjection when rowRDD is another DataFrame and
the applied
// schema differs from the existing schema on any field data type.
val catalystRows = if (needsConversion) {
- val converter =
CatalystTypeConverters.createToCatalystConverter(schema)
- rowRDD.map(converter(_).asInstanceOf[InternalRow])
+ val encoder = RowEncoder(schema)
--- End diff --
So, we already do null check in RowEncoder, right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-217902758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58147/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-217902756 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15192][SQL] null check for SparkSession...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/13008#issuecomment-217902709 **[Test build #58147 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58147/consoleFull)** for PR 13008 at commit [`8f0a0bf`](https://github.com/apache/spark/commit/8f0a0bfdb044acf185548b214e904b00d3ba3be3). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
