[GitHub] spark issue #14679: [SPARK-17102][SQL] bypass UserDefinedGenerator for json ...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14679 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14679: [SPARK-17102][SQL] bypass UserDefinedGenerator for json ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14679 **[Test build #63902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63902/consoleFull)** for PR 14679 at commit [`c04084d`](https://github.com/apache/spark/commit/c04084d29c3d049053a5929d6860006549c96573). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14679: [SPARK-17102][SQL] bypass UserDefinedGenerator for json ...
Github user clockfly commented on the issue: https://github.com/apache/spark/pull/14679 @cloud-fan Can you post the stacktrace of the problem somewhere? By the way, what you mean by saying ``` e.g. DataFrameSuite.simple explode, this test always fail in branch 1.6, but pass in master, because of the different scala versions. ``` What scala version Spark 1.6 use, and What version Spark master use? And why different scala version causes this problem? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14679: [SPARK-17102][SQL] bypass UserDefinedGenerator for json ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14679 **[Test build #63902 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63902/consoleFull)** for PR 14679 at commit [`c04084d`](https://github.com/apache/spark/commit/c04084d29c3d049053a5929d6860006549c96573). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14679: [SPARK-17102][SQL] bypass UserDefinedGenerator for json ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14679 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63902/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14679: [SPARK-17102][SQL] bypass UserDefinedGenerator for json ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14679 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14679: [SPARK-17102][SQL] bypass UserDefinedGenerator for json ...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14679 Now I have put the stacktrace in the JIRA, sorry for that. I didn't look into this, seems like the behavior of scala reflection methods is different between scala 2.10 and 2.11. Anyway the json format for function object is not supported, we should just skip it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14679: [SPARK-17102][SQL] bypass UserDefinedGenerator for json ...
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14679 Looks like it is caused by `Seq[(DataType, Boolean, String)]` in `UserDefinedGenerator`? I think it is fine to bypass it for now. But, it will be good to address issues that causing toJson crash. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14679: [SPARK-17102][SQL] bypass UserDefinedGenerator for json ...
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14679 I am merging this to master, branch 2.0, and branch 1.6. So, we will not hit this issue. I have created https://issues.apache.org/jira/browse/SPARK-17109 to investigate the root cause. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
