[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user inouehrs commented on the issue: https://github.com/apache/spark/pull/16391 I agree that it is quite nice to have multiple record types in the benchmark to reveal the source of overheads! I also observed the relative performances of RDD, DataFrame, Dataset become quite different when only a primitive is used as the record type as discussed in an old PR (https://github.com/apache/spark/pull/13459/commits/f1e49f35e03c1d69662273b91b67b734d95b117b). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/16391 Just reverted from master. btw, I think we can keep existing cases and then add new cases. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/16391 Seems there is no explicit LGTM. I am reverting this change from master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16391 thanks for the review, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16391 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70619/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16391 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16391 **[Test build #70619 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70619/testReport)** for PR 16391 at commit [`2a4d22d`](https://github.com/apache/spark/commit/2a4d22db999330320ae18b1281d6b355b565c4d1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16391 **[Test build #70619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70619/testReport)** for PR 16391 at commit [`2a4d22d`](https://github.com/apache/spark/commit/2a4d22db999330320ae18b1281d6b355b565c4d1). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16391 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70553/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16391 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16391 **[Test build #70553 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70553/testReport)** for PR 16391 at commit [`ff89719`](https://github.com/apache/spark/commit/ff89719aec3785cb27315810791f14787d5ade26). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Literal (value: Any, dataType: DataType) extends LeafExpression with CodegenFallback ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16391 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70552/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16391 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16391 **[Test build #70552 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70552/testReport)** for PR 16391 at commit [`43da0af`](https://github.com/apache/spark/commit/43da0af12b40e1425e95fc944723b2a6539dbabc). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `case class Literal (value: Any, dataType: DataType) extends LeafExpression ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16391 cc @yhuai @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16391 **[Test build #70553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70553/testReport)** for PR 16391 at commit [`ff89719`](https://github.com/apache/spark/commit/ff89719aec3785cb27315810791f14787d5ade26). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16391: [SPARK-18990][SQL] make DatasetBenchmark fairer for Data...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16391 **[Test build #70552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70552/testReport)** for PR 16391 at commit [`43da0af`](https://github.com/apache/spark/commit/43da0af12b40e1425e95fc944723b2a6539dbabc). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
