[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17164 This pr added an new SQL option `spark.sql.aggregate.preferSortAggregate` to preferably select `SortAggregate` for easy-to-test in `DataFrameAggregateSuite.scala`. In some cases (e.g., input data is already sorted in cache), sort aggregate is faster than hash one (See: https://issues.apache.org/jira/browse/SPARK-18591). But, you know, the current spark does not adaptively select sort aggregate in these cases. So, I probably think this option is some useful to control aggregate strategies by user. What do u think? cc: @hvanhovell If yes, I'd like to make another pr to add this option before this pr reviewed. https://github.com/apache/spark/compare/master...maropu:SPARK-16844-3 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74066/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #74066 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74066/testReport)** for PR 17164 at commit [`fc01d07`](https://github.com/apache/spark/commit/fc01d079fc09286921b0cc4800435178ea1ccb35). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait AggregateCodegenHelper ` * `abstract class AggregateExec extends UnaryExecNode ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #74066 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74066/testReport)** for PR 17164 at commit [`fc01d07`](https://github.com/apache/spark/commit/fc01d079fc09286921b0cc4800435178ea1ccb35). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73993/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73993 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73993/testReport)** for PR 17164 at commit [`2a018cb`](https://github.com/apache/spark/commit/2a018cbf37458bfe650b02e28b5f88655b86e903). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73993 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73993/testReport)** for PR 17164 at commit [`2a018cb`](https://github.com/apache/spark/commit/2a018cbf37458bfe650b02e28b5f88655b86e903). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17164 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73986/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73986 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73986/testReport)** for PR 17164 at commit [`d5cc0f0`](https://github.com/apache/spark/commit/d5cc0f08624448362e535f46b9deea889f2467ab). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait AggregateCodegenHelper ` * `abstract class AggregateExec extends UnaryExecNode ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73986/testReport)** for PR 17164 at commit [`d5cc0f0`](https://github.com/apache/spark/commit/d5cc0f08624448362e535f46b9deea889f2467ab). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17164 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73974/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73974/testReport)** for PR 17164 at commit [`d5cc0f0`](https://github.com/apache/spark/commit/d5cc0f08624448362e535f46b9deea889f2467ab). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73952/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73952/testReport)** for PR 17164 at commit [`6af8064`](https://github.com/apache/spark/commit/6af80640d7be8eced379b064e6815c6403d97522). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait AggregateCodegenHelper ` * `abstract class AggregateExec extends UnaryExecNode ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73952/testReport)** for PR 17164 at commit [`6af8064`](https://github.com/apache/spark/commit/6af80640d7be8eced379b064e6815c6403d97522). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73949/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73949 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73949/testReport)** for PR 17164 at commit [`29c713b`](https://github.com/apache/spark/commit/29c713b64a35bef9a36b7a4e3a52c9c74d48e045). * This patch **fails to build**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait AggregateCodegenHelper ` * `abstract class AggregateExec extends UnaryExecNode ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73948 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73948/testReport)** for PR 17164 at commit [`8413dd7`](https://github.com/apache/spark/commit/8413dd7f700fbdf0e920f9b36f1643fdb9f389d4). * This patch **fails to build**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait AggregateCodegenHelper ` * `abstract class AggregateExec extends UnaryExecNode ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73948/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73949/testReport)** for PR 17164 at commit [`29c713b`](https://github.com/apache/spark/commit/29c713b64a35bef9a36b7a4e3a52c9c74d48e045). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73948/testReport)** for PR 17164 at commit [`8413dd7`](https://github.com/apache/spark/commit/8413dd7f700fbdf0e920f9b36f1643fdb9f389d4). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73935/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73935/testReport)** for PR 17164 at commit [`5b138f7`](https://github.com/apache/spark/commit/5b138f708f34b66cf4d811e6cacab8f905e94955). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait AggregateCodegenHelper ` * `abstract class AggregateExec extends UnaryExecNode ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73935/testReport)** for PR 17164 at commit [`5b138f7`](https://github.com/apache/spark/commit/5b138f708f34b66cf4d811e6cacab8f905e94955). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73926/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73926/testReport)** for PR 17164 at commit [`b29c22d`](https://github.com/apache/spark/commit/b29c22daf40d5e7405b5afc7b77df1d36de0baa8). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait AggregateCodegenHelper ` * `abstract class AggregateExec extends UnaryExecNode ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73925/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73925 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73925/testReport)** for PR 17164 at commit [`a45048a`](https://github.com/apache/spark/commit/a45048ad74aaaee7aae2854af3e248b0b79e8602). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait AggregateCodegenHelper ` * `abstract class AggregateExec extends UnaryExecNode ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73924/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73924/testReport)** for PR 17164 at commit [`f63e663`](https://github.com/apache/spark/commit/f63e663dd9a6b6d5528fd807be79e59be3ba6cb6). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `abstract class AggregateExec extends UnaryExecNode ` * `trait CodegenAggregateSupport extends CodegenSupport ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73926/testReport)** for PR 17164 at commit [`b29c22d`](https://github.com/apache/spark/commit/b29c22daf40d5e7405b5afc7b77df1d36de0baa8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73925/testReport)** for PR 17164 at commit [`a45048a`](https://github.com/apache/spark/commit/a45048ad74aaaee7aae2854af3e248b0b79e8602). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73924/testReport)** for PR 17164 at commit [`f63e663`](https://github.com/apache/spark/commit/f63e663dd9a6b6d5528fd807be79e59be3ba6cb6). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17164 @maropu I think this is pretty exciting. This is very useful in situations where we have a lot of groups, in that case I will happily take a 2x performance improvement any day. This is still pretty decent if you consider that this aggregate is dominate by sorting. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73908/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17164 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73908 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73908/testReport)** for PR 17164 at commit [`9a26a0a`](https://github.com/apache/spark/commit/9a26a0a0e9c7f9d0e90dc5257eb5038eafeb206c). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `abstract class AggregateExec extends UnaryExecNode ` * `trait CodegenAggregateSupport extends CodegenSupport ` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17164 @hvanhovell I reworked #14481 though, I'm not sure it is still worth trying this codegen. Could you give me insight first? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/17164
```
import org.apache.spark.sql.execution.debug._
spark.conf.set("spark.sql.aggregate.preferSortAggregate", "true")
val df = spark.range(10).selectExpr("id % 2 AS key", "rand() AS value")
df.groupBy().count.debugCodegen
Found 2 WholeStageCodegen subtrees.
== Subtree 1 / 2 ==
*SortAggregate(key=[], functions=[partial_count(1)], output=[count#51L])
+- *Project
+- *Range (0, 10, step=1, splits=Some(4))
Generated code:
/* 001 */ public Object generate(Object[] references) {
/* 002 */ return new GeneratedIterator(references);
/* 003 */ }
/* 004 */
/* 005 */ final class GeneratedIterator extends
org.apache.spark.sql.execution.BufferedRowIterator {
/* 006 */ private Object[] references;
/* 007 */ private scala.collection.Iterator[] inputs;
/* 008 */ private boolean sagg_initAgg;
/* 009 */ private boolean sagg_bufIsNull;
/* 010 */ private long sagg_bufValue;
/* 011 */ private org.apache.spark.sql.execution.metric.SQLMetric
range_numOutputRows;
/* 012 */ private org.apache.spark.sql.execution.metric.SQLMetric
range_numGeneratedRows;
/* 013 */ private boolean range_initRange;
/* 014 */ private long range_number;
/* 015 */ private TaskContext range_taskContext;
/* 016 */ private InputMetrics range_inputMetrics;
/* 017 */ private long range_batchEnd;
/* 018 */ private long range_numElementsTodo;
/* 019 */ private scala.collection.Iterator range_input;
/* 020 */ private UnsafeRow range_result;
/* 021 */ private
org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder range_holder;
/* 022 */ private
org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter
range_rowWriter;
/* 023 */ private org.apache.spark.sql.execution.metric.SQLMetric
sagg_numOutputRows;
/* 024 */ private org.apache.spark.sql.execution.metric.SQLMetric
sagg_aggTime;
/* 025 */ private UnsafeRow sagg_result;
/* 026 */ private
org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder sagg_holder;
/* 027 */ private
org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter
sagg_rowWriter;
/* 028 */
/* 029 */ public GeneratedIterator(Object[] references) {
/* 030 */ this.references = references;
/* 031 */ }
/* 032 */
/* 033 */ public void init(int index, scala.collection.Iterator[] inputs)
{
/* 034 */ partitionIndex = index;
/* 035 */ this.inputs = inputs;
/* 036 */ sagg_initAgg = false;
/* 037 */
/* 038 */ this.range_numOutputRows =
(org.apache.spark.sql.execution.metric.SQLMetric) references[0];
/* 039 */ this.range_numGeneratedRows =
(org.apache.spark.sql.execution.metric.SQLMetric) references[1];
/* 040 */ range_initRange = false;
/* 041 */ range_number = 0L;
/* 042 */ range_taskContext = TaskContext.get();
/* 043 */ range_inputMetrics =
range_taskContext.taskMetrics().inputMetrics();
/* 044 */ range_batchEnd = 0;
/* 045 */ range_numElementsTodo = 0L;
/* 046 */ range_input = inputs[0];
/* 047 */ range_result = new UnsafeRow(1);
/* 048 */ this.range_holder = new
org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder(range_result, 0);
/* 049 */ this.range_rowWriter = new
org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter(range_holder,
1);
/* 050 */ this.sagg_numOutputRows =
(org.apache.spark.sql.execution.metric.SQLMetric) references[2];
/* 051 */ this.sagg_aggTime =
(org.apache.spark.sql.execution.metric.SQLMetric) references[3];
/* 052 */ sagg_result = new UnsafeRow(1);
/* 053 */ this.sagg_holder = new
org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder(sagg_result, 0);
/* 054 */ this.sagg_rowWriter = new
org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter(sagg_holder,
1);
/* 055 */
/* 056 */ }
/* 057 */
/* 058 */ private void sagg_doAggregateWithoutKey() throws
java.io.IOException {
/* 059 */ // initialize aggregation buffer
/* 060 */ sagg_bufIsNull = false;
/* 061 */ sagg_bufValue = 0L;
/* 062 */
/* 063 */ // initialize Range
/* 064 */ if (!range_initRange) {
/* 065 */ range_initRange = true;
/* 066 */ initRange(partitionIndex);
/* 067 */ }
/* 068 */
/* 069 */ while (true) {
/* 070 */ while (range_number != range_batchEnd) {
/* 071 */ long range_value = range_number;
/* 072 */ range_number += 1L;
/* 073 */
/* 074 */ // do aggregate
/* 075 */ // common sub-expressions
/* 076 */
/* 077 */ // evaluate aggregate function
/* 078 */ boolean sagg_isNull1 = false;
/* 079
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/17164
```
import org.apache.spark.sql.execution.debug._
spark.conf.set("spark.sql.aggregate.preferSortAggregate", "true")
val df = spark.range(10).selectExpr("id % 2 AS key", "rand() AS value")
df.groupBy($"key").sum("value").debugCodegen
Found 2 WholeStageCodegen subtrees.
== Subtree 1 / 2 ==
*SortAggregate(key=[key#3L], functions=[sum(value#4)], output=[key#3L,
sum(value)#12])
+- *Sort [key#3L ASC NULLS FIRST], false, 0
+- Exchange hashpartitioning(key#3L, 200)
+- *SortAggregate(key=[key#3L], functions=[partial_sum(value#4)],
output=[key#3L, sum#17])
+- *Sort [key#3L ASC NULLS FIRST], false, 0
+- *Project [(id#0L % 2) AS key#3L, rand(-2342342825239413884)
AS value#4]
+- *Range (0, 10, step=1, splits=Some(4))
Generated code:
/* 001 */ public Object generate(Object[] references) {
/* 002 */ return new GeneratedIterator(references);
/* 003 */ }
/* 004 */
/* 005 */ final class GeneratedIterator extends
org.apache.spark.sql.execution.BufferedRowIterator {
/* 006 */ private Object[] references;
/* 007 */ private scala.collection.Iterator[] inputs;
/* 008 */ private org.apache.spark.sql.execution.metric.SQLMetric
sagg_numOutputRows;
/* 009 */ private boolean sort_needToSort;
/* 010 */ private org.apache.spark.sql.execution.SortExec sort_plan;
/* 011 */ private org.apache.spark.sql.execution.UnsafeExternalRowSorter
sort_sorter;
/* 012 */ private org.apache.spark.executor.TaskMetrics sort_metrics;
/* 013 */ private scala.collection.Iterator sort_sortedIter;
/* 014 */ private scala.collection.Iterator inputadapter_input;
/* 015 */ private org.apache.spark.sql.execution.metric.SQLMetric
sort_peakMemory;
/* 016 */ private org.apache.spark.sql.execution.metric.SQLMetric
sort_spillSize;
/* 017 */ private org.apache.spark.sql.execution.metric.SQLMetric
sort_sortTime;
/* 018 */ private UnsafeRow sagg_currentGroupingKey;
/* 019 */ private boolean sagg_bufIsNull;
/* 020 */ private double sagg_bufValue;
/* 021 */ private UnsafeRow sagg_result;
/* 022 */ private
org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder sagg_holder;
/* 023 */ private
org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter
sagg_rowWriter;
/* 024 */ private UnsafeRow sagg_result1;
/* 025 */ private
org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder sagg_holder1;
/* 026 */ private
org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter
sagg_rowWriter1;
/* 027 */ private
org.apache.spark.sql.execution.aggregate.SortAggregateExec sagg_sortAggregate;
/* 028 */ private UnsafeRow sagg_result2;
/* 029 */ private
org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder sagg_holder2;
/* 030 */ private
org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter
sagg_rowWriter2;
/* 031 */ private org.apache.spark.sql.execution.metric.SQLMetric
wholestagecodegen_numOutputRows;
/* 032 */
/* 033 */ public GeneratedIterator(Object[] references) {
/* 034 */ this.references = references;
/* 035 */ }
/* 036 */
/* 037 */ public void init(int index, scala.collection.Iterator[] inputs)
{
/* 038 */ partitionIndex = index;
/* 039 */ this.inputs = inputs;
/* 040 */ wholestagecodegen_init_0();
/* 041 */ wholestagecodegen_init_1();
/* 042 */
/* 043 */ }
/* 044 */
/* 045 */ private void wholestagecodegen_init_0() {
/* 046 */ this.sagg_numOutputRows =
(org.apache.spark.sql.execution.metric.SQLMetric) references[0];
/* 047 */ sort_needToSort = true;
/* 048 */ this.sort_plan = (org.apache.spark.sql.execution.SortExec)
references[1];
/* 049 */ sort_sorter = sort_plan.createSorter();
/* 050 */ sort_metrics =
org.apache.spark.TaskContext.get().taskMetrics();
/* 051 */
/* 052 */ inputadapter_input = inputs[0];
/* 053 */ this.sort_peakMemory =
(org.apache.spark.sql.execution.metric.SQLMetric) references[2];
/* 054 */ this.sort_spillSize =
(org.apache.spark.sql.execution.metric.SQLMetric) references[3];
/* 055 */ this.sort_sortTime =
(org.apache.spark.sql.execution.metric.SQLMetric) references[4];
/* 056 */ sagg_currentGroupingKey = null;
/* 057 */
/* 058 */ sagg_result = new UnsafeRow(1);
/* 059 */ this.sagg_holder = new
org.apache.spark.sql.catalyst.expressions.codegen.BufferHolder(sagg_result, 0);
/* 060 */ this.sagg_rowWriter = new
org.apache.spark.sql.catalyst.expressions.codegen.UnsafeRowWriter(sagg_holder,
1);
/* 061 */ sagg_result1 = new UnsafeRow(1);
/* 062 */ this.sagg_holder1 = new
[GitHub] spark issue #17164: [SPARK-16844][SQL][WIP] Support codegen for sort-based a...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17164 **[Test build #73908 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73908/testReport)** for PR 17164 at commit [`9a26a0a`](https://github.com/apache/spark/commit/9a26a0a0e9c7f9d0e90dc5257eb5038eafeb206c). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
