[GitHub] [spark] SparkQA commented on pull request #32014: [SPARK-34922][SQL] Use a relative cost comparison function in the CBO

SparkQA commented on pull request #32014: URL: https://github.com/apache/spark/pull/32014#issuecomment-812475460 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41429/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32015: [SPARK-34821][INFRA] Set up a workflow for developers to run benchmark in their fork

AmplabJenkins removed a comment on pull request #32015: URL: https://github.com/apache/spark/pull/32015#issuecomment-812464939 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136846/

[GitHub] [spark] SparkQA commented on pull request #32014: [SPARK-34922][SQL] Use a relative cost comparison function in the CBO

SparkQA commented on pull request #32014: URL: https://github.com/apache/spark/pull/32014#issuecomment-812464987 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41429/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #32015: [SPARK-34821][INFRA] Set up a workflow for developers to run benchmark in their fork

AmplabJenkins commented on pull request #32015: URL: https://github.com/apache/spark/pull/32015#issuecomment-812464939 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136846/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32015: [SPARK-34821][INFRA] Set up a workflow for developers to run benchmark in their fork

SparkQA removed a comment on pull request #32015: URL: https://github.com/apache/spark/pull/32015#issuecomment-812356191 **[Test build #136846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136846/testReport)** for PR 32015 at commit

[GitHub] [spark] SparkQA commented on pull request #32015: [SPARK-34821][INFRA] Set up a workflow for developers to run benchmark in their fork

SparkQA commented on pull request #32015: URL: https://github.com/apache/spark/pull/32015#issuecomment-812463873 **[Test build #136846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136846/testReport)** for PR 32015 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29871: [SPARK-32995][SQL] CostBasedJoinReorder optimizer rule should be idempotent

AmplabJenkins removed a comment on pull request #29871: URL: https://github.com/apache/spark/pull/29871#issuecomment-812461275 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41428/

[GitHub] [spark] AmplabJenkins commented on pull request #29871: [SPARK-32995][SQL] CostBasedJoinReorder optimizer rule should be idempotent

AmplabJenkins commented on pull request #29871: URL: https://github.com/apache/spark/pull/29871#issuecomment-812461275 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41428/ --

[GitHub] [spark] SparkQA commented on pull request #29871: [SPARK-32995][SQL] CostBasedJoinReorder optimizer rule should be idempotent

SparkQA commented on pull request #29871: URL: https://github.com/apache/spark/pull/29871#issuecomment-812461242 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

AmplabJenkins removed a comment on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812460528 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136852/

[GitHub] [spark] SparkQA removed a comment on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

SparkQA removed a comment on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812459967 **[Test build #136852 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136852/testReport)** for PR 32036 at commit

[GitHub] [spark] SparkQA commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

SparkQA commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812460511 **[Test build #136852 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136852/testReport)** for PR 32036 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

AmplabJenkins commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812460528 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136852/ -- This

[GitHub] [spark] SparkQA commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

SparkQA commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812459967 **[Test build #136852 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136852/testReport)** for PR 32036 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31980: [SPARK-34807][SQL] Transpose Window nodes with Project between them

AmplabJenkins removed a comment on pull request #31980: URL: https://github.com/apache/spark/pull/31980#issuecomment-812458880 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41425/

[GitHub] [spark] AmplabJenkins commented on pull request #31980: [SPARK-34807][SQL] Transpose Window nodes with Project between them

AmplabJenkins commented on pull request #31980: URL: https://github.com/apache/spark/pull/31980#issuecomment-812458880 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41425/ --

[GitHub] [spark] SparkQA commented on pull request #31980: [SPARK-34807][SQL] Transpose Window nodes with Project between them

SparkQA commented on pull request #31980: URL: https://github.com/apache/spark/pull/31980#issuecomment-812450247 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

AmplabJenkins removed a comment on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-812443242 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41427/

[GitHub] [spark] AmplabJenkins commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

AmplabJenkins commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-812443242 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41427/ --

[GitHub] [spark] SparkQA commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

SparkQA commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-812443229 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41427/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31968: [SPARK-34873][SQL] Avoid wrapped in withNewExecutionId twice when run SQL with side effects

AmplabJenkins removed a comment on pull request #31968: URL: https://github.com/apache/spark/pull/31968#issuecomment-812431877 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41426/

[GitHub] [spark] SparkQA commented on pull request #31968: [SPARK-34873][SQL] Avoid wrapped in withNewExecutionId twice when run SQL with side effects

SparkQA commented on pull request #31968: URL: https://github.com/apache/spark/pull/31968#issuecomment-812431845 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] AmplabJenkins commented on pull request #31968: [SPARK-34873][SQL] Avoid wrapped in withNewExecutionId twice when run SQL with side effects

AmplabJenkins commented on pull request #31968: URL: https://github.com/apache/spark/pull/31968#issuecomment-812431877 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41426/ --

[GitHub] [spark] SparkQA commented on pull request #32014: [SPARK-34922][SQL] Use a relative cost comparison function in the CBO

SparkQA commented on pull request #32014: URL: https://github.com/apache/spark/pull/32014#issuecomment-812428311 **[Test build #136851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136851/testReport)** for PR 32014 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31257: [SPARK-33630][SQL] Support SHOW TABLES command as table valued function

AmplabJenkins removed a comment on pull request #31257: URL: https://github.com/apache/spark/pull/31257#issuecomment-812427221 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41424/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

AmplabJenkins removed a comment on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812427218 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41420/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32015: [SPARK-34821][INFRA] Set up a workflow for developers to run benchmark in their fork

AmplabJenkins removed a comment on pull request #32015: URL: https://github.com/apache/spark/pull/32015#issuecomment-812427219 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41423/

[GitHub] [spark] AmplabJenkins commented on pull request #31257: [SPARK-33630][SQL] Support SHOW TABLES command as table valued function

AmplabJenkins commented on pull request #31257: URL: https://github.com/apache/spark/pull/31257#issuecomment-812427221 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41424/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

AmplabJenkins commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812427218 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41420/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32015: [SPARK-34821][INFRA] Set up a workflow for developers to run benchmark in their fork

AmplabJenkins commented on pull request #32015: URL: https://github.com/apache/spark/pull/32015#issuecomment-812427219 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41423/ --

[GitHub] [spark] SparkQA commented on pull request #32015: [SPARK-34821][INFRA] Set up a workflow for developers to run benchmark in their fork

SparkQA commented on pull request #32015: URL: https://github.com/apache/spark/pull/32015#issuecomment-812422089 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41423/ -- This is an automated message from the

[GitHub] [spark] Ngone51 commented on a change in pull request #32033: [SPARK-34939][CORE] Throw fetch failure exception when unable to deserialize broadcasted map statuses

Ngone51 commented on a change in pull request #32033: URL: https://github.com/apache/spark/pull/32033#discussion_r606133973 ## File path: core/src/main/scala/org/apache/spark/shuffle/FetchFailedException.scala ## @@ -68,5 +68,6 @@ private[spark] class FetchFailedException(

[GitHub] [spark] maropu commented on a change in pull request #32022: [SPARK-34932][SQL] Ignore the groupBy expressions in GROUP BY ... GROUPING SETS

maropu commented on a change in pull request #32022: URL: https://github.com/apache/spark/pull/32022#discussion_r606129184 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala ## @@ -914,19 +914,19 @@ class AstBuilder extends

[GitHub] [spark] maropu commented on a change in pull request #32022: [SPARK-34932][SQL] Ignore the groupBy expressions in GROUP BY ... GROUPING SETS

maropu commented on a change in pull request #32022: URL: https://github.com/apache/spark/pull/32022#discussion_r606129184 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala ## @@ -914,19 +914,19 @@ class AstBuilder extends

[GitHub] [spark] SparkQA commented on pull request #31257: [SPARK-33630][SQL] Support SHOW TABLES command as table valued function

SparkQA commented on pull request #31257: URL: https://github.com/apache/spark/pull/31257#issuecomment-812405948 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] Ngone51 commented on pull request #32033: [SPARK-34939][CORE] Throw fetch failure exception when unable to deserialize broadcasted map statuses

Ngone51 commented on pull request #32033: URL: https://github.com/apache/spark/pull/32033#issuecomment-812405408 This seems to be the same issue with https://github.com/apache/spark/pull/27604. cc @liupc @cloud-fan -- This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] SparkQA commented on pull request #32015: [SPARK-34821][INFRA] Set up a workflow for developers to run benchmark in their fork

SparkQA commented on pull request #32015: URL: https://github.com/apache/spark/pull/32015#issuecomment-812405165 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41423/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

SparkQA commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812405061 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] dbtsai commented on a change in pull request #32033: [SPARK-34939][CORE] Throw fetch failure exception when unable to deserialize broadcasted map statuses

dbtsai commented on a change in pull request #32033: URL: https://github.com/apache/spark/pull/32033#discussion_r606123297 ## File path: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ## @@ -953,13 +959,19 @@ private[spark] object MapOutputTracker extends Logging

[GitHub] [spark] dbtsai commented on a change in pull request #32033: [SPARK-34939][CORE] Throw fetch failure exception when unable to deserialize broadcasted map statuses

dbtsai commented on a change in pull request #32033: URL: https://github.com/apache/spark/pull/32033#discussion_r606122357 ## File path: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ## @@ -843,7 +843,13 @@ private[spark] class MapOutputTrackerWorker(conf:

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32022: [SPARK-34932][SQL] Ignore the groupBy expressions in GROUP BY ... GROUPING SETS

AmplabJenkins removed a comment on pull request #32022: URL: https://github.com/apache/spark/pull/32022#issuecomment-812398775 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41422/

[GitHub] [spark] dbtsai commented on a change in pull request #32033: [SPARK-34939][CORE] Throw fetch failure exception when unable to deserialize broadcasted map statuses

dbtsai commented on a change in pull request #32033: URL: https://github.com/apache/spark/pull/32033#discussion_r606122022 ## File path: core/src/main/scala/org/apache/spark/MapOutputTracker.scala ## @@ -843,7 +843,13 @@ private[spark] class MapOutputTrackerWorker(conf:

[GitHub] [spark] SparkQA commented on pull request #32022: [SPARK-34932][SQL] Ignore the groupBy expressions in GROUP BY ... GROUPING SETS

SparkQA commented on pull request #32022: URL: https://github.com/apache/spark/pull/32022#issuecomment-812398763 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41422/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32022: [SPARK-34932][SQL] Ignore the groupBy expressions in GROUP BY ... GROUPING SETS

AmplabJenkins commented on pull request #32022: URL: https://github.com/apache/spark/pull/32022#issuecomment-812398775 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41422/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis r

AmplabJenkins removed a comment on pull request #32032: URL: https://github.com/apache/spark/pull/32032#issuecomment-812397312 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41421/

[GitHub] [spark] AmplabJenkins commented on pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis rules run

AmplabJenkins commented on pull request #32032: URL: https://github.com/apache/spark/pull/32032#issuecomment-812397312 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41421/ --

[GitHub] [spark] SparkQA commented on pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis rules run.

SparkQA commented on pull request #32032: URL: https://github.com/apache/spark/pull/32032#issuecomment-812397290 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #29871: [SPARK-32995][SQL] CostBasedJoinReorder optimizer rule should be idempotent

SparkQA commented on pull request #29871: URL: https://github.com/apache/spark/pull/29871#issuecomment-812397139 **[Test build #136850 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136850/testReport)** for PR 29871 at commit

[GitHub] [spark] SparkQA commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

SparkQA commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-812397113 **[Test build #136849 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136849/testReport)** for PR 30057 at commit

[GitHub] [spark] SparkQA commented on pull request #31968: [SPARK-34873][SQL] Avoid wrapped in withNewExecutionId twice when run SQL with side effects

SparkQA commented on pull request #31968: URL: https://github.com/apache/spark/pull/31968#issuecomment-812396717 **[Test build #136848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136848/testReport)** for PR 31968 at commit

[GitHub] [spark] SparkQA commented on pull request #31980: [SPARK-34807][SQL] Transpose Window nodes with Project between them

SparkQA commented on pull request #31980: URL: https://github.com/apache/spark/pull/31980#issuecomment-812396690 **[Test build #136847 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136847/testReport)** for PR 31980 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32033: [SPARK-34939][CORE] Throw fetch failure exception when unable to deserialize broadcasted map statuses

AmplabJenkins removed a comment on pull request #32033: URL: https://github.com/apache/spark/pull/32033#issuecomment-812396091 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136839/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis r

AmplabJenkins removed a comment on pull request #32032: URL: https://github.com/apache/spark/pull/32032#issuecomment-812396092 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136840/

[GitHub] [spark] AmplabJenkins commented on pull request #32033: [SPARK-34939][CORE] Throw fetch failure exception when unable to deserialize broadcasted map statuses

AmplabJenkins commented on pull request #32033: URL: https://github.com/apache/spark/pull/32033#issuecomment-812396091 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136839/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis rules run

AmplabJenkins commented on pull request #32032: URL: https://github.com/apache/spark/pull/32032#issuecomment-812396092 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136840/ -- This

[GitHub] [spark] HyukjinKwon commented on pull request #31980: [SPARK-34807][SQL] Transpose Window nodes with Project between them

HyukjinKwon commented on pull request #31980: URL: https://github.com/apache/spark/pull/31980#issuecomment-812396108 This looks fine too but it would be great if we can have a sign-off from @hvanhovell too .. he has much better insight than I have -- This is an automated message from

[GitHub] [spark] SparkQA removed a comment on pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis rules r

SparkQA removed a comment on pull request #32032: URL: https://github.com/apache/spark/pull/32032#issuecomment-812315631 **[Test build #136840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136840/testReport)** for PR 32032 at commit

[GitHub] [spark] SparkQA commented on pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis rules run.

SparkQA commented on pull request #32032: URL: https://github.com/apache/spark/pull/32032#issuecomment-812393980 **[Test build #136840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136840/testReport)** for PR 32032 at commit

[GitHub] [spark] maropu commented on a change in pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

maropu commented on a change in pull request #32036: URL: https://github.com/apache/spark/pull/32036#discussion_r606117085 ## File path: python/pyspark/pandas/__init__.py ## @@ -0,0 +1,209 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] [spark] AngersZhuuuu commented on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

AngersZh commented on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812393166 > BTW, @AngersZh does the issue exist in 3.1/3.0/2.4? If so, please, backport the changes. Need to check. I will update here after check this -- This is an

[GitHub] [spark] AngersZhuuuu commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

AngersZh commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-812392769 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA removed a comment on pull request #32033: [SPARK-34939][CORE] Throw fetch failure exception when unable to deserialize broadcasted map statuses

SparkQA removed a comment on pull request #32033: URL: https://github.com/apache/spark/pull/32033#issuecomment-812315600 **[Test build #136839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136839/testReport)** for PR 32033 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

HyukjinKwon commented on a change in pull request #32036: URL: https://github.com/apache/spark/pull/32036#discussion_r606114750 ## File path: python/pyspark/pandas/__init__.py ## @@ -0,0 +1,209 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] [spark] SparkQA commented on pull request #32033: [SPARK-34939][CORE] Throw fetch failure exception when unable to deserialize broadcasted map statuses

SparkQA commented on pull request #32033: URL: https://github.com/apache/spark/pull/32033#issuecomment-812390916 **[Test build #136839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136839/testReport)** for PR 32033 at commit

[GitHub] [spark] maropu commented on a change in pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

maropu commented on a change in pull request #32036: URL: https://github.com/apache/spark/pull/32036#discussion_r606114026 ## File path: python/pyspark/pandas/__init__.py ## @@ -0,0 +1,209 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] [spark] MaxGekk commented on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

MaxGekk commented on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812385399 BTW, @AngersZh does the issue exist in 3.1/3.0/2.4? If so, please, backport the changes. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] viirya commented on a change in pull request #31451: [SPARK-34338][SQL] Report metrics from Datasource v2 scan

viirya commented on a change in pull request #31451: URL: https://github.com/apache/spark/pull/31451#discussion_r606111670 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/PartitionReader.java ## @@ -51,7 +51,8 @@ T get(); /** - * Returns

[GitHub] [spark] MaxGekk closed pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

MaxGekk closed pull request #32018: URL: https://github.com/apache/spark/pull/32018 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] MaxGekk commented on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

MaxGekk commented on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812380712 +1, LGTM. Merging to master. The failed GA is a known issue. Thank you @AngersZh, and @cloud-fan @wangyum for your review. -- This is an automated message from the

[GitHub] [spark] c21 commented on pull request #32034: [SPARK-34940][SQL][TEST] Fix test of BasicWriteTaskStatsTrackerSuite

c21 commented on pull request #32034: URL: https://github.com/apache/spark/pull/32034#issuecomment-812374473 Thank you @HyukjinKwon, @viirya and @cloud-fan for the quick review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

HyukjinKwon commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812363612 Probably we should at least include their GitHub IDs in the commit message. Let me address it. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] HyukjinKwon commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

HyukjinKwon commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812363089 We could .. but that will include too many commits (https://github.com/databricks/koalas/graphs/contributors). I know people use that status for many other purposes, e.g.)

[GitHub] [spark] MaxGekk commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

MaxGekk commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812358687 Is it possible to preserve commits history from the koalas repo (include all commits) otherwise you will hide the work of others contributors to the Koalas project. -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

AmplabJenkins removed a comment on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812357189 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136842/

[GitHub] [spark] SparkQA removed a comment on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

SparkQA removed a comment on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812356080 **[Test build #136842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136842/testReport)** for PR 32036 at commit

[GitHub] [spark] SparkQA commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

SparkQA commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812357176 **[Test build #136842 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136842/testReport)** for PR 32036 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

AmplabJenkins commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812357189 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136842/ -- This

[GitHub] [spark] HyukjinKwon commented on pull request #32034: [SPARK-34940][SQL][TEST] Fix test of BasicWriteTaskStatsTrackerSuite

HyukjinKwon commented on pull request #32034: URL: https://github.com/apache/spark/pull/32034#issuecomment-812356948 Merged to master, branch-3.1 and branch-3.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] HyukjinKwon closed pull request #32034: [SPARK-34940][SQL][TEST] Fix test of BasicWriteTaskStatsTrackerSuite

HyukjinKwon closed pull request #32034: URL: https://github.com/apache/spark/pull/32034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] SparkQA commented on pull request #32015: [SPARK-34821][INFRA] Set up a workflow for developers to run benchmark in their fork

SparkQA commented on pull request #32015: URL: https://github.com/apache/spark/pull/32015#issuecomment-812356191 **[Test build #136846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136846/testReport)** for PR 32015 at commit

[GitHub] [spark] SparkQA commented on pull request #32022: [SPARK-34932][SQL] Ignore the groupBy expressions in GROUP BY ... GROUPING SETS

SparkQA commented on pull request #32022: URL: https://github.com/apache/spark/pull/32022#issuecomment-812356161 **[Test build #136845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136845/testReport)** for PR 32022 at commit

[GitHub] [spark] SparkQA commented on pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

SparkQA commented on pull request #32026: URL: https://github.com/apache/spark/pull/32026#issuecomment-812356123 **[Test build #136844 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136844/testReport)** for PR 32026 at commit

[GitHub] [spark] SparkQA commented on pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis rules run.

SparkQA commented on pull request #32032: URL: https://github.com/apache/spark/pull/32032#issuecomment-812356091 **[Test build #136843 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136843/testReport)** for PR 32032 at commit

[GitHub] [spark] SparkQA commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

SparkQA commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812356080 **[Test build #136842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136842/testReport)** for PR 32036 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31968: [SPARK-34873][SQL] Avoid wrapped in withNewExecutionId twice when run SQL with side effects

AmplabJenkins removed a comment on pull request #31968: URL: https://github.com/apache/spark/pull/31968#issuecomment-812355321 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136837/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32034: [SPARK-34940][SQL][TEST] Fix test of BasicWriteTaskStatsTrackerSuite

AmplabJenkins removed a comment on pull request #32034: URL: https://github.com/apache/spark/pull/32034#issuecomment-812355318 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136838/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

AmplabJenkins removed a comment on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812355320 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41419/

[GitHub] [spark] AmplabJenkins commented on pull request #31968: [SPARK-34873][SQL] Avoid wrapped in withNewExecutionId twice when run SQL with side effects

AmplabJenkins commented on pull request #31968: URL: https://github.com/apache/spark/pull/31968#issuecomment-812355321 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136837/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32034: [SPARK-34940][SQL][TEST] Fix test of BasicWriteTaskStatsTrackerSuite

AmplabJenkins commented on pull request #32034: URL: https://github.com/apache/spark/pull/32034#issuecomment-812355318 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136838/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

AmplabJenkins commented on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812355320 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41419/ --

[GitHub] [spark] HyukjinKwon closed pull request #32035: [SPARK-34938][SQL][TESTS] Benchmark only legacy interval in `ExtractBenchmark`

HyukjinKwon closed pull request #32035: URL: https://github.com/apache/spark/pull/32035 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] HyukjinKwon commented on pull request #32035: [SPARK-34938][SQL][TESTS] Benchmark only legacy interval in `ExtractBenchmark`

HyukjinKwon commented on pull request #32035: URL: https://github.com/apache/spark/pull/32035#issuecomment-812353890 None of tests actually verfiies this changes except comliation and linter which passed. Merged to master -- This is an automated message from the Apache Git

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32015: [SPARK-34821][INFRA] Set up a workflow for developers to run benchmark in their fork

HyukjinKwon commented on a change in pull request #32015: URL: https://github.com/apache/spark/pull/32015#discussion_r606097289 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala ## @@ -92,8 +92,9 @@ object ExtractBenchmark

[GitHub] [spark] imback82 removed a comment on pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis rules

imback82 removed a comment on pull request #32032: URL: https://github.com/apache/spark/pull/32032#issuecomment-812344802 Actually, I need to do the transformation (removing children) after `checkAnalysis`, or just run `checkAnalysis` in the `TransformAfterAnalysis` rule. -- This is an

[GitHub] [spark] itholic commented on a change in pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

itholic commented on a change in pull request #32036: URL: https://github.com/apache/spark/pull/32036#discussion_r606097006 ## File path: python/pyspark/pandas/__init__.py ## @@ -0,0 +1,208 @@ +# +# Copyright (C) 2019 Databricks, Inc. Review comment: Oh, thanks :)

[GitHub] [spark] SparkQA commented on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

SparkQA commented on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812352039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] imback82 edited a comment on pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis rules r

imback82 edited a comment on pull request #32032: URL: https://github.com/apache/spark/pull/32032#issuecomment-812344802 Actually, I need to do the transformation (removing children) after `checkAnalysis`, or just run `checkAnalysis` in the `TransformAfterAnalysis` rule. -- This is an

[GitHub] [spark] HyukjinKwon commented on pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

HyukjinKwon commented on pull request #32036: URL: https://github.com/apache/spark/pull/32036#issuecomment-812350828 cc @ueshin, @xinrong-databricks, @rxin from Koalas dev FYI. It would be great if we have approvals from non-Koalas dev especially to make sure if your plan in PR

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32036: [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark

HyukjinKwon commented on a change in pull request #32036: URL: https://github.com/apache/spark/pull/32036#discussion_r606094415 ## File path: python/pyspark/pandas/__init__.py ## @@ -0,0 +1,208 @@ +# +# Copyright (C) 2019 Databricks, Inc. Review comment: Oh yeah.

[GitHub] [spark] SparkQA removed a comment on pull request #31968: [SPARK-34873][SQL] Avoid wrapped in withNewExecutionId twice when run SQL with side effects

SparkQA removed a comment on pull request #31968: URL: https://github.com/apache/spark/pull/31968#issuecomment-812301835 **[Test build #136837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136837/testReport)** for PR 31968 at commit

<    1   2   3   4   5   >