[GitHub] [spark] SparkQA commented on pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-07-02 Thread GitBox
SparkQA commented on pull request #33200: URL: https://github.com/apache/spark/pull/33200#issuecomment-873358947 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45121/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33200: URL: https://github.com/apache/spark/pull/33200#issuecomment-873358952 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45121/ -- T

[GitHub] [spark] SparkQA commented on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

2021-07-02 Thread GitBox
SparkQA commented on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-873357656 **[Test build #140611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140611/testReport)** for PR 33101 at commit [`6d123ba`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33188: [SPARK-35989][SQL] Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled

2021-07-02 Thread GitBox
SparkQA commented on pull request #33188: URL: https://github.com/apache/spark/pull/33188#issuecomment-873356608 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45123/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-07-02 Thread GitBox
SparkQA commented on pull request #33200: URL: https://github.com/apache/spark/pull/33200#issuecomment-873355831 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45122/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-07-02 Thread GitBox
SparkQA commented on pull request #33200: URL: https://github.com/apache/spark/pull/33200#issuecomment-873355459 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45121/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33188: [SPARK-35989][SQL] Do not remove REPARTITION_BY_NUM shuffle if AQE is enabled

2021-07-02 Thread GitBox
SparkQA commented on pull request #33188: URL: https://github.com/apache/spark/pull/33188#issuecomment-873352553 **[Test build #140610 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140610/testReport)** for PR 33188 at commit [`b2822d5`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-07-02 Thread GitBox
SparkQA commented on pull request #33200: URL: https://github.com/apache/spark/pull/33200#issuecomment-873351326 **[Test build #140609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140609/testReport)** for PR 33200 at commit [`f729d8d`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-873351009 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140608/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-873351009 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140608/ -- This

[GitHub] [spark] imback82 opened a new pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-07-02 Thread GitBox
imback82 opened a new pull request #33200: URL: https://github.com/apache/spark/pull/33200 ### What changes were proposed in this pull request? This PR proposes to migrate the following `ALTER TABLE ... ADD/REPLACE COLUMNS` commands to use `UnresolvedTable` as a `child` to re

[GitHub] [spark] viirya commented on pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

2021-07-02 Thread GitBox
viirya commented on pull request #33142: URL: https://github.com/apache/spark/pull/33142#issuecomment-873349727 @maropu Any more comments? Otherwise I will merge this tomorrow. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

2021-07-02 Thread GitBox
SparkQA removed a comment on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-873332565 **[Test build #140608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140608/testReport)** for PR 33101 at commit [`a958a5e`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

2021-07-02 Thread GitBox
SparkQA commented on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-873346816 **[Test build #140608 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140608/testReport)** for PR 33101 at commit [`a958a5e`](https://github.co

[GitHub] [spark] gengliangwang commented on pull request #33192: [SPARK-35994][INFRA] Publish snapshot from branch-3.2

2021-07-02 Thread GitBox
gengliangwang commented on pull request #33192: URL: https://github.com/apache/spark/pull/33192#issuecomment-873346434 late lgtm -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [spark] Peng-Lei commented on a change in pull request #33175: [SPARK-35973][SQL] DataSourceV2: Support SHOW CATALOGS

2021-07-02 Thread GitBox
Peng-Lei commented on a change in pull request #33175: URL: https://github.com/apache/spark/pull/33175#discussion_r663311709 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala ## @@ -662,6 +662,15 @@ case class ShowCurrentNam

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-873338716 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45120/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33199: URL: https://github.com/apache/spark/pull/33199#issuecomment-873338717 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140607/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33199: URL: https://github.com/apache/spark/pull/33199#issuecomment-873338717 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140607/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-873338716 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45120/ -- T

[GitHub] [spark] Peng-Lei commented on a change in pull request #33175: [SPARK-35973][SQL] DataSourceV2: Support SHOW CATALOGS

2021-07-02 Thread GitBox
Peng-Lei commented on a change in pull request #33175: URL: https://github.com/apache/spark/pull/33175#discussion_r663311709 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala ## @@ -662,6 +662,15 @@ case class ShowCurrentNam

[GitHub] [spark] Peng-Lei commented on a change in pull request #33175: [SPARK-35973][SQL] DataSourceV2: Support SHOW CATALOGS

2021-07-02 Thread GitBox
Peng-Lei commented on a change in pull request #33175: URL: https://github.com/apache/spark/pull/33175#discussion_r663311232 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogManager.scala ## @@ -128,6 +128,10 @@ class CatalogManager(

[GitHub] [spark] Peng-Lei commented on a change in pull request #33175: [SPARK-35973][SQL] DataSourceV2: Support SHOW CATALOGS

2021-07-02 Thread GitBox
Peng-Lei commented on a change in pull request #33175: URL: https://github.com/apache/spark/pull/33175#discussion_r663311232 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogManager.scala ## @@ -128,6 +128,10 @@ class CatalogManager(

[GitHub] [spark] SparkQA removed a comment on pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
SparkQA removed a comment on pull request #33199: URL: https://github.com/apache/spark/pull/33199#issuecomment-873319682 **[Test build #140607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140607/testReport)** for PR 33199 at commit [`78c6a45`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
SparkQA commented on pull request #33199: URL: https://github.com/apache/spark/pull/33199#issuecomment-873336422 **[Test build #140607 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140607/testReport)** for PR 33199 at commit [`78c6a45`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

2021-07-02 Thread GitBox
SparkQA commented on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-873336408 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45120/ -- This

[GitHub] [spark] SparkQA commented on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

2021-07-02 Thread GitBox
SparkQA commented on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-873332565 **[Test build #140608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140608/testReport)** for PR 33101 at commit [`a958a5e`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33199: URL: https://github.com/apache/spark/pull/33199#issuecomment-873332196 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45119/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-873332195 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140604/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33199: URL: https://github.com/apache/spark/pull/33199#issuecomment-873332196 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45119/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-873332195 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140604/ -- This

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

2021-07-02 Thread GitBox
dongjoon-hyun commented on a change in pull request #33091: URL: https://github.com/apache/spark/pull/33091#discussion_r663305160 ## File path: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala ## @@ -41,10 +41,17 @@ import org.apache.spark.sql.streaming.Si

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31611: [SPARK-34488][CORE] Support task Metrics Distributions and executor Metrics Distributions in the REST API call for a specif

2021-07-02 Thread GitBox
dongjoon-hyun commented on a change in pull request #31611: URL: https://github.com/apache/spark/pull/31611#discussion_r663305042 ## File path: core/src/main/scala/org/apache/spark/status/api/v1/api.scala ## @@ -279,7 +292,9 @@ class StageData private[spark]( val resourceP

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31948: [SPARK-34848][CORE] Add duration to TaskMetricDistributions

2021-07-02 Thread GitBox
dongjoon-hyun commented on a change in pull request #31948: URL: https://github.com/apache/spark/pull/31948#discussion_r663304856 ## File path: core/src/main/scala/org/apache/spark/status/api/v1/api.scala ## @@ -342,6 +342,7 @@ class ShuffleWriteMetrics private[spark]( class T

[GitHub] [spark] SparkQA commented on pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
SparkQA commented on pull request #33199: URL: https://github.com/apache/spark/pull/33199#issuecomment-873329809 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45119/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-02 Thread GitBox
SparkQA removed a comment on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-873272508 **[Test build #140604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140604/testReport)** for PR 32049 at commit [`1648070`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-02 Thread GitBox
SparkQA commented on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-873328214 **[Test build #140604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140604/testReport)** for PR 32049 at commit [`1648070`](https://github.co

[GitHub] [spark] Shockang commented on a change in pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

2021-07-02 Thread GitBox
Shockang commented on a change in pull request #33101: URL: https://github.com/apache/spark/pull/33101#discussion_r663302600 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -285,9 +285,10 @@ private[spark] object Utils extends Logging { */ def c

[GitHub] [spark] AmplabJenkins commented on pull request #33178: [SPARK-35980][CORE] ThreadAudit logs whether thread is daemon

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33178: URL: https://github.com/apache/spark/pull/33178#issuecomment-873327425 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] SparkQA commented on pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
SparkQA commented on pull request #33199: URL: https://github.com/apache/spark/pull/33199#issuecomment-873325641 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45119/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873325219 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45118/

[GitHub] [spark] AmplabJenkins commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873325219 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45118/ -- T

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
dongjoon-hyun commented on a change in pull request #33199: URL: https://github.com/apache/spark/pull/33199#discussion_r663296840 ## File path: project/MimaBuild.scala ## @@ -86,7 +86,7 @@ object MimaBuild { def mimaSettings(sparkHome: File, projectRef: ProjectRef): Seq[Se

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #33196: [SPARK-35996][BUILD] Setting version to 3.3.0-SNAPSHOT

2021-07-02 Thread GitBox
dongjoon-hyun edited a comment on pull request #33196: URL: https://github.com/apache/spark/pull/33196#issuecomment-873250800 I fully agree with your points and I believe we can do that in a transitive manner in theory as you describe. That will remove the hiding cases you mentioned.

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873320323 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45118/ -- This is an automated message from the A

[GitHub] [spark] dongjoon-hyun commented on pull request #33196: [SPARK-35996][BUILD] Setting version to 3.3.0-SNAPSHOT

2021-07-02 Thread GitBox
dongjoon-hyun commented on pull request #33196: URL: https://github.com/apache/spark/pull/33196#issuecomment-873320336 I made a PR for MiMa and the MiMa-detected changes from 3.1.1. - https://github.com/apache/spark/pull/33199 -- This is an automated message from the Apache Git Service

[GitHub] [spark] dongjoon-hyun commented on pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
dongjoon-hyun commented on pull request #33199: URL: https://github.com/apache/spark/pull/33199#issuecomment-873320180 Hi, All. Although I checked the PR, I need your confirmation for this MiMa-detected change from 3.1.1. cc @gengliangwang -- This is an automated message from

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
dongjoon-hyun commented on a change in pull request #33199: URL: https://github.com/apache/spark/pull/33199#discussion_r663296376 ## File path: project/MimaExcludes.scala ## @@ -34,12 +34,42 @@ import com.typesafe.tools.mima.core.ProblemFilters._ */ object MimaExcludes { -

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
dongjoon-hyun commented on a change in pull request #33199: URL: https://github.com/apache/spark/pull/33199#discussion_r663296258 ## File path: project/MimaExcludes.scala ## @@ -34,12 +34,42 @@ import com.typesafe.tools.mima.core.ProblemFilters._ */ object MimaExcludes { -

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
dongjoon-hyun commented on a change in pull request #33199: URL: https://github.com/apache/spark/pull/33199#discussion_r663296098 ## File path: project/MimaExcludes.scala ## @@ -34,12 +34,42 @@ import com.typesafe.tools.mima.core.ProblemFilters._ */ object MimaExcludes { -

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
dongjoon-hyun commented on a change in pull request #33199: URL: https://github.com/apache/spark/pull/33199#discussion_r663296376 ## File path: project/MimaExcludes.scala ## @@ -34,12 +34,42 @@ import com.typesafe.tools.mima.core.ProblemFilters._ */ object MimaExcludes { -

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
dongjoon-hyun commented on a change in pull request #33199: URL: https://github.com/apache/spark/pull/33199#discussion_r663296258 ## File path: project/MimaExcludes.scala ## @@ -34,12 +34,42 @@ import com.typesafe.tools.mima.core.ProblemFilters._ */ object MimaExcludes { -

[GitHub] [spark] SparkQA commented on pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
SparkQA commented on pull request #33199: URL: https://github.com/apache/spark/pull/33199#issuecomment-873319682 **[Test build #140607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140607/testReport)** for PR 33199 at commit [`78c6a45`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
dongjoon-hyun commented on a change in pull request #33199: URL: https://github.com/apache/spark/pull/33199#discussion_r663296098 ## File path: project/MimaExcludes.scala ## @@ -34,12 +34,42 @@ import com.typesafe.tools.mima.core.ProblemFilters._ */ object MimaExcludes { -

[GitHub] [spark] dongjoon-hyun opened a new pull request #33199: [SPARK-36004][INFRA] Update MiMa and audit API changes

2021-07-02 Thread GitBox
dongjoon-hyun opened a new pull request #33199: URL: https://github.com/apache/spark/pull/33199 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### H

[GitHub] [spark] dgd-contributor commented on pull request #32951: [SPARK-33603][SQL] Grouping execution/command

2021-07-02 Thread GitBox
dgd-contributor commented on pull request #32951: URL: https://github.com/apache/spark/pull/32951#issuecomment-873319022 cc @allisonwang-db -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873318107 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140605/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873318107 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140605/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873308873 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45117/

[GitHub] [spark] SparkQA removed a comment on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA removed a comment on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873290590 **[Test build #140605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140605/testReport)** for PR 33162 at commit [`7a34527`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873317838 **[Test build #140605 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140605/testReport)** for PR 33162 at commit [`7a34527`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873315228 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45118/ -- This is an automated message from the Apache

[GitHub] [spark] github-actions[bot] closed pull request #29754: [SPARK-32875][CORE][TEST] TaskSchedulerImplSuite: For the pattern of submitTasks + resourceOffers + assert, extract the general method.

2021-07-02 Thread GitBox
github-actions[bot] closed pull request #29754: URL: https://github.com/apache/spark/pull/29754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: re

[GitHub] [spark] github-actions[bot] commented on pull request #31598: [SPARK-34478][SQL] When build SparkSession, we should check config keys

2021-07-02 Thread GitBox
github-actions[bot] commented on pull request #31598: URL: https://github.com/apache/spark/pull/31598#issuecomment-873311100 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] HyukjinKwon commented on pull request #33197: [SPARK-35995][INFRA][3.2][FOLLOWUP] Fix the reference branches and PR builder

2021-07-02 Thread GitBox
HyukjinKwon commented on pull request #33197: URL: https://github.com/apache/spark/pull/33197#issuecomment-873309041 Thanks Takuya! I was about to take a look yesterday but was stuck in some personal works ... -- This is an automated message from the Apache Git Service. To respond to th

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873308868 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45117/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873308873 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45117/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33185: [SPARK-35986][PYSPARK] Fix type hint for RDD.histogram's buckets

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33185: URL: https://github.com/apache/spark/pull/33185#issuecomment-873308627 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] SparkQA removed a comment on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA removed a comment on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873306368 **[Test build #140606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140606/testReport)** for PR 33162 at commit [`780fc1f`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873306599 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140606/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873306599 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140606/ -- This

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873306592 **[Test build #140606 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140606/testReport)** for PR 33162 at commit [`780fc1f`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873306368 **[Test build #140606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140606/testReport)** for PR 33162 at commit [`780fc1f`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33197: [SPARK-35995][INFRA][3.2][FOLLOWUP] Fix the reference branches and PR builder

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33197: URL: https://github.com/apache/spark/pull/33197#issuecomment-873305879 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-873305878 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45116/

[GitHub] [spark] AmplabJenkins commented on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-873305878 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45116/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33197: [SPARK-35995][INFRA][3.2][FOLLOWUP] Fix the reference branches and PR builder

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33197: URL: https://github.com/apache/spark/pull/33197#issuecomment-873305879 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] dongjoon-hyun commented on pull request #33196: [SPARK-35996][BUILD] Setting version to 3.3.0-SNAPSHOT

2021-07-02 Thread GitBox
dongjoon-hyun commented on pull request #33196: URL: https://github.com/apache/spark/pull/33196#issuecomment-873305311 @srowen . I took a look quickly. As you were worried, it seems that Apache Spark 3.2.0 seems to have hidden MiMa failure since 3.1.1. In this case, that should be an offic

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873302133 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45117/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33197: [SPARK-35995][INFRA][3.2][FOLLOWUP] Fix the reference branches and PR builder

2021-07-02 Thread GitBox
SparkQA removed a comment on pull request #33197: URL: https://github.com/apache/spark/pull/33197#issuecomment-873252026 **[Test build #140600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140600/testReport)** for PR 33197 at commit [`2e4151b`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33197: [SPARK-35995][INFRA][3.2][FOLLOWUP] Fix the reference branches and PR builder

2021-07-02 Thread GitBox
SparkQA commented on pull request #33197: URL: https://github.com/apache/spark/pull/33197#issuecomment-873300208 **[Test build #140600 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140600/testReport)** for PR 33197 at commit [`2e4151b`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33197: [SPARK-35995][INFRA][3.2][FOLLOWUP] Fix the reference branches and PR builder

2021-07-02 Thread GitBox
SparkQA removed a comment on pull request #33197: URL: https://github.com/apache/spark/pull/33197#issuecomment-873249038 **[Test build #140598 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140598/testReport)** for PR 33197 at commit [`45fc550`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33197: [SPARK-35995][INFRA][3.2][FOLLOWUP] Fix the reference branches and PR builder

2021-07-02 Thread GitBox
SparkQA commented on pull request #33197: URL: https://github.com/apache/spark/pull/33197#issuecomment-873297921 **[Test build #140598 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140598/testReport)** for PR 33197 at commit [`45fc550`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-02 Thread GitBox
SparkQA commented on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-873296652 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45116/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #33183: [SPARK-35972][SQL] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

2021-07-02 Thread GitBox
SparkQA removed a comment on pull request #33183: URL: https://github.com/apache/spark/pull/33183#issuecomment-873192390 **[Test build #140596 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140596/testReport)** for PR 33183 at commit [`d3262ed`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33183: [SPARK-35972][SQL] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33183: URL: https://github.com/apache/spark/pull/33183#issuecomment-873294316 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140596/ -- This

[GitHub] [spark] SparkQA commented on pull request #33183: [SPARK-35972][SQL] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

2021-07-02 Thread GitBox
SparkQA commented on pull request #33183: URL: https://github.com/apache/spark/pull/33183#issuecomment-873293726 **[Test build #140596 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140596/testReport)** for PR 33183 at commit [`d3262ed`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873290590 **[Test build #140605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140605/testReport)** for PR 33162 at commit [`7a34527`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33197: [SPARK-35995][INFRA][3.2][FOLLOWUP] Fix the reference branches and PR builder

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33197: URL: https://github.com/apache/spark/pull/33197#issuecomment-873289722 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873289719 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33195: [SPARK-35785][SS][FOLLOWUP] Ignore concurrent update and cleanup test

2021-07-02 Thread GitBox
AmplabJenkins removed a comment on pull request #33195: URL: https://github.com/apache/spark/pull/33195#issuecomment-873289718 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140590/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33197: [SPARK-35995][INFRA][3.2][FOLLOWUP] Fix the reference branches and PR builder

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33197: URL: https://github.com/apache/spark/pull/33197#issuecomment-873289726 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #33195: [SPARK-35785][SS][FOLLOWUP] Ignore concurrent update and cleanup test

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33195: URL: https://github.com/apache/spark/pull/33195#issuecomment-873289718 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140590/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
AmplabJenkins commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873289720 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] SparkQA commented on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-02 Thread GitBox
SparkQA commented on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-873287265 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45116/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873285123 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45115/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA removed a comment on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873272164 **[Test build #140603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140603/testReport)** for PR 33162 at commit [`219732f`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873282978 **[Test build #140603 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140603/testReport)** for PR 33162 at commit [`219732f`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33162: [WIP][SPARK-35615] Make all basic operators data-type-based

2021-07-02 Thread GitBox
SparkQA commented on pull request #33162: URL: https://github.com/apache/spark/pull/33162#issuecomment-873281908 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45114/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33197: [SPARK-35995][INFRA][3.2][FOLLOWUP] Fix the reference branches and PR builder

2021-07-02 Thread GitBox
SparkQA commented on pull request #33197: URL: https://github.com/apache/spark/pull/33197#issuecomment-873281500 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45112/ -- This is an automated message from the A

  1   2   3   4   5   6   7   >