[GitHub] [spark] SparkQA commented on pull request #30819: [SPARK-33819][CORE][3.1] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

SparkQA commented on pull request #30819: URL: https://github.com/apache/spark/pull/30819#issuecomment-747277614 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37539/

[GitHub] [spark] dongjoon-hyun commented on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

dongjoon-hyun commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747277478 Thanks, I merged two issues by copying all description into one JIRA (SPARK-33822 Bug). This is an

[GitHub] [spark] SparkQA commented on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

SparkQA commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747277413 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37540/

[GitHub] [spark] HyukjinKwon commented on pull request #30818: [SPARK-33823][SQL] Use the `CastSupport.cast` method in HashJoin

HyukjinKwon commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747275647 Yeah let's port it back. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] MaxGekk commented on a change in pull request #30776: [SPARK-33787][SQL] Add the `purge` parameter to `dropPartition()` of `SupportsPartitionManagement`

MaxGekk commented on a change in pull request #30776: URL: https://github.com/apache/spark/pull/30776#discussion_r544878422 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/SupportsAtomicPartitionManagement.java ## @@ -82,4 +82,27 @@ void

[GitHub] [spark] MaxGekk opened a new pull request #30821: [SPARK-33787][SQL][3.1] Add the `purge` parameter to `dropPartition()` of `SupportsPartitionManagement`

MaxGekk opened a new pull request #30821: URL: https://github.com/apache/spark/pull/30821 ### What changes were proposed in this pull request? Add default methods `dropPartition()` in `SupportsPartitionManagement` and `dropPartitions()` in `SupportsAtomicPartitionManagement()` with the

[GitHub] [spark] maropu commented on a change in pull request #30790: [SPARK-33798][SQL] Add new rule to push down the foldable expressions through CaseWhen/If

maropu commented on a change in pull request #30790: URL: https://github.com/apache/spark/pull/30790#discussion_r544876405 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala ## @@ -528,6 +528,28 @@ object SimplifyConditionals

[GitHub] [spark] SparkQA removed a comment on pull request #30717: [SPARK-33599][SQL] Group exception messages in catalyst/analysis

SparkQA removed a comment on pull request #30717: URL: https://github.com/apache/spark/pull/30717#issuecomment-747175935 **[Test build #132917 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132917/testReport)** for PR 30717 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #30816: [SPARK-33818][DOC] Doc `spark.sql.parser.quotedRegexColumnNames`

SparkQA removed a comment on pull request #30816: URL: https://github.com/apache/spark/pull/30816#issuecomment-747259249 **[Test build #132938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132938/testReport)** for PR 30816 at commit

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #30818: [SPARK-33823][SQL] Use the `CastSupport.cast` method in HashJoin

dongjoon-hyun edited a comment on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747267974 BTW, @maropu . Why do you make another JIRA for this? Also, the new JIRA seems to be an `Improvement`? Do we need more patch to handle Q5? If you don't mind,

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #30818: [SPARK-33823][SQL] Use the `CastSupport.cast` method in HashJoin

dongjoon-hyun edited a comment on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747267974 BTW, @maropu . Why do you make another JIRA for this? Also, the new JIRA seems to be an `Improvement`? If you don't mind, shall we use the original issue,

[GitHub] [spark] SparkQA commented on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

SparkQA commented on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747268531 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37536/

[GitHub] [spark] SparkQA commented on pull request #30815: [SPARK-33817][SQL] CACHE TABLE uses a logical plan when caching a query to avoid creating a dataframe

SparkQA commented on pull request #30815: URL: https://github.com/apache/spark/pull/30815#issuecomment-747268149 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37537/

[GitHub] [spark] dongjoon-hyun commented on pull request #30818: [SPARK-33823][SQL] Use the `CastSupport.cast` method in HashJoin

dongjoon-hyun commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747267974 BTW, @maropu . Why do you make another JIRA for this? Also, the new JIRA seems to be an `Improvement`?

[GitHub] [spark] kiszk commented on a change in pull request #30810: [WIP][ML] Add a vectorized BLAS implementation

kiszk commented on a change in pull request #30810: URL: https://github.com/apache/spark/pull/30810#discussion_r544868623 ## File path: mllib-local/src/main/scala/org/apache/spark/ml/linalg/BLAS.scala ## @@ -18,28 +18,46 @@ package org.apache.spark.ml.linalg import

[GitHub] [spark] sarutak commented on a change in pull request #30573: [SPARK-26341][WEBUI]Expose executor memory metrics at the stage level, in the Stages tab

sarutak commented on a change in pull request #30573: URL: https://github.com/apache/spark/pull/30573#discussion_r544868604 ## File path: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ## @@ -687,6 +687,9 @@ private[spark] class AppStatusListener(

[GitHub] [spark] AmplabJenkins commented on pull request #30717: [SPARK-33599][SQL] Group exception messages in catalyst/analysis

AmplabJenkins commented on pull request #30717: URL: https://github.com/apache/spark/pull/30717#issuecomment-747264692 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132917/

[GitHub] [spark] AmplabJenkins commented on pull request #30816: [SPARK-33818][DOC] Doc `spark.sql.parser.quotedRegexColumnNames`

AmplabJenkins commented on pull request #30816: URL: https://github.com/apache/spark/pull/30816#issuecomment-747264601 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132938/

[GitHub] [spark] SparkQA commented on pull request #30816: [SPARK-33818][DOC] Doc `spark.sql.parser.quotedRegexColumnNames`

SparkQA commented on pull request #30816: URL: https://github.com/apache/spark/pull/30816#issuecomment-747264450 **[Test build #132938 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132938/testReport)** for PR 30816 at commit

[GitHub] [spark] SparkQA commented on pull request #30717: [SPARK-33599][SQL] Group exception messages in catalyst/analysis

SparkQA commented on pull request #30717: URL: https://github.com/apache/spark/pull/30717#issuecomment-747263881 **[Test build #132917 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132917/testReport)** for PR 30717 at commit

[GitHub] [spark] gengliangwang commented on a change in pull request #30573: [SPARK-26341][WEBUI]Expose executor memory metrics at the stage level, in the Stages tab

gengliangwang commented on a change in pull request #30573: URL: https://github.com/apache/spark/pull/30573#discussion_r544864373 ## File path: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ## @@ -687,6 +687,9 @@ private[spark] class AppStatusListener(

[GitHub] [spark] SparkQA commented on pull request #30820: [SPARK-33819][CORE][3.0] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

SparkQA commented on pull request #30820: URL: https://github.com/apache/spark/pull/30820#issuecomment-747262059 **[Test build #132941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132941/testReport)** for PR 30820 at commit

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-747260714 **[Test build #132940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132940/testReport)** for PR 29695 at commit

[GitHub] [spark] dongjoon-hyun opened a new pull request #30820: [SPARK-33819][CORE][3.0] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

dongjoon-hyun opened a new pull request #30820: URL: https://github.com/apache/spark/pull/30820 ### What changes were proposed in this pull request? This PR aims to convert `EventLogFileReader`'s derived classes into `package private`. - SingleFileEventLogFileReader -

[GitHub] [spark] SparkQA commented on pull request #30818: [SPARK-33823][SQL] Use the `CastSupport.cast` method in HashJoin

SparkQA commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747259797 **[Test build #132937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132937/testReport)** for PR 30818 at commit

[GitHub] [spark] SparkQA commented on pull request #30816: [SPARK-33818][DOC] Doc `spark.sql.parser.quotedRegexColumnNames`

SparkQA commented on pull request #30816: URL: https://github.com/apache/spark/pull/30816#issuecomment-747259249 **[Test build #132938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132938/testReport)** for PR 30816 at commit

[GitHub] [spark] SparkQA commented on pull request #30790: [SPARK-33798][SQL] Add new rule to push down the foldable expressions through CaseWhen/If

SparkQA commented on pull request #30790: URL: https://github.com/apache/spark/pull/30790#issuecomment-747259282 **[Test build #132939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132939/testReport)** for PR 30790 at commit

[GitHub] [spark] SparkQA commented on pull request #30819: [SPARK-33819][CORE][3.1] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

SparkQA commented on pull request #30819: URL: https://github.com/apache/spark/pull/30819#issuecomment-747259209 **[Test build #132936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132936/testReport)** for PR 30819 at commit

[GitHub] [spark] huaxingao commented on a change in pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

huaxingao commented on a change in pull request #29695: URL: https://github.com/apache/spark/pull/29695#discussion_r544860215 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala ## @@ -21,6 +21,7 @@ import

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30814: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

AmplabJenkins removed a comment on pull request #30814: URL: https://github.com/apache/spark/pull/30814#issuecomment-747258018 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30543: [SPARK-33597][SQL] Support REGEXP_LIKE for consistent with mainstream databases

AmplabJenkins removed a comment on pull request #30543: URL: https://github.com/apache/spark/pull/30543#issuecomment-747258020 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37535/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30312: [SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuffle blocks after successful map task completion

AmplabJenkins removed a comment on pull request #30312: URL: https://github.com/apache/spark/pull/30312#issuecomment-747258016 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132924/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30790: [SPARK-33798][SQL] Add new rule to push down the foldable expressions through CaseWhen/If

AmplabJenkins removed a comment on pull request #30790: URL: https://github.com/apache/spark/pull/30790#issuecomment-747258015 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37538/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #24990: [WIP][SPARK-28191][SS] New data source - state - reader part

AmplabJenkins removed a comment on pull request #24990: URL: https://github.com/apache/spark/pull/24990#issuecomment-747258021 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132916/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

AmplabJenkins removed a comment on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747258019 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37531/

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-747258014 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132914/

[GitHub] [spark] AmplabJenkins commented on pull request #30543: [SPARK-33597][SQL] Support REGEXP_LIKE for consistent with mainstream databases

AmplabJenkins commented on pull request #30543: URL: https://github.com/apache/spark/pull/30543#issuecomment-747258020 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37535/

[GitHub] [spark] AmplabJenkins commented on pull request #24990: [WIP][SPARK-28191][SS] New data source - state - reader part

AmplabJenkins commented on pull request #24990: URL: https://github.com/apache/spark/pull/24990#issuecomment-747258021 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132916/

[GitHub] [spark] AmplabJenkins commented on pull request #30790: [SPARK-33798][SQL] Add new rule to push down the foldable expressions through CaseWhen/If

AmplabJenkins commented on pull request #30790: URL: https://github.com/apache/spark/pull/30790#issuecomment-747258015 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37538/

[GitHub] [spark] AmplabJenkins commented on pull request #30312: [SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuffle blocks after successful map task completion

AmplabJenkins commented on pull request #30312: URL: https://github.com/apache/spark/pull/30312#issuecomment-747258016 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132924/

[GitHub] [spark] AmplabJenkins commented on pull request #30814: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

AmplabJenkins commented on pull request #30814: URL: https://github.com/apache/spark/pull/30814#issuecomment-747258018 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

AmplabJenkins commented on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747258019 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37531/

[GitHub] [spark] dongjoon-hyun opened a new pull request #30819: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

dongjoon-hyun opened a new pull request #30819: URL: https://github.com/apache/spark/pull/30819 ### What changes were proposed in this pull request? This PR aims to convert `EventLogFileReader`'s derived classes into `package private`. - SingleFileEventLogFileReader -

[GitHub] [spark] maropu commented on pull request #30818: [SPARK-33823][SQL] Use the `CastSupport.cast` method in HashJoin

maropu commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747256274 cc: @dongjoon-hyun @cloud-fan @HyukjinKwon This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] SparkQA commented on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

SparkQA commented on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747255650 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37536/

[GitHub] [spark] SparkQA commented on pull request #30815: [SPARK-33817][SQL] CACHE TABLE uses a logical plan when caching a query to avoid creating a dataframe

SparkQA commented on pull request #30815: URL: https://github.com/apache/spark/pull/30815#issuecomment-747255085 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37537/

[GitHub] [spark] SparkQA removed a comment on pull request #30814: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

SparkQA removed a comment on pull request #30814: URL: https://github.com/apache/spark/pull/30814#issuecomment-747196136 **[Test build #132926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132926/testReport)** for PR 30814 at commit

[GitHub] [spark] SparkQA commented on pull request #30814: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

SparkQA commented on pull request #30814: URL: https://github.com/apache/spark/pull/30814#issuecomment-747254566 **[Test build #132926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132926/testReport)** for PR 30814 at commit

[GitHub] [spark] maropu commented on pull request #30818: [SPARK-33823][SQL] Use the `CastSupport.cast` method in HashJoin

maropu commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747253568 To add unit tests, I'm looking for a simpler query to reproduce this. But, I've not found it yet. This is an

[GitHub] [spark] dongjoon-hyun commented on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

dongjoon-hyun commented on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747253610 Thank you, @HyukjinKwon ! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] maropu opened a new pull request #30818: [SPARK-33823][SQL] Use the `CastSupport.cast` method in HashJoin

maropu opened a new pull request #30818: URL: https://github.com/apache/spark/pull/30818 ### What changes were proposed in this pull request? This PR intends to fix the bug that throws a unsupported exception when running [the TPCDS

[GitHub] [spark] MaxGekk commented on a change in pull request #30799: [SPARK-33803][SQL] Sort table properties by key in DESCRIBE TABLE command

MaxGekk commented on a change in pull request #30799: URL: https://github.com/apache/spark/pull/30799#discussion_r544854547 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala ## @@ -388,7 +388,8 @@ case class CatalogTable( def

[GitHub] [spark] dongjoon-hyun commented on pull request #30814: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

dongjoon-hyun commented on pull request #30814: URL: https://github.com/apache/spark/pull/30814#issuecomment-747250785 Thank you! I'll make backporting PRs. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #30312: [SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuffle blocks after successful map task completion

SparkQA removed a comment on pull request #30312: URL: https://github.com/apache/spark/pull/30312#issuecomment-747193023 **[Test build #132924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132924/testReport)** for PR 30312 at commit

[GitHub] [spark] SparkQA commented on pull request #30312: [SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuffle blocks after successful map task completion

SparkQA commented on pull request #30312: URL: https://github.com/apache/spark/pull/30312#issuecomment-747250596 **[Test build #132924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132924/testReport)** for PR 30312 at commit

[GitHub] [spark] MaxGekk commented on a change in pull request #30799: [SPARK-33803][SQL] Sort table properties by key in DESCRIBE TABLE command

MaxGekk commented on a change in pull request #30799: URL: https://github.com/apache/spark/pull/30799#discussion_r544852303 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala ## @@ -388,7 +388,8 @@ case class CatalogTable( def

[GitHub] [spark] MaxGekk commented on a change in pull request #30799: [SPARK-33803][SQL] Sort table properties by key in DESCRIBE TABLE command

MaxGekk commented on a change in pull request #30799: URL: https://github.com/apache/spark/pull/30799#discussion_r544852303 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala ## @@ -388,7 +388,8 @@ case class CatalogTable( def

[GitHub] [spark] HyukjinKwon closed pull request #30814: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

HyukjinKwon closed pull request #30814: URL: https://github.com/apache/spark/pull/30814 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #30814: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

HyukjinKwon commented on pull request #30814: URL: https://github.com/apache/spark/pull/30814#issuecomment-747248600 Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon closed pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

HyukjinKwon closed pull request #30817: URL: https://github.com/apache/spark/pull/30817 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

HyukjinKwon commented on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747247902 Merged to master and branch-3.1. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA commented on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

SparkQA commented on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747247389 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37531/

[GitHub] [spark] sarutak commented on a change in pull request #30573: [SPARK-26341][WEBUI]Expose executor memory metrics at the stage level, in the Stages tab

sarutak commented on a change in pull request #30573: URL: https://github.com/apache/spark/pull/30573#discussion_r544849995 ## File path: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ## @@ -687,6 +687,9 @@ private[spark] class AppStatusListener(

[GitHub] [spark] SparkQA removed a comment on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

SparkQA removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-747162438 **[Test build #132914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132914/testReport)** for PR 29695 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #24990: [WIP][SPARK-28191][SS] New data source - state - reader part

SparkQA removed a comment on pull request #24990: URL: https://github.com/apache/spark/pull/24990#issuecomment-747163826 **[Test build #132916 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132916/testReport)** for PR 24990 at commit

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-747246451 **[Test build #132914 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132914/testReport)** for PR 29695 at commit

[GitHub] [spark] SparkQA commented on pull request #24990: [WIP][SPARK-28191][SS] New data source - state - reader part

SparkQA commented on pull request #24990: URL: https://github.com/apache/spark/pull/24990#issuecomment-747246397 **[Test build #132916 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132916/testReport)** for PR 24990 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #30814: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

SparkQA removed a comment on pull request #30814: URL: https://github.com/apache/spark/pull/30814#issuecomment-747193231 **[Test build #132922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132922/testReport)** for PR 30814 at commit

[GitHub] [spark] SparkQA commented on pull request #30814: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

SparkQA commented on pull request #30814: URL: https://github.com/apache/spark/pull/30814#issuecomment-747244715 **[Test build #132922 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132922/testReport)** for PR 30814 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30790: [SPARK-33798][SQL] Simplify EqualTo(CaseWhen/If, Literal) always false

AmplabJenkins removed a comment on pull request #30790: URL: https://github.com/apache/spark/pull/30790#issuecomment-747161688 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30790: [SPARK-33798][SQL] Simplify EqualTo(CaseWhen/If, Literal) always false

SparkQA commented on pull request #30790: URL: https://github.com/apache/spark/pull/30790#issuecomment-747233718 **[Test build #132935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132935/testReport)** for PR 30790 at commit

[GitHub] [spark] SparkQA commented on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

SparkQA commented on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747233514 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37531/

[GitHub] [spark] SparkQA commented on pull request #30815: [SPARK-33817][SQL] CACHE TABLE uses a logical plan when caching a query to avoid creating a dataframe

SparkQA commented on pull request #30815: URL: https://github.com/apache/spark/pull/30815#issuecomment-747233417 **[Test build #132934 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132934/testReport)** for PR 30815 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

AmplabJenkins removed a comment on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-747214936 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132910/

[GitHub] [spark] viirya edited a comment on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

viirya edited a comment on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-747232971 > So IMHO the right direction would be either trying our best to unload inactive state ASAP, or considering the replication as the further improvement. Not somewhere in

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

AmplabJenkins removed a comment on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747214209 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132928/

[GitHub] [spark] viirya commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

viirya commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-747232971 > So IMHO the right direction would be either trying our best to unload inactive state ASAP, or considering the replication as the further improvement. Not somewhere in between.

[GitHub] [spark] SparkQA commented on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

SparkQA commented on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747232716 **[Test build #132933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132933/testReport)** for PR 30817 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

AmplabJenkins removed a comment on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-747231846 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37528/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30809: [SPARK-33812][SQL] Split the histogram column stats when saving to hive metastore as table property

AmplabJenkins removed a comment on pull request #30809: URL: https://github.com/apache/spark/pull/30809#issuecomment-747231834 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30813: [SPARK-33815][SQL] Migrate ALTER TABLE ... SET [SERDE|SERDEPROPERTIES] to use UnresolvedTable to resolve the identifier

AmplabJenkins removed a comment on pull request #30813: URL: https://github.com/apache/spark/pull/30813#issuecomment-747175650 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30814: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

AmplabJenkins removed a comment on pull request #30814: URL: https://github.com/apache/spark/pull/30814#issuecomment-747231841 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30312: [SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuffle blocks after successful map task completion

AmplabJenkins removed a comment on pull request #30312: URL: https://github.com/apache/spark/pull/30312#issuecomment-747231842 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37527/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30772: [SPARK-33733][SQL][2.4] PullOutNondeterministic should check and collect deterministic field

AmplabJenkins removed a comment on pull request #30772: URL: https://github.com/apache/spark/pull/30772#issuecomment-747231845 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37534/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30816: [SPARK-33818][DOC] Doc `spark.sql.parser.quotedRegexColumnNames`

AmplabJenkins removed a comment on pull request #30816: URL: https://github.com/apache/spark/pull/30816#issuecomment-747213459 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30813: [SPARK-33815][SQL] Migrate ALTER TABLE ... SET [SERDE|SERDEPROPERTIES] to use UnresolvedTable to resolve the identifier

AmplabJenkins commented on pull request #30813: URL: https://github.com/apache/spark/pull/30813#issuecomment-747231839 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132911/

[GitHub] [spark] AmplabJenkins commented on pull request #30816: [SPARK-33818][DOC] Doc `spark.sql.parser.quotedRegexColumnNames`

AmplabJenkins commented on pull request #30816: URL: https://github.com/apache/spark/pull/30816#issuecomment-747231832 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37532/

[GitHub] [spark] AmplabJenkins commented on pull request #30772: [SPARK-33733][SQL][2.4] PullOutNondeterministic should check and collect deterministic field

AmplabJenkins commented on pull request #30772: URL: https://github.com/apache/spark/pull/30772#issuecomment-747231845 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37534/

[GitHub] [spark] AmplabJenkins commented on pull request #30312: [SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuffle blocks after successful map task completion

AmplabJenkins commented on pull request #30312: URL: https://github.com/apache/spark/pull/30312#issuecomment-747231842 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37527/

[GitHub] [spark] AmplabJenkins commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

AmplabJenkins commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-747231846 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37528/

[GitHub] [spark] AmplabJenkins commented on pull request #30814: [SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

AmplabJenkins commented on pull request #30814: URL: https://github.com/apache/spark/pull/30814#issuecomment-747231843 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30809: [SPARK-33812][SQL] Split the histogram column stats when saving to hive metastore as table property

AmplabJenkins commented on pull request #30809: URL: https://github.com/apache/spark/pull/30809#issuecomment-747231836 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30816: [SPARK-33818][DOC] Doc `spark.sql.parser.quotedRegexColumnNames`

SparkQA commented on pull request #30816: URL: https://github.com/apache/spark/pull/30816#issuecomment-747229732 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37532/

[GitHub] [spark] jzhuge commented on a change in pull request #30806: [SPARK-33808][SQL] DataSource V2: Build logical writes in the optimizer

jzhuge commented on a change in pull request #30806: URL: https://github.com/apache/spark/pull/30806#discussion_r544832653 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala ## @@ -188,15 +189,20 @@ class

[GitHub] [spark] SparkQA commented on pull request #30816: [SPARK-33818][DOC] Doc `spark.sql.parser.quotedRegexColumnNames`

SparkQA commented on pull request #30816: URL: https://github.com/apache/spark/pull/30816#issuecomment-747227772 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37532/

[GitHub] [spark] wangyum commented on a change in pull request #30790: [SPARK-33798][SQL] Simplify EqualTo(CaseWhen/If, Literal) always false

wangyum commented on a change in pull request #30790: URL: https://github.com/apache/spark/pull/30790#discussion_r544829536 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala ## @@ -523,6 +532,14 @@ object SimplifyConditionals

[GitHub] [spark] SparkQA commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

SparkQA commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-747221956 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37528/

[GitHub] [spark] cloud-fan commented on pull request #30659: [SPARK-33697][SQL] RemoveRedundantProjects should require column ordering by default

cloud-fan commented on pull request #30659: URL: https://github.com/apache/spark/pull/30659#issuecomment-747221963 @allisonwang-db can you open backport PRs for other branches? Thanks! This is an automated message from the

[GitHub] [spark] cloud-fan closed pull request #30659: [SPARK-33697][SQL] RemoveRedundantProjects should require column ordering by default

cloud-fan closed pull request #30659: URL: https://github.com/apache/spark/pull/30659 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #30659: [SPARK-33697][SQL] RemoveRedundantProjects should require column ordering by default

cloud-fan commented on pull request #30659: URL: https://github.com/apache/spark/pull/30659#issuecomment-747221660 thanks, merging to master/3.1! This is an automated message from the Apache Git Service. To respond to the

  1   2   3   4   5   6   7   8   9   10   >