[GitHub] [spark] SparkQA commented on pull request #33655: [SPARK-36430][SQL] Adaptively calculate the target size when coalescing shuffle partitions in AQE

SparkQA commented on pull request #33655: URL: https://github.com/apache/spark/pull/33655#issuecomment-894020840 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46643/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33574: [SPARK-36339][SQL] References to grouping that not part of aggregation should be replaced

AmplabJenkins removed a comment on pull request #33574: URL: https://github.com/apache/spark/pull/33574#issuecomment-894020210 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46640/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33646: [SPARK-36388][SPARK-36386][PYTHON] Fix DataFrame groupby-rolling and groupby-expanding to follow pandas 1.3

AmplabJenkins removed a comment on pull request #33646: URL: https://github.com/apache/spark/pull/33646#issuecomment-894020209 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142133/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33624: [SPARK-35881][SQL][FOLLOWUP] Add a boolean flag in AdaptiveSparkPlanExec to ask for columnar output

AmplabJenkins removed a comment on pull request #33624: URL: https://github.com/apache/spark/pull/33624#issuecomment-894020235 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142132/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33556: [SPARK-36324][CORE] Replace revertPartialWritesAndClose with close in ExternalSorter.spill and ExternalAppendOnlyMap.spill

AmplabJenkins removed a comment on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-894020208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

AmplabJenkins removed a comment on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-894020211 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46642/

[GitHub] [spark] AmplabJenkins commented on pull request #33556: [SPARK-36324][CORE] Replace revertPartialWritesAndClose with close in ExternalSorter.spill and ExternalAppendOnlyMap.spill

AmplabJenkins commented on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-894020208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33574: [SPARK-36339][SQL] References to grouping that not part of aggregation should be replaced

AmplabJenkins commented on pull request #33574: URL: https://github.com/apache/spark/pull/33574#issuecomment-894020210 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46640/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33624: [SPARK-35881][SQL][FOLLOWUP] Add a boolean flag in AdaptiveSparkPlanExec to ask for columnar output

AmplabJenkins commented on pull request #33624: URL: https://github.com/apache/spark/pull/33624#issuecomment-894020235 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142132/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

AmplabJenkins commented on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-894020211 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46642/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33646: [SPARK-36388][SPARK-36386][PYTHON] Fix DataFrame groupby-rolling and groupby-expanding to follow pandas 1.3

AmplabJenkins commented on pull request #33646: URL: https://github.com/apache/spark/pull/33646#issuecomment-894020209 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142133/ -- This

[GitHub] [spark] SparkQA commented on pull request #33624: [SPARK-35881][SQL][FOLLOWUP] Add a boolean flag in AdaptiveSparkPlanExec to ask for columnar output

SparkQA commented on pull request #33624: URL: https://github.com/apache/spark/pull/33624#issuecomment-894019220 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46644/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33574: [SPARK-36339][SQL] References to grouping that not part of aggregation should be replaced

SparkQA commented on pull request #33574: URL: https://github.com/apache/spark/pull/33574#issuecomment-894016539 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46640/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #33646: [SPARK-36388][SPARK-36386][PYTHON] Fix DataFrame groupby-rolling and groupby-expanding to follow pandas 1.3

SparkQA removed a comment on pull request #33646: URL: https://github.com/apache/spark/pull/33646#issuecomment-894004368 **[Test build #142133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142133/testReport)** for PR 33646 at commit

[GitHub] [spark] SparkQA commented on pull request #33646: [SPARK-36388][SPARK-36386][PYTHON] Fix DataFrame groupby-rolling and groupby-expanding to follow pandas 1.3

SparkQA commented on pull request #33646: URL: https://github.com/apache/spark/pull/33646#issuecomment-894015793 **[Test build #142133 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142133/testReport)** for PR 33646 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33624: [SPARK-35881][SQL][FOLLOWUP] Add a boolean flag in AdaptiveSparkPlanExec to ask for columnar output

SparkQA removed a comment on pull request #33624: URL: https://github.com/apache/spark/pull/33624#issuecomment-894000678 **[Test build #142132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142132/testReport)** for PR 33624 at commit

[GitHub] [spark] SparkQA commented on pull request #33624: [SPARK-35881][SQL][FOLLOWUP] Add a boolean flag in AdaptiveSparkPlanExec to ask for columnar output

SparkQA commented on pull request #33624: URL: https://github.com/apache/spark/pull/33624#issuecomment-894014335 **[Test build #142132 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142132/testReport)** for PR 33624 at commit

[GitHub] [spark] SparkQA commented on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

SparkQA commented on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-894012650 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46642/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33556: [SPARK-36324][CORE] Replace revertPartialWritesAndClose with close in ExternalSorter.spill and ExternalAppendOnlyMap.spill

SparkQA commented on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-894012607 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46646/ --

[GitHub] [spark] itholic commented on a change in pull request #33634: [SPARK-36369][PYTHON] Fix Index.union to follow pandas 1.3

itholic commented on a change in pull request #33634: URL: https://github.com/apache/spark/pull/33634#discussion_r683955183 ## File path: python/pyspark/pandas/indexes/base.py ## @@ -2292,9 +2292,7 @@ def union( sdf_self =

[GitHub] [spark] SparkQA removed a comment on pull request #33556: [SPARK-36324][CORE] Replace revertPartialWritesAndClose with close in ExternalSorter.spill and ExternalAppendOnlyMap.spill

SparkQA removed a comment on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-894010160 **[Test build #142134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142134/testReport)** for PR 33556 at commit

[GitHub] [spark] SparkQA commented on pull request #33556: [SPARK-36324][CORE] Replace revertPartialWritesAndClose with close in ExternalSorter.spill and ExternalAppendOnlyMap.spill

SparkQA commented on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-894010928 **[Test build #142134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142134/testReport)** for PR 33556 at commit

[GitHub] [spark] SparkQA commented on pull request #33556: [SPARK-36324][CORE] Replace revertPartialWritesAndClose with close in ExternalSorter.spill and ExternalAppendOnlyMap.spill

SparkQA commented on pull request #33556: URL: https://github.com/apache/spark/pull/33556#issuecomment-894010160 **[Test build #142134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142134/testReport)** for PR 33556 at commit

[GitHub] [spark] itholic commented on a change in pull request #33634: [SPARK-36369][PYTHON] Fix Index.union to follow pandas 1.3

itholic commented on a change in pull request #33634: URL: https://github.com/apache/spark/pull/33634#discussion_r683955183 ## File path: python/pyspark/pandas/indexes/base.py ## @@ -2292,9 +2292,7 @@ def union( sdf_self =

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33648: [SPARK-36420][Graphx] Use `isEmpty` to improve performance in Pregel‘s superstep

AmplabJenkins removed a comment on pull request #33648: URL: https://github.com/apache/spark/pull/33648#issuecomment-894008333 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46636/

[GitHub] [spark] SparkQA commented on pull request #33648: [SPARK-36420][Graphx] Use `isEmpty` to improve performance in Pregel‘s superstep

SparkQA commented on pull request #33648: URL: https://github.com/apache/spark/pull/33648#issuecomment-894008313 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46636/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33648: [SPARK-36420][Graphx] Use `isEmpty` to improve performance in Pregel‘s superstep

AmplabJenkins commented on pull request #33648: URL: https://github.com/apache/spark/pull/33648#issuecomment-894008333 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46636/ --

[GitHub] [spark] dongjoon-hyun commented on pull request #33660: [SPARK-36441][INFRA] Fix GA failure related to downloading lintr dependencies

dongjoon-hyun commented on pull request #33660: URL: https://github.com/apache/spark/pull/33660#issuecomment-894008058 +1, LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] sunchao edited a comment on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

sunchao edited a comment on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-894007146 I'm not aware of any issue at the moment - we've already been using this (although a slightly different internal Hadoop version) for a while now. @steveloughran can

[GitHub] [spark] sunchao commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

sunchao commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-894007146 I'm not aware of any issue at the moment - we've already been using this (although a slightly different internal Hadoop version) for a while now. @steveloughran can offer more

[GitHub] [spark] dongjoon-hyun commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

dongjoon-hyun commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-894006737 +1 for backporting! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] beliefer commented on pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

beliefer commented on pull request #33654: URL: https://github.com/apache/spark/pull/33654#issuecomment-894004996 @cloud-fan @HyukjinKwon Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #33646: [SPARK-36388][SPARK-36386][PYTHON] Fix DataFrame groupby-rolling and groupby-expanding to follow pandas 1.3

SparkQA commented on pull request #33646: URL: https://github.com/apache/spark/pull/33646#issuecomment-894004368 **[Test build #142133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142133/testReport)** for PR 33646 at commit

[GitHub] [spark] gengliangwang commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

gengliangwang commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-894003073 @sunchao @dongjoon-hyun @steveloughran Are there any other known issues of Hadoop 3.3.1? Should we have this upgrade in Spark 3.2? -- This is an automated message from

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

AmplabJenkins removed a comment on pull request #33654: URL: https://github.com/apache/spark/pull/33654#issuecomment-894002105 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46639/

[GitHub] [spark] SparkQA commented on pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

SparkQA commented on pull request #33654: URL: https://github.com/apache/spark/pull/33654#issuecomment-894002089 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46639/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

AmplabJenkins commented on pull request #33654: URL: https://github.com/apache/spark/pull/33654#issuecomment-894002105 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46639/ --

[GitHub] [spark] viirya commented on a change in pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

viirya commented on a change in pull request #33651: URL: https://github.com/apache/spark/pull/33651#discussion_r683948307 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -2008,6 +2014,32 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

AmplabJenkins removed a comment on pull request #33654: URL: https://github.com/apache/spark/pull/33654#issuecomment-894001478 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46635/

[GitHub] [spark] SparkQA commented on pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

SparkQA commented on pull request #33654: URL: https://github.com/apache/spark/pull/33654#issuecomment-894001438 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46635/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

AmplabJenkins commented on pull request #33654: URL: https://github.com/apache/spark/pull/33654#issuecomment-894001478 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46635/ --

[GitHub] [spark] viirya commented on a change in pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

viirya commented on a change in pull request #33651: URL: https://github.com/apache/spark/pull/33651#discussion_r683947320 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -2008,6 +2014,32 @@ class

[GitHub] [spark] viirya commented on a change in pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

viirya commented on a change in pull request #33651: URL: https://github.com/apache/spark/pull/33651#discussion_r683947320 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -2008,6 +2014,32 @@ class

[GitHub] [spark] SparkQA commented on pull request #33624: [SPARK-35881][SQL][FOLLOWUP] Add a boolean flag in AdaptiveSparkPlanExec to ask for columnar output

SparkQA commented on pull request #33624: URL: https://github.com/apache/spark/pull/33624#issuecomment-894000678 **[Test build #142132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142132/testReport)** for PR 33624 at commit

[GitHub] [spark] SparkQA commented on pull request #33655: [SPARK-36430][SQL] Adaptively calculate the target size when coalescing shuffle partitions in AQE

SparkQA commented on pull request #33655: URL: https://github.com/apache/spark/pull/33655#issuecomment-894000661 **[Test build #142131 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142131/testReport)** for PR 33655 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33661: [SPARK-36431][SQL] Support comparison of ANSI intervals with different fields

AmplabJenkins removed a comment on pull request #33661: URL: https://github.com/apache/spark/pull/33661#issuecomment-894000248 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142122/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32355: [SPARK-35221][SQL] Add the check of supported join hints

AmplabJenkins removed a comment on pull request #32355: URL: https://github.com/apache/spark/pull/32355#issuecomment-894000251 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142116/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33659: [SPARK-36433][WEBUI] Fix log message in WebUI

AmplabJenkins removed a comment on pull request #33659: URL: https://github.com/apache/spark/pull/33659#issuecomment-894000250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

AmplabJenkins removed a comment on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-894000249 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142117/

[GitHub] [spark] cloud-fan commented on pull request #33640: [SPARK-36409][SQL][TESTS] Splitting test cases from datetime.sql

cloud-fan commented on pull request #33640: URL: https://github.com/apache/spark/pull/33640#issuecomment-894000402 also merged to 3.2 as it's test-only and we can avoid code conflicts when backporting other PRs. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] AmplabJenkins commented on pull request #32355: [SPARK-35221][SQL] Add the check of supported join hints

AmplabJenkins commented on pull request #32355: URL: https://github.com/apache/spark/pull/32355#issuecomment-894000251 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142116/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

AmplabJenkins commented on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-894000249 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142117/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33659: [SPARK-36433][WEBUI] Fix log message in WebUI

AmplabJenkins commented on pull request #33659: URL: https://github.com/apache/spark/pull/33659#issuecomment-894000250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33661: [SPARK-36431][SQL] Support comparison of ANSI intervals with different fields

AmplabJenkins commented on pull request #33661: URL: https://github.com/apache/spark/pull/33661#issuecomment-894000248 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142122/ -- This

[GitHub] [spark] Ngone51 edited a comment on pull request #28072: [SPARK-31314][CORE] Revert SPARK-29285 to fix shuffle regression caused by creating temporary file eagerly

Ngone51 edited a comment on pull request #28072: URL: https://github.com/apache/spark/pull/28072#issuecomment-893964514 @zhuqi-lucas Spark has the exclusion (a.k.a blacklisting) feature (see `spark.excludeOnFailure.enabled`) and Spark on YARN also has its own exclusion feature (see

[GitHub] [spark] cloud-fan closed pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

cloud-fan closed pull request #33654: URL: https://github.com/apache/spark/pull/33654 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan commented on pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

cloud-fan commented on pull request #33654: URL: https://github.com/apache/spark/pull/33654#issuecomment-893999193 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] cloud-fan edited a comment on pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

cloud-fan edited a comment on pull request #33654: URL: https://github.com/apache/spark/pull/33654#issuecomment-893999193 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] SparkQA commented on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

SparkQA commented on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-893999199 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46642/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33659: [SPARK-36433][WEBUI] Fix log message in WebUI

SparkQA commented on pull request #33659: URL: https://github.com/apache/spark/pull/33659#issuecomment-893999031 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46638/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

SparkQA commented on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-893999009 **[Test build #142117 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142117/testReport)** for PR 33651 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

SparkQA removed a comment on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-893901647 **[Test build #142117 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142117/testReport)** for PR 33651 at commit

[GitHub] [spark] cloud-fan commented on pull request #33624: [SPARK-35881][SQL][FOLLOWUP] Add a boolean flag in AdaptiveSparkPlanExec to ask for columnar output

cloud-fan commented on pull request #33624: URL: https://github.com/apache/spark/pull/33624#issuecomment-893998671 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA commented on pull request #33574: [SPARK-36339][SQL] References to grouping that not part of aggregation should be replaced

SparkQA commented on pull request #33574: URL: https://github.com/apache/spark/pull/33574#issuecomment-893998496 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46640/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33661: [SPARK-36431][SQL] Support comparison of ANSI intervals with different fields

SparkQA removed a comment on pull request #33661: URL: https://github.com/apache/spark/pull/33661#issuecomment-893951128 **[Test build #142122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142122/testReport)** for PR 33661 at commit

[GitHub] [spark] SparkQA commented on pull request #33661: [SPARK-36431][SQL] Support comparison of ANSI intervals with different fields

SparkQA commented on pull request #33661: URL: https://github.com/apache/spark/pull/33661#issuecomment-893997101 **[Test build #142122 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142122/testReport)** for PR 33661 at commit

[GitHub] [spark] thejdeep commented on pull request #33659: [SPARK-36433][WEBUI] Fix log message in WebUI

thejdeep commented on pull request #33659: URL: https://github.com/apache/spark/pull/33659#issuecomment-893996472 @AngersZh the logInfo [call](spark-root-org.apache.spark.deploy.history.HistoryServer-proxy-ltx1-hcl14797.grid.linkedin.com.out) should we made after we assign the

[GitHub] [spark] gengliangwang commented on a change in pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

gengliangwang commented on a change in pull request #33615: URL: https://github.com/apache/spark/pull/33615#discussion_r683940726 ## File path: docs/configuration.md ## @@ -3134,3 +3134,111 @@ The stage level scheduling feature allows users to specify task and executor res

[GitHub] [spark] SparkQA removed a comment on pull request #32355: [SPARK-35221][SQL] Add the check of supported join hints

SparkQA removed a comment on pull request #32355: URL: https://github.com/apache/spark/pull/32355#issuecomment-893887130 **[Test build #142116 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142116/testReport)** for PR 32355 at commit

[GitHub] [spark] SparkQA commented on pull request #32355: [SPARK-35221][SQL] Add the check of supported join hints

SparkQA commented on pull request #32355: URL: https://github.com/apache/spark/pull/32355#issuecomment-893993124 **[Test build #142116 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142116/testReport)** for PR 32355 at commit

[GitHub] [spark] venkata91 commented on a change in pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

venkata91 commented on a change in pull request #33615: URL: https://github.com/apache/spark/pull/33615#discussion_r683934920 ## File path: docs/configuration.md ## @@ -3134,3 +3134,111 @@ The stage level scheduling feature allows users to specify task and executor res This

[GitHub] [spark] venkata91 commented on a change in pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

venkata91 commented on a change in pull request #33615: URL: https://github.com/apache/spark/pull/33615#discussion_r683934920 ## File path: docs/configuration.md ## @@ -3134,3 +3134,111 @@ The stage level scheduling feature allows users to specify task and executor res This

[GitHub] [spark] SparkQA commented on pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

SparkQA commented on pull request #33654: URL: https://github.com/apache/spark/pull/33654#issuecomment-893988621 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46639/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33648: [SPARK-36420][Graphx] Use `isEmpty` to improve performance in Pregel‘s superstep

SparkQA commented on pull request #33648: URL: https://github.com/apache/spark/pull/33648#issuecomment-893987865 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46636/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33659: [SPARK-36433][WEBUI] Fix log message in WebUI

SparkQA commented on pull request #33659: URL: https://github.com/apache/spark/pull/33659#issuecomment-893987001 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46638/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon commented on pull request #33652: [SPARK-36425] [PYSPARK][ML] Support CrossValidatorModel get standard deviation of metrics for each paramMap

HyukjinKwon commented on pull request #33652: URL: https://github.com/apache/spark/pull/33652#issuecomment-893986919 Yeah .. since this is a new feature, I wouldn't merge it into 3.2 unless there's a reason to merge it in 3.2. -- This is an automated message from the Apache Git Service.

[GitHub] [spark] AngersZhuuuu commented on pull request #33659: [SPARK-36433][WEBUI] Fix log message in WebUI

AngersZh commented on pull request #33659: URL: https://github.com/apache/spark/pull/33659#issuecomment-893985815 Can you clarify more clear about what behavior I have changed in my last pr? I am a little confused -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #33654: [SPARK-36429][SQL] JacksonParser should throw exception when data type unsupported.

SparkQA commented on pull request #33654: URL: https://github.com/apache/spark/pull/33654#issuecomment-893982507 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46635/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #33662: [SPARK-36162][SQL] Support estimate equal null safe join

AmplabJenkins commented on pull request #33662: URL: https://github.com/apache/spark/pull/33662#issuecomment-893982381 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33646: [SPARK-36388][SPARK-36386][PYTHON] Fix DataFrame groupby-rolling and groupby-expanding to follow pandas 1.3

AmplabJenkins removed a comment on pull request #33646: URL: https://github.com/apache/spark/pull/33646#issuecomment-893981752 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46634/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

AmplabJenkins removed a comment on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-893981750 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46632/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33661: [SPARK-36431][SQL] Support comparison of ANSI intervals with different fields

AmplabJenkins removed a comment on pull request #33661: URL: https://github.com/apache/spark/pull/33661#issuecomment-893981751 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46633/

[GitHub] [spark] AmplabJenkins commented on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

AmplabJenkins commented on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-893981750 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46632/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33646: [SPARK-36388][SPARK-36386][PYTHON] Fix DataFrame groupby-rolling and groupby-expanding to follow pandas 1.3

AmplabJenkins commented on pull request #33646: URL: https://github.com/apache/spark/pull/33646#issuecomment-893981752 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46634/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33661: [SPARK-36431][SQL] Support comparison of ANSI intervals with different fields

AmplabJenkins commented on pull request #33661: URL: https://github.com/apache/spark/pull/33661#issuecomment-893981751 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46633/ --

[GitHub] [spark] changvvb opened a new pull request #33662: Fix null safe join estimation

changvvb opened a new pull request #33662: URL: https://github.com/apache/spark/pull/33662 ### What changes were proposed in this pull request? Support estimate equal null safe join ### Why are the changes needed? Support estimate equal null safe join ### Does this PR

[GitHub] [spark] senthh commented on pull request #33577: [SPARK-36327][SQL] Spark sql creates staging dir inside database directory rather than creating inside table directory

senthh commented on pull request #33577: URL: https://github.com/apache/spark/pull/33577#issuecomment-893980100 @dongjoon-hyun Could you please let me know if anything I can do to make this PR to the further steps? Note: I have created a jira [1]i and working n order to look

[GitHub] [spark] SparkQA commented on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

SparkQA commented on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-893977873 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46632/ -- This is an automated message from the

[GitHub] [spark] gengliangwang commented on a change in pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

gengliangwang commented on a change in pull request #33615: URL: https://github.com/apache/spark/pull/33615#discussion_r683924179 ## File path: docs/configuration.md ## @@ -3134,3 +3134,111 @@ The stage level scheduling feature allows users to specify task and executor res

[GitHub] [spark] SparkQA commented on pull request #33661: [SPARK-36431][SQL] Support comparison of ANSI intervals with different fields

SparkQA commented on pull request #33661: URL: https://github.com/apache/spark/pull/33661#issuecomment-893976568 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46633/ -- This is an automated message from the

[GitHub] [spark] HyukjinKwon closed pull request #33638: [SPARK-36415][SQL][DOCS] Add docs for try_cast/try_add/try_divide

HyukjinKwon closed pull request #33638: URL: https://github.com/apache/spark/pull/33638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon commented on pull request #33638: [SPARK-36415][SQL][DOCS] Add docs for try_cast/try_add/try_divide

HyukjinKwon commented on pull request #33638: URL: https://github.com/apache/spark/pull/33638#issuecomment-893975489 Merged to master and branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #33621: [SPARK-36390][SQL] Replace SessionState.close with SessionState.detachSession

HyukjinKwon commented on pull request #33621: URL: https://github.com/apache/spark/pull/33621#issuecomment-893974905 cc @juliuszsompolski FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA removed a comment on pull request #33648: [SPARK-36420][Graphx] Use `isEmpty` to improve performance in Pregel‘s superstep

SparkQA removed a comment on pull request #33648: URL: https://github.com/apache/spark/pull/33648#issuecomment-893965368 **[Test build #142124 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142124/testReport)** for PR 33648 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33648: [SPARK-36420][Graphx] Use `isEmpty` to improve performance in Pregel‘s superstep

AmplabJenkins removed a comment on pull request #33648: URL: https://github.com/apache/spark/pull/33648#issuecomment-893973157 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142124/

[GitHub] [spark] AmplabJenkins commented on pull request #33648: [SPARK-36420][Graphx] Use `isEmpty` to improve performance in Pregel‘s superstep

AmplabJenkins commented on pull request #33648: URL: https://github.com/apache/spark/pull/33648#issuecomment-893973157 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142124/ -- This

[GitHub] [spark] ulysses-you commented on a change in pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

ulysses-you commented on a change in pull request #33651: URL: https://github.com/apache/spark/pull/33651#discussion_r683919431 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/LogicalQueryStage.scala ## @@ -54,4 +54,6 @@ case class

[GitHub] [spark] SparkQA commented on pull request #33648: [SPARK-36420][Graphx] Use `isEmpty` to improve performance in Pregel‘s superstep

SparkQA commented on pull request #33648: URL: https://github.com/apache/spark/pull/33648#issuecomment-893973018 **[Test build #142124 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142124/testReport)** for PR 33648 at commit

[GitHub] [spark] ulysses-you commented on a change in pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

ulysses-you commented on a change in pull request #33651: URL: https://github.com/apache/spark/pull/33651#discussion_r683919296 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala ## @@ -2008,6 +2014,32 @@ class

[GitHub] [spark] SparkQA commented on pull request #33651: [SPARK-36424][SQL] Support eliminate limits in AQE Optimizer

SparkQA commented on pull request #33651: URL: https://github.com/apache/spark/pull/33651#issuecomment-893972893 **[Test build #142130 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142130/testReport)** for PR 33651 at commit

  1   2   3   4   5   6   7   8   9   10   >