[GitHub] [spark] AmplabJenkins commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-671009262 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-671009262 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] jiangxb1987 commented on a change in pull request #29228: [SPARK-31847][CORE][TESTS] DAGSchedulerSuite: Rewrite the test framework to support apply specified spark configurations.

jiangxb1987 commented on a change in pull request #29228: URL: https://github.com/apache/spark/pull/29228#discussion_r467539295 ## File path: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ## @@ -295,7 +298,20 @@ class DAGSchedulerSuite extends

[GitHub] [spark] SparkQA commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

SparkQA commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-671009126 **[Test build #127231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127231/testReport)** for PR 29389 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

SparkQA removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670991633 **[Test build #127231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127231/testReport)** for PR 29389 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

dongjoon-hyun commented on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-671009033 Hi, @Fokko . Since the PR is working at least and SPARK-10520 has been a long standing issue, could you send an email to `dev@spark` with the summary of your findings

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671008612 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671008612 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

SparkQA commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671008527 **[Test build #127234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127234/testReport)** for PR 29393 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

dongjoon-hyun commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671008077 Retest this please. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun commented on pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

dongjoon-hyun commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-671007726 Merged to master for Apache Spark 3.1.0 on December 2020. This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun closed pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

dongjoon-hyun closed pull request #29265: URL: https://github.com/apache/spark/pull/29265 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] viirya commented on a change in pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

viirya commented on a change in pull request #29066: URL: https://github.com/apache/spark/pull/29066#discussion_r467536043 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2Writes.scala ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671004352 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671004350 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671004350 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

SparkQA commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671004293 **[Test build #127232 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127232/testReport)** for PR 29393 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

SparkQA removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995549 **[Test build #127232 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127232/testReport)** for PR 29393 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

AmplabJenkins removed a comment on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-671003151 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

AmplabJenkins commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-671003151 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

SparkQA removed a comment on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670984511 **[Test build #127230 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127230/testReport)** for PR 29391 at commit

[GitHub] [spark] SparkQA commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

SparkQA commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-671002999 **[Test build #127230 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127230/testReport)** for PR 29391 at commit

[GitHub] [spark] maropu commented on a change in pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on a change in pull request #29389: URL: https://github.com/apache/spark/pull/29389#discussion_r467527399 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -125,7 +125,7 @@ private[execution] object

[GitHub] [spark] maropu commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670997405 You, too, @leanken ;) thanks for the work, anyway. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670996658 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670996658 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

SparkQA commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670996570 **[Test build #127233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127233/testReport)** for PR 29389 at commit

[GitHub] [spark] leanken commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

leanken commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670996334 @maropu updated. thanks for doing this in weekend. ^_^ This is an automated message from the Apache Git

[GitHub] [spark] leanken commented on a change in pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

leanken commented on a change in pull request #29389: URL: https://github.com/apache/spark/pull/29389#discussion_r467525605 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -125,7 +125,7 @@ private[execution] object

[GitHub] [spark] AmplabJenkins commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995621 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995621 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

SparkQA commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995549 **[Test build #127232 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127232/testReport)** for PR 29393 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995101 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] dongjoon-hyun commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

dongjoon-hyun commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995478 Thank you, @WangGuangxin . This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] dongjoon-hyun commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

dongjoon-hyun commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995457 ok to test This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins commented on pull request #29393: [SPARK-32559][SQL] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995101 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995018 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #29393: [SPARK-32559][SQL] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995018 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] dongjoon-hyun commented on pull request #29383: [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms

dongjoon-hyun commented on pull request #29383: URL: https://github.com/apache/spark/pull/29383#issuecomment-670995006 I raised the priority of SPARK-31703 to `Blocker` with `Target Version` 2.4.7 and 3.0.1. This is an

[GitHub] [spark] WangGuangxin opened a new pull request #29393: [SPARK-32559][SQL] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

WangGuangxin opened a new pull request #29393: URL: https://github.com/apache/spark/pull/29393 This is a backport of https://github.com/apache/spark/pull/29375 This is an automated message from the Apache Git Service. To

[GitHub] [spark] dongjoon-hyun commented on pull request #29383: [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms

dongjoon-hyun commented on pull request #29383: URL: https://github.com/apache/spark/pull/29383#issuecomment-670994867 cc @ScrapCodes since this is a correctness issue. This is an automated message from the Apache Git

[GitHub] [spark] maropu commented on a change in pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on a change in pull request #29389: URL: https://github.com/apache/spark/pull/29389#discussion_r467523167 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -125,7 +125,7 @@ private[execution] object

[GitHub] [spark] maropu commented on a change in pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on a change in pull request #29389: URL: https://github.com/apache/spark/pull/29389#discussion_r467523040 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala ## @@ -82,6 +82,8 @@ case class

[GitHub] [spark] yanxiaole edited a comment on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration

yanxiaole edited a comment on pull request #29392: URL: https://github.com/apache/spark/pull/29392#issuecomment-670993081 It happens when data structure getting modified, mainly `delete` causes the problem. Take `LevelDB` implementation for example, in its iterator's `next` function

[GitHub] [spark] yanxiaole commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration

yanxiaole commented on pull request #29392: URL: https://github.com/apache/spark/pull/29392#issuecomment-670993081 It happened when data structure getting modified, mainly `delete` caused the problem. Take `LevelDB` implementation for example, in its iterator's `next` function it

[GitHub] [spark] maropu commented on a change in pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on a change in pull request #29389: URL: https://github.com/apache/spark/pull/29389#discussion_r467522246 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/EliminateAntiJoin.scala ## @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670991720 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670991720 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670852395 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] SparkQA commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

SparkQA commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670991633 **[Test build #127231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127231/testReport)** for PR 29389 at commit

[GitHub] [spark] maropu commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670991526 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

AmplabJenkins removed a comment on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670989942 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

AmplabJenkins commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670989942 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

maropu commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670989945 Thanks, @dongjoon-hyun ! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA removed a comment on pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

SparkQA removed a comment on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670980002 **[Test build #127229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127229/testReport)** for PR 29265 at commit

[GitHub] [spark] SparkQA commented on pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

SparkQA commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670989801 **[Test build #127229 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127229/testReport)** for PR 29265 at commit

[GitHub] [spark] viirya commented on a change in pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

viirya commented on a change in pull request #29066: URL: https://github.com/apache/spark/pull/29066#discussion_r467515565 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/write/WriteBuilder.java ## @@ -23,17 +23,34 @@ import

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

agrawaldevesh commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r467509103 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +504,102 @@ class

[GitHub] [spark] dongjoon-hyun commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

dongjoon-hyun commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670985037 Merged to master/3.0. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] dongjoon-hyun closed pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

dongjoon-hyun closed pull request #29391: URL: https://github.com/apache/spark/pull/29391 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

AmplabJenkins commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670984586 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

AmplabJenkins removed a comment on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670984586 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

SparkQA commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670984511 **[Test build #127230 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127230/testReport)** for PR 29391 at commit

[GitHub] [spark] maropu commented on a change in pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

maropu commented on a change in pull request #29391: URL: https://github.com/apache/spark/pull/29391#discussion_r467514179 ## File path: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala ## @@ -46,11 +47,7 @@ class TPCDSQuerySuite extends BenchmarkQueryTest

[GitHub] [spark] maropu commented on a change in pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

maropu commented on a change in pull request #29391: URL: https://github.com/apache/spark/pull/29391#discussion_r467513958 ## File path: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala ## @@ -104,3 +101,23 @@ class TPCDSQuerySuite extends BenchmarkQueryTest

[GitHub] [spark] maropu commented on pull request #29390: [SPARK-32564][SQL][TEST][3.0] Inject data statistics to simulate plan generation on actual TPCDS data

maropu commented on pull request #29390: URL: https://github.com/apache/spark/pull/29390#issuecomment-670983959 Thanks a lot, @dongjoon-hyun ! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] maropu commented on a change in pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

maropu commented on a change in pull request #28804: URL: https://github.com/apache/spark/pull/28804#discussion_r467472877 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -838,13 +880,17 @@ case class

[GitHub] [spark] maropu commented on a change in pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

maropu commented on a change in pull request #28804: URL: https://github.com/apache/spark/pull/28804#discussion_r467472877 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -838,13 +880,17 @@ case class

[GitHub] [spark] maropu commented on a change in pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

maropu commented on a change in pull request #28804: URL: https://github.com/apache/spark/pull/28804#discussion_r467472877 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -838,13 +880,17 @@ case class

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467512687 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +88,122 @@ case class

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467512524 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +88,122 @@ case class

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

agrawaldevesh commented on a change in pull request #29211: URL: https://github.com/apache/spark/pull/29211#discussion_r467511469 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -277,12 +282,59 @@ private[spark] class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29265: [SPARK-32462][WEBUI] Don't save the previous search text for datatable

AmplabJenkins removed a comment on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670980082 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29265: [SPARK-32462][WEBUI] Don't save the previous search text for datatable

AmplabJenkins commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670980082 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29265: [SPARK-32462][WEBUI] Don't save the previous search text for datatable

SparkQA commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670980002 **[Test build #127229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127229/testReport)** for PR 29265 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #29265: [SPARK-32462][WEBUI] Don't save the previous search text for datatable

dongjoon-hyun commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670979794 Retest this please. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun closed pull request #29372: [SPARK-32555][SQL] Add unique ID on query execution

dongjoon-hyun closed pull request #29372: URL: https://github.com/apache/spark/pull/29372 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] c21 commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

c21 commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467505445 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +88,122 @@ case class

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467503454 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -91,13 +96,18 @@ private[execution] object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29372: [SPARK-32555][SQL] Add unique ID on query execution

AmplabJenkins removed a comment on pull request #29372: URL: https://github.com/apache/spark/pull/29372#issuecomment-670972473 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29372: [SPARK-32555][SQL] Add unique ID on query execution

AmplabJenkins commented on pull request #29372: URL: https://github.com/apache/spark/pull/29372#issuecomment-670972473 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29372: [SPARK-32555][SQL] Add unique ID on query execution

SparkQA commented on pull request #29372: URL: https://github.com/apache/spark/pull/29372#issuecomment-670972247 **[Test build #127225 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127225/testReport)** for PR 29372 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29372: [SPARK-32555][SQL] Add unique ID on query execution

SparkQA removed a comment on pull request #29372: URL: https://github.com/apache/spark/pull/29372#issuecomment-670944925 **[Test build #127225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127225/testReport)** for PR 29372 at commit

[GitHub] [spark] c21 commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

c21 commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467502239 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -58,8 +65,19 @@ case class ShuffledHashJoinExec(

[GitHub] [spark] srowen commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration

srowen commented on pull request #29392: URL: https://github.com/apache/spark/pull/29392#issuecomment-670970887 I'm not sure, what is the problem here? is it this data structure getting modified or something else underneath it in the FS? if the latter, then this synchronization might not

[GitHub] [spark] AmplabJenkins commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-670970688 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-670970688 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

SparkQA commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-670970447 **[Test build #127226 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127226/testReport)** for PR 29360 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

SparkQA removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-670944935 **[Test build #127226 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127226/testReport)** for PR 29360 at commit

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29387: [SPARK-32481] Support truncate table to move data to trash

dongjoon-hyun commented on a change in pull request #29387: URL: https://github.com/apache/spark/pull/29387#discussion_r467498414 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -269,6 +269,23 @@ private[spark] object Utils extends Logging {

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

dongjoon-hyun commented on a change in pull request #29391: URL: https://github.com/apache/spark/pull/29391#discussion_r467498022 ## File path: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala ## @@ -46,11 +47,7 @@ class TPCDSQuerySuite extends

[GitHub] [spark] dongjoon-hyun commented on pull request #29368: [SPARK-32554][K8S][DOCS] Update the k8s document according to the current development status

dongjoon-hyun commented on pull request #29368: URL: https://github.com/apache/spark/pull/29368#issuecomment-670966586 I also have the same opinion with @srowen . We can declare K8s GA in Apache Spark 3.1.0 with AS-IS status and add new features more. I don't think any new feature blocks

[GitHub] [spark] dongjoon-hyun commented on pull request #29351: [MINOR][DOCS] Fix typos at ExecutorAllocationManager.scala

dongjoon-hyun commented on pull request #29351: URL: https://github.com/apache/spark/pull/29351#issuecomment-670966067 Merged to master/3.0/2.4. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] dongjoon-hyun closed pull request #29351: [MINOR][DOCS] Fix typos at ExecutorAllocationManager.scala

dongjoon-hyun closed pull request #29351: URL: https://github.com/apache/spark/pull/29351 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] yanxiaole commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration

yanxiaole commented on pull request #29392: URL: https://github.com/apache/spark/pull/29392#issuecomment-670961239 > What code that modifies this object is also synchronizing on this? Sorry, I'll update them. @srowen , do you think I should add synchronizing at every write and

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r467491962 ## File path: streaming/src/test/scala/org/apache/spark/streaming/scheduler/ExecutorAllocationManagerSuite.scala ## @@ -83,12 +96,26 @@ class

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r467491912 ## File path: core/src/test/scala/org/apache/spark/scheduler/WorkerDecommissionSuite.scala ## @@ -76,7 +76,9 @@ class WorkerDecommissionSuite extends

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r467491854 ## File path: core/src/main/scala/org/apache/spark/scheduler/dynalloc/ExecutorMonitor.scala ## @@ -326,18 +352,33 @@ private[spark] class

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r467491346 ## File path: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala ## @@ -204,7 +205,12 @@ private[spark] class

[GitHub] [spark] holdenk commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r467491189 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +504,102 @@ class

  1   2   3   >