[GitHub] [spark] AmplabJenkins removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-671015466 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-671015466 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

SparkQA removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670996570 **[Test build #127233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127233/testReport)** for PR 29389 at commit [`2063288`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

SparkQA commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-671015287 **[Test build #127233 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127233/testReport)** for PR 29389 at commit [`2063288`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-671014588 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-671014588 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

SparkQA commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-671014481 **[Test build #127236 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127236/testReport)** for PR 29360 at commit [`c1b318a`](https://github.com

[GitHub] [spark] viirya commented on a change in pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

viirya commented on a change in pull request #29360: URL: https://github.com/apache/spark/pull/29360#discussion_r467544190 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2690,6 +2690,18 @@ object SQLConf { .booleanConf

[GitHub] [spark] viirya commented on a change in pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

viirya commented on a change in pull request #29360: URL: https://github.com/apache/spark/pull/29360#discussion_r467544147 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2690,6 +2690,18 @@ object SQLConf { .booleanConf

[GitHub] [spark] Karl-WangSK commented on a change in pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

Karl-WangSK commented on a change in pull request #29360: URL: https://github.com/apache/spark/pull/29360#discussion_r467544123 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -1823,3 +1824,32 @@ object OptimizeLimitZero

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-671013934 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-671013934 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] viirya commented on a change in pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

viirya commented on a change in pull request #29360: URL: https://github.com/apache/spark/pull/29360#discussion_r467543947 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -1823,3 +1824,32 @@ object OptimizeLimitZero exten

[GitHub] [spark] SparkQA commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

SparkQA commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-671013827 **[Test build #127235 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127235/testReport)** for PR 29360 at commit [`b744272`](https://github.com

[GitHub] [spark] viirya commented on a change in pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

viirya commented on a change in pull request #29360: URL: https://github.com/apache/spark/pull/29360#discussion_r467543192 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2690,6 +2690,18 @@ object SQLConf { .booleanConf

[GitHub] [spark] kiszk commented on pull request #29383: [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms

kiszk commented on pull request #29383: URL: https://github.com/apache/spark/pull/29383#issuecomment-671011567 Good catch. This change looks good. Could you please add test cases to [ColumnarBatchSuite.scala](https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apac

[GitHub] [spark] AmplabJenkins commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-671009262 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-671009262 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] jiangxb1987 commented on a change in pull request #29228: [SPARK-31847][CORE][TESTS] DAGSchedulerSuite: Rewrite the test framework to support apply specified spark configurations.

jiangxb1987 commented on a change in pull request #29228: URL: https://github.com/apache/spark/pull/29228#discussion_r467539295 ## File path: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ## @@ -295,7 +298,20 @@ class DAGSchedulerSuite extends SparkFun

[GitHub] [spark] SparkQA commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

SparkQA commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-671009126 **[Test build #127231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127231/testReport)** for PR 29389 at commit [`25dd444`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

SparkQA removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670991633 **[Test build #127231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127231/testReport)** for PR 29389 at commit [`25dd444`](https://gi

[GitHub] [spark] dongjoon-hyun commented on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

dongjoon-hyun commented on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-671009033 Hi, @Fokko . Since the PR is working at least and SPARK-10520 has been a long standing issue, could you send an email to `dev@spark` with the summary of your findings (

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671008612 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671008612 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

SparkQA commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671008527 **[Test build #127234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127234/testReport)** for PR 29393 at commit [`3fffc60`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

dongjoon-hyun commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671008077 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] dongjoon-hyun commented on pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

dongjoon-hyun commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-671007726 Merged to master for Apache Spark 3.1.0 on December 2020. This is an automated message from the Apache Git

[GitHub] [spark] dongjoon-hyun closed pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

dongjoon-hyun closed pull request #29265: URL: https://github.com/apache/spark/pull/29265 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] viirya commented on a change in pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

viirya commented on a change in pull request #29066: URL: https://github.com/apache/spark/pull/29066#discussion_r467536043 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2Writes.scala ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Sof

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671004352 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/127

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671004350 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671004350 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

SparkQA commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-671004293 **[Test build #127232 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127232/testReport)** for PR 29393 at commit [`3fffc60`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

SparkQA removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995549 **[Test build #127232 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127232/testReport)** for PR 29393 at commit [`3fffc60`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

AmplabJenkins removed a comment on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-671003151 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

AmplabJenkins commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-671003151 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

SparkQA removed a comment on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670984511 **[Test build #127230 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127230/testReport)** for PR 29391 at commit [`43fcf8a`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

SparkQA commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-671002999 **[Test build #127230 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127230/testReport)** for PR 29391 at commit [`43fcf8a`](https://github.co

[GitHub] [spark] maropu commented on a change in pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on a change in pull request #29389: URL: https://github.com/apache/spark/pull/29389#discussion_r467527399 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -125,7 +125,7 @@ private[execution] object HashedRel

[GitHub] [spark] maropu commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670997405 You, too, @leanken ;) thanks for the work, anyway. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670996658 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670996658 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

SparkQA commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670996570 **[Test build #127233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127233/testReport)** for PR 29389 at commit [`2063288`](https://github.com

[GitHub] [spark] leanken commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

leanken commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670996334 @maropu updated. thanks for doing this in weekend. ^_^ This is an automated message from the Apache Git Service.

[GitHub] [spark] leanken commented on a change in pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

leanken commented on a change in pull request #29389: URL: https://github.com/apache/spark/pull/29389#discussion_r467525605 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -125,7 +125,7 @@ private[execution] object HashedRe

[GitHub] [spark] AmplabJenkins commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995621 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995621 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

SparkQA commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995549 **[Test build #127232 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127232/testReport)** for PR 29393 at commit [`3fffc60`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995101 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] dongjoon-hyun commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

dongjoon-hyun commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995478 Thank you, @WangGuangxin . This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [spark] dongjoon-hyun commented on pull request #29393: [SPARK-32559][SQL][3.0] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

dongjoon-hyun commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995457 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] AmplabJenkins commented on pull request #29393: [SPARK-32559][SQL] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995101 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29393: [SPARK-32559][SQL] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins removed a comment on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995018 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #29393: [SPARK-32559][SQL] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

AmplabJenkins commented on pull request #29393: URL: https://github.com/apache/spark/pull/29393#issuecomment-670995018 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] dongjoon-hyun commented on pull request #29383: [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms

dongjoon-hyun commented on pull request #29383: URL: https://github.com/apache/spark/pull/29383#issuecomment-670995006 I raised the priority of SPARK-31703 to `Blocker` with `Target Version` 2.4.7 and 3.0.1. This is an autom

[GitHub] [spark] WangGuangxin opened a new pull request #29393: [SPARK-32559][SQL] Fix the trim logic in UTF8String.toInt/toLong did't handle non-ASCII characters correctly

WangGuangxin opened a new pull request #29393: URL: https://github.com/apache/spark/pull/29393 This is a backport of https://github.com/apache/spark/pull/29375 This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] dongjoon-hyun commented on pull request #29383: [SPARK-31703][SQL] Parquet RLE float/double are read incorrectly on big endian platforms

dongjoon-hyun commented on pull request #29383: URL: https://github.com/apache/spark/pull/29383#issuecomment-670994867 cc @ScrapCodes since this is a correctness issue. This is an automated message from the Apache Git Service

[GitHub] [spark] maropu commented on a change in pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on a change in pull request #29389: URL: https://github.com/apache/spark/pull/29389#discussion_r467523167 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -125,7 +125,7 @@ private[execution] object HashedRel

[GitHub] [spark] maropu commented on a change in pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on a change in pull request #29389: URL: https://github.com/apache/spark/pull/29389#discussion_r467523040 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala ## @@ -82,6 +82,8 @@ case class BroadcastExchang

[GitHub] [spark] yanxiaole edited a comment on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration

yanxiaole edited a comment on pull request #29392: URL: https://github.com/apache/spark/pull/29392#issuecomment-670993081 It happens when data structure getting modified, mainly `delete` causes the problem. Take `LevelDB` implementation for example, in its iterator's `next` function

[GitHub] [spark] yanxiaole commented on pull request #29392: [SPARK-32574][CORE] Race condition in FsHistoryProvider listing iteration

yanxiaole commented on pull request #29392: URL: https://github.com/apache/spark/pull/29392#issuecomment-670993081 It happened when data structure getting modified, mainly `delete` caused the problem. Take `LevelDB` implementation for example, in its iterator's `next` function it wi

[GitHub] [spark] maropu commented on a change in pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on a change in pull request #29389: URL: https://github.com/apache/spark/pull/29389#discussion_r467522246 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/EliminateAntiJoin.scala ## @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache S

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670991720 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670991720 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

AmplabJenkins removed a comment on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670852395 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

SparkQA commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670991633 **[Test build #127231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127231/testReport)** for PR 29389 at commit [`25dd444`](https://github.com

[GitHub] [spark] maropu commented on pull request #29389: [SPARK-32573][SQL] Eliminate Anti Join when BuildSide is Empty

maropu commented on pull request #29389: URL: https://github.com/apache/spark/pull/29389#issuecomment-670991526 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

AmplabJenkins removed a comment on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670989942 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

AmplabJenkins commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670989942 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] maropu commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

maropu commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670989945 Thanks, @dongjoon-hyun ! This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [spark] SparkQA removed a comment on pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

SparkQA removed a comment on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670980002 **[Test build #127229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127229/testReport)** for PR 29265 at commit [`3ef408d`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29265: [SPARK-32462][WEBUI] Reset previous search text for datatable

SparkQA commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670989801 **[Test build #127229 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127229/testReport)** for PR 29265 at commit [`3ef408d`](https://github.co

[GitHub] [spark] viirya commented on a change in pull request #29066: [WIP][SPARK-23889] DataSourceV2: required sorting and clustering for writes

viirya commented on a change in pull request #29066: URL: https://github.com/apache/spark/pull/29066#discussion_r467515565 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/write/WriteBuilder.java ## @@ -23,17 +23,34 @@ import org.apache.spark.sql.connec

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29367: [SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

agrawaldevesh commented on a change in pull request #29367: URL: https://github.com/apache/spark/pull/29367#discussion_r467509103 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -503,6 +504,102 @@ class CoarseGrain

[GitHub] [spark] dongjoon-hyun commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

dongjoon-hyun commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670985037 Merged to master/3.0. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun closed pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

dongjoon-hyun closed pull request #29391: URL: https://github.com/apache/spark/pull/29391 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

AmplabJenkins commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670984586 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

AmplabJenkins removed a comment on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670984586 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

SparkQA commented on pull request #29391: URL: https://github.com/apache/spark/pull/29391#issuecomment-670984511 **[Test build #127230 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127230/testReport)** for PR 29391 at commit [`43fcf8a`](https://github.com

[GitHub] [spark] maropu commented on a change in pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

maropu commented on a change in pull request #29391: URL: https://github.com/apache/spark/pull/29391#discussion_r467514179 ## File path: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala ## @@ -46,11 +47,7 @@ class TPCDSQuerySuite extends BenchmarkQueryTest wit

[GitHub] [spark] maropu commented on a change in pull request #29391: [SPARK-32564][SQL][TEST][FOLLOWUP] Re-enable TPCDSQuerySuite with empty tables

maropu commented on a change in pull request #29391: URL: https://github.com/apache/spark/pull/29391#discussion_r467513958 ## File path: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala ## @@ -104,3 +101,23 @@ class TPCDSQuerySuite extends BenchmarkQueryTest w

[GitHub] [spark] maropu commented on pull request #29390: [SPARK-32564][SQL][TEST][3.0] Inject data statistics to simulate plan generation on actual TPCDS data

maropu commented on pull request #29390: URL: https://github.com/apache/spark/pull/29390#issuecomment-670983959 Thanks a lot, @dongjoon-hyun ! This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [spark] maropu commented on a change in pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

maropu commented on a change in pull request #28804: URL: https://github.com/apache/spark/pull/28804#discussion_r467472877 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -838,13 +880,17 @@ case class HashAggregateEx

[GitHub] [spark] maropu commented on a change in pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

maropu commented on a change in pull request #28804: URL: https://github.com/apache/spark/pull/28804#discussion_r467472877 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -838,13 +880,17 @@ case class HashAggregateEx

[GitHub] [spark] maropu commented on a change in pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

maropu commented on a change in pull request #28804: URL: https://github.com/apache/spark/pull/28804#discussion_r467472877 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -838,13 +880,17 @@ case class HashAggregateEx

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467512687 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +88,122 @@ case class ShuffledHashJoinEx

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467512524 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +88,122 @@ case class ShuffledHashJoinEx

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

agrawaldevesh commented on a change in pull request #29211: URL: https://github.com/apache/spark/pull/29211#discussion_r467511469 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -277,12 +282,59 @@ private[spark] class Coarse

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29265: [SPARK-32462][WEBUI] Don't save the previous search text for datatable

AmplabJenkins removed a comment on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670980082 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29265: [SPARK-32462][WEBUI] Don't save the previous search text for datatable

AmplabJenkins commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670980082 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29265: [SPARK-32462][WEBUI] Don't save the previous search text for datatable

SparkQA commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670980002 **[Test build #127229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127229/testReport)** for PR 29265 at commit [`3ef408d`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on pull request #29265: [SPARK-32462][WEBUI] Don't save the previous search text for datatable

dongjoon-hyun commented on pull request #29265: URL: https://github.com/apache/spark/pull/29265#issuecomment-670979794 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] dongjoon-hyun closed pull request #29372: [SPARK-32555][SQL] Add unique ID on query execution

dongjoon-hyun closed pull request #29372: URL: https://github.com/apache/spark/pull/29372 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] c21 commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

c21 commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467505445 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +88,122 @@ case class ShuffledHashJoinExec(

[GitHub] [spark] viirya commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

viirya commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467503454 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -91,13 +96,18 @@ private[execution] object HashedRel

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29372: [SPARK-32555][SQL] Add unique ID on query execution

AmplabJenkins removed a comment on pull request #29372: URL: https://github.com/apache/spark/pull/29372#issuecomment-670972473 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29372: [SPARK-32555][SQL] Add unique ID on query execution

AmplabJenkins commented on pull request #29372: URL: https://github.com/apache/spark/pull/29372#issuecomment-670972473 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29372: [SPARK-32555][SQL] Add unique ID on query execution

SparkQA commented on pull request #29372: URL: https://github.com/apache/spark/pull/29372#issuecomment-670972247 **[Test build #127225 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127225/testReport)** for PR 29372 at commit [`3f1bea2`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29372: [SPARK-32555][SQL] Add unique ID on query execution

SparkQA removed a comment on pull request #29372: URL: https://github.com/apache/spark/pull/29372#issuecomment-670944925 **[Test build #127225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127225/testReport)** for PR 29372 at commit [`3f1bea2`](https://gi

[GitHub] [spark] c21 commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

c21 commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r467502239 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -58,8 +65,19 @@ case class ShuffledHashJoinExec(

  1   2   3   >