[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460267914 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -323,11 +374,20 @@ private[joins] object UnsafeHash

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460269289 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -923,9 +1006,9 @@ case class HashedRelationBroadcas

[GitHub] [spark] rdblue commented on a change in pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions Catalog APIs on DataSourceV2

rdblue commented on a change in pull request #28617: URL: https://github.com/apache/spark/pull/28617#discussion_r460270048 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/SupportsPartitions.java ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apach

[GitHub] [spark] SparkQA commented on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

SparkQA commented on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-663718462 **[Test build #126507 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126507/testReport)** for PR 29193 at commit [`e54f221`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lo

SparkQA removed a comment on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-663655001 **[Test build #126507 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126507/testReport)** for PR 29193 at commit [`e54f221`](https://gi

[GitHub] [spark] rdblue commented on a change in pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions Catalog APIs on DataSourceV2

rdblue commented on a change in pull request #28617: URL: https://github.com/apache/spark/pull/28617#discussion_r460271013 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/SupportsPartitions.java ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apach

[GitHub] [spark] AmplabJenkins commented on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

AmplabJenkins commented on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-663719259 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor

AmplabJenkins removed a comment on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-663719259 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460272135 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -896,22 +969,32 @@ private[joins] object LongHashed

[GitHub] [spark] viirya commented on pull request #29146: [SPARK-32257][SQL] Reports explicit errors for invalid usage of SET command

viirya commented on pull request #29146: URL: https://github.com/apache/spark/pull/29146#issuecomment-663720249 > If we don't allow space in the config name by default(requires quoting), I think we can do that for other special chars as well. Then the parser rule can be very simple: >

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor

AmplabJenkins removed a comment on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-663719262 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460272135 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -896,22 +969,32 @@ private[joins] object LongHashed

[GitHub] [spark] rdblue commented on a change in pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions Catalog APIs on DataSourceV2

rdblue commented on a change in pull request #28617: URL: https://github.com/apache/spark/pull/28617#discussion_r460272432 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/SupportsPartitions.java ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apach

[GitHub] [spark] rdblue commented on a change in pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions Catalog APIs on DataSourceV2

rdblue commented on a change in pull request #28617: URL: https://github.com/apache/spark/pull/28617#discussion_r460272921 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/SupportsPartitions.java ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apach

[GitHub] [spark] rdblue commented on a change in pull request #28617: [SPARK-31694][SQL] Add SupportsPartitions Catalog APIs on DataSourceV2

rdblue commented on a change in pull request #28617: URL: https://github.com/apache/spark/pull/28617#discussion_r460273149 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/SupportsPartitions.java ## @@ -0,0 +1,105 @@ +/* + * Licensed to the Apach

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460273465 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -896,22 +969,32 @@ private[joins] object LongHashed

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460273465 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -896,22 +969,32 @@ private[joins] object LongHashed

[GitHub] [spark] SparkQA commented on pull request #29190: [do not review][wip] Trying to reduce the flakyness of BlockManagerDecommissionIntegrationSuite

SparkQA commented on pull request #29190: URL: https://github.com/apache/spark/pull/29190#issuecomment-663723064 **[Test build #126508 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126508/testReport)** for PR 29190 at commit [`2f6264d`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29190: [do not review][wip] Trying to reduce the flakyness of BlockManagerDecommissionIntegrationSuite

SparkQA removed a comment on pull request #29190: URL: https://github.com/apache/spark/pull/29190#issuecomment-663655061 **[Test build #126508 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126508/testReport)** for PR 29190 at commit [`2f6264d`](https://gi

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460275713 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala ## @@ -133,10 +142,38 @@ case class BroadcastHashJ

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460276050 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -323,11 +374,20 @@ private[joins] object UnsafeHash

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460276191 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -896,22 +969,32 @@ private[joins] object LongHashed

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460275829 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -81,6 +81,34 @@ private[execution] sealed trait Has

[GitHub] [spark] AmplabJenkins commented on pull request #29190: [do not review][wip] Trying to reduce the flakyness of BlockManagerDecommissionIntegrationSuite

AmplabJenkins commented on pull request #29190: URL: https://github.com/apache/spark/pull/29190#issuecomment-663723914 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29190: [do not review][wip] Trying to reduce the flakyness of BlockManagerDecommissionIntegrationSuite

AmplabJenkins removed a comment on pull request #29190: URL: https://github.com/apache/spark/pull/29190#issuecomment-663723914 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

SparkQA commented on pull request #29104: URL: https://github.com/apache/spark/pull/29104#issuecomment-663725492 **[Test build #126513 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126513/testReport)** for PR 29104 at commit [`0b2a565`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

AmplabJenkins commented on pull request #29104: URL: https://github.com/apache/spark/pull/29104#issuecomment-663725865 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

AmplabJenkins removed a comment on pull request #29104: URL: https://github.com/apache/spark/pull/29104#issuecomment-663725865 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] gaborgsomogyi closed pull request #29223: DO_NOT_MERGE Just testing active thread count during test execution

gaborgsomogyi closed pull request #29223: URL: https://github.com/apache/spark/pull/29223 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

agrawaldevesh commented on a change in pull request #29211: URL: https://github.com/apache/spark/pull/29211#discussion_r460282833 ## File path: core/src/test/scala/org/apache/spark/storage/BlockManagerDecommissionIntegrationSuite.scala ## @@ -213,10 +213,7 @@ class BlockManage

[GitHub] [spark] dongjoon-hyun commented on pull request #29124: [WIP][SPARK-31168][BUILD] Upgrade Scala to 2.12.12

dongjoon-hyun commented on pull request #29124: URL: https://github.com/apache/spark/pull/29124#issuecomment-663729350 Given the current status, do we need to skip Scala `2.12.12` like `2.12.11`, too? This is an automated me

[GitHub] [spark] c21 commented on pull request #29216: [SPARK-32420][SQL] Add handling for unique key in non-codegen hash join

c21 commented on pull request #29216: URL: https://github.com/apache/spark/pull/29216#issuecomment-663731196 cc @cloud-fan and @sameeragarwal if you guys can help take a look. Thanks! This is an automated message from

[GitHub] [spark] SparkQA commented on pull request #29224: [SPARK-32430][SQL] Extend SparkSessionExtensions to inject rules into AQE query stage preparation

SparkQA commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663733269 **[Test build #126505 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126505/testReport)** for PR 29224 at commit [`182d4f9`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29224: [SPARK-32430][SQL] Extend SparkSessionExtensions to inject rules into AQE query stage preparation

SparkQA removed a comment on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663608930 **[Test build #126505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126505/testReport)** for PR 29224 at commit [`182d4f9`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29224: [SPARK-32430][SQL] Extend SparkSessionExtensions to inject rules into AQE query stage preparation

AmplabJenkins commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663734229 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29224: [SPARK-32430][SQL] Extend SparkSessionExtensions to inject rules into AQE query stage preparation

AmplabJenkins removed a comment on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663734229 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

SparkQA commented on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-663734707 **[Test build #126509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126509/testReport)** for PR 29020 at commit [`895f5fd`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

SparkQA removed a comment on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-663674372 **[Test build #126509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126509/testReport)** for PR 29020 at commit [`895f5fd`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

AmplabJenkins commented on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-663735266 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

AmplabJenkins removed a comment on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-663735266 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

AmplabJenkins removed a comment on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-663735280 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126

[GitHub] [spark] sarutak commented on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

sarutak commented on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-663738936 ok to test. This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

AmplabJenkins removed a comment on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-657332189 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

AmplabJenkins commented on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-663739451 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

AmplabJenkins removed a comment on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-663739451 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

SparkQA commented on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-663740138 **[Test build #126510 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126510/testReport)** for PR 29211 at commit [`484f8e2`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

SparkQA removed a comment on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-663683416 **[Test build #126510 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126510/testReport)** for PR 29211 at commit [`484f8e2`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

AmplabJenkins removed a comment on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-663740503 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

AmplabJenkins commented on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-663740503 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29211: [SPARK-31197][CORE] Shutdown executor once we are done decommissioning

AmplabJenkins removed a comment on pull request #29211: URL: https://github.com/apache/spark/pull/29211#issuecomment-663740510 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126

[GitHub] [spark] SparkQA commented on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

SparkQA commented on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-663741571 **[Test build #126514 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126514/testReport)** for PR 29082 at commit [`af19675`](https://github.com

[GitHub] [spark] holdenk commented on pull request #29179: [WIP][SPARK-32381][CORE][SQL] Explore allowing parallel listing & non-location sensitive listing in core

holdenk commented on pull request #29179: URL: https://github.com/apache/spark/pull/29179#issuecomment-663753558 > > Interesting. Is this specific to the S3A impl or is there a higher base class? I want to make it work with multiple file formats if possible. > > it's in hadoop common

[GitHub] [spark] SparkQA commented on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

SparkQA commented on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-663753855 **[Test build #126512 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126512/testReport)** for PR 29020 at commit [`ddcb772`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

SparkQA removed a comment on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-663695235 **[Test build #126512 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126512/testReport)** for PR 29020 at commit [`ddcb772`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

AmplabJenkins commented on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-663754437 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] viirya commented on a change in pull request #29216: [SPARK-32420][SQL] Add handling for unique key in non-codegen hash join

viirya commented on a change in pull request #29216: URL: https://github.com/apache/spark/pull/29216#discussion_r460309645 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashJoin.scala ## @@ -108,47 +108,73 @@ trait HashJoin extends BaseJoinExec {

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29020: [SPARK-23431][CORE] Expose stage level peak executor metrics via REST API

AmplabJenkins removed a comment on pull request #29020: URL: https://github.com/apache/spark/pull/29020#issuecomment-663754437 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] viirya commented on a change in pull request #29216: [SPARK-32420][SQL] Add handling for unique key in non-codegen hash join

viirya commented on a change in pull request #29216: URL: https://github.com/apache/spark/pull/29216#discussion_r460310001 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashJoin.scala ## @@ -172,14 +208,26 @@ trait HashJoin extends BaseJoinExec {

[GitHub] [spark] wypoon commented on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-663758369 retest this please This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [spark] SparkQA commented on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

SparkQA commented on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-663758723 **[Test build #126515 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126515/testReport)** for PR 29193 at commit [`e54f221`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

AmplabJenkins commented on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-663758945 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor

AmplabJenkins removed a comment on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-663758945 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] wypoon commented on pull request #29182: [SPARK-32003][CORE][2.4] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on pull request #29182: URL: https://github.com/apache/spark/pull/29182#issuecomment-663759256 @dongjoon-hyun this backport has a clean [build](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126349/) in the most recent retry. This can be merged independ

[GitHub] [spark] c21 commented on a change in pull request #29216: [SPARK-32420][SQL] Add handling for unique key in non-codegen hash join

c21 commented on a change in pull request #29216: URL: https://github.com/apache/spark/pull/29216#discussion_r460318542 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashJoin.scala ## @@ -172,14 +208,26 @@ trait HashJoin extends BaseJoinExec {

[GitHub] [spark] c21 commented on a change in pull request #29216: [SPARK-32420][SQL] Add handling for unique key in non-codegen hash join

c21 commented on a change in pull request #29216: URL: https://github.com/apache/spark/pull/29216#discussion_r460318341 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashJoin.scala ## @@ -108,47 +108,73 @@ trait HashJoin extends BaseJoinExec {

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

agrawaldevesh commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r460322584 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -0,0 +1,401 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

agrawaldevesh commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r460322584 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -0,0 +1,401 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [spark] SparkQA commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

SparkQA commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-663767316 **[Test build #126516 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126516/testReport)** for PR 29014 at commit [`737d5ba`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins removed a comment on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-663767580 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-663767580 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

SparkQA removed a comment on pull request #29198: URL: https://github.com/apache/spark/pull/29198#issuecomment-663686246 **[Test build #126511 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126511/testReport)** for PR 29198 at commit [`27f2fb1`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

SparkQA commented on pull request #29198: URL: https://github.com/apache/spark/pull/29198#issuecomment-663777176 **[Test build #126511 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126511/testReport)** for PR 29198 at commit [`27f2fb1`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

AmplabJenkins removed a comment on pull request #29198: URL: https://github.com/apache/spark/pull/29198#issuecomment-663777586 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29198: [SPARK-32401][SQL] Migrate function related commands to new resolution framework

AmplabJenkins commented on pull request #29198: URL: https://github.com/apache/spark/pull/29198#issuecomment-663777586 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] c21 commented on a change in pull request #29216: [SPARK-32420][SQL] Add handling for unique key in non-codegen hash join

c21 commented on a change in pull request #29216: URL: https://github.com/apache/spark/pull/29216#discussion_r460334659 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashJoin.scala ## @@ -172,14 +208,26 @@ trait HashJoin extends BaseJoinExec {

[GitHub] [spark] agrawaldevesh opened a new pull request #29226: Make the block manager decommissioning test be less flaky

agrawaldevesh opened a new pull request #29226: URL: https://github.com/apache/spark/pull/29226 ### What changes were proposed in this pull request? It's possible for this test to schedule the 3 tasks on just 2 out of 3 executors and then end up decommissioning the third one. Since

[GitHub] [spark] SparkQA commented on pull request #29226: Make the block manager decommissioning test be less flaky

SparkQA commented on pull request #29226: URL: https://github.com/apache/spark/pull/29226#issuecomment-663780471 **[Test build #126517 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126517/testReport)** for PR 29226 at commit [`7659d22`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #29226: Make the block manager decommissioning test be less flaky

AmplabJenkins commented on pull request #29226: URL: https://github.com/apache/spark/pull/29226#issuecomment-663780638 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29226: Make the block manager decommissioning test be less flaky

AmplabJenkins removed a comment on pull request #29226: URL: https://github.com/apache/spark/pull/29226#issuecomment-663780638 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

SparkQA removed a comment on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-663741571 **[Test build #126514 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126514/testReport)** for PR 29082 at commit [`af19675`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

SparkQA commented on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-663781151 **[Test build #126514 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126514/testReport)** for PR 29082 at commit [`af19675`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

AmplabJenkins commented on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-663781538 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

AmplabJenkins removed a comment on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-663781538 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] agrawaldevesh commented on pull request #29226: Fix flakyness of BlockManagerDecommissionIntegrationSuite

agrawaldevesh commented on pull request #29226: URL: https://github.com/apache/spark/pull/29226#issuecomment-663782885 cc: @holdenk @attilapiros for review please. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29226: Fix flakyness of BlockManagerDecommissionIntegrationSuite

AmplabJenkins commented on pull request #29226: URL: https://github.com/apache/spark/pull/29226#issuecomment-663783050 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29226: Fix flakyness of BlockManagerDecommissionIntegrationSuite

AmplabJenkins removed a comment on pull request #29226: URL: https://github.com/apache/spark/pull/29226#issuecomment-663783050 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] dongjoon-hyun commented on pull request #29182: [SPARK-32003][CORE][2.4] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

dongjoon-hyun commented on pull request #29182: URL: https://github.com/apache/spark/pull/29182#issuecomment-663783307 @wypoon . Please see my comment. I didn't say this is blocked by Jenkins. This is blocked by the Apache Spark backporting policy. To prevent a regression at higher version

[GitHub] [spark] dongjoon-hyun commented on pull request #29182: [SPARK-32003][CORE][2.4] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

dongjoon-hyun commented on pull request #29182: URL: https://github.com/apache/spark/pull/29182#issuecomment-663783581 There is no `independent` backporting. And, we do sometimes *forward-port*, but that is not a recommend way in Apache Spark. It's allowed exceptionally. -

[GitHub] [spark] github-actions[bot] closed pull request #26161: [SPARK-27900][CORE][K8s] Add `spark.driver.killOnOOMError` flag in cluster mode

github-actions[bot] closed pull request #26161: URL: https://github.com/apache/spark/pull/26161 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] github-actions[bot] commented on pull request #28034: [SPARK-31268][CORE]Initial Task Executor Metrics with latestMetrics

github-actions[bot] commented on pull request #28034: URL: https://github.com/apache/spark/pull/28034#issuecomment-663783593 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #29182: [SPARK-32003][CORE][2.4] When external shuffle service is used, unregister outputs for executor on fetch failure after executor

dongjoon-hyun edited a comment on pull request #29182: URL: https://github.com/apache/spark/pull/29182#issuecomment-663783307 @wypoon . Please see my comment. I didn't say this is blocked by Jenkins. This is blocked by the Apache Spark backporting policy. To prevent a regression at higher

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #29182: [SPARK-32003][CORE][2.4] When external shuffle service is used, unregister outputs for executor on fetch failure after executor

dongjoon-hyun edited a comment on pull request #29182: URL: https://github.com/apache/spark/pull/29182#issuecomment-663783581 There is no `independent` backporting. And, we do sometimes *forward-porting*, but that is not a recommend way in Apache Spark. It's allowed exceptionally. > Thi

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #29182: [SPARK-32003][CORE][2.4] When external shuffle service is used, unregister outputs for executor on fetch failure after executor

dongjoon-hyun edited a comment on pull request #29182: URL: https://github.com/apache/spark/pull/29182#issuecomment-663783581 There is no `independent` backporting. And, we do sometimes *forward-port*, but that is not a recommend way in Apache Spark. It's allowed exceptionally. > This ca

[GitHub] [spark] SparkQA commented on pull request #29226: Fix flakyness of BlockManagerDecommissionIntegrationSuite

SparkQA commented on pull request #29226: URL: https://github.com/apache/spark/pull/29226#issuecomment-663784088 **[Test build #126518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126518/testReport)** for PR 29226 at commit [`558702e`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

SparkQA commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-663786611 **[Test build #126516 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126516/testReport)** for PR 29014 at commit [`737d5ba`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-663786704 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins removed a comment on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-663786704 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

SparkQA removed a comment on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-663767316 **[Test build #126516 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126516/testReport)** for PR 29014 at commit [`737d5ba`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins removed a comment on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-663786710 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/126

[GitHub] [spark] leanken commented on pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on pull request #29104: URL: https://github.com/apache/spark/pull/29104#issuecomment-663787183 > ensure Thanks for your great help during the review. And i sure will make sure that the BHJ is not regressed by this change. --

<    1   2   3   4   5   6   7   8   >