[GitHub] [spark] huaxingao closed pull request #29159: [SPARK-32310][ML][PySpark][3.0] ML params default value parity

huaxingao closed pull request #29159: URL: https://github.com/apache/spark/pull/29159 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29223: DO_NOT_MERGE Just testing active thread count during test execution

AmplabJenkins removed a comment on pull request #29223: URL: https://github.com/apache/spark/pull/29223#issuecomment-663635392 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29223: DO_NOT_MERGE Just testing active thread count during test execution

AmplabJenkins commented on pull request #29223: URL: https://github.com/apache/spark/pull/29223#issuecomment-663635392 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29223: DO_NOT_MERGE Just testing active thread count during test execution

SparkQA removed a comment on pull request #29223: URL: https://github.com/apache/spark/pull/29223#issuecomment-663551388 **[Test build #126502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126502/testReport)** for PR 29223 at commit

[GitHub] [spark] SparkQA commented on pull request #29223: DO_NOT_MERGE Just testing active thread count during test execution

SparkQA commented on pull request #29223: URL: https://github.com/apache/spark/pull/29223#issuecomment-663634283 **[Test build #126502 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126502/testReport)** for PR 29223 at commit

[GitHub] [spark] moomindani commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

moomindani commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-663632834 @viirya Thanks for looking into this. Do we have any remaining tasks for this? This is an automated message

[GitHub] [spark] attilapiros commented on a change in pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

attilapiros commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r460168600 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -0,0 +1,401 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29168: [SPARK-32375][SQL] Basic functionality of table catalog v2 for JDBC

AmplabJenkins removed a comment on pull request #29168: URL: https://github.com/apache/spark/pull/29168#issuecomment-663629542 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29168: [SPARK-32375][SQL] Basic functionality of table catalog v2 for JDBC

AmplabJenkins commented on pull request #29168: URL: https://github.com/apache/spark/pull/29168#issuecomment-663629542 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29168: [SPARK-32375][SQL] Basic functionality of table catalog v2 for JDBC

SparkQA removed a comment on pull request #29168: URL: https://github.com/apache/spark/pull/29168#issuecomment-663493700 **[Test build #126499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126499/testReport)** for PR 29168 at commit

[GitHub] [spark] SparkQA commented on pull request #29168: [SPARK-32375][SQL] Basic functionality of table catalog v2 for JDBC

SparkQA commented on pull request #29168: URL: https://github.com/apache/spark/pull/29168#issuecomment-663628861 **[Test build #126499 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126499/testReport)** for PR 29168 at commit

[GitHub] [spark] attilapiros commented on a change in pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

attilapiros commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r460165492 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -0,0 +1,401 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] shaneknapp commented on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test

shaneknapp commented on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663626838 @dongjoon-hyun thanks for the breakdown! This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29225: [SPARK-32287][TESTS] Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

AmplabJenkins removed a comment on pull request #29225: URL: https://github.com/apache/spark/pull/29225#issuecomment-663623399 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on pull request #29146: [SPARK-32257][SQL] Reports explicit errors for invalid usage of SET command

cloud-fan commented on pull request #29146: URL: https://github.com/apache/spark/pull/29146#issuecomment-663623599 BTW after https://github.com/apache/spark/pull/29202 , let's make sure SET and RESET are consistent after this PR.

[GitHub] [spark] AmplabJenkins commented on pull request #29225: [SPARK-32287][TESTS] Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

AmplabJenkins commented on pull request #29225: URL: https://github.com/apache/spark/pull/29225#issuecomment-663623399 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29225: [SPARK-32287][TESTS] Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

SparkQA commented on pull request #29225: URL: https://github.com/apache/spark/pull/29225#issuecomment-663622742 **[Test build #126506 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126506/testReport)** for PR 29225 at commit

[GitHub] [spark] tgravescs commented on pull request #29225: [SPARK-32287][TESTS] Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

tgravescs commented on pull request #29225: URL: https://github.com/apache/spark/pull/29225#issuecomment-663620542 @Ngone51 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] tgravescs opened a new pull request #29225: [SPARK-32287][TESTS] Flaky Test: ExecutorAllocationManagerSuite.add executors default profile

tgravescs opened a new pull request #29225: URL: https://github.com/apache/spark/pull/29225 ### What changes were proposed in this pull request? I wasn't able to reproduce the failure but the best I can tell is that the allocation manager timer triggers and call

[GitHub] [spark] squito commented on pull request #29193: [SPARK-32003][CORE][3.0] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

squito commented on pull request #29193: URL: https://github.com/apache/spark/pull/29193#issuecomment-663619316 The failure in test run 126434, "BarrierTaskContextSuite.global sync by barrier() call" was supposedly fixed here: SPARK-31730

[GitHub] [spark] tgravescs commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

tgravescs commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663616748 thanks for looking @dongjoon-hyun, I went back and forth on the improvement vs bug. I can see it both ways, I decided to file it as a bug because without it we can't properly

[GitHub] [spark] dongjoon-hyun closed pull request #29202: [SPARK-32406][SQL] Make RESET syntax support single configuration reset

dongjoon-hyun closed pull request #29202: URL: https://github.com/apache/spark/pull/29202 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

agrawaldevesh commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r460151742 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -0,0 +1,401 @@ +/* + * Licensed to the Apache

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

agrawaldevesh commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r460150908 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -0,0 +1,401 @@ +/* + * Licensed to the Apache

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

agrawaldevesh commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r460150480 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ## @@ -939,12 +941,43 @@ private[spark] class

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460149058 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala ## @@ -454,6 +491,48 @@ case class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

AmplabJenkins removed a comment on pull request #28972: URL: https://github.com/apache/spark/pull/28972#issuecomment-663612956 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

AmplabJenkins commented on pull request #28972: URL: https://github.com/apache/spark/pull/28972#issuecomment-663612956 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

SparkQA removed a comment on pull request #28972: URL: https://github.com/apache/spark/pull/28972#issuecomment-663541118 **[Test build #126500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126500/testReport)** for PR 28972 at commit

[GitHub] [spark] SparkQA commented on pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

SparkQA commented on pull request #28972: URL: https://github.com/apache/spark/pull/28972#issuecomment-663612336 **[Test build #126500 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126500/testReport)** for PR 28972 at commit

[GitHub] [spark] peter-toth commented on a change in pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

peter-toth commented on a change in pull request #28885: URL: https://github.com/apache/spark/pull/28885#discussion_r460109287 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/ExchangeSuite.scala ## @@ -156,4 +158,46 @@ class ExchangeSuite extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins removed a comment on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663609604 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663609604 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

SparkQA commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663608930 **[Test build #126505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126505/testReport)** for PR 29224 at commit

[GitHub] [spark] andygrove commented on a change in pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

andygrove commented on a change in pull request #29224: URL: https://github.com/apache/spark/pull/29224#discussion_r460142890 ## File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensions.scala ## @@ -96,8 +96,10 @@ class SparkSessionExtensions { type

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

agrawaldevesh commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r460141178 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala ## @@ -454,6 +491,48 @@ case class

[GitHub] [spark] dongjoon-hyun closed pull request #29218: [SPARK-32408][BUILD] Enable crossPaths back to prevent side effects

dongjoon-hyun closed pull request #29218: URL: https://github.com/apache/spark/pull/29218 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] srowen commented on pull request #29159: [SPARK-32310][ML][PySpark][3.0] ML params default value parity

srowen commented on pull request #29159: URL: https://github.com/apache/spark/pull/29159#issuecomment-663606097 Same as the version for master - looks OK if there are no API or behavior changes (that are not bug fixes).

[GitHub] [spark] srowen commented on pull request #29153: [SPARK-32310][ML][PySpark] ML params default value parity in feature and tuning

srowen commented on pull request #29153: URL: https://github.com/apache/spark/pull/29153#issuecomment-663605712 Looks good if there are no API or behavior changes (that are not bug fixes). This is an automated message from

[GitHub] [spark] agrawaldevesh edited a comment on pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

agrawaldevesh edited a comment on pull request #29104: URL: https://github.com/apache/spark/pull/29104#issuecomment-663382186 No :) I am trying to figure out how to get this breaking thing. I have also asked a bunch of other folks who know codegen more to opine here.

[GitHub] [spark] viirya commented on a change in pull request #28911: [SPARK-32077][CORE] Support host-local shuffle data reading when external shuffle service is disabled

viirya commented on a change in pull request #28911: URL: https://github.com/apache/spark/pull/28911#discussion_r460136310 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/BlockStoreClient.java ## @@ -61,4 +78,62 @@ public MetricSet

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

dongjoon-hyun commented on a change in pull request #29224: URL: https://github.com/apache/spark/pull/29224#discussion_r460131565 ## File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensions.scala ## @@ -96,8 +96,10 @@ class SparkSessionExtensions {

[GitHub] [spark] dongjoon-hyun commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

dongjoon-hyun commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663598678 Thank you, @andygrove and @tgravescs . I'll take a look, too. This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663595664 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins removed a comment on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663595664 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

SparkQA commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663595067 **[Test build #126504 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126504/testReport)** for PR 29224 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test

dongjoon-hyun commented on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663593043 Yep. I understand, @ifilonenko . :) This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test

dongjoon-hyun edited a comment on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663593043 Thanks. I understand, @ifilonenko . :) This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins removed a comment on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663592153 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663592153 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] ifilonenko commented on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test

ifilonenko commented on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663591000 > Of course, I confirmed that 2020-07-20T22:25:51-07:00 test result is missing Run SparkR on simple dataframe.R example test. In short, I suspect that something happens on

[GitHub] [spark] tgravescs commented on a change in pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

tgravescs commented on a change in pull request #29224: URL: https://github.com/apache/spark/pull/29224#discussion_r460120367 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SparkSessionExtensionSuite.scala ## @@ -731,6 +755,30 @@ class MyExtensions extends

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #29217: [SPARK-32422][SQL][TESTS] Use python3 executable instead of python3.6 in IntegratedUDFTestUtils

dongjoon-hyun commented on a change in pull request #29217: URL: https://github.com/apache/spark/pull/29217#discussion_r460119469 ## File path: sql/core/src/test/scala/org/apache/spark/sql/IntegratedUDFTestUtils.scala ## @@ -197,7 +197,7 @@ object IntegratedUDFTestUtils

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins removed a comment on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663589071 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

SparkQA removed a comment on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663587886 **[Test build #126503 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126503/testReport)** for PR 29224 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins removed a comment on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663589066 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663589066 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

SparkQA commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663589050 **[Test build #126503 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126503/testReport)** for PR 29224 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins removed a comment on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663588586 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663588586 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

SparkQA commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663587886 **[Test build #126503 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126503/testReport)** for PR 29224 at commit

[GitHub] [spark] peter-toth commented on a change in pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

peter-toth commented on a change in pull request #28885: URL: https://github.com/apache/spark/pull/28885#discussion_r460115721 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/util/ReuseMap.scala ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] tgravescs commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

tgravescs commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663586691 @cloud-fan this is fixing another issue with AQE (https://github.com/apache/spark/pull/29134) and overriding would your PR cover something like this as well?

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins removed a comment on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663584614 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] tgravescs commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

tgravescs commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663586055 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins removed a comment on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663584133 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663584614 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

AmplabJenkins commented on pull request #29224: URL: https://github.com/apache/spark/pull/29224#issuecomment-663584133 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] peter-toth commented on a change in pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

peter-toth commented on a change in pull request #28885: URL: https://github.com/apache/spark/pull/28885#discussion_r460109287 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/ExchangeSuite.scala ## @@ -156,4 +158,46 @@ class ExchangeSuite extends

[GitHub] [spark] MaxGekk commented on a change in pull request #29220: [SPARK-32424][SQL] Fix silent data change for timestamp parsing if overflow happens

MaxGekk commented on a change in pull request #29220: URL: https://github.com/apache/spark/pull/29220#discussion_r460108583 ## File path: sql/core/src/test/resources/sql-tests/results/datetime-formatting-legacy.sql.out ## @@ -32,17 +32,17 @@ struct -- !query -select col,

[GitHub] [spark] andygrove opened a new pull request #29224: [SPARK-32430] [SQL] Allow plugins to inject rules into AQE query stage preparation

andygrove opened a new pull request #29224: URL: https://github.com/apache/spark/pull/29224 ### What changes were proposed in this pull request? Provide a generic mechanism for plugins to inject rules into the AQE "query prep" stage that happens before query stage creation.

[GitHub] [spark] huaxingao commented on pull request #29153: [SPARK-32310][ML][PySpark] ML params default value parity in feature and tuning

huaxingao commented on pull request #29153: URL: https://github.com/apache/spark/pull/29153#issuecomment-663580120 cc @srowen @viirya @zhengruifeng This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] huaxingao commented on pull request #29159: [SPARK-32310][ML][PySpark][3.0] ML params default value parity

huaxingao commented on pull request #29159: URL: https://github.com/apache/spark/pull/29159#issuecomment-663579419 @srowen Are you Okay with this? This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] attilapiros commented on a change in pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

attilapiros commented on a change in pull request #29014: URL: https://github.com/apache/spark/pull/29014#discussion_r460102245 ## File path: core/src/test/scala/org/apache/spark/deploy/DecommissionWorkerSuite.scala ## @@ -0,0 +1,401 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] squito commented on a change in pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

squito commented on a change in pull request #28885: URL: https://github.com/apache/spark/pull/28885#discussion_r460099447 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/ExchangeSuite.scala ## @@ -156,4 +158,46 @@ class ExchangeSuite extends

[GitHub] [spark] AmplabJenkins commented on pull request #29210: [WIP][SPARK-24497][SQL] Support recursive SQL query

AmplabJenkins commented on pull request #29210: URL: https://github.com/apache/spark/pull/29210#issuecomment-663572577 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29210: [WIP][SPARK-24497][SQL] Support recursive SQL query

AmplabJenkins removed a comment on pull request #29210: URL: https://github.com/apache/spark/pull/29210#issuecomment-663572577 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] tgravescs commented on pull request #29002: [SPARK-32175][CORE] Fix the order between initialization for ExecutorPlugin and starting heartbeat thread

tgravescs commented on pull request #29002: URL: https://github.com/apache/spark/pull/29002#issuecomment-663572512 ok looked at this a bit more and tested a few scenarios, I think this is fine, I would like it put right after heartbeater unless we have reason and would be nice to get test

[GitHub] [spark] SparkQA removed a comment on pull request #29210: [WIP][SPARK-24497][SQL] Support recursive SQL query

SparkQA removed a comment on pull request #29210: URL: https://github.com/apache/spark/pull/29210#issuecomment-663442924 **[Test build #126490 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126490/testReport)** for PR 29210 at commit

[GitHub] [spark] SparkQA commented on pull request #29210: [WIP][SPARK-24497][SQL] Support recursive SQL query

SparkQA commented on pull request #29210: URL: https://github.com/apache/spark/pull/29210#issuecomment-663571706 **[Test build #126490 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126490/testReport)** for PR 29210 at commit

[GitHub] [spark] cloud-fan edited a comment on pull request #29146: [SPARK-32257][SQL] Reports explicit errors for invalid usage of SET command

cloud-fan edited a comment on pull request #29146: URL: https://github.com/apache/spark/pull/29146#issuecomment-663569282 If we don't allow space in the config name by default(requires quoting), I think we can do that for other special chars as well. Then the parser rule can be very

[GitHub] [spark] cloud-fan edited a comment on pull request #29146: [SPARK-32257][SQL] Reports explicit errors for invalid usage of SET command

cloud-fan edited a comment on pull request #29146: URL: https://github.com/apache/spark/pull/29146#issuecomment-663569282 If we don't allow space in the config name by default(requires quoting), I think we can do that for other special chars as well. Then the parser rule can be very

[GitHub] [spark] cloud-fan commented on pull request #29146: [SPARK-32257][SQL] Reports explicit errors for invalid usage of SET command

cloud-fan commented on pull request #29146: URL: https://github.com/apache/spark/pull/29146#issuecomment-663569282 If we don't allow space in the config name by default(requires quoting), I think we can do that for other special chars as well. Then the parser rule can be very simple:

[GitHub] [spark] cloud-fan edited a comment on pull request #29146: [SPARK-32257][SQL] Reports explicit errors for invalid usage of SET command

cloud-fan edited a comment on pull request #29146: URL: https://github.com/apache/spark/pull/29146#issuecomment-663569282 If we don't allow space in the config name by default(requires quoting), I think we can do that for other special chars as well. Then the parser rule can be very

[GitHub] [spark] Fokko commented on a change in pull request #29121: [SPARK-32319][PYSPARK] Remove unused imports

Fokko commented on a change in pull request #29121: URL: https://github.com/apache/spark/pull/29121#discussion_r460090784 ## File path: python/pyspark/ml/tests/test_stat.py ## @@ -40,7 +40,7 @@ def test_chisquaretest(self): if __name__ == "__main__": -from

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29220: [SPARK-32424][SQL] Fix silent data change for timestamp parsing if overflow happens

AmplabJenkins removed a comment on pull request #29220: URL: https://github.com/apache/spark/pull/29220#issuecomment-663567296 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29220: [SPARK-32424][SQL] Fix silent data change for timestamp parsing if overflow happens

AmplabJenkins removed a comment on pull request #29220: URL: https://github.com/apache/spark/pull/29220#issuecomment-663567280 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29220: [SPARK-32424][SQL] Fix silent data change for timestamp parsing if overflow happens

AmplabJenkins commented on pull request #29220: URL: https://github.com/apache/spark/pull/29220#issuecomment-663567280 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on a change in pull request #29220: [SPARK-32424][SQL] Fix silent data change for timestamp parsing if overflow happens

cloud-fan commented on a change in pull request #29220: URL: https://github.com/apache/spark/pull/29220#discussion_r460089650 ## File path: sql/core/src/test/resources/sql-tests/inputs/datetime-parsing.sql ## @@ -1,5 +1,18 @@ --- TESTS FOR DATETIME PARSING FUNCTIONS --- +--

[GitHub] [spark] cloud-fan commented on a change in pull request #29220: [SPARK-32424][SQL] Fix silent data change for timestamp parsing if overflow happens

cloud-fan commented on a change in pull request #29220: URL: https://github.com/apache/spark/pull/29220#discussion_r460089355 ## File path: sql/core/src/test/resources/sql-tests/inputs/datetime-parsing-invalid.sql ## @@ -1,5 +1,14 @@ --- TESTS FOR DATETIME PARSING FUNCTIONS

[GitHub] [spark] cloud-fan commented on a change in pull request #29220: [SPARK-32424][SQL] Fix silent data change for timestamp parsing if overflow happens

cloud-fan commented on a change in pull request #29220: URL: https://github.com/apache/spark/pull/29220#discussion_r460088732 ## File path: sql/core/src/test/resources/sql-tests/results/datetime-formatting-legacy.sql.out ## @@ -32,17 +32,17 @@ struct -- !query -select

[GitHub] [spark] SparkQA removed a comment on pull request #29220: [SPARK-32424][SQL] Fix silent data change for timestamp parsing if overflow happens

SparkQA removed a comment on pull request #29220: URL: https://github.com/apache/spark/pull/29220#issuecomment-663464888 **[Test build #126494 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126494/testReport)** for PR 29220 at commit

[GitHub] [spark] SparkQA commented on pull request #29220: [SPARK-32424][SQL] Fix silent data change for timestamp parsing if overflow happens

SparkQA commented on pull request #29220: URL: https://github.com/apache/spark/pull/29220#issuecomment-663566585 **[Test build #126494 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126494/testReport)** for PR 29220 at commit

[GitHub] [spark] zero323 commented on pull request #29180: [SPARK-17333][PYSPARK] Enable mypy on the repository

zero323 commented on pull request #29180: URL: https://github.com/apache/spark/pull/29180#issuecomment-663564404 > Sounds good to me. I'm in favor of having it inline, but having the annotation in the pyi is a good first step :) There are trade-offs related to both strategies and

[GitHub] [spark] dongjoon-hyun commented on pull request #29219: Revert "[SPARK-32253][INFRA] Show errors only for the sbt tests of github actions"

dongjoon-hyun commented on pull request #29219: URL: https://github.com/apache/spark/pull/29219#issuecomment-663562682 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29215: [SPARK-32363][PYTHON][BUILD][3.0] Fix flakiness in pip package testing in Jenkins

AmplabJenkins removed a comment on pull request #29215: URL: https://github.com/apache/spark/pull/29215#issuecomment-663561688 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] dongjoon-hyun closed pull request #29215: [SPARK-32363][PYTHON][BUILD][3.0] Fix flakiness in pip package testing in Jenkins

dongjoon-hyun closed pull request #29215: URL: https://github.com/apache/spark/pull/29215 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins commented on pull request #29215: [SPARK-32363][PYTHON][BUILD][3.0] Fix flakiness in pip package testing in Jenkins

AmplabJenkins commented on pull request #29215: URL: https://github.com/apache/spark/pull/29215#issuecomment-663561688 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29215: [SPARK-32363][PYTHON][BUILD][3.0] Fix flakiness in pip package testing in Jenkins

SparkQA commented on pull request #29215: URL: https://github.com/apache/spark/pull/29215#issuecomment-663560559 **[Test build #126496 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126496/testReport)** for PR 29215 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29215: [SPARK-32363][PYTHON][BUILD][3.0] Fix flakiness in pip package testing in Jenkins

SparkQA removed a comment on pull request #29215: URL: https://github.com/apache/spark/pull/29215#issuecomment-663486167 **[Test build #126496 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126496/testReport)** for PR 29215 at commit

<    1   2   3   4   5   6   7   8   >