[GitHub] [spark] jiangxb1987 commented on a change in pull request #28839: [SPARK-32000][CORE][TESTS] Fix the flaky testcase for partially launched task in barrier-mode.

jiangxb1987 commented on a change in pull request #28839: URL: https://github.com/apache/spark/pull/28839#discussion_r441298122 ## File path: core/src/test/scala/org/apache/spark/scheduler/BarrierTaskContextSuite.scala ## @@ -275,7 +275,8 @@ class BarrierTaskContextSuite

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

AmplabJenkins removed a comment on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645168383 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

AmplabJenkins commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645168383 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

SparkQA removed a comment on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645116392 **[Test build #124151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124151/testReport)** for PR 28847 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28842: [SPARK-32006][SQL] Create date/timestamp formatters once before collect in `hiveResultString()`

cloud-fan commented on a change in pull request #28842: URL: https://github.com/apache/spark/pull/28842#discussion_r441297086 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/HiveResult.scala ## @@ -72,47 +90,32 @@ object HiveResult { } } - //

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

AmplabJenkins removed a comment on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-645167728 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

AmplabJenkins commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-645167728 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

SparkQA commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645167716 **[Test build #124151 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124151/testReport)** for PR 28847 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

SparkQA removed a comment on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-645093212 **[Test build #124146 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124146/testReport)** for PR 27066 at commit

[GitHub] [spark] SparkQA commented on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

SparkQA commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-645167153 **[Test build #124146 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124146/testReport)** for PR 27066 at commit

[GitHub] [spark] cloud-fan commented on pull request #28778: [SPARK-31949][SQL] Add spark.default.parallelism in SQLConf for isolated across session

cloud-fan commented on pull request #28778: URL: https://github.com/apache/spark/pull/28778#issuecomment-645167188 After more thoughts, I think the file partitions split logic itself is problematic. Its target is to make the number of partitions the same as the total number of cores,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645157060 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645156028 **[Test build #124154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124154/testReport)** for PR 28818 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645157066 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645157060 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645157055 **[Test build #124154 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124154/testReport)** for PR 28818 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645156370 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645156370 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-645156028 **[Test build #124154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124154/testReport)** for PR 28818 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

AmplabJenkins removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-64515 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

AmplabJenkins removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645152215 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

SparkQA removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645151419 **[Test build #124153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124153/testReport)** for PR 28848 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

AmplabJenkins commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645152215 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

SparkQA commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645152202 **[Test build #124153 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124153/testReport)** for PR 28848 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

AmplabJenkins removed a comment on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645149561 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

SparkQA commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645151419 **[Test build #124153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124153/testReport)** for PR 28848 at commit

[GitHub] [spark] wypoon commented on a change in pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

wypoon commented on a change in pull request #28848: URL: https://github.com/apache/spark/pull/28848#discussion_r441280737 ## File path: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ## @@ -540,6 +540,46 @@ class DAGSchedulerSuite extends

[GitHub] [spark] wypoon commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

wypoon commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645149861 @attilapiros This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins removed a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645149534 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645149534 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

AmplabJenkins commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-645149561 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

SparkQA commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645149223 **[Test build #124152 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124152/testReport)** for PR 28846 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

AmplabJenkins removed a comment on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-645149128 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] wypoon opened a new pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

wypoon opened a new pull request #28848: URL: https://github.com/apache/spark/pull/28848 …after executor is lost ### What changes were proposed in this pull request? If an executor is lost, the `DAGScheduler` handles the executor loss by removing the executor but does not

[GitHub] [spark] AmplabJenkins commented on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

AmplabJenkins commented on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-645149128 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

SparkQA removed a comment on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-645067909 **[Test build #124143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124143/testReport)** for PR 28761 at commit

[GitHub] [spark] SparkQA commented on pull request #28761: [SPARK-25557][SQL] Nested column predicate pushdown for ORC

SparkQA commented on pull request #28761: URL: https://github.com/apache/spark/pull/28761#issuecomment-645148461 **[Test build #124143 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124143/testReport)** for PR 28761 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins removed a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645146303 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins removed a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645146298 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

SparkQA removed a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645102759 **[Test build #124149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124149/testReport)** for PR 28846 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645146298 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

SparkQA commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645146152 **[Test build #124149 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124149/testReport)** for PR 28846 at commit

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r441273689 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1238,6 +1239,18 @@ package object config { s"The

[GitHub] [spark] siknezevic commented on a change in pull request #27246: [SPARK-30536][CORE][SQL] Sort-merge join operator spilling performance improvements

siknezevic commented on a change in pull request #27246: URL: https://github.com/apache/spark/pull/27246#discussion_r441273689 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1238,6 +1239,18 @@ package object config { s"The

[GitHub] [spark] cchighman closed pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman closed pull request #28841: URL: https://github.com/apache/spark/pull/28841 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cchighman closed pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

cchighman closed pull request #28841: URL: https://github.com/apache/spark/pull/28841 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] dongjoon-hyun closed pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

dongjoon-hyun closed pull request #28847: URL: https://github.com/apache/spark/pull/28847 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

dongjoon-hyun commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645134832 Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

dongjoon-hyun commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645122517 Thank you, @HyukjinKwon . This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon closed pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

HyukjinKwon closed pull request #28845: URL: https://github.com/apache/spark/pull/28845 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

HyukjinKwon commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645120469 Merged to master and branch-3.0. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] HyukjinKwon commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

HyukjinKwon commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645120390 Thanks @WeichenXu123 for review. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] HyukjinKwon commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

HyukjinKwon commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645120346 Let me just merge and go ahead since it's just all about a documentation and warning at the end where we can change without many restrictions. Let me know if you guys

[GitHub] [spark] HyukjinKwon closed pull request #28827: [SPARK-31989][SQL] Generate JSON rebasing files w/ 30 minutes step

HyukjinKwon closed pull request #28827: URL: https://github.com/apache/spark/pull/28827 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #28827: [SPARK-31989][SQL] Generate JSON rebasing files w/ 30 minutes step

HyukjinKwon commented on pull request #28827: URL: https://github.com/apache/spark/pull/28827#issuecomment-645119748 Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

AmplabJenkins removed a comment on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645114911 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

SparkQA commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645116392 **[Test build #124151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124151/testReport)** for PR 28847 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

dongjoon-hyun commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645115062 cc @tgravescs This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins commented on pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

AmplabJenkins commented on pull request #28847: URL: https://github.com/apache/spark/pull/28847#issuecomment-645114911 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun opened a new pull request #28847: [SPARK-29148][CORE][FOLLOWUP] Fix warning message to show a correct executor id

dongjoon-hyun opened a new pull request #28847: URL: https://github.com/apache/spark/pull/28847 ### What changes were proposed in this pull request? This aims to replace `executorIdsToBeRemoved` with `executorIdToBeRemoved`. ### Why are the changes needed? Since a wrong

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-645106680 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] fqaiser94 edited a comment on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

fqaiser94 edited a comment on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-645047815 @cloud-fan I've made fairly significant changes to the implementation in order to address the issue where poorly optimized physical plans resulted in massive amounts

[GitHub] [spark] AmplabJenkins commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

AmplabJenkins commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-645106680 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

SparkQA commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-645106412 **[Test build #124150 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124150/testReport)** for PR 28835 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

AmplabJenkins removed a comment on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645105255 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

AmplabJenkins commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645105255 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

SparkQA removed a comment on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645095283 **[Test build #124147 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124147/testReport)** for PR 28845 at commit

[GitHub] [spark] SparkQA commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

SparkQA commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645104956 **[Test build #124147 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124147/testReport)** for PR 28845 at commit

[GitHub] [spark] yaooqinn commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

yaooqinn commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-645104925 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins removed a comment on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645102991 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

AmplabJenkins commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645102991 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

SparkQA commented on pull request #28846: URL: https://github.com/apache/spark/pull/28846#issuecomment-645102759 **[Test build #124149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124149/testReport)** for PR 28846 at commit

[GitHub] [spark] viirya opened a new pull request #28846: [SPARK-32012][SQL] Incrementally create and materialize query stage to avoid unnecessary local shuffle

viirya opened a new pull request #28846: URL: https://github.com/apache/spark/pull/28846 ### What changes were proposed in this pull request? This patch changes the current way of creating query stages in AQE. Instead of creating query stages in batch, incrementally

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28844: [SPARK-32009][ML][PySpark] Remove deprecated method BisectingKMeansModel.computeCost

AmplabJenkins removed a comment on pull request #28844: URL: https://github.com/apache/spark/pull/28844#issuecomment-645096303 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #28844: [SPARK-32009][ML][PySpark] Remove deprecated method BisectingKMeansModel.computeCost

SparkQA removed a comment on pull request #28844: URL: https://github.com/apache/spark/pull/28844#issuecomment-645095273 **[Test build #124148 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124148/testReport)** for PR 28844 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28844: [SPARK-32009][ML][PySpark] Remove deprecated method BisectingKMeansModel.computeCost

AmplabJenkins removed a comment on pull request #28844: URL: https://github.com/apache/spark/pull/28844#issuecomment-645096296 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28844: [SPARK-32009][ML][PySpark] Remove deprecated method BisectingKMeansModel.computeCost

AmplabJenkins commented on pull request #28844: URL: https://github.com/apache/spark/pull/28844#issuecomment-645096296 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28844: [SPARK-32009][ML][PySpark] Remove deprecated method BisectingKMeansModel.computeCost

SparkQA commented on pull request #28844: URL: https://github.com/apache/spark/pull/28844#issuecomment-645096285 **[Test build #124148 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124148/testReport)** for PR 28844 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

AmplabJenkins removed a comment on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645095627 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

AmplabJenkins commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645095627 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28844: [SPARK-32009][ML][PySpark] Remove deprecated method BisectingKMeansModel.computeCost

SparkQA commented on pull request #28844: URL: https://github.com/apache/spark/pull/28844#issuecomment-645095273 **[Test build #124148 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124148/testReport)** for PR 28844 at commit

[GitHub] [spark] SparkQA commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

SparkQA commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645095283 **[Test build #124147 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124147/testReport)** for PR 28845 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

HyukjinKwon commented on pull request #28845: URL: https://github.com/apache/spark/pull/28845#issuecomment-645094746 cc @squito too FYI This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon opened a new pull request #28845: [SPARK-32011][PYTHON][CORE] Remove warnings about pin-thread modes and guide to use collectWithJobGroup

HyukjinKwon opened a new pull request #28845: URL: https://github.com/apache/spark/pull/28845 ### What changes were proposed in this pull request? This PR proposes to remove the warning about multi-thread in local properties, and change the guide to use `collectWithJobGroup` for

[GitHub] [spark] fqaiser94 commented on a change in pull request #27066: [SPARK-31317][SQL] Add withField method to Column

fqaiser94 commented on a change in pull request #27066: URL: https://github.com/apache/spark/pull/27066#discussion_r441230512 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/complexTypesSuite.scala ## @@ -452,4 +453,91 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28840: [SPARK-31999][SQL] Add refresh function command

AmplabJenkins removed a comment on pull request #28840: URL: https://github.com/apache/spark/pull/28840#issuecomment-645093592 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28844: [SPARK-32009][ML][PySpark] Remove deprecated method BisectingKMeansModel.computeCost

AmplabJenkins removed a comment on pull request #28844: URL: https://github.com/apache/spark/pull/28844#issuecomment-645093544 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28844: [SPARK-32009][ML][PySpark] Remove deprecated method BisectingKMeansModel.computeCost

AmplabJenkins commented on pull request #28844: URL: https://github.com/apache/spark/pull/28844#issuecomment-645093544 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28840: [SPARK-31999][SQL] Add refresh function command

AmplabJenkins commented on pull request #28840: URL: https://github.com/apache/spark/pull/28840#issuecomment-645093592 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on a change in pull request #28840: [SPARK-31999][SQL] Add refresh function command

maropu commented on a change in pull request #28840: URL: https://github.com/apache/spark/pull/28840#discussion_r441229539 ## File path: docs/sql-ref-syntax-aux-refresh-function.md ## @@ -0,0 +1,59 @@ +--- +layout: global +title: REFRESH FUNCTION +displayTitle: REFRESH

[GitHub] [spark] fqaiser94 commented on a change in pull request #27066: [SPARK-31317][SQL] Add withField method to Column

fqaiser94 commented on a change in pull request #27066: URL: https://github.com/apache/spark/pull/27066#discussion_r441229325 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala ## @@ -39,7 +41,18 @@ object

[GitHub] [spark] SparkQA commented on pull request #27066: [SPARK-31317][SQL] Add withField method to Column

SparkQA commented on pull request #27066: URL: https://github.com/apache/spark/pull/27066#issuecomment-645093212 **[Test build #124146 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124146/testReport)** for PR 27066 at commit

[GitHub] [spark] SparkQA commented on pull request #28840: [SPARK-31999][SQL] Add refresh function command

SparkQA commented on pull request #28840: URL: https://github.com/apache/spark/pull/28840#issuecomment-645093211 **[Test build #124145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124145/testReport)** for PR 28840 at commit

[GitHub] [spark] huaxingao commented on pull request #28844: [SPARK-32009][ML][PySpark] Remove deprecated method BisectingKMeansModel.computeCost

huaxingao commented on pull request #28844: URL: https://github.com/apache/spark/pull/28844#issuecomment-645093196 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] ulysses-you commented on a change in pull request #28840: [SPARK-31999][SQL] Add refresh function command

ulysses-you commented on a change in pull request #28840: URL: https://github.com/apache/spark/pull/28840#discussion_r441228944 ## File path: docs/sql-ref-syntax-aux-refresh-function.md ## @@ -0,0 +1,59 @@ +--- +layout: global +title: REFRESH FUNCTION +displayTitle: REFRESH

[GitHub] [spark] fqaiser94 commented on a change in pull request #27066: [SPARK-31317][SQL] Add withField method to Column

fqaiser94 commented on a change in pull request #27066: URL: https://github.com/apache/spark/pull/27066#discussion_r441228301 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala ## @@ -22,7 +22,9 @@ import

[GitHub] [spark] vanzin closed pull request #28635: [SPARK-31337][SQL]Support MS SQL Kerberos login in JDBC connector

vanzin closed pull request #28635: URL: https://github.com/apache/spark/pull/28635 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] vanzin commented on pull request #28635: [SPARK-31337][SQL]Support MS SQL Kerberos login in JDBC connector

vanzin commented on pull request #28635: URL: https://github.com/apache/spark/pull/28635#issuecomment-645091699 Merging to master. This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] HeartSaVioR edited a comment on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' for mixed string/array

HeartSaVioR edited a comment on pull request #28831: URL: https://github.com/apache/spark/pull/28831#issuecomment-644538409 cc. @kiszk @viirya as initial reviewers as I can see they're contributed the code part. cc. @bersprockets who helped to review initial patch.

[GitHub] [spark] ulysses-you commented on pull request #28840: [SPARK-31999][SQL] Add refresh function command

ulysses-you commented on pull request #28840: URL: https://github.com/apache/spark/pull/28840#issuecomment-645083564 Hive support `reload functions` that reload all function. `refresh function` just like `refresh table`, invalid cache for one function.

  1   2   3   4   5   6   7   >