[GitHub] [spark] SparkQA commented on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read

2021-02-05 Thread GitBox
SparkQA commented on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774420334 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39534/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774415544 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134949/ -

[GitHub] [spark] SparkQA commented on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read

2021-02-05 Thread GitBox
SparkQA commented on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774415968 **[Test build #134951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134951/testReport)** for PR 31489 at commit [`f33b5a8`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774415544 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134949/ -

[GitHub] [spark] SparkQA removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
SparkQA removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774380337 **[Test build #134949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134949/testReport)** for PR 24559 at commit [`15e127f`](https://gi

[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774410185 **[Test build #134949 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134949/testReport)** for PR 24559 at commit [`15e127f`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-774407572 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39533/

[GitHub] [spark] AmplabJenkins commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-774407572 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39533/ -

[GitHub] [spark] SparkQA commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

2021-02-05 Thread GitBox
SparkQA commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-774402870 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39533/ ---

[GitHub] [spark] SparkQA commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

2021-02-05 Thread GitBox
SparkQA commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-774402314 **[Test build #134950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134950/testReport)** for PR 31495 at commit [`df4dd75`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774401013 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39532/

[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774401013 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39532/ -

[GitHub] [spark] SparkQA commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

2021-02-05 Thread GitBox
SparkQA commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-774399753 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39533/ -

[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774397966 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39532/ ---

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase on SS

2021-02-05 Thread GitBox
HeartSaVioR commented on a change in pull request #31495: URL: https://github.com/apache/spark/pull/31495#discussion_r571355810 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLog.scala ## @@ -46,6 +47,23 @@ import org.apache.spark.sql.c

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase on SS

2021-02-05 Thread GitBox
HeartSaVioR commented on a change in pull request #31495: URL: https://github.com/apache/spark/pull/31495#discussion_r571355810 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLog.scala ## @@ -46,6 +47,23 @@ import org.apache.spark.sql.c

[GitHub] [spark] HeartSaVioR opened a new pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase on SS

2021-02-05 Thread GitBox
HeartSaVioR opened a new pull request #31495: URL: https://github.com/apache/spark/pull/31495 ### What changes were proposed in this pull request? This PR proposes to optimize WAL commit phase via following changes: * cache offset log to avoid FS get operation per batch * us

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774394722 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134945/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774394722 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134945/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774340507 **[Test build #134945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134945/testReport)** for PR 31494 at commit [`dbbde92`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774388999 **[Test build #134945 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134945/testReport)** for PR 31494 at commit [`dbbde92`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774387534 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39532/ -

[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
imback82 commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r571342363 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala ## @@ -437,9 +437,8 @@ class DataSourceV2

[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
imback82 commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r571342363 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala ## @@ -437,9 +437,8 @@ class DataSourceV2

[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774380337 **[Test build #134949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134949/testReport)** for PR 24559 at commit [`15e127f`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774379652 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134941/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774379653 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134947/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774379654 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134946/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774379654 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134946/ -

[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774379653 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134947/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774379652 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134941/ -

[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
imback82 commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r571282171 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala ## @@ -116,7 +116,11 @@ private[sql] object Catalog

[GitHub] [spark] imback82 commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
imback82 commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774375335 cc @cloud-fan @MaxGekk This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [spark] SparkQA removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
SparkQA removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774336759 **[Test build #134946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134946/testReport)** for PR 31493 at commit [`233ec5a`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774290518 **[Test build #134941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134941/testReport)** for PR 31494 at commit [`0065170`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774371375 **[Test build #134946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134946/testReport)** for PR 31493 at commit [`233ec5a`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774371350 **[Test build #134941 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134941/testReport)** for PR 31494 at commit [`0065170`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
SparkQA removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774370078 **[Test build #134947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134947/testReport)** for PR 24559 at commit [`86c6ce7`](https://gi

[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774371188 **[Test build #134947 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134947/testReport)** for PR 24559 at commit [`86c6ce7`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774370078 **[Test build #134947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134947/testReport)** for PR 24559 at commit [`86c6ce7`](https://github.com

[GitHub] [spark] SparkQA removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
SparkQA removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774368595 **[Test build #134948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134948/testReport)** for PR 24559 at commit [`8f1084e`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774369530 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134948/ -

[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774369530 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134948/ -

[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774369521 **[Test build #134948 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134948/testReport)** for PR 24559 at commit [`8f1084e`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774369327 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39531/

[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774369327 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39531/ -

[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774368595 **[Test build #134948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134948/testReport)** for PR 24559 at commit [`8f1084e`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774367636 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39528/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774367634 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39530/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774367635 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39529/

[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774367634 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39530/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774367636 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39528/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774367635 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39529/ -

[GitHub] [spark] github-actions[bot] commented on pull request #30115: [WIP][SPARK-33200][CORE][K8S] Verify all shuffle blocks migrated in K8s integration testing

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #30115: URL: https://github.com/apache/spark/pull/30115#issuecomment-774362521 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] github-actions[bot] closed pull request #28938: [WIP][SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

2021-02-05 Thread GitBox
github-actions[bot] closed pull request #28938: URL: https://github.com/apache/spark/pull/28938 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] github-actions[bot] commented on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements

2021-02-05 Thread GitBox
github-actions[bot] commented on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-774362528 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] github-actions[bot] closed pull request #30158: [SPARK-33249][CORE][UI] Add status plugin for live application

2021-02-05 Thread GitBox
github-actions[bot] closed pull request #30158: URL: https://github.com/apache/spark/pull/30158 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774359284 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39529/ ---

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774357618 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39528/ ---

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774353696 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774353694 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134943/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774353695 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134940/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774353695 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134940/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774353694 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134943/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774353697 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774352078 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39529/ -

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774350161 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39528/ -

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774346760 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39527/ ---

[GitHub] [spark] SparkQA removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation

2021-02-05 Thread GitBox
SparkQA removed a comment on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774295061 **[Test build #134943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134943/testReport)** for PR 31450 at commit [`d37289d`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation

2021-02-05 Thread GitBox
SparkQA commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774345612 **[Test build #134943 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134943/testReport)** for PR 31450 at commit [`d37289d`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774340507 **[Test build #134945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134945/testReport)** for PR 31494 at commit [`dbbde92`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-774339535 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134939/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-774339535 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134939/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

2021-02-05 Thread GitBox
SparkQA removed a comment on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-774230546 **[Test build #134939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134939/testReport)** for PR 31491 at commit [`222be6e`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType

2021-02-05 Thread GitBox
SparkQA commented on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-774338901 **[Test build #134939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134939/testReport)** for PR 31491 at commit [`222be6e`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
SparkQA removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774296346 **[Test build #134942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134942/testReport)** for PR 31493 at commit [`54e0bce`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774338270 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134942/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774338270 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134942/ -

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774337794 **[Test build #134942 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134942/testReport)** for PR 31493 at commit [`54e0bce`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774336759 **[Test build #134946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134946/testReport)** for PR 31493 at commit [`233ec5a`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774336633 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39527/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774336116 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39526/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774336114 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39525/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774336115 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39524/

[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774336114 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39525/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774336116 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39526/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774336115 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39524/ -

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774322983 **[Test build #134944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134944/testReport)** for PR 31494 at commit [`c757b6d`](https://github.com

[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
imback82 commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r571282171 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala ## @@ -116,7 +116,11 @@ private[sql] object Catalog

[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
imback82 commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r571240319 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/TableChange.java ## @@ -64,7 +64,20 @@ static TableChange setProperty(St

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774319136 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39524/ ---

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774318369 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39525/ ---

[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation

2021-02-05 Thread GitBox
SparkQA commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774318333 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39526/ ---

[GitHub] [spark] holdenk commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

2021-02-05 Thread GitBox
holdenk commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774316948 I think the Kafka failures are unrelated. This is an automated message from the Apache Git Service. To respond t

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31492: [SPARK-34346][CORE][SQL][3.0] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31492: URL: https://github.com/apache/spark/pull/31492#issuecomment-774314203 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134938/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read

2021-02-05 Thread GitBox
AmplabJenkins removed a comment on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774314202 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39523/

[GitHub] [spark] AmplabJenkins commented on pull request #31492: [SPARK-34346][CORE][SQL][3.0] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may caus

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31492: URL: https://github.com/apache/spark/pull/31492#issuecomment-774314203 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134938/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read

2021-02-05 Thread GitBox
AmplabJenkins commented on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774314202 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39523/ -

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

2021-02-05 Thread GitBox
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774311918 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39524/ -

[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation

2021-02-05 Thread GitBox
SparkQA commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774306946 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39526/ -

  1   2   3   4   5   6   7   8   >