[GitHub] [spark] dongjoon-hyun edited a comment on pull request #32830: [SPARK-32975][K8S][FOLLOWUP] Avoid None.get exception from SPARK-32975

2021-06-08 Thread GitBox
dongjoon-hyun edited a comment on pull request #32830: URL: https://github.com/apache/spark/pull/32830#issuecomment-857402802 BTW, in the PR title, `[SPARK-32975][K8S][FOLLOWUP]` implies `from SPARK-32975`. So, you don't need to say `from SPARK-32975`. ```

[GitHub] [spark] dongjoon-hyun commented on pull request #32830: [SPARK-32975][K8S][FOLLOWUP] Avoid None.get exception from SPARK-32975

2021-06-08 Thread GitBox
dongjoon-hyun commented on pull request #32830: URL: https://github.com/apache/spark/pull/32830#issuecomment-857402802 BTW, `[SPARK-32975][K8S][FOLLOWUP]` implies `from SPARK-32975`. So, you don't need to say `from SPARK-32975`. ``` [SPARK-32975][K8S][FOLLOWUP] Avoid None.get

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #32824: [SPARK-30993][FOLLOWUP][SQL] Refactor LocalDateTimeUDT as YearUDT in UserDefinedTypeSuite

2021-06-08 Thread GitBox
dongjoon-hyun edited a comment on pull request #32824: URL: https://github.com/apache/spark/pull/32824#issuecomment-857401557 Hi, @gengliangwang . SPARK-30993 was released long time ago by 2.4.6 and 3.0.0. If you don't mind, could you use a new JIRA instead of the follow-up next

[GitHub] [spark] dongjoon-hyun commented on pull request #32834: [SPARK-35690][SS] Stream-stream join keys should be reordered properly

2021-06-08 Thread GitBox
dongjoon-hyun commented on pull request #32834: URL: https://github.com/apache/spark/pull/32834#issuecomment-857402212 cc @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #32824: [SPARK-30993][FOLLOWUP][SQL] Refactor LocalDateTimeUDT as YearUDT in UserDefinedTypeSuite

2021-06-08 Thread GitBox
dongjoon-hyun edited a comment on pull request #32824: URL: https://github.com/apache/spark/pull/32824#issuecomment-857401557 Hi, @gengliangwang . SPARK-30993 was released long time ago by 2.4.6 and 3.0.0. If you don't mind, could you use a new JIRA instead of the follow-up next

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #32824: [SPARK-30993][FOLLOWUP][SQL] Refactor LocalDateTimeUDT as YearUDT in UserDefinedTypeSuite

2021-06-08 Thread GitBox
dongjoon-hyun edited a comment on pull request #32824: URL: https://github.com/apache/spark/pull/32824#issuecomment-857401557 Hi, @gengliangwang and @cloud-fan . SPARK-30993 was released long time ago by 2.4.6 and 3.0.0. If you don't mind, could you use a new JIRA instead of the

[GitHub] [spark] dongjoon-hyun commented on pull request #32824: [SPARK-30993][FOLLOWUP][SQL] Refactor LocalDateTimeUDT as YearUDT in UserDefinedTypeSuite

2021-06-08 Thread GitBox
dongjoon-hyun commented on pull request #32824: URL: https://github.com/apache/spark/pull/32824#issuecomment-857401557 Hi, @gengliangwang . SPARK-30993 was released long time ago by 2.4.6 and 3.0.0. If you don't mind, could you use a new JIRA instead of the follow-up next time?

[GitHub] [spark] SparkQA removed a comment on pull request #32834: [SPARK-35690][SS] Stream-stream join keys should be reordered properly

2021-06-08 Thread GitBox
SparkQA removed a comment on pull request #32834: URL: https://github.com/apache/spark/pull/32834#issuecomment-857393164 **[Test build #139542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139542/testReport)** for PR 32834 at commit

[GitHub] [spark] SparkQA commented on pull request #32834: [SPARK-35690][SS] Stream-stream join keys should be reordered properly

2021-06-08 Thread GitBox
SparkQA commented on pull request #32834: URL: https://github.com/apache/spark/pull/32834#issuecomment-857396323 **[Test build #139542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139542/testReport)** for PR 32834 at commit

[GitHub] [spark] SparkQA commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-06-08 Thread GitBox
SparkQA commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-857396320 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44052/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #32693: [SPARK-35556][SQL][TESTS] Avoid log NoSuchMethodError when running multiple Hive version related tests

2021-06-08 Thread GitBox
SparkQA removed a comment on pull request #32693: URL: https://github.com/apache/spark/pull/32693#issuecomment-857347621 **[Test build #139530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139530/testReport)** for PR 32693 at commit

[GitHub] [spark] SparkQA commented on pull request #32693: [SPARK-35556][SQL][TESTS] Avoid log NoSuchMethodError when running multiple Hive version related tests

2021-06-08 Thread GitBox
SparkQA commented on pull request #32693: URL: https://github.com/apache/spark/pull/32693#issuecomment-857395651 **[Test build #139530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139530/testReport)** for PR 32693 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #32830: [SPARK-32975][K8S][FOLLOWUP] Avoid None.get exception from SPARK-32975

2021-06-08 Thread GitBox
dongjoon-hyun commented on pull request #32830: URL: https://github.com/apache/spark/pull/32830#issuecomment-857395784 cc @cchriswu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-08 Thread GitBox
SparkQA removed a comment on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-857393663 **[Test build #139543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139543/testReport)** for PR 32303 at commit

[GitHub] [spark] Ngone51 commented on a change in pull request #31102: [SPARK-34054][CORE] BlockManagerDecommissioner code cleanup

2021-06-08 Thread GitBox
Ngone51 commented on a change in pull request #31102: URL: https://github.com/apache/spark/pull/31102#discussion_r647983037 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManagerDecommissioner.scala ## @@ -66,86 +66,102 @@ private[storage] class

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-08 Thread GitBox
SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-857394986 **[Test build #139543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139543/testReport)** for PR 32303 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31102: [SPARK-34054][CORE] BlockManagerDecommissioner code cleanup

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #31102: URL: https://github.com/apache/spark/pull/31102#issuecomment-853599863 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139268/

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-08 Thread GitBox
SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-857393663 **[Test build #139543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139543/testReport)** for PR 32303 at commit

[GitHub] [spark] SparkQA commented on pull request #32834: [SPARK-35690][SS] Stream-stream join keys should be reordered properly

2021-06-08 Thread GitBox
SparkQA commented on pull request #32834: URL: https://github.com/apache/spark/pull/32834#issuecomment-857393164 **[Test build #139542 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139542/testReport)** for PR 32834 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #32830: [SPARK-32975][K8S][FOLLOWUP] Avoid None.get exception from SPARK-32975

2021-06-08 Thread GitBox
dongjoon-hyun commented on pull request #32830: URL: https://github.com/apache/spark/pull/32830#issuecomment-857392835 Thank you, @yaooqinn ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] CBribiescas edited a comment on pull request #32813: [SPARK-34591][MLLIB] Disable decision tree pruning

2021-06-08 Thread GitBox
CBribiescas edited a comment on pull request #32813: URL: https://github.com/apache/spark/pull/32813#issuecomment-857390387 To elaborate on the merge condition part, can I propose the following. Expose a `pruning_strategy` (both scala/pyspark) behavior which is as follows: - `None` -

[GitHub] [spark] ulysses-you commented on pull request #32833: [SPARK-35687][SQL][TEST] PythonUDFSuite move assume into its methods

2021-06-08 Thread GitBox
ulysses-you commented on pull request #32833: URL: https://github.com/apache/spark/pull/32833#issuecomment-857392102 thank you @HyukjinKwon , seems something wrong with test, I have seen this error many times recently. ``` [error] ## Exception when compiling 479 sources to

[GitHub] [spark] c21 commented on pull request #32834: [SPARK-35690][SS] Stream-stream join keys should be reordered properly

2021-06-08 Thread GitBox
c21 commented on pull request #32834: URL: https://github.com/apache/spark/pull/32834#issuecomment-857391723 cc @cloud-fan if you have time to take a look, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] SparkQA removed a comment on pull request #32833: [SPARK-35687][SQL][TEST] PythonUDFSuite move assume into its methods

2021-06-08 Thread GitBox
SparkQA removed a comment on pull request #32833: URL: https://github.com/apache/spark/pull/32833#issuecomment-857387124 **[Test build #139536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139536/testReport)** for PR 32833 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32833: [SPARK-35687][SQL][TEST] PythonUDFSuite move assume into its methods

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32833: URL: https://github.com/apache/spark/pull/32833#issuecomment-857391245 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139536/

[GitHub] [spark] c21 opened a new pull request #32834: [SPARK-35690][SS] Stream-stream join keys should be reordered properly

2021-06-08 Thread GitBox
c21 opened a new pull request #32834: URL: https://github.com/apache/spark/pull/32834 ### What changes were proposed in this pull request? For query with multiple stream-stream joins, unnecessary shuffle can be happened as we don't reorder join keys for stream-stream join.

[GitHub] [spark] AmplabJenkins commented on pull request #32833: [SPARK-35687][SQL][TEST] PythonUDFSuite move assume into its methods

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32833: URL: https://github.com/apache/spark/pull/32833#issuecomment-857391245 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139536/ -- This

[GitHub] [spark] SparkQA commented on pull request #32833: [SPARK-35687][SQL][TEST] PythonUDFSuite move assume into its methods

2021-06-08 Thread GitBox
SparkQA commented on pull request #32833: URL: https://github.com/apache/spark/pull/32833#issuecomment-857391213 **[Test build #139536 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139536/testReport)** for PR 32833 at commit

[GitHub] [spark] CBribiescas edited a comment on pull request #32813: [SPARK-34591][MLLIB] Disable decision tree pruning

2021-06-08 Thread GitBox
CBribiescas edited a comment on pull request #32813: URL: https://github.com/apache/spark/pull/32813#issuecomment-857390387 To elaborate on the merge condition part, can I propose the following. Expose a `pruning_strategy` (both scala/pyspark) behavior which is as follows: - `None` -

[GitHub] [spark] viirya commented on pull request #32828: [SPARK-35689][SS] Add log warn when keyWithIndexToValue returns null value

2021-06-08 Thread GitBox
viirya commented on pull request #32828: URL: https://github.com/apache/spark/pull/32828#issuecomment-857390880 As this is extended with its scope, I opened a JIRA for it now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32814: [SPARK-35664][SQL] Support java.time.LocalDateTime as an external type of TimestampWithoutTZ type

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32814: URL: https://github.com/apache/spark/pull/32814#issuecomment-857389942 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139538/

[GitHub] [spark] SparkQA removed a comment on pull request #32814: [SPARK-35664][SQL] Support java.time.LocalDateTime as an external type of TimestampWithoutTZ type

2021-06-08 Thread GitBox
SparkQA removed a comment on pull request #32814: URL: https://github.com/apache/spark/pull/32814#issuecomment-857387110 **[Test build #139538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139538/testReport)** for PR 32814 at commit

[GitHub] [spark] CBribiescas commented on pull request #32813: [SPARK-34591][MLLIB] Disable decision tree pruning

2021-06-08 Thread GitBox
CBribiescas commented on pull request #32813: URL: https://github.com/apache/spark/pull/32813#issuecomment-857390387 To elaborate on the merge condition part, can I propose the following. Expose a `pruning_strategy` behavior which is as follows: - `None` - No pruning to be done -

[GitHub] [spark] SparkQA commented on pull request #32814: [SPARK-35664][SQL] Support java.time.LocalDateTime as an external type of TimestampWithoutTZ type

2021-06-08 Thread GitBox
SparkQA commented on pull request #32814: URL: https://github.com/apache/spark/pull/32814#issuecomment-857389904 **[Test build #139538 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139538/testReport)** for PR 32814 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32814: [SPARK-35664][SQL] Support java.time.LocalDateTime as an external type of TimestampWithoutTZ type

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32814: URL: https://github.com/apache/spark/pull/32814#issuecomment-857389942 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139538/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-08 Thread GitBox
SparkQA removed a comment on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-857387520 **[Test build #139540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139540/testReport)** for PR 32303 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-857389131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-08 Thread GitBox
SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-857389118 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] AmplabJenkins commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-857389131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-06-08 Thread GitBox
SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-857389090 **[Test build #139541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139541/testReport)** for PR 32816 at commit

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-06-08 Thread GitBox
SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-857388691 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44056/ -- This is an automated message from the Apache

[GitHub] [spark] ulysses-you commented on pull request #32815: [SPARK-35675][SQL] EnsureRequirements remove shuffle should respect PartitioningCollection

2021-06-08 Thread GitBox
ulysses-you commented on pull request #32815: URL: https://github.com/apache/spark/pull/32815#issuecomment-857387581 cc @maropu @cloud-fan @yaooqinn -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-08 Thread GitBox
SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-857387520 **[Test build #139540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139540/testReport)** for PR 32303 at commit

[GitHub] [spark] SparkQA commented on pull request #32738: [SPARK-35474] Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-08 Thread GitBox
SparkQA commented on pull request #32738: URL: https://github.com/apache/spark/pull/32738#issuecomment-857387251 **[Test build #139539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139539/testReport)** for PR 32738 at commit

[GitHub] [spark] SparkQA commented on pull request #32814: [SPARK-35664][SQL] Support java.time.LocalDateTime as an external type of TimestampWithoutTZ type

2021-06-08 Thread GitBox
SparkQA commented on pull request #32814: URL: https://github.com/apache/spark/pull/32814#issuecomment-857387110 **[Test build #139538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139538/testReport)** for PR 32814 at commit

[GitHub] [spark] SparkQA commented on pull request #32828: [SPARK-35659][SS][FOLLOWUP] Add log warn when keyWithIndexToValue returns null value

2021-06-08 Thread GitBox
SparkQA commented on pull request #32828: URL: https://github.com/apache/spark/pull/32828#issuecomment-857387141 **[Test build #139537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139537/testReport)** for PR 32828 at commit

[GitHub] [spark] SparkQA commented on pull request #32833: [SPARK-35687][SQL][TEST] PythonUDFSuite move assume into its methods

2021-06-08 Thread GitBox
SparkQA commented on pull request #32833: URL: https://github.com/apache/spark/pull/32833#issuecomment-857387124 **[Test build #139536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139536/testReport)** for PR 32833 at commit

[GitHub] [spark] viirya commented on a change in pull request #32828: [SPARK-35659][SS][FOLLOWUP] Add log warn when keyWithIndexToValue returns null value

2021-06-08 Thread GitBox
viirya commented on a change in pull request #32828: URL: https://github.com/apache/spark/pull/32828#discussion_r647975834 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/SymmetricHashJoinStateManager.scala ## @@ -274,6 +274,9 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32738: [SPARK-35474] Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32738: URL: https://github.com/apache/spark/pull/32738#issuecomment-857385130 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44051/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32819: [WIP][SPARK-35677][Core][SQL] Support dynamic range of executor numbers for dynamic allocation

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32819: URL: https://github.com/apache/spark/pull/32819#issuecomment-857385135 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44044/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32831: [WIP][SPARK-35685][SQL] Prompt recreating the view when there is an incompatible schema issue

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32831: URL: https://github.com/apache/spark/pull/32831#issuecomment-857385132 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32830: [SPARK-32975][K8S][FOLLOWUP] Avoid None.get exception from SPARK-32975

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32830: URL: https://github.com/apache/spark/pull/32830#issuecomment-857385136 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44049/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32781: [SPARK-35650][SQL] Enhance `RepartitionByExpression` to make it coalesce partitions efficiently by AQE

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32781: URL: https://github.com/apache/spark/pull/32781#issuecomment-857385141 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139514/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32582: [SPARK-35436][SS] RocksDBFileManager - save checkpoint to DFS

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32582: URL: https://github.com/apache/spark/pull/32582#issuecomment-857385133 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44054/

[GitHub] [spark] AmplabJenkins commented on pull request #32781: [SPARK-35650][SQL] Enhance `RepartitionByExpression` to make it coalesce partitions efficiently by AQE

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32781: URL: https://github.com/apache/spark/pull/32781#issuecomment-857385141 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139514/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32738: [SPARK-35474] Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32738: URL: https://github.com/apache/spark/pull/32738#issuecomment-857385130 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44051/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32582: [SPARK-35436][SS] RocksDBFileManager - save checkpoint to DFS

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32582: URL: https://github.com/apache/spark/pull/32582#issuecomment-857385133 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44054/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32830: [SPARK-32975][K8S][FOLLOWUP] Avoid None.get exception from SPARK-32975

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32830: URL: https://github.com/apache/spark/pull/32830#issuecomment-857385136 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44049/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32819: [WIP][SPARK-35677][Core][SQL] Support dynamic range of executor numbers for dynamic allocation

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32819: URL: https://github.com/apache/spark/pull/32819#issuecomment-857385135 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44044/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32831: [WIP][SPARK-35685][SQL] Prompt recreating the view when there is an incompatible schema issue

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32831: URL: https://github.com/apache/spark/pull/32831#issuecomment-857385134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-08 Thread GitBox
SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-857384983 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44053/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32582: [SPARK-35436][SS] RocksDBFileManager - save checkpoint to DFS

2021-06-08 Thread GitBox
SparkQA commented on pull request #32582: URL: https://github.com/apache/spark/pull/32582#issuecomment-857383789 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44054/ --

[GitHub] [spark] gengliangwang commented on a change in pull request #32814: [SPARK-35664][SQL] Support java.time.LocalDateTime as an external type of TimestampWithoutTZ type

2021-06-08 Thread GitBox
gengliangwang commented on a change in pull request #32814: URL: https://github.com/apache/spark/pull/32814#discussion_r647973376 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala ## @@ -634,6 +634,39 @@ class

[GitHub] [spark] SparkQA commented on pull request #32738: [SPARK-35474] Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-08 Thread GitBox
SparkQA commented on pull request #32738: URL: https://github.com/apache/spark/pull/32738#issuecomment-857381187 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44051/ -- This is an automated message from the

[GitHub] [spark] MaxGekk commented on a change in pull request #32814: [SPARK-35664][SQL] Support java.time.LocalDateTime as an external type of TimestampWithoutTZ type

2021-06-08 Thread GitBox
MaxGekk commented on a change in pull request #32814: URL: https://github.com/apache/spark/pull/32814#discussion_r647972577 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala ## @@ -634,6 +634,39 @@ class DateTimeUtilsSuite

[GitHub] [spark] MaxGekk commented on a change in pull request #32814: [SPARK-35664][SQL] Support java.time.LocalDateTime as an external type of TimestampWithoutTZ type

2021-06-08 Thread GitBox
MaxGekk commented on a change in pull request #32814: URL: https://github.com/apache/spark/pull/32814#discussion_r647972335 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala ## @@ -634,6 +634,39 @@ class DateTimeUtilsSuite

[GitHub] [spark] SparkQA removed a comment on pull request #32781: [SPARK-35650][SQL] Enhance `RepartitionByExpression` to make it coalesce partitions efficiently by AQE

2021-06-08 Thread GitBox
SparkQA removed a comment on pull request #32781: URL: https://github.com/apache/spark/pull/32781#issuecomment-857281347 **[Test build #139514 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139514/testReport)** for PR 32781 at commit

[GitHub] [spark] SparkQA commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-06-08 Thread GitBox
SparkQA commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-857379517 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44052/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32781: [SPARK-35650][SQL] Enhance `RepartitionByExpression` to make it coalesce partitions efficiently by AQE

2021-06-08 Thread GitBox
SparkQA commented on pull request #32781: URL: https://github.com/apache/spark/pull/32781#issuecomment-857379516 **[Test build #139514 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139514/testReport)** for PR 32781 at commit

[GitHub] [spark] SparkQA commented on pull request #32819: [WIP][SPARK-35677][Core][SQL] Support dynamic range of executor numbers for dynamic allocation

2021-06-08 Thread GitBox
SparkQA commented on pull request #32819: URL: https://github.com/apache/spark/pull/32819#issuecomment-857379361 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44044/ -- This is an automated message from the

[GitHub] [spark] HeartSaVioR commented on a change in pull request #32828: [SPARK-35659][SS][FOLLOWUP] Add log warn when keyWithIndexToValue returns null value

2021-06-08 Thread GitBox
HeartSaVioR commented on a change in pull request #32828: URL: https://github.com/apache/spark/pull/32828#discussion_r647971086 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/SymmetricHashJoinStateManager.scala ## @@ -274,6 +274,9 @@

[GitHub] [spark] SparkQA removed a comment on pull request #32831: [WIP][SPARK-35685][SQL] Prompt recreating the view when there is an incompatible schema issue

2021-06-08 Thread GitBox
SparkQA removed a comment on pull request #32831: URL: https://github.com/apache/spark/pull/32831#issuecomment-857342321 **[Test build #139525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139525/testReport)** for PR 32831 at commit

[GitHub] [spark] SparkQA commented on pull request #32831: [WIP][SPARK-35685][SQL] Prompt recreating the view when there is an incompatible schema issue

2021-06-08 Thread GitBox
SparkQA commented on pull request #32831: URL: https://github.com/apache/spark/pull/32831#issuecomment-857378536 **[Test build #139525 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139525/testReport)** for PR 32831 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #32582: [SPARK-35436][SS] RocksDBFileManager - save checkpoint to DFS

2021-06-08 Thread GitBox
HeartSaVioR commented on pull request #32582: URL: https://github.com/apache/spark/pull/32582#issuecomment-857378260 Thanks @xuanyuanking for your contribution! I merged this to master branch. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] HeartSaVioR closed pull request #32582: [SPARK-35436][SS] RocksDBFileManager - save checkpoint to DFS

2021-06-08 Thread GitBox
HeartSaVioR closed pull request #32582: URL: https://github.com/apache/spark/pull/32582 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] HeartSaVioR commented on pull request #32582: [SPARK-35436][SS] RocksDBFileManager - save checkpoint to DFS

2021-06-08 Thread GitBox
HeartSaVioR commented on pull request #32582: URL: https://github.com/apache/spark/pull/32582#issuecomment-857377223 Thanks! Merging to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #32830: [SPARK-32975][K8S][FOLLOWUP] Avoid None.get exception from SPARK-32975

2021-06-08 Thread GitBox
SparkQA commented on pull request #32830: URL: https://github.com/apache/spark/pull/32830#issuecomment-857374448 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44049/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32831: [WIP][SPARK-35685][SQL] Prompt recreating the view when there is an incompatible schema issue

2021-06-08 Thread GitBox
SparkQA commented on pull request #32831: URL: https://github.com/apache/spark/pull/32831#issuecomment-857374281 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44050/ -- This is an automated message from the

[GitHub] [spark] ulysses-you commented on pull request #32833: [SPARK-35687][SQL][TEST] PythonUDFSuite move assume into its methods

2021-06-08 Thread GitBox
ulysses-you commented on pull request #32833: URL: https://github.com/apache/spark/pull/32833#issuecomment-857371473 cc @HyukjinKwon @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] ulysses-you opened a new pull request #32833: [SPARK-35687][SQL][TEST] PythonUDFSuite move assume into its methods

2021-06-08 Thread GitBox
ulysses-you opened a new pull request #32833: URL: https://github.com/apache/spark/pull/32833 ### What changes were proposed in this pull request? Move `assume` into methods at `PythonUDFSuite`. ### Why are the changes needed? When we run Spark test with such

[GitHub] [spark] beliefer commented on pull request #32513: [SPARK-35378][SQL] Eagerly execute commands in QueryExecution instead of caller sides

2021-06-08 Thread GitBox
beliefer commented on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-857368997 @cloud-fan Thanks for your hard work! @viirya @yaooqinn Thanks for review too. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] cloud-fan closed pull request #32513: [SPARK-35378][SQL] Eagerly execute commands in QueryExecution instead of caller sides

2021-06-08 Thread GitBox
cloud-fan closed pull request #32513: URL: https://github.com/apache/spark/pull/32513 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] cloud-fan commented on pull request #32513: [SPARK-35378][SQL] Eagerly execute commands in QueryExecution instead of caller sides

2021-06-08 Thread GitBox
cloud-fan commented on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-857368535 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA commented on pull request #32738: [SPARK-35474] Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-08 Thread GitBox
SparkQA commented on pull request #32738: URL: https://github.com/apache/spark/pull/32738#issuecomment-857366829 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44051/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-06-08 Thread GitBox
SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-857366765 **[Test build #139535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139535/testReport)** for PR 32303 at commit

[GitHub] [spark] SparkQA commented on pull request #32800: [SPARK-35661][SQL] Allow deserialized off-heap memory entry

2021-06-08 Thread GitBox
SparkQA commented on pull request #32800: URL: https://github.com/apache/spark/pull/32800#issuecomment-857366545 **[Test build #139534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139534/testReport)** for PR 32800 at commit

[GitHub] [spark] cloud-fan commented on pull request #32800: [SPARK-35661][SQL] Allow deserialized off-heap memory entry

2021-06-08 Thread GitBox
cloud-fan commented on pull request #32800: URL: https://github.com/apache/spark/pull/32800#issuecomment-857366415 > Do we have some context about why we don't support off-heap for deserialized storage before? I'm not very sure but it seems like just a missed use case. -- This is

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-857365152 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44055/

[GitHub] [spark] SparkQA commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view

2021-06-08 Thread GitBox
SparkQA commented on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-857365132 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44055/ --

[GitHub] [spark] SparkQA removed a comment on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-06-08 Thread GitBox
SparkQA removed a comment on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-857362820 **[Test build #139532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139532/testReport)** for PR 32816 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-857364900 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139531/

[GitHub] [spark] AmplabJenkins commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-857365152 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44055/ --

[GitHub] [spark] SparkQA removed a comment on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view

2021-06-08 Thread GitBox
SparkQA removed a comment on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-857362781 **[Test build #139531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139531/testReport)** for PR 32832 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-06-08 Thread GitBox
AmplabJenkins removed a comment on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-857365002 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139532/

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-06-08 Thread GitBox
SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-857364988 **[Test build #139532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139532/testReport)** for PR 32816 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-857365002 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139532/ -- This

[GitHub] [spark] SparkQA commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view

2021-06-08 Thread GitBox
SparkQA commented on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-857364878 **[Test build #139531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139531/testReport)** for PR 32832 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32832: [WIP][SPARK-35686][SQL] Not allow using auto-generated alias when creating view

2021-06-08 Thread GitBox
AmplabJenkins commented on pull request #32832: URL: https://github.com/apache/spark/pull/32832#issuecomment-857364900 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139531/ -- This

[GitHub] [spark] SparkQA commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

2021-06-08 Thread GitBox
SparkQA commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-857362972 **[Test build #139533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139533/testReport)** for PR 32473 at commit

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-06-08 Thread GitBox
SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-857362820 **[Test build #139532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139532/testReport)** for PR 32816 at commit

  1   2   3   4   5   6   7   8   9   10   >