[GitHub] [spark] SparkQA commented on pull request #30790: [SPARK-33798][SQL] Add new rule to push down the foldable expressions through CaseWhen/If

2020-12-17 Thread GitBox
SparkQA commented on pull request #30790: URL: https://github.com/apache/spark/pull/30790#issuecomment-747278272 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37542/ -

[GitHub] [spark] maropu commented on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

2020-12-17 Thread GitBox
maropu commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747279127 > BTW, @maropu . Why do you make another JIRA for this? Also, the new JIRA seems to be an Improvement? Do we need more patch to handle Q5? Yea, it's just my mistake... --

[GitHub] [spark] maropu edited a comment on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

2020-12-17 Thread GitBox
maropu edited a comment on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747279127 > BTW, @maropu . Why do you make another JIRA for this? Also, the new JIRA seems to be an Improvement? Do we need more patch to handle Q5? Yea, it's just my mistake.

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-12-17 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-747281637 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37543/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747283178 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37536/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30815: [SPARK-33817][SQL] CACHE TABLE uses a logical plan when caching a query to avoid creating a dataframe

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30815: URL: https://github.com/apache/spark/pull/30815#issuecomment-747283180 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37537/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

2020-12-17 Thread GitBox
AmplabJenkins removed a comment on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747283178 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37536/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30815: [SPARK-33817][SQL] CACHE TABLE uses a logical plan when caching a query to avoid creating a dataframe

2020-12-17 Thread GitBox
AmplabJenkins removed a comment on pull request #30815: URL: https://github.com/apache/spark/pull/30815#issuecomment-747283180 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37537/

[GitHub] [spark] SparkQA commented on pull request #30543: [SPARK-33597][SQL] Support REGEXP_LIKE for consistent with mainstream databases

2020-12-17 Thread GitBox
SparkQA commented on pull request #30543: URL: https://github.com/apache/spark/pull/30543#issuecomment-747283574 **[Test build #132932 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132932/testReport)** for PR 30543 at commit [`e1cd719`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #30543: [SPARK-33597][SQL] Support REGEXP_LIKE for consistent with mainstream databases

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30543: URL: https://github.com/apache/spark/pull/30543#issuecomment-747283906 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132932/ -

[GitHub] [spark] SparkQA commented on pull request #30820: [SPARK-33819][CORE][3.0] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

2020-12-17 Thread GitBox
SparkQA commented on pull request #30820: URL: https://github.com/apache/spark/pull/30820#issuecomment-747284569 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37544/ -

[GitHub] [spark] SparkQA commented on pull request #30821: [SPARK-33787][SQL][3.1] Add the `purge` parameter to `dropPartition()` of `SupportsPartitionManagement`

2020-12-17 Thread GitBox
SparkQA commented on pull request #30821: URL: https://github.com/apache/spark/pull/30821#issuecomment-747284686 **[Test build #132942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132942/testReport)** for PR 30821 at commit [`9c6ef11`](https://github.com

[GitHub] [spark] wangyum commented on a change in pull request #30790: [SPARK-33798][SQL] Add new rule to push down the foldable expressions through CaseWhen/If

2020-12-17 Thread GitBox
wangyum commented on a change in pull request #30790: URL: https://github.com/apache/spark/pull/30790#discussion_r544894004 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala ## @@ -528,6 +528,28 @@ object SimplifyConditionals e

[GitHub] [spark] HyukjinKwon commented on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

2020-12-17 Thread GitBox
HyukjinKwon commented on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-747287036 Let's drop this anyway since it looks going to break other users' applications very easily given the discussion above. -

[GitHub] [spark] HyukjinKwon commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

2020-12-17 Thread GitBox
HyukjinKwon commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-747287335 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [spark] HyukjinKwon closed pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

2020-12-17 Thread GitBox
HyukjinKwon closed pull request #28386: URL: https://github.com/apache/spark/pull/28386 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HyukjinKwon commented on pull request #28386: [SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in mutate

2020-12-17 Thread GitBox
HyukjinKwon commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-747288346 Thanks all for reviewing and proposing this change! This is an automated message from the Apache Git Service

[GitHub] [spark] SparkQA commented on pull request #30819: [SPARK-33819][CORE][3.1] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

2020-12-17 Thread GitBox
SparkQA commented on pull request #30819: URL: https://github.com/apache/spark/pull/30819#issuecomment-747288804 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37539/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #30819: [SPARK-33819][CORE][3.1] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30819: URL: https://github.com/apache/spark/pull/30819#issuecomment-747288832 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37539/ -

[GitHub] [spark] SparkQA commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-17 Thread GitBox
SparkQA commented on pull request #30387: URL: https://github.com/apache/spark/pull/30387#issuecomment-747289339 **[Test build #132920 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132920/testReport)** for PR 30387 at commit [`f3aad97`](https://github.co

[GitHub] [spark] cloud-fan commented on pull request #30776: [SPARK-33787][SQL] Add the `purge` parameter to `dropPartition()` of `SupportsPartitionManagement`

2020-12-17 Thread GitBox
cloud-fan commented on pull request #30776: URL: https://github.com/apache/spark/pull/30776#issuecomment-747289280 also cc @stczwd This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [spark] SparkQA commented on pull request #30772: [SPARK-33733][SQL][2.4] PullOutNondeterministic should check and collect deterministic field

2020-12-17 Thread GitBox
SparkQA commented on pull request #30772: URL: https://github.com/apache/spark/pull/30772#issuecomment-747289670 **[Test build #132931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132931/testReport)** for PR 30772 at commit [`cf4a7bb`](https://github.co

[GitHub] [spark] HeartSaVioR commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

2020-12-17 Thread GitBox
HeartSaVioR commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-747289740 My preference is unloading state as soon as possible if it's not being used. So if this is achievable I don't think we need to investigate alternatives. The difference

[GitHub] [spark] AmplabJenkins commented on pull request #30772: [SPARK-33733][SQL][2.4] PullOutNondeterministic should check and collect deterministic field

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30772: URL: https://github.com/apache/spark/pull/30772#issuecomment-747290256 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132931/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30387: URL: https://github.com/apache/spark/pull/30387#issuecomment-747290360 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132920/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747290573 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37540/ -

[GitHub] [spark] SparkQA commented on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

2020-12-17 Thread GitBox
SparkQA commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747290544 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37540/ ---

[GitHub] [spark] SparkQA commented on pull request #30790: [SPARK-33798][SQL] Add new rule to push down the foldable expressions through CaseWhen/If

2020-12-17 Thread GitBox
SparkQA commented on pull request #30790: URL: https://github.com/apache/spark/pull/30790#issuecomment-747291186 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37542/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #30790: [SPARK-33798][SQL] Add new rule to push down the foldable expressions through CaseWhen/If

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30790: URL: https://github.com/apache/spark/pull/30790#issuecomment-747291216 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37542/ -

[GitHub] [spark] ulysses-you commented on pull request #30771: [SPARK-33733][SQL][3.0] PullOutNondeterministic should check and collect deterministic field

2020-12-17 Thread GitBox
ulysses-you commented on pull request #30771: URL: https://github.com/apache/spark/pull/30771#issuecomment-747293026 cc @maropu @cloud-fan This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] ulysses-you commented on pull request #30772: [SPARK-33733][SQL][2.4] PullOutNondeterministic should check and collect deterministic field

2020-12-17 Thread GitBox
ulysses-you commented on pull request #30772: URL: https://github.com/apache/spark/pull/30772#issuecomment-747293070 cc @maropu @cloud-fan This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #30820: [SPARK-33819][CORE][3.0] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

2020-12-17 Thread GitBox
SparkQA commented on pull request #30820: URL: https://github.com/apache/spark/pull/30820#issuecomment-747294036 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37544/ ---

[GitHub] [spark] ulysses-you commented on pull request #30772: [SPARK-33733][SQL][2.4] PullOutNondeterministic should check and collect deterministic field

2020-12-17 Thread GitBox
ulysses-you commented on pull request #30772: URL: https://github.com/apache/spark/pull/30772#issuecomment-747294097 Seems the jenkins build env is different if the machine is different ? The last passed test also skip the pandas test ``` test_coerce (pyspark.sql.tests.GroupedMapPanda

[GitHub] [spark] SparkQA removed a comment on pull request #30543: [SPARK-33597][SQL] Support REGEXP_LIKE for consistent with mainstream databases

2020-12-17 Thread GitBox
SparkQA removed a comment on pull request #30543: URL: https://github.com/apache/spark/pull/30543#issuecomment-747210304 **[Test build #132932 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132932/testReport)** for PR 30543 at commit [`e1cd719`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #30772: [SPARK-33733][SQL][2.4] PullOutNondeterministic should check and collect deterministic field

2020-12-17 Thread GitBox
SparkQA removed a comment on pull request #30772: URL: https://github.com/apache/spark/pull/30772#issuecomment-747210119 **[Test build #132931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132931/testReport)** for PR 30772 at commit [`cf4a7bb`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-17 Thread GitBox
SparkQA removed a comment on pull request #30387: URL: https://github.com/apache/spark/pull/30387#issuecomment-747180679 **[Test build #132920 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132920/testReport)** for PR 30387 at commit [`f3aad97`](https://gi

[GitHub] [spark] manbuyun commented on pull request #30572: [SPARK-33628][SQL] Use the Hive.getPartitionsByNames method instead of Hive.getPartitions in the HiveClientImpl

2020-12-17 Thread GitBox
manbuyun commented on pull request #30572: URL: https://github.com/apache/spark/pull/30572#issuecomment-747297343 @wangyum Can you help me verify this patch? This is an automated message from the Apache Git Service. To respon

[GitHub] [spark] manbuyun commented on pull request #30225: [SPARK-33187][SQL] Add a check on the number of returned partitions in the HiveShim#getPartitionsByFilter method

2020-12-17 Thread GitBox
manbuyun commented on pull request #30225: URL: https://github.com/apache/spark/pull/30225#issuecomment-747297765 @sunchao @wangyum Can you help me verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-12-17 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-747298040 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37543/ ---

[GitHub] [spark] cloud-fan commented on pull request #30772: [SPARK-33733][SQL][2.4] PullOutNondeterministic should check and collect deterministic field

2020-12-17 Thread GitBox
cloud-fan commented on pull request #30772: URL: https://github.com/apache/spark/pull/30772#issuecomment-747298086 thanks, merging to 2.4! This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] aokolnychyi commented on a change in pull request #30808: [WIP] [SPARK-33784][SQL] Rename dataSourceRewriteRules batch

2020-12-17 Thread GitBox
aokolnychyi commented on a change in pull request #30808: URL: https://github.com/apache/spark/pull/30808#discussion_r544908719 ## File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensions.scala ## @@ -200,19 +200,20 @@ class SparkSessionExtensions {

[GitHub] [spark] cloud-fan closed pull request #30772: [SPARK-33733][SQL][2.4] PullOutNondeterministic should check and collect deterministic field

2020-12-17 Thread GitBox
cloud-fan closed pull request #30772: URL: https://github.com/apache/spark/pull/30772 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] cloud-fan commented on pull request #30771: [SPARK-33733][SQL][3.0] PullOutNondeterministic should check and collect deterministic field

2020-12-17 Thread GitBox
cloud-fan commented on pull request #30771: URL: https://github.com/apache/spark/pull/30771#issuecomment-747298550 thanks, merging to 3.0! This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] cloud-fan closed pull request #30771: [SPARK-33733][SQL][3.0] PullOutNondeterministic should check and collect deterministic field

2020-12-17 Thread GitBox
cloud-fan closed pull request #30771: URL: https://github.com/apache/spark/pull/30771 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] aokolnychyi commented on a change in pull request #30808: [WIP] [SPARK-33784][SQL] Rename dataSourceRewriteRules batch

2020-12-17 Thread GitBox
aokolnychyi commented on a change in pull request #30808: URL: https://github.com/apache/spark/pull/30808#discussion_r544908719 ## File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensions.scala ## @@ -200,19 +200,20 @@ class SparkSessionExtensions {

[GitHub] [spark] SparkQA commented on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

2020-12-17 Thread GitBox
SparkQA commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747304418 **[Test build #132937 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132937/testReport)** for PR 30818 at commit [`7e33df3`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

2020-12-17 Thread GitBox
SparkQA removed a comment on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747259797 **[Test build #132937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132937/testReport)** for PR 30818 at commit [`7e33df3`](https://gi

[GitHub] [spark] aokolnychyi commented on a change in pull request #30808: [WIP] [SPARK-33784][SQL] Rename dataSourceRewriteRules batch

2020-12-17 Thread GitBox
aokolnychyi commented on a change in pull request #30808: URL: https://github.com/apache/spark/pull/30808#discussion_r544917345 ## File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensions.scala ## @@ -200,19 +200,20 @@ class SparkSessionExtensions {

[GitHub] [spark] SparkQA commented on pull request #30821: [SPARK-33787][SQL][3.1] Add the `purge` parameter to `dropPartition()` of `SupportsPartitionManagement`

2020-12-17 Thread GitBox
SparkQA commented on pull request #30821: URL: https://github.com/apache/spark/pull/30821#issuecomment-747307923 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37545/ -

[GitHub] [spark] HyukjinKwon commented on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

2020-12-17 Thread GitBox
HyukjinKwon commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747308314 retest this please This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [spark] SparkQA commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-17 Thread GitBox
SparkQA commented on pull request #30387: URL: https://github.com/apache/spark/pull/30387#issuecomment-747308839 **[Test build #132927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132927/testReport)** for PR 30387 at commit [`0d30d13`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-17 Thread GitBox
SparkQA removed a comment on pull request #30387: URL: https://github.com/apache/spark/pull/30387#issuecomment-747196830 **[Test build #132927 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132927/testReport)** for PR 30387 at commit [`0d30d13`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30815: [SPARK-33817][SQL] CACHE TABLE uses a logical plan when caching a query to avoid creating a dataframe

2020-12-17 Thread GitBox
SparkQA commented on pull request #30815: URL: https://github.com/apache/spark/pull/30815#issuecomment-747309494 **[Test build #132921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132921/testReport)** for PR 30815 at commit [`d9dcfb3`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30815: [SPARK-33817][SQL] CACHE TABLE uses a logical plan when caching a query to avoid creating a dataframe

2020-12-17 Thread GitBox
SparkQA removed a comment on pull request #30815: URL: https://github.com/apache/spark/pull/30815#issuecomment-747193234 **[Test build #132921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132921/testReport)** for PR 30815 at commit [`d9dcfb3`](https://gi

[GitHub] [spark] beliefer commented on pull request #30543: [SPARK-33597][SQL] Support REGEXP_LIKE for consistent with mainstream databases

2020-12-17 Thread GitBox
beliefer commented on pull request #30543: URL: https://github.com/apache/spark/pull/30543#issuecomment-747311692 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins commented on pull request #30820: [SPARK-33819][CORE][3.0] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30820: URL: https://github.com/apache/spark/pull/30820#issuecomment-747312213 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37544/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747312218 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132937/ -

[GitHub] [spark] AmplabJenkins commented on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-747312214 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37543/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30816: [SPARK-33818][DOC] Doc `spark.sql.parser.quotedRegexColumnNames`

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30816: URL: https://github.com/apache/spark/pull/30816#issuecomment-747312212 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37541/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30815: [SPARK-33817][SQL] CACHE TABLE uses a logical plan when caching a query to avoid creating a dataframe

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30815: URL: https://github.com/apache/spark/pull/30815#issuecomment-747312211 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132921/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30387: URL: https://github.com/apache/spark/pull/30387#issuecomment-747312209 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132927/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30820: [SPARK-33819][CORE][3.0] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

2020-12-17 Thread GitBox
AmplabJenkins removed a comment on pull request #30820: URL: https://github.com/apache/spark/pull/30820#issuecomment-747312213 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37544/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30815: [SPARK-33817][SQL] CACHE TABLE uses a logical plan when caching a query to avoid creating a dataframe

2020-12-17 Thread GitBox
AmplabJenkins removed a comment on pull request #30815: URL: https://github.com/apache/spark/pull/30815#issuecomment-747312211 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132921/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30387: [SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT NULLS ]

2020-12-17 Thread GitBox
AmplabJenkins removed a comment on pull request #30387: URL: https://github.com/apache/spark/pull/30387#issuecomment-747209940 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-12-17 Thread GitBox
AmplabJenkins removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-747178057 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30816: [SPARK-33818][DOC] Doc `spark.sql.parser.quotedRegexColumnNames`

2020-12-17 Thread GitBox
AmplabJenkins removed a comment on pull request #30816: URL: https://github.com/apache/spark/pull/30816#issuecomment-747264601 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

2020-12-17 Thread GitBox
AmplabJenkins removed a comment on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747290573 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30818: [SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin

2020-12-17 Thread GitBox
SparkQA commented on pull request #30818: URL: https://github.com/apache/spark/pull/30818#issuecomment-747314093 **[Test build #132943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132943/testReport)** for PR 30818 at commit [`7e33df3`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #30809: [SPARK-33812][SQL] Split the histogram column stats when saving to hive metastore as table property

2020-12-17 Thread GitBox
cloud-fan commented on a change in pull request #30809: URL: https://github.com/apache/spark/pull/30809#discussion_r544929116 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -3075,7 +3085,9 @@ object SQLConf { RemovedConfig("s

[GitHub] [spark] SparkQA commented on pull request #30543: [SPARK-33597][SQL] Support REGEXP_LIKE for consistent with mainstream databases

2020-12-17 Thread GitBox
SparkQA commented on pull request #30543: URL: https://github.com/apache/spark/pull/30543#issuecomment-74731 **[Test build #132946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132946/testReport)** for PR 30543 at commit [`e1cd719`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30543: [SPARK-33597][SQL] Support REGEXP_LIKE for consistent with mainstream databases

2020-12-17 Thread GitBox
AmplabJenkins removed a comment on pull request #30543: URL: https://github.com/apache/spark/pull/30543#issuecomment-747283906 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132932/ -

[GitHub] [spark] SparkQA commented on pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
SparkQA commented on pull request #30784: URL: https://github.com/apache/spark/pull/30784#issuecomment-747314973 **[Test build #132945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132945/testReport)** for PR 30784 at commit [`43b5d13`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30809: [SPARK-33812][SQL] Split the histogram column stats when saving to hive metastore as table property

2020-12-17 Thread GitBox
SparkQA commented on pull request #30809: URL: https://github.com/apache/spark/pull/30809#issuecomment-747315187 **[Test build #132944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132944/testReport)** for PR 30809 at commit [`0722f60`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
SparkQA commented on pull request #30784: URL: https://github.com/apache/spark/pull/30784#issuecomment-747316656 **[Test build #132948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132948/testReport)** for PR 30784 at commit [`e7b9e18`](https://github.com

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544932654 ## File path: pom.xml ## @@ -2508,6 +2512,9 @@ -feature -explaintypes -target:jvm-1.8 + ${sc

[GitHub] [spark] SparkQA commented on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

2020-12-17 Thread GitBox
SparkQA commented on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747317160 **[Test build #132933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132933/testReport)** for PR 30817 at commit [`f50a5f5`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

2020-12-17 Thread GitBox
SparkQA removed a comment on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747232716 **[Test build #132933 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132933/testReport)** for PR 30817 at commit [`f50a5f5`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30809: [SPARK-33812][SQL] Split the histogram column stats when saving to hive metastore as table property

2020-12-17 Thread GitBox
SparkQA commented on pull request #30809: URL: https://github.com/apache/spark/pull/30809#issuecomment-747317497 **[Test build #132947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132947/testReport)** for PR 30809 at commit [`4e8e984`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

2020-12-17 Thread GitBox
AmplabJenkins commented on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747318092 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132933/ -

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544932654 ## File path: pom.xml ## @@ -2508,6 +2512,9 @@ -feature -explaintypes -target:jvm-1.8 + ${sc

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30817: [SPARK-33821][BUILD] Upgrade SBT to 1.4.5

2020-12-17 Thread GitBox
AmplabJenkins removed a comment on pull request #30817: URL: https://github.com/apache/spark/pull/30817#issuecomment-747318092 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132933/ -

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544932654 ## File path: pom.xml ## @@ -2508,6 +2512,9 @@ -feature -explaintypes -target:jvm-1.8 + ${sc

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544934685 ## File path: pom.xml ## @@ -2508,6 +2512,9 @@ -feature -explaintypes -target:jvm-1.8 + ${sc

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544935411 ## File path: pom.xml ## @@ -2508,6 +2512,9 @@ -feature -explaintypes -target:jvm-1.8 + ${sc

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544935411 ## File path: pom.xml ## @@ -2508,6 +2512,9 @@ -feature -explaintypes -target:jvm-1.8 + ${sc

[GitHub] [spark] SparkQA commented on pull request #30821: [SPARK-33787][SQL][3.1] Add the `purge` parameter to `dropPartition()` of `SupportsPartitionManagement`

2020-12-17 Thread GitBox
SparkQA commented on pull request #30821: URL: https://github.com/apache/spark/pull/30821#issuecomment-747325332 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37545/ ---

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544943855 ## File path: pom.xml ## @@ -2521,6 +2528,13 @@ ${java.version} -Xlint:all,-serial,-path,-try +

[GitHub] [spark] viirya commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

2020-12-17 Thread GitBox
viirya commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-747325973 I see. It is much clearer now. This is already over the original scope of this PR. The purpose of this change is pretty simple: making store unloading more consistent. The

[GitHub] [spark] viirya closed pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

2020-12-17 Thread GitBox
viirya closed pull request #30770: URL: https://github.com/apache/spark/pull/30770 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544944855 ## File path: pom.xml ## @@ -3233,6 +3247,10 @@ 2.13.4 2.13 +1.7.1 + + -P:silencer:globalFilters=.*f

[GitHub] [spark] viirya edited a comment on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

2020-12-17 Thread GitBox
viirya edited a comment on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-747325973 I see. It is much clearer now. This is already over the original scope of this PR. The purpose of this change is pretty simple: making store unloading more consisten

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544945803 ## File path: pom.xml ## @@ -3233,6 +3247,10 @@ 2.13.4 2.13 +1.7.1 + + -P:silencer:globalFilters=.*f

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544946239 ## File path: pom.xml ## @@ -3243,6 +3261,31 @@ + + + + + net.alchim31.

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544934685 ## File path: pom.xml ## @@ -2508,6 +2512,9 @@ -feature -explaintypes -target:jvm-1.8 + ${sc

[GitHub] [spark] SparkQA commented on pull request #30819: [SPARK-33819][CORE][3.1] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

2020-12-17 Thread GitBox
SparkQA commented on pull request #30819: URL: https://github.com/apache/spark/pull/30819#issuecomment-747329545 **[Test build #132936 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132936/testReport)** for PR 30819 at commit [`132d50a`](https://github.co

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544949522 ## File path: pom.xml ## @@ -3220,9 +3217,33 @@ scala-2.12 + Review comment: Address 4fe256 changed a way to achieve the

[GitHub] [spark] SparkQA removed a comment on pull request #30819: [SPARK-33819][CORE][3.1] SingleFileEventLogFileReader/RollingEventLogFilesFileReader should be `package private`

2020-12-17 Thread GitBox
SparkQA removed a comment on pull request #30819: URL: https://github.com/apache/spark/pull/30819#issuecomment-747259209 **[Test build #132936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132936/testReport)** for PR 30819 at commit [`132d50a`](https://gi

[GitHub] [spark] SparkQA commented on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-12-17 Thread GitBox
SparkQA commented on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-747332847 **[Test build #132940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132940/testReport)** for PR 29695 at commit [`710ff80`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29695: [SPARK-22390][SPARK-32833][SQL] [WIP]JDBC V2 Datasource aggregate push down

2020-12-17 Thread GitBox
SparkQA removed a comment on pull request #29695: URL: https://github.com/apache/spark/pull/29695#issuecomment-747260714 **[Test build #132940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132940/testReport)** for PR 29695 at commit [`710ff80`](https://gi

[GitHub] [spark] LuciferYang commented on a change in pull request #30784: [SPARK-33560][test-maven][BUILD] Add "unused-import" check to Maven compilation process

2020-12-17 Thread GitBox
LuciferYang commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r544955190 ## File path: pom.xml ## @@ -2521,6 +2528,13 @@ ${java.version} -Xlint:all,-serial,-path,-try +

  1   2   3   4   5   6   7   8   >