[GitHub] [spark] SparkQA commented on pull request #28362: [SPARK-31570][R][DOCS] R combine gapply dapply

2020-04-27 Thread GitBox
SparkQA commented on pull request #28362: URL: https://github.com/apache/spark/pull/28362#issuecomment-619819692 **[Test build #121892 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121892/testReport)** for PR 28362 at commit [`6ad2085`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #28366: [WIP][SPARK-31365][SQL] Enable nested predicate pushdown per data sources

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-619820342 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28362: [SPARK-31570][R][DOCS] R combine gapply dapply

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28362: URL: https://github.com/apache/spark/pull/28362#issuecomment-619820262 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28362: [SPARK-31570][R][DOCS] R combine gapply dapply

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28362: URL: https://github.com/apache/spark/pull/28362#issuecomment-619820262 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28366: [WIP][SPARK-31365][SQL] Enable nested predicate pushdown per data sources

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28366: URL: https://github.com/apache/spark/pull/28366#issuecomment-619820342 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] MichaelChirico opened a new pull request #28367: [SPARK-31573][R] apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
MichaelChirico opened a new pull request #28367: URL: https://github.com/apache/spark/pull/28367 ### What changes were proposed in this pull request? For regex functions in base R (`gsub`, `grep`, `grepl`, `strsplit`, `gregexpr`), supplying the `fixed=TRUE` option will be

[GitHub] [spark] AmplabJenkins commented on pull request #28361: [SPARK-31572][SQL][CORE] Improve task logs at executor side

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28361: URL: https://github.com/apache/spark/pull/28361#issuecomment-619823023 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28361: [SPARK-31572][SQL][CORE] Improve task logs at executor side

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28361: URL: https://github.com/apache/spark/pull/28361#issuecomment-619823023 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28361: [SPARK-31572][SQL][CORE] Improve task logs at executor side

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28361: URL: https://github.com/apache/spark/pull/28361#issuecomment-619816052 **[Test build #121890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121890/testReport)** for PR 28361 at commit [`5099431`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28367: [SPARK-31573][R] apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619823119 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] SparkQA commented on pull request #28361: [SPARK-31572][SQL][CORE] Improve task logs at executor side

2020-04-27 Thread GitBox
SparkQA commented on pull request #28361: URL: https://github.com/apache/spark/pull/28361#issuecomment-619822977 **[Test build #121890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121890/testReport)** for PR 28361 at commit [`5099431`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28361: [SPARK-31572][SQL][CORE] Improve task logs at executor side

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28361: URL: https://github.com/apache/spark/pull/28361#issuecomment-619823033 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28367: [SPARK-31573][R] apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619823119 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #28367: [SPARK-31573][R] apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619823651 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] HeartSaVioR commented on pull request #28326: [SPARK-27340][SS] Alias on TimeWindow expression cause watermark metadata lost

2020-04-27 Thread GitBox
HeartSaVioR commented on pull request #28326: URL: https://github.com/apache/spark/pull/28326#issuecomment-619831846 > Each Attribute/Alias has its own metadata and can easily be hidden by the outer-most Alias. Yeah I see the concern - I'm not sure the column metadata was considered

[GitHub] [spark] HeartSaVioR edited a comment on pull request #28326: [SPARK-27340][SS] Alias on TimeWindow expression cause watermark metadata lost

2020-04-27 Thread GitBox
HeartSaVioR edited a comment on pull request #28326: URL: https://github.com/apache/spark/pull/28326#issuecomment-619831846 > Each Attribute/Alias has its own metadata and can easily be hidden by the outer-most Alias. Yeah I see the concern - I'm not sure the column metadata was cons

[GitHub] [spark] maropu commented on pull request #28322: [SPARK-31550][SQL][DOCS] Set nondeterministic configurations with general meanings in sql configuration doc

2020-04-27 Thread GitBox
maropu commented on pull request #28322: URL: https://github.com/apache/spark/pull/28322#issuecomment-619835884 late LGTM, thanks for the update, @yaooqinn This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] maropu commented on a change in pull request #28215: [SPARK-31272][SQL] Support DB2 Kerberos login in JDBC connector

2020-04-27 Thread GitBox
maropu commented on a change in pull request #28215: URL: https://github.com/apache/spark/pull/28215#discussion_r415642664 ## File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DB2KrbIntegrationSuite.scala ## @@ -0,0 +1,89 @@ +/* + * Licensed

[GitHub] [spark] cloud-fan commented on a change in pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
cloud-fan commented on a change in pull request #28109: URL: https://github.com/apache/spark/pull/28109#discussion_r415645175 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CustomShuffleReaderExec.scala ## @@ -121,12 +122,22 @@ case class CustomS

[GitHub] [spark] SparkQA commented on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
SparkQA commented on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619846231 **[Test build #121893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121893/testReport)** for PR 28109 at commit [`2a26813`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619847107 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #28215: [SPARK-31272][SQL] Support DB2 Kerberos login in JDBC connector

2020-04-27 Thread GitBox
gaborgsomogyi commented on a change in pull request #28215: URL: https://github.com/apache/spark/pull/28215#discussion_r415647618 ## File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/jdbc/DB2KrbIntegrationSuite.scala ## @@ -0,0 +1,89 @@ +/* + * L

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619847107 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28362: [SPARK-31570][R][DOCS] R combine gapply dapply

2020-04-27 Thread GitBox
SparkQA commented on pull request #28362: URL: https://github.com/apache/spark/pull/28362#issuecomment-619848483 **[Test build #121892 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121892/testReport)** for PR 28362 at commit [`6ad2085`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28362: [SPARK-31570][R][DOCS] R combine gapply dapply

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28362: URL: https://github.com/apache/spark/pull/28362#issuecomment-619848709 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28362: [SPARK-31570][R][DOCS] R combine gapply dapply

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28362: URL: https://github.com/apache/spark/pull/28362#issuecomment-619819692 **[Test build #121892 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121892/testReport)** for PR 28362 at commit [`6ad2085`](https://gi

[GitHub] [spark] HyukjinKwon commented on pull request #28367: [SPARK-31573][R] apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
HyukjinKwon commented on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619849863 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28362: [SPARK-31570][R][DOCS] R combine gapply dapply

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28362: URL: https://github.com/apache/spark/pull/28362#issuecomment-619848709 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28367: [SPARK-31573][R] apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619823651 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28367: [SPARK-31573][R] apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28367: URL: https://github.com/apache/spark/pull/28367#discussion_r415651442 ## File path: R/pkg/R/sparkR.R ## @@ -606,7 +606,7 @@ getClientModeSparkSubmitOpts <- function(submitOps, sparkEnvirMap) { # process only if --o

[GitHub] [spark] SparkQA commented on pull request #28367: [SPARK-31573][R] apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
SparkQA commented on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619851054 **[Test build #121894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121894/testReport)** for PR 28367 at commit [`8676253`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
SparkQA commented on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619851103 **[Test build #121895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121895/testReport)** for PR 28109 at commit [`2e512b4`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28367: [SPARK-31573][R] apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619851725 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619851779 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619851779 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28367: [SPARK-31573][R] apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619851725 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
SparkQA commented on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619852234 **[Test build #121895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121895/testReport)** for PR 28109 at commit [`2e512b4`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619852256 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619851103 **[Test build #121895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121895/testReport)** for PR 28109 at commit [`2e512b4`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619852256 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28294: [SPARK-31519][SQL] Cast in having aggregate expressions returns the wrong result

2020-04-27 Thread GitBox
SparkQA commented on pull request #28294: URL: https://github.com/apache/spark/pull/28294#issuecomment-619853417 **[Test build #121884 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121884/testReport)** for PR 28294 at commit [`d4ac6d7`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619852268 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121

[GitHub] [spark] AmplabJenkins commented on pull request #28294: [SPARK-31519][SQL] Cast in having aggregate expressions returns the wrong result

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28294: URL: https://github.com/apache/spark/pull/28294#issuecomment-619853722 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28294: [SPARK-31519][SQL] Cast in having aggregate expressions returns the wrong result

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28294: URL: https://github.com/apache/spark/pull/28294#issuecomment-619783728 **[Test build #121884 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121884/testReport)** for PR 28294 at commit [`d4ac6d7`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28294: [SPARK-31519][SQL] Cast in having aggregate expressions returns the wrong result

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28294: URL: https://github.com/apache/spark/pull/28294#issuecomment-619853722 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28294: [SPARK-31519][SQL] Cast in having aggregate expressions returns the wrong result

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28294: URL: https://github.com/apache/spark/pull/28294#issuecomment-619853727 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415655632 ## File path: R/pkg/R/sparkR.R ## @@ -439,8 +439,11 @@ sparkR.session <- function( rPackageVersion <- paste0(packageVersion("SparkR")) if (jvm

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415655342 ## File path: R/pkg/R/utils.R ## @@ -354,8 +354,10 @@ varargsToStrEnv <- function(...) { } else { value <- pairs[[name]] if (

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415655186 ## File path: R/pkg/R/utils.R ## @@ -369,8 +371,9 @@ varargsToStrEnv <- function(...) { } if (length(ignoredNames) != 0) { -warning(paste0

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415656778 ## File path: R/pkg/R/install.R ## @@ -293,7 +289,7 @@ sparkCachePath <- function() { Sys.getenv("XDG_CACHE_HOME", file.path(Sys.getenv("HOME

[GitHub] [spark] SparkQA commented on pull request #28349: [SPARK-30642][ML][PYSPARK] LinearSVC blockify input vectors

2020-04-27 Thread GitBox
SparkQA commented on pull request #28349: URL: https://github.com/apache/spark/pull/28349#issuecomment-619855561 **[Test build #121896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121896/testReport)** for PR 28349 at commit [`9275258`](https://github.com

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415657792 ## File path: R/pkg/R/install.R ## @@ -231,24 +230,21 @@ getPreferredMirror <- function(version, packageName) { directDownloadTar <- function(mirro

[GitHub] [spark] AmplabJenkins commented on pull request #28349: [SPARK-30642][ML][PYSPARK] LinearSVC blockify input vectors

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28349: URL: https://github.com/apache/spark/pull/28349#issuecomment-619856382 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415658286 ## File path: R/pkg/R/install.R ## @@ -103,12 +103,14 @@ install.spark <- function(hadoopVersion = "2.7", mirrorUrl = NULL, # can use dir.exists(p

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415657976 ## File path: R/pkg/R/install.R ## @@ -201,11 +200,11 @@ robustDownloadTar <- function(mirrorUrl, version, hadoopVersion, packageName, pa # remo

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28349: [SPARK-30642][ML][PYSPARK] LinearSVC blockify input vectors

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28349: URL: https://github.com/apache/spark/pull/28349#issuecomment-619856382 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415659034 ## File path: R/pkg/R/client.R ## @@ -102,10 +102,17 @@ checkJavaVersion <- function() { javaVersionNum <- as.integer(versions[1]) } if (ja

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415659247 ## File path: R/pkg/R/SQLContext.R ## @@ -207,7 +209,8 @@ getSchema <- function(schema, firstRow = NULL, rdd = NULL) { names <- lapply(names, fun

[GitHub] [spark] HyukjinKwon commented on pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619858697 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415660015 ## File path: R/pkg/R/DataFrame.R ## @@ -2587,18 +2589,18 @@ setMethod("join", if (is.null(joinType)) { sdf <- callJMe

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415660279 ## File path: R/pkg/R/DataFrame.R ## @@ -829,8 +829,11 @@ setMethod("repartitionByRange", jcol <- lapply(cols, function(c) { c@jc })

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619816305 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
SparkQA commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-61985 **[Test build #121897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121897/testReport)** for PR 28365 at commit [`1940eb8`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619860511 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28365: [SPARK-31571][R] overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619860511 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] HyukjinKwon commented on a change in pull request #28172: [SPARK-31402][SQL] Fix rebasing of BCE dates/timestamps

2020-04-27 Thread GitBox
HyukjinKwon commented on a change in pull request #28172: URL: https://github.com/apache/spark/pull/28172#discussion_r415664502 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala ## @@ -66,15 +66,53 @@ object RebaseDateTime { /

[GitHub] [spark] SparkQA commented on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
SparkQA commented on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619863720 **[Test build #121898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121898/testReport)** for PR 28109 at commit [`8eec016`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619864473 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28109: [SPARK-31524][SQL][followup] Add metric to the split task number for skew optimization

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28109: URL: https://github.com/apache/spark/pull/28109#issuecomment-619864473 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] MaxGekk commented on a change in pull request #28328: [SPARK-31553][SQL] Fix isInCollection for collection sizes above the optimisation threshold

2020-04-27 Thread GitBox
MaxGekk commented on a change in pull request #28328: URL: https://github.com/apache/spark/pull/28328#discussion_r415676504 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -828,11 +828,12 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] MichaelChirico commented on a change in pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
MichaelChirico commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415680667 ## File path: R/pkg/R/install.R ## @@ -293,7 +289,7 @@ sparkCachePath <- function() { Sys.getenv("XDG_CACHE_HOME", file.path(Sys.getenv("H

[GitHub] [spark] SparkQA removed a comment on pull request #28367: [SPARK-31573][R] Apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619851054 **[Test build #121894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121894/testReport)** for PR 28367 at commit [`8676253`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28367: [SPARK-31573][R] Apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619876485 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28367: [SPARK-31573][R] Apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
SparkQA commented on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619876238 **[Test build #121894 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121894/testReport)** for PR 28367 at commit [`8676253`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28367: [SPARK-31573][R] Apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619876485 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28367: [SPARK-31573][R] Apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28367: URL: https://github.com/apache/spark/pull/28367#issuecomment-619876494 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121

[GitHub] [spark] MichaelChirico commented on a change in pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
MichaelChirico commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415683919 ## File path: R/pkg/R/install.R ## @@ -293,7 +289,7 @@ sparkCachePath <- function() { Sys.getenv("XDG_CACHE_HOME", file.path(Sys.getenv("H

[GitHub] [spark] MichaelChirico commented on a change in pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
MichaelChirico commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415687299 ## File path: R/pkg/R/install.R ## @@ -293,7 +289,7 @@ sparkCachePath <- function() { Sys.getenv("XDG_CACHE_HOME", file.path(Sys.getenv("H

[GitHub] [spark] MichaelChirico commented on a change in pull request #28367: [SPARK-31573][R] Apply fixed=TRUE as appropriate to regex usage in R

2020-04-27 Thread GitBox
MichaelChirico commented on a change in pull request #28367: URL: https://github.com/apache/spark/pull/28367#discussion_r415688361 ## File path: R/pkg/R/sparkR.R ## @@ -606,7 +606,7 @@ getClientModeSparkSubmitOpts <- function(submitOps, sparkEnvirMap) { # process only if

[GitHub] [spark] AmplabJenkins commented on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619885492 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
SparkQA commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619885451 **[Test build #121897 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121897/testReport)** for PR 28365 at commit [`1940eb8`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-61985 **[Test build #121897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121897/testReport)** for PR 28365 at commit [`1940eb8`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619885492 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619885497 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121

[GitHub] [spark] MaxGekk commented on a change in pull request #28328: [SPARK-31553][SQL] Fix isInCollection for collection sizes above the optimisation threshold

2020-04-27 Thread GitBox
MaxGekk commented on a change in pull request #28328: URL: https://github.com/apache/spark/pull/28328#discussion_r415701513 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -828,11 +828,12 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28328: [SPARK-31553][SQL] Fix isInCollection for collection sizes above the optimisation threshold

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28328: URL: https://github.com/apache/spark/pull/28328#issuecomment-619894210 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28328: [SPARK-31553][SQL] Fix isInCollection for collection sizes above the optimisation threshold

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28328: URL: https://github.com/apache/spark/pull/28328#issuecomment-619894210 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] MichaelChirico commented on a change in pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
MichaelChirico commented on a change in pull request #28365: URL: https://github.com/apache/spark/pull/28365#discussion_r415704429 ## File path: R/pkg/R/DataFrame.R ## @@ -2587,18 +2589,18 @@ setMethod("join", if (is.null(joinType)) { sdf <- call

[GitHub] [spark] SparkQA removed a comment on pull request #28356: [SPARK-31485][CORE][3.0] Avoid application hang if only partial barrier tasks launched

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28356: URL: https://github.com/apache/spark/pull/28356#issuecomment-619779844 **[Test build #121882 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121882/testReport)** for PR 28356 at commit [`0f9a186`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28356: [SPARK-31485][CORE][3.0] Avoid application hang if only partial barrier tasks launched

2020-04-27 Thread GitBox
SparkQA commented on pull request #28356: URL: https://github.com/apache/spark/pull/28356#issuecomment-619896941 **[Test build #121882 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121882/testReport)** for PR 28356 at commit [`0f9a186`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #28328: [SPARK-31553][SQL] Fix isInCollection for collection sizes above the optimisation threshold

2020-04-27 Thread GitBox
SparkQA commented on pull request #28328: URL: https://github.com/apache/spark/pull/28328#issuecomment-619897888 **[Test build #121899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121899/testReport)** for PR 28328 at commit [`4bc0e26`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
SparkQA commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619897815 **[Test build #121900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121900/testReport)** for PR 28365 at commit [`e4b8ca9`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #28356: [SPARK-31485][CORE][3.0] Avoid application hang if only partial barrier tasks launched

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28356: URL: https://github.com/apache/spark/pull/28356#issuecomment-619898517 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619898728 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28356: [SPARK-31485][CORE][3.0] Avoid application hang if only partial barrier tasks launched

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28356: URL: https://github.com/apache/spark/pull/28356#issuecomment-619898517 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619898728 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] MaxGekk commented on pull request #28328: [SPARK-31553][SQL] Fix isInCollection for collection sizes above the optimisation threshold

2020-04-27 Thread GitBox
MaxGekk commented on pull request #28328: URL: https://github.com/apache/spark/pull/28328#issuecomment-619904603 I have update PR's description and added a column w/o optimization. I got the numbers by running the code: ```scala test("isInCollection benchmark") { def testExpl

[GitHub] [spark] SparkQA commented on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
SparkQA commented on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619922643 **[Test build #121900 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121900/testReport)** for PR 28365 at commit [`e4b8ca9`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
SparkQA removed a comment on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619897815 **[Test build #121900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121900/testReport)** for PR 28365 at commit [`e4b8ca9`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28365: [SPARK-31571][R] Overhaul stop/message/warning calls to be more translation-friendly/canonical

2020-04-27 Thread GitBox
AmplabJenkins removed a comment on pull request #28365: URL: https://github.com/apache/spark/pull/28365#issuecomment-619923072 This is an automated message from the Apache Git Service. To respond to the message, please log on

  1   2   3   4   5   6   7   8   9   10   >