[GitHub] spark pull request #21427: [SPARK-24324][PYTHON] Pandas Grouped Map UDF shou...

2018-05-25 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21427#discussion_r191040210 --- Diff: python/pyspark/sql/tests.py --- @@ -4931,6 +4931,63 @@ def foo3(key, pdf): expected4 = udf3.func((), pdf

[GitHub] spark issue #21402: SPARK-24355 Spark external shuffle server improvement to...

2018-05-24 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21402 while we are here, could we also add or at least propose some metrics around this, such as number of open block failure, or even number of block threads? we have suffer a lot from

[GitHub] zeppelin issue #2975: ZEPPELIN-3475: Bump up version of Apache Thrift

2018-05-22 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/2975 hmmm - agreed maybe shade is the way to go ---

[GitHub] spark pull request #21368: [SPARK-16451][repl] Fail shell if SparkSession fa...

2018-05-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21368#discussion_r189781578 --- Diff: python/pyspark/sql/session.py --- @@ -547,6 +547,40 @@ def _create_from_pandas_with_arrow(self, pdf, schema, timezone): df

[GitHub] spark pull request #21368: [SPARK-16451][repl] Fail shell if SparkSession fa...

2018-05-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21368#discussion_r189781240 --- Diff: repl/scala-2.12/src/main/scala/org/apache/spark/repl/SparkILoop.scala --- @@ -37,7 +37,14 @@ class SparkILoop(in0: Option[BufferedReader

[GitHub] spark issue #21382: [SPARK-24332][SS][MESOS]Fix places reading 'spark.networ...

2018-05-21 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21382 @susanxhuynh --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21067: [SPARK-23980][K8S] Resilient Spark driver on Kubernetes

2018-05-20 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21067 Jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21342: [SPARK-24294] Throw SparkException when OOM in Br...

2018-05-20 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21342#discussion_r189471396 --- Diff: core/src/main/java/org/apache/spark/memory/SparkOutOfMemoryException.java --- @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #21294: [SPARK-24197][SparkR][SQL] Adding array_sort function to...

2018-05-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21294 looked into a bit not sure why.. - no related changes to testthat - nothing in the R releases around this - no config about default type of NA etc. - tested this out

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-19 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r189447446 --- Diff: python/pyspark/sql/dataframe.py --- @@ -78,6 +78,12 @@ def __init__(self, jdf, sql_ctx): self.is_cached = False self

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-19 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r189447423 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -3056,7 +3059,6 @@ class Dataset[T] private[sql]( * view, e.g

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-19 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r189447465 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,9 +236,13 @@ class Dataset[T] private[sql]( * @param truncate

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-05-19 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r189447477 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -237,9 +236,13 @@ class Dataset[T] private[sql]( * @param truncate

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21370 we will wait for the tests to be fixed first. @xuanyuanking could you update the PR description to clarify which is "before" which

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-05-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21370 @HyukjinKwon @holdenk @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #21374: [SPARK-24323][SQL] Fix lint-java errors

2018-05-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21374 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21368: [SPARK-16451][repl] Fail shell if SparkSession fa...

2018-05-19 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21368#discussion_r189427017 --- Diff: repl/scala-2.11/src/main/scala/org/apache/spark/repl/SparkILoop.scala --- @@ -44,7 +44,14 @@ class SparkILoop(in0: Option[BufferedReader

[GitHub] spark issue #21294: [SPARK-24197][SparkR][SQL] Adding array_sort function to...

2018-05-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21294 hmm, not sure yet, will need to look into --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21294: [SPARK-24197][SparkR][SQL] Adding array_sort function to...

2018-05-18 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21294 What are the versions of R you are running with? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmar...

2018-05-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21288#discussion_r189175140 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -32,14 +32,14 @@ import org.apache.spark.util.{Benchmark

[GitHub] spark issue #21349: [MINOR][PROJECT-INFRA] Check if 'original_head' variable...

2018-05-17 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21349 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21350: [SPARK-24303][PYTHON] Update cloudpickle to v0.4....

2018-05-17 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21350#discussion_r188846130 --- Diff: python/pyspark/cloudpickle.py --- @@ -801,10 +801,10 @@ def save_ellipsis(self, obj): def save_not_implemented(self, obj

[GitHub] zeppelin issue #2969: ZEPPELIN-3463 pip freeze statsmodel 0.8.0

2018-05-15 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/2969 can you update the PR description, include link to JIRA https://issues.apache.org/jira/browse/ZEPPELIN/ etc.? ---

[GitHub] spark pull request #21325: [R][backport-2.2] backport lint fix

2018-05-15 Thread felixcheung
Github user felixcheung closed the pull request at: https://github.com/apache/spark/pull/21325 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

spark git commit: [R][BACKPORT-2.2] backport lint fix

2018-05-15 Thread felixcheung
es #21325 from felixcheung/rlintfix22. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/8c223b65 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/8c223b65 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/8c223b65 Bra

[GitHub] spark issue #21325: [R] backport lint fix

2018-05-14 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21325 also need this in branch-2.1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21325: [R] backport lint fix

2018-05-14 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/21325 [R] backport lint fix ## What changes were proposed in this pull request? backport part of the commit that addresses lintr issue You can merge this pull request into a Git repository

spark git commit: [SPARK-23780][R] Failed to use googleVis library with new SparkR

2018-05-14 Thread felixcheung
ted? manual test, unit tests Author: Felix Cheung <felixcheun...@hotmail.com> Closes #21315 from felixcheung/googvis. (cherry picked from commit 9059f1ee6ae13c8636c9b7fdbb708a349256fb8e) Signed-off-by: Felix Cheung <felixche...@apache.org> Project: http://git-wip-us.apache.org/repos

[GitHub] spark issue #21315: [SPARK-23780][R] Failed to use googleVis library with ne...

2018-05-14 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21315 merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

spark git commit: [SPARK-23780][R] Failed to use googleVis library with new SparkR

2018-05-14 Thread felixcheung
ted? manual test, unit tests Author: Felix Cheung <felixcheun...@hotmail.com> Closes #21315 from felixcheung/googvis. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9059f1ee Tree: http://git-wip-us.apache.org/repos/asf/s

[GitHub] spark pull request #21314: [SPARK-24263][R] SparkR java check breaks with op...

2018-05-13 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21314#discussion_r187817762 --- Diff: R/pkg/R/client.R --- @@ -82,7 +82,7 @@ checkJavaVersion <- function() { }) javaVersionFilter <-

[GitHub] spark issue #21313: [SPARK-24187][R][SQL]Add array_join function to SparkR

2018-05-13 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21313 sure, let's try 4-5 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21314: [SPARK-24263][R] SparkR java check breaks with op...

2018-05-13 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21314#discussion_r187816505 --- Diff: R/pkg/R/client.R --- @@ -82,7 +82,7 @@ checkJavaVersion <- function() { }) javaVersionFilter <-

[GitHub] spark issue #21315: [SPARK-23780][R] Failed to use googleVis library with ne...

2018-05-13 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21315 @shivaram --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21314: [SPARK-24263][R] SparkR java check breaks with op...

2018-05-13 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21314#discussion_r187815502 --- Diff: R/pkg/R/client.R --- @@ -82,7 +82,7 @@ checkJavaVersion <- function() { }) javaVersionFilter <-

[GitHub] spark pull request #21315: [SPARK-23780][R] Failed to use googleVis library ...

2018-05-13 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/21315 [SPARK-23780][R] Failed to use googleVis library with new SparkR ## What changes were proposed in this pull request? change generic to get it to work with googleVis also fix lintr

[GitHub] spark issue #21315: [SPARK-23780][R] Failed to use googleVis library with ne...

2018-05-13 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21315 the main change is https://github.com/apache/spark/compare/master...felixcheung:googvis?expand=1#diff-8e3d61ff66c9ffcd6ffb7a8eedc08409

[GitHub] spark issue #21314: [SPARK-24263] SparkR java check breaks with openjdk

2018-05-13 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21314 @vanzin FYI - need to fix this before release --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21314: [SPARK-24263] SparkR java check breaks with openjdk

2018-05-13 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21314 @shivaram --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21314: [SPARK-24263] SparkR java check breaks with openj...

2018-05-13 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/21314 [SPARK-24263] SparkR java check breaks with openjdk ## What changes were proposed in this pull request? Change text to grep for. ## How was this patch tested? manual

[GitHub] spark pull request #21313: [SPARK-24187][R][SQL]Add array_join function to S...

2018-05-13 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21313#discussion_r187810848 --- Diff: R/pkg/R/functions.R --- @@ -3006,6 +3008,28 @@ setMethod("array_contains",

[GitHub] spark pull request #21307: [SPARK-24186][R][SQL]change reverse and concat to...

2018-05-12 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21307#discussion_r187788406 --- Diff: R/pkg/R/functions.R --- @@ -2055,20 +2058,10 @@ setMethod("countDistinct", #' @details #' \code{concat}: Concatenate

[GitHub] spark pull request #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmar...

2018-05-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21288#discussion_r187764083 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -32,14 +32,14 @@ import org.apache.spark.util.{Benchmark

[GitHub] spark pull request #21298: [SPARK-24198][SparkR][SQL] Adding slice function ...

2018-05-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21298#discussion_r187763906 --- Diff: R/pkg/R/functions.R --- @@ -3138,6 +3139,23 @@ setMethod("size", column(jc) }) +#

zeppelin git commit: [Zeppelin 3388] Correcting documentation link to zeppelin-context documentation

2018-05-10 Thread felixcheung
Repository: zeppelin Updated Branches: refs/heads/branch-0.8 de1a25c73 -> 1b4d37639 [Zeppelin 3388] Correcting documentation link to zeppelin-context documentation ### What is this PR for? A small change was needed in the documentation of the JDBC interpreter also, but had been missed. The

zeppelin git commit: [Zeppelin 3388] Correcting documentation link to zeppelin-context documentation

2018-05-10 Thread felixcheung
Repository: zeppelin Updated Branches: refs/heads/master 3712ce697 -> 941ccbbbd [Zeppelin 3388] Correcting documentation link to zeppelin-context documentation ### What is this PR for? A small change was needed in the documentation of the JDBC interpreter also, but had been missed. The

[GitHub] zeppelin issue #2960: [Zeppelin 3388] Correcting documentation link to zeppe...

2018-05-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/2960 merged to master and 0.8 ---

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r187514621 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,48 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedA

[GitHub] spark pull request #21298: [SPARK-24198][SparkR][SQL] Adding slice function ...

2018-05-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21298#discussion_r187514450 --- Diff: R/pkg/R/functions.R --- @@ -3124,6 +3125,23 @@ setMethod("size", column(jc) }) +#

[GitHub] zeppelin issue #2960: [Zeppelin 3388] Correcting documentation link to zeppe...

2018-05-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/2960 @zjffdu can I merge this to branch-0.8? ---

[GitHub] spark pull request #21294: [SPARK-24197][SparkR][SQL] Adding array_sort func...

2018-05-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21294#discussion_r187470081 --- Diff: R/pkg/R/functions.R --- @@ -208,6 +208,7 @@ NULL #' head(select(tmp, array_contains(tmp$v1, 21), size(tmp$v1))) #' head(select(tmp

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r187469392 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,48 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedA

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r187469216 --- Diff: R/pkg/DESCRIPTION --- @@ -13,6 +13,7 @@ Authors@R: c(person("Shivaram", "Venkataraman", role = c("aut", "

[GitHub] spark issue #21255: [SPARK-24186][R][SQL]change reverse and concat to collec...

2018-05-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21255 Yes From: Hyukjin Kwon <notificati...@github.com> Sent: Thursday, May 10, 2018 9:16:28 AM To: apache/spark Cc: Felix Cheung; M

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r187238820 --- Diff: R/pkg/DESCRIPTION --- @@ -13,6 +13,7 @@ Authors@R: c(person("Shivaram", "Venkataraman", role = c("aut", "

[GitHub] spark pull request #21255: [SPARK-24186][R][SQL]change reverse and concat to...

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21255#discussion_r187238518 --- Diff: R/pkg/R/functions.R --- @@ -219,7 +219,8 @@ NULL #' head(select(tmp3, map_values(tmp3$v3))) #' head(select(tmp3, element_at(tmp3$v3

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r187237615 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,48 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedA

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r187237024 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,48 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedA

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21278 nice I like it... they also say ``` When specifying a minimum Java version please use the official version names, which are (confusingly) 1.1 1.2 1.3 1.4 5.0 6 7 8 9 10

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21278 it fails with ``` Quitting from lines 65-67 (sparkr-vignettes.Rmd) Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics: Java version check failed. Please

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21278 I see yes, maybe grep for java version --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21278 ``` Quitting from lines 65-67 (sparkr-vignettes.Rmd) Error: processing vignette 'sparkr-vignettes.Rmd' failed with diagnostics: Java version check failed Execution halted

[GitHub] spark pull request #21255: [SPARK-24186][R][SQL]change reverse and concat to...

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21255#discussion_r186942863 --- Diff: R/pkg/R/functions.R --- @@ -2047,17 +2049,15 @@ setMethod("countDistinct", #' \code{concat}: Concatenates multiple input column

[GitHub] spark pull request #21255: [SPARK-24186][R][SQL]change reverse and concat to...

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21255#discussion_r186942776 --- Diff: R/pkg/R/functions.R --- @@ -2047,17 +2049,15 @@ setMethod("countDistinct", #' \code{concat}: Concatenates multiple input column

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186940485 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,39 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedA

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186940268 --- Diff: R/pkg/R/sparkR.R --- @@ -163,6 +163,10 @@ sparkR.sparkContext <- function( submitOps <- getClientModeSparkSubm

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186939722 --- Diff: R/pkg/R/utils.R --- @@ -756,7 +756,7 @@ launchScript <- function(script, combinedArgs, wait = FALSE) { # stdout = F means disc

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186940332 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,39 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedA

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186940416 --- Diff: R/pkg/R/client.R --- @@ -60,13 +60,39 @@ generateSparkSubmitArgs <- function(args, sparkHome, jars, sparkSubmitOpts, pack combinedA

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21278#discussion_r186939913 --- Diff: R/pkg/R/sparkR.R --- @@ -163,6 +163,10 @@ sparkR.sparkContext <- function( submitOps <- getClientModeSparkSubm

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-09 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21278 yea, I don't think Spark builds on Java 9 (at least from what I've seen) I see the package is gone from CRAN so the test results are brief but maybe related to https://cran.r

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21278 also I think test fails on Java 9 - is there a way to exclude - doc sounds like it's a minimal version --- - To unsubscribe

[GitHub] spark pull request #21255: [SPARK-24186][SparR][SQL]change reverse and conca...

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21255#discussion_r186634471 --- Diff: R/pkg/R/functions.R --- @@ -2043,34 +2033,6 @@ setMethod("countDistinct", column(jc) }) -#

[GitHub] spark pull request #21255: [SPARK-24186][SparR][SQL]change reverse and conca...

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21255#discussion_r186633965 --- Diff: R/pkg/R/functions.R --- @@ -1253,19 +1256,6 @@ setMethod("quarter", column(jc) }) -#

[GitHub] spark pull request #21255: [SPARK-24186][SparR][SQL]change reverse and conca...

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21255#discussion_r186624915 --- Diff: R/pkg/R/functions.R --- @@ -2043,34 +2033,6 @@ setMethod("countDistinct", column(jc) }) -#

[GitHub] spark pull request #21255: [SPARK-24186][SparR][SQL]change reverse and conca...

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21255#discussion_r186624766 --- Diff: R/pkg/R/functions.R --- @@ -1253,19 +1256,6 @@ setMethod("quarter", column(jc) }) -#

[GitHub] spark pull request #21255: [SPARK-24186][SparR][SQL]change reverse and conca...

2018-05-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21255#discussion_r186624379 --- Diff: R/pkg/R/functions.R --- @@ -209,6 +209,7 @@ NULL #' head(select(tmp, array_max(tmp$v1), array_min(tmp$v1))) #' head(select(tmp

[GitHub] spark pull request #21092: [SPARK-23984][K8S] Initial Python Bindings for Py...

2018-05-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21092#discussion_r186326478 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/bindings/python/Dockerfile --- @@ -0,0 +1,34 @@ +# +# Licensed

[GitHub] spark issue #21198: [SPARK-24126][pyspark] Use build-specific temp directory...

2018-05-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21198 yes, for R, during package check it can only write to the R tempdir() (which can be changed in a number of ways

[GitHub] zeppelin issue #2960: [Zeppelin 3388] Correcting documentation link to zeppe...

2018-05-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/2960 merging if no more comment ---

[GitHub] spark pull request #21244: [SPARK-24185][SparkR][SQL]add flatten function to...

2018-05-05 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21244#discussion_r186278752 --- Diff: R/pkg/R/generics.R --- @@ -918,6 +918,10 @@ setGeneric("explode_outer", function(x) { standardGeneric("explode_outer") }

[GitHub] spark pull request #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in ...

2018-05-05 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18447#discussion_r186278707 --- Diff: R/pkg/R/functions.R --- @@ -679,6 +679,19 @@ setMethod("hash", column(jc) }) +#

[GitHub] spark pull request #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in ...

2018-05-05 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18447#discussion_r186278702 --- Diff: R/pkg/R/functions.R --- @@ -679,6 +679,19 @@ setMethod("hash", column(jc) }) +#

[GitHub] spark pull request #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in ...

2018-05-05 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18447#discussion_r186278685 --- Diff: R/pkg/R/functions.R --- @@ -679,6 +679,19 @@ setMethod("hash", column(jc) }) +#

[GitHub] zeppelin issue #2555: [ZEPPELIN-2885] Have Logger subclass StringIO and over...

2018-05-04 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/2555 hi any update on this? ---

[GitHub] spark pull request #21217: [SPARK-24151][SQL] Fix CURRENT_DATE, CURRENT_TIME...

2018-05-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21217#discussion_r186252704 --- Diff: docs/sql-programming-guide.md --- @@ -1812,6 +1812,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark pull request #21241: [SPARK-24135][K8s] Resilience to init-container e...

2018-05-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21241#discussion_r186252592 --- Diff: docs/running-on-kubernetes.md --- @@ -561,6 +561,13 @@ specific to Spark on Kubernetes. This is distinct from spark.executor.cores

[GitHub] spark pull request #21241: [SPARK-24135][K8s] Resilience to init-container e...

2018-05-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21241#discussion_r186252598 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala

[GitHub] spark pull request #21228: [SPARK-24171] Adding a note for non-deterministic...

2018-05-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21228#discussion_r186252477 --- Diff: R/pkg/R/functions.R --- @@ -818,6 +818,7 @@ setMethod("factorial", #' first(df$c, TRUE) #' } #' @note first(charact

[GitHub] spark pull request #21228: [SPARK-24171] Adding a note for non-deterministic...

2018-05-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21228#discussion_r186252480 --- Diff: R/pkg/R/functions.R --- @@ -963,6 +964,7 @@ setMethod("kurtosis", #' last(df$c, TRUE) #' } #' @note last s

[GitHub] spark pull request #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in ...

2018-05-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18447#discussion_r186252381 --- Diff: R/pkg/R/functions.R --- @@ -679,6 +679,19 @@ setMethod("hash", column(jc) }) +#

[GitHub] spark pull request #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in ...

2018-05-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18447#discussion_r186252423 --- Diff: R/pkg/NAMESPACE --- @@ -236,6 +236,7 @@ exportMethods("%<=>%", "current_date",

[GitHub] spark pull request #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in ...

2018-05-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18447#discussion_r186252398 --- Diff: R/pkg/R/functions.R --- @@ -679,6 +679,19 @@ setMethod("hash", column(jc) }) +#

[GitHub] spark pull request #21198: [SPARK-24126][pyspark] Use build-specific temp di...

2018-05-03 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21198#discussion_r185705560 --- Diff: python/run-tests.py --- @@ -77,13 +79,33 @@ def run_individual_python_test(test_name, pyspark_python): 'PYSPARK_PYTHON': which

[GitHub] spark issue #21201: [SPARK-24128][SQL] Mention configuration option in impli...

2018-05-03 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21201 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in ...

2018-05-03 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18447#discussion_r185703059 --- Diff: R/pkg/R/functions.R --- @@ -653,6 +653,25 @@ setMethod("hash", column(jc) }) +#'

[GitHub] spark issue #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in SQL fun...

2018-05-03 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18447 can you rebase? (there is a conflict) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in SQL fun...

2018-05-03 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18447 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21197: [SPARK-23971] [BACKPORT-2.3] Should not leak Spark sessi...

2018-05-02 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21197 Is this the error? Seems like intermittent problem from CRAN. Let me know if you see this again. Also its just the log text repeated, but the test run. checking CRAN incoming

[GitHub] spark issue #21200: [SPARK-24039][SS] Do continuous processing writes with m...

2018-05-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21200 Jenkins, add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

<    3   4   5   6   7   8   9   10   11   12   >