[GitHub] spark issue #23012: [SPARK-26014][R] Deprecate R prior to version 3.4 in Spa...

2018-11-12 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/23012 Also I think the warning should be in .First in general.R --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #23012: [SPARK-26014][R] Deprecate R prior to version 3.4 in Spa...

2018-11-12 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/23012 I think this should say unsupported (ie could still work) instead of deprecated Also the compareVersion should check both major and minor ie 3.4.0

[GitHub] spark pull request #22939: [SPARK-25446][R] Add schema_of_json() and schema_...

2018-11-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22939#discussion_r232500194 --- Diff: R/pkg/R/functions.R --- @@ -2230,6 +2237,32 @@ setMethod("from_json", signature(x = "Column", schema = &q

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232500065 --- Diff: R/pkg/R/SQLContext.R --- @@ -172,36 +257,72 @@ getDefaultSqlSource <- function() { createDataFrame <- function(data, schema

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232499902 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,91 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232499848 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -307,6 +307,64 @@ test_that("create DataFrame from RDD", { unsetH

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232499794 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -307,6 +307,64 @@ test_that("create DataFrame from RDD", { unsetH

[GitHub] spark pull request #22993: [SPARK-24421][BUILD][CORE] Accessing sun.misc.Cle...

2018-11-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22993#discussion_r232499645 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java --- @@ -67,6 +67,59 @@ unaligned = _unaligned

[GitHub] spark pull request #23007: [SPARK-26010] fix vignette eval with Java 11

2018-11-11 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/23007 [SPARK-26010] fix vignette eval with Java 11 ## What changes were proposed in this pull request? changes in vignette only to disable eval ## How was this patch tested

[GitHub] spark pull request #22993: [SPARK-24421][BUILD][CORE] Accessing sun.misc.Cle...

2018-11-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22993#discussion_r232477875 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java --- @@ -67,6 +67,59 @@ unaligned = _unaligned

[GitHub] spark issue #22993: [SPARK-24421][BUILD][CORE] Accessing sun.misc.Cleaner in...

2018-11-11 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22993 what settings we need to allow `illegal reflective access` --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22989: [SPARK-25986][Build] Banning throw new OutOfMemoryErrors

2018-11-11 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22989 and catching Error or Throwable.. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #22989: [SPARK-25986][Build] Banning throw new OutOfMemor...

2018-11-11 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22989#discussion_r232477783 --- Diff: scalastyle-config.xml --- @@ -240,6 +240,18 @@ This file is divided into 3 sections: ]]> + +throw

[GitHub] spark issue #22977: [BUILD] Bump previousSparkVersion in MimaBuild.scala to ...

2018-11-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22977 right, I mean both this and that should be part of the process "post-release" --- - To unsubscribe, e-mai

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232477365 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,55 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232477325 --- Diff: R/pkg/R/SQLContext.R --- @@ -189,19 +238,67 @@ createDataFrame <- function(data, schema = NULL, samplingRatio = 1.0,

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232477271 --- Diff: R/pkg/R/SQLContext.R --- @@ -189,19 +238,67 @@ createDataFrame <- function(data, schema = NULL, samplingRatio = 1.0,

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232477257 --- Diff: R/pkg/R/SQLContext.R --- @@ -189,19 +238,67 @@ createDataFrame <- function(data, schema = NULL, samplingRatio = 1.0,

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232477171 --- Diff: R/pkg/R/SQLContext.R --- @@ -172,10 +221,10 @@ getDefaultSqlSource <- function() { createDataFrame <- function(data, schema

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232477155 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,55 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-10 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232477131 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -225,4 +226,25 @@ private[sql] object SQLUtils extends Logging

[GitHub] spark issue #22997: SPARK-25999: make-distribution.sh failure with --r and -...

2018-11-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22997 btw, please see the page https://spark.apache.org/contributing.html and particularly "Pull Request" on

[GitHub] spark issue #22997: SPARK-25999: make-distribution.sh failure with --r and -...

2018-11-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22997 thx, but I'm not sure about this approach. this step will now cause hadoop jar to be packaged into the release tarball of hadoop-provided, which is undoing the point of hadoop-provided

[GitHub] spark issue #22977: [BUILD] Bump previousSparkVersion in MimaBuild.scala to ...

2018-11-10 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22977 I think also there is a hive metastore test that downloads spark release jar? --- - To unsubscribe, e-mail

[GitHub] spark pull request #22967: [SPARK-25956] Make Scala 2.12 as default Scala ve...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22967#discussion_r232178323 --- Diff: docs/sparkr.md --- @@ -133,7 +133,7 @@ specifying `--packages` with `spark-submit` or `sparkR` commands, or if initiali

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232170936 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,55 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232176721 --- Diff: R/pkg/R/SQLContext.R --- @@ -189,19 +238,67 @@ createDataFrame <- function(data, schema = NULL, samplingRatio = 1.0,

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232172687 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,55 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232169938 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,55 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232173367 --- Diff: R/pkg/R/SQLContext.R --- @@ -189,19 +238,67 @@ createDataFrame <- function(data, schema = NULL, samplingRatio = 1.0,

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232173043 --- Diff: R/pkg/R/SQLContext.R --- @@ -172,10 +221,10 @@ getDefaultSqlSource <- function() { createDataFrame <- function(data, schema

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232167634 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,55 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232172546 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,55 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232177132 --- Diff: R/pkg/R/SQLContext.R --- @@ -189,19 +238,67 @@ createDataFrame <- function(data, schema = NULL, samplingRatio = 1.0,

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232167926 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -225,4 +226,25 @@ private[sql] object SQLUtils extends Logging

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232176774 --- Diff: R/pkg/R/SQLContext.R --- @@ -189,19 +238,67 @@ createDataFrame <- function(data, schema = NULL, samplingRatio = 1.0,

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232171176 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,55 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232167480 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,55 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [SPARK-25981][R] Enables Arrow optimization from ...

2018-11-09 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r232167110 --- Diff: R/pkg/R/SQLContext.R --- @@ -215,14 +278,16 @@ createDataFrame <- function(data, schema = NULL, samplingRatio =

[GitHub] spark pull request #22939: [SPARK-25446][R] Add schema_of_json() and schema_...

2018-11-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22939#discussion_r232166370 --- Diff: R/pkg/R/functions.R --- @@ -2230,6 +2237,32 @@ setMethod("from_json", signature(x = "Column", schema = &q

[GitHub] spark pull request #22967: [SPARK-25956] Make Scala 2.12 as default Scala ve...

2018-11-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22967#discussion_r231819016 --- Diff: docs/sparkr.md --- @@ -133,7 +133,7 @@ specifying `--packages` with `spark-submit` or `sparkR` commands, or if initiali

[GitHub] spark issue #22932: [SPARK-25102][SQL] Write Spark version to ORC/Parquet fi...

2018-11-07 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22932 Does it have different values for new native ORC writer, old Hive ORC writer

[GitHub] spark pull request #22948: [SPARK-25944][R][BUILD] AppVeyor change to latest...

2018-11-07 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22948#discussion_r231598045 --- Diff: dev/appveyor-install-dependencies.ps1 --- @@ -115,7 +115,7 @@ $env:Path += ";$env:HADOOP_HOME\bin" Po

[GitHub] spark pull request #22921: [SPARK-25908][CORE][SQL] Remove old deprecated it...

2018-11-07 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22921#discussion_r231596680 --- Diff: R/pkg/R/functions.R --- @@ -1663,9 +1692,24 @@ setMethod("toDegrees", #' @aliases toRadians toRadians,Column-method

[GitHub] spark pull request #22921: [SPARK-25908][CORE][SQL] Remove old deprecated it...

2018-11-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22921#discussion_r231403827 --- Diff: R/pkg/R/functions.R --- @@ -319,6 +319,27 @@ setMethod("acos", column(jc) }) +#

[GitHub] spark pull request #22939: [SPARK-25446][R] Add schema_of_json() and schema_...

2018-11-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22939#discussion_r231403096 --- Diff: R/pkg/R/functions.R --- @@ -2230,6 +2237,32 @@ setMethod("from_json", signature(x = "Column", schema = &q

[GitHub] spark pull request #22954: [DO-NOT-MERGE][POC] Enables Arrow optimization fr...

2018-11-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r231402726 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,30 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [DO-NOT-MERGE][POC] Enables Arrow optimization fr...

2018-11-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r231402235 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,30 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [DO-NOT-MERGE][POC] Enables Arrow optimization fr...

2018-11-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r231402297 --- Diff: R/pkg/R/SQLContext.R --- @@ -172,15 +196,17 @@ getDefaultSqlSource <- function() { createDataFrame <- function(data, schema

[GitHub] spark pull request #22954: [DO-NOT-MERGE][POC] Enables Arrow optimization fr...

2018-11-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r231402063 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,30 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] spark pull request #22954: [DO-NOT-MERGE][POC] Enables Arrow optimization fr...

2018-11-06 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22954#discussion_r231401994 --- Diff: R/pkg/R/SQLContext.R --- @@ -147,6 +147,30 @@ getDefaultSqlSource <- function() { l[["spark.sql.sources

[GitHub] zeppelin issue #3206: [ZEPPELIN-3810] Support Spark 2.4

2018-11-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/3206 yes, everything is out (except for the announcement) ---

[GitHub] zeppelin issue #3217: [ZEPPELIN-3848] zeppelin-web dev mode does not work

2018-11-06 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/zeppelin/pull/3217 LGTM ---

[GitHub] spark pull request #22939: [SPARK-25446][R] Add schema_of_json() and schema_...

2018-11-05 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22939#discussion_r231025592 --- Diff: R/pkg/R/functions.R --- @@ -205,11 +205,18 @@ NULL #' also supported for the schema. #' \item \code{from_csv

[GitHub] spark pull request #22939: [SPARK-25446][R] Add schema_of_json() and schema_...

2018-11-05 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22939#discussion_r231025282 --- Diff: R/pkg/R/functions.R --- @@ -2230,6 +2237,32 @@ setMethod("from_json", signature(x = "Column", schema = &q

[GitHub] spark pull request #22921: [SPARK-25908][CORE][SQL] Remove old deprecated it...

2018-11-05 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22921#discussion_r231024007 --- Diff: R/pkg/R/functions.R --- @@ -1641,30 +1641,30 @@ setMethod("tanh", }) #' @details -#' \code{toDegrees}

[GitHub] spark pull request #22921: [SPARK-25908][CORE][SQL] Remove old deprecated it...

2018-11-05 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22921#discussion_r231023768 --- Diff: R/pkg/R/generics.R --- @@ -748,7 +748,7 @@ setGeneric("add_months", function(y, x) { standardGeneric("add_months") })

[GitHub] spark pull request #22939: [SPARK-25446][R] Add schema_of_json() and schema_...

2018-11-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22939#discussion_r230649513 --- Diff: R/pkg/R/functions.R --- @@ -205,11 +205,18 @@ NULL #' also supported for the schema. #' \item \code{from_csv

[GitHub] spark pull request #22939: [SPARK-25446][R] Add schema_of_json() and schema_...

2018-11-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22939#discussion_r230650176 --- Diff: R/pkg/R/functions.R --- @@ -2230,6 +2237,32 @@ setMethod("from_json", signature(x = "Column", schema = &q

[GitHub] spark pull request #22939: [SPARK-25446][R] Add schema_of_json() and schema_...

2018-11-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22939#discussion_r230649120 --- Diff: R/pkg/R/functions.R --- @@ -2230,6 +2237,32 @@ setMethod("from_json", signature(x = "Column", schema = &q

[GitHub] spark pull request #22939: [SPARK-25446][R] Add schema_of_json() and schema_...

2018-11-04 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22939#discussion_r230649693 --- Diff: R/pkg/R/functions.R --- @@ -2260,6 +2293,32 @@ setMethod("from_csv", signature(x = "Column", schema

[GitHub] spark issue #22683: [SPARK-25696] The storage memory displayed on spark Appl...

2018-11-03 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22683 Jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22921: [SPARK-25908][CORE][SQL] Remove old deprecated it...

2018-11-03 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22921#discussion_r230568058 --- Diff: R/pkg/R/generics.R --- @@ -748,7 +748,7 @@ setGeneric("add_months", function(y, x) { standardGeneric("add_months") })

[GitHub] spark pull request #22921: [SPARK-25908][CORE][SQL] Remove old deprecated it...

2018-11-03 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22921#discussion_r230568079 --- Diff: R/pkg/R/functions.R --- @@ -1641,30 +1641,30 @@ setMethod("tanh", }) #' @details -#' \code{toDegrees}

[GitHub] spark pull request #22921: [SPARK-25908][CORE][SQL] Remove old deprecated it...

2018-11-03 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22921#discussion_r230568088 --- Diff: R/pkg/R/functions.R --- @@ -319,23 +319,23 @@ setMethod("acos", }) #' @details -#' \code{approxCou

[GitHub] spark issue #22864: [SPARK-25861][Minor][WEBUI] Remove unused refreshInterva...

2018-11-01 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22864 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22866: [SPARK-12172][SPARKR] Remove internal-only RDD me...

2018-10-27 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/22866 [SPARK-12172][SPARKR] Remove internal-only RDD methods ## What changes were proposed in this pull request? Remove non-public internal only methods for RDD in SparkR ## How

[GitHub] spark issue #22863: [SPARK-25859][ML]add scala/java/python example and doc f...

2018-10-27 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22863 please close this PR. thx --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

spark git commit: [SPARK-25859][ML] add scala/java/python example and doc for PrefixSpan

2018-10-27 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.4 313a1f0a7 -> f575616db [SPARK-25859][ML] add scala/java/python example and doc for PrefixSpan ## What changes were proposed in this pull request? add scala/java/python example and doc for PrefixSpan in branch 2.4 ## How was this

[GitHub] spark issue #22863: [SPARK-25859][ML]add scala/java/python example and doc f...

2018-10-27 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22863 merged to 2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22863: [SPARK-25859][ML]add scala/java/python example an...

2018-10-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22863#discussion_r228727912 --- Diff: examples/src/main/scala/org/apache/spark/examples/ml/PrefixSpanExample.scala --- @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #22865: [DOC] Fix doc for spark.sql.parquet.recordLevelFilter.en...

2018-10-27 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22865 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

spark git commit: [SPARK-16693][SPARKR] Remove methods deprecated

2018-10-27 Thread felixcheung
ile registerTempTable saveAsParquetFile unionAll createExternalTable dropTempTable ## How was this patch tested? jenkins Author: Felix Cheung Closes #22843 from felixcheung/rrddapi. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/com

[GitHub] spark issue #22843: [SPARK-16693][SPARKR] Remove methods deprecated

2018-10-27 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22843 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #22843: [SPARK-16693][SPARKR] Remove methods deprecated

2018-10-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22843#discussion_r228701261 --- Diff: docs/sparkr.md --- @@ -717,4 +717,5 @@ You can inspect the search path in R with [`search()`](https://stat.ethz.ch/R-ma

[GitHub] spark pull request #22843: [SPARK-16693][SPARKR] Remove methods deprecated

2018-10-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22843#discussion_r228701032 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -3477,39 +3447,6 @@ test_that("Window functions on a DataFrame", { expect_eq

[GitHub] spark pull request #22843: [SPARK-16693][SPARKR] Remove methods deprecated

2018-10-26 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22843#discussion_r228701116 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -3477,39 +3447,6 @@ test_that("Window functions on a DataFrame", { expect_eq

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-26 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21710 Sure From: Huaxin Gao Sent: Friday, October 26, 2018 1:16:38 PM To: apache/spark Cc: Felix Cheung; Mention Subject: Re: [apache

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-10-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21588 Sounds like we should try this then --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22843: [SPARK-16693][SPARKR] Remove methods deprecated

2018-10-25 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/22843 [SPARK-16693][SPARKR] Remove methods deprecated ## What changes were proposed in this pull request? Remove deprecated functions which includes: SQLContext/HiveContext stuff

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-10-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21588 does Apache Hive 2.3.2 have all the fixes we need? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22815: [SPARK-25821][SQL] Remove SQLContext methods deprecated ...

2018-10-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22815 Because it was in L457 - multiple function doc blob merged together and CRAN checks for the final version for missing param Thx for catching

[GitHub] spark issue #22821: [SPARK-25832][SQL] remove newly added map related functi...

2018-10-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22821 Please ping me on the R removal. --- - To unsubscribe, e-mail: reviews

spark git commit: [SPARK-24572][SPARKR] "eager execution" for R shell, IDE

2018-10-25 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 19ada15d1 -> ddd1b1e8a [SPARK-24572][SPARKR] "eager execution" for R shell, IDE ## What changes were proposed in this pull request? Check the `spark.sql.repl.eagerEval.enabled` configuration property in SparkDataFrame `show()` method. If

[GitHub] spark issue #22455: [SPARK-24572][SPARKR] "eager execution" for R shell, IDE

2018-10-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22455 LGTM. merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22810: [SPARK-24516][K8S] Change Python default to Python3

2018-10-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22810 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

spark git commit: [SPARK-24516][K8S] Change Python default to Python3

2018-10-25 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master b2e325625 -> 19ada15d1 [SPARK-24516][K8S] Change Python default to Python3 ## What changes were proposed in this pull request? As this is targeted for 3.0.0 and Python2 will be deprecated by Jan 1st, 2020, I feel it is appropriate to

[GitHub] spark pull request #22815: [SPARK-25821][SQL] Remove SQLContext methods depr...

2018-10-25 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22815#discussion_r228045893 --- Diff: R/pkg/R/SQLContext.R --- @@ -343,7 +343,6 @@ setMethod("toDF", signature(x = "RDD"), #' path <- "p

[GitHub] spark issue #22668: [SPARK-25675] [Spark Job History] Job UI page does not s...

2018-10-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22668 ouch sorry --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22801: [SPARK-25656][SQL][DOC][EXAMPLE] Add a doc and ex...

2018-10-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22801#discussion_r227466613 --- Diff: examples/src/main/python/sql/datasource.py --- @@ -57,6 +57,15 @@ def basic_datasource_example(spark): format

[GitHub] spark pull request #22801: [SPARK-25656][SQL][DOC][EXAMPLE] Add a doc and ex...

2018-10-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22801#discussion_r227466415 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -118,6 +118,10 @@ df <- read.df("examples/src/main/resources/people.csv"

[GitHub] spark pull request #22791: [SPARK-25795][R][EXAMPLE] Fix CSV SparkR SQL Exam...

2018-10-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22791#discussion_r227465232 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -114,7 +114,7 @@ write.df(namesAndAges, "namesAndAges.parquet",

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-10-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r227462444 --- Diff: docs/running-on-yarn.md --- @@ -121,6 +121,43 @@ To use a custom metrics.properties for the application master and executors, upd

[GitHub] spark issue #21306: [SPARK-24252][SQL] Add catalog registration and table ca...

2018-10-21 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21306 where are we on this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-21 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21710 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

spark git commit: [SPARK-24207][R] add R API for PrefixSpan

2018-10-21 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 4c6c6711d -> fc64e83f9 [SPARK-24207][R] add R API for PrefixSpan ## What changes were proposed in this pull request? add R API for PrefixSpan ## How was this patch tested? add test in test_mllib_fpm.R Author: Huaxin Gao Closes #21710

[GitHub] spark pull request #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-10-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21710#discussion_r226875038 --- Diff: examples/src/main/python/ml/prefixspan_example.py --- @@ -0,0 +1,48 @@ +# --- End diff -- yes, ping me on that and I can

[GitHub] spark issue #22305: [SPARK-24561][SQL][Python] User-defined window aggregati...

2018-10-21 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22305 where are we on this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22455: [SPARK-24572][SPARKR] "eager execution" for R she...

2018-10-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22455#discussion_r226874107 --- Diff: R/pkg/R/DataFrame.R --- @@ -244,11 +246,33 @@ setMethod("showDF", #' @note show(SparkDataFrame) since 1.4.0 setMe

[GitHub] spark pull request #22455: [SPARK-24572][SPARKR] "eager execution" for R she...

2018-10-21 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22455#discussion_r226874469 --- Diff: R/pkg/tests/fulltests/test_eager_execution.R --- @@ -0,0 +1,52 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one

<    1   2   3   4   5   6   7   8   9   10   >