[GitHub] spark issue #14173: [SPARKR][SPARK-16507] Add a CRAN checker, fix Rd aliases

2016-07-13 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14173 cc @felixcheung @sun-rui @mengxr @junyangq --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #14173: [SPARKR][SPARK-16507] Add a CRAN checker, fix Rd ...

2016-07-13 Thread shivaram
GitHub user shivaram opened a pull request: https://github.com/apache/spark/pull/14173 [SPARKR][SPARK-16507] Add a CRAN checker, fix Rd aliases ## What changes were proposed in this pull request? Add a check-cran.sh script that runs `R CMD check` as CRAN. Also fixes

[GitHub] spark pull request #14090: [SPARK-16112][SparkR] Programming guide for gappl...

2016-07-11 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14090#discussion_r70346974 --- Diff: docs/sparkr.md --- @@ -306,6 +306,64 @@ head(ldf, 3) {% endhighlight %} + Run a given function on a large dataset

[GitHub] spark issue #13993: [SPARK-16144][SPARKR] update R API doc for mllib

2016-07-11 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13993 Change LGTM. Thanks @junyangq and @yanboliang for reviewing. Merging this to master and branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #14011: [SparkR] [Doc] SparkR ML user guides update for 2.0

2016-07-11 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14011 Thanks @yanboliang - LGTM. Merging this to master and branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #14090: [SPARK-16112][SparkR] Programming guide for gappl...

2016-07-10 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14090#discussion_r70202560 --- Diff: docs/sparkr.md --- @@ -306,6 +306,64 @@ head(ldf, 3) {% endhighlight %} + Run a given function on a large dataset

[GitHub] spark pull request #14090: [SPARK-16112][SparkR] Programming guide for gappl...

2016-07-10 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14090#discussion_r70202064 --- Diff: docs/sparkr.md --- @@ -306,6 +306,64 @@ head(ldf, 3) {% endhighlight %} + Run a given function on a large dataset

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SQL] Make CSV cast null value...

2016-07-09 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14118 No - I just noticed a JIRA that said it was a regression, so I wanted to make sure you caught this in the RC cycle --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #14118: [SPARK-16462][SPARK-16460][SQL] Make CSV cast null value...

2016-07-09 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14118 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #14011: [SparkR] [Doc] SparkR ML user guides update for 2.0

2016-07-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14011 @yanboliang is this good to go ? Would be good to merge this soon --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14082: [SPARK-16381][SQL][SparkR] Update SQL examples and progr...

2016-07-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14082 Thanks @keypointt -- LGTM. Will merge this once Jenkins passes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #14082: [SPARK-16381][SQL][SparkR] Update SQL examples and progr...

2016-07-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14082 Thanks @keypointt -- Changes look good to me. @felixcheung any other comments ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request #14082: [SPARK-16381][SQL][SparkR] Update SQL examples an...

2016-07-08 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14082#discussion_r70125056 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -0,0 +1,198 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #14082: [SPARK-16381][SQL][SparkR] Update SQL examples an...

2016-07-08 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14082#discussion_r70124201 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -0,0 +1,198 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #14082: [SPARK-16381][SQL][SparkR] Update SQL examples an...

2016-07-08 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14082#discussion_r70122513 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -0,0 +1,175 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #14082: [SPARK-16381][SQL][SparkR] Update SQL examples an...

2016-07-08 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14082#discussion_r70122361 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -0,0 +1,198 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #14082: [SPARK-16381][SQL][SparkR] Update SQL examples an...

2016-07-08 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14082#discussion_r70121939 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -0,0 +1,198 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #14082: [SPARK-16381][SQL][SparkR] Update SQL examples an...

2016-07-08 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14082#discussion_r70121544 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -0,0 +1,198 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #14082: [SPARK-16381][SQL][SparkR] Update SQL examples an...

2016-07-08 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14082#discussion_r70121398 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -0,0 +1,198 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #14082: [SPARK-16381][SQL][SparkR] Update SQL examples an...

2016-07-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14082#discussion_r70012365 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -0,0 +1,175 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark issue #14096: [SPARK-16425][R] `describe()` should not fail with non-n...

2016-07-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14096 LGTM. Thanks @dongjoon-hyun -- Merging this to master, branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #14082: [SPARK-16381][SQL][SparkR] Update SQL examples an...

2016-07-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14082#discussion_r70009456 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -0,0 +1,175 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #14082: [SPARK-16381][SQL][SparkR] Update SQL examples an...

2016-07-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14082#discussion_r70008001 --- Diff: examples/src/main/r/RSparkSQLExample.R --- @@ -0,0 +1,175 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #14096: [SPARK-16425][R] `describe()` should not fail wit...

2016-07-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14096#discussion_r70006591 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -1804,11 +1804,11 @@ test_that("describe() and summarize() on a Data

[GitHub] spark pull request #14096: [SPARK-16425][R] `describe()` should not fail wit...

2016-07-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14096#discussion_r70003846 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -1804,11 +1804,11 @@ test_that("describe() and summarize() on a Data

[GitHub] spark issue #14096: [SPARK-16425][R] `describe()` should consider numeric/st...

2016-07-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14096 Yeah we should just call it with empty columns (instead of all the columns) and let the Scala side do the appropriate thing. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #14082: [SPARK-16381][SQL][SparkR] Update SQL examples and progr...

2016-07-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14082 I'll take a look at this today. Also cc @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #13984: [SPARK-16310][SPARKR] R na.string-like default for csv s...

2016-07-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13984 LGTM. Merging this to master and branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #13984: [SPARK-16310][SPARKR] R na.string-like default fo...

2016-07-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13984#discussion_r69997940 --- Diff: R/pkg/R/SQLContext.R --- @@ -744,6 +747,9 @@ read.df.default <- function(path = NULL, source = NULL, schema = NULL, ...) { if (is.n

[GitHub] spark issue #14096: [SPARK-16425][R] `describe()` should consider numeric/st...

2016-07-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14096 I'm not sure this is something we should be fixing just on R frontend. What happens when we run the query from Scala / Python ? If we get the same error we should be fixing it in Scala

[GitHub] spark pull request #13984: [SPARK-16310][SPARKR] R na.string-like default fo...

2016-07-07 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13984#discussion_r69945470 --- Diff: R/pkg/R/SQLContext.R --- @@ -744,6 +747,9 @@ read.df.default <- function(path = NULL, source = NULL, schema = NULL, ...) { if (is.n

[GitHub] spark issue #14090: [SPARK-16112][SparkR] Programming guide for gapply/gappl...

2016-07-07 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14090 cc @felixcheung @mengxr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #14041: [SPARK-16359][STREAMING][KAFKA] unidoc skip kafka 0.10

2016-07-04 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14041 I just noticed that our nightly docs build has been failing with an error related to kafka (Example [1]). Will this PR fix this or should we open a new JIRA for this ? [1] https

[GitHub] spark pull request #13984: [SPARK-16310][SPARKR] R na.string-like default fo...

2016-07-04 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13984#discussion_r69487213 --- Diff: R/pkg/R/SQLContext.R --- @@ -744,6 +747,9 @@ read.df.default <- function(path = NULL, source = NULL, schema = NULL, ...) { if (is.n

[GitHub] spark pull request #14046: [SPARK-16366][SPARKR] Fix time comparison failure...

2016-07-04 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/14046#discussion_r69486820 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -1258,10 +1258,12 @@ test_that("date functions on a DataFrame", { df2 <- cr

[GitHub] spark issue #14019: [SPARK-16233][R][TEST] ORC test should be enabled only w...

2016-07-01 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14019 @dongjoon-hyun I merged this into branch-2.0, so it should be present in the next RC. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #14019: [SPARK-16233][R][TEST] ORC test should be enabled only w...

2016-07-01 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/14019 Thanks @dongjoon-hyun - I manually tested this as well and it works fine. LGTM. Will merge after Jenkins passes --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #13984: [SPARK-16310][SPARKR] R na.string-like default fo...

2016-07-01 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13984#discussion_r69359023 --- Diff: R/pkg/R/SQLContext.R --- @@ -744,6 +746,9 @@ read.df.default <- function(path = NULL, source = NULL, schema = NULL, ...) { if (is.n

[GitHub] spark issue #13975: [SPARK-16299][SPARKR] Capture errors from R workers in d...

2016-07-01 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13975 Thanks @sun-rui -- I just tested this and this fixes the issue with the `pipeRDD` unit test on my machine. LGTM. Merging this to master, branch-2.0 --- If your project is set up for it, you

[GitHub] spark issue #13760: [SPARK-16012][SparkR] Implement gapplyCollect which will...

2016-07-01 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13760 Thanks all. LGTM. Merging this to master and branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13975: [SPARK-16299][SPARKR] Capture errors from R workers in d...

2016-06-29 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13975 @sun-rui This is a great find ! I will test this locally as well cc @JoshRosen who was running into the same problem with pipeRDD tests --- If your project is set up for it, you can

[GitHub] spark issue #13904: [SPARKR] add csv tests

2016-06-28 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13904 Sure - LGTM. Merging this to master, branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #13946: [MINOR][SparkR] update sparkR DataFrame.R comment

2016-06-28 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13946 merging this to master and branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13760: [SPARK-16012][SparkR] implement gapplyCollect whi...

2016-06-26 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13760#discussion_r68508656 --- Diff: R/pkg/R/group.R --- @@ -243,17 +236,73 @@ setMethod("gapply", signature(x = "GroupedData"), func

[GitHub] spark issue #13885: [SPARK-16184][SPARKR] conf API for SparkSession

2016-06-26 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13885 LGTM. Thanks @felixcheung - Merging this into master and branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #13885: [SPARK-16184][SPARKR] conf API for SparkSession

2016-06-25 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13885#discussion_r68491306 --- Diff: R/pkg/R/SQLContext.R --- @@ -110,11 +110,46 @@ infer_type <- function(x) { } } -getDefaultSqlSource <- fu

[GitHub] spark issue #13760: [SPARK-16012][SparkR] implement gapplyCollect which will...

2016-06-24 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13760 @felixcheung Any other comments on this ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13839: [SPARK-16128][SQL] Allow setting length of charac...

2016-06-24 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13839#discussion_r68480005 --- Diff: R/pkg/R/DataFrame.R --- @@ -194,7 +194,12 @@ setMethod("isLocal", setMethod("showDF", signature

[GitHub] spark pull request #13839: [SPARK-16128][SQL] Allow setting length of charac...

2016-06-24 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13839#discussion_r68433503 --- Diff: R/pkg/R/DataFrame.R --- @@ -194,7 +194,12 @@ setMethod("isLocal", setMethod("showDF", signature

[GitHub] spark pull request #13885: [SPARK-16184][SPARKR] conf API for SparkSession

2016-06-24 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13885#discussion_r68423467 --- Diff: R/pkg/NAMESPACE --- @@ -10,6 +10,7 @@ export("sparkR.session") export("sparkR.init") export("

[GitHub] spark issue #13839: [SPARK-16128][SQL] Allow setting length of characters to...

2016-06-24 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13839 Thanks @ScrapCodes -- R changes LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13877: [SPARK-16142] [R] group naiveBayes method docs in a sing...

2016-06-23 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13877 The new document in the screenshot looks pretty good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #13877: [SPARK-16142] [R] group naiveBayes method docs in...

2016-06-23 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13877#discussion_r68352034 --- Diff: R/pkg/R/mllib.R --- @@ -390,23 +376,41 @@ setMethod("predict", signature(object = "KMeansModel"), return(d

[GitHub] spark pull request #13839: [SPARK-16128][SQL] Allow setting length of charac...

2016-06-23 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13839#discussion_r68317660 --- Diff: R/pkg/R/DataFrame.R --- @@ -194,7 +194,12 @@ setMethod("isLocal", setMethod("showDF", signature

[GitHub] spark pull request #13839: [SPARK-16128][SQL] Allow setting length of charac...

2016-06-23 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13839#discussion_r68317336 --- Diff: R/pkg/R/DataFrame.R --- @@ -177,8 +177,8 @@ setMethod("isLocal", #' @param x A SparkDataFrame #' @param numRows The numb

[GitHub] spark issue #13838: [SPARK-16088][SPARKR] update setJobGroup, cancelJobGroup...

2016-06-23 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13838 LGTM. Merging this to master, branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #13760: [SPARK-16012][SparkR] implement gapplyCollect whi...

2016-06-23 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13760#discussion_r68268602 --- Diff: R/pkg/R/group.R --- @@ -199,17 +199,10 @@ createMethods() #' Applies a R function to each group in the input GroupedData

[GitHub] spark issue #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-22 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13660 @felixcheung @jkbradley any more comments on this ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #13760: [SPARK-16012][SparkR] gapplyCollect - applies a R...

2016-06-22 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13760#discussion_r68090167 --- Diff: R/pkg/R/group.R --- @@ -199,17 +199,10 @@ createMethods() #' Applies a R function to each group in the input GroupedData

[GitHub] spark pull request #13838: [SPARK-16088][SPARKR] update setJobGroup, cancelJ...

2016-06-22 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13838#discussion_r68089154 --- Diff: R/pkg/R/sparkR.R --- @@ -392,47 +392,81 @@ sparkR.session <- function( #' Assigns a group ID to all the jobs started by this thread un

[GitHub] spark pull request #13760: [SPARK-16012][SparkR] gapplyCollect - applies a R...

2016-06-22 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13760#discussion_r68087556 --- Diff: R/pkg/R/group.R --- @@ -242,18 +235,73 @@ createMethods() setMethod("gapply", signature(x = &q

[GitHub] spark pull request #13838: [SPARK-16088][SPARKR] update setJobGroup, cancelJ...

2016-06-22 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13838#discussion_r68086736 --- Diff: R/pkg/R/sparkR.R --- @@ -392,47 +392,81 @@ sparkR.session <- function( #' Assigns a group ID to all the jobs started by this thread un

[GitHub] spark issue #13803: [SPARK-16109][SPARKR][DOC] R more doc fixes

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13803 Irrespective of people reading the guide, it'll at least be useful to point out what is the expected behavior in a code review etc. But yeah automatic style checks would be really cool

[GitHub] spark issue #13820: [SPARK-16107] [R] group glm methods in documentation

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13820 I think the `#D` comes from using `dontrun` https://github.com/wch/r-source/blob/e5b21d0397c607883ff25cca379687b86933d730/src/library/tools/R/Rd2ex.R#L72 I don't see an easy way to disable

[GitHub] spark issue #13584: [SPARK-15509][ML][SparkR] R MLlib algorithms should supp...

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13584 cc @mengxr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13805: [SPARK-16096][SPARKR] add union and deprecate unionAll

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13805 Alright - yeah lets leave `explode` as is for now. LGTM. Merging this to master, branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #13803: [SPARK-16109][SPARKR][DOC] R more doc fixes

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13803 Yeah - thanks for all the work in cleaning this up. One thing that i was wondering is we could add some style guide checks or contribution guide rules on how to maintain documentation. We can

[GitHub] spark pull request #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-21 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13660#discussion_r67920626 --- Diff: docs/sparkr.md --- @@ -262,6 +262,83 @@ head(df) {% endhighlight %} +### Applying User-defined Function +In SparkR, we

[GitHub] spark issue #13803: [SPARK-16109][SPARKR][DOC] R more doc fixes

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13803 LGTM. Merging this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13805: [SPARK-16096][SPARKR] add union and deprecate unionAll

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13805 Just wondering -- is there a list of SparkSQL deprecations for 2.0.0 ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #13803: [SPARK-16109][SPARKR][DOC] R more doc fixes

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13803 Thanks - rebuilt locally and the docs look good. Will merge after Jenkins passes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark issue #13805: [SPARK-16096][SPARKR] add union and deprecate unionAll

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13805 cc @liancheng Code changes look good to me --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #13803: [SPARK-16109][SPARKR][DOC] R more doc fixes

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13803 Other than the comment about the `family` link the rest of the changes look good --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request #13803: [SPARKR][DOC] R more doc fixes

2016-06-21 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13803#discussion_r67911376 --- Diff: R/pkg/R/stats.R --- @@ -33,7 +32,7 @@ setOldClass("jobj") #' of `col2`. The name of the first column will be `$col1_$col

[GitHub] spark issue #13803: [SPARKR][DOC] R more doc fixes

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13803 @felixcheung Can you change the JIRA in the title to https://issues.apache.org/jira/browse/SPARK-16109 ? I created a sub-task for statfunctions --- If your project is set up for it, you can

[GitHub] spark issue #13803: [SPARKR][DOC] R more doc fixes

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13803 cc @mengxr --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-21 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13660#discussion_r67910379 --- Diff: docs/sparkr.md --- @@ -262,6 +262,83 @@ head(df) {% endhighlight %} +### Applying User-defined Function +In SparkR, we

[GitHub] spark issue #13790: [SPARK-16082][SparkR]remove duplicated docs in dapply

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13790 Thanks -- I should have noticed it before the merge, but missed it. I have resolved the JIRA and put in a link to the PR there, so I think its all fine. --- If your project is set up for it, you

[GitHub] spark issue #13109: [SPARK-15319][SPARKR][DOCS] Fix SparkR doc layout for co...

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13109 Cool. Thanks - LGTM. Merging this to master, branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #13801: [SPARK-15177.1] [R] make SparkR model params and ...

2016-06-21 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13801#discussion_r67818449 --- Diff: R/pkg/R/mllib.R --- @@ -298,17 +296,17 @@ setMethod("summary", signature(object = "NaiveBayesModel"), #' @expo

[GitHub] spark issue #13801: [SPARK-15177.1] [R] make SparkR model params and default...

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13801 Changes look fine given what was a part of #13023 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #13801: [SPARK-15177.1] [R] make SparkR model params and ...

2016-06-21 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13801#discussion_r67818028 --- Diff: R/pkg/R/mllib.R --- @@ -298,17 +296,17 @@ setMethod("summary", signature(object = "NaiveBayesModel"), #' @expo

[GitHub] spark issue #13798: [SPARKR][DOCS] R code doc cleanup

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13798 Alright I'm merging this to master and branch-2.0 so that it makes it to the RC. We can try and fix minor things going forward Thanks @felixcheung -- This is a much needed clean up

[GitHub] spark issue #13109: [SPARK-15319][SPARKR][DOCS] Fix SparkR doc layout for co...

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13109 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13798: [SPARKR][DOCS] R code doc cleanup

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13798 Thanks for the update. This one looks fine to me now. @dongjoon-hyun Any other comments ? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #13109: [SPARK-15319][SPARKR][DOCS] Fix SparkR doc layout for co...

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13109 @mengxr @felixcheung Can we open a new issue of the form `Separate out rd files for SparkR functions` ? We can then make a list there of everything thats sharing a rd file right now and see what

[GitHub] spark issue #13109: [SPARK-15319][SPARKR][DOCS] Fix SparkR doc layout for co...

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13109 @mengxr Yes - this is true and in #13798 we are making a few more of the methods into individual Rd files. At a high level there is a tradition in R to group together similar methods (https

[GitHub] spark issue #13109: [SPARK-15319][SPARKR][DOCS] Fix SparkR doc layout for co...

2016-06-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13109 LGTM. This version looks good to me. Thanks for iterating on this. Will wait for Jenkins and then merge. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #13109: [SPARK-15319][SPARKR][DOCS] Fix SparkR doc layout...

2016-06-20 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13109#discussion_r67811551 --- Diff: R/pkg/R/stats.R --- @@ -134,9 +129,7 @@ setMethod("freqItems", signature(x = "SparkDataFrame", cols = "character&qu

[GitHub] spark issue #13295: [SPARK-15294][SPARKR][MINOR] Add pivot functionality to ...

2016-06-20 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13295 @mhnatiuk Given that #13786 was merged, can you close this PR ? Only the PR authors have permission to close a PR in the Spark project --- If your project is set up for it, you can reply

[GitHub] spark pull request #13798: [SPARKR][DOCS] R code doc cleanup

2016-06-20 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13798#discussion_r67810125 --- Diff: R/pkg/R/generics.R --- @@ -689,67 +689,67 @@ setGeneric("randomSplit", function(x, weights, seed) { standardGeneric

[GitHub] spark issue #13799: [SPARK-15863][SQL][DOC][SPARKR] sql programming guide up...

2016-06-20 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13799 Thanks LGTM. Will wait for @liancheng to also take a look --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #13798: [SPARKR][DOCS] R code doc cleanup

2016-06-20 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13798 Yeah lets do that in a separate PR with discussion ? We don't lose much by leaving in a few extra functions that we deprecate / remove later. Thanks for the doc cleanup - I'll take one

[GitHub] spark issue #13798: [SPARK-16088][SPARKR][DOCS] Remove setJobGroup, clearJob...

2016-06-20 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13798 Hmm can we document somewhere why we want to remove this functionality ? Is this because spark context is no longer accessible ? --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #13109: [SPARK-15319][SPARKR][DOCS] Fix SparkR doc layout...

2016-06-20 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13109#discussion_r67806444 --- Diff: R/pkg/R/stats.R --- @@ -19,7 +19,8 @@ setOldClass("jobj") -#' crosstab +#' @title SparkDataFrame statistic

[GitHub] spark issue #13786: [SPARK-15294][R] Add `pivot` to SparkR

2016-06-20 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13786 LGTM. Merging this to master, branch-2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13109: [SPARK-15319][SPARKR][DOCS] Fix SparkR doc layout for co...

2016-06-20 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13109 cc @dongjoon-hyun --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #13790: remove duplicated docs in dapply

2016-06-20 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/13790 LGTM -- @felixcheung @sun-rui let me know if you have any comments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-20 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13660#discussion_r67772111 --- Diff: docs/sparkr.md --- @@ -262,6 +262,79 @@ head(df) {% endhighlight %} +### Applying User-defined Function +In SparkR, we

[GitHub] spark pull request #13660: [SPARK-15672][R][DOC] R programming guide update

2016-06-20 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/13660#discussion_r67771773 --- Diff: docs/sparkr.md --- @@ -262,6 +262,79 @@ head(df) {% endhighlight %} +### Applying User-defined Function +In SparkR, we

<    3   4   5   6   7   8   9   10   11   12   >