[GitHub] spark pull request #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in ...
Github user krishnakalyan3 closed the pull request at: https://github.com/apache/spark/pull/14741 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16767: [SPARK-19386][SPARKR][DOC] Bisecting k-means in SparkR d...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/16767 ping @wangmiao1981 @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16767: [SPARK-19386][SPARKR][DOC] Bisecting k-means in SparkR d...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/16767 @wangmiao1981 sorry, had made a erroneous commit. Could you please review the PR?. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16767: [SPARK-19386][SPARKR][DOC] Bisecting k-means in S...
Github user krishnakalyan3 commented on a diff in the pull request: https://github.com/apache/spark/pull/16767#discussion_r98974015 --- Diff: R/pkg/vignettes/sparkr-vignettes.Rmd --- @@ -494,6 +494,8 @@ SparkR supports the following machine learning models and algorithms. * Latent Dirichlet Allocation (LDA) +* Bisecting $k$-means --- End diff -- @felixcheung could you please let me know whats wrong here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16767: [SPARK-19386][SPARKR][DOC]
GitHub user krishnakalyan3 opened a pull request: https://github.com/apache/spark/pull/16767 [SPARK-19386][SPARKR][DOC] ## What changes were proposed in this pull request? Update programming guide, example and vignette with Bisecting k-means. You can merge this pull request into a Git repository by running: $ git pull https://github.com/krishnakalyan3/spark bisecting-kmeans Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/16767.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #16767 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14741 ping @shivaram @davies, I am planning to revisit this PR. Could you please let me know which daemon process on Linux we are trying to interrupt. I am assuming its the R process?. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16242: [SPARK-18628][ML] Update Scala param and Python param to...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/16242 cc @MLnick and @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16242: [SPARK-18628][ML] Update Scala param and Python p...
GitHub user krishnakalyan3 opened a pull request: https://github.com/apache/spark/pull/16242 [SPARK-18628][ML] Update Scala param and Python param to have quotes ## What changes were proposed in this pull request? Updated Scala param and Python param to have quotes around the options making it easier for users to read. ## How was this patch tested? Manually checked the docstrings You can merge this pull request into a Git repository by running: $ git pull https://github.com/krishnakalyan3/spark doc-string Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/16242.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #16242 commit f5ffee7e970297347c25d972d820743d7a72b9f3 Author: krishnakalyan3 <krishnakaly...@gmail.com> Date: 2016-12-10T00:30:45Z add quotes around options --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #15755: [SPARK-15902][PySpark] Add deprecation warning if...
Github user krishnakalyan3 closed the pull request at: https://github.com/apache/spark/pull/15755 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #15755: [SPARK-15902][PySpark] Add deprecation warning if python...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/15755 @srowen will do. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #15755: [SPARK-15902][PySpark] Add deprecation warning if...
GitHub user krishnakalyan3 opened a pull request: https://github.com/apache/spark/pull/15755 [SPARK-15902][PySpark] Add deprecation warning if python version below Python 2.7 ## What changes were proposed in this pull request? Deprecation warning if we detect we are running in Python 2.6. (Please fill in changes proposed in this fix) ## How was this patch tested? locally using different python profiles. You can merge this pull request into a Git repository by running: $ git pull https://github.com/krishnakalyan3/spark warn-python Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/15755.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #15755 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR
Github user krishnakalyan3 commented on the issue:
https://github.com/apache/spark/pull/14741
@shivaram thanks for the advice.
Some Issue being faced by me
- While reading a large file from Rstudio and trying to kill the the
process using `Sys.getpid()`, I tried to interrupt the process using the
signals `pskill(pid, signal = SIGUSR1/SIGCHLD)`. This does not seem to affect
my R session and does not print `Interrupt` (As per my code below).
```
readBinFully <- function(con, what, n = 1L, size = NA_integer_, endian) {
while (n > 0) {
if (con == 0) {
cat("Interrupt")
}
readBin(con, what, n, size, endian = "big")
}
}
```
- `sparkr.zip` obtained after running `install-dev.sh` does not seem to
reflect the changes made in my R session. (Restarting R studio solves this
problem). Code below
```
rm(list=ls())
Sys.setenv(SPARK_HOME="/Users/krishna/Experiment/spark")
.libPaths(c(file.path(Sys.getenv("SPARK_HOME"), "R", "lib"), .libPaths()))
library(SparkR)
```
- To check if there are more bytes to read, I have tried the code below.
Which fails the tests in `run-tests.sh`
```
while (size > 0) {
...
}
```
I see that variable size takes the value `NA`.
Please advice on how I should be approaching these issues.
Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14741 @shivaram @davies - Signature of the readBin function is `readBin(con, what, n, as.integer(size), endian)` What should the value of `what` be when an the process is interrupted in the retry method?. - I am also having problems simulating this Issue. As soon as I send the `kill -9 PID`. My R session restarts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14741 @shivaram I am not sure on how to go about the retry method. Could you please share some example that I could refer to?. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14741: [SPARK-6832][SPARKR][WIP]Handle partial reads in ...
GitHub user krishnakalyan3 opened a pull request: https://github.com/apache/spark/pull/14741 [SPARK-6832][SPARKR][WIP]Handle partial reads in SparkR ## What changes were proposed in this pull request? Handle partial reads in SparkR by implementing a retry method in R that will return partial results. ## How was this patch tested? Locally by running the R test suite. (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) You can merge this pull request into a Git repository by running: $ git pull https://github.com/krishnakalyan3/spark partial-reads Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14741.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14741 commit 82999118699c66363413cf5234b1081028f6b018 Author: krishnakalyan3 <krishnakaly...@gmail.com> Date: 2016-08-21T20:52:14Z readBinFully implementation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/13894 @MLnick @holdenk @jkbradley thanks for the reviews. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/13894 cc @MLnick @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14179: [SPARK-16055][SPARKR] warning added while using sparkPac...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14179 @shivaram @felixcheung thanks for the reviews. Will keep the feedbacks in mind. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14179: [SPARK-16055][SPARKR] warning added while using sparkPac...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14179 @felixcheung @shivaram Is the current state okay? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14179: [SPARK-16055][SPARKR] warning added while using s...
Github user krishnakalyan3 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14179#discussion_r71077269
--- Diff: R/pkg/R/sparkR.R ---
@@ -155,6 +155,10 @@ sparkR.sparkContext <- function(
existingPort <- Sys.getenv("EXISTING_SPARKR_BACKEND_PORT", "")
if (existingPort != "") {
+if (length(sparkPackages) != 0) {
--- End diff --
@shivaram yes you are right, thanks. @felixcheung will make the change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14179: [SPARK-16055][SPARKR] warning added while using sparkPac...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14179 @felixcheung @shivaram I am not sure if the warning message is clear enough. I did the best I could with character limit of 100. I am not sure which SparkR unit tests fail from the logs below https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62410/console --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14179: [SPARK-16055][SPARKR] warning added while using sparkPac...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14179 @felixcheung my local unit test still fail, anyway thanks for the clarification. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14179: [SPARK-16055][SPARKR] warning added while using sparkPac...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14179 @shivaram @felixcheung My patch fails spark unit test. (./R/run-tests.sh) Logs https://gist.github.com/krishnakalyan3/6585a1007b731e82fede1b942ea00bec I am not sure how to go about resolving this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14179: [SPARK-16055][SPARKR] warning added while using s...
Github user krishnakalyan3 commented on a diff in the pull request:
https://github.com/apache/spark/pull/14179#discussion_r70899726
--- Diff: R/pkg/R/sparkR.R ---
@@ -155,6 +155,9 @@ sparkR.sparkContext <- function(
existingPort <- Sys.getenv("EXISTING_SPARKR_BACKEND_PORT", "")
if (existingPort != "") {
+if(sparkPackages != ""){
+warning("--packages flag should be used with with spark-submit")
--- End diff --
@shivaram @felixcheung how about something like `sparkPackages cannot be
used as an argument within sparkR.init please use the --packages flag while
using spark-submit or sparkR shell`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/13894 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #14179: [SPARK-16055][SPARKR] warning added while using sparkPac...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/14179 cc @shivaram --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #14179: [SPARK-16055][SPARKR] warning added while using s...
GitHub user krishnakalyan3 opened a pull request: https://github.com/apache/spark/pull/14179 [SPARK-16055][SPARKR] warning added while using sparkPackages with spark-submit ## What changes were proposed in this pull request? SPARK-16055 parkPackages - argument is passed and we detect that we are in the R script mode, we should print some warning like --packages flag should be used with with spark-submit ## How was this patch tested? In my system locally You can merge this pull request into a Git repository by running: $ git pull https://github.com/krishnakalyan3/spark spark-pkg Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14179.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14179 commit 3d5d49bc0362c2c0bd7faed7170f449b7c179807 Author: krishnakalyan3 <krishnakaly...@gmail.com> Date: 2016-07-13T08:53:15Z warning added while using sparkPackages with spark-submit --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/13894 cc @holdenk @MLnick @jkbradley. Does the current state look good?. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/13894 @holdenk @MLnick sorry for so many changes. Newbie here. Please let me know if the current state is okay?. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Vali...
Github user krishnakalyan3 commented on a diff in the pull request:
https://github.com/apache/spark/pull/13894#discussion_r69567260
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala ---
@@ -57,7 +57,7 @@ private[ml] trait CrossValidatorParams extends
ValidatorParams {
/**
* :: Experimental ::
* CrossValidator begins by splitting the dataset into a set of
non-overlapping randomly
- * partitioned folds which are used as separate training and test datasets
e.g., with k=3 folds,
+ * partitioned folds as separate training and test datasets e.g., with k=3
folds,
--- End diff --
@MLnick is the current description okay?.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/13894 @holdenk @MLnick is the current update okay? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/13894 Updated the doc based on the reviews. Thanks for the review comments @holdenk and @MLnick. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Vali...
Github user krishnakalyan3 commented on a diff in the pull request: https://github.com/apache/spark/pull/13894#discussion_r69103201 --- Diff: python/pyspark/ml/tuning.py --- @@ -266,7 +269,7 @@ class CrossValidatorModel(Model, ValidatorParams): """ .. note:: Experimental -Model from k-fold cross validation. +CrossValidatorModel model returns the best set of estimators for your model. --- End diff -- @holdenk thanks for the review, ran lint-python. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Vali...
Github user krishnakalyan3 commented on a diff in the pull request:
https://github.com/apache/spark/pull/13894#discussion_r69086192
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala ---
@@ -194,8 +194,7 @@ object CrossValidator extends
MLReadable[CrossValidator] {
/**
* :: Experimental ::
- * Pipelines facilitate model selection by making it easy to tune an
entire
- * Pipeline at once, rather than tuning each element in the Pipeline
separately.
+ * CrossValidatorModel model returns the best set of estimators for your
model.
--- End diff --
@holdenk I have updated doc. Could you please review it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Validation ...
Github user krishnakalyan3 commented on the issue: https://github.com/apache/spark/pull/13894 cc @holdenk --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #13894: [SPARK-15254][DOC] Improve ML pipeline Cross Vali...
GitHub user krishnakalyan3 opened a pull request: https://github.com/apache/spark/pull/13894 [SPARK-15254][DOC] Improve ML pipeline Cross Validation Scaladoc & PyDoc ## What changes were proposed in this pull request? Updated ML pipeline Cross Validation Scaladoc & PyDoc. ## How was this patch tested? (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) You can merge this pull request into a Git repository by running: $ git pull https://github.com/krishnakalyan3/spark kfold-cv Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/13894.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #13894 commit d7ffe7a442a6e29b5f3afdf32894d108eb04bb35 Author: krishnakalyan3 <krishnakaly...@gmail.com> Date: 2016-06-24T14:11:37Z Improve ML pipeline Cross Validation Scaladoc & PyDoc --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12071][Doc] Document the behaviour of N...
Github user krishnakalyan3 commented on the pull request: https://github.com/apache/spark/pull/13268#issuecomment-221126159 @holdenk @shivaram added the reverse conversion details. Please let me know if its okay. Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12071][Doc] Document the behaviour of N...
Github user krishnakalyan3 commented on the pull request: https://github.com/apache/spark/pull/13268#issuecomment-221124498 @holdenk @shivaram will add that. Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-12071][Doc] Document the behaviour of N...
GitHub user krishnakalyan3 opened a pull request: https://github.com/apache/spark/pull/13268 [SPARK-12071][Doc] Document the behaviour of NA in R ## What changes were proposed in this pull request? Under Upgrading From SparkR 1.5.x to 1.6.x section added the information, SparkSQL converts `NA` in R to `null`. ## How was this patch tested? Document update, no tests. You can merge this pull request into a Git repository by running: $ git pull https://github.com/krishnakalyan3/spark spark-12071-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/13268.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #13268 commit e5e136462cac039b9f11f8a3703a70fefdbd5756 Author: Krishna Kalyan <krishnakaly...@gmail.com> Date: 2016-05-23T23:00:10Z [SPARK-12071][Doc] Document the behaviour of NA in R Under Upgrading From SparkR 1.5.x to 1.6.x section added the information, SparkSQL converts `NA` in R to `null`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
