[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-62468718 Merged into master and branch-1.2. Thanks! (We should find some time and clean really old examples.) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-62313396 @varadharajan Good suggestion about documenting algs for LR; I'll make a note to do that for the upcoming release. Thank you for the PR! LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user varadharajan commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-62350096 @jkbradley Thanks :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user varadharajan commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-62259256 @jkbradley Makes sense. I've updated the warnings, please let me know if wordings can be improved. Also i just noticed that pyspark classification model does not have LR-LBFGS implementation. I'll probably create a new issue and work on it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-62259296 [Test build #23102 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23102/consoleFull) for PR 2894 at commit [`5f9406b`](https://github.com/apache/spark/commit/5f9406b858ae1c80f228bd77d22bb1eebc6b1200). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user varadharajan commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-62259405 Also i think it would help users if we can document in the LR section of the MLlib guide, which algorithm should be preferred in which scenarios. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-62262246 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23102/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-62262242 [Test build #23102 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23102/consoleFull) for PR 2894 at commit [`5f9406b`](https://github.com/apache/spark/commit/5f9406b858ae1c80f228bd77d22bb1eebc6b1200). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-62223607 Since this is MLlib related, @mengxr or @jkbradley, could one of you do the final sign-off + commit on this? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-62226699 @varadharajan Thanks for adding the warnings! My main comment is that LogisticRegressionModel is a model, rather than an algorithm. Users would really want the algorithm which they can run to produce the model. Could you instead direct users to the algorithms: LogisticRegressionWithSGD and LogisticRegressionWithLBFGS? (It is awkward that there are 2 algorithms to direct users towards, but it is hard to get around that.) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-61563547 LGTM, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user varadharajan commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-61589586 Thanks :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-61027290 Jenkins, ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-61027638 [Test build #22498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22498/consoleFull) for PR 2894 at commit [`252f595`](https://github.com/apache/spark/commit/252f595eece96771e385aab1ffce7031097b2c1a). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-61032900 [Test build #22498 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/22498/consoleFull) for PR 2894 at commit [`252f595`](https://github.com/apache/spark/commit/252f595eece96771e385aab1ffce7031097b2c1a). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-61032904 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/22498/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-60394239 Are there other examples that should have the same warning? I think there are many more than this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user varadharajan commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-60408059 Here are list of scala examples that i think is similar / naive implementation of algorithms from MLlib or graphx. 1. LocalALS 2. LocalFileLR 3. LocalKMeans 4. LocalLR 5. SparkALS 6. SparkHdfsLR 7. SparkKMeans 8. SparkLR 9. SparkPageRank (*) 10. SparkTachyonHdfsLR (*) Python examples: 1. ALS 2. kmeans 3. logistic_regression 4. pagerank (*) Java examples: 1. JavaHdfsLR (*) 2. JavaPageRank (*) * - Examples with missing warnings. I've updated JIRA with these details and also added warning for them I've also corrected class names of existing LR examples. They were pointing to org.apache.spark.mllib.classification.LogisticRegression instead of org.apache.spark.mllib.classification.LogisticRegressionModel I've excluded examples that compute transitive closures on graphs because i'm was not able to find corresponding implementations in graphx. Please let me know if i'm missing something --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
GitHub user varadharajan opened a pull request: https://github.com/apache/spark/pull/2894 [SPARK-4047] - Generate runtime warnings for example implementation of PageRank Based on SPARK-2434, this PR generates runtime warnings for example implementations (Python, Scala) of PageRank. You can merge this pull request into a Git repository by running: $ git pull https://github.com/varadharajan/spark SPARK-4047 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/2894.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2894 commit 5c2bf5453834ad406a62d9e95d8ce128a65a2680 Author: Varadharajan Mukundan srinath...@gmail.com Date: 2014-10-22T16:47:59Z [SPARK-4047] - Generate runtime warnings for example implementation of PageRank --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-4047] - Generate runtime warnings for e...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2894#issuecomment-60116178 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
