[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58251926 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread Ishiihara
Github user Ishiihara commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58252347 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58254457 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58255069 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/279/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58257811 @Ishiihara Could you try to merge master? Maybe the python doc conf changed. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58267109 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/279/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread Ishiihara
Github user Ishiihara commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58270419 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58270752 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21408/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58270916 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58270914 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21408/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread Ishiihara
Github user Ishiihara commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58271152 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread Ishiihara
Github user Ishiihara commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58271779 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58271977 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21411/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58272541 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21412/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58278852 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58278879 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58278875 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21412/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58278846 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21411/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/2356 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-07 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58281398 LGTM. Merged into master. Thanks! I created a JIRA to remember add Python code example to the user guide: https://issues.apache.org/jira/browse/SPARK-3838 . Not a high

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18486326 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,58 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18486384 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,192 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18486359 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,192 @@ +# --- End diff -- Please rename the file to `feature.py` to make `Word2Vec`

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18486387 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,192 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18486381 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,192 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18486395 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,192 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18486413 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,192 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18486524 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,192 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18486706 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,58 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18486738 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,58 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58118924 @Ishiihara Another file to update is `python/docs/pyspark.mllib.rst`. We need a section for `pyspark.mllib.feature` module. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-06 Thread Ishiihara
Github user Ishiihara commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-58119086 @mengxr will take care of that and other comments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57881048 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21279/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57888189 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21279/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-10-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57888194 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18184054 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,54 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18184065 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,54 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18184061 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,54 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18184091 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,124 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18184098 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,151 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18184093 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,151 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18184062 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,54 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18184088 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,151 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18184057 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,54 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-29 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18184121 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,151 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18122598 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,124 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18122597 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,124 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread Ishiihara
Github user Ishiihara commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57046286 @mengxr Repartition is very slow when caching at Python side. It takes 9 minutes to do the repartition where as caching in Java only takes 5s. --- If your project is

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57046312 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20915/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57047727 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20915/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57047728 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18117490 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18117593 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,123 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18117584 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18117604 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18117608 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18117647 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,123 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18118109 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread Ishiihara
Github user Ishiihara commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18120761 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57037893 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20894/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57039639 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20894/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-57039641 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18121798 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,124 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18121803 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,124 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18121806 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,42 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-26 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18121812 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,124 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread Ishiihara
Github user Ishiihara commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-56869195 @mengxr PR updated to use new pickle SerDe. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-56869584 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20816/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-56878764 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20816/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-56878776 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18061848 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18061937 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,123 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-56888002 Could you add some tests? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18063655 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -40,11 +40,12 @@ import org.apache.spark.mllib.tree.impurity._

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18063657 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18063661 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18063656 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18063664 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/PythonMLLibAPI.scala --- @@ -284,6 +285,80 @@ class PythonMLLibAPI extends Serializable {

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18063706 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,123 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-25 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/2356#discussion_r18063701 --- Diff: python/pyspark/mllib/Word2Vec.py --- @@ -0,0 +1,123 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-22 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-56420439 Now that #2378 has been merged, is this unblocked? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-22 Thread Ishiihara
Github user Ishiihara commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-56420682 We need to modify the implementation to use the new SerDe mechanism. Working on that now. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-15 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-55684719 @davies Thanks for working on MLlib's SerDe! It definitely simplifies future Python API implementations. We will wait #2378 . --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-12 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-55375085 @davies Could you take a look at this PR and see whether there is an easier way for SerDe? Thanks! --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-12 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-55425713 @mengxr I'm looking into this, could we block this a few days until we find out the scalable way to do serialization? --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-55299387 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20163/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-55308654 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20163/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-11 Thread Ishiihara
GitHub user Ishiihara opened a pull request: https://github.com/apache/spark/pull/2356 [SPARK-3486][MLlib][PySpark] PySpark support for Word2Vec @mengxr Added PySpark support for Word2Vec Change list (1) PySpark support for Word2Vec (2) SerDe support of string

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-55248249 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20148/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-55248343 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20148/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-55253328 [QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20153/consoleFull) for PR 2356 at commit

[GitHub] spark pull request: [SPARK-3486][MLlib][PySpark] PySpark support f...

2014-09-11 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2356#issuecomment-55258752 [QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/20153/consoleFull) for PR 2356 at commit