[GitHub] spark pull request #16465: [SPARK-19064][PySpark]Fix pip installing of sub c...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16465 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16465: [SPARK-19064][PySpark]Fix pip installing of sub c...
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16465#discussion_r97710172 --- Diff: dev/pip-sanity-check.py --- @@ -18,6 +18,8 @@ from __future__ import print_function from pyspark.sql import SparkSession +from pyspark.ml.param import Params +from pyspark.mllib.linalg import * --- End diff -- ok. i think this should be enough. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16465: [SPARK-19064][PySpark]Fix pip installing of sub c...
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/16465#discussion_r94707290 --- Diff: dev/pip-sanity-check.py --- @@ -18,6 +18,8 @@ from __future__ import print_function from pyspark.sql import SparkSession +from pyspark.ml.param import Params +from pyspark.mllib.linalg import * --- End diff -- This just checks one sub component from each, we could import each with rename I suppose but not sure it would do much? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16465: [SPARK-19064][PySpark]Fix pip installing of sub c...
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16465#discussion_r94707156 --- Diff: dev/pip-sanity-check.py --- @@ -18,6 +18,8 @@ from __future__ import print_function from pyspark.sql import SparkSession +from pyspark.ml.param import Params +from pyspark.mllib.linalg import * --- End diff -- Is it better to import `pyspark.ml.linalg` or both? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #16465: [SPARK-19064][PySpark]Fix pip installing of sub c...
GitHub user holdenk opened a pull request: https://github.com/apache/spark/pull/16465 [SPARK-19064][PySpark]Fix pip installing of sub components ## What changes were proposed in this pull request? Fix instalation of mllib and ml sub components, and more eagerly cleanup cache files during test script & make-distribution. ## How was this patch tested? Updated sanity test script to import mllib and ml sub-components. You can merge this pull request into a Git repository by running: $ git pull https://github.com/holdenk/spark SPARK-19064-fix-pip-install-sub-components Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/16465.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #16465 commit 21e224a1d94c65814e553935f88ca65cb3a2ee69 Author: Holden Karau Date: 2017-01-04T00:33:51Z Fix pip installing of sub components, and update the test and make-distribution scripts to be more explicit about cleanup. Also add pypandoc to dev requirements file since we want it for publishing commit b28d9ca5e553e453b34d6199549d845ff5b6e1e2 Author: Holden Karau Date: 2017-01-04T01:06:24Z Install numpy so we can check the mllib linalg imports --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
