[GitHub] spark pull request #23200: [SPARK-26033][SPARK-26034][PYTHON][FOLLOW-UP] Sma...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23200 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23200: [SPARK-26033][SPARK-26034][PYTHON][FOLLOW-UP] Sma...
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/23200#discussion_r238454041 --- Diff: python/pyspark/mllib/tests/test_linalg.py --- @@ -22,33 +22,18 @@ from numpy import array, array_equal, zeros, arange, tile, ones, inf import pyspark.ml.linalg as newlinalg +from pyspark.serializers import PickleSerializer from pyspark.mllib.linalg import Vector, SparseVector, DenseVector, VectorUDT, _convert_to_vector, \ DenseMatrix, SparseMatrix, Vectors, Matrices, MatrixUDT from pyspark.mllib.regression import LabeledPoint -from pyspark.testing.mllibutils import make_serializer, MLlibTestCase - -_have_scipy = False -try: -import scipy.sparse -_have_scipy = True -except: -# No SciPy, but that's okay, we'll skip those tests -pass - - -ser = make_serializer() - - -def _squared_distance(a, b): -if isinstance(a, Vector): -return a.squared_distance(b) -else: -return b.squared_distance(a) +from pyspark.testing.mllibutils import MLlibTestCase +from pyspark.testing.utils import have_scipy --- End diff -- Oh that's good, didn't realize have_scipy was there --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #23200: [SPARK-26033][SPARK-26034][PYTHON][FOLLOW-UP] Sma...
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/23200 [SPARK-26033][SPARK-26034][PYTHON][FOLLOW-UP] Small cleanup and deduplication in ml/mllib tests ## What changes were proposed in this pull request? This PR is a small follow up that puts some logic and functions into smaller scope and make it localized, and deduplicate. ## How was this patch tested? Manually tested. Jenkins tests as well. You can merge this pull request into a Git repository by running: $ git pull https://github.com/HyukjinKwon/spark followup-SPARK-26034-SPARK-26033 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/23200.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #23200 commit cbe13bf2fdc99f033746f5f3ff67c73fe7f27200 Author: Hyukjin Kwon Date: 2018-12-02T10:00:18Z Small cleanup and deduplication --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
