Joseph K. Bradley created SPARK-7432: ----------------------------------------
Summary: Flaky test in PySpark CrossValidator doc test Key: SPARK-7432 URL: https://issues.apache.org/jira/browse/SPARK-7432 Project: Spark Issue Type: Bug Components: ML, PySpark Affects Versions: 1.4.0 Reporter: Joseph K. Bradley Priority: Critical There was a test failure in the doc test in Python CrossValidator: [https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32058/consoleFull] Here's the full doc test: {code} >>> from pyspark.ml.classification import LogisticRegression >>> from pyspark.ml.evaluation import BinaryClassificationEvaluator >>> from pyspark.mllib.linalg import Vectors >>> dataset = sqlContext.createDataFrame( ... [(Vectors.dense([0.0, 1.0]), 0.0), ... (Vectors.dense([1.0, 2.0]), 1.0), ... (Vectors.dense([0.55, 3.0]), 0.0), ... (Vectors.dense([0.45, 4.0]), 1.0), ... (Vectors.dense([0.51, 5.0]), 1.0)] * 10, ... ["features", "label"]) >>> lr = LogisticRegression() >>> grid = ParamGridBuilder().addGrid(lr.maxIter, [0, 1, 5]).build() >>> evaluator = BinaryClassificationEvaluator() >>> cv = CrossValidator(estimator=lr, estimatorParamMaps=grid, evaluator=evaluator) >>> cvModel = cv.fit(dataset) >>> expected = lr.fit(dataset, {lr.maxIter: 5}).transform(dataset) >>> cvModel.transform(dataset).collect() == expected.collect() True {code} Here's the failure message: {code} Running test: pyspark/ml/tuning.py ... ********************************************************************** File "pyspark/ml/tuning.py", line 108, in __main__.CrossValidator Failed example: cvModel.transform(dataset).collect() == expected.collect() Expected: True Got: False ********************************************************************** 1 of 11 in __main__.CrossValidator ***Test Failed*** 1 failures. Had test failures; see logs. [error] Got a return code of 255 on line 240 of the run-tests script. {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org