[ https://issues.apache.org/jira/browse/SPARK-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joseph K. Bradley updated SPARK-7432: ------------------------------------- Assignee: Xiangrui Meng > Flaky test in PySpark CrossValidator doc test > --------------------------------------------- > > Key: SPARK-7432 > URL: https://issues.apache.org/jira/browse/SPARK-7432 > Project: Spark > Issue Type: Bug > Components: ML, PySpark > Affects Versions: 1.4.0 > Reporter: Joseph K. Bradley > Assignee: Xiangrui Meng > Priority: Critical > > There was a test failure in the doc test in Python CrossValidator: > [https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32058/consoleFull] > Here's the full doc test: > {code} > >>> from pyspark.ml.classification import LogisticRegression > >>> from pyspark.ml.evaluation import BinaryClassificationEvaluator > >>> from pyspark.mllib.linalg import Vectors > >>> dataset = sqlContext.createDataFrame( > ... [(Vectors.dense([0.0, 1.0]), 0.0), > ... (Vectors.dense([1.0, 2.0]), 1.0), > ... (Vectors.dense([0.55, 3.0]), 0.0), > ... (Vectors.dense([0.45, 4.0]), 1.0), > ... (Vectors.dense([0.51, 5.0]), 1.0)] * 10, > ... ["features", "label"]) > >>> lr = LogisticRegression() > >>> grid = ParamGridBuilder().addGrid(lr.maxIter, [0, 1, 5]).build() > >>> evaluator = BinaryClassificationEvaluator() > >>> cv = CrossValidator(estimator=lr, estimatorParamMaps=grid, > evaluator=evaluator) > >>> cvModel = cv.fit(dataset) > >>> expected = lr.fit(dataset, {lr.maxIter: 5}).transform(dataset) > >>> cvModel.transform(dataset).collect() == expected.collect() > True > {code} > Here's the failure message: > {code} > Running test: pyspark/ml/tuning.py ... > ********************************************************************** > File "pyspark/ml/tuning.py", line 108, in __main__.CrossValidator > Failed example: > cvModel.transform(dataset).collect() == expected.collect() > Expected: > True > Got: > False > ********************************************************************** > 1 of 11 in __main__.CrossValidator > ***Test Failed*** 1 failures. > Had test failures; see logs. > [error] Got a return code of 255 on line 240 of the run-tests script. > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org