[
https://issues.apache.org/jira/browse/MAHOUT-1603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14089599#comment-14089599
]
ASF GitHub Bot commented on MAHOUT-1603:
----------------------------------------
Github user pferrel commented on the pull request:
https://github.com/apache/mahout/pull/40#issuecomment-51514939
made changes to use the test context in the driver and tests seem to
complete correctly up to the point they try to read the output file, which does
contain the correct results.
```
val indicatorLines = mahoutCtx.textFile(OutPath +
"/indicator-matrix/part-00000")
```
The part file is created in the driver using
```rdd.saveAsTextFile(dest)```. It seems like something was getting done before
by shutting down the context, maybe I need to close the output file(s) (not
sure how to do that since it's created inside the saveAsTextFile call)?
```
java.lang.NullPointerException
at
org.apache.spark.SparkContext.defaultParallelism(SparkContext.scala:1215)
at
org.apache.spark.SparkContext.defaultMinPartitions(SparkContext.scala:1222)
at
org.apache.spark.SparkContext.textFile$default$2(SparkContext.scala:456)
at
org.apache.mahout.drivers.ItemSimilarityDriverSuite$$anonfun$4.apply$mcV$sp(ItemSimilarityDriverSuite.scala:303)
```
> Tweaks for Spark 1.0.x
> -----------------------
>
> Key: MAHOUT-1603
> URL: https://issues.apache.org/jira/browse/MAHOUT-1603
> Project: Mahout
> Issue Type: Task
> Affects Versions: 0.9
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Fix For: 1.0
>
>
> Tweaks necessary current codebase on top of spark 1.0.x
--
This message was sent by Atlassian JIRA
(v6.2#6252)
