[ 
https://issues.apache.org/jira/browse/MAHOUT-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15903320#comment-15903320
 ] 

Pat Ferrel commented on MAHOUT-1951:
------------------------------------

A quick way to test this is:

1) get Spark and HDFS running locally in pseudo-cluster mode.
2) build the version of Mahout under test I use simply "mvn clean install 
-DskipTests"
3) "hdfs dfs -rm -r test-results" to removes any old results
4) run the script below and look for exceptions in the output, they will look 
like the above errors

#!/usr/bin/env bash
#begin script
mahout spark-itemsimilarity \
    --input test.csv \
    --output test-result \
    --master spark://Maclaurin.local:7077 \
    --filter1 purchase \
    --filter2 view \
    --itemIDColumn 2 \
    --rowIDColumn 0 \
    --filterColumn 1
#end-script

test.csv file for the script

u1,purchase,iphone
u1,purchase,ipad
u2,purchase,nexus
u2,purchase,galaxy
u3,purchase,surface
u4,purchase,iphone
u4,purchase,galaxy
u1,view,iphone
u1,view,ipad
u1,view,nexus
u1,view,galaxy
u2,view,iphone
u2,view,ipad
u2,view,nexus
u2,view,galaxy
u3,view,surface
u3,view,nexus
u4,view,iphone
u4,view,ipad


> Drivers don't run with remote Spark
> -----------------------------------
>
>                 Key: MAHOUT-1951
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1951
>             Project: Mahout
>          Issue Type: Bug
>          Components: Classification, CLI, Collaborative Filtering
>    Affects Versions: 0.13.0
>         Environment: The command line drivers spark-itemsimilarity and 
> spark-naivebayes using a remote or pseudo-clustered Spark
>            Reporter: Pat Ferrel
>            Assignee: Pat Ferrel
>            Priority: Blocker
>             Fix For: 0.13.0
>
>
> Missing classes when running these jobs because the dependencies-reduced jar, 
> passed to Spark for serialization purposes, does not contain all needed 
> classes.
> Found by a user. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Reply via email to