Hello,
I've achieved to run a recommender over hadoop using the command line
/bin/mahout org.apache.mahout.cf.taste.hadoop.item.RecommenderJob --input
.
I'm happy with it but now I want to launch this using Java.
What is the easiest way to do this ? I tried to run the MahoutDriver but it
runs
You could just run the main() method with an array of the same arguments you
passed on the command line. It's a little ugly but entirely works.
2011/8/10 Clément Notin
> Hello,
>
> I've achieved to run a recommender over hadoop using the command line
> /bin/mahout org.apache.mahout.cf.taste.hado
Yes I agree it's ugly ;)
I tried with the params
"org.apache.mahout.cf.taste.hadoop.item.RecommenderJob
-Dmapred.input.dir=mb-recouser-input/input.csv
-Dmapred.output.dir=mb-recouser-output/reco.csv --numRecommendations 3
--booleanData true --similarityClassname SIMILARITY_EUCLIDEAN_DISTANCE" (of
I don't believe it's actually cleaned out then. Hadoop thinks the temp
directory exists from a previous run, which perhaps failed. Make sure it is
deleted in HDFS. This is, at least, what the error is trying to tell you.
Are you running two jobs that might both want this directory?
2011/8/10 Cléme
Ho I'm sorry I thought it wasn't running on HDFS because of the local
/tmp/... folder. You're right thanks !
But (yes I know...) it's using LocalJobRunner (in the log msg) so I assume
it doesn't run on the cluster. I have in the classpath the "masters" and
"slaves" files, and also core-site.xml an
Should I set HADOOP_HOME and make a local install of hadoop ? But I don't
want the "launching" machine to be part of the cluster...
Le 10 août 2011 15:18, Clément Notin a écrit :
> Ho I'm sorry I thought it wasn't running on HDFS because of the local
> /tmp/... folder. You're right thanks !
>
>
You should not need to configure any classpath. Use the "job" files which
contains all dependencies. You can run it locally or in a cluster.
2011/8/10 Clément Notin
> Ho I'm sorry I thought it wasn't running on HDFS because of the local
> /tmp/... folder. You're right thanks !
>
> But (yes I kno
Just to be clear, is it possible to run an hadoop job from a machine outside
of the cluster (what I'm trying to do) ? I'm wondering now...
I think I will run this on one of the cluster's machines.
2011/8/10 Sean Owen
> You should not need to configure any classpath. Use the "job" files which
> c
Oh my yes. The Java program that launches the job is quite separate from the
cluster. It could be on the same machine -- but by no mean needs to be.
2011/8/10 Clément Notin
> Just to be clear, is it possible to run an hadoop job from a machine
> outside
> of the cluster (what I'm trying to do) ?
Ho yes it's running ! Great !
I think there was missing the mapred-site.xml file
Thank you !
2011/8/10 Sean Owen
> Oh my yes. The Java program that launches the job is quite separate from
> the
> cluster. It could be on the same machine -- but by no mean needs to be.
>
> 2011/8/10 Clément Notin
10 matches
Mail list logo