You also need to add the jar to the classpath so it's available in
your main. You can do soemthing like this:
HADOOP_CLASSPATH=/usr/local/mahout/math/target/mahout-math-0.6-SNAPSHOT.jar
hadoop jar ...
-Joey
On Tue, Jan 31, 2012 at 1:38 PM, Daniel Quach <danqu...@cs.ucla.edu> wrote:
> For Hadoop 0.20.203 (the latest stable), is it sufficient to do this to parse
> the lib jars from the command line?
>
> public static main (String args[]) {
> Configuration conf = new Configuration();
> String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs();
>
> Job job = new Job(conf, "makevector");
> job.setJarByClass(MakeVector.class);
> // etc other code for mappers/reducers
> }
>
> I'm thinking I am missing a step here which is why it won't load the mahout
> jar while running the map reduce.
>
> On Jan 30, 2012, at 11:10 PM, Daniel Quach wrote:
>
>> I compiled using javac:
>>
>> javac -classpath
>> :/usr/local/hadoop/hadoop-core-0.20.203.0.jar:/usr/local/hadoop/lib/commons-cli-1.2.jar:/usr/local/mahout/math/target/mahout-math-0.6-SNAPSHOT.jar
>> -d makevector_classes/ MakeVector.java;
>>
>> If I don't include the mahout-math jar, it gives me a compile error because
>> of DenseVector.
>>
>>
>> On Jan 30, 2012, at 10:42 PM, Prashant Kommireddi wrote:
>>
>>> How are you building the mapreduce jar? Try not to include the Mahout dist
>>> while building MR jar, and include it only on "-libjars" option.
>>>
>>> On Mon, Jan 30, 2012 at 10:33 PM, Daniel Quach <danqu...@cs.ucla.edu> wrote:
>>>
>>>> I have been compiling my mapreduce with the jars in the classpath, and I
>>>> believe I need to also add the jars as an option to -libjars to hadoop.
>>>> However, even when I do this, I still get an error complaining about
>>>> missing classes at runtime. (Compilation works fine).
>>>>
>>>> Here is my command:
>>>> hadoop jar makevector.jar org.myorg.MakeVector -libjars
>>>> /usr/local/mahout/math/target/mahout-math-0.6-SNAPSHOT.jar input/ output/
>>>>
>>>> This is the error I receive:
>>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>>> org/apache/mahout/math/DenseVector
>>>>
>>>> I wonder if I am using the GenericOptionsParser incorrectly? I'm not sure
>>>> if there is a deeper problem here.
>>>>
>>
>
--
Joseph Echeverria
Cloudera, Inc.
443.305.9434
