I built and hacked together 0.4-snapshot from src
It now finds the class files - hurrah!
However, I now get an ArrayIndexOutOfBoundsException
10/08/09 16:07:14 INFO mapred.JobClient: Task Id :
attempt_201005101218_0012_m_000000_2, Status : FAILED
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.mahout.cf.taste.hadoop.item.ItemIDIndexMapper.map(ItemIDIndexMapper.java:47)
at
org.apache.mahout.cf.taste.hadoop.item.ItemIDIndexMapper.map(ItemIDIndexMapper.java:31)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Looking at the source code, the issue is on the array indexing on tokens
below, which seems to be an issue
with: TasteHadoopUtils.splitPrefTokens(value.toString());
@Override
protected void map(LongWritable key,
Text value,
Context context) throws IOException,
InterruptedException {
String[] tokens = TasteHadoopUtils.splitPrefTokens(value.toString());
long itemID = Long.parseLong(tokens[transpose ? 0 : 1]);
int index = TasteHadoopUtils.idToIndex(itemID);
context.write(new VarIntWritable(index), new VarLongWritable(itemID));
}
Any ideas? Please note, i suspect that this might be an issue with how I
hacked together my package since I can't figure out how to create a proper
binary release from src.
If not, I'm off to the debugger!
Cheers,
Simon
On Mon, Aug 9, 2010 at 11:19 AM, Sean Owen <[email protected]> wrote:
> I don't really get it either -- the .job file ought to have
> everything. That class isn't even in a separate module.
> But I can tell you the next good step is to use the latest code from
> Subversion instead of 0.3, since I 99% know that works.
>
> On Mon, Aug 9, 2010 at 10:16 AM, Simon Reavely <[email protected]>
> wrote:
> > #!/bin/sh
> > # mapred.input.dir = file/dir containing the history to process
> > # usersFile = user ids to produce recommendations for
> > # This will run a co-occurrence algorithm on it
> > hadoop jar mahout-0.3/mahout-core-0.3.job
> > org.apache.mahout.cf.taste.hadoop.item.RecommenderJob --input
> > /user/sreavely/mahout-boolean-enduseraction-input.csv --output
> > /user/sreavely/mahout-output --usersFile
> > /user/sreavely/mahout-users-to-recommend-for.txt
> >
> > This is what i get...
> >
> > Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> > org.apache.mahout.cf.taste.hadoop.item.ItemIDIndexMapper
> > at
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809)
> > at
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:833)
> > ... 12 more
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.mahout.cf.taste.hadoop.item.ItemIDIndexMapper
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> > at java.lang.Class.forName0(Native Method)
> > at java.lang.Class.forName(Class.java:247)
> > at
> >
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
> > at
> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
> > ... 13 more
> >
> > I was assuming that the .job file (being a jar file) would have all this
> > stuff in it. Any thoughts what is going wrong?
> >
> > Cheers,
> > Simon
> >
> > --
> > Simon Reavely
> > [email protected]
> >
>
--
Simon Reavely
[email protected]
