Re: hadoop MapReduce and stop words

2009-05-16 Thread Stefan Will
15:48:23 +0200 > To: > Subject: Re: hadoop MapReduce and stop words > > Try and google "binary tree java" and you will get loads of hits... > > This is a simple implementation but I am sure there are better ones > that handle balancing better. > > Cheers

Re: hadoop MapReduce and stop words

2009-05-16 Thread tim robertson
Try and google "binary tree java" and you will get loads of hits... This is a simple implementation but I am sure there are better ones that handle balancing better. Cheers Tim public class BinaryTree { public static void main(String[] args) { BinaryTree bt = new Binary

Re: hadoop MapReduce and stop words

2009-05-16 Thread PORTO aLET
Can you please elaborate more about in memory index? What kind of software did you used to implement this ? Regards On Sat, May 16, 2009 at 8:55 PM, tim robertson wrote: > Perhaps some kind of in memory index would be better than iterating an > array? Binary tree or so. > I did similar with polyg

Re: hadoop MapReduce and stop words

2009-05-16 Thread tim robertson
Perhaps some kind of in memory index would be better than iterating an array? Binary tree or so. I did similar with polygon indexes and point data. It requires careful memory planning on the nodes if the indexes are large (mine were several GB). Just a thought, Tim On Sat, May 16, 2009 at 1:56

hadoop MapReduce and stop words

2009-05-16 Thread PORTO aLET
Hi, I am trying to include the stop words into hadoop map reduce, and later on, into hive. What is the accepted solution regarding the stop words in hadoop? All I can think is to load all the stop words into an array in the mapper, and then check each token against the stop words..(this would be