Hey Phatak, With your way the mapper o/p will be some thing like
<W1_W2 0> <W1_W3 1> <W1_W4 0> ...... Bu then this way I will miss the pair <W3_W5 0> Correct me if I am wrong. Thanks, Praveen On Mon, Apr 2, 2012 at 12:52 PM, madhu phatak <phatak....@gmail.com> wrote: > Hi, > Yes using Map/Reduce its possible . > > 1. In Mapper, > Read the words in line and make pair with calculated distance. Output key > of mapper will be the this word pair and value will be distance > > 2.In Reducer, > Just sum the distance for each pair. > > On Wed, Mar 28, 2012 at 7:42 AM, Praveen Kumar K J V S < > praveenkjvs.develo...@gmail.com> wrote: > >> Hi All, >> >> I have a file in HDFS spanning across many blocks. Say the file has many >> words in it from W1, W2 , W3 ...Wn. >> >> I want to find the edit distance between all pairs of words. Is this is >> possible in Mapreduce. >> >> For example I have 2 blocks for the file: And an edit distance function >> which returns value between 0 to 1(0 means two words are very close t0 each >> other). Say edit distance between even and odd words is 1 >> >> Block 1 has the words W1, W2, W3, W4 >> Block 2 has the words W1, W2, W5, W6 >> >> Now is there a way to find all pairs edit distance. If so how? >> >> Thanks, >> Praveen >> > > > > -- > https://github.com/zinnia-phatak-dev/Nectar > >
