Re: MapRed ports

2010-02-09 Thread Eric Sammer
Pedro: I can answer at least some of these questions for you. See below. On 2/10/10 12:47 AM, psdc1978 wrote: > Hi, > > I've some question about the MapRed ports and how a reduce knows where > the map output is to fetch. > > I know that MapRed uses jetty has a webserver. > > - The JobTracker s

MapRed ports

2010-02-09 Thread psdc1978
Hi, I've some question about the MapRed ports and how a reduce knows where the map output is to fetch. I know that MapRed uses jetty has a webserver. - The JobTracker send tasks to the TaskTracker execute them through port 50060? - Which port TaskTracker uses to send status about the task that

Re: Barrier between reduce and map of the next round

2010-02-09 Thread Felix Halim
Hi Arun, Ah yes.. the first comment by Owen O'Malley is exactly what I have in mind. Thanks, Felix Halim On Wed, Feb 10, 2010 at 3:04 AM, Arun C Murthy wrote: > Felix, you might want to follow > https://issues.apache.org/jira/browse/MAPREDUCE-1434. > We are discussing ideas very similar to wha

Re: Barrier between reduce and map of the next round

2010-02-09 Thread Arun C Murthy
Felix, you might want to follow https://issues.apache.org/jira/browse/MAPREDUCE-1434 . We are discussing ideas very similar to what you've just described over there. Arun On Feb 8, 2010, at 9:49 PM, Felix Halim wrote: Hi, Currently the barrier between r(i) and m(i+1) is the Job barrier. Th

Re: avoiding data redistribution in iterative mapreduce

2010-02-09 Thread Raghava Mutharaju
Hi, No problem, I am thankful that someone has replied to my question. Known location -- can/will it be the HDFS or some distributed key-value store? Regards, Raghava. On Tue, Feb 9, 2010 at 12:40 AM, Amogh Vasekar wrote: > Hi, > AFAIK no. I’m not sure how much of a task it is to write a

Re: Strange behaviour from a custom Writable

2010-02-09 Thread James Hammerton
Thanks guys for your help. I've got my code working (albeit re-buffering) and will look into secondary sorting later! Regards, James On Tue, Feb 9, 2010 at 3:50 PM, Owen O'Malley wrote: > > On Feb 8, 2010, at 9:37 PM, Amogh Vasekar wrote: > > Hi, > Yes the same location is populated with diffe

Re: Strange behaviour from a custom Writable

2010-02-09 Thread Owen O'Malley
On Feb 8, 2010, at 9:37 PM, Amogh Vasekar wrote: Hi, Yes the same location is populated with different values ( returned by iter.next() ) for optimization reasons. As previously mentioned, if you are trying to sort the values, please refer to the secondary sort example. It shows how to u