Re: Ideal number of mappers and reducers; any physical limits?

2008-10-28 Thread Edward J. Yoon
Hi, I'm interested in graph algorithms. In single machine, as far as we know graph can be stored to linked list or matrix. Do you know about difference benefit between linked list and matrix? So, I guess google's web graph will be stored as a matrix in a bigTable. Have you seen my 2D block

Re: Ideal number of mappers and reducers; any physical limits?

2008-10-28 Thread Ted Dunning
On Tue, Oct 28, 2008 at 5:15 PM, Edward J. Yoon [EMAIL PROTECTED]wrote: ... In single machine, as far as we know graph can be stored to linked list or matrix. Since the matrix is normally very sparse for large graphs, these two approaches are pretty similar. ... So, I guess google's web

Ideal number of mappers and reducers; any physical limits?

2008-07-14 Thread Lukas Vlcek
Hi, I have a couple of *basic* questions about Hadoop internals. 1) If I understood correctly the ideal number of Reducers is equal to number of distinct keys (or custom Partitioners) emitted from from all Mappers at given Map-Reduce iteration. Is that correct? 2) In configuration there can be