Re: Number of concurrent workers

2015-01-28 Thread Eli Reisman
Hi, Lukas is correct, due to the fact that all workers must be available to send and receive messages during each superstep, Giraph needs all mappers up and running at once to begin a computation. On Jan 27, 2015 11:01 AM, Walaa Eldin Moustafa wa.moust...@gmail.com wrote: Thanks! Are not there

Re: Number of concurrent workers

2015-01-27 Thread Walaa Eldin Moustafa
Thanks! Are not there any options to relax this restriction? My end goal is to have small input splits for each worker so that workers can finish processing input splits without throwing out of memory exceptions. On Tue, Jan 27, 2015 at 1:14 AM, Lukas Nalezenec lukas.naleze...@firma.seznam.cz

Re: Number of concurrent workers

2015-01-27 Thread Lukas Nalezenec
On 23.1.2015 00:40, Walaa Eldin Moustafa wrote: Hi, I am experimenting with a memory-intensive Giraph application on top of a large graph (50 million nodes), on a 14 node cluster. When setting the number of workers to a large number (500 in this example), I get errors for not being able to

Re: Number of concurrent workers

2015-01-26 Thread Walaa Eldin Moustafa
Just bumping up this thread. Thanks in advance for the help. Thanks, Walaa. On Thu, Jan 22, 2015 at 3:40 PM, Walaa Eldin Moustafa wa.moust...@gmail.com wrote: Hi, I am experimenting with a memory-intensive Giraph application on top of a large graph (50 million nodes), on a 14 node