> How often does the slots in your Hadoop cluster change? The total # of slots (free and busy) change only when more nodes are added to the cluster. But, the # number of free slots change constantly (as other jobs gets completed and started). The free slots could be assigned to the bsp job in question at a later stage also, similar to an mr job.
> I don't think it is needed for the BSP model to use additional tasks, Adding additional tasks might make the job complete faster. > This is really not how BSP works. Let's define it :) Praveen On Thu, Apr 5, 2012 at 8:59 PM, Thomas Jungblut < [email protected]> wrote: > How often does the slots in your Hadoop cluster change? In mine it changes > once every 2 months when I get a hardware upgrade or a failure occurs. > I don't think it is needed for the BSP model to use additional tasks, also > it is not helpful to schedule just a chunk of tasks at the same time. > This is really not how BSP works. > > Am 5. April 2012 16:36 schrieb Praveen Sripati <[email protected]>: > > > So, if more slots are available after the initial 10 slots then Hama > can't > > use them because the assignment has already been done. Looks like not an > > efficient use of the cluster. Hadoop is able to use the additional slots > > effectively. > > > > It would be nice to evaluate the Hadoop way also for Hama. > > > > Praveen > > > > On Thu, Apr 5, 2012 at 4:42 PM, Thomas Jungblut < > > [email protected]> wrote: > > > > > No this kind of execution is so Hadoop-y. > > > It will use the maximum slots (10 in your case) and tries to fit the > > input > > > to the 10 slots e.G by assigning multiple files/blocks to a single > task. > > > > > > Am 5. April 2012 12:37 schrieb Praveen Sripati < > [email protected] > > >: > > > > > > > Hi, > > > > > > > > If the number of InputSplits is more than the number of bsp task > slots > > > > available in the cluster (which is the case is most of the scenarios) > > how > > > > is this handled in the Hama? Will the tasks run in multiple > iterations > > > > storing the intermediate messages in HDFS? For example, lets say > there > > > are > > > > 100 InputSplits and 10 bsp slots. So, it will require 10 iterations > of > > 10 > > > > bsp tasks to complete the job. > > > > > > > > Praveen > > > > > > > > > > > > > > > > -- > > > Thomas Jungblut > > > Berlin <[email protected]> > > > > > > > > > -- > Thomas Jungblut > Berlin <[email protected]> >
