How often does the slots in your Hadoop cluster change? In mine it changes once every 2 months when I get a hardware upgrade or a failure occurs. I don't think it is needed for the BSP model to use additional tasks, also it is not helpful to schedule just a chunk of tasks at the same time. This is really not how BSP works.
Am 5. April 2012 16:36 schrieb Praveen Sripati <[email protected]>: > So, if more slots are available after the initial 10 slots then Hama can't > use them because the assignment has already been done. Looks like not an > efficient use of the cluster. Hadoop is able to use the additional slots > effectively. > > It would be nice to evaluate the Hadoop way also for Hama. > > Praveen > > On Thu, Apr 5, 2012 at 4:42 PM, Thomas Jungblut < > [email protected]> wrote: > > > No this kind of execution is so Hadoop-y. > > It will use the maximum slots (10 in your case) and tries to fit the > input > > to the 10 slots e.G by assigning multiple files/blocks to a single task. > > > > Am 5. April 2012 12:37 schrieb Praveen Sripati <[email protected] > >: > > > > > Hi, > > > > > > If the number of InputSplits is more than the number of bsp task slots > > > available in the cluster (which is the case is most of the scenarios) > how > > > is this handled in the Hama? Will the tasks run in multiple iterations > > > storing the intermediate messages in HDFS? For example, lets say there > > are > > > 100 InputSplits and 10 bsp slots. So, it will require 10 iterations of > 10 > > > bsp tasks to complete the job. > > > > > > Praveen > > > > > > > > > > > -- > > Thomas Jungblut > > Berlin <[email protected]> > > > -- Thomas Jungblut Berlin <[email protected]>
