Hi, If there are input data files, the number of tasks is determined by the number of HDFS blocks in the input files. You can force the number of tasks using the partition function (it means that creating N partitions), but it's not recommended.
On Thu, Sep 26, 2013 at 1:10 AM, Anveshi Charuvaka <[email protected]> wrote: > Hello Everyone, > > What is the impact of setting the number of bsp tasks using > BSPJob.setNumBspTasks(N) ? Is the number of bsp tasks spawned ℗ by Hama equal > to the set parameter value (N) or is it just a recommendation to the > framework. Does the number of input (key,value) pairs provided have any > influence over the number of tasks spawned? > > Context: Let N = parameter of setNumBspTasks, P = # of bsp tasks > spawned by hama (inferred from console output). I have created a small BSP > program and I am running it in local mode with different inputs and > different values of number of jobs (provided through BSPJob.setNumBspTasks). > When I set the input format as NullInputFormat then, P == N. If I use > TextInputFormat or KeyValueInputFormat the P is not always equal to N, > usually P < N and never P > N. > > > Thanks > Anvesh > > -- Best Regards, Edward J. Yoon @eddieyoon
