Pig is a little too smart when dealing with data. It has a feature called split combination. If you set it to false, you should see more mappers.
SET pig.noSplitCombination true; On Tue, Apr 22, 2014 at 12:14 PM, Patcharee Thongtra < patcharee.thong...@uni.no> wrote: > Hi, > > I wrote a custom InputFormat. When I ran the pig script Load function > using this InputFormat, the number of InputSplit = 16, but there was only 2 > map tasks handling these splits. Apparently the no. of map tasks = the no. > of input files. > > Does the number of Map task not correspond to the number of splits? > > I think the job will be done quicker if there are more Map tasks? > > Patcharee >
