Do you get the same effect if you specify exactly one of those?
If this then also happens and these files are these files gzipped: you may
be using a buggy inputformat.
See this
https://issues.apache.org/jira/plugins/servlet/mobile#issue/MAPREDUCE-2094
On Dec 24, 2014 5:54 PM, "Rodrigo Ferreira" <web...@gmail.com> wrote:
> Hi everyone, happy holidays!
>
> I have a Pig script that reads from 4 different folders in Amazon S3. This
> is the code:
>
> load_1 = LOAD 's3n://mybucket/{folder_1,folder_2,folder_3,folder_4}'
> USING...;
>
> It happens that instead of reading each folder just once and appending the
> files Pig/Hadoop reads each folder 4 times.
>
> The input should have 62174 records, but in the end I get 248696.
>
> Why is that? Any ideas?
>
> Thanks,
> Rodrigo.
>
