we are processing 1 day data in pig and it generates each hour 1 .gz file like 23 files. out of 23 files 1 file has 11GB remaining max 2.5 GB. 11GB file is not able to process when running the pig script.
Can someone please suggest what could be the reason?
