Hello All, I am creating 2 files in the constructor of my reducer and I store the file Handles as member variables. I write some data in these files for each call to reduce method. For some reason the files are created but only the larger file has data(The larger file is 400mb).The smaller file is created but no data is written to it (The size should be around 4MB). I checked the hadoop configuration and the buffer-size parameter is 4096.
I padded some 100MB data to the smaller file and than this file gets around 64MB data. I am clueless what is happening and if there is a solution to this. Please share if you have faced similar problem or if you know the solution. Thanks, Parth Brahmbhatt