Thanks Joe for quick response. I will follow your suggestion. On Tue, May 17, 2016 at 11:20 AM, Joe Witt <[email protected]> wrote:
> Madhu, > > Absolutely. You can use MergeContent, for example, to pack together a > bunch of smaller files to create a larger bundle. I'd recommend if > you will bundle 10s of thousands or hundreds of thousands or more of > things that you use two MergeContent processors together where the > first one merges at most 10,000 items and then the second merges the > next 10,000 or so. Hope that helps > > Joe > > On Tue, May 17, 2016 at 11:11 AM, Madhukar Thota > <[email protected]> wrote: > > Friends, > > > > Is it possible to set file size like 500 mb or 1 GB before writing to > hdfs? > > I want to write large files instead of lot of smaller files to hdfs? > > > > > > If possible, what processor i need to use to achive the size? > > > > -Madhu >
