On Tue, Feb 21, 2012 at 9:25 AM, Bejoy Ks <bejoy.had...@gmail.com> wrote:
> Mohit > Rather than just appending the content into a normal text file or > so, you can create a sequence file with the individual smaller file content > as values. > > Thanks. I was planning to use pig's > org.apache.pig.piggybank.storage.XMLLoader for processing. Would it work with sequence file? This text file that I was referring to would be in hdfs itself. Is it still different than using sequence file? > Regards > Bejoy.K.S > > On Tue, Feb 21, 2012 at 10:45 PM, Mohit Anchlia <mohitanch...@gmail.com > >wrote: > > > We have small xml files. Currently I am planning to append these small > > files to one file in hdfs so that I can take advantage of splits, larger > > blocks and sequential IO. What I am unsure is if it's ok to append one > file > > at a time to this hdfs file > > > > Could someone suggest if this is ok? Would like to know how other do it. > > >
