Re: Sort with customized input/output !!

2010-09-08 Thread Matthew John
Thanks for the reply Ted !! What I understand is that a SequenceFile will have a header followed by the records in a format : Recordlength,Keylength,Key,Value with a sync marker coming at some regular interval.. It would be great if someone can take a look at the following.. Q 1) The thing is my

Re: Sort with customized input/output !!

2010-09-07 Thread Ted Yu
Please get hadoop source code and read the comment at the beginning of SequenceFile.java: * Essentially there are 3 different formats for SequenceFiles ... On Tue, Sep 7, 2010 at 8:13 PM, Matthew John wrote: > Hey , > M pretty new to Hadoop . > > I need to Sort a Metafile (TBs) and thought of us

Sort with customized input/output !!

2010-09-07 Thread Matthew John
Hey , M pretty new to Hadoop . I need to Sort a Metafile (TBs) and thought of using Hadoop Sort (in examples) for it. My input metafile looks like this --> binary stream (only 1's and 0's). It basically contains records of 40 bytes. Every record goes like this : long a; --> 8 bytes. The rest of