Re: A record version mismatch occured. Expecting v6, found v32
Thanks, Tom The problem that content was different was that I converted one sample to Base64 byte-by-byte, and converted the other from-byte-array to-byte-array (Strange, that they cause different outputs). Thanks for good points. Rasit 2009/2/2 Tom White > The SequenceFile format is described here: > > http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/io/SequenceFile.html > . > The format of the keys and values depends on the serialization classes > used. For example, BytesWritable writes out the length of its byte > array followed by the actual bytes in the array (see the write() > method in BytesWritable). > > Hope this helps. > Tom > > On Mon, Feb 2, 2009 at 3:21 PM, Rasit OZDAS wrote: > > I tried to use SequenceFile.Writer to convert my binaries into Sequence > > Files, > > I read the binary data with FileInputStream, getting all bytes with > > reader.read(byte[]) , wrote it to a file with SequenceFile.Writer, with > > parameters NullWritable as key, BytesWritable as value. But the content > > changes, > > (I can see that by converting to Base64) > > > > Binary File: > > 73 65 65 65 81 65 65 65 65 65 81 81 65 119 84 81 65 111 67 81 65 52 57 81 > 65 > > 103 54 81 65 65 97 81 65 65 65 81 ... > > > > Sequence File: > > 73 65 65 65 65 69 65 65 65 65 65 65 65 69 66 65 65 77 66 77 81 103 67 103 > 67 > > 69 77 65 52 80 86 67 65 73 68 114 ... > > > > Thanks for any points.. > > Rasit > > > > 2009/2/2 Rasit OZDAS > > > >> Hi, > >> I tried to use SequenceFileInputFormat, for this I appended "SEQ" as > first > >> bytes of my "binary" files (with hex editor). > >> but I get this exception: > >> > >> A record version mismatch occured. Expecting v6, found v32 > >> at > >> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1460) > >> at > >> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1428) > >> at > >> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1417) > >> at > >> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1412) > >> at > >> > org.apache.hadoop.mapred.SequenceFileRecordReader.(SequenceFileRecordReader.java:43) > >> at > >> > org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58) > >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321) > >> at org.apache.hadoop.mapred.Child.main(Child.java:155) > >> > >> What could it be? Is it not enough just to add "SEQ" to binary files? > >> I use Hadoop v.0.19.0 . > >> > >> Thanks in advance.. > >> Rasit > >> > >> > >> different *version* of *Hadoop* between your server and your client. > >> > >> -- > >> M. Raşit ÖZDAŞ > >> > > > > > > > > -- > > M. Raşit ÖZDAŞ > > > -- M. Raşit ÖZDAŞ
Re: A record version mismatch occured. Expecting v6, found v32
The SequenceFile format is described here: http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/io/SequenceFile.html. The format of the keys and values depends on the serialization classes used. For example, BytesWritable writes out the length of its byte array followed by the actual bytes in the array (see the write() method in BytesWritable). Hope this helps. Tom On Mon, Feb 2, 2009 at 3:21 PM, Rasit OZDAS wrote: > I tried to use SequenceFile.Writer to convert my binaries into Sequence > Files, > I read the binary data with FileInputStream, getting all bytes with > reader.read(byte[]) , wrote it to a file with SequenceFile.Writer, with > parameters NullWritable as key, BytesWritable as value. But the content > changes, > (I can see that by converting to Base64) > > Binary File: > 73 65 65 65 81 65 65 65 65 65 81 81 65 119 84 81 65 111 67 81 65 52 57 81 65 > 103 54 81 65 65 97 81 65 65 65 81 ... > > Sequence File: > 73 65 65 65 65 69 65 65 65 65 65 65 65 69 66 65 65 77 66 77 81 103 67 103 67 > 69 77 65 52 80 86 67 65 73 68 114 ... > > Thanks for any points.. > Rasit > > 2009/2/2 Rasit OZDAS > >> Hi, >> I tried to use SequenceFileInputFormat, for this I appended "SEQ" as first >> bytes of my "binary" files (with hex editor). >> but I get this exception: >> >> A record version mismatch occured. Expecting v6, found v32 >> at >> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1460) >> at >> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1428) >> at >> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1417) >> at >> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1412) >> at >> org.apache.hadoop.mapred.SequenceFileRecordReader.(SequenceFileRecordReader.java:43) >> at >> org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321) >> at org.apache.hadoop.mapred.Child.main(Child.java:155) >> >> What could it be? Is it not enough just to add "SEQ" to binary files? >> I use Hadoop v.0.19.0 . >> >> Thanks in advance.. >> Rasit >> >> >> different *version* of *Hadoop* between your server and your client. >> >> -- >> M. Raşit ÖZDAŞ >> > > > > -- > M. Raşit ÖZDAŞ >
Re: A record version mismatch occured. Expecting v6, found v32
I tried to use SequenceFile.Writer to convert my binaries into Sequence Files, I read the binary data with FileInputStream, getting all bytes with reader.read(byte[]) , wrote it to a file with SequenceFile.Writer, with parameters NullWritable as key, BytesWritable as value. But the content changes, (I can see that by converting to Base64) Binary File: 73 65 65 65 81 65 65 65 65 65 81 81 65 119 84 81 65 111 67 81 65 52 57 81 65 103 54 81 65 65 97 81 65 65 65 81 ... Sequence File: 73 65 65 65 65 69 65 65 65 65 65 65 65 69 66 65 65 77 66 77 81 103 67 103 67 69 77 65 52 80 86 67 65 73 68 114 ... Thanks for any points.. Rasit 2009/2/2 Rasit OZDAS > Hi, > I tried to use SequenceFileInputFormat, for this I appended "SEQ" as first > bytes of my "binary" files (with hex editor). > but I get this exception: > > A record version mismatch occured. Expecting v6, found v32 > at > org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1460) > at > org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1428) > at > org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1417) > at > org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1412) > at > org.apache.hadoop.mapred.SequenceFileRecordReader.(SequenceFileRecordReader.java:43) > at > org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321) > at org.apache.hadoop.mapred.Child.main(Child.java:155) > > What could it be? Is it not enough just to add "SEQ" to binary files? > I use Hadoop v.0.19.0 . > > Thanks in advance.. > Rasit > > > different *version* of *Hadoop* between your server and your client. > > -- > M. Raşit ÖZDAŞ > -- M. Raşit ÖZDAŞ
A record version mismatch occured. Expecting v6, found v32
Hi, I tried to use SequenceFileInputFormat, for this I appended "SEQ" as first bytes of my "binary" files (with hex editor). but I get this exception: A record version mismatch occured. Expecting v6, found v32 at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1460) at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1428) at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1417) at org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1412) at org.apache.hadoop.mapred.SequenceFileRecordReader.(SequenceFileRecordReader.java:43) at org.apache.hadoop.mapred.SequenceFileInputFormat.getRecordReader(SequenceFileInputFormat.java:58) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:321) at org.apache.hadoop.mapred.Child.main(Child.java:155) What could it be? Is it not enough just to add "SEQ" to binary files? I use Hadoop v.0.19.0 . Thanks in advance.. Rasit different *version* of *Hadoop* between your server and your client. -- M. Raşit ÖZDAŞ
