Re: Using JavaSerialzation and SequenceFileInput

Cool, thanks for the answer. On Wed, Sep 17, 2008 at 12:35 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote: > The problem is that the Java serialization works for SequenceFile, but > doesn't work with RecordReader. The problem is that Java serialization > always returns a new object and the RecordRea

Re: Using JavaSerialzation and SequenceFileInput

The problem is that the Java serialization works for SequenceFile, but doesn't work with RecordReader. The problem is that Java serialization always returns a new object and the RecordReader interface looks like: boolean next(Object key, Object value) throws IOException; where the outer cont

Re: Using JavaSerialzation and SequenceFileInput

I read HADOOP-3413 a bit more closely - it updates SequenceFile.Reader, not SequenceFileInputFormat, which is what M.R. framework uses... looks like you have to write your own input format, or have your mappers/reducers take raw bytes, and deseria

Re: Using JavaSerialzation and SequenceFileInput

I just found this one this morning, looks like a fix should be in 0.18.0 according to the bug tracker: https://issues.apache.org/jira/browse/HADOOP-3413 I'm going to go double check all my code, as I'm pretty sure I am on 0.18.0 already -jg- On Tue, Sep 16, 2008 at 9:10 PM, Alex Loddengaard <[

Re: Using JavaSerialzation and SequenceFileInput

Unfortunately I don't know of a solution to your problem, but I've been experiencing the exact same issues while trying to implement a Protocol Buffer serialization. Take a look: I hope this helps others to diagnose your problem. Alex On Wed,

Re: Using JavaSerialzation and SequenceFileInput

*HeadlineDocument *in the code below is equivalent to *MyObject* - I forgot to obfuscate that one... opps... On Tue, Sep 16, 2008 at 11:46 AM, Jason Grey <[EMAIL PROTECTED]>wrote: > I'm trying to use JavaSerialization for a series of MapReduce jobs, and > when it comes to reading a SequenceFile u

Using JavaSerialzation and SequenceFileInput

I'm trying to use JavaSerialization for a series of MapReduce jobs, and when it comes to reading a SequenceFile using SequenceFileInputFormat with JavaSerialized objects, something breaks down. I've added "org.apache.hadoop.io.serializer.JavaSerialization" to the io.serializations property in my c