User defined class as Map/Reduce output value

2010-04-27 Thread Farhan Husain
Hello, I want to output a class which I have written as the value of the map phase. The obvious was is to implement the Writable interface but the problem is the class has other classes as its member properties. The DataInput and DataOutput interfaces used by the read and write methods of the

Re: User defined class as Map/Reduce output value

2010-04-27 Thread Ted Yu
Take a look at the sample given in Javadoc of Writable.java You need to serialize your data yourself: @Override public void readFields(DataInput in) throws IOException { h = Text.readString(in); sc = in.readFloat(); ran = in.readInt (); } On Tue, Apr 27, 2010 at

Re: User defined class as Map/Reduce output value

2010-04-27 Thread Farhan Husain
Can I use the Serializable interface? Alternatively, is there any way to specify OutputFormatter for mappers like we can do for reducers? Thanks, Farhan On Tue, Apr 27, 2010 at 1:19 PM, Ted Yu yuzhih...@gmail.com wrote: Take a look at the sample given in Javadoc of Writable.java You need to

Re: User defined class as Map/Reduce output value

2010-04-27 Thread Farhan Husain
I tried to use a class which implements the Serializable interface and got the following error: java.lang.NullPointerException at org.apache.hadoop.io.serializer.SerializationFactory.getSerializer(SerializationFactory.java:73) at

Re: User defined class as Map/Reduce output value

2010-04-27 Thread Ted Yu
Can you try adding 'org.apache.hadoop.io.serializer.JavaSerialization,' to the following config ? C:\hadoop-0.20.2\src\core\core-default.xml(87,9): nameio.serializations/name By default, only org.apache.hadoop.io.serializer.WritableSerialization is included. On Tue, Apr 27, 2010 at 3:55 PM,