Hi,
The test case is separated out as follows. The call to rdd2.first() breaks when
spark version is changed to 1.1.0, reporting exception NullWritable not
serializable. However, the same test passed with spark 1.0.2. The pom.xml file
is attached. The test data README.md was copied from spark
To: Du Li l...@yahoo-inc.com.invalidmailto:l...@yahoo-inc.com.invalid,
user@spark.apache.orgmailto:user@spark.apache.org
user@spark.apache.orgmailto:user@spark.apache.org,
d...@spark.apache.orgmailto:d...@spark.apache.org
d...@spark.apache.orgmailto:d...@spark.apache.org
Subject: Re: NullWritable
To: Du Li l...@yahoo-inc.com.invalid, user@spark.apache.org
user@spark.apache.org, d...@spark.apache.org d...@spark.apache.org
Subject: Re: NullWritable not serializable
Hi Du,
I don't think NullWritable has ever been serializable, so you must be doing
something differently from your previous
Hi,
I was trying the following on spark-shell (built with apache master and hadoop
2.4.0). Both calling rdd2.collect and calling rdd3.collect threw
java.io.NotSerializableException: org.apache.hadoop.io.NullWritable.
I got the same problem in similar code of my app which uses the newly
Hi Du,
I don't think NullWritable has ever been serializable, so you must be doing
something differently from your previous program. In this case though, just use
a map() to turn your Writables to serializable types (e.g. null and String).
Matie
On September 12, 2014 at 8:48:36 PM, Du Li
