Hi,
I apologize, I spoke too soon.
Those transient member variables may not be the issue.
To clarify my test case I am creating a LinkedHashMap with two elements in
a map expression on an RDD.
Note that the LinkedHashMaps are being created on the worker JVMs (not the
driver JVM) and THEN
Thanks Renato.
I forgot to reply all last time. I apologize for the rather confusing
example.
All that the snipet code did was
1. Make an RDD of LinkedHashMaps with size 2
2. On the worker side get the sizes of the HashMaps (via a map(hash =>
hash.size))
3. On the driver call collect on the
Hi Rahul,
You have probably already figured this one out, but anyway...
You need to register the classes that you'll be using with Kryo because it
does not support all Serializable types and requires you to register the
classes you’ll use in the program in advance. So when you don't register
the
Hi,
Just sending this again to see if others have had this issue.
I recently switched to using kryo serialization and I've been running into
errors
with the mutable.LinkedHashMap class.
If I don't register the mutable.LinkedHashMap class then I get an
ArrayStoreException seen below.
If I do
Hi,
I recently switched to using kryo serialization and I've been running into
errors
with the mutable.LinkedHashMap class.
If I don't register the mutable.LinkedHashMap class then I get an
ArrayStoreException seen below.
If I do register the class, then when the LinkedHashMap is collected on
