Hi guys, I'm trying to do a a job with Spark, using Java.
The thing is I need to have an index of words of about 3 GB in each machine, so I'm trying to broadcast custom objects to represent the index and the interface with it. I'm using java standard serialization, so I tried to implement serializable interface in each class involved, but some objects come from libraries so I can't go any further. Is there another way to make it works? Should I try with Kryo? Is there a way to work with non-serializable objects? I use a fat-jar, so the code is available in all workers really. I thing it should be a way to use it instead of being serializing and deserializing everything. Thanks, Pedro
