Sandesh and I found an issue in almost all the operators that use Kryo to clone operator itself in definePartition method. They all use default Kryo instance without setting context classloader from the thread that execute the code. It would fail in local mode if it tries to load class that only in app package classpath.
When I'm trying to fix it in all the places, I found repeated pattern in definePartitions. And it's very error-prone since no one actually did it in right way until we found the issue. I'm thinking to make a Util functin to create instance from operator itself and also hide the Kryo dependency from the operator. Do you have any better suggestions to do this? Thanks! Siyuan
