[ https://issues.apache.org/jira/browse/SPARK-2878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Wendell updated SPARK-2878: ----------------------------------- Fix Version/s: 1.0.3 > Inconsistent Kryo serialisation with custom Kryo Registrator > ------------------------------------------------------------ > > Key: SPARK-2878 > URL: https://issues.apache.org/jira/browse/SPARK-2878 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 1.0.0, 1.0.2 > Environment: Linux RedHat EL 6, 4-node Spark cluster. > Reporter: Graham Dennis > Assignee: Graham Dennis > Priority: Critical > Fix For: 1.1.0, 1.0.3 > > > The custom Kryo Registrator (a class with the > org.apache.spark.serializer.KryoRegistrator trait) is not used with every > Kryo instance created, and this causes inconsistent serialisation and > deserialisation. > The Kryo Registrator is sometimes not used because of a ClassNotFound > exception that only occurs if it *isn't* the Worker thread (of an Executor) > that tries to create the KryoRegistrator. > A complete description of the problem and a project reproducing the problem > can be found at https://github.com/GrahamDennis/spark-kryo-serialisation > I have currently only tested this with Spark 1.0.0, but will try to test > against 1.0.2. -- This message was sent by Atlassian JIRA (v6.2#6252) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org