Chen Xin Yu created TINKERPOP-1426: -------------------------------------- Summary: GryoSerializer should implement Java serialization interface Key: TINKERPOP-1426 URL: https://issues.apache.org/jira/browse/TINKERPOP-1426 Project: TinkerPop Issue Type: Bug Components: io Affects Versions: 3.2.1 Reporter: Chen Xin Yu
In spark description for Serializer, there is: * Implementations of this trait should implement: * * 1. a zero-arg constructor or a constructor that accepts a [[org.apache.spark.SparkConf]] * as parameter. If both constructors are defined, the latter takes precedence. * * 2. Java serialization interface. Tinkerepop Class GryoSerializer extends Serializer, but does not implement serialisation. It works well before Spark 2.0. But with Spark 2.0, with the change for SPARK-13926 for Dependency,scala. Gyro must implement Java serialisation interface, otherwise hundreds of test cases are failed as: Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task not serializable: java.io.NotSerializableException: org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer Serialization stack: - object not serializable (class: org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer, value: org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer@1b12ec8e) - field (class: org.apache.spark.ShuffleDependency, name: serializer, type: class org.apache.spark.serializer.Serializer) - object (class org.apache.spark.ShuffleDependency, org.apache.spark.ShuffleDependency@7a4f876a) - field (class: scala.Tuple2, name: _2, type: class java.lang.Object) - object (class scala.Tuple2, (MapPartitionsRDD[1] at mapToPair at InputFormatRDD.java:46,org.apache.spark.ShuffleDependency@7a4f876a)) -- This message was sent by Atlassian JIRA (v6.3.4#6332)