[
https://issues.apache.org/jira/browse/TINKERPOP-1426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458260#comment-15458260
]
Chen Xin Yu commented on TINKERPOP-1426:
----------------------------------------
There is performance issue with the two change sets together.
This JIRA is not related to Spark 2.0, even without Spark 2.0, Class
GryoSerializer should also implement Java serialization interface.
So I submit a separate PR on github.
https://github.com/apache/tinkerpop/pull/398
> GryoSerializer should implement Java serialization interface
> ------------------------------------------------------------
>
> Key: TINKERPOP-1426
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1426
> Project: TinkerPop
> Issue Type: Bug
> Components: io
> Affects Versions: 3.2.1
> Reporter: Chen Xin Yu
>
> There is description for Serializer in spark:
> * Implementations of this trait should implement:
> *
> * 1. a zero-arg constructor or a constructor that accepts a
> [[org.apache.spark.SparkConf]]
> * as parameter. If both constructors are defined, the latter takes
> precedence.
> *
> * 2. Java serialization interface.
> Class GryoSerializer in Tinkerepop extends Serializer, but does not implement
> java.io.Serializable.
> It works well before Spark 2.0. But with Spark 2.0, it changed by SPARK-13926
> for Dependency,scala.
> Gyro and all its fields must implement Java serialisation interface,
> otherwise hundreds of test cases are failed as:
> Caused by: org.apache.spark.SparkException: Job aborted due to stage failure:
> Task not serializable: java.io.NotSerializableException:
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer
> Serialization stack:
> - object not serializable (class:
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer, value:
> org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer@1b12ec8e)
> - field (class: org.apache.spark.ShuffleDependency, name: serializer,
> type: class org.apache.spark.serializer.Serializer)
> - object (class org.apache.spark.ShuffleDependency,
> org.apache.spark.ShuffleDependency@7a4f876a)
> - field (class: scala.Tuple2, name: _2, type: class java.lang.Object)
> - object (class scala.Tuple2, (MapPartitionsRDD[1] at mapToPair at
> InputFormatRDD.java:46,org.apache.spark.ShuffleDependency@7a4f876a))
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
