[
https://issues.apache.org/jira/browse/TINKERPOP-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17622877#comment-17622877
]
ASF GitHub Bot commented on TINKERPOP-2817:
-------------------------------------------
spmallette commented on code in PR #1835:
URL: https://github.com/apache/tinkerpop/pull/1835#discussion_r1002767340
##########
hadoop-gremlin/src/main/java/org/apache/tinkerpop/gremlin/hadoop/structure/io/graphson/GraphSONRecordReader.java:
##########
@@ -56,11 +59,33 @@ public GraphSONRecordReader() {
public void initialize(final InputSplit genericSplit, final
TaskAttemptContext context) throws IOException {
this.lineRecordReader.initialize(genericSplit, context);
this.hasEdges =
context.getConfiguration().getBoolean(Constants.GREMLIN_HADOOP_GRAPH_READER_HAS_EDGES,
true);
- this.graphsonReader = GraphSONReader.build().mapper(
- GraphSONMapper.build().
-
version(GraphSONVersion.valueOf(context.getConfiguration().get(Constants.GREMLIN_HADOOP_GRAPHSON_VERSION,
"V3_0"))).
- typeInfo(TypeInfo.PARTIAL_TYPES).
-
addRegistries(IoRegistryHelper.createRegistries(ConfUtil.makeApacheConfiguration(context.getConfiguration()))).create()).create();
+ GraphSONVersion graphSONVersion =
+
GraphSONVersion.valueOf(context.getConfiguration().get(Constants.GREMLIN_HADOOP_GRAPHSON_VERSION,
"V3_0"));
+ Mapper mapper = null;
+
+ if (graphSONVersion == GraphSONVersion.V2_0) {
Review Comment:
might be nice if you make this code a bit more compact - like, create a
builder instance with the common things and then just `addCustomModule(...)` in
the if-then parts. i can do it after it's merged if you don't have time.
> "Could not find a type identifier for the class : class java.lang.Byte"
> occurs when dumping graph to graphson format
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: TINKERPOP-2817
> URL: https://issues.apache.org/jira/browse/TINKERPOP-2817
> Project: TinkerPop
> Issue Type: Bug
> Components: hadoop
> Affects Versions: 3.4.6
> Reporter: Redriver
> Priority: Major
>
> When I used hadoop-gremlin 3.4.6 to export a graph to HDFS, I encountered an
> error.
> My gremlin query looks like:
> {code:java}
> graph = GraphFactory.open('conf/fdb-psave-export.properties')
> graph.compute(SparkGraphComputer).program(CloneVertexProgram.build().create()).submit().get()
> {code}
> {code:java}
> org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Could not
> find a type identifier for the class : class java.lang.Byte. Make sure the
> value to serialize has a type identifier registered for its class.
> at
> org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider._wrapAsIOE(DefaultSerializerProvider.java:509)
> ~[gremlin-shaded-3.4.6.jar:3.4.6]
> at
> org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:482)
> ~[gremlin-shaded-3.4.6.jar:3.4.6]
> at
> org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
> ~[gremlin-shaded-3.4.6.jar:3.4.6]
> at
> org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3906)
> ~[gremlin-shaded-3.4.6.jar:3.4.6]
> at
> org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValue(ObjectMapper.java:3177)
> ~[gremlin-shaded-3.4.6.jar:3.4.6]
> at
> org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONWriter.writeVertex(GraphSONWriter.java:82)
> ~[gremlin-core-3.4.6.jar:3.4.6]
> at
> org.apache.tinkerpop.gremlin.hadoop.structure.io.graphson.GraphSONRecordWriter.write(GraphSONRecordWriter.java:72)
> ~[hadoop-gremlin-3.4.6.jar:3.4.6]
> at
> org.apache.tinkerpop.gremlin.hadoop.structure.io.graphson.GraphSONRecordWriter.write(GraphSONRecordWriter.java:42)
> ~[hadoop-gremlin-3.4.6.jar:3.4.6]
> at
> org.apache.spark.internal.io.HadoopMapReduceWriteConfigUtil.write(SparkHadoopWriter.scala:356)
> ~[spark-core_2.11-2.4.0.jar:2.4.0]
> at
> org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$4.apply(SparkHadoopWriter.scala:130)
> ~[spark-core_2.11-2.4.0.jar:2.4.0]
> at
> org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$4.apply(SparkHadoopWriter.scala:127)
> ~[spark-core_2.11-2.4.0.jar:2.4.0]
> at
> org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1394)
> [spark-core_2.11-2.4.0.jar:2.4.0]
> at
> org.apache.spark.internal.io.SparkHadoopWriter$.org$apache$spark$internal$io$SparkHadoopWriter$$executeTask(SparkHadoopWriter.scala:139)
> [spark-core_2.11-2.4.0.jar:2.4.0]
> at
> org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$3.apply(SparkHadoopWriter.scala:83)
> [spark-core_2.11-2.4.0.jar:2.4.0]
> at
> org.apache.spark.internal.io.SparkHadoopWriter$$anonfun$3.apply(SparkHadoopWriter.scala:78)
> [spark-core_2.11-2.4.0.jar:2.4.0]
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
> [spark-core_2.11-2.4.0.jar:2.4.0]
> at org.apache.spark.scheduler.Task.run(Task.scala:121)
> [spark-core_2.11-2.4.0.jar:2.4.0]
> at
> org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:416)
> [spark-core_2.11-2.4.0.jar:2.4.0]
> at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
> [spark-core_2.11-2.4.0.jar:2.4.0]
> at
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:422)
> [spark-core_2.11-2.4.0.jar:2.4.0]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_202]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_202]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_202]
> {code}
> It looks like the default GraphSONRecordWriter/GraphSONRecordReader does not
> support the serialization/deserialization of java.lang.Byte. But according to
> https://tinkerpop.apache.org/docs/3.4.6/dev/io/#_extended_2, adding the
> custom module GraphSONXModuleV2d0/GraphSONXModuleV3d0 can fix this. The
> experient with a quick fix proves the fix is working.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
