This v3 looks like a <3 heart where the top broke off. Sad.
-----Original Message----- From: Stephen Mallette [mailto:spmalle...@gmail.com] Sent: Wednesday, September 06, 2017 1:45 PM To: dev@tinkerpop.apache.org Subject: Re: io and graphson-v3 never mind - i see what's messed up On Wed, Sep 6, 2017 at 1:43 PM, Stephen Mallette <spmalle...@gmail.com> wrote: > Which tests in particular are failing for SerializationTest? all of them? > > On Wed, Sep 6, 2017 at 1:35 PM, pieter gmail <pieter.mar...@gmail.com> > wrote: > >> Hi, >> >> Pulled TINKERPOP-1767 branch, changed SqlgGraph's io method and ran >> the tests. >> >> All the io tests are passing. >> Only SerializationTest fails for the same reason. It too needs the >> version specified. I did that locally and then all tests passes. >> >> Thanks >> Pieter >> >> On 06/09/2017 18:09, Stephen Mallette wrote: >> >>> Pieter, I created this issue: >>> >>> https://issues.apache.org/jira/browse/TINKERPOP-1767 >>> >>> and made an effort to try to figure a way to fix it: >>> >>> https://github.com/apache/tinkerpop/tree/TINKERPOP-1767 >>> >>> Note the change to TinkerGraph and its io() method. I suppose you >>> could do something similar to get the right registry in play? could >>> you have a look and see if what i did helps? if that works then i'll >>> issue a PR and we can get it reviewed/merged. >>> >>> >>> On Tue, Sep 5, 2017 at 12:10 PM, pieter gmail >>> <pieter.mar...@gmail.com> >>> wrote: >>> >>> Ok, at present there is only one SimpleModule, the default. I can >>> make it >>>> v2 or v3 but not both. >>>> >>>> Lets say I make the SimpleModule support V2. >>>> Then when calling IoEdgeTest for >>>> >>>> {"graphson-v3", true, true, >>>> (Function<Graph, GraphReader>) g -> g.io >>>> (IoCore.graphson()).reader >>>> ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V3_0)). >>>> mapper().create()).create(), >>>> (Function<Graph, GraphWriter>) g -> g.io >>>> (IoCore.graphson()).writer >>>> ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V3_0)). >>>> mapper().create()).create()}, >>>> >>>> then the deserializers that run are for V2, >>>> >>>> static class SchemaTableIdJacksonDeserializerV2d0 extends >>>> AbstractObjectDeserializer<SchemaTable> { >>>> SchemaTableIdJacksonDeserializerV2d0() { >>>> super(SchemaTable.class); >>>> } >>>> >>>> @Override >>>> public SchemaTable createObject(final Map data) { >>>> return SchemaTable.of((String)data.get("schema"), >>>> (String) data.get("table")); >>>> } >>>> } >>>> >>>> when createObject fires the map data has V3 elements in it like >>>> @type and @value whilst its expecting "schema" and "table" >>>> >>>> If we make the SimpleModule support V3 then graphson-v2 will fail. >>>> >>>> {"graphson-v2", false, false, >>>> (Function<Graph, GraphReader>) g -> g.io >>>> (IoCore.graphson()).reader >>>> ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V2_0)). >>>> mapper().typeInfo(TypeInfo.NO_TYPES).create()).create(), >>>> (Function<Graph, GraphWriter>) g -> g.io >>>> (IoCore.graphson()).writer >>>> ().mapper(g.io(GraphSONIo.build(GraphSONVersion.V2_0)). >>>> >>>> mapper().typeInfo(TypeInfo.NO_TYPES).create()).create()}, >>>> >>>> Now the deserializers are for V3. >>>> >>>> static class SchemaTableJacksonDeserializerV3d0 extends >>>> StdDeserializer<SchemaTable> { >>>> public SchemaTableJacksonDeserializerV3d0() { >>>> super(RecordId.class); >>>> } >>>> >>>> @Override >>>> public SchemaTable deserialize(final JsonParser >>>> jsonParser, final DeserializationContext deserializationContext) >>>> throws IOException, JsonProcessingException { >>>> final Map<String, Object> data = >>>> deserializationContext.readValue(jsonParser, >>>> Map.class); >>>> return SchemaTable.of((String)data.get("schema"), >>>> (String) data.get("table")); >>>> } >>>> >>>> @Override >>>> public boolean isCachable() { >>>> return true; >>>> } >>>> } >>>> >>>> This does not fire at all. Eventually I get a detached edge with an >>>> id that is a map. It never deserialized. >>>> >>>> So basically it only works if the SimpleModule version, i.e. >>>> serialize/deserialize code matches up with the version. >>>> >>>> Sqlg serializes RecordId to, >>>> >>>> static class RecordIdJacksonSerializerV3d0 extends >>>> StdScalarSerializer<RecordId> { >>>> public RecordIdJacksonSerializerV3d0() { >>>> super(RecordId.class); >>>> } >>>> @Override >>>> public void serialize(final RecordId recordId, final >>>> JsonGenerator jsonGenerator, final SerializerProvider >>>> serializerProvider) >>>> throws IOException, JsonGenerationException { >>>> final Map<String, Object> m = new HashMap<>(); >>>> m.put("schemaTable", recordId.getSchemaTable()); >>>> m.put("id", recordId.getId()); >>>> jsonGenerator.writeObject(m); >>>> } >>>> } >>>> >>>> and >>>> >>>> static class SchemaTableJacksonSerializerV3d0 extends >>>> StdScalarSerializer<SchemaTable> { >>>> SchemaTableJacksonSerializerV3d0() { >>>> super(SchemaTable.class); >>>> } >>>> >>>> @Override >>>> public void serialize(final SchemaTable schemaTable, final >>>> JsonGenerator jsonGenerator, final SerializerProvider >>>> serializerProvider) >>>> throws IOException, JsonGenerationException { >>>> // when types are not embedded, stringify or resort to >>>> JSON primitive representations of the >>>> // type so that non-jvm languages can better >>>> interoperate with the TinkerPop stack. >>>> final Map<String, Object> m = new LinkedHashMap<>(); >>>> m.put("schema", schemaTable.getSchema()); >>>> m.put("table", schemaTable.getTable()); >>>> jsonGenerator.writeObject(m); >>>> } >>>> >>>> } >>>> >>>> Hope it all makes some sense, >>>> Pieter >>>> >>>> >>>> On 05/09/2017 17:31, Stephen Mallette wrote: >>>> >>>> I guess I'm trying to understand why it matters for purpose of the test. >>>>> If >>>>> you mix/match versions I can't think of why the test would care >>>>> one way or the other. does sqlg serialize its id to a JSON Map? >>>>> >>>>> On Tue, Sep 5, 2017 at 11:19 AM, pieter gmail >>>>> <pieter.mar...@gmail.com >>>>> > >>>>> wrote: >>>>> >>>>> I looked at TinkerGraph's implementation. In fact I copied it. >>>>> TinkerGraph >>>>> >>>>>> does not have any special id serialization. In fact both its >>>>>> TinkerIoRegistryV3d0 andTinkerIoRegistryV2d0 registry uses >>>>>> TinkerModuleV2d0. >>>>>> >>>>>> >>>>>> In IoCustomTest tests you call g.io(GraphSONIo.build(GraphSON >>>>>> Version.V2_0)).mapper().addCustomModule(moduleV2d0). >>>>>> I.e. the test manually registers the appropriate SimpleModule for >>>>>> each version. >>>>>> >>>>>> For IoEdgeTest the same needs to happen somehow. Currently I have >>>>>> only one default V3 SimpleModule same as TinkerGraph. I have >>>>>> written a V2 SimpleModule but how in IoEdgeTest will the correct >>>>>> IoRegistry or SimpleModule be selected? The test itself does not >>>>>> call >>>>>> addCustomModule() >>>>>> and between the builders, mappers, registries and modules I don't >>>>>> see how to add it. >>>>>> >>>>>> Thanks, >>>>>> Pieter >>>>>> >>>>>> On 05/09/2017 16:30, Stephen Mallette wrote: >>>>>> >>>>>> You have registries for each version as well and default to v3. >>>>>> Please >>>>>> >>>>>>> see >>>>>>> TinkerGraph: >>>>>>> >>>>>>> https://github.com/apache/tinkerpop/blob/master/tinkergraph- >>>>>>> gremlin/src/main/java/org/apache/tinkerpop/gremlin/ >>>>>>> tinkergraph/structure/TinkerGraph.java#L198 >>>>>>> >>>>>>> If the user wants to override that then that's their choice, but >>>>>>> they have to rig it all up. We probably need a better system >>>>>>> than this. IO is way too complicated and confusing. >>>>>>> >>>>>>> On Tue, Sep 5, 2017 at 9:52 AM, pieter gmail < >>>>>>> pieter.mar...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>> Afraid I still don't quite get it. How do I register the >>>>>>> different >>>>>>> >>>>>>> SimpleModules depending on the version. >>>>>>>> >>>>>>>> Currently it starts in SqlgGraph, >>>>>>>> >>>>>>>> @Override >>>>>>>> public <I extends Io> I io(final Io.Builder<I> builder) { >>>>>>>> return (I) builder.graph(this).onMapper(mapper -> >>>>>>>> mapper.addRegistry(SqlgIoRegistry.getInstance())).create(); >>>>>>>> } >>>>>>>> >>>>>>>> and the SqlgIoRegistry registers the SimpleModule >>>>>>>> >>>>>>>> private SqlgIoRegistry() { >>>>>>>> final SqlgSimpleModule sqlgSimpleModule = new >>>>>>>> SqlgSimpleModule(); >>>>>>>> register(GraphSONIo.class, null, sqlgSimpleModule); >>>>>>>> register(GryoIo.class, RecordId.class, null); } >>>>>>>> >>>>>>>> >>>>>>>> Is the SqlgGraph.io(...) method suppose to interrogate the >>>>>>>> builder to check the version and add a corresponding IoRegistry >>>>>>>> and SimpleModule? >>>>>>>> >>>>>>>> Thanks >>>>>>>> Pieter >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 05/09/2017 13:30, Stephen Mallette wrote: >>>>>>>> >>>>>>>> I think you should just create a new SimpleModule for each >>>>>>>> version - >>>>>>>> >>>>>>>> don't >>>>>>>>> try to put them in the same SqlgSimpleModule. It's a naming >>>>>>>>> convention that users will have to follow rather than >>>>>>>>> something explicitly enforced through code. >>>>>>>>> >>>>>>>>> On Sun, Sep 3, 2017 at 3:20 PM, pieter gmail < >>>>>>>>> pieter.mar...@gmail.com >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I am getting IO tests failures on 3.3.0. >>>>>>>>> >>>>>>>>>> Sqlg has a SimpleModule which add serializers for its custom id. >>>>>>>>>> >>>>>>>>>> SqlgSimpleModule() { >>>>>>>>>> super("custom"); >>>>>>>>>> // addSerializer(RecordId.class, new >>>>>>>>>> RecordId.RecordIdJacksonSerial izerV2d0()); >>>>>>>>>> // addDeserializer(RecordId.class, new >>>>>>>>>> RecordId.RecordIdJacksonDeserializerV2d0()); >>>>>>>>>> // addSerializer(SchemaTable.class, new >>>>>>>>>> SchemaTable.SchemaTableIdJacksonSerializerV2d0()); >>>>>>>>>> // addDeserializer(SchemaTable.class, new >>>>>>>>>> SchemaTable.SchemaTableIdJacksonDeserializerV2d0()); >>>>>>>>>> >>>>>>>>>> addSerializer(RecordId.class, new >>>>>>>>>> RecordId.RecordIdJacksonSerial izerV3d0()); >>>>>>>>>> addDeserializer(RecordId.class, new >>>>>>>>>> RecordId.RecordIdJacksonDeseri alizerV3d0()); >>>>>>>>>> addSerializer(SchemaTable.class, new >>>>>>>>>> SchemaTable.SchemaTableJacksonSerializerV3d0()); >>>>>>>>>> addDeserializer(SchemaTable.class, new >>>>>>>>>> SchemaTable.SchemaTableJacksonDeserializerV3d0()); >>>>>>>>>> } >>>>>>>>>> >>>>>>>>>> How is it suppose to distinguish between v2 and v3? >>>>>>>>>> >>>>>>>>>> An example of a failure is 'IoEdgeTest.shouldReadWriteEdge' >>>>>>>>>> >>>>>>>>>> If ...V2d0.. is added to the serializers then 'graphson-v3' fails. >>>>>>>>>> If ...V3d0.. is added to the serializers then 'graphson-v2' fails. >>>>>>>>>> >>>>>>>>>> TinkerPop's own CustomId tests do not rely on default >>>>>>>>>> behavior and manually creates SimpleModules for each scenario. >>>>>>>>>> >>>>>>>>>> Are they both suppose to work somehow? >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Pieter >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >> >