Hi, I thought it would be easier to base this on some code, so I submitted a patch to get us started:
https://issues.apache.org/jira/browse/FELIX-5332 <https://issues.apache.org/jira/browse/FELIX-5332> Cheers, =David > On Aug 24, 2016, at 6:07 AM, David Leangen <[email protected]> wrote: > > > Hi David B., > > You are definitely right: when you don’t need the type information > serialised, it would be much too noisy and intrusive. If you need it, it > would be very handy. The default should be that it not be there (i.e. as it > works currently). If there were a way to configure to have the type > information serialised, personally I think it would be a very nice feature. > > I’m not sure if via ConfigAdmin would be the right path. IIUC, it would make > it “all or nothing”. I think this type serialisation is required only in > certain circumstances. I’m not sure how it could be done, but is there a way > of using something like an Adaptor, or something else as a one-off mechanism? > > > Cheers, > =David > > >> On Aug 24, 2016, at 12:35 AM, David Bosschaert <[email protected]> >> wrote: >> >> Hi David, >> >> In that case you'd need to store the type information somewhere which, >> depending on the target serialization format may or may not be possible. >> For example in plain JSON you don't really have a place to store such >> metadata, although you could add it in a 'special' "__metadata" key or >> something. >> >> I think this may be a use-case that could for some people be handy but for >> others it might pollute the generated serialization with stuff that they >> don't really want to see. >> >> I guess it could be a special feature of a certain Codec. Codecs could be >> configured via some mechanism (e.g. ConfigAdmin) to support this feature. >> Then you should be able to get the behaviour you are looking for by calling: >> >> MyDTO dto = (MyDTO) myCodec.decode(Object.class).from(json) >> >> So my feeling is that something like this might already be possible given >> the current API, although not every codec might support it. Maybe we can >> try to expand one of the codecs that is in the Felix codebase to support >> this enabled via a configuration setting? >> >> Thoughts? >> >> David >> >> On 23 August 2016 at 01:50, David Leangen <[email protected]> wrote: >> >>> >>> Thanks, David B., >>> >>> The use cases are indeed very similar, but for one difference, I think. >>> This line: >>> >>> MyDTO dto2 = jsonCodec.decode(MyDTO.class).from(json); >>> >>> You see, when you decode from JSON in this line, you already know in >>> advance that you are deserialising a MyDTO object, so you feed it into the >>> decoder. Easy schmeasy. But what do you do if you don’t know in advance >>> what type the object is? That is the problem with deserialisation. >>> >>> A generic serialiser has this very problem. Its job is to serialise (very >>> easy) and deserialise (difficult because it somehow needs to know what the >>> type is). If this were XML, you would somehow need to first determine the >>> schema before you could deserialise into a statically-typed object. If you >>> know the schema, then you can convert. JSON is no different. What is really >>> cool about the DTO, though, is that the DTO class becomes the schema. So, >>> all we would need to do so solve this problem is serialise the DTO type >>> along with the data. Upon deserialisation, we could instantiate a class >>> object (assuming with Class.forName(“dtoTypeName”)), then we’re good. >>> >>> So, my suggestion is to, as part of the API, serialise the type >>> information along with the data. That way, when calling the API, however, >>> all you would need to do to deserialise is this: >>> >>> // Note the absence of type information, which is different from the >>> line above. >>> // The line above requires knowledge of the type in advance. >>> Object dto2 = jsonCodec.deserializeFrom(json); >>> >>> >>> Of course, this cruft could be done outside of the service. However, it >>> seems like an important use case to consider, and would make the >>> DTO/Converter/Codec more complete and more compelling, for such a small >>> addition. >>> >>> wdyt? >>> >>> Cheers, >>> =David >>> >>> >>>> On Aug 23, 2016, at 6:23 AM, David Bosschaert < >>> [email protected]> wrote: >>>> >>>> Hi David, >>>> >>>> Maybe I'm missing something, but isn't this the same use-case as what is >>>> done in the unit test JsonCodecTest.testDTO()? [1] >>>> >>>> Basically there it has a DTO with an embedded DTO. >>>> This is then converted into JSON with the JSON codec. The JSON contains >>> the >>>> embedded content as an embedded JSON object. >>>> Then that JSON is converted back into the DTO and the embedded DTO is >>>> automatically filled as part of that process. >>>> >>>> Or is your case different? >>>> >>>> Cheers, >>>> >>>> David >>>> >>>> [1] >>>> https://svn.apache.org/viewvc/felix/trunk/converter/src/ >>> test/java/org/apache/felix/converter/impl/json/JsonCodecTest.java?view= >>> markup#l102 >>>> >>>> On 21 August 2016 at 06:46, David Leangen <[email protected]> wrote: >>>> >>>>> >>>>> Hi Johan, >>>>> >>>>> Thanks for your thoughts. Are you able to elaborate a little more? >>>>> >>>>>> Whenever you serialize and say “Hey’ let’s use polymorphism", you are >>>>> pretty >>>>>> much bound to fail. >>>>> >>>>> I’m not sure why you say this. I thought that by including the schema >>>>> information, that is exactly what we were avoiding… >>>>> >>>>> I’m not sure I’m getting your idea about the command pattern. Would you >>>>> mind showing me a quick example? >>>>> >>>>> Cheers, >>>>> =David >>>>> >>>>> >>>>> >>>>>> On Aug 21, 2016, at 2:42 PM, Johan Edstrom <[email protected]> wrote: >>>>>> >>>>>> Whenever you serialize and say “Hey’ let’s use polymorphism", you are >>>>> pretty >>>>>> much bound to fail. >>>>>> >>>>>> Use something like a command pattern identifying what you are doing >>>>>> so you can pick the de-serializer. >>>>>> >>>>>> >>>>>> >>>>>>> On Aug 20, 2016, at 11:32 PM, David Leangen <[email protected]> wrote: >>>>>>> >>>>>>> >>>>>>> I had a few thoughts about this topic. >>>>>>> >>>>>>> In order to properly deserialise, we would need the schema. Since the >>>>> DTO type *is* the schema, then somehow the schema needs to be >>> serialised as >>>>> well, or at least known upon serialisation. >>>>>>> >>>>>>> If the API were to have a toSchema() method, that would simplify >>> things >>>>> a lot. A client should also be able to do something like this: >>>>>>> >>>>>>> SomeDTO dto = ... >>>>>>> codec.encode( dto ).serializeTo( out ); >>>>>>> >>>>>>> Or maybe: >>>>>>> >>>>>>> codec.withSchema( SomeDTO.class).encode( dto ).to( out ); >>>>>>> >>>>>>> The SomeDTO schema would also be serialised (perhaps simply using the >>>>> fully-qualified class name as an alias?), so then the client could do >>> this: >>>>>>> >>>>>>> Object o = codec.deserializeFrom( in ) >>>>>>> >>>>>>> Since the schema is included in the input data, the serializer will >>>>> behind the scene do something like: >>>>>>> >>>>>>> >>>>>>> Map<String, Object> map = codec.decode( Map.class ).from( in ); >>>>>>> // Get the schema type, which is SomeDTO.class >>>>>>> ... >>>>>>> Object o = converter.convert( map.get(“data”) ).to( SomeDTO.class ); >>>>>>> >>>>>>> >>>>>>> wdyt? >>>>>>> >>>>>>> =David >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On Aug 20, 2016, at 3:23 PM, David Leangen <[email protected]> wrote: >>>>>>>> >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I am trying to use the Convert/Codec as a serializer, but it has been >>>>> a bit of a struggle so far. >>>>>>>> >>>>>>>> I have a “deep” object structure (i.e. at least one level of embedded >>>>> objects). Writing the data as a JSON string works just fine. However, >>> when >>>>> deserialising, I am having trouble. The data object uses generics, so >>> it is >>>>> not possible to determine the type at runtime. My embedded objects end >>> up >>>>> being deserialised as Maps, which of course causes a ClassCastException >>>>> later on in the code. >>>>>>>> >>>>>>>> I have tried adding a Rule using an adapter, and setting the >>>>> codec.with(thatAdapter), but that didn’t work out so well, either. >>>>>>>> >>>>>>>> Here is an example of my attempt so far. I am working with Prevayler, >>>>> so I need to serialize/deserialze a command object, which contains some >>>>> data. >>>>>>>> >>>>>>>> I think that this code _should_ work, which means that there is >>> likely >>>>> a bug in the code. Even so… >>>>>>>> >>>>>>>> This is a very convoluted way of working, which doesn’t seem right to >>>>> me. Am I missing some concept? >>>>>>>> >>>>>>>> >>>>>>>> Example below. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> =David >>>>>>>> >>>>>>>> >>>>>>>> public class DTOSerializer<E> >>>>>>>> implements Serializer // This is the interface provided by >>>>> Prevayler >>>>>>>> { >>>>>>>> private final Converter converter; >>>>>>>> private final Codec codec; >>>>>>>> private final Class<E> type; >>>>>>>> >>>>>>>> public DTOSerializer( Converter aConverter, Codec aCodec, Class<E> >>>>> aType ) >>>>>>>> { >>>>>>>> // Very convoluted attempt at adding a rule to try to make the >>>>> conversion of my “PutCommand” work during deserialization >>>>>>>> converter = aConverter.getAdapter() >>>>>>>> .rule( >>>>>>>> PutCommand.class, >>>>>>>> Map.class, >>>>>>>> dto -> { Map map = new HashMap<>(); map.put( >>>>> "key", dto.key ); map.put( "entity", dto.entity ); return map; }, >>>>>>>> map -> { PutCommand c = new PutCommand(); c.key = >>>>> (String)map.get( "key" ); c.entity = aConverter.convert( map.get( >>> "entity" >>>>> ) ).to( aType ); return c; } ); >>>>>>>> codec = aCodec.with( converter ); >>>>>>>> } >>>>>>>> >>>>>>>> @Override >>>>>>>> public Object readObject( InputStream in ) >>>>>>>> throws Exception >>>>>>>> { >>>>>>>> // This is the raw data >>>>>>>> final Map<String, Object> map = codec.decode( Map.class ).from( >>>>> in ); >>>>>>>> // The name of the object type to cast to >>>>>>>> final String commandTypeName = (String)map.get( "command" ); >>>>>>>> final Class<?> commandType = Class.forName( commandTypeName ); >>>>>>>> // This indeed returns an object of the correct type, >>>>>>>> // HOWEVER, the embedded objects are not of the correct type, they >>>>> are of type Map >>>>>>>> final Object command = converter.convert( map.get( "payload" ) >>>>> ).to( commandType ); >>>>>>>> return command; >>>>>>>> } >>>>>>>> >>>>>>>> @Override >>>>>>>> public void writeObject( OutputStream out, Object object ) >>>>>>>> throws Exception >>>>>>>> { >>>>>>>> final Map<String, Object> map = new HashMap<>(); >>>>>>>> // Serialize the name of the command so I know what to cast it do >>>>> when deserializing >>>>>>>> map.put( "command", object.getClass().getName() ); >>>>>>>> // This is the actual payload, which is nothing more than the >>>>> command object >>>>>>>> map.put( "payload", object ); >>>>>>>> codec.encode( map ).to( out ); >>>>>>>> } >>>>>>>> } >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>> >>> >
