I think aliases should be fine. Vova, can you please provide an example of aliases, so we are all on the same page?
D. On Mon, Dec 21, 2015 at 7:03 AM, Vladimir Ozerov <voze...@gridgain.com> wrote: > Well, if having only aliases as a way to resolve such conflicts is fine, > then there is not need for the things I described. > > On Mon, Dec 21, 2015 at 5:49 PM, Dmitriy Setrakyan < > dsetrak...@gridgain.com> > wrote: > > > Are you talking about SQL queries? I thought that we agreed to use field > > aliases in case of name conflicts, no? > > > > D. > > > > > On Dec 21, 2015, at 4:57 AM, Vladimir Ozerov <voze...@gridgain.com> > > wrote: > > > > > > Several additional problems appeared: > > > 1) We must use fully-qualified class names because simple class names > > might > > > also be the same. E.g: class package1.A extends package2.A > > > 2) Our query engine doesn't like dots and "$" from class names. Because > > of > > > this fields with these characters cannot be queried. > > > > > > To workaorund these problems I did the following: > > > 1) Fully qualified class names are used; > > > 2) "." is replaced with "_" and "$" is replaced with "__". E.g. field > > > org.my.package.MyClass$MyInnerClass.x is written as " > > > org_my_package_MyClass__MyInnerClass_x"; > > > 3) If user would like to query the field, he can call special helper > > > method BinaryUtils.qualifiedFieldName(Class > > > cls, String fieldName) returning correct field name. > > > > > > As this problem is not likely to occur in real quering scenarios, I > think > > > this solution is more or less fine. > > > > > > Thoughts? > > > > > > > > > On Mon, Dec 21, 2015 at 12:28 PM, Sergey Kozlov <skoz...@gridgain.com> > > > wrote: > > > > > >> From my standpoint Vova's appoach very close to SQL behavior if two > > joined > > >> tables have same column names. > > >> > > >> On Mon, Dec 21, 2015 at 12:23 PM, Dmitriy Setrakyan < > > dsetrak...@apache.org > > >>> > > >> wrote: > > >> > > >>> Got it. I understand the reasoning now for not throwing an exception. > > >> Let’s > > >>> make sure we document this behavior. > > >>> > > >>> If there is no additional field-name-parsing overhead, then the > > proposed > > >>> API looks very nice. > > >>> > > >>> D. > > >>> > > >>> On Mon, Dec 21, 2015 at 1:19 AM, Vladimir Ozerov < > voze...@gridgain.com > > > > > >>> wrote: > > >>> > > >>>> Dima, > > >>>> > > >>>> Here is how proposed design works: > > >>>> > > >>>> class A { > > >>>> int x = 1; > > >>>> } > > >>>> class B { > > >>>> int x = 2; > > >>>> } > > >>>> > > >>>> BinaryObject obj = ...; > > >>>> > > >>>> Object val = obj.field("A.x"); // Returns "1"; > > >>>> Object val = obj.field("B.x"); // Returns "2"; > > >>>> Object val = obj.field("x"); // Returns null; > > >>>> > > >>>> boolean exists = obj.hasField("A.x"); // Returns "true"; > > >>>> boolean exists = obj.hasField("B.x"); // Returns "true"; > > >>>> boolean exists = obj.hasField("x"); // Returns "false"; > > >>>> > > >>>> Looks clean and consistent for me. Remember that we are talking > about > > >>> very > > >>>> specific use case. It is very unlikely that user will operate on > > >> objects > > >>>> conflicting fields in binary form. > > >>>> Also, there will be no parsing at all. We use field name passed by > > user > > >>>> directly. > > >>>> > > >>>> Vladimir. > > >>>> > > >>>> On Mon, Dec 21, 2015 at 12:10 PM, Dmitriy Setrakyan < > > >>> dsetrak...@apache.org > > >>>>> > > >>>> wrote: > > >>>> > > >>>>> Vova, > > >>>>> > > >>>>> We cannot return null in case of a conflict, as user won’t be able > to > > >>>>> differentiate between a conflict and missing field. We should throw > > >> an > > >>>>> exception. > > >>>>> > > >>>>> Also, I don’t like parsing field names looking for a dot for every > > >>> field. > > >>>>> It will introduce a performance overhead for the cases that do not > > >> have > > >>>>> conflicts. Instead, we should add another API for this use case, > > >>>> something > > >>>>> like field(typeName, fieldName). > > >>>>> > > >>>>> D. > > >>>>> > > >>>>> On Mon, Dec 21, 2015 at 1:01 AM, Vladimir Ozerov < > > >> voze...@gridgain.com > > >>>> > > >>>>> wrote: > > >>>>> > > >>>>>> Denis, > > >>>>>> Yes, as we do not know which field to pick, we return null. > > >>>>>> > > >>>>>> On Mon, Dec 21, 2015 at 12:00 PM, Denis Magda < > dma...@gridgain.com > > >>> > > >>>>> wrote: > > >>>>>> > > >>>>>>> Sounds good for me. I would go for this approach. > > >>>>>>> > > >>>>>>> In addition if to consider your example below and the user > > >> decides > > >>> to > > >>>>>> look > > >>>>>>> up a field by its simple name then he/she will get nothing or > > >>>> exception > > >>>>>>> (depending on the API), correct? > > >>>>>>> As an example for this case the method will return null > > >>>>>>> > > >>>>>>> BinaryObject obj = ...; > > >>>>>>> Object val = obj.field("x"); // null will be returned cause we > > >>> don't > > >>>>> know > > >>>>>>> what particular 'x' we have to return > > >>>>>>> > > >>>>>>> > > >>>>>>> -- > > >>>>>>> Denis > > >>>>>>> > > >>>>>>>> On 12/21/2015 11:48 AM, Vladimir Ozerov wrote: > > >>>>>>>> > > >>>>>>>> Folks, > > >>>>>>>> > > >>>>>>>> I thought about the solution a bit more and came to the > > >> following > > >>>>>> design. > > >>>>>>>> > > >>>>>>>> *Scenario:* > > >>>>>>>> class A { > > >>>>>>>> int x; > > >>>>>>>> } > > >>>>>>>> class B : extends A { > > >>>>>>>> int y; > > >>>>>>>> } > > >>>>>>>> > > >>>>>>>> *Solution:* > > >>>>>>>> 1) Field A.x is written as *"A.x"*, field B.x is written as > > >>> *"B.x"*. > > >>>>>> I.e. > > >>>>>>>> *both > > >>>>>>>> conflicting fields are prefixed* with simple name of the owning > > >>>> class. > > >>>>>>>> 2) API is unchanged. User manipulates these fields on all public > > >>>>> methods > > >>>>>>>> in > > >>>>>>>> exactly the same way: "A.x" and "B.x". > > >>>>>>>> > > >>>>>>>> *Rationale:* > > >>>>>>>> 1) We cannot prefix only some of conflicting fields. E.g. if > > >>> decide > > >>>> to > > >>>>>>>> prefix only A.x, then it is not clear how to handle this case: > > >>>>>>>> > > >>>>>>>> class B extends A implements Binarylizable { > > >>>>>>>> void write(BinaryWriter writer) { > > >>>>>>>> writer.writeInt("B.x", x); // User intentionally > > >> written > > >>>>> field > > >>>>>> as > > >>>>>>>> "B.x". > > >>>>>>>> } > > >>>>>>>> } > > >>>>>>>> > > >>>>>>>> BinaryObject obj = ...; > > >>>>>>>> Object val = obj.field("B.x"); // Should we lookup for "B.x" as > > >>> user > > >>>>>> asked > > >>>>>>>> us, or just for "x"? > > >>>>>>>> > > >>>>>>>> Prefixing all conflicting fields with class name resolves the > > >>>> problem. > > >>>>>>>> > > >>>>>>>> 2) If we add methods to manipulate fields not only by name, but > > >> by > > >>>>>>>> (typeName + fieldName) as well, then we will have to add *9 new > > >>>>> methods* > > >>>>>>>> to > > >>>>>>>> > > >>>>>>>> API: > > >>>>>>>> BinaryType.fieldTypeName(String typeName, String fieldName); > > >>>>>>>> BinaryType.field(String typeName, String fieldName); > > >>>>>>>> BinaryObject.field(String typeName, String fieldName); > > >>>>>>>> BinaryObject.hasField(String typeName, String fieldName); > > >>>>>>>> BinaryObjectBuilder.getField(String typeName, String fieldName); > > >>>>>>>> BinaryObjectBuilder.setField(String typeName, String fieldName, > > >>>> ...); > > >>>>>> // 3 > > >>>>>>>> overloads > > >>>>>>>> BinaryObjectBuilder.removeField(String typeName, String > > >>> fieldName); > > >>>>>>>> > > >>>>>>>> This is definitely an overkill for such a rare scenario. > > >>>>>>>> > > >>>>>>>> Thoughts? > > >>>>>>>> > > >>>>>>>> On Mon, Dec 21, 2015 at 11:22 AM, Vladimir Ozerov < > > >>>>> voze...@gridgain.com > > >>>>>>> > > >>>>>>>> wrote: > > >>>>>>>> > > >>>>>>>> Agree, prefixing parent class fields sound like a better option. > > >>>>>>>>> Regarding aliases - I need some time to understand internal > > >>>>> mechanics. > > >>>>>>>>> Will answer this a bit later. > > >>>>>>>>> > > >>>>>>>>> On Mon, Dec 21, 2015 at 11:18 AM, Dmitriy Setrakyan < > > >>>>>>>>> dsetrak...@apache.org > > >>>>>>>>> > > >>>>>>>>>> wrote: > > >>>>>>>>>> Vova, > > >>>>>>>>>> > > >>>>>>>>>> Shouldn’t it be the other way around? Class B writes the field > > >>> as > > >>>>> “a”, > > >>>>>>>>>> and > > >>>>>>>>>> class A writes it with a prefix (possibly the hash code of the > > >>>> class > > >>>>>>>>>> name). > > >>>>>>>>>> > > >>>>>>>>>> Also, we should clearly document how the SQL queries are > > >>> affected > > >>>> by > > >>>>>>>>>> this. > > >>>>>>>>>> AFAIK, we should be using field aliases here, no? > > >>>>>>>>>> > > >>>>>>>>>> D. > > >>>>>>>>>> > > >>>>>>>>>> On Sun, Dec 20, 2015 at 10:08 AM, Vladimir Ozerov < > > >>>>>> voze...@gridgain.com > > >>>>>>>>>>> > > >>>>>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>> May be we can use normal field names by default and add some > > >>>>>>>>>>> > > >>>>>>>>>> prefix/suffix > > >>>>>>>>>> > > >>>>>>>>>>> if conflict is found? E.g.: > > >>>>>>>>>>> > > >>>>>>>>>>> class A { > > >>>>>>>>>>> int a; // Write as "a"; > > >>>>>>>>>>> } > > >>>>>>>>>>> > > >>>>>>>>>>> class B extends A { > > >>>>>>>>>>> int a; // Write as "B_a"; > > >>>>>>>>>>> } > > >>>>>>>>>>> > > >>>>>>>>>>> On Fri, Dec 18, 2015 at 10:34 PM, Valentin Kulichenko < > > >>>>>>>>>>> valentin.kuliche...@gmail.com> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>> Folks, > > >>>>>>>>>>>> > > >>>>>>>>>>>> It seems to me that issue here is not with 3rd-party > > >>> libraries. > > >>>> We > > >>>>>>>>>>>> > > >>>>>>>>>>> just > > >>>>>>>>>> > > >>>>>>>>>>> don't properly support class hierarchy in binary format. Any > > >>>> class > > >>>>>>>>>>>> > > >>>>>>>>>>> that > > >>>>>>>>>> > > >>>>>>>>>>> extends another class and has the field with the same name as > > >>>>> parent > > >>>>>>>>>>>> > > >>>>>>>>>>> has > > >>>>>>>>>> > > >>>>>>>>>>> will fail unless user provides custom serialization logic > > >> that > > >>>> will > > >>>>>>>>>>>> > > >>>>>>>>>>> handle > > >>>>>>>>>>> > > >>>>>>>>>>>> it. > > >>>>>>>>>>>> > > >>>>>>>>>>>> What if we prepend the field name with the simple class name > > >>> in > > >>>>> this > > >>>>>>>>>>>> > > >>>>>>>>>>> case? > > >>>>>>>>>>> > > >>>>>>>>>>>> Say, we have two classes: > > >>>>>>>>>>>> > > >>>>>>>>>>>> class A { > > >>>>>>>>>>>> private int id; > > >>>>>>>>>>>> } > > >>>>>>>>>>>> > > >>>>>>>>>>>> class B extends A { > > >>>>>>>>>>>> private int id; > > >>>>>>>>>>>> } > > >>>>>>>>>>>> > > >>>>>>>>>>>> In this case we will get two fields: "A.id" and "B.id". The > > >>> only > > >>>>>>>>>>>> > > >>>>>>>>>>> issue is > > >>>>>>>>>> > > >>>>>>>>>>> that if there are no name conflict, we should be able to > > >>> resolve > > >>>> by > > >>>>>>>>>>>> > > >>>>>>>>>>> both > > >>>>>>>>>> > > >>>>>>>>>>> names - with or without prepended type name. I.e., if A is > > >>>>>> serialized, > > >>>>>>>>>>>> > > >>>>>>>>>>> you > > >>>>>>>>>>> > > >>>>>>>>>>>> can get the field value by "id" or "A.id". This is similar > > >> to > > >>>> how > > >>>>> it > > >>>>>>>>>>>> > > >>>>>>>>>>> works > > >>>>>>>>>>> > > >>>>>>>>>>>> if you join two SQL tables with the same column names. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Any thoughts on whether it's doable or not? > > >>>>>>>>>>>> > > >>>>>>>>>>>> -Val > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Fri, Dec 18, 2015 at 10:05 AM, Andrey Kornev < > > >>>>>>>>>>>> > > >>>>>>>>>>> andrewkor...@hotmail.com> > > >>>>>>>>>>> > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>>> > > >>>>>>>>>>>> In this particular case, the class that fails is a > > >> non-static > > >>>>> inner > > >>>>>>>>>>>>> > > >>>>>>>>>>>> class > > >>>>>>>>>>> > > >>>>>>>>>>>> that extends another non-static inner class, so they both > > >> end > > >>> up > > >>>>>>>>>>>>> > > >>>>>>>>>>>> having > > >>>>>>>>>> > > >>>>>>>>>>> the > > >>>>>>>>>>>> > > >>>>>>>>>>>>> compiler-generated "this$0" field. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Regards > > >>>>>>>>>>>>> Andrey > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Date: Fri, 18 Dec 2015 20:44:12 +0300 > > >>>>>>>>>>>>>> Subject: Re: CacheEntry serialization failure > > >>>>>>>>>>>>>> From: voze...@gridgain.com > > >>>>>>>>>>>>>> To: dev@ignite.apache.org > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> The most straightforward solution which comes to my mind - > > >>> *do > > >>>>> not > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> ever > > >>>>>>>>>>> > > >>>>>>>>>>>> use > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> BinaryMarshaller by default*. Always fallback to > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> OptimizedMarshaller > > >>>>>>>>>> > > >>>>>>>>>>> unless > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> user explicitly asked us to use binary format (e.g. > > >> through > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> package > > >>>>>>>>>> > > >>>>>>>>>>> wildcards). > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> BTW, we already do this for Externalizable and > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> readObject/writeObject. > > >>>>>>>>>>> > > >>>>>>>>>>>> On Fri, Dec 18, 2015 at 8:41 PM, Vladimir Ozerov < > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>> voze...@gridgain.com > > >>>>>>>>>>> > > >>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Ah, I saw your problem with DirectedSpecifics. We need to > > >>>> think > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>> about > > >>>>>>>>>>> > > >>>>>>>>>>>> how > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> to solve it. Here is the case: > > >>>>>>>>>>>>>>> 1) Class is Serilzable and cannot be changed; > > >>>>>>>>>>>>>>> 2) There are several duplicated field names; > > >>>>>>>>>>>>>>> => BinaryMarshaller cannot handle it. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Any thoughts? > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> On Fri, Dec 18, 2015 at 8:34 PM, Vladimir Ozerov < > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>> voze...@gridgain.com > > >>>>>>>>>>>> > > >>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> I fixed the problem, it was a bug actually. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> By default classes which has some custom Java logic > > >> (e.g. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Externalizable, > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> or with writeObject/readObject methods) will be written > > >>> using > > >>>>>>>>>>>>>>>> OptimizedMarshaller, so similar field names is not a > > >>>> problem. > > >>>>>>>>>>>>>>>> If you want to serialize such class in binary format and > > >>>> have > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> duplicate > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> field names, you should provide your own BinarySerializer, > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> which > > >>>>>>>>>> > > >>>>>>>>>>> will > > >>>>>>>>>>>> > > >>>>>>>>>>>>> write > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> these fields with different names. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> On Fri, Dec 18, 2015 at 8:07 PM, Andrey Kornev < > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> andrewkor...@hotmail.com> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> How am I supposed to handle this situation if the class > > >>>> comes > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> from > > >>>>>>>>>>> > > >>>>>>>>>>>> a > > >>>>>>>>>>>> > > >>>>>>>>>>>>> 3d > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> party I can't modify? > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> Thanks > > >>>>>>>>>>>>>>>>> Andrey > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> Date: Fri, 18 Dec 2015 09:12:22 +0300 > > >>>>>>>>>>>>>>>>>> Subject: Re: CacheEntry serialization failure > > >>>>>>>>>>>>>>>>>> From: voze...@gridgain.com > > >>>>>>>>>>>>>>>>>> To: dev@ignite.apache.org > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> I'll take a look. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> On Fri, Dec 18, 2015 at 4:37 AM, Valentin Kulichenko < > > >>>>>>>>>>>>>>>>>> valentin.kuliche...@gmail.com> wrote: > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Folks, > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> It looks like CacheEntry implementation (i.e., the > > >>> entry > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> that > > >>>>>>>>>> > > >>>>>>>>>>> contains > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> version) can't be properly serialized with the > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> BinaryMarshaller. > > >>>>>>>>>>>> > > >>>>>>>>>>>>> I > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> created > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> the test and the ticket: > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-2203 > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Can someone take a look? > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> -Val > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > >> > > >> > > >> -- > > >> Sergey Kozlov > > >> > > >