Thanks, Ilya. Can you put more context on this? I don’t familiar with these issues.
> 19 мая 2021 г., в 13:02, Ilya Kasnacheev <ilya.kasnach...@gmail.com> > написал(а): > > Hello! > > Obvious issues are Lazy SQL, Event Driven Services, Sort Binary Object > Fields. > > Regards, > -- > Ilya Kasnacheev > > > ср, 19 мая 2021 г. в 12:56, Nikolay Izhikov <nizhi...@apache.org>: > >> Hello, >> >>> However, for internal platform and services implementations we should >> fix the root cause: >>> avoid extra deserialization->serialization pass completely. >>> This will also improve performance. >> >> Pavel, thanks for the feedback. >> If I understand correctly, your suggestion is to know data size at the >> start of reading service parameters. >> Is it correct? >> >> Right now, when the service method invoked we pass an array of parameters >> through platform reader/writer machinery. >> On java side parameters read one by one and we don’t know the size of the >> data on the read start. >> >> AFAICU, this will require x2 memory on the .Net or thin client-side. >> >> >> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/platform/services/PlatformServices.java#L289 >> >> >>> if we are to break compatibility, I would like to see it done for some >> really common pain point. >> >> Ilya, can you, please, provide a list of common issues with Ignite that >> can be resolved >> only with compatibility breakage? >> >>> 4 мая 2021 г., в 12:58, Ilya Kasnacheev <ilya.kasnach...@gmail.com> >> написал(а): >>> >>> Hello! >>> >>> If we really decide to break some compatibility then Array to >> BinaryObject >>> serialization will be very, very low on my personal list. >>> >>> I just don't see how this issue is relevant. I have been reading and >>> answering user list for a few years now, and I don't remember a single >>> question about storing ConcreteType[] as value and complaining about type >>> information loss. >>> >>> If you have a good scenario how do we keep persistent store binary >>> compatibility here, without adding a lot of new code and still checking >> for >>> both old and new approaches - you can go forward for sure. >>> >>> However, it does seem questionable where we have a new wrapper class >>> specifically for top level arrays. You can have this wrapper in your own >>> client code and it should work OK. >>> >>> Bottom line, if we are to break compatibility, I would like to see it >> done >>> for some really common pain point. >>> >>> Regards, >>> -- >>> Ilya Kasnacheev >>> >>> >>> пт, 30 апр. 2021 г. в 17:34, Nikolay Izhikov <nizhi...@apache.org>: >>> >>>> Hello, Ilya. >>>> >>>> Thanks for the feedback! >>>> >>>>> For me it sounds like something we would like to do in 3.0 >>>> >>>> Ignite 3 is a very long way to go, so I prefer to target this fix in >>>> Ignite 2.x. >>>> >>>>> I think making it default "true" is a breaking change and is not >>>> possible in a minor release >>>> >>>> Yes, you are correct it is a breaking change. >>>> It seems for me, we all agreed that breaking changes are possible in >> minor >>>> releases. >>>> >>>> Anyway, if we will decide do not to enable this feature by default it’s >> OK >>>> for me. >>>> We still can implement it and improve the binary SerDe mechanism. >>>> >>>> >>>>> 30 апр. 2021 г., в 17:23, Ilya Kasnacheev <ilya.kasnach...@gmail.com> >>>> написал(а): >>>>> >>>>> Hello! >>>>> >>>>> For me it sounds like something we would like to do in 3.0 (if indeed >> it >>>>> will have arrays as possible value (or key) type), but doing it in 2.x >>>>> raises concerns whether it has enough time left to stabilize. >>>>> >>>>> Also, I think making it default "true" is a breaking change and is not >>>>> possible in a minor release, case in point, >>>>> IGNITE_BINARY_SORT_OBJECT_FIELDS is still waiting for 3.0 and it is >> less >>>>> destructive. >>>>> >>>>> Of course I would also like to hear what other community members think. >>>>> >>>>> Regards, >>>>> -- >>>>> Ilya Kasnacheev >>>>> >>>>> >>>>> пт, 30 апр. 2021 г. в 17:16, Nikolay Izhikov <nizhi...@apache.org>: >>>>> >>>>>> Igniters, >>>>>> >>>>>> Want to clarify my proposal about new array store format. >>>>>> I think we should store array in special binary wrapper that will keep >>>>>> original component type >>>>>> >>>>>> ``` >>>>>> public class BinaryArrayWrapper implements BinaryObjectEx, >>>> Externalizable { >>>>>> /** Type ID. */ >>>>>> private int compTypeId; >>>>>> >>>>>> /** Raw data. */ >>>>>> private String compClsName; >>>>>> >>>>>> /** Value. */ >>>>>> private Object[] arr; >>>>>> >>>>>> // Further implementation. >>>>>> } >>>>>> ``` >>>>>> >>>>>> >>>>>>> 30 апр. 2021 г., в 16:31, Nikolay Izhikov <nizhikov....@gmail.com> >>>>>> написал(а): >>>>>>> >>>>>>> Hello, Igniters. >>>>>>> >>>>>>> Currently, binary marshaller works as follows(Say, we have a class >>>>>> `User` then): >>>>>>> >>>>>>> IgniteBinary#toBinary(User)` -> BinaryObject >>>>>>> IgniteBinary#toBinary(User[])` -> Object[] >>>>>>> IgniteBinary#toBinary(Object[])` -> Object[] >>>>>>> >>>>>>> This means, that we lose array component type information during >> binary >>>>>> serialization. >>>>>>> AFAIK, it’s a design choice made during binary infrastructure >>>>>> development. >>>>>>> >>>>>>> This lead to the following disadvantages: >>>>>>> >>>>>>> 1. `IgniteBinary` can’t be used as a universal SerDe mechanism. >>>>>>> 2. Ignite internals(service grid, .Net calls) contains many tweaks >> and >>>>>> hacks to deal with custom user array and still has many issues [1] >>>>>>> >>>>>>> I propose to make breaking changes and fix the custom user array SeDe >>>> as >>>>>> follows: >>>>>>> >>>>>>> 1. Implement binary serialization that correctly Ser and Deser >>>>>> array using some kind of the wrapper (BinaryArrayWrapper). >>>>>>> >>>>>>> IgniteBinary#toBinary(User)` -> BinaryObject >>>>>>> IgniteBinary#toBinary(User[])` -> BinaryObject >>>>>>> IgniteBinary#toBinary(Object[])` -> BinaryObject >>>>>>> >>>>>>> 2. Introduce system flag `IGNITE_USE_BINARY_ARRAY` that enables >>>>>> correct SerDe of arrays. The default value is false to keep backward >>>>>> compatibility in the next Ignite release(2.11). >>>>>>> >>>>>>> 3. Set `IGNITE_USE_BINARY_ARRAY` to `true` in the ongoing Ignite >>>>>> releases (2.12). >>>>>>> >>>>>>> WDYT? >>>>>>> >>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-14299 >>>>>> >>>>>> >>>> >>>> >> >>
