Igniters. Just to clarify the issue:
``` public class BinaryObjectTest extends GridCommonAbstractTest { /** */ @Test public void testArray() throws Exception { Ignite ign = startGrid(); IgniteCache<Integer, TestClass1[]> cache = ign.createCache("my-cache"); cache.put(1, new TestClass1[] {new TestClass1(), new TestClass1()}); TestClass1[] obj = cache.get(1); //This will fail with ClassCastException. assertEquals(TestClass1[].class, obj.getClass()); } } ``` > 19 мая 2021 г., в 13:04, Nikolay Izhikov <nizhikov....@gmail.com> написал(а): > > Thanks, Ilya. > > Can you put more context on this? > I don’t familiar with these issues. > >> 19 мая 2021 г., в 13:02, Ilya Kasnacheev <ilya.kasnach...@gmail.com> >> написал(а): >> >> Hello! >> >> Obvious issues are Lazy SQL, Event Driven Services, Sort Binary Object >> Fields. >> >> Regards, >> -- >> Ilya Kasnacheev >> >> >> ср, 19 мая 2021 г. в 12:56, Nikolay Izhikov <nizhi...@apache.org>: >> >>> Hello, >>> >>>> However, for internal platform and services implementations we should >>> fix the root cause: >>>> avoid extra deserialization->serialization pass completely. >>>> This will also improve performance. >>> >>> Pavel, thanks for the feedback. >>> If I understand correctly, your suggestion is to know data size at the >>> start of reading service parameters. >>> Is it correct? >>> >>> Right now, when the service method invoked we pass an array of parameters >>> through platform reader/writer machinery. >>> On java side parameters read one by one and we don’t know the size of the >>> data on the read start. >>> >>> AFAICU, this will require x2 memory on the .Net or thin client-side. >>> >>> >>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/platform/services/PlatformServices.java#L289 >>> >>> >>>> if we are to break compatibility, I would like to see it done for some >>> really common pain point. >>> >>> Ilya, can you, please, provide a list of common issues with Ignite that >>> can be resolved >>> only with compatibility breakage? >>> >>>> 4 мая 2021 г., в 12:58, Ilya Kasnacheev <ilya.kasnach...@gmail.com> >>> написал(а): >>>> >>>> Hello! >>>> >>>> If we really decide to break some compatibility then Array to >>> BinaryObject >>>> serialization will be very, very low on my personal list. >>>> >>>> I just don't see how this issue is relevant. I have been reading and >>>> answering user list for a few years now, and I don't remember a single >>>> question about storing ConcreteType[] as value and complaining about type >>>> information loss. >>>> >>>> If you have a good scenario how do we keep persistent store binary >>>> compatibility here, without adding a lot of new code and still checking >>> for >>>> both old and new approaches - you can go forward for sure. >>>> >>>> However, it does seem questionable where we have a new wrapper class >>>> specifically for top level arrays. You can have this wrapper in your own >>>> client code and it should work OK. >>>> >>>> Bottom line, if we are to break compatibility, I would like to see it >>> done >>>> for some really common pain point. >>>> >>>> Regards, >>>> -- >>>> Ilya Kasnacheev >>>> >>>> >>>> пт, 30 апр. 2021 г. в 17:34, Nikolay Izhikov <nizhi...@apache.org>: >>>> >>>>> Hello, Ilya. >>>>> >>>>> Thanks for the feedback! >>>>> >>>>>> For me it sounds like something we would like to do in 3.0 >>>>> >>>>> Ignite 3 is a very long way to go, so I prefer to target this fix in >>>>> Ignite 2.x. >>>>> >>>>>> I think making it default "true" is a breaking change and is not >>>>> possible in a minor release >>>>> >>>>> Yes, you are correct it is a breaking change. >>>>> It seems for me, we all agreed that breaking changes are possible in >>> minor >>>>> releases. >>>>> >>>>> Anyway, if we will decide do not to enable this feature by default it’s >>> OK >>>>> for me. >>>>> We still can implement it and improve the binary SerDe mechanism. >>>>> >>>>> >>>>>> 30 апр. 2021 г., в 17:23, Ilya Kasnacheev <ilya.kasnach...@gmail.com> >>>>> написал(а): >>>>>> >>>>>> Hello! >>>>>> >>>>>> For me it sounds like something we would like to do in 3.0 (if indeed >>> it >>>>>> will have arrays as possible value (or key) type), but doing it in 2.x >>>>>> raises concerns whether it has enough time left to stabilize. >>>>>> >>>>>> Also, I think making it default "true" is a breaking change and is not >>>>>> possible in a minor release, case in point, >>>>>> IGNITE_BINARY_SORT_OBJECT_FIELDS is still waiting for 3.0 and it is >>> less >>>>>> destructive. >>>>>> >>>>>> Of course I would also like to hear what other community members think. >>>>>> >>>>>> Regards, >>>>>> -- >>>>>> Ilya Kasnacheev >>>>>> >>>>>> >>>>>> пт, 30 апр. 2021 г. в 17:16, Nikolay Izhikov <nizhi...@apache.org>: >>>>>> >>>>>>> Igniters, >>>>>>> >>>>>>> Want to clarify my proposal about new array store format. >>>>>>> I think we should store array in special binary wrapper that will keep >>>>>>> original component type >>>>>>> >>>>>>> ``` >>>>>>> public class BinaryArrayWrapper implements BinaryObjectEx, >>>>> Externalizable { >>>>>>> /** Type ID. */ >>>>>>> private int compTypeId; >>>>>>> >>>>>>> /** Raw data. */ >>>>>>> private String compClsName; >>>>>>> >>>>>>> /** Value. */ >>>>>>> private Object[] arr; >>>>>>> >>>>>>> // Further implementation. >>>>>>> } >>>>>>> ``` >>>>>>> >>>>>>> >>>>>>>> 30 апр. 2021 г., в 16:31, Nikolay Izhikov <nizhikov....@gmail.com> >>>>>>> написал(а): >>>>>>>> >>>>>>>> Hello, Igniters. >>>>>>>> >>>>>>>> Currently, binary marshaller works as follows(Say, we have a class >>>>>>> `User` then): >>>>>>>> >>>>>>>> IgniteBinary#toBinary(User)` -> BinaryObject >>>>>>>> IgniteBinary#toBinary(User[])` -> Object[] >>>>>>>> IgniteBinary#toBinary(Object[])` -> Object[] >>>>>>>> >>>>>>>> This means, that we lose array component type information during >>> binary >>>>>>> serialization. >>>>>>>> AFAIK, it’s a design choice made during binary infrastructure >>>>>>> development. >>>>>>>> >>>>>>>> This lead to the following disadvantages: >>>>>>>> >>>>>>>> 1. `IgniteBinary` can’t be used as a universal SerDe mechanism. >>>>>>>> 2. Ignite internals(service grid, .Net calls) contains many tweaks >>> and >>>>>>> hacks to deal with custom user array and still has many issues [1] >>>>>>>> >>>>>>>> I propose to make breaking changes and fix the custom user array SeDe >>>>> as >>>>>>> follows: >>>>>>>> >>>>>>>> 1. Implement binary serialization that correctly Ser and Deser >>>>>>> array using some kind of the wrapper (BinaryArrayWrapper). >>>>>>>> >>>>>>>> IgniteBinary#toBinary(User)` -> BinaryObject >>>>>>>> IgniteBinary#toBinary(User[])` -> BinaryObject >>>>>>>> IgniteBinary#toBinary(Object[])` -> BinaryObject >>>>>>>> >>>>>>>> 2. Introduce system flag `IGNITE_USE_BINARY_ARRAY` that enables >>>>>>> correct SerDe of arrays. The default value is false to keep backward >>>>>>> compatibility in the next Ignite release(2.11). >>>>>>>> >>>>>>>> 3. Set `IGNITE_USE_BINARY_ARRAY` to `true` in the ongoing Ignite >>>>>>> releases (2.12). >>>>>>>> >>>>>>>> WDYT? >>>>>>>> >>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-14299 >>>>>>> >>>>>>> >>>>> >>>>> >>> >>> >