Re: [DISCUSSION] Array to BinaryObject serialization

2021-11-02 Thread Nikolay Izhikov
Hello, Igniters. I've prepared implementation of support of BinaryArray. Please, join the review. Ticket - https://issues.apache.org/jira/browse/IGNITE-14742 PR - https://github.com/apache/ignite/pull/9490 ср, 19 мая 2021 г. в 14:44, Ilya Kasnacheev : > Hello! > > We don't have to do it in to

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-19 Thread Ilya Kasnacheev
Hello! We don't have to do it in toBinary() - we may preserve its behavior if you like, and only do that transition for Cache API, where additional support may be added to e.g. org.apache.ignite.internal.processors.cache.binary.CacheObjectBinaryProcessorImpl Regards, -- Ilya Kasnacheev ср, 19

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-19 Thread Nikolay Izhikov
Ilya. Actually, current behaviour described even in documentation [1] > Note that not all objects are converted to the binary object format. The > following classes are never converted (e.g., the toBinary(Object) method > returns the original object, and instances of these classes are stored

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-19 Thread Ilya Kasnacheev
Hello! Why do you need to take compatibility into account here at all? You can start writing entries to cache in new format while reading both old and new format. I consider returning Object[] instead of ConcreteType[] a bug so it's totally OK to start returning the "better" ConcreteType[] instea

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-19 Thread Nikolay Izhikov
Ilya, > Maybe we should just automate that, e.g., whenever user stores Type[], always > store one-field ArrayWrapper object, and automatically unwrap it on get() Yes, I’m talking about this opportunity from the beginning of the thread. > 1. Implement binary serialization that correctly Ser and

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-19 Thread Ilya Kasnacheev
Hello! Yes, it does not look pretty, I agree. But I'm saying that I've not encountered this issue on the user list before, and it can probably be easily countered by storing some one-field ArrayWrapper object in the cache as value. Maybe we should just automate that, e.g., whenever user stores Ty

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-19 Thread Nikolay Izhikov
Igniters. Just to clarify the issue: ``` public class BinaryObjectTest extends GridCommonAbstractTest { /** */ @Test public void testArray() throws Exception { Ignite ign = startGrid(); IgniteCache cache = ign.createCache("my-cache"); cache.put(1, new TestCla

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-19 Thread Nikolay Izhikov
Thanks, Ilya. Can you put more context on this? I don’t familiar with these issues. > 19 мая 2021 г., в 13:02, Ilya Kasnacheev > написал(а): > > Hello! > > Obvious issues are Lazy SQL, Event Driven Services, Sort Binary Object > Fields. > > Regards, > -- > Ilya Kasnacheev > > > ср, 19 ма

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-19 Thread Ilya Kasnacheev
Hello! Obvious issues are Lazy SQL, Event Driven Services, Sort Binary Object Fields. Regards, -- Ilya Kasnacheev ср, 19 мая 2021 г. в 12:56, Nikolay Izhikov : > Hello, > > > However, for internal platform and services implementations we should > fix the root cause: > > avoid extra deserializ

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-19 Thread Nikolay Izhikov
Hello, > However, for internal platform and services implementations we should fix the > root cause: > avoid extra deserialization->serialization pass completely. > This will also improve performance. Pavel, thanks for the feedback. If I understand correctly, your suggestion is to know data siz

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-04 Thread Ilya Kasnacheev
Hello! If we really decide to break some compatibility then Array to BinaryObject serialization will be very, very low on my personal list. I just don't see how this issue is relevant. I have been reading and answering user list for a few years now, and I don't remember a single question about st

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-03 Thread Pavel Tupitsyn
Nikolay, I agree that our binary array handling has some limitations - Ignite loses array element type in many cases (cache.Put -> cache.Get, Binary Mode, etc). However, for internal platform and services implementations we should fix the root cause: avoid extra deserialization->serialization pas

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-01 Thread Valentin Kulichenko
Hi Ivan, Thanks for chiming in. My comments are below. -Val On Sat, May 1, 2021 at 12:20 AM Ivan Daschinsky wrote: > Hi! > First of all, when array is serialized, marshaller actually DO > PRESERVE type of element (seel > org.apache.ignite.internal.binary.BinaryUtils#doReadObjectArray and > org

Re: [DISCUSSION] Array to BinaryObject serialization

2021-05-01 Thread Ivan Daschinsky
Hi! First of all, when array is serialized, marshaller actually DO PRESERVE type of element (seel org.apache.ignite.internal.binary.BinaryUtils#doReadObjectArray and org.apache.ignite.internal.binary.BinaryWriterExImpl#doWriteObjectArray). AFAIK, the motivation of Nickolay proposal, is the fact, th

Re: [DISCUSSION] Array to BinaryObject serialization

2021-04-30 Thread Valentin Kulichenko
Hi Nikolay, Is there a specific motivation behind your proposal? I do acknowledge that the semantics of the toBinary method is a little weird, but my concern is that the way it works with arrays is just an example. Are you suggesting changing collections, primitives, and other "first citizen" data

Re: [DISCUSSION] Array to BinaryObject serialization

2021-04-30 Thread Nikolay Izhikov
Hello, Ilya. Thanks for the feedback! > For me it sounds like something we would like to do in 3.0 Ignite 3 is a very long way to go, so I prefer to target this fix in Ignite 2.x. > I think making it default "true" is a breaking change and is not possible in > a minor release Yes, you are cor

Re: [DISCUSSION] Array to BinaryObject serialization

2021-04-30 Thread Ilya Kasnacheev
Hello! For me it sounds like something we would like to do in 3.0 (if indeed it will have arrays as possible value (or key) type), but doing it in 2.x raises concerns whether it has enough time left to stabilize. Also, I think making it default "true" is a breaking change and is not possible in a

Re: [DISCUSSION] Array to BinaryObject serialization

2021-04-30 Thread Nikolay Izhikov
Igniters, Want to clarify my proposal about new array store format. I think we should store array in special binary wrapper that will keep original component type ``` public class BinaryArrayWrapper implements BinaryObjectEx, Externalizable { /** Type ID. */ private int compTypeId;

[DISCUSSION] Array to BinaryObject serialization

2021-04-30 Thread Nikolay Izhikov
Hello, Igniters. Currently, binary marshaller works as follows(Say, we have a class `User` then): IgniteBinary#toBinary(User)` -> BinaryObject IgniteBinary#toBinary(User[])` -> Object[] IgniteBinary#toBinary(Object[])` -> Object[] This means, that we lose array component type information during