Ivan, > why do you use PooledMessageBufferOutput in benchmarks?
To make it fair. Ignite uses thread-local reusable buffers, see [1]. > why packer from msgpack-core show better performance than > BinaryWriter. And I suppose that benchmark is not quite fair. MsgPack writes and reads less bytes, so it should be faster. Benchmark is not 100% fair, there are some small extra things that BinaryWriter does. However: 1. I don't think we care about super-precise benchmarks here, just the ballpark. 2. We are discussing the format, not the implementation. Important takeaway is: The format does not prevent someone from implementing it efficiently. [1] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryWriterExImpl.java#L101 On Thu, Jun 17, 2021 at 10:40 PM Ivan Daschinsky <ivanda...@gmail.com> wrote: > Andrey, here we discuss serialization format, as far as I understand. > Current implementation of ignite binary object serialization can be > rewritten. > If we do not care about fast (O(1)) field lookup, about schema validation > and so on, msgpack is a really good option. It is also good for client > binary protocol, i.e. > tarantool uses it. > > >> Binarilizable interface forces user to write serialization code > I am talking about speed comparison. You can see from Pavel's data, > jackson-msgpack shows a pathetic performance comparing with a ignite's > default binary marshaller. If you want really fast serialization -- the > only option is to write code by yourself or use code generation. Default > packer from msgpack-core java package is similar to BinaryWriter. So I am > wondering why packer from msgpack-core show better performance than > BinaryWriter. And I suppose that benchmark is not quite fair. > > > чт, 17 июн. 2021 г. в 22:19, Andrey Mashenkov <andrey.mashen...@gmail.com > >: > > > Ivan, thankd for clarification. > > > > Binarilizable interface forces user to write serialization code. We can > > support this or similar interface. > > But I'd like Ignite has some default serializer in addition. It can be > also > > useful e.g. in compute for param and result serialization. > > > > BinaryObjectBuider requires an Ignite node for object construction, but > we > > are looking for a detached builder and won't care about schemas. > > > > AFAIR, BinaryObject creates an objectReader on every single field read > > operation. > > So, BO solution produces a lot of garbage and BO has noticable overhead > > which affects the object footprint. > > > > чт, 17 июн. 2021 г., 21:41 Ivan Daschinsky <ivanda...@gmail.com>: > > > > > >> Double checked -- there is not any links to PR either in IEP or in > > jira > > > issue > > > Sorry, there is a link in IEP, but not in jira ticket. > > > > > > чт, 17 июн. 2021 г. в 21:39, Ivan Daschinsky <ivanda...@gmail.com>: > > > > > > > Andrey, > > > > >> arbitrary object graph > > > > Also, that is not true, msgpack format doesn't handle circular > graphs. > > > > Think about msgpack as binary json. You couldn't understand full > > > structure > > > > of message if you didn't deserialize it fully before, maps and arrays > > are > > > > serialized just as contiguos chunks > > > > of values/kv-pairs. Msgpack is a really dumb and simple format. > > > > > > > > Also, as for me, I cannot understand why current ignite serialization > > > > (BinaryObjectBuilder or Binarilizable) is slower than raw message > pack > > > > serializer. > > > > I suppose that this is an issue and we should investigate it. > > > > > > > > Pavel, why do you use PooledMessageBufferOutput in benchmarks? I'm > > > > sorry, but is it fair to use it? > > > > > > > > >> The code is linked in the IEP [2] > > > > Double checked -- there is not any links to PR either in IEP or in > jira > > > > issue > > > > > > > > > > > > > -- > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > -- > Sincerely yours, Ivan Daschinskiy >
