Igniters, Looks like there are no objections and we can accept the proposal. I will close it tomorrow and move on to the thin client protocol itself.
On Fri, Jun 18, 2021 at 12:10 PM Ivan Daschinsky <ivanda...@gmail.com> wrote: > >> To make it fair. Ignite uses thread-local reusable buffers, see [1]. > I know, but PooledMessageBufferOutput is not about thread-local, isn't it? > > I'm not against about MsgPack, I'm for fair and not biased comparison. > > I suppose that MsgPack is an ideal candidate for thin client binary > protocol, not only for serializing some user data. > I am analyzed some tarantool connectors [1] [2] [3] and found msgpack based > protocol is a really good idea. > Also there is realy super fast and just 1 header library with liberal BSD-2 > licence for C -- msgpuck [4]. It used in tarantool itself > and in [1] and is stable and bullet proof. > > [1] -- https://github.com/igorcoding/asynctnt > [2] -- https://github.com/tarantool/tarantool-python/ > [3] -- https://github.com/tarantool/go-tarantool > [4] -- https://github.com/rtsisyk/msgpuck > > пт, 18 июн. 2021 г. в 11:44, Pavel Tupitsyn <ptupit...@apache.org>: > > > Ivan, > > > > > why do you use PooledMessageBufferOutput in benchmarks? > > > > To make it fair. Ignite uses thread-local reusable buffers, see [1]. > > > > > > > why packer from msgpack-core show better performance than > > > BinaryWriter. And I suppose that benchmark is not quite fair. > > > > MsgPack writes and reads less bytes, so it should be faster. > > Benchmark is not 100% fair, there are some small extra things that > > BinaryWriter does. > > > > However: > > 1. I don't think we care about super-precise benchmarks here, just the > > ballpark. > > 2. We are discussing the format, not the implementation. > > > > Important takeaway is: > > The format does not prevent someone from implementing it efficiently. > > > > > > > > [1] > > > > > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryWriterExImpl.java#L101 > > > > On Thu, Jun 17, 2021 at 10:40 PM Ivan Daschinsky <ivanda...@gmail.com> > > wrote: > > > > > Andrey, here we discuss serialization format, as far as I understand. > > > Current implementation of ignite binary object serialization can be > > > rewritten. > > > If we do not care about fast (O(1)) field lookup, about schema > validation > > > and so on, msgpack is a really good option. It is also good for client > > > binary protocol, i.e. > > > tarantool uses it. > > > > > > >> Binarilizable interface forces user to write serialization code > > > I am talking about speed comparison. You can see from Pavel's data, > > > jackson-msgpack shows a pathetic performance comparing with a ignite's > > > default binary marshaller. If you want really fast serialization -- the > > > only option is to write code by yourself or use code generation. > Default > > > packer from msgpack-core java package is similar to BinaryWriter. So I > am > > > wondering why packer from msgpack-core show better performance than > > > BinaryWriter. And I suppose that benchmark is not quite fair. > > > > > > > > > чт, 17 июн. 2021 г. в 22:19, Andrey Mashenkov < > > andrey.mashen...@gmail.com > > > >: > > > > > > > Ivan, thankd for clarification. > > > > > > > > Binarilizable interface forces user to write serialization code. We > can > > > > support this or similar interface. > > > > But I'd like Ignite has some default serializer in addition. It can > be > > > also > > > > useful e.g. in compute for param and result serialization. > > > > > > > > BinaryObjectBuider requires an Ignite node for object construction, > but > > > we > > > > are looking for a detached builder and won't care about schemas. > > > > > > > > AFAIR, BinaryObject creates an objectReader on every single field > read > > > > operation. > > > > So, BO solution produces a lot of garbage and BO has noticable > overhead > > > > which affects the object footprint. > > > > > > > > чт, 17 июн. 2021 г., 21:41 Ivan Daschinsky <ivanda...@gmail.com>: > > > > > > > > > >> Double checked -- there is not any links to PR either in IEP or > in > > > > jira > > > > > issue > > > > > Sorry, there is a link in IEP, but not in jira ticket. > > > > > > > > > > чт, 17 июн. 2021 г. в 21:39, Ivan Daschinsky <ivanda...@gmail.com > >: > > > > > > > > > > > Andrey, > > > > > > >> arbitrary object graph > > > > > > Also, that is not true, msgpack format doesn't handle circular > > > graphs. > > > > > > Think about msgpack as binary json. You couldn't understand full > > > > > structure > > > > > > of message if you didn't deserialize it fully before, maps and > > arrays > > > > are > > > > > > serialized just as contiguos chunks > > > > > > of values/kv-pairs. Msgpack is a really dumb and simple format. > > > > > > > > > > > > Also, as for me, I cannot understand why current ignite > > serialization > > > > > > (BinaryObjectBuilder or Binarilizable) is slower than raw message > > > pack > > > > > > serializer. > > > > > > I suppose that this is an issue and we should investigate it. > > > > > > > > > > > > Pavel, why do you use PooledMessageBufferOutput in benchmarks? > > I'm > > > > > > sorry, but is it fair to use it? > > > > > > > > > > > > >> The code is linked in the IEP [2] > > > > > > Double checked -- there is not any links to PR either in IEP or > in > > > jira > > > > > > issue > > > > > > > > > > > > > > > > > > > > > -- > > > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > > > > > > > > > > > -- > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > -- > Sincerely yours, Ivan Daschinskiy >
