> I agree. But this was decided before in IEP-54, and is out of scope for current IEP. Would you like to start a separate thread to discuss this? Or I can do this a bit later.
Great idea, let's discuss it. I suppose this will simplify many aspects of realization and improve performance a lot чт, 1 июл. 2021 г., 17:50 Ivan Daschinsky <ivanda...@gmail.com>: > > Here is the description of TUPLE_GET_ALL: > - UUID: table ID > - int: schema ID > - arr of arr: array of rows with values for all columns in given schema > > I suppose that we should describe this more verbose and explicit. I > nevertheless suggest to also consider writing values this way: > - arr of fields names (if name is missed, corresponding field is nil) > - arr of rows (row as array, length equal to fields array) > > It is quite simple and if we use str8 (it is more than enough for any > utf-8 reasonable field name), overhead will be negligible, but realization > of a client will be way simpler > > чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn <ptupit...@apache.org>: > >> > No it isn't, I have carefully read code and IEP, in your code you write >> > schema id in each tuple. >> >> There is no code for batch operations yet. >> >> Here is the description of TUPLE_GET_ALL: >> - UUID: table ID >> - int: schema ID >> - arr of arr: array of rows with values for all columns in given schema >> (nil when value is missing for a column) >> >> As you can see, schema ID is written once for all rows. >> A row is just a set of values according to the schema. >> >> >> > Also, my biggest concern -- extra serde step. I suppose we should pass >> > bytearray to internal api, and use msgpack throughout all wire >> protocols, >> > as tarantool does. >> >> I agree. But this was decided before in IEP-54, and is out of scope for >> current IEP. >> Would you like to start a separate thread to discuss this? Or I can do >> this >> a bit later. >> >> On Thu, Jul 1, 2021 at 4:41 PM Ivan Daschinsky <ivanda...@gmail.com> >> wrote: >> >> > > This is described in all operations that include multiple tuples. >> > No it isn't, I have carefully read code and IEP, in your code you write >> > schema id in each tuple. >> > >> > Also, my biggest concern -- extra serde step. I suppose we should pass >> > bytearray to internal api, and use msgpack throughout all wire >> protocols, >> > as tarantool does. >> > >> > чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn <ptupit...@apache.org>: >> > >> > > Ivan, >> > > >> > > > that there is not neccesary to write schema versions in each row >> > > > in collectionof tuples >> > > >> > > This is described in all operations that include multiple tuples. >> > > >> > > >> > > > it is not clear from your code (probably >> > > > mistake?) how differ key tuples and value tuples from each other >> > > >> > > Key tuples include only key columns. Key columns come first in the >> > schema. >> > > Value tuples include all columns, key and value. Added "Key tuples" >> > > section. >> > > >> > > >> > > > As for me, these excercises with schema's doesn't worth a lot >> > > >> > > I'll add a benchmark and we'll see. >> > > >> > > >> > > On Thu, Jul 1, 2021 at 3:17 PM Ivan Daschinsky <ivanda...@gmail.com> >> > > wrote: >> > > >> > > > I suppose, that there is not neccesary to write schema versions in >> each >> > > row >> > > > in collectionof tuples. Also it is not clear from your code >> (probably >> > > > mistake?) how differ key tuples and value tuples from each other. In >> > > > readTuple you always read full schema and check for full length. As >> for >> > > me, >> > > > these excercises with schema's doesn't worth a lot. I.e. postgres >> just >> > > > writes field names and then simpy rows with data. Saving few bytes >> > > doesn't >> > > > make much deal. Btw, msgpack has special types for short strings >> (i.e. >> > > > str8). It is much easier use it and write field name as is. >> > > > >> > > > чт, 1 июл. 2021 г., 14:56 Pavel Tupitsyn <ptupit...@apache.org>: >> > > > >> > > > > Ivan, tuple serialization section added to the IEP, let me know >> if it >> > > is >> > > > > clear enough. >> > > > > >> > > > > Thanks! >> > > > > >> > > > > On Thu, Jul 1, 2021 at 2:06 PM Ivan Daschinsky < >> ivanda...@gmail.com> >> > > > > wrote: >> > > > > >> > > > > > I can't find any description of tuple serialization in IEP, >> only in >> > > > code >> > > > > > >> > > > > > чт, 1 июл. 2021 г., 13:59 Pavel Tupitsyn <ptupit...@apache.org >> >: >> > > > > > >> > > > > > > Ivan, >> > > > > > > >> > > > > > > 0. The IEP is not in progress, it is ready for review and >> > > discussion. >> > > > > > > 1. Tuple serialization is described in the IEP and >> demonstrated >> > in >> > > > the >> > > > > > PoC >> > > > > > > (see ClientMessageHandler#readTuple), let me know if more >> details >> > > are >> > > > > > > required >> > > > > > > 2. Tuple schema serialization is described in SCHEMAS_GET >> > section. >> > > > > Table >> > > > > > > schema (configuration) needs more details, you are right - >> I'll >> > add >> > > > > them. >> > > > > > > 3. This IEP is about tables (tuple-based) API only, since it >> is >> > the >> > > > > only >> > > > > > > API that we have right now, as noted in Risks and Assumptions. >> > > > > > > >> > > > > > > On Thu, Jul 1, 2021 at 1:53 PM Ivan Daschinsky < >> > > ivanda...@gmail.com> >> > > > > > > wrote: >> > > > > > > >> > > > > > > > Also, is there any clear information about KV api? Is there >> any >> > > > plan >> > > > > to >> > > > > > > > implement it? Or is there any proposal about it? >> > > > > > > > >> > > > > > > > чт, 1 июл. 2021 г., 13:51 Ivan Daschinsky < >> ivanda...@gmail.com >> > >: >> > > > > > > > >> > > > > > > > > Pavel, but IEP is in progress, isn't it? >> > > > > > > > > >> > > > > > > > > 1. There is not any information about tuple serialization. >> > And >> > > > > there >> > > > > > > > isn't >> > > > > > > > > a clear consensus about it. >> > > > > > > > > 2. There is not any information about schrma serialization >> > > > format. >> > > > > > And >> > > > > > > > > AFAIK, there isn't a clear consensus also. >> > > > > > > > > >> > > > > > > > > чт, 1 июл. 2021 г., 13:26 Pavel Tupitsyn < >> > ptupit...@apache.org >> > > >: >> > > > > > > > > >> > > > > > > > >> Igniters, >> > > > > > > > >> >> > > > > > > > >> Please review the IEP for thin client protocol in 3.0 >> [1]. >> > > > > > > > >> PoC is in progress [2] >> > > > > > > > >> >> > > > > > > > >> [1] >> > > > > > > > >> >> > > > > > > > >> >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0 >> > > > > > > > >> >> > > > > > > > >> [2] https://github.com/apache/ignite-3/pull/191 >> > > > > > > > >> >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
