Ivan,

KV view does work over the tuples. Nested objects and arbitrary structures
can be stored as blobs. So if you need a basic KV cache, you can always
create a table with two blob fields - one for key and one for value - and
store anything there.

-Val

On Thu, Jul 1, 2021 at 9:55 AM Ivan Daschinsky <ivanda...@gmail.com> wrote:

> Val, am I right, that kv view over the tuples is just simple mapping from
> POJO to tuple? No collections, no nested objects? I have discussed this in
> private with Igor and Pavel and they told me different info.
>
> чт, 1 июл. 2021 г., 19:43 Valentin Kulichenko <
> valentin.kuliche...@gmail.com
> >:
>
> > Ivan,
> >
> > I was answering your question about the KV API. The API I provided has
> been
> > discussed and agreed upon. One of the goals of the protocol is to
> implement
> > this API, so it should give you a clear idea of what we're looking for.
> >
> > Of course, I agree with you that the protocol should be simple and
> flexible
> > enough to allow other implementations for different languages and
> > platforms.
> >
> > -Val
> >
> > On Thu, Jul 1, 2021 at 9:38 AM Ivan Daschinsky <ivanda...@gmail.com>
> > wrote:
> >
> > > Andrey, yep, you are right. This was just a quick idea. As for me, I
> just
> > > don't want to repeat the same problem with compactFooter in thin client
> > api
> > > of ignite 2.x.
> > >
> > > чт, 1 июл. 2021 г., 19:22 Andrey Mashenkov <andrey.mashen...@gmail.com
> >:
> > >
> > > > >
> > > > > I suppose that we should describe this more verbose and explicit. I
> > > > > nevertheless suggest to also consider writing values this way:
> > > > > - arr of fields names (if name is missed, corresponding field is
> nil)
> > > > > - arr of rows (row as array, length equal to fields array)
> > > >
> > > >
> > > > Ivan,
> > > > I think GET and PUT operation parameters should be consistent.
> > > > With PUT operation this way may be tricky.
> > > >
> > > > SQL INSERT operation (which is similar PUT operation) semantic allows
> > > > skipping columns that have a default value.
> > > > Assume we have smth like this:
> > > >
> > > > CREATE TABLE t1 (
> > > >    'id' INT;
> > > >    'colname' VARCHAR DEFAULT "abc";
> > > > )
> > > > INSERT INTO t1 VALUES(1)
> > > >
> > > > Actually, this will add a row (1, "abc")
> > > >
> > > > Your suggestion related to missed fields will not work this way as it
> > is
> > > > impossible to distinct
> > > > case with 'null' value from the case with a default value.
> > > >
> > > >
> > > > On Thu, Jul 1, 2021 at 5:51 PM Ivan Daschinsky <ivanda...@gmail.com>
> > > > wrote:
> > > >
> > > > > > Here is the description of TUPLE_GET_ALL:
> > > > > - UUID: table ID
> > > > > - int: schema ID
> > > > > - arr of arr: array of rows with values for all columns in given
> > schema
> > > > >
> > > > > I suppose that we should describe this more verbose and explicit. I
> > > > > nevertheless suggest to also consider writing values this way:
> > > > > - arr of fields names (if name is missed, corresponding field is
> nil)
> > > > > - arr of rows (row as array, length equal to fields array)
> > > > >
> > > > > It is quite simple and if we use str8 (it is more than enough for
> any
> > > > utf-8
> > > > > reasonable field name), overhead will be negligible, but
> realization
> > > of a
> > > > > client will be way simpler
> > > > >
> > > > > чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn <ptupit...@apache.org>:
> > > > >
> > > > > > > No it isn't, I have carefully read code and IEP, in your code
> you
> > > > write
> > > > > > > schema id in each tuple.
> > > > > >
> > > > > > There is no code for batch operations yet.
> > > > > >
> > > > > > Here is the description of TUPLE_GET_ALL:
> > > > > > - UUID: table ID
> > > > > > - int: schema ID
> > > > > > - arr of arr: array of rows with values for all columns in given
> > > schema
> > > > > > (nil when value is missing for a column)
> > > > > >
> > > > > > As you can see, schema ID is written once for all rows.
> > > > > > A row is just a set of values according to the schema.
> > > > > >
> > > > > >
> > > > > > > Also, my biggest concern -- extra serde step. I suppose we
> should
> > > > pass
> > > > > > > bytearray to internal api, and use msgpack throughout all wire
> > > > > protocols,
> > > > > > > as tarantool does.
> > > > > >
> > > > > > I agree. But this was decided before in IEP-54, and is out of
> scope
> > > for
> > > > > > current IEP.
> > > > > > Would you like to start a separate thread to discuss this? Or I
> can
> > > do
> > > > > this
> > > > > > a bit later.
> > > > > >
> > > > > > On Thu, Jul 1, 2021 at 4:41 PM Ivan Daschinsky <
> > ivanda...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > > This is described in all operations that include multiple
> > tuples.
> > > > > > > No it isn't, I have carefully read code and IEP, in your code
> you
> > > > write
> > > > > > > schema id in each tuple.
> > > > > > >
> > > > > > > Also, my biggest concern -- extra serde step. I suppose we
> should
> > > > pass
> > > > > > > bytearray to internal api, and use msgpack throughout all wire
> > > > > protocols,
> > > > > > > as tarantool does.
> > > > > > >
> > > > > > > чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn <ptupit...@apache.org
> >:
> > > > > > >
> > > > > > > > Ivan,
> > > > > > > >
> > > > > > > > >  that there is not neccesary to write schema versions in
> each
> > > row
> > > > > > > > > in collectionof tuples
> > > > > > > >
> > > > > > > > This is described in all operations that include multiple
> > tuples.
> > > > > > > >
> > > > > > > >
> > > > > > > > > it is not clear from your code (probably
> > > > > > > > > mistake?) how differ key tuples and value tuples from each
> > > other
> > > > > > > >
> > > > > > > > Key tuples include only key columns. Key columns come first
> in
> > > the
> > > > > > > schema.
> > > > > > > > Value tuples include all columns, key and value. Added "Key
> > > tuples"
> > > > > > > > section.
> > > > > > > >
> > > > > > > >
> > > > > > > > > As for me, these excercises with schema's doesn't worth a
> lot
> > > > > > > >
> > > > > > > > I'll add a benchmark and we'll see.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Jul 1, 2021 at 3:17 PM Ivan Daschinsky <
> > > > ivanda...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > I suppose, that there is not neccesary to write schema
> > versions
> > > > in
> > > > > > each
> > > > > > > > row
> > > > > > > > > in collectionof tuples. Also it is not clear from your code
> > > > > (probably
> > > > > > > > > mistake?) how differ key tuples and value tuples from each
> > > other.
> > > > > In
> > > > > > > > > readTuple you always read full schema and check for full
> > > length.
> > > > As
> > > > > > for
> > > > > > > > me,
> > > > > > > > > these excercises with schema's doesn't worth a lot. I.e.
> > > postgres
> > > > > > just
> > > > > > > > > writes field names and then simpy rows with data. Saving
> few
> > > > bytes
> > > > > > > > doesn't
> > > > > > > > > make much deal. Btw, msgpack has special types for short
> > > strings
> > > > > > (i.e.
> > > > > > > > > str8). It is much easier use it and write field name as is.
> > > > > > > > >
> > > > > > > > > чт, 1 июл. 2021 г., 14:56 Pavel Tupitsyn <
> > ptupit...@apache.org
> > > >:
> > > > > > > > >
> > > > > > > > > > Ivan, tuple serialization section added to the IEP, let
> me
> > > know
> > > > > if
> > > > > > it
> > > > > > > > is
> > > > > > > > > > clear enough.
> > > > > > > > > >
> > > > > > > > > > Thanks!
> > > > > > > > > >
> > > > > > > > > > On Thu, Jul 1, 2021 at 2:06 PM Ivan Daschinsky <
> > > > > > ivanda...@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > I can't find any description of tuple serialization in
> > IEP,
> > > > > only
> > > > > > in
> > > > > > > > > code
> > > > > > > > > > >
> > > > > > > > > > > чт, 1 июл. 2021 г., 13:59 Pavel Tupitsyn <
> > > > ptupit...@apache.org
> > > > > >:
> > > > > > > > > > >
> > > > > > > > > > > > Ivan,
> > > > > > > > > > > >
> > > > > > > > > > > > 0. The IEP is not in progress, it is ready for review
> > and
> > > > > > > > discussion.
> > > > > > > > > > > > 1. Tuple serialization is described in the IEP and
> > > > > demonstrated
> > > > > > > in
> > > > > > > > > the
> > > > > > > > > > > PoC
> > > > > > > > > > > > (see ClientMessageHandler#readTuple), let me know if
> > more
> > > > > > details
> > > > > > > > are
> > > > > > > > > > > > required
> > > > > > > > > > > > 2. Tuple schema serialization is described in
> > SCHEMAS_GET
> > > > > > > section.
> > > > > > > > > > Table
> > > > > > > > > > > > schema (configuration) needs more details, you are
> > right
> > > -
> > > > > I'll
> > > > > > > add
> > > > > > > > > > them.
> > > > > > > > > > > > 3. This IEP is about tables (tuple-based) API only,
> > since
> > > > it
> > > > > is
> > > > > > > the
> > > > > > > > > > only
> > > > > > > > > > > > API that we have right now, as noted in Risks and
> > > > > Assumptions.
> > > > > > > > > > > >
> > > > > > > > > > > > On Thu, Jul 1, 2021 at 1:53 PM Ivan Daschinsky <
> > > > > > > > ivanda...@gmail.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Also, is there any clear information about KV api?
> Is
> > > > there
> > > > > > any
> > > > > > > > > plan
> > > > > > > > > > to
> > > > > > > > > > > > > implement it? Or is there any proposal about it?
> > > > > > > > > > > > >
> > > > > > > > > > > > > чт, 1 июл. 2021 г., 13:51 Ivan Daschinsky <
> > > > > > ivanda...@gmail.com
> > > > > > > >:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Pavel, but IEP is in progress, isn't it?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 1. There is not any information about tuple
> > > > > serialization.
> > > > > > > And
> > > > > > > > > > there
> > > > > > > > > > > > > isn't
> > > > > > > > > > > > > > a clear consensus about it.
> > > > > > > > > > > > > > 2. There is not any information about schrma
> > > > > serialization
> > > > > > > > > format.
> > > > > > > > > > > And
> > > > > > > > > > > > > > AFAIK, there isn't a clear consensus also.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > чт, 1 июл. 2021 г., 13:26 Pavel Tupitsyn <
> > > > > > > ptupit...@apache.org
> > > > > > > > >:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >> Igniters,
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Please review the IEP for thin client protocol
> in
> > > 3.0
> > > > > [1].
> > > > > > > > > > > > > >> PoC is in progress [2]
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> [1]
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> [2] https://github.com/apache/ignite-3/pull/191
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Andrey V. Mashenkov
> > > >
> > >
> >
>

Reply via email to