Pavel, - Treat uLong as long in Java (bitwise representation is the same)
That way is impossible. Assume, you have a .NET class with a uByte field and map it to 'uint8' column. Then you set the field value to "250" and put the object into a table, field value perfectly fits to a single byte 'int8' column. But in Java you can't deserialize it to directly the Java object field of 'byte' type, so we should map uint8 type to Java 'short' type because the one expected to see "250" as a value which doesn't fit to the signed type. For uLong the one will need a BigInteger field in Java. SQL index either can't treat column value as Java 'byte' as is, because after reading you will get a negative value, so it should be cast to short at first. (converted to BigInteger for uint64) So, index on signed type will require a different comparator. That way doesn't look simpler. On Tue, Nov 24, 2020 at 4:23 PM Pavel Tupitsyn <ptupit...@apache.org> wrote: > Andrey, > > I don't think range narrowing is a good idea. > Do you see any problems with the simple approach I described? > > > On Tue, Nov 24, 2020 at 4:01 PM Andrey Mashenkov < > andrey.mashen...@gmail.com> > wrote: > > > Pavel, > > > > If you are ok with narrowing range for unsigned types then we could add a > > constraint for unsigned types on schema level (like nullability flag) > > and treat them as signed types in storage. > > > > We are going with a separate storage type-system and binary protocol > > type-system, however most of type will match 1 to 1 with storage (native) > > type. > > On .Net side you will either have a separate type id or treat serialized > > value regarding a schema (signed or unsigned flag). > > > > Igor, > > > > I'm not sure users can ever foresee the consequences of using unsigned > > types. > > > > Assume, a user used to unsigned types perfectly works with some database, > > then he turns into Ignite successor confession with our "native" > > unsigned-types support. > > But later, he finds that he can use the power of Ignite Compute on Java > for > > some tasks or a new app. > > Finally, the user will either fail to use his unsigned data on Java due > or > > face performance issues due to natural Java type system limitations e.g. > > conversion uLong to BigInteger. > > > > I believe that natively supported types with possible value ranges and > > limitations should be known. > > So, the only question is what trade-off we found acceptable: narrowing > > unsigned type range or use types of wider range on systems like Java. > > > > On Tue, Nov 24, 2020 at 3:25 PM Igor Sapego <isap...@apache.org> wrote: > > > > > Actually, I think it is not so hard to implement comparison of unsigned > > > numbers in > > > SQL even in Java, so it does not seem to be a big issue from my > > > perspective. > > > > > > Now to the usage of unsigned types from Java - I think, if a user uses > > > unsigned type > > > in a schema and is going to interact with it from Java he knows what he > > is > > > doing. > > > > > > Mostly they are for use from platforms where they have native support > and > > > widely > > > used, like in C++ or .NET, where users currently have to make a manual > > type > > > casting > > > or even just stop using unsigned types when they use Ignite. > > > > > > Best Regards, > > > Igor > > > > > > > > > On Tue, Nov 24, 2020 at 3:06 PM Pavel Tupitsyn <ptupit...@apache.org> > > > wrote: > > > > > > > Andrey, > > > > > > > > I think it is much simpler: > > > > - Add protocol support for those types (basically, just add more type > > > ids) > > > > - Treat uLong as long in Java (bitwise representation is the same) > > > > > > > > ANSI SQL does not have unsigned integers, so we can simply say that > > > > unsigned value relative comparison is not supported in SQL (equality > > will > > > > work). > > > > > > > > > > > > On Tue, Nov 24, 2020 at 2:40 PM Andrey Mashenkov < > > > > andrey.mashen...@gmail.com> > > > > wrote: > > > > > > > > > Thanks, Pavel and Igor. > > > > > > > > > > I like your ideas to have i8 or int8 instead of Integer. > > > > > But the naming doesn't address the issue. > > > > > > > > > > I agree internal types should be portable across different systems > > with > > > > and > > > > > without unsigned type support. > > > > > The only issue here is that unsigned types cover different ranges. > > > > > > > > > > Let's assume we want to introduce a uLong. > > > > > It doesn't look like a big deal to add uLong type support at > storage > > > > level > > > > > and fit it to a 8 bytes and then use it in e.g. .Net only. > > > > > But how we could support it in e.g. Java? > > > > > > > > > > Let's keep in mind Long range is about (2^-63 .. 2^63) and uLong > > range > > > is > > > > > (0 .. 2^64) > > > > > 1. The first option is to restrict range to (0 .. 2^63). This > allows > > to > > > > use > > > > > signed in e.g. > > > > > Java with no conversion, but doesn't look like a 'real' unsigned > > uLong > > > > > support. Things go worse when the user will use uByte, as > limitation > > > can > > > > > make uByte totally unusable. > > > > > > > > > > 2. The second one is to map unsigned types to a type of wider type > > and > > > > add > > > > > a constraint for negative values. E.g. uLong to BigInteger. > > > > > So, we can't use primitive Java type for Long here. However, it is > > > still > > > > > possible to store uLong in 8 bytes, but have a special comparator > for > > > > > unsigned types to avoid unwanted deserialization. > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 24, 2020 at 2:04 PM Pavel Tupitsyn < > ptupit...@apache.org > > > > > > > > wrote: > > > > > > > > > > > Agree, let's get rid of "long, short, byte" in the protocol > > > definition. > > > > > > > > > > > > We can use Rust style, which is concise and unambiguous: > > > > > > i8, u8, i16, u16, etc > > > > > > > > > > > > On Tue, Nov 24, 2020 at 1:58 PM Igor Sapego <isap...@apache.org> > > > > wrote: > > > > > > > > > > > > > Pavel, > > > > > > > > > > > > > > I totally support that. Also, if we are aiming for > > > > > > > stronger platform-independance, > > > > > > > in our schemas we may want to support bit-notation (int32, > > uint64)? > > > > For > > > > > > > example > > > > > > > "long" can mean a different type on different platforms and > it's > > > easy > > > > > to > > > > > > > confuse > > > > > > > them (happens often when using ODBC for example). > > > > > > > > > > > > > > Best Regards, > > > > > > > Igor > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 24, 2020 at 1:34 PM Pavel Tupitsyn < > > > ptupit...@apache.org > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > I think we should support unsigned data types: > > > > > > > > uByte, uShort, uInt, uLong > > > > > > > > > > > > > > > > Java does not have them, but many other languages do, > > > > > > > > and with the growing number of thin clients this is > important. > > > > > > > > > > > > > > > > For example, in current Ignite.NET implementation we store > > > unsigned > > > > > > > values > > > > > > > > as signed internally, > > > > > > > > but this is a huge pain when it comes to metadata, binary > > > objects, > > > > > etc. > > > > > > > > (it is easy to deserialize int as uint when you have a class, > > but > > > > not > > > > > > > with > > > > > > > > BinaryObject.GetField) > > > > > > > > > > > > > > > > Any objections? > > > > > > > > > > > > > > > > On Tue, Nov 24, 2020 at 12:28 PM Andrey Mashenkov < > > > > > > > > andrey.mashen...@gmail.com> wrote: > > > > > > > > > > > > > > > > > Denis, > > > > > > > > > > > > > > > > > > Good point. Both serializers use reflection API. > > > > > > > > > However, we will allow users to configure static schema > along > > > > with > > > > > > > > 'strict' > > > > > > > > > schema mode, we still need to validate user classes on > client > > > > nodes > > > > > > > > against > > > > > > > > > the latest schema in the grid and reflection API is the > only > > > way > > > > > to > > > > > > do > > > > > > > > it. > > > > > > > > > One can find a few articles on the internet on how to > enable > > > > > > reflection > > > > > > > > in > > > > > > > > > GraalVM. > > > > > > > > > > > > > > > > > > I'll create a task for supporting GraalVM, and maybe > someone > > > who > > > > is > > > > > > > > > familiar with GraalVM will suggest a solution or a proper > > > > > workaround. > > > > > > > Or > > > > > > > > > I'll do it a bit later. > > > > > > > > > If no workaround is found, we could allow users to write > it's > > > own > > > > > > > > > serializer, but I don't think it is a good idea to expose > any > > > > > > internal > > > > > > > > > classes to the public. > > > > > > > > > > > > > > > > > > On Tue, Nov 24, 2020 at 2:55 AM Denis Magda < > > dma...@apache.org > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Andrey, thanks for the update, > > > > > > > > > > > > > > > > > > > > Does any of the serializers take into consideration the > > > > > > > > > > native-image-generation feature of GraalVM? > > > > > > > > > > https://www.graalvm.org/reference-manual/native-image/ > > > > > > > > > > > > > > > > > > > > With the current binary marshaller, we can't even > generate > > a > > > > > native > > > > > > > > image > > > > > > > > > > for the code using our thin client APIs. > > > > > > > > > > > > > > > > > > > > - > > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 23, 2020 at 4:39 AM Andrey Mashenkov < > > > > > > > > > > andrey.mashen...@gmail.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Igniters, > > > > > > > > > > > > > > > > > > > > > > I'd like to continue discussion of IEP-54 (Schema-first > > > > > > approach). > > > > > > > > > > > > > > > > > > > > > > Hope everyone who is interested had a chance to get > > > familiar > > > > > with > > > > > > > the > > > > > > > > > > > proposal [1]. > > > > > > > > > > > Please, do not hesitate to ask questions and share your > > > > ideas. > > > > > > > > > > > > > > > > > > > > > > I've prepared a prototype of serializer [2] for the > data > > > > layout > > > > > > > > > described > > > > > > > > > > > in the proposal. > > > > > > > > > > > In prototy, I compared 2 approaches to (de)serialize > > > objects, > > > > > the > > > > > > > > first > > > > > > > > > > one > > > > > > > > > > > uses java reflection/unsafe API and similar to one we > > > already > > > > > use > > > > > > > in > > > > > > > > > > Ignite > > > > > > > > > > > and the second one generates serializer for particular > > user > > > > > class > > > > > > > and > > > > > > > > > > uses > > > > > > > > > > > Janino library for compilation. > > > > > > > > > > > Second one shows better results in benchmarks. > > > > > > > > > > > I think we can go with it as default serializer and > have > > > > > > > > > reflection-based > > > > > > > > > > > implementation as a fallback if someone will have > issues > > > with > > > > > the > > > > > > > > first > > > > > > > > > > > one. > > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > > > > > There are a number of tasks under the umbrella ticket > [3] > > > > > waiting > > > > > > > for > > > > > > > > > the > > > > > > > > > > > assignee. > > > > > > > > > > > > > > > > > > > > > > BTW, I'm going to create more tickets for schema > manager > > > > modes > > > > > > > > > > > implementation, but would like to clarify some details. > > > > > > > > > > > > > > > > > > > > > > I thought schemaManager on each node should held: > > > > > > > > > > > 1. Local mapping of "schema version" <--> validated > > local > > > > > > > key/value > > > > > > > > > > > classes pair. > > > > > > > > > > > 2. Cluster-wide schema changes history. > > > > > > > > > > > On the client side. Before any key-value API operation > we > > > > > should > > > > > > > > > > validate a > > > > > > > > > > > schema for a given key-value pair. > > > > > > > > > > > If there is no local-mapping exists for a given > key-value > > > > pair > > > > > or > > > > > > > if > > > > > > > > a > > > > > > > > > > > cluster wide schema has a more recent version then the > > > > > key-value > > > > > > > pair > > > > > > > > > > > should be validated against the latest version and > local > > > > > mapping > > > > > > > > should > > > > > > > > > > be > > > > > > > > > > > updated/actualized. > > > > > > > > > > > If an object doesn't fit to the latest schema then it > > > depends > > > > > on > > > > > > > > schema > > > > > > > > > > > mode: either fail the operation ('strict' mode) or a > new > > > > > mapping > > > > > > > > should > > > > > > > > > > be > > > > > > > > > > > created and a new schema version should be propagated > to > > > the > > > > > > > cluster. > > > > > > > > > > > > > > > > > > > > > > On the server side we usually have no key-value classes > > and > > > > we > > > > > > > > operate > > > > > > > > > > with > > > > > > > > > > > tuples. > > > > > > > > > > > As schema change history is available and a tuple has > > > schema > > > > > > > version, > > > > > > > > > > then > > > > > > > > > > > it is possible to upgrade any received tuple to the > last > > > > > version > > > > > > > > > without > > > > > > > > > > > desialization. > > > > > > > > > > > Thus we could allow nodes to send key-value pairs of > > > previous > > > > > > > > versions > > > > > > > > > > (if > > > > > > > > > > > they didn't receive a schema update yet) without > > reverting > > > > > schema > > > > > > > > > changes > > > > > > > > > > > made by a node with newer classes. > > > > > > > > > > > > > > > > > > > > > > Alex, Val, Ivan did you mean the same? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach > > > > > > > > > > > [2] > > > > > > > > > > > https://github.com/apache/ignite/tree/ignite-13618/modules/commons > > > > > > > > > > > [3] https://issues.apache.org/jira/browse/IGNITE-13616 > > > > > > > > > > > > > > > > > > > > > > On Thu, Sep 17, 2020 at 9:21 AM Ivan Pavlukhin < > > > > > > > vololo...@gmail.com> > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Folks, > > > > > > > > > > > > > > > > > > > > > > > > Please do not ignore history. We had a thread [1] > with > > > many > > > > > > > bright > > > > > > > > > > > > ideas. We can resume it. > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Applicability-of-term-cache-to-Apache-Ignite-td36541.html > > > > > > > > > > > > > > > > > > > > > > > > 2020-09-10 0:08 GMT+03:00, Denis Magda < > > > dma...@apache.org > > > > >: > > > > > > > > > > > > > Val, makes sense, thanks for explaining. > > > > > > > > > > > > > > > > > > > > > > > > > > Agree that we need to have a separate discussion > > thread > > > > for > > > > > > the > > > > > > > > > > "table" > > > > > > > > > > > > and > > > > > > > > > > > > > "cache" terms substitution. I'll appreciate it if > you > > > > start > > > > > > the > > > > > > > > > > thread > > > > > > > > > > > > > sharing pointers to any relevant IEPs and reasoning > > > > behind > > > > > > the > > > > > > > > > > > suggested > > > > > > > > > > > > > change. > > > > > > > > > > > > > > > > > > > > > > > > > > - > > > > > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 8, 2020 at 6:01 PM Valentin Kulichenko > < > > > > > > > > > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > >> Hi Denis, > > > > > > > > > > > > >> > > > > > > > > > > > > >> I guess the wording in the IEP is a little bit > > > > confusing. > > > > > > All > > > > > > > it > > > > > > > > > > means > > > > > > > > > > > > is > > > > > > > > > > > > >> that you should not create nested POJOs, but > rather > > > > inline > > > > > > > > fields > > > > > > > > > > > into a > > > > > > > > > > > > >> single POJO that is mapped to a particular schema. > > In > > > > > other > > > > > > > > words, > > > > > > > > > > > > nested > > > > > > > > > > > > >> POJOs are not supported. > > > > > > > > > > > > >> > > > > > > > > > > > > >> Alex, is this correct? Please let me know if I'm > > > missing > > > > > > > > > something. > > > > > > > > > > > > >> > > > > > > > > > > > > >> As for the "cache" term, I agree that it is > > outdated, > > > > but > > > > > > I'm > > > > > > > > not > > > > > > > > > > sure > > > > > > > > > > > > >> what we can replace it with. "Table" is tightly > > > > associated > > > > > > > with > > > > > > > > > SQL, > > > > > > > > > > > but > > > > > > > > > > > > >> SQL is optional in our case. Do you want to > create a > > > > > > separate > > > > > > > > > > > discussion > > > > > > > > > > > > >> about this? > > > > > > > > > > > > >> > > > > > > > > > > > > >> -Val > > > > > > > > > > > > >> > > > > > > > > > > > > >> On Tue, Sep 8, 2020 at 4:37 PM Denis Magda < > > > > > > dma...@apache.org > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > >> > > > > > > > > > > > > >>> Val, > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> I've checked the IEP again and have a few > > questions. > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> Arbitrary nested objects and collections are not > > > > allowed > > > > > as > > > > > > > > > column > > > > > > > > > > > > >>> values. > > > > > > > > > > > > >>> > Nested POJOs should either be inlined into > > schema, > > > or > > > > > > > stored > > > > > > > > as > > > > > > > > > > > BLOBs > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> Could you provide a DDL code snippet showing how > > the > > > > > > inlining > > > > > > > > of > > > > > > > > > > > POJOs > > > > > > > > > > > > >>> is > > > > > > > > > > > > >>> supposed to work? > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> Also, we keep using the terms "cache" and "table" > > > > > > throughout > > > > > > > > the > > > > > > > > > > IEP. > > > > > > > > > > > > Is > > > > > > > > > > > > >>> it > > > > > > > > > > > > >>> the right time to discuss an alternate name that > > > would > > > > > > > replace > > > > > > > > > > those > > > > > > > > > > > > >>> too? > > > > > > > > > > > > >>> Personally, the "table" should stay and the > "cache" > > > > > should > > > > > > go > > > > > > > > > > > > >>> considering > > > > > > > > > > > > >>> that SQL is one of the primary APIs in Ignite and > > > that > > > > > DDL > > > > > > is > > > > > > > > > > > supported > > > > > > > > > > > > >>> out-of-the-box. > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> - > > > > > > > > > > > > >>> Denis > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> On Mon, Sep 7, 2020 at 12:26 PM Valentin > > Kulichenko < > > > > > > > > > > > > >>> valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > Ivan, > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > I see your point. I agree that with the > automatic > > > > > updates > > > > > > > we > > > > > > > > > step > > > > > > > > > > > > into > > > > > > > > > > > > >>> the > > > > > > > > > > > > >>> > schema-last territory. > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > Actually, if we support automatic evolution, we > > can > > > > as > > > > > > well > > > > > > > > > > support > > > > > > > > > > > > >>> > creating a cache without schema and inferring > it > > > from > > > > > the > > > > > > > > first > > > > > > > > > > > > >>> > insert. > > > > > > > > > > > > >>> In > > > > > > > > > > > > >>> > other words, we can have both "schema-first" > and > > > > > > > > "schema-last" > > > > > > > > > > > modes. > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > Alexey, what do you think? > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > -Val > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > On Mon, Sep 7, 2020 at 5:59 AM Alexey > Goncharuk < > > > > > > > > > > > > >>> > alexey.goncha...@gmail.com> > > > > > > > > > > > > >>> > wrote: > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > > Ivan, > > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > Thank you, I got your concern now. As it is > > > mostly > > > > > > > > regarding > > > > > > > > > > the > > > > > > > > > > > > >>> > > terminology, I am absolutely fine with > changing > > > the > > > > > > name > > > > > > > to > > > > > > > > > > > > whatever > > > > > > > > > > > > >>> fits > > > > > > > > > > > > >>> > > the approach best. Dynamic or evolving schema > > > > sounds > > > > > > > > great. I > > > > > > > > > > > will > > > > > > > > > > > > >>> make > > > > > > > > > > > > >>> > > corresponding changes to the IEP once we > settle > > > on > > > > > the > > > > > > > > name. > > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > пн, 7 сент. 2020 г. в 11:33, Ivan Pavlukhin < > > > > > > > > > > vololo...@gmail.com > > > > > > > > > > > >: > > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > Hi Val, > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > Thank you for your answer! > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > My understanding is a little bit different. > > > Yes, > > > > > > schema > > > > > > > > > > > evolution > > > > > > > > > > > > >>> > > > definitely should be possible. But I see a > > main > > > > > > > > difference > > > > > > > > > in > > > > > > > > > > > > "how > > > > > > > > > > > > >>> > > > schema is updated". I treat a common SQL > > > approach > > > > > > > > > > schema-first. > > > > > > > > > > > > >>> Schema > > > > > > > > > > > > >>> > > > and data manipulation operations are > clearly > > > > > > separated > > > > > > > > and > > > > > > > > > it > > > > > > > > > > > > >>> enables > > > > > > > > > > > > >>> > > > interesting capabilities, e.g. preventing > > > > untended > > > > > > > schema > > > > > > > > > > > changes > > > > > > > > > > > > >>> > > > by > > > > > > > > > > > > >>> > > > mistaken data operations, restricting user > > > > > > permissions > > > > > > > to > > > > > > > > > > > change > > > > > > > > > > > > >>> > > > schema. > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > > Schema-first means that schema exists in > > > > advance > > > > > > and > > > > > > > > all > > > > > > > > > > the > > > > > > > > > > > > >>> stored > > > > > > > > > > > > >>> > > data > > > > > > > > > > > > >>> > > > is compliant with it - that's exactly what > is > > > > > > proposed. > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > A schema-last approach mentioned in [1] > also > > > > > assumes > > > > > > > that > > > > > > > > > > > schema > > > > > > > > > > > > >>> > > > exists, but it is inferred from data. Is > not > > it > > > > > more > > > > > > > > > similar > > > > > > > > > > to > > > > > > > > > > > > >>> > > > the > > > > > > > > > > > > >>> > > > proposing approach? > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > And I would like to say, that my main > concern > > > so > > > > > far > > > > > > is > > > > > > > > > > mostly > > > > > > > > > > > > >>> > > > about > > > > > > > > > > > > >>> > > > terminology. And I suppose if it confuses > me > > > then > > > > > > > others > > > > > > > > > > might > > > > > > > > > > > be > > > > > > > > > > > > >>> > > > confused as well. My feeling is closer to > > > > "dynamic > > > > > or > > > > > > > > > liquid > > > > > > > > > > or > > > > > > > > > > > > >>> > > > may > > > > > > > > > > > > >>> be > > > > > > > > > > > > >>> > > > evolving schema". > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > [1] > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > 2020-09-07 0:47 GMT+03:00, Valentin > > Kulichenko > > > < > > > > > > > > > > > > >>> > > > valentin.kuliche...@gmail.com>: > > > > > > > > > > > > >>> > > > > Hi Ivan, > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > >>> > > > > I don't see an issue with that. > > Schema-first > > > > > means > > > > > > > that > > > > > > > > > > > schema > > > > > > > > > > > > >>> exists > > > > > > > > > > > > >>> > > in > > > > > > > > > > > > >>> > > > > advance and all the stored data is > > compliant > > > > with > > > > > > it > > > > > > > - > > > > > > > > > > that's > > > > > > > > > > > > >>> exactly > > > > > > > > > > > > >>> > > > what > > > > > > > > > > > > >>> > > > > is proposed. There are no restrictions > > > > > prohibiting > > > > > > > > > changes > > > > > > > > > > to > > > > > > > > > > > > >>> > > > > the > > > > > > > > > > > > >>> > > schema. > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > >>> > > > > -Val > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > >>> > > > > On Sat, Sep 5, 2020 at 9:52 PM Ivan > > > Pavlukhin < > > > > > > > > > > > > >>> vololo...@gmail.com> > > > > > > > > > > > > >>> > > > wrote: > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > >>> > > > >> Alexey, > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> I am a little bit confused with > > terminology. > > > > My > > > > > > > > > > > understanding > > > > > > > > > > > > >>> > conforms > > > > > > > > > > > > >>> > > > >> to a survey [1] (see part X Semi > > Structured > > > > > Data). > > > > > > > Can > > > > > > > > > we > > > > > > > > > > > > >>> > > > >> really > > > > > > > > > > > > >>> > treat > > > > > > > > > > > > >>> > > > >> a "dynamic schema" approach as a kind of > > > > > > > > "schema-first"? > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> [1] > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> 2020-09-02 1:53 GMT+03:00, Denis Magda < > > > > > > > > > dma...@apache.org > > > > > > > > > > >: > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> However, could you please elaborate > on > > > the > > > > > > > relation > > > > > > > > > > > between > > > > > > > > > > > > >>> > Ignite > > > > > > > > > > > > >>> > > > and > > > > > > > > > > > > >>> > > > >> >> ORM? > > > > > > > > > > > > >>> > > > >> >> Is there a use case for Hibernate > > running > > > > on > > > > > > top > > > > > > > of > > > > > > > > > > > Ignite > > > > > > > > > > > > >>> > > > >> >> (I > > > > > > > > > > > > >>> > > haven't > > > > > > > > > > > > >>> > > > >> >> seen > > > > > > > > > > > > >>> > > > >> >> one so far)? If so, what is missing > > > exactly > > > > > on > > > > > > > the > > > > > > > > > > Ignite > > > > > > > > > > > > >>> side to > > > > > > > > > > > > >>> > > > >> support > > > > > > > > > > > > >>> > > > >> >> this? In my understanding, all you > need > > > is > > > > > SQL > > > > > > > API > > > > > > > > > > which > > > > > > > > > > > we > > > > > > > > > > > > >>> > already > > > > > > > > > > > > >>> > > > >> have. > > > > > > > > > > > > >>> > > > >> >> Am I missing something? > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > > >>> > > > >> > Good point, yes, if all the ORM > > > integrations > > > > > use > > > > > > > > > Ignite > > > > > > > > > > > SQL > > > > > > > > > > > > >>> APIs > > > > > > > > > > > > >>> > > > >> > internally, then they can easily > > translate > > > > an > > > > > > > Entity > > > > > > > > > > > object > > > > > > > > > > > > >>> into > > > > > > > > > > > > >>> > an > > > > > > > > > > > > >>> > > > >> > INSERT/UPDATE statement that lists all > > the > > > > > > > object's > > > > > > > > > > > fields. > > > > > > > > > > > > >>> > Luckily, > > > > > > > > > > > > >>> > > > >> > our > > > > > > > > > > > > >>> > > > >> > Spring Data integration is already > based > > > on > > > > > the > > > > > > > > Ignite > > > > > > > > > > SQL > > > > > > > > > > > > >>> > > > >> > APIs > > > > > > > > > > > > >>> > and > > > > > > > > > > > > >>> > > > >> > needs > > > > > > > > > > > > >>> > > > >> > to be improved once the schema-first > > > > approach > > > > > is > > > > > > > > > > > supported. > > > > > > > > > > > > >>> That > > > > > > > > > > > > >>> > > would > > > > > > > > > > > > >>> > > > >> > solve a ton of usability issues. > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > > >>> > > > >> > I would revise the Hibernate > integration > > > as > > > > > well > > > > > > > > > during > > > > > > > > > > > the > > > > > > > > > > > > >>> Ignite > > > > > > > > > > > > >>> > > 3.0 > > > > > > > > > > > > >>> > > > >> dev > > > > > > > > > > > > >>> > > > >> > phase. Can't say if it's used a lot > but > > > > Spring > > > > > > > Data > > > > > > > > is > > > > > > > > > > > > >>> > > > >> > getting > > > > > > > > > > > > >>> > > > traction > > > > > > > > > > > > >>> > > > >> for > > > > > > > > > > > > >>> > > > >> > sure. > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > > >>> > > > >> > @Michael Pollind, I'll loop you in as > > long > > > > as > > > > > > > you've > > > > > > > > > > > started > > > > > > > > > > > > >>> > working > > > > > > > > > > > > >>> > > > on > > > > > > > > > > > > >>> > > > >> the > > > > > > > > > > > > >>> > > > >> > Ignite support for Micornaut Data > > > > > > > > > > > > >>> > > > >> > < > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > https://micronaut-projects.github.io/micronaut-data/latest/guide/> > > > > > > > > > > > > >>> > > > and > > > > > > > > > > > > >>> > > > >> > came across some challenges. Just > watch > > > this > > > > > > > > > discussion. > > > > > > > > > > > > >>> > > > >> > That's > > > > > > > > > > > > >>> > what > > > > > > > > > > > > >>> > > > is > > > > > > > > > > > > >>> > > > >> > coming in Ignite 3.0. > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > > >>> > > > >> > - > > > > > > > > > > > > >>> > > > >> > Denis > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > > >>> > > > >> > On Mon, Aug 31, 2020 at 5:11 PM > Valentin > > > > > > > Kulichenko > > > > > > > > < > > > > > > > > > > > > >>> > > > >> > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > > >>> > > > >> >> Hi Denis, > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> Generally speaking, I believe that > the > > > > > > > schema-first > > > > > > > > > > > > approach > > > > > > > > > > > > >>> > > natively > > > > > > > > > > > > >>> > > > >> >> addresses the issue if duplicate > fields > > > in > > > > > key > > > > > > > and > > > > > > > > > > value > > > > > > > > > > > > >>> objects, > > > > > > > > > > > > >>> > > > >> because > > > > > > > > > > > > >>> > > > >> >> schema will be created for a cache, > not > > > for > > > > > an > > > > > > > > > object, > > > > > > > > > > as > > > > > > > > > > > > it > > > > > > > > > > > > >>> > > happens > > > > > > > > > > > > >>> > > > >> now. > > > > > > > > > > > > >>> > > > >> >> Basically, the schema will define > > whether > > > > > there > > > > > > > is > > > > > > > > a > > > > > > > > > > > > primary > > > > > > > > > > > > >>> key > > > > > > > > > > > > >>> > or > > > > > > > > > > > > >>> > > > >> >> not, > > > > > > > > > > > > >>> > > > >> >> and which fields are included in case > > > there > > > > > is > > > > > > > one. > > > > > > > > > Any > > > > > > > > > > > API > > > > > > > > > > > > >>> that > > > > > > > > > > > > >>> > we > > > > > > > > > > > > >>> > > > >> would > > > > > > > > > > > > >>> > > > >> >> have must be compliant with this, so > it > > > > > becomes > > > > > > > > > fairly > > > > > > > > > > > easy > > > > > > > > > > > > >>> > > > >> >> to > > > > > > > > > > > > >>> > work > > > > > > > > > > > > >>> > > > >> >> with > > > > > > > > > > > > >>> > > > >> >> data as with a set of records, rather > > > than > > > > > > > > key-value > > > > > > > > > > > pairs. > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> However, could you please elaborate > on > > > the > > > > > > > relation > > > > > > > > > > > between > > > > > > > > > > > > >>> > Ignite > > > > > > > > > > > > >>> > > > and > > > > > > > > > > > > >>> > > > >> >> ORM? > > > > > > > > > > > > >>> > > > >> >> Is there a use case for Hibernate > > running > > > > on > > > > > > top > > > > > > > of > > > > > > > > > > > Ignite > > > > > > > > > > > > >>> > > > >> >> (I > > > > > > > > > > > > >>> > > haven't > > > > > > > > > > > > >>> > > > >> >> seen > > > > > > > > > > > > >>> > > > >> >> one so far)? If so, what is missing > > > exactly > > > > > on > > > > > > > the > > > > > > > > > > Ignite > > > > > > > > > > > > >>> side to > > > > > > > > > > > > >>> > > > >> support > > > > > > > > > > > > >>> > > > >> >> this? In my understanding, all you > need > > > is > > > > > SQL > > > > > > > API > > > > > > > > > > which > > > > > > > > > > > we > > > > > > > > > > > > >>> > already > > > > > > > > > > > > >>> > > > >> have. > > > > > > > > > > > > >>> > > > >> >> Am I missing something? > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> -Val > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> On Mon, Aug 31, 2020 at 2:08 PM Denis > > > > Magda < > > > > > > > > > > > > >>> dma...@apache.org> > > > > > > > > > > > > >>> > > > wrote: > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > Val, > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > I would propose adding another > point > > to > > > > the > > > > > > > > > > motivations > > > > > > > > > > > > >>> > > > >> >> > list > > > > > > > > > > > > >>> > > which > > > > > > > > > > > > >>> > > > >> >> > is > > > > > > > > > > > > >>> > > > >> >> > related to the ORM frameworks such > as > > > > > Spring > > > > > > > > Data, > > > > > > > > > > > > >>> Hibernate, > > > > > > > > > > > > >>> > > > >> Micronaut > > > > > > > > > > > > >>> > > > >> >> and > > > > > > > > > > > > >>> > > > >> >> > many others. > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > Presently, the storage engine > > requires > > > to > > > > > > > > > distinguish > > > > > > > > > > > key > > > > > > > > > > > > >>> > objects > > > > > > > > > > > > >>> > > > >> >> > from > > > > > > > > > > > > >>> > > > >> >> the > > > > > > > > > > > > >>> > > > >> >> > value ones that complicate the > usage > > of > > > > > > Ignite > > > > > > > > with > > > > > > > > > > > those > > > > > > > > > > > > >>> ORM > > > > > > > > > > > > >>> > > > >> >> > frameworks > > > > > > > > > > > > >>> > > > >> >> > (especially if a key object > comprises > > > > > several > > > > > > > > > > fields). > > > > > > > > > > > > >>> > > > >> >> > More > > > > > > > > > > > > >>> on > > > > > > > > > > > > >>> > > this > > > > > > > > > > > > >>> > > > >> can > > > > > > > > > > > > >>> > > > >> >> be > > > > > > > > > > > > >>> > > > >> >> > found here: > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Key-and-Value-fields-with-same-name-and-SQL-DML-td47557.html > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > It will be nice if the new > > schema-first > > > > > > > approach > > > > > > > > > > allows > > > > > > > > > > > > us > > > > > > > > > > > > >>> to > > > > > > > > > > > > >>> > > work > > > > > > > > > > > > >>> > > > >> with > > > > > > > > > > > > >>> > > > >> >> > a > > > > > > > > > > > > >>> > > > >> >> > single entity object when it comes > to > > > the > > > > > > ORMs. > > > > > > > > > With > > > > > > > > > > no > > > > > > > > > > > > >>> need to > > > > > > > > > > > > >>> > > > >> >> > split > > > > > > > > > > > > >>> > > > >> >> > the > > > > > > > > > > > > >>> > > > >> >> > entity into a key and value. Just > > want > > > to > > > > > be > > > > > > > sure > > > > > > > > > > that > > > > > > > > > > > > the > > > > > > > > > > > > >>> > Ignite > > > > > > > > > > > > >>> > > > >> >> > 3.0 > > > > > > > > > > > > >>> > > > >> >> > has > > > > > > > > > > > > >>> > > > >> >> > all the essential public APIs that > > > would > > > > > > > support > > > > > > > > > the > > > > > > > > > > > > >>> > > single-entity > > > > > > > > > > > > >>> > > > >> >> > based > > > > > > > > > > > > >>> > > > >> >> > approach. > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > What do you think? > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > - > > > > > > > > > > > > >>> > > > >> >> > Denis > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > On Fri, Aug 28, 2020 at 3:50 PM > > > Valentin > > > > > > > > > Kulichenko < > > > > > > > > > > > > >>> > > > >> >> > valentin.kuliche...@gmail.com> > > wrote: > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > Igniters, > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > > >>> > > > >> >> > > One of the big changes proposed > for > > > > > Ignite > > > > > > > 3.0 > > > > > > > > is > > > > > > > > > > the > > > > > > > > > > > > >>> > so-called > > > > > > > > > > > > >>> > > > >> >> > > "schema-first approach". To add > > more > > > > > > clarity, > > > > > > > > > I've > > > > > > > > > > > > >>> > > > >> >> > > started > > > > > > > > > > > > >>> > > > writing > > > > > > > > > > > > >>> > > > >> >> > > the > > > > > > > > > > > > >>> > > > >> >> > IEP > > > > > > > > > > > > >>> > > > >> >> > > for this change: > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > > >>> > > > >> >> > > Please take a look and let me > know > > if > > > > > there > > > > > > > are > > > > > > > > > any > > > > > > > > > > > > >>> immediate > > > > > > > > > > > > >>> > > > >> >> > > thoughts, > > > > > > > > > > > > >>> > > > >> >> > > suggestions, or objections. > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > > >>> > > > >> >> > > -Val > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> -- > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> Best regards, > > > > > > > > > > > > >>> > > > >> Ivan Pavlukhin > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > -- > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > Best regards, > > > > > > > > > > > > >>> > > > Ivan Pavlukhin > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > Best regards, > > > > > > > > > > > Andrey V. Mashenkov > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Best regards, > > > > > > > > > Andrey V. Mashenkov > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > Andrey V. Mashenkov > > > > > > > > > > > > > > > > > > -- > > Best regards, > > Andrey V. Mashenkov > > > -- Best regards, Andrey V. Mashenkov
