Folks, Please do not ignore history. We had a thread [1] with many bright ideas. We can resume it.
[1] http://apache-ignite-developers.2346864.n4.nabble.com/Applicability-of-term-cache-to-Apache-Ignite-td36541.html 2020-09-10 0:08 GMT+03:00, Denis Magda <dma...@apache.org>: > Val, makes sense, thanks for explaining. > > Agree that we need to have a separate discussion thread for the "table" and > "cache" terms substitution. I'll appreciate it if you start the thread > sharing pointers to any relevant IEPs and reasoning behind the suggested > change. > > - > Denis > > > On Tue, Sep 8, 2020 at 6:01 PM Valentin Kulichenko < > valentin.kuliche...@gmail.com> wrote: > >> Hi Denis, >> >> I guess the wording in the IEP is a little bit confusing. All it means is >> that you should not create nested POJOs, but rather inline fields into a >> single POJO that is mapped to a particular schema. In other words, nested >> POJOs are not supported. >> >> Alex, is this correct? Please let me know if I'm missing something. >> >> As for the "cache" term, I agree that it is outdated, but I'm not sure >> what we can replace it with. "Table" is tightly associated with SQL, but >> SQL is optional in our case. Do you want to create a separate discussion >> about this? >> >> -Val >> >> On Tue, Sep 8, 2020 at 4:37 PM Denis Magda <dma...@apache.org> wrote: >> >>> Val, >>> >>> I've checked the IEP again and have a few questions. >>> >>> Arbitrary nested objects and collections are not allowed as column >>> values. >>> > Nested POJOs should either be inlined into schema, or stored as BLOBs >>> >>> >>> Could you provide a DDL code snippet showing how the inlining of POJOs >>> is >>> supposed to work? >>> >>> Also, we keep using the terms "cache" and "table" throughout the IEP. Is >>> it >>> the right time to discuss an alternate name that would replace those >>> too? >>> Personally, the "table" should stay and the "cache" should go >>> considering >>> that SQL is one of the primary APIs in Ignite and that DDL is supported >>> out-of-the-box. >>> >>> >>> - >>> Denis >>> >>> >>> On Mon, Sep 7, 2020 at 12:26 PM Valentin Kulichenko < >>> valentin.kuliche...@gmail.com> wrote: >>> >>> > Ivan, >>> > >>> > I see your point. I agree that with the automatic updates we step into >>> the >>> > schema-last territory. >>> > >>> > Actually, if we support automatic evolution, we can as well support >>> > creating a cache without schema and inferring it from the first >>> > insert. >>> In >>> > other words, we can have both "schema-first" and "schema-last" modes. >>> > >>> > Alexey, what do you think? >>> > >>> > -Val >>> > >>> > On Mon, Sep 7, 2020 at 5:59 AM Alexey Goncharuk < >>> > alexey.goncha...@gmail.com> >>> > wrote: >>> > >>> > > Ivan, >>> > > >>> > > Thank you, I got your concern now. As it is mostly regarding the >>> > > terminology, I am absolutely fine with changing the name to whatever >>> fits >>> > > the approach best. Dynamic or evolving schema sounds great. I will >>> make >>> > > corresponding changes to the IEP once we settle on the name. >>> > > >>> > > пн, 7 сент. 2020 г. в 11:33, Ivan Pavlukhin <vololo...@gmail.com>: >>> > > >>> > > > Hi Val, >>> > > > >>> > > > Thank you for your answer! >>> > > > >>> > > > My understanding is a little bit different. Yes, schema evolution >>> > > > definitely should be possible. But I see a main difference in "how >>> > > > schema is updated". I treat a common SQL approach schema-first. >>> Schema >>> > > > and data manipulation operations are clearly separated and it >>> enables >>> > > > interesting capabilities, e.g. preventing untended schema changes >>> > > > by >>> > > > mistaken data operations, restricting user permissions to change >>> > > > schema. >>> > > > >>> > > > > Schema-first means that schema exists in advance and all the >>> stored >>> > > data >>> > > > is compliant with it - that's exactly what is proposed. >>> > > > >>> > > > A schema-last approach mentioned in [1] also assumes that schema >>> > > > exists, but it is inferred from data. Is not it more similar to >>> > > > the >>> > > > proposing approach? >>> > > > >>> > > > And I would like to say, that my main concern so far is mostly >>> > > > about >>> > > > terminology. And I suppose if it confuses me then others might be >>> > > > confused as well. My feeling is closer to "dynamic or liquid or >>> > > > may >>> be >>> > > > evolving schema". >>> > > > >>> > > > [1] >>> > > > >>> > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf >>> > > > >>> > > > 2020-09-07 0:47 GMT+03:00, Valentin Kulichenko < >>> > > > valentin.kuliche...@gmail.com>: >>> > > > > Hi Ivan, >>> > > > > >>> > > > > I don't see an issue with that. Schema-first means that schema >>> exists >>> > > in >>> > > > > advance and all the stored data is compliant with it - that's >>> exactly >>> > > > what >>> > > > > is proposed. There are no restrictions prohibiting changes to >>> > > > > the >>> > > schema. >>> > > > > >>> > > > > -Val >>> > > > > >>> > > > > On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin < >>> vololo...@gmail.com> >>> > > > wrote: >>> > > > > >>> > > > >> Alexey, >>> > > > >> >>> > > > >> I am a little bit confused with terminology. My understanding >>> > conforms >>> > > > >> to a survey [1] (see part X Semi Structured Data). Can we >>> > > > >> really >>> > treat >>> > > > >> a "dynamic schema" approach as a kind of "schema-first"? >>> > > > >> >>> > > > >> [1] >>> > > > >> >>> > > >>> https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf >>> > > > >> >>> > > > >> 2020-09-02 1:53 GMT+03:00, Denis Magda <dma...@apache.org>: >>> > > > >> >> >>> > > > >> >> However, could you please elaborate on the relation between >>> > Ignite >>> > > > and >>> > > > >> >> ORM? >>> > > > >> >> Is there a use case for Hibernate running on top of Ignite >>> > > > >> >> (I >>> > > haven't >>> > > > >> >> seen >>> > > > >> >> one so far)? If so, what is missing exactly on the Ignite >>> side to >>> > > > >> support >>> > > > >> >> this? In my understanding, all you need is SQL API which we >>> > already >>> > > > >> have. >>> > > > >> >> Am I missing something? >>> > > > >> > >>> > > > >> > >>> > > > >> > Good point, yes, if all the ORM integrations use Ignite SQL >>> APIs >>> > > > >> > internally, then they can easily translate an Entity object >>> into >>> > an >>> > > > >> > INSERT/UPDATE statement that lists all the object's fields. >>> > Luckily, >>> > > > >> > our >>> > > > >> > Spring Data integration is already based on the Ignite SQL >>> > > > >> > APIs >>> > and >>> > > > >> > needs >>> > > > >> > to be improved once the schema-first approach is supported. >>> That >>> > > would >>> > > > >> > solve a ton of usability issues. >>> > > > >> > >>> > > > >> > I would revise the Hibernate integration as well during the >>> Ignite >>> > > 3.0 >>> > > > >> dev >>> > > > >> > phase. Can't say if it's used a lot but Spring Data is >>> > > > >> > getting >>> > > > traction >>> > > > >> for >>> > > > >> > sure. >>> > > > >> > >>> > > > >> > @Michael Pollind, I'll loop you in as long as you've started >>> > working >>> > > > on >>> > > > >> the >>> > > > >> > Ignite support for Micornaut Data >>> > > > >> > < >>> > https://micronaut-projects.github.io/micronaut-data/latest/guide/> >>> > > > and >>> > > > >> > came across some challenges. Just watch this discussion. >>> > > > >> > That's >>> > what >>> > > > is >>> > > > >> > coming in Ignite 3.0. >>> > > > >> > >>> > > > >> > >>> > > > >> > - >>> > > > >> > Denis >>> > > > >> > >>> > > > >> > >>> > > > >> > On Mon, Aug 31, 2020 at 5:11 PM Valentin Kulichenko < >>> > > > >> > valentin.kuliche...@gmail.com> wrote: >>> > > > >> > >>> > > > >> >> Hi Denis, >>> > > > >> >> >>> > > > >> >> Generally speaking, I believe that the schema-first approach >>> > > natively >>> > > > >> >> addresses the issue if duplicate fields in key and value >>> objects, >>> > > > >> because >>> > > > >> >> schema will be created for a cache, not for an object, as it >>> > > happens >>> > > > >> now. >>> > > > >> >> Basically, the schema will define whether there is a primary >>> key >>> > or >>> > > > >> >> not, >>> > > > >> >> and which fields are included in case there is one. Any API >>> that >>> > we >>> > > > >> would >>> > > > >> >> have must be compliant with this, so it becomes fairly easy >>> > > > >> >> to >>> > work >>> > > > >> >> with >>> > > > >> >> data as with a set of records, rather than key-value pairs. >>> > > > >> >> >>> > > > >> >> However, could you please elaborate on the relation between >>> > Ignite >>> > > > and >>> > > > >> >> ORM? >>> > > > >> >> Is there a use case for Hibernate running on top of Ignite >>> > > > >> >> (I >>> > > haven't >>> > > > >> >> seen >>> > > > >> >> one so far)? If so, what is missing exactly on the Ignite >>> side to >>> > > > >> support >>> > > > >> >> this? In my understanding, all you need is SQL API which we >>> > already >>> > > > >> have. >>> > > > >> >> Am I missing something? >>> > > > >> >> >>> > > > >> >> -Val >>> > > > >> >> >>> > > > >> >> On Mon, Aug 31, 2020 at 2:08 PM Denis Magda < >>> dma...@apache.org> >>> > > > wrote: >>> > > > >> >> >>> > > > >> >> > Val, >>> > > > >> >> > >>> > > > >> >> > I would propose adding another point to the motivations >>> > > > >> >> > list >>> > > which >>> > > > >> >> > is >>> > > > >> >> > related to the ORM frameworks such as Spring Data, >>> Hibernate, >>> > > > >> Micronaut >>> > > > >> >> and >>> > > > >> >> > many others. >>> > > > >> >> > >>> > > > >> >> > Presently, the storage engine requires to distinguish key >>> > objects >>> > > > >> >> > from >>> > > > >> >> the >>> > > > >> >> > value ones that complicate the usage of Ignite with those >>> ORM >>> > > > >> >> > frameworks >>> > > > >> >> > (especially if a key object comprises several fields). >>> > > > >> >> > More >>> on >>> > > this >>> > > > >> can >>> > > > >> >> be >>> > > > >> >> > found here: >>> > > > >> >> > >>> > > > >> >> > >>> > > > >> >> >>> > > > >> >>> > > > >>> > > >>> > >>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Key-and-Value-fields-with-same-name-and-SQL-DML-td47557.html >>> > > > >> >> > >>> > > > >> >> > It will be nice if the new schema-first approach allows us >>> to >>> > > work >>> > > > >> with >>> > > > >> >> > a >>> > > > >> >> > single entity object when it comes to the ORMs. With no >>> need to >>> > > > >> >> > split >>> > > > >> >> > the >>> > > > >> >> > entity into a key and value. Just want to be sure that the >>> > Ignite >>> > > > >> >> > 3.0 >>> > > > >> >> > has >>> > > > >> >> > all the essential public APIs that would support the >>> > > single-entity >>> > > > >> >> > based >>> > > > >> >> > approach. >>> > > > >> >> > >>> > > > >> >> > What do you think? >>> > > > >> >> > >>> > > > >> >> > - >>> > > > >> >> > Denis >>> > > > >> >> > >>> > > > >> >> > >>> > > > >> >> > On Fri, Aug 28, 2020 at 3:50 PM Valentin Kulichenko < >>> > > > >> >> > valentin.kuliche...@gmail.com> wrote: >>> > > > >> >> > >>> > > > >> >> > > Igniters, >>> > > > >> >> > > >>> > > > >> >> > > One of the big changes proposed for Ignite 3.0 is the >>> > so-called >>> > > > >> >> > > "schema-first approach". To add more clarity, I've >>> > > > >> >> > > started >>> > > > writing >>> > > > >> >> > > the >>> > > > >> >> > IEP >>> > > > >> >> > > for this change: >>> > > > >> >> > > >>> > > > >> >> > > >>> > > > >> >> > >>> > > > >> >> >>> > > > >> >>> > > > >>> > > >>> > >>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach >>> > > > >> >> > > >>> > > > >> >> > > Please take a look and let me know if there are any >>> immediate >>> > > > >> >> > > thoughts, >>> > > > >> >> > > suggestions, or objections. >>> > > > >> >> > > >>> > > > >> >> > > -Val >>> > > > >> >> > > >>> > > > >> >> > >>> > > > >> >> >>> > > > >> > >>> > > > >> >>> > > > >> >>> > > > >> -- >>> > > > >> >>> > > > >> Best regards, >>> > > > >> Ivan Pavlukhin >>> > > > >> >>> > > > > >>> > > > >>> > > > >>> > > > -- >>> > > > >>> > > > Best regards, >>> > > > Ivan Pavlukhin >>> > > > >>> > > >>> > >>> >> > -- Best regards, Ivan Pavlukhin
