Maxim, 2.x and 3.x will have to coexist for some time - I don't see how we can avoid this considering the set of proposed changes. That said, we effectively will need to have two "masters" - one for each major version. Master for 3.x can technically be a branch in the existing repo, but having a separate repo seems cleaner, simply because it will not be a "branch" in the traditional sense.
Note that the new repo will still be under the Apache org, with the same set of committers, managed by the community, etc. All the development happening for 3.0 must follow the rules that we currently have (if anything, it's an opportunity to improve those rules). As I said during the call on Friday, I strongly believe that if there is a transparency issue, it will exist regardless of the approach we choose for 3.0. If community members develop without IEPs or public discussions, this will happen for both 2.x and 3.x unless we address this separately. I don't see how this is related to Alexey's suggestion, which targets *technical* issues with the product more than anything else. This a way to achieve better modularity, introduce better coverage with unit tests, reduce conflicts during development, etc. Coming back to transparency, let's identify the issues and fix them. It probably makes sense to have a separate discussion on this topic. -Val On Tue, Nov 10, 2020 at 1:05 PM Maxim Muzafarov <mmu...@apache.org> wrote: > Sergey, > > > Your summary makes sense to me. > > However, how we come up from *Development transparency* to *create a > separate public repository dedicated for 3.0*? For me *development > transparency* is about making changes in the master branch. These > changes will definitely be seen by all the Ignite developers. > > A dedicated public repository is technically public and visible for > everyone, but it allows development without IEPs, without public > discussion (since all the code changes are not related to the master > branch) it also allows a large number of assumptions and deviations > (like code-style violations). It also not about *development > transparency* since developers which are working on 3.0 is only a > subset of all Ignite developers which may continue working on 2.x. For > me, this would be a huge step backwards. > > Ignite veterans should remember how long the branch stabilization took > for the 2.x version with the PDS. > > > I think each breaking change should be passed through the master branch. > > On Tue, 10 Nov 2020 at 22:18, Alexei Scherbakov > <alexey.scherbak...@gmail.com> wrote: > > > > Makes sense to me. > > > > вт, 10 нояб. 2020 г. в 18:47, Sergey Chugunov <sergey.chugu...@gmail.com > >: > > > > > Igniters, > > > > > > I thought over Friday meeting ideas and concerns and summarized them in > > > these three points: > > > > > > > > > 1. *Components design unification approach.* New proposed components > > > will be developed by different contributors, but they need to be > unified > > > and should integrate with each other easily. To ensure that I > suggest > > > calling an architecture group that will create design guidelines > for all > > > components and high-level overview of overall architecture. How > code is > > > split into components, what are component boundaries, how component > > > lifecycle works and what are its interfaces - all these and other > > > questions > > > should be covered. > > > > > > 2. *Scope management.* Apache 3.0 should be implemented within a > > > reasonable time, so we need some procedure to decide whether a > > > particular > > > feature should be dropped from the scope of 3.0 and postponed to 3.1 > > > release. To do so I suggest to range all features by two parameters: > > > criticality for 3.0 and amount of breaking changes. 3.0 scope should > > > include features of high criticality AND features with a big amount > of > > > breaking changes. All other features can be made optional. > > > > > > 3. *Development transparency.* Development of all components should > be > > > made as transparent for everyone as possible. Any contributor > should be > > > able to look over any component at any stage of development. To > achieve > > > this I suggest to create a separate public repository dedicated for > 3.0 > > > development. It will make the code available for everyone but when > > > development of 3.0 is done we won't loose any stars of our current > > > repository as we merge dev repo into main one and drop dev. > > > > > > Do these ideas make sense to you? Are there any concerns not covered by > > > these suggestions? > > > > > > On Fri, Nov 6, 2020 at 7:36 PM Kseniya Romanova < > romanova.ks....@gmail.com > > > > > > > wrote: > > > > > > > Here are the slides from Alexey Goncharuk. Let's think this over and > > > > continue on Monday: > > > > > > > > > > > > https://go.gridgain.com/rs/491-TWR-806/images/Ignite_3_Plans_and_development_process.pdf > > > > > > > > чт, 5 нояб. 2020 г. в 11:13, Anton Vinogradov <a...@apache.org>: > > > > > > > > > Folks, > > > > > > > > > > Should we perform cleanup work before (r)evolutional changes? > > > > > My huge proposal is to get rid of things which we don't need anyway > > > > > - local caches, > > > > > - strange tx modes, > > > > > - code overcomplexity because of RollingUpgrade feature never > attended > > > at > > > > > AI, > > > > > - etc, > > > > > before choosing the way. > > > > > > > > > > On Tue, Nov 3, 2020 at 3:31 PM Valentin Kulichenko < > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > Ksenia, thanks for scheduling this on such short notice! > > > > > > > > > > > > As for the original topic, I do support Alexey's idea. We're not > > > going > > > > to > > > > > > rewrite anything from scratch, as most of the components are > going to > > > > be > > > > > > moved as-is or with minimal modifications. However, the changes > that > > > > are > > > > > > proposed imply serious rework of the core parts of the code, > which > > > are > > > > > not > > > > > > properly decoupled from each other and from other parts. This > makes > > > the > > > > > > incremental approach borderline impossible. Developing in a new > repo, > > > > > > however, addresses this concern. As a bonus, we can also > refactor the > > > > > code, > > > > > > introduce better decoupling, get rid of kernel context, and > develop > > > > unit > > > > > > tests (finally!). > > > > > > > > > > > > Basically, this proposal only affects the *process*, not the set > of > > > > > changes > > > > > > we had discussed before. Ignite 3.0 is our unique chance to make > > > things > > > > > > right. > > > > > > > > > > > > -Val > > > > > > > > > > > > On Tue, Nov 3, 2020 at 3:06 AM Kseniya Romanova < > > > > > romanova.ks....@gmail.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Pavel, all the interesting points will be anyway published > here in > > > > > > English > > > > > > > (as the principal "if it's not on devlist it doesn't happened" > is > > > > still > > > > > > > relevant). This is just a quick call for a group of developers. > > > Later > > > > > we > > > > > > > can do a separate presentation of idea and discussion in > English as > > > > we > > > > > > did > > > > > > > for the Ignite 3.0 draft of changes. > > > > > > > > > > > > > > вт, 3 нояб. 2020 г. в 13:52, Pavel Tupitsyn < > ptupit...@apache.org > > > >: > > > > > > > > > > > > > > > Kseniya, > > > > > > > > > > > > > > > > Thanks for scheduling this call. > > > > > > > > Do you think we can switch to English if non-Russian speaking > > > > > community > > > > > > > > members decide to join? > > > > > > > > > > > > > > > > On Tue, Nov 3, 2020 at 1:32 PM Kseniya Romanova < > > > > > > > romanova.ks....@gmail.com > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Let's do this community discussion open. Here's the link on > > > zoom > > > > > call > > > > > > > in > > > > > > > > > Russian for Friday 6 PM: > > > > > > > > > > > > > > > https://www.meetup.com/Moscow-Apache-Ignite-Meetup/events/274360378/ > > > > > > > > > > > > > > > > > > вт, 3 нояб. 2020 г. в 12:49, Nikolay Izhikov < > > > > nizhi...@apache.org > > > > > >: > > > > > > > > > > > > > > > > > > > Time works for me. > > > > > > > > > > > > > > > > > > > > > 3 нояб. 2020 г., в 12:40, Alexey Goncharuk < > > > > > > > > alexey.goncha...@gmail.com > > > > > > > > > > > > > > > > > > > > написал(а): > > > > > > > > > > > > > > > > > > > > > > Nikolay, > > > > > > > > > > > > > > > > > > > > > > I am up for the call. I will try to explain my > reasoning in > > > > > > greater > > > > > > > > > > detail > > > > > > > > > > > and will be glad to hear the concerns. Will this > Friday, > > > Nov > > > > > 6th, > > > > > > > > work? > > > > > > > > > > > > > > > > > > > > > > вт, 3 нояб. 2020 г. в 10:09, Nikolay Izhikov < > > > > > > nizhi...@apache.org > > > > > > > >: > > > > > > > > > > > > > > > > > > > > > >> Igniters, should we have a call for this topic? > > > > > > > > > > >> > > > > > > > > > > >>> 2 нояб. 2020 г., в 18:53, Pavel Tupitsyn < > > > > > ptupit...@apache.org > > > > > > > > > > > > > > > > > >> написал(а): > > > > > > > > > > >>> > > > > > > > > > > >>>> not intend to rewrite everything from scratch > > > > > > > > > > >>> > > > > > > > > > > >>>> Every single test from Ignite 2.x should be moved to > > > > Ignite > > > > > 3 > > > > > > > > > > >>>> regardless of how we choose to proceed. > > > > > > > > > > >>> > > > > > > > > > > >>> Alexey, thank you for the explanation, this > addresses all > > > > of > > > > > my > > > > > > > > > > concerns. > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > > >>> > > > > > > > > > > >>> On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov < > > > > > > > > > > >> andrey.mashen...@gmail.com> > > > > > > > > > > >>> wrote: > > > > > > > > > > >>> > > > > > > > > > > >>>> Hi, Igniters. > > > > > > > > > > >>>> > > > > > > > > > > >>>> * AFAIU, we need a new repo if we want to apply > > > different > > > > > > > > > restrictions > > > > > > > > > > >> to > > > > > > > > > > >>>> pull requests, > > > > > > > > > > >>>> otherwise I see no difference for myself. > > > > > > > > > > >>>> E.g. make static analysis (do we have?), compile, > > > styles, > > > > > and > > > > > > > > > javadoc > > > > > > > > > > >>>> checks mandatory. > > > > > > > > > > >>>> > > > > > > > > > > >>>> I think that relaxed requirements here will lead to > bad > > > > > > product > > > > > > > > > > quality. > > > > > > > > > > >>>> > > > > > > > > > > >>>> * Agree with Pavel, we should 'keep' integrations > tests > > > > > > somehow. > > > > > > > > > > >>>> During active development tests will be broken most > of > > > > time, > > > > > > so, > > > > > > > > > > >>>> I'd port them e.g. suite-by-suite once we will have > a > > > > stable > > > > > > and > > > > > > > > > > >> featured > > > > > > > > > > >>>> environment to run them and of course make test's > code > > > > clear > > > > > > and > > > > > > > > > avoid > > > > > > > > > > >>>> bad/non-relevant ones. > > > > > > > > > > >>>> > > > > > > > > > > >>>> * I like bottom-up approach. > > > > > > > > > > >>>> With it we could make a better framework. I mean > clear > > > > > > component > > > > > > > > > > >> lifecycle, > > > > > > > > > > >>>> component wiring mechanics, general methods to > approach > > > > core > > > > > > > > > > components > > > > > > > > > > >>>> such as exchange/communication > > > > > > > > > > >>>> to avoid code mess like we have in ExchangeFuture > with > > > all > > > > > > these > > > > > > > > > > custom > > > > > > > > > > >>>> callbacks for each component, interfaces like > > > > > > > > > > >>>> PartitionsExchangeAware, > IgniteChangeGlobalStateSupport > > > > and > > > > > > > > > > >>>> a pack of > > > > > > > > > > >> > > > > > start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected > > > > > > > > > > >>>> and so on in various unexpected places. > > > > > > > > > > >>>> Hope, we will be able to port most of the good code > to > > > the > > > > > new > > > > > > > > > > framework > > > > > > > > > > >>>> version. > > > > > > > > > > >>>> > > > > > > > > > > >>>> > > > > > > > > > > >>>> > > > > > > > > > > >>>> On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk < > > > > > > > > > > >>>> alexey.goncha...@gmail.com> > > > > > > > > > > >>>> wrote: > > > > > > > > > > >>>> > > > > > > > > > > >>>>> Nikolay, Pavel, > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> Thanks for the feedback! First of all, I wanted to > > > stress > > > > > > that > > > > > > > I > > > > > > > > do > > > > > > > > > > not > > > > > > > > > > >>>>> intend to rewrite everything from scratch (I never > used > > > > > this > > > > > > > > > phrase). > > > > > > > > > > >>>> There > > > > > > > > > > >>>>> are significant parts of code that would be moved > with > > > > > > minimal > > > > > > > > > > >>>>> modifications. Second, I never said that we will > get > > > rid > > > > of > > > > > > the > > > > > > > > old > > > > > > > > > > >> tests > > > > > > > > > > >>>>> codebase. Every single test from Ignite 2.x should > be > > > > moved > > > > > > to > > > > > > > > > > Ignite 3 > > > > > > > > > > >>>>> regardless of how we choose to proceed. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> My point is that for some parts of the code a clean > > > > > bottom-up > > > > > > > > > > >>>>> implementation will be cheaper in many ways. Let me > > > give > > > > > you > > > > > > a > > > > > > > > few > > > > > > > > > > >>>> concrete > > > > > > > > > > >>>>> examples: > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> - I think no one can object that we need a cleanly > > > > > separated > > > > > > > > > > >>>> persistence > > > > > > > > > > >>>>> layer for Ignite. There is a very raw draft IEP > for > > > this > > > > > > > > already. > > > > > > > > > On > > > > > > > > > > >>>> the > > > > > > > > > > >>>>> other hand, I think we also can agree that we > need a > > > > > > > split-brain > > > > > > > > > > >>>>> resistant > > > > > > > > > > >>>>> replication protocol for caches. There is also an > IEP > > > > for > > > > > > > this. > > > > > > > > > > >>>> Neither > > > > > > > > > > >>>>> of > > > > > > > > > > >>>>> the changes is a good fit for 2.x because they are > > > > likely > > > > > to > > > > > > > > > > >> introduce > > > > > > > > > > >>>>> breaking changes in the persistence layer, > > > configuration > > > > > and > > > > > > > > > > >> behavior. > > > > > > > > > > >>>>> Additionally, these components are now tightly > > > coupled, > > > > so > > > > > > > there > > > > > > > > > is > > > > > > > > > > >> no > > > > > > > > > > >>>>> way > > > > > > > > > > >>>>> these two changes can be implemented in parallel > and > > > > then > > > > > > > merged > > > > > > > > > > >>>>> together > > > > > > > > > > >>>>> easily. So what we will end up with is having to > > > > implement > > > > > > > these > > > > > > > > > > >>>> changes > > > > > > > > > > >>>>> sequentially, fixing all existing tests twice, and > > > > > > essentially > > > > > > > > > > >>>> throwing > > > > > > > > > > >>>>> away half of the work done because the other part > of > > > the > > > > > > > change > > > > > > > > is > > > > > > > > > > >>>>> re-implemented > > > > > > > > > > >>>>> - Similar example goes with getting rid of > > > > > > > IgniteInternalFuture > > > > > > > > > and > > > > > > > > > > >>>>> replacing it with CompletableFuture, and any other > > > > change > > > > > > that > > > > > > > > > > >> touches > > > > > > > > > > >>>>> the > > > > > > > > > > >>>>> asynchronous part of the code. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> Third, I do not see how this choice affects the UX > of > > > > > Ignite. > > > > > > > The > > > > > > > > > end > > > > > > > > > > >>>> user > > > > > > > > > > >>>>> experience must be fixed in the IEP regardless of > the > > > > > > > development > > > > > > > > > > >> process > > > > > > > > > > >>>>> and the fact that we have gaps in this area in > Ignite > > > 2.x > > > > > > just > > > > > > > > > > confirms > > > > > > > > > > >>>>> that. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> Pavel, agree that a repo/branch is a technicality, > I > > > > guess > > > > > if > > > > > > > > > > >>>> reformulate, > > > > > > > > > > >>>>> my point is that we might agree to have a single > > > > > development > > > > > > > > master > > > > > > > > > > >>>> branch > > > > > > > > > > >>>>> with 'disassembled' end-to-end functionality for > some > > > > > period > > > > > > of > > > > > > > > > time > > > > > > > > > > to > > > > > > > > > > >>>>> speed up development, and re-assemble the core > features > > > > > after > > > > > > > > > having > > > > > > > > > > >>>>> submodules tested independently. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> Nikolay, > > > > > > > > > > >>>>>> We have many features that have to evolve. > > > > > > > > > > >>>>>> Snapshots, rebalance, tooling, tracing, zookeeper > > > > support, > > > > > > > etc. > > > > > > > > > > >>>>> This is not very specific. In the end, resources > are > > > > > limited > > > > > > > and > > > > > > > > we > > > > > > > > > > >> will > > > > > > > > > > >>>>> not be able to drive both tracks simultaneously, > > > > especially > > > > > > > > after a > > > > > > > > > > >>>> couple > > > > > > > > > > >>>>> of features having been implemented for Ignite > 3.0. If > > > > > there > > > > > > > are > > > > > > > > > > indeed > > > > > > > > > > >>>>> some major changes that we want to do in Ignite 2.x > > > > instead > > > > > > of > > > > > > > > > > putting > > > > > > > > > > >>>>> effort into 3.0 - let's discuss them. I am just not > > > aware > > > > > of > > > > > > > any, > > > > > > > > > > >> that's > > > > > > > > > > >>>>> why I am eager to move forward with Ignite 3.0. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>>> We have bugs and issues that can be fixed in 2.x > > > without > > > > > > > > breaking > > > > > > > > > > >>>> backward > > > > > > > > > > >>>>> compatibility. > > > > > > > > > > >>>>>> We have many users who are happy with the 2.x > with all > > > > > it’s > > > > > > > > > issues. > > > > > > > > > > >>>>> These changes will be covered by end-to-end tests > and > > > > > > migrated > > > > > > > to > > > > > > > > > > >> Ignite > > > > > > > > > > >>>>> 3.0, so I see no issues here. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> Finally, Anton & Nikolay > > > > > > > > > > >>>>> I do not have an estimate for this simply because > the > > > > > > activity > > > > > > > is > > > > > > > > > > >>>>> community-driven and it depends on the number of > people > > > > > > willing > > > > > > > > to > > > > > > > > > > >>>>> contribute. With the current pace, I would hope to > have > > > > an > > > > > RC > > > > > > > of > > > > > > > > > > Ignite > > > > > > > > > > >>>> 3.0 > > > > > > > > > > >>>>> to be ready by the end of 2021. My gut feeling is > that > > > by > > > > > > > moving > > > > > > > > > with > > > > > > > > > > >>>>> incremental changes, we will not be able to > implement > > > > even > > > > > > half > > > > > > > > of > > > > > > > > > > the > > > > > > > > > > >>>>> wishlist by that time. > > > > > > > > > > >>>>> I doubt that releasing several major releases with > > > > breaking > > > > > > > > changes > > > > > > > > > > >> will > > > > > > > > > > >>>>> make Ignite users happy either because each upgrade > > > will > > > > > cost > > > > > > > > > Ignite > > > > > > > > > > >>>> users > > > > > > > > > > >>>>> money, so the fewer major versions we release, the > > > > better. > > > > > > Thus > > > > > > > > my > > > > > > > > > > wish > > > > > > > > > > >>>> to > > > > > > > > > > >>>>> include all breaking changes in one release. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> I'll be now quiet for a while, let's see what other > > > > > community > > > > > > > > > members > > > > > > > > > > >>>>> think. > > > > > > > > > > >>>>> > > > > > > > > > > >>>>> пн, 2 нояб. 2020 г. в 15:52, Pavel Tupitsyn < > > > > > > > > ptupit...@apache.org > > > > > > > > > >: > > > > > > > > > > >>>>> > > > > > > > > > > >>>>>> 1. Rewriting from scratch is never a good idea. > > > > > > > > > > >>>>>> We don't want to follow the path of Netscape and > lose > > > > all > > > > > > our > > > > > > > > > users > > > > > > > > > > >>>>>> by the time we have a working 3.0 [1] > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> 2. Not sure about new repo - seems like some pain > and > > > no > > > > > > gain, > > > > > > > > > > what's > > > > > > > > > > >>>> the > > > > > > > > > > >>>>>> problem with a branch? > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> 3. We should keep existing integration tests when > > > > > possible. > > > > > > > > > > >>>>>> We have accumulated a lot of edge case knowledge > over > > > > the > > > > > > > years, > > > > > > > > > > >>>>>> it is not a good idea to send all of that down the > > > > drain. > > > > > > > > > > >>>>>> Yes, integration tests are slow, but they are the > most > > > > > > > valuable. > > > > > > > > > > >>>>>> I think we can move more stuff into nightly runs > and > > > > have > > > > > a > > > > > > > fast > > > > > > > > > and > > > > > > > > > > >>>>> modern > > > > > > > > > > >>>>>> basic suite. > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> Alexey, you are much more familiar with the Ignite > > > core > > > > > > > codebase > > > > > > > > > > than > > > > > > > > > > >>>>> most > > > > > > > > > > >>>>>> of us, > > > > > > > > > > >>>>>> can you please explain in more detail which > particular > > > > > > > feature, > > > > > > > > in > > > > > > > > > > >> your > > > > > > > > > > >>>>>> opinion, > > > > > > > > > > >>>>>> mandates this "start from scratch" approach? > > > > > > > > > > >>>>>> Is it really not possible at all to follow a less > > > > radical > > > > > > way? > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> [1] > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>> > > > > > > > > > > >>>> > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>> On Mon, Nov 2, 2020 at 2:25 PM Nikolay Izhikov < > > > > > > > > > nizhi...@apache.org > > > > > > > > > > > > > > > > > > > > > >>>>>> wrote: > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>>> Hello, Alexey. > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>> I think that «rewriting from scratch» approach > has a > > > > high > > > > > > > risk > > > > > > > > to > > > > > > > > > > >>>> make > > > > > > > > > > >>>>>> new > > > > > > > > > > >>>>>>> features unusable. > > > > > > > > > > >>>>>>> At the time Ignite2 was started no-one wants to > do > > > bad > > > > UX > > > > > > or > > > > > > > > bad > > > > > > > > > > >>>>>> features. > > > > > > > > > > >>>>>>> Nevertheless, it happen. > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>> I think we can avoid it with the Ignite3 and > > > successors > > > > > if > > > > > > we > > > > > > > > > will > > > > > > > > > > >>>> move > > > > > > > > > > >>>>>>> step by step without keeping backward > compatibility > > > > > > > > > > >>>>>>> With the step by step approach, we can focus on > each > > > > > > > component > > > > > > > > > > >>>>>> separately. > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>>> What new features are we planning to implement > for > > > > > Ignite > > > > > > > 2.x? > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>> We have many features that have to evolve. > > > > > > > > > > >>>>>>> Snapshots, rebalance, tooling, tracing, zookeeper > > > > > support, > > > > > > > etc. > > > > > > > > > > >>>>>>> We have bugs and issues that can be fixed in 2.x > > > > without > > > > > > > > breaking > > > > > > > > > > >>>>>> backward > > > > > > > > > > >>>>>>> compatibility. > > > > > > > > > > >>>>>>> We have many users who are happy with the 2.x > with > > > all > > > > > it’s > > > > > > > > > issues. > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>>> 2 нояб. 2020 г., в 14:09, Anton Vinogradov < > > > > > a...@apache.org > > > > > > > > > > > > > > > > > >>>>> написал(а): > > > > > > > > > > >>>>>>>> > > > > > > > > > > >>>>>>>> Alexey, > > > > > > > > > > >>>>>>>> > > > > > > > > > > >>>>>>>> Do we have any estimates of how fast we'll be > able > > > to > > > > > gain > > > > > > > > > > >>>>>>> production-ready > > > > > > > > > > >>>>>>>> AI 3.0 in case of a "new repo" choice? > > > > > > > > > > >>>>>>>> > > > > > > > > > > >>>>>>>> On Mon, Nov 2, 2020 at 2:01 PM Alexey Goncharuk > < > > > > > > > > > > >>>>>>> alexey.goncha...@gmail.com> > > > > > > > > > > >>>>>>>> wrote: > > > > > > > > > > >>>>>>>> > > > > > > > > > > >>>>>>>>> Nikolay, > > > > > > > > > > >>>>>>>>> > > > > > > > > > > >>>>>>>>> What new features are we planning to implement > for > > > > > Ignite > > > > > > > > 2.x? > > > > > > > > > I > > > > > > > > > > >>>>> think > > > > > > > > > > >>>>>>> once > > > > > > > > > > >>>>>>>>> we commence working on Ignite 3.0, we should > > > > gradually > > > > > > > cease > > > > > > > > > the > > > > > > > > > > >>>>>>> activity > > > > > > > > > > >>>>>>>>> on Ignite 2.x to mere bugfixes because such > > > parallel > > > > > > > > > development > > > > > > > > > > >>>>> will > > > > > > > > > > >>>>>> be > > > > > > > > > > >>>>>>>>> overwhelming regardless of how we choose to > > > proceed. > > > > > > > > > > >>>>>>>>> > > > > > > > > > > >>>>>>>>> пн, 2 нояб. 2020 г. в 13:38, Nikolay Izhikov < > > > > > > > > > > nizhi...@apache.org > > > > > > > > > > >>>>> : > > > > > > > > > > >>>>>>>>> > > > > > > > > > > >>>>>>>>>> To be clear: > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>>>> I would suggest creating a new repository for > > > > Ignite > > > > > > 3.0 > > > > > > > > > > >>>>> (perhaps, a > > > > > > > > > > >>>>>>>>> new > > > > > > > > > > >>>>>>>>>> clean branch, but a new repo looks nicer to > me) > > > and > > > > a > > > > > > new > > > > > > > > > Ignite > > > > > > > > > > >>>>> 3.0 > > > > > > > > > > >>>>>>>>>> TeamCity project. > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>>> +1 for new Team City project. > > > > > > > > > > >>>>>>>>>> +1 for new branch for Ignite3. > > > > > > > > > > >>>>>>>>>> -1 for new repo. > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>>>> 2 нояб. 2020 г., в 13:35, Nikolay Izhikov < > > > > > > > > > > >>>> nizhikov....@gmail.com > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>>>>>>> написал(а): > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>> Hello, Alexey. > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>> I think it will hurt our project more than > help. > > > > > > > > > > >>>>>>>>>>> Developing new features for 2 separate > branches > > > > with > > > > > > the > > > > > > > > > > >>>> different > > > > > > > > > > >>>>>>> APIs > > > > > > > > > > >>>>>>>>>> and internal structure is overwhelming > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>> Maybe we should relax a bit requirements for > > > > Ignite3? > > > > > > > > > > >>>>>>>>>>> Maybe we should move step by step and make > > > Ignite3 > > > > > with > > > > > > > new > > > > > > > > > > >>>>>>>>>> configuration than Ignite4 with new > transactions, > > > > etc? > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> 2 нояб. 2020 г., в 13:14, Alexey Goncharuk < > > > > > > > > > > >>>>>>>>> alexey.goncha...@gmail.com> > > > > > > > > > > >>>>>>>>>> написал(а): > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> Igniters, > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> I wanted to pitch a rather radical idea > > > regarding > > > > > the > > > > > > > > Ignite > > > > > > > > > > >>>> 3.0 > > > > > > > > > > >>>>>>>>>>>> development which has occurred to me some > time > > > > ago. > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> We already have several IEPs targeted to > Ignite > > > > 3.0 > > > > > > > which > > > > > > > > > > imply > > > > > > > > > > >>>>>> major > > > > > > > > > > >>>>>>>>>>>> changes to the codebase (the change in > > > replication > > > > > > > > protocol > > > > > > > > > > and > > > > > > > > > > >>>>>> thus > > > > > > > > > > >>>>>>>>>>>> transactions, change in binary format, > updated > > > > > > > > metastorage, > > > > > > > > > > >>>> etc). > > > > > > > > > > >>>>>> We > > > > > > > > > > >>>>>>>>>> also > > > > > > > > > > >>>>>>>>>>>> planned significant changes in public APIs: > > > > > > > configuration > > > > > > > > > > >>>> format > > > > > > > > > > >>>>>>>>> change, > > > > > > > > > > >>>>>>>>>>>> improvements in cache APIs, SQL APIs, > > > transaction > > > > > mode > > > > > > > > > rework. > > > > > > > > > > >>>>> The > > > > > > > > > > >>>>>>>>>> wishlist > > > > > > > > > > >>>>>>>>>>>> of changes for Ignite 3.0 is huge. > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> So, I was wondering whether it makes sense > to > > > try > > > > to > > > > > > > > change > > > > > > > > > > the > > > > > > > > > > >>>>> old > > > > > > > > > > >>>>>>>>>>>> codebase, or start with a new baseline and > move > > > > old > > > > > > > pieces > > > > > > > > > of > > > > > > > > > > >>>>> code > > > > > > > > > > >>>>>>>>> that > > > > > > > > > > >>>>>>>>>> do > > > > > > > > > > >>>>>>>>>>>> not require significant rework. Personally, > I > > > > would > > > > > go > > > > > > > > with > > > > > > > > > > the > > > > > > > > > > >>>>>>> second > > > > > > > > > > >>>>>>>>>>>> option for the following reasons: > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> - We have a chance to shift the development > > > > paradigm > > > > > > in > > > > > > > > the > > > > > > > > > > >>>>> project > > > > > > > > > > >>>>>>>>> and > > > > > > > > > > >>>>>>>>>>>> introduce the practice of true unit-tests. > In > > > the > > > > > new > > > > > > > > > baseline > > > > > > > > > > >>>> at > > > > > > > > > > >>>>>> the > > > > > > > > > > >>>>>>>>>>>> beginning there will be no ability to run an > > > > > > end-to-end > > > > > > > > > > >>>> scenario, > > > > > > > > > > >>>>>>>>> thus > > > > > > > > > > >>>>>>>>>> we > > > > > > > > > > >>>>>>>>>>>> will be forced to write unit-tests. So far, > such > > > > > > > practice > > > > > > > > > was > > > > > > > > > > >>>>> hard > > > > > > > > > > >>>>>> to > > > > > > > > > > >>>>>>>>>>>> implement because of tight coupling between > > > Ignite > > > > > > > > > components > > > > > > > > > > >>>> and > > > > > > > > > > >>>>>>>>>> inability > > > > > > > > > > >>>>>>>>>>>> to instantiate components without an > instance of > > > > > > > > > > KernalContext. > > > > > > > > > > >>>>> For > > > > > > > > > > >>>>>>>>>>>> example, we should be able to thoroughly > test > > > > > internal > > > > > > > > > > >>>>> primitives, > > > > > > > > > > >>>>>>>>>> such as > > > > > > > > > > >>>>>>>>>>>> replication protocol (without actual > > > > communication), > > > > > > > > > > >>>> distributed > > > > > > > > > > >>>>>>>>>>>> metastorage contracts, persistence layer, > etc. > > > > > > > > > > >>>>>>>>>>>> - We will significantly reduce the > development > > > > cycle > > > > > > in > > > > > > > > the > > > > > > > > > > >>>>>> beginning > > > > > > > > > > >>>>>>>>>>>> (right now the RunAll takes two hours of > > > > > astronomical > > > > > > > time > > > > > > > > > > with > > > > > > > > > > >>>>>> empty > > > > > > > > > > >>>>>>>>>> TC; > > > > > > > > > > >>>>>>>>>>>> in the new approach developer will be able > to > > > run > > > > > ALL > > > > > > > > tests > > > > > > > > > > >>>>> locally > > > > > > > > > > >>>>>>>>> in > > > > > > > > > > >>>>>>>>>> a > > > > > > > > > > >>>>>>>>>>>> matter of minutes) > > > > > > > > > > >>>>>>>>>>>> - We can get rid of TC bot and enforce > green TC > > > by > > > > > > > > > integrating > > > > > > > > > > >>>> TC > > > > > > > > > > >>>>>>>>> build > > > > > > > > > > >>>>>>>>>>>> results with GitHub PRs (the same way > Travis is > > > > > > > currently > > > > > > > > > > >>>>>> integrated > > > > > > > > > > >>>>>>>>>> to PR > > > > > > > > > > >>>>>>>>>>>> check). We should restrict PR merge without > a TC > > > > > check > > > > > > > > > > >>>>>>>>>>>> - We will still have to re-write all tests, > but > > > > only > > > > > > > once. > > > > > > > > > If > > > > > > > > > > >>>> we > > > > > > > > > > >>>>>> try > > > > > > > > > > >>>>>>>>> to > > > > > > > > > > >>>>>>>>>>>> modify the old codebase, we would need to > modify > > > > all > > > > > > the > > > > > > > > > tests > > > > > > > > > > >>>>> for > > > > > > > > > > >>>>>>>>>> every > > > > > > > > > > >>>>>>>>>>>> major change (public API change, > configuration > > > > > change) > > > > > > > > > > >>>>>>>>>>>> - We will have fewer conflicts when working > > > > > together. > > > > > > > For > > > > > > > > > > >>>>> example, > > > > > > > > > > >>>>>> I > > > > > > > > > > >>>>>>>>>>>> cannot imagine how one would merge two > changes > > > of > > > > > > > getting > > > > > > > > > rid > > > > > > > > > > >>>> of > > > > > > > > > > >>>>>>>>>>>> IgniteFuture and changes in replication > > > protocol, > > > > > for > > > > > > > > > example > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> Technically, I would suggest creating a new > > > > > repository > > > > > > > for > > > > > > > > > > >>>> Ignite > > > > > > > > > > >>>>>> 3.0 > > > > > > > > > > >>>>>>>>>>>> (perhaps, a new clean branch, but a new repo > > > looks > > > > > > nicer > > > > > > > > to > > > > > > > > > > me) > > > > > > > > > > >>>>>> and a > > > > > > > > > > >>>>>>>>>> new > > > > > > > > > > >>>>>>>>>>>> Ignite 3.0 TeamCity project. > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> While it may seem quite radical, I do > believe > > > that > > > > > > this > > > > > > > > > > >>>> approach > > > > > > > > > > >>>>>> will > > > > > > > > > > >>>>>>>>>> give > > > > > > > > > > >>>>>>>>>>>> us more benefits than trying to make such > major > > > > > > changes > > > > > > > in > > > > > > > > > the > > > > > > > > > > >>>>>>>>> existing > > > > > > > > > > >>>>>>>>>>>> codebase. If needed, let's schedule a > discord > > > chat > > > > > > like > > > > > > > > > before > > > > > > > > > > >>>> to > > > > > > > > > > >>>>>>>>>> discuss > > > > > > > > > > >>>>>>>>>>>> this. > > > > > > > > > > >>>>>>>>>>>> > > > > > > > > > > >>>>>>>>>>>> WDYT? > > > > > > > > > > >>>>>>>>>>> > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>>> > > > > > > > > > > >>>>>>>>> > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>>> > > > > > > > > > > >>>>>> > > > > > > > > > > >>>>> > > > > > > > > > > >>>> > > > > > > > > > > >>>> > > > > > > > > > > >>>> -- > > > > > > > > > > >>>> Best regards, > > > > > > > > > > >>>> Andrey V. Mashenkov > > > > > > > > > > >>>> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > Alexei Scherbakov >
