Val,
Why *creating a new repo* is the main point we faced with? Would it be better to discuss the components design approach and scope management first suggested by Sergey Chugunov? I doubt that new repo will solve move us forward. Currently, I'm -1 to create a new repo with the inputs above. In addition to Nikolay's answer I see the following drawbacks of creating new repo: - we have very few positive examples of finalizing really huge improvements to *production-ready* states the others remains incomplete (MVCC, Calcite, Zookeeper, Tracing, Thread per Partition, etc) - AFAIK, the Native Persistence took a very long period of stabilization even after it has been developed (we must take it into account for developing new features like IEP-61) - feature development for a long period of time (like 3.0) without any releases will lead to all these changes became obsolete at the moment of release (AFAIK the 2.8 which released a year ago still has no big deployments) - human resources -- some of the Igniters may lose their interest for 3.0 during development, some of them may switch to different projects, etc. - do we all estimating the scope of 3.0 correct? The 2.8 release took 1.5 years. Have I missed something? I suggest the following plan: - initiate 3.0 development in the master branch (after 2.10 release change version to 3.0-SNAPSHOT instead of 2.11-SNAPSHOT) - cleanup and collapse all the current APIs (see To Be Removed List For Discussion on Apache Ignite 3.0 Wishlist) - reduce the scope for 3.0 even more. I suggest focusing on two things: Calcite + Schema-first approach - create feature branches for proposed IEPs (for 3.0 only) - create the release road map (allocate e.g. IEP-61 to 4.0 etc.) On Fri, 13 Nov 2020 at 14:03, Ivan Daschinsky <ivanda...@gmail.com> wrote: > > >> b. Implement IEP-61 - Common Replication Infrastructure > I suppose, that this is the main cause of the current discussion. > I hardly believe that this activity can be done without at least creating a > completely new branch. > > пт, 13 нояб. 2020 г. в 11:12, Nikolay Izhikov <nizhi...@apache.org>: > > > My suggestion: > > > > 1. Reduce Ignite3 scope to the following: > > a. Delete all deprecated API and support of obsolete internal > > protocols. > > b. Implement IEP-61 - Common Replication Infrastructure > > c. Implement new Ignite management tool ignitectl as suggested > > during Ignite3 discussion. > > > > 2. Implement and release following improvements like transactions, Calcite > > based SQL, etc in the ongoing releases - Ignite 4, 5, 6 > > > > My concern against separate Ignite 3 repo is the following: > > > > 1. We spread community to the two very separated part - Ignite3 developers > > and Ignite2 maintainers. believe it’s bad for our community. > > That can lead to the situation when we don’t fix critical or > > blocker issueds «because they will not exists in Ignite3» > > That will lead to the solutions never reviewed or reviewed poorly. > > > > 2. It seems for me that current scope of Ignite3 is too big to be > > implemented in any reasonable time. > > > > > > > 13 нояб. 2020 г., в 10:57, Nikolay Izhikov <nizhikov....@gmail.com> > > написал(а): > > > > > > Hello, Valentin. > > > > > >> Nikolay, Maxim, are you OK with this route? > > > > > > -1 to have another repo for Ignite3 development. > > > > > >> 13 нояб. 2020 г., в 03:04, Valentin Kulichenko < > > valentin.kuliche...@gmail.com> написал(а): > > >> > > >> Folks, > > >> > > >> We already have multiple IEPs for Ignite 3.0, and as far as I know, > > there are contributors that would like to work on them (or probably already > > started). That said, we should make a decision as soon as possible. > > >> > > >> At this point, it doesn't seem that there are any strong objections to > > the technical side of things. So I would suggest the following: > > >> > > >> 1. Proceed with Alexey's approach to the development process, as it > > seems to be the best (in my opinion - the only) way to address all the > > technical concerns and issues expressed in the thread. We'll start by > > creating a new repo and a new TC project. > > >> 2. Start a separate discussion around transparency. If there are any > > changes we need to make to our contributor guidelines, I am happy to talk > > them through, but I don't think it's reasonable to delay feature > > development because of this. In the short term, I will make sure that > > everything that happens within the new repo is as open to the community as > > possible. > > >> > > >> Nikolay, Maxim, are you OK with this route? > > >> > > >> -Val > > >> > > >> On Tue, Nov 10, 2020 at 4:55 PM Valentin Kulichenko < > > valentin.kuliche...@gmail.com> wrote: > > >> Maxim, > > >> > > >> 2.x and 3.x will have to coexist for some time - I don't see how we can > > avoid this considering the set of proposed changes. That said, we > > effectively will need to have two "masters" - one for each major version. > > Master for 3.x can technically be a branch in the existing repo, but having > > a separate repo seems cleaner, simply because it will not be a "branch" in > > the traditional sense. > > >> > > >> Note that the new repo will still be under the Apache org, with the > > same set of committers, managed by the community, etc. All the development > > happening for 3.0 must follow the rules that we currently have (if > > anything, it's an opportunity to improve those rules). > > >> > > >> As I said during the call on Friday, I strongly believe that if there > > is a transparency issue, it will exist regardless of the approach we choose > > for 3.0. If community members develop without IEPs or public discussions, > > this will happen for both 2.x and 3.x unless we address this separately. I > > don't see how this is related to Alexey's suggestion, which targets > > *technical* issues with the product more than anything else. This a way to > > achieve better modularity, introduce better coverage with unit tests, > > reduce conflicts during development, etc. > > >> > > >> Coming back to transparency, let's identify the issues and fix them. It > > probably makes sense to have a separate discussion on this topic. > > >> > > >> -Val > > >> > > >> On Tue, Nov 10, 2020 at 1:05 PM Maxim Muzafarov <mmu...@apache.org> > > wrote: > > >> Sergey, > > >> > > >> > > >> Your summary makes sense to me. > > >> > > >> However, how we come up from *Development transparency* to *create a > > >> separate public repository dedicated for 3.0*? For me *development > > >> transparency* is about making changes in the master branch. These > > >> changes will definitely be seen by all the Ignite developers. > > >> > > >> A dedicated public repository is technically public and visible for > > >> everyone, but it allows development without IEPs, without public > > >> discussion (since all the code changes are not related to the master > > >> branch) it also allows a large number of assumptions and deviations > > >> (like code-style violations). It also not about *development > > >> transparency* since developers which are working on 3.0 is only a > > >> subset of all Ignite developers which may continue working on 2.x. For > > >> me, this would be a huge step backwards. > > >> > > >> Ignite veterans should remember how long the branch stabilization took > > >> for the 2.x version with the PDS. > > >> > > >> > > >> I think each breaking change should be passed through the master branch. > > >> > > >> On Tue, 10 Nov 2020 at 22:18, Alexei Scherbakov > > >> <alexey.scherbak...@gmail.com> wrote: > > >>> > > >>> Makes sense to me. > > >>> > > >>> вт, 10 нояб. 2020 г. в 18:47, Sergey Chugunov < > > sergey.chugu...@gmail.com>: > > >>> > > >>>> Igniters, > > >>>> > > >>>> I thought over Friday meeting ideas and concerns and summarized them > > in > > >>>> these three points: > > >>>> > > >>>> > > >>>> 1. *Components design unification approach.* New proposed components > > >>>> will be developed by different contributors, but they need to be > > unified > > >>>> and should integrate with each other easily. To ensure that I > > suggest > > >>>> calling an architecture group that will create design guidelines > > for all > > >>>> components and high-level overview of overall architecture. How > > code is > > >>>> split into components, what are component boundaries, how component > > >>>> lifecycle works and what are its interfaces - all these and other > > >>>> questions > > >>>> should be covered. > > >>>> > > >>>> 2. *Scope management.* Apache 3.0 should be implemented within a > > >>>> reasonable time, so we need some procedure to decide whether a > > >>>> particular > > >>>> feature should be dropped from the scope of 3.0 and postponed to 3.1 > > >>>> release. To do so I suggest to range all features by two parameters: > > >>>> criticality for 3.0 and amount of breaking changes. 3.0 scope should > > >>>> include features of high criticality AND features with a big amount > > of > > >>>> breaking changes. All other features can be made optional. > > >>>> > > >>>> 3. *Development transparency.* Development of all components should > > be > > >>>> made as transparent for everyone as possible. Any contributor > > should be > > >>>> able to look over any component at any stage of development. To > > achieve > > >>>> this I suggest to create a separate public repository dedicated for > > 3.0 > > >>>> development. It will make the code available for everyone but when > > >>>> development of 3.0 is done we won't loose any stars of our current > > >>>> repository as we merge dev repo into main one and drop dev. > > >>>> > > >>>> Do these ideas make sense to you? Are there any concerns not covered > > by > > >>>> these suggestions? > > >>>> > > >>>> On Fri, Nov 6, 2020 at 7:36 PM Kseniya Romanova < > > romanova.ks....@gmail.com > > >>>>> > > >>>> wrote: > > >>>> > > >>>>> Here are the slides from Alexey Goncharuk. Let's think this over and > > >>>>> continue on Monday: > > >>>>> > > >>>>> > > >>>> > > https://go.gridgain.com/rs/491-TWR-806/images/Ignite_3_Plans_and_development_process.pdf > > >>>>> > > >>>>> чт, 5 нояб. 2020 г. в 11:13, Anton Vinogradov <a...@apache.org>: > > >>>>> > > >>>>>> Folks, > > >>>>>> > > >>>>>> Should we perform cleanup work before (r)evolutional changes? > > >>>>>> My huge proposal is to get rid of things which we don't need anyway > > >>>>>> - local caches, > > >>>>>> - strange tx modes, > > >>>>>> - code overcomplexity because of RollingUpgrade feature never > > attended > > >>>> at > > >>>>>> AI, > > >>>>>> - etc, > > >>>>>> before choosing the way. > > >>>>>> > > >>>>>> On Tue, Nov 3, 2020 at 3:31 PM Valentin Kulichenko < > > >>>>>> valentin.kuliche...@gmail.com> wrote: > > >>>>>> > > >>>>>>> Ksenia, thanks for scheduling this on such short notice! > > >>>>>>> > > >>>>>>> As for the original topic, I do support Alexey's idea. We're not > > >>>> going > > >>>>> to > > >>>>>>> rewrite anything from scratch, as most of the components are going > > to > > >>>>> be > > >>>>>>> moved as-is or with minimal modifications. However, the changes > > that > > >>>>> are > > >>>>>>> proposed imply serious rework of the core parts of the code, which > > >>>> are > > >>>>>> not > > >>>>>>> properly decoupled from each other and from other parts. This makes > > >>>> the > > >>>>>>> incremental approach borderline impossible. Developing in a new > > repo, > > >>>>>>> however, addresses this concern. As a bonus, we can also refactor > > the > > >>>>>> code, > > >>>>>>> introduce better decoupling, get rid of kernel context, and develop > > >>>>> unit > > >>>>>>> tests (finally!). > > >>>>>>> > > >>>>>>> Basically, this proposal only affects the *process*, not the set of > > >>>>>> changes > > >>>>>>> we had discussed before. Ignite 3.0 is our unique chance to make > > >>>> things > > >>>>>>> right. > > >>>>>>> > > >>>>>>> -Val > > >>>>>>> > > >>>>>>> On Tue, Nov 3, 2020 at 3:06 AM Kseniya Romanova < > > >>>>>> romanova.ks....@gmail.com > > >>>>>>>> > > >>>>>>> wrote: > > >>>>>>> > > >>>>>>>> Pavel, all the interesting points will be anyway published here in > > >>>>>>> English > > >>>>>>>> (as the principal "if it's not on devlist it doesn't happened" is > > >>>>> still > > >>>>>>>> relevant). This is just a quick call for a group of developers. > > >>>> Later > > >>>>>> we > > >>>>>>>> can do a separate presentation of idea and discussion in English > > as > > >>>>> we > > >>>>>>> did > > >>>>>>>> for the Ignite 3.0 draft of changes. > > >>>>>>>> > > >>>>>>>> вт, 3 нояб. 2020 г. в 13:52, Pavel Tupitsyn <ptupit...@apache.org > > >>>>> : > > >>>>>>>> > > >>>>>>>>> Kseniya, > > >>>>>>>>> > > >>>>>>>>> Thanks for scheduling this call. > > >>>>>>>>> Do you think we can switch to English if non-Russian speaking > > >>>>>> community > > >>>>>>>>> members decide to join? > > >>>>>>>>> > > >>>>>>>>> On Tue, Nov 3, 2020 at 1:32 PM Kseniya Romanova < > > >>>>>>>> romanova.ks....@gmail.com > > >>>>>>>>>> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> Let's do this community discussion open. Here's the link on > > >>>> zoom > > >>>>>> call > > >>>>>>>> in > > >>>>>>>>>> Russian for Friday 6 PM: > > >>>>>>>>>> > > >>>>>> > > https://www.meetup.com/Moscow-Apache-Ignite-Meetup/events/274360378/ > > >>>>>>>>>> > > >>>>>>>>>> вт, 3 нояб. 2020 г. в 12:49, Nikolay Izhikov < > > >>>>> nizhi...@apache.org > > >>>>>>> : > > >>>>>>>>>> > > >>>>>>>>>>> Time works for me. > > >>>>>>>>>>> > > >>>>>>>>>>>> 3 нояб. 2020 г., в 12:40, Alexey Goncharuk < > > >>>>>>>>> alexey.goncha...@gmail.com > > >>>>>>>>>>> > > >>>>>>>>>>> написал(а): > > >>>>>>>>>>>> > > >>>>>>>>>>>> Nikolay, > > >>>>>>>>>>>> > > >>>>>>>>>>>> I am up for the call. I will try to explain my reasoning in > > >>>>>>> greater > > >>>>>>>>>>> detail > > >>>>>>>>>>>> and will be glad to hear the concerns. Will this Friday, > > >>>> Nov > > >>>>>> 6th, > > >>>>>>>>> work? > > >>>>>>>>>>>> > > >>>>>>>>>>>> вт, 3 нояб. 2020 г. в 10:09, Nikolay Izhikov < > > >>>>>>> nizhi...@apache.org > > >>>>>>>>> : > > >>>>>>>>>>>> > > >>>>>>>>>>>>> Igniters, should we have a call for this topic? > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> 2 нояб. 2020 г., в 18:53, Pavel Tupitsyn < > > >>>>>> ptupit...@apache.org > > >>>>>>>> > > >>>>>>>>>>>>> написал(а): > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> not intend to rewrite everything from scratch > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Every single test from Ignite 2.x should be moved to > > >>>>> Ignite > > >>>>>> 3 > > >>>>>>>>>>>>>>> regardless of how we choose to proceed. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Alexey, thank you for the explanation, this addresses all > > >>>>> of > > >>>>>> my > > >>>>>>>>>>> concerns. > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> On Mon, Nov 2, 2020 at 6:43 PM Andrey Mashenkov < > > >>>>>>>>>>>>> andrey.mashen...@gmail.com> > > >>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Hi, Igniters. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> * AFAIU, we need a new repo if we want to apply > > >>>> different > > >>>>>>>>>> restrictions > > >>>>>>>>>>>>> to > > >>>>>>>>>>>>>>> pull requests, > > >>>>>>>>>>>>>>> otherwise I see no difference for myself. > > >>>>>>>>>>>>>>> E.g. make static analysis (do we have?), compile, > > >>>> styles, > > >>>>>> and > > >>>>>>>>>> javadoc > > >>>>>>>>>>>>>>> checks mandatory. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> I think that relaxed requirements here will lead to bad > > >>>>>>> product > > >>>>>>>>>>> quality. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> * Agree with Pavel, we should 'keep' integrations tests > > >>>>>>> somehow. > > >>>>>>>>>>>>>>> During active development tests will be broken most of > > >>>>> time, > > >>>>>>> so, > > >>>>>>>>>>>>>>> I'd port them e.g. suite-by-suite once we will have a > > >>>>> stable > > >>>>>>> and > > >>>>>>>>>>>>> featured > > >>>>>>>>>>>>>>> environment to run them and of course make test's code > > >>>>> clear > > >>>>>>> and > > >>>>>>>>>> avoid > > >>>>>>>>>>>>>>> bad/non-relevant ones. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> * I like bottom-up approach. > > >>>>>>>>>>>>>>> With it we could make a better framework. I mean clear > > >>>>>>> component > > >>>>>>>>>>>>> lifecycle, > > >>>>>>>>>>>>>>> component wiring mechanics, general methods to approach > > >>>>> core > > >>>>>>>>>>> components > > >>>>>>>>>>>>>>> such as exchange/communication > > >>>>>>>>>>>>>>> to avoid code mess like we have in ExchangeFuture with > > >>>> all > > >>>>>>> these > > >>>>>>>>>>> custom > > >>>>>>>>>>>>>>> callbacks for each component, interfaces like > > >>>>>>>>>>>>>>> PartitionsExchangeAware, IgniteChangeGlobalStateSupport > > >>>>> and > > >>>>>>>>>>>>>>> a pack of > > >>>>>>>>>>>>> > > >>>>>> start/stop/onKernalStart/onPluginStart/onActivate/onDisconnected > > >>>>>>>>>>>>>>> and so on in various unexpected places. > > >>>>>>>>>>>>>>> Hope, we will be able to port most of the good code to > > >>>> the > > >>>>>> new > > >>>>>>>>>>> framework > > >>>>>>>>>>>>>>> version. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> On Mon, Nov 2, 2020 at 6:18 PM Alexey Goncharuk < > > >>>>>>>>>>>>>>> alexey.goncha...@gmail.com> > > >>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Nikolay, Pavel, > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Thanks for the feedback! First of all, I wanted to > > >>>> stress > > >>>>>>> that > > >>>>>>>> I > > >>>>>>>>> do > > >>>>>>>>>>> not > > >>>>>>>>>>>>>>>> intend to rewrite everything from scratch (I never used > > >>>>>> this > > >>>>>>>>>> phrase). > > >>>>>>>>>>>>>>> There > > >>>>>>>>>>>>>>>> are significant parts of code that would be moved with > > >>>>>>> minimal > > >>>>>>>>>>>>>>>> modifications. Second, I never said that we will get > > >>>> rid > > >>>>> of > > >>>>>>> the > > >>>>>>>>> old > > >>>>>>>>>>>>> tests > > >>>>>>>>>>>>>>>> codebase. Every single test from Ignite 2.x should be > > >>>>> moved > > >>>>>>> to > > >>>>>>>>>>> Ignite 3 > > >>>>>>>>>>>>>>>> regardless of how we choose to proceed. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> My point is that for some parts of the code a clean > > >>>>>> bottom-up > > >>>>>>>>>>>>>>>> implementation will be cheaper in many ways. Let me > > >>>> give > > >>>>>> you > > >>>>>>> a > > >>>>>>>>> few > > >>>>>>>>>>>>>>> concrete > > >>>>>>>>>>>>>>>> examples: > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> - I think no one can object that we need a cleanly > > >>>>>> separated > > >>>>>>>>>>>>>>> persistence > > >>>>>>>>>>>>>>>> layer for Ignite. There is a very raw draft IEP for > > >>>> this > > >>>>>>>>> already. > > >>>>>>>>>> On > > >>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>> other hand, I think we also can agree that we need a > > >>>>>>>> split-brain > > >>>>>>>>>>>>>>>> resistant > > >>>>>>>>>>>>>>>> replication protocol for caches. There is also an IEP > > >>>>> for > > >>>>>>>> this. > > >>>>>>>>>>>>>>> Neither > > >>>>>>>>>>>>>>>> of > > >>>>>>>>>>>>>>>> the changes is a good fit for 2.x because they are > > >>>>> likely > > >>>>>> to > > >>>>>>>>>>>>> introduce > > >>>>>>>>>>>>>>>> breaking changes in the persistence layer, > > >>>> configuration > > >>>>>> and > > >>>>>>>>>>>>> behavior. > > >>>>>>>>>>>>>>>> Additionally, these components are now tightly > > >>>> coupled, > > >>>>> so > > >>>>>>>> there > > >>>>>>>>>> is > > >>>>>>>>>>>>> no > > >>>>>>>>>>>>>>>> way > > >>>>>>>>>>>>>>>> these two changes can be implemented in parallel and > > >>>>> then > > >>>>>>>> merged > > >>>>>>>>>>>>>>>> together > > >>>>>>>>>>>>>>>> easily. So what we will end up with is having to > > >>>>> implement > > >>>>>>>> these > > >>>>>>>>>>>>>>> changes > > >>>>>>>>>>>>>>>> sequentially, fixing all existing tests twice, and > > >>>>>>> essentially > > >>>>>>>>>>>>>>> throwing > > >>>>>>>>>>>>>>>> away half of the work done because the other part of > > >>>> the > > >>>>>>>> change > > >>>>>>>>> is > > >>>>>>>>>>>>>>>> re-implemented > > >>>>>>>>>>>>>>>> - Similar example goes with getting rid of > > >>>>>>>> IgniteInternalFuture > > >>>>>>>>>> and > > >>>>>>>>>>>>>>>> replacing it with CompletableFuture, and any other > > >>>>> change > > >>>>>>> that > > >>>>>>>>>>>>> touches > > >>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>> asynchronous part of the code. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Third, I do not see how this choice affects the UX of > > >>>>>> Ignite. > > >>>>>>>> The > > >>>>>>>>>> end > > >>>>>>>>>>>>>>> user > > >>>>>>>>>>>>>>>> experience must be fixed in the IEP regardless of the > > >>>>>>>> development > > >>>>>>>>>>>>> process > > >>>>>>>>>>>>>>>> and the fact that we have gaps in this area in Ignite > > >>>> 2.x > > >>>>>>> just > > >>>>>>>>>>> confirms > > >>>>>>>>>>>>>>>> that. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Pavel, agree that a repo/branch is a technicality, I > > >>>>> guess > > >>>>>> if > > >>>>>>>>>>>>>>> reformulate, > > >>>>>>>>>>>>>>>> my point is that we might agree to have a single > > >>>>>> development > > >>>>>>>>> master > > >>>>>>>>>>>>>>> branch > > >>>>>>>>>>>>>>>> with 'disassembled' end-to-end functionality for some > > >>>>>> period > > >>>>>>> of > > >>>>>>>>>> time > > >>>>>>>>>>> to > > >>>>>>>>>>>>>>>> speed up development, and re-assemble the core features > > >>>>>> after > > >>>>>>>>>> having > > >>>>>>>>>>>>>>>> submodules tested independently. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Nikolay, > > >>>>>>>>>>>>>>>>> We have many features that have to evolve. > > >>>>>>>>>>>>>>>>> Snapshots, rebalance, tooling, tracing, zookeeper > > >>>>> support, > > >>>>>>>> etc. > > >>>>>>>>>>>>>>>> This is not very specific. In the end, resources are > > >>>>>> limited > > >>>>>>>> and > > >>>>>>>>> we > > >>>>>>>>>>>>> will > > >>>>>>>>>>>>>>>> not be able to drive both tracks simultaneously, > > >>>>> especially > > >>>>>>>>> after a > > >>>>>>>>>>>>>>> couple > > >>>>>>>>>>>>>>>> of features having been implemented for Ignite 3.0. If > > >>>>>> there > > >>>>>>>> are > > >>>>>>>>>>> indeed > > >>>>>>>>>>>>>>>> some major changes that we want to do in Ignite 2.x > > >>>>> instead > > >>>>>>> of > > >>>>>>>>>>> putting > > >>>>>>>>>>>>>>>> effort into 3.0 - let's discuss them. I am just not > > >>>> aware > > >>>>>> of > > >>>>>>>> any, > > >>>>>>>>>>>>> that's > > >>>>>>>>>>>>>>>> why I am eager to move forward with Ignite 3.0. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> We have bugs and issues that can be fixed in 2.x > > >>>> without > > >>>>>>>>> breaking > > >>>>>>>>>>>>>>> backward > > >>>>>>>>>>>>>>>> compatibility. > > >>>>>>>>>>>>>>>>> We have many users who are happy with the 2.x with all > > >>>>>> it’s > > >>>>>>>>>> issues. > > >>>>>>>>>>>>>>>> These changes will be covered by end-to-end tests and > > >>>>>>> migrated > > >>>>>>>> to > > >>>>>>>>>>>>> Ignite > > >>>>>>>>>>>>>>>> 3.0, so I see no issues here. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> Finally, Anton & Nikolay > > >>>>>>>>>>>>>>>> I do not have an estimate for this simply because the > > >>>>>>> activity > > >>>>>>>> is > > >>>>>>>>>>>>>>>> community-driven and it depends on the number of people > > >>>>>>> willing > > >>>>>>>>> to > > >>>>>>>>>>>>>>>> contribute. With the current pace, I would hope to have > > >>>>> an > > >>>>>> RC > > >>>>>>>> of > > >>>>>>>>>>> Ignite > > >>>>>>>>>>>>>>> 3.0 > > >>>>>>>>>>>>>>>> to be ready by the end of 2021. My gut feeling is that > > >>>> by > > >>>>>>>> moving > > >>>>>>>>>> with > > >>>>>>>>>>>>>>>> incremental changes, we will not be able to implement > > >>>>> even > > >>>>>>> half > > >>>>>>>>> of > > >>>>>>>>>>> the > > >>>>>>>>>>>>>>>> wishlist by that time. > > >>>>>>>>>>>>>>>> I doubt that releasing several major releases with > > >>>>> breaking > > >>>>>>>>> changes > > >>>>>>>>>>>>> will > > >>>>>>>>>>>>>>>> make Ignite users happy either because each upgrade > > >>>> will > > >>>>>> cost > > >>>>>>>>>> Ignite > > >>>>>>>>>>>>>>> users > > >>>>>>>>>>>>>>>> money, so the fewer major versions we release, the > > >>>>> better. > > >>>>>>> Thus > > >>>>>>>>> my > > >>>>>>>>>>> wish > > >>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>> include all breaking changes in one release. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> I'll be now quiet for a while, let's see what other > > >>>>>> community > > >>>>>>>>>> members > > >>>>>>>>>>>>>>>> think. > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> пн, 2 нояб. 2020 г. в 15:52, Pavel Tupitsyn < > > >>>>>>>>> ptupit...@apache.org > > >>>>>>>>>>> : > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> 1. Rewriting from scratch is never a good idea. > > >>>>>>>>>>>>>>>>> We don't want to follow the path of Netscape and lose > > >>>>> all > > >>>>>>> our > > >>>>>>>>>> users > > >>>>>>>>>>>>>>>>> by the time we have a working 3.0 [1] > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> 2. Not sure about new repo - seems like some pain and > > >>>> no > > >>>>>>> gain, > > >>>>>>>>>>> what's > > >>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>> problem with a branch? > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> 3. We should keep existing integration tests when > > >>>>>> possible. > > >>>>>>>>>>>>>>>>> We have accumulated a lot of edge case knowledge over > > >>>>> the > > >>>>>>>> years, > > >>>>>>>>>>>>>>>>> it is not a good idea to send all of that down the > > >>>>> drain. > > >>>>>>>>>>>>>>>>> Yes, integration tests are slow, but they are the most > > >>>>>>>> valuable. > > >>>>>>>>>>>>>>>>> I think we can move more stuff into nightly runs and > > >>>>> have > > >>>>>> a > > >>>>>>>> fast > > >>>>>>>>>> and > > >>>>>>>>>>>>>>>> modern > > >>>>>>>>>>>>>>>>> basic suite. > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> Alexey, you are much more familiar with the Ignite > > >>>> core > > >>>>>>>> codebase > > >>>>>>>>>>> than > > >>>>>>>>>>>>>>>> most > > >>>>>>>>>>>>>>>>> of us, > > >>>>>>>>>>>>>>>>> can you please explain in more detail which particular > > >>>>>>>> feature, > > >>>>>>>>> in > > >>>>>>>>>>>>> your > > >>>>>>>>>>>>>>>>> opinion, > > >>>>>>>>>>>>>>>>> mandates this "start from scratch" approach? > > >>>>>>>>>>>>>>>>> Is it really not possible at all to follow a less > > >>>>> radical > > >>>>>>> way? > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> [1] > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> > > https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> On Mon, Nov 2, 2020 at 2:25 PM Nikolay Izhikov < > > >>>>>>>>>> nizhi...@apache.org > > >>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> Hello, Alexey. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> I think that «rewriting from scratch» approach has a > > >>>>> high > > >>>>>>>> risk > > >>>>>>>>> to > > >>>>>>>>>>>>>>> make > > >>>>>>>>>>>>>>>>> new > > >>>>>>>>>>>>>>>>>> features unusable. > > >>>>>>>>>>>>>>>>>> At the time Ignite2 was started no-one wants to do > > >>>> bad > > >>>>> UX > > >>>>>>> or > > >>>>>>>>> bad > > >>>>>>>>>>>>>>>>> features. > > >>>>>>>>>>>>>>>>>> Nevertheless, it happen. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> I think we can avoid it with the Ignite3 and > > >>>> successors > > >>>>>> if > > >>>>>>> we > > >>>>>>>>>> will > > >>>>>>>>>>>>>>> move > > >>>>>>>>>>>>>>>>>> step by step without keeping backward compatibility > > >>>>>>>>>>>>>>>>>> With the step by step approach, we can focus on each > > >>>>>>>> component > > >>>>>>>>>>>>>>>>> separately. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> What new features are we planning to implement for > > >>>>>> Ignite > > >>>>>>>> 2.x? > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> We have many features that have to evolve. > > >>>>>>>>>>>>>>>>>> Snapshots, rebalance, tooling, tracing, zookeeper > > >>>>>> support, > > >>>>>>>> etc. > > >>>>>>>>>>>>>>>>>> We have bugs and issues that can be fixed in 2.x > > >>>>> without > > >>>>>>>>> breaking > > >>>>>>>>>>>>>>>>> backward > > >>>>>>>>>>>>>>>>>> compatibility. > > >>>>>>>>>>>>>>>>>> We have many users who are happy with the 2.x with > > >>>> all > > >>>>>> it’s > > >>>>>>>>>> issues. > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> 2 нояб. 2020 г., в 14:09, Anton Vinogradov < > > >>>>>> a...@apache.org > > >>>>>>>> > > >>>>>>>>>>>>>>>> написал(а): > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> Alexey, > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> Do we have any estimates of how fast we'll be able > > >>>> to > > >>>>>> gain > > >>>>>>>>>>>>>>>>>> production-ready > > >>>>>>>>>>>>>>>>>>> AI 3.0 in case of a "new repo" choice? > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>> On Mon, Nov 2, 2020 at 2:01 PM Alexey Goncharuk < > > >>>>>>>>>>>>>>>>>> alexey.goncha...@gmail.com> > > >>>>>>>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> Nikolay, > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> What new features are we planning to implement for > > >>>>>> Ignite > > >>>>>>>>> 2.x? > > >>>>>>>>>> I > > >>>>>>>>>>>>>>>> think > > >>>>>>>>>>>>>>>>>> once > > >>>>>>>>>>>>>>>>>>>> we commence working on Ignite 3.0, we should > > >>>>> gradually > > >>>>>>>> cease > > >>>>>>>>>> the > > >>>>>>>>>>>>>>>>>> activity > > >>>>>>>>>>>>>>>>>>>> on Ignite 2.x to mere bugfixes because such > > >>>> parallel > > >>>>>>>>>> development > > >>>>>>>>>>>>>>>> will > > >>>>>>>>>>>>>>>>> be > > >>>>>>>>>>>>>>>>>>>> overwhelming regardless of how we choose to > > >>>> proceed. > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> пн, 2 нояб. 2020 г. в 13:38, Nikolay Izhikov < > > >>>>>>>>>>> nizhi...@apache.org > > >>>>>>>>>>>>>>>> : > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> To be clear: > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> I would suggest creating a new repository for > > >>>>> Ignite > > >>>>>>> 3.0 > > >>>>>>>>>>>>>>>> (perhaps, a > > >>>>>>>>>>>>>>>>>>>> new > > >>>>>>>>>>>>>>>>>>>>> clean branch, but a new repo looks nicer to me) > > >>>> and > > >>>>> a > > >>>>>>> new > > >>>>>>>>>> Ignite > > >>>>>>>>>>>>>>>> 3.0 > > >>>>>>>>>>>>>>>>>>>>> TeamCity project. > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> +1 for new Team City project. > > >>>>>>>>>>>>>>>>>>>>> +1 for new branch for Ignite3. > > >>>>>>>>>>>>>>>>>>>>> -1 for new repo. > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> 2 нояб. 2020 г., в 13:35, Nikolay Izhikov < > > >>>>>>>>>>>>>>> nizhikov....@gmail.com > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> написал(а): > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> Hello, Alexey. > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> I think it will hurt our project more than help. > > >>>>>>>>>>>>>>>>>>>>>> Developing new features for 2 separate branches > > >>>>> with > > >>>>>>> the > > >>>>>>>>>>>>>>> different > > >>>>>>>>>>>>>>>>>> APIs > > >>>>>>>>>>>>>>>>>>>>> and internal structure is overwhelming > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>> Maybe we should relax a bit requirements for > > >>>>> Ignite3? > > >>>>>>>>>>>>>>>>>>>>>> Maybe we should move step by step and make > > >>>> Ignite3 > > >>>>>> with > > >>>>>>>> new > > >>>>>>>>>>>>>>>>>>>>> configuration than Ignite4 with new transactions, > > >>>>> etc? > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> 2 нояб. 2020 г., в 13:14, Alexey Goncharuk < > > >>>>>>>>>>>>>>>>>>>> alexey.goncha...@gmail.com> > > >>>>>>>>>>>>>>>>>>>>> написал(а): > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> Igniters, > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> I wanted to pitch a rather radical idea > > >>>> regarding > > >>>>>> the > > >>>>>>>>> Ignite > > >>>>>>>>>>>>>>> 3.0 > > >>>>>>>>>>>>>>>>>>>>>>> development which has occurred to me some time > > >>>>> ago. > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> We already have several IEPs targeted to Ignite > > >>>>> 3.0 > > >>>>>>>> which > > >>>>>>>>>>> imply > > >>>>>>>>>>>>>>>>> major > > >>>>>>>>>>>>>>>>>>>>>>> changes to the codebase (the change in > > >>>> replication > > >>>>>>>>> protocol > > >>>>>>>>>>> and > > >>>>>>>>>>>>>>>>> thus > > >>>>>>>>>>>>>>>>>>>>>>> transactions, change in binary format, updated > > >>>>>>>>> metastorage, > > >>>>>>>>>>>>>>> etc). > > >>>>>>>>>>>>>>>>> We > > >>>>>>>>>>>>>>>>>>>>> also > > >>>>>>>>>>>>>>>>>>>>>>> planned significant changes in public APIs: > > >>>>>>>> configuration > > >>>>>>>>>>>>>>> format > > >>>>>>>>>>>>>>>>>>>> change, > > >>>>>>>>>>>>>>>>>>>>>>> improvements in cache APIs, SQL APIs, > > >>>> transaction > > >>>>>> mode > > >>>>>>>>>> rework. > > >>>>>>>>>>>>>>>> The > > >>>>>>>>>>>>>>>>>>>>> wishlist > > >>>>>>>>>>>>>>>>>>>>>>> of changes for Ignite 3.0 is huge. > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> So, I was wondering whether it makes sense to > > >>>> try > > >>>>> to > > >>>>>>>>> change > > >>>>>>>>>>> the > > >>>>>>>>>>>>>>>> old > > >>>>>>>>>>>>>>>>>>>>>>> codebase, or start with a new baseline and move > > >>>>> old > > >>>>>>>> pieces > > >>>>>>>>>> of > > >>>>>>>>>>>>>>>> code > > >>>>>>>>>>>>>>>>>>>> that > > >>>>>>>>>>>>>>>>>>>>> do > > >>>>>>>>>>>>>>>>>>>>>>> not require significant rework. Personally, I > > >>>>> would > > >>>>>> go > > >>>>>>>>> with > > >>>>>>>>>>> the > > >>>>>>>>>>>>>>>>>> second > > >>>>>>>>>>>>>>>>>>>>>>> option for the following reasons: > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> - We have a chance to shift the development > > >>>>> paradigm > > >>>>>>> in > > >>>>>>>>> the > > >>>>>>>>>>>>>>>> project > > >>>>>>>>>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>>>>>>>>> introduce the practice of true unit-tests. In > > >>>> the > > >>>>>> new > > >>>>>>>>>> baseline > > >>>>>>>>>>>>>>> at > > >>>>>>>>>>>>>>>>> the > > >>>>>>>>>>>>>>>>>>>>>>> beginning there will be no ability to run an > > >>>>>>> end-to-end > > >>>>>>>>>>>>>>> scenario, > > >>>>>>>>>>>>>>>>>>>> thus > > >>>>>>>>>>>>>>>>>>>>> we > > >>>>>>>>>>>>>>>>>>>>>>> will be forced to write unit-tests. So far, such > > >>>>>>>> practice > > >>>>>>>>>> was > > >>>>>>>>>>>>>>>> hard > > >>>>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>>>>>>> implement because of tight coupling between > > >>>> Ignite > > >>>>>>>>>> components > > >>>>>>>>>>>>>>> and > > >>>>>>>>>>>>>>>>>>>>> inability > > >>>>>>>>>>>>>>>>>>>>>>> to instantiate components without an instance of > > >>>>>>>>>>> KernalContext. > > >>>>>>>>>>>>>>>> For > > >>>>>>>>>>>>>>>>>>>>>>> example, we should be able to thoroughly test > > >>>>>> internal > > >>>>>>>>>>>>>>>> primitives, > > >>>>>>>>>>>>>>>>>>>>> such as > > >>>>>>>>>>>>>>>>>>>>>>> replication protocol (without actual > > >>>>> communication), > > >>>>>>>>>>>>>>> distributed > > >>>>>>>>>>>>>>>>>>>>>>> metastorage contracts, persistence layer, etc. > > >>>>>>>>>>>>>>>>>>>>>>> - We will significantly reduce the development > > >>>>> cycle > > >>>>>>> in > > >>>>>>>>> the > > >>>>>>>>>>>>>>>>> beginning > > >>>>>>>>>>>>>>>>>>>>>>> (right now the RunAll takes two hours of > > >>>>>> astronomical > > >>>>>>>> time > > >>>>>>>>>>> with > > >>>>>>>>>>>>>>>>> empty > > >>>>>>>>>>>>>>>>>>>>> TC; > > >>>>>>>>>>>>>>>>>>>>>>> in the new approach developer will be able to > > >>>> run > > >>>>>> ALL > > >>>>>>>>> tests > > >>>>>>>>>>>>>>>> locally > > >>>>>>>>>>>>>>>>>>>> in > > >>>>>>>>>>>>>>>>>>>>> a > > >>>>>>>>>>>>>>>>>>>>>>> matter of minutes) > > >>>>>>>>>>>>>>>>>>>>>>> - We can get rid of TC bot and enforce green TC > > >>>> by > > >>>>>>>>>> integrating > > >>>>>>>>>>>>>>> TC > > >>>>>>>>>>>>>>>>>>>> build > > >>>>>>>>>>>>>>>>>>>>>>> results with GitHub PRs (the same way Travis is > > >>>>>>>> currently > > >>>>>>>>>>>>>>>>> integrated > > >>>>>>>>>>>>>>>>>>>>> to PR > > >>>>>>>>>>>>>>>>>>>>>>> check). We should restrict PR merge without a TC > > >>>>>> check > > >>>>>>>>>>>>>>>>>>>>>>> - We will still have to re-write all tests, but > > >>>>> only > > >>>>>>>> once. > > >>>>>>>>>> If > > >>>>>>>>>>>>>>> we > > >>>>>>>>>>>>>>>>> try > > >>>>>>>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>>>>>>> modify the old codebase, we would need to modify > > >>>>> all > > >>>>>>> the > > >>>>>>>>>> tests > > >>>>>>>>>>>>>>>> for > > >>>>>>>>>>>>>>>>>>>>> every > > >>>>>>>>>>>>>>>>>>>>>>> major change (public API change, configuration > > >>>>>> change) > > >>>>>>>>>>>>>>>>>>>>>>> - We will have fewer conflicts when working > > >>>>>> together. > > >>>>>>>> For > > >>>>>>>>>>>>>>>> example, > > >>>>>>>>>>>>>>>>> I > > >>>>>>>>>>>>>>>>>>>>>>> cannot imagine how one would merge two changes > > >>>> of > > >>>>>>>> getting > > >>>>>>>>>> rid > > >>>>>>>>>>>>>>> of > > >>>>>>>>>>>>>>>>>>>>>>> IgniteFuture and changes in replication > > >>>> protocol, > > >>>>>> for > > >>>>>>>>>> example > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> Technically, I would suggest creating a new > > >>>>>> repository > > >>>>>>>> for > > >>>>>>>>>>>>>>> Ignite > > >>>>>>>>>>>>>>>>> 3.0 > > >>>>>>>>>>>>>>>>>>>>>>> (perhaps, a new clean branch, but a new repo > > >>>> looks > > >>>>>>> nicer > > >>>>>>>>> to > > >>>>>>>>>>> me) > > >>>>>>>>>>>>>>>>> and a > > >>>>>>>>>>>>>>>>>>>>> new > > >>>>>>>>>>>>>>>>>>>>>>> Ignite 3.0 TeamCity project. > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> While it may seem quite radical, I do believe > > >>>> that > > >>>>>>> this > > >>>>>>>>>>>>>>> approach > > >>>>>>>>>>>>>>>>> will > > >>>>>>>>>>>>>>>>>>>>> give > > >>>>>>>>>>>>>>>>>>>>>>> us more benefits than trying to make such major > > >>>>>>> changes > > >>>>>>>> in > > >>>>>>>>>> the > > >>>>>>>>>>>>>>>>>>>> existing > > >>>>>>>>>>>>>>>>>>>>>>> codebase. If needed, let's schedule a discord > > >>>> chat > > >>>>>>> like > > >>>>>>>>>> before > > >>>>>>>>>>>>>>> to > > >>>>>>>>>>>>>>>>>>>>> discuss > > >>>>>>>>>>>>>>>>>>>>>>> this. > > >>>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>>>> WDYT? > > >>>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> -- > > >>>>>>>>>>>>>>> Best regards, > > >>>>>>>>>>>>>>> Andrey V. Mashenkov > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >>> > > >>> -- > > >>> > > >>> Best regards, > > >>> Alexei Scherbakov > > > > > > > > > > > -- > Sincerely yours, Ivan Daschinskiy
