>> 1. Automatic tests scaling for new versions. >> 2. Automatic dependency management. Let's just improve the current solution. See no problems here.
You'll have my *Veto* on merge without real reasons to replace solution instead of improvement. On Wed, Feb 20, 2019 at 3:55 PM Pavel Kovalenko <jokse...@gmail.com> wrote: > Anton, > > In my first message, I already noticed the usability problems of the > current framework and showed how they can be solved using the new > framework. > It gives us 2 major advantages: > 1. Automatic tests scaling for new versions. > 2. Automatic dependency management. > Described approach simplifies tests writing and supporting. > > I don't see now how we can avoid manual versions adding and supporting in > test suites and avoid manual dependency management using custom java api > for Maven in the current version of the framework. > If you have ideas how these problems can be resolved in the current version > please share it. > > Nikolay, > > The new framework is designed for persistence compatibility first but can > be used for thin client testing as well. > It works exactly as you described. It collects all available testing > versions by scanning pom files. Then it builds jars for all old versions > using Maven. > Test framework run compatibility tests against all detected old versions. > The number of versions to test is controlled by a property. > Some versions can be excluded for particular test suite or the user can > bound it using versions range. > In all of these cases, there are no needs to copy-paste methods for each of > testing Ignite versions. > > ср, 20 февр. 2019 г. в 12:45, Nikolay Izhikov <nizhi...@apache.org>: > > > Hello, Pavel. > > > > Please, clarify. > > > > What exactly your new compatibility framework should check? > > I know at lease 2 compatible subsystem in Ignite that should work with > > previous versions: > > > > 1. Persistence. > > 2. Thin client. > > > > > A new version of Ignite has released and a developer should update > > compatibility tests to run it against the new version. > > > > I propose to change current approach - check version X with version Y. > > We should check X with X - 1 with X - 2 and so on, depending on our > > compatibility policies. > > > > This approach should lead to the following: > > > > 1. Tests should contains only number of previous versions we should > check. > > 2. Test framework should build versions list internally and use it to run > > tests. > > > > What do you think? > > > > > > > > ср, 20 февр. 2019 г. в 11:41, Pavel Kovalenko <jokse...@gmail.com>: > > > > > Vyacheslav, > > > > > > Thank you for answers! > > > > > > >> I'm not sure what is a problem here? > > > At the moment it's a little bit hard to understand the impact of such a > > > change because the number of test suites is small. > > > Let's Imagine you have X compatibility test suites where X is a > > relatively > > > big number, let's say 20. > > > A new version of Ignite has released and a developer should update > > > compatibility tests to run it against the new version. > > > A developer should manually find all of these 20 test suites and in > each > > of > > > these suites and add a new version to test it (new method). > > > This is a long process and there could be developer mistakes, some of > the > > > test suites can be missed and nobody will know about it before real > > > compatibility problem appears. > > > We already faced with the situation when after new version release > > > (2.5-2.6) no compatibility tests were modified to support new version > > > because nobody remembers about it and there is no such process after > > > release. > > > Adding new pom is a very simple step and can be done by any human who > > even > > > may not be familiar with the compatibility module and may not know > > anything > > > which compatibility tests are used. If we include adding pom file to > > > post-release process all existing tests will automatically detect new > > > version and next TC run will immediately show to us results. If there > are > > > some compatibility problems during development new version TC run will > > show > > > it. At the current approach, compatibility tests are out of focus and > we > > > may not see potential issues. > > > > > > >> This is not about usability, it's about the framework's internals > > > At the current approach, a developer should take care of dependency > > > management for the old versions. E.g. if you look > > > at IgnitePKIndexesMigrationToUnwrapPkTest you can see "magic" H2 > > dependency > > > version. I still don't understand why this version was explicitly added > > and > > > not just inherited from ignite-indexing. If we delegate all dependency > > > management to Maven itself it will simplify tests development. Tests > will > > > look much simpler. > > > > > > >> Please, describe the issue in more details? > > > Suppose you have a closure that invokes some deprecated method which > will > > > be removed in next release. After method removal, such tests will be > > > broken. At the current framework, I can't see a resolution of that > > problem. > > > In a new framework, you can keep the old version of closure, mark it > with > > > special annotation and place to separate module with old Ignite version > > > dependency. This closure will be compiled using an old version of > Ignite > > > and the user can still run it on the old version. > > > > > > > > > > > > вт, 19 февр. 2019 г. в 21:44, Vyacheslav Daradur <daradu...@gmail.com > >: > > > > > > > Hi, Pavel! > > > > > > > > First of all, I'd like to clarify that the Compatibility Testing > > > > Framework was designed to work with a cluster of multi-version nodes. > > > > The main idea is to run a test to verify backward compatibility or do > > > > some kind of rolling upgrades. > > > > > > > > It's not about persistence compatibility, but actually, it is used > for > > > > this now. > > > > > > > > >> 1) It's uncomfortable to add and support new versions of Ignite > > > product. > > > > > > > > I'm not sure what is a problem here? > > > > As I understand you suggest to do similar things - adding new pom.xml > > > > when new a version is released. Pay attention that not all tests > > > > should be tested on all versions, some test may want to test specific > > > > versions each other. > > > > > > > > >> 2) Manual maven dependency through java api. > > > > > > > > This is not about usability, it's about the framework's internals. If > > > > you have issues let's fix it and improve approach if needed. > > > > > > > > >> 3) It doesn't cover the case when some code which works on the > > current > > > > version will not work on older versions due to compile/runtime > > > > incompatibility. > > > > > > > > Please, describe the issue in more details? > > > > > > > > On Tue, Feb 19, 2019 at 7:55 PM Pavel Kovalenko <jokse...@gmail.com> > > > > wrote: > > > > > > > > > > Anton, > > > > > > > > > > >> What about the current compatibility framework? > > > > > Current compatibility framework will be removed after final > adjusting > > > to > > > > > new framework. > > > > > > > > > > >> Could you please share examples for each feature you mentioned? > > > > > You can see example in PR e.g. file > > > > > modules/compatibility/ignite-versions/2.1.0/pom.xml > > > > > < > > > > > > > > > > https://github.com/apache/ignite/pull/5974/files#diff-3c44ef223c31de9ff1e10bd7699cbcdc > > > > > > > > > > > > > > > I think such representing of Ignite product version is more cleaner > > > than > > > > > existing approach with Java classes with dependencies and manual > > > > dependecy > > > > > management. > > > > > > > > > > >> Anyway. I don't like the idea to implement something new instead > > of > > > > > improving the existing. > > > > > If you have concrete proposals how to improve current framework > > please > > > > > share it. > > > > > > > > > > > > > > > вт, 19 февр. 2019 г. в 19:11, Anton Vinogradov <a...@apache.org>: > > > > > > > > > > > +5,327 −59 > > > > > > What about the current compatibility framework? > > > > > > I see no removal or updates. > > > > > > > > > > > > >> Each new version is represented by a single pom > > > > > > Sound not good. > > > > > > Could you please share examples for each feature you mentioned? > > > > > > > > > > > > Anyway. I don't like the idea to implement something new instead > of > > > > > > improving the existing. > > > > > > > > > > > > On Tue, Feb 19, 2019 at 6:48 PM Pavel Kovalenko < > > jokse...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > I would like to start a discussion about replacement existing > > > > > > > persistence compatibility test framework with the newer > version. > > > > > > > The main purpose of that action is simplifying compatibility > > tests > > > > > > > development and support. > > > > > > > > > > > > > > The current version of the test framework has 3 disadvantages: > > > > > > > 1) It's uncomfortable to add and support new versions of Ignite > > > > product. > > > > > > > When a new version has released a developer must manually add > new > > > > test > > > > > > > methods to all existing test suites to support the new product > > > > version. > > > > > > > 2) Manual maven dependency through java api. > > > > > > > If a new version of Ignite product has some specific dependency > > > > version > > > > > > > (like H2) a developer should take care about and manually cover > > > this > > > > case > > > > > > > using java api. > > > > > > > 3) It doesn't cover the case when some code which works on the > > > > current > > > > > > > version will not work on older versions due to compile/runtime > > > > > > > incompatibility. > > > > > > > > > > > > > > A new version of the framework that is under development right > > now > > > > [1] > > > > > > > doesn't have such problems. > > > > > > > Here is a list of key features and possibilities: > > > > > > > > > > > > > > 1) One codebase - multiple versions support. > > > > > > > The key feature of the new framework is significant simplifying > > > > adding > > > > > > and > > > > > > > supporting the new versions of Ignite product. > > > > > > > Each new version is represented by a single pom file that > > contains > > > > Ignite > > > > > > > dependencies of specific version (core, indexing, etc.). All > > > > > > subdependecies > > > > > > > like H2 are covered by Maven automatically. > > > > > > > To add a new version for compatibility a developer just need to > > add > > > > a new > > > > > > > pom file with a new version of Ignite. All existing tests will > > > > > > > automatically detect the new version and will run tests against > > > this > > > > > > > version without additional changes. This feature covers p. 1 > and > > 2 > > > > of the > > > > > > > existing framework. > > > > > > > 2) Unified API to access and run code on old and new versions > of > > > > Ignite. > > > > > > > Each of Ignite instance is represented by remote JVM with a > > single > > > > point > > > > > > of > > > > > > > interaction - running abstract closures. Each closure is just a > > > class > > > > > > which > > > > > > > implement a "runner interface". Each closure object is > serialized > > > to > > > > a > > > > > > file > > > > > > > through Xstream library and executed on remote Ignite JVM. This > > > > approach > > > > > > > gives unified access to interact with both old and new versions > > of > > > > Ignite > > > > > > > and simplifies overall tests development. > > > > > > > 3) Ignite versions support for closures. Sometimes a closure > > > > couldn't be > > > > > > > run on newer or older Ignite version due to compile/runtime > > > > > > incompatibility > > > > > > > of that code. To resolve this problem a special annotation has > > > > > > introduced. > > > > > > > This annotation named as "@Since" contains minimal possible > > Ignite > > > > > > product > > > > > > > version where annotated closure can be compiled and run. Such > > > > closures > > > > > > are > > > > > > > processed on Ignite versions compile phase and this makes it > > > > possible to > > > > > > > use one closure for multiple Ignite versions without additional > > > code > > > > > > > changes. Closures and versioning resolve p. 3 of the existing > > > > framework. > > > > > > > > > > > > > > [1] - https://issues.apache.org/jira/browse/IGNITE-11133 > > > > > > > > > > > > > > I would like to hear any opinions about the new framework. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Best Regards, Vyacheslav D. > > > > > > > > > >
